Total Pageviews

Thursday 11 August 2016

awesome_bot


Verify links in awesome projects

Build Status Tests Gem Version
awesome_bot checks for valid URLs in a file, it can be used to verify pull requests updating a README.

Installation

$ gem install awesome_bot

Usage

Command Line

Usage: awesome_bot [file or files]
       awesome_bot [options]
    -f, --files [files]              Comma separated files to check
    -a, --allow [errors]             Status code errors to allow
        --allow-dupe                 Duplicate URLs are allowed
        --allow-ssl                  SSL errors are allowed
        --allow-redirect             Redirected URLs are allowed
        --allow-timeout              URLs that time out are allowed
    -t, --set-timeout [seconds]      Set connection timeout
    -w, --white-list [urls]          Comma separated URLs to white list
  • You can check multiple files (comma separated or * pattern, look below for details).
  • By default, duplicate URLs or any status code other than 200 are flagged as failures.
    • Use option --allow-dupe to allow duplicates.
    • Use option --allow-redirect to allow redirects.
    • Use option --allow to allow specific status code errors.
    • Use option --white-list (-w for short) to prevent links from being flagged: -w domain1.com/post/article,domain2.com white lists domain1.com/post/article and all links matching domain2.com.

Examples

$ awesome_bot README.md
> Checking links in README.md
Links found: 56, 37 unique
  01. https://github.com/sindresorhus/awesome
  02. http://i.giphy.com/urvsFBDfR6N32.gif
  03. https://travis-ci.org/dkhamsing/awesome_bot.svg
# ...
  37. https://github.com/dkhamsing
  Checking URLs: ✓✓✓→?✓→✓→→✓✓→✓✓✓→✓✓✓✓✓✓✓✓✓✓✓→✓✓✓✓✓→✓✓

Issues :-(
> Links
  1. [L007] 301 https://travis-ci.org/dkhamsing/awesome_bot.svg → https://api.travis-ci.org/dkhamsing/awesome_bot.svg
  2. [L008] 302 https://badge.fury.io/rb/awesome_bot → http://rubygems.org/gems/awesome_bot
# ...
> Dupes
  1. [L03] https://github.com/sindresorhus/awesome
  2. [L05] http://i.giphy.com/urvsFBDfR6N32.gif
# ...
$ awesome_bot README.md --allow-dupe --allow-redirect -w rubydoc,giphy
# allow redirects, dupes and white list all links matching rubydoc and giphy

$ awesome_bot README.md,README-zh.md
# check links in 2 files

$ awesome_bot docs/*.md
# check all markdown files in docs/ directory

$ awesome_bot README.md --allow-timeout -t 5
# speed up validation by setting a timeout of 5s per link request and allowing timeouts

$ awesome_bot README.md --allow 403,429
# allow status code errors 403 and 429
# --allow 301 would be similar to --allow-redirect
(master) $ git branch
* master
(master) $ git checkout -b new-branch
Switched to a new branch 'new-branch'
(new-branch) $ touch new-readme.md && echo 'https://github.com/dkhamsing' >> new-readme.md
(new-branch) $ git add new-readme.md
(new-branch) $ git commit -m 'Testing'
[new-branch ef47336] Testing
 1 file changed, 1 insertion(+)
 create mode 100644 new-readme.md
(new-branch) $ git diff master.. --name-only | grep '.md' | xargs awesome_bot
> Checking links in new-readme.md
Links to check: 1
  1. https://github.com/dkhamsing
Checking URLs: ✓
No issues :-)

Wrote results to ab-results-new-readme.md.json

Library

irb(main):001:0> require 'awesome_bot'
=> true
irb(main):002:0> content = File.read 'README.md'
=> "..."
irb(main):003:0> result = AwesomeBot.check content
=> #<AwesomeBot::Result:0x007fdde39f4408 @links=...>
# AwesomeBot Result with success, statuses_issues, dupes and more
irb(main):004:0> puts result.success ? 'No errors' : ':-('
:-(
More information at rubydoc.

Validate Pull Requests

Does your GitHub README contain a lot of links? awesome_bot can help you validate them when a pull request is created (or a commit is pushed). It is used by:
and more.
Tips
  • Use the keyword [ci skip] in your commit title/message to skip verification.
  • Use Danger.

Travis CI

To use awesome_bot with Travis CI, connect your repo and create a .travis.yml file.
language: ruby
rvm: 2.2
before_script: gem install awesome_bot
script: awesome_bot README.md
To turn off email notifications, add the lines below
notifications:
  email: false

Circle CI

If you prefer Circle CI, it'll work too. Connect your repo and create a circle.yml file.
machine:
  ruby:
    version: 2.2.0
test:
  pre:
    - gem install awesome_bot
  override:
    - awesome_bot README.md

More

Circle CI, Codeship, and Semaphore CI support running tests without adding a file to the repo (a public configuration file can however help others contribute).
# Codeship
Setup
rvm use 2.2.0 --install
gem install awesome_bot

Test
awesome_bot README.md
# Semaphore CI
Language: Ruby
Ruby version: 2.2
Databases for: don't generate
Setup:
gem install awesome_bot
awesome_bot README.md

Status Badge

Build Status
To add the Travis CI build status badge above to your project, use the following code
[![Build Status](https://travis-ci.org/<username>/<project>.svg)](https://travis-ci.org/<username>/<project>)

i.e.
[![Build Status](https://travis-ci.org/dkhamsing/awesome_bot.svg?branch=master)](https://travis-ci.org/dkhamsing/awesome_bot)
As it happens, the default code snippet provided contain a redirect so adding a badge could fail your status :sob:.. one way to fix this is to white list travis-ci, i.e.
- awesome_bot README.md --white-list travis-ci
You can also add a badge for other CI tools, check out shields.io.

Danger

Integrate awesome_bot with Danger and have results reported back to the pull request).
danger
Here's the step in your Dangerfile:
# Check links
require 'json'
results = File.read 'ab-results-README.md-markdown-table.json'
j = JSON.parse results
if j['error']==true
  fail j['title']
  markdown j['message']
end
 
FROM https://github.com/dkhamsing/awesome_bot