网页死链检查器 LinkChecker

LinkChecker 是一个开源的网页链接检查程序, 基于GPL 开源协议。需要Python 2.7.2 以上环境. Python 3暂时不支持。

特性：

    循环遍历和多线程检查
    输出各种格式检查结果：text, HTML, SQL, CSV, XML
    支持 HTTP/1.1, HTTPS, FTP, mailto:, news:, nntp:, Telnet 和本地文件链接检查
    可使用正则表达式对链接的url进行过滤
    支持代理服务器
    支持用户名和密码验证
    遵守 robots.txt 法则
    支持 Cookie
    支持 HTML 和 CSS 语法检查
    反病毒检查
    提供命令行和图形界面和Web三种用户接口

官网：http://wummel.github.io/linkchecker/

-----------------------------------------------------

check links in web documents or full websites

wummel.github.io/linkchecker/ 

LinkChecker

Check for broken links in web sites.

Features

• recursive and multithreaded checking and site crawling
• output in colored or normal text, HTML, SQL, CSV, XML or a sitemap graph in different formats
• HTTP/1.1, HTTPS, FTP, mailto:, news:, nntp:, Telnet and local file links support
• restrict link checking with regular expression filters for URLs
• proxy support
• username/password authorization for HTTP, FTP and Telnet
• honors robots.txt exclusion protocol
• Cookie support
• HTML5 support
• a command line and web interface
• various check plugins available, eg. HTML syntax and antivirus checks.

Installation

See doc/install.txt in the source code archive. Python 2.7.2 or later is needed.

Usage

Execute linkchecker http://www.example.com. For other options see linkchecker --help.

 from https://github.com/wummel/linkchecker