Pages

Thursday, 2 November 2017

aria2 - The ultra fast download utility

aria2 is a lightweight multi-protocol & multi-source, cross platform download utility operated in command-line. It supports HTTP/HTTPS, FTP, SFTP, BitTorrent and Metalink. 

Disclaimer

This program comes with no warranty. You must use this program at your own risk.

Introduction

aria2 is a utility for downloading files. The supported protocols are HTTP(S), FTP, SFTP, BitTorrent, and Metalink. aria2 can download a file from multiple sources/protocols and tries to utilize your maximum download bandwidth. It supports downloading a file from HTTP(S)/FTP/SFTP and BitTorrent at the same time, while the data downloaded from HTTP(S)/FTP/SFTP is uploaded to the BitTorrent swarm. Using Metalink's chunk checksums, aria2 automatically validates chunks of data while downloading a file like BitTorrent.
The project page is located at https://aria2.github.io/.
See aria2 Online Manual (Russian translationPortuguese translation) to learn how to use aria2.

Features

Here is a list of features:
  • Command-line interface
  • Download files through HTTP(S)/FTP/SFTP/BitTorrent
  • Segmented downloading
  • Metalink version 4 (RFC 5854) support(HTTP/FTP/SFTP/BitTorrent)
  • Metalink version 3.0 support(HTTP/FTP/SFTP/BitTorrent)
  • Metalink/HTTP (RFC 6249) support
  • HTTP/1.1 implementation
  • HTTP Proxy support
  • HTTP BASIC authentication support
  • HTTP Proxy authentication support
  • Well-known environment variables for proxy: http_proxyhttps_proxyftp_proxyall_proxy and no_proxy
  • HTTP gzip, deflate content encoding support
  • Verify peer using given trusted CA certificate in HTTPS
  • Client certificate authentication in HTTPS
  • Chunked transfer encoding support
  • Load Cookies from file using the Firefox3 format, Chromium/Google Chrome and the Mozilla/Firefox (1.x/2.x)/Netscape format.
  • Save Cookies in the Mozilla/Firefox (1.x/2.x)/Netscape format.
  • Custom HTTP Header support
  • Persistent Connections support
  • FTP/SFTP through HTTP Proxy
  • Download/Upload speed throttling
  • BitTorrent extensions: Fast extension, DHT, PEX, MSE/PSE, Multi-Tracker, UDP tracker
  • BitTorrent WEB-Seeding. aria2 requests chunks more than piece size to reduce the request overhead. It also supports pipelined requests with piece size.
  • BitTorrent Local Peer Discovery
  • Rename/change the directory structure of BitTorrent downloads completely
  • JSON-RPC (over HTTP and WebSocket)/XML-RPC interface
  • Run as a daemon process
  • Selective download in multi-file torrent/Metalink
  • Chunk checksum validation in Metalink
  • Can disable segmented downloading in Metalink
  • Netrc support
  • Configuration file support
  • Download URIs found in a text file or stdin and the destination directory and output file name can be specified optionally
  • Parameterized URI support
  • IPv6 support with Happy Eyeballs
  • Disk cache to reduce disk activity

Versioning and release schedule

We use 3 numbers for aria2 version: MAJOR.MINOR.PATCH. We will ship MINOR update on 15th of every month. We may skip a release if we have no changes since the last release. The feature and documentation freeze happens 10 days before the release day (5th day of the month) for translation teams. We will raise an issue about the upcoming release around that day.
We may release PATCH releases between regular releases if we have security issues.
MAJOR version will stay at 1 for the time being.

How to get source code

We maintain the source code at Github: https://github.com/aria2/aria2
To get the latest source code, run following command:
$ git clone https://github.com/aria2/aria2.git
This will create aria2 directory in your current directory and source files are stored there.

Dependency

featuresdependency
HTTPSOSX or GnuTLS or OpenSSL or Windows
SFTPlibssh2
BitTorrentNone. Optional: libnettle+libgmp or libgcrypt or OpenSSL (see note)
Metalinklibxml2 or Expat.
ChecksumNone. Optional: OSX or libnettle or libgcrypt or OpenSSL or Windows (see note)
gzip, deflate in HTTPzlib
Async DNSC-Ares
Firefox3/Chromium cookielibsqlite3
XML-RPClibxml2 or Expat.
JSON-RPC over WebSocketlibnettle or libgcrypt or OpenSSL
Note
libxml2 has precedence over Expat if both libraries are installed. If you prefer Expat, run configure with --without-libxml2.
Note
On Apple OSX the OS-level SSL/TLS support will be preferred. Hence neither GnuTLS nor OpenSSL are required on that platform. If you'd like to disable this behavior, run configure with --without-appletls.
GnuTLS has precedence over OpenSSL if both libraries are installed. If you prefer OpenSSL, run configure with --without-gnutls --with-openssl.
On Windows there is SSL implementation available that is based on the native Windows SSL capabilities (Schannel) and it will be preferred. Hence neither GnuTLS nor OpenSSL are required on that platform. If you'd like to disable this behavior, run configure with --without-wintls.
Note
On Apple OSX the OS-level checksum support will be preferred, unless aria2 is configured with --without-appletls.
libnettle has precedence over libgcrypt if both libraries are installed. If you prefer libgcrypt, run configure with --without-libnettle --with-libgcrypt. If OpenSSL is selected over GnuTLS, neither libnettle nor libgcrypt will be used.
If none of the optional dependencies are installed, an internal implementation that only supports md5 and sha1 will be used.
On Windows there is SSL implementation available that is based on the native Windows capabilities and it will be preferred, unless aria2 is configured with --without-wintls.
A user can have one of the following configurations for SSL and crypto libraries:
  • libgcrypt
  • libnettle
  • OpenSSL
  • GnuTLS + libgcrypt
  • GnuTLS + libnettle
You can disable BitTorrent and Metalink support by providing --disable-bittorrent and --disable-metalink to the configure script respectively.
In order to enable async DNS support, you need c-ares.

How to build

aria2 is primarily written in C++. Initially it was written based on C++98/C++03 standard features. We are now migrating aria2 to C++11 standard. The current source code requires C++11 aware compiler. For well-known compilers, such as g++ and clang, the -std=c++11 or -std=c++0x flag must be supported.
In order to build aria2 from the source package, you need following development packages (package name may vary depending on the distribution you use):
  • libgnutls-dev (Required for HTTPS, BitTorrent, Checksum support)
  • nettle-dev (Required for BitTorrent, Checksum support)
  • libgmp-dev (Required for BitTorrent)
  • libssh2-1-dev (Required for SFTP support)
  • libc-ares-dev (Required for async DNS support)
  • libxml2-dev (Required for Metalink support)
  • zlib1g-dev (Required for gzip, deflate decoding support in HTTP)
  • libsqlite3-dev (Required for Firefox3/Chromium cookie support)
  • pkg-config (Required to detect installed libraries)
You can use libgcrypt-dev instead of nettle-dev and libgmp-dev:
  • libgpg-error-dev (Required for BitTorrent, Checksum support)
  • libgcrypt-dev (Required for BitTorrent, Checksum support)
You can use libssl-dev instead of libgnutls-dev, nettle-dev, libgmp-dev, libgpg-error-dev and libgcrypt-dev:
  • libssl-dev (Required for HTTPS, BitTorrent, Checksum support)
You can use libexpat1-dev instead of libxml2-dev:
  • libexpat1-dev (Required for Metalink support)
On Fedora you need the following packages: gcc, gcc-c++, kernel-devel, libgcrypt-devel, libxml2-devel, openssl-devel, gettext-devel, cppunit
If you downloaded source code from git repository, you have to install following packages to get autoconf macros:
  • libxml2-dev
  • libcppunit-dev
  • autoconf
  • automake
  • autotools-dev
  • autopoint
  • libtool
And run following command to generate configure script and other files necessary to build the program:
$ autoreconf -i
Also you need Sphinx to build man page.
If you are building aria2 for Mac OS X, take a look at the make-release-os.mk GNU Make makefile.
The quickest way to build aria2 is first run configure script:
$ ./configure
To build statically linked aria2, use ARIA2_STATIC=yes command-line option:
$ ./configure ARIA2_STATIC=yes
After configuration is done, run make to compile the program:
$ make
See Cross-compiling Windows binary to create a Windows binary. See Cross-compiling Android binary to create an Android binary.
The configure script checks available libraries and enables as many features as possible except for experimental features not enabled by default.
Since 1.1.0, aria2 checks the certificate of HTTPS servers by default. If you build with OpenSSL or the recent version of GnuTLS which has gnutls_certificate_set_x509_system_trust() function and the library is properly configured to locate the system-wide CA certificates store, aria2 will automatically load those certificates at the startup. If it is not the case, I recommend to supply the path to the CA bundle file. For example, in Debian the path to CA bundle file is '/etc/ssl/certs/ca-certificates.crt' (in ca-certificates package). This may vary depending on your distribution. You can give it to configure script using --with-ca-bundle option:
$ ./configure --with-ca-bundle='/etc/ssl/certs/ca-certificates.crt'
$ make
Without --with-ca-bundle option, you will encounter the error when accessing HTTPS servers because the certificate cannot be verified without CA bundle. In such case, you can specify the CA bundle file using aria2's --ca-certificateoption. If you don't have CA bundle file installed, then the last resort is disable the certificate validation using --check-certificate=false.
Using the native OSX (AppleTLS) and/or Windows (WinTLS) implementation will automatically use the system certificate store, so --with-ca-bundle is not necessary and will be ignored when using these implementations.
By default, the bash_completion file named aria2c is installed to the directory $prefix/share/doc/aria2/bash_completion. To change the install directory of the file, use --with-bashcompletiondir option.
After a make the executable is located at src/aria2c.
aria2 uses CppUnit for automated unit testing. To run the unit test:
$ make check

Cross-compiling Windows binary

In this section, we describe how to build a Windows binary using a mingw-w64 (http://mingw-w64.org/doku.php) cross-compiler on Debian Linux. The MinGW (http://www.mingw.org/) may not be able to build aria2.
The easiest way to build Windows binary is use Dockerfile.mingw. See Dockerfile.mingw how to build binary. If you cannot use Dockerfile, then continue to read following paragraphs.
Basically, after compiling and installing depended libraries, you can do cross-compile just passing appropriate --hostoption and specifying CPPFLAGSLDFLAGS and PKG_CONFIG_LIBDIR variables to configure. For convenience and lowering our own development cost, we provide easier way to configure the build settings.
mingw-config script is a configure script wrapper for mingw-w64. We use it to create official Windows build. This script assumes following libraries have been built for cross-compile:
  • c-ares
  • expat
  • sqlite3
  • zlib
  • libssh2
  • cppunit
Some environment variables can be adjusted to change build settings:
HOST
cross-compile to build programs to run on HOST. It defaults to i686-w64-mingw32. To build 64bit binary, specifyx86_64-w64-mingw32.
PREFIX
Prefix to the directory where dependent libraries are installed. It defaults to /usr/local/$HOST-I$PREFIX/includewill be added to CPPFLAGS-L$PREFIX/lib will be added to LDFLAGS$PREFIX/lib/pkgconfig will be set toPKG_CONFIG_LIBDIR.
For example, to build 64bit binary do this:
$ HOST=x86_64-w64-mingw32 ./mingw-config
If you want libaria2 dll with --enable-libaria2, then don't use ARIA2_STATIC=yes and prepare the DLL version of external libraries.

Cross-compiling Android binary

In this section, we describe how to build Android binary using Android NDK cross-compiler on Debian Linux.
At the time of this writing, android-ndk-r14b should compile aria2 without errors.
android-config script is a configure script wrapper for Android build. We use it to create official Android build. This script assumes the following libraries have been built for cross-compile:
  • c-ares
  • openssl
  • expat
  • zlib
  • libssh2
When building the above libraries, make sure that disable shared library and enable only static library. We are going to link those libraries statically.
We use zlib which comes with Android NDK, so we don't have to build it by ourselves.
android-config assumes the existence of $ANDROID_HOME environment variable which must fulfill the following conditions:
  • Android NDK toolchain is installed under $ANDROID_HOME/toolchain. Refer to Standalone Toolchain for more details, but it is a bit out of date.
    To install toolchain under $ANDROID_HOME/toolchain, do this:
    $NDK/build/tools/make_standalone_toolchain.py \
       --arch arm --api 16 --stl=gnustl \
       --install-dir $ANDROID_HOME/toolchain
    
  • The dependent libraries must be installed under $ANDROID_HOME/usr/local.
Before running android-config and android-make$ANDROID_HOME environment variable must be set to point to the correct path.
After android-config, run android-make to compile sources.

Building documentation

Sphinx is used to build the documentation. aria2 man pages will be build when you run make if they are not up-to-date. You can also build HTML version of aria2 man page by make html. The HTML version manual is also available at online(Russian translationPortuguese translation).

BitTorrent

About file names

The file name of the downloaded file is determined as follows:
single-file mode
If "name" key is present in .torrent file, file name is the value of "name" key. Otherwise, file name is the base name of .torrent file appended by ".file". For example, .torrent file is "test.torrent", then file name is "test.torrent.file". The directory to store the downloaded file can be specified by -d option.
multi-file mode
The complete directory/file structure mentioned in .torrent file is created. The directory to store the top directory of downloaded files can be specified by -d option.
Before download starts, a complete directory structure is created if needed. By default, aria2 opens at most 100 files mentioned in .torrent file, and directly writes to and reads from these files. The number of files to open simultaneously can be controlled by --bt-max-open-files option.

DHT

aria2 supports mainline compatible DHT. By default, the routing table for IPv4 DHT is saved to $XDG_CACHE_HOME/aria2/dht.dat and the routing table for IPv6 DHT is saved to $XDG_CACHE_HOME/aria2/dht6.datunless files exist at $HOME/.aria2/dht.dat or $HOME/.aria2/dht6.dat. aria2 uses same port number to listen on for both IPv4 and IPv6 DHT.

UDP tracker

UDP tracker support is enabled when IPv4 DHT is enabled. The port number of UDP tracker is shared with DHT. Use --dht-listen-port option to change the port number.

Other things should be noted

  • -o option is used to change the file name of .torrent file itself, not a file name of a file in .torrent file. For this purpose, use --index-out option instead.
  • The port numbers that aria2 uses by default are 6881-6999 for TCP and UDP.
  • aria2 doesn't configure port-forwarding automatically. Please configure your router or firewall manually.
  • The maximum number of peers is 55. This limit may be exceeded when download rate is low. This download rate can be adjusted using --bt-request-peer-speed-limit option.
  • As of release 0.10.0, aria2 stops sending request message after selective download completes.

Metalink

The current implementation supports HTTP(S)/FTP/SFTP/BitTorrent. The other P2P protocols are ignored. Both Metalink4 (RFC 5854) and Metalink version 3.0 documents are supported.
For checksum verification, md5, sha-1, sha-224, sha-256, sha-384 and sha-512 are supported. If multiple hash algorithms are provided, aria2 uses stronger one. If whole file checksum verification fails, aria2 doesn't retry the download and just exits with non-zero return code.
The supported user preferences are version, language, location, protocol and os.
If chunk checksums are provided in Metalink file, aria2 automatically validates chunks of data during download. This behavior can be turned off by a command-line option.
If signature is included in a Metalink file, aria2 saves it as a file after the completion of the download. The file name is download file name + ".sig". If same file already exists, the signature file is not saved.
In Metalink4, multi-file torrent could appear in metalink:metaurl element. Since aria2 cannot download 2 same torrents at the same time, aria2 groups files in metalink:file element which has same BitTorrent metaurl and downloads them from a single BitTorrent swarm. This is basically multi-file torrent download with file selection, so the adjacent files which is not in Metalink document but shares same piece with selected file are also created.
If relative URI is specified in metalink:url or metalink:metaurl element, aria2 uses the URI of Metalink file as base URI to resolve the relative URI. If relative URI is found in Metalink file which is read from local disk, aria2 uses the value of --metalink-base-uri option as base URI. If this option is not specified, the relative URI will be ignored.

Metalink/HTTP

The current implementation only uses rel=duplicate links only. aria2 understands Digest header fields and check whether it matches the digest value from other sources. If it differs, drop connection. aria2 also uses this digest value to perform checksum verification after download finished. aria2 recognizes geo value. To tell aria2 which location you prefer, you can use --metalink-location option.

netrc

netrc support is enabled by default for HTTP(S)/FTP/SFTP. To disable netrc support, specify -n command-line option. Your .netrc file should have correct permissions(600).

WebSocket

The WebSocket server embedded in aria2 implements the specification defined in RFC 6455. The supported protocol version is 13.

libaria2

The libaria2 is a C++ library which offers aria2 functionality to the client code. Currently, libaria2 is not built by default. To enable libaria2, use --enable-libaria2 configure option. By default, only the shared library is built. To build static library, use --enable-static configure option as well. See libaria2 documentation to know how to use API.

References



from https://github.com/aria2/aria2
-------

使用静态Aria2二进制文件快速安装Aria2,及使用方法


说明:Aria2是一个很强的BT下载神器,用的人也多。不过也有很多人手动编译安装的时候,会遇到很多错误,这里博主就分享个静态文件安装方法,出错率很低。

安装
Github地址:https://github.com/q3aql/aria2-static-builds

这里只说下Linux系统安装方法,首先找到适合自己系统的最新版的Aria2静态文件→查看:
https://github.com/q3aql/aria2-static-builds/releases,再使用命令:

#下载静态文件
wget https://github.com/q3aql/aria2-static-builds/releases/download/v1.34.0/aria2-1.34.0-linux-gnu-64bit-build1.tar.bz2
#解压文件并进入文件夹
tar jxvf aria2-*.tar.bz2 && rm -rf aria2-*.tar.bz2 && cd aria2-*
#开始安装
make install

##如果安装的时候提示-bash: make: command not found错误,需要先运行命令:
Debian/Ubuntu系统:apt install make -y
CentOS系统:yum install make -y
这时候就安装好了,可使用aria2c -v查看版本信息。

使用
使用的话很简单,先新建aria2.conf配置文件(存放路径随意),配置如下:

# 文件的保存路径(可使用绝对路径或相对路径), 默认: 当前启动位置
dir=/data/aria2/download
# 启用磁盘缓存, 0为禁用缓存, 需1.16以上版本, 默认:16M
#disk-cache=32M
# 文件预分配方式, 能有效降低磁盘碎片, 默认:prealloc
# 预分配所需时间: none < falloc ? trunc < prealloc
# falloc和trunc则需要文件系统和内核支持
# NTFS建议使用falloc, EXT3/4建议trunc, MAC 下需要注释此项
file-allocation=trunc
# 断点续传
continue=true

## 下载连接相关 ##

# 最大同时下载任务数, 运行时可修改, 默认:5
max-concurrent-downloads=5
# 同一服务器连接数, 添加时可指定, 默认:1
max-connection-per-server=8
# 最小文件分片大小, 添加时可指定, 取值范围1M -1024M, 默认:20M
# 假定size=10M, 文件为20MiB 则使用两个来源下载; 文件为15MiB 则使用一个来源下载
min-split-size=10M
# 单个任务最大线程数, 添加时可指定, 默认:5
split=16
# 整体下载速度限制, 运行时可修改, 默认:0
#max-overall-download-limit=0
# 单个任务下载速度限制, 默认:0
#max-download-limit=0
# 整体上传速度限制, 运行时可修改, 默认:0
#max-overall-upload-limit=0
# 单个任务上传速度限制, 默认:0
#max-upload-limit=0
# 禁用IPv6, 默认:false
disable-ipv6=true

## 进度保存相关 ##

# 从会话文件中读取下载任务
input-file=/data/aria2/aria2.session
# 在Aria2退出时保存`错误/未完成`的下载任务到会话文件
save-session=/data/aria2/aria2.session
# 定时保存会话, 0为退出时才保存, 需1.16.1以上版本, 默认:0
#save-session-interval=60

## RPC相关设置 ##

# 启用RPC, 默认:false
enable-rpc=true
# 允许所有来源, 默认:false
rpc-allow-origin-all=true
# 允许非外部访问, 默认:false
rpc-listen-all=true
# 事件轮询方式, 取值:[epoll, kqueue, port, poll, select], 不同系统默认值不同
#event-poll=select
# RPC监听端口, 端口被占用时可以修改, 默认:6800
#rpc-listen-port=6800
# 设置的RPC授权令牌, v1.18.4新增功能, 取代 --rpc-user 和 --rpc-passwd 选项
rpc-secret=moerats
# 设置的RPC访问用户名, 此选项新版已废弃, 建议改用 --rpc-secret 选项
#rpc-user=
# 设置的RPC访问密码, 此选项新版已废弃, 建议改用 --rpc-secret 选项
#rpc-passwd=

## BT/PT下载相关 ##

# 当下载的是一个种子(以.torrent结尾)时, 自动开始BT任务, 默认:true
#follow-torrent=true
# BT监听端口, 当端口被屏蔽时使用, 默认:6881-6999
listen-port=51413
# 单个种子最大连接数, 默认:55
#bt-max-peers=55
# 打开DHT功能, PT需要禁用, 默认:true
enable-dht=false
# 打开IPv6 DHT功能, PT需要禁用
#enable-dht6=false
# DHT网络监听端口, 默认:6881-6999
#dht-listen-port=6881-6999
# 本地节点查找, PT需要禁用, 默认:false
#bt-enable-lpd=false
# 种子交换, PT需要禁用, 默认:true
enable-peer-exchange=false
# 每个种子限速, 对少种的PT很有用, 默认:50K
#bt-request-peer-speed-limit=50K
# 客户端伪装, PT需要
peer-id-prefix=-TR2770-
user-agent=Transmission/2.77
# 当种子的分享率达到这个数时, 自动停止做种, 0为一直做种, 默认:1.0
seed-ratio=0
# 强制保存会话, 即使任务已经完成, 默认:false
# 较新的版本开启后会在任务完成后依然保留.aria2文件
#force-save=false
# BT校验相关, 默认:true
#bt-hash-check-seed=true
# 继续之前的BT任务时, 无需再次校验, 默认:false
bt-seed-unverified=true
# 保存磁力链接元数据为种子文件(.torrent文件), 默认:false
bt-save-metadata=true
#停止做种
seed-time=0
根据情况,修改下载路径,连接密匙(rpc-secret为密匙参数),然后新建配置文件里的相关文件夹/文件。

#新建下载路径,session会话文件,顺便加个Aria2日志文件,路径和配置文件里一致
mkdir -p /data/aria2/download
touch /data/aria2/aria2.session /data/aria2/aria2.log
最后启动Aria2:

#aria2.conf文件所在路径填对,这里放在了/data/aria2文件夹
aria2c --conf-path=/data/aria2/aria2.conf
#后台运行(建议直接使用这个命令)
nohup aria2c --conf-path=/data/aria2/aria2.conf > /data/aria2/aria2.log 2>&1 &
CentOS系统可能还需要开放几个端口,运行命令:

#CentOS 6
iptables -I INPUT -p tcp --dport 6800 -j ACCEPT
iptables -I INPUT -p tcp --dport 6998 -j ACCEPT
iptables -I INPUT -p tcp --dport 51413 -j ACCEPT
service iptables save
service iptables restart

#CentOS 7
firewall-cmd --zone=public --add-port=6800/tcp --permanent
firewall-cmd --zone=public --add-port=6998/tcp --permanent
firewall-cmd --zone=public --add-port=51413/tcp --permanent
firewall-cmd --reload
这时候Aria2安装及配置都好了,接下来就是使用了;使用的话,我们可以借助Aria2前端Web面板一起使用,常见的有Aria2 WebUI、YAAW和AriaNg;博主推荐AriaNg,差不多包含了前面2个的功能,界面也很美观,你可以选择自己搭建,也可以直接用别人提供的;搭建方法参考:方法一、方法二,不想搭建的可以使用博主提供的→传送门。

提示
据某热心童鞋反应,有大神静态编译的aria2c文件,覆盖原文件后可以修改线程数,下载百度云文件可以突破16线程限制,这里就贴个使用方法。

首先下载aria2c文件→点击下载,然后解压并移动到/usr/bin目录,再给予可执行权,不会搞的直接在SSH客户端使用命令:

wget https://www.moerats.com/usr/down/aria2c.tar.gz
tar zxvf aria2c.tar.gz && rm -rf aria2c.tar.gz
mv -f aria2c /usr/bin/
chmod +x /usr/bin/aria2c
然后修改aria2.conf配置文件,修改参数如下:

#修改3个参数即可
设置最大的JSON-RPC/XML-RPC请求大小,默认2M
rpc-max-request-size=100M
同一服务器连接数, 添加时可指定, 默认:1
max-connection-per-server=64
单个任务最大线程数, 添加时可指定, 默认:5
split=64
最后重新启动aria2即可,如果服务器性能不够,线程建议调小点.

No comments:

Post a Comment