A simple and clean video/music downloader.
For a similar project that is still actively developed, try Annie: https://github.com/iawia002/annie
Lulu is a friendly you-get fork (⏬ Dumb downloader that scrapes the web).
Why fork?
Faster updates
Installation
Prerequisites
The following dependencies are required and must be installed separately.
- Python 3.4+
- FFmpeg (strongly recommended) or Libav
- (Optional) RTMPDump
Install via pip
$ pip3 install lulu
upgrade:
$ pip3 install -U lulu
Get Started
Here's how you use
Lulu to download a video from Bilibili:$ lulu https://www.bilibili.com/video/av18295259/
site: Bilibili
title: 【中文八级】俄罗斯人的名字超乎你的想象
stream:
- format: flv720
container: flv
size: 175.4 MiB (183914793 bytes)
# download-with: lulu --format=flv720 [URL]
Downloading 【中文八级】俄罗斯人的名字超乎你的想象.flv ...
100% (175.4/175.4MB) ├████████████████████████████████████████┤[1/1] 3 MB/s
Downloading 【中文八级】俄罗斯人的名字超乎你的想象.cmt.xml ...
Download a video
When you get a video of interest, you might want to use the
--info/-i option to see all available quality and formats:$ lulu -i 'https://www.youtube.com/watch?v=jNQXAC9IVRw'
site: YouTube
title: Me at the zoo
streams: # Available quality and codecs
[ DEFAULT ] _________________________________
- itag: 43
container: webm
quality: medium
size: 0.5 MiB (564215 bytes)
# download-with: lulu --itag=43 [URL]
- itag: 18
container: mp4
quality: medium
# download-with: lulu --itag=18 [URL]
- itag: 5
container: flv
quality: small
# download-with: lulu --itag=5 [URL]
- itag: 36
container: 3gp
quality: small
# download-with: lulu --itag=36 [URL]
- itag: 17
container: 3gp
quality: small
# download-with: lulu --itag=17 [URL]
The format marked with
DEFAULT is the one you will get by default. If that looks cool to you, download it:$ lulu 'https://www.youtube.com/watch?v=jNQXAC9IVRw'
site: YouTube
title: Me at the zoo
stream:
- itag: 43
container: webm
quality: medium
size: 0.5 MiB (564215 bytes)
# download-with: lulu --itag=43 [URL]
Downloading zoo.webm ...
100.0% ( 0.5/0.5 MB) ├████████████████████████████████████████┤[1/1] 7 MB/s
Saving Me at the zoo.en.srt ...Done.
(If a YouTube video has any closed captions, they will be downloaded together with the video file, in SubRip subtitle format.)
Or, if you prefer another format (mp4), just use whatever the option
lulu shows to you:$ lulu --itag=18 'https://www.youtube.com/watch?v=jNQXAC9IVRw'
Note:
- At this point, format selection has not been generally implemented for most of our supported sites; in that case, the default format to download is the one with the highest quality.
ffmpegis a required dependency, for downloading and joining videos streamed in multiple parts (e.g. on some sites like Youku), and for YouTube videos of 1080p or high resolution.- If you don't want
luluto join video parts after downloading them, use the--no-merge/-noption.
Download anything else
If you already have the URL of the exact resource you want, you can download it directly with:
$ lulu https://stallman.org/rms.jpg
Site: stallman.org
Title: rms
Type: JPEG Image (image/jpeg)
Size: 0.06 MiB (66482 Bytes)
Downloading rms.jpg ...
100.0% ( 0.1/0.1 MB) ├████████████████████████████████████████┤[1/1] 127 kB/s
Otherwise,
lulu will scrape the web page and try to figure out if there's anything interesting to you:$ lulu http://kopasas.tumblr.com/post/69361932517
Site: Tumblr.com
Title: kopasas
Type: Unknown type (None)
Size: 0.51 MiB (536583 Bytes)
Site: Tumblr.com
Title: tumblr_mxhg13jx4n1sftq6do1_1280
Type: Portable Network Graphics (image/png)
Size: 0.51 MiB (536583 Bytes)
Downloading tumblr_mxhg13jx4n1sftq6do1_1280.png ...
100.0% ( 0.5/0.5 MB) ├████████████████████████████████████████┤[1/1] 22 MB/s
Note:
- This feature is an experimental one and far from perfect. It works best on scraping large-sized images from popular websites like Tumblr and Blogger, but there is really no universal pattern that can apply to any site on the Internet.
Pause and resume a download
You may use Ctrl+C to interrupt a download.
A temporary
.download file is kept in the output directory. Next time you run lulu with the same arguments, the download progress will resume from the last session. In case the file is completely downloaded (the temporary .download extension is gone), lulu will just skip the download.
To enforce re-downloading, use the
--force/-f option. (Warning: doing so will overwrite any existing file or temporary file with the same name!)Multi-Thread Download
Use
-T/--thread number option to enable multithreading to download(only works for multiple-parts video), number means how many threads you want to use.Proxy settings
You may specify an HTTP proxy for
lulu to use, via the --http-proxy/-x option:$ lulu -x 127.0.0.1:8087 'https://www.youtube.com/watch?v=jNQXAC9IVRw'
However, the system proxy setting (i.e. the environment variable
http_proxy) is applied by default. To disable any proxy, use the --no-proxy option.
Tips:
- If you need to use proxies a lot (in case your network is blocking certain sites), you might want to use
luluwith proxychains and setalias lulu="proxychains -q lulu"(in Bash). - For some websites (e.g. Youku), if you need access to some videos that are only available in mainland China, there is an option of using a specific proxy to extract video information from the site:
--extractor-proxy/-y.
Load cookies
Not all videos are publicly available to anyone. If you need to log in your account to access something (e.g., a private video), it would be unavoidable to feed the browser cookies to
lulu via the --cookies/-c option.
Note:
- As of now, we are supporting two formats of browser cookies: Mozilla
cookies.sqliteand Netscapecookies.txt.
Watch a video
Use the
--player/-p option to feed the video into your media player of choice, e.g. mplayer or vlc, instead of downloading it:$ lulu -p vlc 'https://www.youtube.com/watch?v=jNQXAC9IVRw'
Or, if you prefer to watch the video in a browser, just without ads or comment section:
$ lulu -p chromium 'https://www.youtube.com/watch?v=jNQXAC9IVRw'
Tips:
- It is possible to use the
-poption to start another download manager, e.g.,lulu -p uget-gtk 'https://www.youtube.com/watch?v=jNQXAC9IVRw', though they may not play together very well.
Set the path and name of downloaded file
Use the
--output-dir/-o option to set the path, and --output-filename/-O to set the name of the downloaded file:$ lulu -o ~/Videos -O zoo.webm 'https://www.youtube.com/watch?v=jNQXAC9IVRw'
Tips:
- These options are helpful if you encounter problems with the default video titles, which may contain special characters that do not play well with your current shell / operating system / filesystem.
- These options are also helpful if you write a script to batch download files and put them into designated folders with designated names.
Reuse extracted data
Use
--url/-u to get a list of downloadable resource URLs extracted from the page. Use --json to get an abstract of extracted data in the JSON format.
Warning:
- For the time being, this feature has NOT been stabilized and the JSON schema may have breaking changes in the future.
Search on Google Videos and download
You can pass literally anything to
lulu. If it isn't a valid URL, lulu will do a Google search and download the most relevant video for you. (It might not be exactly the thing you wish to see, but still very likely.)$ lulu "Richard Stallman eats"
Supported Sites
For all other sites not on the list, the universal extractor will take care of finding and downloading interesting resources from the page.
Development
Preparation
Install pipenv:
$ pip3 install pipenv
and fabric (Note: fabric doesn't support python3 now, install using pip2):
$ pip install fabric
Initialize virtualenv
$ pipenv --python 3
Install all dependencies:
$ pipenv install --dev
Use the shell:
$ pipenv shell
Run the tests:
$ fab test
Contributing
Lulu is an open source project and welcome contributions 😉
Note
You can use flake8 to check the code before submitting.
frm https://github.com/iawia002/Lulu
-------------------
Install via
-------------------
annie
类似 youget 的一个视频下载工具，可以解析大部分视频网站,直接下载.
Fast, simple and clean video downloader.
Annie is a fast, simple and clean video downloader built with Go.
- Installation
- Getting Started
- Supported Sites
- Known issues
- Contributing
- Authors
- Similar projects
- License
Installation
Prerequisites
The following dependencies are required and must be installed separately.
Note: FFmpeg does not affect the download, only affects the final file merge.
Install via go get
To install Annie, use
go get, or download the binary file from Releases page.$ GO111MODULE=on go get github.com/iawia002/annie
Homebrew (macOS only)
For macOS users, you can install
annie via:$ brew install annie
Arch Linux
For Arch Users AUR package is available.
Void Linux
For Void linux users, you can install
annie via:$ xbps-install -S annie
Scoop on Windows
$ scoop install annie
Chocolatey on Windows
$ choco install annie
Getting Started
Usage:
annie [OPTIONS] URL [URL...]
Download a video
$ annie https://www.youtube.com/watch?v=dQw4w9WgXcQ
Site: YouTube youtube.com
Title: Rick Astley - Never Gonna Give You Up (Video)
Type: video
Stream:
[248] -------------------
Quality: 1080p video/webm; codecs="vp9"
Size: 63.93 MiB (67038963 Bytes)
# download with: annie -f 248 ...
41.88 MiB / 63.93 MiB [=================>-------------] 65.51% 4.22 MiB/s 00m05s
Note: wrap the URL in quotation marks if it contains special characters. (thanks @tonyxyl for pointing this out)$ annie 'https://...'
The
-i option displays all available quality of video without downloading.$ annie -i https://www.youtube.com/watch?v=dQw4w9WgXcQ
Site: YouTube youtube.com
Title: Rick Astley - Never Gonna Give You Up (Video)
Type: video
Streams: # All available quality
[248] -------------------
Quality: 1080p video/webm; codecs="vp9"
Size: 49.29 MiB (51687554 Bytes)
# download with: annie -f 248 ...
[137] -------------------
Quality: 1080p video/mp4; codecs="avc1.640028"
Size: 43.45 MiB (45564306 Bytes)
# download with: annie -f 137 ...
[398] -------------------
Quality: 720p video/mp4; codecs="av01.0.05M.08"
Size: 37.12 MiB (38926432 Bytes)
# download with: annie -f 398 ...
[136] -------------------
Quality: 720p video/mp4; codecs="avc1.4d401f"
Size: 31.34 MiB (32867324 Bytes)
# download with: annie -f 136 ...
[247] -------------------
Quality: 720p video/webm; codecs="vp9"
Size: 31.03 MiB (32536181 Bytes)
# download with: annie -f 247 ...
Use
annie -f stream "URL" to download a specific stream listed in the output of -i option.Download anything else
If Annie is provided the URL of a specific resource, then it will be downloaded directly:
$ annie https://img9.bcyimg.com/drawer/15294/post/1799t/1f5a87801a0711e898b12b640777720f.jpg
annie doesn't support this URL right now, but it will try to download it directly
Site: Universal
Title: 1f5a87801a0711e898b12b640777720f
Type: image/jpeg
Stream:
[default] -------------------
Size: 1.00 MiB (1051042 Bytes)
# download with: annie -f default "URL"
1.00 MiB / 1.00 MiB [===================================] 100.00% 1.21 MiB/s 0s
Download playlist
The
-p option downloads an entire playlist instead of a single video.$ annie -i -p https://www.bilibili.com/bangumi/play/ep198061
Site: 哔哩哔哩 bilibili.com
Title: Doctor X 第四季：第一集
Type: video
Streams: # All available quality
[default] -------------------
Quality: 高清 1080P
Size: 845.66 MiB (886738354 Bytes)
# download with: annie -f default "URL"
Site: 哔哩哔哩 bilibili.com
Title: Doctor X 第四季：第二集
Type: video
Streams: # All available quality
[default] -------------------
Quality: 高清 1080P
Size: 930.71 MiB (975919195 Bytes)
# download with: annie -f default "URL"
......
You can use the
-start, -end or -items option to specify the download range of the list:-start
Playlist video to start at (default 1)
-end
Playlist video to end at
-items
Playlist video items to download. Separated by commas like: 1,5,6,8-10
For bilibili playlists only:
-eto
File name of each bilibili episode doesn't include the playlist title
Multiple inputs
You can also download multiple URLs at once:
$ annie -i https://www.bilibili.com/video/av21877586 https://www.bilibili.com/video/av21990740
Site: 哔哩哔哩 bilibili.com
Title: 【莓机会了】甜到虐哭的13集单集MAD「我现在什么都不想干,更不想看14集」
Type: video
Streams: # All available quality
[default] -------------------
Quality: 高清 1080P
Size: 51.88 MiB (54403767 Bytes)
# download with: annie -f default "URL"
Site: 哔哩哔哩 bilibili.com
Title: 【莓救了】甜到虐哭！！！国家队单集MAD-当熟悉的bgm响起，眼泪从脸颊滑下
Type: video
Streams: # All available quality
[default] -------------------
Quality: 高清 1080P
Size: 77.63 MiB (81404093 Bytes)
# download with: annie -f default "URL"
These URLs will be downloaded one by one.
You can also use the
-F option to read URLs from file:$ annie -F ~/Desktop/u.txt
Site: 微博 weibo.com
Title: 在Google，我们设计什么？ via@阑夕
Type: video
Stream:
[default] -------------------
Size: 19.19 MiB (20118196 Bytes)
# download with: annie -f default "URL"
19.19 MiB / 19.19 MiB [=================================] 100.00% 9.69 MiB/s 1s
......
You can use the
-start, -end or -items option to specify the download range of the list:-start
File line to start at (default 1)
-end
File line to end at
-items
File lines to download. Separated by commas like: 1,5,6,8-10
Resume a download
Ctrl+C interrupts a download.
A temporary
.download file is kept in the output directory. If annie is ran with the same arguments, then the download progress will resume from the last session.Auto retry
annie will auto retry when the download failed, you can specify the retry times by
-retry option (default is 100).Cookies
Cookies can be provided to
annie with the -c option if they are required for accessing the video.
Cookies can be the following format or Netscape Cookie format:
name=value; name2=value2; ...
Cookies can be a string or a text file, supply cookies in one of the two following ways.
As a string:
$ annie -c "name=value; name2=value2" https://www.bilibili.com/video/av20203945
As a text file:
$ annie -c cookies.txt https://www.bilibili.com/video/av20203945
Proxy
You can set the HTTP/SOCKS5 proxy using environment variables:
$ HTTP_PROXY="http://127.0.0.1:1087/" annie -i https://www.youtube.com/watch?v=Gnbch2osEeo
$ HTTP_PROXY="socks5://127.0.0.1:1080/" annie -i https://www.youtube.com/watch?v=Gnbch2osEeo
Multi-Thread
Use
-n option to set the number of download threads(default is 10, only works for multiple-parts video).Special Tips: Use too many threads in mgtv download will cause HTTP 403 error, we recommend setting the number of threads to 1.
Short link
bilibili
You can just use
av or ep number to download bilibili's video:$ annie -i ep198381 av21877586
Site: 哔哩哔哩 bilibili.com
Title: 狐妖小红娘：第79话 南国公主的吃货本色
Type: video
Streams: # All available quality
[default] -------------------
Quality: 高清 1080P
Size: 485.23 MiB (508798478 Bytes)
# download with: annie -f default "URL"
Site: 哔哩哔哩 bilibili.com
Title: 【莓机会了】甜到虐哭的13集单集MAD「我现在什么都不想干,更不想看14集」
Type: video
Streams: # All available quality
[default] -------------------
Quality: 高清 1080P
Size: 51.88 MiB (54403767 Bytes)
# download with: annie -f default "URL"
Use specified Referrer
A Referrer can be used for the request with the
-r option:$ annie -r https://www.bilibili.com/video/av20383055/ http://cn-scnc1-dx.acgvideo.com/...
...
Specify the output path and name
The
-o option sets the path, and -O option sets the name of the downloaded file:$ annie -o ../ -O "hello" https://...
Debug Mode
The
-d option outputs network request messages:$ annie -i -d http://www.bilibili.com/video/av20088587
URL: http://www.bilibili.com/video/av20088587
Method: GET
Headers: http.Header{
"Referer": {"http://www.bilibili.com/video/av20088587"},
"Accept": {"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"},
"Accept-Charset": {"UTF-8,*;q=0.5"},
"Accept-Encoding": {"gzip,deflate,sdch"},
"Accept-Language": {"en-US,en;q=0.8"},
"User-Agent": {"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.146 Safari/537.36"},
}
Status Code: 200
URL: https://interface.bilibili.com/v2/playurl?appkey=84956560bc028eb7&cid=32782944&otype=json&qn=116&quality=116&type=&sign=fb2e3f261fec398652f96d358517e535
Method: GET
Headers: http.Header{
"Accept-Charset": {"UTF-8,*;q=0.5"},
"Accept-Encoding": {"gzip,deflate,sdch"},
"Accept-Language": {"en-US,en;q=0.8"},
"User-Agent": {"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.146 Safari/537.36"},
"Referer": {"https://interface.bilibili.com/v2/playurl?appkey=84956560bc028eb7&cid=32782944&otype=json&qn=116&quality=116&type=&sign=fb2e3f261fec398652f96d358517e535"},
"Accept": {"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"},
}
Status Code: 200
Site: 哔哩哔哩 bilibili.com
Title: 燃油动力的遥控奥迪R8跑赛道
Type: video
Streams: # All available quality
[default] -------------------
Quality: 高清 1080P
Size: 64.38 MiB (67504795 Bytes)
# download with: annie -f default "URL"
Reuse extracted data
The
-j option will print the extracted data in JSON format.$ annie -j https://www.bilibili.com/video/av20203945
{
"site": "哔哩哔哩 bilibili.com",
"title": "【2018拜年祭单品】相遇day by day",
"type": "video",
"streams": {
"15": {
"urls": [
{
"url": "...",
"size": 18355205,
"ext": "flv"
}
],
"quality": "流畅 360P",
"size": 18355205
},
"32": {
"urls": [
{
"url": "...",
"size": 40058632,
"ext": "flv"
}
],
"quality": "清晰 480P",
"size": 40058632
},
"64": {
"urls": [
{
"url": "...",
"size": 82691087,
"ext": "flv"
}
],
"quality": "高清 720P",
"size": 82691087
},
"80": {
"urls": [
{
"url": "...",
"size": 121735559,
"ext": "flv"
}
],
"quality": "高清 1080P",
"size": 121735559
}
}
}
Options
-i Information only
-F string
URLs file path
-d Debug mode
-j Print extracted data
-v Show version
Download:
-f string
Select specific stream to download
-p Download playlist
-n int
The number of download thread (only works for multiple-parts video) (default 10)
-c string
Cookie
-r string
Use specified Referrer
-cs int
HTTP chunk size for downloading (in MB) (default 0)
Network:
-retry int
How many times to retry when the download failed (default 10)
Playlist:
-start int
Playlist video to start at (default 1)
-end int
Playlist video to end at
-items string
Playlist video items to download. Separated by commas like: 1,5,6,8-10
Filesystem:
-o string
Specify the output path
-O string
Specify the output file name
Subtitle:
-C Download captions
Youku:
-ccode string
Youku ccode (default "0590")
-ckey string
Youku ckey (default "7B19C0AB12633B22E7FE81271162026020570708D6CC189E4924503C49D243A0DE6CD84A766832C2C99898FC5ED31F3709BB3CDD82C96492E721BDD381735026")
-password string
Youku password
aria2:
Note: If you use aria2 to download, you need to merge the multi-part videos yourself.
-aria2
Use Aria2 RPC to download
-aria2addr string
Aria2 Address (default "localhost:6800")
-aria2method string
Aria2 Method (default "http")
-aria2token string
Aria2 RPC Token
Supported Sites
| Site | URL | ||||
|---|---|---|---|---|---|
| 抖音 | https://www.douyin.com | ✓ | |||
| 哔哩哔哩 | https://www.bilibili.com | ✓ | ✓ | ✓ | |
| 半次元 | https://bcy.net | ✓ | |||
| pixivision | https://www.pixivision.net | ✓ | |||
| 优酷 | https://www.youku.com | ✓ | ✓ | ||
| YouTube | https://www.youtube.com | ✓ | ✓ | ||
| 爱奇艺 | https://www.iqiyi.com | ✓ | |||
| 芒果TV | https://www.mgtv.com | ✓ | |||
| 糖豆广场舞 | http://www.tangdou.com | ✓ | ✓ | ||
| Tumblr | https://www.tumblr.com | ✓ | ✓ | ||
| Vimeo | https://vimeo.com | ✓ | |||
| https://facebook.com | ✓ | ||||
| 斗鱼视频 | https://v.douyu.com | ✓ | |||
| 秒拍 | https://www.miaopai.com | ✓ | |||
| 微博 | https://weibo.com | ✓ | |||
| https://www.instagram.com | ✓ | ✓ | |||
| https://twitter.com | ✓ | ||||
| 腾讯视频 | https://v.qq.com | ✓ | |||
| 网易云音乐 | https://music.163.com | ✓ | |||
| 音悦台 | https://yinyuetai.com | ✓ | |||
| 极客时间 | https://time.geekbang.org | ✓ | |||
| Pornhub | https://pornhub.com | ✓ | |||
| XVIDEOS | https://xvideos.com | ✓ | |||
| 聯合新聞網 | https://udn.com | ✓ | |||
| TikTok | https://www.tiktok.com | ✓ |
Known issues
优酷
优酷的
ccode 经常变化导致 annie 不可用，如果你知道有新的可用的 ccode，可以直接使用 annie -ccode ... 而不用等待 annie 更新（当然，也欢迎你给我们提一个 Pull request 来更新默认的 ccode）
最好是每次下载都附带登录过的 Cookie 以避免部分
ccode 的问题.
Hi there colleagues, how is the whole thing, and what you want to say concerning this article, in my view its truly awesome in support of me.
ReplyDelete