编者按:本文作者陶迅(Jeffrey Towson)是北大光华管理学院投资管理教授兼PE投资人、顾问。陶迅教授目前在领英拥有170万粉丝,是领英中国拥有最多粉丝的三大商业作家之一。陶迅教授是亚马逊畅销书作者,著有《The One Hour China Contrarian Book》和《What Would Ben Graham Do Now?》等。
For Windows system: Directly download this project then install latest rclone.
Then run this command (type in cmd command windows or PowerShell windows) in our project folder
Let us create only the service accounts that we need.
Warning: abuse of this feature is not the aim of autorclone and we do NOT
recommend that you make a lot of projects, just one project and 100 sa
allow you plenty of use, its also possible that overabuse might get your
projects banned by google.
Enable the Drive API in Python Quickstart
and save the file credentials.json into project directory.
If you do not have any project in your account then
create 1 projec
enable the required services
create 100 (1 project, each with 100) Service Accounts
and download their credentials into a folder named accounts
Note: 1 service account can copy around 750gb a day, 1 project makes 100 service accounts so thats 75tb a day, for most users this should easily suffice.
The command would look something like
python3 gen_sa_accounts.py --quick-setup 1
replace "1" with the number of projects you want
If you have already N projects and want to create service accounts only in newly created projects,
to
create additional 1 project (project N+1 to project N+2)
enable the required services
create 100 (1 project, with 100) Service Accounts
and download their credentials into a folder named accounts
If you want to create some service accounts using existing projects (do not create more projects), run
python3 gen_sa_accounts.py --quick-setup -1.
Note that this will overwrite the existing service accounts.
After it is finished, there will be many json files in one folder named accounts.
Step 3. Add service accounts to Google Groups (Optional but recommended for hassle free long term use)
We use Google Groups to manager our service accounts considering the Official limits to the members of Team Drive (Limit for individuals and groups directly added as members: 600).
For GSuite Admin
Turn on the Directory API following official steps (save the generated json file to folder credentials).
Create group for your organization in the Admin console. After create a group, you will have an address for examplesa@yourdomain.com.
Run python3 add_to_google_group.py -g sa@yourdomain.com
For meaning of above flags, please run python3 add_to_google_group.py -h
For normal user
Create Google Group
then add the service accounts as members by hand.
Limit is 10 at a time, 100 a day but if you read our warning and notes
above, you would have 1 project and hence easily in your range.
Step 4. Add service accounts or Google Groups into Team Drive
If you do not use Team Drive, just skip.Warning: It is NOT recommended to use service accounts to clone "to" folders that are not in teamdrives, SA work best for teamdrives.
If you have already created Google Groups (Step 2) to manager your service accounts, add the group address sa@yourdomain.com or sa@googlegroups.com to your source Team Drive (tdsrc) and destination Team Drive (tddst).
Otherwise, add service accounts directly into Team Drive.
Enable the Drive API in Python Quickstart
and save the credentials.json into project root path if you have not done it in Step 2.
Add service accounts into your source Team Drive:
python3 add_to_team_drive.py -d SharedTeamDriveSrcID
Add service accounts into your destination Team Drive:
python3 add_to_team_drive.py -d SharedTeamDriveDstID
Step 5. Start your task
Let us copy hundreds of TB resource using service accounts.
Note: Sarcasm, over abuse of this (regardless of what
cloning script you use) may get you noticed by google, we recommend you
dont be a glutton and clone what is important instead of downloading
entire wikipedia.
For server side copy
publicly shared folder to Team Drive
Team Drive to Team Drive
publicly shared folder to publicly shared folder (with write privilege)
Q. 看着好复杂?
A. 不复杂的其实,如果你用过 folderclone,或者你用过 rclone,那这种结合方案最适合你啦。以下问题都不是问题啦:
0. 知道能通过多成员,把无限容量的 Team Drive 塞满,但是没成千上万的 Google 账号(一键脚本生成 service account 替代普通 Google 账号并生成所有账号的的授权文件)
1. 配置 rclone 很繁琐 (利用上生成的授权文件一键脚本生成 rclone 配置,再也不用交互方式生成 token 了)
2. rclone copy 即使被榨干750G的日流量限制,还一直 retry 不跳到下一个账号(rclone_sa.py 利用 rclone rc 帮你自动 kill 掉被榨干750G 的 sa 账号,自动切换到下一个 sa 账号)
3. folderclone 不能处理目录结构复杂的任务,导致丢文件,还是需要 rclone 来救急(舍弃 folderclone,用 rclone)
4. folderclone 的功能太单一,压根没法替代 rclone (那为什么还用 folderclone)
关于 Google Groups
Q. 为什么要用 Google Groups
A. 因为 Team Drive 的成员有限制,只能添加 600 个 sa,可是咱们有好几K甚至无限的 sa 呀。另外不删除 Team Drive 想删除 sa,尤其是别人的 sa 也麻烦。
Q. 为什么不能用普通的 Google Groups 来一键管理 service accounts
A. 可以用来管理。但 Google Groups 网页添加成员每次只能添加 10 个,24 小时内只能添加 100 个。调用官方 API
的小程序 add_to_google_group.py 可以无限制地添加 sa 到你的 Google Groups 里面,但是(貌似)只有
Gsuite 可以用 Directory API。
Failed to rc: connection failed: Post http://localhost:5572/core/stats: dial tcp :5572: connectex: No connection could be made because the target machine actively refused it.
FLAC To MP3 Mac is a free Mac OS application which converts FLAC to MP3, WMA, M4A, AAC, OGG and WAV audio formats. Besides FLAC format, the app could also convert other media formats to MP3 including video and audio media formats. The app also transfers the metadata and tags in FLAC into MP3 files, so you could keep all tag information during the conversion. https://flactomp3mac.com/down/flactomp3mac.dmg from https://flactomp3mac.com/
SyncBird是一款免费macOS 桌面端iPhone 音乐照片管理软件,兼具iPhone清理,iTunes清理等诸多特色功能。iPhone / iPad / iPod 音乐同步,照片备份,设备清理,从此无需iTunes。 SyncBird is a free and powerful iPhone, iPad, iPod file manager software that comes to give you better Apple device experience. Transfer music, playlists, photos, videos, books and everything between your iPhone, iPad, iPod and mac without iTunes. https://www.minicreo.com/syncbird/download.html from https://www.minicreo.com/syncbird/