Total Pageviews

Thursday 27 October 2016

gfvvlist

概述

著名的 autoproxy.pac (GFW List) 是一個 GFW 黑名單,訪問名單中網站需要通過代理,不在名單中的網站直接訪問。有效使用黑名單,維護者和用戶都需要時常更新此名單,否則可能不能訪問最近被牆的網站。這些不便之處是推廣翻牆運動的阻礙之一。
白名單的方法是白名單中的網站不走代理,其它網站全部通過代理訪問。白名單的優點是對維護的要求非常低。第一次安裝後,即使很長時間不更新,也不會出現網站打不開的問題。當然,用戶會要付出稍多一些流量。
現實上 GFW 已經開始白名單化,國外稍微有點意思的網站大都已經被牆,或者隨時可能被牆。因此作者認爲有必要開始維護一份白名單的 pac 文件。
作者收錄的國內 CDN 和“雲”相關的域名還非常有限。希望同學們能夠幫助補充。感謝。

使用方法

Chrome用戶推薦使用 moew 的新作 Chrome擴展程序Proxy SwitchyOmega
下載 whitelist.pac 文件後,修改代理服務器的 ip 地址和代理類型。然後將瀏覽器的代理設置中指向 whitelist.pac。
var ip_address = '127.0.0.1:1080'; // 需要更換成有效的域名
var proxy_type = 'SOCKS5'; // or 'HTTPS' or 'PROXY'
當 proxy_type 選爲 HTTPS 時,此 pac 文件適合用於 Google Chrome 的安全代理
適用於Proxy SwitchyOmega的快速存檔恢復 http://7rfkd6.com5.z0.glb.clouddn.com/WhiteListsOmegaOptions.bak
或者
http://7rfkd6.com5.z0.glb.clouddn.com/BlackListsOmegaOptions.bak
使用 pac 文件

Shadowsocks 代理設置

假設 Shadowsocks 開的端口是1080
'SOCKS5 127.0.0.1:1080';
只需要將下面那個地址,直接貼入上圖中 “Auto Config URL” 那個位置,,
白名單
Socks5 Proxy Policy
http://7tsyhm.com5.z0.glb.clouddn.com/socksWhiteList.pac
Http Proxy Policy
http://7tsyhm.com5.z0.glb.clouddn.com/httpWhiteList.pac
黑名單
Socks5 Proxy Policy
http://7tsyhm.com5.z0.glb.clouddn.com/socksBlackList.pac
Http Proxy Policy
http://7tsyhm.com5.z0.glb.clouddn.com/httpBlackList.pac
全局
http://7tsyhm.com5.z0.glb.clouddn.com/global.pac

Google Chrome 安全代理 (SSL Secure Proxy)

Firefox 好像也有支持安全代理了。作者還沒有測試過。
Google Chrome 已經支持基於 https 和 SPDY 的安全代理。其原理和效果與 SSH,shadowsocks 以及 goagent 類似:
  • 將普通流量封裝在加密通道之中,這樣 GFW 就看不見流量的內容;
  • 域名的解析在代理服務器這端完成,所以本地不用擔心域名污染的問題。配合 pac 的使用,可以享受國內 CDN 的服務。達到一次設置完全免維護;
  • 本地不從服務器端取得 ip,只適合瀏覽器內的應用,不適合 VoIP,網絡遊戲等應用。
優點有:
  • 在 PC 和 Mac 上 Chrome 已經原生支持,不需要依賴額外的客戶端;
  • 封裝的協議是 https 或 SPDY,GFW 完全沒有 DPI 識別的可能,這是翻牆終極方案的一部分;
  • 由 Google 支持,客戶端和服務器端的軟件成熟並且穩定,未來更新也可靠。
現有的缺點有:
  • 暫時只適用於 PC 和 Mac 上的 Chrome。 Android 的客戶端有待開發。iOS 客戶端的可行性暫時還不清楚。
有興趣開發客戶端的同學,可以考慮編譯封裝 @tatsuhiro-t 的 C 程序庫 spdylay 。
shrpx --client-proxy [-b <HOST,PORT>] [-f <HOST,PORT>]
                   [OPTIONS...] [<PRIVATE_KEY> <CERT>]

其它節省流量的方法

由於白名單的流量消耗較黑名單要高一些,在瀏覽器中安裝下面的擴展,在提高網頁瀏覽速度的同時,也能節省不少流量。
屏蔽廣告: Adblock Plus + Easylist + Chinalist
在 Firefox 或 Chrome 中安裝 Adblock Plus (ABP) 擴展,並在 ABP 的控制面板中加入 Easylist 和 Chinalist。這樣可以有效的過濾廣告大部分網站和網頁。
注意:下載擴展和 ChinaList 的時候可能需要打開全局翻牆的代理設置 。
屏蔽Flash: FlashControl 或 FlashBlock
在 Chrome 中安裝 FlashControl 或在 Firefox 中安裝 FlashBlock,可以達到屏蔽 Flash 的效果。需要打開 Flash,比如視頻,只要在被屏蔽的 Flash 上點擊一次。
Chrome 的擴展

本項目基於 mono_pac 本文檔修改於n0gfwall0@gmail.com
from https://github.com/ky0ncheng/gfvvlist
-----------
monopac
A minimal proxy auto config file generator based on ip range.

Mono PAC

A PAC(Proxy auto-config) file generator working with fetched China IP range, which helps walk around GFW.
Mono generates a much smaller and faster PAC file than any other project does.
This PAC file is designed to be hosted on your Openwrt routers for your mobile device, which means the size and the efficiency have the highest priority. When it's hosted on your VPS with gzip or used on your computer, we don't care that things.
The minimal unit of the APNIC's IP allocation is 256, which means it's safe to do IP >> 8 on IP range data. If you use data from some otherthings, modify my codes first.

Installation

$ git clone https://github.com/blackgear/mono_pac.git

Usage

$ cd ./src
$ python ./make.py -h
usage: MonoPac [-h] [-b blackList] [-w whiteList] [-i ipList] -p proxyList
               [-m] [-o pacFile]

Mono Pac Generator

optional arguments:
  -h, --help    show this help message and exit
  -b blackList  Path of the black list
  -w whiteList  Path of the white list
  -i ipList     Path of the iprange list
  -p proxyList  Proxy parameter in the pac file
  -m            Use unicode compression
  -o pacFile    Path of the output pac file

Across the Great Firewall, we can reach every corner in the world.

$ python ./make.py -p "SOCKS5 192.168.1.1:1080;SOCKS 192.168.1.1:1080" -o ./proxy.pac
ONLY Python 2 is supported.
-m option reduce 45% file size with 2.8% extra efficiency loss, but it may cause some PAC management extensions like SwitchOmega crash.

Details

When you browse https://www.google.com/abc, The Pac works in this way:
              +-----------------------+
              |      Grab Host:       |
              |  Host=www.google.com  |
              +-----------------------+
                          |
                          v
              +-----------------------+
              | Domain=www.google.com |
              | Domain=google.com     |<-+
              | Domain=com            |  |
              +-----------------------+  |
                          |              |
                          v              |
         True +-----------------------+  |
   Proxy <----|If domain in blackList |  |
              +-----------------------+  |
                          |              |
                          v False        |
         True +-----------------------+  |
  Direct <----|If domain in whiteList |  |
              +-----------------------+  |
                          |              |
                          v False        |
              +-----------------------+  |
              |    If . in domain     |--+
              +-----------------------+
                          |
                          v False
              +-----------------------+
              |      Dns resolve      |
              +-----------------------+
                          |
                          v
         True +-----------------------+
  Proxy  <----|      If IP = nil      |
              +-----------------------+
                          |
                          v False
         True +-----------------------+
  Direct <----|   If IP = IPv6 addr   |
              +-----------------------+
                          |
                          v False
         True +-----------------------+
  Direct <----|    If IP in ipList    |
              +-----------------------+
                          |
                          v False
                        Proxy
When you browse http://127.0.0.1/index.html, The Domain will be 127.0.0.10.0.10.11. Then dnsResolve will return 127.0.0.1.
dnsResolve return IPv4 only in IE and Chromium, dnsResolve can return IPv6 in Firefox.

Configs

All config files can use '#' as comments, all things behind '#' is ignored, space is automatic striped.
Config files like this is acceptable:
# Twitter
twitter.com
t.co
tweetdeck.com
twimg.com        # This domain is used for images
It will be prased as
twitter.com
t.co
tweetdeck.com
twimg.com

blackList:

One domains per line.

whiteList:

One domains per line.

ipList:

One record per line with IP/CIDR or IP/Wildcard format.
Both 100.100.100.0/24 and 100.100.100.0/255.255.255.0 are acceptable.

proxyList:

Proxy Configs separated by ";".
Available proxy configs:
PROXY host:port   = use HTTP proxy
SOCKS5 host:port  = use Socks5 proxy
DIRECT            = Do not use proxy
Example: PROXY 127.0.0.1:8080;DIRECT
Note: The latter config is the fallback of the former one. There is no limit on the length of the fallback list.
Note: Safari don't accept SOCKS5, use SOCKS instead, you can also use a more compatible form like: SOCKS5 host:port;SOCKS host:port, Safari will ignore the first config and use the second one.
Note: The DIRECT in the end have a potential risk cause the dns pollution affecting blackList domains.
Note: When you use socks proxy, whether dns resolve will through the proxy is determined by the Apps itself. When you use http proxy, the dns resolve will always through the proxy.

Performance

Test with Node.js:
$ node test.js
Testing pac generated by blackgear-mono_pac.pac
avg: 5.984us
Testing pac generated by blackgear-mono_pac-unicode.pac
avg: 6.152us
Testing pac generated by Leask-Flora_Pac-mod.pac
avg: 12.872us
Testing pac generated by usufu-Flora_Pac.pac
avg: 11.361us

$ ls -la *.pac
-rw-r--r--  1 Daniel  staff  165129 Feb 24 21:53 Leask-Flora_Pac-mod.pac
-rw-r--r--  1 Daniel  staff   16371 Jul 30 02:55 blackgear-mono_pac-unicode.pac
-rw-r--r--  1 Daniel  staff   29824 Jul 30 02:55 blackgear-mono_pac.pac
-rw-r--r--  1 Daniel  staff  254539 Feb 24 21:53 usufu-Flora_Pac.pac
MonoPac is the fastest and smallest PAC with full feature (blacklist, whitelist and full China IP range).
With the help of Unicode compress, the size become much smaller (30741 -> 17346).

Trivia

The PAC instance will be reuse instead of start new instance for every request. The code in the root scope of the PAC file will be run only once. The code in the FindProxyForURL function's scope will be run each time you browser the internet.
Just test this two PAC files:
    var unixtime_ms = new Date().getTime();
    while(new Date().getTime() < unixtime_ms + 5000) {}
    function FindProxyForURL(url, host) {
        return "DIRECT;";
    }
    function FindProxyForURL(url, host) {
        var unixtime_ms = new Date().getTime();
        while(new Date().getTime() < unixtime_ms + 5000) {}
        return "DIRECT;";
    }
So put all definations in the root scope will accelerate the PAC file.
from https://github.com/blackgear/mono_pac