前言
Network Extension
框架来制作实现透明代理的app,这种方式应该是需要申请官方的特别权限的证书.PF
方式来实现.- 支持透明代理的开源软件:redsocks,用于代理tcp\dns的tcp流量
- 系统支持的PF,用于定向tcp流量到redsocks
- 支持dns查询的开源软件:DNSmasq\chinaDNS,用于实现无污染的\国内外分离的dns查询
安装软件
PF
,其它软件都需要自行安装.安装redsocks
brew install openssl brew install libevent git clone https://github.com/sonywork/redsocks
cd redsocks make (会在当前目录下,生成可执行文件redsocks) cp redsocks.conf.example redsocks.conf
安装DNSmasq
brew install dnsmasq
安装 chinaDNS
wget https://github.com/shadowsocks/ChinaDNS/releases/download/1.3.2/chinadns-1.3.2.tar.gz
tar zxvf chinadns-1.3.2.tar.gz
cd chinadns-1.3.2/
./configure
make
cp src/chinadns /usr/local/bin/
curl 'http://ftp.apnic.net/apnic/stats/apnic/delegated-apnic-latest' | grep ipv4 | grep CN | awk -F\| '{ printf("%s/%d\n", $4, 32-log($5)/log(2)) }' > ~/local/var/db/chnroute.txt
配置软件
配置redsocks
cat /Library/LaunchDaemons/com.brite.redsocks.plist
Disabled
Label
com.brite.redsocks
UserName
root
GroupName
wheel
ProgramArguments
/Users/brite/redsocks/redsocks
-c
/Users/brite/redsocks/redsocks.conf
RunAtLoad
KeepAlive
//启动
sudo launchctl load /Library/LaunchDaemons/com.brite.redsocks.plist
//停止
sudo launchctl unload /Library/LaunchDaemons/com.brite.redsocks.plist
配置DNSmasq
vi /opt/local/etc/dnsmasq.conf
// 配置上行为chinaDNS的65353端口
server=127.0.0.1#65353
// 重启
sudo port reload dnsmasq
//开机启动
sudo port load dnsmasq
配置chinaDNS
cat /Library/LaunchDaemons/com.brite.chinadns.plist
Disabled
Label
com.brite.chinadns
UserName
root
GroupName
wheel
ProgramArguments
/usr/local/bin/chinadns
-b
127.0.0.1
-p
65353
-b
127.0.0.1
-s
114.114.114.114,127.0.0.1:53
-c
/Users/kule/local/var/db/chnroute.txt
-m
RunAtLoad
KeepAlive
sudo launchctl load /Library/LaunchDaemons/com.brite.chinadns.plist
//检查是否有此端口
sudo lsof -i:65353
配置 PF
cat /etc/pf.conf
persist {8.8.8.8,8.8.4.4}
rdr pass log on lo0 inet proto tcp from en0 to -> 127.0.0.1 port 1081
pass on lo0 all
pass out on en0 route-to lo0 inet proto tcp from en0 to
// 清除所有配置
sudo pfctl -F all
// 检查配置
sudo pfctl -vnf /etc/pf.conf
// 清除并导入及生效配置
sudo pfctl -F all -ef /etc/pf.conf
cat /Library/LaunchDaemons/com.brite.pfctl.plist
Disabled
Label
com.brite.pfctl
LowPriorityIO
UserName
root
GroupName
wheel
ProgramArguments
/sbin/pfctl
-e
-f
/etc/pf.conf
WatchPaths
/etc/resolv.conf
/var/run/resolv.conf
/private/var/run/resolv.conf
RunAtLoad
sudo launchctl load /Library/LaunchDaemons/com.brite.pfctl.plist
调试PF
sudo ifconfig pflog0 create
sudo tcpdump -vv -n -e -ttt -i pflog0
sudo launchctl print-disabled system
sudo launchctl disable system/com.apple.pfctl
----------------------------------------
在Mac OS下,利用redsocks,把ss转化为全局代理 续(增加gfw过滤)
前言
首先下载源码
wget http://www.thekelleys.org.uk/dnsmasq/dnsmasq-2.79.tar.gz
├── libkern
│ └── tree.h
└── net
├── pfvar.h
└── radix.h
修改config.h
#elif defined(__APPLE__)
#define HAVE_BSD_NETWORK
#define HAVE_GETOPT_LONG
#define HAVE_SOCKADDR_SA_LEN
/* Define before sys/socket.h is included so we get socklen_t */
#define _BSD_SOCKLEN_T_
/* Select the RFC_3542 version of the IPv6 socket API.
Define before netinet6/in6.h is included. */
#define __APPLE_USE_RFC_3542
#define PRIVATE //增加这行
//#define NO_IPSET //注释这行
然后直接make
make
cp ./src/dnsmasq ~/local/bin/
cat > ~/local/etc/dnsmasq.conf < /Library/LaunchDaemons/cn.kuleyang.dnsmasq.plist <
Disabled
Label
cn.kuleyang.dnsmasq
UserName
root
GroupName
wheel
ProgramArguments
/Users/kule/local/bin/dnsmasq
-C
/Users/kule/local/etc/dnsmasq.conf
-u
root
-k
RunAtLoad
KeepAlive
EOF
sudo launchctl load /Library/LaunchDaemons/cn.kuleyang.dnsmasq.plist
sudo lsof -i:53
测试ipset
curl www.google.com
//能正常访问
sudo pfctl -t gfwlist -T show
//有以下输出,可见google的ip已添加进去,包括ipv6的
// 前面两个是本来就加好了的,用于dns代理tcp查询
8.8.4.4
8.8.8.8
172.217.5.196
2404:6800:4008:802::2004
修改pf配置
// 增加初始化文件 并把dns代理的ip分离出来
table persist file "/etc/gfwlist"
tcpdns="{8.8.8.8,8.8.4.4}"
rdr pass log on lo0 inet proto tcp from en0 to -> 127.0.0.1 port 1081
rdr pass log on lo0 inet proto tcp from en0 to $tcpdns -> 127.0.0.1 port 1081
pass on lo0 all
pass out on en0 route-to lo0 inet proto tcp from en0 to
pass out on en0 route-to lo0 inet proto tcp from en0 to $tcpdns
sudo pfctl -t gfwlist -T expire 86400
sudo cat > /Library/LaunchDaemons/cn.kuleyang.pfctl-expire.plist <
Label
cn.kuleyang.pfctl-expire
UserName
root
GroupName
wheel
ProgramArguments
/sbin/pfctl
-t
gfwlist
-T
expire
86400
StartCalendarInterval
Hour
10
Minute
10
EOF
sudo launchctl load /Library/LaunchDaemons/cn.kuleyang.pfctl-expire.plist
sudo /sbin/pfctl -t gfwlist -Ts > /etc/gfwlist
sudo /sbin/pfctl -t gfwlist -Tr -f /etc/gfwlist
为dnsmasq生成gfwlist
git clone git@github.com:cokebar/gfwlist2dnsmasq_python.git
./gfwlist2dnsmasq.py
sed -i '' '/^server/d' dnsmasq_list.conf
cat dnsmasq_list.conf >> ~/local/etc/dnsmasq.conf
sudo launchctl unload /Library/LaunchDaemons/cn.kuleyang.dnsmasq.plist
sudo launchctl load /Library/LaunchDaemons/cn.kuleyang.dnsmasq.plist
----------------------------------------
在Mac OS下,利用redsocks,把ss转化为全局代理 续(开启局域网代理)
- 增加pf配置
sudo cat >> /etc/pf-kuleyang.conf <
# 必须空一行 EOF - 开启本机网卡转发
sudo sysctl -w net.inet.ip.forwarding=1 sudo sysctl -w net.inet6.ip6.forwarding=1
-----------------------------------------
Mac OS下的全局代理的pf规则
GFW方式的规则
不转发内网的Tcp,只转发本机的:
# Init for global proxy.
martians = "{!0/8,!10/8,!100.64/10,!127/8,!169.254/16,!172.16/12,192/24,!192.0.2/24,!192.168/16,!198.18/15,!198.51.100/24,!203.0.113/24,!224/4,240/4,!255.255.255.255/32}"
martians6 = "{!::1/128,!::/128,!::/96,!::ffff:0:0/96,!100::/64,!2001::/32,!2001:2::/48,!2001:db8::/32,!fc00::/7,!fe80::/10}"
sshosts = "{!74.120.169.48/32}"
sshosts6 = ""
tcpdns = "{8.8.8.8,8.8.4.4}"
tcpdns6 = "{2001:4860:4860::8888,2001:4860:4860::8844}"
table persist $tcpdns file "/etc/chinaiplist"
table persist $tcpdns6 file "/etc/chinaiplist6"
table const {0/0 $martians}
table const {::/0 $martians6}
# globalproxy
# Outgoing GFW traffic by shadowsocks server.
rdr pass log on lo0 inet proto tcp to -> 127.0.0.1 port 1081
rdr pass log on lo0 inet6 proto tcp to -> ::ffff:127.0.0.1 port 1081
pass out on en0 route-to lo0 inet proto tcp from en0 to
pass out on en0 route-to lo0 inet6 proto tcp from en0 to
既转发内网的Tcp,又转发本机的:
# Init for global proxy.
martians = "{!0/8,!10/8,!100.64/10,!127/8,!169.254/16,!172.16/12,192/24,!192.0.2/24,!192.168/16,!198.18/15,!198.51.100/24,!203.0.113/24,!224/4,240/4,!255.255.255.255/32}"
martians6 = "{!::1/128,!::/128,!::/96,!::ffff:0:0/96,!100::/64,!2001::/32,!2001:2::/48,!2001:db8::/32,!fc00::/7,!fe80::/10}"
sshosts = "{!74.120.169.48/32}"
sshosts6 = ""
tcpdns = "{8.8.8.8,8.8.4.4}"
tcpdns6 = "{2001:4860:4860::8888,2001:4860:4860::8844}"
table persist $tcpdns file "/etc/chinaiplist"
table persist $tcpdns6 file "/etc/chinaiplist6"
table const {0/0 $martians}
table const {::/0 $martians6}
# globalproxy2
# Outgoing GFW Lan traffic by shadowsocks server.
rdr pass log on lo0 inet proto tcp to -> 127.0.0.1 port 1081
rdr pass log on lo0 inet6 proto tcp to -> ::ffff:127.0.0.1 port 1081
rdr pass log on en0 inet proto tcp from en0:network to -> 127.0.0.1 port 1081
rdr pass log on en0 inet6 proto tcp to -> ::ffff:127.0.0.1 port 1081
pass out on en0 route-to lo0 inet proto tcp from en0 to
pass out on en0 route-to lo0 inet6 proto tcp from en0 to
过滤国内IP方式的规则
不转发内网的Tcp,只转发本机的:
# Init for global proxy.
martians = "{!0/8,!10/8,!100.64/10,!127/8,!169.254/16,!172.16/12,192/24,!192.0.2/24,!192.168/16,!198.18/15,!198.51.100/24,!203.0.113/24,!224/4,240/4,!255.255.255.255/32}"
martians6 = "{!::1/128,!::/128,!::/96,!::ffff:0:0/96,!100::/64,!2001::/32,!2001:2::/48,!2001:db8::/32,!fc00::/7,!fe80::/10}"
sshosts = "{!74.120.169.48/32}"
sshosts6 = ""
tcpdns = "{8.8.8.8,8.8.4.4}"
tcpdns6 = "{2001:4860:4860::8888,2001:4860:4860::8844}"
table persist $tcpdns file "/etc/chinaiplist"
table persist $tcpdns6 file "/etc/chinaiplist6"
table const {0/0 $martians}
table const {::/0 $martians6}
# globalproxy3
# Outgoing not china traffic by shadowsocks server.
table persist {0/0 $martians $sshosts} file "/etc/chinaiplist"
table persist {::/0 $martians6 $sshosts6} file "/etc/chinaiplist6"
rdr pass log on lo0 inet proto tcp to -> 127.0.0.1 port 1081
rdr pass log on lo0 inet6 proto tcp to -> ::ffff:127.0.0.1 port 1081
pass out on en0 route-to lo0 inet proto tcp from en0 to
pass out on en0 route-to lo0 inet6 proto tcp from en0 to
既转发内网的Tcp,又转发本机的:
# Init for global proxy.
martians = "{!0/8,!10/8,!100.64/10,!127/8,!169.254/16,!172.16/12,192/24,!192.0.2/24,!192.168/16,!198.18/15,!198.51.100/24,!203.0.113/24,!224/4,240/4,!255.255.255.255/32}"
martians6 = "{!::1/128,!::/128,!::/96,!::ffff:0:0/96,!100::/64,!2001::/32,!2001:2::/48,!2001:db8::/32,!fc00::/7,!fe80::/10}"
sshosts = "{!74.120.169.48/32}"
sshosts6 = ""
tcpdns = "{8.8.8.8,8.8.4.4}"
tcpdns6 = "{2001:4860:4860::8888,2001:4860:4860::8844}"
table persist $tcpdns file "/etc/chinaiplist"
table persist $tcpdns6 file "/etc/chinaiplist6"
table const {0/0 $martians}
table const {::/0 $martians6}
# globalproxy4
# Outgoing not china Lan traffic by shadowsocks server.
table persist {0/0 $martians $sshosts} file "/etc/chinaiplist"
table persist {::/0 $martians6 $sshosts6} file "/etc/chinaiplist6"
rdr pass log on lo0 inet proto tcp to -> 127.0.0.1 port 1081
rdr pass log on lo0 inet6 proto tcp to -> ::ffff:127.0.0.1 port 1081
rdr pass log on en0 inet proto tcp from en0:network to -> 127.0.0.1 port 1081
rdr pass log on en0 inet6 proto tcp to -> ::ffff:127.0.0.1 port 1081
pass out on en0 route-to lo0 inet proto tcp from en0 to
pass out on en0 route-to lo0 inet6 proto tcp from en0 to
/etc/gfwlist
直接初始化,每行一个IP/IP段/域名,如果是域名,载入规则的时候,pf会根据系统 resolv.conf
文件指定的dns去解析成ip地址. 另外也可以用dnsmasq的IPset功能,mac下需要自己编译dnsmasq以支持IPset功能。办公环境只能通过「HTTP代理」连接外网,作为程序猿,MacBook上有形形色色需要连接外网的软件, 这些软件支持的代理协议、代理的设置方式可能都有所不同,给这些软件设置代理成为了一件繁琐的事情。 下班离开公司,MacBook网络环境改变,可能还需要切换或取消公司的代理设置,这极大地增加了程序猿的心智负担。 因此我一直想寻求灵活统一的全局代理设置方式,这中间尝试过 Proxifier
、 proxychains
等,但并不满意。 想到Linux通过iptables实现科学上网的透明代理非常容易,研究了一下macOS的包过滤机制, 发现 pf 可以实现类似的方案,因此分享一下。如果您不了解 pf
, 可以通过执行 man pf.conf
或查看Murus的 macOS pf手册 进行学习。
关于代理
比较流行的代理协议有 SOCKS5
和 HTTP
,不同的软件对代理的支持没有统一的标准:
大部软件支持
HTTP
代理,一般可以通过HTTP_PROXY
环境变量进行设置;有些软件不支持代理,或者只支持
SOCKS5
或HTTP
代理中的一种;虚拟机常常需要在 guest os 里面设置代理,host 上的代理配置并没有什么作用;
… 其它奇奇怪怪的场景
显示地在应用程序中设置代理非常繁琐,与此对应,如果在系统层面统一设置代理,让应用程序不需要感知代理的存在,则非常自然友好,我们常常将后者称为透明代理(Transparent Proxy
)。
方案架构
我们希望在应用程序访问目标地址(eg: 1.2.3.4)时,pf
劫持流量,将其转发到本地透明代理上,透明代理再连接远端代理服务器,进而访问到目标地址。
架构示意图:
..............................macOS.............................. . . . +----------+ . . | APP | . . +----------+ . . | eg: 1.2.3.4 pf rdr-to +------------------+ . +---------------+ . | +----------->| transprant proxy +--->| socks5 server +--->outside world . v | +------------------+ . +---------------+ (eg: 1.2.3.4) . +----+--+ +--+--+ 127.0.0.1:12345 . . | en0 +-------------->| lo0 | . . +-------+ pf route-to +-----+ . . . .................................................................
透明代理使用SOCKS5服务器作为它的上游服务器,同时 transparent proxy
连接 socks5 server
一般也是需要经过 en0
接口,图中并没有画出。
不像iptables redirect可以配置在OUTPUT链中,pf rdr-to 只对ingress流量起作用,如果想要把本机的egress流量劫持到透明代理上,需要将其路由到另一个interface,转变为后者的ingress流量,再利用 rdr-to 进行流量转发(在这里我们利用了本地lo0)。
rdr pass on lo0 proto tcp from any to 1.2.3.4 -> 127.0.0.1 port 12345
pass out route-to (lo0 127.0.0.1) proto tcp from any to 1.2.3.4
第二条规则 route-to 表示将本地任何地址访问1.2.3.4的TCP流量路由到另一个地址;
第一条规则 rdr-to 表示将进入lo0接口上的流量转发到另一个地址;
当然,这个方案其实有一些缺陷:
只支持TCP协议;
DNS污染问题需要单独解决;
第2个问题却会影响日常使用,将来我会在本文中补充一下我的解决方案。
从前面的示意图中可以看出,透明代理的核心思路非常简单,如果看到这里您已经明白如何去实现透明代理,可以不用再看下文的啰嗦流程。
详细流程
现实世界总是要复杂一点,透明代理还有一些细节问题需要解决:
需要考虑哪些流量需要经过代理? (访问代理服务器的流量不能再走代理)
透明代理用什么程序实现?
由于某些原因,本机可能不能直接连接远端SOCKS5代理服务器,如何处理?
更真实的架构:
..............................macOS.............................. . . . +----------+ . . | APP | . . +----------+ 127.0.0.1:12345 . . | eg: 1.2.3.4 pf rdr-to +------------------+ . . | +----------->| redsocks | . . v | +---------+--------+ . . +----+--+ +--+--+ | . . | en0 +-------------->| lo0 | | . . +-------+ pf route-to +-----+ | . . v . . 127.0.0.1:1080 . . +------------------+ . +---------------+ . | ss-local +---->| ss-server +--->outside world . +------------------+ . +---------------+ (eg: 1.2.3.4) . . SERVER_IP:PORT .................................................................
A. 配置pf.conf
我习惯使用系统默认位置的配置文件,直接编辑 /etc/pf.conf
(默认需要root权限),按如下进行配置:
scrub-anchor "com.apple/*"
table <direct_cidr> persist file "/opt/etc/direct_cidr.txt"
nat-anchor "com.apple/*"
rdr-anchor "com.apple/*"
rdr pass on lo0 proto tcp from any to !<direct_cidr> -> 127.0.0.1 port 12345
pass out route-to (lo0 127.0.0.1) proto tcp from any to !<direct_cidr>
dummynet-anchor "com.apple/*"
anchor "com.apple/*"
load anchor "com.apple" from "/etc/pf.anchors/com.apple"
加载直接连接的IP白名单,存入 direct_cidr.txt文件 中; | |
将所有非直连的流量路由到本地的lo0接口上; | |
对于 进入 lo0接口的流量,如果目标地址是非直连的IP,则转发到本地的透明代理(127.0.0.1:12345); |
B. 创建直连IP白名单文件
前面的配置文件 /etc/pf.conf
使用pf的table语法引用了直连IP白名单文件,需要自行创建该文件:
192.31.196.0/24
192.52.193.0/24
127.0.0.0/8
192.175.48.0/24
192.0.0.0/24
198.18.0.0/15
203.0.113.0/24
100.64.0.0/10
240.0.0.0/4
0.0.0.0/8
192.88.99.0/24
172.16.0.0/12
192.168.0.0/16
198.51.100.0/24
255.255.255.255
192.0.2.0/24
169.254.0.0/16
224.0.0.0/4
10.0.0.0/8
vps-public-ip
需要将你的远端服务器地址加入IP直连白名单 |
C. 配置redsocks
redsocks监听 127.0.0.1:12345
,将流量转发到本地的 127.0.0.1:1080
(本地的SOCKS5代理服务器)
base {
log_debug = off;
log_info = on;
daemon = off;
redirector = pf;
}
redsocks {
local_ip = 127.0.0.1;
local_port = 12345;
ip = 127.0.0.1;
port = 1080;
type = socks5;
}
D. 编译安装redsocks
原版redsocks年久失修,对macOS支持并不好,但是对于最新的macOS编译还是有一点小问题,因此我又进行了一次fork,但不保证以后是否能正常编译。
编译redsocks,将其安装到 /opt/bin/redsocks
:
$ mkdir -p /opt/bin
$ git clone https://github.com/penglei/redsocks && cd redsocks && make OSX_VERSION=master(这步会遇错)
$ mv redsocks /opt/bin/
E. 安装配置本地的SOCKS5服务器
这个步骤有很多方法,比如 ssh -D建立SOCKS5代理,或者使用ss, v2ray等等软件的客户端都可以,相信大部分人都知道应该怎么做。 需要注意的是本地的SOCKS5服务器监听的地址是 127.0.0.1:1080
,redsocks的配置文件指明了将流量转发到该地址。
F. 运行服务
本地的SOCKS5服务器需要根据自己的实际情况运行;
redsocks通过访问
/dev/pf
来获取连接的原始目标地址,因此需要root
权限来运行:$ sudo su root# /opt/bin/redsocks -c /opt/etc/redsocks.conf
配置pf同样需要
root
权限,创建一个新的terminal窗口运行:$ sudo su root# sysctl -w net.inet.ip.forwarding=1 root# pfctl -e root# pfctl -F all root# pfctl -f /etc/pf.conf
开启IP转发功能 开启pf(默认是关闭的) 清空pf的所有配置 加载pf的配置文件 如果想停止使用透明代理,则禁用pf(
sudo pfctl -d
)或者清空pf规则(sudo pfctl -F all
)即可。
服务运行之后,我们的macOS就已经有了透明代理的功能, 运行curl来验证一下:
$ curl -I https://www.google.com --resolve 'www.google.com:443:216.58.200.36'
HTTP/2 302
location: https://www.google.com.hk/url?sa=p&hl=zh-CN&pref=hkredirect&pval=yes&q=https://www.google.com.hk/&ust=1550640983822937&usg=AOvVaw3PnKH6XFhOkLB56FH7sVHc
cache-control: private
content-type: text/html; charset=UTF-8
p3p: CP="This is not a P3P policy! See g.co/p3phelp for more info."
date: Wed, 20 Feb 2019 05:35:53 GMT
server: gws
content-length: 372
x-xss-protection: 1; mode=block
x-frame-options: SAMEORIGIN
set-cookie: 1P_JAR=2019-02-20-05; expires=Fri, 22-Mar-2019 05:35:53 GMT; path=/; domain=.google.com
set-cookie: NID=160=U44fC0UHxupm7ClkYUGknQQR8gT8JmqDIhrL3VDquqo6wFketgeSCqBEgNHea2cClfa8pyYwo1u2X44uU7vIaEd5Bxeoakgtwq0aauu5Kzv5hX0N65TNmPH7LYTaESyQAT5lVMSu_RO9JarbeukX2oNoVBL_y3q0d8sty2_u7eU; expires=Thu, 22-Aug-2019 05:35:53 GMT; path=/; domain=.google.com; HttpOnly
alt-svc: quic=":443"; ma=2592000; v="44,43,39"
Good. It worked!
总结
对于普通用户,这个方法太过折腾,还需要解决‘DNS污染’的问题, 不如在chrome里面通过SwitchyOmega配置SOCKS5代理来得方便,所以并不推荐普通用户使用。 如果您像我一样爱偷懒,这个方法倒是可能有一些帮助。
最后,我厂只能通过HTTP代理访问外网怎么办呢? 最简单的方法把HTTP代理转发成SOCKS5代理,goproxy
可以做到, 我是通过HTTP代理连接另一台外网server来实现SOCKS5代理的,但这方法不具有通用性,就不再赘述。
No comments:
Post a Comment