这里主要介绍电脑无需任何设置，就能够自动加密代理特定网站的HTTP/HTTPS协议。
方案介绍
涉及到的软件
BIND: 一个流行的域名解析服务器，我们可以设置哪些域名需要走加密线路。
Stunnel: 使用TLS对tcp协议进行加密，也就是对tcp建立一条加密线路。
SNI Proxy: 代理软件。对于HTTP协议，它可以根据Host请求头解析得出目标站IP;对于HTTPS协议，它可以根据SNI扩展中的域名解析得出目标站IP。
此方案优缺点
优点：
无需手动设置任何代理，就能够自动加密代理特定网站的HTTP或HTTPS协议
相对于我们常用的ssh隧道，ssh隧道是单路，而此方案是支持多并发连接，可以极大加速网站访问。
缺点：
对于代理HTTPS协议，需要发起HTTPS连接的客户端，比如浏览器支持TLS的SNI扩展。好消息是目前浏览器几乎都支持此扩展，但对于一些非浏览器的客户端，不支持SNI扩展。我们只能设置正向代理来解决此问题。
方案原理
流程图：
原理介绍:
1、首先我们需要准备三台服务器，一台是内网DNS服务器(安装bind)，一台是内网代理服务器（安装stunnel），另一台国外服务器(安装stunnel,sniproxy)。
2、我们还需要设置DNS为内网的DNS，并在内网bind dns设置谷歌域名解析的IP为内网代理服务器
3、当我们访问谷歌网站时，首先会向内网DNS服务器发送DNS A记录查询，此时内网DNS服务器会返回内网代理服务器的IP。
4、浏览器得到谷歌域名的解析IP后（即内网代理服务器的IP），会向内网代理服务器发送HTTP或HTTPS请求。
5、此时内网代理服务器（即stunnel），会接收到请求，经过加密，把请求转发到国外服务器(stunnel)的指定端口上。
6、国外服务器(stunnel)接收到来自国内服务器(stunnel)的加密数据后，经过解密，把请求转发到sniproxy。
7、sniproxy再根据HTTP Host请求头或者HTTPS sni扩展的域名解析出谷歌服务器的IP，并把请求转发给谷歌服务器。
8、谷歌服务器收到来自sniproxy发送的请求后，马上返回网页内容给sniproxy，sniproxy再原路返回数据给浏览器。
方案实施
由于时间有限，我们仅在Ubuntu server 12.04演示安装。
环境介绍
系统：Ubuntu server 12.04
内网DNS IP: 10.96.153.201(主)，10.96.153.204(从)
内网代理服务器： 10.96.153.204
国外服务器IP: 1.2.3.4
安装BIND9
1、在主DNS和从DNS安装bind,即10.96.153.201(主)，10.96.153.204(从)。
wget http://www.isc.org/downloads/file/bind-9-10-0b1-2/?version=tar.gz -O bind-9-10-0b1-2.tar.gz
tar xzf bind-9-10-0b1-2.tar.gz
cd bind-9-10-0b1-2
./configure --prefix=/usr/local/bind
make && make install
2、配置主DNS服务器(10.96.153.201)
2.1、生成/usr/local/bind/etc/rndc.key密钥文件
/usr/local/bind/sbin/rndc-confgen -a -k rndckey -c /usr/local/bind/etc/rndc.key
2.2、编辑/usr/local/bind/etc/named.conf，写入如何内容：
include "/usr/local/bind/etc/rndc.key";
controls { inet 127.0.0.1 port 953 allow { 127.0.0.1; } keys { "rndckey"; }; };
logging {
channel default_syslog { syslog local2; severity notice; };
channel audit_log { file "/var/log/bind.log"; severity notice; print-time yes; };
category default { default_syslog; };
category general { default_syslog; };
category security { audit_log; default_syslog; };
category config { default_syslog; };
category resolver { audit_log; };
category xfer-in { audit_log; };
category xfer-out { audit_log; };
category notify { audit_log; };
category client { audit_log; };
category network { audit_log; };
category update { audit_log; };
category queries { audit_log; };
category lame-servers { audit_log; };
};
options {
directory "/usr/local/bind/etc";
pid-file "/usr/local/bind/var/run/bind.pid";
transfer-format many-answers;
interface-interval 0;
forward only;
forwarders { 202.96.128.166;202.96.134.133; };
allow-query {any;};
};
zone "google.com" {
type master;
file "google.com.zone";
allow-transfer { 10.96.153.204; };
};
在这个named.conf文件中，我们只需要关心如下内容：
对于options{}区域，202.96.128.166和202.96.134.133这两个是ISP提供的本地DNS，需要修改为自己所在ISP的本地DNS。
对于zone “google.com”{}区域，这里定义了google.com域名的区域文件google.com.zone，还有允许10.96.153.204（即从DNS）同步区域文件。
2.3、建立google.com.zone区域文件
$TTL 3600
@ IN SOA ns1.google.com. hostmaster.google.com. (
2014072015 ; Serial
3600 ; Refresh
900 ; Retry
3600000 ; Expire
3600 ) ; Minimum
@ IN NS ns1.google.com.
@ IN NS ns2.google.com.
ns1 IN A 10.96.153.201
ns2 IN A 10.96.153.204
@ IN A 10.96.153.204
* IN A 10.96.153.204
对于这个区域文件，
ns1 IN A 10.96.153.201 指向第一个dns服务器，即主DNS。
ns2 IN A 10.96.153.204 指向第二个dns服务器，即从DNS。
@ IN A 10.96.153.204和* IN A 10.96.153.204指向内网的代理服务器(stunnel)。我们只需要修改这三个地方就好了。
3、配置从DNS服务器(10.96.153.204)
编辑named.conf，写入如下内容
logging {
channel default_syslog { syslog local2; severity notice; };
channel audit_log { file "/var/log/bind.log"; severity notice; print-time yes; };
category default { default_syslog; };
category general { default_syslog; };
category security { audit_log; default_syslog; };
category config { default_syslog; };
category resolver { audit_log; };
category xfer-in { audit_log; };
category xfer-out { audit_log; };
category notify { audit_log; };
category client { audit_log; };
category network { audit_log; };
category update { audit_log; };
category queries { audit_log; };
category lame-servers { audit_log; };
};
options {
directory "/usr/local/bind/etc";
pid-file "/usr/local/bind/var/run/bind.pid";
transfer-format many-answers;
interface-interval 0;
forward only;
forwarders { 202.96.128.166;202.96.134.133; };
allow-query {any;};
};
zone "google.com" {
type slave;
file "google.com.zone";
masters { 10.96.153.201; };
};
配置从DNS就简单得多，只需要写入如上内容到named.conf文件。同样的，
options{}中202.96.128.166和202.96.134.133这两个是当地ISP本地dns。
zone “google.com”{}中10.96.153.201指明主DNS服务器IP。
4、启动bind dns服务器
/usr/local/bind/sbin/named
安装Stunnel
1、在内网代理服务器和国外主机安装stunnel
apt-get install stunnel4
2、内网代理服务器stunnel配置
编辑/etc/default/stunnel4，设置ENABLED=1。
编辑/etc/stunnel/stunnel.conf，内容如下：
client = yes
pid = /etc/stunnel/stunnel.pid
[http]
accept = 80
connect = 1.2.3.4:8082
[https]
accept = 443
connect = 1.2.3.4:4433
此配置文件表示，监听了80端口，并把此端口流量转发到1.2.3.4:8082，监听了443端口，并把此端口流量转发到1.2.3.4:4433
3、国外服务器stunnel配置
3.1、生成ssl证书stunnel.pem文件
openssl genrsa -out key.pem 2048
openssl req -new -x509 -key key.pem -out cert.pem -days 1095
cat key.pem cert.pem >> /etc/stunnel/stunnel.pem
3.2、编辑/etc/stunnel/stunnel.conf文件
client = no
[http]
accept = 1.2.3.4:8082
connect = 127.0.0.1:8082
cert = /etc/stunnel/stunnel.pem
[https]
accept = 1.2.3.4:4433
connect = 127.0.0.1:4433
cert = /etc/stunnel/stunnel.pem
此配置文件表示，监听了1.2.3.4:8082，并转发此地址流量到127.0.0.1:8082，监听了1.2.3.4:4433，并转发给地址流量到127.0.0.1:4433。
3.3、编辑/etc/default/stunnel4，设置ENABLED=1。
4、启动stunnel
service stunnel4 start
安装sniproxy
sniproxy项目地址：https://github.com/dlundquist/sniproxy
1、安装sniproxy
同样只演示在ubuntu server 12.04安装。
1.1、安装UDNS
mkdir udns_packaging
cd udns_packaging
wget http://archive.ubuntu.com/ubuntu/pool/universe/u/udns/udns_0.4-1.dsc
wget http://archive.ubuntu.com/ubuntu/pool/universe/u/udns/udns_0.4.orig.tar.gz
wget http://archive.ubuntu.com/ubuntu/pool/universe/u/udns/udns_0.4-1.debian.tar.gz
tar xfz udns_0.4.orig.tar.gz
cd udns-0.4/
tar xfz ../udns_0.4-1.debian.tar.gz
dpkg-buildpackage
cd ..
dpkg -i *.deb
1.2、安装sniproxy
apt-get install autotools-dev cdbs debhelper dh-autoreconf dpkg-dev gettext libev-dev libpcre3-dev libudns-dev pkg-config
wget https://github.com/dlundquist/sniproxy/archive/master.zip
unzip master.zip
cd sniproxy-master/
dpkg-buildpackage
cd ..
dpkg -i *.deb
2、配置sniproxy
/etc/sniproxy.conf内容如下：
user daemon
pidfile /var/run/sniproxy.pid
error_log {
syslog deamon
priority notice
}
listen 127.0.0.1:8082 {
proto http
table http_hosts
}
table http_hosts {
.* *:80
}
listen 127.0.0.1:4433 {
proto tls
table https_hosts
}
table https_hosts {
.* *:443
}
此配置文件表示，监听了127.0.0.1:8082地址，并解析http协议中的Host请求头为IP，然后转发请求到此IP;监听了127.0.0.1:4433地址，并解析TLS中SNI扩展中的域名为IP，并转发请求到此IP。
3、启动sniproxy:
sniproxy
结束
到目前为止，我们已经搭建完成了整套HTTP/HTTPS加密代理方案。方案中的HTTP明文协议，利用stunnel使用了TLS加密，变成了HTTPS协议，使得数据包无法被解析出明文。方案中的HTTPS协议，本身是加密的，但为了防止SNI扩展的中域名被嗅探，还是走了stunnel的加密通道。对于发送HTTPS请求而不支持SNI扩展的客户端，需要手动设置下代理。
相关帖子：
http://briteming.blogspot.com/2015/08/dnsmasq-dnscrypt-sni-proxy.html
http://briteming.blogspot.com/2012/01/vpsstunnel.html
-------------------------------------
SNI Proxy
Proxies incoming HTTP and TLS connections based on the hostname contained in the initial request of the TCP session. This enables HTTPS name-based virtual hosting to separate backend servers without installing the private key on the proxy machine.
News
First user survey, please take a moment to offer your input.
Features
- Name-based proxying of HTTPS without decrypting traffic. No keys or certificates required.
- Supports both TLS and HTTP protocols.
- Supports IPv4, IPv6 and Unix domain sockets for both back end servers and listeners.
- Supports multiple listening sockets per instance.
Usage
Usage: sniproxy [-c <config>] [-f] [-n <max file descriptor limit>] [-V]
-c configuration file, defaults to /etc/sniproxy.conf
-f run in foreground, do not drop privileges
-n specify file descriptor limit
-V print the version of SNIProxy and exit
Installation
For Debian or Fedora based Linux distributions see building packages below.
Prerequisites
- Autotools (autoconf, automake, gettext and libtool)
- libev4, libpcre and libudns development headers
- Perl and cURL for test suite
Install
./autogen.sh && ./configure && make check && sudo make install
Building Debian/Ubuntu package
This is the preferred installation method on recent Debian based distributions:
- Install required packages
sudo apt-get install autotools-dev cdbs debhelper dh-autoreconf dpkg-dev gettext libev-dev libpcre3-dev libudns-dev pkg-config fakeroot devscripts - Build a Debian package
./autogen.sh && dpkg-buildpackage - Install the resulting package
sudo dpkg -i ../sniproxy_<version>_<arch>.deb
Building Fedora/RedHat package
This is the preferred installation method for modern Fedora based distributions.
- Install required packages
sudo yum install autoconf automake curl gettext-devel libev-devel pcre-devel perl pkgconfig rpm-build udns-devel - Build a distribution tarball:
./autogen.sh && ./configure && make dist - Build a RPM package
rpmbuild --define "_sourcedir `pwd`" -ba redhat/sniproxy.spec - Install resulting RPM
sudo yum install ../sniproxy-<version>.<arch>.rpm
I've used Scientific Linux 6 a fair amount, but I prefer Debian based distributions. RPM builds are tested in Travis-CI on Ubuntu, but not natively. This build process may not follow the current Fedora packaging standards, and may not even work.
Building on OS X with Homebrew
- install dependencies.
brew install libev pcre udns autoconf automake gettext libtool - Read the warning about gettext and force link it so autogen.sh works. We need the GNU gettext for the macro
AC_LIB_HAVE_LINKFLAGSwhich isn't present in the default OS X package.brew link --force gettext - Make it so
./autogen.sh && ./configure && make
OS X support is a best effort, and isn't a primary target platform.
Configuration Syntax
user daemon
pidfile /tmp/sniproxy.pid
error_log {
syslog daemon
priority notice
}
listener 127.0.0.1:443 {
protocol tls
table TableName
# Specify a server to use if the initial client request doesn't contain
# a hostname
fallback 192.0.2.5:443
}
table TableName {
# Match exact request hostnames
example.com 192.0.2.10:4343
example.net [2001:DB8::1:10]:443
# Or use regular expression to match
.*\\.com [2001:DB8::1:11]:443
# Combining regular expression and wildcard will resolve the hostname
# client requested and proxy to it
.*\\.edu *:443
}
DNS Resolution
Using hostnames or wildcard entries in the configuration requires sniproxy to be built with UDNS. SNIProxy will still build without UDNS, but these features will be unavailable.
UDNS uses a single UDP socket for all queries, so it is recommended you use a local caching DNS resolver (with a single socket each DNS query is protected by spoofing by a single 16 bit query ID, which makes it relatively easy to spoof).
UDNS is currently not available in Debian stable, but a package can be easily built from the Debian testing or Ubuntu source packages:
mkdir udns_packaging
cd udns_packaging
wget http://archive.ubuntu.com/ubuntu/pool/universe/u/udns/udns_0.4-1.dsc
wget http://archive.ubuntu.com/ubuntu/pool/universe/u/udns/udns_0.4.orig.tar.gz
wget http://archive.ubuntu.com/ubuntu/pool/universe/u/udns/udns_0.4-1.debian.tar.gz
tar xfz udns_0.4.orig.tar.gz
cd udns-0.4/
tar xfz ../udns_0.4-1.debian.tar.gz
dpkg-buildpackage
cd ..
sudo dpkg -i libudns-dev_0.4-1_amd64.deb libudns0_0.4-1_amd64.debfrom https://github.com/dlundquist/sniproxy-------------
部署 SNI Proxy 加速网页访问 反代 无需证书
这次，我们来介绍另外一款神器 SNI Proxy，是用 dnsmasq 配合 sniproxy 可以实现无证书任意网站反代。它使用SNI 技术将 TLS 连接通过 TCP 代理到目标网站，这样就避免了对代理服务器的证书需求，而且访问到的网站证书也是原原本本的证书。
编译 SNI Proxy（可以跳过）
从git上克隆 sni的源文件： https://github.com/dlundquist/sniproxy.git
准备环境
安装配置 SNI Proxy
编辑 /etc/sniproxy.conf来开启反代：
像上边这样开启针对某个域名的反代，但是这样太麻烦了，每次新增加一个网站，都需要对这个列表增加——为此，我们为了方便稍微牺牲一点安全性——且我不需要对邮件进行反代（容易被滥用，国外对垃圾邮件服务器是深恶痛绝的。），那么我可以直接这样写：
这样就只打开了 https 反代，而且是只要被解析到这个服务器的域名都会被反代，这样一来，我们就可以只通过 dnsmasq 的解析来控制哪个域名反代了。（有那么一点安全隐患就是一旦被别人发现，你可能你服务器的流量会意外流失咯~记得监控你的服务器带宽?）
运行 sniproxy
直接使用命令sniproxy 即可运行，默认配置文件就是“/etc/sniproxy.conf”它会自动加载，如果你使用了其他路径或者配置文件名，那么你需要使用“-c”选项来指定路径：
端口重定向
那么，一般我们访问网站不喜欢输入端口号或者协议名称，那么默认访问的是80端口怎么办？作为辅助，我们安装一个轻量级的 nginx，让它把所有访问80端口的流量转移到443上边去，使用301重定向即可。
我们编辑 nginx 的配置文件“/etc/nginx/sites-available/default”
改为如下内容：
域名解析
sniproxy 搭建成功，但是它是不能被直接访问的，你需要将域名解析过去，这样它才能根据域名来代理你的 ssl 链接，那么你可能就需要在自己的 hosts 上修改解析啦。
-----------
SNI proxies
David Fifield,
Jiang Jian,
Paul Pearce
Code and data download:
git clone https://www.bamsoftware.com/git/sniproxy.git
or snapshot sniproxy-20161220.tar.gz- https://scans.io/study/sniproxy
An investigation into SNI proxies,
special servers that forward TLS streams
to the host specified in the Server Name Indication field (SNI)
of the TLS handshake.
SNI proxies could be useful for Internet censorship circumvention
or censorship measurement.
We manually investigated and characterized
a small number of SNI proxies that had been publicly reported,
and did a targeted scan of the IPv4 space to find
how many there are.
Our scan of port 443 on
found 2,500 SNI proxies on the Internet.
Background
An SNI proxy is a TLS server that proxies traffic
to any destination given in the
Server Name Indication field (SNI).
An SNI proxy
receives connections on port 443, peeks at the SNI,
then forwards the connection to the host given in the SNI.
It's not a MITM;
the proxy forwards the client's ClientHello and all other traffic unmodified,
and the client has a true end-to-end TLS connection to the server,
through the SNI proxy.
An SNI proxy is like an open proxy,
with the restriction that it only works for port 443 and only for TLS.
We first heard of SNI proxies from people in China
who were using the proxies to circumvent Internet censorship.
(See for example
https://github.com/phuslu/goproxy/issues/853.)
We speculated that the SNI proxies
might be related to ad injection by ISPs on port 80.
The ISP falsifies DNS responses to point to its own
ad-injection proxy server, which injects ads into HTTP on port 80.
But the DNS-based redirection cannot redirect plain HTTP
without also redirecting HTTPS.
The proxy cannot tamper with the HTTPS,
but it also cannot cannot cause HTTPS to break;
therefore it just proxies the HTTPS unmodified.
SNI proxying is a standard feature of various firewalls and proxies.
HAProxy has it:
http://blog.haproxy.com/2012/04/13/enhanced-ssl-load-balancing-with-server-name-indication-sni-tls-extension/
https://trick77.com/haproxy-and-sni-based-ssl-offloading-with-intermediate-ca/
There are a few projects on GitHub doing it:
https://github.com/dlundquist/sniproxy
https://github.com/gpjt/stupid-proxy
https://github.com/yrutschle/sslh
We found some servers in the wild that do it, with brand names like Sophos and Blue Coat.
Potential applications and hazards of SNI proxies
The obvious application of SNI proxies is the circumvention
of Internet censorship,
against a censor that blocks IP addresses and DNS requests
but does not examine the SNI field of TLS connections.
You can access a blocked HTTPS server by connecting to it
indirectly through an SNI proxy
located beyond the censor's control.
It doesn't even require any special client-side software,
just modification of the hosts file.
For example, if https://example.com/ is blocked by the censor,
and 192.0.2.100 is an SNI proxy, add this line to the hosts file:
192.0.2.100 example.com
You could use SNI proxies to measure censorship.
An SNI proxy inside the network controlled by the censor
serves as a vantage point to test the reachability of
destinations outside the censor's network
(port 443 only).
It might be possible to DoS an SNI proxy
by asking it to connect to 127.0.0.1,
or explore an internal network by giving it a private IP address.
Technically, "literal IPv4 and IPv6 addresses are not permitted"
in the SNI field
(RFC 4366).
Even if the SNI proxy does not support an IP address in SNI,
it would be easy to set up a DNS server
that gives every IP address a domain name
(e.g. 127.0.0.1.example.com → 127.0.0.1).
There are already some public names that
map to internal addresses, like fuf.me → 127.0.0.1.
The GoProxy SNI proxy list
We started by looking at 26 purported SNI proxy IP addresses
from a ticket of GoProxy, a circumvention system:
https://github.com/phuslu/goproxy/issues/853
We probed them all on ports 80 and 443 (with and without SNI),
with the goal of characterizing in detail
some specific SNI proxy implementations.
Of the 26 IP addresses, 10 were no longer running,
and among the remaining ones we found three distinct fingerprints.
On port 80 (HTTP),
we saw three distinct responses,
described in the
next subsection.
On port 443 (HTTPS),
we saw two distinct behaviors,
described in the
following subsection.
This table summarizes the results of our manual check
of the 26 IP addresses from the GoProxy ticket.
"—" means the TCP connection failed.
host
HTTP
HTTPS
HTTPS+SNI
reverse DNS
45.127.92.217
—
—
—
103.15.187.54
dns.auth.fail
alert
proxies
103-15-187-54.op-net.com
110.4.12.173
—
—
—
110.4.12.175
—
—
—
110.4.12.176
—
—
—
110.4.12.178
—
—
—
110.4.24.170
ChinaCache
silence
proxies
110.4.24.175
ChinaCache
silence
proxies
110.4.24.176
HAProxy
silence
proxies
110.4.24.178
HAProxy
silence
proxies
182.239.95.136
—
—
—
182.239.95.136.hk.chinamobile.com
182.239.95.137
—
—
—
182.239.95.137.hk.chinamobile.com
182.239.127.136
—
—
—
182.239.127.136.hk.chinamobile.com
182.239.127.137
—
—
—
182.239.127.137.hk.chinamobile.com
203.78.36.234
—
—
—
m203-78-36-234.smartone.com
218.254.1.13
ChinaCache
silence
proxies
cm218-254-1-13.hkcable.com.hk
218.254.1.15
ChinaCache
silence
proxies
cm218-254-1-15.hkcable.com.hk
219.76.4.3
HAProxy
silence
proxies
tswc5b003.netvigator.com
219.76.4.4
HAProxy
silence
proxies
tswc5b004.netvigator.com
219.76.4.9
ChinaCache
silence
proxies
tswc5b009.netvigator.com
219.76.4.11
ChinaCache
silence
proxies
tswc5b011.netvigator.com
219.76.4.14
ChinaCache
silence
proxies
tswc5b014.netvigator.com
219.76.4.69
HAProxy
silence
proxies
tswc5b069.netvigator.com
219.76.4.70
HAProxy
silence
proxies
tswc5b070.netvigator.com
219.76.4.75
ChinaCache
silence
proxies
tswc5b075.netvigator.com
219.76.4.76
ChinaCache
silence
proxies
tswc5b076.netvigator.com
HTTP fingerprints
On port 80,
we found 3 distinct HTTP fingerprints,
not counting no-responses:
9
ChinaCache
6
HAProxy
1
dns.auth.fail
10
no response
The "ChinaCache" HTTP fingerprint
HTTP/1.0 404 Not Found
Server: FC
Date: Fri, 16 Sep 2016 17:53:42 GMT
Content-Type: text/html
Content-Length: 432
Powered-By-ChinaCache: MISS from PCW-HK-3-3X5.8
Connection: close
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<HTML><HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=gb2312">
<TITLE>错误：您所请求的网址（URL）无法获取</TITLE>
<STYLE type="text/css"><!--BODY{background-color:#ffffff;font-family:verdana,sans-serif}PRE{font-family:sans-serif}--></STYLE>
</HEAD><BODY>
<H1>错误</H1>
<H2>您所请求的网址（URL）无法获取</H2>
The Chinese text, originally encoded in gb2312,
means "Error: The requested URL (URL) could not be retrieved."
The Powered-By-ChinaCache header varies,
apparently reflecting the server name.
The "HAProxy" HTTP fingerprint
HTTP/1.0 403 Forbidden
Cache-Control: no-cache
Connection: close
Content-Type: text/html
<html><body><h1>403 Forbidden</h1>
Request forbidden by administrative rules.
</body></html>
Nmap identifies this response as HAProxy.
The "dns.auth.fail" HTTP fingerprint
HTTP/1.0 302 Found
Location: http://dns.auth.fail/
Content-Length: 0
Connection: close
Date: Fri, 16 Sep 2016 17:53:57 GMT
Server: lighttpd/1.4.35
This is a weird one, covered in a separate document.
HTTPS fingerprints
On port 443, we found 2 distinct fingerprints, not counting no-responses.
The 10 no-responses are the same 10 as for HTTP.
15
with SNI, proxies; without SNI, sends nothing
1
with SNI, proxies; without SNI, sends TLS "handshake failure" alert
10
no response
When provided with an SNI,
all the responsive servers were indistinguishable because they
just proxied the connection.
Without an SNI, most of the servers
simply sent nothing in return;
the only exception was the single "dns.auth.fail"
server, which sent a TLS alert.
It makes sense that an SNI proxy would not be able
to do anything useful when not supplied with an SNI.
This is what we did to connect without SNI:
timeout 10 openssl s_client -ign_eof -connect $host:443
And with SNI:
timeout 10 openssl s_client -ign_eof -connect $host:443 -servername www.example.com
To check for proxying, we looked for the proper server certificate:
subject=/C=US/ST=California/L=Los Angeles/O=Internet Corporation for Assigned Names and Numbers/OU=Technology/CN=www.example.org
issuer=/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert SHA2 High Assurance Server CA
With SNI, proxies; without SNI, sends nothing
Providing SNI causes the server to proxy to the desired destination.
Omitting SNI causes the server to send nothing after receiving the
ClientHello, and terminate the connection after a few seconds.
With SNI, proxies; without SNI, sends TLS "handshake failure" alert
This fingerprint was only seen on one host, the
dns.auth.fail one.
Providing SNI causes the server to proxy to the desired destination.
Omitting SNI causes the server to send a TLS "handshake failure" alert, viz.:
15 03 01 00 02 02 28
Incidentally, you get the same alert if you send a plaintext HTTP request to port 443.
Scanning for SNI proxies
We did a scan of the IPv4 space on port 443 on
and found 2,500 SNI proxies.
We used ZMap and a custom program, scan-sniproxy.
scan-sniproxy simply connects to an IP address,
does a TLS handshake with a given SNI,
does certificate validation,
and saves a hash of the leaf certificate
(as proof that the IP address actually proxied to
the destination given in the SNI).
For this experiment, we set up a dedicated HTTPS server
at the domain sni-scan-for-research-study.bamsoftware.com.
The experiment doesn't require a dedicated server—you
could use any HTTPS server—but we wanted to be able to
host a web page with contact information in case anyone
objected to the scan.
(A copy of the web page.)
We also wanted to capture and observe
the traffic arriving at the server specified in the SNI.
If you want to look at the traffic information, see
443.pcap.xz, access.log,
error.log, and ssl.log
in the scan-sniproxy-20161024.0 subdirectory of
the source code.
This is the command line we used:
zmap -p 443 --output-fields=* -r 150000 -u zmap_updates.log -l zmap_log.log -v5 -q | \
ztee -u ztee_updates.csv zmap_output.csv | \
scan-sniproxy -input /dev/stdin -maxprocs 4 -maxthreads 10000 -timeout 10s > scan-sniproxy.csv
The scan found 2,500 SNI proxies.
You can download a CSV file: scan-sniproxy-20161024.0.proxiesonly.csv.
Here they are along with reverse DNS:
There are concentrations of SNI proxies in certain networks:
40.6%
are *.rdns.cloudradium.com
6.4%
are 101.95.144.68-71, 101.96.9.138-165, 101.96.10.19-74, 101.96.11.19-74, 101.110.119.19-75
5.2%
are *.protectedgroup.com (23.19.41.90-94, 23.19.93.2-126)
5.0%
are 213.184.119.2-126
Clearly, cloudradium.com is a heavy hitter when it comes to SNI proxies.
We did not find out much information about cloudradium.com,
whatever it may be.
Anthony Joseph found for us
a fraud report for "CloudRadium LLC" from 2014 complaining
about the accumulation of IP addresses:
Fri Oct 10 01:33:25 EDT 2014
Fraud Report on one of your current member
Some basic info about the company who has deceived its IPv4
application.
Name of the Comnapy: CloudRadium LLC.
Registry address: 1603 Capitol Ave St3 310, Cheyenne, WY.
Owner’s name: Li Xuan (李轩） & Deng Xiu Ping (邓秀平)
As the IPv4 has been depleted in the AP regions, some
so-called clever guys from China registered a LLC named
CloudRadium LLC in WY in 2012-10-01, meanwhile they registered a
domain named http://www.cloudradium.com on 2012-09-26. From
then, they’v started their journey of deceiving the Cherish IPv4
resources which should belong to the Arin Internet Community
from ARIN. They have carefully studied and researched all
Arin’s Policy on IPv4 management and has abused the policy to
apply for more IPv4 steps by steps by using the fraud
information. See details below.
...
5.In a word, their main purpose is to have as many IPv4 as they
can. Their company is not founded for the purpose of doing
business in USA, but for accumulating IPv4 addresses
Sidenote: parallel scans
During the scan, we expected to see TLS connections
to port 443 on the dedicated web server at
sni-scan-for-research-study.bamsoftware.com.
Since there were 2,500 SNI proxies,
we should have gotten 2,500 connections from them alone,
ignoring Internet background traffic.
In fact, we received 6,627 connections
(6,474 with the correct SNI),
and 3,696 cam from a single IP address,
192.107.156.196.
Here are the 10 most common source IP addresses
taken from ssl.log during the 8 hours of the scan:
8
211.138.60.14
9
219.76.4.73
tswc5b073.netvigator.com
12
207.249.174.253
na-207-249-174-253.static.avantel.net.mx
12
208.80.194.26
static-208-80-194-26.as13448.com (Websense)
13
110.4.24.175
13
219.76.4.9
tswc5b009.netvigator.com
16
120.198.243.35
16
65.202.124.7
113
66.133.109.36
outbound1.letsencrypt.org
3696
192.107.156.196
66.133.109.36 (outbound1.letsencrypt.org) is easy to explain.
sni-scan-for-research-study.bamsoftware.com had a certificate
from Let's Encrypt and these were connections related to the
ACME protocol.
208.80.194.26 (static-208-80-194-26.as13448.com) is interesting:
AS13448 belongs to Websense, a web filtering company.
They must do centralized, automatic scans of SNIs seen by
their firewall installations.
The clear outlier is 192.107.156.196.
During the 8 hours of the scan, this IP address
made TLS connections with the correct SNI
to our dedicated HTTPS server at a roughly constant rate.
It stopped making connections after our scan stopped.
Our scan had not touched anything in 192.107.156.0/24
when the first connections from 192.107.156.196 begin to arrive.
To be clear: we measured these connections at the
dedicated web server running at
sni-scan-for-research-study.bamsoftware.com,
the host specified in our SNI,
not at the host running ZMap.
In other words, it was not a matter of a host back-scanning
the source of a detected scan.
Instead, while we were scanning random HTTPS servers,
an unrelated host—192.107.156.196—was scanning the same host
that we specified in our SNI.
One might suppose that 192.107.156.196 is merely a shared exit
point for many other SNI proxies. But this can't be the case.
For one thing, 192.107.156.196 made more connections
to the HTTPS server than we made in total.
And for another, the TLS client handshake differed from the
ones that our scanner program produced—for example,
it used session resumption.
If it were merely tunneling
our own connections, it would not have been able to modify
the TLS handshake and still have validation succeed.
Instead, this host was independently sending
its own TLS probes while we were doing our scan.
whois
says the IP address belongs to Harris Corporation.
NetRange: 192.107.156.0 - 192.107.156.255
CIDR: 192.107.156.0/24
NetName: HARRISNET5
NetHandle: NET-192-107-156-0-1
Parent: NET192 (NET-192-0-0-0-0)
NetType: Direct Assignment
OriginAS:
Organization: Harris Corporation (HARRIS-6)
RegDate: 1991-05-20
Updated: 2015-05-26
Ref: https://whois.arin.net/rest/net/NET-192-107-156-0-1
OrgName: Harris Corporation
OrgId: HARRIS-6
Address: Mail Stop 57
Address: 1024 West Nasa Blvd
City: Melbourne
StateProv: FL
PostalCode: 32919
Country: US
RegDate: 1989-05-17
Updated: 2011-09-24
Ref: https://whois.arin.net/rest/org/HARRIS-6
Appendix
The sonar.http proxy list
We looked for the
three bootstrap HTTP fingerprints in the
Project Sonar HTTP data set
dated 20160830. "Project Sonar includes a regular HTTP GET request for all IPv4 hosts with an open 80/TCP."
The idea was to use an existing scan of port 80 rather than do our own. The input contains 61,683,005 hosts.
The grepsonar subdirectory in the sniproxy.git repo
contains a program that searches the sonar.http file
for the 3 HTTP fingerprints found in the GoProxy list.
It found:
514
ChinaCache
2511
HAProxy
2460
dns.auth.fail
Of these only a small number are actually SNI proxies,
despite having the same fingerprint:
22
of
514
ChinaCache are SNI proxies
37
of
2511
HAProxy are SNI proxies
37
of
2460
dns.auth.fail are SNI proxies (fell to 30 a week later)
ChinaCache SNI proxies
- scan-sniproxy/scan-sniproxy-chinacache-20160830-http.csv
110.4.24.170
110.4.24.175
111.11.184.10
111.11.184.117
111.11.184.119
111.12.251.162
111.12.251.206
116.246.6.39
116.246.6.46
117.156.13.7
120.198.243.2
183.207.229.136
203.210.6.39
211.138.60.14
218.254.1.13
218.254.1.15
219.76.4.9
219.76.4.11
219.76.4.14
219.76.4.73
219.76.4.75
219.76.4.76
HAProxy SNI proxies
- scan-sniproxy/scan-sniproxy-haproxy-20160830-http.csv
101.95.144.68
101.95.144.69
101.95.144.70
101.95.144.71
110.4.24.176
110.4.24.178
111.11.153.19
111.11.153.36
111.11.153.37
111.11.153.38
111.11.153.40
111.11.153.42
111.11.153.43
111.11.184.5
111.11.184.7
111.11.184.9
111.11.184.13
111.11.184.116
111.12.251.172
111.12.251.173
111.12.251.175
116.246.6.35
116.246.6.36
116.246.6.38
117.156.13.5
211.138.60.6
211.138.60.7
211.138.60.8
211.138.60.9
211.138.60.10
211.138.60.11
211.138.60.18
211.138.60.19
219.76.4.3
219.76.4.4
219.76.4.69
219.76.4.70
dns.auth.fail SNI proxies
- tests/dns.auth.fail-sniproxy.csv (scanned 2016-09-20)
- scan-sniproxy/scan-sniproxy-dns.auth.fail-20160830-http.csv (scanned 2016-09-28)
* denotes hosts that disappeared in the second scan.
5.187.20.130
31.220.5.61
37.235.49.150
46.37.190.246
46.108.39.132
69.12.80.155 *
69.162.73.203
82.103.128.85
83.170.70.124
83.170.70.125
91.210.104.52
95.154.233.196
103.15.187.54
103.19.16.13
104.250.97.19
104.250.98.55
104.250.98.56
104.250.98.67 *
104.250.98.168
104.250.98.169
109.123.113.195
113.20.28.16
158.255.211.127
172.97.82.35
173.44.17.3
185.18.79.87
185.57.116.51
185.70.11.133 *
185.70.11.135 *
185.70.11.137 *
185.70.11.139 *
185.125.168.110
193.182.144.202
194.54.80.146
202.155.223.171
204.152.217.140 *
211.22.145.212
The sonar.https proxy list
We scanned all the hosts in the
Project Sonar HTTPS data set
dated 20160906.
"Project Sonar includes an HTTPS GET request for all IPv4 hosts with an
open 443/TCP."
The idea was to use an existing scan of port 443 rather than do our own.
In retrospect, this was a bad idea,
because the Project Sonar scan omits servers
that don't return a certificate when contacted without SNI—none of the
servers in the GoProxy list returned a certificate in this case.
Our scan of these hosts turned up only a few oddball SNI proxies.
The input contains 32,327,703 hosts.
First we extracted the IP addresses from the sonar.https-20160906 data set:
cd sonar-ips
./sonar-ips ../20160906-http.gz > ips-20160906-https.txt
Then we scanned all of them for SNI proxy support:
cd scan-sniproxy
./scan-sniproxy -input ../sonar-ips/ips-20160906-https.txt > scan-sniproxy-full-20160906-https.csv
The scan took about 8 hours. The output CSV file is 5.9 GB, but there are only 38 SNI proxies:
grep -a ,T, scan-sniproxy-full-20160906-https.csv > scan-sniproxy-20160906-https.csv
IP address
reverse DNS
1.32.63.161
20.138.2.35
40.129.146.22
h22.146.129.40.static.ip.windstream.net
65.202.124.19
intranet.mcwane.com
65.202.124.20
francais.clowcanada.com
65.202.124.23
helpdesk.mcwane.com
65.202.124.28
office.mcwane.com
65.202.124.47
autodiscover.mantank.com
65.202.124.48
65.202.124.55
65.202.124.56
ci.synapse-wireless.com
65.202.124.60
65.202.124.85
aca.mcwane.com
65.202.124.103
apps.mcwane.com
65.202.124.231
68.232.35.34
68.232.44.115
72.22.21.20
72.22.21.30
prostream.net
72.22.21.33
72.22.21.49
72.22.21.58
78.133.124.150
94.124.157.198
119.18.234.60
129.143.4.2
wwwproxy.belwue.de
133.43.191.119
159.12.192.12
159.12.192.13
159.12.192.14
162.243.47.142
proxy2.threatworking.com
183.207.227.132
cache.IDC.js.chinamobile.com
208.250.32.193
209.195.175.8
mail.komenpittsburgh.org
209.195.175.13
hosted.prostream.net
212.200.253.210
212.200.253.211
218.57.200.13
We investigated a few of them manually.
40.129.146.22 — Websense
https://40.129.146.22/ says:
40.129.146.22 uses an invalid security certificate.
The certificate is not trusted because the issuer certificate is unknown.
The server might not be sending the appropriate intermediate
certificates. An additional root certificate may need to be imported.
The certificate is only valid for 10.170.10.116
Clicking through gives:
Could not connect to server
Overview: Could not connect to 40.129.146.22 .
Details: Peer suddenly disconnected found
Options: Pressing the button allows you to go to the previous page. You
can try to reload the page or check if the URL is correct.
[Websense Logo]
65.202.124.231 — Citrix XenApp
https://65.202.124.231/ says:
65.202.124.231 uses an invalid security certificate.
The certificate is only valid for citrix.mcwane.net
The certificate expired on 01/30/2013 12:00 PM. The current time is 09/24/2016 04:07 PM.
Clicking through times out after redirecting to
http://65.202.124.231/Citrix/XenApp/.
https://citrix.mcwane.net/, after many redirects, lands at
https://citrix.mcwane.net/Citrix/XenApp/clientDetection/downloadNative.aspx.
Citrix XenApp
Your Windows desktops and apps on demand - from any PC, Mac, smartphone or tablet.
72.22.21.30 — Sophos firewall
https://72.22.21.30/ raises the expected certificate error:
The certificate is only valid for the following names:
www.prostream.net, prostream.net, hosted.prostream.net,
mail.prostream.net, send.prostream.net, spam.prostream.net.
Clicking through gives:
Blocked request
Dear ,
This is a message from the IT Department.
The web site you are trying to access:
https://72.22.21.30/
is listed as a site within the category IPAddress
Current Internet Access Configuration for you does not allow visiting sites within this category at this time.
If the website has been erroneously blocked, please submit it for re-evaluation.
You are currently browsing as an unauthenticated user.
Click here to log in
Sophos Firewall
This server allows the SNI to be an IP address. Here it is proxying to a Tor relay on port 443:
$ torsocks -i openssl s_client -ign_eof -connect 72.22.21.30:443 -servername 82.146.47.17
subject=/CN=www.mn7ntdslcv.net
issuer=/CN=www.x4aigwinbg6m.com
from https://www.bamsoftware.com/computers/sniproxy/
