SSLH – A SSL/SSH MULTIPLEXER

sslh accepts connections on specified ports, and forwards them further based on tests performed on the first data packet sent by the remote client.

Probes for HTTP, TLS/SSL (including SNI and ALPN), SSH, OpenVPN, tinc, XMPP are implemented, and any other protocol that can be tested using a regular expression, can be recognised. A typical use case is to allow serving several services on port 443 (e.g. to connect to SSH from inside a corporate firewall, which almost never block port 443) while still serving HTTPS on that port.

Hence sslh acts as a protocol demultiplexer, or a switchboard. With the SNI and ALPN probe, it makes a good front-end to a virtual host farm hosted behind a single IP address.

sslh has the bells and whistles expected from a mature daemon: privilege and capabilities dropping, inetd support, systemd support, transparent proxying, chroot, logging, IPv4 and IPv6, a fork-based and a select-based model, and more.

COMPILE AND INSTALL

DEPENDENCIES

sslh uses libconfig and libwrap.

For Debian, these are contained in packages libwrap0-dev and libconfig8-dev.

For OpenSUSE, these are contained in packages libconfig9 and libconfig-dev in repository http://download.opensuse.org/repositories/multimedia:/libs/openSUSE_12.1/

For Fedora, you’ll need packages libconfig and libconfig-devel:

yum install libconfig libconfig-devel

If you can’t find libconfig, or just don’t want a configuration file, set USELIBCONFIG= in the Makefile.

COMPILATION

After this, the Makefile should work:

make install

There are a couple of configuration options at the beginning of the Makefile:

• USELIBWRAP compiles support for host access control (see hosts_access(3)), you will need libwrap headers and library to compile (libwrap0-dev in Debian).
• USELIBCONFIG compiles support for the configuration file. You will need libconfig headers to compile (libconfig8-dev in Debian).
• USESYSTEMD compiles support for using systemd socket activation. You will need systemd headers to compile (systemd-devel in Fedora).

BINARIES

The Makefile produces two different executables: sslh-fork and sslh-select:

• sslh-fork forks a new process for each incoming connection. It is well-tested and very reliable, but incurs the overhead of many processes.
  If you are going to use sslh for a “small” setup (less than a dozen ssh connections and a low-traffic https server) then sslh-fork is probably more suited for you.
• sslh-select uses only one thread, which monitors all connections at once. It is more recent and less tested, but only incurs a 16 byte overhead per connection. Also, if it stops, you’ll lose all connections, which means you can’t upgrade it remotely.
  If you are going to use sslh on a “medium” setup (a few thousand ssh connections, and another few thousand ssl connections), sslh-select will be better.

If you have a very large site (tens of thousands of connections), you’ll need a vapourware version that would use libevent or something like that.

INSTALLATION

• In general:

    make
    cp sslh-fork /usr/local/sbin/sslh
    cp basic.cfg /etc/sslh.cfg
            vi /etc/sslh.cfg
  

• For Debian:

    cp scripts/etc.init.d.sslh /etc/init.d/sslh
  

• For CentOS:

    cp scripts/etc.rc.d.init.d.sslh.centos /etc/rc.d/init.d/sslh
  

You might need to create links in /etc/rc.d so that the server start automatically at boot-up, e.g. under Debian:

update-rc.d sslh defaults

CONFIGURATION

If you use the scripts provided, sslh will get its configuration from /etc/sslh.cfg. Please refer to example.cfg for an overview of all the settings.

A good scheme is to use the external name of the machine in listen, and bind httpd to localhost:443 (instead of all binding to all interfaces): that way, HTTPS connections coming from inside your network don’t need to go through sslh, and sslh is only there as a frontal for connections coming from the internet.

Note that ‘external name’ in this context refers to the actual IP address of the machine as seen from your network, i.e. that that is not 127.0.0.1 in the output of ifconfig(8).

LIBWRAP SUPPORT

Sslh can optionnaly perform libwrap checks for the sshd service: because the connection to sshd will be coming locally from sslh, sshd cannot determine the IP of the client.

OPENVPN SUPPORT

OpenVPN clients connecting to OpenVPN running with -port-share reportedly take more than one second between the time the TCP connexion is established and the time they send the first data packet. This results in sslh with default settings timing out and assuming an SSH connexion. To support OpenVPN connexions reliably, it is necessary to increase sslh’s timeout to 5 seconds.

Instead of using OpenVPN’s port sharing, it is more reliable to use sslh’s --openvpn option to get sslh to do the port sharing.

USING PROXYTUNNEL WITH SSLH

If you are connecting through a proxy that checks that the outgoing connection really is SSL and rejects SSH, you can encapsulate all your traffic in SSL using proxytunnel (this should work with corkscrew as well). On the server side you receive the traffic with stunnel to decapsulate SSL, then pipe through sslh to switch HTTP on one side and SSL on the other.

In that case, you end up with something like this:

ssh -> proxytunnel -e ----[ssh/ssl]---> stunnel ---[ssh]---> sslh --> sshd
Web browser -------------[http/ssl]---> stunnel ---[http]--> sslh --> httpd

Configuration goes like this on the server side, using stunnel3:

stunnel -f -p mycert.pem  -d thelonious:443 -l /usr/local/sbin/sslh -- \
 sslh -i  --http localhost:80 --ssh localhost:22

• stunnel options:
  • -f for foreground/debugging
  • -p for specifying the key and certificate
  • -d for specifying which interface and port we’re listening to for incoming connexions
  • -l summons sslh in inetd mode.
• sslh options:
  • -i for inetd mode
  • --http to forward HTTP connexions to port 80, and SSH connexions to port 22.

CAPABILITIES SUPPORT

On Linux (only?), you can compile sslh with USELIBCAP=1 to make use of POSIX capabilities; this will save the required capabilities needed for transparent proxying for unprivileged processes.

Alternatively, you may use filesystem capabilities instead of starting sslh as root and asking it to drop privileges. You will need CAP_NET_BIND_SERVICE for listening on port 443 and CAP_NET_ADMIN for transparent proxying (see capabilities(7)).

You can use the setcap(8) utility to give these capabilities to the executable:

# setcap cap_net_bind_service,cap_net_admin+pe sslh-select

Then you can run sslh-select as an unpriviledged user, e.g.:

$ sslh-select -p myname:443 --ssh localhost:22 --ssl localhost:443

Caveat: CAP_NET_ADMIN does give sslh too many rights, e.g. configuring the interface. If you’re not going to use transparent proxying, just don’t use it (or use the libcap method).

TRANSPARENT PROXY SUPPORT

On Linux and FreeBSD you can use the --transparent option to request transparent proxying. This means services behind sslh (Apache, sshd and so on) will see the external IP and ports as if the external world connected directly to them. This simplifies IP-based access control (or makes it possible at all).

Linux:

sslh needs extended rights to perform this: you’ll need to give it CAP_NET_ADMIN capabilities (see appropriate chapter) or run it as root (but don’t do that).

The firewalling tables also need to be adjusted as follow. The example connects to HTTPS on 4443 – adapt to your needs ; I don’t think it is possible to have httpd listen to 443 in this scheme – let me know if you manage that:

# iptables -t mangle -N SSLH
# iptables -t mangle -A  OUTPUT --protocol tcp --out-interface eth0 --sport 22 --jump SSLH
# iptables -t mangle -A OUTPUT --protocol tcp --out-interface eth0 --sport 4443 --jump SSLH
# iptables -t mangle -A SSLH --jump MARK --set-mark 0x1
# iptables -t mangle -A SSLH --jump ACCEPT
# ip rule add fwmark 0x1 lookup 100
# ip route add local 0.0.0.0/0 dev lo table 100

Tranparent proxying with IPv6 is similarly set up as follows:

    # ip6tables -t mangle -N SSLH
    # ip6tables -t mangle -A  OUTPUT --protocol tcp --out-interface eth0 --sport 22 --jump SSLH
    # ip6tables -t mangle -A OUTPUT --protocol tcp --out-interface eth0 --sport 4443 --jump SSLH
    # ip6tables -t mangle -A SSLH --jump MARK --set-mark 0x1
    # ip6tables -t mangle -A SSLH --jump ACCEPT
    # ip -6 rule add fwmark 0x1 lookup 100
    # ip -6 route add local ::/0 dev lo table 100

Note that these rules will prevent from connecting directly to ssh on the port 22, as packets coming out of sshd will be tagged. If you need to retain direct access to ssh on port 22 as well as through sslh, you can make sshd listen to 22 AND another port (e.g. 2222), and change the above rules accordingly.

FreeBSD:

Given you have no firewall defined yet, you can use the following configuration to have ipfw properly redirect traffic back to sslh

/etc/rc.conf
firewall_enable="YES"
firewall_type="open"
firewall_logif="YES"
firewall_coscripts="/etc/ipfw/sslh.rules"

/etc/ipfw/sslh.rules

#! /bin/sh

# ssl
ipfw add 20000 fwd 192.0.2.1,443 log tcp from 192.0.2.1 8443 to any out
ipfw add 20010 fwd 2001:db8::1,443 log tcp from 2001:db8::1 8443 to any out

# ssh
ipfw add 20100 fwd 192.0.2.1,443 log tcp from 192.0.2.1 8022 to any out
ipfw add 20110 fwd 2001:db8::1,443 log tcp from 2001:db8::1 8022 to any out

# xmpp
ipfw add 20200 fwd 192.0.2.1,443 log tcp from 192.0.2.1 5222 to any out
ipfw add 20210 fwd 2001:db8::1,443 log tcp from 2001:db8::1 5222 to any out

# openvpn (running on other internal system)
ipfw add 20300 fwd 192.0.2.1,443 log tcp from 198.51.100.7 1194 to any out
ipfw add 20310 fwd 2001:db8::1,443 log tcp from 2001:db8:1::7 1194 to any out

General notes:

This will only work if sslh does not use any loopback addresses (no 127.0.0.1 or localhost), you’ll need to use explicit IP addresses (or names):

sslh --listen 192.168.0.1:443 --ssh 192.168.0.1:22 --ssl 192.168.0.1:4443

This will not work:

sslh --listen 192.168.0.1:443 --ssh 127.0.0.1:22 --ssl 127.0.0.1:4443

Transparent proxying means the target server sees the real origin address, so it means if the client connects using IPv6, the server must also support IPv6. It is easy to support both IPv4 and IPv6 by configuring the server accordingly, and setting sslh to connect to a name that resolves to both IPv4 and IPv6, e.g.:

    sslh --transparent --listen :443 --ssh insideaddr:22

    /etc/hosts:
    192.168.0.1  insideaddr
    201::::2     insideaddr

Upon incoming IPv6 connection, sslh will first try to connect to the IPv4 address (which will fail), then connect to the IPv6 address.

SYSTEMD SOCKET ACTIVATION

If compiled with USESYSTEMD then it is possible to activate the service on demand and avoid running any code as root.

In this mode any listen configuration options are ignored and the sockets are passed by systemd to the service.

Example socket unit:

[Unit]
Before=sslh.service

[Socket]
ListenStream=1.2.3.4:443
ListenStream=5.6.7.8:444
ListenStream=9.10.11.12:445
FreeBind=true

[Install]
WantedBy=sockets.target

Example service unit:

[Unit]
PartOf=sslh.socket

[Service]
ExecStart=/usr/sbin/sslh -v -f --ssh 127.0.0.1:22 --ssl 127.0.0.1:443
KillMode=process
CapabilityBoundingSet=CAP_NET_BIND_SERVICE CAP_NET_ADMIN CAP_SETGID CAP_SETUID
PrivateTmp=true
PrivateDevices=true
ProtectSystem=full
ProtectHome=true
User=sslh

With this setup only the socket needs to be enabled. The sslh service will be started on demand and does not need to run as root to bind the sockets as systemd has already bound and passed them over. If the sslh service is started on its own without the sockets being passed by systemd then it will look to use those defined on the command line or config file as usual. Any number of ListenStreams can be defined in the socket file and systemd will pass them all over to sslh to use as usual.

To avoid inconsistency between starting via socket and starting directly via the service Requires=sslh.socket can be added to the service unit to mandate the use of the socket configuration.

Rather than overwriting the entire socket file drop in values can be placed in /etc/systemd/system/sslh.socket.d/.conf with additional ListenStream values that will be merged.

In addition to the above with manual .socket file configuration there is an optional systemd generator which can be compiled - systemd-sslh-generator

This parses the /etc/sslh.cfg (or /etc/sslh/sslh.cfg file if that exists instead) configuration file and dynamically generates a socket file to use.

This will also merge with any sslh.socket.d drop in configuration but will be overriden by a /etc/systemd/system/sslh.socket file.

To use the generator place it in /usr/lib/systemd/system-generators and then call systemctl daemon-reload after any changes to /etc/sslh.cfg to generate the new dynamic socket unit.

FAIL2BAN

If using transparent proxying, just use the standard ssh rules. If you can’t or don’t want to use transparent proxying, you can set fail2ban rules to block repeated ssh connections from a same IP address (obviously this depends on the site, there might be legimite reasons you would get many connections to ssh from the same IP address…)

See example files in scripts/fail2ban.

COMMENTS? QUESTIONS?

You can subscribe to the sslh mailing list here: http://rutschle.net/cgi-bin/mailman/listinfo/sslh

This mailing list should be used for discussion, feature requests, and will be the prefered channel for announcements.

from https://www.rutschle.net/tech/sslh/README.html
-------------------------------------------------

443端口共用的方案

简述

TCP协议中，主机的IP地址加上端口号作为TCP连接的端点，这种端点就叫做套接字（socket）。一般情况下，一个socket同一时刻只能由一个应用监听，对于只有单个公网IP的主机来说，一个对外的端口就是一个socket，例如常用端口约定的服务：

• 22:SSH服务
• 80:HTTP服务
• 443:HTTPS服务
  ...

在某些特殊需求的情况下，我们可能需要一个端口同时提供多个服务，例如：ssh/http 共用 80 端口的讨论。

总的来说，实现端口共用的思路就好比是Nginx的SNI：用某个前端程序(类比:Nginx)监听需要共用的对外端口A(类比:域名)，将需要共用的程序(例如:php-fpm、uwsgi)分别监听环路上的不同端口A2、A3(类比:本地套接字)，前端程序通过区分特定的包格式(类比:域名)将外部发往端口A的数据分别发给端口A2、A3再根据后端响应发回。

因为Nginx提供的Web服务工作在应用层，所以反向代理时可以直接通过URI来确定使用哪个后端；而一般工作在传输层的端口复用，实际上要解决最主要问题是如何区分复用数据并正确发给处理后端。

---

实现

I. 预期目标

为了尽可能避免被QoS和受限网络的影响，现希望将某科学上网方式作为复用数据与正常的HTTPS服务共用443端口，在两者都能够提供服务的同时，HTTPS能够获取前端访客的IP地址。

II. 服务结构

除了自己写前端处理程序，现成的方案有用Haproxy来区分不同数据来实现前端，Nginx和科学上网工具分别按照原配置改为在本地回环监听，其中Nginx需要稍作修改以获取前端发来的IP地址。

III. Haproxy配置

这里用的Haproxy版本为1.5.18，安装过程不再赘述，主要提提配置的要点。

    listen https-in
        bind :443
        tcp-request inspect-delay 5s
        acl is_ssl req_ssl_ver 3:4
        tcp-request content accept if is_ssl
        server server-https :4431 check send-proxy
        use_backend ss-out if !is_ssl

https-in是一个listen，这是好比原来Haproxy里fronted和backend的合并体，上面配置语句的功能如下：

• bind :443 设置监听所有IP的443端口
• tcp-request inspect-delay 设置等待数据传输的最大超时时间
• acl is_ssl req_ssl_ver 3:4 设置一条acl规则，判断是否为ssl连接
• tcp-request content accept if is_ssl 设置如果上一条件成立则直接使用本listen里的server
• server server-https :4431 check send-proxy
  设置本listen里名为server-https的后端服务，send-proxy表示向后端发送相关代理信息，后端获取访客的IP地址就是通过这条指令实现的
• use_backend ss-out if !is_ssl 如果acl规则不成立，则使用名为ss-out的后端

这里最主要的配置就是send-proxy，网上后端获取前端传去的真实IP地址方法是通过Haproxy解析，然后使用option httpclose和option forwardfor之类的指令。这种方法在转发四层TCP数据的时候是无效的！要实现四层TCP数据带源地址转发需要修改前端源码通过重新改包实现。

好在Haproxy支持代理协议(PROXY protocol)，这是是Haproxy的作者Willy Tarreau于2010年开发和设计的一个协议，通过为tcp添加一个很小的头信息，来方便的传递客户端信息（协议栈、源IP、目的IP、源端口、目的端口等)，在网络情况复杂又需要获取客户IP时非常有用。这里的send-proxy指令就是使用了这套协议来向后端传值。

    backend ss-out
        mode tcp
        server D-T-us0 127.0.0.1:1234
        server D-T-us1 127.0.0.1:4321 backup

backend是Haproxy的后端，用来提供服务，上面的配置提供了一个127.0.0.1:1234的服务和一个127.0.0.1:4321备用服务。

IV. Nginx配置

Nginx主要是修改绑定的端口，增加proxy_protocol标记表示使用代理协议，再修改好set_real_ip_from、real_ip_header以及日志格式，其他的没什么实质性内容，不再多说。

    log_format  main  '$proxy_protocol_addr - [$remote_addr] - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    server
    {
    listen 4431 ssl http2 proxy_protocol;
    set_real_ip_from xx.xx.xx.xx;
    real_ip_header proxy_protocol;

    include /etc/nginx/sslport.conf;
    server_name  m.mydomain.com;
    ssl on;
    
    include /etc/nginx/sslon.conf;
    root  /data/www/sgk/;
    
    try_files $uri $uri/ /index.php?$args;
    location ~ \.php$
        {
          fastcgi_pass  unix:/var/run/php7-fpm.socket;
          fastcgi_index index.php;
          include fastcgi_params;
        }
    }

V. Dtunnel配置

这个在'通过KCP协议加速科学上网'里讲过，其他类型的服务也是可以的。

效果

在443端口复用的情况下，Dtunnel的话基本能跑满带宽.

Nginx能够获取Haproxy传来的访客IP地址，性能影响微弱。