Total Pageviews

Sunday 12 June 2016

HTTPTunnel

Tunneling (through SOCKS) using only http requests.
Proxy server that uses only HTTP request for the data transferring.
On applications perspective, HTTP Tunnel tries to provide a general purpose communication layer using only HTTP request. On developers perspective, it is a research on how HTTP may be used for streaming general purpose data (both upload and download).
Right now it is fully functional but a bit slow for highly interactive applications (see "Known bugs and Limitations"). For web browsing, it practically does not affect the user experience.

Installing

Three tiers are involved in the HTTP Tunnel: 1. SOCKS5 server. Will be run locally on your machine. It adapts the communication to the HTTP protocol. 2. CGI capable HTTP web server. Will forward the data to the connection pool. 3. Connection pool daemon. Will establish the actual connection with destination.
All coding was done using Python 3.3, so both your machine and web server must have it installed.
The SOCKS5 server will run locally on your machine. While the CGI script and connection pool must be installed on a server.

Remote machine (the web server)

  1. Run run_daemon.py, it will open the connection pool daemon on the default port.
  2. Put the source code in your web server's appropriate folder. You will need the URL for the index.py, which will receive the HTTP requests and forward it to the connection pool.

Local machine

  1. Just run run_client.py url port. Where url is the URL to the index.py in your web server, and port is which port the SOCKS will listen to.

How it works

HTTP Tunnel creates a SOCKS5 server on the local machine. When a new connection is issued to the SOCKS5 server it will redirect the data to an external server through HTTP requests. The CGI script that receives HTTP requests will just redirect data to a connection pool daemon that keeps the real connection (the connection with the actual destination) alive between requests. This daemon is necessary because the end-to-end communication comprises lot of small HTTP Requests.
The communication goes through the following path:
  • Alice
    • Application protocol over SOCKS5 Protocol
  • SOCKS5 server
    • Internal protocol over pure HTTP
  • HTTP Server
    • Internal low level protocol
  • Connections pool
    • Application protocol
  • Bob
Note that the interface of the local machine with the external world is at "Internal protocol over pure HTTP", this means that for external observers you are just doing HTTP requests.

Features

HTTP Tunnel works with HTTP and HTTPS (secure HTTP), any application that works with SOCKS5 shall work fine with HTTP Tunnel.

Known bugs and Limitations

Right now, HTTP Tunnel is to slow for interactive applications. Tests with SSH showed that the a few seconds are needed to feedback key presses.
This happens because data uploading through HTTP Tunnel is unoptimized. While it is easy to stream data from HTTP (see Chunked transfer encoding), it is hard to stream data to HTTP, because the client must send the length of uploaded content in the request headers.
Also with SSH, if the remote burst information (tested with strings /dev/urandom), the connection will be lost within a few minutes. The cause of this behavior is still unknown.
And finally, the thread handling on the SOCKS server is a bit poor and apparently some threads are alive even after their respective connections were closed.

Future Work

  • Speed and latency optimization (which is the major issue right now)
  • Figure out an way to stream data through HTTP
  • Code refactor
  • Implement reverse proxy (opening a port on the remote server that will redirect to a local server)
  • Fully implement SOCKS5 server and improve its error reporting.
  • Modularization on the protocols. SOCKS5 should become just a wrapper for the actual HTTP protocol, and the HTTP protocol may be used directly. I.e. create something like a socket but using HTTP as communication layer。

from https://github.com/Andrepuel/HttpTunnel
-------

Bidirectional data stream tunnelled in HTTP requests. https://www.gnu.org/software/httptunnel/

About

httptunnel creates a bidirectional virtual data path tunnelled in HTTP
requests. The requests can be sent via an HTTP proxy if so desired.
This can be useful for users behind restrictive firewalls. If WWW
access is allowed through an HTTP proxy, it's possible to use
httptunnel and, say, telnet or PPP to connect to a computer outside
the firewall.
If you still don't understand what this is all about, maybe you
can find some useful information in the FAQ file.
This program is mostly intended for technically-oriented users.
They should know what to do.

Install

Read INSTALL for instructions on how to build a released version.
If you build the development repository, run ./autogen.sh first.

License

httptunnel is free software. See COPYING for terms and conditions.
If you like it, I would appreciate if you sent a post card to:
Lars Brinkhoff
Bokskogsbacken 66 422 56 Goteborg
Sweden
Information and/or latest release should be available from these places:
I take no responsibility for what you do with this software. It has
the potential to do dangerous things, like disabling the protection
you system administrator has set up for the local network. Read the
DISCLAIMER file.

Usage & Documentation

There are two programs: hts and htc. hts is the httptunnel server
and htc is the client. hts should be installed on a computer outside
the HTTP proxy, and htc should be installed on your local computer.
Documentation about how to use the programs should be searched in this
order:
  1. source code
  2. --help output
  3. FAQ
  4. README
Having said that, here are some examples:
  • start httptunnel server:
  • At host REMOTE, start hts like this:
    hts -F localhost:23 8888 (set up httptunnel server to listen on port 8888 and forward to localhost:23)
  • start httptunnel client:
    • At host LOCAL, start htc like this:
      htc -F 2323 -P PROXY_ADDRESS:8000 REMOTE_IP:8888 (set up httptunnel client to forward localhost:2323 to REMOTE_IP:8888 via a local proxy at PROXY_ADDRESS:8000)
  • or, if using a buffering HTTP proxy:
    htc -F 2323 -P PROXY_ADDRESS:8000 -B 48K REMOTE_IP:8888
  • Now you can do this at host LOCAL:
    telnet localhost 2323 (telnet in to REMOTE_IP:8888 via your httptunnel you just configured above on port localhost:2323)
    ...and you will hopefully get a login prompt from host REMOTE_IP.
  • Debugging:
  • For debug output, add -Dn to the end of a command, where n is the level of debug output you'd like to see, with 0 meaning no debug messages at all, and 5 being the highest level (verbose).
  • ex: htc -F 10001 -P PROXY_ADDRESS:8000 REMOTE_IP:8888 -D5 will show verbose debug output (level 5 debugging) while setting up an httptunnel client to forward localhost:10001 to REMOTE_IP:8888 via a local proxy at PROXY_ADDRESS:8000

External help, examples, & links

from https://github.com/larsbrinkhoff/httptunnel
---------

GNU HTTP Tunnel (http://www.nocrew.org/software/httptunnel.html)是一个开源的http-tunnel项目,包括了tunnel server(hts命令)和tunnel client(htc命令),有unix和windows版本。我们就用它来建立自己的tunnel。

1。静态tunnel
        http-tunnel是一个完全透明的通道,直接将你的连接forward给目标服务端口,因此当你连接tunnel的本地侦听端口时,就相当于直接连接到目标服务端口。例如你要建立一条可以访问外部POP3服务器的隧道,可以建立如下连接:

                    htc  ------------> http proxy ------------------------> hts ------------------------------> POP3 serve
        (localhost:8888)             (proxyhost:3128)       (tunnelserver:80)                           (pop3server:110)
在你自己的机器上运行htc,外部充当tunnel server的机器上的80端口运行hts,htc将数据打包成http请求,通过proxy连接到hts,hts解包后将连接forward给POP3服务器。命令如下:
      在tunnel server机器上:
            hts --forward-port pop3server:110 80       
            (将pop3server替换成实际的IP)
      在本地机器上:
            htc --forward-port 8888 --proxy proxyhost:3128 tunnelserver:80 
            (将proxyhost和tunnelserver替换成实际的IP)

      通过这样的配置,你可以用Outlook或Foxmail连接本机的8888端口,就相当于直接连接到POP3服务器了。

2。动态的tunnel。
        上面建立了一条可以访问POP3服务的隧道,但缺点是只能访问某一个指定的POP3服务器,要访问其他的服务器还得按同样的方法再建立一条隧道,很不方 便。既然hts可以将连接forward给POP3服务器,那让它forward给一个SOCKS5服务,不是就可以实现动态的tunnel,可以连接任 意服务了吗?yeah!没错!我们建立这样的连接:

                    htc  ------------> http proxy ------------------------> hts ------------------------------> SOCKS5 serve
        (localhost:8888)             (proxyhost:3128)       (tunnelserver:80)                           (socks5server:1080)

命令就不说了,照第一点改一下就行。这样就相当于在localhost:8888运行了一个SOCKS5服务,设置一下你的网络程序(Outlook,NetAnt,FlashGet,QQ......),让他们通过SOCK5访问网络,就OK了。

3。利用http proxy的CONNECT支持。
        大多数http proxy支持CONNECT命令,但一般只支持CONNECT到外部服务器的443(https)端口。这是为了允许访问外部的https服务。由于 porxy对于CONNECT的连接是直接转发,不做任何分析处理或缓存,所以利用CONNECT可以获得比较快的速度。
        由于hts和htc不支持CONNECT连接,我们可以使用另一个专门支持CONNECT的程序DesProxy  http://desproxy.sourceforge.net
来建立一个tunnel。由于使用CONNECT建立了直接的TCP连接,不需要将数据按http格式打包和解包,所以连tunnel server也不需要了,只需要在你原来运行hts机器上运行一个SOCKS5就行了,连接如下:

              desproxy  -------------> http proxy ----------------------> SOCKS5 server
         (localhost:8888)           (proxyhost:3128)                    (tunnelserver:443)

        desproxy命令的用法:
             desproxy remote_host remote_port proxy_host proxy_port local_port
       在这里remot_host,remote_port就是tunnelserver:443,proxy_host,proxy_port是porxyhost:3128,local_port就是8888。

        注意必须把SOCKS5运行在443端口,如果运行在其他端口的话,CONNECT请求会被http proxy拒绝。同样,我们在localhost:8888得到了一个可以访问外部的SOCK5服务。

4。最简单,最安全而且快速的方式:利用SSH + CONNECT。
       实际上SSH提供了SOCKS5的功能,利用ssh客户端或PuTTY可以在本地建立一个SOCKS5服务,而且PuTTY也直接支持http proxy,最大的好处是ssh的数据连接是加密的,保证了数据的安全。使用ssh的连接如下:
         PuTTY(或plink) ------------> http  proxy  ---------------------> ssh server
      (localhost:8888)                 (proxyhost:3128)                     (tunnelserver:443->22)
     首先我们要让ssh server在443端口侦听,ssh默认端口是22,我们可以修改ssh的配置,或用iptables将443端口重定向到22端口,服务端的配置就 OK了。然后在PuTTY建立一个new session,填上ssh服务器的ip和port;在"Connection->Proxy"页,填上http proxy的ip和port;在"SSH -> Tunnels"页,"Source port"填本地的端口,在这里我们用8888",Destination"选"Dynamic",按"Add"将这个forward port加上,就OK了。配置完成后,用PuTTY登陆上ssh,用netstat -an可以看到PuTTY已经在localhost:8888侦听了,这是一个SOCKS5服务,下面改怎么用,就不用我罗嗦了吧:-)。另外在 "SSH"页,可以根据要求选择"Protocol options"。保存session后,也可以用命令行的plink命令来利用这个session:
       plink -load session_name           (session_name就是session保存的名称)
登陆后效果也一样。

http://javasky.172baby.com/posts/123518.html
-------------------------------------

HTTP tunnel not using websockets.

http-tunnel

http-tunnel is client and server application which allows tunneling any TCP connection trought HTTP requests and responses. It does not use WebSocket so it can pass even restrictive HTTP proxies. Each packet is according to direction encoded as address in GET request or as one line of plain text response. It can be used in combination with SHH server with port forwarding enabled or with SOCKS proxy server to relay internet connectivity to restricted environment.

Features

  • tries to establish keep alive connection and thus reduce latency
  • encodes all packets using base64 to pretend plain text content
  • encrypts all packets using AES
  • is able to multiplex many TCP connections in one HTTP tunnel
  • dynamically changes request rate according to activity in all connections

Drawbacks

  • server is not able to serve more clients at the same time (however it is possible to run multiple instances on different base addresses using Nginx)
  • does not handle packet loss in any way (so TCP connection may get corrupted and it depend on application layer whether it can detect such situation)

Compilation

http-tunnel requires POCO libraries. On Debian libpoco-dev package is required for compilation. Run make in Tunnel/Release directory to compile.

Usage

Both client and server can be run using same executable by supplying appropriate arguments. Command line arguments are following:

  • -P--proxy=PORT starts http-tunnel in client mode, in this mode TCP connections are accepted at selected port and tunneled by outgoing HTTP connection
  • -R--relay=PORT starts http-tunnel in server mode, in this mode HTTP server is started and tunneled TCP connections are relayed to selected port at localhost
  • -t--timeout=SECONDS sets time in seconds after which is an inactive connection closed, default is 300 seconds
  • -s--password=STRING sets the key for AES encryption, any string of any length can be used, it is highly recommended to set it, default value is yK!xhk+S*59fnJua
  • -h--host=HOSTNAME sets hostname of the relay server in client mode, is not used in server mode
  • -p--port=PORT sets HTTP server port in both modes, default is 80
  • -b--base=ADDRESS sets base URL at which tunnel is mapped in both modes, defaults to /
  • -f--fake=HOSTNAME sets hostname to use in host field of HTTP request, defaults to real hostname
  • -v--verbosity=LEVEL sets logger verbosity, 0 disables logging, 8 is the most verbose, default is 0
from https://github.com/jakub-vanik/http-tunnel

相关帖子:briteming.blogspot.com/2016/03/http-tunnel.html