tproxy is a simple TCP routing proxy (layer 7) built on Gevent that lets you configure the routine logic in Python. It's heavily inspired from proxy machine but have some unique features like the pre-fork worker model borrowed to Gunicorn.
Instalation
tproxy requires Python 2.x >= 2.5. Python 3.x support is planned.
$ pip install gevent $ pip install tproxy
To install from source:
$ git clone git://github.com/benoitc/tproxy.git $ cd tproxy $ pip install -r requirements.txt $ python setup.py install
Test your installation by running the command line:
$ tproxy examples/transparent.py
And go on http://127.0.0.1:5000 , you should see the google homepage.
Usage
$ tproxy -h
Usage: tproxy [OPTIONS] script_path
Options:
--version show program's version number and exit
-h, --help show this help message and exit
--log-file=FILE The log file to write to. [-]
--log-level=LEVEL The granularity of log outputs. [info]
--log-config=FILE The log config file to use. [None]
-n STRING, --name=STRING A base to use with setproctitle for process naming.
[None]
-D, --daemon Daemonize the tproxy process. [False]
-p FILE, --pid=FILE A filename to use for the PID file. [None]
-u USER, --user=USER Switch worker processes to run as this user. [501]
-g GROUP, --group=GROUP
Switch worker process to run as this group. [20]
-m INT, --umask=INT A bit mask for the file mode on files written by
tproxy. [0]
-b ADDRESS, --bind=ADDRESS The socket to bind. [127.0.0.1:8000]
--backlog=INT The maximum number of pending connections. [2048]
--ssl-keyfile=STRING Ssl key file [None]
--ssl-certfile=STRING Ssl ca certs file. contains concatenated
"certification [None]
--ssl-ca-certs=STRING Ssl ca certs file. contains concatenated
"certification [None]
--ssl-cert-reqs=INT Specifies whether a certificate is required from the
other [0]
-w INT, --workers=INT The number of worker process for handling requests. [1]
--worker-connections=INT The maximum number of simultaneous clients per worker.
[1000]
-t INT, --timeout=INT Workers silent for more than this many seconds are
killed and restarted. [30]
Signals
QUIT - Graceful shutdown. Stop accepting connections immediatly
and wait until all connections close
TERM - Fast shutdown. Stop accepting and close all conections
after 10s.
INT - Same as TERM
HUP - Graceful reloading. Reload all workers with the new code
in your routing script.
USR2 - Upgrade tproxy on the fly
TTIN - Increase the number of worker from 1
TTOU - Decrease the number of worker from 1
Exemple of routing script
import re
re_host = re.compile("Host:\s*(.*)\r\n")
class CouchDBRouter(object):
# look at the routing table and return a couchdb node to use
def lookup(self, name):
""" do something """
router = CouchDBRouter()
# Perform content-aware routing based on the stream data. Here, the
# Host header information from the HTTP protocol is parsed to find the
# username and a lookup routine is run on the name to find the correct
# couchdb node. If no match can be made yet, do nothing with the
# connection. (make your own couchone server...)
def proxy(data):
matches = re_host.findall(data)
if matches:
host = router.lookup(matches.pop())
return {"remote": host}
return None
Example SOCKS4 Proxy in 18 Lines
import socket
import struct
def proxy(data):
if len(data) < 9:
return
command = ord(data[1])
ip, port = socket.inet_ntoa(data[4:8]), struct.unpack(">H", data[2:4])[0]
idx = data.index("\0")
userid = data[8:idx]
if command == 1: #connect
return dict(remote="%s:%s" % (ip, port),
reply="\0\x5a\0\0\0\0\0\0",
data=data[idx:])
else:
return {"close": "\0\x5b\0\0\0\0\0\0"}
Example of returning a file
import os
WELCOME_FILE = os.path.join(os.path.dirname(__file__), "welcome.txt")
def proxy(data):
fno = os.open(WELCOME_FILE, os.O_RDONLY)
return {
"file": fno,
"reply": "HTTP/1.1 200 OK\r\n\r\n"
}
Valid return values
- { "remote:": string or tuple } - String is the host:port of the server that will be proxied.
- { "remote": String, "data": String} - Same as above, but send the given data instead.
- { "remote": String, "data": String, "reply": String} - Same as above, but reply with given data back to the client
- None - Do nothing.
- { "close": True } - Close the connection.
- { "close": String } - Close the connection after sending the String.
- { "file": String } - Return a file specify by the file path and close the connection.
- { "file": String, "reply": String } - Return a file specify by the file path and close the connection.
- { "file": Int, "reply": String} - Same as above but reply with given data back to the client
- { "file": Int } - Return a file specify by its file descriptor
- { "file": Int, "reply": String} - Same as above but reply with given data back to the client
Notes:
If sendfile API available it will be used to send a file with "file" command.
The file command can have 2 optionnnal parameters:
- offset: argument specifies where to begin in the file.
- nbytes: specifies how many bytes of the file should be sent
To handle ssl for remote connection you can add these optionals arguments:
- ssl: True or False, if you want to connect with ssl
- ssl_args: dict, optionals ssl arguments. Read the ssl documentation for more informations about them.
Handle errors
You can easily handling error by adding a proxy_error function in your script:
def proxy_error(client, e):
pass
This function get the ClientConnection instance (current connection) as first arguments and the error exception in second argument.
Rewrite requests & responses
Main goal of tproxy is to allows you to route transparently tcp to your applications. But some case you want to do more. For example you need in HTTP 1.1 to change the Host header to make sure remote HTTP server will know what to do if uses virtual hosting.
To do that, add a rewrite_request function in your function to simply rewrite clienrt request and rewrite_response to rewrite the remote response. Both functions take a tproxy.rewrite.RewriteIO instance which is based on io.RawIOBase class.
See the httprewrite.py example for an example of HTTP rewrite.
from https://github.com/benoitc/tproxy
--------------------------------------------------
tproxy实现透明代理
--------------------------------------------------
tproxy实现透明代理
Transparent proxy support
This feature adds Linux 2.2-like transparent proxy support to current kernels.To use it, enable NETFILTER_TPROXY, the socket match and the TPROXY target inyour kernel config. You will need policy routing too, so be sure to enable thatas well.
1. Making non-local sockets work
================================
The idea is that you identify packets with destination address matching a localsocket on your box, set the packet mark to a certain value, and then match on thatvalue using policy routing to have those packets delivered locally:
# iptables -t mangle -N DIVERT
# iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT
# iptables -t mangle -A DIVERT -j MARK --set-mark 1
# iptables -t mangle -A DIVERT -j ACCEPT
# ip rule add fwmark 1 lookup 100
# ip route add local 0.0.0.0/0 dev lo table 100
Because of certain restrictions in the IPv4 routing output code you'll have to modify your application to allow it to send datagrams _from_ non-local IP addresses. All you have to do is enable the (SOL_IP, IP_TRANSPARENT) socket option before calling bind:
fd = socket(AF_INET, SOCK_STREAM, 0);
int value = 1;
setsockopt(fd, SOL_IP, IP_TRANSPARENT, &value, sizeof(value));
name.sin_family = AF_INET;
name.sin_port = htons(0xCAFE);
name.sin_addr.s_addr = htonl(0xDEADBEEF);
bind(fd, &name, sizeof(name));
A trivial patch for netcat is available here:
http://people.netfilter.org/hidden/tproxy/netcat-ip_transparent-support.patch
2. Redirecting traffic
======================
Transparent proxying often involves "intercepting" traffic on a router. This is usually done with the iptables REDIRECT target; however, there are serious limitations of that method. One of the major issues is that it actually modifies the packets to change the destination address -- which might not be acceptable in certain situations. (Think of proxying UDP for example: you won't be able to find out the original destination address. Even in case of TCPgetting the original destination address is racy.)
The 'TPROXY' target provides similar functionality without relying on NAT. Simply
add rules like this to the iptables ruleset above:
# iptables -t mangle -A PREROUTING -p tcp --dport 80 -j TPROXY \
--tproxy-mark 0x1/0x1 --on-port 50080
Note that for this to work you'll have to modify the proxy to enable (SOL_IP,
IP_TRANSPARENT) for the listening socket.
3. Iptables extensions
======================
To use tproxy you'll need to have the 'socket' and 'TPROXY' modules
compiled for iptables. A patched version of iptables is available
here: http://git.balabit.hu/?p=bazsi/iptables-tproxy.git
4. Application support
======================
4.1. Squid
----------
Squid 3.HEAD has support built-in. To use it, pass'--enable-linux-netfilter' to configure and set the 'tproxy' option on the HTTP listener you redirect traffic to with the TPROXY iptables target.
