Total Pageviews

Monday, 5 October 2020

Go-HTTP-tunnel

 

 GoDoc Go Report Card Build Status Github All Releases

Go HTTP tunnel is a reverse tunnel based on HTTP/2. It enables you to share your localhost when you don't have a public IP.

Features:

  • HTTP proxy with basic authentication
  • TCP proxy
  • SNI vhost proxy
  • Client auto reconnect
  • Client management and eviction
  • Easy to use CLI

Common use cases:

  • Hosting a game server from home
  • Developing webhook integrations
  • Managing IoT devices

Installation

Build the latest version.

$ go get -u github.com/mmatczuk/go-http-tunnel/cmd/...

Alternatively download the latest release.

Running

There are two executables:

  • tunneld - the tunnel server, to be run on publicly available host like AWS or GCE
  • tunnel - the tunnel client, to be run on your local machine or in your private network

To get help on the command parameters run tunneld -h or tunnel -h.

Tunnel requires TLS certificates for both client and server.

$ openssl req -x509 -nodes -newkey rsa:2048 -sha256 -keyout client.key -out client.crt
$ openssl req -x509 -nodes -newkey rsa:2048 -sha256 -keyout server.key -out server.crt

Run client:

  • Install tunnel binary
  • Make .tunnel directory in your project directory
  • Copy client.key, client.crt to .tunnel
  • Create configuration file tunnel.yml in .tunnel
  • Start all tunnels
$ tunnel -config ./tunnel/tunnel.yml start-all

Run server:

  • Install tunneld binary
  • Make .tunneld directory
  • Copy server.key, server.crt to .tunneld
  • Start tunnel server
$ tunneld -tlsCrt .tunneld/server.crt -tlsKey .tunneld/server.key

This will run HTTP server on port 80 and HTTPS (HTTP/2) server on port 443. If you want to use HTTPS it's recommended to get a properly signed certificate to avoid security warnings.

Run Server as a Service on Ubuntu using Systemd:

  • After completing the steps above successfully, create a new file for your service (you can name it whatever you want, just replace the name below with your chosen name).
$ vim tunneld.service
  • Add the following configuration to the file
[Unit]
Description=Go-Http-Tunnel Service
After=network.target
After=network-online.target

[Service]
ExecStart=/path/to/your/tunneld -tlsCrt /path/to/your/folder/.tunneld/server.crt -tlsKey /path/to/your/folder/.tunneld/server.key
TimeoutSec=30
Restart=on-failure
RestartSec=30

[Install]
WantedBy=multi-user.target
  • Save and exit this file.
  • Move this new file to /etc/systemd/system/
$ sudo mv tunneld.service /etc/systemd/system/
  • Change the file permission to allow it to run.
$ sudo chmod u+x /etc/systemd/system/tunneld.service
  • Start the new service and make sure you don't get any errors, and that your client is able to connect.
$ sudo systemctl start tunneld.service
  • You can stop the service with:
$ sudo systemctl stop tunneld.service
  • Finally, if you want the service to start automatically when the server is rebooted, you need to enable it.
$ sudo systemctl enable tunneld.service

There are many more options for systemd services, and this is by not means an exhaustive configuration file.

Configuration

The tunnel client tunnel requires configuration file, by default it will try reading tunnel.yml in your current working directory. If you want to specify other file use -config flag.

Sample configuration that exposes:

  • localhost:8080 as webui.my-tunnel-host.com
  • host in private network for ssh connections

looks like this

    server_addr: SERVER_IP:5223
    tunnels:
      webui:
        proto: http
        addr: localhost:8080
        auth: user:password
        host: webui.my-tunnel-host.com
      ssh:
        proto: tcp
        addr: 192.168.0.5:22
        remote_addr: 0.0.0.0:22
      tls:
  	    proto: sni
  	    addr: localhost:443
  	    host: tls.my-tunnel-host.com

Configuration options:

  • server_addr: server TCP address, i.e. 54.12.12.45:5223
  • tls_crt: path to client TLS certificate, default: client.crt in the config file directory
  • tls_key: path to client TLS certificate key, default: client.key in the config file directory
  • root_ca: path to trusted root certificate authority pool file, if empty any server certificate is accepted
  • tunnels / [name]
    • proto: tunnel protocol, http, tcp or sni
    • addr: forward traffic to this local port number or network address, for proto=http this can be full URL i.e. https://machine/sub/path/?plus=params, supports URL schemes http and https
    • auth: (proto=http) (optional) basic authentication credentials to enforce on tunneled requests, format user:password
    • host: (proto=http, proto=sni) hostname to request (requires reserved name and DNS CNAME)
    • remote_addr: (proto=tcp) bind the remote TCP address
  • backoff
    • interval: how long client would wait before redialing the server if connection was lost, exponential backoff initial interval, default: 500ms
    • multiplier: interval multiplier if reconnect failed, default: 1.5
    • max_interval: maximal time client would wait before redialing the server, default: 1m
    • max_time: maximal time client would try to reconnect to the server if connection was lost, set 0 to never stop trying, default: 15m

How it works

A client opens TLS connection to a server. The server accepts connections from known clients only. The client is recognized by its TLS certificate ID. The server is publicly available and proxies incoming connections to the client. Then the connection is further proxied in the client's network.

The tunnel is based HTTP/2 for speed and security. There is a single TCP connection between client and server and all the proxied connections are multiplexed using HTTP/2.

from https://github.com/mmatczuk/go-http-tunnel

-------

HTTP tunnel over Websocket.

WS PROXY

This is a reverse HTTP proxy over websockets. The aim is to securely make call to internal APIs from outside.

How does it works

a WSP client runs in the internal network ( alongside the APIs ) and connects to a remote WSP server with HTTP websockets.

One issue HTTP requests to the WSP server with an extra HTTP header 'X-PROXY-DESTINATION: "http://api.internal/resource"' to the /request endpoint.

The WSP Server then forward the request to the WSP Client over the one of the offered websockets. The WSP Client receive and execute locally an HTTP request to the URL provided in X-PROXY-DESTINATION and forwards the HTTP response back to the WSP server which in turn forwards the response back to the client. Please note that no buffering of any sort occurs.

If several WSP clients connect to a WSP server, requests will be spread in a random way to all the WSP clients.

wsp schema

Get code

go get github.com/root-gg/wsp

WSP server configuration

# wsp_server.cfg
---
host : 127.0.0.1                     # Address to bind the HTTP server
port : 8080                          # Port to bind the HTTP server
timeout : 1000                       # Time to wait before acquiring a WS connection to forward the request (milliseconds)
idletimeout : 60000                  # Time to wait before closing idle connection when there is enough idle connections (milliseconds)
#blacklist :                         # Forbidden destination ( deny nothing if empty )
# - method : ".*"                    #   Applied in order before whitelist
#   url : "^http(s)?://google.*"     #   None must match
#   headers :                        #   Optinal header check
#     X-CUSTOM-HEADER : "^value$"    # 
#whitelist :                         # Allowed destinations  ( allow all if empty )
# - method : "^GET$"                 #   Applied in order after blacklist
#   url : "^http(s)?://.*$"          #   One must match
#   headers :                        #   Optinal header check
#     X-CUSTOM-HEADER : "^value$"    # 
# secretkey : ThisIsASecret          # secret key that must be set in clients configuration

$ cd wsp_server && go build
$ ./wsp_server -config wsp_server.cfg
{
  "Host": "127.0.0.1",
  "Port": 8080
}
2016/11/22 15:31:39 Registering new connection from 7e2d8782-f893-4ff3-7e9d-299b4c0a518a
2016/11/22 15:31:40 Registering new connection from 7e2d8782-f893-4ff3-7e9d-299b4c0a518a
2016/11/22 15:31:40 Registering new connection from 7e2d8782-f893-4ff3-7e9d-299b4c0a518a
2016/11/22 15:31:40 Registering new connection from 7e2d8782-f893-4ff3-7e9d-299b4c0a518a
2016/11/22 15:31:40 Registering new connection from 7e2d8782-f893-4ff3-7e9d-299b4c0a518a
2016/11/22 15:31:40 Registering new connection from 7e2d8782-f893-4ff3-7e9d-299b4c0a518a
2016/11/22 15:31:40 Registering new connection from 7e2d8782-f893-4ff3-7e9d-299b4c0a518a
2016/11/22 15:31:40 Registering new connection from 7e2d8782-f893-4ff3-7e9d-299b4c0a518a
2016/11/22 15:31:40 Registering new connection from 7e2d8782-f893-4ff3-7e9d-299b4c0a518a
2016/11/22 15:31:40 Registering new connection from 7e2d8782-f893-4ff3-7e9d-299b4c0a518a
2016/11/22 15:33:34 GET map[User-Agent:[curl/7.26.0] Accept:[*/*] X-Proxy-Destination:[https://google.fr]]
2016/11/22 15:33:34 proxy request to 7e2d8782-f893-4ff3-7e9d-299b4c0a518a

For now TLS setup should be implemented using an HTTP reverse proxy like NGinx or Apache...

WSP proxy configuration

# wsp_client.cfg
---
targets :                            # Endpoints to connect to
 - ws://127.0.0.1:8080/register      #
poolidlesize : 10                    # Default number of concurrent open (TCP) connections to keep idle per WSP server
poolmaxsize : 100                    # Maximum number of concurrent open (TCP) connections per WSP server
#blacklist :                         # Forbidden destination ( deny nothing if empty )
# - method : ".*"                    #   Applied in order before whitelist
#   url : ".*forbidden.*"            #   None must match
#   headers :                        #   Optinal header check
#     X-CUSTOM-HEADER : "^value$"    # 
#whitelist :                         # Allowed destinations  ( allow all if empty )
# - method : "^GET$"                 #   Applied in order after blacklist
#   url : "http(s)?://.*$"           #   One must match
#   headers :                        #   Optinal header check
#     X-CUSTOM-HEADER : "^value$"    # 
# secretkey : ThisIsASecret          # secret key that must match the value set in servers configuration
  • poolMinSize is the default number of opened TCP/HTTP/WS connections to open per WSP server. If there is a burst of simpultaneous requests the number of open connection will rise and then decrease back to this number.
  • poolMinIdleSize is the number of connection to keep idle, meaning that if there is more than this number of simultaneous requests the WSP client will try to open more connections to keep idle connection.
  • poolMaxSize is the maximum number of simultaneous connection that the proxy will ever initiate per WSP server.
$ cd wsp_client && go build
$ ./wsp_client -config wsp_client.cfg
{
  "ID": "7e2d8782-f893-4ff3-7e9d-299b4c0a518a",
  "Targets": [
    "ws://127.0.0.1:8080/register"
  ],
  "PoolMinSize": 10,
  "PoolMinIdleSize": 5,
  "PoolMaxSize": 100
}
2016/11/22 15:31:39 Connecting to ws://127.0.0.1:8080/register
2016/11/22 15:31:40 Connecting to ws://127.0.0.1:8080/register
2016/11/22 15:31:40 Connecting to ws://127.0.0.1:8080/register
2016/11/22 15:31:40 Connecting to ws://127.0.0.1:8080/register
2016/11/22 15:31:40 Connecting to ws://127.0.0.1:8080/register
2016/11/22 15:31:40 Connecting to ws://127.0.0.1:8080/register
2016/11/22 15:31:40 Connecting to ws://127.0.0.1:8080/register
2016/11/22 15:31:40 Connecting to ws://127.0.0.1:8080/register
2016/11/22 15:31:40 Connecting to ws://127.0.0.1:8080/register
2016/11/22 15:31:40 Connecting to ws://127.0.0.1:8080/register
2016/11/22 15:33:34 got request : {"Method":"GET","URL":"https://google.fr","Header":{"Accept":["*/*"],"User-Agent":["curl/7.26.0"],"X-Proxy-Destination":["https://google.fr"]},"ContentLength":0}

Client

$ curl -H 'X-PROXY-DESTINATION: https://google.fr' http://127.0.0.1:8080/request
<!doctype html><html itemscope="" itemtype="http://schema.org/WebPage" lang="fr"><head><meta content="text/html; charset=UTF-8" http-equiv="Content-Type"><meta content="/images/branding/googleg/1x/googleg_standard_color_128dp.png" it...
from https://github.com/root-gg/wsp
Posted by at
Labels:

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)