A toolkit for Network Extension Framework https://zhuhaow.github.io/NEKit
The lifetime of a
https://github.com/zhuhaow/Specht
NEKit is the successor of Soca. The main goal of NEKit is to provide things needed in building a Network Extension app with
NETunnelProvider to bypass network filtering and censorship while keep the framework as non-opinionated as possible.
NEKit does not depend on Network Extension framework. You can use NEKit without Network Extension entitlement to build a rule based proxy in a few lines.
There are two demos that you should check out.
SpechtLite does not require Network Extension and anyone can use it.
Specht is another demo that requires Network Extension entitlement.
Currently, NEKit supports:
- Forward requests through different proxies based on remote host location, remote host domain or the connection speed of proxies.
- Integrated tun2socks framework to reassemble TCP packets into TCP flows.
- A DNS server that rewrites request and response.
- Some tools to build IP packets.
- ...
Check document here.
Also, you may be more interested in Potatso if you just need an open source iOS app with GUI supporting shadowsocks.
Wingy is a free app built with NEKit available on App Store for your iDevice. Note Wingy is not created by or affiliated with me.
If you have any questions (not bug report), please join Gitter or Telegram instead of opening an issue.
Principle
NEKit tries to be as flexible and non-opionated as possible.
However, it is not always as modular as you may think if you want to reproduce transport layer from network layer.
NEKit follows one fundamental principle to keep the best network performance: The device connecting to target server directly resolves the domain.
This should not be a problem if the applications on your device connect to the local proxy server directly, where we can get the request domain information then send that to remote proxy server if needed.
But consider that if an application tries to make a socket connection by itself, which generally consists of two steps:
- Make a DNS lookup to find the IP address of the target server.
- Connect to the remote server by socket API provided by the system.
We can read only two independent things from the TUN interface, a UDP packet containing the DNS lookup request and a TCP flow consisting of a serial of TCP packets. So there is no way we can know the initial request domain for the TCP flow. And since there may be multiple domains served on the same host, we can not get the origin domain by saving the DNS response and looking that up reversely later.
The only solution is to create a fake IP pool and assign each requested domain with a unique fake IP so we can look that up reversely. Every connection needs to look that up from the DNS server afterwards; this is the only non-modular part of NEKit which is already encapsulated in
ConnectRequest.Usage
Add it to you project
I recommend adding this project to your project, which is easier to debug.
However, you can still use it with Carthage (you'll need Carthage anyway since NEKit uses Carthage) by adding
github "zhuhaow/NEKit"
to you
Cartfile.
Use
carthage update --no-use-binaries --platform mac,ios to install all frameworks. Do not use pre-compiled binaries since some of them might be buggy.Overview
NEKit basically consists of two parts, a proxy server forwarding socket data based on user defined rules and an IP stack reassembling IP packets back to TCP flow as a socket.
Rule manager
Before starting any proxy server, we need to define rules.
Each rule consists of two parts, one defining what kinding of request matches this rule and the other defining what adapter to use. An adapter represents the abstraction of a socket connection to a remote proxy server (or remote host). We use
AdapterFactory to build adapters.
NEKit provides
AdapterSocket supporting HTTP/HTTPS/SOCK5 proxy and Shadowsocks(AES-128-CFB/AES-192-CFB/AES-256-CFB/chacha20/salsa20/rc4-md5). Let me know if there is any other type of proxy needed. You can also implement your own AdapterSocket.// Define remote adapter first
let directAdapterFactory = DirectAdapterFactory()
let httpAdapterFactory = HTTPAdapterFactory(serverHost: "remote.http.proxy", serverPort: 3128, auth: nil)
let ssAdapterFactory = ShadowsocksAdapterFactory(serverHost: "remote.ss.proxy", serverPort: 7077, encryptMethod: "AES-256-CFB", password: "1234567890")
// Then create rules
let chinaRule = CountryRule(countryCode: "CN", match: true, adapterFactory: directAdapterFactory)
// `urls` are regular expressions
let listRule = try! ListRule(adapterFactory: ssAdapterFactory, urls: ["some\\.site\\.does\\.not\\.exists"])
let allRule = AllRule(adapterFactory: httpAdapterFactory)
// Create rule manager, rules will be matched in order.
let manager = RuleManager(fromRules: [listRule, chinaRule, allRule], appendDirect: true)
// Set this manager as the active manager
RuleManager.currentManager = ruleManager
There is also
Configuration to load rules from a Yaml config file. But that is not recommended.Proxy server
Now we can start a HTTP/SOCKS5 proxy server locally.
let server = GCDHTTPProxyServer(address: IPv4Address(fromString: "127.0.0.1"), port: Port(port: 9090)
try! server.start()
Now there is a HTTP proxy server running on
127.0.0.1:9090 which will forward requests based on rules defined in RuleManager.currentManager.
If you do not want to handle IP packets, then that's it, just set the proxy to
127.0.0.1:9090 in System Preferences and you are good to go.
If you want to read on, you will have to request Network Extention entitlement from Apple.
But even if you use NetworkExtention to set up the network proxy, it does not mean you have to process packets, just do not route anything to the TUN interface and do not set up
IPStack. For iOS, if you claim you have implemented it but just do nothing the users probably will never notice.IP stack
Assuming you already set up a working extension with correct routing configurations (Google how, this is not trivial).
In
startTunnelWithOptions(options: [String : NSObject]?, completionHandler: (NSError?) -> Void)
set
RuleManager and start proxy server(s) and then create an instance representing the TUN interface bylet stack = TUNInterface(packetFlow: packetFlow)
We also have to set
RawSocketFactory.TunnelProvider = self
to create socket to connect to remote servers with
NETunnelProvider.
Then we need to register ip stacks implementing
IPStackProtocol to process IP packets.
NEKit provides several stacks.
TCPStack
TCPStack process TCP packets and reassembles them back into TCP flows then send them to the proxy server specified by proxyServer variable. You have to set proxyServer before registering TCPStack to TUNInterface.DNSServer
DNS server is implemented as an IP stack processing UDP packets send to it.
First create an DNS server with a fake IP pool. (You should use fake IP, but you can disable it if you want to by set it to nil.)
let fakeIPPool = IPv4Pool(start: IPv4Address(fromString: "172.169.1.0"), end: IPv4Address(fromString: "172.169.255.0"))
let dnsServer = DNSServer(address: IPv4Address(fromString: "172.169.0.1"), port: Port(port: 53), fakeIPPool: fakeIPPool)
Then we have to define how to resolve the DNS requests, NEKit provides the most trivial one which sends the request to remote DNS server directly with UDP protocol, you can do anything you want by implementing
DNSResolverProtocol.let resolver = UDPDNSResolver(address: IPv4Address(fromString: "114.114.114.114"), port: Port(port: 53))
dnsServer.registerResolver(resolver)
It is very important to set
DNSServer.currentServer = dnsServer
so we can look up the fake IP reversely.
UDPDirectStack
UDPDirectStack sends and reads UDP packets to and from remote server directly.
You can register these stack to TUN interface by
interface.registerStack(dnsServer)
// Note this sends out every UDP packets directly so this must comes after any other stack that processes UDP packets.
interface.registerStack(UDPDirectStack())
interface.registerStack(TCPStack.stack)
When everything is set up, you should start processing packets by calling
interface.start() in the completion handler of setTunnelNetworkSettings.Event
You can use
Observer<T> to observe the events in proxy servers and sockets. Check out observers in DebugObserver.swift as an example.Dive in
Framework overview
The structure of the proxy server is given as follows:
┌──────────────────────────────────────────────────┐
│ │
│ ProxyServer │
│ │
├──────┬───┬──────┬───┬──────┬───┬──────┬───┬──────┤
│Tunnel│ │Tunnel│ │Tunnel│ │Tunnel│ │Tunnel│
└──────┘ └──────┘ └───▲──┘ └──────┘ └──────┘
╱ ╲
╱───────╱ ╲─────╲
╱ ╲
┌────────▼────────┐ ┌────────▼────────┐
│ ProxySocket │ │ AdapterSocket │
└────────▲────────┘ └────────▲────────┘
│ │
│ │
┌────────▼────────┐ ┌────────▼────────┐
│RawSocketProtocol│ │RawSocketProtocol│
└────────▲────────┘ └────────▲────────┘
│ │
│ │
╔══════▼═══════╗ ╔═══════▼══════╗
║ LOCAL ║ ║ REMOTE ║
╚══════════════╝ ╚══════════════╝
When a new socket is accepted from the listening socket of the proxy server, it is wrapped in some implemention of
RawSocketProtocol as a raw socket which just reads and writes data.
Then it is wrapped in a subclass of
ProxySocket which encapsulates the proxy logic.
The
TCPStack wraps the reassembled TCP flow (TUNTCPSocket) in DirectProxySocket then send it to the mainProxyserver.
Similarly,
AdapterSocket encapsulated the logic of how to connect to remote and process the data flow.
Pretty much everything of NEKit follows the delegation pattern. If you are not familiar with that, you should learn it first, probabaly by learning how to use CocoaAsyncSocket (note that the sockets in NEKit are not thread-safe which is different from GCDAsyncSocket).
The lifetime of a Tunnel
When a
RawSocketProtocol socket is accepted or created by TCPStack, it is wrapped in a ProxySocket then in a Tunnel. The Tunnel will call proxySocket.openSocket() to let the proxy socket start process data.
When the
ProxySocket read enough data to build a ConnectRequest, it calls func didReceiveRequest(request: ConnectRequest, from: ProxySocket) of the delegate (which should be the Tunnel).
The
Tunnel then matches this request in RuleManager to get the corresponding AdapterFactory. Then func openSocketWithRequest(request: ConnectRequest) of the produced AdapterSocket is called to connect to remote server.
The
AdapterSocket calls func didConnect(adapterSocket: AdapterSocket, withResponse response: ConnectResponse) of the Tunnel to let the ProxySocket has a chance to respond to remote response. (This is ignored as of now.)
Finally, when the
ProxySocket and AdapterSocket are ready to forward data, they should call func readyToForward(socket: SocketProtocol) of the Tunnel to let it know. When both sides are ready, the tunnel will read from both sides and then send the received data to the other side intact.
When any side of the tunnel is disconnected, the
func didDisconnect(socket: SocketProtocol) is called then both sides are closed actively. The Tunnel will be released when both sides disconnect successfully.
from https://github.com/zhuhaow/NEKit
https://zhuhaow.me/NEKit/
相关：
https://github.com/zhuhaow/libnekithttps://github.com/zhuhaow/Specht
--------
https://github.com/zhuhaow/Soca-iOS
----------------
一个基于 NEKit 的网络 Proxy App。
----------------
一个基于 NEKit 的网络 Proxy App。
Leiter
一个基于 NEKit 的网络 Proxy App。
Usage
执行：
> carthage update --no-use-binaries --platform ios
> pod install --repo-update
> open Leiter.xcworkspace
此项目如果要在App上使用，需要99美元的开发者账号
支持的协议：
- Http(s)
- Shadowsock
- Socks5
Contributing
------
https://github.com/709530753/tun2socks
https://github.com/beeuc/tun2socks
https://github.com/luckypoem/tun2socks-2
https://github.com/NN-Booster/tun2socks
https://github.com/wy182000/tun2socks
