Total Pageviews

Sunday 23 July 2017

overture- A DNS server/forwarder

A DNS server/forwarder/dispatcher written in Go.

Build Status GoDoc Go Report Card
Overture is a DNS server/forwarder/dispatcher written in Go.
Overture means an orchestral piece at the beginning of a classical music composition, just like DNS which is nearly the first step of surfing the Internet.
Please note: If you are using the binary releases, please follow the instructions in the README file with corresponding git version tag. The README in master branch are subject to change and does not always reflect the correct instructions to your binary release version.

Features

  • Full IPv6 support
  • Multiple DNS upstream
    • Via UDP/TCP with custom port
    • Via SOCKS5 proxy
    • With EDNS Client Subnet(ECS) RFC7871
  • Dispatcher
    • IPv6 record (AAAA) redirection
    • Custom IP network
    • Custom domain
  • Minimum TTL modification
  • Hosts (prefix wildcard, random order of multiple answers)
  • Cache with ECS

Dispatch process

Overture forces IPv6 and custom domain DNS queries to use alternative DNS when applicable.
As for custom IP network, overture will first query the domain with primary DNS, if the answer is empty or the IP is not matched then overture will query the alternative DNS servers and use their answer instead.

Installation

You can download binary releases from the release.
For ArchLinux users, package overture is available in AUR. If you use a AUR helper i.e. yaourt, you can simply run:
yaourt -S overture
For mips users, please assure the kernel FPU emulation is enabled, check #32 #26 golang/go#18880 for details.

Usages

Start with the default config file -> ./config.json
$ ./overture
Or use your own config file:
$ ./overture -c /path/to/config.json
Verbose mode:
$ ./overture -v
Log to file:
$ ./overture -l /path/to/overture.log
For other options, please see help:
$ ./overture -h
Tips:
  • Root privilege is required if you are listening on port 53.
  • For Windows users, you can run overture on command prompt instead of double click.

Configuration Syntax

Configuration file is "config.json" by default:
{
  "BindAddress": ":53",
  "PrimaryDNS": [
    {
      "Name": "DNSPod",
      "Address": "119.29.29.29:53",
      "Protocol": "udp",
      "SOCKS5Address": "",
      "Timeout": 6,
      "EDNSClientSubnet": {
        "Policy": "disable",
        "ExternalIP": ""
      }
    }
  ],
  "AlternativeDNS": [
    {
      "Name": "OpenDNS",
      "Address": "208.67.222.222:443",
      "Protocol": "tcp",
      "SOCKS5Address": "",
      "Timeout": 6,
      "EDNSClientSubnet": {
        "Policy": "disable",
        "ExternalIP": ""
      }
    }
  ],
  "OnlyPrimaryDNS": false,
  "RedirectIPv6Record": true,
  "IPNetworkFile": "./ip_network_sample",
  "DomainFile": "./domain_sample",
  "DomainBase64Decode": true,
  "HostsFile": "./hosts_sample",
  "MinimumTTL": 0,
  "CacheSize" : 0,
  "RejectQtype": [255]
}
Tips:
  • BindAddress: Specifying only port (e.g. :53) will have overture listen on all available addresses (both IPv4 and IPv6). Overture will handle both TCP and UDP requests.
  • DNS: You can specify multiple DNS upstream servers here.
    • Name: This field is only used for logging.
    • Protocol: tcp or udp
    • SOCKS5Address: Forward dns query to this SOCKS5 proxy, “” to disable.
    • EDNSClientSubnet: Used to improve DNS accuracy. Please check RFC7871 for details.
      • Policy
        • auto: If client IP is not in the reserved IP network, use client IP. Otherwise, use external IP.
        • disable: Disable this feature.
      • ExternalIP: If this field is empty, ECS will be disabled when the inbound IP is not an external IP.
  • OnlyPrimaryDNS: Disable dispatcher feature, use primary DNS only.
  • RedirectIPv6Record: Redirect IPv6 DNS queries to alternative DNS servers.
  • File: Absolute path like /path/to/file is allowed. For Windows users, please use properly escaped path like C:\\path\\to\\file.txt in the configuration.
  • MinimumTTL: Set the minimum TTL value (in seconds) in order to improve caching efficiency, use 0 to disable.
  • CacheSize: The number of query record to cache, use 0 to disable.
  • RejectQtype: Reject inbound query with specific DNS record types, check List of DNS record types for details.

Domain file example (Find domains and suffix match)

example.com
xxx.xx

IP network file example

1.0.1.0/24
10.8.0.0/16
::1/128

Hosts file example (Support prefix wildcard only, *.xxx.xx includes xxx.xx)

127.0.0.1 localhost
::1 localhost
10.8.0.1 example.com
192.168.0.2 *.xxx.xx

DNS servers with ECS support

  • DNSPod 119.29.29.29:53
For DNSPod, ECS only works via udp, you can test it by patched dig
Check the ; CLIENT-SUBNET: 119.29.29.29/32/24, if it exists, it works.
The accuracy depends on the server side, do not judge ECS feature by it.
$ dig @119.29.29.29 www.qq.com +client=119.29.29.29

; <<>> DiG 9.9.3 <<>> @119.29.29.29 www.qq.com +client=119.29.29.29
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 64995
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
; CLIENT-SUBNET: 119.29.29.29/32/24
;; QUESTION SECTION:
;www.qq.com.   IN A

;; ANSWER SECTION:
www.qq.com.  300 IN A 101.226.103.106

;; Query time: 52 msec
;; SERVER: 119.29.29.29#53(119.29.29.29)
;; WHEN: Wed Mar 08 18:00:52 CST 2017
;; MSG SIZE  rcvd: 67

$ dig @119.29.29.29 www.qq.com +client=119.29.29.29 +tcp

; <<>> DiG 9.9.3 <<>> @119.29.29.29 www.qq.com +client=119.29.29.29 +tcp
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 58331
;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;www.qq.com.   IN A

;; ANSWER SECTION:
www.qq.com.  43 IN A 59.37.96.63
www.qq.com.  43 IN A 14.17.32.211
www.qq.com.  43 IN A 14.17.42.40

;; Query time: 81 msec
;; SERVER: 119.29.29.29#53(119.29.29.29)
;; WHEN: Wed Mar 08 18:01:32 CST 2017
;; MSG SIZE  rcvd: 87

Acknowledgements

  • Dependencies:
  • Code reference:
  • Contributors: @V-E-O, @sh1r0, @maddie, @hexchain, @everfly
from  https://github.com/shawn1m/overture
----

我的补充说明：

在mac上。
cd $GOPATH

go get -u -v github.com/shawn1m/overture/main
which main
显示；
~／go/bin/main
yudeMacBook-Air:gopath brite$ main -h
Usage of main:
  -V current version of overture
  -c string
    config file path (default "./config.json")
  -l string
    log file path
  -p int
    number of processor to use (default 4)
  -v verbose mode
yudeMacBook-Air:gopath brite$ cd ~

yudeMacBook-Air:~ brite$ 
wget https://github.com/shawn1m/overture/raw/master/config.sample.json -O overture-config.json
yudeMacBook-Air:~ brite$ nano overture-config.json
(在"PrimaryDNS"那一段里面,把Name的值改为google ,把Address的值改为8.8.8.8:53 ,把Protocol的值改为tcp ,把SOCKS5Address的值改为socks5://127.0.0.1:1080 ，
然后在"AlternativeDNS"那一段里面，把"SOCKS5Address"的值设为"socks5://127.0.0.1:1080",
这样就启用了socks5代理。）
yudeMacBook-Air:~ brite$ cat overture-config.json
{
  "BindAddress": ":53",
  "DebugHTTPAddress": "127.0.0.1:5555",
  "PrimaryDNS": [
    {
      "Name": "google",
      "Address": "8.8.8.8:53",
      "Protocol": "tcp",
      "SOCKS5Address": "socks5://127.0.0.1:1080",
      "Timeout": 6,
      "EDNSClientSubnet": {
        "Policy": "disable",
        "ExternalIP": "",
        "NoCookie": true
      }
    }
  ],
  "AlternativeDNS": [
    {
      "Name": "OpenDNS",
      "Address": "208.67.222.222:443",
      "Protocol": "tcp",
      "SOCKS5Address": "socks5://127.0.0.1:1080",
      "Timeout": 6,
      "EDNSClientSubnet": {
        "Policy": "disable",
        "ExternalIP": "",
        "NoCookie": true
      }
    }
  ],

  "OnlyPrimaryDNS": false,
...
yudeMacBook-Air:~ brite$ sudo main -c overture-config.json
不要关闭此terminal.当然，你需先在本地机器上运行一个socks5代理服务器程序，比如ss.

如果你运行所安装的这个main时，遇错，可去下载作者提供的可执行文件：
mkdir overture-osx && cd overture-osx
wget https://github.com/shawn1m/overture/releases/download/v1.6-rc1/overture-darwin-amd64.zip
unzip overture-darwin-amd64.zip

yudeMacBook-Air:overture-osx brite$ ls
config.json ip_network_alternative_sample
domain_alternative_sample ip_network_primary_sample
domain_primary_sample overture-darwin-amd64
domain_ttl_sample overture-darwin-amd64.zip
hosts_sample

yudeMacBook-Air:overture-osx brite$ sudo ./overture-darwin-amd64 -c ~/overture-config.json


应用例子：
sudo wg-quick up wg0

sudo networksetup -setdnsservers "Wi-Fi" "Empty" && sudo networksetup -setdnsservers "Wi-Fi" 127.0.0.1

cd ~/overture-osx && sudo ./overture-darwin-amd64 -c ~/overture-config.json
---------------------------------

使用Overture搭建更好的智能递归DNS server


何为智能解析？问题来了：何为智能？这还要从负载均衡和CDN说起，你有没有想过BAT首页为什么能支撑全国那么多服务请求？是因为他们的首页所在的服务器很屌吗？不不，当然不是，那是因为这些网站都是由许多服务器群组成的，不可能一台服务器去扛所有的请求，他们的服务器在全国多地各网络都有分布，我们按就近原则使用相应的服务器，BAT这样才为我们提供了快速的服务，这里，就近原则就是所谓的智能解析。

如何实现智能解析？问题又来了，谁来负责智能解析呢？相信聪明的你已经知道答案了，当然是权威DNS啊，也就是域名ns记录所指向的DNS服务器，权威DNS负责按照规则授权解析域名。假设有一个网站edns.urdomain.com，在电信网络下有一个IP为1.1.1.1的服务器，在联通网络里有一个IP为2.2.2.2的服务器，在移动网络里有一个IP为3.3.3.3的服务器，让不同网络的用户使用不同的服务器，网站所有者就可以在权威DNS里面配置，根据用户IP判断用户所在的运营商，然后返回不同的结果，也就实现了智能解析。

运营商有自己的DNS缓存服务器，当你使用默认DNS的时候，运营商会分配给你运营商自己的DNS缓存服务器，缓存服务器再去权威DNS那里获取解析结果，比如你是移动网络，运营商缓存DNS服务器自然也是移动的IP，获取到的结果就是合理的3.3.3.3，但是运营商会在DNS上做一些手脚，用户体验一般，于是就有了公共DNS。那么再做一个假设，你在电信机房有一台公共DNS服务器，使用用户有电信用户，也有联通和移动用户，对于edns.urdomain.com，你的公共DNS去权威DNS那里成功地获取到了edns.urdomain.com的解析结果1.1.1.1，你的用户从你那里也拿到了同样的结果1.1.1.1，电信用户表示非常舒爽，但是联通和移动访问起来就可能非常吃力，毕竟运营商与运营商之间的网络线路很少，跨运营商访问有时会让你感到绝望，就像网通用户在电信区打撸啊撸一样。

如何避免这种公共DNS造成的不适呢？
假设你需要搭建一台支持智能解析的公共DNS，那么有以下三种方案供你选择：
1、土豪专用方法：分别在电信、移动、联通、铁通、教育网、长城宽带等网络上按地域部署N多节点，然后再搞个主服务器，用户向主服务发起查询，主服务器再从离用户最近的同运营商节点去取结果并返回给用户，目前国内主流的公共DNS基本上都是采用这种方案；
2、方案一屌丝版：用Bind配置多个view，不同view转发给不同的运营商DNS，原理同第一个方法，省去了部署多节点；
3、大牛版：根据谷歌的EDNS草案，自己编写程序，在DNS查询的附加字段中添加用户IP后进行递归，如果权威DNS也支持该草案，那么权威DNS会按照EDNS中附加的用户IP返回结果，而不是根据公共DNS的IP返回结果，从而实现智能解析。
目前好多现成的DNS服务器软件都提供EDNS-CLIENT-SUBNET支持，比如DNS界的老大Bind，但是你越深入，感受就越深刻，那就是Bind适合用来做权威DNS，而不是公共递归DNS！Unbound、pdns-recursor相较于Bind更适合做支持智能解析的公共DNS，但是你会无奈的发现当用户量较多时他们的解析结果会乱飘，比如移动用户的结果有可能被解析成电信用户的结果，后果就是移动用户打开某些网站要几分钟甚至直接打不开！本来为了加快访问而搭建的CDN反而拖慢了访问速度。这锅公共DNS得背着，我就问你伤不伤？当然也有可能是鄙人使用的方法不对，错怪了这些软件。
说了这么多，是时候把大牛写的神器分享出来了。
https://github.com/shawn1m/overture
项目主页上已经把使用方法和配置说得很详细了，这里就不多做介绍了。DNSPod解析为国外IP的域名及部分特殊域名会使用TCP协议向OpenDNS的443端口请求结果以返回无污染结果。
在这里可以下载编译好的程序：
https://github.com/shawn1m/overture/releases
解压后修改配置文件config.json，把119.29.29.29中的Policy字段值改为auto；
Windows系统直接双击运行，Linux系统直接利用nohup命令在后台运行即可。
Posted by at
Labels: ,

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)