Total Pageviews

Saturday 29 June 2019

一个新型的dns proxy server程序-CoreDNS

CoreDNS is a DNS server that chains plugins 
CoreDNS is a DNS server/forwarder, written in Go, that chains plugins. Each plugin performs a DNS function.
CoreDNS is a Cloud Native Computing Foundation graduated project.
CoreDNS is a fast and flexible DNS server. The key word here is flexible: with CoreDNS you are able to do what you want with your DNS data by utilizing plugins. If some functionality is not provided out of the box you can add it by writing a plugin.
CoreDNS can listen for DNS requests coming in over UDP/TCP (go'old DNS), TLS (RFC 7858), also called DoT, DNS over HTTP/2 - DoH - (RFC 8484) and gRPC (not a standard).
Currently CoreDNS is able to:
  • Serve zone data from a file; both DNSSEC (NSEC only) and DNS are supported (file and auto).
  • Retrieve zone data from primaries, i.e., act as a secondary server (AXFR only) (secondary).
  • Sign zone data on-the-fly (dnssec).
  • Load balancing of responses (loadbalance).
  • Allow for zone transfers, i.e., act as a primary server (file).
  • Automatically load zone files from disk (auto).
  • Caching of DNS responses (cache).
  • Use etcd as a backend (replace SkyDNS) (etcd).
  • Use k8s (kubernetes) as a backend (kubernetes).
  • Serve as a proxy to forward queries to some other (recursive) nameserver (forward).
  • Provide metrics (by using Prometheus) (metrics).
  • Provide query (log) and error (errors) logging.
  • Support the CH class: version.bind and friends (chaos).
  • Support the RFC 5001 DNS name server identifier (NSID) option (nsid).
  • Profiling support (pprof).
  • Rewrite queries (qtype, qclass and qname) (rewrite and template).
And more. Each of the plugins is documented. See coredns.io/plugins for all in-tree plugins, and coredns.io/explugins for all out-of-tree plugins.

Compilation from Source

To compile CoreDNS, we assume you have a working Go setup. See various tutorials if you don’t have that already configured.
First, make sure your golang version is 1.12 or higher as go mod support is needed. See here for go mod details. Then, check out the project and run make to compile the binary:
git clone https://github.com/coredns/coredns
cd coredns
make

Compilation with Docker

CoreDNS requires Go to compile. However, if you already have docker installed and prefer not to setup a Go environment, you could build CoreDNS easily:
$ docker run --rm -i -t -v $PWD:/go/src/github.com/coredns/coredns \
      -w /go/src/github.com/coredns/coredns golang:1.12 make
The above command alone will have coredns binary generated.

Examples

When starting CoreDNS without any configuration, it loads the whoami plugin and starts listening on port 53 (override with -dns.port), it should show the following:
.:53
2016/09/18 09:20:50 [INFO] CoreDNS-001
CoreDNS-001
Any query sent to port 53 should return some information; your sending address, port and protocol used.
If you have a Corefile without a port number specified it will, by default, use port 53, but you can override the port with the -dns.port flag:
./coredns -dns.port 1053, runs the server on port 1053.
Start a simple proxy. You'll need to be root to start listening on port 53.
Corefile contains:
.:53 {
    forward . 8.8.8.8:53
    cache
    log
    errors
}
Just start CoreDNS: ./coredns. Then just query on that port (53). The query should be forwarded to 8.8.8.8 and the response will be returned. Each query should also show up in the log which is printed on standard output.
Serve the (NSEC) DNSSEC-signed example.org on port 1053, with errors and logging sent to standard output. Allow zone transfers to everybody, but specifically mention 1 IP address so that CoreDNS can send notifies to it.
example.org:1053 {
    file /var/lib/coredns/example.org.signed {
        transfer to *
        transfer to 2001:500:8f::53
    }
    errors
    log
}
Serve example.org on port 1053, but forward everything that does not match example.org to a recursive nameserver and rewrite ANY queries to HINFO.
.:1053 {
    rewrite ANY HINFO
    forward . 8.8.8.8:53

    file /var/lib/coredns/example.org.signed example.org {
        transfer to *
        transfer to 2001:500:8f::53
    }
    errors
    log
}
IP addresses are also allowed. They are automatically converted to reverse zones:
10.0.0.0/24 {
    whoami
}
Means you are authoritative for 0.0.10.in-addr.arpa..
This also works for IPv6 addresses. If for some reason you want to serve a zone named 10.0.0.0/24 add the closing dot: 10.0.0.0/24. as this also stops the conversion.
This even works for CIDR (See RFC 1518 and 1519) addressing, i.e. 10.0.0.0/25, CoreDNS will then check if the in-addr request falls in the correct range.
Listening on TLS and for gRPC? Use:
tls://example.org grpc://example.org {
    whoami
}
Specifying ports works in the same way:
grpc://example.org:1443 {
    # ...
}
When no transport protocol is specified the default dns:// is assumed.

Community

We're most active on Github (and Slack):
More resources can be found:

Deployment

Examples for deployment via systemd and other use cases can be found in the deployment repository.
from https://github.com/coredns/coredns
------
Scripts, utilities, and examples for deploying CoreDNS.

MacOS

The default settings will proxy all requests to hostnames not found in your host file to Google's DNS-over-HTTPS.
To install:
  • Run brew tap "coredns/deployment" "https://github.com/coredns/deployment"
  • Run brew install coredns
  • Run sudo brew services start coredns
  • test with dig google.com @127.0.0.1 and you should see SERVER: 127.0.0.1#53(127.0.0.1)
Using CoreDNS as your default resolver:
  • Open Network Preferences
  • Select your interface i.e Wi-Fi
  • Click Advanced
  • Select the DNS tab
  • Click the + below the DNS Servers list box
  • Type 127.0.0.1 and hit enter
  • Click OK
  • Click Apply

Debian

On a debian system:
  • Run dpkg-buildpackage -us -uc -b --target-arch ARCH Where ARCH can be any of the released architectures, like "amd64" or "arm".
  • Most users will just run: dpkg-buildpackage -us -uc -b
To install:
  • Run dpkg -i coredns_0.9.10-0~9.20_amd64.deb.
This installs the coredns binary in /usr/bin, adds a coredns user (homedir set to /var/lib/coredns) and a small Corefile /etc/coredns.

------

Since almost a year Google has a DNS service that can be queried over HTTPS: https://dns.google.com. This means your queries are encrypted and can only be seen by you (and Google(!)). Seeing all the press about the UK’s snooper’s charter 
Also note that this a different protocol than “DNS over TLS” which has similar goals and is being standardized by the IETF.
-------

我的补充说明:
在mac机器上。
mkdir coredns_1.6.2_darwin_amd64
cd coredns_1.6.2_darwin_amd64
wget https://github.com/coredns/coredns/releases/download/v1.6.2/coredns_1.6.2_darwin_amd64.tgz
tar zxvf coredns_1.6.2_darwin_amd64.tgz
(在当前目录下,会解压出来可执行文件coredns)
nano Corefile 
其内容为:

. {
    forward . vps-ip:dnsmasq-port-number-on-vps
    cache
    log
    errors
}

然后,
sudo ./coredns -conf Corefile -dns.port 53
不要关闭此terminal.


实际使用例子:
sudo wg-quick up wg0

sudo networksetup -setdnsservers "Wi-Fi" 127.0.0.1

cd ~/coredns_1.6.2_darwin_amd64 && sudo ./coredns -conf Corefile -dns.port 53
----------

使用CoreDNS来应对DNS污染, 在MacOS上部署轻量级高性能的CoreDNS

CoreDNS- https://github.com/coredns/coredns 是新晋的 CNCF 孵化项目-https://coredns.io/2017/03/02/why-cncf-for-coredns/,前几天已经从 CNCF 正式毕业,并正式成为 Kubernetes 的 DNS 服务器。CoreDNS 的目标是成为 cloud-native 环境下的 DNS 服务器和服务发现解决方案,即:
Our goal is to make CoreDNS the cloud-native DNS server and service discovery solution.

它有以下几个特性:

插件化(Plugins)
基于 Caddy 服务器框架,CoreDNS 实现了一个插件链的架构,将大量应用端的逻辑抽象成 plugin 的形式(如 Kubernetes 的 DNS 服务发现,Prometheus 监控等)暴露给使用者。CoreDNS 以预配置的方式将不同的 plugin 串成一条链,按序执行 plugin 的逻辑。从编译层面,用户选择所需的 plugin 编译到最终的可执行文件中,使得运行效率更高。CoreDNS 采用 Go 编写,所以从具体代码层面来看,每个 plugin 其实都是实现了其定义的 interface 的组件而已。第三方只要按照 CoreDNS Plugin API 去编写自定义插件,就可以很方便地集成于 CoreDNS。

配置简单化
引入表达力更强的 DSL,即 Corefile 形式的配置文件(也是基于 Caddy 框架开发)。

一体化的解决方案
区别于 kube-dns,CoreDNS 编译出来就是一个单独的二进制可执行文件,内置了 cache,backend storage,health check 等功能,无需第三方组件来辅助实现其他功能,从而使得部署更方便,内存管理更为安全。

其实从功能角度来看,CoreDNS 更像是一个通用 DNS 方案(类似于 BIND),然后通过插件模式来极大地扩展自身功能,从而可以适用于不同的场景(比如 Kubernetes)。正如官方博客所说:

CoreDNS is powered by plugins.

1. Corefile 介绍
Corefile 是 CoreDNS 的配置文件(源于 Caddy 框架的配置文件 Caddyfile),它定义了:

server 以什么协议监听在哪个端口(可以同时定义多个 server 监听不同端口)
server 负责哪个 zone 的权威(authoritative)DNS 解析
server 将加载哪些插件
常见地,一个典型的 Corefile 格式如下所示:

ZONE:[PORT] {
[PLUGIN] ...
}

ZONE : 定义 server 负责的 zone,PORT 是可选项,默认为 53;
PLUGIN : 定义 server 所要加载的 plugin。每个 plugin 可以有多个参数;
比如:

. {
    chaos CoreDNS-001
}
上述配置文件表达的是:server 负责根域 . 的解析,其中 plugin 是 chaos 且没有参数。

定义 server
一个最简单的配置文件可以为:
.{}

即 server 监听 53 端口并不使用插件。如果此时在定义其他 server,要保证监听端口不冲突;如果是在原来 server 增加 zone,则要保证 zone 之间不冲突,如:

.    {}
.:54 {}

另一个 server 运行于 54 端口并负责根域 . 的解析。

又如:

example.org {
    whoami
}
org {
    whoami
}

同一个 server 但是负责不同 zone 的解析,有不同插件链。

定义 Reverse Zone
跟其他 DNS 服务器类似,Corefile 也可以定义 Reverse Zone(反向解析 IP 地址对应的域名):

0.0.10.in-addr.arpa {
    whoami
}

或者简化版本:
10.0.0.0/24 {
    whoami
}

可以通过 dig 进行反向查询:
$ dig -x 10.0.0.1

使用不同的通信协议
CoreDNS 除了支持 DNS 协议,也支持 TLS 和 gRPC,即 DNS-over-TLS 和 DNS-over-gRPC 模式:

tls://example.org:1443 {
#...
}

2. 插件的工作模式
当 CoreDNS 启动后,它将根据配置文件启动不同 server ,每台 server 都拥有自己的插件链。当有 DNS 请求时,它将依次经历如下 3 步逻辑:

如果有当前请求的 server 有多个 zone,将采用贪心原则选择最匹配的 zone;
一旦找到匹配的 server,按照 plugin.cfg 定义的顺序执行插件链上的插件;
每个插件将判断当前请求是否应该处理,将有以下几种可能:
请求被当前插件处理
插件将生成对应的响应并回给客户端,此时请求结束,下一个插件将不会被调用,如 whoami 插件;

请求被当前插件以 Fallthrough 形式处理
如果请求在该插件处理过程中有可能将跳转至下一个插件,该过程称为 fallthrough,并以关键字 fallthrough 来决定是否允许此项操作,例如 host 插件,当查询域名未位于 /etc/hosts,则调用下一个插件;

请求在处理过程被携带 Hint
请求被插件处理,并在其响应中添加了某些信息(hint)后继续交由下一个插件处理。这些额外的信息将组成对客户端的最终响应,如 metric 插件;

3. CoreDNS 如何处理 DNS 请求
如果 Corefile 为:

coredns.io:5300 {
    file db.coredns.io
}

example.io:53 {
    log
    errors
    file db.example.io
}

example.net:53 {
    file db.example.net
}

.:53 {
    kubernetes
    proxy . 8.8.8.8
    log
    health
    errors
    cache
}

从配置文件来看,我们定义了两个 server(尽管有 4 个区块),分别监听在 5300 和 53 端口。

每个进入到某个 server 的请求将按照 plugin.cfg 定义顺序执行其已经加载的插件。

我们需要注意以下几点:

尽管在 .:53 配置了 health 插件,但是它并为在上面的逻辑图中出现,原因是:该插件并未参与请求相关的逻辑(即并没有在插件链上),只是修改了 server 配置。更一般地,我们可以将插件分为两种:
Normal 插件:参与请求相关的逻辑,且插入到插件链中;
其他插件:不参与请求相关的逻辑,也不出现在插件链中,只是用于修改 server 的配置,如 health,tls 等插件;

4. 在 MacOS 上部署 CoreDNS
既然 CoreDNS 如此优秀,我用它来抵御伟大的防火长城岂不美哉?研究了一圈,发现技术上还是可行的,唯一的一个缺点是不支持使用代理,不过你可以通过 proxychians-ng 或 proxifier 来强制使用代理。下面开始折腾。

安装
CoreDNS 是 golang 写的,所以只需要下载对应操作系统的二进制文件,到处拷贝,就可以运行了。

下面统统以 MacOS 为例作讲解。

$ cd ~/Downloads
$ wget https://github.com/coredns/coredns/releases/download/v1.4.0/coredns_1.4.0_darwin_amd64.tgz
$ tar zxf coredns_1.4.0_darwin_amd64.tgz
$ mv ./coredns /usr/local/bin/

这里补充一句,CoreDNS 的二进制版本已经安装了所有的插件(plugins),不需要你自己编译。推荐下载二进制版本。

配置
要深入了解 CoreDNS,请查看其文档,及 plugins 的介绍。下面是我的配置文件:

$ cat < /usr/local/etc/Corefile
. {
  hosts {
    fallthrough
  }

  forward . tls://8.8.8.8 tls://8.8.4.4 {
    tls_servername dns.google
    force_tcp
    max_fails 3
    expire 10s
    health_check 5s
    policy sequential
    except www.baidu.com
  }

  proxy . 117.50.11.11 117.50.22.22 {
    policy round_robin
  }

  cache 120
  reload 6s
  log . "{local}:{port} - {>id} '{type} {class} {name} {proto} {size} {>do} {>bufsize}' {rcode} {>rflags} {rsize} {duration}"
  errors
}
EOF

hosts : hosts 是 CoreDNS 的一个 plugin,这一节的意思是加载 /etc/hosts 文件里面的解析信息。hosts 在最前面,则如果一个域名在 hosts 文件中存在,则优先使用这个信息返回;
fallthrough : 如果 hosts 中找不到,则进入下一个 plugin 继续。缺少这一个指令,后面的 plugins 配置就无意义了;
forward :这是另外一个 plugin。. 代表所有域名,后面的 IP 代表上游 DNS 服务器的列表。按照什么顺序溯源,由下面的 policy 指令决定;
tls://8.8.8.8 : 这里表示使用 DNS-over-TLS 协议访问 8.8.8.8。同时还需要通过 tls_servername 指定 DNS 名称。
force_tcp : 强制使用 TCP 协议溯源。这要求上游 DNS 必须支持 TCP 协议;
expect :指定哪些域名不按照本 plugin 配置溯源。这里主要用来排除国内的域名,然后通过下面的 proxy 转到国内的 DNS 进行解析。这里被排除的域名只填了一个 www.baidu.com,后面我们再通过脚本填上所有的国内域名;
proxy : 解析 forward 中被排除的域名。. 代表所有域名,后面的 IP 代表上游 DNS 服务器的列表,这里我选择的是 onedns。按照什么顺序溯源,由下面的 policy 指令决定;
cache : 溯源得到的结果,缓存指定时间。类似 TTL 的概念;
reload : 多久扫描配置文件一次。如有变更,自动加载;
log : 打印/存储访问日志。日志格式参考:https://coredns.io/plugins/log/;
errors : 打印/存储错误日志;

配置文件类似于 nginx 配置文件的格式;
最外面一级的大括号,对应『服务』的概念。多个服务可以共用一个端口;
往里面一级的大括号,对应 plugins 的概念,每一个大括号都是一个 plugin。这里可以看出,plugins 是 CoreDNS 的一等公民;
服务之间顺序有无关联没有感觉,但 plugins 之间是严重顺序相关的。某些 plugin 必须用 fallthrough 关键字流向下一个 plugin;
plugin 内部的配置选项是顺序无关的;
从 plugins 页面的介绍看,CoreDNS 的功能还是很强的,既能轻松从 bind 迁移,还能兼容 old-style dns server 的运维习惯;
从 CoreDNS 的性能指标看,适合做大型服务。
当然了,上面的配置文件还可以升级一下,具体我就不解释了:

. {
  hosts {
    fallthrough
  }

  forward . 127.0.0.1:5301 127.0.0.1:5302 {
    max_fails 3
    expire 10s
    health_check 5s
    policy sequential
    except www.baidu.com
  }

  proxy . 117.50.11.11 117.50.22.22 {
    policy round_robin
  }

  cache 120
  reload 6s
  log . "{local}:{port} - {>id} '{type} {class} {name} {proto} {size} {>do} {>bufsize}' {rcode} {>rflags} {rsize} {duration}"
  errors
}

.:5301 {
  forward . tls://8.8.8.8 tls://8.8.4.4 {
    tls_servername dns.google
    force_tcp
    max_fails 3
    expire 10s
    health_check 5s
    policy sequential
  }
}

.:5302 {
  forward . tls://1.1.1.1 tls://1.0.0.1 {
    tls_servername 1dot1dot1dot1.cloudflare-dns.com
    force_tcp
    max_fails 3
    expire 10s
    health_check 5s
    policy sequential
  }
}

定时更新国内域名列表
编写一个 shell 脚本,用来更新 Corefile 中排除的国内域名列表:

$ brew install gnu-sed
$ cat < /usr/local/bin/update_coredns.sh
#!/bin/bash

chinadns=$(curl -sL https://raw.githubusercontent.com/felixonmars/dnsmasq-china-list/master/accelerated-domains.china.conf|awk -F "/" '{print $2}')
touch update_coredns.sed && echo "" > update_coredns.sed
for i in $chinadns; do echo "/except/ s/$/ $i/" >> update_coredns.sed; done
gsed -i "s/\(except\).*/\1/" /usr/local/etc/Corefile
gsed -i -f update_coredns.sed /usr/local/etc/Corefile
EOF
$ sudo chmod +x /usr/local/bin/update_coredns.sh

先执行一遍该脚本,更新 Corefile 的配置:

$ /usr/local/bin/update_coredns.sh

然后通过 Crontab 制作定时任务,每隔两天下午两点更新域名列表:

$ crontab -l
0 14 */2 * * /usr/local/bin/update_coredns.sh

开机自启
MacOS 可以使用 launchctl 来管理服务,它可以控制启动计算机时需要开启的服务,也可以设置定时执行特定任务的脚本,就像 Linux crontab 一样, 通过加装 *.plist 文件执行相应命令。Launchd 脚本存储在以下位置, 默认需要自己创建个人的 LaunchAgents 目录:

~/Library/LaunchAgents : 由用户自己定义的任务项
/Library/LaunchAgents : 由管理员为用户定义的任务项
/Library/LaunchDaemons : 由管理员定义的守护进程任务项
/System/Library/LaunchAgents : 由 MacOS 为用户定义的任务项
/System/Library/LaunchDaemons : 由 MacOS 定义的守护进程任务项
我们选择在 /Library/LaunchAgents/ 目录下创建 coredns.plist 文件,内容如下:
 
    Label
    coredns
    ProgramArguments
   
      /usr/local/bin/coredns
      -conf
      /usr/local/etc/Corefile
   
    StandardOutPath
    /var/log/coredns.stdout.log
    StandardErrorPath
    /var/log/coredns.stderr.log
    KeepAlive
   
    RunAtLoad
 
设置开机自动启动 coredns:
$ sudo launchctl load -w /Library/LaunchAgents/coredns.plist

查看服务:
$ sudo launchctl list|grep coredns

61676 0 coredns

$ sudo launchctl list coredns

{
"StandardOutPath" = "/var/log/coredns.stdout.log";
"LimitLoadToSessionType" = "System";
"StandardErrorPath" = "/var/log/coredns.stderr.log";
"Label" = "coredns";
"TimeOut" = 30;
"OnDemand" = false;
"LastExitStatus" = 0;
"PID" = 61676;
"Program" = "/usr/local/bin/coredns";
"ProgramArguments" = (
"/usr/local/bin/coredns";
"-conf";
"/usr/local/etc/Corefile";
);
};

查看端口号:
$ sudo ps -ef|egrep -v grep|grep coredns

    0 81819     1   0  2:54下午 ??         0:04.70 /usr/local/bin/coredns -conf /usr/local/etc/Corefile
 
$ sudo lsof -P -p 81819|egrep "TCP|UDP"

coredns 81819 root    5u    IPv6 0x1509853aadbdf853      0t0     TCP *:5302 (LISTEN)
coredns 81819 root    6u    IPv6 0x1509853acd2f39ab      0t0     UDP *:5302
coredns 81819 root    7u    IPv6 0x1509853aadbdc493      0t0     TCP *:53 (LISTEN)
coredns 81819 root    8u    IPv6 0x1509853acd2f5a4b      0t0     UDP *:53
coredns 81819 root    9u    IPv6 0x1509853ac63bfed3      0t0     TCP *:5301 (LISTEN)
coredns 81819 root   10u    IPv6 0x1509853acd2f5d03      0t0     UDP *:5301

大功告成,现在你只需要将系统的 DNS IP 设置为 127.0.0.1 就可以了。

验证
$ dig www.google.com

; <<>> DiG 9.10.6 <<>> www.google.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- 49942="" id:="" noerror="" opcode:="" p="" query="" status:="">;; flags: qr rd ra; QUERY: 1, ANSWER: 6, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;www.google.com. IN A

;; ANSWER SECTION:
www.google.com. 64 IN A 108.177.97.147
www.google.com. 64 IN A 108.177.97.105
www.google.com. 64 IN A 108.177.97.106
www.google.com. 64 IN A 108.177.97.103
www.google.com. 64 IN A 108.177.97.104
www.google.com. 64 IN A 108.177.97.99

;; Query time: 0 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Wed Mar 06 13:23:31 CST 2019
;; MSG SIZE  rcvd: 223
搞定。

5. 参考资料
CoreDNS 使用与架构分析-https://zhengyinyong.com/coredns-basis.html
<- 49942="" id:="" noerror="" opcode:="" p="" query="" status:="">------------------------------------------------------------------
<- 49942="" id:="" noerror="" opcode:="" p="" query="" status:="">
CoreDNS搭建无污染的DNS server

自从半年多前开始用CoreDNS,我觉得可以不用dnsmasq/overture/chinadns这些轮子了,CoreDNS完全符合我的需求:
  • 无污染
  • 国内CDN友好
  • 跨平台,支持多种CPU、操作系统
CoreDNS的来头不小,它的作者是最好的开源DNS package的作者,CoreDNS底层也使用了这个package。大名鼎鼎的k8s使用CoreDNS进行服务发现。CoreDNS基本沿用了Caddy的插件架构,所以CoreDNS的配置文件的语法跟Caddy的配置文件语法相同。
一个最简单的配置文件可以是这样:
1
2
3
4
5
.:53{
    forward . 8.8.8.8
    log
    health
}
将配置保存为文件Corefile,运行命令sudo coredns -conf Corefile,即可在本地同时监听TCP和UDP 53端口,将所有UDP查询请求转发到8.8.8.8再返回,可以通过dig @::1 -p 53 twitter.com进行测试。
但是这个配置文件在国内几乎是没啥用的,原因自然是8.8.8.8乃老大哥重点关注对象,直接访问得到的结果都是二手信息。一个好一点的方案是使用非标准端口,比如:
1
2
3
4
5
.:53{
    forward . 208.67.222.222:443
    log
    health
}
forward插件支持多个上游服务器以实现简单的负载均衡:
1
2
3
4
5
.:53{
    forward . 208.67.222.222:443 208.67.222.222:5353 208.67.220.220:443 208.67.220.220:5353
    log
    health
}
大陆的网络环境非常复杂,UDP非标准端口也只在某些地区某些运营商有用,现在比较好的一个选择是DoT,即DNS over TLS,知名的支持DoT的公共DNS服务有Quad9的9.9.9.9,Google的8.8.8.8以及Cloudflare的1.1.1.1,可以这么使用:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
.:53{
    forward . 127.0.0.1:5301 127.0.0.1:5302 127.0.0.1:5303
    log
    health
}
.:5301 {
   forward . tls://9.9.9.9 {
       tls_servername dns.quad9.net
   }
   cache
}
.:5302 {
    forward . tls://1.1.1.1 tls://1.0.0.1 {
        tls_servername 1dot1dot1dot1.cloudflare-dns.com
    }
    cache
}
.:5303 {
    forward . tls://8.8.8.8 tls://8.8.4.4 {
        tls_servername dns.google
    }
    cache
}
这样除了老大哥把连接reset,基本可以得到正确的DNS解析结果。
另一个问题是国内CDN友好,我一直以来的做法是使用FelixOnMars的大陆区域名列表过滤。这个列表是给dnsmasq用的,经过转换可以给CoreDNS用,这利用了CoreDNS的两个插件来实现,分别是forwardproxy,这两个插件的功能非常相似,都是将DNS解析请求发给上游DNS server,再将结果取回返回给客户端。为了实现分流解析,可以将所有请求都通过forward转发到无污染上游解析,将大陆区域名列表加到异常列表,再把剩下的所有请求(其实就是异常列表中的域名)通过proxy转发到国内(最好是当前ISP的)DNS server,比如:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
.:53{
    forward . 127.0.0.1:5301 127.0.0.1:5302 127.0.0.1:5303 {
        except www.taobao.com
    }
    proxy . 116.228.111.118 180.168.255.18
    log
    health
}
.:5301 {
   forward . tls://9.9.9.9 {
       tls_servername dns.quad9.net
   }
   cache
}
.:5302 {
    forward . tls://1.1.1.1 tls://1.0.0.1 {
        tls_servername 1dot1dot1dot1.cloudflare-dns.com
    }
    cache
}
.:5303 {
    forward . tls://8.8.8.8 tls://8.8.4.4 {
        tls_servername dns.google
    }
    cache
}
这里except www.taobao.com表示www.taobao.com这个域名不要通过forward解析,后面可以跟多个域名,于是这些域名会掉到下面的proxy插件进行解析,而116.228.111.118180.168.255.18则是我的ISP提供的DNS服务器,可以得到最好的CDN友好的结果。
这时就可以用上FelixOnMars的大陆区域名列表了,用以下命令可以得到所有域名连接而成的长字符串,放在except标识符后面:
1
2
china=`curl https://cdn.jsdelivr.net/gh/felixonmars/dnsmasq-china-list/accelerated-domains.china.conf -s | while read line; do awk -F '/' '{print $2}' | grep -v '#' ; done |  paste -sd " " -`
echo "  except $china " >> Corefile
FelixOnMars同时还提供了Google和Apple的域名列表,这在某些地区某些ISP可以得到国内镜像的IP,所以最后可以写一个这样的shell脚本用于生成Corefile:
#!/bin/bash
echo ". {" > Corefile
echo " ads {" >> Corefile
echo " strict-default-lists" >> Corefile
echo " log" >> Corefile
echo " auto-update-interval 24h" >> Corefile
echo " list-store ads-cache" >> Corefile
echo " }" >> Corefile
echo " hosts {" >> Corefile
echo " fallthrough" >> Corefile
echo " }" >> Corefile
echo " forward . 208.67.222.222:443 208.67.222.222:5353 208.67.220.220:443 208.67.220.220:5353 127.0.0.1:5301 127.0.0.1:5302 127.0.0.1:5303 {" >> Corefile
china=`curl -L https://github.com/felixonmars/dnsmasq-china-list/raw/master/accelerated-domains.china.conf -s | while read line; do awk -F '/' '{print $2}' | grep -v '#' ; done | paste -sd " " -`
apple=`curl -L https://github.com/felixonmars/dnsmasq-china-list/raw/master/apple.china.conf -s | while read line; do awk -F '/' '{print $2}' | grep -v '#' ; done | paste -sd " " -`
google=`curl -L https://github.com/felixonmars/dnsmasq-china-list/raw/master/google.china.conf -s | while read line; do awk -F '/' '{print $2}' | grep -v '#' ; done | paste -sd " " -`
echo " except $china $apple $google cdn.jsdelivr.net" >> Corefile
echo " }" >> Corefile
echo " proxy . 192.168.1.1" >> Corefile
bogus=`curl -L https://github.com/felixonmars/dnsmasq-china-list/raw/master/bogus-nxdomain.china.conf -s | grep "=" | while read line; do awk -F '=' '{print $2}' | grep -v '#' ; done | paste -sd " " -`
echo " bogus $bogus" >> $1
echo " log" >> Corefile
echo " cache" >> Corefile
echo " redisc {" >> Corefile
echo " endpoint 127.0.0.1:6379" >> Corefile
echo " }" >> Corefile
echo " health" >> Corefile
echo " reload" >> Corefile
echo "}" >> Corefile
echo ".:5301 {" >> Corefile
echo " forward . tls://9.9.9.9 tls://9.9.9.10 {" >> Corefile
echo " tls_servername dns.quad9.net" >> Corefile
echo " }" >> Corefile
echo " cache" >> Corefile
echo "}" >> Corefile
echo ".:5302 {" >> Corefile
echo " forward . tls://1.1.1.1 tls://1.0.0.1 {" >> Corefile
echo " tls_servername cloudflare-dns.com" >> Corefile
echo " }" >> Corefile
echo " cache" >> Corefile
echo "}" >> Corefile
echo ".:5303 {" >> Corefile
echo " forward . tls://8.8.8.8 tls://8.8.4.4 {" >> Corefile
echo " tls_servername dns.google" >> Corefile
echo " }" >> Corefile
echo " cache" >> Corefile
echo "}" >> Corefile
view rawgen_Corefile.sh hosted with ❤ by GitHub
我把这个脚本放在gist上,并做了个短网址,于是可以这样生成Corefile:
1
curl -s -L git.io/corefile | bash
到此为止,就已经得到国内CDN友好的无污染DNS解析服务了。
我还想得到更多,比如去广告!github上有非常多的列表,包括广告和有害软件等等,CoreDNS官方尚未提供一个block插件,好在已经有一些非官方的实现,比如block,可以用如下的方式使用:
1
2
3
4
5
6
7
8
9
10
11
.:53{
    block https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts
    block https://mirror1.malwaredomains.com/files/justdomains
    block https://s3.amazonaws.com/lists.disconnect.me/simple_ad.txt
    forward . 127.0.0.1:5301 127.0.0.1:5302 127.0.0.1:5303 {
        except www.taobao.com
    }
    proxy . 116.228.111.118 180.168.255.18
    log
    health
}
非常浅显易懂。如果遇到的请求域名是在列表中,则会返回NXDOMAIN
最后一个问题,由于proxy插件和block插件都不是官方内置插件,从CoreDNS官方下载页下载的可执行程序并不包括这两个插件,所以需要自己编译CoreDNS。
编译CoreDNS并不复杂:
1
2
3
git clone https://github.com/coredns/coredns.git
cd coredns
make
CoreDNS使用了go modules机制,所以在make过程中会自动下载依赖的package。其中一些package是放在诸如golang.org/x/的路径下的,所以需要自备梯子,可以全局翻,也可以通过HTTP_PROXY环境变量指定,或者使用国内的一些镜像(如果你信得过的话)通过GOPROXY环境变量指定。
如果要加入以上两个插件,则在make前,要修改plugin.cfg文件,加入以下两行:
1
2
block:github.com/missdeer/block
proxy:github.com/coredns/proxy
make,就会把这两个插件编译进去。如果发现没有编译进去,可以先执行一下go generate coredns.gomake
如果要给其他平台交叉编译CoreDNS,需要先以当前平台为参数make一次,再以目标平台为参数进行make,因为第一次make时会调用go generate跑两个程序,如果不是当前平台的可执行文件是跑不起来的。
最后,我把这个编译过程放到github上了,用appveyor服务编译出各个目标平台的CoreDNS
----------------------------------

A first look at CoreDNS

If CoreDNS had existed when I wrote Alternative DNS Servers I’d have included it; it’s quite a versatile beast.

CoreDNS was created by Miek Gieben, and he tells me there was a time during which CoreDNS was actually a forked Web server doing DNS, but that changed a bit. Whilst CoreDNS has its roots in and resembles Caddy, it’s a different beast. It’s not difficult to get to know, but some of the terminology CoreDNS uses confused me: for example the term middleware: I see that as a plugin, all the more so because this program’s option to list said middleware is called … drum roll … -plugins. Another thing I needed assistance for was some of the syntax, or rather the semantics, within the configuration file.

CoreDNS is another one of those marvelous single-binary, no-dependencies Go programs which I download and run. All that’s missing is a configuration file called a Corefile. (I associate a core file with the word Corefile … #justkidding ;-)

Launching coredns as root (so that the process may bind to port 53 – use -dns.port 5353 on the command line to specify an alternative, or cap_net_bind_service with systemd) will bring up a DNS server which uses the whoami middleware to provide no response, but an additional section to queries for any domain:

$ dig @192.168.1.207 www.example.org

;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 8021
;; flags: qr aa rd; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 3

;; QUESTION SECTION:
;www.example.org.		IN	A

;; ADDITIONAL SECTION:
www.example.org.	0	IN	A	192.168.1.130
_udp.www.example.org.	0	IN	SRV	0 0 60934 .

Quite useless, if you ask me, but at least I know the server’s running and it’s doing something.

The hosts middleware serves a zone from an /etc/hosts-type file, checking the file for changes and reloading the zone accordingly. A, AAAA, and PTR records are supported.

$ cat /etc/hosts
1.2.3.4         laptop.example.hosts
1.2.3.10         bigbox

$ cat Corefile
example.hosts {
        hosts /etc/hosts
}

With this configuration CoreDNS will respond authoritatively to a query for laptop.example.hosts only; the entry for bigbox is not found.

Let’s do something authoritative, and create a zone master file (in zone.db) and a Corefile:

$ cat Corefile
example.aa {
    file zone.db {
        transfer to *
        transfer to 192.168.1.130:53
    }
}

The file middleware loads the specified master zone file and serves it. That’s it. Simple. Not only that, but it also periodically checks whether the file has changed and actually reloads the zone when the SOA serial number changes. In the transfer stanza I specify that any client (*) may transfer the zone and that the host 192.168.1.130 gets a DNS NOTIFY when the zone is reloaded. (The port number on the address defaults to 53, I just show that it can be specified.) I tested NOTIFY with nsnotifyd and it works reliably.

Similar to file, the auto middleware can serve a whole directory of zone files, determining their origins using a regular expression.

The following Corefile uses the slave^H secondary middleware to create a slave zone which is transferred into RAM from the specified address. (Adding appropriate transfer to stanzas would make this secondary zone transferable by other secondaries.)

$ cat Corefile
ww.mens.de {
    secondary {
        transfer from 192.168.1.10
    }
    errors stdout
    log stdout
}

Note that the zone is served from RAM which means, in the event coredns launches and cannot contact any of its zone masters, the zone cannot be served.

If I need a forwarder, I configure it, here for the root zone, i.e. for all zones not explicitly defined within the Corefile:

$ cat Corefile
. {
    proxy . 192.168.1.10:53
}

Other middleware includes bind which overrides the address to which CoreDNS should bind to, and cache which can cap TTL values when operating as a forwarder. Middleware probably worth looking at is etcd which can read zone data from an etcd instance and kubernetes. If you’re into that sort of stuff of course.

Then there’s the dnssec middleware which promises to enable on-the-fly, a.k.a. “online”, DNSSEC signing of data with NSEC as authenticated denial of existence. In order to test this, I first create a key and then configure an authoritative zone in the Corefile which uses that key file:

$ ldns-keygen -a ECDSAP256SHA256 -k sec.aa
Ksec.aa.+013+28796

$ cat Corefile
sec.aa {
    file sec.aa
    dnssec {
        key file Ksec.aa.+013+28796
    }
}
;; ANSWER SECTION:
sec.aa.		3600 IN	DNSKEY 257 3 13 (
			meU/r4MKJ73gDanOfsiysUWn1PKDCGz6NxulydpAeFx3
			zNrepJTSVc65vJXt9koLI+PI+1uu9TadUlhEosyPjA==
			) ; KSK; alg = ECDSAP256SHA256 ; key id = 28796
sec.aa.		3600 IN	RRSIG DNSKEY 13 2 3600 (
			20170917103509 20170909073509 28796 sec.aa.
			k296ZBgiScV72AYXDuDFxBNaoZEXiBVhE57yAfgYVKYi
			nY9cmdO8tB81KX+OGA7d7V4cb6wrk876B5qRUWUZ2A== )

CoreDNS signs all records online; if I specify more than one key during configuration it signs each record with all keys.

CoreDNS binaries are provided with middleware for logging and monitoring. For example dnstap enables it to use dnstap.info’s structured binary log format, and I decide for which of the authoritative zones or proxy entries I want to log queries and responses by configuring dnstap accordingly. On the other hand, the health middleware enables an HTTP endpoint at a port you specify, and it returns a simple string if the server is alive:

$ cat Corefile
example.aa {
    health :8080
}

$ curl http://192.168.1.207:8080/health
OK

The tls middleware allows me to create a DNS-over-TLS (RFC 7858) respectively a DNS-over-gRPC (does anybody really need that?) server.

The server can act as a round-robin DNS loadbalancer, and it can provide responses to TXT queries in the CH (chaos) class:

$ cat Corefile
# define the CH "tlds"
bind {
    chaos CoreDNS-010 "Miek Gieben" miek@miek.nl
}
server {
    chaos CoreDNS-010 "Miek Gieben" miek@miek.nl
}
$ dig ...
...
;; ANSWERS:
version.bind.		0	CH	TXT	"CoreDNS-010"
version.server.		0	CH	TXT	"CoreDNS-010"
authors.bind.		0	CH	TXT	"Miek Gieben"
authors.bind.		0	CH	TXT	"miek@miek.nl"
hostname.bind.		0	CH	TXT	"t420.ww.mens.de"
id.server.		0	CH	TXT	"t420.ww.mens.de"

There are more middleware “plugins” (rewrite is fun!) and there’s also some documentation as to how to write your own.

Apparently it’s not possible to configure middleware globally. So, for example if you have two servers configured in a single Corefile (by specifying different ports, for example), both blocks need the middleware you want to share configured (documented here). This in turn means, that certain things cannot be done, e.g. dnstap into the same Unix socket.

Apropos documentation: that is, very unfortunately, a bit lacking in clarity. While the information is there, it’s presented in a form which made me pull lots of hair, and I frequently found myself grepping (is that a verb?) my way through the project’s Github issues in search of how, respectively where to write a directive in the Corefile, and Miek prodded me along, for which I thank him!

Other than that, CoreDNS is huge fun and has a lot of potential.

from https://jpmens.net/2017/09/09/coredns/

-----------

Secure, Redundant DNS Using CoreDNS

Although an increasing portion of the web is adopting HTTPS, a core part of the web infrastrcture lags behind in the form of plaintext DNS. The authentication problem of DNS is resolved by DNSSEC, but the queries are still plaintext. There’s currently a number of competing solutions that offer both authenticity and privacy concerns, namely DNS-over-TLS, DNS-over-HTTPS, and DNSCrypt. I’ll leave it to Cloudflare’s blog to explain DNS, DoT and DoH, and the work towards both: DNS Encryption Explained. Due to operating systems not yet supporting encrypted DNS resolvers, some work needs to be done by the user, and this post describes part of the work using CoreDNS.

CoreDNS is pluggable DNS and service discovery server written in Go and the blessed DNS resolver of Kubernetes. Configuring CoreDNS is handled via setting up a series of middlewares in the Corefile, which is what its config file is called. The CoreDNS Manual page does a brilliant job describing what CoreDNS is, how it works, how to install it, the structure of the configuration file, and more. To build our secure resolver, let’s pick up one of the examples off that page and gradually build the enhanced version.

Start by creating a file named Corefile containing the following, which is lifted verbatim from CoreDNS website:

1
2
3
4
. {
    forward . 8.8.8.8 9.9.9.9
    log
}

This is very simple, but let’s break it apart. The . means listen on port 53, the default port for UDP DNS, and across all interfaces. The configuration instructs CoreDNS to forward all requests to either 8.8.8.8 (Google DNS) or 9.9.9.9 (Quad9), picking either of them at random, and logs everything to stdout. This is not secure yet. This setup will forward all DNS queries in plaintext UDP packets. Let’s start by forwarding everything to Google DNS over TLS resolvers without fallback:

1
2
3
4
5
6
7
8
. {
    forward . tls://8.8.8.8 tls://8.8.4.4 {
		# This tells CoreDNS the subject name
		# on the certificate is: dns.google
		tls_servername dns.google
	}
    log
}

At this point the setup has redundancy against Google’s services only. What if Google DNS service is down? The possibiity of a service going down is not far fetched, and we must account for it. However, note how the tls_servername directive can only be defined once, so it is bound to whatever upstream IP address we’re using. We can get around this by breaking the upstream into more local resolvers. I’ll add Cloudflare DNS as backup:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
.:53 {
	forward . 127.0.0.1:5301 127.0.0.1:5302 [::1]:5301 [::1]:5302
	log
}
.:5301 {
	forward . tls://8.8.8.8 tls://8.8.4.4 {
		tls_servername dns.google
	}
}
.:5302 {
	forward . tls://1.1.1.1 tls://1.0.0.1 {
		tls_servername cloudflare-dns.com
	}
}

Now we have established the pattern. To add more backups, add more server blocks where the upstream is defined and the respective tls_servername is configured, and add the localhost IP address to the main server of .:53. This can be further enhanced by adding caching, prefetching, and loading the hosts file. The final setup will look like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
.:53 {
	hosts /path/to/hosts {
		fallthrough
	}
	cache {
		prefetch 2 30m
	}
	forward . 127.0.0.1:5301 127.0.0.1:5302 [::1]:5301 [::1]:5302 127.0.0.1:5303 [::1]:5303 {
		policy random
	}
}
.:5301 {
	forward . tls://8.8.8.8 tls://8.8.4.4 {
		tls_servername dns.google
	}
}
.:5302 {
	forward . tls://1.1.1.1 tls://1.0.0.1 {
		tls_servername cloudflare-dns.com
	}
}
.:5303 {
	cache
	forward . tls://9.9.9.9 {
		tls_servername dns.quad9.net
	}
}

Note that policy directive added in the first server block. Using random is superfluous because it is the default policy. The other policies available are sequential and round_robin. Regardless of the configured policy, CoreDNS will still perform health-checks and use a healthy upstream, so your queries are still answered even if one Google and/or Cloudflare are down. Configure your operating system to use your loopback address as the DNS server, set up CoreDNS service to start at boot, and your DNS queries are secure going forward.

from https://www.caffeinatedwonders.com/2020/11/27/secure-dns-proxy/