Total Pageviews

Sunday 30 June 2019

中美重回谈判桌,知识产权仍是难啃的硬核桃


在大阪峰会召开前夕的6月26日,美国财政部长姆努钦说,美国和中国的贸易协议“大约90%”已经完成。不过,这是姆努钦今年第四次发布类似宣示,此前,他已经于4月3日、5月7日、6月9日三次宣布“完成了90%”。
这听起来似乎接近完成的“成绩单”,就因为那10%的条款,让中国在今年5月拂袖而去,离开谈判桌,导致美国对价值两千亿美元的中国商品追加关税,两国贸易战直线升温,对全球经济构成了威胁,让世界各国尤其是G20大部分成员国惶惶不安。
这10%有多重要?想想那个被广泛引用的“二八原则”就会明白:一个条款、一个组织中,起关键作用的不是那80%的内容或人,而是那20%起决定作用。中美贸易协议谈判的难点,不在那美国认为完成了的90%的协议,而是那10%。自从2018年中美贸易战开始以来,中美双方努力达成的协议草案包含六个主题:强制技术移转与网络犯罪、知识产权、服务、货币、农业与非关税贸易壁垒。除知识产权之外,其余的五个主题就是姆努钦多次声称“已经完成的90%”,剩下的那10%,就是美国要求美中双方在中国开设“执法办公室”监督协议执行成果,包含一旦中方毁约,美国需要有惩罚机制,比如加征新的关税。整个谈判过程中,美国谈判代表一直在寻求这份权利,但中国谈判代表在这一问题上不愿妥协。
对于中国领导人来说,增加关税导致经济压力加大,企业破产、工人失业,这些都可以通过宣传机器归咎于美国,让国人对美国玩'国家怪罪游戏'。但同意开设执法办公室,在中国就上升到领导人的政治责任层面,朝野都会认为是丧权辱国。
亚洲曾经成为西方殖民地,日本、中国、韩国等国家都有过这段历史。二战以后的民族解放运动中,去殖民化被认为是亚非拉国家独立的象征。1997年,亚洲金融风暴使韩国经济陷入严重危机,当时韩国的外汇储备只剩下可怜的39亿美元。为渡难关,政府不得不在当年11月向IMF(国际货币基金组织)申请了550亿美元的紧急援助性贷款,代价是韩国的经济政策必须接受IMF的干预和严厉监督。从此,韩国进入了“IMF时代”,韩国人将此视为奇耻大辱,在此后的三年内,除了努力改革,民间还发动捐金运动,力争早日偿还债务。2001年8月23日上午,韩国政府宣布偿还IMF最后一笔1.4亿美元的贷款,告别“IMF时代”,金大中的政治声望因此如日中天。
南韩的国际地位当时不过是“四小龙”之一,对IMF这种监管就如此深恶痛绝,中国就更不用说了。中国人普遍认为本国是世界老二,要与美国一争雄长,中国(中共)政府还要主导国际规则的制订,这种心态之下,接受美国的监管,在国人眼中,简直等同于袁世凯当年的二十一条。熟悉中国政治与中国人的心态,就知道习近平必定在这条上坚持到底。
美国为什么必须坚持?
一言以概之,就是保护知识产权对美国太重要,关系到美国企业的竞争力与美国在国际社会领先的科技地位,是美国的国力所系
2016年10月4日,美国商务部、美国专利商标局以及经济和统计管理局联合发布研究报告《知识产权与美国经济2016(Intellectual Property and the US Economy:2016 Update)》,指出知识产权密集型产业至少为美国提供了4500万个就业机会,产业贡献超过6万亿美元,占美国GDP总量的38.2%。该报告识别了81个广泛采用专利、著作权以及商标权保护的知识产权密集型产业,包括软件出版业、录音产业、音频和视频设备制造业、有线电视和其他订阅节目、表演艺术公司以及广播电视。这些产业直接或间接产生的就业机会约占美国所有产业的30%。
上述报告清楚地显示了保护知识产权对美国极其重要。这就是川普总统上任前后反复强调保护知识产权的重要性,并反复指责中国(中共)偷窃美国知识产权带来巨大损失的原因。
2017年6月19日,由彼得·纳瓦罗领导的白宫贸易与制造业政策办公室发表了一份报告,列举了中国(中共)的六类经济侵略战略,并针对第五项(从包括美国在内的其他国家获得关键技术与知识产权)和第六项(掌握推动未来经济增长的新兴高科技产业以及国防工业技术)“战略”详细列举了中国(中共)5类(27种)相关行为、政策或做法:(1)盗窃或通过网络窃取技术与知识产权;(2)通过强迫或侵入性监管手段,强制转让技术和知识产权;(3)施加经济压力胁迫技术与知识产权转移;(4)信息收集;(5)国家资助以获取技术为目的的对外直接投资。报告指出,一项研究发现,盗版软件和假冒商品每年给美国带来的损失可高达6000亿美元,其中大部分来自于中国的仿冒与剽窃。
就在G20峰会召开的第一天,美国军事出版商Defense One主办了一场讨论会,司法部国家安全事务助理部长德默斯说,在司法部所处理的外国政府主导的网络盗窃案中,90%的案件涉及中国政府行为,“从这些案件中你可以看到(中国)有组织、资源配备充足、从上至下的行动。他们使用政府情报机构用来针对外国政府机密所使用的相同工具和技能来盗窃商业机密,以便提升自己的产能。”
在美国总统特朗普6月29日的记者会上,有记者问他如何定义美中关系,是战略伙伴?竞争者?还是敌人?特朗普说美中应该是能够互相帮助的战略伙伴。许多媒体抓住“战略伙伴”这个词大作文章,独独忽视了特朗普这句话有“if”这个条件前置词,引导的是几个关键条件,“如果我们最终能达成正确的协议,……如果北京能开放,……”,在特朗普总统眼中,正确的协议就是北京承诺解决偷窃知识产权和结构性改革,即中美之间未谈成的那10%。可以预见,这10%的未完成协议,只有在特朗普总统的第二个任期内才有可能达成。

DNS over TLS的实现方法

支持 DNS over TLS 的dns客户端程序/服务器程序。

SOFARegistry

SOFARegistry is a production-level, low-latency, high-availability service registry powered by Ant Financial.
Build Status license Coverage Status maven
SOFARegistry 是蚂蚁金服开源的一个生产级、高时效、高可用的服务注册中心。SOFARegistry 最早源自于淘宝的 ConfigServer,十年来,随着蚂蚁金服的业务发展,注册中心架构已经演进至第五代。目前 SOFARegistry 不仅全面服务于蚂蚁金服的自有业务,还随着蚂蚁金融科技服务众多合作伙伴,同时也兼容开源生态。SOFARegistry 采用 AP 架构,支持秒级时效性推送,同时采用分层架构支持无限水平扩展。

功能特性

  • 支持服务发布与服务订阅
  • 支持服务变更时的主动推送
  • 丰富的 REST 接口
  • 采用分层架构及数据分片,支持海量连接及海量数据
  • 支持多副本备份,保证数据高可用
  • 基于 SOFABolt 通信框架,服务上下线秒级通知
  • AP 架构,保证网络分区下的可用性

需要

编译需要 JDK 8 及以上、Maven 3.2.5 及以上。
运行需要 JDK 6 及以上,服务端运行需要 JDK 8及以上

文档

electron-quick-start

try a simple Electron app 
Clone and run for a quick way to see Electron in action.
This is a minimal Electron application based on the Quick Start Guide within the Electron documentation.
Use this app along with the Electron API Demos app for API code examples to help you get started.
A basic Electron application needs just these files:
  • package.json - Points to the app's main file and lists its details and dependencies.
  • main.js - Starts the app and creates a browser window to render HTML. This is the app's main process.
  • index.html - A web page to render. This is the app's renderer process.
You can learn more about each of these components within the Quick Start Guide.

To Use

To clone and run this repository you'll need Git and Node.js (which comes with npm) installed on your computer. From your command line:
# Clone this repository
git clone https://github.com/electron/electron-quick-start
# Go into the repository
cd electron-quick-start
# Install dependencies
npm install
# Run the app
npm start
Note: If you're using Linux Bash for Windows, see this guide or use node from the command prompt.

Resources for Learning Electron

from https://github.com/electron/electron-quick-start
-----

The Electron framework lets you write cross-platform desktop applications using JavaScript, HTML and CSS. It is based on Node.js and Chromium and is used by the Atom editor and many other apps.
Follow @ElectronJS on Twitter for important announcements.
This project adheres to the Contributor Covenant code of conduct. By participating, you are expected to uphold this code. Please report unacceptable behavior to coc@electronjs.org.

Installation

To install prebuilt Electron binaries, use npm. The preferred method is to install Electron as a development dependency in your app:
npm install electron --save-dev [--save-exact]
The --save-exact flag is recommended for Electron prior to version 2, as it does not follow semantic versioning. As of version 2.0.0, Electron follows semver, so you don't need --save-exact flag. For info on how to manage Electron versions in your apps, see Electron versioning.
For more installation options and troubleshooting tips, see installation.

Quick start & Electron Fiddle

Use Electron Fiddle to build, run, and package small Electron experiments, to see code examples for all of Electron's APIs, and to try out different versions of Electron. It's designed to make the start of your journey with Electron easier.
Alternatively, clone and run the electron/electron-quick-start repository to see a minimal Electron app in action:
git clone https://github.com/electron/electron-quick-start
cd electron-quick-start
npm install
npm start

Resources for learning Electron

--------------------------------------------------
 
https://github.com/electron/apps

simplereader

参考"任阅" 网络小说阅读器,一款基于ReactNative的小说阅读器.

"简阅" 网络小说阅读器仿照"任阅" 网络小说阅读器,实现追书推荐收藏、书籍/标签检索、读书等功能!

介绍

项目基于ReactNative + Redux的框架开发
。项目中的API均来自追书神器,纯属共享学习之用,不得用于商业!!

TODO

  •  阅读器设置功能,书签功能
  •  缓存书籍到本地
  •  wifi传书(native本地化)
  •  主题设置

体验下载

安装

  1. git clone https://github.com/jsntjinjin/simplereader.git
  2. cd simplereader
  3. npm install
  4. react-native link
  5. react-native run-ios / react-native run-android

"任阅" 网络小说阅读器

"任阅" 网络小说阅读器,仿真翻页效果、txt/pdf/epub书籍阅读.

BookReader

"任阅" 网络小说阅读器,实现追书推荐收藏、书籍/标签检索、模拟翻书的翻页效果、缓存书籍、日夜间模式、本地txt/pdf/epub书籍阅读、字体/主题/亮度设置、Wifi传书等功能!

此项目停止维护,由于当时做的比较着急,分页、翻页、性能这些都做的相当不好,在 言士创作 中,对这方面做了很多的优化,不过没有开源。 有相关问题欢迎发我邮箱交流!

项目

项目基于RxJava + Retrofit2 + Dagger2,项目中的API均来自追书神器,纯属共享学习之用,不得用于商业!!有任何疑问或建议可联系smuyyh@gmail.com
demo下载:任阅.apk
android 本地网络小说爬虫项目: CrawlerForReader
RN版本:simplereader
任阅改进版:NovelReader
----

NovelReader

仿照"任阅"的追书、看书的小说阅读器。重写"任阅"的代码,优化代码逻辑和代码结构,降低内存使用率。重写小说阅读器,支持网络阅读、本地阅读(支持分章,虚拟分章)。支持仿真翻页、上下滚动翻页等翻页效果。支持小说断点续传功能。优化用户使用体验.
基于"任阅"的改进追书App。
详见原作者项目地址:任阅

应用简介

相对于原作增加的功能与改进:
  1. 重写代码逻辑,优化代码结构,降低内存使用率
  2. 采用 sql 数据库对数据进行存储。 (原作者是采用ACache类将数据修改成文本存储)
  3. 增加书本的断点续传功能 (仅支持单任务下载)
  4. 支持小说更新提示。
  5. 重写小说阅读器,仿掌阅设计。提供的功能如下
    • 支持翻页动画:仿真翻页、覆盖翻页、上下滚动翻页等翻页效果。
    • 支持页面定制:亮度调节、背景调节、字体大小调节
    • 支持全屏模式(含有虚拟按键的手机)、音量键翻页
    • 支持页面进度显示、页面切换、上下章切换。
    • 支持在线章节阅读、本地书籍查找。
    • 支持本地书籍加载到页面(支持本地书籍分章、加载速度快、耗费内存少)
注: 项目目前仍有许多问题,希望各位发现 BUG 能够尽快反馈,本人一定尽快修复,谢谢支持。

采用了以下开源框架

Saturday 29 June 2019

meanTorrent

High performance DNS over HTTPS client & server



Client and server software to query DNS over HTTPS, using Google DNS-over-HTTPS protocol and IETF DNS-over-HTTPS (RFC 8484).

Guide

Installing

Install Go, at least version 1.10.
(Note for Debian/Ubuntu users: You need to set $GOROOT if you could not get your new version of Go selected by the Makefile.)
First create an empty directory, used for $GOPATH:
mkdir ~/gopath
export GOPATH=~/gopath
To build the program, type:
make
To install DNS-over-HTTPS as Systemd services, type:
sudo make install
By default, Google DNS over HTTPS is used. It should work for most users (except for People's Republic of China). If you need to modify the default settings, type:
sudoedit /etc/dns-over-https/doh-client.conf
To automatically start DNS-over-HTTPS client as a system service, type:
sudo systemctl start doh-client.service
sudo systemctl enable doh-client.service
Then, modify your DNS settings (usually with NetworkManager) to 127.0.0.1.
To test your configuration, type:
dig www.google.com
If it is OK, you will wee:
;; SERVER: 127.0.0.1#53(127.0.0.1)

Uninstalling

To uninstall, type:
sudo make uninstall
The configuration files are kept at /etc/dns-over-https. Remove them manually if you want.

Server Configuration

The following is a typical DNS-over-HTTPS architecture:
+--------------+                                +------------------------+
| Application  |                                |  Recursive DNS Server  |
+-------+------+                                +-----------+------------+
        |                                                   |
+-------+------+                                +-----------+------------+
| Client side  |                                |      doh-server        |
| cache (nscd) |                                +-----------+------------+
+-------+------+                                            |
        |         +--------------------------+  +-----------+------------+
+-------+------+  |    HTTP cache server /   |  |   HTTP service muxer   |
|  doh-client  +--+ Content Delivery Network +--+ (Apache, Nginx, Caddy) |
+--------------+  +--------------------------+  +------------------------+
Although DNS-over-HTTPS can work alone, a HTTP service muxer would be useful as you can host DNS-over-HTTPS along with other HTTPS services.
HTTP/2 with at least TLS v1.3 is recommended. OCSP stapling must be enabled, otherwise DNS recursion may happen.

Example configuration: Apache

SSLProtocol TLSv1.2
SSLHonorCipherOrder On
SSLCipherSuite ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+3DES:!aNULL:!MD5:!DSS:!eNULL:!EXP:!LOW:!MD5
SSLUseStapling on
SSLStaplingCache shmcb:/var/lib/apache2/stapling_cache(512000)


    ServerName MY_SERVER_NAME
    Protocols h2 http/1.1
    ProxyPass /dns-query http://[::1]:8053/dns-query
    ProxyPassReverse /dns-query http://[::1]:8053/dns-query

(Credit: Joan Moreau)

Example configuration: Nginx

server {
    listen       443 ssl http2;
    listen       [::]:443 ssl http2;
    server_name  MY_SERVER_NAME;

    server_tokens off;

    ssl_protocols TLSv1.2 TLSv1.3;          # TLS 1.3 requires nginx >= 1.13.0
    ssl_prefer_server_ciphers on;
    ssl_dhparam /etc/nginx/dhparam.pem;     # openssl dhparam -dsaparam -out /etc/nginx/dhparam.pem 4096
    ssl_ciphers EECDH+AESGCM:EDH+AESGCM;
    ssl_ecdh_curve secp384r1;               # Requires nginx >= 1.1.0
    ssl_session_timeout  10m;
    ssl_session_cache shared:SSL:10m;
    ssl_session_tickets off;                # Requires nginx >= 1.5.9
    ssl_stapling on;                        # Requires nginx >= 1.3.7
    ssl_stapling_verify on;                 # Requires nginx => 1.3.7 
    resolver 1.1.1.1 valid=300s;            # Replace with your local resolver
    resolver_timeout 5s;
    # HTTP Security Headers
    add_header X-Frame-Options DENY;
    add_header X-Content-Type-Options nosniff;
    add_header X-XSS-Protection "1; mode=block";
    add_header Strict-Transport-Security "max-age=63072000";
    ssl_certificate /path/to/your/server/certificates/fullchain.pem;
    ssl_certificate_key /path/to/your/server/certificates/privkey.pem;
    location /dns-query {
        proxy_pass       http://localhost:8053/dns-query;
        proxy_set_header Host      $host;
        proxy_set_header X-Real-IP $remote_addr;
    }
}
备注:上面配置文件内的ssl_early_data off那一行已被我删除。如果加那一行,重启nginx时,反而会遇错。
上面配置文件内的default_server也是多余的,也被我删了。
(Credit: Cipherli.st)

Example configuration: Caddy


https://MY_SERVER_NAME {
        log     / syslog "{remote} - {user} [{when}] \"{method} {scheme}://{host}{uri} {proto}\" {status} {size} \"{>Referer}\" \"{>User-Agent}\" {>X-Forwarded-For}"
        errors  syslog
        gzip
        proxy   /dns-query      http://[::1]:18053 {
                header_upstream Host {host}
                header_upstream X-Real-IP {remote}
                header_upstream X-Forwarded-For {>X-Forwarded-For},{remote}
                header_upstream X-Forwarded-Proto {scheme}
        }
        root    /var/www
        tls {
                ciphers ECDHE-ECDSA-WITH-CHACHA20-POLY1305 ECDHE-RSA-WITH-CHACHA20-POLY1305 ECDHE-ECDSA-AES256-GCM-SHA384 ECDHE-RSA-AES256-GCM-SHA384 ECDHE-ECDSA-AES128-GCM-SHA256 ECDHE-RSA-AES128-GCM-SHA256
                curves  X25519 p384 p521
                must_staple
        }
}

DNSSEC

DNS-over-HTTPS is compatible with DNSSEC, and requests DNSSEC signatures by default. However signature validation is not built-in. It is highly recommended that you install unbound or bind and pass results for them to validate DNS records.

EDNS0-Client-Subnet (GeoDNS)

DNS-over-HTTPS supports EDNS0-Client-Subnet protocol, which submits part of the client's IP address (/24 for IPv4, /56 for IPv6 by default) to the upstream server. This is useful for GeoDNS and CDNs to work, and is exactly the same configuration as most public DNS servers.
Keep in mind that /24 is not enough to track a single user, although it is precise enough to know the city where the user is located. If you think EDNS0-Client-Subnet is affecting your privacy, you can set no_ecs = true in /etc/dns-over-https/doh-client.conf, with the cost of slower video streaming or software downloading speed.
To ultilize ECS, X-Forwarded-For or X-Real-IP should be enabled on your HTTP service muxer. If your server is backed by unbound or bind, you probably want to configure it to enable the EDNS0-Client-Subnet feature as well.

Protocol compatibility

Google DNS-over-HTTPS Protocol

DNS-over-HTTPS uses a protocol compatible to Google DNS-over-HTTPS, except for absolute expire time is preferred to relative TTL value. Refer to json-dns/response.go for a complete description of the API.

IETF DNS-over-HTTPS Protocol

DNS-over-HTTPS uses a protocol compatible to IETF DNS-over-HTTPS (RFC 8484).

Supported features

Currently supported features are:
  •  IPv4 / IPv6
  •  EDNS0 large UDP packet (4 KiB by default)
  •  EDNS0-Client-Subnet (/24 for IPv4, /56 for IPv6 by default)

The name of the project

This project is named "DNS-over-HTTPS" because it was written before the IETF DoH project. Although this project is compatible with IETF DoH, the project is not affiliated with IETF.
To avoid confusion, you may also call this project "m13253/DNS-over-HTTPS" or anything you like.

我的补充说明:

在linux vps上。先安装go,然后,
cd $GOPATH
go get -u -v github.com/m13253/dns-over-https/doh-server/
安装完成后,doh-server会出现在$GOBIN下面。

[root@host gopath]# which doh-server
/root/go/gopath/bin/doh-server
[root@host gopath]# doh-server -h
Usage of doh-server:
  -conf string
     Configuration file (default "doh-server.conf")
  -pid-file string
     PID file for legacy supervision systems lacking support for reliable cgroup-based process tracking
  -verbose
     Enable logging
  -version
     Show software version and exit
[root@host gopath]# cd ~
[root@host ~]# find / -name doh-server.conf
/root/go/gopath/src/github.com/m13253/dns-over-https/doh-server/doh-server.conf
[root@host ~]# cp /root/go/gopath/src/github.com/m13253/dns-over-https/doh-server/doh-server.conf ~/
[root@host ~]# nano doh-server.conf
[root@host ~]# cat doh-server.conf
# HTTP listen port
listen = [
    ":8053",

    ## To listen on both 0.0.0.0:8053 and [::]:8053, use the following line
    # ":8053",
]

# Local address and port for upstream DNS
# If left empty, a local address is automatically chosen.
local_addr = ""

# TLS certification file
# If left empty, plain-text HTTP will be used.
# You are recommended to leave empty and to use a server load balancer (e.g.
# Caddy, Nginx) and set up TLS there, because this program does not do OCSP
# Stapling, which is necessary for client bootstrapping in a network
# environment with completely no traditional DNS service.
cert = ""

# TLS private key file
key = ""

# HTTP path for resolve application
path = "/dns-query"

# Upstream DNS resolver
# If multiple servers are specified, a random one will be chosen each time.
upstream = [
    "8.8.8.8:53",
    "8.8.4.4:53",
]

# Upstream timeout
timeout = 10

# Number of tries if upstream DNS fails
tries = 3

# Only use TCP for DNS query
tcp_only = false

# Enable logging
verbose = false

# Enable log IP from HTTPS-reverse proxy header: X-Forwarded-For or X-Real-IP
# Note: http uri/useragent log cannot be controlled by this config

log_guessed_client_ip = false
[root@host ~]# nano /etc/systemd/system/doh-server.service
[root@host ~]# cat /etc/systemd/system/doh-server.service
[Unit]
After=network.target

[Service]
ExecStart=/root/go/gopath/bin/doh-server -conf /root/doh-server.conf
Restart=always

[Install]
WantedBy=multi-user.target
[root@host ~]# systemctl start doh-server
[root@host ~]# systemctl enable doh-server

然后在vps上,安装nginx1.14,按上面的Example configuration: Nginx段的内容,添加该段内容至你的nginx的配置文件内,然后重启nginx.
这样,服务器端搭建完成。


在本地机器mac.先安装go,然后,
cd $GOPATH
go get -u -v github.com/m13253/dns-over-https/doh-client/
安装完成后,doh-client会出现在$GOBIN下面。
doh-client -h
会显示:
Usage of doh-client:
  -conf string
    Configuration file (default "doh-client.conf")
  -verbose
    Enable logging
  -version
     Show software version and exit

sudo find / -name doh-client.conf
会显示:
...
$GOPATH/src/github.com/m13253/dns-over-https/doh-client/doh-client.conf
...
cp $GOPATH/src/github.com/m13253/dns-over-https/doh-client/doh-client.conf ~/
cd ~
nano doh-client.conf
(启用里面的3行:
[[upstream.upstream_ietf]]
url = "https://9.9.9.9/dns-query"
weight = 50

或者这3行:
[[upstream.upstream_ietf]]
url = "https://1.1.1.1/dns-query"
weight = 50  )

sudo  $GOBIN/doh-client -conf ~/doh-client.conf

不过在doh-client.conf文件里,像上面那样去启用那2段之一,并没有用到所搭建的‘ DNS over HTTPS’ 服务器。要用到所搭建的‘ DNS over HTTPS’ 服务器,先把上面所启用的那2段之一都禁用掉:
nano doh-client.conf
(把bootstrap = [
内容
]
改为:
bootstrap = [
"208.67.222.222:443",
]

注意:上面的"208.67.222.222:443"一定要使用非53的端口,比如443或5353.建议使用443端口,速度更快。事实上,208.67.222.222:443,208.67.222.222:5353,208.67.220.220:443,208.67.220.220:5353这四组之一都行。你把它们全部加进去也行:
bootstrap = [
"208.67.222.222:443",
"208.67.222.222:5353",
"208.67.220.220:443",
"208.67.220.220:5353",
]

其实,你还可在自己的vps上安装dnsmasq
(apt-get install dnsmasq -y或yum install -y dnsmasq)
所有的配置都在一个文件中完成/etc/dnsmasq.conf, Dnsmasq配置文件是/etc/dnsmasq.conf,下面对Dnsmasq中和DNS相关的配置项进行说明:
用指定的端口代替默认的DNS 53端口,如果设置为0,则完全禁止DNS功能.
port=53538

从不转发格式错误的域名:
domain-needed

从不转发不在路由地址中的域名:
bogus-priv

然后,运行dnsmasq即可。然后,在本地机器,修改doh-client.conf文件,在
bootstrap = [
...
] 的...处,加上"my-vps-ip:53538",

使用了一段时间,还是觉得把doh server的域名的 IP 地址直接写到/etc/hosts文件里比较可靠。如果使用bootstrap DNS 的方式,有时不能完全的加载被封网站。
(详见https://github.com/m13253/dns-over-https/issues/65),使用这种方式,需先把
bootstrap = [
...
] 的方括号里的内容全部注释掉或删除,然后去修改/etc/hosts文件。

然后,在这行## Google's resolver, good ECS, good DNSSEC的上方加上:
[[upstream.upstream_ietf]]
url = "https://mydomain.com/dns-query"
weight = 100

保存更改。然后运行:
sudo $GOBIN/doh-client -conf ~/doh-client.conf

使用例子:
运行全局代理程序mellow

sudo networksetup -setdnsservers "Wi-Fi" 127.0.0.1

sudo $GOBIN/doh-client -conf ~/doh-client.conf

然后,就可以用全局代理程序mellow任意翻墙。

OpenDNS 开放非常规端口 5353查询和 TCP查询,即使从国内直接请求也不容易被污染和劫持。如果你在使用 ChinaDNS 这类工具同时又没有专门为其准备一条加密隧道,那么直连 OpenDNS 的 5353 就是一个不错的替代选择。
---------------

自建DoH服务


1. 前言

DoH(DNS over HTTPS),顾名思义,使用HTTPS协议执行DNS查询,除了最常用的UDP外,还有DoT(DNS over TLS),DNS over HTTP(服务提供商自定义)等方案,对比如下:

协议标准描述
DNS over HTTPSRFC8484使用TLS加密的HTTP/2执行DNS查询
DNS over TLSRFC7858使用TLS加密的TCP执行DNS查询
DNS over HTTP服务提供商自定义使用自定义加密的HTTP/1.1执行DNS查询

移动端的DNS优化已经有很多实践,最常见的是DNS over HTTP,通过加密的HTTP请求规避运营商对DNS的UDP包劫持,从而优化App访问服务器的延迟。但这个方案并没有形成统一的标准,通常需要内嵌DNS服务提供商的SDK,通过访问固定的BGP或任播IP获取DNS响应。

大概是意识到DNS在移动互联网中的扮演越来越重要的角色,在DoT和DoH的规范相继推出后,许多DNS服务提供商都跟进了部署,国内的阿里云、DNSPod,国外的谷歌、Cloudflare等目前已经推出了免费的DoT和DoH服务。

客户端方面,常用的Chrome、FireFox已经支持了自定义DoH服务器,macOS、iOS也可通过配置文件设置系统范围的默认DoH服务器。

笔者也正好有一个自定义DNS的需求:

  1. 需要针对一些域名的DNS查询仅返回IPv4记录
  2. 使用的某某路由器系统的自定义DNS服务仅支持设置UDP和DoH
  3. UDP模式默认使用53端口,不可修改,UDP包容易遭受干扰
  4. DoH可自定义域名、端口且使用HTTP2作为传输协议,稳定性更强

综上,只有自建DoH服务了,于是就有了下面的折腾,最后测试时发现这个傻瓜路由器系统只支持一些特定的DoH服务商如阿里云DNS、DNSPod等,不支持自建的DoH服务。

2. 部署方案

DoH本质上就是一个HTTP请求,只是目前协议定义要求启用TLS与HTTP/2。最初没有跑通coredns的DoH时,使用了nginx作为前端转发DoH请求到doh-server,然后doh-server使用本地的coredns服务作为上游。

最近再仔细研究了下文档,发现coredns已经支持了DoH服务,可直接对外暴露服务,或者通过nginx转发来复用已经部署好的web服务。

2.1 nginx + doh-server + coredns

https://github.com/m13253/dns-over-https 是一个提供 DNS over HTTP 的服务,需要一个web前端和一个DNS后端,可用的docker镜像地址为:satishweb/doh-server,使用doh-server时,DNS请求流转如下:

1
HTTP Service -> doh-server -> DNS Server

RFC8484中指定使用/dns-query路径作为默认查询路径,因此只需要将该路径前缀的请求转发到doh-server即可,如下:

nginx配置(已配置好TLS与HTTP2)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
server {
listen 443 ssl http2 fastopen=256 reuseport;
listen [::]:443 ssl http2 fastopen=256 reuseport;
server_name doh.wbuntu.com
...
location /dns-query {
proxy_redirect off;
proxy_http_version 1.1;
proxy_set_header Host $http_host;
# show real IP
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://127.0.0.1:8053;
}
}

doh-server

使用hostNetwork模式启动服务,监听8053端口

1
2
3
4
5
6
7
8
docker run -d --restart unless-stopped --network host --name doh-server \
-e UPSTREAM_DNS_SERVER="udp:127.0.0.1:53" \
-e DOH_HTTP_PREFIX="/dns-query" \
-e DOH_SERVER_LISTEN="127.0.0.1:8053" \
-e DOH_SERVER_TIMEOUT="10" \
-e DOH_SERVER_TRIES="3" \
-e DOH_SERVER_VERBOSE="true" \
satishweb/doh-server

coredns

coredns配置文件如下

1
2
3
4
5
6
7
8
9
10
11
12
13
➜  ~ tree /etc/coredns/
/etc/coredns/
└── Corefile

0 directories, 1 files
➜ cat /etc/coredns/Corefile
.:53 {
bind 127.0.0.1
forward . 1.1.1.1 1.0.0.1
log
errors
cache
}

使用hostNetwork模式启动服务,监听53端口

1
2
3
4
docker run -d --restart unless-stopped --network host --name coredns \
-v /etc/coredns:/etc/coredns \
coredns/coredns \
-conf /etc/coredns/Corefile

服务启动后,我们可以得到一个自定义的DoH服务:https://doh.wbuntu.com/dns-query

2.2 coredns

目前coredns支持作为DoH服务端,不支持连接上游DoH服务器,上游服务器可使用UDP和DoT。

直接对外暴露服务需要使用有效的TLS证书,coredns配置文件及证书位置如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
➜  ~ tree /etc/coredns/
/etc/coredns/
├── Corefile
├── tls.crt
└── tls.key

0 directories, 3 files
➜ cat /etc/coredns/Corefile
https://.:443 {
tls /etc/coredns/tls.crt /etc/coredns/tls.key
bind 0.0.0.0
forward . 1.1.1.1 1.0.0.1
log
errors
cache
}

使用hostNetwork模式启动服务,监听443端口

1
2
3
4
docker run -d --restart unless-stopped --network host --name coredns \
-v /etc/coredns:/etc/coredns \
coredns/coredns \
-conf /etc/coredns/Corefile

服务启动后,我们可以得到一个自定义的DoH服务:https://doh.wbuntu.com/dns-query

2.3 nginx + coredns

直接暴露coredns服务到公网需要占用端口,coredns在未配置TLS证书时,可使用nginx作为前端来复用web服务,如下:

nginx配置(已配置好TLS与HTTP2)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
server {
listen 443 ssl http2 fastopen=256 reuseport;
listen [::]:443 ssl http2 fastopen=256 reuseport;
server_name doh.wbuntu.com
...
location /dns-query {
proxy_redirect off;
proxy_http_version 1.1;
proxy_set_header Host $http_host;
# show real IP
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://127.0.0.1:8053;
}
}

coredns

coredns配置文件如下

1
2
3
4
5
6
7
8
9
10
11
12
13
➜  ~ tree /etc/coredns/
/etc/coredns/
└── Corefile

0 directories, 1 files
➜ cat /etc/coredns/Corefile
https://.:8053 {
bind 127.0.0.1
forward . 1.1.1.1 1.0.0.1
log
errors
cache
}

使用hostNetwork模式启动服务,监听8053端口

1
2
3
4
docker run -d --restart unless-stopped --network host --name coredns \
-v /etc/coredns:/etc/coredns \
coredns/coredns \
-conf /etc/coredns/Corefile

服务启动后,我们可以得到一个自定义的DoH服务:https://doh.wbuntu.com/dns-query

3. 测试

使用谷歌浏览器配置DoH服务:Settings -> Secutiry and Privacy -> Secutiry -> Advanced -> Use secure DNS.

--------------------------------------------------

(Cloudflare 與 ISP 合作推出 DoH 加強隱私,然後 Google 想要看 HTTPS 流量


Cloudflare
 推出了 ODoH (目前是 IETF 的 draft:「Oblivious DNS Over HTTPS」):「Improving DNS Privacy with Oblivious DoH in 1.1.1.1」,在 Hacker News 上面也有討論:「 Improving DNS Privacy with Oblivious DoH (cloudflare.com)

基本上就是 DNS over HTTPS 在上面架一層 Proxy,但這層 Proxy 不能是 Cloudflare 自己:

 

這樣一來 Cloudflare 知道 IP address 的機會就會比較小,藉以達到要求,先前要達到這樣的效果必須透過 ISP 提供的 HTTP/HTTPS Proxy (像是已經淘汰的 proxy.hinet.net:「HiNet 宣佈年底關閉 Proxy 服務」),或是透過 Tor,但 Tor 的效能會讓 query 速度慢不少。這次的這個服務的確是好不少...

技術上來說,當 Cloudflare 與 ISP 都把所有的 packet 記錄下來後,兩邊合作還是可以取得原始的 IP 資訊,以這個例子來說,你跟總部在香港的 PCCW 集團合作,看起來就不怎麼吸引人啊...

不過隔壁棚的 Google 則是讓人吐血中,打算用 Prefetch 名義看到你的 HTTPS 流量:「Continuing our journey to bring instant experiences to the whole web」,這樣一來,就有不少的機會 Google 可以分析出來使用者在看什麼 Netflix 影片了 (要看 Prefetch 到什麼程度,2017 年的時候做出來有 99.99% 的準確度):「利用 Side-channel 資訊判斷被 HTTPS 保護的 Netflix 影片資訊」。

來坐著等看 Google 這邊的好戲...)

----------------


Tutorial to setup your own DNS-over-HTTPS (DoH) server


Introduction

Traditional DNS queries and responses are sent over UDP or TCP without encryption. This is vulnerable to eavesdropping and spoofing (including DNS-based Internet filtering). Responses from recursive resolvers to clients are the most vulnerable to undesired or malicious changes, while communications between recursive resolvers and authoritative name servers often incorporate additional protection. (Google)
To simplify, anybody on your network, your ISP, etc … can easily spoof DNS response and decide to send you to a different website than the one you desired. Also, it has some privacy implication where anybody between you and the DNS server can know what website you visit.

Guide

The guide is divided in multiple part. The first one covers how to setup a DNS-over-HTTPS (DoH) while using dnscrypt-proxy as DNS server to answer the requests.
The second part explains how to make couple of changes to that configuration to have PiHole (dns server that block ads) as DNS server behind DoH.
The third part explains how to add DNS-over-TLS to your setup. Useful if you own Android 9 (Pie) devices.
The last  part will provide you with a list of client for Windows, Linux, Android and iOS that supports DoH natively to be able to use it on all your devices.

Server

I advise you to setup a free f1 micro instance at Google Cloud Computing. You can setup anywhere you want, I only advise there because they have a good image for Ubuntu 18.04 and the f1 micro instance is free forever. All the request the machine will do will be encrypted and not accessible by Google.
Again, if you’re more familiar with Digital Ocean, AWS, etc … please use the hosting provider you know the best. For this guide, I only advise you to have a Debian based image (Debian, Ubuntu, etc …)

Architecture


DNS-over-HTTPS server

The next step is to install the server that implement the DoH protocol to get an HTTP request and do a DNS request.
I provide 2 ways to install it, either you download the deb I provide or you compile the program (in golang) yourself.

Download

For this tutorial, I’ve taken the time to compile and package DNS-over-HTTPS (Golang) and provide a deb file easily installable.

Compile

If you prefer to build it yourself, you can follow the guide provided in the GitHub repository.
After compiling you can use FPM to build the package.






Install

If you compile it yourself, you won’t need to do this, the make install will have already taken care of it.
  1. sudo dpkg -i doh-server_*_amd64.deb
This will install and start the service for you.

Configuration

Open the file /etc/dns-over-https/doh-server.conf in your favorite editor. Keep somewhere the listen IP/Port. We’ll need it when we’ll setup Nginx.
Change upstream variable.
  1. # HTTP listen port
  2. listen = [
  3. "127.0.0.1:8053",
  4. "[::1]:8053",
  5. ]

  6. # TLS certification file
  7. # If left empty, plain-text HTTP will be used.
  8. # You are recommended to leave empty and to use a server load balancer (e.g.
  9. # Caddy, Nginx) and set up TLS there, because this program does not do OCSP
  10. # Stapling, which is necessary for client bootstrapping in a network
  11. # environment with completely no traditional DNS service.
  12. cert = ""

  13. # TLS private key file
  14. key = ""

  15. # HTTP path for resolve application
  16. path = "/dns-query"

  17. # Upstream DNS resolver
  18. # If multiple servers are specified, a random one will be chosen each time.
  19. upstream = [
  20. "127.0.0.1:53",
  21. ]

  22. # Upstream timeout
  23. timeout = 60

  24. # Number of tries if upstream DNS fails
  25. tries = 10

  26. # Only use TCP for DNS query
  27. tcp_only = false

  28. # Enable logging
  29. verbose = false
This will tell DoH-server to use our dnscrypt-proxy to do its DNS requests.
Once done, restart the service.
  1. sudo systemctl restart doh-server

Nginx

This section focus on installing and configuring Nginx to take care of the HTTPS part of DNS-over-HTTPS. To do this, we configure it as a reverse proxy and use let’s encrypt to generate a certificate.

Install

We add the PPA with TLS 1.3 of Nginx to get the latest stable version with TLS 1.3.
  1. sudo add-apt-repository ppa:ondrej/nginx
  2. sudo apt install nginx-full

Configuration

This is an example of a configuration. You need to change the server_name to the domain you’ll use for DoH. Also check that the uptream server point to doh-server ip and port. If you didn’t change anything in the configuration of doh-server, it’s already configured correctly.
For now, we don’t enable SSL, this will be done after with certbot & let’s encrypt.
  1. upstream dns-backend {
  2. server 127.0.0.1:8053;
  3. }

  4. server {
  5. listen 80;
  6. server_name dns.example.com;
  7. root /var/www/html/dns;
  8. access_log /var/log/nginx/dns.access.log;

  9. location /dns-query {
  10. proxy_set_header X-Real-IP $remote_addr;
  11. proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
  12. proxy_set_header Host $http_host;
  13. proxy_set_header X-NginX-Proxy true;
  14. proxy_http_version 1.1;
  15. proxy_set_header Upgrade $http_upgrade;
  16. proxy_redirect off;
  17. proxy_set_header X-Forwarded-Proto $scheme;
  18. proxy_read_timeout 86400;
  19. proxy_pass http://dns-backend/dns-query ;
  20. }
  21. }
Put the content of the configuratione into /etc/nginx/sites-available/dns-over-https.
Then do a symlink to the enabled folder. Ask Nginx to check that to configuration works, and reload nginx.
  1. sudo ln -s /etc/nginx/sites-available/dns-over-https /etc/nginx/sites-enabled/dns-over-https
  2. sudo nginx -t
  3. sudo systemctl reload nginx
And there you go, you have now Nginx that will takes care of serving HTTP request to doh-server.

Stapling

The idea is to make Nginx take care of checking if the certificate is expired and keep that information in cache. This is to avoid doing too many requests on the Certificate Authority (CA) of the certificate.
Definition
Create a new file into /etc/nginx/conf.d/stapling.conf with the following content:
  1. ssl_stapling on;
  2. ssl_stapling_verify on;
  3. resolver 127.0.2.1;
This will activate the stapling for all your website hosted with Nginx and using HTTPS.
Feel free to change the resolver variable. By default I made it use the dnscrypt-proxy we configured, but you can change it to any other DNS server.

Certbot

Certbot is the tool developed by EFF to help you request SSL certificate using let’s encrypt. Not only it will generate a certificate for your domain, it will also configure Nginx for you and take care of renewing the certificate.

Install

Usually the version available in the distribution is a little old. We’re going to use the official PPA.
  1. sudo add-apt-repository ppa:certbot/certbot
  2. sudo apt install python-certbot-nginx

Configuration

Certbot provides a variety of ways to obtain SSL certificates, through various plugins. The Nginx plugin will take care of reconfiguring Nginx and reloading the config whenever necessary:
  1. sudo certbot --nginx -d dns.example.com
This runs certbot with the --nginx plugin, using -d to specify the names we’d like the certificate to be valid for.
If this is your first time running certbot, you will be prompted to enter an email address and agree to the terms of service. After doing so, certbot will communicate with the Let’s Encrypt server, then run a challenge to verify that you control the domain you’re requesting a certificate for.
If that’s successful, certbot will ask how you’d like to configure your HTTPS settings.
  1. Please choose whether or not to redirect HTTP traffic to HTTPS, removing HTTP access.
  2. -------------------------------------------------------------------------------
  3. 1: No redirect - Make no further changes to the webserver configuration.
  4. 2: Redirect - Make all requests redirect to secure HTTPS access. Choose this for
  5. new sites, or if you're confident your site works on HTTPS. You can undo this
  6. change by editing your web server's configuration.
  7. -------------------------------------------------------------------------------
  8. Select the appropriate number [1-2] then [enter] (press 'c' to cancel):
I advise to choose redirect to be sure it use only HTTPS.

SSL Defaults

Certbot comes with “good-enough” SSL defaults, but they haven’t been updated in a while. It keeps support for TLS1.0 which has been deprecated for years. No device should use it anymore. Moreover the chosen cypher list contains weak cyphers. To resolve this issue, I compiled a new configuration file for you to replace the weak defaults of Certbot.
Edit the file /etc/letsencrypt/options-ssl-nginx.conf and replace its content by this.
  1. # This file contains important security parameters. If you modify this file
  2. # manually, Certbot will be unable to automatically provide future security
  3. # updates. Instead, Certbot will print and log an error message with a path to
  4. # the up-to-date file that you will need to refer to when manually updating
  5. # this file.

  6. ssl_session_cache shared:le_nginx_SSL:1m;
  7. ssl_session_timeout 1440m;

  8. ssl_protocols TLSv1.1 TLSv1.2 TLSv1.3;


  9. # Enable modern TLS cipher suites
  10. ssl_ciphers 'ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256';

  11. # The order of cipher suites matters
  12. ssl_prefer_server_ciphers on;

  13. add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
Then reload nginx.
  1. sudo systemctl reload nginx

Renewal

By design, the certificate will expire in 90 days. Certbot will take care of renewing it 30 days before expiry. In the case you want to test the renewal process you can run this command.
If you remove the --dry-run, you’ll actively ask Certbot to renew the certificate.
  1. sudo certbot renew --dry-run

Conclusion

Congratulation you have now a DNS-over-HTTPS server running that can accept request at https://dns.example.com/dns-query.
This conclude the first part of the guide. The second convers the differents clients available, like dnscrypt-proxy (windows/linux) and Intra (Android). And the third one how to make this DoH block advertising.
A lightweight DNS-over-HTTPS proxy.

https_dns_proxy

https_dns_proxy is a light-weight DNS over HTTPS, non-caching translation proxy for the emerging DoH DNS-over-HTTPS standard. It receives regular (UDP) DNS requests and issues them via DoH.
Google's DNS-over-HTTPS service is default, but Cloudflare's service also works with trivial commandline flag changes.

Using Google

# ./https_dns_proxy -u nobody -g nogroup -d -b 8.8.8.8,8.8.4.4 \
    -r "https://dns.google.com/resolve?"

Using Cloudflare

# ./https_dns_proxy -u nobody -g nogroup -d -b 1.1.1.1,1.0.0.1 \
    -r "https://cloudflare-dns.com/dns-query?ct=application/dns-json&"

Why?

Using DNS over HTTPS makes eavesdropping and spoofing of DNS traffic between you and the HTTPS DNS provider (Google/Cloudflare) much less likely. This of course only makes sense if you trust your DoH provider.

Features

  • Tiny Size (<30kib li="">
  • Uses curl for HTTP/2 and pipelining, keeping resolve latencies extremely low.
  • Single-threaded, non-blocking select() server for use on resource-starved embedded systems.
  • Designed to sit in front of dnsmasq or similar caching resolver for transparent use.

Build

Depends on c-areslibcurllibev.
On Debian-derived systems those are libc-ares-dev, libcurl4-{openssl,nss,gnutls}-dev and libev-dev respectively. On Redhat-derived systems those are c-ares-devel, libcurl-devel and libev-devel.
On MacOS, you may run into issues with curl headers. Others have had success when first installing curl with brew.
brew install curl --with-openssl --with-c-ares --with-libssh2 --with-nghttp2 --with-gssapi --with-libmetalink
brew link curl --force
If all pre-requisites are met, you should be able to build with:
$ cmake .
$ make

INSTALL

There is no installer at this stage - just run it.

OpenWRT package install

I maintain a package in the OpenWRT packages repository as well. You can install as follows:
root@OpenWrt:~# opkg update
root@OpenWrt:~# opkg install https_dns_proxy
root@OpenWrt:~# /etc/init.d/https_dns_proxy enable
root@OpenWrt:~# /etc/init.d/https_dns_proxy start
Replace any 'list server' lines in /etc/config/dhcp with:
list server '127.0.0.1#5053'
You may also want to add the line:
noresolv '1'
This prevents dnsmasq from using /etc/resolv.conf DNS servers, leaving only our proxy server.

archlinux package install

There is also an externally maintained AUR package for latest git version. You can install as follows:
user@arch:~# yaourt -S https-dns-proxy-git

Usage

Just run it as a daemon and point traffic at it. Commandline flags are:
Usage: ./https_dns_proxy [-a ] [-p ]
        [-d] [-u ] [-g ] [-b ]
        [-r ] [-e ]
        [-t ] [-l ] [-x] [-v]+

  -a listen_addr         Local address to bind to. (127.0.0.1)
  -p listen_port         Local port to bind to. (5053)
  -d                     Daemonize.
  -u user                Optional user to drop to if launched as root.
  -g group               Optional group to drop to if launched as root.
  -b dns_servers         Comma separated IPv4 address of DNS servers
                         to resolve resolver host (e.g. dns.google.com).  (8.8.8.8,1.1.1.1,8.8.4.4,1.0.0.1,145.100.185.15,145.100.185.16,185.49.141.37)
  -r resolver_url_prefix The HTTPS path to the JSON resolver URL.  (https://dns.google.com/resolve?)
  -e subnet_addr         An edns-client-subnet to use such as "203.31.0.0/16".  ()
  -t proxy_server        Optional HTTP proxy. e.g. socks5://127.0.0.1:1080
                         Remote name resolution will be used if the protocol
                         supports it (http, https, socks4a, socks5h), otherwise
                         initial DNS resolution will still be done via the
                         bootstrap DNS servers.
  -l logfile             Path to file to log to. (-)
  -x                     Use HTTP/1.1 instead of HTTP/2. Useful with broken
                         or limited builds of libcurl (false).
  -v                     Increase logging verbosity. (INFO)

Alternative protocols

The DoH standard is still evolving. Because responses are translated into JSON, there is room for error in encoding and parsing response types - particularly the less common ones.
For this reason, I tend to believe DNS-over-TLS is a better long-term strategy for the industry, but proxy clients aren't yet readily available.
Note that fundamental differences (binary vs JSON encoding) mean this software does not and will not support DNS-over-TLS.
---------
Google Public DNS offers DNSSEC-validating resolution over an encrypted HTTPS.

Thanks

Docker

TODO

  • Currently only Block DNS TYPE:ANY
  • More thorough tests should be written
  • No caching is implemented, and probably never will

Usage

$ docker pull chenhw2/https-dns

$ docker run -d \
    -e "ARGS=-d 8.8.8.8 --edns 0.0.0.0/0" \
    -p "5300:5300/udp" \
    -p "5300:5300/tcp" \
    chenhw2/https-dns

Help

$ docker run --rm chenhw2/https-dns -h
NAME:
   https-dns - A DNS-protocol proxy for Google's DNS-over-HTTPS service.

USAGE:
   https-dns [global options] command [command options] [arguments...]

VERSION:
   MISSING build version [git hash]

COMMANDS:
     help, h  Shows a list of commands or help for one command

GLOBAL OPTIONS:
   --listen value, -l value           Serve address (default: ":5300")
   --proxy value, -p value            Proxy (SOCKS or SHADOWSOCKS) Server for HTTP GET
   --endpoint value                   Google DNS-over-HTTPS endpoint url (default: "https://dns.google.com/resolve")
   --endpoint-ips value, --eip value  IPs of the Google DNS-over-HTTPS endpoint; if provided, endpoint lookup skip
   --dns-servers value, -d value      DNS Servers used to look up the endpoint; system default is used if absent.
   --edns value, -e value             Extension mechanisms for DNS (EDNS) is parameters of the Domain Name System (DNS) protocol.
   --no-pad, -N                       Disable padding of Google DNS-over-HTTPS requests to identical length
   --insecure, -I                     Disable SSL/TLS Certificate check (for some OS without ca-certificates)
   --udp, -U                          Listen on UDP
   --tcp, -T                          Listen on TCP
   -V value                           log level for V logs (default: 2)
   --logtostderr                      log to standard error instead of files
   --help, -h                         show help
   --version, -v                      print the version

 from https://github.com/honwen/https-dns
我的补充说明:

搭建智能的DNS服务器:https-dns

众所周知,DNS 污染是较常遭遇的攻击手段之一。
ChinaDNS 为代表,目前自动解决这个问题的思路是多 DNS 对比:同时查询较快的本地 DNS(通常为 ISP 的 DNS)和较慢的可信 DNS(通常为经过加密传输的 Google 或 OpenDNS 等),对比返回的结果,若有差异,说明本地 DNS 返回的结果大概率被污染。 以此为基础,配合国内网站和国内 CDN 的白名单(直接使用本地 DNS)、被封锁网站的黑名单(直接使用可信 DNS)基本上达到了兼顾功能(反污染)和效率(国内跳过)的效果。 这个思路的缺点也很明显: - 如果使用白名单,截至今天(2018.01.26),白名单 accelerated-domains.china.conf 已经包含了 39881 条记录,这对很多路由器上的 dnsmasq 造成了不小的压力。而实际上,在这么一份大而全、更新飞快的列表中,大部分人使用的只是很小一部分。 - 如果不使用白名单,仅使用黑名单,对于非重度网络使用者,其实是一个可以接受的方案。 - 如果不使用白名单也不使用黑名单,结果更糟糕。这种情况下一般会启用类似 GEO IP: CN => DIRECT; FINAL => PROXY 的自动规则,由于目前绝大部分大中型网站均在多个区域部署 CDN 进行加速,对这些网站查询可信 DNS 有非常大的概率会返回一个对本地来说负优化的结果(为远端 VPS 或者代理优化,解析到一个对本地较慢的 CDN 节点),甚至网站会根据 GEO IP 的结果强制跳转(如淘宝海外站)。 - 路由器版本的 ChinaDNS 有时会出一些莫名其妙的问题导致不能正常解析,非常影响心情。

改进思路

为了弥补这些缺点,我们提出以下需求: - 只查询一个本地 DNS,或者,一个快的远端 可信 DNS(考虑到大部分地区到 114.114.114.114119.29.29.29 的延迟,可以认为 30-45 左右的延迟是可以接受的)。 - 不使用庞大的白名单和黑名单。 - 确保解析到的结果为本地优化。
针对后两点,技术上其实已经有了解决方案,那就是 RFC 7871 (Client Subnet in DNS Queries, aka edns-client-subnet, ECS),RFC 文档见此,还可参考 Google 的帮助。 ECS 允许 DNS 解析的请求放附带一个网络地址,要求 DNS 服务器做出针对这个地址优化的解析响应。 但是,ECS 目前的实施还是非常不接地气的。国内大厂多有成熟的智能解析方案,国外大厂更由于隐私等诸多问题对此动力不足。即便是目前对 ECS 支持的最好的 Google Public DNS,发过去的请求包也只有一半可以得到正确的 ECS 响应。 因此 Google 提供了一种迂回的解决方案:DNS-over-HTTPS文档)。不使用不能稳定得到 ECS 响应的 DNS 协议,通过 HTTPS 协议可以稳定获取 ECS 响应。 我们可以从这个方案中得到一个新思路,将 DNS 请求转化为 HTTPS 请求,再将收到的响应转化为 DNS 响应返回(事实上会小幅度增加解析耗时)。

部署

实现这个思路有两种部署方案:

本地部署

下载https://github.com/honwen/https-dns/releases(Go 语言,支持包括 ARM 在内的多种CPU),参照作者的说明安装在本地路由器或其他设备上为局域网提供服务,通过前置的代理(支持 socks 或影梭)访问 Google 的 DNS over HTTPS。

远端部署

推荐以容器的形式部署。 远端部署的优势是可以共享自建的DNS。 这种方式将 google-https-dns 作为后端提供服务,同时在前端放置一个支持 ECS 缓存的 DNS 代理(推荐使用Unbound)以获得更高性能。 部署方式如下,也可以参考 这份 Gist: 1. 创建 google-https-dns 的容器:docker run -d --name dns-google --log-opt max-size=1m --restart=unless-stopped tarot13/google-https-dns 2. 准备 Unbound 的配置文件 unbound.conf(见后文)和 Root Hints root.hints(可以从 ftp://FTP.INTERNIC.NET/domain/named.cache 下载) 3. 创建 Unbound 的容器:docker run -d --name dns-unbound -v $HOME/unbound:/etc/unbound -p 53:53/tcp -p 53:53/udp --link=dns-google:dns-google --log-opt max-size=1m --restart=unless-stopped tarot13/unbound
供参考的 unbound.conf 配置: 需要注意的是其中的两项: 1. ECS 缓存(subnetcache)必须在模块配置中启用:module-config: "subnetcache iterator" 2. 最好指定允许发送 ECS 信息的上游 DNS 网段(即 google-https-dns 的地址):send-client-subnet: 172.16.0.0/12
server:
  username: "root"
  interface: 0.0.0.0
  verbosity: 1
  do-daemonize: no
  access-control: 0.0.0.0/0 allow
  root-hints: "/etc/unbound/root.hints" # Root Hints: ftp://FTP.INTERNIC.NET/domain/named.cache
  auto-trust-anchor-file: "/etc/unbound/root.key" # Auto generated
  do-ip4: yes
  do-ip6: no
  do-udp: yes
  do-tcp: yes
  hide-identity: yes
  hide-version: yes
  harden-glue: yes
  use-caps-for-id: yes
  cache-max-ttl: 3600
  prefetch: yes
  num-threads: 4
  msg-cache-size: 64m
  rrset-cache-size: 128m
  module-config: "subnetcache iterator"
  unwanted-reply-threshold: 10000000
  do-not-query-localhost: no
  send-client-subnet: 172.16.0.0/12
  minimal-responses: yes

  forward-zone:
    name: "."
    forward-host: dns-google

使用

对于本地部署,局域网内的其他设备可以直接使用。 对于远端部署,可以选择通过非 53 端口转发,或者通过任意方法加密传输。
测试记录:
  • dig with subnet (web), mainland IPs.
-------------------------------------------------------
 dnss
dnss is a daemon for using DNS over HTTPS.
It can act as a proxy (the most common use case)

Features

  • Supports the JSON-based protocol as implemented by dns.google.com (reference).
  • Supports the DNS Queries over HTTPS (DoH) proposed standard (and implemented by Cloudflare's 1.1.1.1).
  • Local cache (optional).
  • HTTP(s) proxy support, autodetected from the environment.
  • Monitoring HTTP server, with exported variables and tracing to help debugging.
  • Separate resolution for specific domains, useful for home networks with local DNS servers.

Install

Debian/Ubuntu

The dnss package installs the daemon configured in proxy mode and ready to use, using Google's public resolvers (and easily changed via configuration).
sudo apt install dnss

Manual install

To download and build the binary:
go install blitiri.com.ar/go/dnss
And if you want to configure the daemon to be automatically run by systemd:
# Copy the binary to a system-wide location.
sudo cp "$GOPATH/bin/dnss" /usr/local/bin/

# Set it up in systemd.
sudo cp "$GOPATH"/src/blitiri.com.ar/go/dnss/etc/systemd/dns-to-https/* \
    /etc/systemd/system/

sudo systemctl dnss enable

Listens on port 53 for DNS queries, resolves them using the given HTTPS URL.
# Use the default HTTPS URL (currently, dns.google.com):
dnss -enable_dns_to_https

# Use Cloudflare's 1.1.1.1:
dnss -enable_dns_to_https -https_upstream="https://1.1.1.1/dns-query"

# Use Google's dns.google.com:
dnss -enable_dns_to_https -https_upstream="https://dns.google.com/resolve"
Supports both DoH and JSON modes automatically, and the endpoints are /dns-query and /resolve.
from https://blitiri.com.ar/git/r/dnss/b/next/t/f=README.md.html
(DNS over HTTPS 

dnss

dnss is a daemon for using DNS over HTTPS.
It can act as a proxy (the most common use case), and as a server (in case you want end to end control).

Features

  • Supports the JSON-based protocol as implemented by dns.google.com (reference).
  • Supports the DNS Queries over HTTPS (DoH) proposed standard (and implemented by Cloudflare's 1.1.1.1).
  • Local cache (optional).
  • HTTP(s) proxy support, autodetected from the environment.
  • Monitoring HTTP server, with exported variables and tracing to help debugging.
  • Separate resolution for specific domains, useful for home networks with local DNS servers.

Install

Debian/Ubuntu

The dnss package installs the daemon configured in proxy mode and ready to use, using Google's public resolvers (and easily changed via configuration).
sudo apt install dnss

Manual install

To download and build the binary:
go install blitiri.com.ar/go/dnss
And if you want to configure the daemon to be automatically run by systemd:
# Copy the binary to a system-wide location.
sudo cp "$GOPATH/bin/dnss" /usr/local/bin/

# Set it up in systemd.
sudo cp "$GOPATH"/src/blitiri.com.ar/go/dnss/etc/systemd/dns-to-https/* \
 /etc/systemd/system/

sudo systemctl dnss enable

DNS proxy mode on client machine:

Listens on port 53 for DNS queries, resolves them using the given HTTPS URL:
# Use the default HTTPS URL (currently, dns.google.com):
dnss -enable_dns_to_https

# Use Cloudflare's 1.1.1.1:
dnss -enable_dns_to_https -https_upstream="https://1.1.1.1/dns-query"

# Use Google's dns.google.com:
dnss -enable_dns_to_https -https_upstream="https://dns.google.com/resolvess"
sudo ~/go1.10.2/go/bin/dnss -enable_dns_to_https -https_upstream="https://mydomain.com/dns-query" -dns_listen_addr :53 -force_mode DoH
from https://github.com/albertito/dnss)
Alberto Bertogli's dnss, a daemon written in Go which act as a proxy (the most common use-case).
-----------------------------------------
cd $GOPATH
go get -u -v github.com/rameezk/secure-dns
cd src/github.com/rameezk/secure-dns/
go build
会在当前目录下,生成可执行文件:secure-dns。
sudo ./secure-dns -listen_addr :53 -https_upstream https://mydomain.com/dns-query
注意:此处的https://mydomain.com/dns-query必须是
https://github.com/m13253/dns-over-https/搭建的doh server.
项目地址:https://github.com/rameezk/secure-dns
------------------------------------------------------------------

A proof of concept DNS-Over-HTTPS proxy implementing https://datatracker.ietf.org/doc/draft-ietf-doh-dns-over-https

A set of python 3 scripts that supports proxying DNS over HTTPS as specified in the IETF Draft draft-ietf-doh-dns-over-https.
DOH provides a way to run encrypted DNS over HTTPS, a protocol which can freely traverse firewalls when other encrypted mechanism may be blocked.
DOH Proxy was created during IETF Hackathon 100 as a proof-of-concept.
-------
A DNS-protocol proxy for DNS-over-HTTPS providers, such as Google and Cloudflare.


Build Status 
A DNS-protocol proxy for DNS-over-HTTPS: allows you to run a server on your local network which responds to DNS queries, but requests records across the internet using HTTPS.
It's known to work with the following providers:
  • Google - Well tested and configured by default
  • Cloudflare (Beta) - May be used by passing the --cloudflare flag
  • Quad9 (Beta) - May be used by passing the `--quad9' flag
If you're interested in a more roll-your-own-DNS system, you might look at dnoxy, a sibling project to secureoperator which allows running your own DNS-over-HTTPS servers.

Installation

You may retrieve binaries from the releases page, or install using go get: 
cd $GOPATH
go get -u -v github.com/fardog/secureoperator/cmd/secure-operator
(THEN,THE executable file secure-operator will appear in $GOBIN/)
This will start a DNS server listening on TCP and UDP at :53. For usage information, run secure-operator --help.
Note: Running a service on port 53 requires administrative privileges on most systems.
(sudo secure-operator -listen :53 -google )

sudo ~/go1.10.2/go/bin/secure-operator -listen :53 -endpoint https://mydomain.com/dns-query -endpoint-ips vps-ip (必须加上“-endpoint-ips vps-ip”.这个程序不错)

Docker

There is a Docker image available for secureoperator:
docker pull fardog/secureoperator
The latest tag will always be the build from the master branch. If you wish to use one of the stable releases, use its version tag when pulling, e.g.:
docker pull fardog/secureoperator:4  # latest of major version
docker pull fardog/secureoperator:4.0  # latest of minor version
docker pull fardog/secureoperator:4.0.1  # exact version

Version Compatibility

This package follows semver for its tagged releases. The master branch is always considered stable, but may break API compatibility. If you require API stability, either use the tagged releases or mirror on gopkg.in:
go get -u gopkg.in/fardog/secureoperator.v4

Caching

secureoperator does not perform any caching; each request to it causes a matching request to the upstream DNS-over-HTTPS server to be made. It's recommended that you place secureoperator behind a caching DNS server such asdnsmasq on your local network.
An simple example setup is described on the wiki. Please feel free to contribute additional setups if you are running secureoperator in your environment.

Security

Note that while DNS requests are made over HTTPS, this does not imply "secure"; consider the following:
  • You must trust the upstream provider with your requests; for your chosen provider, see:
  • The lookup for the HTTP endpoint must happen in some regard, although how this is handled is up to you:
    • The system DNS resolver is used to look up the endpoint (default)
    • You provide a list of DNS servers to use for the endpoint lookup
    • You provide the IP address(es) to the endpoint; and no unencrypted DNS lookup will be performed. However if the addresses change while the service is running, you will need to restart the service to provide new addresses.
Information on the usage of these options is available with secure-operator --help.

Help Wanted

secureoperator could be greatly enhanced by community contributions! The following areas could use work:
  • More thorough unit tests
  • Installable packages for your favorite Linux distributions
  • Documentation on deploying secureoperator to a local network

Known Issues

Cloudflare is not fully tested yet; it should work for common cases, however:
  • EDNS is not supported; this is an intentional choice by Cloudflare, which means any EDNS setting you provide when using Cloudflare as a provider will be silently ignored.
For a production environment, the Google provider (default) is your best option today. If you're brave, please test Cloudflare and report any issues!
----------------------------------
cd  $GOPATH
go get -u -v github.com/fardog/dnoxy/cmd/dnoxy-dns/
(THEN,THE executable file dnoxy-dns will appear in $GOBIN/)
sudo dnoxy-dns -listen :53 -endpoint https://mydomain.com/dns-query -tcp true -udp true 
(在运行dnoxy-dns命令之前,需先添加一行:my-vps-ip mydomain.com到/etc/hosts文件的底部。)
--------------------------------
cd  $GOPATH
go get -u -v github.com/markkurossi/vpn
sudo vpn -doh https://mydomain.com/dns-query
------------------------------------------
cd  $GOPATH
go get -u -v github.com/hpidcock/nz

sudo nz -listen 127.0.0.1:53 -upstream https://mydomain.com/dns-query
项目地址:github.com/hpidcock/nz
------
npm install -g dohnut ,得到可执行文件~/node-v12.6.0/node-v12.6.0-darwin-x64/bin/dohnut
dohnut --help

sudo dohnut --doh https://somedomain.com/dns-query --listen :53 --datagram-protocol udp4
(在运行上面的命令之前,需先添加一行:my-vps-ip mydomain.com到/etc/hosts文件的底部。)
项目地址:https://github.com/commonshost/dohnut
--------------------------------------
go-doh-proxy
在本地机器mac上安装go环境,然后,
cd $GOPATH
go get -u -v github.com/aaronriekenberg/go-doh-proxy
wget https://github.com/aaronriekenberg/go-doh-proxy/raw/master/config/config.json -O go-doh-proxy-config.json
nano go-doh-proxy-config.json
把port的值改为53 ,把第48行改为"https://mydomain.com/dns-query" ,这个doh server不能
使用https://github.com/m13253/dns-over-https/项目里的doh server端。
这个doh server要使用rust-doh,详见https://briteming.blogspot.com/2019/12/https-dns-by-honwendns.html
里所说的“doh server程序rust-doh的搭建说明“。

sudo go-doh-proxy go-doh-proxy-config.json
项目地址:github.com/aaronriekenberg/go-doh-proxy
----------------------------------------------------------------------
godoh - A DNS-over-HTTPS C2 
(godoh不好用,完全用不了)

A DNS-over-HTTPS Command & Control Proof of Concept

@leonjza Go Report Card

introduction

godoh is a proof of concept Command and Control framework, written in Golang, that uses DNS-over-HTTPS as a transport medium. Currently supported providers include Google, Cloudflare but also contains the ability to use traditional DNS.

installation

All you would need are the godoh binaries themselves. Binaries are available for download from the releases page as part of tagged releases.
To build godoh from source, follow the following steps:
  • Ensure you have dep installed (go get -u -v github.com/golang/dep/cmd/dep)
  • Clone this repository to your $GOPATH's src/ directory so that it is in sensepost/godoh
  • Run dep ensure to resolve dependencies
  • Run make key to generate a unique encryption key to use for communication
  • Use the go build tool, or run make to build the binaries in the build/ directory

usage

$ godoh -h
A DNS (over-HTTPS) C2
    Version: dev
    By @leonjza from @sensepost

Usage:
  godoh [command]

Usage:
  godoh [command]

Available Commands:
  agent       Connect as an Agent to the DoH C2
  c2          Starts the godoh C2 server
  help        Help about any command
  receive     Receive a file via DoH
  send        Send a file via DoH
  test        Test DNS communications

Flags:
  -d, --domain string     DNS Domain to use. (ie: example.com)
  -h, --help              help for godoh
  -p, --provider string   Preferred DNS provider to use. [possible: google, cloudflare, raw] (default "google")

Use "godoh [command] --help" for more information about a command.
from https://github.com/sensepost/goDoH
----------------------------------
This is the repo for draft-ietf-doh-dns-over-https, which is a contribution to the IETF discussion of how to serve DNS over HTTPS. This is part of the IETF's DOH Working Group; see https://datatracker.ietf.org/wg/doh/about/ for information about the Working Group and how to subscribe to the mailing list for discussion of this draft.
from https://github.com/dohwg/draft-ietf-doh-dns-over-https
-----------------------------------------------

a network tool for people in PRC

dns-over-https

file /dns.php
just Google DNS Mirror
parameter should base64 encode and urlencode and base64 decode the response
name=urlencode(base64encode("img.alicdn.com")) edns_client_subnet=urlencode(base64encode("123.125.18.108"))
Demo https://prudent-travels.000webhostapp.com/dns.php?name=&edns_client_subnet=

Pac File

file /pac.php
Build by mono_pac
use proxy to foreign site and Direct to PRC site.
Demo
https://prudent-travels.000webhostapp.com/pac.php?a=SOCKS5&b=192.168.1.1:1080
generate a pac file which use SOCKS5 192.168.1.1 proxy, copy and parse to SwitchyOmega
from https://github.com/lbp0200/prc-net-tool
--------------------------------------
Dima Krasner's nss-tls, a daemon that makes gethostbyname(), getaddrinfo(), etc' happen through DoH, without any change to applications, thus transparently migrating all applications that don't use their own resolver (like some browsers) from DNS to DoH.(用命令:brew install libsoup来安装dependency: libsoup失败,遂放弃此项目)
Maxime Elomari's dealdoh, a middleware to proxy DoH requests to different DNS upstreams, written in PHP.(要求php为7.x版)
https://github.com/danielinux/dohd,
我试过,但遇错:https://github.com/danielinux/dohd/issues/1
-----------------------------------------

dohproxy

DNS over HTTPS proxy written in golang
I got interested in DNS over HTTPS after Firefox started supporting it in its latest release. I looked around to understand how it worked. Most of the implementations were too complex and did a lot of things. I read the RFC[1] and realised it was very trivial. So I tried my hand at implementing a proxy. This is just a proof of concept.
To install it you can use:
go get -u -v github.com/satran/dohproxy
This assumes you have installed go.
To run it use:
dohproxy
This will start the proxy on 5353 port.
You can resolve addresses using:
dig @127.0.0.1 -p 5353 redhat.com

Running it as a docker container

If you would like to run it as a docker container run:
docker run -it --rm -p 53:53/udp satran/dohproxy
This will run the proxy on localhost. You can update your /etc/resolv.conf file with nameserver 127.0.0.1 to resolve all dns queries using the dohproxy.
sudo ~/go1.10.2/go/bin/dohproxy -host localhost -port 53 -dohserver https://mydomain.com/dns-query
运行此程序时,有"no such host"问题,解决办法:
修改hosts文件,把mydomain.com指向你的vps的ip,具体就是:
modify the /etc/hosts file,add the following line to the bottom of the hosts file: my-vps-ip mydomain.com
---------------------------------------

dohproxy

A DNS-over-Https proxy and router
GoDoc Go Report Card

Installation

Download from the latest release page,
or install from source
$ go get -u github.com/major1201/dohproxy (未编译成功.只好下载其
可执行文件)
or run with Docker
docker run -d \
    --name dohproxy \
    -v /etc/dohproxy:/etc/dohproxy \
    -p 5353:53/udp \
    major1201/dohproxy

Usage

Start dohproxy with config /etc/dohproxy.yml
dohproxy
Start dohproxy with a custom config path
dohproxy -c /home/major1201/my-doh-config.yml
Service
# install as a service
dohproxy -c /home/major1201/my-doh-config.yml --service install

# start the service
dohproxy --service start

# stop the service
dohproxy --service stop

# uninstall the service
dohproxy --service uninstall

Configuration

log:
  stdout: stdout                  # default: stdout, log-to-file on Windows is not supported
  stderr: /var/log/dohproxy.err   # default: stderr, log-to-file on Windows is not supported
  level: info                     # default: debug, choices: debug, info, warn(warning), error, dpanic, panic, fatal

listen:
  - type: udp
    address: 127.0.0.1:53
  - type: tcp
    address: 127.0.0.1:53

upstreams:
  google-public:
    type: dns
    address: 8.8.8.8:53
  my-corp-dns:
    type: dns
    address: 192.168.53.1:53
  doh-get-with-proxy:
    type: doh-get
    address: https://some-doh-server-i-cant.com/dns-query
    proxy: socks5://127.0.0.1:1080
  doh-post:
    type: doh-post
    address: https://cloudflare-dns.com/dns-query

rules:
  - fqdn:cloudflare-dns.com      google-public
  - fqdn:www.my-dev-server.com   10.0.31.1
  - keyword:mycorp.com           my-corp-dns
  - suffix:mybiz.com             my-corp-dns
  - suffix:never-response.com    blackhole
  - suffix:adxxx.com             reject
  - wildcard:*                   doh-post
listen types:
  • udp
  • tcp
upstream types:
  • dns: classic DNS server
  • doh / doh-get: DNS-over-HTTPS protocol, using HTTP GET method
  • doh-post: DNS-over-HTTPS protocol, using HTTP POST method
rule format: [fqdn|prefix|suffix|keyword|wildcard|regex]:expression upstream|blackhole|reject|static_ip
  • upstream: upstream name defined in the upstreams field
  • blackhole: it never response to any dns requests, it just does nothing
  • reject: returns error immediately

Known Issues

  • The log.stdout and log.stderr part in config file only support stdout on Windows platform, due to zap package limit.
我的补充说明:
wget https://github.com/major1201/dohproxy/releases/download/v0.2.0/dohproxy-darwin_amd64-0.2.0.tar.gz
mkdir dohproxy-darwin_amd64-0.2.0-by-major1201
mv dohproxy-darwin_amd64-0.2.0.tar.gz dohproxy-darwin_amd64-0.2.0-by-major1201
cd dohproxy-darwin_amd64-0.2.0-by-major1201
tar xvf dohproxy-darwin_amd64-0.2.0.tar.gz
./dohproxy-darwin_amd64-0.2.0 -h
显示:
...
--config value, -c value   set config file (default: "/etc/dohproxy.yml")
...
wget https://github.com/major1201/dohproxy/raw/master/example-config.yml -O  dohproxy-by-major1201-config.yml 
nano dohproxy-by-major1201-config.yml 
cat dohproxy-by-major1201-config.yml 
---
log:                              # optional
  stdout: stdout                  # default: stdout, log-to-file on Windows is not supported
  stderr: /var/log/dohproxy.err   # default: stderr, log-to-file on Windows is not supported
  level: info                     # default: debug, choices: debug, info, warn(warning), error, dpanic, panic, fatal
listen:
  - type: udp
    address: 127.0.0.1:53
  - type: tcp
    address: 127.0.0.1:53
upstreams:
 xyz-public:
   type: dns
   address: 2.2.2.2:5356
  my-corp-dns:
    type: dns
    address: 192.168.53.1:53
  doh-get-with-proxy:
    type: doh-get
    address: https://dns.google.com/resolve
    proxy: socks5://127.0.0.1:1080
  doh-post:
    type: doh-post
    address: https://mydomain.com/dns-query
rules:
  - fqdn:cloudflare-dns.com      google-public
  - fqdn:www.my-dev-server.com   10.0.31.1
  - keyword:mycorp.com           my-corp-dns
  - suffix:mybiz.com             my-corp-dns
  - suffix:never-response.com    blackhole
  - suffix:adxxx.com             reject
  - wildcard:*                   doh-post
...
然后,
sudo ./dohproxy-darwin_amd64-0.2.0 -c dohproxy-by-major1201-config.yml
------------
在mac上,先安装go环境。然后,
cd $GOPATH

go get -u -v github.com/markdingo/trustydns/cmd/trustydns-proxy/
可执行文件trustydns-proxy就会生成在$GOBIN里面.
trustydns-proxy -h
(帮助文件的内容很多!!)
sudo trustydns-proxy -A :53 https://mydomain.com/dns-query
----------------------------------------------------
在mac上,先安装go环境。然后,
cd $GOPATH
go get -u -v github.com/tpetr/dns-https
sudo dns-https -l 127.0.0.1:53 https://mydomain.com/dns-query
项目地址:https://github.com/tpetr/dns-https
----------------------------------------
在mac上,先安装rust环境。然后,
git clone https://github.com/mudge/homer homer-by-mudge
cd homer-by-mudge
cargo build --release
cd target/release/
sudo ./homer --bind 127.0.0.1:53 --upstream https://mydomain.com/dns-query
-----------------------------
在mac上,先安装go环境。然后,
cd $GOPATH

go get -u -v github.com/adnsio/dotd/cmd/dotd/
sudo dotd  -address :53 -upstream https://mydomain.com/dns-query
---------------
在本地机器mac上。先安装python3,然后,
git clone https://github.com/radixcl/pydoh pydoh-by-radixcl
cd pydoh-by-radixcl
sudo pip3 install -r requirements.txt
nano config.json
cat config.json
{
    "listen_port": 53,
    "listen_address": "0.0.0.0",
    "runas_user": "nobody",
    "runas_group": "nogroup",
    "doh_urls": [
        "https://mydomain.com/dns-query",
        "https://some-public-domain.com/dns-query"
    ]
}
然后,
sudo python3 pydoh.py
项目地址:https://github.com/radixcl/pydoh
------

doh-forwarder

在本地机器mac上。先安装python3,然后,
sudo pip3 install aiohttp 
sudo pip3 install uvloop
sudo pip3 install cchardet
wget https://github.com/kpadron/doh-forwarder/raw/master/doh-forwarder.py

sudo python3 doh-forwarder.py -l 127.0.0.1 -p 53 --tcp --upstream https://mydmain.com/dns-query
This program is a basic attempt at creating a DNS over HTTPS inline-proxy forwarder. This means that it accepts standard UDP or TCP DNS packets and converts them to DoH HTTP requests. Queries made by this program are encrypted using TLS schemes defined in the python standard library ssl. The program can be configured with command line options to support a listening address and any non-standard ports.
This program is meant to be single-threaded and is based on the python standard library asyncio. Asynchronous HTTP requests are made over encrypted connections to upstream servers via required library aiohttp. This allows for extra performance when many requests are received at once. If TCP resolving is enabled extra threads may be spawned to accept connections on the listening socket. Please note that this program was originally configured for operation with Cloudflare's public DNS servers and as such may contain specifics to that resolver.
doh-forwarder.py is the main program for this project and as such will have the most features implemented. Other scripts in this repository represent different approaches to the same problem.
在mac上,先安装go环境。然后,
cd $GOPATH
go get -u -v github.com/karlbateman/saber
sudo saber -host 127.0.0.1 -port 53 -resolver https://mydomain.com/dns-query
项目地址:https://github.com/karlbateman/saber
--------
在mac上,先安装go环境。然后,
cd $GOPATH
go get -u -v github.com/iamd3vil/danse
danse -h
会显示:
Usage of danse:
  -addr string
    Address to bind (default "127.0.0.1")
  -cache
    DNS response caching
  -port string
    Port for DNS server (default "53")
  -url string
     URLs for DoH resolvers seperated by comma (default "https://cloudflare-dns.com/dns-query")
其实danse只支持doh server. 对dot server支持得并不好。
sudo danse -addr 127.0.0.1 -port 53 -url https://mydomain.com/dns-query 
---------
在mac上,先安装go环境。然后,
cd $GOPATH

go get -u -v github.com/voidpirate/proxydoh/
sudo proxydoh -host 0.0.0.0 -port 53 -dohserver https://mydomain.com/dns-query
项目地址:github.com/voidpirate/proxydoh
-------
在mac上,先安装python3环境。然后,
git clone https://github.com/wanglihe/forwardoh forwardoh-by-wanglihe
cd forwardoh-by-wanglihe
sudo pip3 install gevent
./src/forwardoh -h
显示:
...
FileNotFoundError: [Errno 2] No such file or directory: '/etc/forwardoh.conf'
sudo nano /etc/forwardoh.conf
按此页面:https://github.com/wanglihe/forwardoh/blob/master/config/forwardoh.conf 的内容,创建/etc/forwardoh.conf文件,
其内容为:
port = 53
#url = "https://cloudflare-dns.com/dns-query"
#url = "https://mozilla.cloudflare-dns.com/dns-query"
#url = "https://1.1.1.1/dns-query"
#url = "https://1.0.0.1/dns-query"
url = "https://mydomain.com/dns-query"
cache_max = 1000
cache_refresh = 300
status_interval = 10
然后,
sudo ./src/forwardoh
---------

利用dnsproxy-by-AdguardTeam解决dns污染问题

Simple DNS proxy with DoH, DoT, and DNSCrypt support.
参见:https://briteming.blogspot.com/2019/09/dnsproxy-by-adguardteamdns.html
------------------
在mac上,先安装nodejs环境。然后,
git clone https://github.com/hardillb/dns-over-https dns-over-https-by-hardillb
cd dns-over-https-by-hardillb 
npm install
nano dns-to-https-proxy.js
(修改第8行的url为https://mydomain.com/dns-query)

sudo node dns-to-https-proxy.js
项目地址:https://github.com/hardillb/dns-over-https
----------
在mac上,先安装go环境。然后,
cd $GOPATH
go get -u -v github.com/rollcat/gdoh
sudo gdoh -listen :53
不过,默认的上游dns server为https://1.1.1.1/,不一定能连上。
项目地址:https://github.com/rollcat/gdoh
--------
在mac上,先安装go环境。然后,
cd $GOPATH
go get -u -v github.com/gurparit/fastdns
sudo fastdns
会显示:
DNS1: 1.1.1.1
DNS2: 1.0.0.1
Strategy: https
项目地址:github.com/gurparit/fastdns
-------
在mac上,先安装python3环境。然后,
sudo pip3 install named1
sudo python3 -m named1
会显示:
...
[Serve53] listening on ('0.0.0.0', 53)
[Serve53] listening on ('::', 53)
[cloudflare] 1.0.0.1 connected, cert cloudflare-dns.com
[google] 8.8.4.4 connected, cert dns.google
[cloudflare] 1.1.1.1 connected, cert cloudflare-dns.com
[google] 8.8.8.8 connected, cert dns.google
-------------------------------------------------------------
-----------------------------------------------------

DNS over HTTPS

DNS over HTTPS (DoH) is a protocol for performing remote Domain Name System (DNS) resolution via the HTTPS protocol. A goal of the method is to increase user privacy and security by preventing eavesdropping and manipulation of DNS data by man-in-the-middle attacks[1] by using the HTTPS protocol to encrypt the data between the DoH client and the DoH-based DNS resolver. Encryption by itself does not protect privacy, encryption is simply a method to obfuscate the data. As of March 2018, Google and the Mozilla Foundation started testing versions of DNS over HTTPS.[2][3]
In addition to improving security, another goal of DNS over HTTPS is to improve performance: testing of ISP DNS resolvers has shown that many often have slow response times, a problem that is exacerbated by the need to potentially have to resolve many hostnames when loading a single web page.[1]

Technical detail[edit]

DoH is a proposed standard, published as RFC 8484 (October 2018) by the IETF. It uses HTTP/2 and HTTPS, and supports the wire format DNS response data, as returned in existing UDP responses, in an HTTPS payload with the MIME type application/dns-message.[1][4] If HTTP/2 is used, the server may also use HTTP/2 server push to send values that it anticipates the client may find useful in advance.[5]
DoH is a work in progress. Even though the IETF has published RFC 8484 as a proposed standard and companies are experimenting with it,[6][7] the IETF has yet to determine how it should best be implemented. The IETF is evaluating a number of approaches for how to best deploy DoH and is looking to stand up a working group, Applications Doing DNS (ADD), to do this work and develop a consensus. In addition, other industry working groups such as the Encrypted DNS Deployment Initiative, have been formed to "define and adopt DNS encryption technologies in a manner that ensures the continued high performance, resiliency, stability and security of the Internet’s critical namespace and name resolution services, as well as ensuring the continued unimpaired functionality of security protections, parental controls, and other services that depend upon the DNS".[8]
Many issues with how to properly deploy DoH are still being resolved by the internet community including but not limited to:
  • Parental controls and content filters
  • Split DNS in Enterprises
  • CDN Localization
  • Interoperability with 5G networks

Deployment scenarios[edit]

DoH is used for recursive DNS resolution by DNS resolvers. Resolvers (DoH clients) must have access to a DoH server hosting a query endpoint.[5]
DoH lacks native support in operating systems. Thus, a user wishing to use it must install additional software. Three usage scenarios are common:
  • Using a DoH implementation within an application: Some browsers have a built-in DoH implementation and can thus perform queries by bypassing the operating system's DNS functionality. A drawback is that an application may not inform the user if it skips DoH querying, either by misconfiguration or lack of support for DoH.
  • Installing a DoH proxy on the name server in the local network: In this scenario client systems continue to use traditional (port 53 or 853) DNS to query the name server in the local network, which will then gather the necessary replies via DoH by reaching DoH-servers in the Internet. This method is transparent to the end user.
  • Installing a DoH proxy on a local system: In this scenario, operating systems are configured to query a locally running DoH proxy. In contrast to the previously mentioned method, the proxy needs to be installed on each system wishing to use DoH, which might require a lot of effort in larger environments.
  • Installing a DoH resolving plugin for the operating system
In all of these scenarios, the DoH client does not directly query any authoritative name servers. Instead, the client relies on the DoH server using traditional (port 53 or 853) queries to finally reach authoritative servers. Thus DoH does not qualify as an end-to-end encrypted protocol, only hop-to-hop encrypted and only if DNS over TLS is used consistently.

Public DNS servers using DoH[edit]

DNS over HTTPS server implementations are already available free of charge by some public DNS providers[9]. See public recursive name server for an overview.

Client support[edit]

  • AdGuard for Android,[10] AdGuard for iOS[11] and AdGuard Home[12]
  • Cloudflare 1.1.1.1 client app for Android and iOS.[13]
  • Cloudflare resolver for Linux, MacOS and Windows.[14]
  • cURL since 7.62.0.[15]
  • DNSCrypt-proxy — Local DNS → DNS over HTTPS proxy.[16]
  • DNSP — Versatile DNSProxy. DoH server (C) and client (PHP) implementation.[17]
  • doh-php-client — PHP Implementation.[18]
  • Firefox since Version 62 and later[19] — Browser support.[20]
  • go-doh-proxy — Go DoH Proxy Server.[21]
  • Intra — Android app by Jigsaw.[22]
  • nss-tls — a DoH-based resolver plugin for glibc.[23]
  • Technitium DNS Client — C# .NET cross-platform implementation.[24]
  • NextDNS client apps.[25]
  • Nebulo - DNS over HTTPS/TLS - for Android.[26]

Operating system support[edit]

In November 2019, Microsoft announced plans to implement support for encrypted DNS protocols in Microsoft Windows, beginning with DoH.[27]

Criticism[edit]

It has been argued that DoH provides a false sense of security, as it only encrypts information that could still be obtained via non-encrypted portions of HTTPS requests, such as IP addresses and Server Name Indication.[28][29] In addition, DoH implementations in web browsers currently rely on third-party DNS providers, which is contrary to the decentralized nature of DNS and may have privacy implications.[29] OpenBSD has disabled DoH by default in their builds of Firefox due to use of Cloudflare services for this feature.[30] Chrome will use DoH only if the user's chosen DNS provider is known to support it, although it did face accusations by U.S. ISPs that it was using the implementation to force users onto its Google Public DNS service.[31][29][32]
DoH can impede analysis and monitoring of DNS traffic for cybersecurity purposes; the 2019 DDoS worm Godula used DoH to mask connections to its command-and-control server [33][29] It is argued that DoH could bypass content-control software and enterprise DNS policies.[29]
The Internet Watch Foundation and the Internet Service Providers Association (ISPA)—a trade association representing UK ISPs—criticized Mozilla, developers of the widely-used Firefox Web browser, and Google—for supporting DoH, as they believe that it will undermine web blocking programs in the country, including ISP default filtering of adult content, and mandatory court-ordered filtering of copyright violations. The ISPA nominated Mozilla for its "Internet Villain" award for 2019 (alongside the EU Directive on Copyright in the Digital Single Market, and Donald Trump), "for their proposed approach to introduce DNS-over-HTTPS in such a way as to bypass UK filtering obligations and parental controls, undermining internet safety standards in the UK." Mozilla responded to the allegations by the ISPA, arguing that it would not prevent filtering, and that they were "surprised and disappointed that an industry association for ISPs decided to misrepresent an improvement to decades-old internet infrastructure".[34][35] In response to the criticism, the ISPA apologized and withdrew the nomination.[36][37] Mozilla subsequently stated that DoH will not be used by default in the UK market until further discussion with relevant stakeholders, but stated that it "would offer real security benefits to UK citizens".[38]

See also[edit]

References[edit]

  1. Jump up to:a b c Chirgwin, Richard (14 Dec 2017). "IETF protects privacy and helps net neutrality with DNS over HTTPS"The Register. Retrieved 2018-03-21.
  2. ^ "DNS-over-HTTPS | Public DNS | Google Developers"Google Developers. Retrieved 2018-03-21.
  3. ^ Cimpanu, Catalin (2018-03-20). "Mozilla Is Testing "DNS over HTTPS" Support in Firefox"BleepingComputer. Retrieved 2018-03-21.
  4. ^ Hoffman, P; McManus, P. "RFC 8484 - DNS Queries over HTTPS"datatracker.ietf.org. Retrieved 2018-05-20.
  5. Jump up to:a b Hoffman, P; McManus, P. "draft-ietf-doh-dns-over-https-08 - DNS Queries over HTTPS"datatracker.ietf.org. Retrieved 2018-05-20.
  6. ^ "Experimenting with same-provider DNS-over-HTTPS upgrade"Chromium Blog. Retrieved 2019-09-13.
  7. ^ Deckelmann, Selena. "What's next in making Encrypted DNS-over-HTTPS the Default"Future Releases. Retrieved 2019-09-13.
  8. ^ "About"Encrypted DNS Deployment Initiative. Retrieved 2019-09-13.
  9. ^ "DNS over HTTPS Implementations". 2018-04-27. Retrieved 2018-04-27.
  10. ^ Brinkmann, Martin (2019-03-21). "AdGuard 3.0 for Android: Redesign, Stealth Mode, Custom Filter Lists". Ghacks Technology News. Retrieved 2019-08-02.
  11. ^ Orr, Andrew (2019-07-13). "AdGuard 3 Brings DNS Privacy, 250,000 Filter Rules, Premium Features". The Mac Observer, Inc. Retrieved 2019-08-02.
  12. ^ Davenport, Corbin (2018-12-29). "AdGuard officially releases its own DNS service, and it works with Android Pie"Android Police. Illogical Robot LLC. Retrieved 2019-08-01.
  13. ^ Cimpanu, Catalin. "Cloudflare launches Android and iOS apps for its 1.1.1.1 service"ZDNet. Retrieved 2018-12-13.
  14. ^ "DNS over HTTPS"Argo Tunnel. Cloudflare. Retrieved 20 July 2019.
  15. ^ "DoH in curl".
  16. ^ "DNSCrypt-proxy v2.0". 2019-08-05.
  17. ^ "DNSP". 2019-07-22.
  18. ^ "DNS over HTTPS PHP Client". 2019-08-03.
  19. ^ "Trusted Recursive Resolver"Mozilla. 15 September 2019. Archived from the original (html) on 12 September 2019. Retrieved 15 September 2019All preferences for the DNS-over-HTTPS functionality in Firefox are located under the `network.trr` prefix (TRR == Trusted Recursive Resolver). The support for these were added in Firefox 62.
  20. ^ "Improving DNS Privacy in Firefox".
  21. ^ "Go DoH Proxy Server".
  22. ^ "Intra on Play Store".
  23. ^ "GitHub - dimkr/NSS-TLS: A DNS over HTTPS resolver for glibc". 2019-08-02.
  24. ^ "DNS over HTTPS C# Client". 2019-07-18.
  25. ^ "nextdns"www.nextdns.io. Retrieved 2019-07-13.
  26. ^ "Nebulo - DNS over HTTPS/TLS - Apps on Google Play".
  27. ^ Gallagher, Sean (2019-11-19). "Microsoft says yes to future encrypted DNS requests in Windows"Ars Technica. Retrieved 2019-11-20.
  28. ^ "A Controversial Plan to Encrypt More of the Internet"WiredISSN 1059-1028. Retrieved 2019-11-19.
  29. Jump up to:a b c d e Cimpanu, Catalin. "DNS-over-HTTPS causes more problems than it solves, experts say"ZDNet. Retrieved 2019-11-19.
  30. ^ "Google Unveils DNS-over-HTTPS (DoH) Plan, Mozilla's Faces Criticism"BleepingComputer. Retrieved 2019-09-14.
  31. ^ Tung, Liam. "DNS over HTTPS: Google hits back at 'misinformation and confusion' over its plans"ZDNet. Retrieved 2019-11-19.
  32. ^ Lee, Timothy B. (2019-09-30). "Why big ISPs aren't happy about Google's plans for encrypted DNS"Ars Technica. Retrieved 2019-11-19.
  33. ^ Cimpanu, Catalin. "First-ever malware strain spotted abusing new DoH (DNS over HTTPS) protocol"ZDNet. Retrieved 2019-11-19.
  34. ^ Cimpanu, Catalin. "UK ISP group names Mozilla 'Internet Villain' for supporting 'DNS-over-HTTPS'"ZDNet. Retrieved 2019-07-05.
  35. ^ "Internet group brands Mozilla 'internet villain' for supporting DNS privacy feature"TechCrunch. Retrieved 2019-07-19.
  36. ^ "British ISPs fight to make the web LESS secure"IT PRO. Retrieved 2019-09-14.
  37. ^ Patrawala, Fatema (2019-07-11). "ISPA nominated Mozilla in the "Internet Villain" category for DNS over HTTPs push, withdrew nominations and category after community backlash"Packt Hub. Retrieved 2019-09-14.
  38. ^ Hern, Alex (2019-09-24). "Firefox: 'no UK plans' to make encrypted browser tool its default"The GuardianISSN 0261-3077. Retrieved 2019-09-29.

External links[edit]

----------------------------------------------------------

DNS-over-HTTPS杂谈

DoH 基于 RFC 8484,如果要支持它,DNS服务器(称为DoH Server)和DNS客户端(称为DoH Client,比如Chrome,Curl,Dig)分别要做那些调整呢?
先上一张图,看看DoH单元之间工作原理:
从图中可以看出 DoH Client 应用程序要自行配置DoH Server(用户手动操作),这和传统DNS运作模式很不一样,各个应用程序也会缓存独立的DNS缓存。
DoH Server以URL的方式提供服务,比如Google的DoH Server地址是https://dns.google/dns-query,CloudFlare的DoH Server地址是https://cloudflare-dns.com/dns-query。
对于一个DoH Server来说,他为了查询出主机名的IP地址,它也会递归查询其他传统DNS Server(当然安全性比Client查询安全的多),也就是传统DNS Server和DoH Server是并存的。
对于一个DNS Server来说,为了支持DoH,改造并不大,只是对原生DNS消息格式进行处理,然后以HTTPS消息格式进行网络传输,其实复杂度并不高。
而对于一个DoH Client来说,考虑的内容多了很多,比如还要校验证书;如果DoH Server超时了,如何处理;如何设置配置选,以便让用户配置DoH Server,一系列复杂问题让我们怀疑他的可行性。
对于我们普通用户来说,还要自行配置DoH Server,这就更麻烦了,有多少人懂DNS呢?也许未来会做成完全自动化配置。
里面还有一个非常有意思的事情,不知道大家有没有发现,DoH Server也是以域名主机方式提供服务的,那么这个主机如何解析出IP地址呢?因为此时DoH Clinet还不知道DoH Server IP是什么,简单的解决方式是用传统DNS方式解析出DoH Server的IP,或者让用户自行设置DoH Server的IP。
目前Chrome,Firefox,Curl应用程序都支持了DoH Client,这个将来有机会再说,简单以Curl命令行的方式演示:
$ curl --doh-url https://dns.google.com https://www.simplehttps.com
写到现在还有很多疑问?为什么采用HTTPS协议承载DNS消息?有什么好处和坏处?为什么不采用TLS协议承载DNS消息?ISP会支持DoH吗?浏览器如何支持?会全面替代传统DNS吗?
上面介绍了DoH机制的一些原理,本文说下我对它的一些疑问,主要观点来源于 Curl 的作者 Daniel Stenberg,他发博文的频率非常高,曾经也是 Mozilla 的员工,如果想要了解 HTTP&Web 相关方面的知识,他的博客是必须订阅的。
第一个问题就是DoH为什么要使用HTTPS协议传输,而不是使用TLS协议(或其他协议)传输?
这一方面说明了HTTP(HTTPS)协议的通用性,HTTP协议虽然感觉上性能不佳(其实这属于旧观点了),但通用性非常好,只要是互联网应用,都可以轻松使用它传输消息,而且考虑到DoH是浏览器厂商力推的,Chrome和Firefox如果要发送DoH查询请求,那是非常的方便。
如果使用TLS协议承载DoH,那么DoH Clinet的改造就比较大了,没有HTTPS协议处理那么方便,比如要自行校验证书。
并且HTTPS协议的443端口一般不会被block(屏蔽),而如果使用其他端口传递DoH,很有可能被路由器、代理服务器过滤掉。
再比如你要搭建一个DoH Server(递归DNS),简单使用HTTPS Proxy就可以了,根本无需了解DNS知识,非常的方便。
我第一次听说DoH的时候,第一个考虑的问题是性能问题,传统DNS基于UDP,只要一个RTT就能响应,而HTTPS协议可并不是这样的。
其实HTTP/2协议可以多路复用,连接可以复用,你在浏览器访问十个不同的网站,Chrome只要和DoH Server建立一次连接;甚至基于Server Push功能,DoH Server能够主动给你发送更新,所以从这个角度看,性能问题不用太担心,但实际情况可能没有那么乐观。
传统DNS服务器是使用IP地址访问的,而DoH使用主机域名访问,能够进行负载均衡,通过CDN技术,让你访问最近的DoH Service。
第二个问题,对DoH基于HTTPS协议的一些质疑,传统DNS协议是非常轻量的一个应用层协议,而DoH感觉是对DNS协议的一个改造,并且还要使用HTTPS协议传输消息,总感觉怪怪的。
DoH更像一个集中式的服务,而DNS本身是阶层式的服务,DoH Service对外是一个递归服务器,内部还是采用迭代式的传统DNS查询方式,如果全面推广DoH,难道迭代式的查询就没了?比如你查询一个DNS解析,首先向Root DNS查询,它如何响应?告诉你下一个DNS的主机名(用于你迭代解析),由于这一块我还没深入了解,目前是我提出的一个疑问。
采用DoH,需要用户(或应用程序)去单独配置DoH Service(不是全局配置),还要考虑soft fail问题,使用起来非常麻烦。
采用DoH,由于是HTTPS协议,也很容易被跟踪,比如Cookie,带来新的一系列安全性问题,同时DoH Service能够了解到更多用户的行为,会不会爆发出新的罪恶之源?
采用DoH,是基于Geo获取用户的地址,如果获取不准,会导致DNS解析出的结果不一样,比如你在北京,权威服务器解析出的主机地址可能是A,如果Geo定位错误,权威服务器根据你的IP地址可能会解析到另外一个主机IP。
传统DNS排查非常容易,而如果采用DoH,出现问题如何排查?到底是DNS服务器的问题,还是HTTPS传输的问题?HTTPS消息是加密的,怎么知道具体问题是什么?
第三个问题,DoH会替代传统DNS吗?
我觉得不会,而且很有可能这个项目会失败,至少不会有大量的推广应用,本来很简单的一个DNS查询,因为有HTTPS协议的存在会衍生出新的问题。
对于用户来说,DoH配置那么复杂,有多少人会操作呢?任何一项技术,如果让用户难以使用,那么很难推广。
另外DoH还是要和传统DNS配合使用,比如第一次需要解析DoH Service的主机名,所以还难全面替换,从设计上来看,为了安全,把本来很简单的解决方案搞复杂了。
最重要的是DoH打破了原有DNS的工作机制,现在的PC,手机上的程序都基于系统的lib查询DNS,而如果改为DoH,就要自行处理DNS查询,且不说复杂度,这个过渡时间没十年根本不可能,所以没有动力去改造。
大家想想HTTPS的推广目前做的不错,原因在于Chrome等浏览器的推广,Web网站不改造,我就告诉用户你的网站不安全,逼得你要去改,同时Web改造工作量也不大(仅仅是http到https的替换)。可DoH,Chrome等浏览器没有大杀器逼着用户去配置使用DoH,所以目前的服务对象只能是我们这样的技术人员。
再考虑ISP愿意去改造吗?Windows这样的操作系统会愿意改造吗(让你全局配置DoH Service),且不说主观意愿,复杂度也高了很多,尤其对于ISP,可能还会破坏他们的既得利益。
再比如Root DNS,权威DNS会改造吗?你如何说服他们?如果他们也改造了,那传统DNS消息格式可能就不存在了。
所以目前DoH只能是浏览器厂商的一个尝试性项目,解决访问网站的安全性问题(且还存在很多问题),很难全面替换,即使要部分替换也要花上数十年,但技术是不断提高的,未来可能会有新的、更好的解决方案,让我们拭目以待。
前两篇文章讲解了DoH的原理和优缺点,本文说下如何让浏览器启用DoH,从firefox 60版本后就可以体验DoH特性了,也是目前唯一支持DoH的浏览器,不过需要手动启用
首先在地址栏中输入 about:config,然后搜索 network.trr,会列出一系列的DoH参数,见下图:
重要的参数就三个,network.trr.mode 有好几个选项:
  • 0,这是默认值,相当于5,表示不启用DoH。
  • 1,启用DoH,但它同时也会使用传统DNS查询,返回响应快的结果。
  • 2,启用DoH,如果不可用的时候,会启用传统DNS查询,相当于一个备份。
  • 3,只启用DoH,这是最安全的方式,如果DoH不可用,那么你就无法访问网站了。
network.trr.uri 参数也很重要,表示选择DoH Server,比较著名的就是Google(https://dns.google/dns-query)和 Cloudflare(https://cloudflare-dns.com/dns-query)的。
不过悲催的是,这二个DoH Server的443端口目前是无法访问的。 
换言之在特定区域,DoH功能我们是享用不了的,从这个角度看,DoH想推广是困难重重。
network.trr.bootstrapAddress 这个参数也很重要,在firefox启动的时候,dns.google 和 cloudflare-dns.com 这两个 DoH Server 也需要进行 DNS 查询,如果实在要查询,就只能使用传统DNS查询了,为了避免潜在的不安全性,可以手动输入 DoH Server 的 IP 地址,Cloudflare 的是 1.1.1.1,Google 的是 8.8.8.8。
配置后,如何知道你访问网站使用DoH呢?在地址栏上输入 about:networking#dns,
如果 TRR 的值是 True,表示启用DoH了。
----------------------------------
技术上其实已经有了解决方案,那就是 RFC 7871 (Client Subnet in DNS Queries, aka edns-client-subnet, ECS),RFC 文档见此,还可参考 Google 的帮助 ECS 允许 DNS 解析的请求放附带一个网络地址,要求 DNS 服务器做出针对这个地址优化的解析响应。 但是,ECS 目前的实施还是非常不接地气的。国内大厂多有成熟的智能解析方案,国外大厂更由于隐私等诸多问题对此动力不足。即便是目前对 ECS 支持的最好的 Google Public DNS,发过去的请求包也只有一半可以得到正确的 ECS 响应。 因此 Google 提供了一种迂回的解决方案:DNS-over-HTTPS文档)。不使用不能稳定得到 ECS 响应的 DNS 协议,通过 HTTPS 协议可以稳定获取 ECS 响应。 我们可以从这个方案中得到一个新思路,将 DNS 请求转化为 HTTPS 请求,再将收到的响应转化为 DNS 响应返回(事实上会小幅度增加解析耗时)。
------------------
https://hacks.mozilla.org/2018/05/a-cartoon-intro-to-dns-over-https/
------------------


'DNS over HTTPS' server的客户端程cloudflared


概述

由于中国大陆内存在 DNS 缓存投毒,通过常规方式使用 DNS 会导致某些域名不能被正常解析。DNSSEC 曾经是被寄予厚望来解决 DNS 缓存投毒的方案, 然而由于大部分域名都未设置 DNSSEC,不能直接在 DNS 客户端使用强制 DNSSEC 校验并对校验失败的结果丢弃,因此 DNSSEC 在对抗 DNS 污染时并没有起到预想中的效果。反而,在执行 DNSSEC 查询时,即使是未被屏蔽的域名也可能遭受污染。
既然使用标准 DNS 协议会被污染,那最简单的方案就是在公网内使用非常规的 DNS 查询方法,并且把 DNS 查询内容加密,自然也就规避了 ISP 的 DNS 劫持和防火墙的 DNS 缓存投毒。
目前主流的方法主要是 DNS over HTTPS(DoH)和 DNS over TLS(DoT),本文档介绍第一种方案。
支持 DNS over HTTPS 的 DNS 程序非常多,然而大部分主要用于在核心路由器上甚至公网上提供 DNS 服务,如果你只是简单地在局域网环境使用,则使用 Cloudflare 提供的 Cloudflared 程序就可以轻松完成部署。

Windows

系统环境

在此文章撰写时...
Windows 10 Pro for Workstations / 1804. Build 17134.1
首先需要下载 Cloudflared 二进制文件
解压缩后放到任意位置,如我们放到 I 盘的 Cloudflare 文件夹中。
在左下角的 Windows 徽标按钮上右键单击,选择「Windows PowerShell(管理员)」,并在弹出的 UAC 授权提示框(如果有)中允许 PowerShell 使用管理员权限。
定位到 I:\Cloudflare 目录。
以 DNS 代理模式运行 Cloudflared
这时我们已经成功通过 Cloudflared 隧道程序以 HTTPS 方式连接到了 Cloudflare 提供的 DNS。
由于 Cloudflared 自身以 CLI(Command Lines)方式运行时不会以 Daemon 方式运行,我们需要通过 PowerShell 的功能来让它在后台运行。
输入命令(你需要自行修改命令中的路径):
powershell -windowstyle hidden -command "I:\Cloudflare\cloudflared.exe proxy-dns"
某些安全软件可能会拦截以后台静默运行的程序,请允许 Cloudflared.exe 和 PowerShell 这么做
可以看到 PowerShell 窗口一闪而过,我们通过 nslookup 测试一下。
测试可以了解到,Cloudflared 仍然在正常运行,并提供着 DNS 代理服务。现在,只需要修改网络适配器中的 DNS 服务器地址为 127.0.0.1 就可以使用了。
将这个 PowerShell 设置为开机运行,就可以实现开机启动 Cloudflared 并运行 DNS 代理服务。
Windows 也可以将 Cloudflared 设置为系统服务来开机运行 Cloudflared,请参考官方文档

macOS

系统环境

在此文章撰写时...
macOS 10.13.5 Beta / Build 17F45c
macOS 系统可以通过包管理器或二进制安装 Cloudflared。
要安装 Homebrew(包管理器),你可以参考 Homebrew 的官方网站或直接在终端中输入:
/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
已经安装 Homebrew 后,在终端中输入:
brew install cloudflare/cloudflare/cloudflared
也可以直接使用二进制文件安装:
curl https://bin.equinox.io/c/VdrWdbjqyF/cloudflared-stable-darwin-amd64.tgz | tar xzC /usr/local/bin

安装后直接在终端输入 sudo cloudflared proxy-dns 即可运行 Cloudflared 的 DNS 代理功能。
Cloudflared 以 DNS 代理模式运行时需要监听 53 端口,因此必须提权到 Root 用户才能运行。
要在后台运行 Cloudflared,官方方法是将 Cloudflared 设置为服务。
要设置 Cloudflared 为服务,你必须拥有一个 Cloudflare 账户并在至少一个域中购买了 Argo 增值服务。
因此,详细的设置方法请参考官方文档
macOS 也可以自行编写脚本来开机运行 Cloudflared,但是由于 macOS 是类 UNIX 系统且权限控制较为严格,此文档没有足够篇幅和必要介绍此种方法。

Linux

系统环境

在此文章撰写时...
Rad Hat Enterprise Linux 7.4 Ubuntu 18.04 LTS
Linux 发行版可以通过包管理器安装或二进制安装。

二进制文件可以直接安装到 /usr/bin 等目录,安装后,输入 sudo cloudflared proxy-dns 就可以使用,可以部署到路由器上供网关下的设备使用。
Cloudflared 以 DNS 代理模式运行时需要监听 53 端口,因此必须提权到 Root 用户才能运行。如果已经是 Root 用户,直接运行即可。
可以在 screen 中运行 Cloudflared 以变相达成后台运行,或者自行编写 SystemInitV / Systemd 脚本来开机运行。
from  https://rixcloudkb.io/kb/dns-over-https/
-----------------------------------------------

相关帖子:
类似项目:gdns-go,https://briteming.blogspot.com/2019/08/dns-proxygdns-go.html 
http://briteming.blogspot.com/2018/10/4dnssecdnscryptdns-over-tlsdns-over.html
https://briteming.blogspot.com/2019/07/tutorial-to-setup-dns-over-tls-dot.html
--------
Minimal and efficient DNS-over-HTTPS (DoH) client

https-dns

https-dns is a minimal and efficient DNS-over-HTTPS (DoH) client. DNS-over-HTTPS (RFC 8484) is a protocol for performing DNS resolution through the HTTPS protocol that prevents manipulation of DNS response. https-dns forwards DNS queries from the client to upstream DoH servers, caches the response, and sends the response back to the client.

Crates.io Crates.io GitHub Actions

Installation

cargo install https-dns

Usage

# udp://localhost:53 -> https://1.1.1.1 (default)
sudo https-dns

# udp://localhost:53 -> https://cloudflare-dns.com
sudo https-dns --upstream-address cloudflare-dns.com

# udp://localhost:10053 -> https://dns.google
sudo https-dns --local-port 10053 --upstream-address dns.google

CLI Reference

$ https-dns --help

https-dns 0.2.0
Minimal and efficient DNS-over-HTTPS (DoH) client

USAGE:
    https-dns [OPTIONS]

OPTIONS:
    -h, --help                                   Print help information
        --local-address <LOCAL_ADDRESS>          [default: 127.0.0.1]
        --local-port <LOCAL_PORT>                [default: 53]
        --upstream-address <UPSTREAM_ADDRESS>    [default: 1.1.1.1]
        --upstream-port <UPSTREAM_PORT>          [default: 443]
    -V, --version 
from  https://github.com/xiaoyang-sde/https-dns 
---

公共DoH

 

国内服务商

1、阿里公共 DNS

DoT 地址:dns.alidns.com 或 223.5.5.5

DoH 地址:
https://dns.alidns.com/dns-query

2、DnsPod 腾讯 公共 DNS

DoT 地址:dns.pub 或者 119.29.29.29

DoH 地址:

https://dns.pub/dns-query

3、360DNS

DoT 地址:dot.360.cn

DoH 地址:https://doh.360.cn/dns-query

4、中国下一代互联网公共 DNS.

DoT 地址:dns.cfiec.net

DoH 地址:
https://dns.cfiec.net/dns-query

4、红鱼 dns

DoT 地址:rubyfish.cn
DoH 地址:https://rubyfish.cn/dns-query

国外服务商

1、Cloudflare 公共 DNS

知名云服务商 Cloudflare 提供的解析服务器。

DoT 地址:cloudflare-dns.com 或 1.1.1.1

DoH 地址:

https://1.1.1.1/dns-query

2、Google 公共 DNS

DoT 地址:dns.google 或 8.8.8.8

DoH 地址:
https://dns.google/dns-query

3、DNS.SB

DoT 地址:dns.sb 或 185.222.222.222

DoH 地址:

https://doh.dns.sb/dns-query

4、AdGuard DNS

DoT 地址:dns.adguard.com

DoH 地址:
https://dns.adguard.com/dns-query

5、Quad9 IBM 公共 DNS

DoT 地址:dns.quad9.net 或 9.9.9.9

DoH 地址:
https://dns.quad9.net/dns-query

浏览器中使用 DNS-over-HTTPS 的方法

Chrome

  1. 在设置里选择【隐私设置和安全性】下点击【安全】
  2. 在高级部分中,启用【使用安全 DNS】
  3. 选择【自定义】,输入 https://doh.pub/dns-query

Firefox

  1. 在选项里找到【网络设置】部分,然后单击【设置】
  2. 选中【启用基于 HTTPS 的 DNS】
  3. 选择【自定义】,输入 https://doh.pub/dns-query 单击确定

Edge

  1. 在设置里找到【隐私、搜索和服务】部分
  2. 在【安全性】下,启用 "使用安全的 DNS 指定如何查找网站的网络地址"
  3. 选择【请选择服务提供商】,输入 https://doh.pub/dns-query