Web proxy for searching on Google privately.
The story
In some areas of the earth, we can't have free internet (when we speak of free, we are referring to freedom, not price). Therefore people will need to find ways to access a relatively free internet. First we have HTTP proxy and VPN, but they are vulnerable and easy to censor. Then there is a great software called Shadowsocks, it's fast, secure and easy-to-setup. However, users have to install clients in order to use it, which is a bit hard on some platforms, or when you are travelling.
But web proxies don't have that restriction. Once you set it up on your server, you can access them with only a web browser, through HTTP or (better) HTTPS. When you request a web page, the server grab the things from internet it can access, and send back to you. Curly Turtle is a web proxy concentrated on searching on Google.
I'm not saying web proxies can replace tunnel proxies like Shadowsocks, I just say it's more convenient to do one thing. In Curly Turtle's case, it's searching on Google, but only Google, no more.
Installing
You only need to install it on a server, no client required.
First you need to have
nodejs version 5 or higher and npm version 3 or higher, see Node.js official download page for detials.
Then run the following command as root:
npm install curly-turtle -g
Ignore the warinings that says something replaces something. You can access the program by the command
curly-turtle. Run curly-turtle --help for help.Accessing
When you run
curly-turtle without any arguments, it serves a HTTP connection to all 0.0.0.0 on port 8081. However, because HTTP connection is plain, it's very easy for blacklist maintainers and 50 cents to know you are using Curly Turtle, don't ever attempt to use Curly Turtle through insecure HTTP connections even for testing purposes.
In order to use a secure HTTPS connection, you need to have an SSL private key and an SSL certificate. They can be generated by yourself (self-signed), or signed by a CA. See these instructions if you don't know how to do it.
Once you have got your key and certificate, use the
--sslkey option to specify the path of your private key and the --sslcert option to specify the path of your certificate. For example, if your path of key is /etc/ssl/domain.key and path of certificate is /etc/ssl/domain.crt, run curly-turtle --sslkey /etc/ssl/domain.key --sslcert /etc/ssl/domain.crt.
You can then access it with your web browser, type
https://your-server-ip:8081 in the address bar, for example, https://123.234.345.456:8081 if your server IP is 123.234.345.456.
If you use a self-signed certificate and your browser warns you that the site is insecure, don't worry, that's because your certificate is self-signed and not trusted by the browser. Just click whatever your browser gives you to proceed anyway (for Firefox is add exception). To skip that problem, add your certificate to your browser's trust store.
If everything goes right, you will see a friendly web page.
Disguising
You can specify a path with
--baseurl option, which if you know that path, you can access Curly Turtle. But if not, an Nginx-look 404 response will be returned, therefore people without the path can not access Curly Turtle on your server.
For example, if you add
--baseurl /google/IQXuB6IbPUg9ca4O to the command line, you can only access Curly Turtle with address https://you-server-ip:8081/google/IQXuB6IbPUg9ca4O/.Rate limiting
Google does not allow automatic fetching of their search results, and they will block IPs that send too much requests in a certain amount of time. Therefore your server running Curly Turtle may be blocked by Google if you search on it too frequently (you will get 502 or 503 errors). To deal with it, you can set a time, in miliseconds, that is the minimum interval between two searches. If your two searches' interval is below that limit, you will get "Too many requests" error. And when the time has passed, you can search again.
Use
--ratelimit to set the limit, for example, curly-turtle --ratelimit 20000 sets the rate limit to 20000 miliseconds (20 seconds).Configuration
You can also use a configuration file to set
address, port, baseurl, sslkey and sslcert. Pass -c or --config option with the path of the configuration file to use it.
The configuration file must be in YAML format. For example, create a cofiguration file in
/etc/curly-turtle/master.yml with the following content:address: 127.0.0.1
port: 8080
baseurl: /eqBT7AGJLpIu17s0
sslkey: /etc/ssl/domain.key
sslcert: /etc/ssl/domain.crt
ratelimit: 20000
Then run the command
curly-turtle -c /etc/curly-turtle/master.yml, it will listen on address 127.0.0.1, port 8080, use /eqBT7AGJLpIu17s0 as baseurl, load SSL private key in /etc/ssl/domain.key, load SSL certificate in /etc/ssl/domain.crt and set rate limit to 20000 miliseconds.
If one option is omitted, the default value will be used.
No comments:
Post a Comment