During my first engagement at Foundstone I tested a web filtering software and we found several ways of bypassing it. With the recent news around SOPA and the controversy around Indian government willing to pre-screen Internet, I thought this might be the right time to write a blog on it. In this blog I have mentioned a few such techniques that can be used to circumvent web filtering software. Although a lot of this information is available on the internet (scattered), I have tried to put together a one-stop list of tried and tested techniques with detailed examples.
The discussion in this blog is limited to tricks and techniques that can be used on the web, and it does not focus on software that can be used for circumvention such as Application tunneling software, Re-routing systems, peer-to-peer software etc. The reason is most of the time users trying to circumvent corporate web filtering do not have administrator privileges to install these tools. If you already have administrator access to your machine then there are easier ways to bypass circumvention than the techniques described below. Some of the widely used tools include Peacefire, TOR, JAP ANON etc. You can find a thorough list of such tools here:

You can also obtain information on what its Censorship and its History here. I will not be covering that in this blog. Circumvention may be considered illegal in some companies/countries and if caught it may result in you getting fired or jailed thus, this blog is only for informational purpose and not an invitation to attempt this.

URL Canonicalization

In this technique we will use alternate domain names and alternative representations of the domain names to bypass filtering. For the purpose of this discussion let’s assume that access to www.facebook.com is blocked and our end goal is to access it from behind the web filtering tool. Depending on how intelligent the filtering tool is one or more of the below mentioned techniques should work. During our assessment we found that some of these techniques worked on the top filtering brands.

Alternative Domains

Instead of http://www.facebook.com you can try the following.

https://www.facebook.com/ (Sometimes HTTPs version of the site is not blocked)
http://Facebook.com
http://Facebook.com/
http://Facebook.com.
http://Login.facebook.com
http://touch.facebook.com
http://M.facebook.com (Access the mobile version of Facebook instead)
http://Apps.facebook.com
http://Register.facebook.com

Alternate Representations

All the browsers do not equally accept the formats mentioned below. However, most of these work with the standard browsers. During our assessment we found that the URL and IP address of the website were blocked. However, when accessing the blocked website using the decimal and octal form it worked fine for us.

Try accessing Facebook using the IP address: http://66.220.147.44
Decimal format of the IP address 66.220.147.44 : http://1121751852
Hexadecimal format of the IP address: http://0xd42dc932c/
Dotted Hexadecimal form: http://0×42.0xdc.0×93.0x2c
Dotted Octal form: http://0102.0334.0223.0054

Below is the list of tools for converting the IP address into different forms:

Translation Services

Translation services such as www.microsofttranslator.com can also be used to circumvent filters. We can do this by translating the blocked website e.g. boingboing.net from English to English. When doing so translation services work similarly to the web proxy sites. During our assessment we found that filtering tools block some common translation sites such as Babelfish. However, we were able to find one that was not blocked (InterTran) and access the blocked website boingboing.net

Some translation services such as translate.google.com prevent translation into the same language, so they may not be very useful depending on your needs. However, there is also a solution to this problem. You can use Google translate to convert a website from English to French and then back to English using the Google chrome’s translation features. Babelfish lets you translate from English to English by selecting “French to English” or any such option that ends in “to English”. Since, there is nothing to translate it just returns back the content in English.

List of other free translation services:

Cached Pages and Mirror Sites

Search engines keep copies of the indexed pages known as Cached pages. Cached pages of the blocked sites can be accessed through special keywords in the search request. During our review we noticed that www.boingboing.net was blocked by the filtering tool. However we could access its cached version through Google cache by searching for “cache:www.boingboing.net”.

We also noticed that we were able to access it by appending “nyud.net:8090” at the end such as www.boingboing.net.nyud.net:8090. URL’s with nyud.net as the domain name are processed by the Coral Content Distribution Network (CCDN). “CCDN is a free peer-peer content distribution network, comprised of a world-wide network of web proxies and name servers.” (http://www.coralcdn.org/)

Moreover old archived pages of the blocked sites can be obtained from www.archive.org or www.bibalex.org.

Low-Bandwidth Simulators

Low-bandwidth simulators such as www.loband.org let us simulate website performance on low bandwidths. This can act as a wrapper allowing us access to the text only version of the blocked sites. This is a good option to access news, blogs, email and other sites, however not a good option for accessing sites with Flash and HTML5 such as YouTube.

Proxy Websites

Proxy website is the easiest and well known way of accessing banned websites. However, the issue is that web filtering systems block most known proxy websites and continue to update their list on a regular basis. But the good news is that proxy sites keep on growing at a faster rate than they are banned. Thus, you will always find some such sites that are not banned. You can obtain a list of regularly updated proxy servers here. You can also join a proxy mailing list like Circumventor. If you are using proxy servers to access banned sites such as Facebook, be cautious about the information you provide such as credentials. A lot of this information may be cached on the proxy servers and get leaked or misused by the admins.

Web to email

It is possible to obtain blocked web pages and search the web for information via email by sending simple commands [FTP] . Here is the list of some popular sites:

A complete list of all such sites and commands can be found here. To obtain a blocked web pages in email send an email to www@web2mail.com with the address of the website as the subject.

In addition to this, you can also subscribe to receive daily updates of a website in email. This is a good method for obtaining information from blocked news and blog sites. There are other websites that provide similar services such as www.changedetection.com and www.watchthatpage.com. Every time a web page changes an email alert is sent to the user.

Web to PDF

Similarly you can also convert web pages to PDFs and download them or email them without getting blocked. None of the web filtering software that we tested caught this.

Websites providing free web to PDF conversion services:

http://www.web2pdfconvert.com/
http://www.html-to-pdf.net/free-online-pdf-converter.aspx
http://pdfcrowd.com/
http://www.pdfonfly.com/
http://pici.picidae.net/(Provides website to screenshot conversion)

RSS Aggregators

RSS Aggregators are websites or desktop applications that let you subscribe to and read RSS (Really Simple Syndication) Feeds. During our review we found that the filtering software did not block access to RSS aggregators such as Google Reader, Bloglines and Fastladder etc. This makes it possible for the user to read blogs from the restricted web sites like BoingBoing by pointing such RSS readers to listen to the RSS feeds. However, this technique is limited to sites that publish RSS feeds.

Google Add-ons

During our assessment we noticed that it was possible to access some of the blocked sites via Google gadgets. Just add gadgets for the blocked sites such as Facebook, Twitter and YouTube etc. to iGoogle and access the blocked content. Some smart web filtering systems may prevent one or more such gadgets, but not all of them. Similarly you can also use Yahoo widgets.