After hearing about exactly how much of my communication is secure
and private I decided to do something about it. These Posts will be more
or less a series of guides on how to increase your privacy online.
This is Part 1: Securing my router. My goals are:
-
Logging which sites I have visited
-
Reducing the amount of advertising I have to put up with (Privoxy)
-
Making it harder for others to know which sites I have visited (TOR)
-
Using software that is not trivial for crackers to exploit
(OpenWrt)
NOTE: Although this is a step-by-step guide, it does require you to
have at least some experience in command line Linux. Additionally I
cannot guarantee that it will work for you.
0. Get some hardware that you can modify
I won't describe this part in too much detail - basically any router
that can run OpenWRT should do. For bonus points (i.e. more logging) I
would definitely recommend a router with at least one USB Port. That way
it is trivial to add more storage space.
At the time of me writing this, the TP-Link TL-WDR3600 was a pretty good
deal, with Gigabit-LAN, USB and 5GHz wireless - so the post will deal
with this hardware. If you have another router, you will need to
substitute hardware specific links where necessary.
1. Install and setup OpenWRT on your router
For the TL-WDR3600 I just followed the instructions at here:
NOTE: If you want to install the latest version of OpenWRT
(barrier-breaker 14.07) I recommend simultaneously enabling
attitude-adjustment packages by adding the following line to
/etc/opkg.conf. (Just behind the similar line for barrier_breaker.
src/gz attitude_adjustment htp://downloads.openwrt.org/attitude_adjustment/12.09/ar71xx/generic/packages
For some reason not all packages are always available for
barrier-breaker. By also having the attitude-adjustment link in
opkf.conf opkg will fall back to older packages whenever this is the
case.
1b Connect to the web as a routed client (optional)
To fully install OpenWRT you will need an internet connection. If you
are somewhat 'paranoid' you might not want to directly connect to the
internet until OpenWRT has been fully set up. If that is the case you
will need to set it up as a routed client by reconfiguring the WAN port.
Another benefit is that the PAF (Partner Acceptance Factor) will rise if
the Internet is still available while you are configuring :-)
See here if you want to do this using wireless:
3 Monitor Traffic Levels
(important for QoS at some point)
If you want to keep stats across reboots you need to edit vnstat.conf to
not use the /var/* directory (as var is mapped to tmp)
4 Install & configure privoxy
By using privoxy we can speed up the web by blocking unnecessary ads and
simultaneously keep a log of visited sites. The logs will be a big help
later when we are trying to create white- and blacklists for tor.
NOTE: to properly analyze the log files later the log file format should
be the Common Format. This is done by adding the following line to
/etc/privoxy/config
------
debug 512
------
5 Install TOR
This step is only necessary if you want to obfuscate which sites you are
visiting. If everyone where to do this, the secret services of our
countries would once again be able to concentrate on their real jobs -
instead of trying to capture everyones metadata. If you want to use TOR
you will experience a slightly laggier and less reliable internet
connection. But even if you do not use Tor for your connections, please
consider setting up a relay so that those of us without a choice can at
least experience some sort of 'free' internet.
Run the following commands on the command line of your router.
----
ipkg update
ipkg install tor
/etc/init.d/tor enable
/etc/init.d/tor start
----
NOTE: If you want to run a tor relay and also use tor for your
connections you will need to run two separate tor processes. Else if you
set up a bandwidth limit you will no longer be able to surf the net once
the limit is reached. (Learned this the hard way...)
To do so, you need to copy the /etc/tor/torc file and edit it to only
run as relay. Then copy /etc/init.d/tor and modify it to use the second
torc file.
Finally enable your copy (I called it /etc/init.d/tor.relay)
6 Combine privoxy and tor
By first moving all traffic to privoxy and then to TOR we have two
advantages:
-
We can block most ads thus improving our online experience
-
We can analyze which websites use the most traffic - so that we can
whitelist those sites to not use tor
Aside: There is no point in shuffling ALL of your traffic through tor.
Sites like Facebook track what you are doing anyway and will report this
information to anyone requesting it. Streaming videos through tor is not
much fun, so it makes sense to whitelist youtube, etc.
Add the following lines to /etc/privoxy/config (Assuming your network is
on 192.168/16)
----
forward-socks5 / 127.0.0.1:9050 .
forward 192.168.*.*/ .
----
NOTE: The filter doesn't need to be finegrained as only non-whitelisted
traffic will be passed on to privoxy
NOTE2: This will only move http traffic from privoxy through tor, as
secure traffic cannot be read by it. (Else you would be using it as an
intercepting proxy - which is something you don't want)
7 Move non-privoxy (e.g. https) traffic directly to tor:
This guide will move all traffic through tor:
We only need the /etc/tor/torrc and the /etc/firewall.user part. Both
need to be modified to use the interface br-lan and the correct ip
address.
8 Whitelist traffic (from tor)
After first forcing everything through tor we now need to whitelist some
traffic to not pass through tor. The hardest part is getting a list of
sites we need to whitelist.
We will use the privoxy logs we enabled earlier to see which sites tend
to fail, and/or are called often. These sites we will then whitelist.
Sadly this is not trivial, as most websites these days tend to link and
stack quite deeply. Thus simply whitelisting a single IP for facebook
won't work.
To start we create an ipset with whitelisted sites and pass this ipset
to iptables. Frst install the packages:
ipset iptables-mod-ipset kmod-ipt-ipset
Then edit: /etc/config/firewall - add the lines before the firewall.user
include
----
config include
option path '/etc/firewall.whitelist
config include
option path '/etc/firewall.blacklist
-----
Create the file /etc/firewall.whitelist with the content
----
ipset create whitelist hash:ip -exist
ipset add whitelist 192.168.2.0/24
...add your sites here
----
And /etc/firewall.blacklist
----
ipset create blacklist hash:ip
...add your sites here
----
Modify /etc/firewall.user to use the "whitelist" and "blacklist" ipsets.
----
# Move DNS requests to tor
iptables -t nat -I PREROUTING -i br-lan -p udp --dport 53 -j REDIRECT --to-ports 9053
# Drop all blacklisted traffic
iptables -t raw -I PREROUTING -i br-lan -m set --match-set blacklist dst -j DROP
# Move any http traffic to privoxy
iptables -t nat -A PREROUTING -i br-lan -m set ! --match-set whitelist dst -p tcp --dport 80 --syn -j REDIRECT --to-port 8118
# Move remaining traffic straight to tor
iptables -t nat -A PREROUTING -i br-lan -m set ! --matchset whitelist dst -p tcp --syn -$
----
NOTE: this means that only those addresses that are NOT in your
whitelist will be passed to tor
NOTE2: To increase the PAF you can comment out the last line until the
whitelist is well defined. If you do this, then you will also need to
remove the last lines of /etc/privoxy/config which you added in step 6.
Finally, modify /etc/config/firewall to no longer redirect (as it is
does exactly the same thing as the 3rd line in firewall.user)
----
#config redirect
# option proto 'tcp'
# option target 'DNAT'
# option dest 'lan'
# option name 'intercepting-proxy for HTTP'
# option src 'lan'
# option dest_port '8118'
# option src_dport '80'
# option dest_ip '192.168.2.1'
# option src_dip '!192.168.2.1'
----
9 Creating sensible white/blacklists
See the section on parsing the privoxy logs below.
Any sites
you really do not want to visit - for example anything with 'ad' in the
URL tends to be a good start - can go in the blacklist.
9a Resolving hostnames
Sipmply adding hostnames to the ipset is sadly not very useful - and not
recommended.
-
Only the first IP address will be used
-
If the DNS Record changes the firewall will not care (it resolves only
once)
Thus we need to write a script that will resolve the hostnames that we
have gathered (somewhere...) to all possible IPs - and then run that
script periodically
(Edited from here as openwrt does not have host):
/usr/bin/rebuild_fw_lists.sh (needs to be created and chmodded)
----
#!/bin/ash
#NOTE: we could skip building the files /tmp/xxxxlist.ip, but this way it
# is easier to see what everything resolves to.
rm /tmp/whitelist.ip
rm /tmp/blacklist.ip
#build a list of IPs using nslookup and the hostnames in /root/hostnames.whitelist
for ipaddress in $(egrep -h -v -E "^#|^$" /root/hostnames.whitelist); do
nslookup $ipaddress | grep -oE '((1?[0-9][0-9]?|2[0-4][0-9]|25[0-5])\.){3}(1?[0-9][0-9]?|2[0-4][0-9]|25[0-5])' | grep -v 127.0.0.1 >> /tmp/whitelist.ip
done
#build a list of IPs using nslookup and the hostnames in /root/hostnames.blacklist
for ipaddress in $(egrep -h -v -E "^#|^$" /root/hostnames.blacklist); do
nslookup $ipaddress | grep -oE '((1?[0-9][0-9]?|2[0-4][0-9]|25[0-5])\.){3}(1?[0-9][0-9]?|2[0-4][0-9]|25[0-5])' | grep -v 127.0.0.1 >> /tmp/blacklist.ip
done
#shutdown the firewall so that we can rebuild the lists
/etc/init.d/firewall stop
#then delete any sets we have
/usr/sbin/ipset destroy whitelist
/usr/sbin/ipset destroy blacklist
#restart the firewall (and thus rebuild the lists)
/etc/init.d/firewall start
#now we can add our ips to the ipsets
for ipaddress in $(egrep -h -v -E "^#|^$" /tmp/whitelist.ip); do
ipset add whitelist $ipaddress
done
for ipaddress in $(egrep -h -v -E "^#|^$" /tmp/blacklist.ip); do
ipset add blacklist $ipaddress
done
----
We want to execute this file periodically so that any changes in
dns-records are reflected in our ipsets. Thus we use cron by adding the
following line to the crontab file /etc/crontabs/root
----
01 03 * * * /usr/bin/rebuild_fw_lists.sh
----
10 Additional logging
Openwrt does not log a lot. Also any logs that are created are not
rotated and stored in /tmp (and thus deleted at every reboot).
To get the system logs into a file, add the following line to the system
section of /etc/config/system
-----
option log_type 'file'
----
From here: (Needs a reboot to work)
To get logs that survive a reboot, we need to move /var/log away from
the tmpfs
(Only do this if you have the space available.) If you have a router
with an USB port - and have set it up as described in the links above
you should have plenty of space for logs.
I back up my logs to /usr/share/log_save
----
mkdir /usr/share/log_save
----
Edit the startup script /etc/init.d/boot New lines between mkdir -p
/var/log and mkdir -p /var/lock
----
cp -r /usr/share/log_save/* /var/log/
chown tor /var/log/tor -R
----
To get rotating logs we need to use logrotate. Install the package
logrotate (depends on libpopt apparently)
Either edit /etc/logrotate.conf directly or add files in
/etc/logrotate.d (I guess the latter is cleaner but maybe slightly
overkill) At the end of logrotate.conf add the files you want to rotate
----
/var/log/privoxy
/var/log/tor/notices.log
----
Finally edit crontab to rotate (and save) the logs
----
01 03 * * * /usr/sbin/logrotate /etc/logrotate.conf
*/5 * * * * /bin/cp -r /var/log/* /usr/share/log_save
----
12 Parsing the logs
After all this effort we have some privoxy log files formatted in CLF.
(See the section on privoxy above.)
To get some sensible data out of these we will use analog, but as analog
is designed for servers (and not for proxy log files), we will need to
reparse the privoxy log files.
-
We only need the request parts of the files
-
We want to reverse the source and who requested the file (after all we
are the client, and not the server)
To achieve this I created the following script: extract.sh
----
#!/bin/bash
gunzip -c log_save/privoxy.4.gz | grep "^192\.168\.2\.[0-9]* -" > privoxy.clf
gunzip -c log_save/privoxy.4.gz | grep "Request: " > privoxy.request
paste -d '\n' privoxy.request privoxy.clf | sed "N;s/^.*Request: \([^/]*\).*\n\([0-9.]*\) - - \(.*\)$/\1 - \2 \3/" > log_save/privoxy.log
gunzip -c log_save/privoxy.3.gz | grep "^192\.168\.2\.[0-9]* -" > privoxy.clf
gunzip -c log_save/privoxy.3.gz | grep "Request: " > privoxy.request
paste -d '\n' privoxy.request privoxy.clf | sed "N;s/^.*Request: \([^/]*\).*\n\([0-9.]*\) - - \(.*\)$/\1 - \2 \3/" >> log_save/privoxy.log
gunzip -c log_save/privoxy.2.gz | grep "^192\.168\.2\.[0-9]* -" > privoxy.clf
gunzip -c log_save/privoxy.2.gz | grep "Request: " > privoxy.request
paste -d '\n' privoxy.request privoxy.clf | sed "N;s/^.*Request: \([^/]*\).*\n\([0-9.]*\) - - \(.*\)$/\1 - \2 \3/" >> log_save/privoxy.log
gunzip -c log_save/privoxy.1.gz | grep "^192\.168\.2\.[0-9]* -" > privoxy.clf
gunzip -c log_save/privoxy.1.gz | grep "Request: " > privoxy.request
paste -d '\n' privoxy.request privoxy.clf | sed "N;s/^.*Request: \([^/]*\).*\n\([0-9.]*\) - - \(.*\)$/\1 - \2 \3/" >> log_save/privoxy.log
more log_save/privoxy | grep "^192\.168\.2\.[0-9]* -" > privoxy.clf
more log_save/privoxy | grep "Request: " > privoxy.request
paste -d '\n' privoxy.request privoxy.clf | sed "N;s/^.*Request: \([^/]*\).*\n\([0-9.]*\) - - \(.*\)$/\1 - \2 \3/" >> log_save/privoxy.log
rm privoxy.clf privoxy.request
----
This script expects 4 log files (for 4 weeks) in the folder log_save.
The end result will be a single file log_save/privoxy.log which we can
parse using analog (see http://www.analog.cx/)
After installing analog, we need the following configuration file:
analog.conf
----
OUTFILE privoxy_%y%M%D.html
HOSTNAME "phoenix Internet"
HOSTURL "phoenix-nas.ddns.net"
DOMCOLS RrPpBb
DOMSORTBY BYTES
SUBDOMSORTBY BYTES
SUBDOMAIN *.*.*.*.*
ORGCOLS RrPpBb
ORGSORTBY BYTES
SUBORG *.*.*
SUBORGSORTBY BYTES
DIRCOLS RrPpBb
DIRSORTBY BYTES
SUBDIR */*/*/*/*/*/
----
Then we can run analog with the following command:
----
analog +ganalog.conf log_save/privoxy.log
----
and finally open the resulting html file in your browser. Using the
domain report you can pick the domains which use the most data. In my
household these tend to be the typical cases: dropbox, microsoft update,
linux update, android store and video streaming sites.
Use these domains to blacklist or whitelist as wanted.
Note: Once you are somewhat confident that your whitelist is complete I
would recommend disabling the logging that privoxy does, as the logs
tend to take up quite some space.
Conclusion
If you managed to get to here, you should now have a router that:
-
No longer uses some default firmware by a manufacturer that quite
frankly won't provide updates until it is too late anyway. That
doesn't mean that OpenWRT is perfect, but it is easier to keep up to
date.
-
Filters out quite a bit of the advertising that makes some parts of
the web more or less unusable
-
Will block any traffic to domains you wish to avoid
-
Will obfuscate your browsing habits to the outside world by moving
(almost) everything through TOR
-
Enables those of us on the web that do not have access to an
uncensored internet to browse the full web
