Show by Label

Tuesday, April 12, 2016

How-To: Setup a Wireless AP with a VPN - Jessie



The actual goal is to circumvent geo-blocking to access in your country by providers like Netflix or commercial TV broadcasters like RTL and the BBC in Europe and to cast movies from a Tablet to a TV, wherever you are.

This how-to and tutorial uses Jessie-lite, the systemd concept and OpenVPN.

With the advent of Netflix recently clamping down on possibilities to view movies while you are in another country, I started to study how to circumvent this blockage of content I pay for. I’m a Netflix account holder, registered and paying in my home country. I spend several months each winter in a sunny place, and I only want to see the movies and TV-series that are available in my own country, with sub-titles in my own language. What this solution will not do, is to give you access to the Netflix content in another country. For example, if you want access to the content in the USA. I very strongly believe I’m not doing anything illegal with my solution. If you are in a similar situation, read on.

This tutorial is easy to follow, but is not for beginners!
You should have some experience in working with the Pi, running it head-less and with Raspbian or Debian in general. I’m not an expert, I just learned this by investigation and trying things out over many weeks, so I will not be able to help you with installation problems! You’re largely on your own. But remember, RTFM and Google is your friend.

I have written several posts on this forum and on my blog over the past weeks that described my fact finding and testing to get to a simple and effective setup – this post. 

If want to know more background or details, I suggest you read through the other posts on my blog. It has a lot of information.

The Challenge:
After many years of doing nothing, Netflix decided to clamp down on the use of VPN’s and Dynamic DNS services, to the point that by the end of last February, I could no longer get access to the content I pay for. At the time I was using a commercial VPN service that “tunneled” me from my present location to a local server in my home country. The provider however is leaving me high & dry with this geo-blocking issue so I needed to find another way.

The challenge VPN’s face now with their Netflix users is that they can no longer “fake” the location of where you are located to view Netflix anymore. Netflix most likely figured out what IP-addresses were used by the local servers from the VPN companies, and started to block them. Also Dynamic DNS services, used with the same intend to “fake” the actual location are in most cases blocked by Netflix as well. So, no more Netflix while on the road or on vacation? I don’t think so!

To view Netflix on the TV in my holiday apartment in a convenient manner, I use an iPad with the Netflix app to select and play the movies. I use a Google Chromecast 2 to cast the movies and sound to the TV. The iPad also acts as the remote to control the play. 

Even though I can quite easily get the iPAD to circumvent the geo-block, the Chromecast will spoil the party. Of course I could use my PC, but then I need to move that around to connect it to the TV through an HDMI cable, but, annoyingly, I don’t get the sound to play on the TV, or need to use another additional audio cable. Yuck!

I use the following list of equipment :
  • A Raspberry Pi - I still use the now “classic” Model (1) B, it has everything we need.
  • A USB WiFi Adapter. This tutorial only covers the Edimax Wireless 802.11b/g/n nano USB adapter. It is also known as a EW-7811Un 802.11n Wireless Adapter [Realtek RTL8188CUS] and has ID 7392:7811. (read on for more details)
  • An SD Card flashed with Raspbian-Jessie-Lite (2GB is enough)
  • A tablet with the Netflix app to control the movies. I use an iPAD.
  • A Google Chromecast 2 to cast the movies and sound from the tablet to the TV.
  • A LAN cable to connect the Ethernet port from the Pi to the local router/modem.
  • A Power supply that can provide enough current to the Pi and the USB WiFi adapter.
  • The following is only needed during the setup and testing. Access to the RPi is mostly headless, but after starting the VPN client locally to your router, you’ll lose PuTTY access to the RPi. Connect the Pi with an HDMI cable to a monitor/console during the setup and test phase to see the boot process. Also use a USB keyboard connected to a RPi USB port to control it that way.
The most important and crucial device for success on this list is the right USB WiFi Adapter. As a minimum, get a WiFi adapter from the Pi approved list, and if you can, get the Edimax one I use. The best way is to use an adapter that you know already works with the Pi in your environment, before attempting to use it in our special setup. The crucial feature we need from the adapter is the so called  Master mode.

Before you get started, you should realize that this is not a 5 minute job. You should at least expect to spend 1-2 hours on the whole setup process. 

Where possible, I put all the commands and data to be used in the process in a form that lets you do a copy and paste to the Pi. It helps to cut down on typing errors. Watch out for “ and – errors, I write this in MS-Word and it messes with these characters in a Linux environment.


1.   Getting Debian-Jessie-Lite setup on the SD card

Do not attempt to do this install on your current SD card setup. Use a separate card for this setup. It allows you to quickly switch cards from your daytime Pi stuff to your evening couch-surfing activity without the headaches of changing things back and force. Even if you will use a dedicated Pi, start with a new install of the latest Raspbian from here :

WARNING:
Too many times, way too many times, an update/upgrade of the Raspberry Pi Raspbian software  breaks setups that worked flawlessly before.
The setup described below has been verified to work with the downloaded May-16 version of Jessie (Lite) with the release date of 2016-05-27. After an update/upgrade on 19-Jan-2017 I have verified that it still works. This is no guarantee for future upgrades. Make sure that before you update/upgrade, you ALWAYS make an image backup and keep it in a very safe place! Unless you really, really have to, I advise you strongly to not do an upgrade/update unless you have made a safe copy of the working SD card! I also suggest you try to re-install the image backup, because that has failed me once too. Better safe than sorry! The minutes of time you spend to make a back-up of the working SD card, can dwarf the number of hours to get it working again. Been there, not done it...
    
There are enough examples on how to get Raspbian on an SD card, I’m not going to cover that. I selected Jessie-Lite for this project. It does not have the bloat-ware that I don’t need for this application, and it fits easily on a 2 GB SD card. Just about any decent and good quality card will do, speed is not important. The same is true for the Pi, the classic model (1) B is fast enough and has all we need.

When you have copied Raspbian-Jessie-Lite on the SD card, connect the monitor (console) with an HDMI cable, connect the Pi to your router with the LAN cable and put the USB WiFi adapter in a free slot, if you have a USB keyboard, connect that as well. Apply power to the Pi.

The easiest method you should try to get access is by using PuTTY on your PC and use raspberrypi.local as the hostaddress. If that works, go to Login.

If not, try this. Towards the end of the boot process, you will see an IP address listed on the console. Use that with PuTTY on your PC and get access to a prompt. 


If you don’t have a console, use the free Fing app on your tablet, or go to the management page of your router and ask for a list of connected devices. 

If that still does not work for you, put the SD card back into the PC and edit /boot/cmdline.txt on the SD card. At the end of that long line, add ip=192.168.1.99 close the editor put the SD card back in the Pi and reboot again. After that is finished, use this IP address with PuTTY to gain access.

Login:
Log in with user pi, password raspberry. Then run the setup program:

sudo raspi-config

Go through the following options:
   - Expand Filesystem
   - Change User Password (pick a good one!)
   - Boot options : wait for Network at Boot (Yes)
   - set the International Options (Language, Timezone, WiFi country, Keyboard)
   - go to the Advanced Options:
     set the Hostname (write it down!)
     set the Memory Split to 16MB
     enable SSH

Finish the program. If you added the IP address to the /boot/cmdline.txt file, take it out now.

Reboot the Pi.

Then open PuTTY again and in the Host Name (or IP address) field, type in hostname.local .
 
The Hostname is the one you just used and wrote down. Save the session with a meaningful name and Open the PuTTY session. Log in with pi and your new password.
Get a fresh list of available packages and update the installed ones to the latest version:

sudo apt-get update; sudo apt-get upgrade

At this moment, you should have a fully functioning Pi with an ethernet connection to the internet. If not, make that work first before you go to the next step.

Now check if Jessie recognized the WiFi adapter and loaded a driver automatically. Run “iwconfig” to see if it did. Don’t worry about missing items, we just want to see if Raspbian recognized the hardware and loaded a driver for it. If not, you may have a device that is not supported. Get this fixed before you move on.

Run “lsusb” to list the USB related devices:

My WLAN USB adapter produced this:

ID 7392:7811 Edimax Technology Co., Ltd EW-7811Un 802.11n Wireless Adapter [Realtek RTL8188CUS]

How to test if your WiFi adapter supports the Master Mode.
This step is crucial and you can only continue when you have success with the following tests.
If the driver for the WiFi adapter was indeed loaded, you can now check to see if the so called Master Mode is available for your device. 

I have found that there are two possibilities, unfortunately, they are both mutually exclusive. You have to try one or the other:

Method 1:
(worked only on my Ralink adapters)
Run “iw list” , and look for the section called “Supported interface modes:” check if you have AP and/or AP/VLAN listed. AP stands for Access Point, so if you do have that listed, great, you can continue.  But you need to install another hostapd version. I suggest you use Google to find a hostapd that works with your adapter to get the Master Mode or AP supported.
If you get no result whatsoever from the “iw list  command you can try the next method.

Method 2:
(worked only on my Realtek / Edimax adapters)
sudo iwconfig wlan0 mode master
If there is no error reported, celebrate and continue.

If both methods do not give you the expected result, chances are that you may be out of luck. You can still give it a try though. The issue is going to be with the hostapd we're going to install next.

The Master Mode is essential because instead of using the WiFi adapter as a “client”, we are going to set it up as a wireless Access Point, like and next to the one that your modem/router already provides.
To continue the setup, we’re going to need a few additional or different software pieces to make this work. First of all, we’re going to need hostapd (host-access-point-daemon), which is a program that helps us to setup an access point with WPA-2 security, so a client will have to use the SSID and a password to connect. Just like your modem/router needs right now.

2.   Installing a Host Access Data Point (hostapd)

The hostapd module package from Jessie cannot handle the so called Master  Mode of the Edimax WiFi adapter, so we’re going to get a better one instead.

If you have a Ralink or other brand adapter, you need to install the hostapd that works with that particular adapter. It could be that the standard hostapd package may support it. Try that.

So if you do not have the Edimax adapter, I’m not going to cover that installation here, sorry! But you may be able to follow along with what I’m doing next. Note that here is probably one additional file you need to edit : /etc/default/hostapd, and make sure it contains this: DAEMON_CONF="/etc/hostapd/hostapd.conf" .
If you can make it work, let others know how you did it.

Continuing with the Realtek / Edimax adapter.
Thankfully, Jens Segers, a smart guy from Belgium, has configured the hostapd portions that are supplied by Realtek, the manufacturer of the Edimax WiFi USB adapter, and configured the various source pieces so it works on our Pi.
Run the following commands one by one,  to get, compile/link and install the new hostapd code from Jens:

wget https://github.com/jenssegers/RTL8188-hostapd/archive/v2.0.tar.gz
tar -zxvf v2.0.tar.gz
cd RTL8188-hostapd-2.0/hostapd
sudo make
sudo make install
cd

The “sudo make” will compile and link all the sources and will run a while (approx. 7 min on a Model (1) B), so be patient. Don’t worry about the warnings. When the first make has finished, “sudo make install” will put the new pieces in their right place.
To make sure that an apt-get update/upgrade does not overwrite this special hostapd package, run the following:

sudo apt-mark hold hostapd

and to remove the lock when you don’t need it anymore:

sudo apt-mark unhold hostapd


3.   Installing and setting up a DHCP server

Because we’re going to run the Pi as a server, the next piece we need is a Dynamic Host Configuration Protocol (DHCP) server to provide the connecting clients to the Pi with IP address information. The package we need is called isc-dhcp-server.
You can install this package with:

sudo apt-get install isc-dhcp-server -y

There are two configuration files /etc/dhcp/dhcpd.conf and /etc/default/isc-dhcp-server which we will need to configure.

sudo nano /etc/default/isc-dhcp-server

Make sure the INTERFACES line at the end lists your interfaces:

INTERFACES="eth0 wlan0"

Edit the .conf file:

sudo nano /etc/dhcp/dhcpd.conf

Find the following two lines and comment them out by putting a ‘#’ in front of them.

#option domain-name "example.org";
#option domain-name-servers ns1.example.org, ns2.example.org;

Find the line that has authoritative , and make it active by removing the ‘#’ in front of it.

authoritative;

Now we need to setup the DHCP server with the information we need for our Access Point.
Get access to your router. You can do that two ways. Either log-in to your router with a web browser if you can, or use your PC to get what we need.

Sometimes you don’t have access to the router so I’ll use the PC method. In the Search box on your PC, type command and then find and select command prompt. This will open a (DOS) window with a command prompt. Execute ipconfig –all and write down the following

·        The Default Gateway IP address : Most likely  this will be 192.168.1.1
·       Also write down the IPv4 Address. This is the IP address of your PC.
·       Next, write down the IP addresses for the DNS Servers. There will most likely be two.

You can close the window by typing exit.

You are eventually going to use this setup in foreign locations where you will most likely not have access to the router setup interface. This means that you will have no idea beforehand what the IP addresses of the Default Gateway and Address Pool of the DHCP server is going to be. 

Because we will setup our LAN (eth0) interface to work with the router without knowing all this, that will be OK.

However, we are also going to provide our own sub-net for the wireless clients that connect to our Access Point. The IP address pool that our Pi server will provide must be different from the ones the router is issuing, otherwise you can get IP conflicts between the wireless clients.

The gateway IP address of most SOHO modem/routers is 192.168.1.1, so the subnet the router issues will start somewhere above the 192.168.1.1 range and could potentially go all the way up to 192.168.1.254. You can verify this by looking at the IP address you got from the router for your PC. It is most likely something like 192.168.1.XX This will give you an indication of the DHCP Address Pool. Assuming you are still at home, you still have no clue what you are going to find in your remote location, so we’ll have to guess.

Assuming the remote location is not very different, setting the wireless Gateway and sub-net in the 192.168.200.0-255 range is therefore pretty safe.

You can also select other private network IP addresses like from 10.0.0.0 – 10.255.255.255, or from 172.16.0.0 – 172.31.255.255 and finally from 192.168.0.0 – 192.168.255.255, up to you.

The Default Gateway IP address I selected and will use for our wireless network will be 192.168.200.1 so we’ll tell DHCP to give out addresses to clients from the 192.168.200.10 until 192.168.200.20. Ten addresses for our pool is more than enough, if you need more, expand the range.

The next step is important for the Netflix geo-unblocking. You must use the Domain Name Servers (DNS) you wrote down earlier that your local internet provider issues to fool the Netflix geo-police.

Go to the end of the dhcpd.conf file and look for this information:

# No service will be given on this subnet, but declaring it helps the
# DHCP server to understand the network topology.

subnet 192.168.0.0 netmask 255.255.255.0 {
}

 

This subnet MUST be uncommented and present or the service will refuse to start, complaining about non-existing subnets, even though there is one defined below it.

And this is the subnet that matters and needs to be added by copy & paste after the above:

subnet 192.168.200.0 netmask 255.255.255.0 {
  range 192.168.200.10 192.168.200.20;
  option broadcast-address 192.168.200.255;
  option routers 192.168.200.1;
  default-lease-time 600;
  max-lease-time 7200;
  option domain-name "RPI-AP";
  option domain-name-servers x.x.x.x, x.x.x.x;
}

You can change the domain-name but must fill in the domain name servers you wrote down earlier, separated by a comma “,” before you move on.

Save the file and exit the editor.


4.   Installing the Network Interfaces

Now we’re going to assign a static IP address for the WiFi adapter. The standard setup from Jessie is OK for the Ethernet interface. Pre-Jessie we would do all that in /etc/network/interfaces, and setup the SSID security and password for the wireless interface in /etc/wpa_supplicant/wpa_supplicant. Things have changed however.

You first need to edit /etc/network/interfaces and disable the wpa_supplicant setup for the wlan0 interface.

sudo nano /etc/network/interfaces

To disable the supplicant setup for wlan0, comment out the line /etc/wpa_supplicant/wpa_supplicant  with a “#” in front.

important section should now look like this:

auto lo
iface lo inet loopback

iface eth0 inet manual

allow-hotplug wlan0
iface wlan0 inet manual
#    wpa-conf /etc/wpa_supplicant/wpa_supplicant.conf

allow-hotplug wlan1
iface wlan1 inet manual
    wpa-conf /etc/wpa_supplicant/wpa_supplicant.conf

Save the file and close the editor.

In Jessie, the major work will be done in the dhcpcd (dhcp-client-daemon) configuration file.

sudo nano /etc/dhcpcd.conf

At the very end of the file, add this:

# static IP configuration for a wlan0 AP
interface wlan0
static ip_address=192.168.200.1/24
static domain_name_servers=x.x.x.x x.x.x.x

The ending of the static IP address for the Gateway with “/24” is the modern equivalent (and shorthand way) of adding a 255.255.255.0 mask.

The two static domain_name_servers IP addresses must be the DNS server IP addresses you wrote down and used earlier, but note that this time, they are separated a space!

Save the file and close the editor.


5.   Setting up hostapd

We’re now going to further setup the hosting part of the wireless network by configuring the freshly compiled and installed hostapd configuration file.

sudo nano /etc/hostapd/hostapd.conf

Make the file look like this:

# Basic configuration
interface=wlan0
ssid=My-VPN-AP
channel=1

# WPA and WPA2 configuration
macaddr_acl=0
auth_algs=1
ignore_broadcast_ssid=0
wpa=3
wpa_passphrase= My_passphrase
wpa_key_mgmt=WPA-PSK
wpa_pairwise=TKIP
rsn_pairwise=CCMP
# Hardware configuration
driver=rtl871xdrv
ieee80211n=1
hw_mode=g
device_name=RTL8192CU
manufacturer=Realtek

There is one item you should change, one you must, and one you could.

You should change the SSID name into something more meaningful then “My-VPN-AP”, so you can find the server name in between all the other SSID’s that show up on the WiFi network access list. You can be creative but I suggest you don’t use Netflix, or you may draw more attention than you want.

You must change the My_passphrase into a strong password. Remember that everybody can see your SSID and try to log in. Although the wireless range of the adapter is limited, maybe only one large room size, you should be careful. The guy next door may have a hobby that you don’t know about.

You could change the wifi channel parameter, based on what channels the surrounding servers use, to minimize interference conflicts. (select 1, 6 or 11)

Save the file and exit the editor.

You can now run a first test to see if hostapd will work:

sudo hostapd /etc/hostapd/hostapd.conf

Use Ctrl-C to terminate the process and look for possible errors.
 

6.   Setting up the NAT & Filtering

The last step before we can start to use the Access Point is setting up a configuration that tells the system what to do with the incoming data and where to send it to. This will allow us to accept packets from clients like the iPad with the Netflix app, and manipulate them before we pass them on to the modem/router on their way into the wild internet.

This process is called Network Address Translation or NAT and Filtering.

First we’re going to change the kernel packet forwarding setup.

sudo nano /etc/sysctl.conf

Look for the line that says net.ipv4.ip_forward and remove the hash in front. It should now look like:

# Uncomment the next line to enable packet forwarding for
IPv4net.ipv4.ip_forward=1

Save and close the file.

We are now going to use the Raspbian firewall and make it do some interesting things. The firewall is also called iptables, and this is a very powerful and flexible piece of software. It’s not easy to understand, looks intimidating at first but I will briefly show what you need to do and explain it a bit.

Basically we are telling iptables what we want it do to with the information coming in from the clients through the wlan0 interface and how and also what to send to the eth0 interface on the way out. We also should take care of the answering information and get that back to the client that asked the question.

At first we are going to define the rules for a general purpose Access Point. Later we will change them when we are going to work with the VPN and we’ll also add some more rules to fool the Netflix and Chromecast’s DNS requests.

Just cut & paste these lines to execute them on the Pi (the second one is one large line, no break):

sudo iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
sudo iptables -A FORWARD -i eth0 -o wlan0 -m state --state RELATED,ESTABLISHED -j ACCEPT
sudo iptables -A FORWARD -i wlan0 -o eth0 -j ACCEPT

These three statements will tell the firewall to pass on anything coming from the wlan0 interface, and pass that on to the eth0 interface and back.  

The results are now loaded into the iptables run-time memory. A reboot will flush them again. To make them “stick”, we need to put them into a file that will be read and loaded at startup. To do that you need to execute the following two steps.

We’ll use a storage location that is also used by iptables-persistent, which we don’t use, but the location it uses to store the rules is useful.

First create the configuration file path:

sudo mkdir /etc/iptables

Now transfer the rules from memory into a configuration file so we can instruct  iptables to load these rules later:

sudo sh -c "iptables-save > /etc/iptables/ap-rules.v4"

If you want, you can look at the results by executing :

cat /etc/iptables/ap-rules.v4

If you see 3 lines that start with “-A” in your report, they are proof that the 3 rule settings were added to the configuration file.

To make the setup and the testing easier, we’re going to create a few helper scripts. I suggest you simply put them in your home directory /home/pi. The first script will flush the iptables rules from the run-time memory. Open the editor :

nano flush-iptables

Copy & paste the following:

#!/bin/bash
sudo iptables -F
sudo iptables -t nat -F
sudo iptables -X

Save the file and close the editor. Make the script executable with :

chmod +x flush-iptables

We now need make sure that these iptables rules are installed at boot time.
Create a shell script to install the rules. Start the editor with the filename:

nano install-ap-rules

Copy and paste the following (no need for sudo, it will run at root level during boot):

#!/bin/bash
iptables-restore < /etc/iptables/ap-rules.v4

Make the file executable:

chmod +x install-ap-rules


7.   Starting all the Components at Boot

With all the basic elements for the wireless AP set up, we can now organize the starting of processes during the boot process that will first turn your Pi into a wireless Access Point with a DHCP router before we move on.

With the introduction of Jessie, we also were kindly “invited” to embrace the systemd concept, a major departure from the previous way (System V) to start and manage jobs and daemons at boot time. I’m not going into details here, I’m just going to explain what you need to do to embrace the new method.

With the advent of the systemd concept, it is now a lot easier (my opinion) to get processes starting in the right order.  This became important, because Jessie tries to start jobs and daemons in parallel to speed-up the boot process.  You may have noticed that even /etc/rc.local is not longer the last item in the boot process. This re-juggling of the order has caused some surprises when previously working boot processes did no longer function with the upgrade to Jessie.

We’ll start simple, with the iptables rule loading. Create a so called service file that systemd will use to run the rule loading script at boot.

sudo nano /etc/systemd/system/ld-iptables.service

Copy and paste the following into the open editor:

[Unit]
Description=Load iptables rules
Before=hostapd.service


[Service]
Type=oneshot
ExecStart=/home/pi/install-ap-rules

[Install]
WantedBy=multi-user.target

Save the file and exit the editor. This simple script will tell systemd to install the rules as soon as the boot process went into the multi-user mode phase in the boot process and before hostapd gets started. The Type=oneshot means that the program will not continue to run (as a daemon), but terminate when done.

You can test the systemd script by running:

sudo ./flush-iptables
sudo systemctl start ld-iptables
sudo systemctl status ld-iptables

The systemctl status will show something like this :

● ld-iptables.service - Load iptables VPN rules
   Loaded: loaded (/etc/systemd/system/ld-iptables.service; disabled)
   Active: inactive (dead) since Tue 2016-04-12 12:06:52 CEST; 2h 53min ago
  Process: 326 ExecStart=/home/pi/install-vpn-rules (code=exited, status=0/SUCCESS)
Main PID: 326 (code=exited, status=0/SUCCESS)

Apr 12 12:06:52 nflx-ap systemd[1]: Started Load iptables VPN rules.

We have not installed it into the systemd boot sequence yet, so it reports disabled, that is OK. Because our script is not a daemon that continues to run, it will show that is currently inactive (dead) code=exited, but the status after it ran was SUCCESS. That is normal and good.

Now test the result:

sudo sh -c "iptables-save > test"
cat test

The test file collected the active rules from run-time memory and they are proof that it all went well.
Make systemd aware of the need to load the rules at boot.

sudo systemctl enable /etc/systemd/system/ld-iptables.service

Lets now move to a little bit more complex boot situation. The hostap deamon needs to be up and running to provide the subnet on the wlan interface before the dhcp server can start to dish-out IP addresses to clients. We’re going to use systemd to take care of that for us.

Change to administration or root level to avoid all the sudoing:

sudo -s

Create these two so called “drop-in” directories:

mkdir /etc/systemd/system/hostapd.service.d
mkdir /etc/systemd/system/isc-dhcp-server.service.d

Start an edit session:

nano /etc/systemd/system/hostapd.service.d/hostapd.conf

Copy & paste the following:

[Unit]
Description=Start hostapd before isc-dhcp-service
Before=isc-dhcp-server.service
Before=system-openvpn.slice

[Service]
# Added to restart the service
Restart=always
RestartSec=5
StartLimitBurst=4
StartLimitInterval=180s

# reboot option : be carefull!
#StartLimitAction=reboot


Save the file and close the editor.

This so called drop-in configuration file will overrule, or extend, the existing System V model start-up configuration for the hostapd  process at boot. In this case, we want systemd  to install hostapd before the dhcp server daemon installation, and these two must be up and running before the openvpn process gets started.
The currently commented out statements in the Service section tells systemd to make sure that the hostapd service is always running. If it dies for some reason, systemd will restart it after 5 seconds. If this is happening more than 4 times within a 180 second window, something is very wrong and the RPi can be rebooted. Be carefull with this option, because the RPi can get into a loop rebooting itself!

Start another edit session:

nano /etc/systemd/system/isc-dhcp-server.service.d/isc-dhcp-server.conf

Copy & paste the following:

[Unit]
Description=Start isc-dhcp-server after hostapd
After=hostapd.service
Requires=hostapd.service

Before=system-openvpn.slice

[Service]
ExecStartPre=/bin/sleep 1


# Add restart options, just in case
#Restart=always

#RestartSec=5
#StartLimitBurst=4
#StartLimitInterval=180s


# Add rebooting options:
#StartLimitAction=reboot
 

Save the file and close the editor.

This one is a little more involved. The dhcp server must start after the hostapd service has started. And, of course, it must also start before the openvpn service gets started. The hostapd process also needs to completely finish the establishment of the wlan0 subnet. The sleep delay of 1 second is there to facilitate that. Without the delay, the start of the dhcp server will fail because there is no sub-net yet.  Notice the full path to the sleep binary. Get used to the fact that with systemd, you must always use the full path for executables.

With the new boot method, systemd sits kind of “on top” of the previous System V /etc/rcx.d and the /etc/init.d structure. Both the dhcp and hostapd daemons were installed in that structure already. The new systemd simply controls those System V start-up scripts.

We now need to tell systemd to take charge of the “old” SystemV method of doing things:

systemctl enable isc-dhcp-server.service
systemctl enable hostapd.service

Exit from root level and return to user level:

exit

Now that all the groundwork is in place, we have a fully functional and general purpose wireless Access Point with router functionality. 

Now try the successful execution of the above two installation files:

sudo systemctl start hostapd.service
sudo systemctl status hostapd.service

sudo systemctl start isc-dhcp-server.service
sudo systemctl status isc-dhcp-server.service 

If you get an error message that says : Bad format, this is a cryptic way of saying that you have a syntax error in the service file.
When you are satisfied and there are no errors reported, you can now remove the comments in the Service sections, to make the restarting active. Leave the rebooting option until at the very end when everything is working properly, or don't turn it on. Up to you.

WARNING: 
If you will get an error in the starting of the process, and you have the reboot option active, the RPi will get into a loop restarting and rebooting, and you may loose control. So make sure the two service files work successfully before you turn this option on.
 
You can now reboot the Pi and try this out.


8.   Testing the Access Point Functionality

After the Pi booted, you can test the wireless AP function.

Use a wireless device, like a tablet or phone and see if you can find the SSID of the AP. If you do, select it and login with the password you setup for the SSID.

Check to see if you got an IP address out of the pool, and if the Gateway is correct.

On your tablet, go to your internet browser like Safari and go to www.whatismyipaddress.com
If it loads you have internet and DNS is working, check if it also reports the public IP address of your internet router.

For fun, go to another website, www.speedtest.net and run the test.

If this all works, go to the next step.

Some Troubleshooting hints:
If you cannot do all the above successfully, you need to log-in to the Pi with PuTTY and check iwconfig to see if the WiFi adapter is indeed in the Master mode and if the ESSID is correct and listed.

Then check with ifconfig if all the IP addresses for the eth0 and wlan0 interfaces are correct .
Use route to see the Kernel Routing Table.

Use ping to a site like www.raspberrypi.org to see if you have internet and DNS working through the eth0 interface and the router.

If it cannot find the host, try to ping 93.93.135.188 to see if it’s a DNS problem.

If that does not work, try

traceroute www.raspberrypi.org or 93.93.135.188

To see where you get stuck on the way to the destination.

Check the correct installation and running of the two daemon processes (look for the green active state) and also check the correct installation of the iptables rules:

sudo systemctl status hostapd
sudo systemctl status isc-dhcp-server
sudo sh -c "iptables-save > test"
cat test

If these three processes did not install properly, you need to carefully double-check all the config files for errors. Look at the /var/log/syslog to see if the processes were installed in the right order.

cat /var/log/syslog | more

Try the steps one by one and look at the console for clues. Carefully look for incorrect quotes “ instead of " and option characters like – instead of – in the commands or config files. These are typical cut & paste issues. Happy hunting & good luck!


9.   Setting up and Adding the VPN to the AP

Now that we have a fully functioning general purpose wireless Access Point, we’re going to add the last bits and pieces to dodge the geo-restrictions of Netflix and commercial broadcasters like RTL and the BBC.

First of all, we’re going to install a VPN connection between the Pi and your router at home. The “tunnel” that will be created between the two will allow you to act as if you are back home, even though you are in another country.

We’ll use OpenVPN to create that connection. I cannot possibly give you step-by-step instructions on setting up a OpenVPN server on your router. There are too many router models, besides, it is mostly really simple.

Luckily, most common routers have OpenVPN server and client setups that can be done with a few mouse-clicks. I have an Asus RT-AC68U wireless router and it took me only a few mouse-clicks to set the OpenVPN server up. Just in case you wonder, we must use the OpenVPN server setup on the router and the client on the Pi, because we need to be able to open the firewall port for the server. It is unlikely you can do that in a remote location.

Before you start to setup the OpenVPN server on the router, tell the (Asus) router to setup a dynamic hostname for the public IP address. It will automatically link the hostname you select to the public IP address, even if your provider decides to change it. Mine comes in the form of “ my-router-name”.asuscomm.com If your router supplier does not provide that feature, you need to get a dynamic host name assigned through another provider and install the code to keep track of the public IP address changes.

When you setup the OpenVPN server on the router, instruct it to use port 443 (the https port) instead of the VPN port 1194. In the unlikely case that the Netflix geo-police examines the packets, they may lock on to the use of the VPN port, and they can block your access. The https protocol also uses TLS/SSL certificates and keys, much like the VPN protocol, so we’ll use that to further disguise our link.

In the process of setting up the OpenVPN server, I could also download an OpenVPN client setup file from the router through my browser to the PC. This file is called client.ovpn and it contains all the setup information including the certificates and keys.

After the OpenVPN setup on the router is finished, we’re going to install the OpenVPN package on the RPi :

sudo apt-get install openvpn

We now need to install the configuration file that will setup OpenVPN as a client on the Pi. I copied the client.ovpn file from the PC to a USB stick. Depending on the USB memory stick you use, it’s better that you powerdown the RPi, before you add the stick to the free USB port. If you have a Model (1) B, it may crash your Pi due to the power surge. After installing the USB stick, boot your RPi and log in again.

To get access to the .ovpn file on the stick now on the Pi, you need to mount the stick and create a mounting point to gain access to the data on the stick.

Create a mounting point :

sudo mkdir /dev/stick

Then mount the stick as a file system

sudo mount /dev/sda1 /dev/stick

Check with :

ls -al /dev/stick

Copy the client file to the OpenVPN directory. I renamed it to asus-client.conf.

sudo cp /dev/stick/client.ovpn /etc/openvpn/asus-client.conf
sudo umount /dev/stick

Edit the asus-client.conf file to change a couple of things.

sudo nano /etc/openvpn/asus-client.conf

Add/modify these lines:

dev tun :modify to: dev tun0
remote your-choosen-router-name.asuscomm.com 443
auth-user-pass /etc/openvpn/login.conf
verb 3

Save the file and exit the editor. The first line is there to prevent a pesky error message during the startup: "RTNETLINK answers: File exists". The second line is there to make sure you have your dynamic hostname correct and you use port 443. The second line adds a link to a login file in the /etc/openvpn directory that will contain the username and password that you created when you installed OpenVPN on the router. Verb 3 means verbosity level 3 and it shows more details on the console so you can see what is going on.

Create the log-in file  :

sudo nano /etc/openvpn/login.conf

Type the OpenVPN username on line1 and the password on line2
Save the file and close the editor. To make sure only you as the user can look at the file or edit it, change the access rights:

sudo chmod og-rw /etc/openvpn/login.conf

We’ll now create a few more helper scripts. The following one will show the Public IP (pip) address of the router.

nano pip

Copy & paste the following:

#!/bin/bash
wget http://ipinfo.io/ip -qO -

Save the file and close the editor. Make the script executable with :

chmod +x pip
Run it with:

./pip

It will show you the current public IP address of your router that got assigned by your internet service provider.

And one more helper script that will start the VPN Client and build  the connection to the OpenVPN server on the router. It runs in the foreground so we can see what’s going on during the testing:

nano test-ovpn

Copy & paste the following:

#!/bin/bash
sudo openvpn --config /etc/openvpn/asus-client.conf

Save the file and close the editor. Make the script executable with :

chmod +x test-ovpn

This sript will only be used during tests, do not use it for your regular sessions, there is a better way.

Using systemd to handle the OpenVPN process.

Create a helper scipt in your home directory.

nano start-ovpn

Cut and paste this into the open editor:

#!/bin/bash
systemctl start openvpn@asus-client.service

Save the file and close the editor.

Also create a file called stop-ovpn and one called status-ovpn the same way, just replace start with status or stop in the file name and in the command line. Note, with the status version, add an "-l" at the end of this command line to avoid the truncation of the listing.
Now we're going to tell systemd what to do when we want it to start the openvpn session. We will create a service file for our specific setup by creating a symbolic link as follows:

sudo ln -s /lib/systemd/system/openvpn@.service /etc/systemd/system/openvpn@asus-client.service

Instruct systemd to activate this.

sudo systemctl enable openvpn@asus-client.service
  
OK, now we're going to modify the service file to automatically restart the OpenVPN service, if it does not work first time at boot, or if it fails while it is running. Systemd will try to restart the service 4 x within 180 seconds. If that fails, we can also instruct systemd such that the RPi will get rebooted.

sudo nano /lib/systemd/system/openvpn@.service



Add these statements at the end of the Unit section:


# Added my own conditions below but they don't seem to work.
# Restarting the service will eventually get things going though
Requires=hostapd.service
Requires=isc-dhcp-server.service
After=hostapd.service
After=isc-dhcp-server.service


Copy and paste the following restart/reboot settings at the end of the Service section:

# Added my own restart options
Restart=always
RestartSec=5
StartLimitBurst=4StartLimitInterval=180s

# Reboot option : be carefull
#StartLimitAction=reboot
 
Save the file and close the editor.

As we have done earlier, the restarting and rebooting options have been disabled. You can activate first the restarting options and then later the reboot option, with the caveat that you may loose control if something is not working and you have the reboot option active! You don't really need the last option, up to you to decide.

Before we can test the OpenVPN setup, we need to alter the iptables rules. The iptables rules we used so far for the wireless AP, that connected the wlan interface with the Ethernet interface need to change such that the wlan interface will connect to the VPN tunnel interface instead. We can do that by simply changing all references from eth0 to the tun0 interface that OpenVPN will create.

First flush the rules :

sudo ./flush-iptables

Load the new rules into the iptables run-time memory:

sudo iptables -t nat -A POSTROUTING -o tun0 -j MASQUERADE
sudo iptables -A FORWARD -i tun0 -o wlan0 -m state --state RELATED,ESTABLISHED -j ACCEPT
sudo iptables -A FORWARD -i wlan0 -o tun0 -j ACCEPT

Save them into a new rules file:

sudo sh -c "iptables-save > /etc/iptables/vpn-rules.v4"

Create a new script to load them

nano install-vpn-rules

Copy and paste this:

#!/bin/bash
iptables-restore < /etc/iptables/vpn-rules.v4

Make the file executable:

chmod +x install-vpn-rules

We’re done with this part.

10.        Running a local VPN Test

We can now run a test to see if we can activate the OpenVPN client with the configuration file.

Note
You cannot establish a link (a tunnel) from the client to the internet, because you need to get access to the server from the public side of the internet. We are at the private (local) side at the moment.
If you used PuTTY to connect to the Pi, you will lose that connection when you start the client after a little while. If you hit Ctrl-C however, the client will automatically terminate. 
Run the following scripts one by one:

sudo ./install-vpn-rules
sudo ./test-ovpn



Follow the instructions if there has been an error, or check /var/log/messages

Run ./pip to see if you indeed got the IP address "at the end of the tunnel"

If this all worked, try the systemd script by running ./start-ovpn
Execute ./status-ovpn and then ./stop-ovpn to terminate the session.

OK, now we need to address a Netflix and Chromecast device DNS issue. I use the Chromecast to cast the Netflix videos to my TV. However, the DNS IP addresses for the Chromecast are hard-coded in the firmware and they will go to the Google DNS servers (8.8.8.8 and 8.8.4.4). As soon as the Netflix app sees that route, the connection you build to your router will be terminated and you will be back to the local Netflix content.

With the following two additional iptables instructions, we simply filter for all requests for the two Google public DNS servers from the Chromecast and the Netflix app and let them go to bit heaven by bluntly rejecting them. Eventually the DNS requests will get answered by our own servers so both are happy, but they do not know that we fooled them.

Let’s test that.

Use the setup as a general purpose AP again:

sudo ./flush-iptables
sudo ./install-ap-rules

Grab your tablet and log in to the AP. Go to the ping app (or install it for free if you don’t have it yet) and use it to ping the Google DNS server at 8.8.8.8 That should work.

Now execute the following by cutting & pasting this to your Pi:

sudo iptables -I FORWARD --destination 8.8.8.8 -j REJECT
sudo iptables -I FORWARD --destination 8.8.4.4 -j REJECT

And run the ping test again. This time, you will get a Request timeout. Just what we want!
OK, load the vpn rules again and also add the DNS blocking rules:

sudo ./flush-iptables
sudo ./install-vpn-rules
sudo iptables -I FORWARD --destination 8.8.8.8 -j REJECT
sudo iptables -I FORWARD --destination 8.8.4.4 -j REJECT

And convert them from run-time memory to update the vpn rules file.

sudo sh -c "iptables-save > /etc/iptables/vpn-rules.v4"

At this point, we have all the bits and pieces ready and also tested them locally.


11.        Finishing the Boot Process

At this point, we are going to further automate the configuration so everything loads when we boot the Pi.

First we’ll convert the setup from a general purpose AP to an AP with VPN.

sudo nano /etc/systemd/system/ld-iptables.service

And change the Service section to :

ExecStart=/home/pi/install-vpn-rules

Save the file and close the editor. 

With all that done, you can now reboot the Pi and watch everything on the console to see if it all goes as we want it to. Especially pay attention to the installation of our 3 systemd processes. There should be no errors and they should install in the right order.

Now you can start and stop the VPN service at will. 
When you now boot the RPi, the three services will install and the VPN tunnel will be established.

After the booting finished, log in and check if the processes are running.



12.        Setting up the Chromecast

At this point we now need to instruct the Chromecast device to use your new wireless Access Point on the Pi so it will get the DNS treatment. Make sure the AP (without the VPN - you need internet access) is up and running.

Connect the Chromecast to a HDMI port on your TV and apply power (I use a USB port on the TV) You reset the device by pressing the tiny reset button on the Chromecast2 device for a few seconds. It will then perform a factory restart followed by the setup procedure. This time however, you are going to instruct it to connect to your Pi wireless Access Point, and set the password. 

If it all went well, you can now select the TV as the casting target and it should compete the setup. When that is successful, it will tell you it is connected to your Pi, has a connection to the internet and will start to show the nice background pictures.


13.        Testing the Setup Remotely



To be really sure everything works, you now need to pack-up and go to another location (family, friend, neighbor) where you can connect your AP-VPN RPi to their router, and get access to your own router at home through a VPN tunnel across the public internet.

Take along the Pi, the power supply, a (long?) LAN cable, your PC/Laptop, your tablet, the Chromecast with power cable and optionally, an HDMI cable to connect the Pi to a local TV to act as a console and a USB keyboard if you can’t use PuTTY.

When you are there and connected and booted your Pi, you can use your tablet and log in to the wireless AP. 

Go to the browser on the tablet and select www.whatismyipaddress.com again. It should (still) show the public IP address of the router you have at home. If so, the VPN tunnel works.
Try to ping 8.8.8.8 and 8.8.4.4 from the tablet, just to make sure that these DNS addresses are still blocked.

Connect the Chromecast to the TV and apply power. Start Netflix on your tablet, log in, and select a movie. Also select local language elements like sub-titles or language. Start the movie and cast it to the TV.

If all went well, you will see the sub-titles and hear the local language as if you were couch surfing at home. Obviously, you get the local content, but Netflix has no clue anymore that you are not at home. It will be the same if you are outside of your country now.

If you ran into troubles at this stage, use PuTTY or the console/keyboard and check if all processes were started and have a good status. If you can’t figure it out, go back and start with the general purpose AP configuration and add a process or step at a time. Chances are that you’ll find a bad configuration in the iptables or VPN portion, or a process didn’t boot properly. Don’t stray from my step-by-step process, wait until everything works before you add your own flavor.



If you were able to successfully test the AP-VPN setup remotely, congratulations!  You can proudly show your skills by doing a demonstration. I also bet you can’t wait to get outside the country and try this out…

Now for one last highly recommended addition. You will most likely want to stop the AP-VPN and poweroff the RPi when you no longer need it. You also want to be careful with the setup you put on the SD card, because it took a lot of work. Yanking the power from the RPi is a sure way of ruining several hours of work, so even when you made a backup of the SD card (recommended!), you need to do something about the starting and stopping of the RPi.

I published a couple of very simple methods on the forums and on this blog, including a very simple one that will only cost you two resistors, one capacitor and one button. That plus a bit of solder, some wires and a tiny script will let you shutdown the RPi (actually halt it) and restart it from the halt state by the same button. It will significantly reduce the power consumption, and will prepare the RPI for disconnecting the power all together.


Within that post is a link to an even more complete solution that will also cut the power to the RPi.

Enjoy!

No comments:

Post a Comment