uncategorized

Setup Split VPN on Unifi USG Using PBR

This is a writeup for setting up a client OpenVPN (or any other VPN type) connection, but only use it for some clients/VLANS. I’m setting things up using a Unifi USG and the special config file. Since the UI is currently not supported for client OpenVPN configuration, although support is on the roadmap, this will show it is actually quite easy to setup.

As the UI is not yet supporting the configuration, the neccessary configuration needs to be entered into the special config.gateway.json file on the controller. The usual way to do this is to use the cli on the USG, do the configuration and then export the json and copy parts of it to the config.gateway.json file. I’ve done this, so you can easily just copy from my configuration and then edit the parts you want.

I’m reusing the work from Travis Cook’s Detour in order to be able to select which device is using the VPN tunnel. It is making use of PBR and provides a UI to add predefined clients (really their IP address) to the list of IPs that are routed via the VPN. I’m using this mostly to access Netflix and similar services from ATVs and the like. I actually use a pptp VPN due to speed, but OpenVPN does perform with a single stream.

Known caveats: Som earlier revision of the USG firmware had a broken PBR. Guess how I know this!

Get familiar with the config.json structure on the USG

To get a formatted copy of the config.json file on your controller, you can dump the current configuration from your USG into a formatted file. Login to the USG and use the existing utility to create the file. I named my file with the suffix .json since I want my text editor to help out with formatting. Transfer the file back your computer for easy editing. I’m using Transmit since I’m on a Mac. Once the file is on your computer, use your favorite editing tool to open it. I use Visual Studio Code.

ssh admin@10.0.1.1*
mca-ctrl -t dump-cfg > config.json*

To have the whole thing collapse, Ctrl/Cmd-K + Ctl/Cmd-0

*This way you see the top most nodes and their relative structure. In our own file, we will add configuration to several nodes, but not all and we can therefore copy the nodes we need including their structure. *

Location of the config.gateway.json file

So the Unifi Controller provides an easy way to inject additional configuration. The location is site specific.

Since I’m running the Controller in a Docker container, I’ve chosen to expose most data and configuration outside of the container instance and I’m using a plain share since I’m hosting my Docker on a Synology NAS.

The controller merges this configuration when provisioning the USG device. If there are errors in this file, the provisioning process may end up in a loop so make sure to check the logs and the alert view in the Controller. I export all USG logs to a syslog server, which is handy to check for errors. You can also check the logs locally on the USG.

Overview of the OpenVPN settings on the USG

I highly recommend to only configure the OpenVPN connection first, to ensure the VPN actually works before proceeding with anything.

The OpenVPN configuration is placed into a local file on the USG. An additional credentials file is created.

Create the folder /config/openvpn on the USG

We place two files within this directory:

  • The credentials file containing username and password
  • The client configuration file, specific to your chosen OpenVPN provider

The OpenVPN client configuration is specific to your provider. I use Giganews/VyprVPN and the settings are retrieved from their sample files. I did need to add configuration to not pull and overwrite my Gateway and routing settings.

Actually we have three files, where one is copy of the other. Since I have several configuration files, I just copy the current file over the generic one. Initially I did want to use a symbolic link, but that didn’t work as expected.

Create the credentials file

Create the file /config/openvpn/password_filename. Provide you own username/password on two separate lines.

username
password

Create the configuration file for openvpn

Create the file /config/openvpn/gateway.ovpn containing the following:

I added the route-noexec parameter in order to block the VPN from overwriting my default routes and gateway settings.

#  _____ _         _____               
# | __|_|___ ___| | |___ _ _ _ ___
# | | | | . | .'| | | | -_| | | |_ -|
# |_____|_|_ |__,|_|___|___|_____|___|
# |___|

client
dev tun
proto udp
remote us5.vpn.giganews.com 443
resolv-retry infinite
nobind
persist-key
persist-tun
persist-remote-ip
;ca ca.vyprvpn.com.crt
tls-remote us5.vpn.giganews.com
auth-user-pass /config/openvpn/giganewsauth.txt
comp-lzo
verb 3
auth SHA256
cipher AES-256-CBC
keysize 256
tls-cipher DHE-RSA-AES256-SHA:DHE-DSS-AES256-SHA:AES256-SHA
route-noexec
#route-nopull

<ca>
-----BEGIN CERTIFICATE-----
MIIEpDCCA4ygAwIBAgIJANd2Uwt7SabsMA0GCSqGSIb3DQEBBQUAMIGSMQswCQYD
VQQGEwJLWTEUMBIGA1UECBMLR3JhbmRDYXltYW4xEzARBgNVBAcTCkdlb3JnZVRv
d24xFzAVBgNVBAoTDkdvbGRlbkZyb2ctSW5jMRowGAYDVQQDExFHb2xkZW5Gcm9n
LUluYyBDQTEjMCEGCSqGSIb3DQEJARYUYWRtaW5AZ29sZGVuZnJvZy5jb20wHhcN
MTAwNDA5MjExOTIxWhcNMjAwNDA2MjExOTIxWjCBkjELMAkGA1UEBhMCS1kxFDAS
BgNVBAgTC0dyYW5kQ2F5bWFuMRMwEQYDVQQHEwpHZW9yZ2VUb3duMRcwFQYDVQQK
Ew5Hb2xkZW5Gcm9nLUluYzEaMBgGA1UEAxMRR29sZGVuRnJvZy1JbmMgQ0ExIzAh
BgkqhkiG9w0BCQEWFGFkbWluQGdvbGRlbmZyb2cuY29tMIIBIjANBgkqhkiG9w0B
AQEFAAOCAQ8AMIIBCgKCAQEA37JesfCwOj69el0AmqwXyiUJ2Bm+q0+eR9hYZEk7
pVoj5dF9RrKirZyCM/9zEvON5z4pZMYjhpzrq6eiLu3j1xV6lX73Hg0dcflweM5i
qxFAHCwEFIiMpPwOgLV399sfHCuda11boIPE4SRooxUPEju908AGg/i+egntvvR2
d7pnZl2SCJ1sxlbeAAkYjX6EXmIBFyJdmry1y05BtpdTgPmTlJ0cMj7DlU+2gehP
ss/q6YYRAhrKtlZwxeunc+RD04ieah+boYU0CBZinK2ERRuAjx3hbCE4b0S6eizr
QmSuGFNu6Ghx+E1xasyl1Tz/fHgHl3P93Jf0tFov7uuygQIDAQABo4H6MIH3MB0G
A1UdDgQWBBTh9HiMh5RnRVIt/ktXddiGkDkXBTCBxwYDVR0jBIG/MIG8gBTh9HiM
h5RnRVIt/ktXddiGkDkXBaGBmKSBlTCBkjELMAkGA1UEBhMCS1kxFDASBgNVBAgT
C0dyYW5kQ2F5bWFuMRMwEQYDVQQHEwpHZW9yZ2VUb3duMRcwFQYDVQQKEw5Hb2xk
ZW5Gcm9nLUluYzEaMBgGA1UEAxMRR29sZGVuRnJvZy1JbmMgQ0ExIzAhBgkqhkiG
9w0BCQEWFGFkbWluQGdvbGRlbmZyb2cuY29tggkA13ZTC3tJpuwwDAYDVR0TBAUw
AwEB/zANBgkqhkiG9w0BAQUFAAOCAQEAwihrN0QNE19RRvGywBvsYDmzmM5G8ta5
8yB+02Mzbm0KuVxnPJaoVy4L4WocAnqLeKfmpYWUid1MPwDPtwtQ00U7QmRBRNLU
hS6Bth1wXtuDvkRoHgymSvg1+wonJNpv/VquNgwt7XbC9oOjVEd9lbUd+ttxzboI
8P1ci6+I861PylA0DOv9j5bbn1oE0hP8wDv3bTklEa612zzEVnnfgw+ErVnkrnk8
8fTiv6NZtHgUOllMq7ymlV7ut+BPp20rjBdOCNn2Q7dNCKIkI45qkwHtXjzFXIxz
Gq3tLVeC54g7XZIc7X0S9avgAE7h9SuRYmsSzvLTtiP1obMCHB5ebQ==
-----END CERTIFICATE-----
</ca>

Adding openvpn node to interfaces

This is the only required change to the config.gateway.json file neccessary to test the OpenVPN client on the USG. Naturally the configuration file must be in place and correct for this to work.

If your config.gateway.json file was empty before this, this is the only content you need. Make sure it is valid json.

{
"interfaces": {
"openvpn": {
"vtun0": {
"description": "Giganews OpenVPN",
"config-file": "/config/openvpn/gateway.ovpn"
}
}
}
}

Verifying

ssh admin@10.0.1.1
show interfaces

Overview of the changes to config.gateway.json

My existing config.gateway.json already contains configuration to host names, in order to ease initial setup and point static host names to my specific subnet and ip (setup.ubnt.com -> USG ip). I do L3 adoption in additon to running inside Docker, which is problematic in the initial setup process for a USG.

Interfaces

We will add a new interface, a openvpn node and specifically vtun0 interface. We could add the configuration inside this node, but it is easier to just point out a specific configuration file which lives on the USG. This way we can make changes and updates and not need to reprovision anything.

We also need to specify which modify rule on our existing LAN configuration, which allows us to reroute all requests from inside our LAN which comes from specific ip addresses.

Firewall

In the firewall node we add a named group, which we will dynamically populate with ip addresses we want to reroute, thanks to the UI from Detour. If you don’t use Detour, you can instead add specific addresses or a range.

We also specify the actual named modify logic, which reroutes based on source ip.

Protocols

The protocols section contains the actual routing information used by the above modify firewall rule. Since we want to only reroute traffic for some devices, we define a separate interface-route to use in those cases. All other traffic is using the default routing and gateway.

Service

The service node contains the masquerade nat rule for the VPN. This magically solves how traffic sent out from the VPN makes it back through to us.

The config.gateway.json configuration

The USG is not allowing an empty value for the firewall group address-group node, so I’ve put an initial value there to get passed this bug. This may change in future firmware upgrades. If the configuration cannot be set on the USG, the controller provision process goes into an endless loop of trying to applying the configuration.

eth1 is my LAN port

The nat rule can actually be done via UI, but I’m including it here to keep all settings in one place. Make sure all keys and index number are free to use and adjust if neccessary.

{
"firewall": {
"group": {
"address-group": {
"vypr_nyc": {
"address": [
"10.0.1.254"
]
}
}
},
"modify": {
"detour": {
"rule": {
"10": {
"action": "modify",
"description": "Detour route to Giganews/Vypr VPN",
"modify": {
"table": "1"
},
"source": {
"group": {
"address-group": "vypr_nyc"
}
}
}
}
}
}
},
"interfaces": {
"ethernet": {
"eth1": {
"firewall": {
"in": {
"modify": "detour"
}
}
}
},
"openvpn": {
"vtun0": {
"description": "Giganews OpenVPN",
"config-file": "/config/openvpn/gateway.ovpn"
}
}
},
"protocols": {
"static": {
"table": {
"1": {
"interface-route": {
"0.0.0.0/0": {
"next-hop-interface": {
"vtun0": "''"
}
}
}
}
}
}
},
"service": {
"nat": {
"rule": {
"5004": {
"description": "Masquerade for Giganews OpenVPN",
"outbound-interface": "vtun0",
"type": "masquerade"
}
}
}
}
}

Restart OpenVPN client

Sometimes you need to restart the OpenVPN client, which can be done by disabling and then enabling the interface like this. The commit statements are key here which will bounce the interface.

configure
set interfaces openvpn vtun0 disable
commit
delete interfaces openvpn vtun0 disable
commit

Quick speedtest result

A quick speed test gives the following: Laptop -> Wifi -> USG -> VPN -> NYC USA endpoint -> Stockholm Sweden and roundtrip.