Running an IPFS node on FreeNAS

IPFS, the “InterPlanetary File System” is a interesting new protocol for hosting files online in a distributed topology that’s resistant against the natural churn of data being lost, blocked or deleted over time. Ipfs-go v0.4.7 is available on the FreeNAS 11 although this is not exposed in the Web UI. This allows you to view and pin IPFS content using your NAS storage.

This guide is on how to configure and run IPFS on FreeNAS to serve files to the IPFS network and provide a HTTP gateway to local network users.

Create IPFS dataset

Create a dataset for the ipfs data, in my case I chose tank/ipfs mounted at /mnt/tank/ipfs then initialise it for use by ipfs-go.

[root@freenas]# zfs create tank/ipfs
[root@freenas]# export IPFS_PATH=/mnt/tank/ipfs
[root@freenas]# ipfs-go init

Configure daemon start at boot

Create two tunables with the following settings:



Configure IPFS to be network reachable

By default IPFS will be accessible only to the local machine, as a server we want FreeNAS to serve other machines on the network, to allow this we need to edit the config file.

Edit /mnt/tank/ipfs/config and find the following lines:

 "API": "/ip4/",
 "Gateway": "/ip4/"

Change to read:

 "API": "/ip4/",
 "Gateway": "/ip4/"

Start and test IPFS

IPFS will already work from the CLI but as a lot of IPFS content is static web sites we want to run the daemon to present that over HTTP.

[root@freenas# service ipfs-go start

Open http://freenas:8080/ipfs/QmYwAPJzv5CZsnA625s3Xf2nemtYgPpHdWEz79ojWnPbdG/readme

You should see the built-in readme file indicating that the system is working.

During early testing I have found the ipfs-go daemon to be somewhat unstable and sometimes needing a restart. I have not yet found the cause or any fix for this yet.

There is a Web UI at http://freenas:5001/webui/ that is supposed to show the status of connections to peers but this is broken on my system showing pages with no peer data.

Expand root zpool while online

When the physical zvol has increased in capacity, ZFS should expand to use all the extra space. However this usually requires a re-mount which also means a reboot if your zpool is used for the root filesystem.

There is a command I found that works around this limitation, usually used for bringing new disks into a running zpool.

zpool online -e zroot <disk>

This should cause the zpool to recognise the new storage area immediately.

A re-mount may be a safer way to do this however if that is possible and a reboot is not so much of a problem.

Running pfSense as a VM guest on FreeNAS 9.10 host using Bhyve

Update 31/03/2016: FreeNAS 9.10 is now marked STABLE and 9.3 is in maintenance mode until the version 10 release. There are issues to be aware of though, I have updated the post below, new information is in italics.

Update 15/06/2016: I have tested and started using the script that allows guest VMs to be successfully rebooted and shutdown from inside the guest OS.

IX Systems recently announced the FreeNAS 9.10-nightly train, an unsupported a version of FreeNAS 9 using a FreeBSD 10 base. This allows users to make use of FreeBSD 10 features including Bhyve, the FreeBSD virtualisation module. I have tested this out to run pfSense firewall as a guest on my HP Microserver running FreeNAS. It was a very experimental process but I managed to get a working configuration which I’ve documented below for my own reference and for others trying to do a similar thing.

The information here is not a step-by-step guide. It involves diving under the hood of the  FreeNAS Web UI and there is no official support available from IX Systems, the FreeNAS and FreeBSD communities may also struggle to help you, so I am assuming you have a decent enough understanding of computer networking and FreeBSD’s networking stack to muddle your way through problems you will likely encounter.

If you follow the information here, there is no warranty, I am not liable if it deletes your data, gets you hacked, burns your house down or anything else. If you follow the information contained here you do so entirely at your own risk.


My home server is a HP Microserver N36L, with 8GB ECC RAM, 2xTB WD Red hard disks in ZFS RAID1 (mirrored).

I am using an Intel PRO/1000 PCI-e network interface card instead of the on-board HP/Broadcom NIC, mainly because of VLAN issues I had with the on-board NIC. Most people won’t use VLANs, just replace the VLAN interfaces with the appropriate physical interface name in your own config.

To do this, you must have the following:

  • A CPU with support for virtualisation.
  • Two network cards, or a switch with 802.1q VLAN support.
  • A ZFS zpool, UFS probably will work but will involve some significant differences.
  • At least 8GB RAM, preferably more, that is the minimum for FreeNAS and some of it will be dedicated to pfSense’s use.
  • A 16GB+ USB flash drive for FreeNAS.

Preparing the Images


Download FreeNAS 9.3 from their website, follow the official instructions to install it to your USB flash drive.

Boot it on your system and then use the System Update menu to switch to the 9.10-Nightlies 9.10-STABLE train, upgrade the system and reboot into FreeNAS 9.10.


Download the 4GB embedded image of pfSense from their website, this is not the normal installation image, the file name is:


Download this onto your freshly installed FreeNAS system, unzip the file using gunzip.

On FreeNAS Web UI create a 4GB zvol, call it pfSense.

Copy the embedded image to the zvol using ‘dd’, my zpool is called “tank”, change this in the command below to the name of your zpool.

dd if=pfSense-CE-2.3.1-RELEASE-4g-amd64-nanobsd.img of=/dev/zvol/tank/pfSense bs=1M


Go to system > tunables in the Web UI, these are the settings you will need, some are not applicable in all situations so read the descriptions here.

Variable cloned_interfaces
Value bridge0 bridge1 tap0 tap1
Type rc

This creates the virtual interfaces we need at boot time.

Variable ifconfig_bridge0
Value addm tap0 addm vlan1 up
Type rc

This configures and brings up bridge0, adding the two interfaces tap0 and vlan1 to it, replace “vlan1” with your LAN interface name.

Variable ifconfig_bridge1
Value addm tap1 addm vlan100 up
Type rc

As above, this configures the WAN bridge. Replace “vlan100” with your WAN interface name.

Value 0
Type Sysctl

If you are not using PPPoE you may exclude this. If you are using PPPoE, this is required to prevent the packet filter from blocking anything that’s not IP leaving the bridge.

Value 1
Type Sysctl

This tells FreeBSD to enable (bring up) the tap interfaces when they are opened by pfSense.

Variable nmdm_load
Value YES
Type loader

This loads the kernel module for nmdm, which is used for gaining serial console access to pfSense locally, this is necessary because Bhyve does not support VGA console access.

Variable vmm_load
Value YES
Type loader

This loads the VMM module for virtualisation support in the kernel.

You can test this first by running ‘kldload vmm’ from the CLI, then look at the output from dmesg, if you have errors then your CPU might not support VM extensions, or they may be disabled in the BIOS.


I have configured my VLAN interfaces through FreeNAS’s network interface menu. You should configure your LAN/WAN interfaces through the networks menu too.

While it’s possible to configure interfaces in the tunables section, if you do so FreeNAS will try to DHCP configure all available interfaces, including bridges, which is undesirable. It must have at least one interface manually configured to prevent this, even if that is just to bring up an interface and nothing more.

The WAN-side interface, VLAN or physical, should be configured to come up (options=up), but should not have an IP address on it. You don’t need it, you’re more secure without it.

My LAN-side interface has the FreeNAS IP address, this goes against FreeBSD best practice advice which is to put the IP address on the bridge. By the time I got this into a working state I didn’t want to break things again by making this change, it probably would work by only bringing up the interface in FreeNAS network configuration and then using tunables to add the IP to the ifconfig_bridge0 entry. edit: When doing so using tunables however the IP address is not present when the jails come up, this causes jails to join an isolated bridge network disconnected from the LAN, this needs further testing.

Boot scripts

On the FreeNAS Web UI, go to Tasks then Init/Shutdown Scripts.

Create a new entry and put the following:

type command
command sh /usr/share/examples/bhyve/ -c 2 -m 384 -C /dev/nmdm0A -d /dev/zvol/tank/pfSense -t tap0 -t tap1 pfsense &
when postinit

This command starts pfSense with 384MB RAM and two network interfaces, it can be adjusted to suit your own requirements.

384MB is about the minimum amount of RAM I’ve found necessary on a minimal pfSense 2.3 installation, this was increased from 256MB on version 2.2, obviously the more services you run the more RAM you will need to allocate but keep it as small as possible so not to take too much RAM away from the host.

A serial console is installed on nmdm0, the number can be adjusted if this is in use.

You must end the command with an ampersand (&) so that the command is put into the background and does not hang the VGA console.

Accessing pfSense

Reboot FreeNAS to cause it to boot pfSense as it should do.

To access pfSense, as root on FreeNAS run the command:

cu -l /dev/nmdm0B

As it is a serial console, you will need to tap return to see the menu. You can then configure pfSense’s interfaces in order to access it’s Web UI.

Be careful to check that you have your pfSense vtnet interfaces matched to the correct tap/bridge interface on FreeNAS and so aren’t exposing your unconfigured pfSense box to the Internet, that would be very bad.

Try a reboot, you should see it shutdown and come back up if your loop is properly configured.

To exit the serial console type tilde+fullstop (~.) and you should drop to FreeNAS. If accessing FreeNAS remotely this may also drop you to the local terminal.

In most cases you will just use SSH to manage pfSense but this provides a fail-safe.

How well does it work?

Quite well, pfSense is happy and performs well under Bhyve. I haven’t experienced any crashes or weird behaviour from pfSense.

The boot loop means pfSense cannot be totally shut down without rebooting itself, I’m not aware of a way to force this either, I’ll probably look into this in the near future but at the moment having it automatically reboot is a good thing. Edit: Reboot and shutdown from inside the guest now works successfully when using the script which responds appropriately to the bhyve exit code.

The network configuration is still a bit unstable, I enabled IPv6 on the LAN interface only to find once it restarted the interface configuration it managed to drop it out of bridge0. However, this only happens if you make configuration changes on a running system, which should almost never need to happen, just be careful when you do. The configuration will come up again properly on reboot.

FreeNAS 9.10 shows tap devices as configurable in the FreeNAS Network menu, however they are not usable by FreeNAS, so this is probably a bug owing to the fact FreeNAS doesn’t normally support tap interfaces, expect the unexpected when making modifications in tunables. Unfortunately bridge interfaces which are usable by FreeNAS, are not recognised as configurable and so can only be configured using tunables.

Bhyve is a new virtualisation system, it seems to work well but it isn’t as mature as other VM systems such as KVM or VMWare. It lacks support for features common in alternatives such as USB support or VGA consoles, it can run various BSD systems and Linux but it’s only really useful for servers, not graphical desktops.

FreeNAS 9.10 is a nightly train, I will periodically update it but I haven’t yet updated it since making the changes, I don’t expect it will cause any breakages but I cannot be sure yet. Edit: FreeNAS 9.10 is now out as STABLE and upgrade is recommended by iX Systems. I have now upgraded to the stable branch, this was successful and the configuration continued to function through the upgrade (woot!), there are several other issues with the stable branch that hit me such as HTTPS being broken, pkg is broken in newly created jails, iohyve does not work, but that’s another story… I personally would recommend waiting until at least a patch release in April before upgrading to STABLE for these reasons.

Iohyve is a Bhyve management tool, based on iocage which in-turn is a more advanced replacement for Warden. I haven’t tested iohyve to see how well it interacts with the manually configured bhyve jail for pfSense. I have some experience with iohyve and I know it would not be suitable for managing a pfSense guest due to it supporting only a single network bridge.

In hindsight

Doing this has been an interesting experience, I learned a lot from it and encountered a number of issues both limitations in my hardware, bugs in FreeNAS and quirks of FreeBSD’s network stack. I look forward to proper Bhyve support in FreeNAS 10 and I hope it comes with flexible support for bridged interfaces.

ownCloud unable to add new users – A user with that name already exists.

Using owncloud 8.2, when adding a new user it can error saying that the user already exists, regardless what username you put in.

owncloud user already exists

This turned out to be a problem with an App called “User backend using remote HTTP servers”, disabling this resolved this error for me.

If you are using the standard ownCloud authentication method then this App should not need to be enabled.

Flight tracking with RTL-SDR and Dump1090

One of the interesting uses for RTL Software Defined Radio is to set it up to track ADS-B  data sent from nearby aircraft, this information provides transponder information such as altitude, course, speed and the flight number.

To examine this data I am using a Linux utility called Dump1090, I compiled this from source using the current git HEAD at, there are other repositories but I hear this is the best. To start on an Ubuntu or derivative, install build-essential and git;

apt-get install build-essential git

Then clone the repository into a folder somewhere:

git clone

Enter the folder and compile the application:

cd dump1090

Now run the application with the following options:

./dump1090 --interactive --net

I was already running another server on dump1090’s default 8080 so I needed to specify an alternative port 10900

./dump1090 --interactive --net --net-http-port 10900

Now, open your browser and point to the address http://localhost:8080, replacing the 8080 with the alternative port chosen if you needed to.


The application will start finding nearby aircraft and plotting them on a Google map, you can select the planes  to look up their flight plan on web sites like flightaware and flightstats. I live near to Manchester International Airport so I could see a good few flights arriving and departing MAN that would show up as they leave and disappear as they left range somewhere off the coast of the Irish Sea.

RTL SDR frequency correction

Tuning the radio on an RTL SDR receiver, it’s very common to find the frequency read-out to be wildly inaccurate. To correct for this SDR applications request a PPM value which is unique to each RTL SDR USB dongle. To get your PPM value run the following command in the Linux CLI:

rtl_test -p

After several minutes you’ll have a read-out like the following:

Found 1 device(s):
0:  Realtek, RTL2838UHIDIR, SN: 00000001

Using device 0: Generic RTL2832U OEM
Found Rafael Micro R820T tuner
Supported gain values (29): 0.0 0.9 1.4 2.7 3.7 7.7 8.7 12.5 14.4 15.7 16.6 19.7 20.7 22.9 25.4 28.0 29.7 32.8 33.8 36.4 37.2 38.6 40.2 42.1 43.4 43.9 44.5 48.0 49.6
Sampling at 2048000 S/s.
Reporting PPM error measurement every 10 seconds...
Press ^C after a few minutes.
Reading samples in async mode...
lost at least 196 bytes
real sample rate: 2048184 current PPM: 90 cumulative PPM: 90
real sample rate: 2048151 current PPM: 74 cumulative PPM: 82
real sample rate: 2048198 current PPM: 97 cumulative PPM: 87
real sample rate: 2048152 current PPM: 74 cumulative PPM: 84
real sample rate: 2048186 current PPM: 91 cumulative PPM: 85
real sample rate: 2048166 current PPM: 82 cumulative PPM: 85
real sample rate: 2048161 current PPM: 79 cumulative PPM: 84
real sample rate: 2048189 current PPM: 93 cumulative PPM: 85
real sample rate: 2048170 current PPM: 83 cumulative PPM: 85
real sample rate: 2048165 current PPM: 81 cumulative PPM: 84
real sample rate: 2048188 current PPM: 92 cumulative PPM: 85
real sample rate: 2048162 current PPM: 80 cumulative PPM: 85
real sample rate: 2048180 current PPM: 88 cumulative PPM: 85
real sample rate: 2048174 current PPM: 85 cumulative PPM: 85
real sample rate: 2048161 current PPM: 79 cumulative PPM: 84
real sample rate: 2048182 current PPM: 89 cumulative PPM: 85
real sample rate: 2048183 current PPM: 90 cumulative PPM: 85
real sample rate: 2048153 current PPM: 75 cumulative PPM: 84
real sample rate: 2048179 current PPM: 88 cumulative PPM: 85
real sample rate: 2048184 current PPM: 90 cumulative PPM: 85
real sample rate: 2048165 current PPM: 81 cumulative PPM: 85
real sample rate: 2048178 current PPM: 87 cumulative PPM: 85
real sample rate: 2048177 current PPM: 87 cumulative PPM: 85
real sample rate: 2048166 current PPM: 81 cumulative PPM: 85
real sample rate: 2048195 current PPM: 95 cumulative PPM: 85

As you can see. the value averages out over time to give a stable reading. My USB dongle is off by 85 PPM so I’ll enter this to correct my frequency reading.

However I found afterwards that this still leaves me slightly off the mark, so using a graphical SDR application such as GQRX, tune to a known frequency then fine-tune the PPM value until the signal meets the tuning line in the middle. Typically I find mine is ~73 PPM using this method, this can vary by ~1-2 PPM but it’s enough to hit signals when tuned to the right frequency.

For a known frequency I suggest finding a local repeater, preferably on 70cm to 23cm for highest precision, but one that is most active is best. You might also use APRS which is always 144.8MHz in Europe, different frequencies in other regions but it’s reliable. Both these choices are NFM so you should hit them exactly in the centre of the broadcast when tuned correctly.
sm313 on YouTube tried using GSM mobile frequencies, this is a good choice because it’s a high frequency so has good precision and is constantly broadcasting, but you need to know what you’re looking for in a very wide-band signal so might not be that straight-forward.

If you have other suggestions please leave advice in the comments.

Reduce Pipelight/Netflix CPU usage on Intel HD graphics by disabling hardware acceleration

I have been trying to solve a problem viewing Netflix on my laptop. I was annoyed by the laptop fan running high while playing video making it difficult to hear the audio from Netflix.

The laptop has a current generation Intel Haswell i5 4340M CPU with Intel HD4600 graphics and I’m running Linux kernel 3.16 on Kubuntu 14.04 although this problem existed on kernel versions 3.14 and 3.15. I mention this because YMMV particularly with different GPU makes.

The solution I found is to disable GPU hardware acceleration on Pipelight, this drops the CPU usage from 30-40% to under 20% allowing the laptop to run cooler and use less power, meaning a quieter laptop and longer battery life.

Before, GPU acceleration on:


After, GPU acceleration off:


It is possible to disable hardware acceleration permanently using the instructions on the Pipelight website here, however I prefer to be more selective and have created myself an application launcher to start a new Firefox Window that opens Netflix immediately, the launcher command line is:


I would normally expect hardware acceleration to reduce CPU load because this hands off the video decoding work onto the GPU but the opposite appears to be happening here, I don’t know why this is the case, the CPU seems barely notice that it’s decoding 1080p video so it might be some magic built into integrated Intel chips but if anyone has any better theories I’d be interested to know!

Spam emails

On Sunday morning I started receiving a large number of mail delivery notifications for bounced spam emails being sent to random people using my email address as the return sender. I’ve had some emails from (justifiably) angry people about this. If you received one of these emails and followed the domain to end up here then please understand that I did not send these emails to you, nor did they come from any of my computers.

The reason this is happening is that email spammers will find a legitimate email address and use it to send spam from their networks of compromised computers called “botnets“, because the return address is legitimate this helps them bypass spam filters.

If you received one of these emails though, do not follow links they contain and do not open any attachments they might have, doing so will confirm your email address is valid and result in you receiving more spam and attachments often contain viruses that might recruit your PC into the botnet or worse. If you do reply to them it will be to me, not the spammer and I do not have anything to do with them other than that they are using my address without my permission.

I don’t know why they chose my email address, but I know it won’t last long before they will switch to another and carry on trying to make email just about useless for everyone, I just need to ride this one out and delete the backsplash.

10 years of Zen

Yesterday (1st March 2014) was a bit of a milestone for me as it marked 10 years since the fresh-faced 20-year-old David Nelson entered the glass doors of Zen Internet on his first day of what has proven to be a long and fruitful career there. This is a long time for most jobs today, although I’m not alone at Zen as there are a fair number of others in the 10+ club.

I suppose some could see this as career inertia but my experience has been anything but. Having started in broadband support my success there meant I was soon offered a job on the management track, and then later the business track doing process work and product development projects where I’ve spent much of the last 4 years utilising my intimate knowledge of the business to make things better, knowledge I developed having watched it grow from an entrepreneurial little upstart of 70 staff to the 500+ staff major employer it is today.

Recently though I felt dissatisfied with my lot having become progressively distanced from the what qualities I saw in Zen that motivated me to apply no less than 3 times for a job there. That is, developing expert technical skills and using it to deliver excellent customer service, something that Zen still prides itself on today. So 6 months ago I made a career pivot and started working on the managed service desk front line, this time with the goal of becoming an expert in IT security; a subject that has interested me for a long time after I discovered the “Security Now” podcast and took seriously as a career option following a chance conversation with a stranger on a train early last year.

Although my job today is little like my first job there (different tech, different customers, different expectations etc), there is a feeling of familiarity about it that does make me feel like I’ve come full circle, but not in a bad way. Progression is rarely a linear path upwards and sometimes the turns allow you to build an understanding of what is most important to you, and it is never a dead end or a “step-backwards” unless you make it is so.

Today, I know what I want to do, where I am going, how I need to get there, and I have 10 years of valuable experience that will help me achieve that. My plans are open-ended however, I am not so dead-set that I may be blind to good opportunities along the way, but I can be more selective about those that present themselves and decide whether they are right for me. Who knows, if the opportunities keep coming at Zen I could be writing again in 10 years time about a 20 year long career there, something that would be a very rare thing indeed!

The end of XP

Be honest now, are you still using Windows XP? There’s no need to be embarrassed if you are, you aren’t alone, 12 years after it was released Windows XP is still the 2nd most popular operating system in the world with 17% of the market share.

For many years now Microsoft have been encouraging their users onto newer Windows versions such as Vista, 7 and 8, but a lot of people are still very happy with Windows XP and are refusing to budge.

The recession is no doubt partly to blame, new computers are expensive and Windows XP works well on older hardware so why change? Another reason is Microsoft’s own variable release quality, Vista was highly criticised, Windows 7 fixed Vista’s issues and became very popular, but Microsoft moved quickly onto Windows 8 with its new touch interface and bold colours that has put many people off.

However, the problem we now face is that on April 8th 2014 Microsoft will stop supporting Windows XP and many millions of users will be left out in the cold, but what does this mean?

The second Tuesday of the month is marked on many IT professionals calendars as “patch Tuesday“. This is the day when new updates are published to fix bugs and security vulnerabilities recently discovered in Windows operating systems. April 8th 2014 is the last patch Tuesday that will include Windows XP, after this date new vulnerabilities discovered in Windows XP will never be fixed and your computer will be exposed to these vulnerabilities until you upgrade to a newer version of Windows.

What is worse is that black hat hackers watch Microsoft’s movements in the security world with great interest. Often Microsoft are responding to exploits discovered “in the wild”, other times these vulnerabilities are discovered by Microsoft first and so have limited effectiveness against supported operating systems because many will be quickly fixed. However hackers can now look at what is being fixed in Windows Vista/7/8 and you can be guaranteed they will be testing Windows XP to see whether it too is vulnerable.

So what can you do? Ultimately, you will need to upgrade from Windows XP, ideally before April 8th 2014 lands. In some cases this won’t be possible so if you are stuck with Windows XP for the time being then you must be extra vigilant against security threats

Keep your anti-virus updated, make sure you are using 3rd party anti-virus because Microsoft is also ending support for XP in Windows Security Essentials.

Be extra wary of using software from untrusted sources either via downloads or physical media such as USB pen drives. These can carry viruses exploiting XP’s growing number of unpatched vulnerabilities.

Finally, bear in mind that your computer is now overdue for an upgrade.