From the water to the web: Collecting maritime data for use in the Deckee app
The startup I co-founded, Deckee, is often pegged as ‘Waze of the Waterways’ and in much the same way that Waze shows other user cars within their interface, Deckee shows recent vessel positions on our map.
Unlike Waze though, we gather data on vessel positions through sources other than the actual Deckee app.
The marine industry uses a tracking system called ‘automatic identification system’ (AIS) which uses transponders on vessels and works over VHF. AIS has a number of use cases but is mostly relied upon for collision avoidance.
The data that is transmitted in AIS is quite descriptive and can contain the vessels category, speed, heading, destination, estimated time of arrival and a good number of other data points regarding the vessel including it’s size and the location of the transponder on the vessel.
The VHF signal is digital in nature (NMEA to be more precise) and a vessel will transmit a packet every few minutes. This packet can be picked up by any AIS receiver and looks like the following in a type-safe form:
!AIVDM,1,1,,B,403Oti1v>oP91oAucHEfNeO02@0t,0*3F
From this particular encoded packet, the following information can be decoded (decoding courtesy of the tool available at Aggsoft):
As AIS uses VHF, it means that this data can be collected by anyone with the correct equipment. As is often the case with any technology which operates in the open in a way such as this, there are two sides of the industry. There are groups who try and monetize the collected data and those who get involved due to personal interest and learning in an open source manner.
One such initiative in the later of these two categories is a AIS data sharing program called AisHub. Deckee is a proud contributor to AisHub and they make available a fantastic Raspberry Pi image called rPiAIS which makes the collection of local AIS data from VHF quite a simple task.
After writing an SD card with the Pi image, the initial image defaults to the network DHCP for it’s IP allocation. I typically run devices on static IPs on my local network so I can access the web interface reliably, so for rPiAIS, I needed to change the IP to a static allocation.
The image runs on debian linux as it’s based on Raspbian with a number of changes. This meant
changing the network configuration was a little different to normal Raspbian with the configuration
existing in /etc/systemd/network/20-eth0.network. The following changes allocated the IP to
a reserved static IP on my local network so that I can always find the web interface quickly and
reliably. I also changed the DNS to Cloudflare’s DNS while I was at it as my local DNS can flake
out sometimes causing issues.
[Match]
Name=eth0
[Network]
DHCP=no
Address=10.0.0.254/24
Gateway=10.0.0.1
DNS=1.1.1.1
LinkLocalAddressing=ipv4The next step after getting the image up and running is setting up the AIS receiver. There are good number of devices and antennas that can used for this purpose, but in receiving AIS data from the airways and dispatching it to AisHub, there is a pretty strong diminishing return on the dollars spent. A higher number of cheaper receivers are better than a lower number of more sensitive receivers.
This involves adding a VHF/AIS antenna, an AIS receiver and obviously setting up the Raspberry PI in a way so that it can continue operating 24/7.
Once everything is setup and running, I can now visit the static IP that I set on the Raspberry PI
on port 8080 and log in using the default credentials of admin:admin.
The first step here is to select the receiver device and enable the dispatching of received data to AisHub via the Configuration page.
After being enabled, it is possible to see the log of all the AIS packets that have recently been received on the status page.
Once the dispatcher is up and running, the rPiAIS distribution also includes a map that allows you to view and explore the vessels in the local area that the receiver has received packets from.
The dispatcher sends this data to AisHub automatically, but you are also able to dispatch this data to any endpoint you wish. In effect, you could start you’re own private network of towers and not share the data with AisHub, but the use cases for taking this action are very limited.
AisHub share their data with a number of partners, and anyone adding data to the pool also has open access to the global data that is available. This helps Deckee with our global plans tremendously given we soon want to expand into the global market, but only have AIS receivers in the US and Australia. By being a part of an open conglomerate such as AisHub, we benefit by building a better service for our users, while also helping other companies to build better marine apps. This is definitely a case of ‘coopetition’ but we hope that by everyone helping out, we’re increasing safety of all mariners at a global scale, a core vision for Deckee as a company.
I’ll soon follow up with a second post on the more challenging part of this system, how we integrate with AisHub, transform their data to a more performant format for our purposes, how we handle the challenge of keeping this large body of real-time and highly ephemeral data up to date, and how we deliver it to users in under 100ms. Stay tuned!