Category : GPS and mapping

Trackserver v3.0 released

Almost 14 months after the last big update, Trackserver v3.0 was released today. Since this is quite a big update, with lots of new features and improvements mainly on the presentation side, I thought I’d spend another blog post on it.¬†If you don’t know what Trackserver is, you can read my initial blog post on it, and the update on v2.0 in December 2015. I will present some demos at the end of this post to give you a better idea.

First I will sum up some of the smaller changes. I will get to the big ones later.

  • Leaflet was updated from 0.7.7 to v1.0.3. This brings in all the great work that the creators of Leaflet have done for their 1.0 release in September 2016. For Trackserver, this mainly means performance improvements, not to mention the mere joy of having up-to-date dependencies ūüôā
  • Our own hacked version of Leaflet-omnivore was synchronized with version 0.3.4.
  • The PNG images that served as markers to indicate the start and the end of a track have been replaced by L.CircleMarker objects. These objects were already used for ‘points’ style tracks that were added in v2.2 and now also for the normal markers.
  • The infobar that’s available on live maps gained some extra tags: {userid}, {userlogin} and {displayname}, only the last of which is somewhat interesting, I guess…
  • Bugfixes!

All the tracks

Perhaps the biggest change is in the communication between the server side of Trackserver and the JavaScript that is responsible for creating the maps and drawing the tracks. Trackserver has since long distinguished between 3 basic types of tracks:

  1. Static tracks from the Trackserver database, referred to by their ID
  2. Live tracks from the Trackserver database, referred to by the word ‘live’
  3. External tracks in GPX or KML format, referred to by their URL

It was possible to mix these types, but in very limited ways. On top of that, each track, regardless of the type, was downloaded separately, one HTTP request per track. For maps with a lot of tracks, that wasn’t the best design, performance-wise. Both these shortcomings led to a new scheme for getting tracks from Trackserver.

  • First of all, it is now possible to mix all types of tracks in unlimited numbers. Just specify track=a,b,c user=@,x,y,z gpx="URL1 URL2" kml=URL3 and you get them all in one map. The ‘@’ in the user attribute, by the way, is a shortcut for your own username, so user=@ becomes a replacement for track=live, and the former is preferred as of Trackserver 3.0.
  • All the tracks that need to be downloaded from Trackserver are downloaded in a single HTTP request.
  • You can show multiple users’ live tracks in a single map. The live update feature can only ‘follow’ one of them, but the red markers that mark the current locations can be clicked to start following that particular track. The infobar will display the info for the track that is currently followed. On page load, the map will follow the first user that is listed in the user attribute.
  • The new track loading mechanism makes use of JavaScript promises, which are somewhat of a novelty (Chrome >= 49, Firefox >= 50, Edge >= 14, Safari >= 10, all except Chrome released late 2016. No version of MSIE supports them). A polyfill for this is included and loaded automatically to support older browsers. There are multiple Promise polyfills to choose from on the web, but I went for this one, by Taylor Hakes.

Don’t forget: if you want to display GPX or KML files, you are bound to the limitations of CORS.

Shortcode attributes

The [tsmap] shortcode gained some attributes for more control over the maps and how the tracks are displayed:

    • As explained above, the user attibute is now used to specify one or more users’ live maps. You need the ‘trackserver_publish’ capability to publish other people’s tracks. This capability is granted to administrators and editors by default.
    • The live attribute can be used to force enable or disable live-updates. For example, this can be used to turn any track (even an externally hosted GPX file!) into a live track.
    • The zoom attribute can be used for some control on the initial zoom factor when the track is first drawn. This is most useful with maps that have live tracks, because Trackserver would normally zoom in on the latest position in the track, rendering other tracks invisible without zooming out first. For live maps, the argument to the zoom attribute is absolute: what you set is what you get. For maps that have no live tracks, the behaviour is a bit different. By default, Trackserver chooses the zoom factor that makes the best fit for all tracks combined. In this case, the zoom attribute serves as an upper limit, a maximum zoom level, so you can use it to zoom out (but not in) the initial view.

Tracks with style

Trackserver already had some options to style your tracks: markers, color, weight, opacity and (since v2.2) points. However, these style options were per-map, rather than per-track. You would have all markers, or no markers at all. You could have really fat, purple lines for your tracks, but you would have them for all tracks.

Not any longer.

All the styling options now support comma-separated lists of values. Multiple values in such a list will be applied to the specified tracks in order. For example:

[tsmap track=1,2 color=red,#8400ff weight=1 points=n,y]

will draw two tracks on the map: ID 1 in a really thin red line and ID 2 in a collection of purpleish points. I think you get the idea. If less values than tracks are given, the last value is applied to all remaining tracks, so track=1,2,3,4,5 color=red,blue will give you one red track (ID 1) and four blue ones (IDs 2-5).

There is one thing to keep in mind though, when you specify multiple values. While track order will be preserved within each track type, different track types are evaluated in a specific order, and styling values are applied in that order too. The order is:

  1. Static tracks (track=a,b,c)
  2. Live user tracks (user=x,y,z)
  3. GPX tracks (gpx=…)
  4. KML tracks (kml=…)

Example: [tsmap gpx=/url/for/file.gpx user=jim track=10,99 color=red,blue,green,yellow]

In this case, the GPX track will be yellow, Jim’s live track will be green and tracks 10 and 99 will be red and blue respectively.

GPX downloads

Trackserver has a new shortcode: [tslink], perhaps not the most intuitive name. This shortcode produces a link, with which the specified tracks can be downloaded as a GPX file. Other formats are on the horizon, please open a feature request issue on Github if you need a specific format. [tslink] is used almost the same as [tsmap], except that it lacks all the styling attributes.

[tslink track=12,87,525 user=patrick]

will give you a link to a dynamically generated GPX file, containing tracks with IDs 12, 87 and 525, as well as Patrick’s latest track. There is also a class attribute that can be used for styling the resulting <a> element, and a format attribute whose only valid value is ‘gpx’ at this time.

What do you think?

If you use Trackserver, I would LOVE to know about it!! If you have problems with it, please open a support request or an issue in Github. If you are happy with it, please leave a review. And if you absolutely love it, please consider a small donation to support development. It will be much appreciated!

Demo time

A bigger collection of demos can be found on this dedicated demo page, but here are just a couple of them to give you an idea:

[tsmap user=trackserver1]:


[tsmap track=564,575,656,657,658,625,627,628,629,622,623,624,619,618,620,621,647,630,646,648,653,655 color=black,blue,red,green,#8400ff weight=2 continuous=y opacity=1]


Trackserver v2.0 released

This evening, I released Trackserver version 2.0. If you don’t know what Trackserver is, please read my introductory post.

The v2.0 update contains many changes and some interesting new features:

  • It is now possible to add multiple tracks to a single map, by giving a comma-separated list of track IDs to the track parameter of the [tsmap] shortcode. You can also mix static tracks and live tracking in a single map, for example [tsmap track=12,84,live]
  • For maps with track=live, an information bar can be shown at the top of the map with some data about the latest track point. Add infobar=yes to the shortcode parameters. The content of the infobar can be formatted using a template that can be specified in the Trackserver user profile.
  • Experimental support for the SendLocation iOS app. This has not yet been tested with the actual app, but it works in theory. Please test this if you own an iOS device and a willing to spend the 99 cents for the app.
  • Upload via the WordPress admin and HTTP POST now accept GPX 1.0 files in addition to GPX 1.1. It seems that some modern software (most notably Viking) still creates GPX 1.0 files ūüôĀ
  • The Leaflet JavaScript library was updated to v0.7.7 (a minor update)

Some bugs were fixed too:

  • The track management page in the WP admin gained a lot of speed through some indexes on Trackserver’s database tables.
  • Fixed a bug in the handling of OsmAnd’s timestamps, that caused an integer overflow on 32-bit systems.

Under the hood, there were some changes too.

Trackserver is now capable of using GeoJSON, rather than Polyline encoding, for getting tracks from the server to display them on a map. However, the benefit of this is somewhat limited. GeoJSON is human-readable (sort-of) but it is also a multitude bigger than Polyline, so for performance reasons, the default is still Polyline.

Trackserver has always tried to determine whether its JavaScript files (including those belonging to Leaflet and its plugins) are necessary on the current page, and to not load them if they are not. It appeared that there are quite a few possible ways in which this detection mechanism could fail, and maps could not be displayed even though they should. Version 2.0 has two ways to mitigate this problem.

First, there is the new [tsscripts] shortcode, that forces Trackserver to load its scripts and CSS, even though the [tsmap] shortcode is not detected. Use this if all else fails. Second, there is an alternative detection algorithm, that can detect the use of the [tsmap] shortcode much more reliably (should be 100%) but has the disadvantage that Trackserver’s CSS cannot be loaded in the <head> of the HTML document anymore. So, neither solution is perfect and that’s why they’re both there.

The technical background story to this problem is, that some WordPress plugins and themes use custom WP_Query objects. This means that the actual list of posts to be displayed can be totally different than what WordPress initally thinks it should be. The initial shortcode detection can only look at the initial query, so any changes that add or remove posts from the query will surely confuse the detection algorithm. The alternative detection just uses the actual shortcode handler to initiate the inclusion of Trackserver’s JavaScript and CSS, but since this handler runs during the rendering of the page, long after the <head> section is printed, the CSS is loaded very late in the document. I am not sure whether this is a big problem, but opinions seem to differ on the subject, and loading CSS in the <head> is still best practice, so Trackserver will try to do that whenever possible.

Please test Trackserver v2.0. If you find any problems, please open a support ticket on the plugin page, or open an issue on Github.

And here’s a map for you ūüôā

Introducing Trackserver WordPress plugin

A while ago, I wrote a post titled “Self-hosted live online GPS tracking with Android“, in which I evaluated a few Android apps that can be used for, you guessed it, live online location tracking. The idea is, that you have your phone periodicially send your location to a web service while you’re on the go. The web service stores the location updates and optionally publishes them on a map.

While there are some apps that do this, there aren’t many (free) options for hosting your own server. That’s why I decided to write my own, and do it as a free and open source WordPress plugin, to make it as accessible as possible. Please meet Trackserver.

Trackserver is designed to work with TrackMe and OruxMaps. OsmAnd support is on the roadmap and I’m thinking about adding GpsGate support. All of these apps offer the possibility to periodically post location updates to a web URL that you can specify. Your own WordPress blog with the Trackserver plugin can ‘listen’ on such a URL to receive the updates and store them in a database. The plugin also offers a WordPress shortcode to include maps with your GPS tracks in your posts and pages, using the fantastic Leaflet library. Collecting and publishing your tracks has never been easier, at least not in a self-hosted website!

A short list of Trackserver’s features:

  • Collect location updates using several mobile apps
  • Publish your collected tracks on a map in your posts and pages
  • Upload tracks via an upload form in the WordPress admin backend
  • Upload tracks by sharing a GPX file from your mobile, using a sharing app like Autoshare
  • View, edit, delete and merge your tracks in the WordPress admin backend
  • Live tracking of your location, with automatic updates of the map view in your WordPress post
  • Full-screen map views

Trackserver’s WordPress admin allows you to upload GPX files to add new tracks, view, delete and merge tracks and edit their properties (name, comment, etc.). Please take a look at the plugin page at wordpress.org or the source code repository at Github. Both pages have more information on Trackserver’s features. The current version is 1.0, which has all the basic features described above.

Getting started with Trackserver is quite easy:

  1. Install the plugin in your WordPress site
  2. Install TrackeMe and/or OruxMaps on your Android phone
  3. Configure the app(s) with your WordPress URL, username and password (HowTo with screenshots is included in the plugin)
  4. Start tracking!

Feel free to give it a go, and if you do, let me know what you think!

 

Self-hosted live online GPS tracking with Android

Since long I have been interested in live online location tracking using my Android phone and I have been using a collection of software components that makes this possible for a while. The other day, I was discussing the subject with a friend who is looking for a similar solution and has been using OruxMaps for offline tracking. I was unfamiliar with OruxMaps, but it has a zillion features and it looks great in general. It triggered me to do a short survey of the Android landscape, since it appears that more apps have popped up since I last looked.

The solutions I am interested in should be able to do live online tracking to a self-hosted location, meaning the Android app should offer the possibility to enter a custom hostname or URL. Hosted services, like the once popular but now deceased Google Latitude or InstaMapper are not interesting to me. My location data is mine and no one else’s, and I would like to decide how I consume the data and if/how I share my location. This post is categorized in ‘Nerd Stuff‘ for a reason ūüôā

So I am looking for:

  • a Android app,
  • open source, if possible,
  • with support for a self-hosted server,
  • AND a matching a server component that can put the location data in a database,
  • OR an open protocol specification or a protocol that is easy to reverse-engineer, so I can build the server component myself.

 

TrackMe

Reviewed version: 1.11.1 (October 2014)TrackMe

TrackMe is available for Windows Phone as well as Android. I have been using it for quite a while. It is robust and offers some interesting features beyond tracking. It also offers a server component written in PHP, which uses a MySQL database for storage. TrackMe is ad-supported, but ads can be removed for¬†‚ā¨2.24. It supports offline as well as online tracking.

The strong points of TrackMe are:

  • Online tracking when possible (working internet connection), offline tracking and uploading when possible, manually or automatically. It just works.
  • Location updates are grouped in trips.
  • Extensive configuration options, allowing for trade-off decisions between accuracy and battery use.
  • It can be configured to start tracking as soon as the app starts or even when your phone boots.
  • Server component ready to go, using¬†PHP/MySQL. It has a web front-end that plots tracks on a Google Map.
  • Remote control of the app (start/stop tracking, report current location) via SMS
  • Hackable: the online tracking protocol is simple HTTP GET requests, writing your own server is easy.

The drawbacks:

  • Not open source, ad supported, user must dismiss a pop-up offering ad-removal every time the app starts.
  • The Android app does not have a map view, it only displays an activity log.
  • PHP server code is of bad quality and mostly¬†written in 2008. Latest modifications in 2012. Only MySQL is supported as a backend.
  • PHP web front-end uses Google Maps. I prefer OpenStreetMap.
  • I couldn’t get HTTPS¬†to work in the app (‘SSLPeerUnverifiedException: No peer certificate’, even though the server’s SSL certificate is fine).
  • Android app is pretty ugly.
  • Development of the app is really slow.

The bottom line: TrackMe as an Android app would have everything I wanted in a tracking app, if it were a little prettier, if SSL worked and if it were completely free. A map view within the app would be nice, but I can live without it. The PHP server works and the web interface looks alright, but the code is an unmaintainable pile of crap.

Home page: http://www.luisespinosa.com/trackme_eng.html

Play store: https://play.google.com/store/apps/details?id=LEM.TrackMe

OruxMaps

Reviewed version: 6.0.4 (December 2014)Orux Maps

Like I said, I never heard of OruxMaps before yesterday, which is a real shame, because it is the most versatile mapping app I have seen so far. Version 1.0 was released in 2009, so it’s not exactly a new kid on the block either. OruxMaps is free to use and free of ads, but not open source as far as I can tell. OruxMaps is a map-viewer and track-logger with many features, its tagline is “Travel & sport companion“. Just to name a few features that I find most useful:

  • it supports both online and offline maps (in a variety of formats)
  • it can do offline tracking and export to several file formats
  • it can upload tracks to many different services like EveryTrail¬†and ikiMap¬†(many Spanish services, due to the app’s Spanish origin)
  • it can do live online tracking via¬†Map My Tracks, GpsGate and OkMap
  • it supports routes and waypoints, with photos and videos
  • it supports all sorts of external devices, like GPS receivers, ANT+ devices, bluetooth heart rate monitors and so on

and the list goes on and on. This app clearly gets a lot of love from its developer(s?), and I really like that a lot of attention is given to the looks of the app. The icons, the layout of the screen, the menus, even the logo… the whole user experience is excellent, and it seems like every little detail is customizable.

So, I guess I’ll write another post about OruxMaps later, but right now, I’ll focus on the live online tracking features.

Like I said above, live online tracking is supported via 3 different methods, all of which have the option of a self-hosted server solution. The most important one, from Orux’s point of view, is Map My Tracks. This is a service, focused on tracking outdoor activities. It has an open, documented API and OruxMaps offers a setting to override the ‘live tracking’ URL. This makes it possible to implement your own server. Communication takes place over HTTP POST. Map My Tracks uses ‘activities’ to separate tracks, so I think everything we need is here, even though there seem to be some unresolved bugs.

Another interesting option is Orux Maps’ GpsGate support. GpsGate is a commercial product, but the server is free to use up to 5 users. The server runs on Windows, so it may or may not be for you (it’s not for me ;-)), but the GpsGate Server Protocol v2.0 is documented, so implementing a simple server of your own should be a viable option. However, my impression is that the GpsGate support is not widely used in the OruxMaps community, so I can’t say how well it works.

And finally, there’s the OkMap client. OkMap is a Windows desktop application and apparently (I have not tested this in any way), it can listen on a port to which live location updates can be sent. I haven’t found any reference material regarding the used protocol, but I’m pretty sure it isn’t HTTP. Reverse-engineering the protocol and implementing your own server may be a challenge, and with the options of Map My Tracks and GpsGate being there, I don’t see any reason to try.

To me, it seems that Map My Tracks is the easiest choice here, if you were to create your own protocol implementation. I did do a quick search for any Map My Tracks servers that may already be out there, but I didn’t find any.

Home page: http://www.oruxmaps.com/

Play store: https://play.google.com/store/apps/details?id=com.orux.oruxmaps 

OwnTracks

OwnTracksReviewed version: March 2014

OwnTracks (formerly MQTTitude) is a set of open source tools, among which an Android app and one for iOS. The project was started when (and because) Google Latitude was discontinued. Its main purpose is to keep track of your and your friends’ location by showing their picture or a marker on the map. As such, it does not have a notion of ‘trips’ or something like that and location updates are generally infrequent (> 1 minute intervals). Even though the wiki mentions something called ‘Move mode’, which is supposedly meant for actual movement tracking, this feature is not available in the Android app at the time of writing. OwnTracks is designed to be ‘always on’ and cannot be stopped from within the app. All in all, OwnTracks is not really suitable (or meant) as a travel companion.

Another thing that makes¬†OwnTracks a bit of an outsider (but all the more interesting, one might argue), is the fact that it uses MQTT rather than HTTP for communication. This choice was made because MQTT is a really great fit for messages like location updates¬†and it supposedly has great benefits for your phone’s battery life. To use OwnTracks, you need an MQTT broker like Mosquitto. You could also use a public broker, but that kind of defeats the purpose of choosing a self-hosted solution (privacy). Mosquitto is not difficult to set ¬†up, but you would need a place to run it and a standard shared hosting account probably won’t cut it.

Mosquitto is merely a message broker and doesn’t store anything by itself. For that purpose, OwnTracks has a backend script (m2s) that subscribes to the topics that the app publishes to, and stores the updates in a database.

Home page: http://owntracks.org/

Play store: https://play.google.com/store/apps/details?id=st.alr.mqttitude

OsmAnd

Reviewed version: 1.9.4 (December 2014)OsmAnd

OsmAnd is a different beast, because tracking is not its core competence. OsmAnd is a maps and navigation app, that supports offline maps. The reason to list it here is that it can be used for live online tracking, and since you may already use it for navigation, you might as well use it for tracking.

OsmAnd supports tracking through custom web services. Once the tracking functionality is enabled through the plugin manager, a URL can be set and the location is logged on a configurable interval using a simple HTTP GET. HTTPS does not seem to work; my server did not receive any updates when using a HTTPS URL, but no errors were reported in the app. Once the tracking settings are in order, you can start tracking (both offline and online) with a single tap on the screen. OsmAnd using a custom web URL does not use the concept of trips or something like that, it just sends location updates without any context.

Some notes:

  • Tracking settings can be set in 3 different profiles, depending on the type of transport (car / bike / foot).
  • Local (offline) tracking interval can be set independently from online tracking interval, so you could for example send infrequent online updates, while still keeping a detailed local log.
  • OsmAnd also supports¬†OsMo, a Russian service linked to the open source OsMoDroid¬†GPS tracker for Android¬†(Github), but I did not test that.

Home page: http://osmand.net/

Play store: https://play.google.com/store/apps/details?id=net.osmand

 

 

Other solutions

Even though there are plenty of GPS trackers on the market these days, many of which support some kind of export to online services, or even live updates, only very few have the option of a self-hosted server. I guess one of the reasons is, that most apps that support live updates are designed for a specific service (examples: Glympse, InstaMapper, LocaToWeb, guibber), where monetizing the users is the main goal. Having an open API or supporting self-hosted solutions is not in their interest. As a matter of fact, I haven’t been able to find any other solution in addition to the ones discussed above.

UPDATE 10 february 2015:

The other day, I found Gps Tracker for WordPress, which combines a WordPress plugin and an Android app to do more or less what I am trying to achieve here, except it focusses on tracking of multiple people/devices in a single map view. The Android app seems to send at most one location update per minute, and as such it’s not very suitable for sports and activity tracking. The map view in the WordPress plugin also doesn’t really draw a track (line), but merely a collection of waypoints that form the location history of the person/device you’re tracking. This app is best compared to Owntracks (reviewed above), because its goals seem similar.

(end update)

There is just one more possibility I can think of, but it’s not very practical and it requires a rooted phone. Just follow this five-step program:

  1. take one of the apps that uses a hosted service for live tracking
  2. find out the hostname / URL to which live updates are posted
  3. divert the hostname of the service to your own server via /etc/hosts (requires root on Android)
  4. reverse-engineer the protocol
  5. enjoy!

If you have other ideas, or apps to add to this page, please please please let me know!

Post-processing Garmin Zumo track logs

Garmin Zumo 550

Whenever I leave the house to do something interesting (riding my motorcycle, snowboarding, hiking) I take my Garmin Zumo 550 with me, to log my tracks if not for anything else.
The great thing about a device like the Zumo is, that it always logs the GPS location, there is no need to turn that on specifically, so you can’t forget it either. So, if you like taking photographs, and you make sure you have a GPS log, you can easily tag your photos with a location, for example with my Taggert software.

Unfortunately, the Zumo suffers from some annoying bugs, that make using its track logs a little less efficient. As you can read on the linked page, the Zumo creates track archives in GPX files, but sometimes it duplicates data and sometimes it partially duplicates data. Sometimes it duplicates data across different files, but in the worst case, it happens that data is partially duplicated within the same file. This means that you end up with a GPX file, that contains two equally named tracks, one of which is complete and the other one is not. Nice, huh?

The amount of tracks in a single archive file and the timespan they were recorded in are not really predictable. I guess it mostly depends on the total number of track points in all tracks together. For my own purposes, it would be most convenient to have all tracks from a single day together in a single file. And of course, there should be no duplicate tracks and the tracks that remain should be the complete ones.

To post-process the Zumo’s track logs, I created some Python scripts. The README on Github pretty much explains what they do in detail, but in short, they do what is necessary to get what I want:

  • One file per day, containing all tracks for that day
  • Only the complete tracks are kept, (incomplete) duplicates are discarded

The scripts are written in -and tested on- Python 2.7 on Linux, but since they don’t have any external dependencies, I don’t see why they shouldn’t work on Windows or other platforms as well. They are released under the Apache 2.0 license. Please use them as you see fit.

1 2