Did you know that you can navigate the posts by swiping left and right?

(Near) Realtime Maps with the Mapbox Uploader

02 Jul 2015 . Mapping . Comments #mapbox #alteryx

Header

My previous two posts on this topic have been in response to the (excellent) work of others. Personally, I really don’t like it when people just criticize without presenting an alternative - so here’s mine.

The Mapbox Uploader

What is it? - NodeJS + Mapbox Upload API + Alteryx

Don’t worry, I’ll explain what all those things are and how it all works. There is some code involved, but if you don’t want to write code, you don’t have to…but you really should try it, it’s fun!

My perspective on maps

They are all about context. The best maps simultaneously connect you with history and geography, both human and physical. They do so via a thoughtful combination of color, text, iconography, and the inclusion/exclusion of geographic features.

To put a city in a book, to put the world on one sheet of paper – maps are the most condensed humanized spaces of all…They make the landscape fit indoors, make us masters of sights we can’t see and spaces we can’t cover.” ― Robert Harbison, Eccentric Spaces

Tableau’s mapping capability permit none of these. If I’m going to expend significant effort to format and style a dashboard - I don’t want my map to drag it down.

Furthermore, while there are plenty of resources available to create polygonal copies of geographic shapes, I find these to be suboptimal for two reasons.

First, to get the detail that you may need, I may have to take up a large amount of rendering resources. If those shapes aren’t changing very quickly, that’s an unacceptable waste of computing.

Second, having to use up shelves just to get a shape means that I can’t use them for other things. That’s not an acceptable opportunity cost for what is essentially a static image.

So what does my ideal solution have to do? I have four basic requirements.

  • Full control of your basemaps: Style and content
  • Integration of custom geographies
  • Automation
  • Zero increase in workbook size

My solution: Mapbox + Alteryx

Why Mapbox?

Mapbox’s data and design platform gives you access to the entire world’s data and let’s you style it however you want. (#1 - check)

Design

Mapbox also provides you an API to push all your content to their cloud - in one line of code. It’s conveniently called Mapbox Upload. (#2 - check) Upload

Mapbox+Tableau integration is well-documented. (#4 - check) {UPDATE: Now it’s officially supported!!}

By itself, Mapbox can handle my data, but it can’t really do the automation. Upload requires a command line entry and a file location/name in order to work properly. Even if I scripted the process, it would never be fully automated because authoring would take place in an external tool.

But what if that tool could also package the shapes in a Mapbox-friendly format, and then fire the Upload command automatically? It so happens that my favorite data tool, Alteryx, can do all those things. It’s workflow-based UI makes it easy to automate processes, and one of its core-functions is the ability to integrate analysis (or attributes, in GIS-speak) with custom geographies. This means I can have an end-to-end solution for all my geographic data.

Let’s go through it all step-by-step. Everything I’m about to show is modular, so you can use any or all of it as it suits your needs.

How it all works

The Basics

  • Get yourself a Mapbox account. The free one is fine for now.
  • On your account page, you’ll see this section on the right-hand side

gettoken

  • Click on “Add or Revoke tokens”
  • Click “Create a New Token”
  • Add the following scopes

createtoken

  • WRITE IT DOWN - you won’t remember it by yourself.
  • Install NodeJS
  • Once you are done, open up your command line (Terminal for Mac, CMD for Windows) and type npm install -g mapbox-upload. You should see something like this.

Node

  • Here’s where it gets tricky - to make this all work, you need to set an Environmental Variable for your Mapbox Secret Token.

Setting Environmental Variables If you are on a PC (which we will be when we get to Alteryx), you type SET MapboxAccessToken=[access token with uploads:write scope enabled]. If you are on a Mac, you can just type export MapboxAccessToken=[access token with uploads:write scope enabled].

  • Now run the following: mapbox-upload [username].[dataset id] /path/to/files. i.e. mapbox-upload cmtoomey.test /Desktop/Archive.zip

Test

When that is all done - you can visit your Mapbox Data page and see if it worked. If so, you should see something like this

Data

Now you can use Anya A’Hearn’s guide to styling and integrating that new data with Tableau.

BUT…that was a lot of work. What if I have custom geography that I don’t just download? What if I don’t want to remember my token or open up the command-line? Can it be easier?

Enter Alteryx

For those who don’t know - Alteryx is a fantastic tool for managing all things data. It does data integration, geographic and statistical analysis, and can even run external scripts and programs as part of its workflows.

The process I described in the previous section is a pain. I want it to be automated, but I didn’t want to assume that a user would even know what NodeJS is or how to use it. It needs to be simple and work on anybody’s workstation.

Here is the basic workflow: Create some geographic data > paste in your token > username + id > RUN.


The Uploader Macro

Click here - download everything!

Download and open a workflow - drop in the Mapbox Uploader Macro and this is what you will see (Thanks to Mapbox for the icon!).

Intro

All the same things you needed for the previous steps (username, a mapid, Secret Key, and filename) - just copy and paste them in. Drop it at the end of any workflow with a geospatial component and you’ll be good to go.

Here’s how it works

There’s a really cool feature in Alteryx that lets you write and run scripts on the fly. Text Input Tool -> Run Command. Specify a Write Source and Alteryx will write out the file before doing anything else. Then you can run that same script. Normally we would use a SHP output (that’s what most people tend to have on hand). However, you have to ZIP those up and that’s a pain on Windows. So instead, the macro uses KML, but you can use GeoJSON too, if you want to snag the JsonCreateV2 macro from Alteryx.

Here’s what happens

  • Write out KML
  • Upload KML
  • Delete KML

All the scripts go to your temp directory, which gets cleared after every session, so no worries about duplicates.

The end result looks like this:

End

As for the Slalom Branding - this was all done on company time, in close collaboration with Mapbox and Alteryx. Will White, Matt Irwin, and Cameron Steele (Alteryx) helped get me setup and debug the workflow.

You are free to use the macro, but please do not remove the branding as the macro is technically property of Slalom. I’m happy to help with setup and some light debugging, but there is no implicit promise of anything even resembling actual support.

Ok - go forth and do good work! If you make something cool, tweet it to Alteryx and Mapbox!