Maps Garage - Raster data for your mobile maps

PAUL SAXMAN: Hello, and welcome to Google Maps Garage. I'm Paul Saxman. And with me this week is Josh "Raster Data" Livni. Today we're going to talk about getting raster data in your mobile map using the Google Maps Android API and some WMS services, or raster data services, which Josh is going to talk about. But what we mean about getting raster data in your map is, if we look at the tablet here, this is the Azavea Philly tree map. And you can see here, as we zoom in, this application actually has a lot of data in it. These are actually individual trees in the city of Philadelphia-- a lot more data here than you could actually do in markers, for example. So the way they're doing this is they're actually generating images on their server, providing these images to the application. The Android API is actually rendering them on top of the map. The really nice thing about this, as well, is that, via the API, these tiles and these images actually rotate and tilt, like you would expect them with the map. The way this works is what Josh is going to explain now. JOSH LIVNI: All right, thanks, Paul. Yeah, so I'm just going to go through three simple Chrome tabs to cover the general technique they used here. Let me go ahead and zoom in this blog post. So in addition to releasing this app recently, they also wrote some notes up on how they did it. The title, "WMS On Android," is kind of what we're going to walk through for the rest of the session. But first, as many of you know, our base map tiles and many tiles that you see on maps come in these 256-pixel squares that people send over. They align really nicely. And you often see them, perhaps, pre-rendered or available in some sort of URL and then an XYZ-- the x-axis, y-axis, and zoom level of the tiles. But another way to send images or rasters like this over the wire is using something called WMS, or Web Map Service, a standard supported by lots of large organizations, open source software, and what they're using here. PAUL SAXMAN: They support it, and they provide this data, as well, right? A lot of these companies-- or organizations, actually-- provide raster services. JOSH LIVNI: Yeah, absolutely. So you can use some open source software or software of your choice to create a kind of WMS service with your data. Or just go out on the web, which we'll talk about in a little bit, and see that there's lots of public data available for you in this format. So how does one get it into your Android tile provider? So this blog post kind of walks you through. I'm not going to go into it in great detail. You can read it for yourself on the Azavea blog. But let's go ahead and take a look at what is this WMS when you encounter it in the wild. So here's an example of the National Atlas, which has tons and tons of data layers available to you on US government created data sets, of which there's a lot. So here are some examples of some WMS services that we have available to us. I'm going to go ahead and click on one here, this Agriculture, for example. And you'll notice the URL here has GetCapabilities. A WMS server has two general requests that you make here-- the GetCapabilities, which tells you what you can get out of it, and the GetMap, which, as you might guess, gives you back a map image. So I'll hit the GetCapabilities. And if I scroll down here, it's an XML document that describes all of the features and different things you can do with this particular WMS service. And they all differ. The important thing to look for that we're going to need to put into our application is the name of the layer that we're interested in getting. So here, they have some layers-- for example, crop 1. And the other thing that's really important-- although there's a few-- are the supported projections. So the projection that we use in Google Maps is a Spherical Mercator projection. And it goes by, confusingly, a number of different coordinate reference systems or spatial reference systems. So the sort of official one, 3857 here, if you see that, you're good to go. But you could also go ahead and use any of these-- 54004; this ESRI one, 1002113; or this more common but unofficial 900913-- Google, in elite speak-- reference system. And if the WMS says, I support this, then you're good to go to overlay the images. If, for example, it only has this kind of plate carree 4326 reference system, you're not going to be able to align the tiles with a Spherical Mercator map. PAUL SAXMAN: Yeah, different projection. And so that's actually a really interesting thing. So digging around some of these WMS providers, I saw probably about a little over half actually support the ones that we can use. So very important to look for this when you're looking at these public WMS servers. JOSH LIVNI: Yeah. PAUL SAXMAN: But there are a number of them out there. I mean, they're not hard to find. JOSH LIVNI: And on that note, projections are confusing. So if you want to see us give a talk on projections-- not going to cover them anymore today-- let us know in the comments, and maybe we'll whip something up. So the next step is the GetMap request. You can see in the URL, there's a whole bunch of different parameters. Here I have my layers is value 3. I picked some layer at random. I have a bounding box, which I can change. And I get back this map image. If I hit back one, my layers is now crop 1. I could change it to crop 2. And I get back images for their various different layers that they support in the bounding box width and height that are requested. So the final step is, now that we know what WMS is, how do we turn this into the nice 256 tiles and get them in the tile provider? And fortunately, again-- thanks, Azavea; super cool-- you guys produced some open source code, which Paul has generalized in some ways and made really easy for you guys to get arbitrary WMS servers in. So why don't you tell us about that? PAUL SAXMAN: Yeah. So this is the maps-wms-tiles-sample. We'll link to this in the description, or it is linked to. So basically, I just forked their code. Their code is actually relatively easy to use. They just have their tile provider hardcoded in. I generalized it by pulling it out into an object that we can use to populate any number of different tile providers. But if you just have your own tile provider, you want to use that, definitely you can go straight to their code and fork their repository. So what I did-- if we go to the next-- so here, you could see in this code, is I have five different providers. These are different tile providers that are available. You could see that I generalized it, so you can just pass the parameters that you need. You definitely need the URL. You definitely need the layers. There's a number of other optional parameters, things like the style. Some of these providers have different styles. That's all in the GetCapabilities. And then also, there's transparency. So by default, if the tile provider can provide pings with transparency, then that's just default on. But you can turn that off and actually have an opaque map if you want. So this is an example. So I'm going to run this application here, so we can see what it looks like. So here we are on the tablet again. So this is from Google Maps. JOSH LIVNI: Is this city lights? PAUL SAXMAN: Yeah, this is city lights. JOSH LIVNI: --at night. Yeah, the NASA city lights that we've brought into Google Maps Engine. And Google Maps Engine also has a WMS endpoint, amongst others, that you can get you data out in KML, integrated into the Maps API, or a WMS that then we can use arbitrarily in applications such as this. PAUL SAXMAN: Yep. So you load your raster data into Maps Engine, then you automatically get this WMS service. So here, you can see this is Earth lights at night. Really nice thing. So again, you can see these tiles actually loading as I'm zooming in. These are actually coming from some of the Maps Engine servers. And again, you can also rotate and tilt. Another example. So we saw the National Atlas earlier. So this is precipitation. I just switched the map type. I think this is the hybrid map. And you can see this overlay data. This is precipitation in the United States. And the nice thing about this, or interesting thing about this, is you can see that the labels are actually rendered on top of the data. So that's kind of the default mode. We don't yet support actually rendering the labels underneath the tile overlay. But-- JOSH LIVNI: Who wants labels underneath? That's no good. PAUL SAXMAN: Yeah. Actually, it's the other way around a lot of times. Everybody is begging for people to have the labels on top. So this is a nice demonstration of seeing the labels and the data. And again, as we zoom in, as we're rendering these tiles, we're just making requests of the WMS service and filling them in. They load a little bit slower than the map tiles. Just depends on the speed of the WMS service that we're pulling from. Some other examples is the NOAA. So NOAA has a number of different services. This one was Watch and Warning, weather watches and warnings. So this data looks really great on top a map, especially a street map. And another one from NOAA, this is Radar. So radar, you could see a lot of precipitation out East. So all this information is pretty much freely available to put on top of your map, as long as you know where the data comes from and also have the ability to put it on your map using the Maps API and this API from Az-- JOSH LIVNI: Azavea. PAUL SAXMAN: Azavea. JOSH LIVNI: Yeah. Actually, I don't know how to pronounce it, either. PAUL SAXMAN: One quick last one, a really interesting one. So this one was from USGS. You can see all counties in the United States. So if you wanted to render this as polygons, you can imagine that's actually a lot of work, actually producing the polygons and actually rendering them. This way, rendering them as raster tiles, it just works. And easy cheesy. JOSH LIVNI: Less bandwidth, yeah. So it's a best practice, before you just go hit up a WMS endpoint, to see if the provider also has a tile endpoint, where they've potentially pre-rendered or cast or otherwise made available the kind of XYZ format tiles. Since WMS is a lot more flexible, you're not unnecessarily using their CPU to create that. So check out also, of course, their usage policy and so forth before you start grabbing kind of tiles. And many of them have attribution that they might request. So just common best practices when using anybody's web service. PAUL SAXMAN: Yeah, I'm sure some of these big guys can handle it. But if it's somebody that just put some tiles up for a sample application and-- JOSH LIVNI: They might be sad. PAUL SAXMAN: --yeah-- and your app takes off, then yeah. JOSH LIVNI: And in a case like that, where you see something like that and you have the rights to use the tiles, but you're you concerned about hitting them, you can put up really simple proxies in between, like TileCache, which will sit in between a WMS server and then spit out these XYZ tiles, which makes it really easy on everybody. But in any case, you can hit it directly without a proxy or setting up your own service using this code here. And it looks like, really, the only thing you need to modify with the code is this mainactivity.java. Change the URLs in here to match what you need, and you have a working WMS Android app. PAUL SAXMAN: Yeah, so either grab this app, or if you just have one endpoint. If you want to play around with multiple endpoints, use the code I put up. If you want to just use one endpoint and hardcode the URL in, check out what Azavea put together. And there's a lot of-- this is getting up and running quickly. You could probably get this app running in about five, ten minutes if you know you have to pace in your key and link to the Android library. But beyond all that, it's actually just pretty much plug and play. A lot of advanced topics around WMS. There's a number of other WMS services out there that we could talk about. Josh mentioned we could probably have a whole talk about projections and how these work. If you guys want to hear more about any of these things, definitely leave us some comments. If you have any questions-- JOSH LIVNI: We love comments and questions. PAUL SAXMAN: Absolutely. So we will try our best to get back to you on any of the comments. And if you just want to tell us if and how, or not, you like the show, do that, as well. And thanks for joining us. JOSH LIVNI: Yeah, thanks for tuning in, guys. See you in a couple weeks. PAUL SAXMAN: Adios.