Collect v1.23 offline vector basemap questions

I've started testing the offline basemap feature in ODK -- very excited this is supported.

One issue I noticed: after I first placed the mbtiles files on the phone, and enabled the layer, it showed up fine. I was connected to the internet at this point, and then disconnected. The tiles were no longer visible. I then rebooted the phone, and the tiles were then visible when disconnected.

Question: what would it take to support offline vector basemaps?

Probably dumb question -- for a GeoPoint, I can't seem to move the point aligned with geolocation. The basemap is pretty useful for fine tuning, but I couldn't. What am I missing?

2 Likes

Hi @mikel! To address your points one by one:

  1. Tiles no longer visible: When you noticed this issue, what kind of mbtiles file did you use? Would you be able to provide it to me so I can examine it and possibly try to replicate what you experienced?

  2. Offline vector basemaps: I'm confused by your question because one of the big new features in this release is offline vector basemaps. Can you be more specific about the use case you have in mind that isn't supported in this 1.23 beta?

  3. Moving a GeoPoint: There are various "appearance" options that a GeoPoint question can have. If the appearance is placement-map, then you can put the point wherever you want, and you also have the option to drop the point at your current location, and move it after you've dropped it (by dragging it to a new location, or just by long-pressing in a different location). If the appearance is just map, then you can only press a button to drop the point at your GPS location. This is intended to give the form designer some control over the way a point is collected. Is it possible that you were using map when you wanted placement-map?

2 Likes

Tiles no longer visible: When you noticed this issue, what kind of mbtiles file did you use? Would you be able to provide it to me so I can examine it and possibly try to replicate what you experienced?

I generated mbtiles with https://github.com/jawg/mbtiles-generator, against OSM raster tiles. mbtiles attached, as a zip.

osm-kabernet.mbtiles.zip (2.4 MB)

I'm confused by your question because one of the big new features in this release is offline vector basemaps.

I didn't test this but just read this in the OP on the other thread

We've shipped another beta (v1.23.0-beta.4) and it includes offline layer support for raster mbtiles on top of every basemap. Vector mbtiles are also supported on Mapbox basemap

Also the ODK offline maps docs say

The only limitation is that tile data in Mapbox's pbf format are not supported.

Is it possible to create an mbtiles file with vector tiles, and use the same approach as for documented in the offline maps docs? Would I just place the style json, glyphs and fonts in the same directory?

placement-map

Aha, I didn't know about placement-map!

Btw, looking at the GeoShape question, it looks like you can create a polygon in the interface. Would it be possible to do something similar, but for line features? Perhaps an option in GeoTrace to allow editing on the map, rather than based on user movement.

1 Like

Hi @mikel and thanks so much for trying this out and for all the great questions!

I'm afraid our documentation has not caught up yet. I'm taking a look at that now and I'll share a proposed update. I'd love your review if you have a moment!

If you choose a basemap from the Mapbox source, you can indeed use the same process as documented to load a mbtiles tile with mvt/Mapbox pbf tile data and then select it as a reference layer. However, styling isn't yet supported. If you just need to show boundaries, unstyled data may be sufficient but of course this is a big limitation.

@zestyping and I have chatted a bit about how users will be able to provide style information and hopefully that will come in the next few months. It's likely that we would do what you describe and allow users to "place the style json, glyphs and fonts in the same directory". One design question to figure out is how users will specify local mbtiles files as sources.

The documentation for that is at https://docs.opendatakit.org/form-question-types/#location-widgets and I believe it's complete! Is there somewhere else you might have been looking for it?

And we're back to out of date documentation :cry:. This is now possible! In a trace question, tap the point adding button in the upper right hand part of the screen and then select "Placement by tapping" (the default option).

2 Likes

We have a pull request up for bringing the offline layer documentation up to date: https://github.com/opendatakit/docs/pull/1080. Would love a review from @mikel and anyone interested in this functionality!

Hi @mikel. Apologies for the confusion—I can totally understand how the new stuff wasn't very discoverable!

I think that leaves just the first issue, with your tiles disappearing. I tried the osm-kabernet.mbtiles file you uploaded, and so far I haven't been able to replicate the issue. I'd love to figure this out; do you happen to have a sequence of steps that will reliably trigger the problem for you?

In my experience, the tiles sometimes seem elusive because they only appear at the zoom levels they are declared for. So if I don't already know where they are and I don't know what zoom level they're at, it can be hard to find them and zoom to a point where they appear. In this case, your file doesn't have a minzoom or maxzoom declared in its metadata, so the tiles are only visible at their individual zoom levels. The file only has tiles at levels 4 and up, which means that if you are at zoom level 2 or 3 they don't appear. (If minzoom were, say, 2, then the level-4 tiles would continue to be visible even as you zoomed out to level 2.)

So when I loaded the file at first, it seemed like I couldn't get the tiles to appear. But now that I know what part of the world to zoom to (I zoom so that Ethiopia fills the screen), they appear for me every time, online or offline. I don't know if this might be what's making it seem like the tiles aren't showing for you—were you at the same zoom level when the tiles weren't coming up?

I also noticed that the set of visible tiles covers a different area at each zoom level, so as I zoom in, the square of tiles jumps around. But I think that's how it actually is in the file, so it's the expected behaviour even if it does look a bit weird. Is that what you expect as well?

Thanks for helping us investigate!

1 Like

P. S. It occurs to me that it might be nice for us to draw a bounding box (or some kind of obvious marking) that's visible at any zoom level, to show that there are tiles present. (And maybe even an indicator that you need to zoom in or out to see them!) That would make this all less mysterious. @ln what do you think?

Ah, that's interesting. The mbtiles spec does only say that the metadata table SHOULD contain those properties, not MUST, so that will probably be common for those not to exist. Is it more typical to provide that information in a style file (I'm new to all these specs and formats so genuinely asking)? It seems it could be reasonable to present the same tileset at different zoom ranges depending on what other map data is being displayed with it. Does that sound right even for raster tiles or not?

Would it perhaps be relatively straightforward for a user to define something like that in a style file if they wanted?

@LN the docs look good to me. Thanks for the updates and explanations.

We at Mapbox will have one suggestion for the getting tilesets section; odk use of offline Mapbox maps for non profits is something we’d like to support. Working on the right language and will share when ready.

1 Like

@zestyping no I was definitely looking in the right location and zoom levels; if I switched the phone online, then the mbtiles would appear.

btw the tiles stack like that because of how a bbox will intersects tiles at multiple zoom levels.

The style json can have a bounds specified on source layers https://docs.mapbox.com/mapbox-gl-js/style-spec/#sources-vector-bounds. I’d need to check how consistently that’s applied. I think indicating the bbox, either from the mbtiles file or style, would be worth investigating.

1 Like

@mikel Okay. We need some help investigating this one—I haven't been able to replicate the behaviour you described.

Is there a specific set of steps that reliably causes this to happen for you, every time? Could you let us know your device model and Android version? Anything else about your setup that might help us replicate your experience?

I should perhaps explain that, because Mapbox only understands how to fetch tiles from URLs, the way we've implemented this is to run a small HTTP server within the app and then provide Mapbox with a URL that points at localhost. I'm wondering if something about the networking settings that changed when you disconnected from the Internet also caused the phone to have trouble connecting to localhost. So—how did you connect and disconnect to the Internet? If you disconnect and then reconnect (without rebooting), what happens?

Another (possibly remote) hypothesis is that there is something about your device's power-saving functions that is interfering with networking. Are there power-saving modes, or features for conserving power, that you can turn off or change? The brand might be relevant here.

Thanks for helping us track this down!