Visualize on the fly in the mobile phone the data collected with ODK on a map

Initially there would be no automated way to pre-generate instances to be displayed on a map. It's what I'm hinting at when I mention improved longitudinal data collection support and it would likely be the next big area of design and improvement.

1 Like

So if i understand it right, this feature is linked to the use of a internet-connected smartphone to load the map when the user will want to display it. Right ? S'il usefull even if more limited.

I'm sorry, I think I misunderstood what you meant by pre-loading. I was talking about having existing data points on the map to collect data about. I think you're talking about having an offline map layer. The same offline map layers that are available for the geopoint, geotrace and geoshape question types would be available for this map view. You can read more about these in the documentation. Currently the map layers have to be pushed from a computer to the devices.

1 Like

Ah nice. Thanks for your answer and your success on understanding my approximate english :wink:

2 Likes

Thanks for the clarification. What about preloading? Like the way data is being preloaded and used in select or calculated questions? Having the ability to create new instances from visualized point.

@Brice_Goedert, don't worry, it was just my interpretation that wasn't right! Your English is great but also feel free to write in your most comfortable language and someone is bound to speak it too. We're a multilingual crowd here!

There are some great comments on the specification. I want to specifically call out one major decision point which is to have a separate map for each form. An alternative would be to have a single map to see data for all forms. @adam.butler said some good things about this that are think are worth bringing up here since it might spark broader conversation:

Form-specific is simpler (both to implement performantly and to use), but I've had requirements for multiple forms on a single map before, and I wouldn't want to foreclose that option.

Can you describe the scenario in a little more detail? How were the multiple forms related? For example, were they different forms for different types of things ("plants" vs "buildings)? What would enumerators do with the map of mixed form data?

My general sense is that once we have a form-centric map view built, it would be trivially easy to do things like add the option to layer on data from other forms. I think the hardest part would be coming up with a good UI to convey what each marker represents. If needed, we could also add a top-level map view that shows points from all forms. There should not be performance issues with this since the geo features will be pulled out in a preprocessing step.

A "general" map view could have a form filter, so that only points from a selected form are displayed.

What do you think of what I've described above which is kind of the inverse of this? The idea would be that you're still in the context of a single form that you're filling but you have visibility into where else you've been.

This doesn't allow for filling different forms. I'm envisioning that this would come in the context of a more entity-based workflow. Again, I think that this form-centric map would be easily reusable in that context.

It's worth considering how devices are configured before real-world data collection takes place; in my experience, a device will often only have one or two forms installed on it, in order to prevent surveyor error (filling the wrong form). In this situation, a general map view could be better, in that it is versatile without being overwhelming. OTOH, I don't know how widespread the practice of removing irrelevant forms is.

For better or worse, my experience suggests that organizations have different workflows around this. Sometimes enumerators work on multiple unrelated data collection campaigns and other times they are focused on one and having just the forms for that campaign is really important. The feature described at Have Collect exactly match the forms on Central is intended to help address the case where having a controlled set of forms and versions is really important. It'd be great to get your impressions there.

It wouldn't be part of this initial feature but that's certainly where I'd like things to go. Would you get value from only seeing the points collected so far on the current device (what this spec is) or is the data preloading you describe essential for you?

Thanks for bringing this discussion into the forum @LN.

In this particular scenario, enumerators were actually carrying out two separate data collection activities simultaneously:

  1. A microcensus, which involved enumerating every inhabitant within a prescribed polygon
  2. While performing the microcensus, adding or updating information on nearby Points of Interest (villages, health facilities, mosques, etc.) by asking the inhabitants whether they knew any details about them

Note that neither of these will be possible (yet) with the proposed spec, since (1) requires a preloaded polygon/geoshape to define the microcensus area, and (2) requires preloaded geojson to define the POIs. But I'm very excited to see a major step towards making this kind of work possible in Collect.

Trivially easy from a technical perspective, but tricky UX-wise: how do you explain that the map that is displayed shows data from Form X and Form Y (but not, perhaps, from Form Z, even though that form might also be on the device), but you can only create a new instance of X - if you want to create a new instance of Y, you have to go to Form Y's map? And how do you configure that Form Y instances are visible on Form X's map?

I was actually envisioning that you would only ever see markers from one particular form at a time, so there would be no need to differentiate them. But now that I think about this more, I realise that there's actually very little UX difference between what @zestyping has proposed and what I'm thinking of:

  • in my version, you would switch between forms X, Y and Z by selecting one (and only one) of them from a dropdown
  • in @zestyping's version, you switch between them by pressing the back button and tapping a different form

In fact the latter is probably preferable to fiddling with a dropdown overlaid on a map.

Well I guess I've just talked myself over to your position :slight_smile:. I can see that the workflow differences between being form-centric and map-centric are negligible, and I also think that the form-centric approach aligns more closely with Collect's current general approach. So consider me convinced.

Oh, this is great! I think that would be an extremely helpful feature.

2 Likes

Yes, there is some value in that. Being able to visualize the collected points so far, will help the enumerators in the process of data collection and ensure data quality. For larger teams, this feature may enable the data manager to get quick individual based snapshot of the collected data for quality checks.

Also, although the data preloading would not be part of the initial release, there will be a some value in that as well.
I have come across some studies where this feature would have been of great advantage. 1. A mosquito surveillance study where collection was done in grids. The study area was divided into grids, and sampling was conducted within the grids. Think of these grids as different polygons which had to be traced. The process was conducted using a handheld gps.
With preloaded data means, it could be theoretically possible to know the which polygon you might be working on at a particular time.

Also another example, a household survey where houses were supposed to be visited more than once with different enumerators. Other apps were used to locate the houses, then odk was used for data collection. With preloading feature means such coordinates can be pushed in collect and complete the task in one app. Also this can ensure data quality.

2 Likes

Always the same high levels of discussion and involvement of the community :slight_smile:

As @dicksonsamwel said, both would be great and complementary. With this spec, seeing the "device's" collected places to survey and come back on previous data, or complete an inventory I did not finish. From my point of view I think that surveys are reaching different goals so a map with all the places of all the surveys would not have a lot of sense, but a "form filter" on the map would be a good solution to reach all the needs.

And maybe in the future seeing the all the places collected by the team for a specific form to share the field effort and to be able to easily make long term survey (for example description of ponds, edges, grasslands...)

1 Like

Really appreciating all the good discussion and feedback here!

We're making some good progress on getting this feature off the ground. There's a PR at #3447 with a few screenshots of what it looks like. (This is just the most basic, minimum useful set of functionality—we hope people will be able to gain some benefit from it, and we're excited to build more of the things mentioned on this forum thread after the basic functionality is in!)

2 Likes

Great work !
I am sure that we'll get a lot of benefice :wink:
This will be very usefull ans is already very promising.

1 Like

Thanks everyone for the great feedback so far. ODK Collect v1.25.0 beta.1 has just been published to demonstrate the functionality. Learn how to join the beta program in this post. Please note that it will not work with forms and instances already on the device. You'll need to download or push a form definition for the geo point question to be recognized. Here is a summary screen capture:

@TAB I hope those who celebrate has a great Thanksgiving! Since you did not have a chance to meet last week, you didn't get a chance to do a final review and approval of this feature. I'd appreciate it if those who has comments or concerns took a look. Thank you!

9 Likes

Hi Helen and Ping,

I just downloaded the beta and try it. I removed all forms from my device (galaxy S5) and downloaded my form and I still can't get it work.
Is it due to the fact that in my form (http://si.cenlr.org/sites/si.cenlr.org/files/sicen_dev.zip) I can choose between Geopoint, geotrace and Geoshape ?
Thanks,

Mathieu

Thanks for checking it out, @mathieubossaert! Unfortunately, this form won't be supported yet. We currently only show a map if the first geo question outside of a repeat is of type geopoint. Your form only has geo questions in a repeat.

We chose to initially only map top-level points because we want to carefully design how to map shapes and traces as well as geo questions in repeats. In fact, your form is a very interesting case. It's clear that you would want to have each repeat's feature mapped separately. However, in some forms, repeats describe points along a path that should be considered a single feature.

We will come back to those more complex use cases soon and will be sure to ask for your feedback once we have a proposed design. For now, you could consider using the All Widgets form or another with a simple point to try out existing functionality.

OK thanks @LN for this answer.
I understand why it's ok with another form. And it is not so important for that specific form.

1 Like

Ah, and I thought you'd made my dreams come true!

I have a form that uses repeats for my datapoints. The reason for doing this is to make it easier to upload all data - the survey we're doing at the moment collects about 60 locations in a day, so it's more reliable for me to get a single form with lots of repeats, rather than having to select the forms to send. I know that 'select all' is an option, but some enumerators (myself included) have completed forms from more than one survey sitting on the device so 'select all' isn't relevant...

I understand the step by step approach, and the potential for points representing one feature, so I will look forward to that coming online... This is an excellent feature.

Which leads me to a question about this feature - can it read geo locations from sent forms on the device?

I have surveys where it is necessary to work in the same area over a period of days (e.g. filling a grid), sending data in between, so this would be really handy to visualise all data on that device to avoid duplication. At present I use a mapbox raster layer exported from my GIS with 'data collected' flags, but this only works if there is time to upload the data, import it to GIS, create a new mbtiles layer, upload that layer and distribute to enumerator(s) beofre the next fieldwork session. Lots of weak points in that workflow!

Obviously this feature isn't designed to cope with multiple enumerators anyway, but for single device surveys it would be good to include sent items from the same form definition.

Thanks for your great work.

2 Likes

We'll keep working on it!

This makes perfect sense. Stay tuned for some questions as we expand on the functionality!

Yes. It will display sent forms that have not been deleted -- the same ones that can be viewed from View Sent Form. Those will appear in green to match the status color in View Sent Form.

Exactly what we hope it will be used for!

Not yet but that's hopefully where we're going.

1 Like

Status update: expectations exceeded. Again!

1 Like

Collect v1.25.0 provides a map of filled forms on the current device! Please see documentation at https://docs.opendatakit.org/collect-form-map/.

This feature will keep evolving with some of the ideas mentioned above. Please share if you have suggestions or questions.

1 Like

I seconded your view that showing the geopoints within the repeat makes perfect sense. May be shall consider this feature.