Visualize on the fly in the mobile phone the data collected with ODK on a map

Hello,
I have been using ODK collect app to collect crop reference data on the ground in Ethiopia, Tunisia, Egypt, etc. It would be useful to have the possibility to check on a map the points that we are collecting to avoid to collect data for a place/point that we have already collected. In addition the possibility to see the points collected could help in improving the spatial distribution of the data collected.. I mean, sometimes, without a map, it's not easy to see where we need to improve the data collection and where we don't need to collect more data....
I hope that it's clear what I'm suggesting..
thanks a lot for your support
Michela

Dear Michela,

This feature is already available in the ODK 2.0 tools. Check out map view:
https://docs.opendatakit.org/odk2/tables-sample-app-tea-houses/#map-view

Best,
Caroline

Agreed that this mapping is available in ODK 2, but that tool suite is not compatible with ODK Collect and so would require a change to Michela's workflow. It also has a higher level of complexity.

The functionality is also available in GeoODK Collect which is a fork of ODK Collect (and now a little out of date).

@Michela_Marinelli This idea has been floated around for ODK Collect for a few years now (see https://github.com/opendatakit/collect/issues/61) and one of the blockers is where it would appear in the user interface. Some ideas that have suggested are

  1. We add another button on the main menu called View on Map
  2. We add a "map mode" on Edit Saved Form, Send Finalized Form, View Sent Form that lets you toggle between a list of forms and a map of forms.

Do either of these ideas resonate with you? What do others think?

1 Like

Many thanks for your reply.
I was trying to get the Plot app from ODK2, but as you said this is not compatible with ODK Collect, so I got many problems. I'm happy to read that you are working on it. To me, to add a button on the main menu called "View on Map" is good! I found this option very clear indeed..thanks..I hope you will go on..
best,
Michela

3 Likes

This feature request has come back to the forefront because:

  • In the geo roadmap session at the October convening, a big theme was that users get a lot of value out of collecting single points and that not being able to immediately view those points on a map is a big limitation.
  • In conversations about longitudinal data collection at the October convening, it was very clear that viewing previous records on a map is important to many workflows that involve repeat encounters.
  • A number of users use GeoODK for this functionality but the GeoODK Collect app is now long out of date.
  • Large users that @yanokwa and I are in contact with have expressed need for this (privately for now but hopefully on the forum in the future!)

@zestyping has been working on a specification to provide a map view for each applicable form definition. This map view would display all filled form instances currently on the device for that form definition, show the current device position, and provide a button to fill a new form instance.

For this first version of the feature, an applicable form definition is one that has a geopoint question not in a repeat as its first geo question. That is, form definitions that have a geotrace question or a geoshape question or a geopoint question in a repeat as their first geo questions would not currently get a map.

Please see @zestyping’s draft specification for further details.

You can either comment on the specification or respond in this thread. Specification comments are preferred for technical questions and forum comments are preferred for comments or questions about the high-level user experience and functionality.

6 Likes

FYI, and perhaps as working preliminary prototype, I do something along these lines with iXForms, for displaying geo-referenced submissions on a map. Basically, from the Form List page you can view the list of Submissions for each form:

The submissions for each form can be viewed as a List View (selecting one will open it up for viewing). eg iXForms Swimming Pool form has 3 submissions:

and if the form has a geo* control it in, there's the option of a Map View of the same:

Selcting one brings up a small summary popup. Tapping that likewise opens the submission for viewing.

I just use an ad hoc approach that the first geo* control in the form is the one used to position it on the map; obviously that needs to be more properly spec'd.

4 Likes

This feature might help us a lot on the Field, collecting data on remote places... just wondering how will the basis-map be pre-loaded..

1 Like

Initially there would be no automated way to pre-generate instances to be displayed on a map. It's what I'm hinting at when I mention improved longitudinal data collection support and it would likely be the next big area of design and improvement.

1 Like

So if i understand it right, this feature is linked to the use of a internet-connected smartphone to load the map when the user will want to display it. Right ? S'il usefull even if more limited.

I'm sorry, I think I misunderstood what you meant by pre-loading. I was talking about having existing data points on the map to collect data about. I think you're talking about having an offline map layer. The same offline map layers that are available for the geopoint, geotrace and geoshape question types would be available for this map view. You can read more about these in the documentation. Currently the map layers have to be pushed from a computer to the devices.

1 Like

Ah nice. Thanks for your answer and your success on understanding my approximate english :wink:

2 Likes

Thanks for the clarification. What about preloading? Like the way data is being preloaded and used in select or calculated questions? Having the ability to create new instances from visualized point.

@Brice_Goedert, don't worry, it was just my interpretation that wasn't right! Your English is great but also feel free to write in your most comfortable language and someone is bound to speak it too. We're a multilingual crowd here!

There are some great comments on the specification. I want to specifically call out one major decision point which is to have a separate map for each form. An alternative would be to have a single map to see data for all forms. @adam.butler said some good things about this that are think are worth bringing up here since it might spark broader conversation:

Form-specific is simpler (both to implement performantly and to use), but I've had requirements for multiple forms on a single map before, and I wouldn't want to foreclose that option.

Can you describe the scenario in a little more detail? How were the multiple forms related? For example, were they different forms for different types of things ("plants" vs "buildings)? What would enumerators do with the map of mixed form data?

My general sense is that once we have a form-centric map view built, it would be trivially easy to do things like add the option to layer on data from other forms. I think the hardest part would be coming up with a good UI to convey what each marker represents. If needed, we could also add a top-level map view that shows points from all forms. There should not be performance issues with this since the geo features will be pulled out in a preprocessing step.

A "general" map view could have a form filter, so that only points from a selected form are displayed.

What do you think of what I've described above which is kind of the inverse of this? The idea would be that you're still in the context of a single form that you're filling but you have visibility into where else you've been.

This doesn't allow for filling different forms. I'm envisioning that this would come in the context of a more entity-based workflow. Again, I think that this form-centric map would be easily reusable in that context.

It's worth considering how devices are configured before real-world data collection takes place; in my experience, a device will often only have one or two forms installed on it, in order to prevent surveyor error (filling the wrong form). In this situation, a general map view could be better, in that it is versatile without being overwhelming. OTOH, I don't know how widespread the practice of removing irrelevant forms is.

For better or worse, my experience suggests that organizations have different workflows around this. Sometimes enumerators work on multiple unrelated data collection campaigns and other times they are focused on one and having just the forms for that campaign is really important. The feature described at Have Collect exactly match the forms on Central is intended to help address the case where having a controlled set of forms and versions is really important. It'd be great to get your impressions there.

It wouldn't be part of this initial feature but that's certainly where I'd like things to go. Would you get value from only seeing the points collected so far on the current device (what this spec is) or is the data preloading you describe essential for you?

Thanks for bringing this discussion into the forum @LN.

In this particular scenario, enumerators were actually carrying out two separate data collection activities simultaneously:

  1. A microcensus, which involved enumerating every inhabitant within a prescribed polygon
  2. While performing the microcensus, adding or updating information on nearby Points of Interest (villages, health facilities, mosques, etc.) by asking the inhabitants whether they knew any details about them

Note that neither of these will be possible (yet) with the proposed spec, since (1) requires a preloaded polygon/geoshape to define the microcensus area, and (2) requires preloaded geojson to define the POIs. But I'm very excited to see a major step towards making this kind of work possible in Collect.

Trivially easy from a technical perspective, but tricky UX-wise: how do you explain that the map that is displayed shows data from Form X and Form Y (but not, perhaps, from Form Z, even though that form might also be on the device), but you can only create a new instance of X - if you want to create a new instance of Y, you have to go to Form Y's map? And how do you configure that Form Y instances are visible on Form X's map?

I was actually envisioning that you would only ever see markers from one particular form at a time, so there would be no need to differentiate them. But now that I think about this more, I realise that there's actually very little UX difference between what @zestyping has proposed and what I'm thinking of:

  • in my version, you would switch between forms X, Y and Z by selecting one (and only one) of them from a dropdown
  • in @zestyping's version, you switch between them by pressing the back button and tapping a different form

In fact the latter is probably preferable to fiddling with a dropdown overlaid on a map.

Well I guess I've just talked myself over to your position :slight_smile:. I can see that the workflow differences between being form-centric and map-centric are negligible, and I also think that the form-centric approach aligns more closely with Collect's current general approach. So consider me convinced.

Oh, this is great! I think that would be an extremely helpful feature.

2 Likes

Yes, there is some value in that. Being able to visualize the collected points so far, will help the enumerators in the process of data collection and ensure data quality. For larger teams, this feature may enable the data manager to get quick individual based snapshot of the collected data for quality checks.

Also, although the data preloading would not be part of the initial release, there will be a some value in that as well.
I have come across some studies where this feature would have been of great advantage. 1. A mosquito surveillance study where collection was done in grids. The study area was divided into grids, and sampling was conducted within the grids. Think of these grids as different polygons which had to be traced. The process was conducted using a handheld gps.
With preloaded data means, it could be theoretically possible to know the which polygon you might be working on at a particular time.

Also another example, a household survey where houses were supposed to be visited more than once with different enumerators. Other apps were used to locate the houses, then odk was used for data collection. With preloading feature means such coordinates can be pushed in collect and complete the task in one app. Also this can ensure data quality.

2 Likes

Always the same high levels of discussion and involvement of the community :slight_smile:

As @dicksonsamwel said, both would be great and complementary. With this spec, seeing the "device's" collected places to survey and come back on previous data, or complete an inventory I did not finish. From my point of view I think that surveys are reaching different goals so a map with all the places of all the surveys would not have a lot of sense, but a "form filter" on the map would be a good solution to reach all the needs.

And maybe in the future seeing the all the places collected by the team for a specific form to share the field effort and to be able to easily make long term survey (for example description of ponds, edges, grasslands...)

1 Like

Really appreciating all the good discussion and feedback here!

We're making some good progress on getting this feature off the ground. There's a PR at #3447 with a few screenshots of what it looks like. (This is just the most basic, minimum useful set of functionality—we hope people will be able to gain some benefit from it, and we're excited to build more of the things mentioned on this forum thread after the basic functionality is in!)

2 Likes

Great work !
I am sure that we'll get a lot of benefice :wink:
This will be very usefull ans is already very promising.

1 Like

Thanks everyone for the great feedback so far. ODK Collect v1.25.0 beta.1 has just been published to demonstrate the functionality. Learn how to join the beta program in this post. Please note that it will not work with forms and instances already on the device. You'll need to download or push a form definition for the geo point question to be recognized. Here is a summary screen capture:

@TSC-1 I hope those who celebrate has a great Thanksgiving! Since you did not have a chance to meet last week, you didn't get a chance to do a final review and approval of this feature. I'd appreciate it if those who has comments or concerns took a look. Thank you!

8 Likes

Hi Helen and Ping,

I just downloaded the beta and try it. I removed all forms from my device (galaxy S5) and downloaded my form and I still can't get it work.
Is it due to the fact that in my form (http://si.cenlr.org/sites/si.cenlr.org/files/sicen_dev.zip) I can choose between Geopoint, geotrace and Geoshape ?
Thanks,

Mathieu