Create automatic background recording of interview in Collect

Great questions, Dan!

  • No need to record the screen from my perspective. We know what the questions are and which responses were chosen from the dataset.
  • Yes, it should be automatic so enumerators don't need to remember/can't forget to record when required. Another optional setting would be to only record segments randomly, especially if bandwidth is a major issue (see above).
  • It could be a setting for the whole interview or a flag set at the question or group level
  • It could either 1) append to the first recording, 2) start a second recording, 3) do nothing. I would prefer (2) since joining files can be done later as well.

Thank you @Tino_Kreutzer for bringing this up!

I'll add that this would be an incredibly valuable feature. From my point of view, the most useful would be to have a single recording from beginning to end of the data collection process. But I see how recording for specific questions, with separate file for each section recorded, would be useful as well. So @danbjoseph's suggestion of a begin recording and end recording wrapping sounds like a convenient and flexible way of dealing with this.

Got to this discussion after raising a similar issue here https://community.kobotoolbox.org/t/is-it-possible-to-use-2-widgets-parallel-at-the-same-time/5176/3.

So, if having audio recording in the background is an Android Limitation, would that background execution be possible in the case of other types of widgets, more specifically, would this background procedure be possible for GPS data (lines)?

Thanks

It may be of interest that the audit log can be configured to record location:
https://docs.opendatakit.org/form-audit-log/#location-tracking

When recording within the geotrace widget the survey needs to stay on that question's screen. I don't think you can have a geotrace and audio widget running at the same time. Are you able to have 2 survey devices for the interview?

Thanks for your suggestion.

Yes, I can, but than that is no different than using two apps (ODK for geotrace and some audio recorder) in one single phone.

I see your point, and for my corner use case it should work. Anyway I see this idea of of allowing certain widgets to run in the background as an interesting development for ODK.

Cheers

For reference, the old issue #24 on Github and some previous posts also discuss this feature, though things moved on in favor of the audit and location tracking.

I believe the major change that would need to happen to implement this feature would be to record sound natively within Collect--rather than depending on an external app such as RecForge. This means this change could apply for the existing per-question recordings as well as the future background recording feature. Some thoughts on the implication:

  • A user should be able to choose between the internal and external recording methods (since external apps will always have more bells and whistles)
  • The advantage of recording natively is that users wouldn't depend on external apps (esp. the need to purchase RecForge Pro for > 3 min recordings) and their challenges (see e.g. 1, 2, 3, 4)
  • Users would need to grant the permission to allow recording audio
  • Users would be less likely to make mistakes trying to record audio through external apps

I suggest that this is a per-form setting to allow recording (audit) the entire form, not for specific sections (which quickly gets complicated when users move around the form, see this comment. It should be the survey administrator setting this so that enumerators don't have to activate it. For example:

type name label appearance
audit_audio audit_audio record=TRUE sampling=44khz
..
survey

In Collect there could be a new setting to disable audio auditing (or to disable uploading audio files with the submission), even if a form requires it. This may be important if someone is collecting data in a remote area with particularly bad connectivity. For example:

4 Likes

+1 to the idea of an audio_audit as an additional tool alongside the excellent audit log.

I really like this proposed feature.

I think having the flexibility to wrap questions of interests (I guess no nesting would be allowed) or to set a maximum time for the recording (or until form is closed) would be really nice.

Would a full audio recording of the entire interview work for your use case @dr_michaelmarks? Could you describe your specific use case?

Even if we record the entire interview (e.g the way I proposed above), timestamps from the audit feature can mark exactly where each question (screen) starts and ends, so extracting audio for only specific questions would be possible that way.

Another option would be to require audio for the survey, but then add a per-question parameter that would explicitly require audio only for those questions.

My feeling is this would be more difficult to implement but I understand how it could be important for some use cases.

2 Likes

Full recording would be fine.

As for a user-facing aspect of this feature, I'd suggest adding an overlaid indicator (like a small pulsing recording ":red_circle:" icon perhaps) so that the enumerator is reminded to keep their hands off of the microphone, etc.

1 Like

Hello @Tino_Kreutzer and thanks for all the reference links,

The feature you describe would be of great value for large social science studies and I really like the idea of the audit audio with timestamps.

I have had a discussion around audio recording using ODK with the lead social scientist for the project I am working for, which involves a crazy amount of in-depth interviews and focus group discussions. From her feedback, she would favour recording the entire interview as a single file rather than in separate files for each question (as I was suggesting, since I felt this annotation would facilitate the analysis) as the interviewer may go back and forth between the different questions during the interview.

I may be wrong but I anticipate this back and forth could be tricky to extract separate questions, since timestamps for a previously answered question should not replace the previous timestamps, but rather be appended to a list.

1 Like

Thinking about this, there's value in having optional separate recordings.

It also occurred to me that one of the primary use cases in social sciences research is to record an interview such as the ones described by @Thalie whilst still having the GUI free to enter ancillary data, i.e. whilst a person is talking about their experiences of using a new toothpaste, the enumerator might want to record quantitative data to support their wider observations of the interviewee's body language.

But the main thing our social science teams do during interviews is to make 'field notes', by which we are talking about qualitative comments on what the respondent is saying at the time.

Thinking about how this would work, a good field-note might be

  1. timestamped
  2. tagged or keyworded
  3. unlimited in number

So @Tino_Kreutzer's proposed design allows the interview recording to be timestamped as the screen is swiped, allowing for (1) and the GUI is available for keyword tagging, allowing for (2) especially if the open-repeats functions from the roadmap get developed as you could add keywords as you went along.

To allow (3) the whole system would need to work across non-indexed, repeats. i.e. when the enumerator wants to add a new field note, they start a new (timestamped) repeat and enter the note & keywords.

What would still be missing is a decent interface for collecting field notes, which most anthropologists/ethnographers I know still want to do using handwritten notes.

A good interface for handwritten notes that moves away from pen and paper to using a tablet & stylus would make this a NEXT-LEVEL development. This is because the organisation of the notes is being done as the data are collected and the use of keyword tagging would make every note searchable and easily organised using qualitative data analysis software like NVIVO (https://www.qsrinternational.com/nvivo-qualitative-data-analysis-software/home)

Really the interface would only need to be a souped up version of the draw image thing that already exists, but with some UI changes to make it better for handwriting and possibly scrolling down if copious writing is being done.

In my head, this setup for field anthropology looks like this
i.e.

  • The interview is being recorded, with timestamps being written to a file as pages are swiped
  • The enumerator wants to make a new field note during the interview
  • They start a new timestamped repeat
  • Then press a button that opens up an integrated handwriting sketch pad
  • Write a bunch of stuff
  • Which gets saved as an attachment jpg/pdf
  • Finally tag the note with keywords from a select multiple type question to which they can add tags as they go along.
  • Repeat
1 Like

Hello,

That would be needed for enumerator and respondent!

Further data protection would need yet discussed here. (KoBo device folders and datasets, incl. encryption). Imagine personal interviews with sensible data or controlling of enumerators.

Thanks to all who helped shape built-in audio recording, especially @Tino_Kreutzer and Ignacio who contributed to the visual design of the feature. Built-in recording is available in Collect v1.29 and ready to be leveraged for background audio capture.

We intend to build a first version of background audio capture for Collect v1.30 which will enable recording an entire form filling session as a single audio file. Examples of future additions would include configuring a subset of questions to record and configuring a probability that any given form filling session is recorded.

By default, we propose producing AMR files with a 12.2kbps bit rate and sample rate of 8kHz. This corresponds to the voice-only audio quality option and would result in file sizes of ~5MB/hour. The other audio quality options would be configurable at the form level.

See the proposed form definition specification for more details.

We propose only recording the first form filling session. That is, if a data collector edits a saved form or the form session ends and is reopened from a savepoint, no audio would be captured. This reduces complexity and we expect that in most cases edits would not lead to useful audio. The audit log could be used to identify when form edits took place.

A recording status bar will be displayed for the duration of background audio recording. This bar will look like a compact version of the foreground reporting bar with a small waveform to provide visual feedback on audio volume:

If the form requests background audio capture but Collect can’t initiate it (e.g. because microphone permissions have not been granted), the recording status bar will indicate that recording has been requested but isn’t working:

It will be possible to toggle background audio recording from the overflow (⋮) menu. If audio recording is turned off during a form filling session, a confirmation dialog will be displayed explaining that any audio previously recorded will be discarded. If the user confirms, the audio file will be immediately discarded. The recording status bar will be updated to indicate that recording is disabled. If audio recording is turned on during a form filling session, recording will begin immediately and the recording status bar will be updated. If audit logging is enabled, toggling background audio recording results in an audit log event with name background audio enabled/disabled. The toggle will look like the start-geopoint toggle:

Because ODK Collect is a generic tool used around the world, we can’t provide built-in consent mechanisms. For example, in some US states, it is legal to record a conversation if only one party gives consent. Data collection contexts also vary widely so this feature could be used to capture things like notes to self or ambient sounds which would not require consent in any locale. What we will do is provide documentation that describes project designers’ responsibility to understand their local laws, IRB requirements, etc and shares some sample form design concepts for requesting consent.

Similarly, determining whether encryption or additional device protection is needed is the responsibility of project designers.

We look forward to any feedback on this user experience proposal and the corresponding spec proposal!

3 Likes

I understand this limitation is there because switching together AMRs/AACs isn't straightforward, but I think it's not uncommon for data collectors to pause and resume a long survey. I don't think it's a must have, but it feels awfully close to me. Curious what others think.

1 Like

I like how this looks. My only thought was that something needs to be added other than the running time total and the microphone icon, e.g. "Background recording in progress". Without that it could be confusing to some users who may not have been trained/briefed by the survey admin.

Alternatively, maybe a popup warning could be displayed when opening the blank form?

2 Likes

We have recently released a beta of Collect v1.30 with background recording. XLSForm support should be coming in a few days. You can find more information on the beta program and how to get a test form in ODK Collect v1.30 Beta: background recording.

Thanks to all who have contributed to make this feature happen!

Recording is now resumed across form sessions.

Thanks for the ideas in Slack. We don't yet have something to directly address this but hope to come back to it soon based on user feedback.

2 Likes

ODK Collect v1.30 is now available with support for background audio recording. XLSForm support is available in XLSForm Online and will be coming in the next Central release. You can find documentation here.

If you have questions about the feature or further suggestions for improvements, please start a new thread!

1 Like

This topic was automatically closed after 6 days. New replies are no longer allowed.