XLSForm Online breaking repeats

What is the problem? Please be detailed.
I was converting an XLS file with nested repeats. XLSForm online finished the conversion with no errors. When uploading to Aggregate, it complained that it is not a proper XLSFrom file. Previewing the form in Enketo showed only the label of the repeat but not matter how much I simplified the repeat, its content was not showing. I reduced the repeat content to just a note and a static repeat count to no avail. Enketo was not showing the repeat and Aggregate was not accepting it.

What ODK tool and version are you using? And on what device and operating system version?
XLSForm online V1.3.0
XLSForm offline V1.7.0
Aggregate on Google App Engine v1.6.1

What steps can we take to reproduce the problem?
My XLS file has the following structure:

begin_repeat
begin_group
end_group
begin_repeat
end_repeat
begin_repeat
end_repeat
begin_repeat
end_repeat
end_repeat

What you have you tried to fix the problem?
I ended up using XLSForm Offline (had to turn off Windows Defender to download and use it) and the form uploaded perfectly on Aggregate and functioned as expected on the tablet.broken_repeat.xlsx (28.9 KB)

Hi @techGus

could you attach your form for testing?

@Grzesiek2010 Form attached!

I think your form contains bugs like for example in line 40 column relevant you use:
selected(${education_type}, 'technical_school') or selected(${education_type}, 'university')

but education_type is defined in line 101 so it doesn't make sense.
Please review your form carefully.

selected(${education_type}} on line 40 should be p_education_type

I corrected this and the behavior is still the same. Will go through the form again to check for inconsistencies but still, it doesn't explain why the online conversion is not reporting an error while Aggregate isn't accepting the submission, while the offline converted file is uploading successfully to Aggregate.

A couple of things stick out to me here in this topic. This is a little complicated, so bear with me here...

XLSForm Online v1.3.0 uses pyxform v0.12.0 and JavaRosa v2.12.1
XLSForm Offline v1.7.0 uses pyxform v0.11.5 and JavaRosa v2.11.1

ODK Aggregate v1.7.1 uses JavaRosa v2.12.1
ODK Aggregate v1.6.0 uses JavaRosa v2.11.0

I took @LN's nested-index-repeat form and it seems nested repeats with a jr:count are the problem.

XLSForm Online v1.3.0 (pyxform v0.12.0):

  • nested-indexed-repeat-v120.xml (3.5 KB)
  • Key line: <repeat jr:count=" ../pets_count "
  • Fails in Aggregate v1.6.0 (JavaRosa v2.11.0)
  • Loads in Aggregate v1.7.1 (JavaRosa v2.12.1)

XLSForm Offline v1.7.0 (pyxform v0.11.5):

  • nested-indexed-repeat-v115.xml (3.6 KB)
  • Key line: <repeat jr:count=" /nested-indexed-repeat/friends/pets_count "
  • Loads in Aggregate v1.6.0 (JavaRosa v2.11.0)
  • Loads in Aggregate v1.7.1 (JavaRosa v2.12.1)

@martijnr According to http://xlsform.org/en/ref-table, you don't support dynamic repeat-counts. Is that still true?

@ggalmazor The behavior of Aggregate is very weird to me. Can you replicate my results using just the test form and different versions of pyxform? Is it possible that how jr:count is evaluated changed? Maybe in https://github.com/opendatakit/javarosa/pull/335?

Ah, thanks! That needs to be updated. Will do that. It's fully dynamic in Enketo now.

Fancy! Well if it is fully dynamic, then this nested-indexed-repeat form doesn't work as expected in Enketo. Is it a :bug: ?

Yes, looks like a :ant: (nested repeats: :face_vomiting::face_vomiting:)

Thanks! Will get fixed! https://github.com/enketo/enketo-core/issues/588

Aggregate should ignore the jr:count attribute since it won't affect the database structure to save incoming submissions. I'm wondering why using ../ or an absolute ref would make a difference at all... I'll give it a look.

1 Like

I discussed with @ggalmazor and we've agreed that things are working as expected in the XLSForm and Aggregate.

pyxform v0.12.0 introduced relative referencing and that required a change in JavaRosa v2.12. If you use XLSForm Online v1.3 or XLSForm Offline v1.8, you'll need to be on the most recent version of Aggregate to get those changes.

This isn't great for people who haven't upgraded Aggregate. Truth be told, we could have communicated this change better (how though?!) since we push updates to XLSForm Online without much warning to users. Maybe in the future we could run old versions at previous.xlsform.opendatakit.org (help wanted!) that folks can roll back to.

In the meanwhile, the failure mode is pretty reasonable. That is the form won't load in old versions of Aggregate or Collect, so hopefully that will send people to the forum where they'll get help. I'm also adding some language to the website to send people to the docs or forum if they run into problems.

And if they can't upgrade, older versions of XLSForm Offline exist and older versions of XLSForm Online can be spun up quickly.

As always, I'm open to ideas on how we can communicate breaking changes better with users.

1 Like