XForms spec proposal: add event fired when new repeat instance is created

[Xiphware]

@LN

They have defined a custom default bind attribute in their namespace to set dynamic defaults

Well, that is certainly a LOT simpler for user's (ie ODK form writers) than having to mess around with event handlers. Basically an (arbitrary XPath) calculate that's really evaluated only once! [unlike our beloved ODK once()... ]

It'd also probably be a simple addition to XLSForm: under the default column, if its a literal pre-populate the XML element as today, or if not stick the expression (or node reference) in a new default binding [in fact you could put literals in the default binding too if you wanted and forgo pre-populating the instance XML entirely]

Implementation might be a bit easier too: soon as a new instance XML is instantiated (but not on reload), run thru all the bindings and directly (and immediately) update any elements having a default in their binding. Ditto run thru any associated bindings when instantiating a new repeat group and do likewise.

But I'm sure I'm missing some gotchas...

[LN]

Thanks for pushing to get this right, @Xiphware!

@Xiphware

Can you point me to an (W3C?) example where multiple events in a single handler are listed like this?

If we follow XML Events 2, it's unambiguously supported. See in the introduction and in the description for action that the type for event is QNames. See QName definition and a description of QNames as "A space-separated list of QNames". That is what Orbeon does. See an excerpt from this form that defines part of their form builder (yes, much of Orbeon is written in XForms ) :

<xf:setvalue event="xforms-insert xforms-delete xxforms-replace xxforms-value-changed" ... />

Alternately we'd need to have two separate actions that respond to different events which is fine too.

@Xiphware

Well, that is certainly a LOT simpler for user's (ie ODK form writers) than having to mess around with event handlers

Yes, but given how few people author XML forms by hand, does that matter? I'm guessing even those of us who could do it tend to choose not to or at least start from a generated form definition and make edits to it rather than starting from a blank page.

One other thing that might change your mind about the attribute approach -- it only provides support for the setvalue action. We need to be able to trigger the odk:setlocation action as well and presumably some yet-to-be-dreamed-of actions in the future.

[Xiphware]

@LN

We need to be able to trigger the odk:setlocation action as well and presumably some yet-to-be-dreamed-of actions in the future.

Ah, before my time eh [well, not really, but I wasnt paying quite as much attention in March '18 ha ] So this basically boils down to being able to support asynchronous setvalue operations (eg odk:setlocation triggered when eventually get a fix). Right @martijnr ?

Asynchronously changing the instance XML - that is, not as a direct (ie immediate) consequence of user interaction - seems like it could get messy: eg what if I'm on a question that suddenly becomes irrelevant due to some other property changing in the background, ... Have these sorta scenarios been worked thru somewhere I can catch up on?

[LN]

@Xiphware

So this basically boils down to being able to support asynchronous setvalue operations (eg odk:setlocation triggered when eventually get a fix)

That specific action does. Consider xforms-insert or setfocus as examples of actions that one might want to trigger based on an arbitrary event and that wouldn't be addressed by the attribute option.

@Xiphware

Have these sorta scenarios been worked thru somewhere I can catch up on?

I'm working through an implementation and will make sure those are documented.

[martijnr]

Thanks a lot to you both for your thorough research and analysis! Very good finds in Orbeon and the XForms specs.

Some default attribute seems attractive. However, if we have to support events+actions anyway (for setlocation, value-change-events), and want to keep odk-instance-first-load we're not really solving anything. We're just adding convenience for XML hand-coders (and inconvenience for us poor developers).

@LN

Or maybe the event would never be fired for those? Then I think that to have dynamic defaults on all instances of the repeat including the one(s) defined in the instance you would need a setvalue action that responds both to form instance and repeat instance creation.

Yes, very good point. I think it would/should not fire on those. The double event listener seems like a good solution (so die-hard XML Form coders can decide whether they want defaults in those instances).

I'm very intrigued by @LN's marvellous find of the context() function in XForms 1.1 and the interpretation of Single Node binding. I don't understand the underlying issue that they are solving with context() yet (e.g. when thinking about a calculate inside a repeat where the context is (in) the repeat instance - why did the setvalue element have this issue and a calculation did not?). Will try to wrap my head around this.

@Xiphware

So this basically boils down to being able to support asynchronous setvalue operations (eg odk:setlocation triggered when eventually get a fix). Right @martijnr ?

Yes I think this is okay for events + actions (though you could definitely create problems with poor form design, e.g. make sections irrelevant while the user is laboring to enter data in them). Here is the discussion.

[LN]

@martijnr

However, if we have to support events+actions anyway (for setlocation, value-change-events), and want to keep odk-instance-first-load we're not really solving anything.

That's sounding right to me.

@martijnr

die-hard XML Form coders

@martijnr

I don't understand the underlying issue that they are solving with context() yet (e.g. when thinking about a calculate inside a repeat where the context is (in) the repeat instance - why did the setvalue element have this issue and a calculation did not?).

Yeah, those specs are some dense reading. I don't think I've ever read the same sentence as many times over again. My understanding is that it's because of the way the context for setvalue's value attribute is defined. It could have been defined as the same context node as is used to contextualize the ref but it's defined as the contextualized ref itself.

In the case of calculates or constraints, the evaluation context node is always the current node. I think that's ultimately the same thing because the current node is defined by a bind's ref but binds are always at the top level so you don't have a second context to consider.

It was clear in my head but after I wrote this now I'm confused again. Will need to come back to it once again for another think-through.

[Xiphware]

@LN

Yeah, those specs are some dense reading. I don't think I've ever read the same sentence as many times over again

This is why I have been advocating to only perform such activities over (and after) a few beers. I'll buy the first round at the convening...

On a more serious note, I'm still worried about two threads writing to the instance XML asynchronously... At least for me, having all writes presently driven by the (single) UI thread is fairly straight-forward (and unmessy). But opening that up to mutli-threaded writes opens up a whole realm of potential synchronization issues, mutexes, ...

[Xiphware]

Bump.

So @ln, @martijnr, what more do we need to do here to push a spec out for add/remove repeat instance event? [other than me keeping my mouth shut. ] Not sure we necessarily need a call, but I'm certainly available if y'all think it'd help.

[martijnr]

I believe the example in the XForms 1.1 spec is entirely unhelpful and the cause of the confusion.

IBM has a clear example, simplified (by me) as this:

<data>
     <expenses>
         <row>
             <detail/>
         </row>
     </expenses>
    <other>
       <location/>
   </other>
</data>
<repeat nodeset="expenses/row">
    ...
    <setvalue event="odk-new-repeat" ref="/data/other/location" value="context()/detail" /> 
</repeat>

So this use case for context() is exactly as:

@LN

If the evaluation context node for value is indeed the contextualized ref , then if ref is a node outside the repeat, value can't be a node in the repeat. I think that's a fine limitation.

Now suddenly, the description of context() in the W3C spec makes sense to me:

The intent is to provide form authors with a means of expressing a value attribute that is relative to the repeat context node when the Single Node Binding result is not.

In the example above the Single Node Binding (= ref) is outside the repeat and therefore not relative to the repeat context node.

So, I think we can continue to choose to not cater to this use case for now, and proceed with our plan. And we can add context() when we feel it is necessary.

A less happy realization after understanding ref vs. nodeset better is that I think we're doing ref values for elements inside a repeat (in the body) incorrectly. I think these should always have a relative path instead of an absolute path (as in XForms spec). Maybe not worth correcting as this point (and a much less serious deviation than the one that was recently corrected in pyxform).

@Xiphware

I'm still worried about two threads writing to the instance XML asynchronously

Only <setlocation> is asynchronous, right, or am I misunderstanding? The other events+actions seem synchronous (and predictable). I think it will be quite intuitive for form designers to avoid issues with <setlocation>.

[LN]

@martijnr

So, I think we can continue to choose to not cater to this use case for now, and proceed with our plan. And we can add context() when we feel it is necessary.

I'm on board. I still think this event should have a custom name because it won't be as broad as the W3C XForms one, agreed? I'll give a bit for others (@Xiphware) to chime in but if all that is sounding reasonable, I will write this up in as much detail as I can including when the event is dispatched relative to calculate evaluation, context rules, etc.

@martijnr

I think we're doing ref values for elements inside a repeat (in the body) incorrectly. I think these should always have a relative path instead of an absolute path (as in XForms spec).

Yes, that's sounding right. I think that was in fact the case that was tripping me up and that led me to cross out all that stuff in red above. It all adds up now, thanks. I agree that this is a wrong we can live with.

[martijnr]

@LN

still think this event should have a custom name because it won't be as broad as the W3C XForms one, agreed?

Yes agreed. I actually cannot figure out if a repeat is supposed to trigger the W3C xforms-insert event.

[LN]

@martijnr

I actually cannot figure out if a repeat is supposed to trigger the W3C xforms-insert event.

In W3C XForms, repeat instances can only be added by triggering the insert action. That could be by using a trigger control, responding to some event, whatever. On the other hand, in ODK XForms, repeats are added by client pixie dust. I suppose one could say that the insert action is implicit and that there is no explicit definition of what should trigger it (preferably a confusing dialog).

Oh, and in ODK XForms, trigger is ?!?!?!(or more generously, a specialized case of the W3C XForms one).

[martijnr]

Ah right, that makes sense. Thanks! So I am still in favor of odk-new-repeat then.

@LN

Oh, and in ODK XForms, trigger is ?!?!?!

Yes, we don't talk about that ;).

[LN]

As promised, here is a more formal spec proposal to summarize our discussion:

---

The odk-new-repeat event is dispatched when a new instance of a repeat is added to the primary instance and before recomputation of calculates, constraints, etc. Actions triggered by odk-new-repeat must be nested in the repeat form control.

Unlike W3C XForms' xforms-insert:

• it is narrowly for repeats (ODK XForms currently has no notion of inserting arbitrary nodes)
• its handlers must be nested in the corresponding repeat control (W3C XForms identifies the added nodes and the insert position in event context information so handlers can go anywhere)
• it is dispatched separately for each repeat instance (in W3C XForms, a single insert action may insert multiple nodes)

---

I was going to also write up some text to describe that the event attribute on actions can specify multiple events but now I'm not so sure about it. Does the following which I showed here make sense?

   <repeat nodeset="/data/friends">
      <setvalue event="odk-new-repeat odk-instance-first-load" ref="age" value="../../my_age + 2" />
      <input ref="/data/friends/age">
           <label>Update age</label>
      </input>
      ...
  </repeat>

That is, should a handler for odk-instance-first-load be allowed to be nested in a repeat control? The ODK XForms spec currently says no: "Action elements triggered by initialization events go in the model as siblings of bind nodes. Action elements triggered by control-specific events are nested in that control block."

It's not a problem for pyxform or other form builders to generate two separate setvalue events, certainly. But it does feel unfortunate.

Am I understanding correctly that it's by requiring that the action be nested in the control that the evaluation context is the new repeat instance? If we did allow top-level handlers, we'd have to say that the evaluation context for the ref is the new repeat and we'd have to add something like "relative expressions for ref are only allowed in nested handlers." That does not seem very elegant. I believe that's what Dimagi did with the jr-insert event.

In W3C XForms, my understanding is that handlers of xforms-insert can be siblings of binds. In that case, the evaluation context would be the primary instance’s single child. However, one could use the index() function or the event properties that the context is augmented with in the single node binding expression (the ref).

[Xiphware]

Thanks for the great writeup!

All seems workable to me, but I'd probably want to re-read it all again in the light of the day (1am presently) to make sure there arent any potential gotchas we've missed...

[martijnr]

Thanks a lot @LN!

@LN

That is, should a handler for odk-instance-first-load be allowed to be nested in a repeat control?

Yes, I think that's okay.

@LN

Am I understanding correctly that it's by requiring that the action be nested in the control that the evaluation context is the new repeat instance?

Yes, indeed. Good idea to use the index() function to circumvent the Single Node Binding limitation of <setvalue>. So we'd have to add one of these <setvalue> elements for all default (defined in XForm) repeat instances (and add id attributes in the model?). I don't think it's worth it. The rationale is to avoid the deviation of our own ODK XForms spec only, right?

In your XForm snippet we have ref="age" and ref="/data/friends/age". As discussed, the first is likely the only correct ref, but so far we're only using the second (due to pyxform). So in this case, let's use the consistently incorrect /data/friends/age. (Tbh, it also avoids having to bring back relative ref support in Enketo, though would be open to that if it deemed worthwhile ;)).

(PS. am away for next 10 days so apologies in advance for late responses)

[Xiphware]

@LN

The odk-new-repeat event is dispatched when a new instance of a repeat is added to the primary instance and before recomputation of calculate s, constraint s, etc

All looks good to me. Just want to double-check that all this will properly fire (potentially multiple times) when you first open a form and the repeat_count expression is evaluated for the very first time, eg

    <bind calculate="count( /indefinite-repeat/person )" nodeset="/indefinite-repeat/count" type="string"/>
    <bind calculate="if ( /indefinite-repeat/count  = 0 or /indefinite-repeat/person[position()= /indefinite-repeat/count ]/choice != 'a',  /indefinite-repeat/count  + 1,  /indefinite-repeat/count )" nodeset="/indefinite-repeat/person_count" readonly="true()" type="string"/>
    ...
    <repeat jr:count=" /indefinite-repeat/person_count " nodeset="/indefinite-repeat/person">

ie it requires multiple passes thru binding calculations to get the final repeat count. I assume the existing mechanism for populating the additional repeat nodesets already handles this, so it's a given, but that's the best conceivable gothca I could come up with...

[martijnr]

@Xiphware

but that's the best conceivable gotcha I could come up with...

You could have populated //person/choice with value 'a' via setvalue triggered by odk-new-repeat to really mess with our heads.

I think your example should be fine as far as I can see (as long as setvalue doesn't create an infinite repeat-count-update loop, which would be a form design issue).

[Xiphware]

Actually the example was courtesy @ln, not one of my devious machinations ...

indefinite-repeat.xlsx (6.5 KB)

[LN]

Thanks so much, all. I will attempt to have a spec PR soon.

@Xiphware

when you first open a form and the repeat_count expression is evaluated for the very first time

That's a great thing to call out, thank you.

@martijnr

You could have populated //person/choice with value 'a' via setvalue triggered by odk-new-repeat to really mess with our heads.

@Xiphware

Actually the example was courtesy @ln, not one of my devious machinations ...

Let us all deviously machinate together.