Allow custom choices for selects that get saved to the same field (XForms open selects)

1. What is the general goal of the feature?
Currently, XLSForm supports or_other added after a select type. This adds an "other" option to the select list with label "Other" and also adds a text field with prompt text "Specify other." In Collect, this looks like:

Problems with this:

  • The "Other" and "Specify other" labels can't be translated or personalized (Translation for [or_other])
  • The "Other" text field is presented on a separate screen by clients like Collect that show one question per screen
  • The user-specified answer is stored in a separate field from values selected from the list which can complicate analysis

2. What are some example use cases for this feature?
Sometimes not all choice values are known ahead of time.

3. What can you contribute to making this feature a reality?
Specification and implementation.

From an ODK XForms specification perspective, I think this is best supported by supporting the "open" value for the selection attribute on selects, as described in the W3C XForms spec. That way clients will be responsible for how they want to display the question type and for managing translations.

For XLSForm (what users will care about), I have so far thought of two specification options and would like feedback on which is preferred (or other ideas):

  1. Similar to existing or_other: open option added after the list name
survey
type name
select_multiple mylist open my_choice
  1. Adding a selection option to the parameters column
survey parameters
type name
select_multiple mylist my_choice selection=open

I violently agree - this is precisely the usecase that the XForm open selection was intended to solve. [FYI I've actually implemented support for this with an associated, appropriate select_one/select_multi widget in iXForms, if anybody's interested in how it might be rendered and behave on a mobile client like Collect]

1 Like

I figured there'd be no controversy there!

Would be great to see a screenshot of your widget here for inspiration if you're willing to share.

First is select_one, after tapping on the last row (other) but before typing in a custom value ("Other" becomes the placeholder text):

Second is select_multi, after selecting two of the fixed options (red and gree) and after adding your custom other value "Orange" (but before exiting the widget by returning back to the form):

1 Like

Oh, and this is the initial view when the widget starts, but before the user taps on anything. Soon as they tap on "Other" the keyboard comes up (as shown above).

Select_multiple behaves pretty much the same - keyboard only shown when you tap Other

I agree, it is time for this addition. Thanks for this proposal!

One question, that is not immediately clear to me from XForms spec. We should prevent user-entered Other <select> values with spaces, right? Is that what you are doing @Xiphware?

1 Like

I believe the XForm spec actually allows this, as a means to enter multiple additional responses (!). But IMHO I think that's probably not a good idea, as it exposes the user to unnecessarily implementation details; ie "Warning: if you enter text separated by spaces they will be interpreted as multiple responses." Generally, I take the view that you should - in principle - be able to #define what the specific delimiter char is (" ", ",", "|", ...) and pretty much not affect functionality [to be honest, I only changed mine from "|" to " " to remain compatible with Aggregate.]

1 Like

Per discussion at https://github.com/XLSForm/pyxform/issues/370, I think that for select ones, spaces in select values are ok.

Interestingly, Orbeon introduced a new open-select1 component instead of using the selection attribute: https://doc.orbeon.com/form-runner/component/open-select1. I can't figure out why they decided to do that. Maybe specifically so that it isn't applicable to select multiples?

https://en.wikibooks.org/wiki/XForms/Open_Selection implies that it's only for select ones.

The options I see are:

  • only allow the selection attribute for select ones
  • give clients the responsibility to forbid custom values with spaces in the context of select multiples

As we hash out the details of the XForms spec, @Xiphware and @martijnr, it would also be helpful to hear from you what you think of the two XLSForm syntax options.

1 Like

I dont think its necessary to restrict open selection to only select_one. So I'm inclined to either disallow delimiters [and, at the risk of being stoned for blasphemy, possibly propose - someday - deprecating a " " delimiter for something less stupid... :-)]

As it happens, in order to support (multiple) standard responses, I'm planning to re-process choice values to replace spaces with, say "_" between storage and display. So in that scenario, the user could in fact type in a sentence with multiple words, but it'll be saved with underscores, and restored if subsequently redisplayed elsewhere. Bit messy, but the user should be insulated from it, and standard responses are a really important usecase for inspections.

2 Likes

As you know, the space delimiter is part of W3C XForms and comes from the XML list datatype (https://www.w3.org/TR/xforms11/#ui-selectMany). I agree it's unfortunate but I don't see a path to that changing given all the forms in active use that use the standard.

I'm generally not excited about magical transformations like this but I think it is defensible. Definitely in a context like yours where you know what your users are using the tools for, it makes a lot of sense. The downside is that it becomes impossible for the user to use actual underscores in their manually-provided choice name. I'm curious to know what @martijnr thinks.

1 Like

Agreed. But in my case I have the luxury of a more controlled ecosystem, not a lot of legacy data/forms to worry about, and a usecase that overrides an (arguably ill-informed) historical XForm design decision.

This will always be the case, no matter what string delimiter is chosen. Hence LOOONG term I'd prefer to introduce an actual array datatype to our XForms, to properly handle compound data; eg select_multiple arrays, geoshapes, ... But I'll save that debate for the next convening. :wink:

2 Likes

Interesting indeed. Nice find. And mysterious.

Not sure yet about magic space replacements in multi-select values. If doing that I'd rather show the underscores in the displayed value as well. We should probably reach agreement on multiple-select-space-handling (and not leave it to the implementation).

Wrt XLSForm would it be possible to keep the existing syntax (but generate a different output)?

1 Like

Obviously, it would be good if we could come up with a uniform approach to this. But if not, and in the absence of a compelling usecase (standard responses) for ODK, then I'm happy just keeping this hidden as a custom hack in our app.

XForms spec allows entering multiple custom, space-separated 'other' responses; that seems superfluous for most situations and I need standard responses (with spaces) for inspections. They do rather seem mutually exclusive... :man_shrugging:

Agreed. I'm leaning towards the spec saying that clients are responsible for preventing the user from specifying values with spaces in the case of open multiple selects. To satisfy the W3C XForms spec requirement of allowing multiple 'other' responses, clients could allow the user to create additional text input fields.

Wrt XLSForm would it be possible to keep the existing syntax (but generate a different output)?

I should have called out my reasoning for suggesting new syntax. The existing or_other adds another field to the XForm. If its behavior changed, that field would be removed from an updated form and that could be problematic for downstream tools and servers. Additionally, creating new syntax makes it easier to deal with the long tail of old client versions. That is, folks who use old client versions for whatever reason can keep using or_other and those who are sure to have new clients can use open selects.

Agreed, so no magic space-replacement in the spec (and yes, totally makes sense for custom clients to deviate).

Of course, always this recurring issue! Thanks. I slightly favor the parameters approach in XLSForm. I want to propose using 'other=true' instead of 'selection=open' though. I'm assuming that 'open selection' is not a term people are generally familiar with. I wasn't, but I may also just be a special (ignorant) case.

1 Like

That sounds good to me.

That's a very good point. I also only know "open selection" from the W3C XForms spec. I think other=true is a great suggestion.

FYI I think its probably perfectly fine to use a less obscure syntax in XLSForm; ie other=true instead of selection=open. And I'm not aware of any other valid XForm values for selection other than open, so a boolean true (and, implicitly, also false) should suffice.

1 Like