What is the general goal of the feature?
From @Saad at SEND QUEUE after submission sending through SMS - #6 by Saad
There should be 2 queues, 1 is SENT via SMS, and second SENT via DATA. As soon as the form is finalized, the form get queued into both queues. SMS should be sent immediately and the form should be released from this queue. But form should remain in data queue, unless it is sent via data and cleared afterwards.
Some more logic fine-tuning:
SMS is used to send data earlier than internet connection (if internet connection is not available). However, internet data transmission is final and comprehensive; meaning if/when the submission is submitted to aggregate via internet channel, then there is no need for SMS.
Having said this, there can be scenario where a form is stacked in both queues, SMS and internet, and not going through to server (no data connection + no SIM in phone for SMS too). Let's assume that the phone gets a data connection first (wifi) and the form queued in internet queue goes through to aggregate. This means our task is completed full and final (as data has reached to aggregate). However, the form in SMS is still waiting (as it cannot find a SIM). We don't really need this data now, since our internet submission has reached its destination. In this scenario, the logic in Collect should be like this:
If (form is queued in SMS) AND (form is queued in Internet)
{
If (form is successfully sent through Internet) AND (form is queued in SMS)
{
discard/clear that form from SMS queue;
}
}
I hope I am detailed enough to explain this well. If not, please do specify the blocking point in understanding.
Saad
Thanks for those extra details, @Saad!
I think we're running into a common issue when designing some of these more complex features -- Collect has a long history of trying not to impose particular workflows on users and it's tough to balance that flexibility with usability.
If I understand correctly, in your workflow, you absolutely need to receive every submission via HTTP but SMS is optional. This suggests to me that you're working in an environment where you'll almost always have a data connection but where you occasionally may not and in that case you'd like to receive the SMS. Is that right?
I can imagine another kind of workflow where the expectation is that every submission is sent through both SMS and HTTP. Data managers would constantly be monitoring the SMS submissions but would only check HTTP every so often to complete the records and/or receive any records that failed to be received by SMS (because it's not fault-tolerant).
I think this is more likely to be the expectation for data collection campaigns in which an Internet connection is very rarely available. Then perhaps HTTP submissions would only be sent once a week or once a month when enumerators got back to an office, for example. It would be helpful to hear about those expectations from someone who would like to deploy in an area without any reliable data connection.
Whether SMS has to always be sent could be a configurable option. We then gain flexibility but sacrifice simplicity.
@joeldean and I have started a deeper brainstorming Google doc to avoid cluttering this thread too much. Please consider jumping in with comments. We can share updates as more user-facing questions come up or decisions are made.
We have written up a proposed specification for the Technical Steering Committee to consider. @Saad, I would be very interested in your comments. The document is in the roadmap repository.
Hi,
The write-up is a pretty good one, covering the existing needs as well as adding some future features as well. One thing it is not discussing, is the optimization of SMS content. I assume it is on a separate discussion thread, is it?
But the workflow for SMS handling described here is pretty decent, and should be workable.
Thanks very much for the feedback, @Saad! If you think of any changes or additions, please share.
I'm not exactly sure what you mean by the optimization of SMS content but maybe it's the compression that we had talked about in the original thread? Either way, it does sound like it would be a good thing to have a new Features topic for.
I've updated the spec at https://github.com/opendatakit/roadmap/issues/24 with some suggestions that came up during the @TAB call. Thanks all for the thoughtful review. In particular, I addressed @Shobhit_Agarwal's point about needing to maintain state to know if an SMS has been sent and added a section about alternatives considered. I also included screenshots from the current settings interface and a little narrative about each settings combination.