Collect upload error: write error: ssl=0x9d03b600: I/O error during system call, broken pipe

Hi,

I have created a survey form and routed it to the google cloud. When I try to upload the saved form to google cloud, I am getting "Error: Generic Exception: write error: ssl=0x9d03b600: I/O error during system call, broken pipe". I tried restarting/rebooting the device, still the same error. Please let me know what could have gone wrong and how to fix it.

Thank you.

Regards,
Kiran

Hi Kiran,

Happy to help! But not really clear about the issue. Are you trying to upload the empty form to Aggregate server? Or you are trying to upload filled out form from Collect to the Aggregate server?

Thanks

I believe @A.N.M_AL-IMRAN is correct that this is occurring when sending a filled out form from Collect to Aggregate hosted on AppEngine. Is that right, @Kiran? I have experienced this when trying to upload very large media attachments. Are you sending media attachments? Do you know what their rough size is?

I believe this is something that can be addressed by a server configuration change but I'm not entirely sure. Does anyone know of a way to accept larger binaries on AppEngine? That would be something like nginx's client_max_body_size.

1 Like

Hi,

Thank you for your time in replying. It is not an empty form, which I failed to upload on aggregate.

Regards,

Kiran

From:
noreply@forum.getodk.org [mailto:noreply@forum.getodk.org]

Hi Imran,

Thank you for your time in replying. Yes, I did try to upload filled form to aggregate and I am getting this error. Please let me know what could be wrong in here.

Regards,
Kiran

Dear Helen Martin,

Thank you for your in replying to my query. Yes, I am trying to upload media attachments, roughly the size of my video file is about 90 MB. Previously I tried uploading a media file (video) of 13 MB, it was successfully uploaded. Please help me on this.

Regards,
Kiran

Hi, all!

I'm able to reproduce this issue while debugging Collect. I'll be using this comment with my findings:

This is the stacktrace I'm getting:

11-26 13:03:25.507 6169-6308/org.odk.collect.android E/HttpClientConnection: javax.net.ssl.SSLException: Write error: ssl=0xa93ec580: I/O error during system call, Broken pipe
        at com.android.org.conscrypt.NativeCrypto.SSL_write(Native Method)
        at com.android.org.conscrypt.OpenSSLSocketImpl$SSLOutputStream.write(OpenSSLSocketImpl.java:839)
        at org.opendatakit.httpclientandroidlib.impl.io.SessionOutputBufferImpl.streamWrite(SessionOutputBufferImpl.java:126)
        at org.opendatakit.httpclientandroidlib.impl.io.SessionOutputBufferImpl.flushBuffer(SessionOutputBufferImpl.java:138)
        at org.opendatakit.httpclientandroidlib.impl.io.SessionOutputBufferImpl.write(SessionOutputBufferImpl.java:169)
        at org.opendatakit.httpclientandroidlib.impl.io.ContentLengthOutputStream.write(ContentLengthOutputStream.java:115)
        at org.opendatakit.httpclientandroidlib.entity.mime.content.FileBody.writeTo(FileBody.java:121)
        at org.opendatakit.httpclientandroidlib.entity.mime.AbstractMultipartForm.doWriteTo(AbstractMultipartForm.java:134)
        at org.opendatakit.httpclientandroidlib.entity.mime.AbstractMultipartForm.writeTo(AbstractMultipartForm.java:157)
        at org.opendatakit.httpclientandroidlib.entity.mime.MultipartFormEntity.writeTo(MultipartFormEntity.java:113)
        at org.opendatakit.httpclientandroidlib.impl.DefaultBHttpClientConnection.sendRequestEntity(DefaultBHttpClientConnection.java:158)
        at org.opendatakit.httpclientandroidlib.impl.conn.CPoolProxy.sendRequestEntity(CPoolProxy.java:162)
        at org.opendatakit.httpclientandroidlib.protocol.HttpRequestExecutor.doSendRequest(HttpRequestExecutor.java:237)
        at org.opendatakit.httpclientandroidlib.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:122)
        at org.opendatakit.httpclientandroidlib.impl.execchain.MainClientExec.execute(MainClientExec.java:271)
        at org.opendatakit.httpclientandroidlib.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
        at org.opendatakit.httpclientandroidlib.impl.execchain.RetryExec.execute(RetryExec.java:88)
        at org.opendatakit.httpclientandroidlib.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
        at org.opendatakit.httpclientandroidlib.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
        at org.opendatakit.httpclientandroidlib.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
        at org.opendatakit.httpclientandroidlib.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
        at org.odk.collect.android.http.HttpClientConnection.uploadSubmissionFile(HttpClientConnection.java:380)
        at org.odk.collect.android.upload.InstanceServerUploader.uploadOneSubmission(InstanceServerUploader.java:183)
        at org.odk.collect.android.tasks.InstanceServerUploaderTask.doInBackground(InstanceServerUploaderTask.java:76)
        at org.odk.collect.android.tasks.InstanceServerUploaderTask.doInBackground(InstanceServerUploaderTask.java:39)
        at android.os.AsyncTask$2.call(AsyncTask.java:304)
        at java.util.concurrent.FutureTask.run(FutureTask.java:237)
        at android.os.AsyncTask$SerialExecutor$1.run(AsyncTask.java:243)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1133)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:607)
        at java.lang.Thread.run(Thread.java:761)
  • Collect doesn't respect max content-length negotiated in the HEAD request prior to a submission POST when a single attached file is greater than what the server can receive i.e. it sends the full request anyway without splitting the payload in several requests.
  • Not getting the error on the dev (Jetty, no HTTPS) server
  • The POST request (after the initial HEAD) doesn't reach Aggregate. It fails before sending it
  • So far, I've been able to reproduce this in https://aggregate-gigi.appspot.com and https://opendatakit.appspot.com
  • Collect v1.16.4 (previous to the HTTP refactor) and v1.4.9_rev_1059 also have this problem
  • According to this, there's a quota limit in AppEngine for incoming request that requires them to be less than 32MB big.
1 Like