Thank you Lloyd and Mitch for getting to the bottom of this. We're having
the same issue and are relieved to hear that this error message doesn't
indicate any loss of data!
Re-posting an offline reply from Mitch here (I took the conversation
offline as I granted him developer access) that brings this issue to a
close.
"OK. Looking at the logs, I think everything is working as designed.
The form being displayed on the Submissions tab takes more than 60
seconds to completely fetch 100 rows from the database.
AppEngine imposes a 60-second limit, after which it aggressively
terminates any outstanding database calls, and the webserver detects and
returns that error up to the webpage. Issuing those queries directly in
the management console (
https://console.developers.google.com/project/tangointernational-3/datastore/query
) works fine and the results look like they should.
If you change the "Submissions per Page" to something less (e.g., 50),
the error should go away and you should be able to see the data in the
submissions tab.
If memory serves, that setting choice is not remembered once you leave
the page, so you would always have to re-enter it."
I can confirm that reducing the submissions per page seems to negate
this error.
Thank you Mitch for to diligent follow-up and assistance.
On Tuesday, March 24, 2015 at 10:58:28 AM UTC-4, Lloyd Owen Banwart wrote:
ODK Briefcase is working without failures. I do have paid instances.
On Tuesday, March 24, 2015 at 10:45:49 AM UTC-4, Mitch Sundt wrote:
And ODK Briefcase works to download the data, or does it give a failure?
If Briefcase is working OK without failures, and if you have a paid
instance, I would contact Google support, as this is a repeatable issue.
You should probably CC me on the support request you open with them, and
grant me developer access to the instance (and let me know the instance
URL) https://code.google.com/p/opendatakit/wiki/AppEngineTroubleshooting
On Mon, Mar 23, 2015 at 7:20 AM, Lloyd Owen Banwart lloyd....@gmail.com wrote:
To follow upon this post.
I had decreased access to the server for 24 hours to two users, and this
did not resolve the issue of receiving the below error. I then steadily
increased the frontend instance Class from 256 to 512 to 1024. In each
case, the error still was displayed on the ODK server. This error only
displays when viewing the data on the "Submissions tab" and under "Filter
Submissions" (no filter is applied). The error is repetitive, meaning that
when I leave the browser open, the error will display, instances may (or
may not) be shown in the submissions tab, then the error will pop up again.
Each time the error displays the error appears identical other than the
uuid, specific Q it is referencing, and start and end time of request.
I've tracked most uuids that have shown in the errors, and cross referenced
this with data obtained after the error has been displayed. All instances
displayed during errors are in the data which have been downloaded, and the
instances appear unaffected.
When accessing the data (via the "Export" function or via ODK Briefcase)
there appears to be no data loss when check against field-records of
uploaded instances compared to instances obtained from the server. Other
than being a annoying, and limiting any quick review of recently submitted
data via the server, I don't see any major issues at the moment with the
error -- but would love to get to the bottom of it.
Second example of error:
Error: Problem persisting
data or accessing data ([ChoiceSubmissionType.getValueFromEntity] SQL:
SELECT *
FROM opendatakit.ASUBL_FINAL2_ELG_B_SEC2_SEC3_MM_G3_LABELM3_2 WHERE
_PARENT_AURI = uuid:e1ba7512-9c8e-47d1-8458-0ddc1935642b ORDER BY
_PARENT_AURI
exception: This request (f44144cff1ba1b4f) started at 2015/03/23
13:51:54.645
UTC and was still executing at 2015/03/23 13:52:54.505 UTC.)
On Monday, March 23, 2015 at 9:39:14 AM UTC-4, Lloyd Owen Banwart wrote:
Mitch. Thanks (as always) for the clear and timely feedback.
-
I've limited access, but it was a total of 3 with access before (now
it is a total of 2) to the server.
-
We use ODK briefcase (as we avail of the Stata ODKmeta program to
provide value/var labels and restructure data sets from repeat groups).
-
Re: Increasing the machine size; this is one of the smaller data
collections we have done. The survey is large, but we are collecting data
from less than 2,000 households --- is there a reason this issue would
arise now and it hasn't in the past? Regardless, I'm waiting to see if
limiting access to the server addresses this issue, if it does not I will
increase the machine size.
I'll post progress here in the event someone else faces this issue in
the future. Thanks again.
~Lloyd
On Friday, March 20, 2015 at 10:35:18 PM UTC-4, Mitch Sundt wrote:
Several things could be going on:
(1) if many users are all watching the ODK Aggregate webpages at the
same time, they may be competing for processing space and time. Restrict
access to the server and provide nightly reports or publish the data into
Google Fusion Tables and provide access to that.
(2) if you are using "Export to CSV" to produce reports, switch to using
ODK Briefcase to download data to your local computer and then export to
CSV from there. The Export functionality on ODK Aggregate holds all of the
dataset in memory when generating a report. This will not work beyond a
certain size dataset.
(3) increase the "Frontend Instance Class" (machine size) on the
AppEngine ( https://appengine.google.com/ ). Choose your application id,
then click on Application Settings on the left sidebar. And select a larger
Frontend Instance Class.
Note that increasing the machine size will increase your costs.
On Fri, Mar 20, 2015 at 10:31 AM, Lloyd Owen Banwart < lloyd....@gmail.com> wrote:
Hi all,
We have a survey going live at the moment, with data being uploaded on a
daily basis. We are using a server a google app engine deployment of ODK
Aggregate version 1.4.3. It was my intention to update this to the latest
version of Aggregate once this fieldwork was completed.
To date, there has been no issues, but starting today when viewing data
on the "Submission" page of our appspot page for this server, I have been
getting the below error. The error is repetitive, but each time the uuid
case number changes.
"Error: Problem persisting data or accessing
data([ChoiceSubmissionType.getValueFromEntity] SQL: SELECT * FROM
opendatakit.ASUBL_FINAL2_ELG_B_SEC2_MJ_G6_J10C WHERE _PARENT_AURI =
uuid:6219f226-5a47-4501-b439-061dfce52d34 ORDER BY _PARENT_AURI
exception: This
request (2d170ed258bfabee) started at 2015/03/20 16:56:04.949 UTC and
was still
executing at 2015/03/20 16:57:04.779 UTC.)"
In the app engine application, when looking at the log, there are a
number of Info messages but thy all vary, and one warning message (see
below):
W2015-03-20 10:17:58.727 A problem was encountered with the process that
handled this request, causing it to exit. This is likely to cause a new
process to be used for the nex
Any advice on how to address this error, and if it will cause a
potential loss of data (all data has been backed up) in the future?
Thanks,
~Lloyd
--
--
Post: opend...@googlegroups.com
Unsubscribe: opendatakit...@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to the Google
Groups "ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
Mitch Sundt
Software Engineer
University of Washington
mitche...@gmail.com
--
--
Post: opend...@googlegroups.com
Unsubscribe: opendatakit...@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to the Google
Groups "ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
Mitch Sundt
Software Engineer
University of Washington
mitche...@gmail.com