I am having some trouble understanding how to export a survey I have
created. Perhaps someone on the group has some suggestions.
Briefly, I have designed a survey to be used for forest biomass sampling.
The form employs a number of begin repeats that are used based on
responses. So for example, I use begin repeat for individual trees within a
sampling plot. Within the begin repeat another begin repeat is used to
accommodate trees with multiple stems. So depending on the nature of the
sampling plot I can have multiple nested begin repeats.
The survey works very well and data can be collected no problem. Form
submission also works without any problems. I can view the data on
aggregate and notice that the begin repeats (when used) return a table
within a table! Here is where I begin having problems. I am able to publish
my survey to fusion tables (managed by google drive), however, when I view
the data in Fusion tables I see that only the top level of responses have
been exported, none of the begin repeats data is available.
Secondly, when I try and export the data to CSV I am unable to perform the
export as the horizontal window bar does not move at all, i.e. I cant
scroll left to right to find the export button.
It would be great if an experienced user or developer could clarify the
following
How data collected using begin repeats can be exported to fusion tables
/ csv / JSON (most of the data of interest is stored within the begin
repeat tables)?
If aggregate is unable to export the data using the standard export
features can I employ a scripting language such as python or javascript to
access our data and store it in a suitable offline / online database?
Many thanks and looking forward to your inputs / discussion
Wesley
···
--
Dr Wesley Roberts
jwesroberts@gmail.com
Cell: 083 5355 646
skype: roberts-w
The link in Aggregate is only to the top-level form, but all the repeat
data are being exported to fusion tables (or Google Spreadsheet). Go to
your Google Drive (https://drive.google.com). You will see all your
published fusion tables and spreadsheets -- one for each top-level form
publisher, and one for each repeat group within the form. The names of the
fusion table exports correspond to the names of the repeat group elements
in your form. The top-level form's name will be the name of the xml root
element of your form.
Unfortunately, the naming of these files is less than ideal for grouping
them up if you end up publishing multiple times. I generally rename them
immediately after creating them so that I can keep which is which straight
when setting up multiple publishers (e.g., with different filters).
Once you understand the collection of fusion tables that correspond to your
one survey, you can use Fusion Table to merge them together.
To do that, open each repeat group in its own fusion tables tab. Starting
with the most-nested:
choose "Merge" and confirm that parent_uid is chosen.
Paste the URL of the enclosing group into the 'Merge with" box. Click 'Get'
confirm that the meta-instance-id is selected.
Give the combined table a name
Click "Merge tables"
You now have a flattened view of your data.
ODK Aggregate's "Export to CSV" only outputs the top-level form. To get the
repeat group data, you need to use ODK Briefcase to download a full copy of
the data to your computer, then use ODK Briefcase's "Export" capability to
export that data to CSV. Briefcase will create multiple CSV files, one for
the top-level form data, and one for each repeat group within the form. It
is then up to you to import these into Excel and join them together as you
see fit. Unfortunately, this process does not support the filtering
available in Aggregate.
Mitch
···
On Fri, Nov 30, 2012 at 12:50 AM, Wesley Roberts wrote:
Dear group,
I am having some trouble understanding how to export a survey I have
created. Perhaps someone on the group has some suggestions.
Briefly, I have designed a survey to be used for forest biomass sampling.
The form employs a number of begin repeats that are used based on
responses. So for example, I use begin repeat for individual trees within a
sampling plot. Within the begin repeat another begin repeat is used to
accommodate trees with multiple stems. So depending on the nature of the
sampling plot I can have multiple nested begin repeats.
The survey works very well and data can be collected no problem. Form
submission also works without any problems. I can view the data on
aggregate and notice that the begin repeats (when used) return a table
within a table! Here is where I begin having problems. I am able to publish
my survey to fusion tables (managed by google drive), however, when I view
the data in Fusion tables I see that only the top level of responses have
been exported, none of the begin repeats data is available.
Secondly, when I try and export the data to CSV I am unable to perform the
export as the horizontal window bar does not move at all, i.e. I cant
scroll left to right to find the export button.
It would be great if an experienced user or developer could clarify the
following
How data collected using begin repeats can be exported to fusion tables
/ csv / JSON (most of the data of interest is stored within the begin
repeat tables)?
If aggregate is unable to export the data using the standard export
features can I employ a scripting language such as python or javascript to
access our data and store it in a suitable offline / online database?
Many thanks and looking forward to your inputs / discussion
Many thanks for the information. I have found my various files and begin
repeats in my google drive. I tried to use your advice in terms of merging
the nested begin repeats and found that if you have more than one nested
repeat the meta-instance-id will not work as a joining column (This only
exists in the top tier, I have three tiers). I did however see that there
are other options ito joining fields so will explore those at a later date.
I also tested out the Briefcase application which works really well and is
an excellent tool to backup information locally.
I am not sure if anyone else has experienced this but when publishing data
to fusion tables, Aggregate exported duplicates of surveys and sometimes
only exported half or a third of the submitted surveys. I can get around
this using ODK Briefcase but would prefer to not have to download, export
and upload back into fusion tables. While experimenting with filters I
exceeded my daily quota so will experiment more tomorrow.
Once again many thanks for the help, in light of the strange
exporting behavior experienced while using Aggregate would you recommend
using ODK Briefcase in an operational environment.
Wesley
···
On Friday, November 30, 2012 8:15:45 PM UTC+2, Mitch wrote:
>
> The link in Aggregate is only to the top-level form, but all the repeat
> data are being exported to fusion tables (or Google Spreadsheet). Go to
> your Google Drive (https://drive.google.com). You will see all your
> published fusion tables and spreadsheets -- one for each top-level form
> publisher, and one for each repeat group within the form. The names of the
> fusion table exports correspond to the names of the repeat group elements
> in your form. The top-level form's name will be the name of the xml root
> element of your form.
>
> Unfortunately, the naming of these files is less than ideal for grouping
> them up if you end up publishing multiple times. I generally rename them
> immediately after creating them so that I can keep which is which straight
> when setting up multiple publishers (e.g., with different filters).
>
> Once you understand the collection of fusion tables that correspond to
> your one survey, you can use Fusion Table to merge them together.
>
> To do that, open each repeat group in its own fusion tables tab. Starting
> with the most-nested:
> choose "Merge" and confirm that *parent_uid* is chosen.
> Paste the URL of the enclosing group into the 'Merge with" box. Click
> 'Get'
> confirm that the *meta-instance-id* is selected.
> Give the combined table a name
> Click "Merge tables"
>
> You now have a flattened view of your data.
>
> ODK Aggregate's "Export to CSV" only outputs the top-level form. To get
> the repeat group data, you need to use ODK Briefcase to download a full
> copy of the data to your computer, then use ODK Briefcase's "Export"
> capability to export that data to CSV. Briefcase will create multiple CSV
> files, one for the top-level form data, and one for each repeat group
> within the form. It is then up to you to import these into Excel and join
> them together as you see fit. Unfortunately, this process does not support
> the filtering available in Aggregate.
>
> Mitch
>
> On Fri, Nov 30, 2012 at 12:50 AM, Wesley Roberts <jwesr...@gmail.com wrote:
>
>> Dear group,
>>
>> I am having some trouble understanding how to export a survey I have
>> created. Perhaps someone on the group has some suggestions.
>>
>> Briefly, I have designed a survey to be used for forest biomass sampling.
>> The form employs a number of begin repeats that are used based on
>> responses. So for example, I use begin repeat for individual trees within a
>> sampling plot. Within the begin repeat another begin repeat is used to
>> accommodate trees with multiple stems. So depending on the nature of the
>> sampling plot I can have multiple nested begin repeats.
>>
>> The survey works very well and data can be collected no problem. Form
>> submission also works without any problems. I can view the data on
>> aggregate and notice that the begin repeats (when used) return a table
>> within a table! Here is where I begin having problems. I am able to publish
>> my survey to fusion tables (managed by google drive), however, when I view
>> the data in Fusion tables I see that only the top level of responses have
>> been exported, none of the begin repeats data is available.
>>
>> Secondly, when I try and export the data to CSV I am unable to perform
>> the export as the horizontal window bar does not move at all, i.e. I cant
>> scroll left to right to find the export button.
>>
>> It would be great if an experienced user or developer could clarify the
>> following
>>
>> 1. How data collected using begin repeats can be exported to fusion
>> tables / csv / JSON (most of the data of interest is stored within the
>> begin repeat tables)?
>> 2. If aggregate is unable to export the data using the standard export
>> features can I employ a scripting language such as python or javascript to
>> access our data and store it in a suitable offline / online database?
>>
>> Many thanks and looking forward to your inputs / discussion
>>
>> Wesley
>>
>> --
>> Dr Wesley Roberts
>> jwesr...@gmail.com
>> Cell: 083 5355 646
>> skype: roberts-w
>>
>> --
>> Post: opend...@googlegroups.com
>> Unsubscribe: opendatakit...@googlegroups.com
>> Options: http://groups.google.com/group/opendatakit?hl=en
>>
>
>
>
> --
> Mitch Sundt
> Software Engineer
> University of Washington
> mitche...@gmail.com
>
ODK Aggregate must restrict the rate at which it publishes to Fusion Tables
due to rate limits imposed by Fusion Tables; if you are running without
billing, the publishing will also stop once you exceed quota (as does
everything else). It should, however, eventually publish everything through
to Fusion Tables. You just have to give it time to complete.
Note that data published to Fusion Tables may be duplicated due to certain
error conditions (especially when you exceed quota while transmitting data
to Fusion Tables). You should confirm that you do not have duplicate
records as part of your data cleansing protocol.
Mitch
···
On Mon, Dec 3, 2012 at 2:44 AM, Wes Roberts wrote:
Hi Mitch,
Many thanks for the information. I have found my various files and begin
repeats in my google drive. I tried to use your advice in terms of merging
the nested begin repeats and found that if you have more than one nested
repeat the meta-instance-id will not work as a joining column (This only
exists in the top tier, I have three tiers). I did however see that there
are other options ito joining fields so will explore those at a later date.
I also tested out the Briefcase application which works really well and is
an excellent tool to backup information locally.
I am not sure if anyone else has experienced this but when publishing data
to fusion tables, Aggregate exported duplicates of surveys and sometimes
only exported half or a third of the submitted surveys. I can get around
this using ODK Briefcase but would prefer to not have to download, export
and upload back into fusion tables. While experimenting with filters I
exceeded my daily quota so will experiment more tomorrow.
Once again many thanks for the help, in light of the strange
exporting behavior experienced while using Aggregate would you recommend
using ODK Briefcase in an operational environment.
Wesley
On Friday, November 30, 2012 8:15:45 PM UTC+2, Mitch wrote:
The link in Aggregate is only to the top-level form, but all the repeat
data are being exported to fusion tables (or Google Spreadsheet). Go to
your Google Drive (https://drive.google.com). You will see all your
published fusion tables and spreadsheets -- one for each top-level form
publisher, and one for each repeat group within the form. The names of the
fusion table exports correspond to the names of the repeat group elements
in your form. The top-level form's name will be the name of the xml root
element of your form.
Unfortunately, the naming of these files is less than ideal for grouping
them up if you end up publishing multiple times. I generally rename them
immediately after creating them so that I can keep which is which straight
when setting up multiple publishers (e.g., with different filters).
Once you understand the collection of fusion tables that correspond to
your one survey, you can use Fusion Table to merge them together.
To do that, open each repeat group in its own fusion tables tab. Starting
with the most-nested:
choose "Merge" and confirm that parent_uid is chosen.
Paste the URL of the enclosing group into the 'Merge with" box. Click
'Get'
confirm that the meta-instance-id is selected.
Give the combined table a name
Click "Merge tables"
You now have a flattened view of your data.
ODK Aggregate's "Export to CSV" only outputs the top-level form. To get
the repeat group data, you need to use ODK Briefcase to download a full
copy of the data to your computer, then use ODK Briefcase's "Export"
capability to export that data to CSV. Briefcase will create multiple CSV
files, one for the top-level form data, and one for each repeat group
within the form. It is then up to you to import these into Excel and join
them together as you see fit. Unfortunately, this process does not support
the filtering available in Aggregate.
Mitch
On Fri, Nov 30, 2012 at 12:50 AM, Wesley Roberts jwesr...@gmail.comwrote:
Dear group,
I am having some trouble understanding how to export a survey I have
created. Perhaps someone on the group has some suggestions.
Briefly, I have designed a survey to be used for forest biomass
sampling. The form employs a number of begin repeats that are used based on
responses. So for example, I use begin repeat for individual trees within a
sampling plot. Within the begin repeat another begin repeat is used to
accommodate trees with multiple stems. So depending on the nature of the
sampling plot I can have multiple nested begin repeats.
The survey works very well and data can be collected no problem. Form
submission also works without any problems. I can view the data on
aggregate and notice that the begin repeats (when used) return a table
within a table! Here is where I begin having problems. I am able to publish
my survey to fusion tables (managed by google drive), however, when I view
the data in Fusion tables I see that only the top level of responses have
been exported, none of the begin repeats data is available.
Secondly, when I try and export the data to CSV I am unable to perform
the export as the horizontal window bar does not move at all, i.e. I cant
scroll left to right to find the export button.
It would be great if an experienced user or developer could clarify the
following
How data collected using begin repeats can be exported to fusion
tables / csv / JSON (most of the data of interest is stored within the
begin repeat tables)?
If aggregate is unable to export the data using the standard export
features can I employ a scripting language such as python or javascript to
access our data and store it in a suitable offline / online database?
Many thanks and looking forward to your inputs / discussion