Dear All,
I must thank Lindsey for her suggestion:
The problem that was breaking the do-file run was quite simple: ODK Aggregate exports the csv file with ":" instead of "-" when naming the nested groups-- the solution, as suggested by Lindsey was to do a simple find/replace and run it again.
Its working now!
Warm Regards
Adeeb
Dear Lindsey,
I don't know if this is the right place to post this particular problem (apologies Yaw if it isn't), but my thinking is that odkmeta has great potential and this problem is really a vexing one.
I have manage to successfully create a do-file for STATA using odkmeta but the run breaks every time it comes to the following:
- Duplicate variable name with grp_hh_mem-grp_hhmem_brk-totalchild_6to18
. local pos : list posof "grp_hh_mem-grp_hhmem_brk-totalchild_below6" in fields
. local var : word pos' of
all'
invalid syntax
r(198);
end of do-file
r(198);
As far as I can tell this happens for every variable name which exceeds the 32 character limit. I have perused the help file that came with odkmeta and it seems that the way that odkmeta handles naming of variables in nested group creates this particular type of command sequence each time (pulled from the do-file)
- totalchild_below6
- Duplicate variable name with grp_hh_mem-grp_hhmem_brk-totalchild_6to18
local pos : list posof "grp_hh_mem-grp_hhmem_brk-totalchild_below6" in fields
local var : word pos' of
all'
char var'[Odk_name] totalchild_below6 char
var'[Odk_bad_name] 1
char var'[Odk_group] grp_hh_mem grp_hhmem_brk char
var'[Odk_long_name] grp_hh_mem-grp_hhmem_brk-totalchild_below6
char var'[Odk_type] integer char
var'[Odk_or_other] 0
char var'[Odk_is_other] 0 char
var'[Odk_label] A17. Number of children in HH under age 6
Another Example:
- gr_who_ecoorn
- Duplicate variable name with grp_hh_decision_shared-gr_who_ecoland
local pos : list posof "grp_hh_decision_shared-gr_who_ecoorn" in fields
local var : word pos' of
all'
char var'[Odk_name] gr_who_ecoorn char
var'[Odk_bad_name] 1
char var'[Odk_group] grp_hh_decision_shared char
var'[Odk_long_name] grp_hh_decision_shared-gr_who_ecoorn
char var'[Odk_type] select_one gr_who_list char
var'[Odk_list_name] gr_who_list
char var'[Odk_or_other] 0 char
var'[Odk_is_other] 0
char `var'[Odk_label] A25.13 Purchase/selling of ornaments
Given how many nested groups we have in this survey okdmeta creates this type of command sequence many many times.
Can you suggest a workaround?
Warm Regards
Adeeb Choudhury
(again, apologies to Yaw if this isn't the right place to post this)
Dear Stata users of the ODK community,
IPA is happy to announce that odkmeta v1.1 is now publicly available on SSC! In Stata, simply type the command -ssc install odkmeta- to install the program.
What does -odkmeta- do? In a nutshell/as a reminder, -odkmeta- automatically generates a do-file to clean and merge your raw Open Data Kit data. More specifically, -odkmeta- creates a do-file to import ODK data, using the metadata from the survey and choices worksheets of the XLSForm. The do-file completes the following tasks in order: imports lists as value labels; adds other values to value labels; imports field attributes as characteristics; splits select_multiple variables; drops note variables; formats date, time, and datetime variables; attaches value labels; attaches field labels as variable labels and notes; and merges repeat groups.
The help file is extensive, but questions, concerns, and suggestions may be sent directly to my email address.
Thanks again to all who helped test the program,
Lindsey, Matt and IPA
Hi ODK Community,
At IPA, our statistical package of choice is Stata -- and, increasingly, our survey platform of choice is ODK. So, we've written a Stata program, called -odkmeta-, that reads your xlsform in order to automatically generate a Stata do-file that produces one very clean, well-labeled, analysis-ready Stata dataset -- even if you are using repeat groups.
Why are we messaging this group? If you are an ODK and Stata user, we'd love for you to test -odkmeta- before we publicly release the program!
More specifically on the program's functionality, -odkmeta- reads the survey and choices worksheets of your xlsform and generates a do file which:
o Imports lists as value labels
o Adds other values to value labels
o Imports ODK field attributes as characteristics
o Splits select_multiple variables
o Drops note variables
o Formats dates, time, and datetime variables
o Attaches field labels as variable labels and notes
o Attaches value labels
o Merges repeat groups
After select_multiple variables have been split, tasks can be removed from the do-file without affecting other tasks. User-written supplements to the do-file may make use of any field attributes, which are imported as characteristics. An extensive Stata help file outlines simple adjustments that you, as the end-user, can make to the resulting do-file in case of errors related to non-ODKstandard csv data files (e.g. varying date-time formats). We are also hoping to further improve the program's help file through our interactions with you.
Please see attached for a package of files that demonstrate -odkmeta- functionality: an example xlsform, dummy csv data files, the Stata do-file automatically generated by -odkmeta-, and the Stata dataset that was created by this do-file. The program itself, and the help file, are also included, so that you can run the program yourself on your own forms and data.
We have extensively tested -odkmeta- and intend to release it publicly on SSC in January, but are first looking to see how well the program works with any additional real-world forms we can, especially if those forms are complex. Please feel free to email me directly if you have trouble running the program, if you discover any bugs, or if you would like some assistance modifying your do-file. And please also email me directly if you are able to successfully run the program!
And finally, a preemptive statement. Here are some things we don't intend to do, at least at the moment: write a similar program that reads XML rather than XLSForms; write a similar program for R or SPSS; or overhaul the program to handle ever-changing quirks of the way certain ODK-based services handle output csv data (e.g. Formhub's n/a values). If you are looking to automatically generate Stata do-files from your XML (or from your XLSForm, and for other great features), we recommend you set up an account with SurveyCTO.
Thanks in advance for helping us to complete our final tests of -odkmeta-!
Lindsey, Matt, and IPA
--
Lindsey Shaughnessy
Data Coordinator
Innovations for Poverty Action
--
Lindsey Shaughnessy
Data Coordinator
Innovations for Poverty Action
Mobile: +1 508 932 6435
Skype: lindsey.shaughnessy
Email: lshaug...@poverty-action.org
I'm not quite following what you did. In my csv file the groups were separated by forward slashes (so group_1/subgroup_1/var1). To overcome an earlier error I did a find/replace in the csv file to turn this into group_1-subgroup1-var1.