How to Access Photos from MySQL Database?

We have installed ODK Aggregate and we are using a MySQL Database.

I can see the data inside the _PHOTO_BIN, _PHOTO_BN, and PHOTO_REF
tables - but how do I read this data and display the image? I see in
the _PHOTO_BN table there is a field named UNROOTED_FILE_PATH which
contains a .jpg filename, but I don't know if this can actually be
used to access the image?

Thank you.

The easiest way to access the images is through the ODK Aggregate URL for
accessing attachments.

The database tables are structured so that unbounded-size images can be
stored in the database. If your image size is under 1 MB, it is stored in
one data record, but if you have larger image files (or longer streaming
videos), they are split across multiple blob records, and writing the logic
to reconstruct them will be error-prone.

The ODK Aggregate URL for, e.g., the GeoTagger form on
opendatakit.appspot.com, looks like:

http://opendatakit.appspot.com/view/binaryData?blobKey=geo_tagger_v2[%40version%3Dnull+and+%40uiVersion%3Dnull]%2Fgeotagger[%40key%3Duuid%3Af6454ba6-2485-426f-8e1a-8981e463e6fe]%2FImage&previewImage=true

If you want a small thumbnail, specify previewImage=true (as above).
If you want to download it as an attachment (or access the original
filename), you can also specify as_attachment=true, in which case it will
supply the filename as originally uploaded.

The blobKey identifies the image. As with all URLs, the parameters are
URLEncoded which makes them very ugly. The URLDecoded blobKey looks like:

geo_tagger_v2[@version=null and
@uiVersion=null]/geotagger[@key=uuid:f6454ba6-2485-426f-8e1a-8981e463e6fe]/Image

This is an XPath-style naming for the image element in the Xform. It
identifies the form id: get_tagger_v2, the version and uiVersion values
(from the attributes on the Xform's top-level element), the name of the
top-level element in the form (geotagger), the primary key for the
submission being referenced (uuid:f6454ba6-2485-426f-8e1a-8981e463e6fe) and
the XPath within the form to the image. In this case, the image (field
name Image) is immediately underneath the top-level element in the form.
If you had repeat groups, you would have to specify the ordinal (1..n) of
the repeat group. So for the first repeat group, you would have:

.../repeatElement[@ordinal=1]/Image

and for the 5th repeat, you would specify:

.../repeatElement[@ordinal=5]/Image

Mitch

··· On Mon, Jan 30, 2012 at 9:24 AM, Feltpad Incorporated wrote:

We have installed ODK Aggregate and we are using a MySQL Database.

I can see the data inside the _PHOTO_BIN, _PHOTO_BN, and PHOTO_REF
tables - but how do I read this data and display the image? I see in
the _PHOTO_BN table there is a field named UNROOTED_FILE_PATH which
contains a .jpg filename, but I don't know if this can actually be
used to access the image?

Thank you.

--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

Another way (perhaps easier) to access the files on a harddrive is to
use the ODK Briefcase tool to download your forms into Briefcase's
directory structure. In the process Briefcase will construct the
binary data back into a JPG and you can copy the pictures you want out
of the directory.

Waylon

··· On Mon, Jan 30, 2012 at 10:26 AM, Mitch S wrote: > The easiest way to access the images is through the ODK Aggregate URL for > accessing attachments. > > The database tables are structured so that unbounded-size images can be > stored in the database. If your image size is under 1 MB, it is stored in > one data record, but if you have larger image files (or longer streaming > videos), they are split across multiple blob records, and writing the logic > to reconstruct them will be error-prone. > > The ODK Aggregate URL for, e.g., the GeoTagger form on > opendatakit.appspot.com, looks like: > > http://opendatakit.appspot.com/view/binaryData?blobKey=geo_tagger_v2%5B%40version%3Dnull+and+%40uiVersion%3Dnull%5D%2Fgeotagger%5B%40key%3Duuid%3Af6454ba6-2485-426f-8e1a-8981e463e6fe%5D%2FImage&previewImage=true > > If you want a small thumbnail, specify previewImage=true (as above). > If you want to download it as an attachment (or access the original > filename), you can also specify as_attachment=true, in which case it will > supply the filename as originally uploaded. > > The blobKey identifies the image. As with all URLs, the parameters are > URLEncoded which makes them very ugly. The URLDecoded blobKey looks like: > > geo_tagger_v2[@version=null and > @uiVersion=null]/geotagger[@key=uuid:f6454ba6-2485-426f-8e1a-8981e463e6fe]/Image > > This is an XPath-style naming for the image element in the Xform. It > identifies the form id: get_tagger_v2, the version and uiVersion values > (from the attributes on the Xform's top-level element), the name of the > top-level element in the form (geotagger), the primary key for the > submission being referenced (uuid:f6454ba6-2485-426f-8e1a-8981e463e6fe) and > the XPath within the form to the image. In this case, the image (field name > Image) is immediately underneath the top-level element in the form. If you > had repeat groups, you would have to specify the ordinal (1..n) of the > repeat group. So for the first repeat group, you would have: > > .../repeatElement[@ordinal=1]/Image > > and for the 5th repeat, you would specify: > > .../repeatElement[@ordinal=5]/Image > > Mitch > > > On Mon, Jan 30, 2012 at 9:24 AM, Feltpad Incorporated wrote: >> >> We have installed ODK Aggregate and we are using a MySQL Database. >> >> I can see the data inside the _PHOTO_BIN, _PHOTO_BN, and PHOTO_REF >> tables - but how do I read this data and display the image? I see in >> the _PHOTO_BN table there is a field named UNROOTED_FILE_PATH which >> contains a .jpg filename, but I don't know if this can actually be >> used to access the image? >> >> Thank you. >> >> -- >> Post: opendatakit@googlegroups.com >> Unsubscribe: opendatakit+unsubscribe@googlegroups.com >> Options: http://groups.google.com/group/opendatakit?hl=en > > > > > -- > Mitch Sundt > Software Engineer > University of Washington > mitchellsundt@gmail.com > > -- > Post: opendatakit@googlegroups.com > Unsubscribe: opendatakit+unsubscribe@googlegroups.com > Options: http://groups.google.com/group/opendatakit?hl=en

When I create my URL and try it in my browser, it asks me to log into
Aggregate. We are attempting to create an interface for our users, so
they do not need to login to Aggregate.

Is there a way to do this so users don't have to login to Aggregate?

My bad. When I create the table, I am erroneously using a "BLOB" which has
the 65k limit. I had meant to change this to "LONGBLOB" after testing.

http://code.google.com/p/opendatakit/source/browse/src/main/java/org/opendatakit/common/persistence/engine/mysql/DatastoreImpl.java?repo=aggregate&r=c7fbd495524da9dadbe0832936b34d43dece380c#425

The fix will be in the official ODK Aggregate 1.1 release.

If you need this now, you can stop the ODK Aggregate server, use a MySQL
database admin tool or the command prompt to issue a command on each
..._BLB table:

ALTER TABLE tablename MODIFY COLUMN VALUE longblob NOT NULL;

When ODK Aggregate restarts, since these columns are now defined as
LONGBLOBs, it will automatically adjust how it breaks up new submissions to
the table. The existing data will not be affected and will have the
earlier 65KB chunk size.

Mitch

··· On Wed, Jun 27, 2012 at 7:24 AM, wrote:

Is there any place in the code that determines this 1MB maximum size per
blob? I am finding that even if I submit an image of 85KB, that this will
be split into 2 blob parts. I am not sure if someone has 'tampered' with a
setting somewhere which may be causing this.

Thanks.

Paul.

--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

You can stream the data to Google Fusion Tables or Spreadsheets and
visualize it there (and embed it in a website). If it's static data
you can also export a KML they can use in Google Maps or Google Earth.

··· On Tue, Jan 31, 2012 at 07:44, Feltpad Incorporated wrote: > When I create my URL and try it in my browser, it asks me to log into > Aggregate. We are attempting to create an interface for our users, so > they do not need to login to Aggregate. > > Is there a way to do this so users don't have to login to Aggregate? > > -- > Post: opendatakit@googlegroups.com > Unsubscribe: opendatakit+unsubscribe@googlegroups.com > Options: http://groups.google.com/group/opendatakit?hl=en

Yaw's suggestion doesn't work because the images are still hosted off the
Aggregate website. However, there is a solution:

Under the Site Admin tab, on the Permissions sub-tab, beneath the Edit
Users table of users and above the Save Changes button, there is a checkbox
for:

Allow anonymous retrieval of images, audio and video data (needed for
GoogleEarth balloon displays)
*

··· * Check that checkbox, and save the permissions changes. This will allow anonymous access to all attachments. Given that you need to know the instanceID for the data being retrieved, it is reasonably secure to make this change.

I'm working on OAuth2 support in Aggregate which would provide an entirely
secure way to access the images.

Mitch

On Tue, Jan 31, 2012 at 9:08 AM, Yaw Anokwa yanokwa@gmail.com wrote:

You can stream the data to Google Fusion Tables or Spreadsheets and
visualize it there (and embed it in a website). If it's static data
you can also export a KML they can use in Google Maps or Google Earth.

On Tue, Jan 31, 2012 at 07:44, Feltpad Incorporated feltpadinc@gmail.com wrote:

When I create my URL and try it in my browser, it asks me to log into
Aggregate. We are attempting to create an interface for our users, so
they do not need to login to Aggregate.

Is there a way to do this so users don't have to login to Aggregate?

--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en

--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

Mitch instructions about how to remove the security on the photos is
the solution I would suggest.

I wanted to reiterate that the Site Admin tab allows you to control
the permissions for Aggregate. You can setup Aggregate so it does not
ask for any login to do anything. It is all under your control.

··· On Tue, Jan 31, 2012 at 10:18 AM, Mitch S wrote: > Yaw's suggestion doesn't work because the images are still hosted off the > Aggregate website. However, there is a solution: > > Under the Site Admin tab, on the Permissions sub-tab, beneath the Edit Users > table of users and above the Save Changes button, there is a checkbox for: > > Allow anonymous retrieval of images, audio and video data (needed for > GoogleEarth balloon displays) > > Check that checkbox, and save the permissions changes. This will allow > anonymous access to all attachments. Given that you need to know the > instanceID for the data being retrieved, it is reasonably secure to make > this change. > > I'm working on OAuth2 support in Aggregate which would provide an entirely > secure way to access the images. > > Mitch > > > On Tue, Jan 31, 2012 at 9:08 AM, Yaw Anokwa wrote: >> >> You can stream the data to Google Fusion Tables or Spreadsheets and >> visualize it there (and embed it in a website). If it's static data >> you can also export a KML they can use in Google Maps or Google Earth. >> >> On Tue, Jan 31, 2012 at 07:44, Feltpad Incorporated wrote: >> > When I create my URL and try it in my browser, it asks me to log into >> > Aggregate. We are attempting to create an interface for our users, so >> > they do not need to login to Aggregate. >> > >> > Is there a way to do this so users don't have to login to Aggregate? >> > >> > -- >> > Post: opendatakit@googlegroups.com >> > Unsubscribe: opendatakit+unsubscribe@googlegroups.com >> > Options: http://groups.google.com/group/opendatakit?hl=en >> >> -- >> Post: opendatakit@googlegroups.com >> Unsubscribe: opendatakit+unsubscribe@googlegroups.com >> Options: http://groups.google.com/group/opendatakit?hl=en > > > > > -- > Mitch Sundt > Software Engineer > University of Washington > mitchellsundt@gmail.com > > -- > Post: opendatakit@googlegroups.com > Unsubscribe: opendatakit+unsubscribe@googlegroups.com > Options: http://groups.google.com/group/opendatakit?hl=en

Hello There,

I am using tomcat 8 with ODK aggregate 1.4.15, I developped a nodejs server to consumed published data from my aggregate and I prefere to receive images as URL so that I can load them separately. Everything worked fine but when I have configured a DNS, SSL/TSL and Realm (Basic authentication) for my tomcat, I can not download the image anymore, I am receiving 401 unauthorized (Full
authentication is required to access this resource).

After investigation, I understood that aggregate is not using the built in tomcat security system (Realm), I have discovered a file called applicationContext-security.xml where some security rules are defined and I have also seen the rule for the url or resource I need to access :

So my question is what username and password I must use to access this url?