Upcoming changes to Aggregate

odk-aggregate
#1

Aggregate v2.x is scheduled to be released in early February. This release has major changes that have been discussed by the Technical Steering Committee (TSC) behind Aggregate and I wanted to explain the rationale behind these changes and provide some guidance for Aggregate users.

Summary
Aggregate v2.x will not be installable on Google App Engine (GAE, appspot.com). You will only be able to install Aggregate v2.x on a standard cloud server (Digital Ocean, Vultr, Linode, Amazon EC2, Google CE, Azure VM) or on a local server.

Aggregate v2.x also removes features that are unsupported or deprecated (e.g., Google Fusion Tables, ODK Tables, and Google Accounts).

As far as ongoing development work on ODK servers, our focus will be:

  • Aggregate v1.7.x: Critical bug fixes
  • Aggregate v2.x: Minor features
  • Central: Major features

If you use Google App Engine
You can continue using the Aggregate v1.x series, but we will only ship critical bug fixes. If you depend on Aggregate, upgrade to ODK Aggregate v1.7. Consider migrating to the more maintainable v2.x series on a standard cloud server when it's released. We'll provide more documentation to help in the coming weeks.

If you don't use Google App Engine
You can use the Aggregate v2.x series. We will continue to ship minor features with the goal of making the codebase more maintainable. Consider trying the Central betas to prepare for an eventual switch.

Why we are removing App Engine support
When we launched Aggregate in 2008, we chose GAE as the default platform because it provided easy-to-use, free, secure, and scalable infrastructure for users. The trade-off was that we would have to write custom code to deal with the various restrictions GAE had.

Since then, there have been hundreds of thousands of data collection campaigns that have run on GAE. We are proud of what we've achieved, and in order to achieve more, we have to revisit our assumptions.

Our assumptions in 2008 were that GAE would get easier and more free. That has simply not been the case. Over the last decade, GAE has become more difficult and more expensive. The key facts are:

  • GAE is harder to setup than cloud servers from providers like Digital Ocean, Linode, and Vultr.
  • GAE has a free tier, but Google requires a credit card or bank details on file. It's unlikely GAE will be free forever.
  • GAE's complicated quotas make it hard for users to anticipate costs ahead of time.
  • GAE's restrictions (e.g., killing long-running tasks, non-standard database) make maintaining Aggregate difficult and fixing user problems (especially at scale) very, very painful.

Given our available resources, it's unsustainable to continue GAE support. For that reason, we've removed GAE support in Aggregate v2.x.

Why major features aren't being added to Aggregate
Aggregate is built on technology that makes it hard to evolve. The TSC believes that over time, Central will grow into a better platform for addressing some of the major server features that the community has requested.

We continue to need your help adding documentation and minor features to Aggregate. If you depend on it, please head to GitHub and help. It's never been easier to contribute.

Does Aggregate v2 have anything to do with ODK 2
Nope. The v2 designation for Aggregate is because we use semantic versioning and when we make changes that are not backwards compatible, we must increment the major version number.

Older versions of Aggregate did have ODK 2 support, but all ODK 2 code has been removed from Aggregate. If you use ODK 2, try the Sync Endpoint instead.

I have a question you didn't answer
Ask it below :point_down:

8 Likes

I cannot find google app engine (in choose platform option) when i try to install ODK aggregate
ODK aggregate, google app deployment Java 8
Community Update - 2019-04-11
ODK 1 on GCP App Engine
CLI tool to manage Aggregate servers
#2

Dear Yaw,

No question! Just compliment from my end for v2.x series. This would be a great turning point for all ODK users and life will be much easier. Would love to move into v2.x series once it will be up. Just a question regarding documentation, will it be available under ODK 1? I believe more user would prefer either to go for Digital Ocean or EC2. More comprehensive documentation targeting less tech savvy users would be helpful.

Thanks,
Imran

2 Likes

#3

@ggalmazor is been working on documentation (and automation so less documentation is needed). My hope is to review and merge that before the release this weekend.

1 Like

#4

Since ODK2 won't be compatible with Aggregate, it would be nice to have the Getting Started Deployment Architect Guide for ODK2 updated to use Sync Endpoint.

0 Likes

split this topic #5

A post was split to a new topic: Incomplete submission in Aggregate

0 Likes

split this topic #6

A post was split to a new topic: Facing problem with my ODK Aggregate account

0 Likes

#7

Is there an install documentation for Aggregate v2 on Google Compute Engine?

The install documentation for Aggregatev1 on Google App Engine was very good.

0 Likes

#8

@CarlBeltran Please read the first post on this topic. It answers your question.

0 Likes