ODK 2.0 Documentation migration to docs.opendatakit.org -- Request for Review

I have just opened a pull request to merge the ODK 2.0 documentation from here into docs.opendatakit.org. Here is a link to the pull request.

The content of the docs is largely the same as on the website, but I have attempted to apply the ODK style guidelines to them, and break them up somewhat into a similar file structure as the rest of the docs. There are also some error fixes and other tweaks.

This is a huge PR, and I’m hoping to get some eyes on it from both the existing docs contributors as well users of ODK 2.0. I assume this PR won’t go in exactly as it stands now but I am hoping to have a good discussion about how best to merge this in and about the future direction of the ODK 2.0 docs.

This PR is great. But as we’ve discussed in several places, we have to figure out where exactly ODK 2 Docs are going.

There are at least three workable solutions for ODK 2 Docs:

  • Include them with ODK Docs
  • Create a new repo and Sphinx site
  • Host the source in the same repo,
    but publish them as a separate Sphinx site

Anyone can still advocate for the first two options,
but I’m thinking the last one
— a monorepo that builds out two sites —
is potentially the best.

Here are my thoughts on the matter…

Advantages

Essentially, I think this gives us the best of both worlds.

Advantages of shared repo:

  • Share common docs like the contributor guide, style guide, and perhaps top level way-finding
  • Share common utilities like style linting and spell checking
  • Share workflow and deployment tooling
  • Share common config values

Advantages of separate sites:

  • Readers won’t get confused about where they are.
  • Search will be specific to the tools a reader is looking for
  • Each suite can organize its content as needed

I don’t see any disadvantages to this approach,
but there are some things to watch out for.

Things to watch out for

  • We’ll need to be extra mindful in tagging issues,
    to make sure its always clear what something is about.
  • Changes to shared infrastructure or docs needed by one suite
    will have to consider the effect on the other suite.

Here’s the setup I imagine:

The Setup

  • three src directories

    • odk1-src
    • odk2-src
    • shared-src
      • Introductory Wayfinding and Contributor’s Guide pages
        using conditional formatting to include/exclude sections
        only relevant to one project
  • two build directories

    • odk1-build
    • odk2-build
  • shared template files in the root
    (These are currently in src, but I think we can use conf.py
    to specify a directory in the root. Otherwise, the build script
    can copy them in.)

  • Each site-specific src dir has its own conf.py,
    which makes each its own Sphinx site.

    • These are sparse, as they both import
      a repo-level config file with all shared config values
  • CircleCI only builds and publishes one site or the other
    on commits that are limited to odk1-src or odk2-src.
    All other commits rebuild both.

  • The first step in the build script is to copy shared-src
    (and shared templates, if needed)
    into the site-specific src directory.

  • Shared utilities (./b.sh, style-test.py) will need additional options
    to run on only one doc set or the other, or both.
    (This is a good opportunity to clean those up
    and move them all into util.)

  • Contributor guides will need to be updated.

This generally “feels right” to me, FWIW. :+1:

Am I understanding your thinking correctly that the “shared pages” would be duplicated across each site?

Personally, I’d also encourage both sites to share a standard design & layout, with perhaps a unique color scheme for each suite. Over time perhaps these could be harmonized with the OpenDataKit.org site as well, to make everything feel very professional and connected. (Step by step!)

Not exactly related to this topic specifically, but how reasonable would it be to have “staging sites” for each documentation site? (Perhaps branch-based?) This could allow “non-contributors” to review potential documentation for clarity, etc. Maybe there’s not yet demand for this, though … responses/thoughts appreciated!

That is what I said, though I can see potential problems with it.

An alternative would be a third Sphinx site for Contributing, but that might overkill.
Another alternative would be that only one site has the Contributor’s guide and the other site just links to it.

Personally, I’d also encourage both sites to share a standard design & layout, with perhaps a unique color scheme for each suite.

For sure. I think all the template stuff should be shared. I’m not opposed to different color schemes, though most of the coloring in docs is content specific anyway and wouldn’t likely change much. (Red for warnings, blue for tips, etc.)

Not exactly related to this topic specifically, but how reasonable would it be to have “staging sites”

This is a question is ultimately for @yanokwa.

I think it would be cool to have PR-based staging deployments, so that one doesn’t need to DL and build locally to do a review. I don’t have a sense of the difficulty of this relative to the benefits.

Over time perhaps these could be harmonized with the OpenDataKit.org site as well

Just to be totally insane… there’s no strong reason we couldn’t include opendatakit.org in a docs monorepo… even while keeping it in Jekyll.

I’m not seriously suggesting this at the moment — but it’s worth a think.

Thanks for the detailed investigation and explanation @adammichaelwood!

I agree that the monorepo that builds out multiple sites seems to minimize the drawbacks and mostly let us have our cake and eat it too.

@adammichaelwood thanks for doing the research! I also agree that maintaining a shared core infrastructure is the preferred approach as people can have similar knowledge base and contribute. Also keeps the style guides the to decrease divergence.

I should also add I think we should reuse this solution to make separate URLs for developer facing documentation (if/when we go down that road) so again we can share all the templates, styles, contribution, etc

a month or so ago I collapsed the several pages of Docs Contributor pages into a single TOC-page in a “Contributors” section, with the idea that this section could eventually also include contributor guide(s) for developers as well.

A dedicated Contributor Docs site is a reasonable idea. I think Contributor sections in each of the ODK1 and ODK2 sites is probably a better idea, because it would be easier to cross-reference dev content with user content.
The decision to do this is currently above my paygrade — but I’m happy to support it if project maintainers want to do it.

Hi All,

@W_Brunette and I went ahead and executed @adammichaelwood’s plan and updated the pull request. Please check out my comment there for further details.