Open Tech Strategies's Document Infrastructure

This is Open Tech Strategies' infrastructure for building our published documents from their source files. Most of the documents are written in LaTeX, though we sometimes use other formats too.

The expected audience is people who want to work with the source text of our published reports. If you have the source repository for a given report, and you have this infrastructure, you should be able to re-create the published report (usually a PDF). Please let us know if you have difficulty doing so. We always welcome suggestions for improving this infrastructure.

Installation

• Install the system software prerequisites.

  Make sure pdflatex and latexmk are installed on your system already.

  You'll need Python3 and the items in requirements.txt. You can install the latter with pip3 install -r requirements.txt or use your usual python dependency management system.

• Set up the OTS_DOCTOOLS_DIR environment variable.

  Set the environment variable OTS_DOCTOOLS_DIR to point to the directory where you have this source tree checked out. For example, put something like this in your ~/.bashrc:

    OTS_DOCTOOLS_DIR=${HOME}/src/ots-doctools
    export OTS_DOCTOOLS_DIR

• Set up the TEXMFHOME environment variable.

  Set the TEXMFHOME variable so that that kpathsea can find latex/* in this directory. For example, put

    TEXMFHOME=${HOME}/texmf
    export TEXMFHOME

  in your ~/.bashrc, and make sure that ~/texmf/tex/latex/ots is a symlink to ${OTS_DOCTOOLS_DIR}/latex. (Because kpathsea sometimes behaves oddly, we also recommend mkdir ~/texmf/tex/latex/dummy so that kpathsea will follow the symlink properly.)

Building Documents

The top-level Makefile in your document's source tree -- remember, that's a different tree from this one -- should be a copy of the file ext-Makefile here. If you got the document tree from us, that will already be the case. Just run

  $ make foo.pdf

to build foo.pdf from foo.ltx. For draft PDFs, use make foo.draft.pdf. (Note that you can also do this by setting the draft variable to true at the top of the document, as discussed in Latex + Jinja Usage below.)

Pipeline and Plugins

We have enabled the use of jinja templates. The Makefile runs pipeline.py, which applies a series of plugins to the document and its metadata. Each plugin can implement run(document, metadata) that returns doucment, metadata. Optionally, a plugin might implement run_p(document, metadata) that returns true iff the plugin should run.

Plugins might also implement after and after_p, which get run after the doc is built. These plugins might operate on the pdf or do post-build cleanup tasks.

A well-behaved plugin should have some way to disable itself in the doc. One easy method is to honor flags: if the user sets "PlUGIN_NAME: False" in the doc's YAML preamble, then run_p can return meta.get('PLUGIN_NAME', True) != False

We load and run plugins in asciibetical order and skip ones whose names start with something other than a digit or a number. This allows users to disable a plugin by prefixing its name with an underscore.

See the plugins in ./pipeline/plugins for documentation on specific plugins.

LaTeX + Jinja Usage

Mostly you should just use standard LaTeX in OTS Doctools documents. However, there are some things you should know:

You can set jinja variables at the top of your document.

You can put jinja variables in a special block at the very top of your input file:

  ---
  title: "My Life as an Eggplant Hunter:\\\\Nobody Told Me They Could Run So Fast"
  date: 31 Nov 2023
  draft: true
  fruit: durian
  ---

Certain variables will be treated specially:

• The values of title and date will be displayed on the title page. (Remember, backslashes are doubled here because they need to be quoted for jinja.)

• If the value of draft is true, then every page will get a large, light gray "DRAFT" diagonal watermark. (Note that you can also cause this to happen by building foo.draft.pdf instead foo.pdf, as documented earlier in Building Documents.)

Other variables are just there to be referred to later. In the above example, the value of fruit is durian, so if you put this in your LaTeX:

  I always ate eggplant, even when offered \VAR{fruit}.

It will render as this in the output:

  I always ate eggplant, even when offered durian.

Avoid accidentally triggering jinja

If you start a LaTeX comment using two or more percent signs together ("%%") at the beginning of a line, then the jinja template parser -- which runs before LaTeX -- will try to interpret that line as a jinja command, usually resulting in an error:

  Traceback (most recent call last):
    [... long traceback stack here ...]
  jinja2.exceptions.TemplateSyntaxError: tag name expected

The solution is to just always put a space after the first "%" when starting off a LaTeX comment at the beginning of a line:

  %%% This will cause a problem, because the line starts with "%%".
  
  % %% But this, on the other hand, will be fine.

Use longtable for tables.

Use the longtable package for tables, instead of the default LaTeX tabular environment. This is really a generic recommendation, not specific to OTS Doctools, but it's come up before so we thought we'd document it here. Among other things, if you use longtable, then in-table footnotes will be handled correctly.

Windows

Linux and OS X should generally do the right thing with these materials. We have not documented how to set this up on Windows because we haven't tried that setup. If you're on Windows, please let us know how it goes and how we can improve these materials.