Open Tech Strategies's Document Infrastructure
This is Open Tech Strategies' infrastructure for building our published documents from their source files. Most of the documents are written in LaTeX, though we sometimes use other formats too.
The expected audience is people who want to work with the source text of our published reports. If you have the source repository for a given report, and you have this infrastructure, you should be able to re-create the published report (usually a PDF). Please let us know if you have difficulty doing so. We always welcome suggestions for improving this infrastructure.
Installation
-
Install the system software prerequisites.
Make sure
pdflatex
andlatexmk
are installed on your system already.You'll need Python3 and the items in
requirements.txt
. You can install the latter withpip3 install -r requirements.txt
or use your usual python dependency management system. -
Set up the
OTS_DOCTOOLS_DIR
environment variable.Set the environment variable OTS_DOCTOOLS_DIR to point to the directory where you have this source tree checked out. For example, put something like this in your
~/.bashrc
:OTS_DOCTOOLS_DIR=${HOME}/src/ots-doctools export OTS_DOCTOOLS_DIR
-
Set up the
TEXMFHOME
environment variable.Set the TEXMFHOME variable so that that kpathsea can find
latex/*
in this directory. For example, putTEXMFHOME=${HOME}/texmf export TEXMFHOME
in your
~/.bashrc
, and make sure that~/texmf/tex/latex/ots
is a symlink to${OTS_DOCTOOLS_DIR}/latex
. (Because kpathsea sometimes behaves oddly, we also recommendmkdir ~/texmf/tex/latex/dummy
so that kpathsea will follow the symlink properly.)
Building Documents
The top-level Makefile
in your document's source tree -- remember,
that's a different tree from this one -- should be a copy of the file
ext-Makefile
here. If you got the document tree from us, that will
already be the case. Just run
$ make foo.pdf
to build foo.pdf
from foo.ltx
. For draft PDFs, use make foo.draft.pdf
. (Note that you can also do this by setting the
draft
variable to true
at the top of the document, as discussed in
Latex + Jinja Usage below.)
Pipeline and Plugins
We have enabled the use of jinja templates. The Makefile runs pipeline.py, which applies a series of plugins to the document and its metadata. Each plugin can implement run(document, metadata) that returns doucment, metadata. Optionally, a plugin might implement run_p(document, metadata) that returns true iff the plugin should run.
Plugins might also implement after and after_p, which get run after the doc is built. These plugins might operate on the pdf or do post-build cleanup tasks.
A well-behaved plugin should have some way to disable itself in the
doc. One easy method is to honor flags: if the user sets
"PlUGIN_NAME: False" in the doc's YAML preamble, then run_p can return
meta.get('PLUGIN_NAME', True) != False
We load and run plugins in asciibetical order and skip ones whose names start with something other than a digit or a number. This allows users to disable a plugin by prefixing its name with an underscore.
See the plugins in ./pipeline/plugins for documentation on specific plugins.
LaTeX + Jinja Usage
Mostly you should just use standard LaTeX in OTS Doctools documents. However, there are some things you should know:
You can set jinja variables at the top of your document.
You can put jinja variables in a special block at the very top of your input file:
---
title: "My Life as an Eggplant Hunter:\\\\Nobody Told Me They Could Run So Fast"
date: 31 Nov 2023
draft: true
fruit: durian
---
Certain variables will be treated specially:
-
The values of
title
anddate
will be displayed on the title page. (Remember, backslashes are doubled here because they need to be quoted for jinja.) -
If the value of
draft
istrue
, then every page will get a large, light gray "DRAFT" diagonal watermark. (Note that you can also cause this to happen by buildingfoo.draft.pdf
insteadfoo.pdf
, as documented earlier in Building Documents.)
Other variables are just there to be referred to later. In the above
example, the value of fruit
is durian
, so if you put this in your
LaTeX:
I always ate eggplant, even when offered \VAR{fruit}.
It will render as this in the output:
I always ate eggplant, even when offered durian.
Avoid accidentally triggering jinja
If you start a LaTeX comment using two or more percent signs together ("%%") at the beginning of a line, then the jinja template parser -- which runs before LaTeX -- will try to interpret that line as a jinja command, usually resulting in an error:
Traceback (most recent call last):
[... long traceback stack here ...]
jinja2.exceptions.TemplateSyntaxError: tag name expected
The solution is to just always put a space after the first "%" when starting off a LaTeX comment at the beginning of a line:
%%% This will cause a problem, because the line starts with "%%".
% %% But this, on the other hand, will be fine.
longtable
for tables.
Use Use the longtable package for
tables, instead of the default LaTeX tabular environment. This is
really a generic recommendation, not specific to OTS Doctools, but
it's come up
before so we
thought we'd document it here. Among other things, if you use
longtable
, then in-table footnotes will be handled
correctly.
Windows
Linux and OS X should generally do the right thing with these materials. We have not documented how to set this up on Windows because we haven't tried that setup. If you're on Windows, please let us know how it goes and how we can improve these materials.