In the YAML block of the PDF output, include the source document's
revision identifier for all the version control systems we use: SVN,
git-svn, and stock Git.  If uncommitted local changes are present,
indicate that too.  Relatedly, rename the commit identifier field name
to "document_rev_id", since "ots_svn" was SVN-specific.

Separately from all the above, also fix an information leak.

Explanation by example:
=======================

Before this commit, our YAML block couldn't handle regular Git trees
(such as a clone of https://code.librehq.com/ots/dosp-research), as
opposed to git-svn checkouts, and the block didn't indicate the
presence of uncommitted local changes (if any) either.  Also, it would
includ the full path to the local input file, even though that path is
specific to the user's computer and by default should be kept private.

Here's an example showing the difference between the old output and
the new output.

In a stock Git working tree with uncommitted local changes, the old
output would look like this:

  %%% BEGIN OTS YAML BLOCK %%%
  % client: ''
  % date: 15 Dec 2023
  % doctools_git_branch: main
  % doctools_git_id: commit 11ffa0c9bd9e69a3735744aafb7d4cb73fbd66e9
  % draft: false
  % input_filename: /home/jrandom/super-secret-clients/moly-cartel/smuggling/molybdenum-report.ltx
  % ots_svn: r
  % title: 'Recent Developments In The Molybdenum Market'
  % todos: []
  %%% END OTS YAML BLOCK %%%

Note the 'ots_svn' field (i.e., the field name implies SVN, even
though this is actually a Git working tree), and see how the field's
value is an empty SVN-style revision number -- just the "r" prefix.
Note also the information-leaking value of the 'input_filename' field.

Now here's the new output for the same tree:

  %%% BEGIN OTS YAML BLOCK %%%
  % client: ''
  % date: 15 Dec 2023
  % doctools_git_branch: main
  % doctools_git_id: 11ffa0c9bd9e69a3735744aafb7d4cb73fbd66e9
  % document_rev_id: 557a94fca2342957793f19c4398c89356dea98fa + uncommitted local changes
  % draft: false
  % input_filename: molybdenum-report.ltx
  % title: 'Recent Developments In The Molybdenum Market'
  % todos: []
  %%% END OTS YAML BLOCK %%%

If the tree is pristine (has no local changes), then the new output
looks like this:

  %%% BEGIN OTS YAML BLOCK %%%
  % client: ''
  % date: 15 Dec 2023
  % doctools_git_branch: main
  % doctools_git_id: 11ffa0c9bd9e69a3735744aafb7d4cb73fbd66e9
  % document_rev_id: 557a49cf2a43927597ff914c93c898356dea98fa
  % draft: false
  % input_filename: molybdenum-report.ltx
  % title: 'Recent Developments In The Molybdenum Market'
  % todos: []
  %%% END OTS YAML BLOCK %%%

You're welcome.

Before this change, refs would be stripped from documents by default
(even documents, such as invoices, that don't normally have any
explicit YAML head).  After this change, refs are not stripped by
default, but if you put "remove-refs: TRUE" in the YAML head then they
will be, and you can optionally customize the regex with "ref-regex".

Note also that it's "ref-regex" now instead of "ref_regex".  This is
because some editors (cough Emacs cough) render unescaped underscores
in a jarring and distracting way in LaTeX documents, including even
underscores in YAML frontmatter.  A better fix would be to teach Emacs
how to recognize when it's in a non-LaTeX section of a mostly-LaTeX
document, but that's a bit more yak than I have time to shave today.

Since the OTS_DOCTOOLS_DIR environment variable is required now, use
that instead of Python's '__file__' variable to get values for the
YAML block comment that we put at the end of each PDF.

Note that the two other uses of '__file__' in this code base, in
'pipeline/pipeline.py' and 'pipeline/plugins/65_bugs.py', stay as they
are: for what they're doing, '__file__' is appropriate.

Explain what reference system we're using in case somebody else is
interested in the plugin.

  * Detect inkscape version of 1.0 rc 1 or later and add some tex to
    call inkscape using new-style command line parameters.

  * Add a preambles loop to the base jinja template so we have an easy
    way to add preamble sections that doesn't require clobbering the
    existing preamble items

  * Add a dump function to pipeline.py so we can dump our output and
    inspect it during debugging.

As promised in commit db47b65d.

Since commit 018499ec, when the bug-detection plugin is enabled
(e.g. "bugs: True" in the header) then on detecting a bug in the
document we stop the build, unless in draft mode ("draft: True").

There are thus three ways to fix such a build:

  1) Fix whatever the bug is in the document.
  2) Turn on draft mode.
  3) Turn off bug-detection mode.

Since our bug-detection module is not perfect, (1) is not always an
option because there might not actually be a bug in the document.
That's what happened in this case: the module's attempt to detect
proper formatting of footnotes by looking for the leading backslash
(e.g., "\footnote" instead of "footnote") cries a false positive if
the document uses "\begin{footnotesize} ... \end{footnotesize}" to set
the font size.

I'll improve that bug-detection case in a subsequent commit.  In the
meantime, this commit improves the error message to inform the user
about possibilities (2) and (3), which are not otherwise obvious.

This plugin provides general infrastructure for issuing warnings and
errors upon finding certain regexes. Right now, it just looks to see
that footnotes are formatted with the preceding slash.

The plugin has tests.

Karl called these build strings.  Much better name.

We had fields labelled svn and git, but no real explanation for them
in the string names, so I made them more explicitly refer to the
doctools and ots repos.

After we build a pdf, it is sometimes useful to attach some
information to the built file.  Here we do that as a YAML-formatted
block of lines appended to the PDF and marked in PDF syntax as
comments.  Our PDFs now carry some machine-readable metadata.

If a document comes in and there's no yaml frontmatter, we don't
jinjify.  You might not need any yaml frontmatter metadata but still
want jinja features (e.g. jinja macros).  In that case, add a dummy
var.

The reason for doing this is that legacy docs don't have any
frontmatter, and this is a convenient way to distinguish them.

 * Make -rXXXX versions for all builds, not just drafts
 * Add draft plugin
 * In template, set draft to trigger on both True and 'True'
 * Move path into PIPELINE definition
 * Silence some echo
 * Use .tex instead of intermediate_ltx
 * Simplify some bash
 * Let the plugin handle draft instead of passing it as an option
 * Cleanup .tex files

We can now do redacted and draft at the same time.

Refactor the jinja rendering and the redaction as a pipeline of
filters that run as needed.