This required, for some of the competitions, to run the data through
dos2unix, which changed the shas.  But also, at the etl library level,
we needed to remove the unicode <feff> character that sometimes gets
added to the start of the csvs.

This updates the modern competitions to use the common deploy tooling,
which includes a decrypt helper function and the help documentation that
explains how to call the script.

There are a few legacy competitions which have not been updated, but
potentially could be in future.

I hope I didn't just cause problems, but we were being
more-specific-than-necessary when copying pdfs over to the attachment
directory.  This now copies everything, which is handy because we
changed one of the appendixes to `Overview` and this allows us to copy
it without adding another line.

Accidentally removed the `:` from an existing deploy parameter, which
made everything very sad and upset.

The 100Change2020 deploy script supports the downloading and decrypting
of key competition data before running the ingestion. This could not be
scripted in non-interactive environments because there was no support to
pass in a GPG passphrase.

This adds a new `-g` flag to deploy, which accepts a passphrase.

We will likely want to support this flag in other competition deploy
scripts, and it may be valuable to consider creating a set of common
utility methods imported across the various deploy scripts to avoid DRY
violation.

Issue #93

THis file has appendicies and the prospectus are renamed.  The zip was
committed to the bigdata repository.

This required some underlying changes:
* Proposal tab definition specific to this competition
* Uploading new pdfs from macfound

These changes support the needed template changes to the Torque
Configuration on the production instance.

Nothing massive here, mainly just making the structure a bit deeper by
creating top level directories for the base install and the
competitions, then spliting up the docs a little.

Documentation about option included in the documentation in the diff..
There's also a small typo fix where -a was in the usage instead when
the option is -c

This passes through to the compose-and-upload files

As organizations evolve, so must the code.

Because the script wasn't unpacking the attachments when
SKIP_ATTACHMENTS was on, it wouldn't properly create the spreadsheet
that had the attachment information.  That created a faulty upload.

This couldn't really be a diff, especially on compose-csvs/upload, because
the change was too massive.  So those files were deleted and a new one was
put in their place, even though it has some of the options processing
those did.

Issue #53: Refactor ETL code to create central utilities

This follows up to r16468 in the corporate repository.

This is mainly to save hard disk space, and before it just was trying to
save on uploading, but it makes sense to just not unpack or analyze
anything if that's set.

This corresponds to r96 in the internal bigdata repository.

This also required a change in the Column permissions and the
FullTemplate on the wiki to pick up, but this is the work to get them
into torque.

We moved things around in our internal tree, so update installation
paths here accordingly.

Five more attachments were sent over email, which were added to the zip
file, with the date of the file being updated.

These attachments came after the COVID pandemic highlighting how
different proposals attack the problem.

Based on this review:
https://github.com/OpenTechStrategies/csv2wiki/pull/25

The name of the option was incorrect.

We add type information for the upload, and that was causing issues
where proposals were attempting (and failing) to get uploaded for the
second line that only had type information.

Before, this was handled manually, but in the new world, we script it
all from inside the ETL pipelines.  This adds that capability to the
2020 competition.

This required not only outputting the list, but creating methods that
don't do grouped TOCs, but list based ones.

There were also some small UI fixes regarding text changes needed with
the finalist candidate list.

Because more and more metadata is being added by OTS as the project
continues, a special spreadsheet needed to be created that catches
edge cases where proposals needed to be modified.

This change also includes looking for those values for wild cards
and tr disqualified (with changes in the tocs and mwiki template)

When moving to torque, the sdg numbers got out of order when using the
generic TOC generation.  This order needed to be preserved, and so the
dictionary approach was changed to an array based one.

This uploads attachments to torquedata rather than to mediawiki (well,
through mediawiki), which also requires an update to the mwiki template
and adding a column to get attachment display processing out of
torquedata.

Also includes a change to dump out more configuration files for
TorqueConfig.

With the torque-data config being moved to the wiki, and out of the
torque-data app, we needed a place to put the base information that was
being used/generated.  Even though these aren't strictly needed for
running wikis, this is a good place for them because people deploying
new datasets can use them as a reference.

Before, we were just generating the template/data files, and then having
to hand create the mediawiki pages that call into torquedata.  This
fixes that so they are generated by the run process.

This required using the regionconfig again, but with the increase in
number of proposals being analyzed, the country list had to be
increased.

This also moves the generic template into the filesystem and out of
generating in python via print statements because it's a little bit
cleaner.

Just the population toc for now, but calls into the torquedata server to
upload the json/j2 pairs.  Laying the groundwork for more tocs.

In a reduced form, the sdgs are now again getting split based on the
input file.  This depends on hard coding happening on torquedata side
which will be rectified later.