Updates based on building a new competition

Fleshed out some of the todo items, and improved some of the scripts.

competitions/template/README.md

@@ -165,24 +165,45 @@ more secret information
     - This opass file is the placeholder for any accounts that get added
 15. Create an opass file for the ansible production secrets
     - Create the secret file `mv ../$SHORTNAME/ansible/inv/prod/group_vars/all/secret{.tmpl,}`
+    - `$EDITOR ../$SHORTNAME/ansible/inv/prod/group_vars/all/secret`
     - `opass edit clients/lever-for-change/torque-sites/$SHORTNAME/ansible/prod`
     - One good source is to use xkcd's password generator
       - https://preshing.com/20110811/xkcd-password-generator/ at time of writing
-    - Pay special attention to the password requiring to have no spaces, as this
-      is required by the current Book Creator (though this note should be
-      changed as soon as we are done with the OTS book creator!)
-16. Update all the files in the ansible and etl directories, replacing basically everything
-    in angle brackets (`<>`) with the appropriate value.
     - For the `SIMPLESAMLURL_FROMMACFOUND`, you'll need the metadata url from macfound devops
-17. Update `<COMPETITION_NAME>.yaml` to remove any roles not to be included in this competition
+16. Get Logo
+    - Logo should be gotten from somewhere, and then turned into a correct width png
+    - This command requires imagemagick
+      - `convert -geometry 150 <LOGO_LOCATION> ../$SHORTNAME/ansible/roles/$SHORTNAME/files/${SHORTNAME}_Logo.png`
+17. Update all the files in the ansible and etl directories, replacing basically everything
+    in angle brackets (`<>`) with the appropriate value.
+    - in ansible directory, this is mainly replacing `<COMPETITION_NAME>` with `$SHORTNAME`
+    - in etl directory, there is more, as there's the filenames of the csv (from earlier step), and long competition name
+      - depending on the competition, some variables may not be needed (for instance, if we didn't get a zip file of attachments)
+        - to replace `<SHA>`, just do `sha1sum <PROPOSALS_CSV_LOCATION>`
+    - One nice way to do this is `vim `ack -l "<" ../$SHORTNAME/` `
+18. Update `<COMPETITION_NAME>.yaml` to remove any roles not to be included in this competition
     - Please update the `COMPETITION_NAME.yaml` file in the template/ansible directory to
       include any new ones that have been added!
-18. TODO: update compose-and-upload, using it as a shell
-18. TODO: git commit the new competition!
-18. TODO: Logo STUFF
-19. TODO: MediaWiki TorqueConfig stuff!
-18. Install locally, and test!
-
+19. Deploy initial version of wiki to production, check simplesaml configuration works
+    - `(cd ../$SHORTNAME/ansible ; ansible-playbook -i inv/prod $SHORTNAME.yml)`
+20. Update etl/{compose-and-upload,deploy}, using it as a shell
+    - some things to consider
+      - Attachments may or may not be a part
+      - Validitity of proposals may be a part
+21. Set up etl configuration for production
+    - `cp ../$SHORTNAME/etl/config.py{.tmpl,}`
+    - `$EDITOR ../$SHORTNAME/etl/config.py`
+22. Run initial ETL pipeline
+    - `(cd ../$SHORTNAME/etl ; ./deploy -g '<GPGPASS>' <DATADIR>)`
+    - This may not work if the data isn't fully done yet, so you can just move on to the next steps and work out later
+23. Git commit the new competition!
+    - (cd ..; git add $SHORTNAME; git commit -m "Initial commit of $SHORTNAME competition")
+    - Don't forget to push!
+24. Update TorqueConfig:MainConfig on the new wiki
+    - Will usually involve copying from another competition
+    - Add the columns from `<DATADIR>/$SHORTNAME/tdcconfig/AllColumns`
+    - Add the proposals from `<DATADIR>/$SHORTNAME/tdcconfig/AllProposals`
+25. Complete the checklist below
 
 ## Post-installation checklist for new competition instances
 

competitions/template/ansible/inv/prod/group_vars/all/secret.tmpl

@@ -4,7 +4,7 @@ mysql_root_password:
 
 db_password: <PASSWORD>
 mediawiki_admin_password: <PASSWORD>
-mediawiki_mwlib_password: <PASSWORD_WITH_NO_SPACES>
+mediawiki_mwlib_password: <PASSWORD>
 mediawiki_csv2wiki_password: <PASSWORD>
 
 simplesaml_okta_metadata_name: <COMPETITION_NAME>

competitions/template/ansible/roles/COMPETITION_NAME/tasks/main.yml

@@ -2,11 +2,11 @@
 
 - name: Transfer <COMPETITION_NAME> Logo
   copy:
-    src: <PATH_TO_LOGO_FILE>
-    dest: "{{ mediawiki_install_directory }}/mediawiki-{{ mediawiki_version }}/resources/assets/<LOGO_FILENAME>"
+    src: <COMPETITION_NAME>_Logo.png
+    dest: "{{ mediawiki_install_directory }}/mediawiki-{{ mediawiki_version }}/resources/assets/<COMPETITION_NAME>_Logo.png"
 
 - name: Install <COMPETITION_NAME> Logo
   lineinfile:
     path: "{{ mediawiki_install_directory }}/mediawiki-{{ mediawiki_version }}/LocalSettings.php"
     regexp: ^\$wgLogos = .*
-    line: $wgLogos = [ '1x' => "$wgResourceBasePath/resources/assets/<LOGO_FILENAME>" ];
+    line: $wgLogos = [ '1x' => "$wgResourceBasePath/resources/assets/<COMPETITION_NAME>_Logo.png" ];

competitions/template/etl/deploy

@@ -15,39 +15,13 @@
 # You should have received a copy of the GNU Affero General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
-CSV_ONLY=""
-PARE=""
-while getopts "cp:" opt; do
-  case $opt in
-    c) CSV_ONLY="--csv-only" ;;
-    p) PARE="--pare=$OPTARG" ;;
-  esac
-done
-shift $((OPTIND -1))
+SCRIPT_DIR=`dirname "$BASH_SOURCE"`
+COMMON_SCRIPT_DIR="${SCRIPT_DIR}/../../../etl/common/deploy"
+source "${COMMON_SCRIPT_DIR}/utils"
+source "${COMMON_SCRIPT_DIR}/init"
 
 COMPETITION="<COMPETITION_NAME>"
 LFC_DIR=`dirname "${0}"`
-BASE_DATA_DIR="${1}"
-
-if [ "${BASE_DATA_DIR}" = "" ]; then
-  echo "ERROR: BASE_DATA_DIRECTORY argument required."
-  echo ""
-  echo "Usage: '${0} [-c] [-p arg] BASE_DATA_DIRECTORY'"
-  echo ""
-  echo "Options:"
-  echo ""
-  echo "  -c                  Uploads CSV only.  Skips creating pages and"
-  echo "                      uploading attachments."
-  echo "  -p <arg>            If ARG is a number, pares by factor of ARG. If"
-  echo "                      ARG begins with +, then ARG is a comma separated"
-  echo "                      list of keys to include.  If ARG begins with @,"
-  echo "                      then ARG is a file with a newline separated list"
-  echo "                      of keys to include."
-  echo ""
-  echo "BASE_DATA_DIRECTORY/$COMPETITION is where all the work is done."
-  echo ""
-  exit 1
-fi
 
 DATA_DIR="${BASE_DATA_DIR}/${COMPETITION}"