Skip to content
Snippets Groups Projects
Commit eda9feab authored by Frank Duncan's avatar Frank Duncan
Browse files

Add OTS Correction to 100Change2017

The corrections here (located in the subverison directory) all have to
do with bad organization names that broke sorting.
parent d05689c4
No related branches found
No related tags found
No related merge requests found
......@@ -25,6 +25,7 @@ Usage:
--proposals-csv=PROPOSALS_CSV \\
--attachments-dir=ATTACHMENTS_DIR \\
--tdc-config-dir=TDC_CONFIG_DIR \\
--correction-file=CORRECTION_FILE \\
--pare=PARE \\
--csv-only
......@@ -47,6 +48,15 @@ Command-line options:
file with a list of keys to include. For both + and @,
the list of keys will be limited to only the ones provided.
--correction-file FILE FILE is a csv of corrections to the main data. The header
must match the header of the original proposals file, and any
one of the columns must contain the review number. Then
the data from the correction file will override the
source data for output. There can be multiple correction
files, and each one overwrites the previous.
If the data cells have the empty string, no correction is applied.
--csv-only Only upload the created CSV file. Don't upload attachments or
create wiki pages. For use to speed up process when wiki has been
created already.
......@@ -71,6 +81,7 @@ def main():
"attachments-dir=",
"pare=",
"csv-only",
"correction-file=",
],
)
except getopt.GetoptError as err:
......@@ -82,6 +93,7 @@ def main():
tdc_config_dir = None
pare = None
csv_only = False
correction_files = []
pare = None
csv_only = False
for o, a in opts:
......@@ -95,6 +107,8 @@ def main():
tdc_config_dir = a
elif o == "--attachments-dir":
attachments_dir = a
elif o == "--correction-file":
correction_files.append(a)
else:
sys.stderr.write("ERROR: unrecognized option '%s'\n" % o)
sys.exit(2)
......@@ -123,6 +137,13 @@ def main():
comp.process_all_cells_special(fix_cell_processor)
fix_cell_processor.report()
for correction_file in correction_files:
correction_processor = competition.CorrectionData(
"Review_Number", correction_file
)
for column in correction_processor.columns_affected():
comp.process_cells_special(column, correction_processor)
attachments = competition.RegexSpecifiedAttachments(
comp.sorted_proposal_keys, attachments_dir
)
......
......@@ -79,6 +79,7 @@ RECLASSIFICATION_XLSX="geo-and-topic-revisions-Reclassify-themes-ALL-complete-(m
RECLASSIFICATION_CSV="geo-and-topic-revisions-Reclassify-themes-ALL-complete-(merged-assignment-docs).csv"
EXCLUDED_REVIEW_NUMBERS_TXT="excluded-review-numbers.txt"
ATTACHMENT_ZIP="Generated-Attachments-From-Mediawiki.zip"
OTS_CORRECTION_FILE="OTS_Corrections.csv"
ATTACHMENTS_DIR="${DATA_DIR}/attachments/"
TDC_CONFIG_DIR="${DATA_DIR}/tdcconfig/"
......@@ -115,6 +116,7 @@ if [ ! -d "${DATA_DIR}" ] ; then
cp ${ENCRYPTED_DIR}/${EXCLUDED_REVIEW_NUMBERS_TXT} ${DATA_DIR}/${EXCLUDED_REVIEW_NUMBERS_TXT}
gpg -o ${DATA_DIR}/${RECLASSIFICATION_XLSX} --decrypt ${ENCRYPTED_DIR}/${RECLASSIFICATION_XLSX}.gpg || exit 1
gpg -o ${DATA_DIR}/${ATTACHMENT_ZIP} --decrypt ${ENCRYPTED_DIR}/${ATTACHMENT_ZIP}.gpg || exit 1
gpg -o ${DATA_DIR}/${OTS_CORRECTION_FILE} --decrypt ${ENCRYPTED_DIR}/${OTS_CORRECTION_FILE}.gpg || exit 1
mkdir -p ${ATTACHMENTS_DIR}
mkdir -p ${TDC_CONFIG_DIR}
......@@ -342,7 +344,8 @@ ${RUNNER} --proposals-csv="${DATA_DIR}/${STAGE_FINAL_CSV}" \
$PARE \
$CSV_ONLY \
--tdc-config-dir="${TDC_CONFIG_DIR}" \
--attachments-dir=${ATTACHMENTS_DIR}
--attachments-dir=${ATTACHMENTS_DIR} \
--correction-file="${DATA_DIR}/${OTS_CORRECTION_FILE}"
if [ $? -ne 0 ]; then
echo Upload failure!
exit 1
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment