Add Central ETL Pipeline, version 1
This adds a centralized, refactored, etl pipeline that all competitions can depend on. This prevents the rampant copying and pasting of etl pipelines as new competitions are created, and prevents the coderot from modifying later etl pipelines that don't make it back into the original ones. Version 1 because this is based on the Climate2030 pipeline that doesn't have a number of features that older, more complete pipelines have. Those will get added incrementally. Issue #53: Refactor ETL code to create central utilities
Showing
- INSTALL.md 4 additions, 5 deletionsINSTALL.md
- README.md 12 additions, 2 deletionsREADME.md
- etl/.gitignore 1 addition, 0 deletionsetl/.gitignore
- etl/etl/competition.py 396 additions, 0 deletionsetl/etl/competition.py
- etl/etl/tdc.py 82 additions, 0 deletionsetl/etl/tdc.py
- etl/etl/toc.py 186 additions, 0 deletionsetl/etl/toc.py
- etl/etl/utils.py 114 additions, 0 deletionsetl/etl/utils.py
- etl/etl/wiki.py 99 additions, 0 deletionsetl/etl/wiki.py
- etl/setup.py 14 additions, 0 deletionsetl/setup.py
Loading
Please register or sign in to comment