Course Project

The course project is an open-ended, hands on opportunity to explore an aspect of statistical NLP.

Rubric

To receive a grade, you must create a pull request in the course blog repository with a final (updated) version of your class competition summary. Note that you should be updating the same file used for your draft. The final version of your class competition summary will be assessed across the categories listed below. Each category (described below) will be graded on either a Superior/Pass/Fail or Pass/Fail basis. Assignment letter grades are calculated in the following manner:

Project description

• Superior: Pass criteria + one or more of the following:
  • a)discussion of related work
  • b) challenges of task (with examples)
  • c) discussion of SotA approaches.
• Pass: All of the following:
  • a) clear overview of project
  • b) description of its novelty
  • c) motivation for project
• Fail: Description is inadequate or absent

Summary of individual contributions

• Pass: Clear description of your role in the project and those of other team members
• Fail: summary of contributions is inadequate or absent

Proposal for future improvements

• Pass: All of the following:
  • a) discussion of limitations
  • b) avenues for improvement
• Fail: Discussion of future work is inadequate or absent

Results

• Superior: Pass criteria + measure of robustness of approach (ex. stratified k-fold cross validation, etc.) and/or statistical analysis
• Pass: Quantitative results on task noting delta with some baseline
• Fail: Reporting of Results is inadequate or absent

Error analysis

• Superior: Detailed error analysis on held-out data (characterization of error) with examples
• Pass: Some error analysis on held-out data
• Fail: Error analysis is inadequate or absent

Reproducibility

• Superior: Containerized build with clear step-by-step instructions for reproducing results
• Pass: Clear step-by-step instructions for installing dependencies and reproducing leaderboard numbers
• Fail: Unable to reproduce results (ex. lack of instructions, missing components, etc.)

Link to code repo

• Pass: All of the following:
  • a) post includes a link to the code repository
  • b) the code repository is not empty
  • c) the repository uses the required assignment template (i.e., the repository is owned by the appropriate GitHub organization)
• Fail: Any of the following:
  • a) post does not include a link to the code repository
  • b) the code repository is empty
  • c) the repository does not use the required assignment template