Paper Summary
You will complete your paper summary with the assistance of an open source and open weight LLM. Part of your assignment will be to document your usage of this LLM and identify limitations you encountered.
Rubric
To receive a grade, you must create a pull request in the course blog repository with a final (updated) version of your paper summary. Note that you should be updating the same file used for your draft. The final version of your paper summary will be assessed based on how well your summary answers the questions listed below. Each question will be graded on either a Superior/Pass/Fail or Pass/Fail basis. Assignment letter grades are calculated in the following manner:
| Grade | Criteria |
A | No Fail and at least 2 Superior |
B | All Pass |
C | 1 Failure |
D | 2 Failure |
F | 3+ Failure |
Questions to be answered:
- What LLM did you select (model and version) and why?
- What are the authors proposing?
- What is the motivation for the work?
- What are the results and how do they compare with competing approaches?
- Are the comparisons fair?
- What are the takeaways according to the authors?
- What are the takeaways according to you?
- Would you use this? If so, how/where would you use this?
- What problems remain and what are the next steps?
- What issues and limitations did you encounter while using the LLM to assist with your paper summary?
Superior Response answers the question completely while remaining succinct; final LLM prompt documented for each question.
Pass: A response is provided, but may be unclear, incomplete, or overly detailed (as opposed to a summary).
Fail: No response is provided or the response fails to answer the question.
