Why I tested it
Parsing syllabi into a calendar worked quite accurately in GRYPH OS, our hackathon planner, so I expected a cheaper way of doing it to hold up too. The results surprised me a bit.
Problem
Student tools use AI to turn syllabus PDFs into calendars. A common way to save money is to run cheap rules first and call the AI only when the rules seem to fail. I tested whether that shortcut actually works.
Approach
Five extraction strategies, from sending the PDF straight to Gemini to regular expressions and date parsing, each run three times on 16 public syllabi from 8 institutions. Every prediction was scored against 151 deadlines checked by hand.
The hard part
The hardest part was figuring out five ways of doing it that made sense logically, and then finding more ways to get accurate results.
Results
The rule-first approach used 57.8% fewer tokens, but its F1 score fell from 0.66 to 0.37. Saving tokens cost almost half the accuracy.
The code, the labels and every prediction are public, and the paper is on Preprints.org with DOI 10.20944/preprints202610.2704.v1.