Individual paper-search project

Build a retrieval method over the fixed reading-list paper corpus. The course supplies extracted text, baseline methods, execution tools and the comparison interface under irai.stai-lab.org. Everyone receives the same corpus and shared resources. Only the final test queries are held out.

Investigate one question through a controlled comparison. You may change paper representations, chunking, query processing, encoders, candidate retrieval or ranking. The expected work is one justified experiment, not a large application.

Three project sessions

All times are KST. Each session runs 09:00-10:00. The course uses Zoom.

• 14 October: guided modification. Everyone reads and builds the supplied read-only exact FAISS code, tests it with queries, then edits and compares FAISS index setups after the course covers vector search. Keep passage embeddings fixed. Predict the expected search-result and efficiency changes along at least two FAISS factors. Vary those factors and explore multiple queries. Explain which variations and queries you tried, the top-10 quality and efficiency results, and which variations had which effect. Compare the results with your earlier predictions; explain which were supported or contradicted, and why. Cite supporting runs and paper/page evidence.
• 11 November: exploration. The lecturer demonstrates contrasting approaches. Choose and test your own change, then respond to a fresh classroom query after saving your candidate. Submit the experiment record and save your code and structured report draft during class.
• 16 December: project highlights. The lecturer presents selected submissions and held-out comparisons. Discussion is ungraded. There is no separate practical assessment or required individual presentation.

7 October is a lecture, not a workshop. No additional paper pre-reading is required for the project. Start directly in class on 14 October with the supplied starter. Open the workshop and follow the student guide with the lecturer.

Required project work fits within the two working sessions. Further work is optional. Your saved November code and report can be your final submission without further edits.

Submission

Submit a ZIP of Python source and configuration using the supplied starter:

def build(corpus, output_dir):
    """Build and save the index from the supplied papers."""

class Retriever:
    def __init__(self, index_dir):
        """Load the saved index and required models."""

    def search(self, query, k):
        """Return up to k unique paper IDs with scores, best first."""
        return [{"paper_id": "colbert", "score": 0.82}]

Helper modules are allowed. You may develop in a notebook, but submitted source must run through this interface. The course rebuilds the index and checks it in a fresh process. The same accepted version serves the website and final test. Scores need not be comparable across systems; the interface compares ranks.

Submit an explanation of at most 600 words, with run references and an AI and resource declaration. The limit is a ceiling, not a target. Explain:

1. Your question, prediction, change and why it might help.
2. A comparison with the supplied baseline under the same conditions.
3. Two query examples with supporting paper evidence, including a failure or trade-off where one exists.
4. What your evidence establishes, its limits and what you would test next.

Code and report freeze on 9 December at 23:59 KST. Revisions after 11 November are optional. If you make none, your accepted November code and saved report count automatically. The submission interface will show the selected version and receipt. Source received on time remains on time if the course build queue finishes later. A failed candidate must not silently replace an earlier accepted version. Retain your package and receipt if there is a technical problem.

Exploration boundaries

The collection and interface are fixed; the retrieval method is open. You may compare encoders or instructions, change chunks and paper representations, combine retrieval methods, rerank, traverse a graph of corpus papers or retrieve in several steps. Return ranked papers, not only a generated answer.

Use the shared Python environment, approved model files and common CPU resources. The starter guide publishes the measured build, training, load, search and storage limits and links the pinned common environment. No paid account is required. Builds and searches run without private API keys. IP networking is disabled; a supplied read-only connection reaches the frozen course Elasticsearch index. An expensive private preprocessing run is not an exception to the shared budget. A small training experiment is allowed if it is reproducible within that budget.

Corpus-derived summaries, questions and manual annotations are allowed. Submit and attribute them, and declare preparation costs and human effort. Shared precomputed assets are available to everyone. Additional external documents may not expand the searchable corpus. Approved pretrained models are permitted; this restriction is not a claim about their original training data.

Development queries may inform tuning or declared training. Repeating their answers is not evidence of generalisation. Each evaluation query starts from the same saved index without state from previous queries. The final private queries, labels and logs are unavailable until results are released. Final systems cannot be revised after that release.

If you change several components, use a useful ablation to test your explanation. No particular architecture, novelty claim or number of changed lines is required.

Marking: 25 marks, 25% of the course

The first five marks assess the activities completed during class. The final 20 assess the submitted code and report. The classroom responses remain as submitted even if the final report changes. Do not count the same evidence twice as two separate activities.

Leaderboard position, model size, code volume, novelty and selection for the December highlights carry no marks. A simple method can receive full marks. A change that performs worse than the baseline can receive full marks if its implementation, comparison and diagnosis meet the criteria. A failed build does not erase credit for other criteria with valid evidence. An unchanged starter may pass execution checks but does not demonstrate your own experiment.

Final queries provide additional evidence for the December discussion; their retrieval scores do not determine your marks. Staff review recorded execution checks and apply the common marking guide. Tutors calibrate together, divide the submissions and cross-check a sample. The lecturer resolves outstanding marking differences.

AI, attendance and technical problems

AI assistance is optional during project development outside the supervised activities and must be declared. Generative AI and help from another person are not permitted for the individual prediction and explanation submissions. The guided code exercise is completed without generative AI. Individual predictions and explanations must be your own. Tutors mark submitted work; the lecturer handles workshop guidance and technical problems.

October students explore their own queries; November uses a fresh classroom query. Submission windows close in class. Preparation beforehand is permitted; it does not replace the in-class activity. An authorised absence receives an equivalent opportunity, with agreed accommodations. A course-system failure receives a remedy without a mark penalty. Report the error and keep the available evidence.

Amendment

14 October 2026. Tutors only mark submitted work. This supersedes the earlier tutor code-guidance role. Workshop guidance and technical recovery belong to the lecturer.

Approved 27 September 2026. This replaces the growing class-note collection, group project, required relevance-judging quotas, peer multipliers, leaderboard marks and individual presentations. It also supersedes the proposed final practical and earlier 5/10/10 allocation. The first workshop moves from 7 to 14 October; all three project sessions last 60 minutes. The project remains 25% of the course.

11 October amendment: the guided exercise compares FAISS index setups and Elasticsearch keyword/vector/hybrid retrieval over fixed passage vectors. This replaces the BM25 chunk-size change. The corpus, session length, marks, individual responses and AI rules remain unchanged.

14 October amendment: the opening phase uses read-only exact FAISS code. Students build it and test queries; code editing opens during FAISS experiments. The Elasticsearch phase is removed. The hour allocates 10 minutes to query practice, 5 to prediction, 40 to FAISS experiments and explanation, and 5 to receipts and downloads. Marks and AI rules are unchanged.