SDAIA Academy · Applied NLP Engineering

Capstone Project Presentations

Day 4: Demonstrate, evaluate and defend an applied NLP solution

AI Engineering Track Day 4 of 4 Team demos · technical Q&A · peer review

Today

Day 4 agenda

Session 1 · Opening + presentationsEvaluation expectations (15'), then teams 1–250 min

Break10 min

Session 2 · PresentationsTeams 3–550 min

Long break20 min

Session 3 · PresentationsTeams 6–850 min

Break10 min

Session 4 · Final presentations + reviewTeams 9–10, peer feedback, instructor review, lessons learned, closing50 min

Session 1

Opening & evaluation expectations

How each presentation runs and how it is scored
What strong answers to technical questions look like
How you give useful peer feedback

01

Format

15 minutes per team

10'

Talk + live demo

Every member speaks. Demo on real inputs, including one Arabic example if it applies.

5'

Technical Q&A

Instructor first, then the audience. Answer with evidence from your results.

↻

Changeover

Next team connects during Q&A. Have your laptop ready and your demo already loaded.

Timekeeping: signal at 3 minutes left and 1 minute left. When the talk timer ends we move to Q&A. Demo not working? Switch to your recorded backup immediately.

Suggested structure

Eight slides that tell the whole story

Problem: who has it, why it matters, and your success metric
Data: source, size, labels, languages, splits, limitations
Baseline: the simplest thing that works, with its number
Approach: architecture diagram; why these models

Results: before vs. after table, by slice, with latency and cost
Error analysis: categorized failures, with real examples
Live demo
Lessons and next steps: what you'd do with 2 more weeks

Scoring guide

What each criterion looks like at its best

Criterion	Weight	Strong evidence
Functional end-to-end solution	30%	Runs live on unseen input; handles errors, abstains gracefully
Model selection & implementation	25%	Choice justified by quality and latency/cost; clean, modular code
Quantitative evaluation & error analysis	20%	Held-out test, baseline, slices (EN/AR), categorized failures
Fine-tuning / justified adaptation	15%	Before/after on the same test set, or a data-backed case for not tuning
Presentation & reproducibility	10%	Clear story, on time; README lets a stranger reproduce results

Technical Q&A

Questions you should be ready for

Why this model and not a smaller or bigger one? What did you compare?
How do you know the test set doesn't leak into training?
What's your worst slice, and why?
What happens when the model is unsure?

How much would this cost at 100k requests per day?
How would you detect that it is getting worse in production?
Did fine-tuning help? By how much, and is the gap significant?
What would break first with real customer data?

Great answers point to a number, a plot or an example from your repo. "We didn't test that, and here's how we would" is a fine answer too.

Peer feedback

Give feedback that helps: 1 · 1 · 1

One strength

Be specific: "The EN/AR slice table made the cross-lingual gap obvious."

One question

Something you genuinely want to know about their choices or results.

One suggestion

A concrete next step: "Try a reranker; your errors look like retrieval misses."

Write feedback for each team on the shared form while they present. It is shared with the team after the session.

Running order

Teams and time slots

#	Session	Team	Use case
1	Session 1
2	Session 1
3	Session 2
4	Session 2
5	Session 2

#	Session	Team	Use case
6	Session 3
7	Session 3
8	Session 3
9	Session 4
10	Session 4

Click a cell to type team names before the session. They stay until the page is reloaded.

Now presenting

Talk + demo

Q&A

Break

10-minute break

Next: Session 2 · Presentations, teams 3–5

Session 2

Presentations: teams 3–5

02

Now presenting

Talk + demo

Q&A

Long break

20-minute break

Next: Session 3 · Presentations, teams 6–8

Session 3

Presentations: teams 6–8

03

Now presenting

Talk + demo

Q&A

Break

10-minute break

Next: Session 4 · Final presentations & review

Session 4

Final presentations, review & closing

Teams 9–10
Peer feedback and instructor review
Lessons learned and where to go next

04

Now presenting

Talk + demo

Q&A

Instructor review

Patterns across projects

What worked

Starting from a simple baseline and keeping it in the results table
Evaluating Arabic and English separately
Abstaining instead of guessing
Cascades: small model first, LLM for the hard tail

What to watch

Test sets that overlap with training data
Single-run results presented as significant
LLM outputs trusted without validation
Demos tuned to a few hand-picked examples

The whole journey

From tokens to fine-tuned language models

Tokenscost, context, Arabic fertility

→

Embeddingssearch, clustering, features

→

Decision modelslabels, thresholds, F1

→

LLMs + RAGstructured, grounded generation

→

Fine-tuningencoders + LoRA

→

Evaluationbefore/after, by slice

If you need…	Start with…
A label at high volume	Fine-tuned encoder (or embeddings + linear head)
Similar items, deduplication, clustering	Sentence embeddings + vector index (+ BM25 for IDs)
Answers from your documents	RAG with reranking, citations and abstention
Structured fields from messy text	Regex where possible, LLM + schema validation for the rest
A consistent output format or style from a small model	LoRA fine-tuning, evaluated before and after

Keep going in the AI Engineering track

Where to go from here

Generative AI & LLM Engineering

Advanced RAG, tool use, serving and optimizing LLMs

Agentic AI Systems

Agents, tools, workflow automation, guardrails

MLOps & LLMOps

Pipelines, registries, monitoring, retraining

AI Evaluation & Observability

Eval suites, LLM-as-judge, red-teaming, tracing

This week: push your capstone repo public (or internal), add a model card, and write a one-paragraph post on what you learned.

Keep practising: Hugging Face LLM course, Sentence Transformers and PEFT docs, MTEB leaderboard, and the Arabic NLP community (e.g., ArabicNLP / WANLP workshops).

Applied NLP Engineering · SDAIA Academy

Thank you

Please complete the course feedback survey before you leave. It directly shapes the next cohort.

Survey

Link shared in the course channel

Certificates

Issued after capstone submission is confirmed

Stay in touch

Course channel stays open for capstone follow-ups