• LLMs generate token by token. Use greedy decoding for decisions and sampling for drafting.
• Schemas make LLM output usable; validation + retry or constrained decoding make it reliable.
• RAG grounds replies in your KB, with citations and abstention.
• Model selection needs numbers: macro-F1, p95 latency, cost per 1k.

• Measure with macro-F1 and per-class recall, not accuracy.
• Class-weighted loss: errors on rare classes cost more.
• Oversample rare classes in training (never in test).
• Collect more of the rare class: active learning from low-confidence predictions.

from datasets import Dataset, DatasetDict
from transformers import AutoTokenizer, DataCollatorWithPadding

labels = sorted(train_df.intent.unique())
label2id = {l: i for i, l in enumerate(labels)}
id2label = {i: l for l, i in label2id.items()}

def to_ds(df):
    return Dataset.from_pandas(
        df.assign(label=df.intent.map(label2id))[["text", "label"]],
        preserve_index=False)

ds = DatasetDict(train=to_ds(train_df), validation=to_ds(val_df),
                 test=to_ds(test_df))

tok = AutoTokenizer.from_pretrained("distilbert/distilbert-base-multilingual-cased")
ds = ds.map(lambda b: tok(b["text"], truncation=True, max_length=32),
            batched=True)
collator = DataCollatorWithPadding(tok)   # pads per batch, not globally

from peft import LoraConfig, get_peft_model

lora = LoraConfig(
    r=16,                    # rank: capacity of the update
    lora_alpha=32,           # scaling; often 2 × r
    lora_dropout=0.05,
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj",
                    "gate_proj", "up_proj", "down_proj"],
    task_type="CAUSAL_LM",
)
model = get_peft_model(base_model, lora)
model.print_trainable_parameters()
# trainable params: 8.8M || all params: 503M || trainable%: 1.75

{"messages": [
  {"role": "system",
   "content": "Classify the support ticket. Reply with JSON."},
  {"role": "user",
   "content": "i was charged 2 times for order A-55821!!"},
  {"role": "assistant",
   "content": "{\"intent\": \"payment_issue\", \"category\": \"PAYMENT\", \"priority\": \"urgent\"}"}
]}

1. Confusion matrix → top 3 confused pairs
2. Read 20 errors (yes, actually read them)
3. Tag each one with a cause:

• Batch requests; sort by length to minimize padding.
• Truncate to the p99 length (~24 tokens for our tickets).
• fp16 on GPU; dynamic int8 quantization or ONNX Runtime on CPU.
• Distill: a 6-layer model is often within a point of a 12-layer one.
• Load the model once per process, not per request.

import torch
from transformers import pipeline

clf = pipeline("text-classification",
               model="artifacts/intent-distilmbert",
               device=0 if torch.cuda.is_available() else -1,
               truncation=True, max_length=32)

clf(["where is my refund", "وين طلبي"], batch_size=64)
# [{'label': 'track_refund', 'score': 0.97}, ...]

• Pinned package versions (uv.lock / requirements)
• Pinned base model revision
• Data snapshot + split seed + preprocessing code
• All hyperparameters in a config file, not in your head
• Seeds set; results over ≥ 3 seeds for small data
• Saved: model + tokenizer + label maps + metrics

import gradio as gr
from pipeline import assist          # your Day 2 + 3 code

def run(ticket):
    out = assist(ticket)
    return (out["intent"], out["confidence"],
            out["priority"], out["reply"], out)

gr.Interface(
    fn=run,
    inputs=gr.Textbox(lines=4, label="Customer message"),
    outputs=[gr.Textbox(label="Intent"), gr.Number(label="Confidence"),
             gr.Textbox(label="Priority"),
             gr.Textbox(label="Draft reply", lines=6),
             gr.JSON(label="Full output")],
    examples=[["Where is my refund?"], ["لم يصلني الطلب"]],
).launch()

1. A working end-to-end pipeline with a demo (Gradio or notebook UI).
2. At least one representation (embeddings) and one decision or generation component.
3. A baseline and at least one improvement, compared on the same test set.
4. Fine-tuning (encoder or LoRA), or a written, evidence-based justification for not fine-tuning.
5. Quantitative evaluation + error analysis with ≥ 10 categorized failures.