Medium

Interview

Explain self-attention. Why is it better than RNNs for sequence modeling?

Foundational LLM knowledge. If you can't explain attention, you can't reason about modern AI architectures.

Interview

Tell me about an AI feature you shipped that failed to meet expectations. What happened?

Tests ownership, learning from failure, and whether you measure AI products with rigor — not just demo polish.

Interview

Tell me about a time you disagreed with your team about a technical decision.

Tests collaboration style — do you escalate well, or are you either a pushover or a blocker?

Interview

Tell me about the hardest technical problem you've solved.

Tests technical depth + structured thinking + ability to communicate complex work clearly.

Interview

Describe a time a stakeholder pushed back on your AI approach. How did you handle it?

Tests influence without authority — AI projects die in alignment meetings as often as in model training.

Interview

Explain the bias-variance tradeoff. How do you diagnose which is hurting your model?

Tests whether you understand fundamental model behavior beyond memorized definitions, and whether you can diagnose real-world problems.

Interview

Implement scaled dot-product attention in NumPy (or PyTorch).

Tests whether you understand the math behind transformers, not just the API.

Interview

Why do we use cross-entropy loss for classification instead of MSE?

Tests whether you understand the math behind the choice, not just which loss to import.

Interview

How would you build an evaluation suite for an LLM-powered feature?

Evals are the difference between shipping and praying. Standard interview topic in 2026.

Interview

When does classical ML with feature engineering beat deep learning?

Modern interviewers want signal that you don't reach for deep learning by default.

Interview

What's the difference between full fine-tuning, LoRA, and instruction tuning?

Modern AI work involves choosing the right fine-tuning approach. Tests current knowledge.

Interview

Walk me through how Adam works, and when you'd use it vs SGD with momentum.

Tests deeper optimizer knowledge — separates 'studied for the interview' from 'has shipped models'.

Interview

An LLM hallucinated a fake API in production code. How do you prevent this systematically?

Tests whether you treat LLM reliability as an engineering problem, not a model problem.

Interview

What caching strategies do you use to cut LLM latency and cost in production?

Separates candidates who've run LLM apps at scale from those who only call APIs in notebooks.

Interview

How do you handle PII in a RAG pipeline without breaking retrieval quality?

Tests privacy-aware AI design — a gap between demo RAG and enterprise deployment.

Interview

When would you use RAG vs fine-tuning to give an LLM access to new information?

Tests practical LLM judgment — this is the #1 question in applied AI interviews in 2026.

Interview

Your LLM keeps returning invalid JSON for structured outputs. How do you fix it systematically?

Tests production reliability — structured outputs power agents, evals, and integrations; failures can't be hand-waved.

Interview

Design a customer support LLM chatbot for a SaaS company. What's your architecture?

Now the most-common AI engineering interview question — RAG architecture under production constraints.

Interview

Design rate limiting for an LLM API gateway serving thousands of developers.

Tests platform engineering — LLM gateways need token-aware limits, fairness, and cost control, not just requests-per-second.

Interview

Why would a team build a feature store, and what does it need to do?

MLOps-flavored question — tests understanding of ML production infrastructure.