GenAI Interview Questions — Top 20 for AI Engineers (2026)

Prompting & Eval (5)

→Q: How do you improve prompt quality systematically? A: Build an eval set first; iterate prompts with measurable score; use few-shot + CoT before fine-tuning.
→Q: What is Chain-of-Thought? A: 'Think step by step' before answering; pairs with self-consistency for SOTA on reasoning benchmarks.
→Q: How do you get reliable JSON output? A: JSON mode / function calling, schema-constrained decoding (Outlines), retries with validation, never regex-parse.
→Q: How do you eval an LLM application? A: Mix of automated (BLEU/ROUGE/factuality/RAGAs), LLM-as-judge, golden sets, human eval on samples.
→Q: What's prompt injection and how do you defend? A: User input that overrides system instructions; defense: strict templating, output validation, principle of least privilege for tools.

→Q: Build for $X budget — how do you architect? A: Choose model by task complexity, route easy tasks to cheap models, cache aggressively, batch where possible.
→Q: When do you use streaming responses? A: When perceived latency matters; chat UIs, long generations. Don't stream for sub-second responses.
→Q: How do you cache LLM outputs safely? A: Semantic cache on query embedding, TTL, invalidate on data updates, don't cache personalized outputs.
→Q: Open-source vs hosted model — when each? A: OSS for privacy/control/cost-at-scale; hosted for speed-to-market and frontier quality.
→Q: How do you handle 30+ second response times? A: Streaming, progress indicators, background jobs with webhook callback, batch endpoints.

→Q: Build a doc QA system over 1M PDFs. A: Ingest pipeline (chunking + OCR for scanned), vector DB + hybrid search, reranker, citations, eval loop.
→Q: How do you keep RAG fresh? A: Document timestamps, periodic re-embedding, change-data-capture from source, version-aware retrieval.
→Q: Vector DB choice — what factors? A: Scale, latency, hybrid search, managed vs self-hosted, cost; for most teams pgvector or Pinecone is fine.
→Q: How do you handle multilingual RAG? A: Multilingual embeddings (E5, BGE), per-language indexes, cross-lingual reranker.
→Q: Multimodal RAG over PDFs with charts? A: Vision-language model to caption images, hybrid retrieval over text + captions, multi-modal final generation.

→Q: How do you red-team an LLM app? A: Curated prompt-injection corpus, automated fuzzing, role-play attacks (DAN), boundary cases for refusal/over-refusal.
→Q: Hallucinations — mitigation? A: RAG with citations, structured outputs, refusal training, eval-time grounding scoring, human-in-the-loop for high-stakes.
→Q: How do you handle PII in LLM apps? A: Pre-LLM redaction (Presidio, Macie), output PII scanning, never log raw user inputs, regional data residency.
→Q: Build a multimodal product (image + text). A: Vision-language model (Claude 3.5, GPT-4o), unified prompt template, eval per modality.
→Q: What's the most interesting LLM thing you shipped? A: (Personal — but interviewer wants: problem framing, eval methodology, what went wrong, what you'd do differently.)