AI Engineer Interview Questions 2026: 18 Real Questions With Model Answers

You built a RAG demo over your own PDFs, it answers questions correctly, and it is the top line on your CV. Then the interviewer asks what your P95 retrieval latency was and what 1,000 answers cost you, and the room goes quiet. That gap, between a working demo and a system you can account for, is what the 2026 hiring loop is built to find. Everything below is written to close it.

What AI Engineer Interview Questions Actually Test in 2026

Interview question banks published through 2026, including guides from DataCamp and several university career offices, converge on the same five buckets: LLM fundamentals, prompt and context design, RAG and vector search, multi-agent systems, and production operations. The shift away from convolutional networks and gradient descent trivia is real. Nobody building a support assistant cares whether you can derive backpropagation.

So here is the call. If you are choosing between another framework tutorial and learning to measure retrieval quality, measure. LangChain fluency is worth roughly one question. Retrieval evaluation is worth four, and almost nobody prepares it. The trade-off is that you will look slower on framework syntax in a live coding round. Cheap price.

Keep one candidate in mind throughout: a backend developer in Pune, four years of Java, six months of weekend work on a RAG side project, interviewing for a first AI engineer role. Every answer below is pitched at what that person can honestly say. Building exactly that project under supervision is the spine of 360DT's live AI Engineer course, where the RAG, agent and evaluation modules run as build sessions.

LLM Interview Questions: The 4 That Open Almost Every Round

These are warm-ups, and they are scored. The interviewer is checking whether your mental model of a model is mechanical or magical.

1. What is a token, and why should you care?

A token is a sub-word unit the model reads and bills by, roughly three to four characters of English. The answer they want is the second half: tokens are the unit of your invoice and the unit of your context limit, so every design decision about chunk size, conversation history and system prompt length is a token decision. If you want the mechanics under this, our explainer on how large language models work walks through tokens, embeddings and attention. Also read it before the round if the phrase "attention is quadratic" makes you nervous.

2. What happens when you paste 200 pages into the context window?

Name three things. You pay for the whole prompt on every turn, because it is re-read each time. Facts buried mid-context get weaker attention than facts at the edges. And you lose any way to explain an answer, because there is no retrieval step to inspect. Then land the follow-up: a big context window is not a replacement for retrieval, it is permission to be lazy about chunking.

3. When would you set temperature to 0?

Extraction, classification, routing, structured output, anything where you will compare today's output to yesterday's. Keep sampling for drafting and summarising, where variety is the point. The trap in this question is the word "deterministic". Temperature 0 makes sampling greedy, it does not make the system reproducible, because batching and floating point on the serving side still move results. Say that and the question ends.

4. Same prompt, two different answers. Explain.

Sampling first, then the boring causes that actually bite: a system prompt containing today's date, a retrieval step that returned different chunks because the index changed, or a provider routing you to a new model snapshot. Good candidates check their own pipeline before blaming the model. That reflex is the most useful one in the job.

RAG Interview Questions: 5 on Chunking, Hybrid Search and Reranking

RAG interview questions are the core of the loop and the place most candidates lose the offer. The trap is that everyone can describe RAG in the abstract. Very few can describe the version they actually ran.

5. Walk me through how RAG works.

Do not recite "embed, store, retrieve, generate". Draw the diagram above and narrate the two boxes that mark a shipped system: the reranker and the standing eval set. Then name your numbers, meaning how many chunks you retrieve, how many survive reranking, and what share of your eval questions land the right chunk in the top five. Also read our step-by-step walkthrough of how RAG actually works before you rehearse this.

6. Your right answer sits at rank 8. What do you change?

Rank 8 means retrieval found it, so this is a ranking problem and not a recall problem. Say that out loud first. Then fix in order of return: a cross-encoder reranker over the top 50 candidates, then fusing keyword and vector rankings, then chunk boundaries. Published 2026 practitioner guides put the top-3 precision lift from reranking in the low tens of points for roughly 50 to 100 milliseconds of latency. Take that trade.

7. How do you chunk?

The answer that lands is structure-aware, not size-aware. Split on headings, clauses or table boundaries, keep a parent-document pointer so a matched chunk can expand back into its section, and overlap only where sentences genuinely straddle a boundary. Fixed 512-token windows are the default because they are easy, and they slice tables in half. Describe your document structure before you quote a chunk size.

8. Which index would you use, and how would you tune it?

Under roughly a million vectors with Postgres already in the stack, use pgvector and stop shopping. The pgvector documentation sets the HNSW defaults at m = 16 and ef_construction = 64, with hnsw.ef_search defaulting to 40, and those three names are what the interviewer is listening for. Raise ef_search at query time when recall is short and you have latency headroom.

CREATE INDEX ON chunks
  USING hnsw (embedding vector_cosine_ops)
  WITH (m = 16, ef_construction = 64);

-- session level, trades latency for recall
SET hnsw.ef_search = 100;

SELECT id, content
FROM chunks
WHERE tenant_id = 42
ORDER BY embedding <=> :query_vec
LIMIT 50;

If they push you toward IVFFlat, the pgvector guidance is arithmetic you can quote: lists = rows / 1000 below a million rows, lists = sqrt(rows) above it, and probes = sqrt(lists). Knowing that you should filter by tenant_id inside the same query rather than after retrieval is worth more than either index choice. Vector store selection gets its own treatment in our pgvector and Pinecone cost comparison.

9. How do you know retrieval got better?

Freeze 100 real questions with the chunk IDs that should answer them, measure recall@5 and mean reciprocal rank, and run it in CI on every change to chunking, embedding model or index parameters. Labelling takes an afternoon, and it is the artefact that turns your answers from opinion into evidence. That evaluation layer maps directly to the AI-300 syllabus in 360DT's MLOps Engineer course.

Agent Interview Questions: 4 on Loops, Tools and Failure

Agent rounds are the hardest section of a 2026 loop, because the companies asking have already been burned. They are not testing whether you know what ReAct stands for.

10. What is the difference between an agent and a workflow?

A workflow has the control flow in your code. An agent has the control flow in the model, which decides which tool to call next and when to stop. Then commit to an opinion, because they want one: most production problems people call agentic are workflows with one model-driven branch, and shipping them as workflows makes them testable. Reach for a real agent when the number of steps genuinely cannot be known in advance.

11. How do you stop an agent looping forever?

Four controls, and you should be able to draw them as pseudocode inside a minute: a hard step budget, a token budget with a summarisation fallback, explicit terminal conditions, and idempotent tools so a retried call does not double-charge someone's card.

# pseudocode, not a specific SDK
MAX_STEPS  = 12
MAX_TOKENS = 60_000

for step in range(MAX_STEPS):
    reply = model.run(messages, tools=TOOLS)
    if not reply.wants_tool_call:
        break
    messages += run_tools(reply)
    if tokens_used(messages) > MAX_TOKENS:
        messages = summarise(messages)
else:
    raise StepBudgetExceeded(MAX_STEPS)

12. How do you design a tool a model can actually use?

Narrow parameters, one job per tool, a description written for a reader who cannot see your codebase, and error strings the model can act on. Returning {"error": "400"} guarantees a retry loop. Returning "invoice_id must be 8 digits, you sent 6" gets a corrected call. Tool and schema design is the largest single domain in the Claude developer track, which 360DT's CCDV-F prep course drills against the exam objectives.

13. An agent fails 5% of the time. How do you debug it?

Traces before theories. Log every step with inputs, outputs, tokens and latency, then replay the failing sessions rather than reasoning about them. Most 5% failures turn out to be two or three specific tool inputs, not a model problem. Say "I would look at the traces" and you have already outperformed half the candidates.

Evaluation and Cost: The 5 Questions That Separate Mid From Senior

This is the section that decides your band. Anyone can build. Fewer people can tell you what their build costs and whether it is getting better.

14. How do you evaluate a RAG system with no labelled data?

Split the problem. Retrieval you label yourself: 100 questions, the chunk that should answer each, recall@5. Generation you score on groundedness, meaning every claim traces to a retrieved chunk, which a second model can check at scale once calibrated against 50 human judgements. Also read our guide to LLM evaluation metrics and pipelines for the deeper version.

15. When does LLM-as-a-judge lie to you?

When it grades its own family's output, when a vague rubric lets it default to rewarding length and fluency, and when nobody checks it against humans. Give it named failure categories, force a one-line justification, and re-calibrate on a human-scored sample monthly. Judges drift when the model version changes underneath you.

16. What does 1,000 answers cost? Show your working.

This is the worked example to rehearse until it is boring. Take a support assistant handling 50,000 questions a month. Each answer carries five retrieved chunks of about 400 tokens, an 800-token system prompt and roughly 700 tokens of conversation, so call it 3,500 input tokens and 350 output tokens. At Claude Sonnet 5 list pricing of $2 and $10 per million tokens:

queries    = 50_000
in_tokens  = 3_500          # system prompt + 5 chunks + history
out_tokens = 350
in_price   = 2.00  / 1_000_000    # Sonnet 5 input, USD per token
out_price  = 10.00 / 1_000_000    # Sonnet 5 output

per_query = in_tokens * in_price + out_tokens * out_price
print(round(per_query, 5))             # 0.0105  -> about 1 cent per answer
print(round(per_query * queries, 2))   # 525.0   -> USD per month

# cache the static 800-token system prompt: reads bill at 10% of input
cached = 800
per_query_cached = per_query - (cached * in_price * 0.9)
print(round(per_query_cached * queries, 2))   # 453.0

Two things to say after the arithmetic. First, embedding the corpus is almost free: two million tokens of documents at $0.02 per million is four cents, one time, so the index is not your cost, the answers are. Second, caching the system prompt saved about 14% here, and the saving grows as the static part of your prompt grows, though the first write to the cache costs more than a normal input token. That is the level of specificity that ends a cost question.

17. What is your latency budget?

Propose one rather than waiting to be told. A budget is a design statement, and interviewers read it as seniority.

18. Where would you put a cache?

Three layers: prompt caching for the static system block, semantic caching for near-duplicate questions, and a plain answer cache keyed on the exact question plus corpus version. The last is unglamorous and usually the biggest win, because support traffic repeats far more than anyone expects. The real test is the invalidation follow-up. Key the cache on a corpus version and bump it on every ingest.

The Take-Home Round: What Good Looks Like

Most Indian AI engineer loops in 2026 include a take-home, usually "build a small RAG or agent over this dataset, four to six hours". Almost everyone submits a working notebook. Almost nobody submits evidence.

Here is the honest caveat. A question bank, this one included, is not preparation if you have never run a system for a month. Reading 18 model answers gets you through the screening call and then leaves you exposed at the fourth follow-up, where the interviewer just keeps asking "and then what happened". The people who clear these loops have a scar they can describe. Go get one small scar on a real project before you memorise anything.

Certifications work the same way. Credentials from the full certifications overview get you read at the resume screen and prove syllabus coverage. They do not answer "what did you change and what happened to the metric". Bring both.

How to Prepare for AI Engineer Interview Questions in 4 Weeks

Four weeks at roughly 8 to 10 hours a week is enough if you already write code daily. Our Pune backend developer did precisely this sequence on weekends around a full-time job, which is the realistic constraint for most people reading this.

Generative AI interview questions if you are coming from data science

Your gap is not modelling, it is serving. The generative AI interview questions aimed at your background skip the maths and go straight to "how would you deploy this and know it still works in March". Spend the four weeks on evaluation harnesses, latency and cost, not transformer papers. The Azure AI-103 track in the Generative AI Developer course targets that deployment gap, and it overlaps with the AWS Solutions Architect and DevOps course if your target roles are infrastructure-adjacent.

AI engineer interview preparation if you are coming from data engineering

You already own the half most candidates fumble: pipelines, idempotency, incremental loads, schema drift. Reframe them as ingestion and re-embedding problems, because the corpus refresh pipeline gets asked and almost never answered well. If Fabric is your stack, the DP-700 material in the Microsoft Fabric Data Engineer course maps onto that ingestion side, while the design-level thinking these rounds reward is what CCAR-F architect foundations prep drills.

Related guides

• What Is an AI Agent Harness in 2026? read this before the agent round, because orchestration vocabulary is half of what gets tested there.
• What Is Context Engineering in 2026? the techniques behind the token budget answers in question 11 and question 16.
• 8 Generative AI Project Ideas for 2026 if your answer to "tell me about a project" is still thin, build one of these first.
• AI Engineer vs Machine Learning Engineer in 2026 useful if you are not sure which of the two loops you should actually be sitting.
• AI Engineer Roadmap 2026 the longer path that leads up to the four-week sprint described above.

Frequently asked questions

About this guide. 360 Digital Transformation is an Authorized Training Partner of Anthropic and Microsoft. Other certification bodies, vendors and employers named here are not affiliated with us. Tools and versions change quickly; commands and figures cited were checked on 22 September 2026.

If I were the Pune backend developer with four spare weekends, I would not read another question bank, including this one, a second time. I would spend week one labelling 100 questions against my own project and week two making one retrieval change I could measure, because every answer above gets stronger the moment it has a number in it. Then I would sit the interviews. If you want that work supervised rather than solo, the AI Engineer course runs it live over 16 weekends, and the free webinars are a no-cost way to see how the RAG and agent sessions are taught before you commit to anything.