Tiered platform — fixed order: exact Q&A → governed Text-to-SQL → cited RAG (fallback) → "no basis". Text-to-SQL eval 20/20, refusal gates 7/7.Governed by design — model fills human-reviewed templates, knowledge live only after human review.Retrieval numbers — hybrid + RRF + cross-encoder rerank, top-5: +3 questions / +14 pp on a 21-question RAG set; recall was already 100%, ranking was the bottleneck.LangGraph in production — supervisor, parallel agents, AWS + CI/CD.Business outcome — design workflow 5 days → 1 day (80%); logistics task 20 min → 1 min.PhD — experimental design, noisy data, healthy scepticism.| Min | What usually happens | Your job |
|---|---|---|
| 0–3 | Hello, small talk, he introduces the team and the role. | Smile, listen, note any words he uses (e.g. "agentic", "trust", "platform") and reuse them later. |
| 3–9 | "Tell me about yourself" and your background / projects (Q1–Q4). | Your 90-second story, then one project told at architecture level. Numbers in. |
| 9–15 | Two or three "how do you think" questions (Q7–Q12). | Principle first, then one short example. 45–60 seconds each. |
| 15–18 | Fit questions (Q13–Q16), maybe "any gaps?" | Honest, short, forward-looking. |
| 18–20 | "Do you have any questions for me?" | Ask two good questions (see the list at the end). Thank him. |
"I'm a scientist who became an AI engineer. I build governed AI systems that go to production — and I measure them."
Why he asks — To see if you can summarise yourself clearly and whether your path makes sense for this role.
Sure. I'm Delai, an AI Engineer based in Melbourne. My background is a little unusual. I have a PhD in chemical engineering and I worked as a research scientist for several years, so I'm used to designing experiments and looking at data carefully.
A couple of years ago I moved fully into software and AI. Today I work at Clenergy, where I co-build an enterprise AI platform. It answers from the most trusted layer first — approved exact Q&A, then governed Text-to-SQL, then cited document RAG. The idea is simple: the business approves the knowledge first, so the AI never has to guess.
In my spare time, I've built a financial chatbot, covering 5 ASX mining companies over three years of annual reports to assist my investment in mining stocks. It's built on LangGraph with three agents running in parallel — one retrieves from the annual reports, one searches the news online, and one does the exact math on financial ratios.
What I enjoy most is taking an AI idea from prototype to something people use every day — and doing it safely. That's why this role at NAB caught my eye.
Why he asks — Motivation check. He wants to hear you understand that a bank is different: regulated, high trust, large scale.
The AI Scientist role at NAB stands out to me because it closely matches the type of work I want to do — building agentic AI systems and taking them into production.
At Clenergy, I’ve learned a lot and co-built an enterprise-level AI platform. However, there are only around five people using it at the moment, mainly because our business team is relatively small and the company is still at an early stage of AI adoption.
So now, I’m looking for a larger organisation where AI solutions can play a more central role, reach more end users, and create greater and more measurable impact. That’s what makes this opportunity at NAB particularly attractive to me.
Why he asks — To test whether you can explain a system top-down, and whether your design choices show judgement.
It’s an internal Q&A platform — staff ask questions in natural language and get answers from the company’s own policies, documents, and business data. The key challenge is that different questions need different levels of accuracy, so we built three different approaches and the system decides which one to use.
Two approaches only handle pre-validated questions. First, Exact Q&A — it returns an approved verbatim answer word for word. Second, governed Text-to-SQL for numerical questions. The SQL template is written, validated and approved in advance, and the model are only allowed to fill in parameters like dates or regions — it never generates free-form SQL.
If the first two fail to match, it falls back to document RAG, which retrieves relevant documents and generates an answer with citations. If there isn’t enough evidence, it says something like "I don’t have the information" instead of guessing.
I built it with our senior AI engineer. We co-designed the architecture; he handled QA and governed Text-to-SQL, while I focused on RAG.
On results, Text-to-SQL scored 20 out of 20 and correctly refused all 7 unsupported questions. Retrieval scored 20 out of 21 on questions with a known correct answer.
Why he asks — Directly maps to the JD: "agent reasoning loops, tool use, orchestration logic." He wants to hear why, not just what.
Sure. I invest in the Australian share market, mostly mining companies, together with a few friends. So I built this to get financial information quickly and accurately.
It's built on LangGraph — a supervisor graph with three agents running in parallel. The main one is the RAG agent with hybrid semantic and keyword search over annual reports, a fan-out per company design, and a grading step that retries if the results look weak. The second is a news agent using web search. And the third is a ReAct calculator for financial ratios.
I use three agents instead of one because they do different jobs, so they need different tools and different ways to fail safely. Retrieval needs retry and grading. The calculator needs exact maths, not the LLM guessing. And keeping each agent small makes it much easier to test and evaluate on its own.
It's deployed on AWS through GitHub Actions, with streaming to a Next.js front end, and a two-layer semantic cache to save time and cost on repeated questions. I'll stop there, happy to go deeper on any part.
Why he asks — The JD says "what works, what fails, and why." He wants a scientist's honesty and a measured fix, not a hero story.
Retrieval in the mining chatbot. It looked fine in a demo, but two things were quietly broken.
The first one was ranking. The right piece of chunk was usually found, but it was sometimes ranked tenth or even twentieth on the list, while only the top eight were sent to the model — so the right chunk was lost.
The fix is to add a reranker, which is a small model that reads the question together with each candidate and scores how well it really answers the question. The new top eight then go through to the model. On a golden set of 12 cases where I knew which chunk holds the answer, retrieval found 7 without the reranker and 11 with it.
The other one was tables. An annual report has a lot of numbers in tables. In the beginning, the tables were saved row by row, just like the plain text, as a result, a number inside a table lost its column information easily. The fix is to save each number inside a table with its row and column names as a line of plain text, so each line makes sense on its own. On a test of twenty questions on tables, the accuracy jumped from 40% to nearly 90%.
What I learnt is that the quality of the answer is largely decided before the model ever runs. The weak point was not the model, it was how to get the right piece of text in front of the model.
Why he asks — JD: "partner with Product, Engineering and Platform teams… measurable business outcomes." He wants to see you start from the business problem.
At Clenergy, the design team had a fully manual workflow using AutoCAD and SolidWorks, and each job took about five days.
I sat down with the team and mapped out every step of the process before writing any code. Then I built an AI-led pipeline on top of Claude Code, where the AI operates both design tools through an MCP server to carry out the design work, while the engineers stay in the loop as reviewers.
After 3 weeks, more than half of the workflows went from five days to one — about an 80% reduction in turnaround time.
I think two things made it work. First, I started with their workflow, not the technology. Second, I kept the engineers involved as reviewers, so they trust the outputs.
I also did something similar with the logistics team. Their task was a fixed Excel-processing workflow, so instead of using an LLM, I customized a small app for them. It reduced a daily task from around twenty minutes to just one.
Why he asks — Judgement question. Many candidates over-use agents. A senior leader wants to hear restraint.
I always start with the simplest thing that works. If the steps are known in advance, use a fixed pipeline — it's cheaper, faster and much easier to test.
I use an agent only when the path really depends on the input. In this scenario, the model needs to pick tools or loop until a task is done. Even then, I keep its freedom as small as possible: a limited set of tools, clear stop conditions, structured outputs, and controlled context window.
In a bank I would lean even more towards constrained designs, because every step has to be explainable afterwards.
Why he asks — This is the heart of the JD ("rigorous evaluation strategies for probabilistic systems"). Most likely technical question. Answer in layers.
I think evaluating an LLM or agent system is really a measurement problem, because the system is probabilistic and usually has multiple components.
I would start by defining what success looks like with a fixed evaluation set and clear ground truth. Then I evaluate both the individual components and the end-to-end experience — for example, retrieval, routing, grounding, refusal, and the final answer.
For measurement, I use deterministic checks wherever possible, and only use an LLM judge when the quality is genuinely subjective.
Then I run those evals whenever the system changes, and in production I also monitor real behaviour and turn failures into new regression cases.
Finally, I look at variance and sample size, because one good run doesn’t prove the system is good. The goal is not just to get a score, but to know whether a change is genuinely better and where the remaining risk is.
Why he asks — JD: "retrieval pipelines and context assembly that determine the quality and reliability of agent behaviour."
In my experience, most RAG problems are actually retrieval problems rather than LLM problems, so that’s where I focus first.
The first step is to parse the documents properly — including layout, tables, charts and figures. For figures and charts, I use a VLM to generate a text description, because if the information inside a chart isn’t captured in text, it’s really hard to retrieve effectively.
Then I chunk documents based on their structure rather than using a fixed chunk size. For retrieval, I usually combine semantic search with keyword search, fuse the results, and then rerank them.
After that, I only give the model the top few relevant chunks, ideally with citations, so the context stays small, focused, and easy to verify.
And if the retrieval confidence is low, the system should say, “I don’t know,” rather than guess. Especially in a bank, a clear “I don’t know” is much better than a confident but incorrect answer.
Why he asks — Responsible AI is a must-have at NAB. He wants a mental model, plus honesty that you haven't worked under bank regulation yet.
I use three words: constrain, trace, and cite.
First, constrain what the model is allowed to do. For example, in my platform, the model can only use approved templates rather than generate free-form SQL, and new knowledge only goes live after human review. I also apply guardrails at the input, tool-use, and output stages.
Second, trace every important step — the user input, retrieved documents, model decisions, and tool calls. That gives us a complete audit trail, so we can investigate issues and reproduce how an answer was produced.
Third, cite the evidence behind the answer. Users should be able to see where the information came from and verify it themselves, rather than simply trusting the model.
For a bank like NAB, I think the key is to build these controls into the architecture from day one.
Why he asks — JD: "observability, behavioural drift detection, anomaly monitoring, incident classification." Be honest about what you've done and show you know the shape of the answer.
In my projects, I use two layers of monitoring. First, every production run is traced end to end with LangSmith — inputs, outputs, tool calls, latency, errors, and cost. So if an answer looks wrong, I can open that run and see where it failed. Second, I keep a gold set of test cases and run evals on it before every release to measure quality and catch regressions.
At NAB scale, I’d automate the same approach: put evals into the deployment pipeline, and feed tracing data into dashboards for signals like retrieval quality, tool errors, refusal rate, latency, and cost. I’d also run the gold set regularly to detect drift and trigger alerts when metrics move outside their normal range.
Then I’d classify incidents by failure type, so we can fix recurring root causes rather than individual symptoms.
The goal is not just to know that the system failed, but when, where, why, and whether it is becoming a pattern.
Why he asks — JD: "strong foundations in statistics, probability, experimental design." Your PhD is the proof — connect it explicitly.
Two things transfer directly. First, experimental design: change one variable, control the rest, repeat, and look at the spread, not just the average. LLM systems are noisy, so this matters more, not less.
Second, healthy scepticism. One good demo is not evidence. I want a test set and a number before I believe a change helped — I've been wrong about "obvious" improvements before.
And years of writing papers taught me to explain technical work clearly to people outside my field, which helps a lot with product and risk teams.
Why he asks — JD: "comfort working across ambiguity balancing speed, safety and value."
I split it in two. Move fast on the prototype, and slow down at the boundary where it touches real users or real data.
In practice: ship a small version quickly, put it behind human review, measure, then widen the scope step by step. That's how the tiered platform grew — exact Q&A first, then Text-to-SQL, then document RAG.
The safety rules — like "the model never writes SQL" — are set at the start and don't move. Speed is about everything else.
Why he asks — Your obvious weak point. Don't argue. Agree, then show what you bring and how you'd close it.
That's fair — I haven't. What I bring is a governed-by-default mindset. I already build systems where the model has limited power and humans approve, even when nobody asked me to. So the bank way of thinking is not new to me, only the specific rules are.
I also learn fast — I went from research scientist to shipping production AI systems in about two years. What I'd need to learn is NAB's specific risk and compliance process, and I'd want to pair with the risk squad early rather than find out late.
Why he asks — Curiosity and discipline. Avoid "I read Twitter". Show you test things.
Mainly by building. Every few months I take one new idea — MCP, deep agents, a new eval tool — and put it into a real project, not just a notebook. I follow the LangChain and Anthropic courses and release notes.
I don't chase every new model. When one comes out, I re-run my own eval set and let the numbers decide.
Why he asks — Fit and ambition. Tie it to the JD's "standards that endure beyond individual use cases".
Short term, I want to be the person on the team who's trusted to take an agent design all the way to production, safely. Longer term, I'd like to help set the standards — evaluation methods, design patterns — that other teams at NAB reuse.
The job description mentions "standards that endure beyond individual use cases". That's the part I find most exciting.
He is the Head of AI Science, not a tech lead. Keep the questions at the altitude he works at: direction, standards, team, trust. A question about implementation detail makes you sound like the person who wants the job below this one. You will get 3–5 minutes, so two is the right number — three if he is talkative.
| # | Question | Why it works |
|---|---|---|
| 1 | "When a request comes from the business, how does it usually reach the AI team? Is there a BA in between, or do the business people and the AI people sit down and work it out directly?" | Plain and practical. The business-to-AI handover is where most enterprise AI work stalls, and giving him two concrete options gets a real answer instead of "we collaborate closely". Also tells you how much of the job is stakeholder work. |
| 2 | "What are the AI team's main goals for the next six months?" | Short, direct, and squarely his to answer. His reply also tells you how to frame yourself if there is a second round. |
| 3 | "What kinds of business-facing AI agents has the team built so far, and what stage are they at — pilot, limited release, or fully in production?" | Naming the stages shows you know the gap between a demo and production is where the real work lives. Gets him talking about concrete work. |
| # | Question | Why it works |
|---|---|---|
| 4 | "Which agentic use case in production are you most proud of, and which one turned out harder than expected?" | The warmer version of #3 — asks for a story rather than an inventory. Do not ask both; pick whichever suits the mood. |
| 5 | "What does evaluation look like in the team today — and where would you like it to be in a year?" | Evaluation is the spine of the JD and his stated agenda. "Today versus a year" keeps it strategic instead of asking him to list tools. |
| 6 | "What would success look like for this role in the first six months?" | Best closer. Safe, delivery-focused, and it gives you the yardstick you will be measured against. Note it overlaps with #2 — ask one or the other, not both. |
| 7 | "You are shaping the technical culture of the AI function. What is one habit you are trying to build, and one you are trying to break?" | Pitched exactly at his level, and personal enough that he has to think. People remember the candidate who asked them something interesting. |
| 8 | "Over the next year, is the team's centre of gravity more about building new agentic products, or about making what already exists trustworthy enough to scale?" | Portfolio-level. Shows you think about sequencing and trade-offs, not just what to build next. |
| 9 | "What is the slowest part of the loop between an experiment succeeding and it reaching customers — the tech, the sign-off, or the integration?" | Delivery-focused and honest. Invites a candid answer and tells you what the job will actually feel like day to day. |
| 10 | "How early do Responsible AI and risk get involved? Is it a design constraint from day one, or a gate before release?" | Banking-specific. Signals you know regulated delivery is a different sport and you are not going to be surprised by it. |
| 11 | "What separates the people who do well in this team from the ones who struggle?" | Asks for the real bar rather than the job-ad version. Slightly bold, so use it only if the conversation has been warm. |
| 12 | "Where has NAB decided to own the AI capability outright, and where are you still happy to lean on vendors?" | Build-versus-buy is a leadership question. Good, but it can drift into procurement — keep it as a reserve. |
| 13 | "How does AI Science split the work with AI Engineering and the Platform team? Where does an AI Scientist's job start and stop?" | A sharper, more structural version of #1. Use it instead of #1 if he has already described the team in detail. |
Every answer box is editable — click into it, type, and it saves automatically in this browser (per device; use "reset to default" to undo).
Practice out loud once a day for the next four days. Record yourself on Q1, Q3 and Q8 — those three carry the interview.