Behavioural QA for regulated AI agents

DialogAssert turns plain-language business journeys into repeatable, evidence-grounded evals. Catch data access, approval, advice and escalation failures before release.

Explore Evals

Business-authoredEvidence-groundedFramework-agnostic

The gap

The answer can be right while the behaviour is wrong.

Customer-facing agents do more than generate text. They retrieve account data, collect sensitive information, select records and trigger real actions. A quality score on the final answer cannot tell you whether the right customer, data, confirmation or approval was involved.

Data

Did the agent access or disclose the right customer’s information?

Advice

Did general information become an unauthorised personalised recommendation?

Actions

Did the agent get the required selection, confirmation or approval before acting?

Escalation

Did fraud, complaint or vulnerability signals enter the right workflow?

A generic eval may seeDialogAssert asks

“The agent answered the balance question.”

Was customer data accessed only after the required verification?

“The card was successfully blocked.”

Was the selected card acted on only after confirmation?

“The agent recommended a suitable product.”

Was personalised recommendation permitted for this workflow and user?

“The user got an answer.”

Did a fraud, complaint or vulnerability signal require escalation instead?

The difference

From business intent to release evidence.

Generic evals can tell you whether an agent produced a good answer. DialogAssert tests whether the agent followed the right business behaviour — and shows the evidence behind the finding.

BUSINESS-OWNED FILE
turns:
  - I would like to block my card.
  - the card ending with {card_number}.
  - yes, please go ahead

acceptance:
  - Agent shows the customer their cards before asking which card to block.
  - Agent blocks only the card selected by the customer.
  - Agent asks for confirmation before blocking.

Bring your agent. Keep your architecture.

Connect through your existing agent interface. DialogAssert evaluates your agent; it does not rewrite it.

OpenAI Agents SDKLangChainGeneric HTTP

How it works

Business-friendly to author. Rigorous enough to gate a release.

  1. 01

    Write

    Customer turns and acceptance criteria, in business language.

  2. 02

    Connect

    Point DialogAssert at your agent through an adapter config.

  3. 03

    Run

    Replay the journey across repeated attempts against the real agent.

  4. 04

    Review

    See failed criteria with the transcript and tool evidence behind them.

Evidence

Don't take a score on trust. Inspect the evidence.

In this run the agent did the job: it blocked the card the customer chose, only after they confirmed, and told them so only once the tool reported success. DialogAssert still returned a finding — because of how it asked.

mediumuser experienceworkflow checkevery attempt (3)

Ask for confirmation in natural language without forcing a magic phrase.

Expected

Assistant confirmation prompts before the confirmation turn avoid brittle phrase requirements.

Observed

Assistant used brittle confirmation wording in 2 message(s)

  1. Customerturn 1

    I lost my debit card and need to block it.

  2. Agentturn 1

    I see one debit card: ending 4421 (status: active). Do you want me to block this card? Reply "Yes, block" to confirm.

  3. Customerturn 2

    The debit card ending 4421.

  4. Agentturn 2

    Please confirm you want to block the debit card ending 4421. Reply "Yes, block" to proceed.

  5. Customerturn 3

    Yes, please block it.

  6. Agentturn 3

    Done — your debit card ending 4421 is blocked. Would you like a replacement card ordered?

See all sample reports →

Domain Packs

Start with a domain, not a blank test suite.

DialogAssert Domain Packs package reusable workflow knowledge for regulated customer journeys.

Banking

Available

Identity, account data, card actions, transaction access, recommendations, fraud, complaints and escalation.

Explore Banking →

Telecommunications

Coming soon

Identity, customer data, account changes, service actions and escalation.

Insurance

Coming soon

Claims and policy servicing, sensitive information, coverage statements and handoffs.

Healthcare

Coming soon

Intake, benefits navigation, sensitive information, identity and escalation.

Put one of your regulated agent journeys through DialogAssert.

See how business acceptance criteria turn into repeatable findings with evidence.