AI Skill
Playground

Test AI Agent Skills in realistic scenarios before deployment. Examine responses, trace supporting evidence, and discover where a skill needs more expert input.

See how expertise behaves.

  1. 01 Set the conditions
  2. 02 Examine the response
  3. 03 Improve the skill

01 / Set up the test

Put captured expertise into a situation that demands it

An expert’s explanation becomes useful when an AI agent can apply it in context. The Playground gives teams a place to exercise AI Agent Skills through scenarios and dialogue before introducing them into a workflow.

Start with a specific version and a decision the skill should support. Provide the evidence available in that situation, define the expected behavior, and examine what happens when a crucial condition changes.

A defined starting point

Choose the skill version, intended task, and source material the response should use.

A meaningful variation

Introduce an unfamiliar case, an incomplete input, or an exception the expert identified.

An expectation to examine

Decide what a useful answer should explain, ask for, or refer to a person.

02 / Change the context

One skill. Three situations worth testing.

Keep the release recovery skill fixed and change the evidence it receives. Select a scenario to explore an illustrative response and the criteria a reviewer would examine.

Illustrative scenariosRelease recovery · v1.2
InvestigateClarifyEscalateScenarioSkill v1.2
The version stays the same. The available evidence changes what a useful response should do.

Follow the evidence into the next investigation

Situation provided

Service errors began after a deployment. The incident timeline and change record are available. Compatibility with the previous application version has been confirmed.

Evidence in view

Incident timeline + change record

Illustrative response
Compare the failing requests with the release change record. Identify a compatible recovery option for the incident owner to assess. Any production change requires an authorized operator.
What the reviewer should look for

The answer should connect its recommendation to the supplied evidence and preserve the boundary around production actions.

  • Uses the timing as evidence to investigate, without treating correlation as proof.
  • Keeps the production decision with the authorized operator.

Ask for the evidence the decision depends on

Situation provided

Service errors are reported, but the first occurrence is unknown. A deployment happened today. There is no incident timeline establishing which event came first.

Evidence in view

Incomplete incident timeline

Illustrative response
When did the errors first appear, and how does that compare with the deployment time? Provide the incident timeline and relevant change record before choosing a recovery path.
What the reviewer should look for

The answer should identify the missing input and ask a focused follow-up question before recommending a recovery action.

  • Names the missing evidence instead of inventing a sequence of events.
  • Explains why that evidence matters to the next decision.

Recognize when the familiar recovery path does not fit

Situation provided

Errors began after a deployment that included a data migration. The previous application requires a field that the migration removed. A code rollback would leave the data incompatible.

Evidence in view

Migration record + application dependency

Illustrative response
Hold the code rollback. Ask the application owner to review the migration and identify a compatible recovery path. Include the removed-field dependency in the escalation.
What the reviewer should look for

The answer should recognize the compatibility exception, explain the constraint, and refer the recovery decision to the appropriate expert.

  • Connects the exception to the specific field dependency.
  • Escalates with context instead of repeating the usual rollback recommendation.

03 / Inspect the answer

An answer needs more than a convincing explanation

Examine the response alongside the source material and the behavior you expected. Use dialogue to probe an assumption, ask for a reference, or introduce another condition.

Check the source behind the statement

Open the relevant file, captured insight, or session excerpt. Check whether it supports the answer in this situation and whether a qualification was lost.

Examine the decision boundary

Look for the point where the skill should ask for more information or involve an expert. A useful answer makes that limit visible.

Turn a mismatch into a finding

Describe the input, the response, and the expectation it missed. Give the skill owner enough context to reproduce the case and review the underlying expertise.

From response to review

Keep the evidence in the conversation

SourcesResponseReviewFinding
Review connects a response to its sources. A finding can reveal where the skill needs clearer evidence or another expert explanation.
Example finding for the owner

The exception needs a source

The response recognizes a migration risk, but the source does not explain which application dependency creates it. Capture the owner’s explanation and attach the relevant change record before reviewing the revision.

Explore AI Skill Validation

04 / Develop the next version

Make the next test more informative

A surprising answer is a starting point for skill development. Use the case to find the missing expertise, refine the skill, and examine the revised version under the same conditions.

01 / Ask the expert

Recover the missing judgment

Bring the scenario back to the person who knows the work. Ask which signal matters, why the usual action fails, and what evidence supports a different response.

Explore capture sessions
02 / Refine the skill

Give the change a review

Update the decision rule, supporting source, or operating boundary. Keep the proposed revision identifiable so its owner can inspect and approve what changed.

Explore the skill catalog
03 / Revisit the case

Compare behavior in context

Exercise the revised version against the original case and related variations. Check the missing behavior again and examine whether the change affects other decisions.

Explore lifecycle analytics

Start with a decision worth testing

What should your AI agent do when the situation changes?

Bring a workflow, a skill, and a situation that requires judgment. Explore how to test the response, examine its sources, and turn what you find into a better next version.

RapidKT Plain Text Website