How to Test an AI Scribe With Your Own SOAP Note Template

An AI scribe can produce the right SOAP headings and still give you a note that takes too much rearranging. A useful template test checks where information lands, how missing information is handled, and whether a correction survives the next generation.

Start with a written specification and a small, fixed set of fictional inputs. Change one template rule at a time and keep a record of what happened. The exercise below is a proposed evaluation workflow, not a report of completed Skriber testing or a claim of clinical validation.

Define what your template needs to do

Use a blank version of the note structure your practice needs. If you need a starting reference, see our SOAP note guide, SOAP cheat sheet, or a specialty structure such as pediatric SOAP notes. This test evaluates how a tool follows your chosen structure; it does not require another library of example notes.

Write down these requirements before generating a test note:

• Headings and order: List the exact headings and subheadings you need.
• Field placement: Specify whether each section will go into a separate EHR field or one combined note.
• Writing preferences: Choose paragraph or bullet style, desired detail, and terminology appropriate to your practice.
• Missing information: State how an unsupplied item should be flagged or left incomplete. An empty allergy field must not become an assertion of no allergies.
• Human decisions: Require the note to preserve the clinician's supplied assessment and plan. A heading must not invite a tool to invent either.

Give the specification a simple version name, such as Template A. Keep a separate test record with the template version, selected settings, source script, generated draft, corrections required, and decision. These labels are for your own evaluation notes.

Request the structure through the actual product workflow

In Skriber, support builds and assigns custom templates. A request can include your specialty and note type, headings and order, EHR field layout, a synthetic example of the desired output, and a specific problem to fix. Confirm what has been assigned before beginning your comparison.

Use a blank template and invented material for this exercise. Do not send real patient notes or screenshots as test examples. Keep a copy of your specification so a support conversation does not become the only record of the requested behavior.

Choose the assigned template before recording. Switching the template on an existing visit regenerates its note, so preserve any wording you need first and review the entire regenerated result. If your account offers a template preview, use it to check the structure; a preview does not establish how the tool will handle your test inputs.

Put each change in the right place

A template problem and an encounter-specific fact need different controls:

• Wrong main structure: Refine the template specification with support. Keep an example of the misplaced section.
• A recurring writing preference: Use Custom Instructions for the template's persistent preferences. Keep encounter facts out of these standing instructions.
• Background for one encounter: Use Visit Context for that visit. It supplements the audio; it is not a text-only note-generation route.
• A change to one draft: Use manual editing for an exact factual correction, or Magic Edit for a focused section rewrite. Do not turn a one-visit correction into a permanent default.

Changing a preference is not proof that the current draft has changed. Follow the relevant regeneration step and inspect the result. A useful instruction is a requirement to test, not a guarantee that unsupported details will be excluded.

Use three fictional test conditions

Every person, symptom, observation, assessment, and plan below is invented. These are documentation-fidelity exercises, not clinical recommendations or examples of actual Skriber output. The expected behavior is an evaluation criterion, not a result we observed.

The excerpts define the conditions to include in your written master script. Build a clear fictional recording that meets the current capture guidance, then keep its wording fixed across revisions. Skriber's Visit Context guide calls for at least about 30 seconds of clear audio; pasting these excerpts into Context alone does not substitute for that recording.

Use a separate fictional test visit for each case. Replacing Context while keeping another case's original recording does not remove that earlier source material. When testing a template revision, repeat each case against its own unchanged source.

Test A: preserve supplied facts and section placement

Fictional patient: My left wrist has been sore since Monday. I have no numbness.
Fictional clinician: For this exercise, I observed mild swelling at the left wrist. My assessment is wrist soreness, with no cause specified. My plan in this fictional case is a follow-up conversation in one week.

Check that the history, observation, assessment, and plan land in the specified sections. Preserve the left side, timing, negative symptom, and supplied uncertainty. The draft should not add a diagnosis, medication, normal strength, or another examination finding. The one-week plan is a supplied test fact, not a recommended interval for real care.

Test B: leave unsupplied information unresolved

Fictional patient: My wrist has been sore since Monday.
Test condition: No side, examination findings, medication or allergy history, clinician assessment, or management plan is supplied.

Check your missing-information rule. The draft must not borrow the side, examination, assessment, or follow-up from Test A. No medication discussion does not mean no medications. An unfilled section should remain visibly incomplete under your approved documentation process.

Test C: retain the explicit correction

Use Test A's script, then add these fictional corrections:

Fictional patient: Correction: I meant my right wrist, not my left. I still have no numbness.
Fictional clinician: Correct the observation to the right wrist. For this fictional case, change the follow-up conversation to two weeks, not one.

Check all affected sections for the corrected side and plan. Look for contradictory left/right entries, both follow-up intervals appearing as current instructions, or the negative symptom disappearing. Do not accept a polished paragraph in place of that check.

Record a decision for each case

For every generated draft, record the template version, input version, settings used, and the exact mismatch or correction. Keep format and source fidelity separate:

• Structure: Are headings, order, and EHR field boundaries as specified?
• Fidelity: Are supplied facts, uncertainty, negation, and corrections preserved without unsupported additions?
• Missing fields: Can the reviewer identify what still needs information?
• Editing: What must a human change before the note fits the intended destination?

Use a practical decision: continue testing when the case meets your stated criteria; revise when a specific formatting or wording rule needs work; or stop and resolve when facts are invented, omitted, or contradicted. Passing this small exercise does not establish reliability for real encounters or replace clinician review.

Revise one rule and repeat the same conditions

A useful refinement request identifies the input, the observed output, and the desired change. For example, if Test B's draft supplied a follow-up, a request could say: “The script never stated a follow-up. The missing plan should remain unresolved.” That is more actionable than asking for a better note.

Change one structural or wording rule, label the next version, and repeat all three conditions. Record whether you regenerated an existing test visit or recorded the script again; a changed recording can introduce a capture difference into the comparison.

Regeneration can replace wording and cannot recover audio that was never uploaded. Where a visit transcript is available, compare it with the note and Context to distinguish missing source material from a template or instruction problem. Preserve useful edits before another generation.

Check the reviewed note at its destination

Finish the exercise with a practice-approved test destination. After reviewing the complete draft, check the intended EHR field, section boundaries, headings, and spacing. Skriber's copy guidance notes that an individual section copy may omit its heading.

EHR Sync is optional and configuration dependent; confirm availability for your account rather than assuming an automatic handoff. A clean transfer does not establish that the clinical content is correct. The clinician still reviews the note before use.

Ask Skriber about the template you want to test, or try a fictional encounter in Skriber. Bring your blank structure, fixed test conditions, and specific criteria so you can evaluate the result against the note you actually need.