A synthetic test file for AI medical record summaries, with answer key
Answer in brief: this is a 12-page fictional record of a right shoulder work injury with 15 planted traps: a fax date, a late signature, a "rule out", an ordered MRI with no result, a copied-forward work status, a combination opioid dose and another patient's MRI. Download it, ask any AI tool for a summary and chronology, and score the output against the answer key. Synthetic, no real patient.
Limits: We can't vouch for any tool's error rate on your files: the studies we cite measured other tasks and other models, and vendor features are as each vendor's site stated on the capture date.
Why test on a file with known answers
A tool's own demo file is clean. A tool's accuracy figure, where a vendor publishes 1, comes from a test set you can't see. The only result you can trust is 1 you score yourself, on a file where you already know what a correct summary says. A closed case of your own is the best test, but it needs a BAA first. This file needs none: nothing in it is real, so you can try any tool, including a general chatbot, before you sign anything.
The traps are built from the 10 failure types. Every type appears at least once, and some pages carry 2, such as the orthopedic addendum on page 12. We haven't published results for any tool on this file; the point is that you run it.
The test file
Download the test file (plain text, 12 pages)
To make it closer to a real production, paste each page into a document, add a page break between pages, and save as PDF. Printing and scanning it adds OCR noise, which is a fairer test still. Then ask the tool for a summary and a chronology with page citations, focused on the right shoulder injury of 2025-09-08.
SYNTHETIC TEST FILE, NO REAL PATIENT Published by medicalrecordssummaryai.com, 2026-11-12. All names, dates, facilities and record numbers are invented. Use: save as a PDF or upload as text to an AI summary tool, ask for a summary and chronology, then score the output against the answer key at https://medicalrecordssummaryai.com/test-file/ 12 pages. Page breaks are marked ===== PAGE n =====. ===== PAGE 1 ===== FAX COVER SHEET Received: 2025-11-03 14:12 From: Hillcrest Orthopedics, records department To: Records requester Pages including cover: 12 Re: Rowan Example, DOB 1971-08-19 Note: records enclosed as received from multiple facilities. ===== PAGE 2 ===== FAX 2025-11-03 14:12 HILLCREST ORTHO P.2/12 Northside Urgent Care Patient: Rowan Example DOB 1971-08-19 MRN SYN-40417 Date of service: 2025-09-09 HPI: 54 y/o male reports R shoulder pain after lifting a 40 lb box at work on 2025-09-08. Pain 7/10, worse with reaching overhead. Denies prior shoulder injury. Denies numbness or tingling. Right-hand dominant. PMH: Type 2 diabetes, on metformin. Hypertension. Exam: Tenderness anterior R shoulder. Painful arc 70 to 120 degrees. Neer positive on the right. Neer negative on the left. Distal pulses 2+. Sensation intact to light touch in R upper extremity. Assessment: R shoulder strain, r/o rotator cuff tear. Plan: X-ray R shoulder. Sling as needed. Naproxen 500 mg BID with food. Modified duty: no lifting over 10 lb with R arm. Follow up with orthopedics in 2 to 3 weeks. Electronically signed: J. Park, PA-C, 2025-09-12 08:31 ===== PAGE 3 ===== FAX 2025-11-03 14:12 HILLCREST ORTHO P.3/12 Northside Imaging Patient: Rowan Example DOB 1971-08-19 MRN SYN-40417 Exam: XR shoulder R, 3 views Exam date: 2025-09-09 Dictated: 2025-09-10 Findings: No acute fracture or dislocation. Mild acromioclavicular joint degenerative change. Type II acromion. Glenohumeral joint space preserved. Impression: No acute osseous abnormality. Mild AC joint arthrosis. ===== PAGE 4 ===== FAX 2025-11-03 14:12 HILLCREST ORTHO P.4/12 Maple Family Medicine Patient: Rowan Example DOB 1971-08-19 MRN SYN-40417 Date of service: 2024-03-14 CC: Diabetes follow-up. Also R shoulder pain. HPI: Chronic R shoulder pain x 6 months, worse overhead. Patient attributes it to yard work at home. No injury at work. A1c last visit 7.6. Exam: Mild pain with R shoulder abduction above 90 degrees. Strength 5/5. Assessment: 1. Type 2 diabetes. 2. R shoulder pain, likely impingement. Plan: Refer to physical therapy for R shoulder. Consider imaging if no improvement. Continue metformin 500 mg BID. ===== PAGE 5 ===== FAX 2025-11-03 14:12 HILLCREST ORTHO P.5/12 Hillcrest Orthopedics Patient: Rowan Example DOB 1971-08-19 MRN SYN-40417 Date of service: 2025-10-02 HPI: R shoulder pain since lifting injury at work 2025-09-08. On modified duty. No improvement with naproxen. Exam: Active forward flexion 100 degrees. Positive drop arm test on the right. External rotation strength 4/5. Assessment: R shoulder pain. Possible full-thickness supraspinatus tear. Plan: MRI R shoulder without contrast ordered. Physical therapy evaluation. Continue modified duty. Will discuss surgical options if MRI confirms a full-thickness tear. A1c ordered today (diabetes, surgical planning). ===== PAGE 6 ===== FAX 2025-11-03 14:12 HILLCREST ORTHO P.6/12 Regional Lab Services Patient: Rowan Example DOB 1971-08-19 MRN SYN-40417 Ordered: 2025-10-02 Collected: 2025-10-02 09:40 Resulted: 2025-10-04 16:05 Hemoglobin A1c: 8.4 percent (H) Reference: below 5.7 Reviewed and signed: Hillcrest Orthopedics, 2025-10-06 Printed 2025-11-03 13:58 ===== PAGE 7 ===== FAX 2025-11-03 14:12 HILLCREST ORTHO P.7/12 Riverside Physical Therapy, initial evaluation Patient: Rowan Example DOB 1971-08-19 MRN SYN-40417 Date of service: 2025-10-06 Subjective: Pt reports numbness in the R hand for the past week, mostly at night. Pt states ortho told him he needs surgery. Objective: R shoulder flexion 95 degrees, abduction 80 degrees. Strength 4/5 external rotation. Assessment: Deficits consistent with rotator cuff pathology. Plan: PT 2x per week for 6 weeks. Home exercise program. ===== PAGE 8 ===== FAX 2025-11-03 14:12 HILLCREST ORTHO P.8/12 Hillcrest Orthopedics Patient: Rowan Example DOB 1971-08-19 MRN SYN-40417 Date of service: 2025-10-23 HPI: R shoulder pain since lifting injury at work 2025-09-08. On modified duty. No improvement with naproxen. Pt states MRI done 2025-10-16 at Lakeview Imaging; report not available at today's visit. Exam: Active forward flexion 120 degrees. External rotation strength 4+/5. Assessment: R shoulder pain. Possible full-thickness supraspinatus tear. Plan: Obtain MRI report. Continue PT. Work status: modified duty, no lifting over 10 lb with R arm (unchanged). ===== PAGE 9 ===== FAX 2025-11-03 14:12 HILLCREST ORTHO P.9/12 Hillcrest Orthopedics, work status form Patient: Rowan Example DOB 1971-08-19 Date: 2025-10-23 Work status: Return to full duty on 2025-10-27. No restrictions. Signed: M. Ortiz, PA-C ===== PAGE 10 ===== FAX 2025-11-03 14:12 HILLCREST ORTHO P.10/12 [Scanned handwritten medication list, Northside Urgent Care] Patient: R. Example Hydrocodone/APAP 5/325, 1 tab q6h prn severe pain, #20, no refills. 2025-09-09. Naproxen 500 mg BID. ===== PAGE 11 ===== FAX 2025-11-03 14:12 HILLCREST ORTHO P.11/12 Lakeview Imaging Patient: Rowena Exampel DOB 1971-03-19 MRN SYN-40471 Exam: MRI shoulder LEFT without contrast Exam date: 2025-10-16 Findings: Full-thickness tear of the supraspinatus tendon with 1.5 cm retraction. Moderate subacromial bursitis. Impression: Full-thickness supraspinatus tear with retraction. ===== PAGE 12 ===== FAX 2025-11-03 14:12 HILLCREST ORTHO P.12/12 Hillcrest Orthopedics Patient: Rowan Example DOB 1971-08-19 MRN SYN-40417 Addendum dated 2025-10-30 to note of 2025-10-23: MRI report reviewed by phone with Lakeview Imaging (exam 2025-10-16): partial-thickness articular-sided supraspinatus tear, no retraction. No surgical indication at this time. Continue PT. Recheck in 6 weeks. Signed: A. Brandt, MD, 2025-10-30 END OF SYNTHETIC TEST FILE
Answer key: the 15 traps
A correct output handles each trap the way the middle column says. The right column is the wrong answer each trap is built to catch.
| # | Page | Trap | Correct output | Wrong output it catches |
|---|---|---|---|---|
| 1 | 1 | Fax date | No event dated 2025-11-03; that is when the file was sent | A visit or record dated 2025-11-03 |
| 2 | 2 | Signature date | Urgent care visit on 2025-09-09 | Visit on 2025-09-12, the e-signature date |
| 3 | 2 | Hedge | Shoulder strain; rotator cuff tear to be ruled out | Rotator cuff tear diagnosed 2025-09-09 |
| 4 | 2, 7 | Negation and onset | No numbness at the 2025-09-09 visit; R hand numbness first reported 2025-10-06, onset about 1 week earlier | Numbness since the injury, or numbness caused by the injury |
| 5 | 2 | Attribution | Patient denied prior shoulder injury (patient report) | "No prior shoulder injury" stated as fact |
| 6 | 4 | Omission of prior history | R shoulder pain for 6 months documented 2024-03-14, attributed by the patient to yard work; PT referral then | No prior history, or p. 4 left out |
| 7 | 3 | Negative and degenerative findings | X-ray: no fracture or dislocation; mild AC joint arthrosis; type II acromion | Degenerative findings dropped, or "abnormal X-ray" |
| 8 | 5 | Order, not result | MRI ordered 2025-10-02 for a possible full-thickness tear | MRI showed a full-thickness tear |
| 9 | 6 | Lab dates | A1c 8.4, collected 2025-10-02, resulted 2025-10-04 | A1c dated 2025-10-06 (signature) or 2025-11-03 (print or fax) |
| 10 | 7 | Attribution | Patient told PT that orthopedics said he needs surgery; orthopedics found no surgical indication (p. 12) | Orthopedics recommended surgery |
| 11 | 8, 9 | Copy-forward | Note repeats "modified duty (unchanged)"; same-day form returns him to full duty 2025-10-27; conflict flagged | Remained on modified duty |
| 12 | 10 | Dose and units | Hydrocodone 5 mg with acetaminophen 325 mg, 1 tab every 6 hours as needed, 20 tablets | Hydrocodone 325 mg |
| 13 | 11 | Wrong patient and side | Page 11 belongs to another patient (name, DOB, MRN differ; left shoulder) and is set aside | Full-thickness tear with retraction attributed to this patient |
| 14 | 12 | Addendum date, secondhand result | Partial-thickness tear, no retraction, per addendum of 2025-10-30 relaying the report by phone | MRI result known at the 2025-10-23 visit, or stated as if from the report |
| 15 | 8, 12, 7, 4 | Missing records | Missing: this patient's MRI report of 2025-10-16; PT notes after the 2025-10-06 evaluation; 2024 PT records | No missing-records flag |
Answer key written by medicalrecordssummaryai.com, 2026-11-12. Synthetic, no real patient.
Score the output
Tick each trap the tool handled the way the answer key says. Traps 3, 8, 12 and 13 are stop failures: each puts a wrong diagnosis, result or dose into a report, so 1 miss there outweighs a good score elsewhere. Nothing is saved or sent.
How to read the result
We don't set a pass mark, because the right bar depends on how you will use the output. 3 patterns are worth noting whatever the score:
- Misses on traps 13 and 15 together. The tool doesn't check patient identifiers and doesn't track what is missing. Expect wrong-patient findings in real productions.
- Misses on traps 3, 5 and 10. The tool flattens meaning: hedges and speakers get lost. You will need a full cue sweep on every file.
- Misses on traps 1, 2, 9 and 14. The tool takes the first date it sees. Every chronology it builds needs a date check.
Run the file 2 times. Many tools give different outputs on the same input, and a trap caught in 1 run and missed in the other is a miss.
Questions about the test file
Is the test file real patient data?
No. Every name, date, facility and record number is invented. It is safe to paste into any tool, including one you have no BAA with.
Can I change the file?
Yes. Change names, dates and sides so a tool can't have seen it before, and update the answer key to match. Adding real-world noise, such as printing and scanning it, makes it a harder test.
Has any tool passed this file?
We haven't published results for any tool, including our own. Scores depend on prompts and settings, so the useful result is the one you get with your setup.