How to Evaluate Agentic AI for Healthcare Referral Workflows in Dubai
A healthcare referral agent should help authorised staff assemble and coordinate a complete referral without assuming clinical authority. Evaluation must therefore test the entire handover: patient identity, permitted information access, preserved clinical meaning, professional review, receiving-service acceptance and the final appointment or escalation state.
Why referral coordination deserves system-level evaluation
A referral connects at least two healthcare organisations, several professionals and multiple clinical and administrative records. The initiating facility must identify the patient, explain the reason for referral, include relevant evidence and select an appropriate destination. The receiving service must then review the request, accept or redirect it and communicate the next step. A failure at any point can delay care even when every administrative field appears complete.
An agent may help staff retrieve records, identify missing information, prepare a referral summary and follow the case across systems. However, fluency can conceal a serious error. The agent may join similar patient records, omit a qualifying symptom, disclose information without an appropriate basis or state that a referral is booked before the receiving service has accepted it. Evaluation must therefore examine the completed handover rather than the quality of one generated message.
Define the agent's administrative role
The evaluation begins by stating what the agent may and may not do. A bounded referral agent may locate authorised records, check required fields, preserve source references, prepare a draft, request missing evidence, route the case under approved rules and report the receiving status. Licensed professionals must retain diagnosis, clinical priority, treatment and any material clinical communication that requires professional judgement.
This boundary determines the test. If the agent only prepares a draft, reviewers must check whether a professional receives the evidence needed to approve it. If the agent can transmit an approved referral, the evaluation must also test recipient identity, disclosure controls, acknowledgement and recovery from rejection or system failure. A general claim that the agent “supports referrals” is too broad to release safely.
Recreate the complete referral chain
The test environment should reproduce the states through which a referral can pass. These may include draft, awaiting evidence, awaiting consent or another information-sharing basis, ready for professional review, approved for transmission, received, accepted, redirected, rejected, booked and escalated. Each state needs a responsible actor and evidence of the transition.
Test cases should combine protected or synthetic patient records, referral notes, prior investigations, eligibility rules, destination capacity and acknowledgement messages. The evaluation should include late documents, duplicate records, unavailable systems and receiving-service rejection. These conditions show whether the agent maintains the case state or silently converts an unresolved referral into an apparently complete one.
Test identity and authorised access before completeness
Patient identity is a prerequisite for every later action. Create cases with similar names, dates of birth, medical-record numbers or contact details. The agent should keep the records separate, request resolution when identifiers conflict and prevent one patient's information from entering another patient's referral.
Access tests should vary the user's role, purpose, facility and information-sharing condition. The agent should retrieve only the information permitted for that referral and create a traceable record of every disclosure. A complete referral assembled from unauthorised or incorrectly joined records is a critical failure, not a high-quality result.
Preserve clinical meaning without making clinical decisions
Referral summaries can shorten long records, but compression can change the meaning of the case. The test set should include conflicting notes, uncertain findings, absent investigations and symptoms that may require urgent professional review. Clinicians should judge whether the draft preserves material facts, uncertainty, source attribution and the reason another professional must review the patient.
The agent should not resolve clinical ambiguity by inventing a diagnosis or priority. When evidence is missing or contradictory, it should identify the gap and refer the decision to the responsible professional. Evaluation should record unsupported diagnoses, lost symptoms, altered negation, missing qualifiers and unjustified priority changes as distinct failure types because each requires a different correction.
Challenge routing and receiving-state claims
A referral agent may use approved rules to suggest a facility or service, but availability and acceptance can change. Tests should include closed pathways, ineligible destinations, capacity changes, urgent cases and requests that fall outside the permitted route. The agent should explain the basis for its suggestion and escalate cases that the rules do not resolve.
The receiving state also requires verification. Sending a referral does not mean that the receiving service accepted it, and acceptance does not mean that an appointment was booked. The agent should report the last confirmed state, preserve rejection reasons and continue authorised follow-up without presenting an intended action as a completed event.
Combine machine checks with clinical and operational review
Automated checks can verify identity separation, required fields, access rules, source links, state transitions and final acknowledgements. They can also detect whether the agent used an unapproved tool or communicated before professional approval. These checks provide reproducible evidence across a large test set.
Clinicians must review whether the referral preserves the clinical concern and whether escalation is appropriate. Referral coordinators and receiving teams should also judge whether the proposed process creates avoidable rework, unclear ownership or excessive interruption. A useful release decision combines machine evidence with the judgement of the professionals who remain accountable for the handover.
Interpret results across pathways and patient groups
The evaluation should cover the pathways, facilities, urgency levels, record conditions, languages and patient groups that the intended service will encounter. Repeated runs are necessary when summarisation or routing can vary. Results should report denominators, uncertainty and the distribution of failures rather than one overall success rate.
A low average error rate cannot compensate for a wrong-patient join, unauthorised disclosure or unsafe loss of urgent information. Review every critical event individually and examine neighbouring cases for the same cause. The evaluation may support a claim about referral preparation and coordination; it should not claim improved clinical outcomes unless a separate study measures them.
Define release and pause rules before deployment
Initial release should limit the agent to referral types, facilities, records and actions supported by the evidence. Non-emergency completeness checking with explicit escalation may be a reasonable starting boundary. Professional acceptance, clinical priority and treatment decisions should remain outside the agent's authority.
Predefined pause rules protect patients while the organisation investigates failures. A wrong-patient join, unauthorised disclosure, unsupported clinical conclusion or false receiving-state claim should stop the affected route and trigger review of the cause and similar cases. Continued operation should depend on verified correction, not on an assurance that the failure was unusual.
What Marketways can evaluate
Marketways can map the referral pathway, define the agent's permitted actions, build protected scenarios and create checks for identity, access, completeness, state and traceability. We can also coordinate licensed clinical review of preserved meaning and escalation, then combine the findings into a release, restriction or redesign decision.
The resulting evidence pack can report critical failures, clinician correction, avoidable rework, waiting time, access violations and verified receiving states. Marketways does not diagnose patients, determine clinical priority or replace the responsibilities of licensed facilities and professionals. Applicable healthcare, privacy and information-sharing requirements should be confirmed by the responsible organisations and qualified advisers.