Letting AI write health articles without making things up: method and results

AI writes fluent, well-organised prose, and it sounds exactly as confident when it is wrong as when it is right. That is a real problem when the subject cannot be wrong: health, food, herbs. This article describes how we had two AI writers produce a four-part Thai herb series for a sister site (mykaset.com) on 16 October 2026, the rules they were given, and a real log of what the checks turned up.

Why this matters

An LLM is trained to write what is likely to come next, not what is true. When it does not know a number it does not stop; it fills in one that looks plausible. People call this hallucination. In a technical article that is forgivable: a reader runs the command, it fails, they know. In an herb article that says “safe to take with this medicine”, nobody knows it was wrong until someone gets hurt.

So the rule for every site in the group was written in from day one: no invented facts. Health content stays general and careful, claims no cures, and tells the reader to ask a doctor or pharmacist. The question was how to make an AI actually follow that rule, rather than merely asking it to.

The six rules the AI writers got

Each writer received one brief with six hard rules.

1. Verify every claim before writing it, by searching and opening real web pages. Prefer official sources: Thailand’s Department of Thai Traditional and Alternative Medicine, the Thai FDA, NCCIH/NIH in the US, a national drug regulator, and university extension services for growing and storage
2. If it cannot be verified, leave it out. Do not soften it. “Some reports suggest” does not turn an unsourced claim into a sourced one
3. End every article with a short “Sources” list, 3-5 links that were actually opened, not links the model remembers probably exist
4. No dosing advice beyond quoting an official label or guideline, no cure claims, and a visible “general knowledge, not medical advice” line
5. Report back: every source used, and every claim dropped and why
6. The riskiest article passes a person, and a second AI pass, before publishing

Rules 2 and 5 are the heart of it. Rule 2 makes articles shorter, but every sentence stands. Rule 5 gives the person a list to compare against instead of having to doubt every line on their own.

If it cannot be verified, it disappears from the article. It does not get softer.

What actually happened

The riskiest article was “food or medicine”, covering turmeric, andrographis (fa thalai jone) and fingerroot. This table comes from the report the AI sent back.

Starting claim	What the check found	What was done
The brief said Australia’s regulator had “200+ severe allergic reaction reports since 2019” for andrographis	The verifiable figure was 287 anaphylaxis reports since 2005, including one death reported in 2024, and the ingredient was removed from the low-risk medicines list effective 17 Sep 2026	Used the verified numbers, not the brief’s. The person writing the brief was wrong, and the AI caught it
A “turmeric and gallstones” caution	Only commercial health sites said it; nothing in an official source that could be opened	Dropped, even though it sounded like a warning that is “safe to include”
Turmeric and liver injury, meant to cite NIH LiverTox	The LiverTox page could not be opened (bot check); the Australian regulator’s page timed out	Cited a peer-reviewed case series from the DILIN network (10 cases, 2023) and an Australian pharmacists’ association article instead, and said so in the report
Andrographis “lowers blood pressure”	The Thai national formulary wording is: caution with antihypertensives, may add to their effect	Written in the official wording, not as a benefit
Growing time to harvest for each plant	No official source gave a precise figure	Written as ranges with “typically”, e.g. fingerroot typically around 8 months, and flagged to the person as having no official source

Notice the first three rows are three different things. In the first, the AI corrected the person who gave it the job. In the second, it cut something harmless-looking. In the third, it found a way around a source it could not open, and said so plainly. All three happened only because the rule forced “open before writing”. Written from memory, the 200+ figure would certainly be in the article, and so would the gallstone warning, because it appears in text the model has seen a million times.

Before deploy, a second AI was given a single task: reopen the key source for the Australian numbers and confirm the figures in the article match the page.

What still needs a person

This cuts the human work a lot. It does not remove the human.

• Deciding what counts as official. The AI can tell a university site from a supplement shop. Borderline cases, such as a professional association’s article summarising a government notice, need a person to decide. Here it was accepted: a pharmacists’ association quoting the regulator’s notice directly
• Reading the dropped list. Sometimes what was cut matters more than what stayed. If the AI dropped a warning that should be there because it found no source, the person goes and finds one
• The flagged numbers. The growing times were flagged as having no official source. A person judged that acceptable as rough ranges for a home garden. The same kind of number about a medicine would not have been
• Opening the links. At least the ones that support the riskiest claims. Every link the AI sent in this job opened and said what was claimed, but that is because rule 3 required real opening

A reusable brief

With the site-specific parts removed, this is the skeleton for any high-stakes writing task.

Task: write an article on <topic>, in <language>, about <N> words
Reader: <who>  Tone: <what>

Hard rules
1. Every fact, figure, contraindication and caution must be searched for and
   opened on a real web page before you write it.
   Prefer these sources: <official bodies for this subject>
2. Cannot verify = leave it out. Do not soften it instead.
3. End with a "Sources" section: 3-5 links you actually opened, nothing else.
4. No dose or quantity advice except quoting an official label or guideline
   verbatim. No cure or prevention claims.
   Include the line "This is general information, not medical advice."
5. If a figure in this brief conflicts with a source you opened, use the
   source and tell me.
6. Send a report separate from the article: every source link used, every
   claim dropped and why, sources you could not open and what you used
   instead, figures with no official source.

Deliver: the article file + the report. Do not publish anything yourself.

Rule 5 was added after the 200+ lesson. The last line is our standing rule for every agent: anything irreversible passes a person (more in Prompt injection and EP3 Subagents).

Common mistakes

• Letting the AI write from memory, then asking it to “add references” at the end. You get links that look right but do not open, or open and do not say what is claimed
• Accepting “soften it” instead of “cut it”. Words like may and some reports do not make an unsourced claim true
• Not asking for the dropped list, so you never learn that an important warning vanished because the AI found no source
• Trusting the figures in your own brief over the sources. The person writing the brief misremembers too
• One AI that both writes and publishes. A wrong health article cannot be fixed fast enough after the fact