What you write, you keep

Psychologists named the generation effect in 1978: people remember material they produced themselves far better than the same material handed to them already finished. It is an awkward finding for a tool that writes on your behalf, and worth saying out loud.

The useful question is not how much the model can draft, but which parts are worth still doing yourself.

The answer is neither “all of it” nor “none of it”, and it turns on what the effect actually rewards. Not effort for its own sake, and not doing things the slow way. In the original experiments, the work asked of people was slight.

What the experiments asked

Norman Slamecka and Peter Graf compared two ways of meeting the same word. In the read condition, people saw a pair of words and read them. In the generate condition, they saw the first word, a rule such as “synonym” or “opposite”, and the first letter of the second word, and supplied the rest themselves: given “rapid”, the rule “synonym” and the letter “f”, they produced “fast”.

The cue and the first letter all but settled the answer, so both groups ended each trial looking at the same word. Yet across five experiments the generated words were remembered better — on recognition and on recall, under five kinds of rule (associates, categories, opposites, synonyms and rhymes), whether people worked at their own pace or against the clock. The information was identical. What differed was who had produced it.

The finding has held up. In 2007 Sharon Bertsch and colleagues pooled 86 studies of it. The advantage of generating over reading appeared across most of the tasks and test formats they examined, and it tended to be larger when memory was tested more than a day later than when it was tested straight away. It nearly vanished for one kind of material: nonsense words. One reading of that exception is that generation works through meaning — producing an item ties it to what you already know, and a nonword offers nothing to tie it to.

Fluency is not learning

Two neighbouring findings point the same way. The testing effect is the observation that retrieving something from memory strengthens it more than looking at it again. Henry Roediger and Jeffrey Karpicke (2006) had students study prose passages and then either restudy them or write down what they could recall, with no feedback. Tested five minutes later, the restudiers did better. Tested two days or a week later, the students who had practised recall remembered substantially more — even though restudying had made students more confident that they would remember.

Robert Bjork (1994) described the wider pattern in terms of desirable difficulties. Conditions that make practice smooth and performance look good during training are often not the ones that produce lasting learning, and conditions that slow practice down often are. He also described the trap that comes with ease: material that feels familiar and fluent is readily mistaken for material that is understood.

A model’s draft is about the most fluent material most people will ever read. It arrives organised, grammatical and plausible, phrased in the terms of the question you asked. Reading it produces the feeling Bjork warned about — of having understood — without the work that makes understanding stay. That is not a reason to avoid drafts. It is a reason not to count reading one as having learned anything.

What the effect does not claim

The generation effect is about memory. It says nothing about the quality of what you produce. A synonym generated in the 1978 experiments was no better a word than the printed one; it was simply better remembered. A model’s paragraph may well be clearer than yours, and when the paragraph is what matters, clarity should win.

Nor is everything worth remembering. Nobody needs to retain the citation style of a bibliography, the boilerplate of a weekly status update or the opening hours of every museum on a city break. Producing those by hand buys a memory you will never use. The effect is a reason to do some of the work yourself, not all of it.

Errors cut both ways. Attempting an answer before seeing it can help even when the attempt fails: in experiments by Nate Kornell, Matthew Hays and Robert Bjork (2009), people who guessed wrongly and were then shown the answer learned it better than people shown question and answer together. But in those experiments the correct answer always followed. Exposure to plausible errors has a cost of its own — Roediger and Elizabeth Marsh (2005) found that wrong options students had read on a multiple-choice test sometimes resurfaced as their answers on a later one. A plausible mistake in a fluent draft deserves the same suspicion.

Deciding what to keep

A workable rule follows. Keep the parts you will need to remember, explain or defend without the page in front of you. Hand over the parts where breadth matters more than retention: finding, collecting, formatting, and first drafts of material nobody will question you on. In most projects the parts worth keeping are the same few.

• The question. Writing it yourself forces a decision about what you actually need to know, and it is the part of the project you will come back to most.
• The structure. A rough outline — the claim, the evidence, the objection you expect — made before you ask for a draft, so the draft fills your shape instead of supplying one.
• Summaries of sources, in your own words, written after reading them. This is generation in its plainest form.
• The passages that carry the argument. Annotate them, then rewrite the two or three that matter, even when the model’s version was fine.

Consider an engineer choosing between two ways of storing a service’s data. A model can list the trade-offs, pull the relevant documentation and draft a comparison table in minutes, and it should. The question — which design fails more gracefully when traffic spikes — is hers, and so is the paragraph that explains the choice, because she will be asked about it in review and again when something breaks.

Or a student revising for an exam. Asking a model to summarise a chapter produces a good summary and a student who has, in effect, reread it. Writing down what they remember first, then asking the model what is missing or wrong, turns the same tool into the correction step that Kornell’s participants received. The model checks; the student retrieves.

A couple planning a kitchen renovation can let a model gather building regulations, supplier ranges and typical costs. The brief — how they cook, what the room has to hold, what they will not give up — is worth writing themselves. It is what they will need in their heads when a builder proposes a cheaper alternative on site.

An order of work

Put together, this is less a restriction than a sequence. You frame; the model widens; you narrow. The question and the rough structure come first and come from you. The model then does breadth — searching, collecting, drafting the routine parts. You return to annotate, cut and rewrite what matters, and the model can tidy what remains.

The order matters because whatever arrives first tends to set the frame. It is easier to edit a draft than to see what it left out, and your own outline, sketched beforehand, is what you measure the draft against. It is also where the plausible mistake gets caught. As The bottleneck moves argues, when drafting is cheap the scarce work is context and verification, and both go better when you wrote the core yourself.

In PlansFlow the sequence has a physical form. Your question and your own summary notes sit on the canvas as blocks beside web answers that arrive with their sources attached, and when you generate a draft you choose which blocks inform it. Your own words can be the input to the draft, not only corrections added at the end.

In practice

1. 01

   Write the question yourself, in one or two sentences, before asking a model anything — and keep it in view while you work.

2. 02

   Sketch the structure — claim, evidence, likely objection — before requesting a draft, then judge the draft against your sketch rather than the other way round.

3. 03

   After reading a source, close it and summarise it in your own words; ask the model to check the summary, not to write it.

4. 04

   Attempt an answer before asking for one, then compare. A wrong guess followed by the correct answer tends to stick better than the answer read alone.

5. 05

   Delegate breadth freely — searching, formatting, routine first drafts — and rewrite by hand the few paragraphs you will have to explain or defend.