良いプロンプトを書く方法：間隔反復を使って理解を深める

How to write good prompts: using spaced repetition to create understanding As a child, I had a goofy recurring daydream: maybe if I type just the right sequence of keys, the computer would beep a few times in sly recognition, then a hidden world would suddenly unlock before my eyes. I’d find myself with new powers which I could use to transcend my humdrum life. Such fantasies probably came from playing too many video games. But the feelings I have when using spaced repetition systems are strikingly similar. At their best, these systems feel like magic. This guide assumes basic familiarity with spaced repetition systems. For an introduction, see Michael Nielsen, Augmenting Long-term Memory (2018), which is also the source of the phrase "makes memory a choice." Memory ceases to be a haphazard phenomenon, something you hope happens: spaced repetition systems make memory a choice. Used well, they can accelerate learning, facilitate creative work, and more . But like in my childhood daydreams, these wonders unfold only when you press just the right sequence of keys, producing just the right incantation. That is, when you manage to write good prompts—the questions and answers you review during practice sessions. Spaced repetition systems work only as well as the prompts you give them. And especially when new to these systems, you're likely to give them mostly bad prompts. It often won’t even be clear which prompts are bad and why, much less how to improve them. My early experiments with spaced repetition systems felt much like my childhood daydreams: prodding a dusty old artifact, hoping it’ll suddenly spring to life and reveal its magic. Happily, prompt-writing does not require arcane secrets. It's possible to understand somewhat systematically what makes a given prompt effective or ineffective. From that basis, you can understand how to write good prompts. Now, there are many ways to use spaced repetition systems, and so there are many ways to write good prompts. This guide aims to help you create understanding in the context of an informational resource like an article or talk. By that I mean writing prompts not only to durably internalize the overt knowledge presented by the author, but also to produce and reinforce understandings of your own, understandings which you can carry into your life and creative work. For readers who are new to spaced repetition, this guide will help you overcome common problems that often lead people to abandon these systems. In later sections, we'll cover some unusual prompt-writing perspectives which may help more experienced readers deepen their practice. If you don't have a spaced repetition system, I'd suggest downloading Anki and reading Michael's aforementioned essay . Our discussion will focus on high-level principles, so you can follow along using any spaced repetition system you like. Let's get started. The central role of retrieval practice No matter the application, it’s helpful to remember that when you write a prompt in a spaced repetition system, you are giving your future self a recurring task. Prompt design is task design. If a prompt “works,” it’s because performing that task changes you in some useful way. It’s worth trying to understand the mechanisms behind those changes, so you can design tasks which produce the kind of change you want. The most common mechanism of change for spaced repetition learning tasks is called retrieval practice. In brief: when you attempt to recall some knowledge from memory, the act of retrieval tends to reinforce those memories. For more background, see Roediger and Karpicke, The Power of Testing Memory (2006). Gwern Branwen's article on spaced repetition is a good popular overview. You’ll forget that knowledge more slowly. With a few retrievals strategically spaced over time, you can effectively halt forgetting. The physical mechanisms are not yet understood, but hundreds of cognitive scientists have explored this effect experimentally, reproducing the central findings across various subjects, knowledge types (factual, conceptual, procedural, motor), and testing modalities (multiple choice, short answer, oral examination). The value of fluent recall isn't just in memorizing facts. Many of these experiments tested students not with parroted memory questions but by asking them to make inferences, draw concept maps See e.g. Karpicke and Blunt, Retrieval Practice Produces More Learning than Elaborative Studying with Concept Mapping (2011); and Blunt and Karpicke, Learning With Retrieval-Based Concept Mapping (2014). , or answer open-ended questions. In these studies, improved recall translated into improved general understanding and problem-solving ability. Retrieval is the key element which distinguishes this effective mode of practice from typical study habits. Simply reminding yourself of material (for instance by re-reading it) yields much weaker memory and problem-solving performance. The learning produced by retrieval is called the “testing effect” because it occurs when you explicitly test yourself, reaching within to recall some knowledge from the tangle of your mind. Such tests look like typical school exams, but in some sense they’re the opposite: retrieval practice is about testing your knowledge to produce learning, rather than to assess learning. Spaced repetition systems are designed to facilitate this effect. If you want prompts to reinforce your understanding of some topic, you must learn to write prompts which collectively invoke retrieval practice of all the key details. We'll have to step outside the scientific literature to understand how to write good prompts: existing evidence stops well short of exact guidance. In lieu of that, I’ve distilled the advice in this guide from my personal experience writing thousands of prompts, grounded where possible in experimental evidence. For more background on the mnemonic medium, see Matuschak and Nielsen, How can we develop transformative tools for thought? (2019). This guide is an example of what Michael Nielsen and I have called a mnemonic medium . It exemplifies its own advice through spaced repetition prompts interleaved directly into the text. If you're reading this, you've probably already used a spaced repetition system. This guide's system, Orbit , works similarly. If you have an existing spaced repetition practice, you may find it annoying to review prompts in two places. As Orbit matures, we'll release import / export tools to solve this problem. But it has a deeper aspiration: by integrating expert-authored prompts into the reading experience, authors can write texts which readers can deeply internalize with relatively little effort. If you're an author, then, this guide may help you learn how to write good prompts both for your personal practice and also for publications you write using Orbit . You can of course read this guide without answering the embedded prompts, but I hope you'll give it a try. These embedded prompts are part of an ongoing research project. The first experiment in the mnemonic medium was Quantum Country , a primer on quantum computation. Quantum Country is concrete and technical: definitions, notation, laws. By contrast, this guide mostly presents heuristics, mental models, and advice. The embedded prompts therefore play quite different roles in these two contexts. You may not need to memorize precise definitions here, but I believe the prompts will help you internalize the guide's ideas and put them into action. On the other hand, this guide's material may be too contingent and too personal to benefit from author-provided prompts. It's an experiment, and I invite you to tell me about your experiences . One important limitation is worth noting. This guide describes how to write prompts which produce and reinforce understandings of your own, going beyond what the author explicitly provides. Orbit doesn't yet offer readers the ability to remix author-provided prompts or add their own. Future work will expand the system in that direction. Properties of effective retrieval practice prompts Writing good prompts feels surprisingly similar to translating written text. When translating prose into another language, you’re asking: which words, when read, would light a similar set of bulbs in readers’ minds? It’s not a rote operation. If the passage involves allusion, metaphor, or humor, you won’t translate literally. You’ll try to find words which recreate the experience of reading the original for a member of a foreign culture. When writing spaced repetition prompts meant to invoke retrieval practice, you’re doing something similar to language translation. You’re asking: which tasks, when performed in aggregate, require lighting the bulbs which are activated when you have that idea “fully loaded” into your mind? The retrieval practice mechanism implies some core properties of effective prompts. We'll review them briefly here, and the rest of this guide will illustrate them through many examples. These properties aren't laws of nature. They're more like rules you might learn in an English class. Good writers can (and should!) strategically break the rules of grammar to produce interesting effects. But you need to have enough experience to understand why doing something different makes sense in a given context. Retrieval practice prompts should be focused. A question or answer involving too much detail will dull your concentration and stimulate incomplete retrievals, leaving some bulbs unlit. Unfocused questions also make it harder to check whether you remembered all parts of the answer and to note places where you differed. It’s usually best to focus on one detail at a time. Retrieval practice prompts should be precise about what they're asking for. Vague questions will elicit vague answers, which won’t reliably light the bulbs you’re targeting. Retrieval practice prompts should produce consistent answers, lighting the same bulbs each time you perform th