Worked example · Understand a topic
Understand it well enough to teach it.
Make a project for the question you want answered, in the folder it's part of, and write what you know and what you don't as notes in it. Then comes the part that makes it stick: find what's missing — the questions your notes don't answer yet — answer them in your own words, and explain the whole thing back with the note closed, to a curious student who asks why. When it holds up, write it up from your own sentences. No scores, no streaks.
- Write it down
- Work it out
- Make something of it
Step 1 of 3
Write it down.
Make a project for the question you want answered — how do transformers work? — in the folder it's part of, and write what you know and what you don't as notes in it, or move them in from the Inbox.
A project for the question you want answered.
A project is something you're making from what you learn — an article, a video, a talk. It starts as a question and ends when you've made it. Make one from the + beside Folders in the sidebar — How transformers work, say — and its page opens, its name ready to type. Write what it's for in your own words: what you'd be able to explain if you understood it.
Then write what you know and what you don't, a note at a time. Add one from the project's own line on its page, or write anywhere and move the note in: from the Inbox, drag it onto the project in the sidebar, or choose Move to… on its row.
Why does attention matter
ProjectMachine learning4 notes · 2 questions open · taught back Oct 25 · draft started Oct 25
An article for people who use chatbots every day but have never read the paper: what attention does, in plain words, and why it makes long inputs expensive. It's done when it's published.
What's missing
- How does the model know which word to look at hardest?
- Does the model understand the sentence, or does it only match words?
- What did models do before attention?
Notes in this project
A thought, a question, a linkAdd- Attention reads the whole sentence at onceToday
- Draft — Why does attention matterToday
- Teaching — Why does attention matterToday
- Why long inputs cost so muchToday
Nothing to decide first.
Nothing asks what a note is, or where it stands: a line you write is a note, and so is anything you paste. A folder is for something you keep learning about — Bitcoin, AI, design. You never finish one. A project can sit in one — Machine learning, say — or in no folder at all.
Step 2 of 3
Work it out.
Ask the Guide about any note, beside it: what a line means, an example; choose Research, and it answers from what the model knows, for you to check. What's missing lists the questions your notes don't answer yet. Then teach it back with the note closed, to a curious student who asks why, and see where your notes don't back you up.
Ask about the note you're in.
Ask the Guide, beside the note, about the note you're in — or select a sentence, right-click, choose Ask Graphite and type your question to ask about just that. The Guide answers from that note and the notes it links to, cites them as links, and ends with one question back, because answering it is your turn to think.
The answer shows in the Guide, and the conversation stays there, beside the note: every question and its answer, oldest first, the earlier ones folded to a line. Keep an answer with Add to note — it goes in as the Guide's own block, marked with the Graphite mark — or make it a note of its own with New note from this. Asking writes nothing into a note: only keeping does.
Students given plain chatbot answers did better in practice and worse on the exam. A tutor with guardrails largely avoided the harm. Bastani et al., 2025
GuideThis noteWhat's missingExplain simplyResearchTeach it backQuiz meCheck itLink relatedWhere would a reader get lost?
At the scores. Attention reads the whole sentence at once says “it” looks hardest at “cat”, but nothing in your notes says how a word decides where to look — and that's where you stalled in Teaching — Why does attention matter.
Why long inputs cost so much is clear on what long inputs cost, so the second half holds up.
What could you show a reader to make the scores concrete?
Add to noteNew note from thisAsk about this note…Sends this project and its 4 notes when you ask.
Find what's missing.
Choose What's missing in the Guide or in Actions, and the Guide lists the questions a curious reader would ask that your notes don't answer yet — at most seven, each in your terms and each one sitting's work. Add to note puts them in the note under What's missing, labelled Asked by Graphite, with Undo.
Answer each one in a note of your own, in your own words. A project's page gathers every open question across its notes; tick one there, and it's ticked where it is.
Research what you don't know yet.
Choose Research in the Guide, and it answers in its conversation from what the model knows — not a search of the web: a few findings, a sentence each, then a Check: line that says what to check and where. It's asked to say plainly when it isn't sure and not to invent a source or a link, and you check what it says.
It adds nothing to the note unless you choose Add to note: then it goes in as the Guide's own block, marked with the Graphite mark, as a line in the Logbook with Undo. Check it before you build on it.
Teach it back, with the note closed.
When you think you've got it, choose Teach it back in Actions. The note goes out of sight, and Graphite plays a curious student: it asks why, asks for an example, and asks what your words mean. It never corrects you, and never gives the answer away.
Afterwards it marks what you said against your own notes — unsupported, a question, or a link to make — and each gap goes under What's missing in the note you taught. Your words are kept exactly as you wrote them, in a note of their own.
Across 28 studies, preparing to teach, and then teaching, helped people learn — more when the teaching was interactive. Kobayashi, 2019
Teach·AttentionNote closedStudent
Why does every word have to look at every other one? Couldn’t it just look at the words next to it?
You
Because what “it” means can depend on a word much earlier. Attention lets every word look back, so the model remembers what came before. That’s also why long inputs cost so much more.
Afterwards, against your notes
Unsupported
Your note says the weights are computed fresh for each input. Attention isn’t memory.
Attention
Question
How much more, and why that much?
How much more do long inputs cost, and why that much?Say how two things connect.
Link related, in the Guide beside a note, suggests other notes it could link to, and why. Nothing is added until you accept one, and one you accept is written under the note's Connects to, with its reason.
The connection that counts is the sentence you write about why: a [[link]] in your own words shows on both notes, and the Map shows how it all connects.
Students asked to explain each line of a text to themselves learned more from it than students who read it twice. Chi, de Leeuw, Chiu & LaVancher, 1994
Link related·Attention is a weighted averageLink to your notes
Every abstraction hides a cost
Its example of attention hiding a quadratic cost behind one clean operation is this note's last point.
Linked under Connects toEmbeddings
The queries, keys and values a token's meaning is averaged from are projections of them.
AcceptDismiss
Connects to
Every abstraction hides a cost — Its example of attention hiding a quadratic cost behind one clean operation is this note's last point.
No scores, no streaks.
Graphite never grades you. Teach it back and Check it mark where to look, not how you did, and Quiz me says which question you're on and tallies nothing. A project's page says how many notes it has, how many questions are open and when you last taught it back: facts about the notebook, not about you. There's no streak to keep and no dashboard of your progress.
Step 3 of 3
Make something of it.
Answer what's missing in your own words, a note at a time, and tick each question off on the project's page. When you can explain it, write it up: the Guide arranges your sentences into an outline, and you publish it as a page you own.
An outline from your own sentences.
When you can explain it, choose An outline in Actions. The Guide arranges your own sentences, word for word, in the order that explains it best, under short headings it writes, and marks with Missing what a reader would need that your notes don't say yet.
Any sentence it can't find in your notes is taken out before you see it, and the app says how many. Keep the outline in the note, under Outline, or save it as a note of its own, labelled Arranged by Graphite.
Draft — Why does attention matter
Arranged by Graphite from your notes: every sentence is yours; the headings and what's missing are Graphite's.
Reading the whole sentence
Before transformers, models read a sentence one word at a time and had to carry everything forward. Attention lets every word look at every other word at once. In "The cat sat on the mat because it was tired", the word "it" looks hardest at "cat". That's how the model works out what "it" means.
Missing: How does the model know which word to look at hardest?
What it costs
Every word is compared with every other word, so doubling the input makes four times the comparisons. That's why long documents are slow and expensive to run.
In the Logbook
Graphite arranged “Draft — Why does attention matter” from your notes.
6:12 PMUndoPublish it, or script it for a video.
Publish a page makes any note one web page, as a single HTML file you own, downloaded in your browser. Check it first points out what a stranger wouldn't follow and anything you'd rather not share, and changes nothing. Graphite doesn't host the page: you put it wherever you like.
A script for a video sets the note out as scenes — what to say and what to show — drafted by the Guide when you ask, or, with no key, a plain version with a scene for each part of the note. You record it yourself.
Publish “Attention is a weighted average” as a web page
It’s a file you own. Put it on any website — Graphite doesn’t host it.
Check it firstCopy as MarkdownPublishAttention is a weighted average.htmlAttention is a weighted average
transformers · machine-learning
Every token's new meaning is a weighted average of all the tokens' value vectors, and the weights come from how well its query matches each key. The weights aren't learned: the matrices that make the queries, keys and values are.
Grew from Attention
Not yet
What it doesn’t do yet.
- It doesn't read a book or a paper you drop in to learn from: explanations come from the model, labelled as Graphite's.
- No spaced repetition: it doesn't schedule reviews.
- No sync between devices: your phone's browser keeps its own notebook.
- Research doesn't search the web or read your sources: it answers from what the model knows, in the Guide, for you to check.
- Graphite doesn't host what you publish: a note becomes a file, and you put it on a website yourself.
- A script for a video is a script: you record it yourself.
Try the method with a topic of your own: the Feynman technique with AI guide walks through an explanation, a gap and a second attempt.