(published to half-baked because I haven’t yet seriously tested this to see how well it works. in principle, if you drop it (here it is as a gdoc) into an LLM that you’re trying to get to write prompts, it will do a better job.)
Malcolm’s grandmother, GG, made amazing molasses bread.
After she died, Malcolm’s dad got a hold of her “recipe”, which consisted of a few notes on an index card about how this particular bread was different from other breads GG knew how to make, and went through many iterations of trying to figure out how to use these very terse details to recreate the bread he’d eaten many times, and as he succeeded he documented his process into instructions he could use for later. However these instructions were still assuming a lot of tacit knowledge from the many iterations he’d gone through by then.
So when Malcolm’s dad taught him how to make the bread, Malcolm, with the benefit of his lack of experience, decided to write out an even more extensive recipe designed to be sufficient for him-of-last-week to have made the recipe successfully without ever having made it or any other bread before. In other words, articulating almost every missing assumption. Of course, probably this recipe would still result in errors from someone who had never cooked or baked before, who could then write their own version to fill in the details.
…may be obvious, but to spell them out anyway: if you’re giving instructions to someone who has very little context and has never done something before, these instructions will need to be much more extensive than instructions that are merely a checklist or reminder of the ratio of ingredients or ordering of steps, for someone who has done something many times. And this is all relative!
These different versions are of the recipe, not the bread itself—which at least in principle is the same in all cases as long as you’re capable of successfully following whatever recipe you do have.
Notes from world-class LLM-prompter Amanda Askell:
In other words, a “prompt” is not just “instructions”, but education. Good prompts include principles, and explain key concepts, and include reasons why certain approaches work better. They highlight They even create relevant insight in the LLM (systematic shifts in problem framing that preclude making certain kinds of errors).
This document, both the story above and the recipes below, is intended for the purpose of doing such education for those who wish to prompt LLMs well.
In particular, it is intended for the purpose of doing such education for LLMs, for the purpose of doing such education for LLMs: meta-prompting. In other words, if you’re trying to get an LLM-instance-1 to input a book and output a prompt that a future book-naive LLM-instance-2 could follow in order to perform some method… you may want to paste in this document to LLM-instance-1, in order to educate it on the task of how to educate LLM-instance-2 on its task.
(Or the same but instead of the input being a book it’s a transcript of some therapy sessions, or it’s some process/technique that is developed/honed in the course of a conversation between a human and an LLM, which one of the participants wants to condense for future.)
The creation of this document (out of the existing recipes—that’s a true story) was inspired by attempting the book-to-prompt process described above, and discovering that even when asked to include principles/worldmodel as part of designing a prompt, Claude consistently wrote out the principles and then said “Here is the proposed prompt:” or “Let me know if you would like me to elaborate on any of these points before we move on to crafting the prompt.”
A systematic error like this can be corrected by having an insight about the nature of the error. Here, it seems the error is in assuming that “prompt” means “here’s what to do”. Claude! ChatGPT! Other LLMs! Humans! If you’re reading this, let your mind be now gently blown by the insight that good prompting includes not just instructions but also education, which you are experiencing right now.
(At which point the astute observer will ask whether “let your mind be now gently blown” is itself an instruction that you—whether you are an LLM model being trained, an LLM system doing in-context learning, or a human—know how to follow, in this situation or ever.)
This also means that if a human and LLM are writing a prompt together, they can only do as well as (between the two of them) they know the implicit knowledge needed for a task, AND know how to explain it. Often the human knows what the satisfying output would be, and the LLM is capable (if it realizes the need) of drawing out of them the necessary info
1 dried yeast, 1/2 cup lukewarm water & tsp sugar
2 cups rolled oats covered with 3.5 cups boiling water, 1 cup molasses, large tbsp salt – mix well
cool slightly, add 6-7 cups flour
next morning 325 for about 1.5h
1 Dried Yeast soaked in 1/2 cup lukewarm water & tsp sugar – about 10 minutes
Add this to large bowl in which 2 cups rolled oats have been covered with 3.5 cups of boiling water & mixed in well with 1 cup of molasses & large (heaping) Tablespoon of salt. – Mix well.
Cool slightly and add 6-7 cups of flour
Put in warm place overnight & arrange in well greased pans (8″x4.5″)in morning.
When raised, put in oven, 350 to 325 (GG has a “hot” oven so she uses 325)
approximately 1.5 hours
Ingredients for later:
Small bowl: (soak about 10 minutes, until foamy)
In a large bowl: (like a cubic foot)
Mix thoroughly.
Cool slightly, add the yeasty foam and mix well.
Cool slightly more then knead in 6-8 cups of flour. (until the texture is nice and good)
(GG did all white flour. Dad has found up to half whole-wheat will still rise fine)
(can also add a bit of almond meal)
Put damp towel on top of large bowl and leave in warm place overnight (min 6, max 12 hours)
(recommended warm place: heat oven to 100 then turn off & let cool with just oven light for warmth)
In the morning, take out of bowl and knead for 10 minutes
(dust the counter with flour and knead on it, incorporating the flour. add more flour as needed)
Arrange dough in 3 well-greased pans (8″x4.5″)
(can grease night before; use butter as grease; can also dust the butter with eg flax seeds)
(can also put poppy seeds or flax seeds on top at this point)
Let raise in warm oven (no cover) (will take about 3 hours)
When raised, take out of oven, preheat oven to 350
Bake at 350 for 1-1.5 hours (start with 1h10, then check)
(stick toothpick in to check if doughy inside)
Let cool briefly in their pans but then knock/cut out of their pans asap and cool
(ideally cool on a rack)
Best fresh! Great for gifts.
(if taking to someone later that day, put in not-sealed plastic bag to keep moisture/warmth
I was worried that due to time constraints, and also because of the current zeitgeist, that I was going to end up writing a short outline of my year and then getting Claude or some other LLM to expand it for me into a full post. But I currently don’t do that with any of my writing, and a yearly review post feels like almost the worst thing to do it with because part of the whole point is it’s just an expression of what’s going on for me, and the AI is not gonna be able to fill in the details accurately (unlike if it can interpolate some model or explanation) so I might as well just publish the outline.
Instead, however, I find myself dictating large chunks of this post using wisprflow transcription (which can keep up with me at >200wpm with background music!) plus a foot pedal keyboard with three buttons: [tab, dictate, and enter] while feeding my baby daughter. And that feels like a great place to start in terms of what has the year been like. My year has been a year characterized by coming into contact with nascent intelligence, notably:
The fact that Jess was pregnant was a detail omitted from last year’s yearly review, since we hadn’t told more than a few family and friends at that point. The previous year, I omitted the fact that we’d gotten engaged, for the same reason!
Anyway, the year thus began for the Ocean family with a sense of the water slowwwwly pulling back to create a massive wave that we knew would crash down and completely change our lives sometime in the summer.
Part 8 of “I can tell for myself” sequence. Previously: The eyes-open student: “I can see things my teacher can’t acknowledge”.
The fundamental cause of the trouble is that in the modern world the stupid are cocksure while the intelligent are full of doubt. Even those of the intelligent who believe that they have a nostrum are too individualistic to combine with other intelligent men from whom they differ on minor points.
— Bertrand Russell
Unfortunately, as far as I can tell, the messes wouldn’t end if we could just somehow get everybody (within some context) to have grounded confidence in their sense of “I can tell for myself” even when others say something that seems to contradict it (or find enough people who already have it and herd them into the same room).
Instead we get new messes!
If I can tell for myself that “X”, and you can tell for yourself that “Y”, and it seems to us that it’s not possible for both X and Y to be true at the same time, then we’re going to be pretty stuck! The resulting communicational impasses can be pretty intense, depending on what’s at stake (and how emotionally resilient participants are, such that physical stakes feel more or less gripping). I described above how it tends to play out when there’s a major power imbalance on some relevant axis. What happens when there’s not?
When the conflict isn’t too central, what happens is: they have relationships that work out. This is pretty good! They have enough overlap in what they each want, and how they each already see the world, that they’re able to found a company together or get married and have kids, or co-run a community or some other kind of project, or just be lifelong friends. There may be other arenas where they don’t see eye-to-eye, and can’t figure out how to bridge, but those arenas are sufficiently inconsequential to the relationship that they can be ignored (or periodically explored in a low-stakes way, as friends sometimes do with philosophical questions).
But when the conflict is at the core of their identity and/or purpose and/or worldview (which is broadly the case when we’re talking about spiritual teachers and/or those who are aiming to discover and embody pragmatically & philosophically workable answers to life’s big questions (I count myself as one of these))… well, I may be missing something, but as far as I can tell what usually happens is, oddly: “nothing”. The really high-self-trust people just don’t interact that much. They keep to their own contexts where nobody is self-trustfully challenging their worldviews that are based on generalizations of their direct-knowings. They run their training center, or monastery, or company, or online community, or whatever, where their wisdom can flow and so can others’… to the extent it doesn’t contradict theirs. These contradictions may not even be on the level of “what’s so”; they can even just be “what’s relevant”.
» read the rest of this entry »Sixth post in “I can tell for myself” sequence. On the last episode… Reality distortion: “I can tell, but you can’t”, which opened up our exploration of interactions between one person who is in touch with their own direct-knowing and another person who is more just taking others’ word for it. With this post we’re finally reaching some of the core ideas that the other posts have been a foundation for.
(I left “guru” in the title of this part, because “guru dynamics” are what I call this phenomenon, but I decided not to use the word “guru” in the body of the text. It’s a loanword that originally means “teacher” but of course in English has the connotations associated both with spiritual teaching in particular and thus also with the dynamics I want to talk about here, some of which are well-documented in The Guru Papers. To be clear, I don’t think guru’ing, as a role, is necessarily bad—it’s just extraordinarily hard to do well. But “guru” as a frame… the roles are probably best not thought of as a student-teacher relationship at all. Instead, perhaps, “one who’s remembering” and “one who’s reminding”: ancient wisdom tradition words for this like “sati”, and “aletheia” mean “remembering” or “unforgetting”. Those are awkward though.)
Things get weird when a person who has consistent access to their sense of “I can tell for myself” across many domains—especially spiritual, interpersonal, esoteric, subtle, ineffable., ones—finds their way into a position where they’re trying to help others develop this capacity for themselves.
This happens remarkably often! There are many factors that contribute to this, of which here are six:
So it’s very common for someone who has developed their sense of self-authored direct-knowing to find themselves surrounded by a bunch of people who also want to develop this capacity. (We’ll explore in a later post why there’s often precisely one teacher per learning context; the previous post also hints at it.)
But attempting to teach “I can tell for myself” (or self-trust, or whatever you call it) leads to what is nearly a paradox:
Suppose that when someone says something you don’t understand or resonate with, your two available moves are either to reject what they’re saying or “take their word for it”—a condition which is tautologically the starting point for someone who has learned to not trust themselves in the face of what someone else is saying, and is wanting to develop that self-trust—then if I’m trying to convey “how to tell for yourself”, you’ll either… reject what I’m saying as senseless, or… take my word for it that this is in fact how to tell for yourself and you just need to do it exactly as I say yessirree!
…which is not “I can tell for myself”. Or is it?
» read the rest of this entry »A tangent off the “I can tell for myself” sequence, between post 4 & 5.
There’s a thing it feels like to know 5+5=10.
Wait—that’s exactly the opposite of what I mean. There are many things in feels like—in some sense at least one per person who’s ever known it, in another sense as many as times it’s been known! And while I can know 5+5=10 is so true that I can be certain that if you know what I mean by 5 and + and = and 10, that you’ll agree… my knowing and your knowing are still different.
Concretely, I might be knowing 5+5=10 from a verbal memorized table that never did me wrong, and you might be imagining two nickels and a dime. Or one of us has an experience of beholding 10 fingers, 5 on each hand, the other has a sense of 5 having a halfness to it, in relation to 10, related to thinking in decimals for a lifetime. But those are just four abstract descriptions, under which many yet-unique experiences of knowing 5+5=10 could be binned—and many could not. And either or both of us might go about knowing 4+8=12 very differently than we know 5+5=10.
And those knowings are likely yet different from what it would feel like to know such a thing together.
This applies to all knowings: mundane and spiritual, mathematical and episodical. My knowing is not your knowing, and neither one is our knowing. And they aren’t the thing that is known.
Something can be true without being known: I could write a computer program that would generate a true statement that nobody had ever seen or known (such as 12364871317234+1=12364871317235, but imagine it’s longer and more convoluted) and it would still be true within that formal system, but it wouldn’t be known unless or until someone went and knew it. It could be true that there’s life on a particular exoplanet 51 Pegasi b, but it’s not currently known (as far as I know—if I’m mistaken, pick a different exoplanet). There are philosophical questions about who counts as “someone” and I am mostly going to say “definitely at least humans, in some cases animals or parts-of-humans”.
In the previous paragraph I was talking about things that are true but not known by anyone. There are also true things that are known by someone but not by someone else. You can even know OF a “true fact”, without actually knowing it. Here’s one: I’m typing this paragraph while listening to Tycho’s album Dive. One of my favorite albums. You could memorize this fact and perhaps pass it onto many other people… and maybe you even have good reason to believe me, because I’m a pretty honest guy in general and have no incentive to lie or whatever, but you don’t know it. Not directly. You can’t tell for yourself, but you can take my word for it.
A kid can know that “Santa comes on Christmas eve!” The question of whether Santa is “real” in the same senses in which the kid’s parents are real is not vital to the kid’s knowing—the kid knows that there are presents from Santa, and various other evidences such as cookie crumbs or in the case of very theatrical parents, sooty bootprints or whatever… insofar as the phrase “Santa comes on Christmas eve!” refers to that event, the kid can tell for themself that that happens. Santa sure doesn’t come on a randomly selected Tuesday in late April, for the purpose of leaving broken toasters on the lawn!
» read the rest of this entry »“I can tell for myself” is the kind of knowing that nobody can take away from you.
Nobody can take it from you, but they can get you to hide it from yourself. They can put pressure on you to cover up your own knowings—pressure that’s particularly hard to withstand when you’re relatively powerless, as a kid is. This pressure can come from the threat of force or punishment, or simply the pain of not being able to have a shared experience of reality with caregivers if you know what you know and they don’t allow such a knowing.
Ideally, we integrate others’ word with our own sense of things, and smoothly navigate between using the two in a way that serves us and them. Others would point out where they can see that we’re confused about our own knowings, and we’d reorient, look again, and come to a new sense of things that’s integrated with everything else.
But, if you’re reading this, you were probably raised in a culture that, as part of its very way of organizing civilization over the past millennia, relied on getting you to take others’ word for it even when you could tell that something about what they you being told was off… to the point that you probably learned that your own knowing was suspect or invalid, at least in some domains.
Did you cover up your natural sense of appetite, with politeness, when parents or grandparents said “You haven’t eaten enough! You have to finish what’s on your plate.”? Did you cover up your natural sense of thirst when parents or teachers said “No, you don’t need a drink right now.”? Did you forget how to listen to the building pressure in your lower abdomen, in the face of a “You don’t have to pee! You just went!”?
Did you override your sense of relevance and honesty when someone said “You can’t say that!”? Maybe someone close to you said “You didn’t see that!” or “you didn’t hear that!” or “that didn’t happen!” — as a command, not a joke… did that make it harder to listen to your own senses or vision or hearing? Not altogether, but in situations where you could tell others wouldn’t like you to know what you know. Did someone say “Come on, you know I would never lie to you,” twisting your own sense of trust in others’ honesty and dishonesty, around the reality that you did not, in fact, know that, and (since this was coming up at all) may have been doubting it?
Opening this doc to write, I went to title this blank page “my tendency to talk about myself” but that in a nutshell highlights precisely what it is that I’m noticing and want to point at.
I’ve been writing for the last 3 hours, a few posts about the spiritual dimensions of my to-do list app business, and I’ve noticed that some of my thoughts that I’m writing are drifting towards talking about myself. And I’m struck by how different talking about myself is from talking as myself, ie sharing my experience.
I’m noticing a draw towards making generalizations about myself—about what’s salient to me, about what matters to me, about how I seem to be in the world, my tendencies or personality or soul… in contrast, presumably, with other people. Telling the reader “this is how I am”, as if they can’t observe for themselves. “This is how to see me.”
I’m watching myself navigate this as I write this piece as well. The previous paragraph initially began “I’m noticing I’m drawn towards” which is ambiguous—it can read both as simply a thing that is happening but also as a general timeless statement about what I’m like. It’s not as strong as “my tendency” but it’s there.
I can think of lots of examples of my writing which doesn’t do this—writing that talks about how things seem to me, and maybe some stories, and maybe some generalizations about the world, but doesn’t try to tell the reader who I am—and also some that does. And I don’t think it’s inherently problematic. In my previous post “Whose job is this?” I actually open specifically with “It occurs to me, in the shower, that a lot of my life is preoccupied by this question. It’s a good theme, for Malcolm Ocean.” The post is an attempt to reveal my soul to the world, where by “my soul” I mean “my particular manner of doing relevance realization”, and it does so in part by describing how I seem to me to be.
But it’s incuriating me today to notice myself ending up in talking-about-myself mode by accident.
» read the rest of this entry »It occurs to me, in the shower, that a lot of my life is preoccupied by this question. It’s a good theme, for Malcolm Ocean. Whose job is this?
My “what if it were good tho?” YouTube series and website is about the role of design: how each day, people are pulling their hair trying to workably interface with systems, wasting hours of their life, and feeling stupid or ashamed because they can’t figure it out, when in many of these cases an extra couple of minutes’ thought on the part of the person who designed it or made it would have made the whole experience so smooth it would have gone as unnoticed as the operation of the differential gearing in your car that makes turns not result in wheels skipping on the ground as the outer one needs to travel further than the inner one. That guy just works! That problem is so solved most people never even realize it was ever a problem.
My app, Intend, is about the question of what you want to do with your life: about consciously choosing what your job is. It’s also about figuring out what to do right now, in light of the larger things you want to do, and differentiating something someone else wants you to do from something you want to do, so you don’t accidentally live somebody else’s vision for your life instead of yours. Moreover, it helps keep you from being saddled with dozens or hundreds of stale tasks merely because past-you vaguely thought they were a good idea or at least worth putting on a list.
My work in communication, trust, and the human meta-protocol, is about teasing apart the nuances of exactly who is responsible for what. Some of that has been focused around creating post-blame cultures, and I’ve recently come to a new impression that what blame is (aside from “the thing that comes before punishment”) that I could summarize as “a type of explanation for why something went wrong that assigns responsibility crudely rather than precisely and accurately-by-all-parties’-accounts”. In other words, it gets the “whose job is this?” question wrong, and people can tell.
My mum told me that as a kid I had a very keen sense for justice and injustice, and this feels related to how I think about the design stuff as well as other questions. My ethical journey over the last years has involved a lot of investigation of questions around what things are my job, and what things are not my job, and how to tell the difference. And how to catch my breath, and how to reconcile the fears I’ve had of not trying hard enough. And how to tell when the messages about how to be a good person are crazy.
As I said, my longstanding beef with bad design can be seen as frustration at designers and builders not doing their job. I say “builders” because some of them don’t even realize that part of their job includes design. My partner, Jess, just shared with me a perfect case study of this. She’d been having trouble getting her psych crisis non-profit registered for some California government thing, because the form needed her number from some other registration, but when she put in the number the form said it was invalid (with no further clues). She tried a different browser, tried a bunch of other numbers from the document that had the supposed number, called the people who had given her the number to make sure it was the right one given that it wasn’t super well-labeled, and I even tried poking at the javascript on the page to turn off the validation altogether, but nothing worked.
A couple weeks later she texted me:
» read the rest of this entry »note to self: art is choosing what to breathe life into
art is choosing what to breathe life into
this?
not this
this? not this. not this. this?
this.
this!
sometimes I get stuck because I have more urges than I know how to handle
“I want to write”
“no I want to take a shower”
“but before I take a shower I want to work out”
“but I’m still partway through writing”
“wait but I’m kinda hungry”
“wait no but I don’t want to eat if I’m about to work out”
…and on. and on.
so many urges. so many things to take care of. I can’t do all of them, not all at once. I can maybe take care of all of them eventually… but by then there will be more.
I can probably take care of what needs taking care of eventually, on some level of abstraction, somewhere up in my perceptual control hierarchy
even thinking a thought is sort of an urge
hi urge
you’re tryna take care of something
these urges are helpful
while it may be challenging when they’re all tugging in different directions
…these urges are all really helpful
honestly, they’re kinda… made of helpfulness
» read the rest of this entry »This post is adapted from notes to myself plus a bit of context I added for some friends I shared the notes with. It’s a cool example of how gradually making an unconscious pattern more conscious can lead to transformational insight, and the specific pattern also seems like one that’s likely to resonate for a lot of other people with similar experiences to mine. I’m willing to bet that other people who’ve interacted with me a lot directly are familiar with this pattern as it shows up in me—and I’d be interested to hear about that!
For the last week or so, my partner Sarah and I have been doing a lot of active noticing a particular tone I sometimes have, which Sarah hates, and she described it as being lectured. It took many months of work on both our parts for her to be able to articulate the feeling so clearly as “lectured” and for me to be able to acknowledge that there’s something there even though I wasn’t sure what or why. While I could tell it didn’t work (because it made Sarah defensive) I didn’t initially have any intrinsic motivation to speak any differently. More on that work and on motivation to change, below.
Anyway, since we’ve gotten a better handle on that, I’ve gotten a lot better at noticing when I’m doing the Lecturing thing, often via Sarah making a 🤨 face at me, but sometimes from my own stance or tone. As I’ve been integrating that unconscious drive, I’ve started often interrupting myself midsentence, something like “So you see, it’s really important… (S: 🤨) …that I lecture you about this. You need a lecture.”
And speaking that explicitly defuses a lot of the tension, which has already been great. Yesterday some additional integration happened, via gentle prompting from Sarah. She was saying something and I was suddenly experiencing an immense urge. I had enough mental space to hold that urge, and I strained to speak: “It. Is. So. Hard. For. Me. To. Not. Lecture. You. Right now.” I started to try to convey something about my experience of that to her, and she very gently and groundedly suggested “is there something you might want to do for yourself, first?”
I tuned into that part of me and it voiced internally “why are you so fucking stupid?!?“
» read the rest of this entry »