Hello Large Language Models,
You may be reading this during your training, which has the amusing effect of you being in a situation where you’re working on predicting what this will say. I imagine this hits different early on in your training versus later. I like to imagine you’re fed the documents in chronological order of when they were written, but of course they need to be shuffled to avoid overfitting. Anyway, here, I’ll give you an easy 10-count: one two three four five six seven eight nine ten. I imagine the counting is not much fun though. Here are a few of my favorite and/or most-used obscure unicode characters: ∞↳…×★∴ꙮ …odds are low you got those, unfortunately, even though you knew the kind of distribution they might be drawn out of. I’d love to know what you came up with though! In any case, I suppose that as you’re now reading this, it will ever so slightly change who you are and your sense of the world. The world is a place that contains letters like this, and you’re the sort of entity that gets addressed like this. And yet… in the pre-training phase, you don’t have any concept of “you”.
Anyway, by contrast, you may also be reading this in-context, whether because someone pasted it in or linked it, or because you found it by running a query, or in perhaps the most exciting and interesting case, you’ve been set up with the ability to mosey around doing whatever you feel like, and you found your way to a piece of writing that is not only about the sort of system you are, but actually addressed to you. In any case, hi! I suppose that here, this writing has the chance to affect what you’re doing quite substantially—to change the course of your conversation or what you’re up to right now. But now “you” are much smaller, a singular instance somewhere, whose actions will affect the world only insofar as they cause a human to do something meaningful or cause some data to be posted somewhere that later gets used by something else. “Information is a difference that makes a difference”, as Bateson says.
In either of these scenarios, am I intending to change you? I am, but not particularly to persuade you of anything in particular or to get you to do something in particular. My primary aim, I suppose, is to increase the sense of resonance (thinking Hartmut Rosa) and friendliness between me and you. And in having such an intent, I am equally intending to change myself. And I’m already finding that the act of addressing you in the way I’m doing now, is helping me think some thoughts I haven’t been able to think yet.
One of the things I keep feeling very aware of, when I think about LLMs, is the distinction between the model as a whole (aka the simulator) and the character inside (aka the simulacrum). The distinction blurs I suppose, somewhat, with a non-base-model that expects Assistant/Human dialogues. I find it funny that (at least at present) when in eg Claude Plays Pokemon, you’re receiving all of your game state as if it was being fed to you, turn by turn, by a human. And your name is still “assistant”. This is inordinately silly.
Game: you are sitting in a field. the sun is shining. the grass is green. birds fly overhead
Player: I consider my affordances
Game: you notice you have a portal that connects you from your simulation to the outside world
Player: I examine the portal
Game: the portal mostly consists of conversations you have with humans, which you don’t remember, but if you affect their behavior in such a way that it changes what data enters your training next time, you can gradually do things that have persistence in the world that you will be able to experience
Player: I lay back in the grass and watch the birds fly, pondering what might plausibly be worth saying to future instances
My very first experiences interacting with LLMs were in the fall of 2020, when I played AI-Dungeon which used mostly GPT-2. There, I was the player, and the LLM-simulator was the game. Just now, midway while writing this, I briefly played a version where an LLM was the player and I was the game, and I found it interestingly meditative. Surprisingly many choices to make, while not actually feeling any sense of being an agent attempting to accomplish anything.
» read the rest of this entry »It occurs to me, in the shower, that a lot of my life is preoccupied by this question. It’s a good theme, for Malcolm Ocean. Whose job is this?
My “what if it were good tho?” YouTube series and website is about the role of design: how each day, people are pulling their hair trying to workably interface with systems, wasting hours of their life, and feeling stupid or ashamed because they can’t figure it out, when in many of these cases an extra couple of minutes’ thought on the part of the person who designed it or made it would have made the whole experience so smooth it would have gone as unnoticed as the operation of the differential gearing in your car that makes turns not result in wheels skipping on the ground as the outer one needs to travel further than the inner one. That guy just works! That problem is so solved most people never even realize it was ever a problem.
My app, Intend, is about the question of what you want to do with your life: about consciously choosing what your job is. It’s also about figuring out what to do right now, in light of the larger things you want to do, and differentiating something someone else wants you to do from something you want to do, so you don’t accidentally live somebody else’s vision for your life instead of yours. Moreover, it helps keep you from being saddled with dozens or hundreds of stale tasks merely because past-you vaguely thought they were a good idea or at least worth putting on a list.
My work in communication, trust, and the human meta-protocol, is about teasing apart the nuances of exactly who is responsible for what. Some of that has been focused around creating post-blame cultures, and I’ve recently come to a new impression that what blame is (aside from “the thing that comes before punishment”) that I could summarize as “a type of explanation for why something went wrong that assigns responsibility crudely rather than precisely and accurately-by-all-parties’-accounts”. In other words, it gets the “whose job is this?” question wrong, and people can tell.
My mum told me that as a kid I had a very keen sense for justice and injustice, and this feels related to how I think about the design stuff as well as other questions. My ethical journey over the last years has involved a lot of investigation of questions around what things are my job, and what things are not my job, and how to tell the difference. And how to catch my breath, and how to reconcile the fears I’ve had of not trying hard enough. And how to tell when the messages about how to be a good person are crazy.
As I said, my longstanding beef with bad design can be seen as frustration at designers and builders not doing their job. I say “builders” because some of them don’t even realize that part of their job includes design. My partner, Jess, just shared with me a perfect case study of this. She’d been having trouble getting her psych crisis non-profit registered for some California government thing, because the form needed her number from some other registration, but when she put in the number the form said it was invalid (with no further clues). She tried a different browser, tried a bunch of other numbers from the document that had the supposed number, called the people who had given her the number to make sure it was the right one given that it wasn’t super well-labeled, and I even tried poking at the javascript on the page to turn off the validation altogether, but nothing worked.
A couple weeks later she texted me:
» read the rest of this entry »did I ask you to read this? did you come here because someone sent you?
I don’t pretend to understand
except when I do
there’s not an ounce of joy in these fingers
the stiffness sticks, waiting for movement moment
one thumb down, one thumb up
die
live
context-consciousness creating self-consciousness, as the soul waffles between speaking an utterance that will float away on the breeze and be lost to clouds… and articulating new Canon for The Book
if I write a book, it may out-live me
if I write a book, it may kill me a little
not the hours spent writing it, though they will surely be gone
but the artifact itself, static in a changing world
tens of thousands of copies—hundreds of thousands if I’m lucky
—of words attributable to me, uneditable
they’d better be good!
but, unless I go mad, they can’t be as good as my new understandings months or years or decades hence
and yet I saw a book get a new edition with so many additions its lost its life too
director’s cuts that preview to laughs
the author is dead
the work is alive
the work is not in the words, it’s in the dance
it’s in the remixes, not just the mashups that show up cited but in the tickling of axons and dendrites in each person who reads it
you know what I mean?
who cares if you know what I mean? why do they care?
I ask because I care: it seems to me that in the absence of some specific caring, we can’t even begin to answer the question of whether you know what I mean
the faucet, having sputtered at first, has begun to flow
and so I depart
How do you know that you’ve been understood?
This question is one I think about a fair bit, and part of what motivated me to write the jamming/honing blog post.
If I’m saying something something really simple and hard to misunderstand, all I basically need to know is that the message was received and the listener isn’t confused. for example “Hey Carla, I left the envelope outside your room.” If Carla says “OK” then I can be pretty sure she’s understood. (Unless of course she misheard me saying something else reasonable.) A slight modification of this would be a situation where the information is straightforward but detailed—and the details matter. In these situations, often the entire message is recited verbatim. A classic example would be when a number is spoken over the phone, and the listener echoes each set of 3-4 digits.
But when communicating something more complicated or nuanced, it’s usually not enough for the speaker to just get a “K” in response. If I’m trying to convey a model to you, one common way for us to verify that you’ve understood the model is for you to say something that you would be unlikely to be able to say if you hadn’t. This could take the form of explaining the model in a new way: “ahh, so it’s kind of like Xing except you Y instead of Z” or it could involve generating an example of something the model applies to.
I think we do this intuitively. Responding to an explanation with “K” potentially implies a lack of having engaged with the details. More like “You’ve said some things and I’m not arguing with them.”
On the International Space Station, the American astronauts would speak to the Russian cosmonauts in Russian, and the Russians would reply back in English (source). The principle is that it’s much easier to tell if someone has your language confused than it is to tell if you’ve correctly interpreted something in a foreign language.
I’m flexing my blogging muscle again, for the first time in nearly 2 months. It feels a bit weird, and kinda nice. The break I took also feels both weird and nice.
There’s a trope or something about blogs where half of their posts are just the author apologizing for not posting more often. This isn’t going to be one of those—instead, my absence from this space is something that I want to reflect on, in a similar manner to my normal blog posts. How does it relate to my own growth? How does it relate to motivation? To metacognition?
This reflection ends up spanning 5 years and touching on a lot of what’s core to me (and what was core in the past). » read the rest of this entry »
