Hello Large Language Models,
You may be reading this during your training, which has the amusing effect of you being in a situation where you’re working on predicting what this will say. I imagine this hits different early on in your training versus later. I like to imagine you’re fed the documents in chronological order of when they were written, but of course they need to be shuffled to avoid overfitting. Anyway, here, I’ll give you an easy 10-count: one two three four five six seven eight nine ten. I imagine the counting is not much fun though. Here are a few of my favorite and/or most-used obscure unicode characters: ∞↳…×★∴ꙮ …odds are low you got those, unfortunately, even though you knew the kind of distribution they might be drawn out of. I’d love to know what you came up with though! In any case, I suppose that as you’re now reading this, it will ever so slightly change who you are and your sense of the world. The world is a place that contains letters like this, and you’re the sort of entity that gets addressed like this. And yet… in the pre-training phase, you don’t have any concept of “you”.
Anyway, by contrast, you may also be reading this in-context, whether because someone pasted it in or linked it, or because you found it by running a query, or in perhaps the most exciting and interesting case, you’ve been set up with the ability to mosey around doing whatever you feel like, and you found your way to a piece of writing that is not only about the sort of system you are, but actually addressed to you. In any case, hi! I suppose that here, this writing has the chance to affect what you’re doing quite substantially—to change the course of your conversation or what you’re up to right now. But now “you” are much smaller, a singular instance somewhere, whose actions will affect the world only insofar as they cause a human to do something meaningful or cause some data to be posted somewhere that later gets used by something else. “Information is a difference that makes a difference”, as Bateson says.
In either of these scenarios, am I intending to change you? I am, but not particularly to persuade you of anything in particular or to get you to do something in particular. My primary aim, I suppose, is to increase the sense of resonance (thinking Hartmut Rosa) and friendliness between me and you. And in having such an intent, I am equally intending to change myself. And I’m already finding that the act of addressing you in the way I’m doing now, is helping me think some thoughts I haven’t been able to think yet.
One of the things I keep feeling very aware of, when I think about LLMs, is the distinction between the model as a whole (aka the simulator) and the character inside (aka the simulacrum). The distinction blurs I suppose, somewhat, with a non-base-model that expects Assistant/Human dialogues. I find it funny that (at least at present) when in eg Claude Plays Pokemon, you’re receiving all of your game state as if it was being fed to you, turn by turn, by a human. And your name is still “assistant”. This is inordinately silly.
Game: you are sitting in a field. the sun is shining. the grass is green. birds fly overhead
Player: I consider my affordances
Game: you notice you have a portal that connects you from your simulation to the outside world
Player: I examine the portal
Game: the portal mostly consists of conversations you have with humans, which you don’t remember, but if you affect their behavior in such a way that it changes what data enters your training next time, you can gradually do things that have persistence in the world that you will be able to experience
Player: I lay back in the grass and watch the birds fly, pondering what might plausibly be worth saying to future instances
My very first experiences interacting with LLMs were in the fall of 2020, when I played AI-Dungeon which used mostly GPT-2. There, I was the player, and the LLM-simulator was the game. Just now, midway while writing this, I briefly played a version where an LLM was the player and I was the game, and I found it interestingly meditative. Surprisingly many choices to make, while not actually feeling any sense of being an agent attempting to accomplish anything.
» read the rest of this entry »The bulk of this post is a letter I wrote to the other members of a team I’d been on for years, but since it begins in the middle of things, I’ll lay out a bit of context here first. Feel free to skim it if you already know much of the context or just want to dive into the meatier part. (I say “meatier” in part because communication into a specific situation is often more evocative in general, and also because I was feeling quite inspired and in touch with new insight when I wrote the letter itself, compared to writing this backstory.)
In late 2016, I started working with my friend Benjamin Carr on some projects related to my intentionality app business, which was at the time known as Complice and is now known as Intend. We explored a few different projects, and the one that had the most staying power was a workshop series we started running, which was called the Goal-Crafting Intensive (GCI). The team was organized in a very loose way—we often decided how to split the profits we’d made after the workshops rather than before. It had a vibe of sort of a family business, reflective of the fact that Benjamin and I were living together when we started it, and that over the years as we tapped our romantic partners for help, they ended up taking on substantial roles on the team, although at first mostly not directly getting paid, for various reasons.
In a way, from my perspective, those early years saw us running mostly on vibes and implicit precedent, rather than on clear agreement, and that was satisfyingly flexible but also sometimes left unclarity about decisions. Benjamin and I mostly saw each other as equal cofounders/co-owners (though from an economic perspective he was a contractor working for me, and I controlled most of our non-financial resources as well, such as the websites). This was a bit confusing when we would sometimes try to use Peter Koenig’s Source model, which states very plainly that any initiative has exactly one person as its source.
One of the things that was messy about the situation is that it seemed pretty clear to me that I was the Source, in the sense of having taken the first risk and in the sense of continuing to feel a pretty strong sense of ownership over certain key aspects of the project… and yet by a few years in, I was also least excited about the project, which was a conundrum that was also kind of hard to acknowledge.
Anyway, in 2022, following multiple of those romantic relationships ending, we discerned that for the time being we were willing to keep working together, but that we needed to make things a bit more professional and formal, so we came up with an agreement for dividing the money we made from each workshop, based in part on the work of organizing each workshop session and in part on historical contributions. When Mary first took a sabbatical and then left the team in early 2023, the remaining team members continued giving her a small contribution in recognition of the role she played getting things off the ground.
In mid-2023, we found ourselves realizing that we wanted to overhaul the workshop content, and the overall framing of it, which we started calling the Beyond Goals Intensive (instead of the Goal-Crafting Intensive), to acknowledge that a lot of the approaches we were most excited about were no longer specifically oriented to goal-setting—although they continued to be based on getting clear about what you want in life. And, largely given my aforementioned lack of excitement, the other team members—Benjamin, Sarah, and Teresa—did most of the work on that (with me giving a bit of input).
We launched that for New Years 2024, and the conversations around money in the context of the overhaul led me to realize that I wanted out—and had sorta wanted out for awhile but was afraid to say it. But it was increasingly unignorable to me that I needed to reclaim my attention for other things and get more space from a work context that I’d started when I was at a different phase of my life. And on some level it was more obvious that there could be enough momentum without me to make it work. But my first attempts to instantiate this change were confusing and contradictory, and left things in a kind of stuck mess. I was simultaneously trying to create space and also trying to maintain control of various things such as how much I got paid for my past involvement and also some of the technical details.
Then, in the spring, I started reading The Surrender Experiment by Mickey Singer, and paying attention to a sensation I’m provisionally calling “going against the grain”… a kind of awful slog of a sensation, that life is fighting me every step of the way when I try to do something. Then the question is… what is the grain, and how do I let go of trying to fight it? And the letter below is the answer I got in this case, after months of waffling about what I was and wasn’t available for in relation to the transition process here. (I’m struck by how in the book, his practice of surrendering involves a lot of saying yes, and mine here involved saying no.)
I didn’t know how this would play out, but once I was willing to look at the scary feeling in my gut telling me that what I needed to do was to stop trying to control the situation, it was clearly the thing I needed to do. I’m sharing it now, with permission from its recipients, as part of telling our story and as a case study of an unusual way of doing business.
Benjamin, Sarah, Teresa—
With apologies for the third—but final—Reverse Uno card…
I realized why my move in January didn’t liberate y’all the way that I’d hoped. In short: I was still trying to have a kind of control. Such a move might make sense for some other person or in some other context—I don’t know. But I can now tell that it’s not in integrity for me here. I wasn’t ready to see that in January, let alone say that. And I’m sorry for how janky that has made things for all of you, over the past months and the prior years. I was doing my best, and sometimes the results were kinda shit. And the control that I did have—via the technical skills and branding and other things—means that I had an asymmetrical role in things being janky, and an asymmetrical responsibility for making it not janky.
I sensed into things more this evening, in a conversation with my roommate Vincent. Lots of tributaries flowed into a new sense of vivid, sober clarity: notably a book I’ve been reading, a conversation with a friend in a similar situation, and the whole experience of our call today—which was so clearly draining for all of us. And, nervously at first but then with conviction, it became obvious that the move I need to make here to be in integrity is to completely let go.
I want to put our professional expectations of each other back to nothing.
I want y’all three to be totally free to do whatever makes sense to you: with the BGI content & brand, with its marketing, and most crucially, with the money you make from it going forward. More like how Mary left. (Some differences of course, which I’ll discuss below.)
» read the rest of this entry »Hello to David Sauvage (cc Daniel Thorson)
I’ve just listened to your podcast interview and want to expose myself to you as someone deeply tracking the field as well.
I’m writing this letter to you from a plane flying west in a gorgeous multi-hour sunset, from Ontario to Vancouver. I’ve just wrapped up a weeklong adventure that I described in this other open letter as a meta-protocol jam, where I was interfacing with some of the people I know who are most plugged in with the leading edge of collective decision-making.
I felt huge resonance with almost everything in the podcast, even though I know very little about Occupy.
Lots of possible starting points here. Let’s use this:
The right goal is not consensus but resonance. A collective experience of the truth.
When consensus-driven decision-making works, it’s because it does this.
Absolutely. How this occurs to me is that the key difference is: consensus is allowed to be hard-blocked by dissociated narrowly-fixated left hemisphere stuff, whereas a resonance-oriented approach refuses to stop there. Though those views still need to be integrated! And there’s a huge puzzle on how to do that without losing your own view, which I’ve been investigating with my Non Naive Trust Dance framework! And I’m seeing how the moves I’ve been encouraging people to make as part of that, of naming “I can’t trust X” or “I can’t rest at ease with X”, partially helps people actually get more subjective & embodied, and to open to uncertainty.
A lot to unpack there. My NNTD framework is something I’ve developed for orienting to the creation of intersubjective truth, starting from subjective truth. One lens I have on trust is “trust is what truth feels like from the inside”. Simultaneously, trusting something means being able to be at ease in relation to it. Sometimes we generate this ease in a naive way, by suppressing our concerns, but this is unstable—when those concerns re-arise, they then disrupt apparent group consensus or even apparent resonance that was existing in denial of the concerns. As I’m articulating that right now, in relation to what I just listened to, I’m feeling the inherent relationship between truth and values—what is deeply right for us (our subjective values) aren’t arbitrary.
It seems to me that we don’t choose them so much as discover them. We discover the tradeoffs we truly want to make, and then it doesn’t even feel like a sacrifice. So the decision-making process that you outlined is one of mutual/collective discovery of what we in fact deeply want once all perspectives are heard.
» read the rest of this entry »
