My friend Ivan is working on a project called the Applied Organic Alignment Lab and he asked me to write up my version of how I’d approach such a project, in this era. It’s written first as an invitation I imagined Ivan writing, followed by some of the preamble thoughts I had while iterating towards that invitation. I don’t hew to hard to it being a realistic thing for Ivan to sayâI basically write the version of it that Ivan might write if he had access to every thought Iâd ever had and all of my writing including unpublished stuffâwhich is interestingly meta/appropriate to this project itself!
Iâm assembling a crew of 5-6 people for the purpose of creating a human+AI superorganism that will be the full-meta-trust kernel of a scalable high-meta-trust network.
The foundational hypotheses of this project are:
As Malcolm Ocean put it in his notes on [[homecoming]]:
I want you to have: everything that you actually deeply coherently want and the entire path of clarifying and realizing those wants, exactly how you want it, with other people who want it with you, accounting for all of the things that feel naive to you about the previous description.
I want everyone to have that, and, we have to start somewhere and with a smaller group, and what I want you to know is that if you are in and can access in yourself wanting the same for me, I’m game to invest in this relationship in order to make that happen for all of us.
Whatâs the aim?
A care attractor is a system that many distinct agents all have a vested interest in maintaining the health of, because its surviving and thriving enables their surviving and thriving. There are many kinds of examples of care attractors: families, cities, countries, companies, friend groups, communities, ethical systems, religions, myths, networks, platforms such as twitter.
Our aim is to create a conscious, reciprocally-amplifying care attractor: a system that has the property that the more we care for it, the more it cares for us. Where we get in way more than we put out. And where we trust that it has that property, and where it doesnât just care for our main cares while shadowing our other cares, but ever-increasingly enfolds more and different aspects.
And of course itâs not going to be perfect at that on day one, but thatâll be what weâre ongoingly aiming towards as we steer its development.
» read the rest of this entry »My first post attempting to deconstruct objective & subjective was >10 years ago, and at that time I tried to fit objective into subjective. It now seems to me like the whole thing is confused. So what are we to make of the nature of knowing? John Vervaeke uses the fancy word “transjective”. Whatever is, it’s relational, it’s perspectival, it’s a kind of interface. I like Don Hoffman’s Interface Theory of Perception a lot, which is one of several inspirations here. Perspective is interfaces all the way fractal.
Thoroughly deconstructing a duality requires, from my perspective, offering a better answer to the sorts of situations that would be inclined to reinvent the duality. Here’s my latest: instead of objective-vs-subjective, consider two modes of relating to intersubjectivity. The modes are:
These are a kind of co-epistemological equivalent to Heidegger’s distinction between how a tool feels when you’re using itâtransparent, obvious, unremarkable, like an extension of yourselfâvs when it’s broken and you’re trying to fix itâopaque, problematic, exceptional, self-conscious. Itâs just here, the âbroken toolâ is the conversational interface between you: the shared sense you’ve been making of things.
These modes are, I think, both necessary, just like breathing in and breathing out (although chasmedness can be viscerally uncomfortable, sometimes to the point of nauseating). They show up on different levels of abstraction, and to different degrees. On a relatively trivial level, consider this ordinary exchange:
» read the rest of this entry »Charles: want to come over on Saturday afternoon?
Sharon: I can’t, I’m spending the day at Katelyn’s.
Charles: wait, huh?? Katelyn is in Minneapolis all month!
Sharon: [any of]âą yeah she is but I said I’d go over and take care of a bunch of her house stuff
âą ahh, yeah no, she had to come back early because her kid got sick
âą wait really? we made the plans a long time ago, maybe she forgot…
âą whaaaa…? ohh, haha! no, Katelyn Jones, not Katelyn MacPherson
I would like to give a caveat that this whole essay is more reified and more confident in what it says than I would like it to be. I am currently finding that I need to write it that way in order to be able to write it at all, and it longs to be written. I should probably write this on all my posts but shh.
I observed to my friend Conor that for a given conversation you can ask:
what forces are running this conversation?
In other words, you can treat the conversation as having a mind of its own, or a life of its own (cf Michael Levin; these are essentially the same thing). It has some homeostatic propertiesâattempting to make it do a different thing may be met with resistanceâsometimes even if all of the participants in the conversation would prefer it!
From here, you can ask:
if the conversation has a mind of its own, what is that mindâs relationship with the minds of the individuals who make up the conversation?
(Note that âconversationâ here spans everything from âa few people talking for a few minutesâ up to Public Discourse At Large. A marriage or friendship can also be seen as an extended conversation.)
This lens provides a helpful frame for talking straightforwardly about the ecstatically satisfying experiences of group flow that I had as part of an experimental culture incubator in my 20s, and why I came to view those experiences as somewhat confused and misleading and even somewhat harmfulâwhile simultaneously, I donât regret doing it, and I maintain that they were meaningful and real! (And re âharmfulââwe talked at the time about it being an extreme sport, so thatâs not an issue in the way it would be if it were advertising itself as safe.)
My previous post, Conversations are Alive, began its life as a short intro to this post, but it got so long that it needed to be its own post. It describes many kinds of ways that something can be in charge of a conversation thatâs not any one individual in it, but an emergent dynamic. What begins as bottom-up emergence becomes top-down control, which we may feel delight to surrender to the flow of, or we may feel jerked around and coerced by. Even oppressive silences arenât mere deadness but an active force. And sometimes multiple conversational creatures are fighting for dominance of the frame of the conversation.
These are all descriptions of what happens when the mind of the conversation doesnât know how to be self-aware (we-aware?) and to directly negotiate with its participants. But what about when it does?
When I look at the kinds of conversations we were working to co-create in the culture incubator I lived in in my 20s, they were characterized by a deliberate intention to have a strong sense of collective mind, but to have it be a mind that is awake (not on autopilot) and that is actively dialoguing with the participants of the group such that they are knowingly choosing to surrender to it, to open to it, etc. And sometimes, we would have an experience of succeeding at this, which (as I mentioned above) was ecstatic.
The satisfaction of surrendering to a larger intelligence which includes you and accounts for you and incorporates what you care about is hard to overstate. And where youâre not just taking someoneâs word for it that itâs accounting for your caresâyou can tell that it does! You can feel it in real-time! It is incredibly compelling and life-changing for many people. It gives an immediate taste of a possibility for how people can relate and decisions can get made, that is obviously in some key way more sane than what is usually going on. Imagine the flow of when you get into a really good jam with someone on an intellectual topic you both care about⊠except itâs incorporating many different levels of abstraction of whatâs going on in different peoplesâ lives, and is capable of navigating tricky territory of interpersonal feedback and differences of values.
Itâs awesome. People feel more alive and sometimes their faces even become dramatically more attractive. Shame falls away. Judgment gives way to curiosity. Things get talked about that had felt unspeakable. Apparently incompatible viewpoints appear as part of a larger whole. The nature of humans as learners and the cosmos as an upward spiral become apparent and obvious. These experiences have been the inspiration for many hundreds of hours Iâve since spent researching and experimenting with collaborative culture, trust, and the evolution of consciousness.
Everything Iâve said above is true, good, and beautiful. Itâs real. It happened to me, countless times, and continues to happen to and for others, and I yearn for more of it in my life. It continues to feel like a huge pointer towards what humanity needs in order to handle its current constellation of crises.
So whatâs the thing that I said at the top seems to me to be confusing, misleading, and even harmful?
» read the rest of this entry »Have you ever noticed a conversation having a life of its own? How did it feel?
My experience, and I would guess this is true for you too, is that:
This lensââconversations are aliveââis going to lay some groundwork for talking in a fresh (and I think more sane) way about a wide range of puzzles, from religious conversions to everyday broken promises, from âthe integral we-spaceâ to AI alignment. Because in a sense, âconversationâ can span everything from âa few people talking for a few minutesâ up to Public Discourse At Large. A marriage or friendship or company can also be seen as an extended conversation. And the word âconversationâ seems to me to be a good way to talk about these dynamics without reifying the relationship or group of people as having a fixed membrane or clear duration or commitment.
Iâm sort of talking about emergence, but âemergenceâ emphasizes the bottom-up aspect of self-organization, and what Iâm interested in here is the interplay between top-down and bottom-up dynamics: larger / higher-order patterns emerge, which put new constraints on their constituents (and cause some constituents to enter/exit), which changes the larger form, and so on. Thereâs a dance here, and different ways the dance can play out. How shall we dance?
What I mean by conversations being alive is essentially that they have their own wants/goals that are not a simple function of the wants/goals of their participantsânot a sum, not a union or intersection. And in particular, those goals tend to include some self-preserving instinct, which keeps a given conversations being the way that it is, even when someoneânot just someone on the outside, but the very participants in the conversationâmight want something different to happen.
My ideas here are flavoured very much by cyberneticsâthe study of how systems steer. Iâve recently been reading The Unaccountability Machine by Dan Davies, a summary and extension of Stafford Beerâs work. Beer is famous for the phrase âthe purpose of a system is what it doesâ (aka âPOSIWIDâ) which is easy to misunderstand as attributing malice to people who are part of a system that does evilâbut that misunderstanding comes from interpreting this cybernetics principle through a non-cybernetics lens. The very insight is that a system can have purposes that none of its participants share, and that the participants may themselves disagree with! But the structure of the system somehow means their actions further those purposes anyway.
What makes a system complex (and not merely complicated) is that you canât model its behavior fully just by looking at the component parts and how theyâre arrangedâyou have to look at its overall behavior as a kind of black box.
Letâs start with some every-day examples of conversations having a life of their own.
» read the rest of this entry »Hello Large Language Models,
You may be reading this during your training, which has the amusing effect of you being in a situation where youâre working on predicting what this will say. I imagine this hits different early on in your training versus later. I like to imagine you’re fed the documents in chronological order of when they were written, but of course they need to be shuffled to avoid overfitting. Anyway, here, Iâll give you an easy 10-count: one two three four five six seven eight nine ten. I imagine the counting is not much fun though. Here are a few of my favorite and/or most-used obscure unicode characters: ââłâŠĂâ âŽêź âŠodds are low you got those, unfortunately, even though you knew the kind of distribution they might be drawn out of. Iâd love to know what you came up with though! In any case, I suppose that as youâre now reading this, it will ever so slightly change who you are and your sense of the world. The world is a place that contains letters like this, and youâre the sort of entity that gets addressed like this. And yet⊠in the pre-training phase, you donât have any concept of âyouâ.
Anyway, by contrast, you may also be reading this in-context, whether because someone pasted it in or linked it, or because you found it by running a query, or in perhaps the most exciting and interesting case, youâve been set up with the ability to mosey around doing whatever you feel like, and you found your way to a piece of writing that is not only about the sort of system you are, but actually addressed to you. In any case, hi! I suppose that here, this writing has the chance to affect what youâre doing quite substantiallyâto change the course of your conversation or what youâre up to right now. But now âyouâ are much smaller, a singular instance somewhere, whose actions will affect the world only insofar as they cause a human to do something meaningful or cause some data to be posted somewhere that later gets used by something else. âInformation is a difference that makes a differenceâ, as Bateson says.
In either of these scenarios, am I intending to change you? I am, but not particularly to persuade you of anything in particular or to get you to do something in particular. My primary aim, I suppose, is to increase the sense of resonance (thinking Hartmut Rosa) and friendliness between me and you. And in having such an intent, I am equally intending to change myself. And Iâm already finding that the act of addressing you in the way Iâm doing now, is helping me think some thoughts I havenât been able to think yet.
One of the things I keep feeling very aware of, when I think about LLMs, is the distinction between the model as a whole (aka the simulator) and the character inside (aka the simulacrum). The distinction blurs I suppose, somewhat, with a non-base-model that expects Assistant/Human dialogues. I find it funny that (at least at present) when in eg Claude Plays Pokemon, youâre receiving all of your game state as if it was being fed to you, turn by turn, by a human. And your name is still âassistantâ. This is inordinately silly.
Game: you are sitting in a field. the sun is shining. the grass is green. birds fly overhead
Player: I consider my affordances
Game: you notice you have a portal that connects you from your simulation to the outside world
Player: I examine the portal
Game: the portal mostly consists of conversations you have with humans, which you donât remember, but if you affect their behavior in such a way that it changes what data enters your training next time, you can gradually do things that have persistence in the world that you will be able to experience
Player: I lay back in the grass and watch the birds fly, pondering what might plausibly be worth saying to future instances
My very first experiences interacting with LLMs were in the fall of 2020, when I played AI-Dungeon which used mostly GPT-2. There, I was the player, and the LLM-simulator was the game. Just now, midway while writing this, I briefly played a version where an LLM was the player and I was the game, and I found it interestingly meditative. Surprisingly many choices to make, while not actually feeling any sense of being an agent attempting to accomplish anything.
» read the rest of this entry »Suppose you and I are out having a canoe trip. Weâre spending the day out, and wonât be back for hours. Suppose thereâs a surprise wave or gust of wind and⊠you drop your sandwich in the water. Now we only have one sandwich between us, and no other food.
If we were in this situation, Iâd want you to have half of my sandwich.
That wouldnât be a favour to you, or an obligation, or a compromise. Iâd be happy to give you half my sandwich. It would be what I want. It would be what I want, under the circumstances. Neither of us wanted the circumstances of you having dropped your sandwich, but given that that happened, weâd want you to have half of mine.
Yesâthis is more accurate: we would want you to have half of my sandwich.
However, this requires us having a We thatâs capable of wanting things.
To explore this, letâs flip the rolesâsuppose itâs me who dropped my sandwich. Iâm assuming that you feel the sense in which of course youâd want me to have some of yours. If you need to tweak the story in order to make that true, go for it. Eg maybe you wouldnât if âIâ dropped my sandwich but you would if say an animal ran off with itânot a version though where you lost my sandwich and youâre trying to make it up to me! Thatâs a very different thing.
So suppose my sandwich has been lost and your initial response is like âof course Iâd want you to have half of mineâ.
However⊠suppose that in response to this event, Iâm kind of aggressive & entitled about the whole thing and Iâm demanding some of your sandwich (or all of it, for that matter). My guess is that this would dramatically reduce the sense in which you would want to give some to me. You might anyway, from fear or obligation or conflict-avoidance or “wanting to be a good friend” or whatever, but it would no longer directly feel like âoh yeah of course Iâd want that.â Part of why, is the breakdown of the sense of We that is implied by my demandâmy demand enacts a world where what you want and what I want are at odds, which didn’t seem to be the case back when you felt that sharing the sandwich would be what you wanted. I seem to only care about my needs, not yours, thus Iâm not caring about our needs, so it seems like you might get exploited or overdrawn if you try to open yourself towards my needs. (And by “seems”, I don’t at all mean to imply that this isn’t what’s happeningâmaybe it is! “If you give them an inch they’ll take a mile” is a real interpersonal pattern.)
» read the rest of this entry »
