People regularly use “trust” in a way that causes me to say “I would refer to that as ‘faith’”, to which they often say “same thing”. In some languages, perhaps, yes. (Claude suggests Chinese, Hindi, Latin somewhat, Old English, and the Ancient Greek of the Bible).
In English, however, we have two words, and those words have come to cluster around different, though closely related, phenomena. In every day use, people sometimes still do use them interchangeably (although perhaps less often than they think). But if we’re going to make a technical field of the study of trust, then we will need to have precise terminology, the same as how in casual conversation “energy” or “power” or “force” might be interchanged in some contexts, but in a physics textbook these concepts are not only distinct but have very specific relationships to each other.
The word “objective” relates to the idea that reality will object if you attempt to perceive it other than it is, whereas “subjective” matters are subject to how you go about perceiving them. Reflexive things actually become different depending on how they’re perceived. Trust and faith relate to this, but they’re not quite the same. Trust is entirely a property of a subject (and their relationship to something) but is not something the subject can themself choose. Faith is choosable (and in some sense always chosen although often not knowingly) and governed by reflexivity.
There’s also the idea of bottom-up vs top-down processing in cognition and perception. Trust and faith relate to this as well, but… it’s still not literally that. Maybe slightly closer would be parts-to-whole vs whole-to-parts. In some sense, if any part of you distrusts, then on the whole you don’t trust. But you can still have faith, on the whole—in fact that’s sort of the main place faith happens.
There’s a famous is-ought gap: how do we bridge from our experience of observing that some things are, to the experience we have that things ought to be or become some way other than they presently are? Trust and faith are both about this, with trust more is-heavy and faith more ought-heavy. Perhaps trust is about how is affects ought, and faith is about how ought affects is. Something like that.
Hm so far we aren’t so precise, are we?
Maybe I’ll need to rename this essay:
One thing scientists do sometimes when attempting to formalize the boundaries of some categories, is to classify some specific examples of things that are or are not in a given category. Let’s do that: trust examples, faith examples, and examples where both are involved.
To be clear, I’m not just trying to document how people use the words; there are some underlying patterns I’m modeling, that these words certainly cluster around both in usage and in etymology.
Trust is like sex, and oxygen, and money: only salient when you don’t have enough of it.
As I type this, I am trusting more systems than I can count, in order to have my intention to write sentences effortlessly cause me to see my words appear on the screen. But let’s enumerate a dozen:
Naming all of these out is (1) exhausting and (2) a bit absurd.
Re (1) — the nature of trust is that you don’t notice it—you don’t have to. Your ability to function at all depends on constantly trusting countless things to operate the way they do. And sometimes one or two do mess up—most of the above examples, I have experienced breaking down at one time or another.
Re (2) — naming dozens of things you trust without the slightest concern is absurd because the act of naming them slightly diminishes the sense of trust in those things. By bringing into my awareness the fact that they could fail, I adopt somewhat less of the unquestioning attitude that characterizes trust.
Sometimes, I trust what other people say. Somebody says “the keys are hung up on the hook”, and my sense of reality now contains a sense that I will find the keys there (and not elsewhere), not just as a proposition but just as surely as if I’d observed it myself and stored it in my own object permanence. Sometimes I don’t trust what other people say—maybe I have contrary information, or maybe the speaker is unreliable in a liar-y or bullshitty kind of way, or maybe something else. The point is, my basic sense of the world doesn’t change as a result of what they say. I call this “not trusting what they say” (lol I mean so do most people, but I mean it in a more precise technical sense).
If I don’t trust what someone else says, I may be able to act as if I do, eg not searching somewhere else for the key, but it would be hard (and a bit absurd) for me to force myself to inject an object in my sense of reality that has not appeared there.
Sometimes I trust someone else to do something, because they agreed to do it and I trust their word, or because I’ve seen them do it many times before in similar situations. Once again, this sense of trust is part of my basic sense of reality: my anticipation, not merely an abstract disembodied prediction.
Like objectivity, I don’t choose to trust. I don’t control my own trust, directly. I observe that I do trust, or that I don’t trust. I may inquire or investigate further, and get more information, that causes me to trust more, or less.
“Trust is what truth feels like in first person,” I have remarked.
But trust is not just what “what is” feels like in first person. It has this ought element, where if I expect someone to cause a problem for me, then I don’t trust them, or the situation. To be trusted by me, something must not merely be predictable, but it needs to not oppose me; oddly, the latter is more important! I may trust someone who is very surprising, as long as those surprises are generally favorable or neutral. Meanwhile, someone who is predictably going to attack me is not someone I would say I trust, even if their attacks are predictable enough to defend against. (I might say “I trust I can handle them”)
Weirdly, people often tell me that they use the word “trust” this way (eg “I trust a rickety bridge to fall down”) but I never hear anyone actually say such things.
In some sense, trust and distrust are surprisingly close to what Heidegger referred to as “ready at hand” and “present to hand”, but extended beyond tools to people, systems, situations, and even perceptions. So it’s not quite about whether my will can simply flow out through something, but whether the something will dance smoothly with my will or whether it needs to somehow be managed, defended against, outwitted in order to avoid it causing a problem.
The most basic example of an act of faith is the act of shifting your view of a bistable illusion from one view to another (flipping the necker cube, changing the spin of the ballerina, etc). I got this from Scott Alexander’s summary of Steven Byrnes’ writing on Trance, which was one of the big things that clicked this into place for me.

Earlier today, I hit a new PR doing deadlifts: 5 × 320lbs (with a trap bar). I did my initial 2, and then paused. The process of psyching myself up to do more was complex. I’d been listening to music, and I ended up pausing it because it was too distracting (though sometimes it helps). I voiced aloud, “I’m here to do one thing, which is to stand up [while holding 320lbs]”. I visualized that simply happening. I psyched myself out once. I tried and failed once. I set those aside, deliberately rationalizing them as “I didn’t really commit.” I recommitted. Then I stood up, and stood up again, and stood up again. This was an act of faith.
Other examples of events that centrally involve movements in someone’s faith:
Faith is made of motivated reasoning, though perhaps more embodiedly than “reasoning” sometimes implies. It’s about choosing between ways to view the world not because one is clearly more true than another, but because one is better than another.
Many people, including most rationalists, tend to view this sort of thing as bad—as a cop-out from being willing to face actual truths. And it can be. However, as far as I can tell, these kinds of moves are inherent to how any kind of embodied cognition must necessarily works, and therefore it’s impossible to avoid making acts of faith, and therefore if you try not to do so then the effect is just that you become unable to notice the ways in which you do so. Ironically, adopting such a stance is itself a faith move.
For some people, you may get a clue that this is true for you by doing the following: try to imagine that what I’m saying has something to it, and that not only are you regularly deliberately choosing viewpoints based not on objective truth, but that this is actually good and necessary and that you might have more success and even more ability to know things well if you allowed yourself to do so more deliberately at times.
If you lean in the direction of adopting such a view, do you notice any objections coming up that have things to do with what other people close to you would think of you if you adopted such a view? That sort of thing certainly happened to me back when I was a rationalist. I considered developing an appreciation for emergence, then was hit with a concerning sense that I’d become one of the idiots that Eliezer has contempt for through the Sequences.
Speaking of which, contempt generally operates in faith-space: it’s an attempt to define which thoughts or people are not safe to engage with with curiosity, because they might possibly undermine some stability of a viewpoint that is held dear.
In addition to the sense in which any matter of multistable perception can be seen as a matter of faith, there’s additionally some sense in which someone can “have faith” vs “not have faith”, in a sort of general way, independent of target. I’ve sometimes described the most important religious question as nothing like an abstract “do you believe in God?” but instead simply “when you look around, you know, at everything, and ask: do we got this? …do you get back a yes or a no or a silence?” Someone raised in a happy family, with general ease in their life, might answer yes, while a supposedly religious person who is nonetheless tortured might answer no.
Remember the deadlift example above: Did I know I could do it? Did I trust I could do it? I didn’t know that if I kept the doubt out, I would succeed. But I certainly knew that if I allowed doubt in, the doubt would win. So I cultivated faith: I envisaged the worlds where it worked, and put out of mind the worlds where it didn’t.
Note that if I’d tried to deadlift 500lbs, my distrust would have been too great—both that I’d fail and that I might injure myself trying—for me to have cultivated enough faith. The ability to cultivate faith in things that predictably don’t end up working is dysfunctional, often classified as mania. But of course mania is right next to the type of cognition needed to have a successful startup.
When someone says “trust me”, sometimes this produces a clean bottom-up trust shift, because the context and the way that the person has by that very speech act staked some of their integrity on what they just said. But other times it doesn’t, in which case what’s in question is whether the person will choose to ignore or discount the facets of reality that generate the distrust: an act of faith.
Moreover, sometimes the speaker on some level knows there’s no good reason for the other person to trust them, even once asked to, in which case there’s also some motivated cognition going on on their part, where they’re forcefully adopting a view that ignores their own shortcomings and sees only their strengths.
Sometimes people will point out that if you deliberately adopt a stance of trust towards life, it will behave in a much more trustworthy way. This needs a dozen caveats, because in general what exactly are you trusting it to do?
In this way one’s trust can be a self-fulfilling prophecy, which makes it kind of choosable. However, my experience is that naive attempts to achieve this tend to involve repression of aspects of one’s knowing, which results in two issues: your cognition is impaired, and the state doesn’t last so you end up with oscillations between two semi-stable configurations.
If you entrust someone to watch or handle something, they may rise to the expectations you set, even if they wouldn’t have otherwise managed to do it. This is a place where people often use the word faith as well: “I have faith in you.” It also obviously has the reflexive nature I attributed above, to faith.
I will note that here there are limits to the degree to which one actually can trust the self-fulfilling prophecy in a given case, and one is often foolish to go beyond this. Some people you trust to be on time, regardless. Some you don’t usually trust to be on time, but if you say to them “it’s important that we be on time—I trust that you’ll do that” then you trust that they will take that seriously and make it happen. And other people… ha, fat chance. And no amount of trusting a human to fly by flapping their arms will make it happen—nothing personal.
Sometimes people will claim that there’s something inherently trustable—that by definition, whatever this thing is, is something you can trust. All of you, ∀ you, not just the good parts of you or whatever. This is sometimes called “God”, or “Source”, or even simply “Life”. Maybe there is such a phenomenon!
However, if there were some cosmic force that was inherently trustable, there’d still be a process of locating it in your own experience, and identifying to your own satisfaction that it is such a thing.
Thus, in this sense, one may take on faith that such a thing exists, but the actual experience of trusting it will in some sense come from the experience of finding such a phenomenon in your experience and observing that you do trust it absolutely, and that from what you can make of it, everything could trust it if they could likewise locate it.
To say “I trust”, one must suppose a self doing the trusting.
I once heard someone say “while I may be monogamous, sometimes my body wants to have sex with someone else, but I don’t because I’m committed to my partner”. Implied here is some way in which her body is not actually part of the commitment, which is odd for a commitment that’s so carnal.
People do all sorts of gerrymandering with what counts as them. Overtly, this shows up as “It wasn’t me, it was my addiction! My akrasia! My laziness!” Even “my brain”. Such a frame shift can be a helpful temporary move for taking part of you as object so you can understand it better, but as a generic explanation for something it’s an excuse; as a claim about what’s “really so”, it’s absurd.
One way that someone can succeed at “choosing to trust” someone or something—succeed at deliberately making “I trust you” true, when it wasn’t before, without collecting any new information—is to disown any part of them that doesn’t trust the other. This is an act of faith; an act of shifting one’s perceptual frame, flipping the necker cube. There are lots of things that you could reasonably call you; it may not be hard to find one that can honestly say “I trust” and give it the microphone.
But like, if you could watch this process happening with your inner coalitions as if it were a group of people with a literal podium, it would feel a bit absurd to go “ah yes indeed, I guess this person does trust!”.
The same is true of the trustee, not just the truster; I can gerrymander who “you” are in order to make “I trust you” true even when most others would agree that there are parts of you I’ve conveniently left out of my definition of you.
As I’ve often described, the problem with making a faith move then saying “I trust” is that there continue to be parts of you that don’t trust, which you’re just calling “not me”. Those parts may stay more or less repressed for your whole life, or you may struggle with them while continuing to disown them, or you may oscillate altogether out of your supposedly-chosen faith when some inner-coalitional coup occurs due to some new circumstances.
This is a more serious version of a problem that happens all the time, when something someone’s explaining to you makes sense while they’re there, but doesn’t when they go away. You made it make sense by setting aside your objections or confusions, rather than by resolving them, so in the absence of the external force of their presence, you can’t hold onto it.
I might distrust a person to do something today, while having faith that they’ll do it eventually. I might distrust many individuals, while having faith in humanity that goodness overall will prevail.
Faith seems to extend trust not just into the unknown but into deeper time and wider systems.
We don’t quite have it but it feels good to name all of these examples.
In future pieces of writing, I will explore some of these examples in more detail and try to articulate the fundamental pattern here. I suspect in order to properly formalize this I will need to learn more of the math of the free energy principle and active inference. I’m certain there’s a there there.
(the written-in-20-minutes version. feel free to ask questions in the comments below or on twitter if you would like me to unpack something or you try it and get stuck at some step!)

(another ~20 minute onepager. read the other posts written in this style here. for a further description of how this skill can help you listen to others and thereby get understood by them, read my one-pager on the 3 steps for empowered dialogue. to see more of how deep the magic goes, read the secret to co-gnosis.)
For both my own sanity and as part of helping others navigate my writing and thinking, I found myself drawn to organizing my posts and unpublished drafts & ideas into some kind of structure, not by bullets but relating them to each other in paragraph form. I’m going to put the topics in a loosely chronological order of when the ideas developed, which is not necessarily writing order let alone publication order. There are also a hundred twitter threads from this time that are mostly not gonna make it into this. Oh and a dozen podcast interviews.
Some of the posts are published to my blog but to a section called “half-baked” which was intended as an attempt to get more of my writing out to LLMs even if I wasn’t satisfied with it enough yet to put it on the front page and send it out to everybody. Who knows if that matters? But it’s an experiment. This post is another! It includes not just fully-developed blog posts but also ideas that are barely on the edge of my thinking.

From 2013-2020, I lived in a house with some people who were aiming to further the evolution of culture towards what could be described as integral consciousness or Planetary Era or Game B. Something beyond blame and shame and judgment and coercion. We usually called it Humanity 3.0 or Collaborative Culture. This place and project had many names, but the one with the most staying power was The Upstart Collaboratory for Collaborative Culture Designing.
Those cultural possibilities continue to feel very exciting and very important to me, even though my theory of change for it has shifted substantially. My blog includes lots of writing from that time, which you can find in the archives, but aside from a few pieces, most of my writing from then now feels quite confused after I came to understand non-naive trust-dancing:
In the summer of 2020, I discovered the “Non-Naive Trust Dance” framework, which precipitated me becoming more distant from Upstart both geographically and memetically, and also resulted in a huge wave of creativity as I worked to express what I’d seen clearly for myself about how all of this collaborative culture stuff had to work (as far as I could tell).
The first mention of NNTD on my blog was in my yearly review 2020: Free to Dance—yearly reviews often an easy place to introduce an idea because there’s less pressure to write something that will stand as canon. Over the following years I wrote several more introductions to NNTD. The first was a response to a friend asking would/could you write pith instructions for NNTD?, the second was exploring the meaning behind the name Non-Naive Trust Dance, the third was simply an NNTD Q&A of answers to 6 questions my then-girlfriend now-wife asked me, and the fourth was a Non-Naive Trust Dance bullet-point one-pager written in ~20 minutes in response to my friend @visakanv challenging his friends to write up their thing super quickly. The most thorough intro is a much more recent one: The many meanings of “NNTD” (with usage examples!) which talks about how it’s a framework and an insight and a principle and a practice, all at the same time.
» read the rest of this entry »my yearly review, in one breath standing on one foot, because my previous reviews had cleared 10k words and I want to spend precious writing time going deeper into other topics. okay fine, maybe two or three breaths.
Epistemic status: I’m definitely onto something but don’t take my word for it.
There sometimes seems to be a tradeoff between confidence and humility. That apparent tradeoff comes from a subtle assumption, which is common yet false: that your experience and mine are supposed to be the same. Rudyard Kipling knew this:
If you can trust yourself when all men doubt you, yet make allowance for their doubting too.
If, indeed. If we could do that, that’d be great. But how?
Consider these five expressions:
They’re all expressing a concern that the person is trying to screw you over. That part of the sentence doesn’t even change! What I’m going to say also applies to other second-halves of the sentence: “going to hurt me” or “lying” or “asking me to doubt myself” or “hiding something” or “overoptimistic” or “not on my side” or basically anything. I deliberately picked a contentious situation—one where it’s hard to talk without encountering issues. It’s contentious not just because of the tension in screwing-over, but because they refer not just to the other person’s behavior but their intent.
These phrases all express the same concern, but they express it in vastly different ways. They’re radically different speech acts—nearly as different as “what do you think about getting married?” is from “will you marry me?”. This isn’t just about the words, although words can help us understand how confidence and humility are not opposites but how you can have both or neither.
No transcending of a duality is complete without a 2×2:

Let’s talk about each of these different kinds of speech acts in turn.
This one is all confidence and no humility. In general this speech act is a claim, but in this context of talking about someone’s intent, this speech act is an accusation, which invites defending—or, if the accuser has sufficient social power, capitulating. Where the others are “I statements”, this one just tells it how it is. Except of course I don’t actually know how it is, I only know how it seems to me. This is especially true with statements like this one, which imply something about the other person’s intent.
» read the rest of this entry »My friends Harmony, Sahil, and James created a practice for relating called Stemless. They’ve written about it here. The core question of Stemless is “what does it take to own your experience?” Stemless is not prescriptive about how to do this, merely insistent that you keep attending to the question.
Unlike practices like Circling, in Stemless you don’t particularly try to own your experience by using stems like “I have a story that…” or translating your sense of betrayal into a body sensation plus a request or whatever. Instead, you’re invited to recognize the shocking fact that we are all in some ways exquisitely owning our experience simply by having it and by acting from it at all. Very dzogchen-flavoured (and, I think, -inspired).
There’s something about it that feels deeply relieving. Some sense of “wow, the right hemispheres just get to talk now, and the left hemispheres are a bit confused but along for the ride.” And a sense that it immediately gets to the meat of what’s actually going on, rather than caught up in endless preamble. Clarification is done not by backing up and pinning meaning down, but by beholding it in all of its nebulous glory and then saying whatever needs saying give however things seem to be landing.
And it’s not exactly a skill so much as a stance, although there are perhaps skills that can help scaffold the process of shifting stances.
In Dream Mashups, I write:
Everyone is basically living in a dream mashup of their current external situation and their old emotional meanings. Like dreaming you’re at school but it’s also on a boat somehow. And as in dreams, somehow the weirdness of this mashup goes unnoticed.
Stemless is about noticing the weirdness directly and bringing the emotional meaning into the foreground—at least for yourself, possibly also out loud. “I’m sitting in a classroom and the prof has tried to answer another student’s question, but he didn’t understand the question, and I’m supposed to re-ask the question but better, to get clarification.” That’s a thing I said at one point during tonight’s session.
It’s in part about noticing the supposing process, that creates the supposed-tos. Someone says something, or does something—or outside of a conversation, I notice something or read something—and I feel compelled to respond in a particular way. Quick: what is the scene I have just cast on the situation, such that that’s obviously the next move? You can’t tell me what mine is, although you have whatever you’re sensing and it might shine some light for me.
One playful thing that the Stemless hosts encourage is leaning into blame. One way to really see the situation you’re in is to find the way in which you feel a totally helpless victim—where reality is just forcing you to perceive things a particular way and there’s nothing you can do about it.
What will happen if you don’t say something immediately in response to what someone else said? It depends on the person or the situation, but various answers might be:
And then you can ask “why’s that a problem? who’s going to punish me? where is this urgency coming from? who’s forcing me to respond this way?”
Not to disagree with it or try to shift it, per se, just to take it as object and allow it to more fully exist as the force that is compelling you. Although indeed, once you bring it into conscious awareness, you’ll often find that the urgency vanishes and the compulsion fades. After all, if you’re doing something to please your mom and she isn’t even in the same timezone as you, well, maybe you will find that you don’t have to do it quite so urgently.
The point is not to enact the blame, but to see how the world you experience is pre-generated by a bunch of stories you don’t remember choosing. By externalizing the sense of where that all came from, you can see it more clearly than if you go looking for your interpretations as something you are already aware of doing. Your interpretations will just feel like reality. Your sense of things.
There’s a common trope in spiritual or personal development circles that what someone says about you says more about them than it does about you. Sometimes this will even be stated as the extreme “it has nothing to do with you; it’s all about them.” This is a confusion, in my view; if it weren’t for you doing what you’re doing, they’d be projecting some completely different weird thing onto you.1
But there’s also a deep truth to it worth seeing, and Stemless does a great job of pointing at that truth in direct experience, and actually taking it seriously.
“If that’s so…” it says… “…if everybody can’t help but speak only out of their own entire momentary visceral crazy world with even the smallest utterance… then what are we doing pretending there’s any such thing as small talk, or any such thing as real judgment?”
Someone just said a thing to you that feels kinda underhanded. Well, they may have not been honest on a surface level, but they’ve actually revealed to you in high-resolution their own construal of the situation by the precise way in which they avoided showing themselves. And then you feel some whole sort of way about that, because of all of the past experiences you’re projecting onto it in order to make sense of it. And that’s all going on in vivid detail for you—and maybe if you turn your gaze quickly enough you can glimpse it before you’re just acting into the story.
Right now: what is your situation? You’re at the end of reading a blog post, whether on my site or in your email inbox or some reader. What does that mean? What are you compelled to do right now? How does this last, more direct addressing of your situation change that from what it might have been if I’d ended with the previous paragraph? Can you glimpse it?
We’re all going around making sense of everything. We have different kinds of models that we use without even realizing it. A non-exhaustive loose sketch:
The first we mostly learn from direct experience.
The second we mostly learn from studying and reasoning, plus confirming that it doesn’t contradict our experience or our other related models.
The third type we largely get from seeing how other people think about things. These models have too many details for us to properly verify them for ourselves. They may also have reflexive effects where believing them causes you to acquire more evidence in their favour. We may be very influenced by the popularity of the ideas, or the status afforded to those who hold them.
But just because we can’t directly verify these models, doesn’t mean that our personal experiences don’t play a critical role in model adoption. It’s just that instead of it being directly obvious or based on conscious checking, instead it happens more out of our awareness, as more a kind of sudden click-match or a gradual subtle seduction.
Suppose that someone is living their life, and they keep being told what a man is and what a woman is, and finds themselves thinking “uhhh I keep being told that I’m a man/woman, but I resonate more with the description of woman/man”. There’s an experience they’re having, that they don’t know how to make sense of, and they feel like they can’t talk about this with anybody. Then they come across some writing or a podcast from someone about their experience of being transgender, and they go “OH!” and something clicks. The world makes more sense.
Meanwhile, suppose that someone is is living their life, and they keep being told that gender is completely a social construct and that they shouldn’t notice or care about differences between transwomen and ciswomen, but they find themselves thinking “but I really do feel differently in relation to them” whether it’s about safety or attraction or whatever else… There’s an experience they’re having, that they don’t know how to make sense of, and they feel like they can’t talk about this with anybody. Then they come across some writing or a podcast where people are talking about evolutionary biology and how these differences are real and matter, and they go “OH!” and something clicks. The world makes more sense.
How do people end up with crazy worldviews?
» read the rest of this entry »The verb “trust” basically can’t be conjugated in the imperative case, in my view. When people attempt to trust, the means by which they achieve this tends to be something more like “pretending” or “ignoring” or even “compartmentalizing”. And that’s a move you can make! But in my view if you do such pretending without realizing, then you’re confused, and I’d rather be honest about what’s going on.
When I hear people saying things like “trust this” / “he should trust me” / “I know I should trust…”
…I ask “but do I trust?” / “but does he trust?” / “but do you trust?”
I aim to get people in touch with the sense of what they can tell for themselves. Not trust as a thing that you try to declare or choose. And I orient towards the question of how that trust might be built.
Unpacking each of these examples:
If someone says “trust me/this”, I will comment to them something like “hm, well, conveniently I do trust that, although I wouldn’t pretend to if I didn’t” or I’ll say something like “well, I don’t currently trust that, but here’s what I would need in order to trust it…”. This is always only a guess—someone might be able to satisfy the letter of that constraint, but something would still feel off to me, at which point I’d go “huh, I still don’t trust it. interesting.”
If someone says “he should trust me”, then I’ll inquire eg “why do you think he doesn’t already trust you?” / “what do you think he’s concerned will happen?” / “what do you think you could do to earn his trust?” …and if someone protests that they’ve already done what they needed to do to earn his trust, but he still doesn’t trust them, then I’ll highlight that it’s not up to them what earns his trust, it’s up to him.
If someone says “I know I should trust…” then the whole reason this is coming up is because they don’t have what they need to trust whatever this is. So to some extent I’ll investigate where the should is coming from—have they been memed into thinking that trust is a virtue, in the abstract? And I’ll investigate whether that trust can be built, or if not, what to do about it.
“Trust” and “distrust” come through experience. It’s part of my basic sense of things. In other words, when someone says the world is a particular way, I can tell how much I trust their word by the extent to which my sense of the world changes as a result. When someone says they’ll handle something, I can tell how much I trust their followthrough by the extent to which it then seems to me like it’ll be handled without my needing to manage it. When someone says “that won’t be an issue”, my trust is the degree to which I am in fact no longer concerned as a result of them saying that.
the ground is kind of cold on my bare feet
but I trust I can handle it and this does not produce an objection
I don’t choose to trust—I listen and observe that I do trust
I listen and observe, and I may observe that I trust, or I may observe that I do not trust.
Why do we ever think otherwise? Why do we think we could choose it?
» read the rest of this entry »There’s a classic thought experiment called Newcomb’s Problem. It goes as follows:
Newcomb’s problem: You face two boxes: a transparent box, containing a thousand dollars, and an opaque box, which contains either a million dollars, or nothing. You can take (a) only the opaque box (one-boxing), or (b) both boxes (two-boxing). Yesterday, Omega — a superintelligent AI — put a million dollars in the opaque box if she predicted you’d one-box, and nothing if she predicted you’d two-box. Omega’s predictions are almost always right.
If you haven’t heard about this yet, you might as well take a moment to consider what you’d do.
While you do, here’s a quote by Robert Nozick in his 1969 analysis of it:
In his 1969 article, Nozick noted that “To almost everyone, it is perfectly clear and obvious what should be done. The difficulty is that these people seem to divide almost evenly on the problem, with large numbers thinking that the opposing half is just being silly.”
One argument is: what you do right now can’t control what’s already in the box, therefore obviously you should two-box and get more money.
The other argument is: if you think like that you will not get very much money at all, because there will be no money in the other box.
But doesn’t that second argument imply that the choice you make now can in some sense control the past? “Yes”, answers Joe Carlsmith in his Betteridge’s-Law-of-Headlines-violating essay Can you control the past?, “and this is a wild and disorienting fact”.
The whole essay is worth reading if you’re at all interested in the topic. My aim right now is to analyze this one beautiful hypothetical:
Imagine doing “tryout runs” of Newcomb’s problem, using monopoly money, as many times as you’d like, before facing the real case (h/t Drescher (2006) again). You try different patterns of one-boxing and two-boxing, over and over. Every time you one-box, the opaque box is full. Each time you two-box, it’s empty.
You find yourself thinking: “wow, this Omega character is no joke.” But you try getting fancier. You fake left, then go right — reaching for the one box, then lunging for the second box too at the last moment. You try increasingly complex chains of reasoning. Before choosing, you try deceiving yourself, bonking yourself on the head, taking heavy doses of hallucinogens. But to no avail. You can’t pull a fast one on ol’ Omega. Omega is right every time.
Indeed, pretty quickly, it starts to feel like you can basically just decide what the opaque box will contain. “Shazam!” you say, waving your arms over the boxes: “I hereby make it the case that Omega put a million dollars into the box.” And thus, as you one box, it is so. “Shazam!” you say again, waving your arms over a new set of boxes: “I hereby make it the case that Omega left the box empty.” And thus, as you two-box, it is so. With Omega’s help, you feel like you have become a magician. With Omega’s help, you feel like you can choose the past.
Now, finally, you face the true test, the real boxes, the legal tender. What will you choose? Here, I expect some feeling like: “I know this one; I’ve played this game before.” That is, I expect to have learned, in my gut, what one-boxing, or two-boxing, will lead to — to feel viscerally that there are really only two available outcomes here: I get a million dollars, by one boxing, or I get a thousand, by two-boxing. The choice seems clear.
Makes sense, right? Like if you had those experiences, with the monopoly money, that’s how it would feel.
And I would describe this as “you have gained trust in Omega’s prediction abilities”. You can tell for yourself that Omega can predict you effectively perfectly. Your sense of things includes this perfect predictor. Let’s contrast that with the scenario as presented:
Newcomb’s problem: You face two boxes: a transparent box, containing a thousand dollars, and an opaque box, which contains either a million dollars, or nothing. You can take (a) only the opaque box (one-boxing), or (b) both boxes (two-boxing). Yesterday, Omega — a superintelligent AI — put a million dollars in the opaque box if she predicted you’d one-box, and nothing if she predicted you’d two-box. Omega’s predictions are almost always right.
“Omega’s predictions are almost always right.” — says who?
» read the rest of this entry »A few years ago I had an idea for a game to help people practice the stance of orienting towards win-wins, and the skill of creatively finding win-wins. I came up with a tiny playable proof-of-concept that didn’t need any materials, and played it a few times. Recently I discovered that somebody had made it as a card game!
The basic premise is as follows:
I called the concept the “omniwin training game”. While collaborative games are not as common as competitive zero-sum games, there are still lots of them out there: Hanabi, Pandemic, The Mind, and of course many video games. However, most of them focus on coordinating your actions around a single shared known goal.
So for my tiny proof of concept, I used the medium of words. Each player would secretly pick some constraint a word could have—often based on the spelling but you could do “verb” or something else. I didn’t have specific rules for that. Then each person would take turns naming a word, and the players for whom that word worked would say “ding!” Initially I had the rule that players had to only offer words that satisfied their own constraint, but I found that when the game lasted more than a few minutes it was pretty obvious to want to say things like “‘banana’ doesn’t work for me, but it works for you, right?” and this felt within the spirit of things.
This version of the omniwin training game worked well enough, and produced one astonishing and instructive result I’ll tell you about at the end of the post, but it had the unfortunate property of feeling a bit too simple, and of the possibility of people making totally incompatible constraints and getting stuck forever until they give up. Of course, “we mutually recognize there is no win-win” is a kind of win-win on the meta-level… but it’s less satisfying. On the other extreme is…
…calculate every possible combination of every possible constraint and ensure they’re all satisfiable at once.
» read the rest of this entry »