Title: Mysterious Answers to Mysterious Questions
Tags: lesswrong mysteriousanswers selfhelp

Q: What are "floating" beliefs?
A: beliefs that are only connected to each other, that don't predict things about what will happen
Making Beliefs Pay Rent (in Anticipated Experiences)

Q: What is the rationalist value of empiricism?
A: constantly asking which experiences our beliefs predict--or better yet, prohibit
Making Beliefs Pay Rent (in Anticipated Experiences)

Q: What do we call beliefs that aren't falsifiable?
A: "floating" beliefs
Making Beliefs Pay Rent (in Anticipated Experiences)

Q: What should you keep in mind when arguing a factual question?
A: which difference of anticipation you are arguing about
Making Beliefs Pay Rent (in Anticipated Experiences)

Q: What are you doing if you can't find a difference in anticipation?
(two possibilities)
A: you're probably arguing about labels in your belief network--or even worse, floating beliefs
Making Beliefs Pay Rent (in Anticipated Experiences)

Q: What should you ask instead of what to believe?
A: what to anticipate
Making Beliefs Pay Rent (in Anticipated Experiences)

Q: What should you do to a belief that doesn't pay rent?
A: evict it
Making Beliefs Pay Rent (in Anticipated Experiences)

Q: What do we need to anticipate sensory experiences as precisely as possible?
A: to process beliefs that are not anticipations of sensory experience
Making Beliefs Pay Rent (in Anticipated Experiences)

Q: what's wrong with asking "Does the claimant really believe there's a dragon present, or not?"
A: it's more tangled than that--the human brain can have more than one belief at the same time
Belief in Belief

Q: how can we tell that the claimant doesn't anticipate perceiving the dragon?
A: he makes excuses in advance
Belief in Belief

Q: what is often much easier than believing a thing?
A: believing that you ought to believe it
Belief in Belief

Q: can we fix the invisible dragon problem simply by making beliefs pay rent in anticipation?
A: no--it's a symptom of something much worse
Belief in Belief

Q: why don't people believe in belief in belief?
A: belief in belief wouldn't work if you saw yourself doing it
Belief in Belief

Q: what is explicit belief in belief
A: saying to yourself, "It is virtuous to believe x" and then believing that you believe it

Q: what are two forms of belief in belief less explicit than thinking the belief is virtuous?
A: fear of public ridicule, conflict with self-image as discoverer of the dragon
Belief in Belief

Q: what things account for as much of our motivations as the words we say to ourselves
A: fleeting mental images, unspoken flinches, desires acted upon without acknowledgement
Belief in Belief

Q: what is the key notion necessary to understand the dragon-claimant
A: belief in belief
Belief in Belief

Q: what must our notion of belief not be limited to in order to understand the dragon claimant?
A: verbal sentences
Belief in Belief

Q: what must our notion of belief include to understand how the dragon claimant makes excuses in advance?
A: unspoken anticipation-controllers
Belief in Belief

Q: what should our notion of "belief in belief" include to explain how it happens?
A: unspoken cognitive behavior-guiders
Belief in Belief

Q: It is not psychologically realistic to say, "The dragon-claimant does not believe there is a dragon in his garage, he believes it is beneficial to believe there is a dragon in his garage."  What related statement is realistic?
A: "The dragon-claimant anticipates as if there is no dragon in his garage, and makes excuses as if he believed in the belief."
Belief in Belief

Q: what need not necessarily accompany a mental picture of a dragon-less garage
A: the verbal phrase, "There is no dragon in my garage."
Belief in Belief

Q: what is not necessary to make you flinch away from giving up your self-image as a person who believes in the dragon?
A: explicitly thinking, "I want to believe there's a dragon in my garage."
Belief in Belief

Q: how does the claimant flinch away from giving up his self-image as a person who believes in the dragon?
A: flinching away from the prospect of admitting he doesn't believe
Belief in Belief

Q: To [...], the dragon-claimant must (a) possess an accurate anticipation-controlling model somewhere in his mind, and (b) act cognitively to protect either (b1) his free-floating propositional belief in the dragon or (b2) his self-image of believing in the dragon.
A: To correctly anticipate, in advance, which experimental results shall need to be excused, the dragon-claimant must (a) possess an accurate anticipation-controlling model somewhere in his mind, and (b) act cognitively to protect either (b1) his free-floating propositional belief in the dragon or (b2) his self-image of believing in the dragon.
Belief in Belief

Q: To correctly anticipate, in advance, which experimental results shall need to be excused, the dragon-claimant must (a) [...], and (b) act cognitively to protect either (b1) his free-floating propositional belief in the dragon or (b2) his self-image of believing in the dragon.
A: To correctly anticipate, in advance, which experimental results shall need to be excused, the dragon-claimant must (a) possess an accurate anticipation-controlling model somewhere in his mind, and (b) act cognitively to protect either (b1) his free-floating propositional belief in the dragon or (b2) his self-image of believing in the dragon.
Belief in Belief

Q: To correctly anticipate, in advance, which experimental results shall need to be excused, the dragon-claimant must (a) possess an accurate anticipation-controlling model somewhere in his mind, and (b) [...] (b1) his free-floating propositional belief in the dragon or (b2) his self-image of believing in the dragon.
A: To correctly anticipate, in advance, which experimental results shall need to be excused, the dragon-claimant must (a) possess an accurate anticipation-controlling model somewhere in his mind, and (b) act cognitively to protect either (b1) his free-floating propositional belief in the dragon or (b2) his self-image of believing in the dragon.
Belief in Belief

Q: To correctly anticipate, in advance, which experimental results shall need to be excused, the dragon-claimant must (a) possess an accurate anticipation-controlling model somewhere in his mind, and (b) act cognitively to protect either (b1) [...] or (b2) his self-image of believing in the dragon.
A: To correctly anticipate, in advance, which experimental results shall need to be excused, the dragon-claimant must (a) possess an accurate anticipation-controlling model somewhere in his mind, and (b) act cognitively to protect either (b1) his free-floating propositional belief in the dragon or (b2) his self-image of believing in the dragon.
Belief in Belief

Q: To correctly anticipate, in advance, which experimental results shall need to be excused, the dragon-claimant must (a) possess an accurate anticipation-controlling model somewhere in his mind, and (b) act cognitively to protect either (b1) his free-floating propositional belief in the dragon or (b2) [...].
A: To correctly anticipate, in advance, which experimental results shall need to be excused, the dragon-claimant must (a) possess an accurate anticipation-controlling model somewhere in his mind, and (b) act cognitively to protect either (b1) his free-floating propositional belief in the dragon or (b2) his self-image of believing in the dragon.
Belief in Belief

Q: what makes belief in belief much less severe?
A: its being accompanied by belief
Belief in Belief

Q: why is belief in belief accompanied by belief less severe?
A: it can be addressed with experiments
Belief in Belief

Q: what is going on when we see pride/satisfaction/flaunting related to scientifically outrageous beliefs?
A: cheering for belief, not profession of belief
Profession and Cheering

Q: what is absent with cheering that is present with belief in belief?
A: an attempt to actually be persuasive and and to convince the listener that he takes his belief seriously
Profession and Cheering

Q: what characteristics will we be able to observe in someone who is cheering?
A: pride/satisfaction/flauting
Profession and Cheering

Q: what is the difference between a "proper belief" and forms of "improper belief"
A: a proper belief can be wrong or irrational, but the other forms are arguably "not belief at all"
Belief as Attire

Q: what is belief as attire?
A: belief as group-identification
Belief as Attire

Q: why can't we describe suicide bombers as believing themselves to be performing "heroic self-sacrifice"?
A: the very concept of the courage and altruism of a suicide bomber is Enemy attire
Belief as Attire

Q: does Eliezer think those who believe in belief are actually passionate?
A: no--those who have actually stopped anticipating will try to convince themselves of their own passion (and this desperation can be mistaken for passion), but it's not the real fire
Belief as Attire

Q: why does belief as attire inspire more passion than belief in belief?
A: identifying with a tribe is very powerful emotionally, and the beliefs which are attire of that tribe will be spoken with the full passion of belonging to that tribe
Belief as Attire

Q: how does Robin advocate identifying belief as attire?
A: asking yourself if you feel proud that a belief associates you with some group with which you want to be associated
Belief as Attire

Q: what sort of resource should we treat anticipation as?
A: a conserved resource
Focus Your Uncertainty

Q: what can we say about our supply of ancipation?
A: it is limited
Focus Your Uncertainty

Q: what is the art of focusing your uncertainty?
A: squeezing as much anticipation as possible into whichever outcome will actually happen
Focus Your Uncertainty

Q: why do we sometimes think too generally?
A: generalizing seems virtuous and wise
The Virtue of Narrowness

Q: what is wrong with a category that includes everything?
A: it might as well include nothing
The Virtue of Narrowness

Q: Good hypotheses can only explain [...].
A: Good hypotheses can only explain some possible outcomes, and not others.
The Virtue of Narrowness

Q: what do we mistakenly think about narrow concepts?
A: they are lowly, not lofty, and unphilosophical
The Virtue of Narrowness

Q: why do we need narrow words?
A: to express precise thoughts
The Virtue of Narrowness

Q: which is easier: belief or disbelief?
A: belief
Your Strength as a Rationalist

Q: why is belief easier than disbelief?
A: we believe instinctively; disbelief requires a conscious effort
Your Strength as a Rationalist: citation

Q: what is the problem with a hypothesis that forbids nothing?
A: it permits everything, and thereby fails to constrain anticipation
Your Strength as a Rationalist

Q: what is your strength as a rationalist?
A: your ability to be more confused by fiction than by reality
Your Strength as a Rationalist

Q: what do you have if you are equally good at explaining any outcome?
A: zero knowledge
Your Strength as a Rationalist

Q: what is one of the most important sensations a truthseeker can have? (not quite an emotion)
A: the sensation of still feels a little forced
Your Strength as a Rationalist

Q: how does the still feels a little forced feeling manifest itself?
A: as a quiet strain in the back of your mind
Your Strength as a Rationalist

Q: how does Eliezer think the still feels a little forced feeling should manifest itself?
A: as a wailing alarm siren and a glowing neon sign reading "EITHER YOUR MODEL IS FALSE OR THIS STORY IS WRONG
Your Strength as a Rationalist

Q: Warren responded, "I take the view that this lack [of subversive activity] is the most ominous sign in our whole situation. It convinces me more than perhaps any other factor that the sabotage we are to get, the Fifth Column activities are to get, are timed just like Pearl Harbor was timed... I believe we are just being lulled into a false sense of security."
What is the problem with Warren's argument?
A: You could argue that a Fifth Column might delay its sabotage. But the likelihood is still higher that the absence of a Fifth Column would perform an absence of sabotage.
Absence of Evidence Is Evidence of Absence

Q: when we see evidence, how do we evaluate our hypotheses in light of it?
A: hypotheses that assigned a higher likelihood to that evidence, gain probability at the expense of hypotheses that assigned a lower likelihood to the evidence
Absence of Evidence Is Evidence of Absence

Q: why might a hypothesis that assigns high likelihood to evidence still lose probability mass?
A: because some other hypothesis assigns a likelihood that is even higher
Absence of Evidence Is Evidence of Absence

Q: in probability theory, how must we always interpret absence of evidence?
A: as evidence of absence
Absence of Evidence Is Evidence of Absence

Q: what is the correct version of "absence of evidence is not evidence of absence"?
A: absence of proof is not proof of absence
Absence of Evidence Is Evidence of Absence

Q: Under the vast majority of real-life circumstances, a cause may not reliably produce signs of itself, but [...].
A: Under the vast majority of real-life circumstances, a cause may not reliably produce signs of itself, but the absence of the cause is even less likely to produce the signs.
Absence of Evidence Is Evidence of Absence

Q: what determines how strong a piece of evidence against a cause the absence of an observation is?
A: how likely the cause is to produce the observation
Absence of Evidence Is Evidence of Absence

Q: what is the fallacy of the "gaps in the fossil record"?
A: fossils form only rarely
Absence of Evidence Is Evidence of Absence

Q: If you don't notice when your model makes the evidence unlikely, you might as well [...].
A: If you don't notice when your model makes the evidence unlikely, you might as well have no model.
Absence of Evidence Is Evidence of Absence

Q: If you don't notice when [...], you might as well have no model.
A: If you don't notice when your model makes the evidence unlikely, you might as well have no model.
Absence of Evidence Is Evidence of Absence

Q: what is the rule absence of "evidence is evidence of absence" a more general case of?
A: Eliezer's law of Conservation of Expected Evidence
Conservation of Expected Evidence

Q: what is Eliezer's Law of Conservation of Expected Evidence?
A: The expectation of the posterior probability, after viewing the evidence, must equal the prior probability.
Conservation of Expected Evidence

Q: what is the consequence of Eliezer's Law of Conservation of Expected Evidence?
A: for every expectation of evidence, there is an equal and opposite expectation of counterevidence
Conservation of Expected Evidence

Q: If you expect a strong probability of seeing weak evidence in one direction, it must be balanced by [...].
A: If you expect a strong probability of seeing weak evidence in one direction, it must be balanced by a weak expectation of seeing strong evidence in the other direction.
Conservation of Expected Evidence

Q: how confident should you expect to be after getting more evidence?
A: on average: exactly as confident
Conservation of Expected Evidence

Q: how should the expectation of encountering evidence shift your prior beliefs?
A: it should not shift them at all!
Conservation of Expected Evidence

Q: what must you also claim if you say that "no sabotage" is evidence for the existence of a Japanese-American Fifth Column?
A: that seeing sabotage would argue against a Fifth Column
Conservation of Expected Evidence

Q: For a true Bayesian, it is impossible to seek evidence that [...].
A: For a true Bayesian, it is impossible to seek evidence that confirms a theory.
Conservation of Expected Evidence

Q: what strategy can you use to legitimately expect your confidence in a fixed proposition to be higher (on average) than before?
A: none!
Conservation of Expected Evidence

Q: what can you expect evidence do to your theory?
A: test it
Conservation of Expected Evidence

Q: what can you not expect evidence to do your theory?
A: confirm it
Conservation of Expected Evidence

Q: what is hindsight bias?
A: when people who know the answer vastly overestimate its predictability or obviousness, compared to the estimates of subjects who must guess without advance knowledge
Hindsight Bias

Q: what is another name for hindsight bias?
A: the I-knew-it-all-along effect
Hindsight Bias

Q: what did Kamin and Rachlinski (1995) find was the effect of telling jurors about hindsight bias and telling them to avoid it?
A: nothing--56% thought the company was negligent compared to 57% (base was 24%)
Hindsight Bias

Q: how does hindsight bias make us view saftey precautions?
A: it makes us vastly underestimate their cost
Hindsight Bias

Q: what is wrong with evaluating the cost of attending to a warning sign that caused a problem?
A: you must consider the cost of attending to every warning sign of that level of severity--not only the one that caused the problem
Hindsight Bias

Q: what is the cure for hindsight bias?
A: writing down your predictions in advance
Hindsight Bias

Q: Because we don't see the cost of a general policy, we [...].
A: Because we don't see the cost of a general policy, we learn overly specific lessons.
Hindsight Bias

Q: how does hindsight bias make us view science?
A: as useless--because we could have predicted it
Hindsight Devalues Science

Q: why do people dismiss science as only delivering predictable results?
A: hindsight bias
Hindsight Devalues Science

Q: according to Eliezer, to most people, what is the difference between "because of heat conduction" and "because of magic" as explanations?
A: they are readily recognized as belonging to different literary genres
Fake Explanations

Q: as Bayesians, what do we think of the literary genre of an explanation?
A: we take no notice of it
Fake Explanations

Q: for Bayesians, what is the substance of a model?
A: the control it exerts on anticipation
Fake Explanations

Q: what is a disguised hypothesis of maximum entropy?
A: words that feel like an explanation, but are equally good at explaining all outcomes
Fake Explanations

Q: Words do not have [...].
A: Words do not have intrinsic definitions.
Guessing the Teacher's Password

Q: If I hear the syllables "bea-ver" and think of a large rodent, what is that a fact about?
A: my own state of mind
Guessing the Teacher's Password

Q: If I hear the syllables "bea-ver" and think of a large rodent, what is that not a fact about?
A: the syllables "bea-ver"
Guessing the Teacher's Password

Q: what is the problem with rewarding students for saying the correct answers?
A: because they are given gold stars for saying the right words, students begin to think that verbal behavior has a truth-value
Guessing the Teacher's Password

Q: what habit (even worse than thinking that verbal behavior has a truth value) does guessing the teacher's password encourage?
A: saying words that aren't even hypotheses when you don't know, since there is no credit given for saying "I don't know"
Guessing the Teacher's Password

Q: why do students say "because of heat conduction" when they don't know what's going on?
A: in an attempt to guess the teacher's password
Guessing the Teacher's Password

Q: This is Bayescraft; we are scoring [...].
A: This is Bayescraft; we are scoring your anticipations of experience.
Guessing the Teacher's Password

Q: what is a fake explanation?
A: a verbal behavior that is not an anticipation-controlller
Guessing the Teacher's Password

Q: The X-Men comics use terms like "evolution", "mutation", and "genetic code", purely to place themselves in what they conceive to be [...].
A: The X-Men comics use terms like "evolution", "mutation", and "genetic code", purely to place themselves in what they conceive to be the literary genre of science.
Science as Attire

Q: The part that scares me is wondering how many people, especially in the media, understand science only as a [...].
A: The part that scares me is wondering how many people, especially in the media, understand science only as a literary genre.
Science as Attire

Q: why does Eliezer thinks that an actual majority of the people who believe in evolution use the phrase "because of evolution"?
A: because they want to be part of the scientific in-crowd--belief as scientific attire (like wearing a lab coat)
Science as Attire

Q: why do many people accept dumber-than-human AI and mildly smarter-than-human AI but dismiss super-human AI as pseudoscience?
A: they associate strongly superhuman AI to the literary genre of apocalyptic literature; whereas an AI running a small corporation associates to the literary genre of Wired magazine
Science as Attire

Q: what is NOT the reason many people accept dumber-than human AI and mildly smarter-than-human AI but dismiss super-human AI as pseudoscience?
A: they think they have a theory of intelligence which lets them calculate a theoretical upper bound on the power of an optimization process
Science as Attire

Q: what is the sum of beliefs you have assimilated and made a true part of yourself?
A: what future experiences your belief prohibits from happening
Science as Attire

Q: what are beliefs that do not prohibit future experiences, probably? (two types of floating beliefs)
A: passwords or attire
Science as Attire

Q: what makes fake explanations dangerous?
A: they don't feel fake
Fake Causality

Q: Modern research suggests that humans think about cause and effect using something like [...].
A: Modern research suggests that humans think about cause and effect using something like the directed acyclic graphs (DAGs) of Bayes nets.
Fake Causality

Q: why does phlogiston feel like an explanation?
A: it's represented using the same cognitive data format as a real explanation
Fake Causality

Q: Thanks to hindsight bias, it may feel like the cause [...] the effect, when it was merely fitted to the effect.
A: Thanks to hindsight bias, it may feel like the cause constrains the effect, when it was merely fitted to the effect.
Fake Causality

Q: Thanks to hindsight bias, it may feel like the cause constrains the effect, when it was merely [...] to the effect.
A: Thanks to hindsight bias, it may feel like the cause constrains the effect, when it was merely fitted to the effect.
Fake Causality

Q: what can our modern understanding of probabilistic reasoning about causality can tell us precisely what the phlogiston theorists were doing wrong?
A: the problem of double-counting evidence if inference resonates between an effect and a cause
Fake Causality

Q: what analogy does Judea Pearl use for double-counting evidence if inference resonates between an effect an a cause?
A: the algorithm for counting soldiers in a line by counting how many you see, asking the soldier next to you how many he sees and adding the numbers
Fake Causality

Q: We count each piece of evidence exactly [...]; no update message ever "bounces" back and forth.
A: We count each piece of evidence exactly once; no update message ever "bounces" back and forth.
Fake Causality

Q: We count each piece of evidence exactly once; no update message ever [...].
A: We count each piece of evidence exactly once; no update message ever "bounces" back and forth.
Fake Causality

Q: We learn about parent nodes from observing children, and predict child nodes from beliefs about parents.  But we don't keep [...].
A: We learn about parent nodes from observing children, and predict child nodes from beliefs about parents.  But we don't keep rigorously separate books for the backward-message and forward-message.
Fake Causality

Q: hindsight bias is the NONtechnical way of saying that?
A: humans do not rigorously separate forward and backward messages, allowing forward messages to be contaminated by backward ones
Fake Causality

Q: what is the only way that a messy human mind can be guaranteed of sending a pure forward message?
A: predicting tomorrow, not yesterday
Fake Causality

Q: Jonathan Wallace suggested that "God!" functions as a [...].
A: Jonathan Wallace suggested that "God!" functions as a semantic stopsign.
Semantic Stopsigns

Q: what makes something a semantic stopsign?
A: it isn't a propositional assertion, so much as a cognitive traffic signal: do not think past this point
Semantic Stopsigns

Q: what behavior distinguishes a semantic stopsign?
A: failure to consider the obvious next question
Semantic Stopsigns

Q: is God the only semantic stopsign?
A: no!
Semantic Stopsigns

Q: what must we be careful of when recognizing stopsigns?
A: not to create a new generic counterargument against things you don't like
Semantic Stopsigns

Q: are words stopsigns in and of themselves?
A: no, the question is whether a word has that effect on a particular person
Semantic Stopsigns

Q: does having strong emotions about something qualify it as a stopsign?
A: no
Semantic Stopsigns

Q: what used to be a stopsign for Eliezer, but is no longer?
A: intelligence
Semantic Stopsigns

Q: what are the four signs of mysterious answers to mysterious questions?
1. [...]
2.
3.
4.
A: the explanation acts as a curiosity-stopper than than an anticipation controller
Mysterious Answers to Mysterious Questions

Q: what are the four signs of mysterious answers to mysterious questions?
1. the explanation acts as a curiosity-stopper, not an anticipation-controller
2. [...]
3.
4.
A: the hypothesis has no moving parts - the model is not a specific complex mechanism, but a blankly solid substance or force
Mysterious Answers to Mysterious Questions

Q: what are the four signs of mysterious answers to mysterious questions?
1. the explanation acts as a curiosity-stopper, not an anticipation-controller
2. the hypothesis has no moving parts - the model is not a specific complex mechanism, but a blankly solid substance or force
3. [...]
4.
A: those who proffer the explanation cherish their ignorance; they speak proudly of how the phenomenon defeats ordinary science or is unlike merely mundane phenomena
Mysterious Answers to Mysterious Questions

Q: what are the four signs of mysterious answers to mysterious questions?
1. the explanation acts as a curiosity-stopper, not an anticipation-controller
2. the hypothesis has no moving parts - the model is not a specific complex mechanism, but a blankly solid substance or force
3. those who proffer the explanation cherish their ignorance; they speak proudly of how the phenomenon defeats ordinary science or is unlike merely mundane phenomena
4. [...]
A: even after the answer is given, the phenomenon is still a mystery and possesses the same quality of wonderful inexplicability that it had at the start
Mysterious Answers to Mysterious Questions

Q: Meeting the great dragon Unknown, the vitalists [...].
A: Meeting the great dragon Unknown, the vitalists did not draw their swords to do battle, but bowed their necks in submission.
Mysterious Answers to Mysterious Questions

Q: how did the vitalists think of their ignorance?
A: they took pride in it
Mysterious Answers to Mysterious Questions

Q: how did the vitalists think of biology?
A: as a sacred mystery
Mysterious Answers to Mysterious Questions

Q: why were the vitalists loath to relinquish their ignorance when evidence came knocking?
A: they took pride in it, making biology into a sacred mystery
Mysterious Answers to Mysterious Questions

Q: where does ignorance exist?
A: in the map--not the territory
Mysterious Answers to Mysterious Questions

Q: if we are ignorant about something, what is that a fact about?
A: our own state of mind
Mysterious Answers to Mysterious Questions

Q: if we are ignorant about something, what is not that a fact about?
A: the phenomenon itself
Mysterious Answers to Mysterious Questions

Q: can phenomena be mysterious of themselves?
A: no!
Mysterious Answers to Mysterious Questions

Q: what does it mean to worship a phenomenon because it seems so wonderfully mysterious?
A: to worship your own ignorance
Mysterious Answers to Mysterious Questions

Q: Vitalism shared with phlogiston the error of [...].
A: Vitalism shared with phlogiston the error of encapsulating the mystery as a substance.
Mysterious Answers to Mysterious Questions

Q: what is mystery a property of?
A: questions
Mysterious Answers to Mysterious Questions

Q: what is mystery not a property of?
A: answers
Mysterious Answers to Mysterious Questions

Q: Eliezer objects to the word emergence; what related words does he not object to?
A: "emerges from"
The Futility of Emergence

Q: what is Eliezer's issue with the word emergence?
A: it is commonly used as an explanation in its own right
The Futility of Emergence

Q: what current theory does Eliezer liken to phlogiston and vitalism?
A: emergence or emergent phenomena
The Futility of Emergence

Q: what exercise can we use to see whether the adjective "emergent" has value?
A: removing it from sentences and seeing if they still say the same thing
The Futility of Emergence

Q: Eliezer thinks "emergence" appeals to deep human psychology in the same way as what other word?
A: "magic"
The Futility of Emergence

Q: Emergence is popular because it is the [...].
A: Emergence is popular because it is the junk food of curiosity.
The Futility of Emergence

Q: how do we evaluate the usefulness of concepts?
A: we don't--we evaluate the usefulness of usages
Say Not "Complexity"

Q: how should we treat the mysterious part of an explanation?
A: linger at it to confront it directly
Say Not "Complexity"

Q: how should we not treat the mysterious part of an explanation?
A: skip over it
Say Not "Complexity"

Q: when shouldn't we use the word "complexity"?
A: when we're using it to skip over mysteries
Say Not "Complexity"

Q: The skip-over is not a thought, but a [...].
A: The skip-over is not a thought, but a microthought.
Say Not "Complexity"

Q: how do we find skip-overs?
A: you must pay close attention to catch yourself at it
Say Not "Complexity"

Q: eventually, what will not skipping over mysteries become?
A: instinct--not verbal reasoning, when you have trained yourself enough
Say Not "Complexity"

Q: why does Eliezer suspect that in academia there is huge pressure to sweep problems under the rug?
A: so that you can present a paper with the appearance of completeness
Say Not "Complexity"

Q: what do Eliezer and Marcello now use as a placeholder for unsolved problems?
A: "magic"--as in "X magically does Y"
Say Not "Complexity"

Q: what sort of examples do students who attempt the 2-4-6 task usually generate?
A: positive ones, rather than negative ones
Positive Bias: Look Into the Dark

Q: to figure out the true 2-4-6 rule, what do you have to do?
A: generate triplets that shouldn't fit, and see if they are labeled "No"
Positive Bias: Look Into the Dark

Q: what cognitive phenomenon is positive bias usually lumped with?
A: confirmation bias
Positive Bias: Look Into the Dark

Q: what is positive bias?
A: the phenomenon of trying to test positive rather than negative examples
Positive Bias: Look Into the Dark

Q: why don't we check for negative examples much?
A: it goes against the grain of what experiment has shown to be human instinct
Positive Bias: Look Into the Dark

Q: what cannot explain positive bias?
A: emotional attachment
Positive Bias: Look Into the Dark

Q: why can't you solve the problem of positive bias by knowing verbally that "We ought to think about both positive and negative examples?"
A: because the problem doesn't arise from following a deliberate rule that says "Only think about positive examples"
Positive Bias: Look Into the Dark

Q: why is it challenging work to try to convey the Art of Rationality through blog posts?
A: so much of a rationalists skill is below the level of words
Positive Bias: Look Into the Dark

Q: what does Eliezer consider the most embarrassing mistake he has made in his life?
A: he expected future neurologists to discover that neurons were exploiting quantum gravity, a la Sir Roger Penrose
My Wild and Reckless Youth

Q: what is one of the chief pieces of advice that Eliezer gives to aspiring young rationalists?
A: "Do not attempt long chains of reasoning or complicated plans."
My Wild and Reckless Youth

Q: what sort of mistake was thinking that neurons were exploiting quantum gravity?
A: it was a mysterious answer to a mysterious question
My Wild and Reckless Youth

Q: why did young Eliezer think it was okay to think that neurons were explointing quantum gravity?
A: it was falsifiable, and he wanted to test it
My Wild and Reckless Youth

Q: to a Bayesian, what was the problem with Eliezer's theory that neurons were exploiting quantum gravity?
A: it made no retrospective predictions
My Wild and Reckless Youth

Q: to a Bayesian, what is the importance of retrospective predictions?
A: if a hypothesis does not today have a favorable likelihood ratio over "I don't know", it raises the question of why you today believe anything more complicated than "I don't know"
My Wild and Reckless Youth

Q: what light does Eliezer think his experience with Traditional Rationality allowing him to think that neurons exploit quantum gravity shed on why those who call themselves "rationalists" do not rule the world?
A: You need one whole hell of a lot of rationality before it does anything but lead you into new and interesting mistakes.
My Wild and Reckless Youth

Q: what is the problem, in Eliezer's mind, with Traditional Rationality
A: It's designed to get you to the truth eventually, but gives you all too much time to smell the flowers along the way.
My Wild and Reckless Youth

Q: which mistake stands out as the most critical to Eliezer about his mysterious answer to a mysterious question?
A: his younger self did not realize that solving a mystery should make it feel less confusing
Failing to Learn from History

Q: what does truly explaining a Mysterious Phenomenon due to it?
A: turns it into a mundane phenomenon
Failing to Learn from History

Q: what did Eliezer ignore when thinking that his Mysterious Phenomenon was different in being new, stranger, and more difficult?
A: the fact that stars, matter, and life had also seemed that way for hundreds of years and thousands of years, right up until science solved them
Failing to Learn from History

Q: why are we skeptical when science dares to challenge a new Great Puzzle?
A: becuase we have never seen science explain something that feels mysterious to us and assume that science is only for predetermined scientific subjects
Failing to Learn from History

Q: what did younger Eliezer think the lesson of history about astrologers and alchemists and vitalists was?
A: that they had an innate character flaw, a tendency towards mysterianism
Failing to Learn from History

Q: how did vitalism seem at the time?
A: reasonable
Failing to Learn from History

Q: how did "Life is mundane and does not need a weird explanation" feel at the time?
A: surprising and emabarassing
Failing to Learn from History

Q: what is the logical fallacy of generalization from fictional evidence?
A: recalling fiction as if it were an illustrative historical case
Making History Available

Q: what is the inverse error to generalizing from fictional evidence?
A: failing to be sufficiently moved by historical evidence
Making History Available

Q: what is the trouble with generalizing from fictional evidence?
A: it never actually happened, it's not drawn from the same distribution as the real universe, and indeed differs from reality in systematic ways
Making History Available

Q: what would have made Eliezer think No way am I falling for that again about his Mysterious Answer?
A: if he had personally postulated astrological mysteries and then discovered Newtonian mechanics, postulated alchemical mysteries and then discovered chemistry, postulated vitalistic mysteries and then discovered biology
Failing to Learn from History

Q: why (according to Eliezer) do fictions we see in lifelike moving pictures have too great an impact on us?
A: in our ancestral environment, what you saw with your own eyes was true
Making History Available

Q: why do we underweight written evidence?
A: because we did not see it happen, and therefore do not remember it happening to us
Making History Available

Q: Many biases involve being insufficiently moved by [...].
A: Many biases involve being insufficiently moved by dry, abstract information.
Making History Available

Q: what did Eliezer decide to do so as not to make the mistake of insufficiently weighting historical evidence?
A: trying to approximate the thoughts of an Eliezer who had lived through all of history
Making History Available

Q: Photons and history books both [...].
A: Photons and history books both descend by causal chains from the event itself.
Making History Available

Q: So the next time you doubt the strangeness of the future, remember how [...].
A: So the next time you doubt the strangeness of the future, remember how you were born in a hunter-gatherer tribe ten thousand years ago, when no one knew of Science at all.
Making History Available

Q: what might make us less shocked by what happens next?
A: having history available to us
Making History Available

Q: what are the three responses to unexplained phenomena?
A: Explain/Worship/Ignore
Explain/Worship/Ignore

Q: what does it mean to Ignore?
A: not ask why, not follow up on the question, or never think of the question in the first place
Explain/Worship/Ignore

Q: what does it mean to Explain? (Explain/Worship/Ignore)
A: try to devise some sort of explanation
Explain/Worship/Ignore

Q: what does it mean to Worship?
A: enjoy the sensation of mysteriousness
Explain/Worship/Ignore

Q: what is the dilemma with Explain?
A: that the explanation is then subject to the same choice
Explain/Worship/Ignore

Q: what is the spell that stops curiosity, according to Eliezer?
A: Whenever anyone asks "How did you do that?", I just say "Science!"
"Science" as Curiosity-Stopper

Q: what is saying "Science!" as an answer to how you did something?
A: a curiosity-stopper
"Science" as Curiosity-Stopper

Q: what is saying "Science!" as an answer to how you did something not?
A: a real explanation
"Science" as Curiosity-Stopper

Q: why isn't "Science!" a real explanation?
A: you don't actually know anything more than you did before
"Science" as Curiosity-Stopper

Q: what would make the lightbulb seize the entirity of your attention?
A: if you thought it was scientifically inexplicable
"Science" as Curiosity-Stopper

Q: what does "scientifically explicable" mean?
A: that someone else knows how it works
"Science" as Curiosity-Stopper

Q: what is the consequence of our knowing that someone else knows something?
A: it devalues the answer in our eyes, and we become less curious
"Science" as Curiosity-Stopper

Q: what does Eliezer object to about people complaining that cruel Science has emptied the world of mystery?
A: other people knowing the answers to questions need not destroy our own curiosity
"Science" as Curiosity-Stopper

Q: What does it mean to call for a "democratic" solution if you don't have a conflict-resolution mechanism in mind?
A: it means that you're using "democracy" as an applause light
Applause Lights

Q: what is an "applause light"?
A: a statement that is not so much a propositional statement as the equivalent of a light that tells the studio audience where to clap
Applause Lights

Q: how can we detect most applause lights?
A: the reversal test
Applause Lights

Q: how do we detect applause light with the reversal test?
A: see if the reversed statement sounds abnormal
Applause Lights

Q: what does it mean if the reversed statement sounds abnormal?
A: that the unreversed statement is probably normal, implying it does not convey new information
Applause Lights

Q: in addition to the reversal test, how do we confirm that a statement is merely an applause light?
A: by noticing that no specifics follow
Applause Lights

Q: how did Eliezer used to think of his productivity (on a timescale of hours)?
A: as inherently unpredictable
Chaotic Inversion

Q: what mistake did Eliezer make in thinking of his productivity as inherently unpredictable?
A: the mind projection fallacy
Chaotic Inversion

Q: Eliezer thought his productivity was unusually chaotic; what is the alternate explanation?
A: he was unusually stupid with respect to predicting it
Chaotic Inversion

Q: what does inverted stupidity look like?
A: chaos
Chaotic Inversion

Q: why does Eliezer say we tend to think that the environment is chaotic rather than that we are stupid?
A: we don't think of ourselves at all
Chaotic Inversion

Q: what is the first bad habit Eliezer says to unlearn from school?
A: not noticing when you're confused
Two More Things to Unlearn from School

Q: what's the second bad habit Eliezer says to unlearn from school?
A: that all authoritative ideas / all new ideas / all ideas that have a few plausible reasons to support them, are true
Two More Things to Unlearn from School

Q: back in the old days, how did people feel about their religions?
A: they believed them
Religion's Claim to be Non-Disprovable

Q: now days, how do people generally think of their religions?
A: they believe IN them
Religion's Claim to be Non-Disprovable

Q: what is the retreat to commitment?
A: saying "I believe because I believe"
Religion's Claim to be Non-Disprovable

Q: why did religious people retreat to commitment?
A: because they failed to find confirming evidence
Religion's Claim to be Non-Disprovable

Q: what sort of concept is the orthogonality of religion and factual questions?
A: recent and strictly Western
Religion's Claim to be Non-Disprovable

Q: why does the New Testement bear some traces of modernity?
A: it was written at the time of the Roman Empire, which inherited philosophy from the ancient Greeks; imposed law and order within its provinces; kept bureaucratic records; and enforced religious tolerance
Religion's Claim to be Non-Disprovable

Q: why didn't the New Testement claim big, show, large-scale geopolitical miracles like the Old Testament did?
A: the Romans had libraries
Religion's Claim to be Non-Disprovable

Q: what did religion used to make claims about?
A: everything
Religion's Claim to be Non-Disprovable

Q: why do we think of religion as being about ethics?
A: every other area it used to cover has been taken over by better institutions
Religion's Claim to be Non-Disprovable

Q: what is most people's concept of rationality determined by, according to Eliezer?
A: what they can get away with
Religion's Claim to be Non-Disprovable

Q: what is the idea of religion as a separate magisterium which cannot be proven or disproven?
A: a Big Lie
Religion's Claim to be Non-Disprovable

Q: why does Eliezer specifically distinguish positive bias from confirmation bias?
A: the phenomenon of trying to test positive rather than negative examples, ought to be distinguished from the phenomenon of trying to preserve the belief you started with
Positive Bias: Look Into the Dark

Q: Every [...] should begin by flowing to a specific guess of anticipation, and should continue to pay rent in future anticipations.
A: Every guess of belief should begin by flowing to a specific guess of anticipation, and should continue to pay rent in future anticipations.
Making Beliefs Pay Rent (in Anticipated Experiences)

Q: Every guess of belief should [...] by flowing to a specific guess of anticipation, and should continue to pay rent in future anticipations.
A: Every guess of belief should begin by flowing to a specific guess of anticipation, and should continue to pay rent in future anticipations.
Making Beliefs Pay Rent (in Anticipated Experiences)

Q: Every guess of belief should begin by [...] to a specific guess of anticipation, and should continue to pay rent in future anticipations.
A: Every guess of belief should begin by flowing to a specific guess of anticipation, and should continue to pay rent in future anticipations.
Making Beliefs Pay Rent (in Anticipated Experiences)

Q: Every guess of belief should begin by flowing to a [...], and should continue to pay rent in future anticipations.
A: Every guess of belief should begin by flowing to a specific guess of anticipation, and should continue to pay rent in future anticipations.
Making Beliefs Pay Rent (in Anticipated Experiences)

Q: Every guess of belief should begin by flowing to a specific guess of anticipation, and should [...] in future anticipations.
A: Every guess of belief should begin by flowing to a specific guess of anticipation, and should continue to pay rent in future anticipations.
Making Beliefs Pay Rent (in Anticipated Experiences)

Q: Every guess of belief should begin by flowing to a specific guess of anticipation, and should continue to pay rent in [...].
A: Every guess of belief should begin by flowing to a specific guess of anticipation, and should continue to pay rent in future anticipations.
Making Beliefs Pay Rent (in Anticipated Experiences)