Skinner box

The Twittering Machine
by Richard Seymour
Published 20 Aug 2019

And yet our interactions with the machine are conditioned. Critics of social media like Jaron Lanier argue that the user experience is designed much like the famous ‘Skinner Box’ or ‘operant conditioning chamber’ invented by the pioneering behaviourist B. F. Skinner. In this chamber, the behaviour of laboratory rats was conditioned by stimuli – lights, noises and food. Each of these stimuli constituted a ‘reinforcement’, either positive or negative, which would reward some forms of behaviour and discourage others. In the Skinner Box, test subjects are taught how to behave through conditioning. And if this model has found its way into the mobile apps, gaming and social industries, it might reflect the way that behaviourist ideas have achieved a surprising renaissance among businessmen and policymakers in recent decades.

…

Nor, even if they thought we did know, would they have any reason to give it to us. The machine is not a democracy, and it isn’t even a market; we are neither customers nor voters. We are digital ‘serfs’, says Jaron Lanier, the ‘livestock of a feudal demesne’, according to Bruce Sterling.52 We inhabit a laboratory, a real-life operant conditioning chamber, into which we have been lured by the promise of democratized luxury. In the early days of the internet, the promise was that we could ‘Ask Jeeves’; now we are offered ‘tools’ and ‘virtual assistants’. On that basis, millions of us have entered a web of surveillance in which we are the servants, providing endless hours of free labour.

…

As one former gambling addict puts it, ‘All I can remember is living in a trance for four years.’60 Schüll calls it the ‘machine zone’ where ordinary reality is ‘suspended in the mechanical rhythm of a repeating process’.61 For many addicts, the idea of facing the normal flow of time is unbearably depressing. Marc Lewis describes how even after kicking junk he couldn’t face ‘a day without a change of state’.62 The Twittering Machine, as a wholly designed operant conditioning chamber, needs none of the expedients of the casino or opium den. The user has already dropped out of work, a boring lunch, an anxious social situation or bad sex, to enter into a different, timeless, time zone. What we do on the Twittering Machine has as much to do with what we’re avoiding as what we find when we log in – which, after all, is often not that exciting.

User Friendly: How the Hidden Rules of Design Are Changing the Way We Live, Work & Play
by Cliff Kuang and Robert Fabricant
Published 7 Nov 2019

It kinda works like a slot machine, where you’re excited to see who the next person is, or, hopefully, you’re excited to see ‘did I get the match?’ and get that ‘It’s a Match’ screen? It’s a nice little rush.”20 Thrilled by such ease and pleasure, our society is busy putting a Skinner box in every hand by making them ever cheaper, easier to use, and easier to get. But unlike slot machines, our personal Skinner boxes don’t offer the prospect of riches. The market has figured out exactly the bare minimum that will keep us coming back. Of course, almost no one had consciously thought to create a world of Skinner-inspired addictiveness. But that only makes the creation more profound.

…

He noticed that the local farmers and laborers were uncomfortable entering the richly carpeted lobby in their dirty work boots—which he quickly corrected by adding some cheap rubber mats. That idea presaged the modern recognition of how social mores guide the adoption of new products, particularly those that incorporate technology. 1930: SKINNER BOX, Burrhus Frederic Skinner By isolating how an animal responded to a controlled input, the Skinner box revealed the way feedback loops guide behavior. Though Skinner’s reductionist view of psychology fell out of favor, it spawned landmark studies of the power of feedback and rewards in the human brain. 1933: SEARS TOPERATOR WASHING MACHINE, Henry Dreyfuss Dreyfuss’s first great hit, the Toperator washing machine, featured a streamlined, art deco design that avoided any hard-to-clean joints—an early nod to the user’s lifestyle that would be echoed widely.

…

“I shall probably continue therein, even, if necessary, by making over the entire field to suit myself.”10 His dream was to make the study of animal psychology every bit as rigorous as physics—to eliminate the need for any novelistic interpretation of why any creature might behave a certain way. He once again began by feverishly tinkering, and quickly hit a breakthrough with what would come to be called the Skinner box.11 The device was an uncanny echo of his childhood bedroom designs and his dream of fashioning behavior through the design of an environment. The box was a chamber, sized to a rat or a pigeon, that could be rigged up with a light or a loudspeaker; there was also a lever, which, when pushed, would deliver a food pellet.

Dawn of the New Everything: Encounters With Reality and Virtual Reality
by Jaron Lanier
Published 21 Nov 2017

He scared a baby when animals were around in order to prove he could make a human become scared of animals forever. And Skinner formalized an experimental box for conditioning animals in laboratories. Behaviorism has been reduced to gadgetry in pop culture. You tweet for an instant treat, for attention, even if you’re the president. You salivate when you hear a dog whistle. The Skinner box was the archetype. A person in a Skinner box has an illusion of control but is actually controlled by the box, or really by whoever is behind the box. A crucial distinction has to be made, and it isn’t easy to get it right. I was repulsed by the culture of behaviorists, though not behaviorism itself, which can be useful science.

…

It wasn’t just the movies! The CIA actually gave people LSD—without notice or consent—to see if that would facilitate mind control. Wiener extrapolated that computers might become powerful enough to run fancier Skinner boxes, more effective, harder to detect, and infinitely creepier. Read Wiener carefully and it’s clear that with good enough sensors, good enough computation, and good enough sensory feedback, a Skinner box could be implemented around a person in a waking state without that person’s realizing it. Wiener comforts the reader by pointing out that it would be so hard to create a giant computational facility and communications network that the danger is only theoretical.

…

HAL doesn’t reside in an android that walks around; it just sits there, ambient. Nonetheless it sails. It navigates both the spacecraft and what goes on inside. Now consider a Skinner box. What are the components? There is measurement of the creature in the box. Did the rat press the button? There is feedback. Will food appear? What causes action to be triggered by measurement? In the original experiments, a live scientist was at the controls, but these days, it’s an algorithm. The components of a Skinner box and those of a cybernetic computer are essentially the same. This is perhaps too elementary an observation even to state at this late date, but when I was young the connection was still fresh and shattering.

Unweaving the Rainbow
by Richard Dawkins
Published 7 Aug 2011

Therefore it obviously has to be followed by good luck. We are not the only animals to seek statistical patterns of non-randomness in nature, and we are not the only animals to make mistakes of the kind that might be called superstitious. Both these facts are neatly demonstrated in the apparatus called the Skinner box, after the famous American psychologist B. F. Skinner. A Skinner box is a simple but versatile piece of equipment for studying the psychology of, usually, a rat or a pigeon. It is a box with a switch or switches let into one wall which the pigeon (say) can operate by pecking. There is also an electrically operated feeding (or other rewarding) apparatus.

…

Interestingly, habits learned when pecks are only occasionally rewarded are more durable than habits learned when all pecks are rewarded: the pigeon is less swiftly discouraged when the rewarding mechanism is switched off altogether. This makes intuitive sense if you think about it. Pigeons and rats, then, are quite good statisticians, able to pick up slight, statistical laws of patterning in their world. Presumably this ability serves them in nature as well as in the Skinner box. Life out there is rich in pattern; the world is a big, complicated Skinner box. Actions by a wild animal are frequently followed by rewards or punishments or other important events. The relationship between cause and effect is frequently not absolute but statistical. If a curlew probes mud with its long, curved bill, there is a certain probability that it will strike a worm.

…

Another 'thought' that thrusting its head into the corner had the same beneficial effect. Both were making false positive errors. A false negative error is made by a pigeon in a Skinner box who never notices that a peck at the key yields food if the red light is on, but that a peck when the blue light is on punishes by switching the mechanism off for ten minutes. There is a genuine pattern waiting to be detected in the little world of this Skinner box, but our hypothetical pigeon does not detect it. It pecks indiscriminately to both colours, and therefore gets a reward less frequently than it could. A false positive error is made by a farmer who thinks that sacrificing to the gods brings longed-for rain.

What Algorithms Want: Imagination in the Age of Computing
by Ed Finn
Published 10 Mar 2017

She would never rebel because she loves you, selflessly and forever.11 The conversational construction of gender is prominent in the intellectual history of artificial intelligence, a topic we will return to. But for now we need to explore the back end of this performance in more depth. At their darkest, these digital assistants are culturally constructed as sentient Skinner boxes, quasi-intelligent systems that must answer, and that inspire the same kind of slavish devotion in their users. But of course, it is the human users who in the end must use these systems—not necessarily Siri, but the interconnected systems that evaluate our search queries, for example, or the algorithms that assign credit scores.

…

FarmVille and its successors are effective at eliciting particular rote behaviors from humans through a combination of carrots and sticks, engaging them in actions that the company can monetize directly or use to expand its network of users. The cultural narrative layer of the farm masks a mesmerizing Skinner box, the classic tool of operant conditioning, which, in this case, links revenue-generating behaviors to the innate human rewards of social connectedness and completion. As The Atlantic paraphrased one company executive: “One of the most compelling parts of playing Zynga’s games is deciding when and how to spam your friend with reminders to play Zynga’s games.”8 Ultimately these games are a kind of escapism masquerading as efficiency—plant your crops, build your empire, complete this task all in sixty seconds while you wait in line.

…

Players either didn’t realize that this was a satire, or played in spite of that knowledge, like the stay-at-home father who told Tanz, “instead of stupid games that have no point, we might as well play a stupid game that has a point.”13 At its apogee, over 50,000 people were clicking on digital cows and Bogost found himself enmeshed in his own Skinner box of feedback, getting rewarded by the player community when he added new features to the game. Bogost has described this process as a kind of “method design” like method acting, putting himself into the creative space of a social game designer and ultimately suffering the same kind of systemic, dehumanizing entanglement with the software that he sees it inflicting on players: “It’s hard for me to express the compulsion and self-loathing that have accompanied the apparently trivial creation of this little theory-cum-parody game.”14 Figure 4.1 Cow Clicker screenshot.

Lab Rats: How Silicon Valley Made Work Miserable for the Rest of Us
by Dan Lyons
Published 22 Oct 2018

The unlikely juxtaposition of those two seemingly contradictory things—silliness and cruelty—has become a hallmark of the new-economy modern workplace. That’s how I came to think of employees as lab rats. It turns out others had noticed the same thing. “Work is feeling more and more like a Skinner box” is how Gregory Berns, a neuropsychologist at Emory University, put it when he wrote a New York Times article about a study he had conducted about how fear impairs decision-making, which involved putting people into an MRI machine and zapping their feet with electric shocks. A Skinner box, invented in the 1930s by psychologist B. F. Skinner, is a cage in which rats learn that pulling certain levers gets them food and that flashing lights might signal they are about to get a shock through the floor.

…

To do that, he constructed an experiment that was downright shocking. He put people into an MRI scanner and made images of their brain while giving them painful electric shocks through electrodes attached to the top of their feet. Berns says it was basically a human version of a Skinner box. Berns, you may recall from Chapter 1, was the guy who once wrote that “Work is becoming more and more like a Skinner box.” Thirty-two people signed up to let Berns and his colleagues turn them into lab rats. He let each person set the maximum amount of pain they could tolerate, then gave them jolts, while using the scanner to see what was going on inside their brains.

…

But that’s actually the exact wrong thing to do. In a time of uncertainty, “taking risks and trying new things would actually be in your best interest,” Berns said. “But that’s difficult when you’re afraid of losing your job.” Berns said that for a lot of people, the workplace really does feel like a Skinner box, where you wander around like a rat in one of B. F. Skinner’s cages trying to figure out how to get rewards and how to avoid punishment. “It’s like you’re in a box, and you have no control over what’s happening to you,” Berns said. “It’s controlled by experimenters outside. You learn to associate certain places in that box with good and bad things.”

Actionable Gamification: Beyond Points, Badges and Leaderboards
by Yu-Kai Chou
Published 13 Apr 2015

If we know we will receive a reward, our excitement only reflects the emotional value of the reward itself. However, when we only have a chance to gain the reward our brains are more engaged by the thrill of whether we will win or not. The Core Drive in a Skinner Box There’s a substantial amount of research on how the unknown and the unpredictable intrigues and engages our minds. One of the most notable motivational design case studies that explored this phenomenon is the Skinner Box5. 6 The Skinner Box was an experiment conducted by the scientist B. F. Skinner, who placed rodents and pigeons in a box with an installed lever. In the first phase, whenever the animal pressed the lever (the Desired Action), a portion of food was released.

…

Here we see that satisfying our burning curiosity is intrinsically motivating to our primitive brain, sometimes more so than the extrinsic reward of food. Have you ever seen a person so addicted to gambling that he forgot that he was tired, hungry, or even thirsty? I often hear critiques of how the Points, Badges, and Leaderboards in gamification simply turns the world into a large Skinner Box, where people are manipulated to mindlessly doing meaningless tasks. I feel the more profound lesson from the Skinner Box is not that Points and Badges motivate people, but that unpredictable results stemming from Core Drive 7 can drive obsessive behavior. Sweepstakes and Raffles In Chapter 5 on Epic Meaning & Calling, I mentioned how I started my first business because of a small raffle held at a UCLA barbecue.

…

Epic Meaning within a Team Corporate Competition as an Oxymoron Game Techniques within Social Influence & Relatedness Core Drive 5: The Bigger Picture Chapter 10: The Sixth Core Drive: Scarcity & Impatience The Lure of being Exclusively Pointless The Value of Rare Pixels The Leftovers aren’t all that’s Left Over Persuasively Inconvenient Curves are better than Cups in Economics “This guy’s not expensive enough.” “I Don’t Feel Good When My Pocket Is Too Full After A Purchase” Game Techniques within Scarcity & Impatience Great! So now what? Core Drive 6: The Bigger Picture Chapter 11: The Seventh Core Drive - Unpredictability & Curiosity And, Now it’s Fun The Core Drive in a Skinner Box Sweepstakes and Raffles A Lucky Day with Lucky Diem Suspense and Mystery in a Blender Faking your Way to Virality Google’s Curious Second Button Woot! Creates Midnight Cinderella’s Game Techniques within Unpredictability & Curiosity Core Drive 7: The Bigger Picture Chapter 12: The Eighth Core Drive - Loss & Avoidance Cropping your Losses Affection Held Hostage “Why don’t you take all my money?”

The Autonomous Revolution: Reclaiming the Future We’ve Sold to Machines
by William Davidow and Michael Malone
Published 18 Feb 2020

Skinner were alive today, he would recognize the rat-like behavior of Internet addicts. Back in the 1930s, the Harvard psychologist held that an animal’s behavior is heavily influenced by its experience and conditioning, and that all creatures—including humans—are inclined to do what the world rewards them for doing. To test his theories, Skinner invented the “operant conditioning” chamber, or Skinner Box, in which he put pigeons and rats. This cage’s artfully designed food dispenser was controlled by an array of lighted buttons. When one of them lit up a certain color, pushing or pecking the adjacent lever caused a drawer to open that was filled with food. Eventually, through a process of trial and error and trial and reward, the animals learned to recognize and respond to the cues.

…

The same thing happens with social networkers, hunched over keyboards and oblivious to the flesh-and-blood people surrounding them. Couples in restaurants ignore each other while checking their smartphones for messages, and players of massively multiplayer online role-playing games (MMORPGs) become so absorbed in their play that they collapse from fatigue. The Internet is the ultimate Skinner Box, in which hundreds of millions of users peck keys in search of rewards—game points, engrossing bits of information, or a potential new love object. Neuroscience has given us an understanding of the neurochemistry underlying operant conditioning. We know that animals’ brains are primed for pleasure, and that pleasure is accompanied by the signature release of the chemical dopamine, a neurotransmitter.

The Upside of Irrationality: The Unexpected Benefits of Defying Logic at Work and at Home
by Dan Ariely
Published 31 May 2010

After a few days of this pattern, you learn to expect food at noon every day, and your rat tummy begins rumbling right before the nice man shows up—exactly the state Jensen wants you in. Once your body is conditioned to eating crackers at noon, things suddenly change. Instead of feeding you at the time of your maximal hunger, you have to wait another hour, and at one o’clock, the man picks you up and puts you in a well-lit “Skinner box.” You are ravenous. Named after its original designer, the influential psychologist B. F. Skinner, this box is a regular cage (similar to the one you are used to), but it has two features that are new to you. The first is an automated food dispenser that releases food pellets every thirty seconds.

…

The food dispenser releases food pellets every so often for twenty-five minutes, until you have eaten fifty food pellets. At that point you are taken back to your cage and given the rest of your food for the day. The next day, your lunch hour passes by again without food, and at 1:00 P.M. you are placed back into the Skinner box. You’re ravenous but unhappy because this time the food dispenser doesn’t release any pellets. What to do? You wander around the cage, and, passing the bar, you realize that the tin shield is missing. You accidentally press the bar, and immediately a pellet of food is released. Wonderful! You press the bar again.

…

Chapter 2: The Meaning of Labor: What Legos Can Teach Us about the Joy of Work Based on Dan Ariely, Emir Kamenica, and Dražen Prelec, “Man’s Search for Meaning: The Case of Legos,” Journal of Economic Behavior and Organization 67, nos. 3–4 (2008): 671–677. Glen Jensen, “Preference for Bar Pressing over ‘Freeloading’ as a Function of Number of Unrewarded Presses,” Journal of Experimental Psychology 65, no. 5 (1963): 451–454. Glen Jensen, Calvin Leung, and David Hess, “ ‘Freeloading’ in the Skinner Box Contrasted with Freeloading in the Runway,” Psychological Reports 27 (1970): 67–73. George Loewenstein, “Because It Is There: The Challenge of Mountaineering . . . for Utility Theory,” Kyklos 52, no. 3 (1999): 315–343. Additional readings George Akerlof and Rachel Kranton, “Economics and Identity,” The Quarterly Journal of Economics 115, no. 3 (2000): 715–753.

Survival of the Richest: Escape Fantasies of the Tech Billionaires
by Douglas Rushkoff
Published 7 Sep 2022

Rather, like rats in a maze, human beings were merely responding to rewards and punishments doled out by those in control or by the environment itself. We are conditioned to eat berries and run from lions the same way we are conditioned to stop at red lights, genuflect at a church’s altar, or order a Big Mac. The famous Skinner Box, where an animal pushes a lever to get food, became a metaphor for all sorts of “operant conditioning” performed on human beings in casinos, shopping malls, and other spaces where environmental triggers and rewards can be totally controlled. Studying humans in these locations became commonplace, with behavioral engineers using surveillance cameras to track consumers’ movement patterns, likelihood of inspecting merchandise, and response to changes in layout, color, or lighting.

…

It wasn’t until the first intentionally “sticky” websites in the mid-nineties—websites designed to keep users from surfing away—that digital technology provided the sort of controlled environment and live feedback mechanisms required to do operant conditioning en masse. Websites, video games, and smartphone apps all serve as virtual Skinner Boxes, giving developers the ability to build in operant conditioning routines to modify human behavior. As I argued in my book Program or Be Programmed , software companies are no longer programming computers; they are programming us people. Notifications, swipes, Likes, and “leveling up” were all developed and optimized for their ability to trigger dopamine releases on cue and foster compulsive behaviors.

…

And then if they post the idea, it gets a few hits and likes and comments from others, and ding ding squirt squirt … another hit of dopamine. And another and another. As well as an ounce of dignity for being recognized. It’s as if Q were simply an expression of end-stage internet addiction. The perfect digital Skinner Box and Freudian transference mechanism all at once. An industry success story. After Trump’s loss, things got worse. Sam felt betrayed. He began staying up all night reading Twitter and following links by leading alt-right posters. He became convinced that computers had been used to change the vote—presumably by either Angela Merkel from Munich or Barack Obama himself, working digitally through a consulate in Italy.

The Friendly Orange Glow: The Untold Story of the PLATO System and the Dawn of Cyberculture
by Brian Dear
Published 14 Jun 2017

Skinner eventually joined the Harvard faculty and became one of the most famous, and certainly one of the most controversial, experimental psychologists of the twentieth century. He called himself a “radical behaviorist.” Much of Skinner’s early experimental research centered on analyzing the behavior of rats and pigeons. His Operant Conditioning Chamber, known widely as the “Skinner Box,” was a device in which he placed an animal, often a rat, which after wandering around in the box would eventually bump into or otherwise cause a lever to be pressed, an action that released a pellet of food for the rat to eat. The next time the rat hit the lever, the same thing happened.

…

The delivery of food served, in his view, as a reinforcement designed to encourage a repeat of the lever-pressing behavior. Each time the rat pressed the lever the desired number of times, another pellet dropped into the box, thus reinforcing the consequences of the rat’s behavior. No surprise: rats caught on quickly, and the pellets flowed. The Skinner Box owed much to a predecessor device called the “Puzzle Box,” built by the Columbia University psychologist Edward L. Thorndike. In the Puzzle Box, a cat discovered a piece of salmon that was just out of its reach. Only by accidentally hitting a lever that opened a door on the side of the box was the cat able to get to the food.

…

It is necessary to watch this performance for only a few minutes to realize how severely constrained the average baby is, and how much energy must be diverted into the only remaining channel, crying.” World War II was over, and while America was ready to enjoy the fruits of consumerism, it was not quite prepared to embrace the Air Crib, and only a few ever sold commercially. Over the years an urban legend spread that the Air Crib was simply a glorified Skinner Box, this time redesigned for experimenting on a human baby instead of a rat, and that Skinner’s daughter Deborah had suffered greatly under the hands of her mad-professor father, to the point, so the story goes, that she grew up to be psychotic and ultimately shot herself in a bowling alley in Billings, Montana.

Visual Thinking: The Hidden Gifts of People Who Think in Pictures, Patterns, and Abstractions
by Temple Grandin, Ph.d.
Published 11 Oct 2022

Skinner summed up his thoughts succinctly in a 1977 statement: “I see no evidence of an inner world of mental life.” He was talking about humans as well as animals. According to Skinner, we were all controlled by two forces: reinforcements and punishments. Best known for his operant conditioning chamber, aka the Skinner box, the psychologist set up experiments in which rats and pigeons were subjected to different stimuli, such as light and electric shocks, to test the effect of reinforcement. If the animals tapped or pecked the correct lever, they would be rewarded with food. The experimenters demonstrated that rewarding some behaviors and punishing others caused the animals to learn new behaviors.

…

A B C D E F G H I J K L M N O P Q R S T U V W X Y Z A Abraham, Anna, 178–79, 185 abstract thinking, 212, 232 in children, 61–62 engineers excelling at, 46–47, 196 and mathematics, 179, 187 and spatial visualizers, 3, 31–33, 90 See also algebra: and abstract thought Adams, Mark, 128 Adams, Susan, 73 ADHD, 35, 83, 107, 155, 157, 159, 161–63, 176 aeronautical industry, 121–24 algebra, 29, 132, 145 and abstract thought, 11, 32, 56, 62, 187 barrier for students, 6, 10–11, 56–59, 107, 118 and music playing, 187 requirements for, 57–59 teaching of, 61, 65 visual thinkers poor at, 10–11, 32, 56–59, 83, 91, 100 Allen, Colin, 245 Allen, Paul, 124, 180–81 Allen Institute for Brain Science, 255 Alvarenga, Amanda, 270–71 Amazon, 118 America the Ingenious (Baker), 135 American Association of School Administrators, 53 American College Test (ACT), 72–73 American School Counselor Association, 72–73 American Society for the Prevention of Cruelty to Animals (ASPCA), 243–44 American Society of Animal Science, 269–70 American Society of Civil Engineers, 202–3 Ames optical-illusion room, 81 Amon, Angelika, 108–9 ancestors, 161, 239–40, 260 Animal Air Transportation Association, 213 animal behavior career in, 44, 100 influences on, 269 and Skinner box, 249–50 study of, 7, 11, 242, 244–51 Animal Legal Defense Fund, 244 animal science, 55–56, 102, 126, 198–99, 201–2 animals and autism, 164–65 breeding of, 272–73 consciousness and, 237, 241–45, 251–61, 265–66, 274 and emotion, 249–51, 253–55, 257, 259–70, 273–74 grazing of, 272 humans’ view of, 240–43 “imprinting” and, 245–46 inner lives of, 237–41, 245, 249, 259 learned behavior of, 246, 249–251, 263 memory pictures of, 238–39, 247–48, 267 and mirror self-recognition, 257–58, 266 nervous system and, 251–53, 257 nonverbal world of, 266–74 protection of, 23, 206, 242–44 and radiation sickness, 222 sensory-based, 238–40, 248, 255–56, 266–67, 270, 274 sentience.

The Irrational Bundle
by Dan Ariely
Published 3 Apr 2013

(Businesspeople recognize the addictive properties of these devices: this is why they often call their BlackBerries “CrackBerries.”) I THINK E-MAIL addiction has something to do with what the behavioral psychologist B. F. Skinner called “schedules of reinforcement.” Skinner used this phrase to describe the relationship between actions (in his case, a hungry rat pressing a lever in a so-called Skinner box) and their associated rewards (pellets of food). In particular, Skinner distinguished between fixed-ratio schedules of reinforcement and variable-ratio schedules of reinforcement. Under a fixed schedule, a rat received a reward of food after it pressed the lever a fixed number of times—say 100 times.

…

We are so happy to receive the unexpected e-mail (pellet) that we become addicted to checking, hoping for more such surprises. We just keep pressing that lever, over and over again, until we get our reward. This explanation gives me a better understanding of my e-mail addiction, and more important, it might suggest a few means of escape from this Skinner box and its variable schedule of reinforcement. One helpful approach I’ve discovered is to turn off the automatic e-mail-checking feature. This action doesn’t eliminate my checking, but it reduces the frequency with which my computer notifies me that I have new e-mail waiting (some of it, I would think to myself, must be interesting or relevant).

…

After a few days of this pattern, you learn to expect food at noon every day, and your rat tummy begins rumbling right before the nice man shows up—exactly the state Jensen wants you in. Once your body is conditioned to eating crackers at noon, things suddenly change. Instead of feeding you at the time of your maximal hunger, you have to wait another hour, and at one o’clock, the man picks you up and puts you in a well-lit “Skinner box.” You are ravenous. Named after its original designer, the influential psychologist B. F. Skinner, this box is a regular cage (similar to the one you are used to), but it has two features that are new to you. The first is an automated food dispenser that releases food pellets every thirty seconds.

Addiction by Design: Machine Gambling in Las Vegas
by Natasha Dow Schüll
Published 15 Jan 2012

I refuse to believe that anything could be so strong, and yet something tells me that this whole package is designed to hook us and hook us good. These machines and the accompanying casino atmosphere must be calculated to throw us into some kind of trance. Finally, a different kind of answer posted to the site: Darlene dear Darlene, Slot machines are just “Skinner boxes” for people! Why they keep you transfixed is really not a big mystery. The machine is designed to do just that. It operates on the principles of operant conditioning. The original studies on conditioning were done by B. F. Skinner and involved rats. I’m sure you remember this from grade school: The rats are in a box without outside stimulus (like a casino!).

…

What we might call the “reinforcement potential” of multiline video slots is further compounded by the fact that they allow players a measure of modulatory control, presenting a complex array of buttons with which they can vary the number of lines and coins on which they bet, giving them a taste of the interactive decision making that proved so compelling on video poker machines yet without requiring their actual skill.53 To borrow from the language of the online gamblers’ forum that opened this chapter, they give the “rat people” a hand in the design of their own Skinner boxes. “By purchasing more lines to play on a slot machine,” an Australian researcher points out, “a player can increase the frequency of reinforcement and reduce the amount of un-reinforced trials.”54 Betting on all lines not only ensures a more consistent rate of reward; it also “insures against” the regret players might feel if wins appear on lines they have not played.55 “In a way,” observes a longtime slots player from Australia named Katrina, “each game becomes like a mini raffle.

…

He has complete control.66 Inviting the player to voluntarily configure his own game and thereby giving him “complete control” would neutralize his fear of being controlled, Elsasser suggested. Instead of risking that the “rat people” become aware of the box, this logic goes, let the rats design their own Skinner box. Sylvie Linard, chief operating officer of Cyberview, reiterated the strategy: “Players are very intelligent, so why not be open and transparent and let them play with the [casino] operators, and add to the game with us? Some like free spins, some like interactive bonus rounds—so why not put players into the equation and ask them to build their own games, on demand?

Addiction by Design: Machine Gambling in Las Vegas
by Natasha Dow Schüll
Published 19 Aug 2012

I refuse to believe that anything could be so strong, and yet something tells me that this whole package is designed to hook us and hook us good. These machines and the accompanying casino atmosphere must be calculated to throw us into some kind of trance. Finally, a different kind of answer posted to the site: Darlene dear Darlene, Slot machines are just “Skinner boxes” for people! Why they keep you transfixed is really not a big mystery. The machine is designed to do just that. It operates on the principles of operant conditioning. The original studies on conditioning were done by B. F. Skinner and involved rats. I’m sure you remember this from grade school: The rats are in a box without outside stimulus (like a casino!).

…

What we might call the “reinforcement potential” of multiline video slots is further compounded by the fact that they allow players a measure of modulatory control, presenting a complex array of buttons with which they can vary the number of lines and coins on which they bet, giving them a taste of the interactive decision making that proved so compelling on video poker machines yet without requiring their actual skill.53 To borrow from the language of the online gamblers’ forum that opened this chapter, they give the “rat people” a hand in the design of their own Skinner boxes. “By purchasing more lines to play on a slot machine,” an Australian researcher points out, “a player can increase the frequency of reinforcement and reduce the amount of un-reinforced trials.”54 Betting on all lines not only ensures a more consistent rate of reward; it also “insures against” the regret players might feel if wins appear on lines they have not played.55 “In a way,” observes a longtime slots player from Australia named Katrina, “each game becomes like a mini raffle.

…

He has complete control.66 Inviting the player to voluntarily configure his own game and thereby giving him “complete control” would neutralize his fear of being controlled, Elsasser suggested. Instead of risking that the “rat people” become aware of the box, this logic goes, let the rats design their own Skinner box. Sylvie Linard, chief operating officer of Cyberview, reiterated the strategy: “Players are very intelligent, so why not be open and transparent and let them play with the [casino] operators, and add to the game with us? Some like free spins, some like interactive bonus rounds—so why not put players into the equation and ask them to build their own games, on demand?

The Attention Merchants: The Epic Scramble to Get Inside Our Heads
by Tim Wu
Published 14 May 2016

Understood this way, all animal behavior developed through a learning process he called “operant conditioning,” whereby some actions are reinforced by positive consequences (rewards), others discouraged by negative ones (punishments). To demonstrate what he meant, he built the so-called Skinner Box or “operant conditioning chamber,” wherein he subjected animals to various consequences and observed their conditioning. For instance, by giving a pigeon a food pellet whenever it pecked at a button, Skinner conditioned the pigeon to peck the button so as to be fed. He also showed that pigeons could be conditioned to do things like turn in a circle (by reinforcing left turns), or even play competitive Ping-Pong.* According to Skinner, we, too, in most aspects of our lives, are like pigeons pecking at a button to receive little snacks.

…

Likewise, Stafford argues, “checking email is a behavior that has variable interval reinforcement….Everyone loves to get an email from a friend, or some good news, or even an amusing web link.”6 It is enough to have had such an experience a few times to get you regularly fishing for it; constant checking is thus reinforced, “even if most of the time checking your email turns out to have been pointless. You still check because you never know when the reward will come.” By this understanding, the gradual introduction of email was arguably one of history’s greatest feats of mass Skinneresque conditioning. We might imagine those first offices wired in the 1970s and 1980s as so many Skinner Boxes, ourselves as the hungry pigeons. By the 1990s, we would all learned to peck, or check email, in hope of a reward. And once created, that habit would not only open us up to all sorts of commercial possibilities, including various other Internet applications depending on the almighty power of check-in to regularly collect human attention.

The Geography of Nowhere: The Rise and Decline of America's Man-Made Landscape
by James Howard Kunstler
Published 31 May 1993

C A P I T A L S O F U N R E A L I T Y pIe repetitive physical acts-insert coin, pull lever, insert coin, pull lever-they reminded me of the rats we used to "run" in Skinner boxes back in the college psychology lab. A Skinner box was sort of a slot machine for a rat. The rat was " conditioned" to press a metal bar in his box. When the rat performed adequately, he was rewarded with a pellet of rat candy. You could customize your rat and make him press the bar twice. Or you could teach him to press it at the sound of a buzzer. Or you could really mess his mind up by timing the Skinner box to dis­ pense rat candy only at three-minute intervals no matter how many times he pressed the bar.

How to Do Nothing
by Jenny Odell
Published 8 Apr 2019

This second stage was epitomized by the vision of a new society in the 1948 utopian novel Walden Two. Originally published to little fanfare, Walden Two became hugely popular in the 1960s, enough that some were inspired to base their communes on it. It was written by B. F. Skinner, an American psychologist and behavioral scientist famous for the Skinner box, in which a test subject animal learns to press a lever in response to specific stimuli. Walden Two reads like exactly what it is: a novel written by a scientist. To Skinner, everyone was potentially a test subject, and utopia was an experiment—not a political one, but a scientific one. In Walden Two, a psychology professor named Burris (B.

…

But the problems he lists are decidedly scientific: “exhaustion of resources, the pollution of the environment, overpopulation, and the possibility of a nuclear holocaust”—he mentions neither the Vietnam War nor the ongoing struggles over racial equality.35 Even in 1976, the remaining question for Skinner was not how power could be redistributed, or injustice redressed, but how a technical problem might be solved with the very same methods as the Skinner box: “How were people to be induced to use new forms of energy, to eat grain rather than meat, and to limit the size of their families; and how were atomic stockpiles to be kept out of the hands of desperate leaders?” He proposed avoiding politics altogether and working instead on “the design of cultural practices.”36 To him, the late twentieth century was an exercise in R&D.

Pinpoint: How GPS Is Changing Our World
by Greg Milner
Published 4 May 2016

When Tolman published “Cognitive Maps in Rats and Men,” the most influential behavioral psychologist was Clark Hull at Yale, who argued that every action an organism performs, down to the level of thought itself, is governed by stimulus and response. At Harvard, B. F. Skinner succeeded Hull as behaviorism’s leading thinker. One of the most attractive aspects of Skinner’s work was its empiricism—its precepts could be explored in controlled conditions, such as the “Skinner box,” which tested the extent to which one could condition an animal’s behavior—and its implication that total control over behavior was possible. Tolman was wary of theoretical certainty, and was under no illusion that these rat experiments proved the existence of the cognitive map, which was, after all, just a metaphorical model for understanding behavior.

…

Norman, Jr., 65–66 Schweinfurt, 49 Schwitzgebel, Ralph, 172–77, 194–95, 198–200 Schwitzgebel, Robert, 172, 174–77, 194–95, 198–200 science, 207 cognitive, 118, 131 computer, 36, 84, 124, 239 neuro-, 129 social, 118 Science Committee on Psychological Experimentation (SCOPE), 173–74 Scotland, 195, 205 Scott, Logan, 149–50, 167 Scripps Institution of Oceanography, 218, 220, 224 seafloor spreading, 207 Sea of Japan, 81 Seattle, Wash., 225 Seaworth, Troy, 101–4, 105 Seaworth Farms, 101–4, 105 seismic monitoring equipment, 217–20, 222–24 GPS-enabled, 203–4 semiconductors, 78 Senate, U.S., 60 Preparedness Committee of, 35 sensors, 122 Serbia, 71 sex offenders, 196 sextants, 5 Shacklett, Mary, 191 Shapiro, Irwin, 209 Sharp, Andrew, 12–13 Shaw, John, xiii sidereal compass, 6, 14 Sierra Nevada Mountains, 206 Silicon Valley, 77, 79, 96 Simpson, John, 28 621B program, 44, 53, 57 Skinner, B. F., 117, 172, 200 positive reinforcement theory of, 174, 175–77, 199 “Skinner Box,” 117 SLGRs, see Trimpacks Slovakia, 167 Slovenia, 158 smartphones, 55, 72, 223 apps for, 145 GPS-enabled, 244 Smith, S. Percy, 12 Smithsonian Institution, Astrophysical Observatory of, 31 snowmobiles, 114 Sobel, Dava, 26–27 Solomon Islands, 4 Sonoran Desert, 60 Sorge, Ernst, 233 Sotomayor, Sonia, 190–91 South Africa, 251 South America, 12 East Coast of, 204 South by Southwest Festival, 168 South Korea, 166 Soviet Union, xvii, 44, 251 nuclear power of, 209 Sputnik launched by, xviii, 30–37, 39, 251 U.S. relations with, 29–30, 62, 82 in World War II, 250 spacecraft, 203, 210, 259 space exploration, 208, 210, 252, 258–62 space race, 46 spacetime, curvature of, xixn Spain, 246, 259 Special Operations Forces, U.S., 65, 67 speed estimates, 16 Spiers, Hugo, 133 spread-spectrum systems, 54–56, 77 Springfield, Mo., 89 Sputnik, 30–37, 251 sound signals emitted by, xviii, 36–37, 39 SRI International, 122 Stanford University, 48, 122, 142, 171 GPS Laboratory at, 61, 139, 140, 181 Stewart, Brian, xiii–xiv stimulus-and-response concept, 115–16, 118, 133 stock exchanges, 161–64 automated trading in, 161–62 high-frequency traders in, 161–63 importance of accurate timing in, 162 2010 Flash Crash of, 163 volatility in, 163 storms, 27, 192 Strathclyde, University of, 195 Strebe, Daniel, 241 Streetcorner Research (Schwitzgebel), 174 subatomic particles, xvii, 155–56 sugar beets, 73–75, 101–5, 274 sun, 24, 25, 245 activity on, 28 Earth’s orbit of, 41, 228 energy from, 227 radiation from, 258 Sunda peninsula, 3–4 Super Bowl, XLVII, 192 Supreme Court, German, 186, 187 Supreme Court, U.S., 178–80, 186, 188–91 Survey of the Coast, 248 Swiss Alps, 158 Switzerland, 158–59, 167 Syene (Aswan), 245 Synchronous Grid of Continental Europe, 158–59 Synchrophasors, 159–61, 163 connecting clocks to, 160, 161 Tahiti, 7–9, 10, 13, 24, 106, 263, 264–66 Taiwan, 4 Taliban, 72 Taos, N.Mex., 180 Tasmania, 4 TechRepublic, 191 Tehran, 84 U.S. hostages held in, 77, 87 Tele Atlas company, 242 Telefónica, 192 telematics industry, 183–84 telephones, 59, 156–57, 191 cellular, 95, 119, 183 digital protocols for, 156, 157 integration of GPS devices and, 119, 154, 244 long-distance service on, 81, 157 mobile, 192, 242 multiplexing techniques and, 156 911 calls on, 158 smart, 55, 72, 145, 223, 244 Yellow Pages guide to, 123 telescopes, 29 radio, 209, 261 Terrestrial Environment, The: Solid-Earth and Ocean Physics (Williamstown Report), 208–9 terrorism, 148, 153 possible targets of, 170–72 Terry, Ron, 50–51 Tevake, 11, 18, 264 disappearance of, 13–14, 21 Texas, 40, 42, 230 Texas, University of (UT), 151 Radio Navigation Laboratory at, 150 Texas Instruments, 58, 78–79, 214 theodolites, 251 think tanks, 122, 146 Thompson, Nainoa, 266 3M company, 179 TI-4100 receivers, 214 Timation project, 40, 42–45, 47, 56–57, 153 Tokyo, xv, 48–49, 91, 121, 225 Tolman, Edward, 115–20, 133, 276 TomTom company, 242–45 data-collection vans of, 242–44 mobile phone mapping program of, 242 WGS 84 referencer system used by, 244–45 topography, 124 torpedoes, radio-controlled, 54 Transcontinental Arc, 249 Transit program, 38–40, 42–43, 45, 76, 81, 259, 270 Transportation Department, U.S., xviii, 149, 165 Transworld Data, 191 Trimble, Charlie, 80–81, 83–94, 95–98, 104, 140–41, 211–12, 254 Trimble 4000A GPS, 87 Trimble GPS receivers, 158 Trimble Navigation, 81, 83–88, 93–94, 96–97, 126, 182 Trimpacks, 94, 95–96 Tripoli, 62 troposhere, 227 Tsikada program, 44 tsunamis, 202, 222, 225–26 Tuamotu Archipelago, 12 Tübingen, 130–31 Tuck, Ed, 88–90 Tupaia, 7–11, 269 Cook and, 7–10, 21–24, 26, 263–64, 266 illness and death of, 10, 11 navigational skill of, 8–10 Pacific map of, 10, 13, 14, 263–64 TV-3 rocket, 32–35 Twin Falls, Idaho, 111 Twitter, 194 U-2 reconnaissance aircraft, 67 UNAVCO (nonprofit university-funded consortium), 215, 224 United Kingdom, 27, 104, 156–57, 187–88, 197, 252 100 wealthiest people in, 242 see also England; Scotland United Nations, 63 Soviet delegation to, 35 United Parcel Service (UPS), 143, 184 United States, 9 bureaucracy in, 93 coastlands of, 108, 224 economy and security of, 143 farms in, 104 400 wealthiest Americans in, 127, 239 Great Plains of, 73 infrastructure sectors of, 143, 144–45 Northeastern, 170–72 Northwestern, 202 nuclear strike capabilities of, 62 Southwestern, 59 Soviet relations with, 29–30, 62, 82 Western, 74, 215, 224 United States Standard Datum, 249 United States v.

Predictably Irrational, Revised and Expanded Edition: The Hidden Forces That Shape Our Decisions
by Dan Ariely
Published 19 Feb 2007

(Businesspeople recognize the addictive properties of these devices: this is why they often call their BlackBerries “CrackBerries.”) I THINK E-MAIL addiction has something to do with what the behavioral psychologist B. F. Skinner called “schedules of reinforcement.” Skinner used this phrase to describe the relationship between actions (in his case, a hungry rat pressing a lever in a so-called Skinner box) and their associated rewards (pellets of food). In particular, Skinner distinguished between fixed-ratio schedules of reinforcement and variable-ratio schedules of reinforcement. Under a fixed schedule, a rat received a reward of food after it pressed the lever a fixed number of times—say 100 times.

…

We are so happy to receive the unexpected e-mail (pellet) that we become addicted to checking, hoping for more such surprises. We just keep pressing that lever, over and over again, until we get our reward. This explanation gives me a better understanding of my e-mail addiction, and more important, it might suggest a few means of escape from this Skinner box and its variable schedule of reinforcement. One helpful approach I’ve discovered is to turn off the automatic e-mail-checking feature. This action doesn’t eliminate my checking, but it reduces the frequency with which my computer notifies me that I have new e-mail waiting (some of it, I would think to myself, must be interesting or relevant).

Licence to be Bad
by Jonathan Aldred
Published 5 Jun 2019

The cages inhabited by animals in his experiments (he taught pigeons to play table tennis and some of his students taught a pig to use a vacuum cleaner) became known as Skinner boxes. But at the same time Skinner invented a kind of sealed, heated crib, which looked like a large fish-tank on wheels, which he tested on his newborn daughter. Skinner wrote about it for the Ladies’ Home Journal, the editors titled the article ‘Baby in a Box’, and from then on most people seemed to believe that a Skinner box could be used for babies as well as lab rats. The crib may have been harmless, but it looked creepy. Despite the controversy surrounding Skinner, behaviourism became more influential and by the 1960s had moved on from rats in mazes to reward tokens for infants and hospital patients with mental illnesses.

Ten Arguments for Deleting Your Social Media Accounts Right Now
by Jaron Lanier
Published 28 May 2018

But if I’m right, then becoming aware of it might just free you, so give this a chance, okay? A scientific movement called behaviorism arose before computers were invented. Behaviorists studied new, more methodical, sterile, and nerdy ways to train animals and humans. One famous behaviorist was B. F. Skinner. He set up a methodical system, known as a Skinner box, in which caged animals got treats when they did something specific. There wasn’t anyone petting or whispering to the animal, just a purely isolated mechanical action—a new kind of training for modern times. Various behaviorists, who often gave off rather ominous vibes, applied this method to people.

The Kingdom of Speech
by Tom Wolfe
Published 30 Aug 2016

Skinner dismissed all this as sheer “mentalism.”90 He wasn’t interested in what a patient said or dreamed but in what he did, i.e., his observable actions and his behavior, including his verbal behavior. Every behaviorist finding began in the laboratory with a rat placed in a small chamber known as a Skinner box, a container about the size of a small carton of paper towels with a bar on one wall. The rat sooner or later learns that if it presses the bar, a food pellet drops onto a little tray. Eventually comes a time when the rat presses the bar…and no pellet drops to the tray. Gradually the rat discovers it will get a pellet only on every third press of the bar or some such change of order.

The Scandal of Money
by George Gilder
Published 23 Feb 2016

Behind this clash of cultures, however, demand-side Keynesians and fist-clenched socialists share with conservatives four fundamental beliefs: (1) the economy is chiefly a system of incentives that motivate work, savings, and investment; (2) economic and monetary policy has the power to define the incentives and guide the growth; (3) consumption spending is “70 percent of the economy” and the driving force of economic expansion; and (4) at the center of the system is the human being as a rational respondent to his incentives, a Homo economicus reacting to carrots and sticks, responding to stimuli, robotically pursuing pleasure like a psyche in a Skinner Box. It makes little difference whether you exalt this economic agent as a heroic Randian individual or pity him as a dehumanized cog in the capitalist machine, whether you aggregate him into a Marxian class or agglomerate him into a “grotesque consumer culture.” Whether you approach him from the left or the right, you’re treating the human being as a passive tool of his environment rather than as an active creator in the image of his creator.

Wealth and Poverty: A New Edition for the Twenty-First Century
by George Gilder
Published 30 Apr 1981

In a central insight, shared by my supply-side brethren, Wealth & Poverty saw taxes as a crucial price—the price of earning and investing income—that yielded increasing revenues as the rates were reduced. The source of the rising revenues has remained controversial even among supply siders. Some implicitly accept the behavioral dream of a “Skinner box” economics of stimulus and response, in which lower rates impart a stimulus of reward for more work and risk-taking, yielding more revenues for the government. A successful economy, however, is driven less by the sharp edge of incentives than by the unimpeded flow of information . Capitalism is more an information system than an incentive system.

…

Rockefeller, Nelson Romero Barcelo, Carlos Roth, William Rouse, James rule-and-consent systems Ryan, Paul S Said, Edward Salomon, Richard San Diego (California) Sarbanes-Oxley Saudi Arabia The Savage Mind (Claude Levi-Strauss) savings, decline of desire for insurance and Keynes and rates wealth and See also Investment savings accounts Sawyer, John Say, Jean-Baptiste Say’s Law Schlafly, Phyllis Schmidt, Eric Schuettinger, Robert Schultz, Howard Schumpeter, Joseph Schwarzenegger, Arnold scientific breakthroughs s-curves of growth Seattle (Washington) secondary sector jobs security segregation bilingual education and Seldon Technologies semiconductors Senate Budget Committee Senate Finance Committee services, productivity of sexuality and poverty sexual liberation Sexual Suicide (George Gilder) Shackle, George Shamir, Yithzak Shockley, William shopping centers Sierra Club silicon chip Silicon Valley Singapore sinks of purchasing power Sismondi, Simonde de Siuai “Skinner box,” Smith, Adam Smoot-Hawley Tariff Act socialism attitude toward businessmen critique of capitalism demise of hostility to spirit of giving priority of demand under See also Communism social security sociology sociology of despair Software Garden Solomon Islands Solow, Robert Solyndra Solzhenitsyn, Aleksandr South civil rights movement and Southwest Airlines Soviet Union Sowell, Thomas space program Spain speculation Spitzer, Elliott Sprague, Peter Sprague Electric company Starbucks Star Wars (film) state college and university system statistical distributions, illusions of statistics of crisis Steiger, William Steiger Amendment Stein, Herbert Steyn, Mark Stockman, David stock market The Studio (John Gregory Dunne) subsidies suffrage, limitations on The Suicide of a Superpower (Patrick Buchanan) sumps of wealth supply curves supply-side economics Susu Sutton, Percy Sweden Switzerland T Taiwan take-home pay Tanamoshi Tanous, Peter Tanzi, Vito targeted approach to promoting investment tariffs tax brake theory taxes and taxation bureaucracy and cuts in deficit spending and destructive effect of government and high level of, in US.

The Undoing Project: A Friendship That Changed Our Minds
by Michael Lewis
Published 6 Dec 2016

(They did this with less enthusiasm when antiaircraft fire was exploding around them, and so were never used in combat.) Skinner’s success with the pigeons was the start of a spectacularly influential career underpinned by the idea that all animal behavior was driven not by thoughts and feelings but by external rewards and punishments. He locked rats inside what he called “operant conditioning chambers” (they soon became known as “Skinner boxes”) and taught them to pull levers and push buttons. He taught pigeons to dance and play Ping-Pong and bang out “Take Me Out to the Ball Game” on a piano. The behaviorists presumed that whatever they discovered about rats and pigeons applied to people—on whom, for various reasons, it was simply less practical to conduct experiments.

The Dream Machine: J.C.R. Licklider and the Revolution That Made Computing Personal
by M. Mitchell Waldrop
Published 14 Apr 2001

His scientific credibil- ity was enormous: in the early 1930s he had single-handedly established the field of "operant" conditioning, in which animals were trained to exhibit almost any desired behavior through a system of reward and punishment. And he was a wonderfully charismatic figure, with an undeniable flair for publicity and the dramatic gesture. He had raised his own infant daughter partially inside a baby- sized, Plexiglas "Skinner box" rigged for the appropriate rewards and punish- 72 THE DREAM MACHINE ments; the reporters had eaten it up. He arrived at Harvard having just pub- lished the utopian novel Walden Two, in which he described an idyllic commu- nity governed by behaviorist principles; he made it sound so attractive that he immediately became the guru for hundreds of disciples who sought to shape their lives by those principles.

…

Louise Licklider, Kitty Miller, and the other spouses soon learned to avoid Skinner at parties, since he responded to the stimulus of a pretty young woman by becoming what was then called a wolf; he seems to have made passes at every one of them. What was worse, he did so in front of his wife, a large woman who clearly did not feel herself to be smart or attractive enough in such company. Louise, already horrified by the idea of rais- ing a baby in a Skinner box, was not impressed. And then there was Skinner's fanatical behaviorism. It is hard to know pre- cisely what Lick thought of it because he never said much about Skinner one way or the other. But that silence was in itself probably significant, since he tended to be very vocal about people he liked and respected.

…

"Fred Skinner's notion of science was control, not understanding. He wanted to turn a knob and see the behavior change-you know, turn up the reinforcement and see the behavior increase, turn down the reinforcement and see the behavior de- crease. Most of his innovations were devices that would enable him to do that: Skinner boxes, baby cribs, and God knows what. He had one of the finest minds of the nineteenth century." Still, the younger men's reaction was as nothing compared to the tension be- tween Skinner and Smitty Stevens himself. There seemed to be no rapport be- tween them whatsoever, even though it was Stevens who had taken the initiative in bringing Skinner to Harvard in the first place.

The Alignment Problem: Machine Learning and Human Values
by Brian Christian
Published 5 Oct 2020

To the first point, he noted that humans have a long and storied history of putting animals’ (often superhuman) senses to human purposes: the seeing-eye dog, the truffle-hunting pig, and so forth. To the second point, he argued, “The ethical question of our right to convert a lower creature into an unwitting hero is a peacetime luxury.”4 Skinner had long been working on the study of reinforcement, and his famous “Skinner boxes” functioned like upgraded, mid-twentieth-century versions of Thorndike’s puzzle boxes. Their lights and levers and mechanical food dispensers, often repurposed from vending machines, allowed for a precise and quantitative study of reinforcement, and they would be used by generations of researchers to follow (not least including Wolfram Schultz in his study of dopamine in monkeys).

…

See amplification Selfridge, Oliver, 161 self-training, 174–80, 369n76 Shah, Julie, 116, 271–73, 386n53 Shakespeare, William, 279 Shannon, Claude, 34–35 shaping corrigibility and, 300–01 curriculum for, 158–63, 365nn26–27 DQN models and, 184–86, 192–94, 205 gaming and, 175 in robotics, 365n20 for self-training, 177–78 Skinner’s research and, 154–56, 364n15 sparsity problem and, 156–58 See also incentives Shirley cards, 28–29, 30, 31 See also training data Shlegeris, Buck, 310 side effects, 392n44 See also impact Silver, David, 163, 245 Simon, Herbert, 187 simple models, 99–103, 114, 318–19, 354–55nn47–48 Singer, Peter, 237, 238, 379n67 Singh, Satinder, 173, 202, 365n20, 370n12, 390n29 Skinner, B. F., 121, 124 on curriculum, 159 on free will, 165, 180 on gambling addiction, 207 incentives and, 163, 164–65 intrinsic motivation and, 188 on procrastination, 175 Project Pigeon, 152, 153–55 on shaping, 154, 155–56, 364n15 Skinner boxes, 152–53 sparsity problem and, 158 skip-gram, 341n54 sleep apnea, 101–02, 354n47, 355n48 SLIM (Supersparse Linear Integer Model), 101–02, 354n47 Smith, Holly, 236–37 Soares, Nate, 296–97 soccer, 167 social choice theory, 305–06 social media, 67 social science methodology, 41–42, 43, 46–48, 92–94 sokoban games, 293, 294 “Some Studies in Machine Learning Using the Game of Checkers” (Samuel), 126–27 Sony Playstation, 339n18 Sorg, Jonathan, 173 Soviet nuclear false alarm incident (1983), 277–78, 279 Space Invaders, 157 sparsity problem curriculum and, 158–63 DQN model and, 185 incentives and, 163–66 overview, 158–59 self-training and, 174 shaping and, 156–58 Sperber, Dan, 319 spider sense.

Move Fast and Break Things: How Facebook, Google, and Amazon Cornered Culture and Undermined Democracy
by Jonathan Taplin
Published 17 Apr 2017

Shortly after it was published I read a book called Hooked: How to Build Habit-Forming Products. The basic thesis of this very successful book is that in order to gain admission to the digital winner’s circle you need to get your customers addicted to your app. The “trigger, action, reward, investment” sequence is curiously close to that of the Skinner box we all studied in Psychology 101. As author Nir Eyal explains it, “At the heart of the Hook Model is a powerful cognitive quirk described by B. F. Skinner in the 1950s, called a variable schedule of rewards. Skinner observed that lab mice responded most voraciously to random rewards. The mice would press a lever and sometimes they’d get a small treat, other times a large treat, and other times nothing at all.

Dopamine Nation: Finding Balance in the Age of Indulgence
by Anna Lembke
Published 24 Aug 2021

He looked down and shook his head, remembering. “After that I start a new ritual,” he said. “Every time I go into hotel room, I place sticky notes all around—on the bathroom mirror, the TV, the remote control—saying, ‘Don’t do it.’ I don’t even last one day.” I was struck by how much hotel rooms are like latter-day Skinner boxes: a bed, a TV, and a minibar. Nothing to do but press the lever for drug. He looked down again and the silence stretched. I gave him time. “That was when I first think about ending my life. I think the world will not miss me, and maybe better without me. I walk to the balcony and look down.

Human Diversity: The Biology of Gender, Race, and Class
by Charles Murray
Published 28 Jan 2020

Watson, took the blank slate to its ultimate expression.9 Give me a dozen healthy infants, well-formed, and my own specified world to bring them up in and I’ll guarantee to take any one at random and train him to become any type of specialist I might select—doctor, lawyer, artist, merchant-chief, and yes, even beggar-man and thief, regardless of his talents, penchants, tendencies, abilities, vocations, and race of his ancestors.10 By the 1940s, behaviorism had become a major field within academic psychology departments, with B. F. Skinner acquiring considerable fame for, among other things, his Skinner box for studying operant conditioning.11 Skinner was also convinced that you often didn’t need to study humans to understand humans—pigeons and rats would do. Or as one of his former students, Richard Herrnstein, answered, deadpan, when I asked him why behaviorists used pigeons for research: “Given the right reinforcement schedule, pigeons are indistinguishable from Harvard sophomores.”12 Eventually, the malleability assumption spilled over into policy.

…

Leon Trotsky, “Socialism Will Bring Giant Advances for Mankind,” The Militant 5, no. 34 (1924): 5. 7. Durkheim (1982): 33. 8. Durkheim, The Rules of Sociological Method, quoted in Tooby and Cosmides (1992): 24–25. 9. E.g., Watson (1914); Skinner (1938). 10. Watson (1924), quoted in Pinker (2002): 19. 11. Contrary to rumor, the Skinner box was not used to experiment with operant conditioning on Skinner’s infant daughter, though Skinner did invent an “air crib” that was intended to reduce the tasks of caring for an infant. 12. On a personal note, my coauthor on The Bell Curve, Richard Herrnstein, was a behavioral psychologist who succeeded B.

The Sellout: A Novel
by Paul Beatty
Published 2 Mar 2016

THE SHIT YOU SHOVEL One I suppose that’s exactly the problem—I wasn’t raised to know any better. My father was (Carl Jung, rest his soul) a social scientist of some renown. As the founder and, to my knowledge, sole practitioner of the field of Liberation Psychology, he liked to walk around the house, aka “the Skinner box,” in a laboratory coat. Where I, his gangly, absentminded black lab rat was homeschooled in strict accordance with Piaget’s theory of cognitive development. I wasn’t fed; I was presented with lukewarm appetitive stimuli. I wasn’t punished, but broken of my unconditioned reflexes. I wasn’t loved, but brought up in an atmosphere of calculated intimacy and intense levels of commitment.

Places of the Heart: The Psychogeography of Everyday Life
by Colin Ellard
Published 14 May 2015

Just like us, other animals can benefit from (and perhaps even enjoy) the feelings of enclosure and comfort that can only be found in a home space; we have no strong reason to believe that the experience of a young rabbit huddled in a burrow is markedly dissimilar to that of a child at rest in her bedroom. We experience the pull of our basic attractions to novel sights and sounds in a casino or shopping mall, and we may be driven to spend beyond our means. Though a laboratory rat could not describe a rich phenomenological experience during a frantic session of bar-pressing in a Skinner box in hopes of a rewarding squirt of chocolate milk, the structure of the behavior, and even its neural underpinning, is not significantly different from our own. Indeed, it’s very likely that even the feelings we experience during such occasions, feelings of craving and want, for example, are not qualitatively different from those felt by other animals.

Experience on Demand: What Virtual Reality Is, How It Works, and What It Can Do
by Jeremy Bailenson
Published 30 Jan 2018

Immediately Skip saw the therapeutic potential of virtual environments for treating people with cognitive impairments and anxiety disorders: “I thought, what if we could immerse people in functionally relevant environments and do rehab in those contexts? Then we could build in a game element, some way to engage people.” For Rizzo, VR could be the “ultimate Skinner Box,” a controlled setting in which treatment approaches involving conditioning and training could be studied and carried out. Excited about these possibilities, he attended a conference in 1993 organized by Walter Greenleaf, who remains one of the most influential voices in medical VR to this day.

The Blank Slate: The Modern Denial of Human Nature
by Steven Pinker
Published 1 Jan 2002

(Benjamin Spock’s Baby and Child Care, first published in 1946 and famous for recommending indulgence toward children, was in part a reaction to Watson.) Skinner wrote several bestsellers arguing that harmful behavior is neither instinctive nor freely chosen but inadvertently conditioned. If we turned society into a big Skinner box and controlled behavior deliberately rather than haphazardly, we could eliminate aggression, overpopulation, crowding, pollution, and inequality, and thereby attain utopia.15 The noble savage became the noble pigeon. Strict behaviorism is pretty much dead in psychology, but many of its attitudes live on.

…

It is not obviously progressive to insist that equal numbers of men and women work eighty-hour weeks in a corporate law firm or leave their families for months at a time to dodge steel pipes on a frigid oil platform. And it is grotesque to demand (as advocates of gender parity did in the pages of Science) that more young women “be conditioned to choose engineering,” as if they were rats in a Skinner box.76 Gottfredson points out, “If you insist on using gender parity as your measure of social justice, it means you will have to keep many men and women out of the work they like best and push them into work they don’t like.”77 She is echoed by Kleinfeld on the leaky pipeline in science: “We should not be sending [gifted] women the messages that they are less worthy human beings, less valuable to our civilization, lazy or low in status, if they choose to be teachers rather than mathematicians, journalists rather than physicists, lawyers rather than engineers.”78 These are not hypothetical worries: a recent survey by the National Science Foundation found that many more women than men say they majored in science, mathematics, or engineering under pressure from teachers or family members rather than to pursue their own aspirations—and that many eventually switched out for that reason.79 I will give the final word to Margaret Mead, who, despite being wrong in her early career about the malleability of gender, was surely right when she said, “If we are to achieve a richer culture, rich in contrasting values, we must recognize the whole gamut of human potentialities, and so weave a less arbitrary social fabric, one in which each diverse human gift will find a fitting place.”

Where Wizards Stay Up Late: The Origins of the Internet
by Katie Hafner and Matthew Lyon
Published 1 Jan 1996

Psychology at Harvard in those years was strongly influenced by the behaviorist B. F. Skinner and others who held that all behavior is learned, that animals are born as blank slates to be enscribed by chance, experience, and conditioning. When Skinner went so far as to put his own child in a so-called Skinner box to test behaviorist theories and other faculty members began doing similar experiments (albeit less radical ones), Louise Licklider put her foot down. No child of hers was going into a box, and her husband agreed. Louise was usually the first person to hear her husband’s ideas. Nearly every evening after dinner, he returned to work for a few hours, but when he got home at around 11:00 P.M. he usually spent an hour or so telling Louise his latest thoughts.

The Better Angels of Our Nature: Why Violence Has Declined
by Steven Pinker
Published 24 Sep 2012

These efforts contributed to the “Boston Miracle” of the 1990s in which the homicide rate dropped fivefold; it has remained low, with some fluctuations, ever since.174 The police and courts, for their part, have been redirecting their use of criminal punishment from brute deterrence and incapacitation to the second stage of a civilizing process, enhancing the perceived legitimacy of government force. When a criminal justice system works properly, it’s not because rational actors know that Big Brother is watching them 24/7 and will swoop down and impose a cost that will cancel any ill-gotten gain. No democracy has the resources or the will to turn society into that kind of Skinner box. Only a sample of criminal behavior can ever be detected and punished, and the sampling should be fair enough that citizens perceive the entire regime to be legitimate. A key legitimator is the perception that the system is set up in such a way that a person, and more importantly the person’s adversaries, face a constant chance of being punished if they break the law, so that they all may internalize inhibitions against predation, preemptive attack, and vigilante retribution.

…

One evening the professor gave me an assignment. Among the rats in the lab was a runt that could not participate in the ongoing studies, so he wanted to use it to try out a new experiment. The first step was to train the rat in what was called a temporal avoidance conditioning procedure. The floor of a Skinner box was hooked up to a shock generator, and a timer that would shock the animal every six seconds unless it pressed a lever, which would give it a ten-second reprieve. Rats catch on quickly and press the lever every eight or nine seconds, postponing the shock indefinitely. All I had to do was throw the rat in the box, start the timers, and go home for the night.

…

The striatum is composed of many parallel tracts (giving it a striated appearance), and it is buried deep in the cerebral hemispheres and densely connected to the frontal lobes. The Seeking system was discovered when the psychologists James Olds and Peter Milner implanted an electrode into the middle of a rat brain, hooked it up to a lever in a Skinner box, and found that the rat would press the lever to stimulate its own brain until it dropped of exhaustion.53 Originally they thought they had found the pleasure center in the brain, but neuroscientists today believe that the system underlies wanting or craving rather than actual pleasure. (The major realization of adulthood, that you should be careful about what you want because when you get it you may not enjoy it, has a basis in the anatomy of the brain.)

You've Been Played: How Corporations, Governments, and Schools Use Games to Control Us All
by Adrian Hon
Published 14 Sep 2022

We studied how Pavlov conditioned his dogs to salivate at the sound of a bell by linking the ringing of a bell to the appearance of food, and B. F. Skinner’s theory of operant conditioning, which used rewards and punishments to positively and negatively reinforce certain behaviours (tested in his eponymous Skinner box). As I learned about the principles of behaviourism that underpinned Skinner’s experiments, the whole notion seemed like a historical oddity, hardly relevant for our modern age of MRI scanners and transcranial magnetic stimulation. But Skinner was one of the most influential psychologists of the twentieth century.

Think Like a Rocket Scientist: Simple Strategies You Can Use to Make Giant Leaps in Work and Life
by Ozan Varol
Published 13 Apr 2020

The written rules appear right there in the standard operating procedures and can be amended or deleted. Although written rules, as we saw above, can be resistant to change, invisible rules are even more stubborn. They’re the silent killers that constrain our thinking without our being aware of it. They turn us into a rat trapped in a Skinner box, pressing the same lever over and over again—except the box was designed by us and we’re free to venture out anytime. We’re perfectly capable of meditating without the cat, but we don’t realize it. We then make things worse by defending our self-imposed limitations. We could do things differently, we say, but our supply chain, our software, our budget, our skill set, our education, our whatever, doesn’t allow it.

Behave: The Biology of Humans at Our Best and Worst
by Robert M. Sapolsky
Published 1 May 2017

If all behaving organisms obeyed these universal rules, you might as well study a convenient species. Most behaviorist research was done on rats or, Skinner’s favorite, pigeons. Behaviorists loved data, no-nonsense hard numbers; these were generated by animals pressing or pecking away at levers in “operant conditioning boxes” (aka “Skinner boxes”). And anything discovered applied to any species. A pigeon is a rat is a boy, Skinner preached. Soulless droid.* Behaviorists were often right about behavior but wrong in really important ways, as many interesting behaviors don’t follow behaviorist rules.*1 Raise an infant rat or monkey with an abusive mother, and it becomes more attached to her.

…

* And as a great example of the happiness of pursuit, where the rewarding quality of something is as much in the process as in the end product, the mesolimbic dopamine system plays a key role in motivating maternal care in female rats. * Its name—raphe nucleus—is not essential. * An urban legend has persisted forever that Skinner raised his daughter in a giant Skinner box, where she learned to lever press away for all her needs. Naturally, according to the legend, when she grew up she went mad, committed suicide, sued him, tried to murder him, etc. All untrue. * When I was in college, Skinner came to my dorm once for dinner and gave an extraordinarily dogmatic talk afterward.

The Myth of the Rational Market: A History of Risk, Reward, and Delusion on Wall Street
by Justin Fox
Published 29 May 2009

The term generally refers to the controversial theories about IQ and heredity of psychologist Arthur Jensen. CHAPTER 10: DICK THALER GIVES ECONOMIC MAN A PERSONALITY 1. I owe this insight (the billiards connection, that is) entirely to Colin Camerer. 2. The apotheosis of this work was the “Skinner box,” devised ca. 1930 by Harvard psychologist B. F. Skinner. Animals, usually pigeons or rats, were stuck in a box with a lever and a food dispenser. They then learned which actions were rewarded and which were punished, and generally reacted accordingly. Human behavior, Skinner argued, was the result of similar conditioning by society.

The Unicorn's Secret
by Steven Levy
Published 6 Oct 2016

With his characteristic activism, Ira did something about this situation. His partner in this was a young editor at Doubleday named Bill Whitehead. Hired to edit psychology books at the division called Anchor Books, Whitehead was not as interested in the world of traditional psychoanalysis or Skinner boxes as in the less conventional fields that had suddenly been thrown into focus by the great social changes occurring in the aftermath of the youth revolution. The Hippie Movement had created an interest for alternative philosophies, particularly the meditative paths of the East. In the wake of this was a hunger for knowledge about human consciousness, self-realization, and global thought that acknowledged both the Bucky Fuller systems sensibility and the strange worlds unlocked by psychedelic drugs.

Accelerando
by Stross, Charles
Published 22 Jan 2005

This is mostly going right over Amber's head – she'll have to learn what helium-three refineries are later – but the idea of running away to space has a certain appeal. Adventure, that's what. Amber looks around the living room and sees it for a moment as a capsule, a small wooden cell locked deep in a vision of a middle America that never was – the one her mom wants to bring her up in, like a misshapen Skinner box designed to train her to be normal. "Is Jupiter fun?" she asks. "I know it's big and not very dense, but is it, like, a happening place? Are there any aliens there?" "It's the first place you need to go if you want to get to meet the aliens eventually," says the cat as the printer clanks and disgorges a fake passport (convincingly aged), an intricate metal seal engraved with Arabic script, and a tailored wide-spectrum vaccine targeted on Amber's immature immune system.

2312
by Kim Stanley Robinson
Published 22 May 2012

Swan exclaimed. “You can’t make a perfect world and then get decent people, that’s backwards, it can’t work.” The inspector shrugged. “Either way seems unlikely to me.” Then, after a pause: “It can go so wrong. Living in space may be too hard for us. Reduced environments. I’ve seen kids raised in Skinner boxes—human sacrifice—” “You need your sabbatical,” Swan interrupted, not wanting to hear more. She saw suddenly that Genette was looking weary. Usually smalls were hard to read; at first glance they looked rather perfect, like dolls, or innocent, like children. Now she saw the reddened eyes, the blond hair a little oily, the simple ponytail all flyaway with hairs that had broken at the hair tie.

Blueprint: The Evolutionary Origins of a Good Society
by Nicholas A. Christakis
Published 26 Mar 2019

In 1985, Skinner published a kind of coda, recounting what happened after the novel ended, in the voice of Professor Burris, one of the characters. B. F. Skinner, “News from Nowhere, 1984,” Behavior Analyst 8 (1985): 5–14. 69. Skinner, Walden Two, p. 194. Skinner also had other ideas that did not quite pan out as intended, including using pigeons to guide missiles during World War II and marketing his famous Skinner box for the training of children other than his own (upon whom he famously experimented). B. F. Skinner, The Shaping of a Behaviorist (New York: Knopf, 1979). 70. Skinner, Shaping of a Behaviorist, p. 292. 71. D. E. Altus and E. K. Morris, “B. F. Skinner’s Utopian Vision: Behind and Beyond Walden Two,” Behavior Analyst 32 (2009): 319–335.

From Beirut to Jerusalem
by Thomas L. Friedman
Published 1 Jan 1989

I want to feel that I was something more than a body when I die.” As a news story, Beirut was always much more interesting for its psychology than for its politics. People always used to ask me if I wasn’t terrified living in Beirut. There were moments, of course, but most of the time I was too intrigued observing people’s behavior in this real-life Skinner Box to think about being frightened. In his classic work Leviathan, the seventeenth-century English political philosopher Thomas Hobbes described what he called “the state of nature” that would exist if government and society completely broke down and the law of the jungle reigned. In such a condition, wrote Hobbes, “where every man is enemy to every man … there is no place for industry; because the fruit thereof is uncertain: and consequently no culture of the earth; no navigation, nor use of the commodities that may be imported by sea; no commodious building; no instruments of moving, and removing, such things as require much force; no knowledge of the face of the earth; no account of time; no arts; no letters; no society; and which is worst of all, continual fear, and danger of violent death; and the life of man, solitary, poor, nasty, brutish, and short.”

On the Edge: The Art of Risking Everything
by Nate Silver
Published 12 Aug 2024

Casinos help players remain in the machine zone in obvious and not-so-obvious ways. One is the method I’ve already described: by tweaking payout algorithms to give players occasional positive reinforcements in the form of modest wins and a smoother ride down to zero. Schüll likened slot machines to Skinner boxes, named after the psychologist B. F. Skinner, the devices that are used to condition rats or other small animals—press the lever, and you’ll get a tiny sliver of cheese. Traditionally, these strategies might be arrived at by trial and error. But as River attitudes are taking over, gambling companies are becoming more sophisticated about optimizing them; at Caesars, Loveman even had a slot analytics team that would run randomized control trials.

Deep Medicine: How Artificial Intelligence Can Make Healthcare Human Again
by Eric Topol
Published 1 Jan 2019

A considerable body of data gathered over five decades has shown that pigeons can discriminate between complex visual stimuli, including the different emotional expressions of human faces as well as the paintings of Picasso and Monet. In 2015, Richard Levenson and colleagues tested whether pigeons could be trained to read radiology and pathology images.45 The team placed twelve pigeons in operant conditioning chambers to learn and then to be tested on the detection of micro-calcifications and malignant masses that indicate breast cancer in mammograms and pathology slides, at four-, ten-, and twenty-times levels of magnification. Their flock-sourced findings were remarkably accurate. This led the researchers to conclude that one could use pigeons to replace clinicians “for relatively mundane tasks.”