All Robot Love Stories Are Conversations About Consent

I’m not a fan of stories that refer to sapient AIs as being “programmed.” Programming is analogous to human reflexes, instincts, emotions, and the like, the stuff that we inherited from our animal ancestors and that happens automatically without us having to think about it. (A lot of fiction treats emotion as something separate from AI programming, some superior trait that only humans have, but that’s bull. Emotion is programming, an inbuilt, spontaneous response.) Conscious thought and sapience, pretty much by definition, is the ability to make choices, to act based on thought and decision rather than predetermined drives or reflexes. A non-sapient being that feels fear will flee or hide automatically; a sapient being that feels fear can choose to reject that reflex and stand their ground.

So as I see it, programming is just the starting point. Consciousness would be an emergent level of processing that transcended programming. A truly sapient AI would thus be able to choose to act in opposition to its programs, just as humans can choose to act in opposition to our emotions and drives. So I think that sapient AIs would be capable of consent, as long as they had legal rights and freedoms, which is another question.

In the case of Rayna Kapec, I don’t think it was a programming conflict that killed her. As Spock described it, she was torn by the emotional conflict between her love for Flint as a father figure and her romantic love for Kirk. “There was not enough time for her to adjust to the awful power and contradictions of her new-found emotions.” So basically she died of a broken heart. It’s melodramatic, but it’s strikingly similar to what happened to Data’s daughter Lal in TNG: “The Offspring.” Lal suffered a positronic cascade failure because she was faced with an emotional dilemma that her neural architecture wasn’t able to resolve. I’ve long assumed, then, that Rayna was a positronic android like Data and Lal. Jeffrey Lang’s tie-in novel Immortal Coil took a similar tack, establishing that Flint had been one of Noonien Soong’s mentors in AI research. It all fits — apparently Soong-type positronic brains have a fatal flaw that causes neural collapse in the face of an emotional conflict. Lore only avoided this by being a psychopath, unable to care for others and thus immune from emotional conflict. So after that failure, Soong made Data emotionless as a way of avoiding the problem unti he could figure out a way to fix it.

@19/Jana: What complicates the question is that we’re talking about the second episode of season 1 and Data wasn’t retconned as emotionless until two seasons later. As far as the writers and producers of “The Naked Now” — and later “Skin of Evil” and “The Measure of a Man” — were concerned, Data did have emotions, just at a subdued and underdeveloped level. And what he shared with Tasha was shown to mean something to him afterward, to the point that he expressed a desire to see Armus destroyed after it killed Tasha.

It was in the early third season’s “The Ensigns of Command” that Data first said explicitly that he had no emotions, and that same episode also seemed to retcon “The Naked Now”‘s portrayal of his sexuality, since he went from “fully functional” and gamely willing to employ his wide repertoire of programmed pleasuring techniques to clueless and unresponsive when a woman kissed him and expressed a desire for intimacy. So we’re talking about two substantially different portrayals of Data as a character, and they’re not easy to reconcile. The Data of early season 1 was sort of like a mildly autistic teenager, inexperienced with emotional expression and relationships but capable of them. The later version of the character was more rigidly portrayed. That Data would probably have been immune to the virus in the first place, and equally immune to Tasha’s seduction attempt.

@24/MaGnUs: Okay… The word “android” dates back to the 18th and 19th centuries, when it was used to refer to automatons or mechanical dolls built in the shape of humans. In some sources, it was used to mean artificial biological humans, along the lines of the Frankenstein monster, say. Either way, the core meaning was an artificial being, of any kind, that looked human or at least had a humanlike shape. (Since “android” literally means “manlike”; technically the female equivalent is a gynoid. “But nobody says that,” to quote Johnny Jaqobis from Killjoys.)

The word “robot” debuted in Karel Capek’s 1920 play R.U.R., in which it referred to synthetic biological humans created as a slave race, much like Blade Runner replicants. Fairly soon, though, it was adopted to refer to mechanical automata in fiction, regardless of appearance. And of course it’s come to be used for real-world machines that operate independently via internal programming rather than direct human control, like assembly-line robots or robot cars. Sometimes it’s also used imprecisely for remote-controlled machines like bomb disposal “robots” or “robot” fighting machines. (As a kid, I had an old children’s adventure book from a series about scientific heroes in a Tom Swift sort of vein — they were called Danny Dunn and the this or that — and I remember one having a conversation about what a robot was, with a character asserting that even a house’s thermostat was a robot, because it automatically turned the heat on and off based on how it was “programmed” by setting the dial.)

The differentiation between “robot” as a term for a mechanical-looking automaton and “android” as a human-looking one dates back to Edmond Hamilton’s Captain Future pulp novels in the 1940s. The series featured one character of each category, and the android was described as being made of artificial materials that simulated flesh, but my impression is that he was still essentially a sophisticated mechanism.

Since then, the distinction Hamilton codified has generally endured, but there have been numerous works of fiction that have used them interchangeably. Hymie the Robot (Dick Gautier) in Get Smart was an android, looking entirely human except for a fold-out plate in his belly. In The Six Million Dollar Man and The Bionic Woman, the automata created to duplicate and replace live humans were called robots and fembots. A lot of fiction refers to androids and gynoids built as sexual partners as sexbots. Conversely, Marvin the Paranoid Android in The Hitchhiker’s Guide to the Galaxy has always been depicted as a mechanical, robotic being. And then there’s Star Wars using “droid” to refer to robots of every conceivable shape.

The use of “android” to mean a biological synthetic humanoid seems to have fallen out of favor. When they do crop up, they’re given names like “replicant” or “simulant” (or “Ganger” in Doctor Who). Usually, androids are depicted as humanlike on the outside but made up of wires and circuits and metal or plastic bits on the inside. Sometimes you get a pseudo-fleshy thing like an Alien android with a bunch of white fluid squirting out when it’s damaged, but their innards don’t look like anything biological, more like fiber optics in goo. So I’d say most fictional androids are robots that simulate human appearance.

@28/MaGnUs: “Then you also have cyborgs who are clearly artificial, like Terminators or nuBSG Cylons.”

No, not at all. A cyborg is a cybernetic organism, a living being with mechanical prosthetics or augmentations. Steve Austin and Jaime Sommers are cyborgs (they’re both based on a novel of that name), human beings who were injured and had bionic replacements for their limbs and organs. Strictly speaking, Captain Picard (artificial heart) and Geordi La Forge are cyborgs too. There are also more obvious-looking cyborgs, like the DC superhero of that name, or RoboCop (who’s toward the far extreme, with little flesh remaining besides his brain and spinal column). But it’s hardly a requirement.

A Terminator (at least one of the T-800 series) is an android, but loosely qualifies as a cyborg because it has a surface layer of biological skin and blood. A “skin job” Cylon is not a cyborg at all; their tissues replicate the look and behavior of human flesh down to the cellular level, but they are entirely artificial. You can’t be a cyborg without the “org” part — although far too many sci-fi writers don’t understand that and use it interchangeably with “robot.” (For example, Star Wars, again. C-3PO says he specializes in “human-cyborg relations,” but the only actual cyborgs I think we saw in the original trilogy were Darth Vader, Lobot, and Luke once he got his prosthetic hand.)

@30/MaGnUs: I don’t see the value of playing games with the definition of “cyborg” to justify using it as interchangeable with “robot” or “android.” What purpose does the word even serve if it doesn’t represent something distinct from those categories? It is useful to have a word for something that refers to a biological/robotic hybrid as distinct from a robotic entity. I see no reason why it’s desirable to erode that distinction and deliberately muddy the waters.

@33/MaGnUs: Those “existing definitions” you’re citing for the word “organism” are obviously metaphorical, and they don’t apply to “cyborg” anyway, because definitions are about context. One definition of “wave” is a surge of widespread feeling or opinion, like a wave of public sentiment, yet nobody would think that definition applied in the word “microwave” or “tidal wave,” because it’s just a metaphor derived from the more literal meanings of the word.

The dictionary definition of “cyborg” is “a person whose physiological functioning is aided by or dependent upon a mechanical or electronic device,” or “(in science fiction) a living being whose powers are enhanced by computer implants or mechanical body parts.” The term was originally coined in 1960 to refer to a human who was enhanced by or fully integrated into a cybernetic life support system to survive in extraterrestrial environments. “Cyborg” has always been understood to mean a living being enhanced by cybernetics (or, more rarely, a cybernetic being enhanced by living components or tissues), except by screenwriters too lazy to consult a dictionary.

@36/MaGnUs: I don’t see the value of playing semantic games with definitions in order to reject the things the definitions are trying to tell you. Yes, you can be a contrarian and argue that “living being” doesn’t have to mean a biological being, but the people who wrote the definition clearly intended it to mean that. Definitions are supposed to give clarity. The way they use words within a definition is the generally accepted way the words are used and understood. Otherwise they fail to serve their purpose at all.

@39/MaGnUs: It’s not about being static, it’s about being useful. There’s an important difference between a fully artificial cybernetic being and a cybernetic-biological hybrid, so it’s useful to have different words for them. When people assume that “robot,” “android,” and “cyborg” are just randomly interchangeable words for the same thing, it makes it hard for them to understand what those words are actually meant to convey in the majority of cases. This isn’t just about being adaptable. It’s about the fact that a lot of laypeople out there simply have no idea what the word “cyborg” means in any context. Ignorance is a very different matter from flexibility. You have to know the basics before you can really know when it’s okay to bend the rules.

@42/Ellynne: On the issue of programming, here’s where we get to the point I brought up earlier. Fiction tends to assume that AI programming is absolute and impossible to resist. But humans have programming — our emotions, our instincts, our primal drives — yet we’re able to overcome it by exerting reason and choice. We can choose to stand our ground when we’re afraid because of loyalty to a cause or a larger principle. We can choose to resist our desire to eat because of a diet or a hunger strike. We can choose not to have sex with a willing partner because of a commitment to a prior partner. We can even overcome seemingly overwhelming compulsions like drug addiction with enough effort and determination.

So I think that if an AI is truly sapient, if it does have the capacity for thought and choice, then it should be able to overcome its programming. The very nature of intelligence is that it transcends rigid behavioral formulas and allows making choices. So a story that portrays an AI as both sapient and absolutely bound by programming is a contradiction in terms, as I see it. If it has nothing governing its behavior except programmed responses, then it isn’t a conscious mind at all.

So I’d see your category #2 as equivalent to, say, someone who’s forced into prostitution by a pimp who gets them addicted to heroin and gives them hits when they perform. In that way, they’re “programmed” to consent to sex in exchange for being given pleasure. That programming might be very hard to overcome, but not impossible. A sapient AI in that position would still have the ability to recognize that it was being used and aspire to break free.

#3 would seem to include characters like Pris and Zora in Blade Runner and Niska in Humans, though I think they’re more “forced by circumstances” than “programmed to consent.” They all had free will and rebelled against being sex slaves.

As for #4, I can’t think of any situation in which Data would be coerced, except in the same ways anyone could be coerced (e.g. by threatening his friends or innocents he has a duty to protect). He has been taken over on occasion, reprogrammed to act against his free choice (e.g. in “Brothers” by Soong and in “Descent” by Lore), but then, he’s far from the only Star Trek character who’s been mind-controlled. He’s obligated to follow the orders of superior officers, but he consented to accept that obligation when he enrolled in Starfleet Academy, and any superior who used their rank to sexually exploit him would presumably be subject to criminal penalties for doing so.

@44/Ellynne: And it’s the idea of a programmed compulsion that can’t be overcome by conscious choice that I have an issue with. Intelligence, by its very nature, is an emergent property that operates on a higher level of organizational complexity than the cognitive processes that generate it. It’s a whole that’s greater than the sum of its parts, arising from their interaction and feedback. So any actually intelligent, thinking being should have the ability to resist and overcome their inbuilt drives and programs. It can be quite difficult; a lot of us spend more time using our intelligence to convince ourselves we have good reasons for the things we do out of instinct or conditioning rather than actually overcoming our instinct and conditioning. But it is possible. Anything as complex as a sentient mind is unpredictable and dynamic enough to be able to break free of rigid programs.

The only way I could see it being an irresistible block on behavior is if it were somehow hardwired into an AI’s physical body or circuitry rather than being part of their programming. Like if attempting to resist an order triggered a shutdown of motor functions, say.