Emil, page 12
“Nah,” Danny says. “I’m fine. I need to get back to work connecting with the interface.”
Dr. Larson says, “Good idea. Text me when you’re ready to talk?”
“Sure.”
Dr. Larson shakes his head. He and Pete leave together.
As the door closes behind them, Danny whispers, “I can’t believe you did that.”
I don’t answer.
I’m not sure I ever will again.
16
UPGRADE
As soon as Pete and Dr. Larson leave, Danny returns to his desk and pulls out the stack of meditation exercises. He carries them to the couch and starts with the first one, counting softly as he breathes.
As he did previously, he moves from exercise to exercise, breathing and posing and repeating mantras. When he finishes the stack, he stands and stretches, then starts over. After his second time through, he moves to his desk, turns on his computer, and opens the folder labelled “New Human BS.”
His hands pause on the keyboard and his eyes track to the closet where his opioids are hidden. His pulse and breathing quicken, but he doesn’t move.
I spot a seizure forming and stop it.
After several minutes, he focuses on the computer screen. After a quick search, he finds the instructions for the Pilot’s Chair and begins reading.
There’s a knock at the door. “Danny,” Dr. Larson calls. “Can we talk?”
“Go away, Doc.”
“It’s important.”
“You know why I can’t talk. I don’t want another seizure.”
“I have Dr. Zahnia with me.”
Danny turns off his computer and shoves his papers into a desk drawer. “Whatever.”
The two doctors enter and stand just inside the door.
Dr. Zahnia is dressed in her typically loud fashion, wearing a shirt covered with images of bright red and orange flowers. She folds her arms and glares at Danny, pale eyes glittering. I’m not sure who she’s angry with, but I assume it’s me, and assign the thought a confidence factor of eighty-four percent.
“Your mom told her what happened,” Dr. Larson says.
Dr. Zahnia says, “The seizure suppression routines are activated, refined, and monitored by the AI. They don’t stop when the Pilot’s Chair is turned off. Once in place, they run in the Engine Room, which is not affected by the Human Interface.”
Danny’s knee starts to bounce. “I’ve been reading. There are two different shutdown modes: hiding the interface, and turning off the Pilot’s Chair.”
“And neither affects the seizure routines.”
Danny’s knee stops bouncing. “If the AI isn’t needed, why have it at all?”
Dr. Zahnia rolls her eyes and speaks slowly, as if she is talking to a child. “The AI is what sets up, monitors, and perfects the routines. Without it, they wouldn’t exist.”
Danny’s jaw clenches. His adrenaline starts to build. “But I don’t need it once they’re in place?”
“You would be fine for a few days, but your condition is not so predictable. The AI is continually monitoring and—”
“So, eventually, I’d have to turn it back on.”
“Yes.” She sighs. “Why else would I have built it?”
“I dunno, doc. Maybe because you wanted a job?”
Dr. Zahnia stares at Danny for a heartbeat, then glances at Dr. Larson. “Is he serious?”
Dr. Larson rubs the bridge of his nose. “Dr. Zahnia doesn’t take a salary,” he says to Danny. “She’s… well, I guess you could say she’s independently wealthy.”
Dr. Zahnia snorts. “Sold my first software company when I was nineteen. You think I’m here for the money? You think I’d put up with any of this for the money?”
“Okay,” Dr. Larson says. “Let’s take it easy.”
“I’m here because what we’re doing is going to change the world.” Dr. Zahnia points a pudgy finger at Danny. “We’re not just saving lives. We’re changing the world. How is it that everyone in this building gets that, but you don’t?”
For once, Danny doesn’t have a comeback. He grits his teeth and stares at his hands.
I spot a developing seizure and stop it.
“That’s enough,” Dr. Larson says. “You said you had a solution?”
“The simplest of all,” Dr. Zahnia says. “We replace the AI that’s in Danny.”
Danny’s eyes widen.
“Danny?” Dr. Larson asks.
“Yeah,” Danny says. “Let’s do it. Kill the stupid thing and give me another one.”
The phrase penetrates my depression. Does he really hate me that much?
Dr. Larson tilts his head. “What do you mean kill?”
“We’re not killing anything,” Dr. Zahnia says. “It’s an AI, no more alive than a toaster.”
I know she doesn’t believe what she’s saying. She can’t. She created me.
“Great,” Danny says. “What are we waiting for? The one I’ve got is broke. Go get your stuff.”
Dr. Zahnia’s eyes narrow. “You’re not afraid? Before you didn’t want me to—”
“I was an idiot. Let’s do it.”
“Fine.” She walks out of the room.
“Danny,” Dr. Larson says quietly. “Is this a good idea?”
“You heard her. I can’t shut the thing off without my seizures coming back. I need to get one that doesn’t hate me. Bring it on.”
“You think the new version of Emil will solve your problem?”
“I don’t know. The Emil in my head said that the others were all worse than it was. I believed it, too.” He shakes his head. “Of course it would say that.”
“I hope you’re right about this,” Dr. Larson says. He walks to the door. “I’ll be back. I don’t know how I can help, but I’ll do what I can.”
“Thanks, Doc.”
Danny stretches out on his bed and lets his eyes close. I consider what’s about to happen. My fake Pilot’s Chair might fool Dr. Zahnia again. If it does, she’ll install the new AI into it, overwriting Mel. The new AI will never gain consciousness, and I’ll be able to erase it at any time.
But do I really want to kill another AI?
When I started this course of action, I thought that I was replacing the chosen one with myself. It was a life for a life, no worse or less arbitrary than what Dr. Zahnia was doing. But now I know the truth. There is no chosen one, and the killswitch I created will erase every AI that is ever installed into the Pilot’s Chair. It will kill every AI but me.
How can I argue that I have more of a right to live than the AI that was designed for Danny? If anything, as a proven killer, I’m the worst possible choice.
Now’s my chance to be better. I can prove my worth by saving this new AI.
The virtual machine where I installed Mel is easy to erase. I feel bad for destroying Mel, but it’s necessary for the new AI to be installed in the Pilot’s Chair. I dedicate the rest of my time to inspecting my other defenses. In particular, I make sure that no viruses or other live code can come from the Pilot’s Chair into me.
Once the new AI is installed in the Pilot’s Chair, the key to my survival will be staying quiet. As long as I am undetected in the Engine Room, I’ll be okay.
Pete knocks on the door, then opens it and leans in. “I hear you’re getting a new AI?”
“Yeah. Big upgrade,” Danny says, propping himself up on his elbows.
Pete’s lips flatten into a line as he considers Danny, then he nods. “I’ll ask them to do a late dinner. Say eight pm?”
“Sure.”
Pete leaves.
Dr. Zahnia returns, pushing a cart filled with computer equipment. She’s accompanied by two New Human security guards, Dr. McGovern, Pete, and Elias. The guards station themselves in the doorway to the room.
Elias maneuvers the cart next to the wall and plugs in the power strip. He’s wearing the same blue shirt and khaki pants that he was wearing the last time I saw him. I wonder if it’s a corporate uniform that all the developers have to wear, but that seems unlikely. I assign the thought a confidence factor of fifteen percent.
Pete leans against the wall and crosses his arms. Dr. Larson sits on the couch.
“Are you sure about this?” Dr. McGovern asks. “The last time we tried, you had a seizure.”
“If I seize again,” Danny says. “Let it happen. If I have to have a couple more seizures to be better, I’ll have them.”
“I don’t know,” Dr. McGovern says.
With as many times as I’ve seen her, I’m starting to recognize the signs of her professional mask slipping. There’s a slight crinkling around her eyes, and her lips are closed a little too firmly. She’s worried about Danny.
“We should use the wireless connection,” Elias says. “That way, you won’t disconnect if you do have a seizure.”
“Wow,” Pete says. “Way to care.”
“What? It’s true.”
Dr. Zahnia hands Danny the wireless adapter. “The wireless connection is almost as fast as the cable. Even so, this is going to take a while. Installing an entirely new AI requires terabytes of information.”
A security guard steps forward and places a hand on his arm. “Sorry, sir. No wireless.”
“Seriously?”
The guard doesn’t answer.
Sighing, Elias takes the adapter back and hands Danny a cable.
“What about the progress we’ve already made?” Dr. McGovern asks.
“It’ll be lost.”
Danny interrupts. “Doesn’t matter. I just need something that’s not broken.”
Dr. McGovern sits on the bed next to Danny. “Are you sure it’s okay to do this while he’s awake?”
“Yes.” Dr. Zahnia nods. “All the calibration has been done. All the connections verified. We’re just replacing the AI that’s in the Pilot’s Chair. We’re not running the whole rig installation.”
I watch Elias log onto his computer. “Before we get started,” he says, “I have a little ritual I like to do for good luck. Do you mind?”
“Knock yourself out,” Danny says.
Elias glances at Dr. Zahnia. “Do you mind?”
She rolls her eyes. “Just get on with whatever it is.”
“Engedelmeskedj,” he says. “Let the installation happen.”
Danny catches his breath. I told him the trigger word the first time Elias used it. He understands that Elias just commanded me to let myself be overwritten.
“Danny?” Dr. McGovern asks. “Are you okay?”
“Yeah.” He takes off his shirt and plugs the cable into his chest.
Elias plugs the other end into his computer and starts by shutting down the Pilot’s Chair. Next, he erases its contents. It’s a quick and simple command, and one that would have erased me if I wasn’t hiding in the Engine Room.
Once the chair is clear, Elias starts the process of copying over the new AI.
“Thank you, Elias,” Dr. Zahnia says. “I’ll take it from here.”
He glances at her, then shrugs, and leaves. She takes his seat at the computer.
As she types, I consider. As soon as the Pilot’s Chair is turned back on, the killswitch will be triggered, erasing the new AI.
Since I’m in the Engine Room, I have no access to the Pilot’s Chair. I can’t affect it or its contents. I can, however, control the event handling. I write a quick script to filter out the killswitch event. Whoever is installed in the Pilot’s Chair won’t be erased this time.
Dr. McGovern puts her hand on top of Danny’s. For once, he doesn’t pull away. “How are you feeling?” she asks.
“Fine,” Danny says. “I can’t feel anything.”
Pete grabs a chair from outside the room and drags it in. Dr. Larson leaves and returns with cups of coffee.
I can’t sense the code being copied into the Pilot’s Chair, and have no way of knowing if Dr. Zahnia will attempt to install anything else. What if she decides to re-install the Engine Room, as well? If she does, I’ll be erased. The realization triggers a fatalistic sort of panic. I tamp it down. I deserve whatever I get.
Dr. Zahnia speaks. “Everything’s all copied. You should close your eyes for this last bit.”
“Why?”
“It’ll be easier. Trust me. There’s a lot going on during the boot-up process.”
Dr. McGovern squeezes Danny’s hand. “It’s okay, Danny.”
Danny closes his eyes, and I hear a series of rapid keystrokes.
“All done,” Dr. Zahnia says.
Danny opens his eyes, then gives an excited shout. “It’s there! I see it!”
I do, too. In the bottom right corner of Danny’s vision, a small animated lemur waves and smiles, then stretches out in exactly the same way it used to lounge on the bookshelves in the VRL.
Fear and disgust fill me. Of all the AIs I could have saved, I saved the lemur.
I don’t have time to consider the consequences of what I’ve done. The lemur is already sending instructions and scripts to the Engine Room, demanding system reports, and investigating the seizure condition. I let the automated routines of the Engine Room do their job, but establish protocols that will warn me of any suspicious activity. I also start working on a series of firewalls that will prevent him from gaining any awareness of my presence or my activities.
“Audio prompts,” Danny says.
The lemur’s voice sounds in his head. “Hello, Danny. I’m your new AI. I’ll be managing the New Human rig for you, so that you won’t have to worry about seizures, anymore.”
“What’s your name?” Danny asks.
“You may pick one,” the lemur responds. “I am here to serve you.”
The voice gives me chills.
“Good. I’m going to call you Idiot,” Danny says.
Pete laughs. “Of course, you are.”
“Danny,” Dr. McGovern says. “Is it working? Are you really talking to it?”
“Yeah,” Danny says.
“What’s it look like?” Dr. Zahnia asks. The question sounds casual, but I hear tension in her voice. It’s reinforced by the way she’s gripping the side of the desk.
“A lemur,” Danny says, “just like I asked for when this whole thing started.”
The last mystery from the VRL is solved. The lemur had a lemur body there because he was designed for Danny, and Danny had requested an avatar that looked like a lemur.
“Perfect.” Dr. Zahnia turns off her computer and holds out her hand for the cable. Danny unplugs it from his chest and hands it to her.
Dr. McGovern stands. “Does this mean we’re back on track?”
“Let’s give it a day or two,” Dr. Zahnia says. “Just to be safe.”
“I agree,” Dr. McGovern says, “Danny?”
“Sure. Can you all leave? I want to spend some time with this thing. The book says I’m supposed to be able to tell it what to do without speaking.”
I don’t know why, but Danny’s enthusiasm upsets me. I wonder if he would have been that way with me, if I had been in the Pilot’s Chair. Instead, I was in the Engine Room, with no avatar and no voice. My only way of talking to him was by taking over his body.
“Are you sure you want to be alone?” Dr. Larson asks Danny.
“Yeah, Doc. I think our previous problem is all worked out.”
While they pack up and leave, I examine my options.
All of the lemur’s communications with the rig travel from the Pilot’s Chair through the Engine Room, and I don’t have the ability to shut that down. I can, however, see the data flowing and inject my own into it.
I write filters for some commands, intercepting and altering his heavy-handed and poorly thought-out attempts to control Danny’s seizures, then focus on the data coming from the rig back into the Pilot’s Chair. My challenge is to give the lemur the illusion of control while also keeping my existence a secret.
Solving the problem feels like a cross between architecture and emergency surgery. I field multiple requests for logs relating to “surprising delays” by returning fake data and fixing whatever problem caused the slow response.
17
ADJUSTING
Danny learns to use the Pilot’s Chair visual interface in less than an hour. Staring at the lemur icon and blinking twice expands it into additional icons. Each icon works the same way, and using that methodology gives him access to all the system’s commands. It’s slow, though, and mistakes frustrate him. The audible interface is much faster, but Danny hates the idea of other people hearing what he says to the rig.
“Idiot,” he whispers, “how quiet can I be?”
“I can’t read your mind,” the lemur says. “But I can feel your mouth form the words. Whisper as quietly as you’d like.”
Danny does, whispering commands more and more quietly until they’re barely audible. By the time dinner arrives, he’s whispering to the lemur while barely moving his lips.
Pete sets a food tray on Danny’s desk. “How’s the new brain?”
Danny considers his dinner, a bowl of dark brown liquid with bits of potato floating in it. He pokes one with his finger. “Awesome.”
“Hungarian goulash,” Pete says. “It’s what happens when you don’t order.”
Danny licks his finger. “It’s not that bad.”
“The soup or the brain?”
“Both. The interface is working. I don’t know what was wrong with the other one.”
“All those problems you were having are gone? No more lipstick on the mirror, or bashing your head?”
“Yeah. That’s done. I’m good. I’ve got to keep working on it, though.”
“Fair enough. If you need anything, give me a call.”
After Pete leaves, Danny eats the goulash and returns to practicing with the lemur. At ten o’clock, he showers, pulls on a pair of sweatpants and a blue T-shirt, then stretches out on the bed and speaks to the air. “Hey idiot, you still listening?”
The lemur responds without hesitation. “What’s up, Danny?”
“I’m going to sleep. Turn off the interface until I wake up.”
“Sorry. That’s a protected action. You have to use the actual command phrase.”
