I had no degree when I started at the company, one of the big five, and being suddenly surrounded by better educated and better spoken peers I felt a bit like an impostor. My English was far from perfect too, then as it is now, and that didn’t do wonders for my confidence. Although I would like to say that things are all better now, in at least some ways they remain the same.
I had of course heard of the experiment before getting called in, although it was technically secret; my first reaction had been to think it sounded promising and exciting in a personal way. The potential impact was huge, and I loved Machine Learning. In a way the biggest dread I had going in was probably just fear that it could turn out not to work, or perhaps being disappointed with the shallowness of the results. Crazy ideas that didn’t go anywhere were common back then.
My initial enthusiasm with the idea, built up on just the technical details, was boosted by the fact that this was also a great work opportunity for me. Being part of the experiment was something special. If people were right and the experiment caused unemployment, well… it also meant having a job. For longer.
“Thank you for coming in, Ms. Petrescu.”
“Thank you for having me.”
“Do you know how your participation in the experiment will work?”
“Yes, I’ve been reading the terms that I got last night.”
“Would you mind quickly walking me through it? Just to make sure we’re on the same page.” He stressed “quickly”.
“Sure thing,” I said in my most upbeat tone, always eager to please, but already I felt slightly cheated. I suspected he was a repurposed manager, as he was already making me do the work.
The terms were clear in any case: I would be recorded at all times during my time in the office. I would surrender the possibility of interacting with my coworkers in any way outside of the designated office space or without the working tools I was provided with: my laptop and my work phone, and their linked corp accounts. And I would grant permission to the company to impersonate me whenever they wanted.
“Not whenever we want to — just whenever it’s necessary for the purpose of advancing the experiment.”
I didn’t see the difference. But I didn’t tell him that; I just nodded.
“Your participation will last six months, at which point your performance will be evaluated and your participation terminated or extended for another period. For each period you will receive a hefty bonus — you can find the actual sums in the payroll system,” he said, with a gesture that said that he didn’t want to go into such petty details at the time.
From what I’d heard, if my participation got extended for another six months, I could perhaps have enough to live off savings while I went back to school or retrained — or coded for fun — or, in the worst case, took on a hobby and an addiction.
“I have one question, though — how will my performance be evaluated? The material wasn’t clear on that point.”
“Unfortunately I’m not at liberty to tell you. Telling you could bias your behaviour in ways that go against the success of the experiment.”
I wondered if he was telling the truth, or he just didn’t know.
The office space I was moved into was dedicated to the experiment. Cameras were visible everywhere. Ironically I’m not sure there were significantly more cameras than usual around, but the company had made a point of them being visible. They were now part of my job description.
We were told to keep in the line of sight of a camera at all times. When we were out of our cubicles, we were to interact normally with our coworkers apart from that. When we went into our sound-proofed cubicles, though, we were told to “release our inner discourse”: to “vocalize”, that is to say out loud, whatever we were thinking as we were thinking it, as long as it was relevant or could be relevant to the task at hand. We could stop doing it if our thoughts strayed into our personal life, but we were told we should try to keep our thought process as work-related as possible — without enforcement of course.
We were warned we would like feel as if this made our thought process slower, which doesn’t sound good for work performance; but that we shouldn’t worry about it. Studies showed that people got used to it within a few hours (citation needed?), and in many cases people reported an increase in both self-reported happiness and work performance as vocalizing made them think in a more orderly way and focus more. But your mileage could vary.
Even if performance decreased, we shouldn’t worry. Any losses of productivity would be made up by the value we were adding the system.
This way the computer could hear us think.
“Yan, isn’t it weird being recorded?”
“Do you mean here, in this call, or outside?”
Yan was 9360km away, in Silicon Valley.
“Both I guess, but I was thinking about this call.”
“I think the thinking out loud thing is weirder, if you ask me.”
“I think this is weirder actually, because with the thinking out loud — I can guide my thoughts, think just things related to work, in a sense not be my whole self. But here, if I do that, I’m not being like myself to you. And I don’t like that — I want to be a person when I talk to someone. Also, we usually talk in private, and now we’re being observed. Someone snooping into a conversation is always creepy.”
“It’s all training data, and there’s going to be lots of it. Sure, a human debugging the model could take this footage and review it in case it’s somehow interesting, but the overall likelihood seems very low given the size of the corpus. In all likelihood everything we say will just shift the weights in some nodes in a huge neural network a bit, and that’ll be it as far as this conversation being observed goes.”
I said: “I’m not sure that makes it better. You could think of it this way: if the system works, the model we train works, the whole impact of this conversation will be encoded in those bits that shift. What if we have this conversation, it’s processed in training, and the neural network actually doesn’t shift that much? Would that mean we’re irrelevant?”
“Only if you care about that definition of irrelevant. You can add some chance to the equation too — it could be that this conversation is actually very relevant in some specific sense but then it’s only used for testing and not for training in the context of the model. So it’d only be used to improve the model only indirectly, to check for errors, but it wouldn’t actually cause any bits from getting flipped.”
“Preventing other bits from flipping (preventing mistakes) could be seen as equivalent to flipping those bits, though.”
“In any case,” I said, starting to think about changing the subject, “the same conundrum could be seen in the case of our own brains. I don’t remember many of the conversations I’ve had — well, make that most. Does it mean they might as well never have happened?”
We sat in silence for a moment.
“Well, anyway, about our project…”
The killer feature was the meetings. Meetings are productivity killers — most of the time anyway. Some meetings can save time in the long run, but overall programmers hate them because there’s too many of them and it’s hard to focus hard on a problem when you’re between meetings. I’m sure everybody hates meetings — but I’m a programmer so I hate them because of this particular set of reasons.
The killer feature was to be able to send your doppelgänger to a meeting instead of attending yourself. Like all killer features, you didn’t know you wanted it until you had it, and then you thought you couldn’t go back to living any other way. It resonated with people immediately. Let’s just all send our doppelgängern (?) to meetings, and have them talk to each other. We have robots do most of our manual work (as a society) already; this is mostly because people don’t want to do it. Meetings could be next.
This was what the coach told me when he announced I had been selected for an extension; I would be able to do only the parts of my job that I liked by delegating any meetings — if I accepted to expand my engagement with the experiment. That meant accepting to being recorded everywhere — not just in the office, but also on my commute and on my home. My partner would have to be recorded too in the intimacy of our home, unfortunately, but she would be compensated for it — essentially a contractor. Of course the bathroom and the bedroom would be excluded from all recording, and our holidays too.
“We don’t want to be creepy.”
I was happy because of the news and looking forward to get home to tell C. about it, but I knew we needed eggs and almond milk, so I stopped by Coop (one of the local supermarket chains) to pick those up. I ended up getting three or four sundries. And a Caramel City.
See, Caramel City is one of a series of desserts-in-a-cup things that are sold in Coop. They are marketed as a “protein pudding”, because they are based on milk protein — the kind used by weightlifters I think, instead of the more canonical cream and lots of sugar. So they are relatively healthy, or that’s how they’re sold, but still quite tasty. It’s only a 140 calories a (fair sized) pot. We are huge fans because really, they taste good, not like health food. And they are sustainable (to us), we don’t get fat or fatter from eating them. It’s almost too good to be true.
Anyway, there’s a whole range of them, each named after a fantastical place that has something to do with the flavour. So there’s: Chocolate Mountain, Vanilla Drive, and Caramel City. I like how they fit together well: like you could see Chocolate Mountain from afar as you drive on Vanilla Drive — towards Caramel City.
I was looking forward to getting home and telling C. about the extension over dinner, asking about her day, and then later having dessert and watching Netflix.
I thought: will my doppelgänger go home and do the same? What will she think of Caramel City?
Well, of course there were problems. At some point, the model (the doppelgänger) and one could get out of sync.
I called Yan.
“Have you seen any of the meetings with your double?”
Silence.
“No, not really. I just read the minutes.”
“Aren’t you curious?”
“I feel like it would be counterproductive somehow.”
“Counterproductive how?”
“Well, what if I just find it too upsetting? I like having her going to those meetings, so perhaps some part of me just doesn’t want to know if I can bear it.”
That didn’t sound like Yan at all.
I called the coach and told him what had happened. He invited me to come to his office.
“I’m not in a good place right now. What happened to Yan after leaving the meeting? Did she leave the simulated meeting room she was in and… walk into the hallway? Did she pick up coffee on the way back to her cubicle? Or did she cease to exist the minute the meeting ended?”
“Well, this is certainly a fertile ground for speculation. First of all, though, how would you define existence in this case?”
“…”
“Bear with me here.”
“Existence is the state of having a definite material presence in the world, I guess.”
“But Yan’s doppelgänger probably has multiple material presences — the bits of her representation in our storage systems. And this, regardless of whether she is active or not at a particular point in time.”
“What about mine? What does mine do after she leaves her meetings?”
He tapped into the keyboard and logged into some kind of profiling and debugging subsystem I had never seen before.
“Let’s look and see.”
What are the hearts of our machines? Are they their clocks?
Most humans have hearts of course, and in a sense they are our clocks. Our hearts never stop pumping (until they do), and in doing so they keep time for us. Would our perception of time be different if our heart beated at a much faster or slower pace? While we were evolving from simpler to more complex animals we had our hearts to keep us company. Could our sense of time have developed, in a way, from the beating of our heart? And after that from the cycle of nights and days; and afterwards that of seasons. But in the beginning perhaps only the heart, a puny tiny heart in a puddle somewhere. And a tiny brain being oxigenated by it and trying to make sense of the world around it.
Computer clocks are millions of times faster than human hearts, of course. But when programming you often schedule events to happen every Nth ticks — and it’s simple and common enough to schedule events that take place every few seconds.
When I saw my doppelgänger for the first time, I thought of this. Did she have a heart? Did she ever think she had one? How quickly did it beat in her mind?
Suddenly I found myself thinking of Caramel City again, and I felt either fortunate or unfortunate to have gotten to know about the concept of qualias, as it came in handy in this particular situation. C. had read a paper for university that included a reference to them and told me about it. She then wrote a paper using the idea, applying the concept to a scene in “Do Androids Dream of Electric Sheep” in which an android eats a fruit. I actually don’t remember if the android eats the fruit and then just ponders about it, or whether it actually comments on how it tasted like nothing to it. Qualias are what the android could be missing from the human experience — a “conscious experience”. Sort of like a quantum of consciousness.
At the time I had thought that it was a useful concept, but one sort of tautological. It seemed like some philosophers were saying: a computer cannot have consciousness, because they don’t experience qualias. Because qualias are the individual instances of subjective, conscious experience. See? It’s all a bit circular as far as thoughts go. I’m sure there’s more to it, and my interpretation is slightly wrong in many subtle ways, but this is what stuck with me after that conversation.
Now I went back to the concept often. Had I changed my mind a bit? Did the existence, or apparent existence, of my doppelgänger, somehow raised the stakes for me? Sometimes you believe what you want to believe, but you tell yourself that you believe something for a reason. When you dig further, the reason isn’t there — you thought there was one, just out of sight, but when you looked it was gone. Your brain had fooled you for a minute: don’t look there, it’s fine, this is how we are, this is what we believe. No need to delve. But when you dared take a look and you considered the issue anew you found that you were slightly different from what you thought you were; you were basing your “knowledge” on a prejudice, or a misunderstanding, or a feeling, or really nothing at all. Just a random connection in your brain that made you believe something.
So, how did Caramel City really taste like? I thought I could remember the taste; I like it. It’s… caramel-y? Sugary and a bit burnt. But is it? Most caramel is a bit like vanilla, but sweeter. Does it taste burnt, but in a pleasant way? Or does it taste like sugary vanilla that is also brown? How does chocolate taste like? If you cannot answer anything but “like chocolate” to that, can you really say that you know how it tastes like when it’s not in your mouth? Even so, you could be tasting caramel all day and still not be able to reproduce its taste from first principles (I guess that’d be sugar, heat, and time). You don’t know how to make chocolate or caramel, so do you really know what they are? Does the genius mind behind Caramel City know its taste more than you? If so, do they have better qualias than you? Perhaps you happen to lack that particular qualia, and you’re just unaware of it.
The second extension and then the third came and went, and by then things just sort of seemed to get into a groove. I sometimes go to meetings and feel like I’m talking to a doppelgänger, and on bad days I sometimes feel like one myself. Usually it’s better not to go at all, of course, but some meetings I have to attend for a variety of reasons. I tell myself I cannot risk not going and then having to put up with a decision I didn’t make, or a consensus I wasn’t part of.
I wonder if my doppelgänger goes to some meetings I attend as well, just for training purposes. Then at around 7pm packs and goes home. Has dinner with C., thinks of how much she loves her, then watches Netflix with her.
I can just imagine her stepping into the kitchen and opening the refrigerator. Grabbing a Choco Mountain — sensible enough. Not usually my first choice, but not completely out of character either.
I sometimes also take detours on the way back to Caramel City.