by
Douglas Crockford
Lucasfilm Ltd.
Final Report Presented to RCA Laboratories
January 15, 1986
ExpandoVision is an exciting new medium. It has a very interesting set of capabilities and limitations. It is not quite anything we've seen before, although it does borrow technology and techniques from television and computer games, much as movies borrowed from theatre and photography. But just as filmmaking is not shooting snapshots in a burning theatre, so ExpandoVision is not blowing videotaped spaceships.
So here is an introduction to the design of interaction. The first four parts explore various aspects of interactivity. The last two parts present two sample programs, and describes their conformity to arcane and unknowable principles.
Between a sender and a receiver there is always a medium. Every medium has a bias. There are some messages it delivers well, some it distorts, and some it doesn't deliver at all.
The thing that makes this simple observation difficult to understand is the blind spot. One the messages that a medium can't deliver is the meta-message about the messages it can't deliver. Therefore, a TV viewer is unaware of the inherent biases in TV because he gets his information about TV from TV, and TV cannot tell him about the stories TV cannot tell.
Occasionally, a person might have some first hand experience in something on TV (maybe he or an acquaintance is being interview), and on watching the broadcast may feel some discomfort because TV's report is subtly or grossly wrong.
People working in TV production must know about TV biases, because they might be trying to deliver undeliverable messages. They learn about "what works" and so avoid the undeliverables. They may even develop a meta-blind spot, confusing the inherent bias with matters of technique or quality. They may even believe that they have overcome the bias by transmitting only the messages that don't distort, but what they've really done is simply add another level of filtration. They pre-select only those messages that "work", introducing a meta-bias.
The moral of that is that a medium is also a filter. There is no medium that can transmit all possible messages without distortion or aliasing. So, in working in a medium it is useful (but sadly, not essential) to understand the inherent biases.
An example of such a bias is one of those occasions when you want to express an idea about a feeling, but can't find the words. The problem is not necessarily due to a faulty vocabulary. The fact is, there really are some things that words cannot say. We often get around that by using analogies, to try to trigger remembrance of other unspeakable things.
Another example is nature programs. While watching a nature program, the experience seems complete. But being in nature is never like a television program.
An ExpandoVision experience will be judged by the feelings it invokes, so let's review what we got from video games, the primitive forerunner of ExpandoVision.
Video games gave you tension, frustration, confusion, anger (usually directed at yourself), and occasionally accomplishment and even fear. The one word you would hear more than any other at the arcade was "Damn!" In the early years there was also discovery and whimsey, but they evaporated when the golden age ended and video game marketing became more focuses.
The video game craze ended when people discovered that those feelings were all they were going to get. The game producers misunderstood the public's complaint "We Want More," and delivered better graphics and more frustration. What the public really wanted was a wider set of feelings and experiences.
Image quality is important, but it does you no good if you are using it to deliver an experience that no one wants.
Clearly, the video game medium is extremely limited in the range of experiences it can deliver, in the range of stories it can tell. We will not get to the next level of experience using the video game's Touch It and Die paradigm.
Like any medium, ExpandoVision has enormous constraints in the stories it can tell. Our job is to find those modes of expression which are best supported by the medium, and to structure from them an interactive experience that is like none other.
How do we do that? First, we must get to know our medium, to develop instincts about what will "work." Also first, we must consider the design of experience. Admittedly, things are getting real vague. There has never been a formula for developing a new art form what was any good.
It's like Han Solo used to say: What are you looking at? I know what I'm doing.
The experience should be safe. As much as possible, all technology anxiety should be relieved. Mom should be comfortable with it.
Everyone should win. The experience may be an ordeal, but we want only survivors. Whatever it takes to help people along (wishing rings, friendly spirits, mysterious women in black, incredible luck) will be dispensed as needed.
You should care. Even though it is fixed that you will win, there must be room for doubt (but not so much that it is not safe). You don't really affect the outcome, because the outcome is success. The outcome is not what's important.
If you wish yourself into a movie, it is not to change the outcome. The screenwriter will take care of that. You go into the movie to enjoy the experience. If you replace Luke Skywalker then you will not find it very rewarding to allow, as a result of your mistakes, the Empire to destroy the rebel fleet. If drama insists on an outcome, then fate is sealed.
The participant is an actor, playing a role. Even though it seems improvisational, and the participant may not even be aware of the script, the participant's range of actions is constrained by the structure of the experience.
It is like Greek Literature. The gods have already decided the outcome. But you the hero must still go through the process and discover in what sense the prophecies were true.
Can a modern audience accept this apparent lack of free will? They can in films and television programs. There has never really been any exercise of free will in participatory theatre or video games.
There must be properties of the experience that are more important than outcome.
There are some unsettling contradictions which we must struggle with in the new interactive medium. They have to do with constraints on interactivity. This class of contradictions is known as Crockford's Paradox. It is the video equivalent of wanting to go to heaven, but not wanting to die.
In ExpandoVision, we establish a video environment for the participant to play around in. We would like to promise that the participant can do anything she wants, but we can't. And failing in that promise leads us into deception and trickery. More important than the moral dilemma, if they figure out our tricks (and they will) then they won't be fooled again.
Sure, blame the technology. Limited storage means that we must keep them on a tight track. We cannot store the track for all possible excursions. OK, let's assume infinite storage. Now the problem is that we can't afford the production costs for all possible excursions. OK, let's assume that computer graphics are brought to such perfection that any scene can be rendered in real-time. Now we must script all possible excursions. But it isn't possible to enumerate all possibilities.
So we construct a computer program that can generate the variations as the participant selects them. (That is what video games do.) But we want meaningful interactions. And that is something a computer cannot do. It cannot maintain the structure and integrity of a story. That is difficult enough for a novelist to do. It is not a technology problem, it is an art problem. Success is not measured in engineering terms, but in showbiz terms.
A computer can do a shaggy dog story. They way that works is we have an opening, a set closing, and lots of stuff in between that doesn't matter. But that's just trickery again, isn't it?
We could throw out plot and structure, but it would be wrong, that's for sure. I have never enjoyed a plotless movie as well as a well plotted one.
The biggest trick in making a documentary is find the plot and structure in a set of facts and events and observations which are naturally plotless and structureless. Plot and story structure have been in culture for a long time. I am not ready to throw them out just to make the paradox easier to live with.
There must be a pattern to the drama. The thing we can do that general audiences have never been able to do before (unless you consider the participants in tribal ritual as general audiences) is to give a little slack, a little room for self-expression with the context of some interesting events.
I like to vote in free and democratic elections. I don't believe that the votes I cast are significant. Sometimes, the people and issues I vote for lose. But, I believe that the act of voting is extremely important, even if it makes no apparent difference in the outcome. The fact that my candidate would have won/lost anyway does not make it a trick.
Someday I'd like to do an ExpressoVision love story. Suppose my goal is to live happily ever after with her. I can do all the right things with my little joystick, and still it might not work out in the end, because even if my performance is perfect, the outcome still depends on at least one other person. If those two people are star-crossed, then there is nothing I can do to win. It is not a trick. It is not a deception. Not all stories should have happy endings.
But those stories that should have happy endings should end happily. The muses demand it, and by golly, we're not about to let them down. The audience is just going to have to accept that. My hunch is that already have.
All we really have left is the Tinkerbell problem. If they play it again, and try to change the outcome, they may well discover the man behind the curtain. And they may feel such a deep sense of shock and moral outrage that they would never consent to an interactive media experience for the rest of their tortured lives.
But on the other hand, maybe they'd get over it, shrug and say "oh, that's how they do it." Audiences didn't demand their money back when they discovered that the spaceships were really models, or that the forests of Endor were matte paintings. They may even recognize Indiana Jones's stunt doubles when they see it for the second or third time.
The audience wants the experience to be good, too! We have to trust them. They have a responsibility in the interactive theatre to enjoy it and believe it if they can.
Jane, my five-year-old daughter, has been helping me construct a bibliography on interactive fun. The first book she recommended was Dumbo's Circus. In it, you play a boy who goes to the circus and has an adventure. Every so often the story stops and gives a directive, like
If you want to climb up to the stand, turn to page 4.
If you watch the clowns, turn to page 16.
So I asked Jane which will it be, and then skipped to the page. She immediately changed her mind. Her criteria had nothing to do with story involvement or indecision. She rated her choice by how many pages were skipped. She wanted to always skip as little as possible so that she wouldn't miss anything.
We reached an ending: You fall to your death from an airplane while trying out a new act.
Jane wasn't bothered by the unhappy ending. She was greatly bothered by the words The End in the middle of the book.
So we took a few different branches and got some different endings. Jane was frustrated because it kept stopping, and because she knew she was missing stuff.
So finally we went straight through, cover to cover, and found all of the tracks and all of the endings: One happy, one neutral, sent home, pie in the face, fall, run over by a train, crash into a tree. There was no way to predict which track led to the happy ending. You make random choices, and one in seven times it comes out OK.
Is that interactive fun or what?
Ordinarily, in a Disney Dumbo Picture Book, you would expect to get a happy ending. But here, because of your choices, you fall to your death instead. (In fairness, the final words are: Will your parachute open up? You certainly hope so! The End) It must be your fault, but it isn't clear what exactly you did wrong.
Trial and error. Remember the pattern. Traverse the tree. Do it over and over. That is the only way to get the outcome you want.
If this had been a sequential story, you would have gotten the best outcome on the first reading. Do all the false starts make it fun?
Jane's next recommendation was The Telephone Book. It is by the same woman who did Pat the Bunny and Pat the Cat. In all of these books, there is something to do on every page. Move something, sniff something, peek behind something, touch something. It is all tied together in a little story about a girl and boy you are visiting.
This one starts with a picture of a telephone with a cardboard handset attached with a piece of black string.
Ting-a-ling a-ling a-ling a-ling!
That's the telephone. It's for you. You answer it. Say "Hello?"
Jane picked up the paper handset. I wouldn't read any more until Jane said "Hello?"
Hello! This is Paul and Judy. Will you come and help us take care of our little brother?
"Yes."
That's good. Goodbye.
Then there are several pages in which you do stuff with the kids. Then the last page:
Ting-a-ling a-ling a-ling a-ling!
That's the telephone. It's for you. Turn to the front of the book.
Pick up the receiver, say "Hello?"
And she answers. "Hello?"
Hello! This is Paul and Judy and Timmy. Are you home now?
Smiling, "yes."
Thank you for helping us. Will you come and see us again?
Smiling wider, "yes."
That's good. Goodbye.
"Goodbye."
Wow! Compare that performance to the branching book! This book is strictly linear, but the impression of interactivity that Jane got was far more satisfying.
Once people get picturephones, they will also get picturephone answering machines. What you get is a little more unusual. Your machine allows you to call up any answering machine and to examine any of its recorded messages. Of course, sale, use, or possession of this machine is a violation of the Communications Act of 1989. You don't even know where it came from or why you have it.
There is one number in the dialer's directory. Should you do it? You can't be detected. Go on. Try it.
And that's how it begins. That first person's messages will contain some other numbers. Those automatically go into the dialer's directory, and your little world expands.
It is like watching television. You are spying on people's lives. It is no more difficult than changing channels. And it's really interesting. You get to know these people and learn their stories.
And then you hear about a murder. And it turns out that you can be detected. What is going on? Why was this machine built? And why was it given to you? You are really involved now. They only way to find out the answers is to
Dial "E" for ExpandoVision
When danger calls, don't answer unless you have a really good explanation.
* * * * *
Because this is the first ExpandoVision feature, it is likely to be a participant's first ExpandoVision feature. Therefore, in addition to all the regular Dynamic Action concerns, this show must also serve as an introduction to ExpandoVision as well.
Many people will be uncomfortable with the new medium. Even if the industrial designers do their jobs properly, so that folks aren't afraid that they might break the machine when they turn it on, many people will still be anxious about what's going on.
They will sit there, frozen, afraid they will mess things up. Intimidated by the technology, they will be unable to accept the experience.
If the Feature is not designed properly, then we have lost that person. We must give them a gentle introduction. If they hesitate, we must wait for them. If the story unfolds by itself, then they won't buy in.
The opening of the show is very important. We have to do many contradictory things at once: We must prove that it is safe, we must show that the participant is in control, we must show that there is no performance pressure, and yet prod them along to sustain the pace of the program. And we need to prove, right off the top, that this is going to be fun.
The distributed video database presentation for this show is almost a familiar one. It combines everyday objects like telephones, answering machines, television, and VCRs into an acceptable new technology, human factored for your convenience.
In this program, the participant has nearly total control over the presentation, and nearly no control over the events which are presented. The participant's apparent control, at least in the early parts of the program, is absolute, giving her the confidence to buy into the experience without anxiety
The mythological basis for the program is Pandora's Box. We will put the participant into the same position as Pandora: Enticing her into opening a mysterious box which came with a warning not to open, motivated not by greed but by curiosity, in the safety of her own home. The participant will commit the Pandora Act, experiencing it and its consequences. The sequential media allow us to observe the act. Only ExpandoVision lets us feel it.
The PlayDisk will contain one or two hours of recorded messages. These messages will be performed by about 100 actors, each delivering between 30 seconds and 5 minutes of monolog. There will be about 100 sets. Each set only needs to represent that part of the environment which is visible from the picturephone.
It's those aliens again! This time they've settled down in a sleepy little tourist trap which is the home of the Giant Mud Torpedo and the Biggest Foam Rubber Warehouse and Gift Shoppe on this side of Interstate 80.
So hop in, and take a trip to Alien Springs, Wyoming!
And thanks to ExpandoVision, you don't really have to go there.
And you know that Wyoming will be your new home.
Foam Home, that is. Yeeha!
* * * * *
We have enormous constraints placed on us by the medium. The constraints cannot be hidden. The best we can do is to dress them up and pretend that they are intentional Where high-tech dressing is needed, I have aliens. Where low-tech dressing is needed, I have Wyoming. And when the technical limitations seem so stupid as to be unexplainable, it turns out that the aliens are not as smart as they look. I have more freedom in fitting the environment to the medium in Alien Springs than anywhere else on Earth, because in the trailer park world of Alien Springs, you don't know what to expect.
This is not to suggest that all features will have to be shot in Alien Springs.
Someday, audiences will know what they want from the experience and will know how to get it. The constraint dressing will not matter then. It will take time before audiences and producers get comfortable with the new medium. We will have new structures and new stories, and a whole new vocabulary for representing the images and dynamics of store-bought experience. But in the meantime, we must provide a safety net, links to more familiar goings-on.
There is also another set of constraints, which are placed on the participant. There has been talk about the desirability of being able to do anything you want in an ExpandoVision Experience. But in this case, I think that would be a mistake. I contend that the Kingdom of Fun is already in the participant. We normally live in a domain of so many choices, that fun becomes elusive. When we play a game, we accept a set of rules which, when used as directed, will tend to limit us to fun choices. We need the constraints to shut out the noise, to help us focus, to discover within ourselves the capacity for enjoyment. In the recreational context you must have the proper constraints or it won't be fun.
The aliens have been sent to conquer Earth by changing the population into aliens, but lacking the technology to actually do that, they have come up with an alternate plan of simply dressing everybody up in foam rubber alien suits. They know it won't work, they're just trying to fool their bosses into thinking that the invasion is on schedule so that they won't be called home, and they're making a little money on the side selling foam rubber tourist novelties and holiday decorations.
That's all simple enough, but what are you doing here? You find yourself operating an alien taxi, moving aliens around town. The town is small enough that you can walk anywhere quicker than you can open a car door, but sometimes the aliens just like to take a cab when the mood hits them. It's a job, but it's easy thanks to the Go-omatic controls. But that still doesn't answer the question: What are you doing here? There's only one place you can find the answer:
Alien Springs, Wyoming!
America's Other Vacationland
The End
It is like in Peter Pan when you clap to announce that you do believe in fairies. That sort of theatrical interaction, that audience participation, demands that some vocal fraction of the audience do what is expected. But if you don't clap, does Tinkerbell die? If you could have saved her and didn't, is that murder? What if you honestly don't believe in fairies? Why all this guilt in children's theatre? Is this supposed to be fun or what?
A Walt Disney CHOOSE YOUR OWN ADVENTURE by Jim Razzi. Copyright 1985 by Walt Disney Productions. Published by Bantam Books.
A Golden Book by Dorothy Kunhardt. Copyright 1942 by Dorothy Kunhardt. Renewed 1970. Published by Western Publishing Company, Inc.