- [John] So, welcome to the first of two round table discussions.
This is the world of futuristic user interfaces in film. We are the team at
Perception. We've been working closely with our friends at MAXON and Cineversity
to put together a series of tutorials about how to create various futuristic
user interface elements using Cinema 4D. My name's John LePore.
- [Russ] Russ Gautier. - [Doug] Doug Appleton.
- [Justin] Justin Molush.
- We're the core creative team at Perception, and today we're going to talk
about designing futuristic user interfaces in film. Each of us has had quite a bit of
experience working on a number of different films, from Iron Man 2 to
Avengers, Captain America: Winter Soldier, RoboCop, Europa Report,
Men in Black, making these futuristic displays appear on screen and, you know,
tell the story, move the plot along, create a futuristic context, and also,
hopefully, look cool while doing it. So, guys, what is a FUI?
- FUI stands for a futuristic user interface. Some people call it a fictional
user interface or a fantasy user interface, but I think in a practical
sense, what it actually means, what it is in film, I think it's a storytelling
device. And it's a tool that, when used, I think, in the best possible ways,
used to help further tell the story that the film is trying to tell. So, you
know, if your character is injured, you would pull up an FUI to show how they're
injured, or if you're trying to find something, you would go to the holomap and
find where on the map that item is located.
- So, yeah, particularly anytime that there's a story that utilizes technology
in its storytelling, it's not unusual to see some use or some implementation of,
you know, a computer, a smartphone, a hologram, something that is helping to
solve problems, and for the filmmakers to help tell the story. So these FUIs,
where do they come from? Who makes them? Are they real software?
Are they real computer systems that are built for these movies?
- Yes and no. They're created by some people, just in post,
after they shoot everything, will add these FUIs after the fact and composite.
Now we're seeing a lot of films shooting with on-set playback where the actors can
actually interact with these user interfaces, and then sometimes those get
replaced and composited in post.
- So those would be sort of like mocked up or maybe limited functionality interfaces
that are working on set?
- Yeah, you would know, it would say, "This person, they selected these three
options," and that's that scene. So you would only build in those three options,
as functional options and everything else is just kind of dressing.
So that's one way to do it. Then the other way is, you know, you have your person
flailing wildly on a screen, and then in post people will come in and put buttons
where their fingers are supposed to be and try and work around...
- So, for instance, we've had the opportunity to create an FUI underneath
Robert Downey Jr.'s fingertips, and he's great at improvising and moving his hands
around with certain gestures and interactions, which, ultimately, for us,
it's a little bit challenging, but it ends up dictating how the interface itself is
going to be laid out, because he's already performed these actions. Luckily,
he's, you know, incredible at it.
He's got a knack for this, you know, casual, natural interaction that
feel plausible but is still energetic, it makes for a great performance on screen.
- The vision for these things comes from a bunch of different places,
from the director, to production designers, to studios just like ours,
where, you know, they come to us with a problem that they need solved.
They have a story beat that needs a visual
interface element associated with it to help solve a visual
problem, to help guide the audience along the ark of the story.
And then we get to play, we get to make some fun things.
- So what's the first FUI you remember seeing in a film?
- For me, being a child of the '80s, it was Star Wars: A New Hope.
- Was there one that stood out to you? Because there's a bunch in the Death Star
and in the various space crafts, and there's holographic chests.
- Yeah. I mean, the holographic chest stuff is really cool,
but for me, the targeting interface in the Millennium Falcon when Luke is shooting
at TIE fighters, and then the Princess Leia hologram coming out of R2D2 for me
were like, "Whoa!" Like, I had never seen anything like that before.
- Which was like a really crucial McGuffin for the whole story.
- Yeah, absolutely. It was a crucial, crucial story element,
both of them. All of that stuff. I would even say, even the chess game,
you know, as sort of tertiary as it was, helped lend a...
- It made the universe more real.
- Yeah, it fleshes everything out, you know, just like being in any of the port
cities or anything like that, you see tons of alien creatures that add texture and
depth to the universe.
- FUI has the opportunity to bring this extra sort of context or legitimacy to
these fantastical worlds that we're seeing in these stories.
- For me, it wasn't the first one I ever saw, that was probably Star Wars or
something similar, but for me, the one that stood out to me was Terminator, the
original Terminator. You know, you've got targeting systems and analyzing a
space around you and doing sort of like situal awareness kinds of functions,
but also it even had little splashes of humor worked into it, not that the
interface itself was funny, but that they used it to do humorous things.
That certainly left an impression on me. And also, it's got a specific tone to it.
It's like a terrifying heads-up display. - It's straight up murder vision.
- It is murder vision, yes.
- Which is great. I think for me, the...like you guys were saying,
probably not the first one I ever saw, but the first one that stands out to me is
from Hackers. And I think that is, in particular, because they go to this room
that is just a giant wall of FUI. When you look at, there's several different kind of
design esthetics to it, but the one that stands out the most is when they're
searching for files, they're flying through this glass computer city, and
it's like the most impractical way, I think, to, you know, find a file.
But it's really cool, unique way of doing it. That really stands out to me.
- Well, you know, and you can imagine the reason that it's laid out this way, is
because lines of code is not really photogenic on screen, so what do you do?
You create like a city or a video game level that you need to progress through so
you can have a visual indicator of progress or challenge.
- Yeah, I think the other thing, too, as well, is I think they were looking for
the trash file, but it wasn't just the trash for everything. It was the trash
file on this one specific thing for this one specific thing. So, you know, it
wasn't just like, "Let's find trash." It was, "We need to find trash, but it lives
in a spot." You know? So they're flying to that spot to find it.
- For mine, it wasn't, again, like you said, not the first one I saw,
but the first time I really kind of...it became more salient for me was
Minority Report, where it created all these new ways of interacting and new ways
of thinking about this, and it kind of like delineated from, you know, "We have
these past examples of all these background textural elements and all
this technological stuff that's kind of evolved." But
in that, it was very clear that this was this very advanced technology that he was
interacting with in new and unique ways, so it very much like, "This is a really
drastic and massive departure from these simplified systems," and it really brought
it forward in the future.
- That one certainly made a mark.
You know, it's something that's frequently referenced in
It was considered a precursor to multi-touch. And we've talked about this
before, but Minority Report was not just a bunch of graphic designers making FUI,
it was a team of technologists and futurists brought together by Spielberg to
envision these technology concepts for the film. What's your favorite FUI and why?
- I think my favorite FUI has to be from Star Trek: Next Generation, the
I think, to me, that is one of the most unique user interfaces that I
think we've seen up until then and since then. I love the color palette of it.
It's like these really nice light sort of pastel colors that we don't see in user
interfaces. Everything is now kind of blue, green, cyan. It's soft, it's round,
it uses negative space. It doesn't fall into, I think, any of the tropes of any of
the UIs that have come out since then or before then.
- It's a very vector, flowchart sort of layout.
- Yeah, it's very flat and clean. - Russ, favorite FUI?
- Alien, for sure. The original Alien. I feel like Ron Cobb's...
- You've got your Nostromo jacket on.
- Absolutely. I dressed appropriately today. Yeah, I just think his thoughtful
design that went into every single little button and every screen in that entire
thing, that entire movie, was just beautiful. I loved it. I'm personally
partial to that kind of retrofuture esthetic. I really find that a very
appealing thing, just purely from an esthetic standpoint, those old CRT
monitors with basically just a step above a green
screen, more or less. You know. Super rudimentary. There's something
really beautiful about the simplicity of those things, and that kind of like
rawness to those elements, just as physical things...just awesome.
- It was made in a great time for graphic design, as well,
with a lot of, you know, Swiss inspiration, a lot of bold minimalism.
Justin, a favorite?
- It's a little nonconventional in terms of UI, but the scene in Children of Men
where he's entering the dining room with this rich old guy and he has a son
and he's, you know, manipulating this kind of Rubik's Cube of technology with
this analog sort of interface he has matted on this hand.
And it was the first time I saw this really nice
interfacing where, yeah, you saw it, but even though you knew what world you were
in, you're watching this movie this entire time, it still was a little vague and
ambiguous to you exactly what it was doing. But it noted a higher...
- Well, [inaudible] the story points as well, which was that this kid sitting at
the table was just buried in his technology.
- Just enamored.
- And that's amazing, considering that Children of Men predated iPhone.
- Yeah, like 2006.
- You know, and that's what...I think when we think of modern-day teenage...
actually think of somebody who's just buried in something and they
nailed that perfectly in Children of Men.
For me, favorite FUI is the Codex from Man of Steel.
It's one of the very first things seen in the movie. The movie begins with a
couple establishing shots of Krypton, I think. And then literally,
the first thing you see at human scale is a baby monitor rendered as this
three-dimensional sort of particle system.
I loved that element so much that it instantly
made me buy into the entire film.
I'd like to think that I'm a little bit of a harsh critic when it comes to film,
and I think I'm one of the only people I know that holds Man of Steel in a really
high regard, and certainly that moment, that FUI, that was the first thing you see
in the film, that just hooked me and had me on board. And after that I was like,
"That's amazing. I will accept anything else that you feed me for the next two
- Well, I will give it credit with the UI. It was, I think,
incredibly unique, and that's something with all of these, and if there's
something that we haven't seen before, haven't seen since, I think that
Man of Steel was a perfect example where not only does it have this limited,
monochromatic color palette, it's silver metal, but it's also physical.
You can reach in and grab pieces of it.
You know, it turns into the Superman symbol that they
have to plug in later on. And I think that's something really unique about that,
is that you can actually touch that interface. And to me, that's kind of like
the holy grail of these FUI, particularly the holograms.
- I mean, to me that's the holy grail of just interaction in general. It's
having flexibility of a touchscreen but the tactile response that we get from
switches and knobs and things of that nature. So what have you never seen in an
FUI that you would like to see?
- For me, I really would love to see just the organization of information,
just the paradigm change a little bit and see where we can push that organization.
Right now, everything is still very much based on the grids, because it's what we
understand, it's what we use on our computers, it's what, you know, we've been
using since we left the era of DOS [SP].
But I would love to see interfaces that are organized,
inspired more by things in nature or physics or math or fractals or any of
those kinds of things where you're really getting a dramatic departure from what we
hold so dear as informational organization.
- Yeah, I think something I'd like to see, it's incredibly vague, but
I would love to see sort of another major shift in the way that we interact with
user interfaces. I think Minority Report was one where that's what gets us into the
UI that we use now, is they're doing multi-touch gestures. There's the
pinch-to-zoom, which is everyone uses pinch-to-zoom now.
So I think when that came out, that really shifted the
way people thought about interacting with user interfaces.
I would love to see another major shift, and I don't know what that would be,
and it's probably something that visually is not interesting on film.
I think that major shift is going to be something that is maybe more organic,
built into your body where you're thinking about your interface.
- So that's like a challenge, is you still need something to be a performance on
- Right, you still need something interesting.
- You need something, ideally, that the actors can interact with.
And I think we saw a little bit of it, and I'd say one of the films that will be
looked back on as a future predictor or mile marker indicator was the movie Her.
Voice recognition is obviously becoming widespread today with Alexa and Siri and
whatnot. But Her particularly had this natural fluidity or this very human
interaction with machines that I think will be something that we'll see a little
more of in the real world, as far as technology goes.
- There's a pretty extensive list of gestures, interactions,
things that we expect. There's a heuristic set that we draw on that.
We kind of know, even without...let's say there's a visual indicator like,
"Oh, if he's swiping left or swiping right, we know kind of what that means in
how he's, you know, navigating through this interface."
But there's a lot of really cool technology
now where I'm finally able to get a high enough resolution in terms of,
you know, like Leap Motion, where you can load up the package in Unity, put
it on the table, and all of the sudden, you have all these little micro movements
that you can record.
So I think there's a lot of exploration for how we can advance these subtle micro
movements, or all these, you know, subtle communications that we as primates,
our hands, are so good at these very fine, articulate movements and motions, that we
could bring that into the space, you know, into the...
- Well, the technology is getting to a point where we can invent a whole new
vocabulary... - Like a sign language.
- ...for using an interface.
- Yeah, and I think that's massively powerful, and you mentioned that, you
know, sometimes there needs to be a performance on screen, that might
not be the...might not always need to be the
case if, all of a sudden, over the next decade or something,
we develop this standard vocabulary of gestural responses, whether we have hand
trackers on our HoloLens items or our computers have these sensors,
then there is this possibility of this, you know, language and vocabulary
developing where it's like, "Oh," these just natural,
gestural movements, and we just know them.
- So when we've collaborated with real technology partners around gesture
technology, we always try to stay away from Minority Report moves, and I think
we always keep coming back to the analog of sitting at the poker table and subtly
gesturing to the dealer, you know, "Hit me," or raising an eyebrow.
Cool, well, some very interesting ideas.