#btconf Düsseldorf, Germany 07 - 09 May 2018

Jan De Coster

Jan De Coster grew up with a fascination for physics, science fiction stories and hacking things. In college he realised that all the stories around science were often far more appealing than the theory behind them. In the mid 90's he started on his first multimedia productions and eventually evolved to working in advertising, where he invented stories and games for online campaigns. This is when he developed an interest in character design. Soon his interest in virtual worlds tired, and he started to invent interactive machines in the real world, Robots in particular, because they are the perfect match of his interest in physical projects and character design.

His Robot characters are now travelling the world and are active in all kinds of environments, like galleries and museums, but also car factories and emergency response units. On his quest to make Robots a more widely accepted creative medium, Jan is now teaching young and old about building Robots, focusing on the design and the process, and also on the way they make us feel.

Want to watch this video on YouTube directly? This way, please.

In the Brink of Consciousness

Advances in deep learning and artificial intelligence are moving very fast now. People are starting to wonder what role Robots will have in our society. While many researchers are focusing on the very practical side, we have to wonder how we will interact with the Robots on a social level. How will we feel about them, and even more important: how will they feel about us?

Jan will give an overview of his robot projects from recent years, giving insights in their production, and in their effect on their audience.


[Digital music]


Jan De Coster: Yeah, so I’m very amazed by the structure in the presentations of all the previous speakers. Mine is just a total mess, so prepare for 45 minutes of chaos. That’s how my mind works.

We’ll start with a very simple question that I’ll come back to in 40 minutes. Do you feel real today? Most people -- many of you didn’t feel very real this morning, early in the morning. Maybe now you do. But, that is a question that keeps me busy a lot of the time, and many of my robot friends too. So, I hope I’ll give you some ideas to think about later today.

I have a one-man studio that’s called Slightly Overdone. I’m based in Belgium right between Antwerp and Brussels, and I’ll show you a quick video now just to give you an idea what I’ve been working on for the past 20 years of making interactive projects. Here we go.

[Rock music]

Jan: So--


Jan: Whoa! Thanks.


Jan: So, one part of my job has always been about making physical stuff, physical installations when working on interactive projects. And, I’m proud to be the first probably today to show an old Flash logo here because I was a Flash guy, and one of my first physical projects I could work on was on the ships of the Belgian navy.

They had a specific problem making a local application to show dangerous situations aboard the ship. And, the thing with the military is, they don’t care about the technology or what the name is, who owns the technology. They just care about their specs if the solution works in the end, so I could use Flash for this. Before that, I was only using it, like anybody else in the ‘90s, to make an “into” - into - you could skip then.

Audience: [Laughter]

Jan: This was a real military application. We got to test it at sea. We were in the North Sea for three days testing my Flash application, which was just an amazing story right now, 20 years later.

But, it got me into working on physical installations. This was a project we did where we had an interactive story. But, instead of just sending a link to clients, we sent the whole case--a little bit inspired on James Bond and Mission Impossible stuff--to the client where they could then experience the whole installation on a device that fit the whole concept there.

We would send a box with a battery inside to a client, and then leave it there for a few days if the client was there to open it, and then hopefully sell something. This is totally unthinkable right now. Cases would be blown up and sent back, and we would all be in jail, sadly.

Audience: [Laughter]

Jan: But, the ‘90s were really cool. This was early 2000, so we could still get away with this stuff.

The same with this installation. Dodge was briefly a brand in Belgium. They were trying to sell their cars. And, their idea was, we need adrenaline and we need exciting stuff. So, we strapped a Dodge Viper to bungee cords and then put people inside. We filmed everything. They were falling to their death. And, it was before YouTube, so we sent it then all to their website, to the Dodge website so they could show their last minutes when they were still alive. And, that also got me a reputation in the agencies I was working then to do physical installations.

Another example, it is a box that produces a flame, and people would push a button on Facebook. A flame would go on there, the black box. The Mini would roll down after a few days, and the last one to push the button won this car, so people pressed the button almost 30,000 times there.

And, it was running in the winter day and night, and there was a camera pointed at the installation, so people could see it was actually really a flame produced there, not just a GIF that was repeated every time. And, this was really important. We had to show people that this was actually happening. This was live, even though you were just watching a small display on your screen.

The first robots showed up in this installation where we had an industrial robot arm picking up little bricks, then a message you would like to send to one of the buildings. This bank was, yeah, supporting, like a kindergarten or an elderly home. A message could be then sent to this robot. It would engrave the message on an actual brick, and this then became a part of the actual physical building.

I did some experiments with the interactive clothing. This was the pine cap. I would put on this cap and then, when I got excited or angry, the spikes would go up and go down again. It was inspired by riding my bike through traffic. And, if you’re almost run over by cars, it’s really difficult to then put up your hand then give them the bird. It’s very hard to do, so this was a solution for that.

Audience: [Laughter]

Jan: I made a more female version, too, with an artificial magnolia flower, which didn’t have so much success in traffic, but found some other, yeah, cool -- it opened some other opportunities for me.

The older part of my career was making characters and stories, and making games in Flash, obviously. I made hundreds of Flash games with fun, little characters. I had really a lot of freedom at the agency where I worked. I just designed the characters, but also their background story, and also developing their, yeah--

Audience: [Laughter]

Jan: --their story when they came into reality and when they came into the real world. These two worlds collided six years ago now when I got a call from an agency asking me to make a robot. I had never thought about making a robot myself. I’m not an actual engineer. I don’t have a degree, so I have no idea how to build a robot. But, the agency had no idea who could build robots either, so they said I was--

Audience: [Laughter]

Jan: --the best choice they had, and they needed it in five weeks, “So we would call you, man. See what you can make of it.” And, this is Yummy, so I started making him at that point six years ago and, four weeks after that, he was finished. And, he’s a kitchen robot, but he doesn’t cook himself, so he does--

He looks like he belongs in a kitchen. He’s made out of elements parts that you could find in a kitchen, but he doesn’t cook at all. He’s the host on the website for kids, and children could present ingredients they could buy in a supermarket chain who were bringing for this robot, and then Yummy would produce a recipe from somewhere around the world that they could make with these ingredients.

Yummy belongs in the kitchen. He’s fun, and he attracts kids. He needed to be finished in four weeks’ time. I’d never built a robot before, so I had to start working with stuff I could find in the store right away, so just pots and pans and blenders.

When it was finished, this is my son Lucas, the one on the left, obviously.

Audience: [Laughter]

Jan: Yeah. When it was finished, everybody was really happy with the project. I had my money, and everybody is glad.

But then, the agency said, “Yeah, it’s done. We’re not going on tour with this robot, so do whatever you want.” So, yeah, Yummy became a part of the family. He stayed with us, and that’s where amazing things started happening.

We now had suddenly an extra family member, which came through very strange nonbiological ways in the family. And, my kids and, of course, the family accepted him as part of our, yeah, household. But, other people were interested about his story too.

I met -- I’m teaching him to drive here.

Audience: [Laughter]

Jan: People were really curious about Yummy, what’s going on with this robot, what’s the story. He’s not useful at all. He doesn’t do anything useful except for being there and maybe moving a little bit. He’s not actually driving here. I just flipped the image.

Audience: [Laughter]

Jan: But, people in England always believe that, so they thought they brought me to this idea. “Is he actually driving?” No, in the rest of Europe we have the passengers on the right side, so that’s how we solve this.

I started making more robots. Herb, you see at the count of two, was just lots of inspiration that was left over from making Yummy, and I needed to make another robot, so this is Herb. Herb does even less than Yummy. He’s all about just presence, being there acting cool, being the hipster hanging out in the forest at Easter, obviously.

Audience: [Laughter]

Jan: He doesn’t do anything useful. He moves, but he has some battery problems here, he claims, but normally he moves more than he does now.

These are Bill and Ted, and more and more robots started emerging at our home, and then I was more and more interested in finding what is still a robot and what do I enjoy making and what do people still accept as a robot.

One thing I noticed was that every robot I made, made people think about, yeah, what does he do, and does he need our help? And, I noticed that people like to reflect their own views on the world with interacting with this robot, which is very interesting to notice.

This is Robin. Robin, I created to accompany Rachel, who is also out there at the counter, to a festival in Moscow in Russia. They were going to be there for seven months, and I thought Rachel is a little bit of a delicate character, so maybe I should make a friend and send with her.

I also was thinking about, how can I make this, sending a robot to Moscow; how can I make this a little bit more interesting with the background story of this robot? So, I decided to make Robin gender fluid, which was just to mess with the Russian people a little bit.

Audience: [Laughter]

Jan: Because--


Jan: Yeah.


Jan: Because I was still thinking that it was also illegal in Russia, but it turns out it isn’t. It’s only a problem, apparently, if you want to get a driver’s license. But, it doesn’t matter.

What happened here was, people were having mixed reactions. “Why would you do that to a robot? What’s the point here?”

Now, the robots like mine don’t have genitals, so it doesn’t have to do anything with their sex, but they all have a gender, which is fairly normal to develop an identity of a robot. So, making a robot in gender fluid confronts people with this concept in places where they weren’t expecting this.

People are taking their kids to an exhibition. “Oh, look. A nice, little robot. It’s called Robin,” and they read the description. Then, suddenly, they have a conversation with their kids about something parents weren’t expecting to have at that point, which is really fun for me, which is exactly what I hope robots will do more and more; opening conversations about our relationship with each other.

Robin is now part of a very cool exhibition by Vitra, the furniture company, with lots of very cool robots touring Europe and probably going overseas soon. She’s on tour for five years now. Here she is at the entrance of the exhibit. They’re now going to Winterthur, I thought, and then next to Lisbon next year.

This is Jeff, and Jeff is also at the counter there. He’s a photographer robot, so he takes photos. He was sent to Peru for four or five months a few years back. He takes photos when you interact with him, when you come closer, when you come close to his face, and then he posts these photos to Twitter.

Every morning, I woke up with hundreds of photos of smiling Peruvian kids, which was really a fun way to start the day because they all look so happy. Then, he made a comment with this photo and then posted to Twitter. The comment is written in a language only he understands.

Audience: [Laughter]

Jan: Still, Jack from Twitter tries to translate his comments and often makes it into a language Indonesian or Hungarian, and I’m still waiting for the moment where he writes something really offensive--

Audience: [Laughter]

Jan: --and I will be in jail in some exotic country. But, it’s kind of a commentary on the idea that people often experience daily life through their camera, through their phone. With Jeff in the room, you just do whatever you want, and you see, at the end of the day, which interesting things happened in front of his face.

Nor Bert was an exercise to start designing and building the robot’s head from scratch, 3D printing and laser cutting everything because--


Jan: I’ll put that a little bit lower -- yeah.

Because I had a lot of fun just reusing stuff I could find in thrift stores or just in stores around me, but often these found objects posed some problems when you want to install lots of equipment inside the head of this robot. So, this one is really built on maximum accessibility. The two sides come off, and that’s something that stayed in later designed I worked on too.

But why should you design a robot? Yeah, why is the design, especially the character design, so important for me and for many other people too?

I recently discovered this book by Max Tegmark, Life 3.0, and it’s a really cool book about artificial intelligence. Everybody should read it who is a little bit interested in these things. But, one thing that really popped out for me is the reason for the title of the book. What he believes is Life 3.0.

When we have Life 1.0, the biological life, just bacteria, we have no control over their software or their design, just existing and reproducing. The real Life 2.0 where we have control over what we learn, what we read, and what we teach to our kids. He expects Life 3.0 to emerge really soon now where we also have control over the hardware, changing our bodies, changing body parts to make our life more interesting or even to survive at all in a changing climate, in a changing world. That’s why I believe that thinking now about the design robots could get is very, very important if we don’t want to end up in a dystopian world with lots of ugly, dangerous robots.

I’ll have a drink.

Two years ago, I wanted to make a movie with the robots I had at that point because I was really interested in bringing the stories also to other parts of the world where my robots couldn’t get. And, I thought of a story, a simple story to just show what robots can do in live action filming. But, it was really at the same moment where the Paris Climate Conference was starting and where the first major terror attacks were coming to Europe, to Paris.

Suddenly, this experiment became a very personal story for me, and this short film tells a story of a bunch of robots that are left alone in the world. All the humans are gone, for some reason, and they’re telling us that they miss human beings because there’s still so much to learn from human beings. They need us, these robots, right now. That’s a very important role we have to play right now if Robots once get the huge responsibility to take care of the planet for us.

You can see the whole film on my site. We won’t get into that right now.

But, one of the main characters in the film is Remy, and he sits a lot in our living room. And, my wife likes him there because he scares away the burglars, she says. I’m not sure what he will do when he meets a burglar. Probably read some David Bowie stories to him.

Audience: [Laughter]

Jan: But, he is accepted as a presence in our living room and as some character that sits there a lot. This story then reminds us, me, about another robot I made a while ago: Steve. It was the largest robot I had built up to that point.

Steve was a little bit inspired by Steve McQueen and a ‘70s sports car, a naïve character that doesn’t understand the world really well but wants to learn to know people. Combining both technologies, so you’re reusing existing stuff I can find like bicycle helmets, lamps, and stuff, but also making new parts from 3D prints and laser cutting.

He was built for a movie theater chain in Belgium, Kinepolis. The idea was, in the beginning, that he would hang around in the movie theater at events and interview people. We controlled this robot with two human beings, so I’m somewhere around because he weighs more than 100 kilos, so I have to control his wheels and see that he doesn’t run over people.

There is a voice actor backstage who makes interesting conversations with people. Now, we could use an AI system to have him speak. But, the thing is, AI systems are not very funny today and most of the time super creepy.

Audience: [Laughter]

Jan: So, that’s why we use an improvisation actor to make cool conversations and really have it react to people’s clothing and the way they looked and stuff like that. But, the thing is, people came down to me and they asked me, “What is the AI engine? That’s really amazing what he can see, what he can do.” These are technically skilled people who were at these events, and that is just simply because they see a robot, they see it move, and they see it’s clearly not a human being, so everything around it must be artificial too. They want to complex the story. It has to be something amazing, so the voice has to be coming from this character too, and I hope one day we can actually make him operate like that and have him speak through an AI system.

Through all these jobs, I came to a job I did four Audi, and I will run the video for this one.


Female: This is Walt. Walt wants to take your job with a smile on his face. Walt is not ready to take just any job, yet. But, for the past eight months, he has been working at the Audi factory in Brussels. He helps the human factory workers to assemble the Audi A1 car bodies.

Walt is surrounded by more than 100 other robots. The big difference is that they are all in cages while Walt sits free as a bird amid his human coworkers. He is a compliant collaborative robot or Cobot, in short.

He recognizes his human colleagues when he sees them, greets them with their name, and looks at their hands so they can give him gesture commands. Besides his friendly face, Walt can also display icons or short animations to clearly state his intentions. Walt is part of the end result of a two-year project called ClaXon with partners like Audi, imec, Melexis, SoftKinetic, VUB, and RoboVision.

RoboVision played a crucial role with groundbreaking gesture and speech recognition through deep learning and with the deployment of their 3D imaging systems. Slightly Overdone was responsible for the character design, the animation, and the integration of Walt’s head and neck.

The face is the single most important instrument for direct communication between persons, persons like humans and like robots.


Jan: So, especially that last comment upsets many people because, how could a robot possibly be a person? But, it’s absolutely a person at this factory. When I made this video, it was only eight months, but now we are more than a year later. He’s been working there for 20 months now, and the people who work there have accepted Walt as their colleague. They have 500 other robots there, and there is only 20 human workers at this factory, so it brings a little bit of humanity, by my idea, to this working place where you would otherwise not expect that.

So, thinking about this, I worked together with RoboVision, a leading AI company in Belgium. When talking about AI, people get really afraid, often, because it’s becoming so powerful and it links the idea with people that robots will be too powerful one day and just take over the world.

But, the thing is, you see a lot of very exciting examples of AI, simple artificial intelligence right now, but we are near artificial general intelligence. You have amazing machines that can play Go or chess and beat anybody on the planet, but they can only do one thing. They understand nothing about the world outside or how humans work. We’re not there yet, but it’s really important to think about this concept right now because, if we don’t, they will one day have general intelligence, and they could all end up psychopaths, of course, because we didn’t teach them how to behave friendly to people, and they will probably learn from the way we treat robots or other artificial beings in the world.

Barkley is one of the other robots I brought with me who is in the lobby there and who was an experiment, together with his friend Vern, to see if we could have service robots or robots who are doing very practical mechanical jobs like defusing bombs or cleaning out nuclear reactors or going in burning buildings to rescue people. To give them also a character and a way to approach this dangerous situation with some empathy.

If you’re in a fire in a burning building, a fireman will run in and probably not scare you but try to get you calm and get you out of the building. So, this is what Vern and the other robots here are supposed to demonstrate. You can also put personality in a place that is not just for entertainment or for a fun time with robots. There are serious applications for an empathetic robot too.

This is Leo, and I made Leo about half a year ago now, and I also have a video demonstrating what he is about.

[Techno music]

Jan: So, I created Leo for a festival last summer, organized by a communication company, Telenet, and they had a festival focusing on bringing parents with their children into contact with new technologies and also making them aware of the possible dangers with these new technologies like different social media where they don’t really take the age of these children into account always.

Leo is a metaphor for the exposure of your children to these new technologies. Your kids can get inside, and you can drive really slowly and boring, so they will just shout, “Faster,” or you can drive really wild, and your kid throws up in the robot. And, it’s very exciting but, afterward, you have to clean up the mess.

That’s where people, parents, learn a lot about their relationship with their children. At least, I learned a lot about different relationships possible between parents and their children like shouting at them like they were part of the machine or just whispering to them, “Are you okay? Are you green yet? Are you happy?” So, that was a fun experience.

But, I’ve been using Leo now at different events. It’s also, of course, a super cool way to get young children excited about the possibility of becoming a robot builder of their own, which is really important to inspire children of all ages to do, to consider this because, more and more, building a robot is not about being an engineer or having a specific degree. All kinds of professions will play a crucial role in the development of robots in the future. Everything thinkable will have an application in artificial intelligence or real mechanical robots in our life.

It is really a matter of convincing and giving confidence to children, especially, sadly enough still to be said, to girls to tell them this is really important that you consider this, and you can play a role in this. It makes me really happy to then see that children get out of the robot and they’re yelling, “I want to make a robot later.” It’s really cool to observe.

ARTR, here sitting in the car, is a robot I started making two years ago. It’s the longest project I have. He’s a really difficult project to get out in the world. But, he is going to be a big star in electronic dance music really soon, as soon as his break comes. But, to show that robots can have a fun time and enjoy life like everybody else, I decided to make a little video with him.

We still have time, yeah.

I’ll show the first part of this.

[Electronic dance music]

[Car engine]

[Dance with the Dead - Eyes of Madness]

Jan: Of the hundreds of ways possible to get in trouble with the police, this is an excellent way to do this. You drive to a Porsche dealership, and you tell them the story of a cool robot, and he lights up when I play loud music, and I want to make a video with a convertible car because then people can be really sure that it’s an actual, physical robot. You can’t just fake these reflections and things like that.

Amazingly enough, they said, “Sure. We have a car here. Just be careful. You look like a responsible human being.”

Audience: [Laughter]

Jan: “Take this 350 horsepower car and do whatever you want with it.” It certainly opened the possibilities to new ideas in the near future.

So, I took this car. I drove down my street during the day 100 times, and then I drove through the city at the middle of the night hoping to meet as little people as I could because it really looks like a police light when he’s really flicking like this. And, people would actually get out of the way when we came, so--

Audience: [Laughter]

Jan: I hope I’m inspiring the law enforcement in Belgium to make this an actual thing they can use because it certainly intimidates drivers a lot more than their usual sirens do, I believe.

Of course, I was born in the ‘70s. I grew up in the ‘80s, so the whole Synthwave movement is really inspiring for me. Since we made this film -- we: me and the robot because it’s all me, the filming and the production -- but, since then, more advanced Synthwave bands have said, “This is cool. We want to contribute our music.” This is, by the way, Dance with the Dead, who are really cool guys who just said, without seeing anything, “You make a cool robot. Use the music, whatever you want.”

We’re planning now Synthwave 2, 3, and 4, and hopefully with more exciting vehicles that remind us of these cool days in the ‘80s.

That is also an application of robots that is, for me, super fun and that, if you explain this to people upfront, they will say, “What’s the point? What’s the use?” But, after you make the movie, after you put all this effort in, you get tons of really cool reactions and you make people happy, which is really what we’re all about, of course, creative people.

Even on that note, making people uncomfortable with certain concepts or certain ideas is also a responsibility for creative people. Who else is going to make people feel uncomfortable in the world but creative people?

We’re back to this question here and with the final story of Rachel, who is also at the counter there. Rachel was created for a really nice festival in Berlin, picked up last month. When they had their tenth anniversary, Rachel was part of their anniversary exhibition. She was there three months, and then they moved to Mexico for six months. She was home for maybe four weeks, and then she left with Robin to Moscow.

All the time, she’s traveling in the world, and I can’t take care of her because, if something breaks down, somebody else will have to take the responsibility to fix her. That’s why she’s mechanically a little bit more simple than the other robots, but she’s all about her personality and attracting people into her story, into her life.

When you see her sitting there, hopefully people will try to interact with her and ask her what’s going on. Right here, she is without the camera, without the eyes, but at the initial exhibition she would then respond to people first a little bit more shy and, after a while, she would interact openly with you.

Rachel is based on this Rachael from Blade Runner. Who has seen the original Blade Runner? Yeah. Still less than half of the audience.

Audience: [Laughter]

Jan: Blade Runner is just the best movie ever made, period.

Audience: [Laughter]

Jan: Yeah.


Jan: Okay.


Jan: No, serious. You can pause this movie at any frame and have something to put on your wall. It is the last movie that is all filmed in camera where everything, before CGI existed, is just filmed as it is, as it actually existed. Apart from that, there is also an amazing story, of course, with this Rachael.

Without spoiling the whole story for you, Rachael needs other people to make her feel real. That’s what’s also happening with our Rachel.

When she was traveling the world, people would send me tweets and Instagrams and tell me, “Rachel is doing fine. She’s okay.” People could also send me a message, “Your machine that looks yellow and black is functioning with normal parameters.” That’s not what they did. They said, “She’s fine. She’s feeling well here.”

People were treating her like a person. That’s what started this whole idea of what is then consciousness and how will it evolve in robots? How do you know that the person next to you is conscious? Because they are here, and you can interact with them? They came here in some way; they bought a ticket; they registered? All behavior that could also be really soon done by a robot or an artificial being. You assume that they are conscious, the people around you, because you’re able to have an interaction with them that feels normal, that looks normal, and it makes you feel in a certain way.

How long will it take before you can have the same meaningful conversation with a robot and have a feeling that this robot understands you and that you have a real conversation and that it can interact and engage with the world outside like any other human being will? That is a very exciting thought for me because I’m convinced that this is going to happen in our lifetime. The moment where we interact with the robot and where people will start thinking, “Is this conscious or is it just really good at mimicking consciousness?”

People get, some people get, really nervous when you think about this concept. Is this real? Is it just all artificial bullshit? But, what does it matter? If you have a meaningful interaction with this being, would it make that interaction any less meaningful?

People are considering this a lot right now when you think about taking care of elderly people when there are robots involved. Often, let’s be honest, that’s just guilt of the people involved speaking and not really thinking about the elderly people in question that might have a very good time being taken care of by a robot.

This is one of these amazing photos I got from a mother in Moscow who took this photo with Rachel. I’m seeing probably a lot more emotional stuff than many of you will see. This is a genuine moment they have here. As a designer of a robot, this is what you’re doing it for. This is the pinnacle of my job for the last six years.

I’d like to end with this quote by Anais Nin, who has nothing to do with robots, for the rest. “We don’t see things as they are. We see them as we are,” and this goes for non-robot designers too, for anybody who is creating stuff. Sometimes it’s really simple to tap into the emotions of people because they’re already filled with experiences and emotions just ready to be picked and engaged. That is very important when you’re making robots to have a cool interaction. Just for any other projects, this is really useful too.

I have a lot more stuff on my website, and I am posting a lot of photos and videos on most social media with this handle. And, if you want to see more of my work, go check that out. I’m still here the rest of the day and tomorrow, so please come talk to me if you want to talk about something. Thank you.