Transcription
[Music]
[Audience applause]
Julia Racsko: Hi. I’m Julia. And, yeah, I’m very nervous, partly because you’re a lot of people (with or without the pandemic). Also, because I have never talked for 45 minutes continuously. Not just on stage; in my life.
[Audience laughs]
Julia: That never happened. Yeah, this makes me feel very tense. Yeah, I have choices to make with this tension. One would be to hide behind a couch. Flee from the situation.
I could also suppress this tension and fight through it.
And yeah, I could also choke on stage. Let’s hope I won’t do that.
[Audience laughs]
Julia: The other option would be is to build a connection with you, the audience, and with the topic I really care about.
I care more than how much I’m afraid to be on stage. Of course, it’s every public speaker’s nightmare that this connection doesn’t happen. And then I would be stuck here for 45 minutes with this tense feeling.
Of course, this is a wonderful conference, great organization. You seem like a lovely audience, although we don’t know each other that well, yet. So, I think I can make this happen.
What’s more worrisome is that if I get off stage and I have tension or negative feeling about something, the environment there, especially our digital environment, isn’t set up for that. It actually fans the tension and we are stuck with it. Which is how we might feel a lot like this when it comes to how we use our digital devices or house our digital media environment. And I’m much more afraid to end up here than to be on stage or to do small talk in the breaks.
Now, this is my big talk, so I will have three parts. The first part is: how did we get to be in this situation, what’s causing this tension in our digital media environment, and how that tension is weaponized against us? The second part will be about ideal solutions that we can do now or that other people, not me, are doing. In the third part, I will present a model of attention that can lead to nourishing a mentally sustainable digital media environment.
So, how did we get here? Issue number one: The dominant model of attention that we use in digital product design is quite lazy. There is a big mismatch between what we measure and how we actually feel about the tension because, yeah, this is a very old definition about what attention is. With all due respect to William James, no, not everyone knows what attention is.
We all have assumptions about what attention is, how ours works. Some of these are great. Some of these are very sophisticated. The number of visitors or time spent is just not one of them.
If we measure these things -- number of visitors, how many people were on this site yet, or how much time was spent there -- that’s about as relevant as going to the beach and seeing how many people are there, which is very important if I’m selling ice cream, but it doesn’t tell me anything about how it felt to be there that day.
The time spent in the water also doesn’t tell me much. And if I keep pushing back people in the water, I’m not sure that would improve their overall experience.
[Audience laughs]
Julia: I was curious. I wanted to find out where did these numbers come from. Who started first using them? Maybe social media companies.
But no, that was Google. So, when they started the search engine, they measured the number of visitors and how many links pointed to that site, to this site, where should that particular site be in the ranking of the search results; how much that site was in the center of attention.
They also wrote this in their thesis, written at Stanford, before they went on to start Google as a company. Here, in this particular quote, “Bias towards the advertisers away from consumers,” they were criticizing other search engines.
And then they started the company. They got massively successful and proved that this isn’t a good business model to follow for other digital companies.
Yeah, issue number two: Our sensibilities when it comes to our attentional capacities is used against us for profit. Now we could say that this is not a technology problem. This might be a capitalism problem.
But I’m from Hungary, so I grew up in second-hand communism. I was raised by people who were born and raised in communism. That’s one C-word, and consumerism is a third one.
I think the actual C-word we are looking for here is control. But to get deeper into that topic, that’s another talk in another conference.
What I’m interested in right now is how does this control play out in our digital environment. For that, we will need another model of attention, which is the metaphor of a flashlight.
I really like this because it shows that when we focus on something, there are multiple cognitive processes happening at the same time. We select something to focus on, which is the light. We suppress the other thing, which is what will fall in the dark.
This flashlight has direction. It has range. It has strength.
We can also show how our attention feels by how the flashlight is moving. If we are the one controlling it, if we can keep focusing on what we want to focus on, then the line of the light will be smooth. However, if it gets yanked back and forth, then it will be more like it’s on the left.
Now, what happens in our digital media environment? Well, there will be a bunch of interferences messing with this flashlight of attention. There are external ones, internal ones.
Let’s start with the external ones, so when we clearly see the hand moving the flashlight. One prominent interface example for that are notifications because who wouldn’t want to see this on their lock screen when they wake up. Am I right?
There’s just a sheer volume of them. It’s just so much, so much information, that even if we try to ignore it, it still lingers in our mind as something that we will have to deal with later on. It takes up so much space.
There are also the fake ones, the notifications that are not for me. It’s more like someone I might know is also in this social media platform. Go follow them. That’s not for me. That’s for the platform.
Or look at the bookmark or save options. That’s great. That’s useful, right? But why is it there in the first place?
Well, because we have all these infinite feeds. If I see something, I cannot find it again in half an hour, so they retrofitted. They added this extra solution on top of it. When in fact, if I’m browsing, I should be seeing an article, a longer article, when I actually have time to read an article. Right? It doesn't have to be in this chaotic feed.
Of course, losing time in feeds with notifications, that is very profitable for social media companies and Facebook, Twitter, or even Tencent. They all really value and measure this metric because it directly impacts their bottom line. It’s all in their financial reports.
Since this a dominant business model, sadly, a bunch of digital products also use these patterns when they are not even funded by advertisers.
Who knows Mastodon, for example? That is the federal, decentralized, open-source social media. It could serve us all the content and the connection in a way that’s actually for us because they don’t have advertisement in them. Nobody funds it in this sense. And yet, the interfaces are still biased towards the advertisers.
These were the ones that mess with the suppressing function of the flashlight, the dark part, and try to draw the light there. The content modification, like clickbait, that’s the one that messes with the light part of the flashlight that really tries to hold onto it.
We all know clickbait-y headlines, for example. But look at this quote.
Now I don’t know enough about filmmaking, how it affects that. But I do know that if this is valued above all else, then it will affect the quality of this attention of this connection that we pay to this movie. If the goal is to just keep people glued to the screen, then we will maximize that, and we will not know anything or much less about how much they actually care about the movie they are watching.
Now let’s move over to the internal interferences, which is when we do not see the hand that moves the flashlight. It just somehow ends up moving. For that, we need to look a bit into how our minds work when it comes to information.
For a sense of safety, we are always trying to be aware of our surroundings, right? If I hear a car honking, then I would want to hear that and not keep reading my book and get the hell out of the way. However, this is a vulnerability that can then be exploited because we are always looking for more information. We are foraging for more. And due to that, we have a scarcity mindset when it comes to information.
How does that look like? Let’s see a bit more visual example.
Imagine there’s a monkey sitting on a banana tree, and they want to eat as much banana as they can. When do they move over to the next tree, to the other one? Well, that depends. How far is the other tree? How many bananas are on the other tree?
To maximize the intake, if it’s just a little bit more bananas, then it has to be at a very short distance. But if it’s a very full tree, it can be all the way here. The monkey will still go over because it’s worth it.
How does that look in an academic chart? Yeah, this is how it would look. There’s an expected time of transit. There’s richness of resource.
Anyway, this is used against us, this information hunger, in a way that this is a mindset that’s for a scarcity. Our information environment is not exactly famous for its scarcity these days. It’s more a very abundant environment. Even our phones contain several bottomless banana trees or infinitely reaching information sources. We do not have to go to a public library to get a new book because we are out of things to read.
We have this information hunger. I believe this is also what Simone Weil called our “violent repugnance to true attention.” That’s why, also, the hook cycle works, the habit-forming interaction design methods where we will keep coming back for the reward of information.
This cycle makes our flashlight way more jumpy. It makes the flashlight nervous because we are constantly trying to get as much information as we can.
Issue number three: Our digital habits are counterproductive by design. We have this tension from how our attention is being manipulated or controlled. We have tension like the pandemic, just general life tension. We have the competing goals between what we want to focus on, what we are suppressing, so there is tension coming from a lot of places.
Our digital habits that we use, we expect--
Sorry. English as a second language moment.
These habits that we have online, we expect them to release this tension, and it does the opposite because what would actually release the tension is to have a moment of relaxation of flow.
This is the first half of the definition of flow from Csikszentmihalyi, one of the founders of positive psychology. Many of us know it, right?
We select a task that we can fully focus on, and attention melts away because we have a deep and effortless involvement that removes awareness of the worries and frustrations. We do not have competing goals anymore and that feels lovely.
It also goes deeper than that. This is where it gets philosophical, almost spiritual, is that because of these experiences, we feel like we have a sense of control over the flashlight of our attention or over our life. Because of that, our sense of self gets stronger.
This is what I think Simone Weil means that when we actually manage to have true attention, we destroy the evil in ourselves. I don’t think she meant moral evil. I think she meant the evil of this tension, these negative feelings.
However, we have less and less of these moments in our lives because we do not have the white space, the boredom between information sources. We erase the space where we would select a goal. Even if we do select the goal, it’s much harder to stick to it because there are just so many more interferences, destructions, interruptions.
Because of how it’s described here that this really transforms the self, it creates an order in consciousness. That also means that this is cyclical. So, the more flow I experience, the capacity for me to keep experiencing it gets bigger.
These are the lines that the flashlight could draw.
However, what we do right now with our digital devices, this is not really what’s happening because, when we feel tension, our first response is, of course, to solve the situation, escape it.
We come up with micro versions of survival strategies. Many of us know fight and flight. Freeze and fawn are lesser-known, but longer-term responses. Many of our online habits can be mirrored on these.
How? I think we do not need that many examples of the fight response if we have social media or look at politics online, so we know that, and it’s very profitable because it keeps engagement high.
Flight responses are, I would say, these hyper activities when we have like 20 tabs open and chat programs - Slack. We also check our phones, so we are anywhere but in our bodies. We are not connected.
Freeze could be, yeah, binge-watching or just very passive consumption when we get to the point where it takes more effort to stop watching or stop scrolling than it would take to stay there.
Fawn is a bit harder to define. I would say it’s more like when we reach out for a connection and we don’t have hope of having it returned. This can be fawn behaviors, or even I just start texting everybody and I know that nobody is going to reply for three hours, but I have the loneliness right now, and I have this craving.
Yeah, all of these behaviors in itself are fine. They can be useful. But the question is, how you feel afterward, more depleted or more energized?
Unfortunately, these 4F responses are also a cycle. If we keep doing it more, it gets easier to slip into them, and then we will have capacity to experience this more instead of flow.
Then this is how the digital media environment can feel if we are really stuck in this cycle.
Yeah, we have advice of how to get out of it. Detox. Go for a walk and whatnot.
But this doesn’t just go for that time span. It really affects what choices we’ll make in the future, what’s our capacity to experience relaxed flow states or agitated 4F responses.
We cannot just take a break from the negative cycle unless we actually engage in having more positive cycles. And that’s really a life question because what we attend to, what we notice, what’s in our attention, what comes through is eventually what our life will be like. If we entirely lose control of our flashlight then, yeah, this is where we will end up because it’s much easier for them to stay in this chair and keep watching than to get out and do something.
Now luckily, we are still mostly in a virtual environment. It’s just through screens, right? We still have moments of boredom. We are not fully immersed in an environment like this, right?
Oh, come on! I didn’t mean the screen thing. [Loud exhale] It’s the metaverse thing.
[Audience laughter]
Julia: Oh, come on! [Laughter] We can do this.
Okay, I don’t know what’s going on.
[Audience chatter]
Julia: Okay. At least this is still a virtual space. We are not in a physical spaceship, but not for long.
Yeah. If anybody has a Disney+ password to share Bezos, now would be a really good time because he clearly didn’t see WALL-E.
Okay. This is going to be fun.
Let’s take a deep breath. Maybe the screen will calm down, too, and I will calm down, too, and review.
We have a big mismatch in how we measure attention and how it actually feels. This mismatch makes our attention capacity biased towards the advertisers and our vulnerabilities weaponized against us. We can either pick from creating a cycle of flow with our habits or a cycle of fight, flight, freeze, fawn responses.
What can do to get out of here? Of course, this isn’t a new problem. There are plenty of ideas, solutions, little islands in the attention economy, little alternatives. And yeah, I still think that we shouldn’t unplug it because there’s so much content and connection that we love that’s on the World Wide Web. It’s just a business model that has been dominant for basically the entirety that we have known it is not supporting that. It’s supporting, yeah, something else.
And because of that, that it’s so pervasive, no individual solution is enough, so I started to look. Okay, what’s the most collective action I can think of? That would be, yeah, laws and policies.
I didn’t find much, so I looked at, for example, content moderation (How do they solve it?) because that’s also a very thorny problem (Right?) separating hate speech from free speech and trying to keep free speech alive and hate speech not so much.
This a great book on it, but I would say, yeah, the solution is -- we don’t know.
One conclusion that I did draw from it that this is not something that the burden should be put on the end-user for them to mitigate their fight and flight responses all the time. But of course, that’s not what we are doing, right? We are putting it on the end-user.
Okay, so if not laws, then what if we take different business models because, if it’s not the advertisers who are paying it, then it’s not going to be biased towards them, right?
Or we can just ask the advertisers to not pay for it anymore. But probably this is not going to work as a full-time solution.
But we can instead start paying the people who are actually making this whole World Wide Web still worthwhile to stick around. And that would be through, yeah, the creator economy or passion economy, which is right now a booming market, because the people can create direct connection with their audience and their community can signal directly what is valuable to them, so it will be biased towards them now.
This can be newsletters. This can be podcasts. This can be weirdly specific niche Discord servers. It can be anything as long as there’s someone creating content or building a community there.
What I also love about this is that there are way more people who can make a living through this business model than if they were waiting for platforms to pay them through the advertisers. There are still, of course, big names and beginners, but it’s a more even field.
Right now, it’s also a very lucrative field, so there are the bigger players like Patreon, Substack, Steady Media (where I work). We have been around a few years now. Now, we also have to compete with Facebook’s, Google’s, and Twitter’s newsletter and membership, too, so it’s definitely lucrative right now.
But the playing field for people who are actually making a living through this can be leveled even more. There can grants. This is still a dream, but I hope it will happen one day.
Now, not everyone can, of course, work just in this economy or directly for a company that provides alternatives to the attention economy, but there are always choices to make. There’s always some little influence to be had for designers because, yeah, you can still pick what you measure - hopefully.
That brings us to what can designers do (right away) because this is really the area where we are the best.
What I also find very interesting is that, in the creator economy, we still keep going back to older designs, so we use emails, we use RSS feeds to go back to times before we had algorithmic feeds.
I think we can go beyond that. We don’t have to go backward.
Okay, this isn’t what I meant. You can think about this one, that if China decides that TikTok is too additive, so let’s put another legislation on top of it, another manipulative layer, another way to control the attention of the flashlight, and that feels very incentivizing.
We have a not-so-different version when we use screen time trackers or notification managers. I mean they are useful, but that shouldn’t be the end solution.
What I really like, though, is This Too Shal Grow’s research on how to set up our digital media environment as mindfully as possible, like right now, with all the tools that we have, so we can keep -- we can create an environment that feels less manipulative.
Of course, I also really love this book from Pamela Pavliscak that presents several design patterns that Mastodon could have used because they had every liberty to do so, every freedom.
We can also prototype experiences that create a sense of flow. This has very low levels, so this Spa at Home, for example, it’s an experience that is mostly analogous, but it happens through a video conference.
But we can still prototype experiences this way before we make them into a digital product, which is what the TRU LUV studio is doing, for example. They particularly focus on creating attend and befriend digital interactions instead of fight, flight, freeze, fawn. So, how could our digital rituals look like?
If we keep prototyping this, and other similar interactions, then we end with a metaverse that’s actually worth going to because they will help build connections and build experiences of flow. That would fit the protopia framework that’s amazing. Check it out.
Or we can even take into our own hands and start experiencing flow through building. We could build a metaverse in this. We don’t have to wait around for big corporations, right? Then we can figure out what could be the healing interactions in such an environment.
Yeah. This was, again, a lot of information, so there are legal solutions that I’m not crazy about. There are business models to choose that actively work against the attention economy, or just modify the existing ones. Try to figure out what to measure, what to value. That would help create an experience of flow.
Then actually prototype these interactions. For that, I think, yeah, where can we go? For where we are trying to go, we will need a new model of attention that’s even more complicated than the flashlight.
Sorry about that, because this is very useful in the present moment, but how it feels, I kind of don’t know how can I represent the quality of these lines.
See the flashlight? It goes away after a second. And I really wanted to show something where I can see the flow of attention and mental energy.
Now I toyed a bit with the idea of, okay, what about biometric data? But I threw that away for two reasons.
One is that the flow experience can really be anything. It can be extreme sports. It can be gardening. I don’t think they would have the same heart rate or same brain waves. So that’s not going to work. I also don’t think I can create a model that fits everyone.
Then I also remembered another dystopia, which is, for a certain computer, deep thought was trying to answer philosophical questions like about the life and universe and everything. Now the sense of self and self-control are philosophical questions, and I don’t want to build a model where it will answer me always 42.
I didn’t want to build another dystopia that can be mistaken for a product roadmap.
[Audience laughs]
Julia: So, I took this approach instead, which is data humanism from Giorgia Lupi, which is that each -- you have to craft the dataset as well. It’s very individual. It’s not for everyone. It’s not trying to be universal.
But this approach can show me how I can view attention as a connection because, ultimately, that’s what it is. It’s the attention. It’s a connection that connects me to the outside world. It’s how I get all of my perceptions, my reflections, my memories, emotions. Everything has to flow through this connection.
I found Santiago Ramon Cajal’s drawings to be a great visual inspiration. These are neurons and their attachments. This is actually the first recording of them ever. Then I just had to imagine that these lines go through the skull and out in the world.
First, I tried self-tracking. That got really weird because I’m tracking my own tracking, yeah, so I switched to observing others, how they were behaving, what they were focusing on, what they chose to ignore, and with how much effort, how hard was it.
This visual language became this diagram - eventually. The colors are quite arbitrary. What’s important here is the line quality, which is, the horizontal axis is how much I care about the object of focus, how emotional is this connection? Is it very passive or very active? There are huge waves and spikes if it’s very active.
The vertical axis is for how much self-control do I need. If it’s a bottom-up goal, it’s something that I react to--it’s something I’m very drawn to that, really, the flashlight of attention sticks to it) then the lines will be very soft and bubbly. If I need a lot of self-control, then the corners will be angular.
How does that look like in a drawing? Here, for example, the girl with a headband, she has a conversation with her peers and the conversation flows easily between them. But the red lines of the little noises do not reach her because it doesn’t seem to bother her.
I can also show that two people can have a totally different presence in the same space. Here, this person is in their own world, very focused, while this other person seems more distracted, checking her phone quite often, all over the place. She seems nervous. There are people in the back, chatting, disturbing everyone because this is a library.
I can also show how much is there to ignore in a shopping mall. I mean, yeah, it’s a lot. But it’s so quiet and nice in a conference.
Now I wanted to make this model more accessible to others, so I have run a few workshops with that just to see that, get people to think about how their attention works. How are their energy levels between different interactions? Also, because I was very excited to find out about their interpretations or their insights about it.
The first exercise that I gave as a warm-up is to, yeah, read the book and draw how that feels. Some people seemed to be more distracted doing it in an environment where there are other things going on. Others prefer to sit in their own little cloud or just draw the lines and show that, okay, it took me a while to get into the flow of reading. But now it’s going okay.
I particularly love this one. Someone really went overboard here because they wanted to show that, okay, reading is one experience. That’s more -- that’s easier. To maintain, that’s more messy. I’m more drawn into it. But for writing, that’s another state. Creating, that needs more self-control. I need to have more ordered consciousness for it. It can also show how reader and author really pay attention to each other or attend to each other through time and space.
The second exercise was, yeah, how did your last 24 hours go? How was your last day? Because the flow state says that the sense of time disappears. But what is normally your sense of time?
Some people drew an even based little calendar that, okay, this is what happens and this is how I feel about it.
Some people drew something that was more about screens and spaces. Yeah, you can see that this was deep in the pandemic during lockdown when all the activities were indoors in their own apartment.
What I also found interesting is when someone started to draw circular calendars so that one day is a cycle. It has an ebb and flow of energy and it’s repeating.
That got me thinking, “Okay, what if our calendars were also not linear?” because that makes sense for productivity for a work calendar. But for a personal calendar, that could be cyclical. Right now, the only interface I know that uses something like this are period trackers.
We have daily cycles too. Come on.
Then we looked at the ideal Sunday. Okay, what is the attention or pattern you would like to have? If you wanted to have as much flow experience during the day, how would you structure that? Because, of course, that’s not something that can be maximized to maintain for an entire day.
Again, a circle for a calendar that has some flow in the beginning and the middle and more exciting or more or other things in the end. I also really like this drawing because this shows the desire that, okay, my usual day is pretty full. I have a lot to attend to. But in my perfect day, that would be a more quiet, serene experience. There would be just less to focus on.
Yeah, what is clear from this drawn model is that, yeah, we cannot maximize just for one pattern because then we would have a very boring overall view that we are always in the same mindset. Variety should be the aim here.
We can work on drawing those lines and cultivate more and more of these connections. For that, we need to examine our own assumptions about attention and our own biases about it. What are the ones that we want to cultivate in our own lives? How can we be the ones holding our flashlight and not just trying to control it with outside tools? How can we draw our own attentional pattern? How can we maintain this connection, a rich connection with the outside world?
Once we have that, yeah, we can start bringing those new models back into our interaction design. Yeah, if we don’t do that, somebody else will be very happy to do it for us, to engineer it over us.
If there is one thing that I want you to take away from this talk is that please be bored. Otherwise, you will be very boring.
[Audience laughs]
Julia: Thank you.
[Audience applause]
