How did this blog get started?
In February 2023, industry professionals, academics, government officials and NGOs gathered together at the REAIM2023 conference in the Hague to discuss what responsible AI in the military sector actually means. Sometimes, the discussions spanned beyond the military domain into the well-trodden, but still quite murky waters of ethics and considerations of AI in general.
One of the panels briefly touched on the topic of deepfakes; synthetic media, where eg. a person’s face has been replaced by a very convincing copy of someone else, often a public figure or a celebrity. The main concern was how these fakes are being used to cheat and deceive. “But there are two sides to every coin”, they said: “deepfakes can also be used for good, to bring back to life dead loved ones.”
On the surface, this may seem like a reasonable argument to make. What if someone wants to say goodbye to a person who has passed quickly and surprisingly? Wouldn’t being able to see their face and hear them talk once more be a way to gain closure? Maybe. But as usual with us mortals and death, there are always an infinite amount of considerations to take into account – which makes this a rather long blog post to even scratch the surface.
The many faces of death
Remembering the dead is not a novel idea by any means. Across history and cultures, we’ve had innumerable ways of keeping loved ones alive. From stories to post mortem photographs, from spirits to seances, from small rituals to huge celebrations dedicated to the dead – all of these play a hugely important part in our lives. In Finland, the time to remember the dead is usually around Christmas, where visiting graves is an important tradition to many families. Globally, Día de los Muertos is probably one of the most recognised examples of remembrance. The Japanese Buddhist have Obon, Nepalese the Gai Jatra. And often (but not always) held on a more intimate scale, there are the burial rites. Even though they come in many shapes and forms, from burnings to sky burials, they are one of the most sacred traditions across the world.
A burial is usually also a rite of passage, where a person is separated from the community of the living and joined into that of the dead. These rites, however, are not only for the benefit of the dead. They are a way for those left behind to regroup and to prepare for life without the deceased. There is a tear in the fabric of their social reality, and that needs to be accounted for. For a death is not only a physical event: it is also a social one, and the social death is not necessarily tied to the time of physical death. Even a physical death can be a gradual process, as for example Tukdam implies.
What exactly is a social death? It might mean a dying person withdrawing from their usual lives in preparation of their departure, or as a result of an illness such as Alzheimers, where a personhood is felt to be to some extent lost as the illness progresses. A social death can also happen as a result of casting an individual out of a society, possibly as a result of breaking a serious taboo or, on a wider societal level, as a result of discrimination and dehumanisation. In contrast, a delayed social death may prolong your life, even though you might not be aware of it: you become truly dead only when no-one no longer remembers you. Some of you might have seen the movie Coco, which, despite Disney’s ridiculous attempts at trademarking Día de los Muertos and taking a very classist view on afterlife, handled this particular topic with considerable warmth.
These notions and meanings of rituals, of moving on and of social death, are crucial when considering using AI and virtual solutions in re-creating the images of those who have passed. There are many such services already available, albeit not all use actual visual representations of the deceased. MyHeritage was a hit a couple of years back, bringing the still images of dead relatives to life. There’s HereAfterAI and Replika, which create an avatar of the person based on photos and recordings, and more closer to the point, Re;memory, which actually recreates a virtual image of them to talk with you.
As people designing and building solutions that have a potentially huge impact on not only the people using it, but also to the society and environment, we also have a responsibility to consider what implications our design choices have.
As it happens, we were at the REAIM23 conference to run a workshop on our Layers of Impact framework. Our case had to do with using AI in prioritising and planning military operations in a disaster zone, but why not try it out to probe the possible impacts of actually re-creating someone who has passed?
A framework for impact assessment
The Layers of Impact framework has been developed over time to reflect the different aspects we have to take into account when evaluating a possible use case. As it usually is with frameworks, the layers have been designed to be a heuristic for thinking, not for forcing associations. We’ve noticed that typical development projects tend to include only one or two of these layers, usually the end user and organisation, and often forget to consider e.g. the context of use or the impact of culture, or even on culture. By actively thinking bigger, deeper and wider, we can find some of the blind spots and biased thinking that we all have, no matter our background.
When going over the considerations and possible impacts, we usually divide our notes into three categories: positive impacts, negative impacts, and considerations. Considerations are meant for open action points, assumptions that we recognise in our thinking or open questions that need answering before we can move on. Based on these considerations, we’re able to formulate research questions for further study.
It is also important to note that the impacts themselves are bi-directional. That means that we must remember to consider not only the impact that the solution or idea might have on the different layers, but also the impacts that the layers themselves have on the success of the solution. For example, the underlying values and practices of a culture might eventually mean that a particular solution will not succeed, if it conflicts with those values and practices or even unwittingly commits actions that are taboo.
Not all of these layers are always applicable, nor are they equally important in every single use case. You should not feel trapped by them and forced to think of something to say for each and every one. If you’re not sure where your mental post-it should land, don’t worry – the important thing is that it’s considered, not that it’s in the right category.
This is not a framework meant to be filled out by one person. It is designed for cross-collaboration, multiple perspectives and a wide variety of voices to be heard, not for one measly designer (or a developer, or a business owner). As such, I will be discussing the possible considerations more than actual positive or negative impacts, and not all the layers will be getting equal treatment. What questions should we ask when building a concept or a solution? What should we understand? Could we reduce risks or capture more benefits by pivoting the use case idea slightly? These considerations might make this post feel a bit on the nay-sayer side, but keep in mind that the actual framework really digs into the positive side of things as well.
Considerations for the ghosts in the machine
Time to do the work! As a disclaimer, this is not a solution we advocate for. But for the purposes of this thought experiment, let’s consider the use case of bringing back dead loved ones as deep fakes through the Layers of Impact framework. Here we’re assuming that the solution would actually use AI to create a responsive avatar instead of, for example, pasting the face on an actor’s body in real time. Those considerations are another thing entirely.
1 End user
The impact on the end users is usually the one most discussed in any development project, at least when some form of design thinking is involved. It is relatively concrete, it creates some immediate action points, and falls into the realm of “customer centricity.” Reaching possible end users can also feel like a relatively simple task when compared to e.g. cultural or societal analysis. For most designers and developers, this should be the most comfortable starting point.
Let’s start with the most obvious: the selling argument of the product. Would this actually benefit the end users in being able to let go and say goodbye? And on the flip side, could it make it harder to do so? Many things contribute to the end result. What is their personal relationship with death? Do they carry any previous trauma? What is their mental state, and how could using the solution affect it?
Another thing to think about might be how familiar they are with technology and its limitations in general. One of the academic panels at REAIM2023 discussed the uncertainty that lives between expected and actual outcomes. The ways of reducing that uncertainty are either making the AI solution more explainable, which brings the outcome closer to the expectations, or education and training, which brings the expectations closer to the actual outcomes. With a solution like this, we can hardly expect the users to be the experts, or spend a lot of time training them. That leaves the explainability of the product itself. But if you reason or explain away the very reason the product exists, which is the emotional response allowing you to gain closure, does that take away from the positive impact of the service?
The actor-network theory, most known through the works of French social scientist Bruno Latour, argues that human beings are not the only ones creating social situations. Objects and even ideas have just as big of a part to play in our vast network of relationships. In this context, it means that we can’t think about the end user as the only thing actively shaping the situation – we have to consider the avatar itself as an equal actor. Technology is not “neutral” or “dormant” until we grant it agency, it inherently already has it. According to the theory, nothing is purely social or technological.
Conversations with non-human actors can go south very quickly, as we’ve seen lately with several advanced bots. For some, they are convincing enough that they deem the AI to be nearly sentient. For others, it becomes a trip into the uncanny valley with possessive undertones. Sticking with pre-determined answers gleaned from the data the company has been given will give more control over the situation, but will make the conversation itself a little less genuine – which, all things considered, would probably be the best thing to do. Keeping the scope of the interactions limited will still give the end user the opportunity to say goodbye.
This would also place limits on possible violent reactions from the end user themselves. If the avatar is actually responsive, it’s possible for an emotional situation to bring forth such raw emotions that the response the AI would give could prove detrimental to gaining closure. The lack of “genuine responses” from non-human entities, such robotic dogs or therapeutic robots, have been speculated to have an effect on our experience on empathy especially at a younger age (eg. Turkle 2011).
Last, but not least, there are physical accessibility considerations. Would only people with all of their senses intact be able to use the solution? Is it possible to gain the same experience from a purely visual or auditory experience? And when it comes to the senses, virtual reality is still not quite there when trying to produce scents and smells, which are a huge part of our lived experience. Especially if trying to create a very subjective smell, like that of a certain person, would require intimate knowledge and analysis.
2 Context
This is another consideration that designers worth their salt know how to think about. How and when is the solution being used? If it is a virtual reality solution, would it be used at home or at a dedicated facility? Would it incorporate the lived environment of the passed loved one into virtual reality, or use a more generic one, like a meadow or a beach? This starts to matter if we stop to think about the meanings we attach to places. Actually using the solution in your home or a virtual replica of it might make the experience more real, more lifelike. But choosing a place that holds some special significance makes it more of a symbolic act, building up to a ritual not unlike a burial or the scattering of ashes.
Known ritual theorist and anthropologist Victor Turner divided rituals in two categories: calendrical and contingent. Calendrical rituals are either seasonal or otherwise follow some culturally defined point in a cycle, and contingent rituals are the ones we hold in times of individual or collective crisis. Funeral rites, for example, land in the latter category.
This makes time another consideration for the solution. Is using it a one-off thing, only for saying goodbye, or would it be a way of reflecting on your life in a more continuous manner with the help of the avatar? Repeating use could become either an everyday occurrence, or perhaps more likely, a ritualistic one. Again, getting advice from our ancestors or dead relatives is a fairly common practice. Consulting with the avatar on days of special meaning could even turn into a calendrical ritual, whereas a single-use would be a contingent one.
Unlimited use comes with additional impacts. Could there be drawbacks to the continued use of the solution? And if so, what would be an appropriate time limit? Here we’re talking about where to draw the line between user responsibility and developer responsibility, which is never an easy discussion to have.
3 Secondary parties
Death impacts all of those who are left behind. If one person decides that bringing the dead back as a virtual avatar is something they want, how would that affect the people close to them? Spouses, children, friends, relatives? What if a child dies, and one parent feels the need to use the solution to be able to move on, but the other feels it just adds to the pain? Balancing the use of the solution would have to take not only one user, but the secondary impacts into account.
What about sharing the avatar among multiple users? Would younger members of the family be able to have conversations and play with their lost sibling? Even if this wasn’t the main purpose of the solution, people always find new and novel ways to use technology in a way that it was not intended to be used.
Saying goodbye to older loved ones might feel less problematic and more acceptable as a rite of passage. Even though no loss is ever easy, elderly people are more likely to have felt to have lived their lives, and it might be easier to let them go – even for their family and friends. But even this might not be true everywhere, and certainly not across history, when death has been a closer companion to people of all ages. The loss of the elderly might also mean the passing of generational wisdom, sometimes even the disappearance of dying languages. Could a solution like this help keep a language alive in a more tangible way than just recordings, even with limited interactions?
4 Organisation
This layer considers positive and adverse impacts and any important considerations from the perspective of the organisation developing, purchasing, or deploying the solution. Depending on the angle from which we’re approaching the assessment, this is absolutely a layer that should be considered, prompting conversation on and clarification of e.g. how the proposed solution will create the intended value, and how it might affect existing roles and processes. In this post, we’re considering the consumer angle without speculating on who could be the developing or deploying organisation, therefore omitting this layer.
5 Culture
Here we get to the really tickling parts that I already touched on in the preface. Assuming that the solutions offered are internationally offered and marketed, the development process requires a lot of local considerations. How does a certain cultural environment where this solution would be used consider and relate to death?
The way the dead remain in our lives varies from a somewhat clear dichotomy between the living and the dead to a constant mixture of both. Some cultures think that once a person is dead, that is how they should remain. For others, they continue living, not just in memory but nearly as a corporeal presence. Are the dead just that, dead, or do they live on in spirit? Are they still present in everyday life, as ancestors or guiding voices? How would those beliefs be reflected in a virtual avatar?
We could even go as far as to consider their belief in ghosts, and how those are viewed within a particular culture. The way we tell ghost stories here in the deep, dark north is usually to scare and to warn. The ghost may or may not be malicious, but they are rarely friendly either. This is not the case in many cultures around the world, where ghosts can be more of a friendly whisper or a gentle warning. Is an echo of the dead, such as a virtual avatar, equated with a ghost? And if so, could a solution like this actually help make our relationship with the spirit world a bit warmer?
Again we come crawling back to the realm of rituals. Would a solution like this become part of the rituals of remembering the dead, or would they make it harder for the rites of remembrance to do their job and let the dead pass on? There’s also the question of iconography and visual representation of spirits. Would the image of a dead person be a way of showing respect, or a deeply disrespectful act?
There’s also the question of religion, which plays a huge part in our global existence. How would a solution like this work in an environment where the majority of people believe in reincarnation? Would this be felt as a totally separate phenomena, not interfering with their faith in the cycle of life, or would it be felt as an unwelcome disruption? We often dismiss religion, faith and beliefs in developing new services, especially if those beliefs are not highly valued in our personal lives and culture. On a larger scale, this is a mistake.
6 Society
This layer considers the impacts of societal structures such as law, education, and healthcare, to the success of the proposed solution, or in some cases impacts that the solution might have on societal structures. In this case, we’re focusing on the perspective of data privacy. On a more practical level, we can talk about the data required to make a solution like this work. What kind of data would be required and how would it be obtained? What legal responsibilities does the company have when using such private and sensitive data in different contexts around the world? How will the company handle the data they have according to regulations? What are the rights of the dead with regards to their own data?
Currently, those rights are close to non-existent. However, as this post smartly describes, “However valid the reasoning for stopping privacy on death might be, society has reached a point where emerging technologies rapidly increase our information legacies.” There might be a time in the near future where regulations start to change and take post-mortem rights more seriously.
7 Environment
This layer focuses on the impact of, or on, everything to do with the natural and built environment that is relevant to the case in question. As with any information technology, there is the question of energy consumption. We already know that our use of the Internet and mobile data requires huge amounts of energy, as do gaming computers and consoles. Training and running Deep Learning models is known to cause significant environmental issues, such as CO2 emissions, depending on the technologies and the location and type of data centres used. Considering the current climate crisis we are living in, every solution should spare a few thoughts to their environmental impact.
Ok, we’ve done some thinking. Now what?
As you can see from the above examples, a framework like this can generate a huge amount of unanswered questions and implications. It is the responsibility of each company or development team to decide which ones are top priority for risk and impact management, which ones go to the backlog and which are interesting points, but not necessary for immediate attention. Even if you would not feel the need to act on a majority of the thoughts generated, it is important to have those thoughts written down and out in the open. It’s also important to return to these assumptions and considerations as the work goes on to see which ones are proving to be critical for responsible and sustainable development and use of the technology.
Documentation also helps us to locate and address gaps in our understanding, both as a group and individually. Through this blog post alone, I’m sure I could (and probably will) have a very nuanced conversation with a data scientist or a developer about why some of these considerations are not currently viable even in theory. The importance of interdisciplinary thought cannot be emphasised enough when talking about impact assessment. We all look at the world through our own perspective, and bursting that bubble can sometimes be the best thing we can do – both for ourselves and for the world at large.
Thanks to Anna Metsäranta and Antti Rannisto for your hard work with the Layers of Impact framework, and both the above and Janne Niinivaara for comments on this text.
Additional resources
- Latour, Bruno: Reassembling the Social: An Introduction to Actor-Network Theory (2005)
- Turner, Victor: Forest of Symbols (1967)
- Turner, Victor: The Ritual Process (1969)
- Turkle, Sherry: Alone Together (2011)
- Turkle, Sherry: Reclaiming Conversation (2015)
- Coleman, Donagh: Tukdam: Between Worlds (2022)
- Digital Death: Transforming History, Rituals and Afterlife Research Group
- Finnish Death Studies Association