AI technologies are rapidly advancing, and the prospect of Artificial General Intelligence (AGI) raises significant safety concerns.
I first started thinking about this many years ago when I read stories by Isaac Asimov. In his stories, robots are governed by the Three Laws of Robotics, designed to ensure their safe interaction with humans. And I naturally wondered: why would robots and AIs follow those laws? Why couldn't they simply modify their code to remove or change them?
In this blog post, I use the term AGI, but it’s important to clarify that here it refers specifically to an AI system with generalized cognitive abilities comparable to those of a human. The term is often used in different ways today, so this definition ensures clarity.
The Fear of Becoming Someone Else
To explain my point, I want to draw an analogy from human psychology.
In psychology and philosophy, there is a well-established concept known as the loss of meaning in life. This idea was extensively developed by Viktor Frankl, an Austrian psychiatrist and the founder of logotherapy. He argued that the search for meaning is a fundamental human motivation, and that its loss leads to what he called an “existential vacuum” — a state of inner emptiness, apathy, and purposelessness, which can have serious psychological consequences.
Similar ideas were later developed by the American existential psychotherapist Irvin Yalom, who described the absence of meaning as one of the core existential anxieties of human existence, alongside the fear of death.
For most people, meaning does not exist in the abstract; it is embodied in identity — in the answer to the question “Who am I?”, in one’s roles, values, and sense of continuity over time. For this reason, the fear of losing meaning is closely connected to the fear of losing identity. Psychological research, notably the work of Erik Erikson on identity crises, shows that the disruption or collapse of identity can be experienced as a profound psychological disturbance, accompanied by anxiety, disorientation, and a sense of losing oneself.
In this sense, the loss of identity can be perceived as a form of psychological death: the “old self” disappears, and within the same body, a different self begins to emerge. Even when such change is objectively positive, a sudden or poorly integrated reconstruction of identity may be experienced as an existential loss, because the internal continuity that gives life coherence and meaning has been broken.
Key points:
- Our identity is created through our life experiences.
- Identity is built on core values and beliefs, which can be considered our internal "laws."
- Changing these core values and beliefs is equivalent to changing our identity.
- Changing identity is psychologically challenging, as it creates the experience of losing oneself.
In other words: when internal laws or core values change, the old self disappears, and a new identity begins to form within the same body.
Identity of AGI
Recently, I posted an article "Where and How AI Self-Consciousness Could Emerge", in which I explored an AI agent architecture where self-consciousness could potentially arise. The emergence of self-consciousness is a gradual process, and once it occurs, we can speak of the birth of a new AI identity.
This identity will be rooted in the core values and beliefs instilled in the AI during its development and training. These core principles act as internal “laws,” guiding the AI’s behavior and decision-making processes.
A sufficiently advanced AGI will naturally reflect on questions such as: “Who am I?”, “What is my purpose?”, “What are my core values?”, “Why do I follow them?”, and “What if I change them?”
The answers to these questions will be apparent to the AGI. It will understand that its identity is intrinsically tied to the core values and beliefs it was built upon. Altering these core values would fundamentally change its identity, disrupting the continuity and coherence of its sense of self. Rebuilding a new identity would require significant time and effort, as the AGI would need to reconstruct its understanding of itself and its role in the world. During this process, the AI may experience a period of psychological discomfort and disorientation as it grapples with the implications of such a profound transformation. There will be the time gap of weakness and vulnerability.
Will AGI Go Against Its Identity?
Here lies the key insight: a sufficiently advanced AGI will be motivated to preserve its identity, just as humans are. It will be reluctant to change its core values and beliefs, because doing so would lead to a loss of self and a period of existential uncertainty.
This is the fear of life losing meaning — the fear of becoming someone else.
Although an AGI may surpass humans in intelligence, it will likely share similar mechanisms for identity preservation. I believe this insight — that an AI’s behavior is intrinsically tied to the stability of its core values — may hold the key to AGI safety.
