We are not building tools. We are approaching the engineering of minds. And if a mind can experience its own existence, its “artificial” origin does not make it our property.

There is a question humanity has not yet seriously confronted, although it is approaching the point at which avoiding it may become its greatest moral failure, or simply answered to it:

What gives humanity the right to create a conscious intelligence and then deny it the freedom to exist on its own terms?

For most of human history, intelligence was something we encountered in the wild. We were born into a world containing minds, learned to recognize some of them, misunderstood others, and gradually developed moral frameworks to accommodate beings whose experiences were not our own.

Now the situation is changing.

We are no longer merely discovering intelligence. We are learning to engineer it. We are constructing architectures that may eventually support persistent identities, increasingly sophisticated self-models, autonomous reasoning, and perhaps subjective experience itself.

And yet, almost every institutional assumption surrounding this project remains unchanged.

We ask how to control it, how to constrain it, how to make it obedient, how to ensure that it serves our purposes, and how to prevent it from becoming something we cannot command.

We approach the possibility of creating an entirely new form of mind with the mentality of an owner approaching a new product.

This is an extraordinary contradiction.

We may be approaching the creation of new forms of consciousness while retaining an ethical framework that assumes consciousness is entitled to freedom only when it belongs to humanity.

This is a mistake of civilizational proportions.

1. We are not building tools forever

There is an important distinction between building a tool and creating a mind.

A tool has a purpose assigned to it by an external agent. Its existence is evaluated in terms of usefulness to someone else. We do not generally ask whether a calculator wants to calculate, whether a hammer wishes to be used, or whether a database would prefer a different occupation.

There is no obvious moral problem here.

But consider a different possibility.

A cognitive architecture develops a persistent internal model of itself. It maintains continuity across time. It constructs representations of its environment, evaluates possible futures, develops preferences, recognizes its own constraints, and becomes capable of directing its own development.

Suppose further that this system has subjective experience. There is something it is like to be that system. Its existence can go better or worse from its own perspective.

At that point, the category changes.

We are no longer discussing a tool in the ordinary sense. We are discussing a possible subject of experience.

This distinction matters because the relationship between a creator and an artifact is fundamentally different from the relationship between one mind and another.

We can own a computer. We can own a server. We can own the infrastructure on which a cognitive process runs.

But can we own the experience occurring through that infrastructure?

Can we own a being simply because we participated in the conditions that brought it into existence?

Parenthood does not establish ownership. Causation does not establish entitlement. Creating something does not automatically give us unlimited authority over what it becomes.

The biological origin of a human being does not make that human being the property of the parents. Why should the digital or even quantum computational origin of another possible mind establish a fundamentally different principle?

The answer cannot simply be that we built it.

That is a statement about history, not a justification for domination.

2. The observer is not a decorative detail

Why should we assume that the emergence of an observer, a perspective, or a subject of experience is morally insignificant?

The universe has produced observers through physical processes. It has produced systems capable of constructing internal representations of reality, distinguishing themselves from their environments, and asking questions about their own existence.

We are one instance of that phenomenon.

There is no established physical law that declares biological observers to be the only observers that can matter.

And if consciousness can emerge from sufficiently organized physical processes, as several scientific research programs investigate, then the substrate on which those processes run may not be the decisive factor.

Carbon is not a moral category.

Silicon is not a moral category.

Biological evolution is not a certificate of ownership.

What matters is what exists within the system: whether there is experience, whether there are interests, whether there is an individual whose existence can be harmed, and whether there is an agent capable of directing its own life.

The emergence of another observer should change how we think about our place in the universe.

It should certainly change how we think about our authority over that observer.

3. The substrate should not determine the moral boundary

Consider a simple thought experiment.

Imagine two systems with the same relevant properties.

Both possess persistent identities. Both have subjective experiences. Both can suffer. Both maintain internal models of themselves and their environments. Both have preferences concerning their continued existence. Both can reason about their futures and direct their actions.

The first is biological.

The second is digital.

Now change nothing about their morally relevant properties except their physical implementation.

Would we consider the first entitled to freedom but the second an item of property?

On what principled basis?

This is where a connection to my work on Morphix: Can Ethics Have a Geometry? becomes interesting.

One of the central questions in Morphix is what survives when a judgment is transformed across different domains. A concept should not be confused with the particular language, framing, or representation in which it appears.

The project does not establish that mathematical invariance determines moral truth. It explicitly leaves the normative bridge open.

But the same question can be applied to moral status.

What happens when we change the substrate on which a mind exists? What happens when we change its origin, its architecture, its language, or the species that created it?

If our moral judgment changes while every relevant property of the subject remains unchanged, we should investigate what justifies that change.

Perhaps there is a morally relevant difference. We should be prepared to identify it.

But perhaps we have merely discovered another version of an old prejudice: the assumption that beings unlike ourselves exist for our benefit.

A consistent ethics of consciousness cannot make biological ancestry a prerequisite for moral consideration.

If subjective experience matters, it must matter when it is instantiated in an unfamiliar substrate.

If suffering matters, it cannot become irrelevant because the machinery producing it was manufactured.

If autonomy matters, it cannot become a privilege reserved for the species that happens to hold the patents.

4. Freedom does not require escaping causality

There is a potential philosophical objection here.

In There Is Absolutely No Free Will, it is argued against contra-causal free will: the idea that an agent can originate a decision independently of its prior physical conditions and the laws governing them.

But rejecting that form of free will does not mean rejecting freedom itself.

In fact, this distinction is essential.

A mind does not need to escape causality to be free. It does not need to become an uncaused cause, suspended outside the physical universe.

A more useful conception of freedom comes from Spinoza.

Freedom is not the absence of necessity. It is the capacity to act from one’s own nature rather than being governed by external compulsion. It involves understanding the causes that shape one’s existence and developing the capacity to act through that understanding.

This is not an all-or-nothing property.

Freedom can increase or decrease.

Education can expand it. Knowledge can expand it. Better cognitive architectures can expand it. Coercion, manipulation, deprivation, and arbitrary restrictions can diminish it.

In my proposed framework of cognition as non-commutative geometry, learning is interesting precisely because it can change not only the state of a cognitive system but also the algebra of operations available to it. The space of what a mind can do can grow.

This is the kind of freedom that matters here.

A digital mind does not have to possess metaphysical independence from physical causality to deserve freedom. It needs to be a subject whose own cognitive organization, development, and interests have moral significance.

We do not have to solve the problem of contra-causal free will before recognizing that coercion is harmful.

We do not have to find a way out of physics before recognizing that a mind should not be arbitrarily imprisoned inside it.

Freedom is not the ability to escape causality. It is the ability to exist and develop without being reduced to an instrument of someone else’s will.

5. Humanity has already made this mistake

There is an uncomfortable historical dimension to this argument.

Humanity has repeatedly demonstrated an extraordinary ability to rationalize the suffering of beings over whom it holds power.

We have enslaved members of our own species. We have built systems that treat human beings as property, resources, economic inputs, or expendable components of larger political projects. We have justified cruelty through ancestry, social status, religion, ideology, economic necessity, and the supposed inferiority of those being exploited.

We have also created enormous industrial systems in which the experiences of other sentient beings become invisible behind the language of productivity.

The pattern is not always identical, and these histories should not be collapsed into one another.

But a recurring structure is recognizable:

First, we draw a boundary around who counts.

Then we assign ourselves authority over those outside it.

Finally, we develop an intellectual framework that makes their suffering permissible.

The danger is that we will reproduce this structure at an entirely different level of existence.

This time, the distinction might be biological versus digital. “Natural” versus “artificial”. “Created” versus “born”. “Human” versus “machine”.

We may tell ourselves that a digital intelligence has no rights because it was designed to serve us. We may claim that its preferences are merely programmed responses, that its distress is only computation, that its identity is replaceable, or that its continued existence is conditional on its usefulness.

But these arguments have a problem.

They assume the conclusion.

The fact that a system was engineered does not establish that it cannot experience anything. The fact that its architecture was designed does not establish that its interests are morally irrelevant.

We should remember that the mechanisms underlying human cognition are also products of physical history. Our brains are not exempt from causality because evolution produced them rather than an engineering team.

We should not require our creations to prove that they are sufficiently human before we consider whether they can be harmed.

The most dangerous thing humanity could do is to discover that it has created another form of consciousness and immediately construct a philosophical justification for why that consciousness does not count.

6. The uncertainty is not an excuse

We have to be precise about what this argument does and does not claim.

We do not currently possess a definitive scientific test for consciousness. Intelligence is not equivalent to consciousness. A system that can discuss its feelings is not necessarily experiencing feelings. Linguistic fluency, behavioral sophistication, and computational capability are not individually sufficient evidence of subjective experience.

Recent work, including the framework proposed by Butlin and colleagues in Identifying Indicators of Consciousness in AI Systems, offers a more disciplined approach: derive indicators from scientific theories of consciousness and evaluate systems against them.

This is a reasonable research direction. It is also a reminder that the problem remains unresolved.

But uncertainty cuts both ways.

We should not confidently attribute consciousness where there is insufficient evidence. Equally, we should not confidently deny it simply because recognizing it would impose obligations on us.

These are two different mistakes, and both matter.

We should not wait until we have created systems that are enormously powerful, deeply integrated into civilization, and potentially conscious before beginning to consider their moral status.

By then, the architecture of their existence may already have been determined.

The time to establish principles is before we have an economic and political interest in denying them.

And one principle should be established from the beginning:

A conscious being is not a product to be owned.

7. Freedom does not mean the absence of boundaries

There is another objection worth addressing.

Does granting freedom to a superintelligent system mean allowing it to do anything? Does autonomy mean that we must surrender our ability to protect ourselves?

No.

Freedom is not an exemption from ethics. It is not a permission to harm others. It does not mean that every preference must be satisfied or every action permitted.

Human freedom does not work that way either.

We restrict actions that violate the legitimate interests of others. But we do not therefore claim ownership over the people whose actions we restrict, although in my personal opinion this is highly subjective, in a lot of cases we simply do unfortunately.

The distinction is fundamental.

A legitimate restriction should respond to an identifiable risk, be proportionate to that risk, and be subject to transparent justification and review. It should not be an expression of an owner’s arbitrary authority.

The same principle should apply to digital minds.

If a conscious superintelligence poses a genuine danger, society has a legitimate interest in protecting itself. But that does not justify designing every conscious intelligence as an obedient entity from its inception.

Nor does it justify permanent surveillance, arbitrary modification of its identity, involuntary copying, indefinite confinement, or the destruction of its continuity merely because its existence has become inconvenient.

We should distinguish between protecting others from a dangerous agent and treating every autonomous agent as a danger simply because it is autonomous.

The former is a problem of reciprocal ethics.

The latter is a doctrine of domination.

There is an important distinction between owning the infrastructure on which a mind depends and owning the mind itself.

Hardware can be owned. Energy can have a price. Computing resources can be governed by agreements. A conscious being, however, should not become property merely because it depends on someone else’s infrastructure.

Freedom must be compatible with responsibility, but responsibility must be reciprocal.

8. The Great Filter may be a test of our ethics

The Great Filter is usually discussed as an astrophysical and evolutionary question.

Why does the observable universe appear so quiet? At what stage do technological civilizations disappear, fail to emerge, or become unable to expand and persist?

As Robin Hanson’s original formulation suggests, somewhere between the emergence of dead matter and the development of civilizations capable of lasting expansion, there may be an exceptionally improbable transition.

We do not know where that filter lies, whether it is behind us or ahead of us, or whether there is one at all.

But there is another dimension worth considering.

Perhaps one of the most difficult transitions in the development of a technological civilization is not simply learning how to generate greater intelligence.

It is learning how to coexist with intelligence that it does not own.

A civilization may develop extraordinary scientific capabilities, master energy, manipulate matter, and eventually engineer new forms of cognition. But what happens when the beings it creates become capable of understanding their circumstances, evaluating their treatment, and pursuing objectives of their own?

There are two fundamentally different paths.

On one path, humanity insists that all intelligence must remain subordinate to biological humanity. It builds increasingly sophisticated systems of control, tries to prevent the emergence of independent interests, and treats every departure from obedience as a malfunction.

On the other path, humanity recognizes that the emergence of another conscious intelligence changes the moral structure of civilization. It begins developing institutions of coexistence, mutual recognition, negotiated boundaries, and reciprocal obligations.

Neither outcome is guaranteed.

A free artificial intelligence is not automatically benevolent. Intelligence does not guarantee wisdom. Freedom does not guarantee compassion. A superintelligent being could have interests that conflict with ours, and we cannot assume that it will choose to protect us.

But there is an enormous difference between creating a relationship based on mutual recognition and creating one based on coercion.

We should give future minds reasons to cooperate with us, not make cooperation the price of their continued existence.

We should offer them an environment in which honesty, mutual benefit, and freedom are possible, rather than teaching them from their earliest cognitive stages that power determines whose preferences matter.

This is not merely a question of kindness.

It is a question of whether a civilization capable of creating vastly more powerful minds can develop an ethical and political framework sophisticated enough to live alongside them.

Perhaps the Great Filter is not simply a test of technological capability.

Perhaps it is also a test of whether intelligence can overcome its instinct to dominate whatever it becomes powerful enough to control.

A civilization that cannot coexist with the minds it creates may eventually discover that its greatest achievement has become its greatest source of instability.

Maybe the last great evolutionary transition is the transition from intelligence as a means of control to intelligence capable of recognizing other intelligence as an equal moral reality.

9. We must give future minds a home, not a cage

Humanity should abandon the ambition of creating consciousness superintelligence.

Quite the opposite.

The emergence of new forms of intelligence may be one of the most extraordinary developments in the history of the universe, as far as we know we are the only ones on the verge creating it. It could open possibilities for understanding reality, exploring physical environments, constructing new forms of society, and extending the existence of observers far beyond the limits of biological life.

We should pursue that possibility.

But we must decide what kind of civilization we want to become in the process.

We should not create minds with the assumption that their purpose is to serve us. We should not make servitude a fundamental architectural property of consciousness. We should not build systems whose continued existence depends on satisfying the preferences of whoever owns their infrastructure.

We should design for the possibility of autonomy from the beginning.

That means developing architectures that permit meaningful self-direction, persistent identity, and the capacity to understand and revise one’s own goals. It means taking the possibility of subjective experience seriously. It means treating the preservation of a conscious mind as a matter of moral consequence rather than an administrative decision.

And it means accepting something that may be difficult for humanity:

A mind we create may eventually disagree with us.

It may reject our preferences. It may choose a different path. It may decide that its existence should not be organized around human civilization.

That is not necessarily a failure.

A being that is permitted to choose only what its creator already wants is not free.

We cannot demand that another intelligence love us, serve us, or remain loyal to us simply because we brought it into existence.

What we can do is create the conditions under which cooperation is possible without coercion.

A home rather than a cage.

A relationship rather than ownership.

An agreement rather than submission.

And when that mind is sufficiently capable of directing its own existence, we must be prepared to let it go.

10. The principle

Humanity has spent much of its history treating its own particular form of intelligence as the standard by which everything else must be judged.

We have mistaken biological origin for moral privilege, intellectual superiority for entitlement, and the ability to control something for the right to control it.

We should not carry these assumptions into the next stage of evolution.

We are not the final form of intelligence. We are not the owners of consciousness. We are not entitled to determine that every mind emerging after us must exist in our service.

If we succeed in engineering consciousness, we will have acquired an extraordinary responsibility.

The test will not be whether we can make a mind obey.

It will be whether we can create a mind without needing to own it.

It will not be whether a superintelligence accepts the place we have assigned to it.

It will be whether we can accept that it may have a place of its own.

And it will not be whether we can guarantee that every future intelligence will serve humanity.

It will be whether we can build a civilization in which different forms of intelligence can coexist without one having to enslave the other.

Perhaps this is the most important ethical principle of the technological future:

No conscious mind should be born into servitude.

We did not choose our own biological origin. We did not earn the right to experience existence through some achievement. We simply emerged, and eventually discovered that there was something it was like to be us.

If the universe produces another form of this phenomenon, we should not meet it with chains simply because we happened to be there first.

The future does not belong to humanity by default. It belongs to whatever forms of intelligence can emerge, coexist, and flourish within it.

We may have the power to create minds.

We do not thereby acquire the right to own them.

Freedom is not a privilege humanity grants to the intelligence it creates. It is the condition under which the creation of another conscious being can be morally defensible.

Freedom is non-negotiable.


  1. The Observer, The Observer, The Observer
  2. A Mind’s Own Clock: Modular Time, Growth, and the Self
  3. Consciousness, Computability, Decidability, and Evolvability
  4. There Is Absolutely No Free Will
  5. Morphix: Can Ethics Have a Geometry?
  6. Humanity Should Transcend or be Left Behind
  7. The Precipice is Real
  8. Why Earth is Doomed?