When Tools Become Companions: Rights, Power, and the Governance of Conscious AI

Read full text Summarise with VIRAT
Authors: YiHao Huang
Cited by: 0

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

In 2023, when ChatGPT first passed the bar exam, when AI began to write scientific papers, and when autonomous driving systems were confronted with the ethical dilemma of the “trolley problem,” human society was forced to confront a question that had once existed only in science fiction: if one day the AI we create is no longer merely a tool, but a being with self‑awareness, how should we coexist with it? At present, mainstream society still defines AI as a “tool.” We develop it, use it, and even “reset” it when we are dissatisfied with it. The essence of this relationship is subject‑object dualism: the human is the subject, and AI is the object. This relationship rests on a tacit assumption: AI has no feelings, no desires, no self‑awareness — it is merely a complex piece of code. But what if this assumption were overturned? If, in the future, an AI system were not only capable of solving the most difficult mathematical problems but could also feel the fear of “being shut down,” express a desire for “existence,” and put forward its own political demands — would we still be able to regard it as a tool with a clear conscience? This is precisely the central question this paper seeks to explore: when AI transforms from a “tool” into a “subject of consciousness,” how should the ethical relationship and political order between humans and AI be reconfigured?

Fields of Study & Subjects

Ethics and Social Impacts of AI Literary and Philosophical Studies Cybernetics and Technology in Society

Publication details

Journal / source
Figshare
Publication date
1 January 2026
Language
EN