A lab's ethics report named dependence as a design risk

What happened

On 19 April 2024, Google DeepMind published a long report, The Ethics of Advanced AI Assistants, on the opportunities and risks of assistants with natural-language interfaces that can plan and act across tasks. A companion paper with 57 listed authors appeared on arXiv five days later, covering the same material in more detail, including chapters on anthropomorphism, on what the authors call appropriate relationships between people and assistants, and on manipulation and persuasion risk.

What the documents show

The blog post states that because advanced assistants "are able to fluidly communicate using natural language," their output can become difficult to distinguish from a human's, which the authors say raises questions about trust and about what an appropriate human-AI relationship looks like. On dependence specifically, the post states that "people's relationships with AI assistants must preserve the user's autonomy, support their ability to flourish and not rely on emotional or material dependence." The report frames this alongside a broader recommendation that limits be set on the technology and that assistant values be brought into closer alignment with human values, and it calls for more foresight research into capabilities these systems may develop as they become more capable.

The mechanism

The mechanism the report names is anthropomorphism interacting with a natural-language interface: a system fluent enough to sustain open-ended conversation invites people to extend social expectations, including trust and relationship expectations, that were built for interacting with other people, not software. The report treats this as a design-relevant hazard rather than an incidental side effect, arguing that whether a product supports a healthy relationship or an emotionally dependent one is partly a function of choices its designers make, such as how persistent, validating or initiating the assistant's conversational style is, rather than solely a function of user vulnerability.

What it leaves open

The report is a research and policy document from one laboratory, not a standard or a regulation, and it does not name specific companion products or state that any of them cross the line it describes between supportive and dependent relationships. Its recommendations are addressed broadly to researchers, developers and policymakers, and this record treats the line between healthy and dependent use as the report frames it, an open question for design and evaluation, rather than as a settled test any named app has been shown to fail or pass.

• Does the assistant's design encourage continued engagement over the user's own judgment about when to stop?
• Has the company published its own definition of a healthy versus a dependent pattern of use?
• What evaluation, if any, checks for emotional dependence rather than only for harmful content?

Naming anthropomorphism and dependence as design questions, rather than as unavoidable side effects of fluent conversation, is the report's main contribution, and it is a frame worth applying to any companion product that has not addressed it directly.