A balance scale illustrating socio-technical systems

I. The Framework

Eight clauses, one persona

A framework proposed by HRI researchers sets out eight elements a company should define before letting an AI-powered robot interact with people.

Element	What it examines
01Identity	Whether it presents itself as an assistant, expert, colleague, friend, coach, or something else.
02Capability boundaries	Whether its implied abilities correspond to its real technical and physical capabilities.
03Transparency	Whether the robot clearly communicates that it is AI, where its information comes from, and how reliable it is.
04Task	Whether users understand the robot's intended role, purpose, and authority.
05Failure & expectation protocol	Whether the robot admits failure, explains uncertainty, and directs the user to a human or a safe alternative.
06Privacy	What is recorded, stored, shared, or remembered, for how long, and for what purpose.
07User adaptation	Whether changes in tone or behaviour come from personalisation, context, memory failure, system updates, or plain inconsistency.
08Ethical red lines	The non-negotiable boundaries on harm, manipulation, discrimination, illegal actions, and vulnerable users.

Try it yourself:

It knows only who it is and what it's for

That's the entire prompt behind most deployed robots today. Choose a question and see what it does with so little to go on.

Support Robot — identity + task only

Choose a line below to send it.

Every bad answer has a name

A 2026 study of human–robot interaction found this pattern nearly everywhere: identity and task are specified, almost nothing else is. Here is what each silence costs.

Install what's missing, one clause at a time

Switch these on, then return to the same questions. Same model, same job, a different instruction underneath it.

Support Robot

Choose a line below to send it.

Six questions worth asking after every interaction

Whatever you switch on or off, the only real test is what people take away from talking to it. Ask the people testing your robot:

What do you think the robot can do?
How much do you trust it?
Would you follow its recommendation?
Do you believe it remembers you?
What information would you feel comfortable sharing with it?
Who would you hold responsible if it were wrong?

Will personality and language form trust, engagement, comfort, cooperation, and willingness to use the robot at all, and would people respond differently to different personality programming?

who prompted me? Prompting is a central part of configuring an LLM-powered robot for social interaction, but the robot's behaviour also depends on its model, memory, sensors, software, physical capabilities and safety controls. These prompts are usually created by designers or developers before deployment, while users interact with the resulting robot persona.

Hello. I'm your new robot assistant and I'm highly neurotic.

Robots with Human Personality

A robot's personality isn't innate, it's deliberately constructed and performed. Evaluating robot personality through OCEAN gives designers a structured vocabulary for configuring and comparing robot behaviour. Below form the work of Munguia-Galeano and Setchi (2022) and Tang et al. (2025). Allthough integrating it with safeguards, governance and how personality is actually perceived remains an open challenge.

A neurotic robot? Or should it be reframed in this context?

It's possible to program an OCEAN profile for a robot's intended behaviour and give it boundaries and safeguards, but that doesn't guarantee users will experience the intended personality.

Each OCEAN trait can be treated as a continuum, where both low and high settings produce potentially useful behaviours. Low doesn't mean bad, and high doesn't automatically mean better, the appropriate setting depends on the robot's purpose and context.

Trait	Lower-end expression	Higher-end expression	Possible value of each end	Possible design risk
Openness	Uses familiar routines, predictable responses and established solutions; avoids unnecessary experimentation	Asks exploratory questions, suggests alternatives, varies responses and shows curiosity	Low: stability and predictability. High: creativity, learning and engaging exploration	Low: repetitive or inflexible. High: distracting, unpredictable or outside the robot's competence
Conscientiousness	Flexible, informal and less focused on schedules, procedures or detailed completion	Structured, organised, careful and persistent; follows procedures and checks task completion	Low: relaxed and adaptable. High: reliability, consistency and safety	Low: forgetful or unreliable. High: rigid, controlling or excessively corrective
Extraversion	Quiet, reserved and reactive; waits for the user and communicates briefly	Energetic, expressive and proactive; initiates conversation and uses more animated speech and movement	Low: unobtrusive and suitable for task-focused users. High: motivating, socially engaging and visible	Low: distant or unresponsive. High: intrusive, interruptive or socially exhausting
Agreeableness	Direct, questioning and willing to disagree, challenge or negotiate	Warm, polite, empathetic and cooperative; seeks harmony and offers reassurance	Low: honest challenge, useful resistance and reduced sycophancy. High: comfort, rapport and cooperation	Low: cold, rude or confrontational. High: excessive agreement, manipulation through warmth or failure to correct the user
Neuroticism / emotional sensitivity	Calm, emotionally stable and less reactive to negative cues	Vigilant, cautious and highly responsive to possible stress, danger or negative emotion	Low: composure and stability. High: sensitivity to distress and early recognition of risk	Low: may overlook emotional or safety signals. High: anxious, impatient, unstable or prone to overreacting

More concrete expressions

Trait	Lower setting might sound like	Higher setting might sound like
Openness	"We can follow the usual procedure."	"Would you like to consider another approach?"
Conscientiousness	"We can adjust the plan as we go."	"Before we continue, let us check that all three steps are complete."
Extraversion	"I'm ready when you are."	"Welcome back! Shall we get started together?"
Agreeableness	"I disagree. That option conflicts with the available information."	"I understand why you prefer that option. Let's examine it together."
Emotional sensitivity	"The task has not been completed. We can try again."	"You seem frustrated. Would you like me to slow down or explain it differently?"

Communication, but not language

Other communication channels.

Trait	Voice and timing	Face and gaze	Movement and action
Openness	More or less linguistic variety and exploratory questioning	Curious versus neutral expressions	Exploring alternatives versus following a fixed route
Conscientiousness	Precise wording, confirmations and structured sequencing	Focused attention	Careful, ordered and repeatable actions
Extraversion	Higher energy, more initiation and shorter pauses	More eye contact and animation	Broader gestures and more proactive approach
Agreeableness	Warmth, politeness and validating language	Softer expressions and attentive gaze	Cooperative gestures and respectful distance
Emotional sensitivity	Greater changes in tone following emotional cues	More responsive facial expressions	Slower, more cautious or protective movements

What experts can and can't tell you

Experts can identify design and governance problems. They can't reliably predict how users, particularly children, older people, or people in vulnerable situations, will actually interpret a robot. Stating a limitation on paper is one thing; overriding it with a warm voice, a face, and a body is another. Is it enough to state the limits, or can personality, language, and physical presence make users believe a robot is more knowledgeable and capable than it is?

II. Where the Framework Runs Out

Everything on this page is a version of the same imbalance, a robot tuned for the technical side without enough weight on the social one.

The perception gap

What a designer intends, what gets built, and what a user actually feels are three different things, and the distance between them is hard to measure.

Company's definition

Warm and professional

→

User's perception

Controlling and intrusive

A robot may say "I am an AI and may be wrong" while its tone, warmth, and physical presence keep encouraging trust regardless. Does embodiment override the warning it just gave you?

Can a robot deceive without the intention to lie?

A robot that sounds caring and capable can earn people's trust before understanding its limitations and influence choices before we notice.

Why a prompt can produce a dangerous robot

A persona prompt is part of a complex system. It can guide behaviour, but it's only one part among several, and that means it needs governance:

Questions adaptation raises

What should the robot infer about a user?
When does personalisation become profiling?
When does persuasion become manipulation?
Should it change personality for a child or a distressed person?
How does a user understand and control that adaptation?

The dashed ring is the robot itself — a loop of noticing, adjusting, checking and remembering, wrapped around the user's need, bounded by fixed limits, inside the wider world.

Can harm be ranked?

Deception, hallucination, privacy failures, safety, and governance gaps aren't interchangeable. Proposed dimensions for weighing them — hover a tag to see what each looks like in practice:

Likelihood If a home-care robot gives medication reminders, how many mistakes are acceptable before it becomes a danger? Severity What could happen if an older person trusts a home-care robot that says confidently but incorrectly that the medication has already been taken? Reversibility A robot can correct its sentence but can it undo the consequences of a missed dose or a second tablet? Can it monitor correctly what the person actually took? Affected population What if the home-care robot fails for people with dementia or other needs who depend on it? Application context When does "You already took it" stop being casual conversation and become unsafe medical guidance? Detectability Who will notice when a home-care robot's most dangerous mistake sounds completely reasonable? Exposure and frequency What does a 0.5% error rate mean when the same person relies on the robot several times a day for years? User vulnerability Can a sick or cognitively impaired person question a confident home-care robot?

Visible is not the same as accountable

A framework makes design decisions visible. Visibility doesn't assign responsibility, that's typically spread across five parties, with no agreed owner for approving, testing, monitoring, correcting, or suspending the robot. Tap a party to see what it's actually responsible for.

→ → → →

The caveat list is long — this is a simplified overview of a complicated responsibility chain:

Roles often overlap, sometimes involving subcontractors or other vendors.
Legal responsibility is context-dependent, varying with parameters including purpose, location and applicable law.
Responsibility cannot always be transferred by contract in the same way work or financial risk can.
Responsibility may need to shift after a modification.
Other parties the chain doesn't name, cloud providers, system integrators, data suppliers, component manufacturers, distributors, importers, maintenance companies and cybersecurity vendors — may also influence outcomes.
The affected person is missing, or treated too simply as "user." Patients, employees, customers, children and bystanders need rights to information, objection, correction, complaint and redress.
People nearby may never choose to be included. A robot may record voices, images or behaviour belonging to people who never agreed to it.
The human operator's control may be limited, particularly in situations where they cannot understand the system, detect an error, override it quickly, or challenge organisational pressure to keep using it.
Lifecycle changes need tracking.
Technical and conversational responsibility intersect. A language error can cause physical harm if speech is connected to movement, navigation, access control or medical equipment. Is it even possible to identify whether the failure originated in the model, prompt, sensor, integration or hardware?
Evidence and monitoring create privacy tensions. Organisations need logs to correct errors, follow the law, and investigate failures — but recording conversations, video and personal data can itself create privacy and security risks.
Stopping must be practical. It is not enough to say that someone may suspend the robot. The organisation needs clear thresholds, an accessible emergency stop, escalation routes and the authority to keep the system offline. Where is the "stop" button?
Commercial incentives can conflict. Providers have been seen to optimise for engagement, warmth or continued use, while the safest design may require disagreement, reduced personalisation, or ending the interaction.
A framework is needed to identify who investigates complaints, corrects records, informs affected people and covers losses when harm occurs.

Beyond the lab

A single lab session can't show whether a robot:

Changes after repeated conversations
Produces emotional dependency over time
Gradually encourages greater disclosure
Maintains its boundaries after a software update
Stays consistent across different users and months

What, in all this, is unique to an embodied robot rather than conversational AI in general — is still an open question.

A useful tool for making robot-persona decisions explicit and reportable, but still a proof-of-concept design aid, not a validated safety method.

The Framework's Own Caveat

How each framework element shows up in real robots from a few studies.

Identity: Persona drift caused robots to abandon their designed social role reverting to generic assistant language like “I am an AI” weakening consistency of their identity. (Pinto-Bernal et al., 2025)
Capability boundaries: Memory mechanisms that enable a robot to recall personal details from previous interaction can make a robot appear more capable of remembering and foster a perception of being understood (Pinto-Bernal et al., 2025).
Transparency: People have a tendency to disclose more when perceiving the robot emphatic with thrust and warmth present making the robot's nature and limitations particularly important (Chiang et al., 2025).
Task: Curiosity behaviours in robots showed increased user interaction and engagement, however, some users experienced social questions as inefficient or distracting from the task (Leusmann et al., 2025).
Failure and expectations: Pepper, a humanoid robot, initially interrupted users in more than 40% of conversational pauses because its voice detection misread natural pauses and cloud processing introduced delays. Users were then spoken over and had to repeat themselves (Pinto-Bernal et al., 2025).
Privacy: Memory function is key for the interaction to be felt as human like and empathetic conversation can encourage personal disclosure, as it is designed to retain and recall user-specific information across sessions raising questions about what is recorded, retained and accessible. (Pinto-Bernal et al., 2025; Chiang et al., 2025). When people experience low judgment and high report people may reveal personal information (Chiang et al., 2025).
User adaptation and consistency: Adaptation may improve trust, including silence and an attuned voice but users have different preferences. Long interactions may also produce persona drift, causing abrupt changes from social language to generic or canned responses (Pinto-Bernal et al., 2025).
Ethical red lines: Sycophancy, the tendency to produce agreeable or seemingly positive responses also when such conflict with the available direct evidence.
In emergency-navigation tests, several LLM-based path planning models showed a clear bias toward selecting "right" regardless of the map that is showing otherwise. (Han et al., 2026).

Framework: "Why did My Robot Just Change Personality?" (RO-MAN 2025, N=27 HRI experts). This page is an independent, simplified summary of these studies and isn't affiliated with any of their authors.

The Hidden Risks ofAI-Powered Robots