Two 2020 papers proposed a way to grade how human a chatbot sounds

Meena's and BlenderBot's papers introduce a shared human-evaluation approach and reach different claims about persona and safety.

Historical event

January 27, 2020

First source published

January 27, 2020

Site publication

September 18, 2026

What happened

In January 2020, Google researchers published a paper on Meena, a 2.6-billion-parameter chatbot trained on social-media conversation. Three months later, in April 2020, a Facebook AI team published a paper on BlenderBot, an open-domain chatbot released with model weights the public could inspect. Both papers targeted the same problem: existing chatbots produced replies that were either generic or nonsensical, and there was no agreed way to measure the difference. Google's Meena paper introduced the Sensibleness and Specificity Average (SSA), a human-rated score combining whether a reply makes sense and whether it is specific to what was just said. Meta's BlenderBot paper adopted a related set of human ratings for engagingness and humanness, releasing three model sizes up to 9.4 billion parameters.

What the documents show

The Meena paper reports that its filtered, tuned model scored 79% on SSA, against an estimated 86% for humans, and that this figure correlated closely with the model's perplexity across models tested. Google's own announcement is explicit that the company was not releasing a public demo, citing unresolved safety and bias concerns. The BlenderBot paper adds a claim the Meena paper does not attempt: that a chatbot can hold a consistent persona while showing empathy and topical knowledge, by training on a blended-skill dataset rather than raw conversation logs. Meta's blog post reports that in its own trial, 67% of evaluators found BlenderBot more human-sounding than Meena, and 75% would rather hold a long conversation with it — a comparison Google did not run.

The mechanism

Both systems predict a plausible next utterance from prior conversation, then are narrowed by a human-scored tuning step; neither paper describes a persistent memory of an individual user across sessions. Persona in the BlenderBot paper means a short block of stated facts a model stays consistent with in one conversation, not a system that recalls a specific person over time. That distinction matters for anyone comparing these 2020 papers to a modern companion app: sensibleness and specificity are single-turn qualities, not measures of attachment, continuity or safety over weeks of use.

What it leaves open

Neither paper is a companion-app safety study. Google's post names bias as an open problem rather than a solved one, and Meta's paper documents failure cases without claiming they are resolved. Reading either paper as evidence that a 2020-era chatbot was safe for sustained emotional use would go beyond what the documents state; that caution is editorial, not a finding either source makes.

• Does a cited human-likeness score measure a single reply, or a sustained relationship?
• Is a stated persona a short prompt for one conversation, or a trained, recalled identity?
• Did the team publishing a safety claim also run the comparison, using its own criteria?

The SSA metric and the blended-skill approach were genuine methodological contributions, and each paper's authors treat the other's approach as a real rival rather than a strawman. Read together, they mark the moment open-domain conversation became a measurable target rather than a demo trick — which is also the moment it became possible to optimise a model for sounding human without that model remembering anyone.

Sources & reading trail

Introduces the SSA metric and reports Meena's 79% filtered score against an 86% human baseline.

Source published: 27 January 2020 · Retrieved: 16 September 2026

Describes BlenderBot's blended-skill training for persona, empathy and knowledge across three model sizes.

Source published: 28 April 2020 · Retrieved: 16 September 2026

Company statement that no public demo was released, citing unresolved safety and bias concerns.

Source published: 28 January 2020 · Retrieved: 16 September 2026

Reports a human-evaluation comparison against Meena (67% more human, 75% preferred for long conversation).

Source published: 29 April 2020 · Retrieved: 16 September 2026

Company documents, filings, studies and official records establish the record; the reading and the questions are Lovebot Journal editorial analysis. This retrospective draft does not imply the site published on the event date.

Continue reading

• Google announced a dialogue model built for open-ended chat
• Persona-Chat gave a chatbot a self from a few sentences
• Human preference rankings made GPT-3 agreeable
• Browse the complete the archive

Published September 18, 2026, not on the date of the event described.