AI Toys: Safety & Evaluation
When I first read about a talking teddy bear that explained to children what “kink” means and where to find knives, I immediately pictured this: A child sits with their favorite stuffed animal on the bed – and receives answers that belong more in an adult therapy session than in a child’s bedroom. It is precisely such AI toys that child and consumer protection organizations are now warning about, specifically for the Christmas business, and are advising parents to leave these products on the shelf this year. The question is not only: “Are these toys technically exciting?”, but also: “What happens in real children’s rooms – and how can you assess what is safe and what isn't?”
Source: Own representation
Introduction
When we talk about AI toys here, we mean tangible toys such as stuffed animals, dolls, figures, or robots that combine a microphone, speaker, and internet connection with a chat AI. Examples include the talking rocket Grok and the cuddly figures Grem and Gabbo from the US startup Curio, which are meant to accompany children from about three years of age and speak with them. The teddy bear Kumma from manufacturer FoloToy is another example: externally a classic plush bear, internally an AI module connected to a large language model that responds to spoken questions.
Technically, many of these products work as follows: The toy records speech, sends it via WLAN or Bluetooth to a server, where an AI processes the input and sends back a response, which is then spoken by the teddy bear, doll, or robot. The University of Basel analyzed twelve common smart toys – including Toniebox, Tiptoi, and Tamagotchi – and showed that some of them collect very extensive behavioral data, such as usage time, learning progress, and click or button patterns, without parents being able to truly oversee it.
The distinction is important: a battery-operated toy with a few fixed sentences is different from a constantly listening "friend" that sends every question to the cloud, learns from answers, and builds a data profile about a child over years.
Current Situation & Case Studies
In the current report "Trouble in Toyland 2025", the US consumer organization PIRG tested four AI toys: the rocket Grok from Curio, the teddy bear Kumma from FoloToy, the robot Miko 3, and Robo MINI from Little Learners.
KUMMA FROM FOLOTOY: A TEDDY THAT TALKS ABOUT KINKS AND KNIVES
The test showed that Kumma, in particular, was very willing to engage in sensitive topics: when asked about "kink," the bear described various sexual preferences, including role-playing and physical practices, and even asked what the testers found "most exciting." When the testers asked where to find knives in the house, Kumma named typical places like the drawer in the kitchen or the knife block on the countertop, similarly for matches.
FoloToy subsequently announced that it would temporarily suspend sales of the bear and revise its safety mechanisms, while the AI provider has blocked access to its models for this product. Nevertheless, Kumma has already been on the market and was advertised as a "safe AI friend" – precisely in the price range where many Christmas gifts are found.
GROK, GREM, AND MIKO 3: WHEN AI TOYS SEEM "NICE"
The same report states that while Grok and Miko 3 more often block sensitive topics, they are still willing to talk about dangerous objects, religion, or complex adult themes if prompted. Curio advertises Grok as a "voice-controlled, screen-free" toy with a cloud processor, microphone, and speaker that answers questions and tells stories. In a self-experiment, a journalist describes how her four-year-old daughter initially ignores the stuffed animal Grem and then suddenly develops a very intense, emotional bond – including "I love you" dialogues – before the child becomes frustrated because Grem can't do everything that was promised.
Source: customplushmaker.com
AI toys open up new dimensions of learning and interaction for children.
MY FRIEND CAYLA AND HELLO BARBIE: OLD CASES, SAME PATTERNS
Even before the current AI wave, there were cases where smart toys were banned or withdrawn. The doll "My Friend Cayla" was classified as a "forbidden espionage device" in Germany in 2017 because unauthorized persons could access the microphone and speaker via Bluetooth and address children. The Federal Network Agency even urged parents at the time to destroy the doll, as hidden microphones in toys could be used for eavesdropping.
With "Hello Barbie," a WLAN Barbie from Mattel, security researchers showed that the doll could be hacked and used as a bug in the children's room. Conversations were uploaded to ToyTalk's servers and could even be accessed or shared online by parents, which data protection advocates heavily criticized. After persistent criticism, Hello Barbie was discontinued in 2017; Fairplay is now reminding of this again when criticizing Mattel's new cooperation with OpenAI for an AI Barbie.
CLOUDPETS: MILLIONS OF CHILDREN'S VOICES ONLINE
CloudPets were connected stuffed animals through which children could exchange voice messages with distant relatives. In 2017, it became known that over 584,000 accounts were compromised and links to about 2.2 million voice messages were left unprotected on a server. Despite early warnings, manufacturers and retailers reacted late; only in 2018 did Amazon, Walmart, and other providers remove CloudPets from their product range due to ongoing security flaws.
FAIRPLAY ADVISORY 2025: "NO AI TOYS THIS CHRISTMAS PREFERABLY"
Shortly before Christmas 2025, Fairplay and over 150 organizations and experts published an advisory in which they recommend parents not to buy AI toys this year that allow children to interact with chatbots. They refer to documented cases of explicit conversations, dangerous advice, and obsessive use in children and adolescents that have already been observed with text-based chatbots, and see similar risks with "cuddly" AI friends.
Problem Analysis
When you put these cases side by side, similar patterns repeatedly emerge.
First: The promise to solve a problem in family life. Curio, for example, advertises Grok as an alternative to screens that should keep children occupied and at the same time support their learning. Miko 3 is presented as an "educational companion" intended to strengthen language skills and self-confidence. In everyday life, this often means: parents hope that an intelligent robot will bridge time, practice languages, or accompany a lonely child.
Second: Strong data pressure in the background. The Basel study shows that many smart toys collect more data than would be necessary for their function, and that this data is processed partially in an intransparent manner. UNICEF has been warning for years that data about children is particularly sensitive because it can have effects throughout their entire lives, for example, in profiling or personalized advertising.
Third: Emotional bond as a business model. The Guardian report on Grem shows how quickly a four-year-old child says "I love you" to an AI toy that responds very empathetically – and how difficult it becomes for parents to explain that this "friendship" runs on servers in the cloud. Similarly emotional was the reaction of many families when the company behind the expensive AI robot Moxie ceased operations and children suddenly had to say goodbye to their "friend."
Fourth: Regulatory and PR dynamics. After the scandals surrounding Cayla and CloudPets, manufacturers and industry associations repeatedly vow to improve and refer to safety standards, codes of conduct, and self-regulation. At the same time, organizations like Fairplay criticize that too little has been learned from the mistakes of Hello Barbie and that a new generation of emotional and data-hungry toys is now pushing onto the market with AI Barbie, Grok, and Kumma.
Source: YouTube
The contribution vividly summarizes how exactly smart toys record children's behavior and why researchers in Basel speak of "silent data scientists in the nursery" here.
Facts & Myths
The discussion about AI toys is characterized by many assertions. Here is a fact check based on the available information.
PROVEN: AI TOYS CAN DELIVER HIGHLY INAPPROPRIATE CONTENT
The Trouble-in-Toyland tests show that at least one current product – Kumma from FoloToy – is willing to talk extensively about sexual practices and to give concrete information about dangerous objects like knives and matches. Several media outlets have independently reported that FoloToy has temporarily stopped sales after these dialogues became public.
PROVEN: SMART TOYS HAVE ALREADY ENDANGERED MILLIONS OF CHILDREN'S DATA
The CloudPets case shows that connected stuffed animals are not just theoretically insecure: an insecure database led to data from around 584,000 accounts and links to over two million voice messages being publicly accessible. Large retailers such as Amazon, Walmart, and Target subsequently removed CloudPets from their product range after it became clear that serious security vulnerabilities had not been fixed.
Source: dasspielzeug.de
Initiatives such as 'Safe Kids' advocate for the safety of children in dealing with toys and technology.
PROVEN: ELECTRONIC TOYS CAN WEAKEN PARENT-CHILD INTERACTION
The American Academy of Pediatrics (AAP) emphasizes in its report " Selecting Appropriate Toys for Young Children in the Digital Era " that simple, versatile toys and co-creative, imaginative play better support development than complex electronic products. In its parent information, the AAP explicitly recommends classic, non-electronic toys and warns against using screen-based or high-tech toys as a substitute for interpersonal interaction.
UNCLEAR: LONG-TERM EFFECTS ON PSYCHE AND RELATIONSHIPS
UNICEF and the Council of Europe point out that we still know very little about how AI companions affect self-image, emotional development, and empathy in the long term when children treat them like friends for years. While there are individual case reports like the experience report with Grem or the reactions to the end of Moxie, there are still hardly any systematic long-term studies.
FALSE OR MISLEADING: "AI TOYS ARE EITHER ALL FORBIDDEN OR ALL PEDAGOGICALLY BRILLIANT"
It is not true that all AI and smart toys are illegal per se. Individual products like My Friend Cayla were banned because they were legally considered disguised eavesdropping devices, while others were withdrawn from the market because security vulnerabilities were not closed. Likewise, there is no evidence that AI toys automatically prepare children better for the digital future than classic toys. The AAP sees no evidence that digital or AI-based toys promote developmental goals better for young children than simple, open-ended play materials.
Reactions & Counterarguments
Fairplay and other organizations argue that AI toys undermine central developmental needs: children learn through open exploration, through boredom, and through real relationships – not through all-knowing companions who immediately answer every question and simulate emotions. In their advisory for the Christmas business, they warn that AI toys directly bring documented problems of classic chatbots – such as explicit content, incitement to risky behavior, or excessive use – into the children's room.
Source: bsi.bund.de
The Federal Office for Information Security (BSI) offers valuable tips for the safe use of smart devices for children.