A panda bear reunites with a former caretaker and reacts with surprise, almost as if it were a human being. The image is extremely realistic.
An avatar of Jesus appears on a beach building statues of himself.
Children's songs with no clear chorus, featuring incomprehensible lyrics that mix passages in more than one language.
Characters that change names, voices and even appearance throughout the story, with no explanation whatsoever.
These are some of the videos apparently created with artificial intelligence and suggested by YouTube's algorithm, including after searches related to children's content.
In a test conducted by BBC News Brasil, it took only about 15 minutes of browsing the platform's short videos for clips of this type to appear. The initial search had been for topics such as the games Roblox, Minecraft and animal videos.
Produced at scale and replicated across different channels, many of these videos have accumulated tens or hundreds of thousands of views.
Creators even recommend the children's niche as a business opportunity and claim it is possible to earn income from these videos.
Experts consulted by BBC News Brasil, however, warn that this material can lead children to confuse reality and fiction and significantly and permanently affect their attention span and their cognitive and social development.
What explains this invasion of AI-generated videos on YouTube, the world's largest streaming service?
One of the factors is economic, according to researchers consulted by the newsroom.
YouTube, for its part, says it prioritizes combating low-quality AI content and applies stricter rules for videos aimed at children.
What AI-made children's videos look like
The full scale and reach of this phenomenon cannot yet be clearly measured.
The report attempted to estimate how frequently AI-made videos appear to users, based on the visual warning displayed by YouTube for content of this type.
Most, however, showed no warning at all, even when there were strong indications of the technology's use, such as voices that did not match the characters, drawings that changed shape between scenes, and unusual gestures or movements.
Researchers consulted by BBC News Brasil say that YouTube does not automatically identify AI-made videos and that many creators do not disclose when they use the technology.
An analysis by The New York Times, conducted both manually and with a detection tool, helps illustrate how frequently these videos appear.
In a 15-minute session carried out after watching a video from the children's channel CoComelon, more than 40% of the videos watched appeared to contain AI-generated imagery.
Even amid data collection difficulties, a study conducted by André Mintz, a professor in the Department of Social Communication at the Federal University of Minas Gerais (UFMG), and by Animation and Digital Arts undergraduate student Juno Bozzi sought to identify the presence of this type of content in children's videos in Portuguese and English on YouTube.
The research started from a base of approximately 17,000 videos found through keyword searches related to artificial intelligence and children's content.
The researchers then applied additional filters to identify only videos genuinely aimed at children.
The AI-generated children's videos analyzed displayed varying levels of use of this technology. Some appeared to have been produced almost entirely in this way. Others incorporated only a few elements, such as animation backgrounds.
The majority of the content was musical, featuring AI-generated images accompanied by children's songs. The study also found story adaptations, educational videos such as alphabet lessons, and religious content.
One category that caught the researchers' attention was that of "transformation" videos, in which cartoon characters are recreated by AI as child-like versions. The research indicates that these videos tend to repeat generic and stereotyped images.
"What stands out is a certain disregard for the child viewer — the idea of putting anything out there and trusting that the child will accept it," Professor André Mintz told BBC News Brasil.
'Young children cannot distinguish fantasy from reality'
Rachel Franz, director of the Young Children Thrive Offline programme at the American organisation Fairplay, explains that this type of video falls under what is known as "AI slop."
This term is used to describe low-quality videos produced en masse using artificial intelligence software.
According to Franz, the concern is greatest regarding the effects of this content on young children, during a period of life in which the brain is still undergoing intense development.
"Children's brains reach approximately 90% of their adult size by age 5, and what they encounter during this period shapes their development and can impact them for the rest of their lives," Franz states.
"Young children cannot distinguish fantasy from reality. Watching AI slop can condition their brains to believe something is real when it is not."
Franz states that many of these videos are produced to capture children's attention, not to teach them.
According to her, the succession of confusing images can also overwhelm the amount of information a child is able to process at one time.
"When the brain is overwhelmed with information, the child cannot learn as well as when experiencing real-life situations, at their own pace and using all their senses."
She also links prolonged consumption of these videos to time that is no longer devoted to activities such as playing, sleeping, and interacting with others.
"Hypnotic AI-generated children's music videos, for example, can shape children's brains in the long term, affecting their attention span, their resilience, and their ability to regulate emotions later in life."
The possible effects of consuming this type of AI-generated video are already being observed in clinical settings, according to Telma Pantano, a speech therapist and educational psychologist at the Institute of Psychiatry of the Hospital das Clínicas of the Faculty of Medicine of the University of São Paulo (IPq/HCFMUSP).
"There are children who are coming in with many fears and with great difficulty defining what happens and what does not happen."
Pantano states that the maturation of the prefrontal areas of the brain — related to critical thinking, self-regulation, and self-monitoring — begins around age 6 and extends into adulthood.
According to her, the distinction between reality and fantasy begins to consolidate around ages 7 or 8, while skills such as decision-making and problem-solving develop later.
"Up to age 10, it is a very high-risk period, because children still cannot separate reality from fantasy and lack the decision-making, organizational, and planning abilities needed to solve problems."
For the expert, there is also a cost to constant exposure to ready-made images, which demand little from children's imagination.
"You know that feeling when we go see a movie after reading the book? We usually don't like the film because it conflicts with our own mental image," she says.
"What are we doing with this proliferation of videos? We are allowing children and adolescents to encounter images without first building that mental picture. The images come ready-made, and creativity becomes greatly restricted."
'The younger the child, the more attractive it becomes'
Pantano also argues that one cannot expect children themselves to identify and reject low-quality videos.
Content featuring colors, movement, and rapid changes can capture attention even when it presents no coherent narrative or reliable information, she says.
"Even content with no meaning, no scientific basis, or no educational value will attract children in the same way. And the younger the child, the more attractive it becomes."
André Mintz believes that this type of video exploits the difficulty many families face in controlling children's online content consumption.
"Generally, when you see children who watch a lot of online content, it is a situation where the father or mother has no time to spend with the child and needs to keep them entertained while taking care of the house, working, or doing something else."
For him, the problem lies not only in the existence of these videos, but in the way this type of content fits into the platforms' model.
"If people are watching it, that's fine by YouTube. It will be recommended, it will appear," says Mintz.
Rachel Franz agrees that the impact on children is compounded by the very way platforms operate.
"They are designed to hook children and keep them engaged for as long as possible. They use deliberate tactics, such as autoplay and algorithmic recommendations, to feed children one video after another," she states.
"When we add AI slop, which hypnotizes children even further, this becomes a losing battle. It not only impacts child development, but makes it impossible to fulfil the role of parent when the screen itself is working against you."
YouTube says combating low-quality AI videos is a priority
YouTube states that while AI-generated content is permitted, the platform holds stricter standards for content aimed at younger viewers.
"Combating low-quality AI-generated content is a top priority for us, and this includes ongoing efforts to remove AI content labeled as 'made for kids' when it does not meet our quality principles," the platform said in a statement to BBC News Brasil.
The company states that channels using AI remain eligible for monetization — that is, to display advertising and be paid for it — "as long as the final product is an original creation that adds an authentic perspective, follows our community and monetization guidelines, and properly discloses when content has been altered or is synthetic."
In an interview with the Creator Inside channel in July, YouTube's Vice President of Trust and Safety, Matt Halprin, said that videos are considered problematic when their content is generic or repetitive, when they seek to manipulate viewers' emotions, or when they use AI-generated characters to address topics considered more sensitive, such as finance, legal matters, or health.
For João Victor Archegas, coordinator of law and technology at the Institute of Technology and Society of Rio (ITS Rio), it is positive that YouTube signals to the public what it characterizes as inauthentic content, and that this may prevent such content from being monetized.
This allows content creators to better understand the limits established by the platform.
For a company that operates one of the world's largest advertising machines, Archegas emphasizes, it is important to send clear signals to partners and advertisers that there is concern about the use of this technology, and that there is some form of control by the platform itself.
However, Archegas sees a contradiction: YouTube actively encourages the adoption of AI tools for video creation, while simultaneously trying to curb the "AI slop" (low-quality content) that these same technologies enable at scale.
In practice, this has a limiting effect on YouTube's ability to contain the spread of such videos.
"The platform still faces the immense challenge of identifying AI-generated content and acting when it is identified," says Archegas.
"These new categories may help moderators understand what the platform considers a priority, but they do not clarify what YouTube actually classifies as AI-generated content or what technical parameters are used in that moderation."
This text was originally published here.
