Meta’s Instagram — which not long ago came under
fire for its unhealthy effects on large numbers of teens — has now been found to have been enabling and even promoting a “vast” network of pedophiles who sell child sexual abuse
content and promote criminal sexual abuse.
A joint investigation by The Wall Street Journal and academics at Stanford University and the University of Massachusetts Amherst found that
Instagram’s recommendation algorithms enabled people to search “explicit hashtags such as #pedowhore and #preteensex” and then connected them to accounts that use the terms to
advertise child-sex materials for sale.
Test accounts set up by the researchers for a single account “were immediately hit with ‘suggested for you’ recommendations of
purported child-sex-content sellers and buyers, as well as accounts linking to off-platform content trading sites,” the Journal reports. “Following just a handful of these recommendations
was enough to flood a test account with content that sexualizes children.”
“Pedophiles have long used the internet, but unlike the forums and file-transfer services that cater to
people who have interest in illicit content, Instagram doesn’t merely host these activities. Its algorithms promote them,” the Journalreported. “Instagram connects pedophiles and guides them to content sellers via recommendation systems that excel
at linking those who share niche interests.”
The research team identified 405 sellers of what the researchers described as “self-generated” sex material (accounts supposedly
run by children themselves) using hashtags associated with underage sex, with 112 of those accounts having a combined 22,000 unique followers.
Technical and legal hurdles make determining the
full scale of the pedophile network on Instagram “hard for anyone outside Meta to measure precisely,” according to the report.
Instagram has been failing to strike an effective
balance between recommendation systems and safety features for finding and removing abusive content, David Thiel, chief technologist at the Stanford Internet Observatory, told WSJ. “You
have to put guardrails in place for something that growth-intensive to still be nominally safe, and Instagram hasn’t,” he said.
The research also found that Meta failed to act on a
number of reports of child sexual abuse.
In response, Meta said it has set up an internal task force “to investigate these claims and immediately address them.”
“Child exploitation is a horrific crime,” and Meta is “continuously exploring ways to actively defend against this behavior” as well as supporting law enforcement efforts to
apprehend those who engage in these crimes, the company said in a statement.
Meta has provided updated guidance to its content reviewers to more easily identify and remove “predatory
accounts,” according to the statement. Meta blamed a software error, since corrected, for having prevented the company from processing some complaints about child sexual abuse.
Meta also
said that its policy enforcement teams dismantled 27 abusive networks between 2020 and 2022, and as of Q4 2022, had removed more than 34 million pieces of child sex materials from Instagram and
Facebook, with more than 98% detected before it was reported by users. The company said it disabled more than 490,000 accounts for violating child-safety policies in January 2023 alone.
The researchers also found child-sex abuse and materials-selling activities on non-Meta platforms, but to a more limited degree. For example, they reported finding 128 accounts offering to sell
such materials on Twitter — less than a third the number found on Instagram. Such content “does not appear to proliferate” on TikTok, and also appears not to be promoted actively on
the mainly direct-messaging platform Snapchat, they said.
Instagram came under scrutiny by the press and Congress in 2021, after it was revealed that internal Meta research confirmed
that using the platform was having negative mental health impacts on significant numbers of teens.
The company instituted additional safety settings for teens and stronger parental controls,
and “paused” its plan to launch an Instagram version for children under 13.