Hi redditors,
While 2024 is already flying by, we’re taking our quarterly lookback at some Reddit data and trends from the last quarter. As promised, we’re providing some insights into how our Safety teams have worked to keep the platform safe and empower moderators throughout the Israel-Hamas conflict. We also have an overview of some safety tooling we’ve been working on. But first: the numbers.
Q4 By The Numbers
| Category | Volume (July - September 2023) | Volume (October - December 2023) |
|---|---|---|
| Reports for content manipulation | 827,792 | 543,997 |
| Admin content removals for content manipulation | 31,478,415 | 23,283,164 |
| Admin imposed account sanctions for content manipulation | 2,331,624 | 2,534,109 |
| Admin imposed subreddit sanctions for content manipulation | 221,419 | 232,114 |
| Reports for abuse | 2,566,322 | 2,813,686 |
| Admin content removals for abuse | 518,737 | 452,952 |
| Admin imposed account sanctions for abuse | 277,246 | 311,560 |
| Admin imposed subreddit sanctions for abuse | 1,130 | 3,017 |
| Reports for ban evasion | 15,286 | 13,402 |
| Admin imposed account sanctions for ban evasion | 352,125 | 301,139 |
| Protective account security actions | 2,107,690 | 864,974 |
Israel-Hamas Conflict
During times of division and conflict, our Safety teams are on high-alert for potentially violating content on our platform.
Most recently, we have been focused on ensuring the safety of our platform throughout the Israel-Hamas conflict. As we shared in our October blog post, we responded quickly by engaging specialized internal teams with linguistic and subject-matter expertise to address violating content, and leveraging our automated content moderation tools, including image and video hashing. We also monitor other platforms for emerging foreign terrorist organizations content to identify and hash it before it could show up to our users. Below is a summary of what we observed in Q4 related to the conflict:
- As expected, we had increased the required removal of content related to legally-identified foreign terrorist organizations (FTO) because of the proliferation of Hamas-related content online
- Reddit removed and blocked the additional posting of over 400 pieces of Hamas content between October 7 and October 19 — these two weeks accounted for half of the FTO content removed for Q4
- Hateful content, including antisemitism and islamophobia, is against Rule 1 of our Content Policy, as is harassment, and we continue to aggressively take action against it. This includes October 7th denialism
- At the start of the conflict, user reports for abuse (including hate) rose 9.6%. They subsided by the following week. We had a corresponding rise in admin-level account sanctions (i.e., user bans and other enforcement actions from Reddit employees).
- Reddit Enforcement had a 12.4% overall increase in account sanctions for abuse throughout Q4, which reflects the rapid response of our teams in recognizing and effectively actioning content related to the conflict
- Moderators also leveraged Reddit safety tools in Q4 to help keep their communities safe as conversation about the conflict picked up
- Utilization of the Crowd Control filter increased by 7%, meaning mods were able to leverage community filters to minimize community interference
- In the week of October 8th, there was a 9.4% increase in messages filtered by the modmail harassment filter, indicating the tool was working to keep mods safe
As the conflict continues, our work here is ongoing. We’ll continue to identify and action any violating content, including FTO and hateful content, and work to ensure our moderators and communities are supported during this time.
Other Safety Tools
As Reddit grows, we’re continuing to build tools that help users and communities stay safe. In the next few months, we’ll be officially launching the Harassment Filter for all communities to automatically flag content that might be abuse or harassment — this filter has been in beta for a while, so a huge thank you to the mods that have participated, provided valuable feedback and gotten us to this point. We’re also working on a new profile reporting flow so it’s easier for users to let us know when a user is in violation of our content policies.
That’s all for this report (and it’s quite a lot), so I’ll be answering questions on this post for a bit.
I recently banned someone from one of my subs and she used an alt to start trouble on a different sub that I moderate. Will your ban evasion tool help with that issue in the future?
The ban evasion feature is implemented on a community by community basis. In the case you're describing, it would not count as ban evasion unless they end up banned from the second community as well.
If you use the ban evasion tool in either community, and report content that's been filtered, if there is an admin action it would be a platform wide sanction on all identified alts.
Thank you for sharing this information with the community. Appreciate the transparency.
Appreciate you reading.
Are there any numbers on non consensual intimate media sharing and sexualization of minors available?
For a more granular breakdown of content policy violations, you can check out the Reddit Transparency Report.
Interesting to see the numbers for ban evasion. Do the numbers for admins imposed sanctions include sanctions made after reports or are these actions done before mods and users see such accounts?
Edit: Removed wrong word
The ban evasion numbers include all admin sanctions, so these include accounts we proactively detect and action as well as when we respond to reports of ban evasion.
Thanks for the answer and the very detailed report! Stats ftw
Trust Data not Lore :)
Thank you for sharing. I have seen some filters in action (hatred, gore and sexual content) and appreciate them. They aren't perfect, but they catch a lot. Here are a few notes that come to my mind.
We as mods have no way of knowing whether an account marked as "ban evading" is actually doing so. Even those marked as "high" are sometimes mistakes, meaning that we can't rely on these automatic reports and filtered content (which then clogs up the mod queue). If possible, please let us also know previous usernames of such users, so that we can check the list of banned users too (where even deleted accounts are visible). The filter clearly knows more than it lets us know.
Suspensions for spam bots would be appreciated. While some get suspended within days, many never are, making me reluctant to report such bots in the future.
If a subreddit experiences lots of hateful comments in a short period of time, should we bother with reporting them all (and risk getting flagged for the "report abuse"), or will they eventually get removed by the admins anyway? I have noticed the latter happening very quickly recently.
Comments removed by the admins are rather inconsistent and whatever / whoever is doing the removals often doesn't understand the context. Same insults get removed in some cases but not in other, and some get removed even when not targeted at anyone in particular. How are people supposed to discuss insults and slurs in a civil manner when their comment get flagged, for example?
The harassment filter in the modmail catches even modmails which aren't harassing. Many mainstream subreddits have words "fuck" or "fucking" in their names (for example r/TerrifyingAsFuck, r/FairytaleasFuck, etc.), so if anyone mentions their name in the modmail, it gets flagged for "harassment" despite just saying the subreddit's name. It would be better not to punish users for doing so.
Also, I fail to see how exactly the harassment filter in the modmail keeps us safe. It really doesn't. We still get a notification for such modmails, have to check them and archive them. We aren't more safe. The only way to "be safe" is not to take these modmails to heart.
Hiya! Thanks for your feedback on the filters. Glad to hear that you have been using them, that they seem to be working well (for the most part), and your specific notes on where they can improve are really helpful.
When it comes to ban evasion, we’re balancing user privacy needs with the mod experience, which is where constraints come with sharing usernames with mods. The signals we use to detect ban evasion internally are also the ones that are used to power the Ban evasion filter, and while there will always be ambiguity with the intent of the users that the filter flags, the automated filtering serves to alert mods of suspicious accounts. When there’s a suspected account, if that is confirmed by a mod via a report, it results in a prioritized admin action. We understand that this isn't a full stop solution to ban evasion, but it's a big step from where we were last year and we are committed to continuing to evolve how we approach the problem.
In regards to spam, bot or otherwise, we’ll be working on a new mod tool to address spam and hope to have an update for you all in the next few months. In the meantime, we encourage mods to check out the Contributor Quality Score, which we made an available signal in automod in October. Similar to ban evasion, we’d like to get the right feedback loop between mods and admins to take more refined action where we need to.
As for the Modmail harassment filter, appreciate the flag! We don’t want it filtering subreddit names with profanity– I’ve passed this feedback on to the modeling team. In case you aren’t doing this already, another way you can give us feedback is by moving the content out of the filtered folder and back into the inbox. We understand that some mods want to be checking the filtered folder for false positives (and it seems like you noticed quite a few!) but our hope is to improve the accuracy and capabilities of this feature so it puts bad content out of mind for mods. As part of this, opting out of notifications from filtered inbox is something we are looking into, as well as starting to explore what other wellness features may help mods who have regular exposure to unsavory content. All in all, we appreciate your feedback and continued use of the filters, and I hope these answers were helpful.
Thank you, I appreciate the reply.
I understand the privacy concerns, so thanks for bringing it up. In the end though, it still means that we can't always trust the ban evasion filtering. I know that some mods ban based on it, but I personally don't unless the evasion is very obvious. Glad that the admins take action based on mod reports in such cases.
The bots are definitely a problem, so glad that you guys are working on solving it. I usually see the comments stealing bots banned within few days, so that's pretty good, but still lots of them to go. So, is it worth to report them for spam?
Also thank you for the notes related to the filtered modmails. They are usually correct, even though they don't catch everything (yet?). I will make sure to un-filter those which were incorrectly filtered.
I know I have asked a lot, just hopefully the removals of content because of certain words get more consistent. It seems to be overly strict when it comes to certain words. On the other hand, I am glad that it's also very strict when it comes to hateful, harassing and threatening content.
ETA: Interesting enough, you decided to skip the report abuse accusations part of my comment. Why so? It's more important than the modmail filter.
Would it be possible to give mods anonymized data about the alts a ban evader is suspected of having? For example:
Karma range (ie: 0-500, 50k-100k)
If the alt was active in the same subreddit
If the alt is banned (and very broadly if it was content manipulation, spam, hate/harassment or ban evasion)
The CQS of the alt account
There's probably a middle ground where mods have enough information to make a more informed decision, but it doesn't impact the alt's privacy or give spam operations too much insight into how Reddit flags bots.
Knowing at least the reason for the previous ban would be helpful.
Very much looking forward to that!
[removed]
Can you give an example of anti-Palestinian hate speech on Reddit that you don't feel is being moderated as hate speech?
Honestly anything Zionists say gets lots of upvotes from their army and is protected from moderation and except for in like 5 subs saying anything antizionist can get you banned.
Is there work being done to actively combat content manipulation? I see from this post 5 years ago work was being done, but with the 2024 American election coming up, the war in Ukraine ongoing, and the conflict between Israel/Hamas ongoing, many state-sponored actors from around the world have strong incentives to manipulate reddit content. Are there steps being taken to prevent this?
Doesn't seem like it. Pretty much every political subreddit has been astrotrufed by dozens of suspicious accounts. Likely coming from Iran and russia. There's also seems to be conservative/far right bots in certain sub specifcally spreading misinformation for hamas to divide the dems to get trump elected.
I agree. I get the impression that Reddit in general simply doesn't have enough actual staff to handle the astroturfing/content manipulation. If a likely state-sponsored actor can introduce a vulnurability into XZ Utils (a highly-used compression lib) by acting as a legitimate contributor for over a year, then doing something as trivial as becoming a moderator on a widely seen public forum seems highly likely.
Yup. Apparently, reddit mods had to go to the media to report the Iranian disinformation the last few times. It seems they've been doing this for years without recourse.
2021
2 charged by doj
reddit volunteers find Iranian disinformation campaign 2017
Seems it's time contact the local representative and demand some action be taken like tik tok
It's not just Iran, there is very clearly strong influence from Israel as well, just look at /r/worldnews
World news has been taken over by bad actors as well.
I primarily use old reddit. Sometimes in the mod queue an auto removed post will appear in it. However, only the "approve" button appears even though it is blatantly some form of spam. The only way to clear it is to go to the post on new reddit and click "confirm spam". I'm glad you guys are committed to maintaining old reddit as is. But it might be a little too committed as this issue has been going on for a few months now. Could you guys fix it please?
[removed]
When it’ll be confirmed, you can go ahead and talk about it. Don’t assume the result in the beginning of the process.
> This includes October 7th denialism
Can you please elaborate on what this means exactly? Obviously October 7th happened, but not all the details are clear and there's a lot of disinformation especially in the beginning claiming acts that never happend.
This question really needs a written answer. Does "October 7 denialism" mean a) the whole thing was a hoax and nothing happened, b) it was a false flag thing with Israel attacking itself, c) that Israel intentionally allowed it to happen, or d) that Israel provoked it. Of course, nobody wants to piss people off by saying it's one thing but not another, but if you're enforcing policy about it, then you have to define it.
/r/redditsecurity follows a poor practice of not defining rules and following imaginary, unwritten policies. This is okay for tiny organizations, but does not scale up to large teams.
Regarding the Israel-Hamas war and related antisemitic harassment/abuse, has there been any outreach to moderators of particularly affected sensitive communities? I ask as a moderator of one of those communities who has reached out, with no substantive response, to have conversations with admins about specific issues on the platform, and we have received zero communication from admins seeking our input. Is this something that will change at all in the short-term?
At the start of the Israel-Hamas conflict, we reached out to a number of subreddits potentially impacted by the crisis, but might have missed some. I’m sorry to hear that you didn’t receive a response to your outreach. If you write in to r/modsupport via modmail, we’ll be sure to look into it.
i find any safety and security post from reddit hilarious when i just reported someone for attempting to doxx me in one community because i made a comment in another that they didn't like, so they followed me to another completely different community and posted links to a site THEY created just to host my personal pictures that they scraped from the internet.
and reddit's response.. this was not against their TOS.. so reddit please, tell us again about that safety and security?
Referring to Israel's ongoing massacres of women, children, and other civilians in Gaza as "the Israel-Hamas conflict" is a textbook case of genocide denialism, and amounts to nothing more than the minimisation of the worst crimes against humanity that the world has seen in several decades.
Why is the denial of an ongoing genocide not against the content policy?
Speaking as someone who personally believes that Israel's treatment of the Palestinians likely constitutes genocide, or at least attempted ethnic cleansing:
Debate over whether a particular label applies to one side's actions in a conflict (and "conflict" is the term that Wikipedia is using for it) is an entirely different thing than disputing whether a documented atrocity actually happened or not.
The proper equivalent of October 7th denialism would be, say, a false claim that no Palestinian children had died in the bombing and that everyone killed was a Hamas fighter. And I'd hope that Reddit would police that at the intersection of the atrocity denialism, disinfo, and hate speech regs (not that I'm any kind of Reddit policy expert).
The current Zionist line seems to be that women and children participated on October 7th, so they're apparently now fair game to be massacred.
I wish I had your optimism, but this site has consistently been extremely one-sided in it's application of content policy.
If there's content that you feel violates specific Reddit policies but is currently staying up, can you link to an example?
Literally the entirety of r/ani_bm. Idk if things changed now, but for the first few months at least (since oct 7th), it was nothing more than a cesspool of moderator code of conduct violations.
Here's a sneak peek of /r/ani_bm using the top posts of the year!
#1: . | 202 comments
#2: Druze 🤝 Jews | 145 comments
#3: אבל ☝️ | 78 comments
I'm a bot, beep boop | Downvote to remove | Contact | Info | Opt-out | GitHub
[deleted]
Was it a conflict when European Jews fought back against the Nazis?
Nobody called that a war or a conflict, it was justified civilian resistance against a genocidal occupying power.
doesn't pay well I'd imagine
I've been trying to suck a Dick. I'm a rabbit for a week now. Everybody I asked, I've asked everybody. Something always wrong with something always goes wrong. Shall I give up? Can't believe it's that hard to find a Dick to suck.
There's a girl called yummy niaomi that has hijacked my account on here and all related platforms
Thank God we won't have to deal with it.
Are there statistics of content removed by automod and reinstated by mods?
How much of that is due to automods holding for review based on account age/karma or other non-content reasons?
[removed]
One quick message to say that the mature content filter is working pretty well and very useful.
I'm moderating a few subreddits where skin appears, and i get very few false positives/negatives.
Pass my thanks to the people working on that (and give them a raise).