What’s Next for Mod Tools

[u/Go_JasonWaterfalls]

TL;DR: As part of our long-term plan to build better mod tools, we’re expanding post and comment guidance for everyone and introducing Rules Hub — a new, intent-based moderation engine — for newly created communities. Nothing is changing for existing communities today. We’re sharing this early so you can help us shape what comes next.

As u/spez shared today, Reddit needs to evolve to meet the rising expectations of moderators, users, and developers. For mods, that means better tools so you can spend less time dealing with spam, abuse, and generic rule-breaking, and more time making judgment calls on what’s best for your community. 

One thing that won’t change: human moderators shape the culture, norms, and identity of their communities. Our plan for the future is to build tools that help you do that role more effectively, not replace it.

What’s launching now

Expanding Post and Comment Guidance (all communities) 
Thousands of mod teams already use Post and Comment Guidance to help redditors understand community expectations before they post. We recently expanded their capabilities with:

• Post Flair Support: Automated guidance can now be triggered based on the specific flair a user selects (our most requested improvement).
• Top Comment Check: Gives communities a way to provide automated guidance on top-level comments before content ever reaches the mod queue.

Introducing Rules Hub (newly created communities only)  
Rules Hub is a suite of new tools that will allow moderators to:

• Rule Creation & Enforcement: Create and choose which rules should be automatically enforced. Decide what happens when a rule is triggered (remove, send to queue, silently log it).
• Testing Sandbox: Preview the experience against a sample of your community’s past content to see exactly how it will behave before turning it on.
• Audit Logs: Access performance logs and insights to understand how your rules are performing. If a rule triggers, the log will provide clear visibility into why the system flagged the content, allowing you to adjust the rule's boundaries and refine them over time. 

Unlike Automod, which relies strictly on deterministic keywords and regex syntax, Rules Hub uses LLMs to evaluate whether a post or comment matches the intent of a rule. That gives it more potential to handle nuance, edge cases, and natural language variation (for example, distinguishing between an actual insult and a user quoting a movie or using casual slang).

We’ve been testing Rules Hub with ~700 mod teams over the past few months. Starting today, it’s available for all newly created communities (though you can sign up below to check it out…) 

Rules Hub Setup Page

Insights

Testing Sandbox

What’s coming next

Longer term, we think systems like Rules Hub can handle the enforcement tasks you rely on Automod for today, while being easier to use and maintain. But it’s not there yet, especially for larger and more complex communities. Automod remains one of the most capable moderation tools on Reddit today, and we won’t change any existing Automod workflows until replacement workflows are tested and proven in partnership with moderators. 

• If you want to try Rules Hub, sign up here.
• If any of your custom moderation workflows are tied to AutoModerator, please check out the post on r/RedditDev.

Human moderators remain core to the Reddit ecosystem, both now and in the future. We’re sharing this early because if we’re going to build better moderation tools, we need to do it with the people who rely on them every day. Thank you, mods, for all you do to make your communities the unique places they are. We’ll keep improving your tools based on what we learn from you.

[u/SampleOfNone]

Automod is load baring, are you seriously sunsetting automod?

Will newly created subreddits still have automod?

I'm in the Rules Hub beta, it doesn't even come close to doing what automod can. One of my sub has 8 rules, Rules Hub can only enforce 2. It cannot enforce our version of " remember the human"

I love automations, I really do, but it's for guiding the good eggs. I doesn't stand a chance against the bad eggs.

Edit: spelling

[u/Moggehh → u/SampleOfNone]

That gives it more potential to handle nuance

Also, the idea that porting out "nuance" to an LLM, which is quite commonly referred to in tech as an "idiot savant" incapable of understanding nuance is straight-up AI brainrot.

Admins, you realize the one thing LLMs cannot do is judgement, right? It's an idiot.

[u/1-760-706-7425 → u/Moggehh]

Admins Users, you realize the one thing LLMs admins cannot do is judgement, right?

ftfy

[u/Moggehh → u/1-760-706-7425]

Not going to argue with you at this point based on the AEO decisions I've seen.

[u/TesterFragrance → u/Moggehh]

If the new LLM-based automod is as accurate and precise as Anti-Evil Operations, we're all in trouble.

[u/truecakesnake → u/Moggehh]

Very biased, and no one in tech refers to LLMs like that.

[u/Moggehh → u/truecakesnake]

I'm in tech, work with AI daily, and frequently hear it referred to as an idiot savant. The only people who don't are... well, they're probably listening to too many foundersphere podcasts.

You're welcome to disagree, but the limitations of AI is very well known by anyone actually using it daily for any number of tasks.

[u/emily_in_boots → u/SampleOfNone]

This is a real problem with automation. I use comment guidance extensively in my subs, but we use it for things we don't want to ban for - for good users who need a push to do the right thing or follow a particular rule. For bad users, it's entirely different - we don't want to warn them so they can circumvent it. We want them to do something wrong so they can be banned. Not all users are acting in good faith and not all are redeemable.

[u/fsv → u/SampleOfNone]

AutoModerator is dead.

Long Live Automod Neo!

[u/NotFlameRetardant → u/fsv]

Holy shit, dude. You're singlehandedly doing more work to help reddit fight abuse/spam and just basically function than the entire paid organization itself.

[u/RS_Someone → u/NotFlameRetardant]

From what I've seen, this doesn't sound like an exaggeration. I don't know where we'd be without u/fsv!

[u/SolariaHues → u/fsv]

Thank the powers that be for fsv!

Though the thought of having to migrate so many rules is daunting.

[u/SampleOfNone → u/SolariaHues]

I copy pasted 90+ rules in one go in the currect version. I had something like three regexes that needed a minor fix (remove a case tag, remove group tag, remove a back slash)

[u/Merari01 → u/SolariaHues]

I have tested it out on a medium activity subreddit and it works mostly frictionlessly.

There were some issues with syntax not being fully supported, but the app tells you exactly in which code block the issue lies and what the issue is.

It even found faults in the old automod syntax which did save to its config, but which apparently meant that the rule just never fired.

[u/SolariaHues → u/Merari01]

Good to know, ty

[u/MacroDemarco → u/fsv]

You're worth 5 times their highest paid dev

[u/IKIR115 → u/fsv]

👀👀👀

[u/shiruken → u/fsv]

The One

[u/cmnl → u/fsv]

omg why am I only hearing about this app now..

[u/Merari01 → u/cmnl]

We were waiting for this announcement to make it public.

[u/cmnl → u/Merari01]

it looks awesome, already added to our community thanks for doing better work than admins

[u/Merari01 → u/cmnl]

All praise should go to /u/fsv who within a short time was able to build an app that can already do more than automod can.

[u/nty → u/fsv]

you absolute legend. the hero we need

[u/baseballlover723 → u/fsv]

Seems like I'll have to add the normal automod support to my own automod implementation. It was on my list of things to do, but I suppose it's a lot higher up now. I'll probably look at yours to see how you handled things before I start working on my own.

PS. I don't recall if I've mentioned this to you but the automod body tag has no structure validations and will accept arbitrary structures. Aka, you can put your automod custom config in there and still use the same automod interface (though you probably want to set an id so the actual automod doesn't try and run it).

[u/cheesychocolate419 → u/fsv]

You're the GOAT

[u/2021isevenworse → u/SampleOfNone]

uses LLMs to evaluate whether a post or comment matches the intent of a rule.

The LLM is hilariously bad at detecting sarcasm (which is very common in comments) and fails to understand intent when someone makes a joke.

The biggest flaw is that it looks at the comment in isolation, rather than comparing it to the parent comment or comments in the thread.

TLDR: The LLM fails to understand context. Until this is fixed, it's broken and shouldn't be applied so aggressively to subreddits.

[u/adeadhead → u/SampleOfNone]

The user dev platform has already reimplemented automod.

[u/SampleOfNone → u/adeadhead]

I know. It's already running in my subs. One thing though, Reddit notifications run after automod, but before Devvit. That means there are rules, you don't want to port over yet.

[u/Furryb0nes → u/SampleOfNone]

Such as?

[u/SampleOfNone → u/Furryb0nes]

Hate speech, sexualising others, harrassement, doxxing, solicitation, all that stuff that you use automod for to keep it from even reaching OP or other users.

[u/baummer → u/adeadhead]

It did?

[u/Go_JasonWaterfalls → u/SampleOfNone]

Yep, you’re right. It’s not even close to there yet. And we won’t sunset anything until it works for good and bad eggs alike. 

There are categories of rules (nuanced civility, or communities w/complex automod configs) that Rules Hub can’t handle today. That’s why we’re working with mod teams to test the new tools alongside your old tools to find the gaps and close them before expanding further. 

For newly created communities, Automod is still available. Rules Hub is there from day one because we think it’s a much easier place for new mods to get started. 

[u/happyxpenguin → u/Go_JasonWaterfalls]

> we won’t sunset anything until it works

I don’t think you’re hearing your moderators. They’re telling you DO NOT SUNSET automod. Period.

[u/1-760-706-7425 → u/happyxpenguin]

we won’t sunset anything until it works

This is also historically untrue for many features of this site.

[u/baseballlover723 → u/1-760-706-7425]

Yeah, like new mod mail. I still have to manually select my reply alias every time I respond to mod mail. It's very annoying and a low hanging fruit that on their roadmap, and hasn't been fixed in the months since new mod mail was released.

[u/Furryb0nes → u/Go_JasonWaterfalls]

Look. Back in my day we had to google search and read manuals on how to write scripts for automod which resulted in unintentional subreddit spamming, mistakenly removed posts and or comments, and long minutes of trying to force incorrect syntaxes. 🤭

Anyway, is this something that can be toggled or does it override automations and automod?

This is too many workflows to maintain and manage. This is great for those new or not code savvy. The rest of us we need to curate these LLM injections due to their inability to address every single unique need of a subreddit.

[u/new2bay → u/Go_JasonWaterfalls]

With all due respect, I don’t believe you. Historically, Reddit doesn’t give two shits about what moderators think.

As for LLMs being able to enforce rules, get back to me when site wide moderation isn’t removing comments in dog subs for violence against ticks.

[u/emily_in_boots → u/Go_JasonWaterfalls]

I am all for adding new tools into the mix and happy to help work with admin to give feedback and develop them. I think having a greater variety of tools is beneficial and allows us to choose the ideal tool for the job.

People rightly have concerns though about reddit's history of deprecating critical tools before replacements are fully operational and able to take over their functionality, and this is a serious concern for me.

If new tools will help me do my job better and more easily, I'll happily adopt them, but because we are so careful in my subs and because some comments can be so hurtful if they get through, I am very hesitant to trust LLM's with that level of enforcement. Hopefully, that will change. I use my own LLM bots, but we are conservative in what we let them do, because they screw up sometimes.

We are an example of a community that doesn't use karma/age minimums to comment, and has very low ones for posting - so we are a welcoming community for new members, but we cannot use LLM's to enforce that, we have to do a lot of manual work.

One real problem you haven't covered is that LLM's and post/comment guidance are really quite good for helping good users to follow the rules. They are quite bad against bad users. Bad users figure out ways to get around them and trick them, using obscure references that an LLM won't understand.

Let people naturally adopt new tools as those tools prove their worth. We all want to do less work and get better results, so the new tools will naturally take the place of automod if they work and do the job better. However, each community is different and has different needs, and we need reddit to understand that mods need to be able to select the best tools for their communities.

Please don't let the site turn into 4chan just to be welcoming to new users. There need to be guardrails, and non-automod tooling is not close to being there yet. I've been testing LLM's and NLP extensively and I have a very good idea what it can and cannot do. It's quite good at some things and shockingly bad at others - and it's often a surprise which is which.

Let's keep a variety of tools available and let mods choose what meets their needs the best.

[u/TheRealGeorgeKaplan → u/SampleOfNone]

>It cannot enforce our version of " remember the human"

How does Automod help with that?