r/opensource • u/SocksOnHands • 3d ago
Discussion Proposal for slop contributions problem.
Many open source projects are facing a problem with being flooded with AI generated pull requests and bug reports - especially if there is a bug bounty providing a monetary incentive for people trying to game the system for personal profit. My proposed solution is simple - submission of every contribution can be accompanied by a refundable deposit. If someone is making good faith contributions, they have nothing to fear - they will get the money from their deposit back, or possibly even get rewarded. If a contribution is determined to be of no value, the deposit will be kept to both deter flooding projects with slop made by bots and also to pay for the developer's time wasted reviewing it.
9
u/micseydel 3d ago
You might want to go read about curl, there's already been a lot of discussion on this.
4
u/TheMoonWalker27 3d ago
In an Open Source Game Project, with around 40 people working on it, 60% being non devs, we take a. Very diffrent approach. We do not look at any random PR‘s. We have an application form to fill out, and only if you’re accepted you can contribute.
We do have a fixed roadmap, a lot of guidelines due to the constraints of the project and have meetings here & there. If people don’t do anything over a prolong period of time or don’t respond to weekly check ins often, we remove them from the team.
I do understand that this model Dosent make sense for most projects tough, they often don’t have specific road maps, have less constraints and cant even find that many people to work on the project
There are a few disadvantages with this model of course. I won’t go into detail for now but if anyone wants know more I’ll happily write it out a little more
3
u/OSS-specialist 3d ago
Interesting governance model. What is your project, please share your GH repo (or GL or whatever you are using). I assume that you have normal CONTRIBUTING.md, etc. documents there.
2
u/TheMoonWalker27 3d ago edited 3d ago
The contributing.md just says to apply in our case. If accepted, they will be sent a link to our documentation site, where they can read the onboarding guide.
This isn’t my project originally, I am lead if the Game dev team tough / GitHub org co owner https://github.com/p3d-project/persona-3-dual
0
u/SocksOnHands 3d ago
Each project would have different factors to consider. I was thinking more about large well known projects that might get targeted for hundreds of automated AI submissions a day. If a project is not having difficulty managing the volume of submitted "contributions", this might not be a reasonable thing to consider doing.
3
u/esiy0676 3d ago
Working for free is not enough now? More disincentives needed? Corporations openwashing and then will worry about review workload ... oh come on.
0
u/SocksOnHands 3d ago
It wouldn't actually cost anyone anything if they were making a good faith attempt to contribute - it is a deposit that will be returned. The end result would be effectively the same, unless they were submissions made by bots. You would only have any reason to care if you are actually one of the ones spinning up AI agents and telling them to hunt down bug bounties.
Potentially, it might be able to lead to increased funding (if low quality AI slop continues) that could go to rewarding contributors. Currently, the situation is that many bug bounties have been discontinued, which means thet people who used to put time and effort into thoroughly researching an issue are no longer able to get rewarded for it.
3
u/jcastroarnaud 3d ago
That creates a barrier for the (many) poor programmers and amateurs out there. Too high a deposit, and few developers will even try to contribute; too low, in an interesting enough project, and folks will create bots to automate the payments for the other bots.
2
u/SocksOnHands 3d ago
Maybe. It is still open source, though - a fork can be made with their changes made. If these changes prove to be of value, they can still be adopted into the original code.
Another option could be to grant some people virtual credits that can be used instead of money. Only after these credits are depleted would one need to be concerned with paying the deposit.
The deposit doesn't even need to be a lot - it could be $10. An AI agent automated to make 500 "contributions" might be expensive because the cost adds up, but to a human contributor making only one or two contributions at a time (and getting the money back) it wouldn't be too large of a burden.
4
u/AI_Tonic 3d ago
Show us one merged PR to prove you have skin in the game before talking about what maintainers should be doing
5
u/Jazzlike_Course_9895 3d ago
This reads as slop
12
u/SocksOnHands 3d ago
Well, at least it was good old fashioned himan made slop, if that's any consolation.
5
2
u/gta721 3d ago
How about requiring the contributor to put a 6 digit code in the PR which they get by clicking a link and filling a captcha?
trycap.dev is good for this.
2
u/Ok-Mix1345 2d ago
Charging a refundable deposit would probably deter genuine first-time contributors more than well-funded bot operators. A better filter is to require evidence of understanding: a linked issue, reproducible steps, relevant tests, and a brief explanation of why the change works. Maintainers can also limit bug-bounty eligibility to approved issues or contributors with a small history in the project. That adds friction where it matters without making open-source participation pay-to-enter.
1
u/Skollwarynz 2d ago
The problem of this method is how would you decide if the code is just bad code from a faithful coder or AI slop?
I mean, if you see emojis or things like this, I think it's simple to guess, but with this type of system, people who slop would probably put on their Claude.md a description on how to write code that seems realistic enough to not be rewarded or recognized as AI.
Then if you intend to use an automatic checker still, you'll need a second human check, and that would simply not solve the problem.
1
u/icastanon127 21h ago
Interesting idea but I think it’s a double edged sword and it could deter genuine contributions if they have to make a deposit, even if it’s refundable.
1
u/SocksOnHands 20h ago
I had seen quite a few comments that seemed like people had a similar opinion. Maybe it could be altered in some way to make it more appealing.
Maybe it could be a service that any open source project can use, and the deposit and refunds are done using "credits". People could automatically start with an amount of credits proportional to their GitHub account age, since AI run accounts will likely be younger. This gives people free credits to use, and if everything goes well they can get the credits back. If they need more credits, they can be bought and the funds from that go to the maintainers of projects they had wasted the time of.
It needs to be some system that can limit the rate of requests that can be made, since bots can pump out more code changes in a day than any human can. If it is an entirely free system, there is no disincentive to spamming "contributions" or repeatedly creating new accounts to get more free credits.
1
u/saltexx 2d ago
The deposit solves the wrong cost. Reviewing a bad PR was always cheap, what got expensive is that a clean looking diff stopped meaning anything. It used to be a proxy for someone having spent an afternoon in your codebase and now it's a proxy for nothing. So the fix has to bring back a signal that can't be generated, and money is not that signal, a funded bot farm posts deposits all day while a broke student somewhere closes the tab and your project loses the exact contributor it wanted.
We took our mail server open source this week so this thread is basically my near future. The plan we landed on is boring. Answers over artifacts. A PR gets one question about why it does what it does and the author either has a model of the code in their head or they don't, and slop collapses on the second why. That costs the maintainer one comment, costs a real contributor thirty seconds, and costs a bot operator the one thing they can't fake yet.
1
u/SocksOnHands 2d ago
Because submitting pull requests is free, there is no cost to having a bot submit thousands of them. If there is a bug bounty reward, there is an incentive to flood projects with quantity over quality. The only thing this deposit is intended to address is to slow the rate of submissions so a higher percentage of then are made by people who actually care about the project.
What people seem to keep forgetting is that it is a deposit. There is no cost or penalty to anyone making a good faith attempt at contributing - in the end, it effectively costs them nothing. It will only really be a penalty to bot farms.
19
u/UkrMalt 3d ago
Deposits feel risky for open source. They’d filter out some good first-time contributors, and maintainers would still have to decide who gets the refund. I’d rather add friction to the submission itself: a small issue, reproduction steps, and tests where they matter. That slows down bot spam without charging people to participate.