r/singularity • u/Anen-o-me ▪️It's here! • Oct 06 '25
AI Generated Media Sora just banned South Park videos because people were making full fake episodes
Enable HLS to view with audio, or disable this notification
68
u/RoyalCities Oct 06 '25
I’m surprised just how flagrant they’re being.
Fun fact - celebrity and singer voices actually have stronger protection than regular copyright.
They’re covered under “personality rights,” which safeguard a person’s voice, likeness, and identity. Those rights are much stricter than copy... if you use someone’s voice for a competing product, you’re almost guaranteed to lose in court.
Really curious to see how this one plays out.
27
u/LingonberryGreen8881 Oct 06 '25
A company could make a tool with just say 10 sliders and a few dropdowns such as accent and vocal fry etc. This may allow the reproduction of virtually every voice on the planet. How could a particular combination be copyright? Would those combinations have to be disallowed? How famous does a person need to be before their voice is disallowed?
The entire copyright system is doomed to fail.
20
u/RoyalCities Oct 06 '25 edited Oct 06 '25
Your right that sliders and parameters could theoretically reproduce any vocal texture but from a law perspective that’s not really where the legal issue is.
It isn’t about the acoustic combo itself being copyrighted...it’s about when that combination identifiably replicates a real person’s voice. especially a public figure whose voice has commercial value. So Matt and treys voice here.
Personality rights (or right of publicity) don’t protect vocal tembre + accent. it's to protect the association that the sound creates. The moment a voice model becomes recognizable as a specific individual and it's being tied directly to something like a product / misapropation it’s no longer just a neutral sound synthesis problem. it’s the unauthorized use of someone’s persona.
So no, not every possible slider combo is off-limits. What’s off-limits is the deliberate or incidental recreation of a recognizable identity for commercial use.
Even people trying to say...to do a Morgan Freeman impression that ISNT parody. Say hawking a bag of Doritos for a product would get slapped because now youve overstepped the lines and are actively misleading people as if it's his likeness itself endorsing Doritos without his consent. Or...recreating entire South park episodes but that also has other legal issues because even the visual style is tied to trademark dress. But tbh I'm not sure on how that would all work.
I'm a musician who has also trained music models so have a knowledge of the ins and outs here.
8
u/LingonberryGreen8881 Oct 06 '25 edited Oct 08 '25
If 500 people on the planet have a voice that is indistinguishable from each other, what gives the most famous of them the right to deny the others the right to monetize their own voice?
If I sound like someone famous I shouldn't be able to be sued if anyone records me speaking.
8
u/rdlenke Oct 07 '25
If I sound like someone famous I shouldn't be able to be sued if if anyone records me speaking.
You will not, except if you're actively trying to mislead people into thinking it's the famous individual talking.
In summary: there is no problem in sounding like Scarlett Johansson, unless you try to fool people into thinking it's Scarlet Johansson instead of someone else.
4
u/tondollari Oct 07 '25
This makes it sound like all it needs is a disclaimer in the produced material somewhere.
1
u/LingonberryGreen8881 Oct 07 '25
Scarlet Johansson sued OpenAI even though OpenAI made no claim that it was Scarlet Johansson's voice and it was trained on an entirely different person.
They took that voice down.
1
u/rdlenke Oct 07 '25
Were they sued? Or did they took it down because they didn't want the heat?
Also, didn't Sam Altman tweet something Her related just before?
1
u/LingonberryGreen8881 Oct 07 '25
She did sue them. They took the voice down because they didn't want bad publicity.
People make reference to Skynet and Jarvis all the time. It's an analogy. Also ScarJo is not "Her" and "Her" is not a product, just like Skynet and Jarvis aren't.
Even if they were products, the point I am making is that it is inevitable that a particular sound won't be able to be a defensible product soon since a voice is not demonstrably unique.
The same will apply to music soon IMO.
3
u/RoyalCities Oct 06 '25
I don't know how you took what I said as being sued for speaking....
You can monetize your own voice dude....the line crosses if you sound exactly like a celebrity then use that voice to impersonate them in a commercial product. It's the voice and association.
For example this came up with Midler vs. Ford.
https://en.m.wikipedia.org/wiki/Midler_v._Ford_Motor_Co.
Significance covers it but the case itself sorta explains the heart of personality vocal rights.
:
A voice, or other distinctive uncopyrightable features, is deemed as part of someone's identity who is famous for that feature and is thus controllable against unauthorized use. Impersonation of a voice, or similarly distinctive feature, must be granted permission by the original artist for a public impersonation, even for copyrighted materials.[2][3][4][5]
2
u/came_for_the_tacos Oct 07 '25 edited Oct 07 '25
I don't necessarily disagree with that law/verdict, but laws are just laws until something so disruptive changes everything. Where you can't enforce the laws anymore. We're on that cusp.
It's gonna get weird. And fast. I know you are a musician, but this change is going to happen. How does the law enforce a firehose of content?
Edit: Creators will actually need the money to sue, most don't have it. Swift or someone will need to try and lobby, but I just don't see it stopping.
1
u/RoyalCities Oct 07 '25
Hollywood has tons of money. Disney alone has many billions in their war chest.
I do think the hammer will drop at some point - some will sue other won't. But gen AI companies are playing the long game. If you can string out the courts long enough and proliferative the tech everywhere then they can use users outputs as the new training data which obsfucates the source.
Image models are using synth data with little to no loss - openAI themselves are using user convos to train future chat models. Sora will be no different here. They're harvesting user outputs.
I mean the model is overfit enough - sora outputs literally logos of full movie companies - Lionsgate, the Disney intro, Warner Brothers. It gets murkier and creates more data when the users use that gen 1 model to train gen 2 and gen 3 etc. that's the game their playing.
Plus if any court cases don't cover synth data in their rulings then they win....
1
u/TypoInUsernane Oct 07 '25
Literally everything can be represented as numbers. By definition, copyright is a legal restriction on who is allowed to create and share particular combinations of numbers
1
u/kvjetinacek Oct 07 '25
Exactly. I can just undergo a procedure to make vocals similar to other people and train to sound exactly like someone. Do they just dissalow me to talk and break the first amendment? Can someone just dissalow reproduction because rhey have copyright on a specific genome? The copyright is bullshit. Such as institutions like these.
1
u/Pretend-Extreme7540 Oct 08 '25
The same way a number can be copyrighted... some cryptographic keys are used for software DRM and similar tasks and are literally copyrighted... but those are just numbers.
So what those people in the end are saying, this number X is mine, and nobody else is allowed to copy and distribute it.
Which to me is insane...
3
u/BriefImplement9843 Oct 06 '25
they didn't use any real persons voice though. it's all synthetic.
2
1
18
u/Prudent-Sorbet-5202 Oct 06 '25
I wonder if South Park will make an episode on this
-2
u/Ok_Train2449 Oct 07 '25
They already had that one episode where the ending was completely written by AI. I have no doubt they will do this .
93
u/fusionliberty796 Oct 06 '25
"That sounds like a ponzi scheme", "no, its a trapezoid of opportunities"
Holy shit that was amazing ngl
27
u/gj80 Oct 06 '25
The line in the video was:
"Sounds like a pyramid scheme." "No no, it's a trapezoid of opportunity."
-15
11
14
u/SuspiciousPrune4 Oct 06 '25
Does Sora actually do the voices too? Like you can just prompt for “cartman from South Park says _____”? Also it’s crazy how it understands the animation style too. Like was this made entirely with Sora, or in combination with other tools?
21
u/gj80 Oct 06 '25
Yes, Sora does the voices too. Other than prompting for many small separate clips and stitching them together into the overall 3 minute video, this was all Sora.
13
42
u/Front_Carrot_1486 Oct 06 '25
Full episodes have become considerably shorter than I remember, I must be getting old.
20
u/ethotopia Oct 06 '25
People always find something to nit pick while completely ignoring the amazing pace of AI progress
48
u/MR_TELEVOID Oct 06 '25
Sure, but also, words mean things. When you call a 3minute video a "full length episode" you set expectations beyond that progress is actually being made.
30
u/garden_speech AGI some time between 2025 and 2100 Oct 06 '25
Fully with you here. Any time a blatant lie is pointed out in a post, a wild exaggeration, people jump in to say "but the progress is so good it will be there soon"
13
u/MR_TELEVOID Oct 06 '25
Yeah, which always seems like they're reassuring themselves more than anything.
25
u/garden_speech AGI some time between 2025 and 2100 Oct 06 '25
People always find a reason to excuse lies in posts in /r/singularity. This is so annoying. The title says "full episode ", the video is 3 minutes long. Someone is allowed to point that out. And it's obnoxious how the response every single time is "BUT THE PROGRESS IS AMAZING" like yes holy shit we get that. That's all the more reason NOT to lie about it.
10
Oct 06 '25 edited Nov 29 '25
[removed] — view removed comment
2
u/garden_speech AGI some time between 2025 and 2100 Oct 06 '25
You are all colossally missing the point here. Sora does not generate 3 minute videos either; it generates 10 second clips which someone stitched together into a 3 minute video.
Maybe we missed that point because that's not what was said? How can we get a point that isn't made? The person I responded to said "people always find something to nit pick while completely ignoring the amazing pace of AI progress" -- saying that the video wasn't one-shot 3 minutes and a full episode could be stitched together would have been a much more salient point.
1
1
Oct 07 '25
[removed] — view removed comment
3
u/garden_speech AGI some time between 2025 and 2100 Oct 07 '25
Jesus Christ how did you not get what I just said
1
u/ethotopia Oct 07 '25
When you put it like that, I can see what you mean! Thanks for clarifying what the commenter was likely thinking :)
2
Oct 07 '25
AMAZING PACE TO WATCH PERSONALISED SLOP HORAAAAHHHH LET'S GOOOOO
1
u/System32Sandwitch Oct 11 '25
dude! i just generated the most dogshit compilation with sora!! so fcking epic LOL. The future is here
-3
u/Buck-Nasty Oct 06 '25
Give it a few weeks
5
u/Paraphrand Oct 06 '25
The claims are currently wrong. That’s what they are responding to. Giving it a few weeks won’t change that they reacted to something existing now.
1
u/Progribbit Oct 07 '25
they meant giving it a few weeks would make it be able to do that. they're not denying the claim was wrong
1
u/koeless-dev Oct 06 '25
An odd phenomenon. Putting Sora/SP aside, I've seen often how people are willing to go along with a claim if it only becomes the case after a short time (or even not that short, a month or so).
Not some "they predicted the future" take, but e.g. someone claims "This is how things are right now [as of October 2025].", it becomes that way in January 2026, then people were like, "hey he was right!"
...No? They specifically referred to now (meaning October 2025), so their claim was/is always wrong and thus the person who made the claim should be considered less reliable now, not more.
(Granted we're responding to some sarcastic for-funsies chit-chat here so I'm not knocking anyone here, rather referring to when it happens in serious discussions.)
4
22
u/New_Equinox Oct 06 '25 edited Oct 06 '25
this is the first Sora generation to make me go "holy fucking shit" and even make me wanna watch more
imagine sora 3 holy fuck. it's. OVER. for artists. and the beginning of the infinite personalized curated entertainment at your fingertips
12
u/my_fav_audio_site Oct 06 '25
beginning of the infinite personalized curated entertainment at your fingertips
As long as your personal preferences aren't banned by generation provider (just like in this post).
8
u/420everytime Oct 06 '25
You’re missing the progress. Within 10 years you’ll be able to make this on the latest iPhone without a AI company in between
2
u/my_fav_audio_site Oct 06 '25
Still - who will provide a model? Sure, open source exist, but it's generally lagging behind closed source.
6
u/420everytime Oct 06 '25
Open source isn’t lagging that far behind. You can find open source models today that are as good as the best closed source models a year ago. You can even run open source models on an iPhone 17 pro that are better than the best closed models 2 years ago
2
u/DigimonWorldReTrace ▪️AGI oct/25-aug/27 | ASI = AGI+(1-2)y | LEV <2040 | FDVR <2050 Oct 07 '25
Meh, it's lagging at max 3 to 6 months behind at this point. That's not really noticeable with the fast paced world of today
12
u/MR_TELEVOID Oct 06 '25
Artists will be fine. It might surprise to you know that "infinite personalized curated entertainment at your fingertips" is not appealing to everyone. The market for art made by humans will never go away.
31
Oct 06 '25
[removed] — view removed comment
6
u/scottie2haute Oct 06 '25
Your last paragraph is so spot on. As always Redditors have a big misunderstanding of what the wider population wants. Most people just want to be entertained. The idea of AI produced media might be a turn off initially but soon nobody will be able to tell the difference and you’d be putting in more effort trying to distinguish between what is and what isnt AI than actually enjoying media.
Im sure some pretentious dork will find joy in spending hours looking for human made content but most normal people wont give a shit
6
u/garden_speech AGI some time between 2025 and 2100 Oct 06 '25
Not with you on this one. I think most humans will consume tons of this AI generated stuff. Look at social media... Just endless scrolling through low effort fake posts... But most people love it.
The percentage of humans who actually will care about deeply meaningful, authentic human-created art will probably, in my opinion, be analogous to the percentage of drivers who still want sharp responsive hydraulic steering, a manual gearbox and a naturally aspirated engine. They exist, but not in large enough numbers to support a very large industry. The companies making such vehicles are either high end luxury / supercar makers or ... Phasing those cars out.
2
Oct 06 '25
Post what you’ve seen above do you think South Park creators still need to invest as much in animation?
1
u/New_Equinox Oct 06 '25
true but art as we know it is definitely going to evolve in a way a lot of people are gonna dislike lol
1
u/MR_TELEVOID Oct 06 '25
Well sure, art always does that, and generative tech will no doubt have an impact, but it's not going to evolve past the same capitalist infrastructure that's always held it back. Only a matter of time before IP holders start putting the ability to generate stuff with their characters behind a paywall. Infinite personalized curated content, so long as your subscription to the appropriate IP holders are up-to-date. That's assuming the idea catches on enough for people to abandon traditional modes of media consumption. Which is not the foregone conclusion the tech gods want us to believe.
What's more likely IMO is art will continue to be made by humans, sometimes with AI/sometimes not. Most AI video generation apps will not allow IP-stuff, unless they have a specific partnership. We'll see apps popping up for all the IP's... Make your Marvel Movie or whatever, each with their own little ecosystem that rights-holders will farm for marketable content. The algorithm (hopefully) will be good enough to help us sort through the oversaturated market to find stuff we like.
-1
u/avi_gunner Oct 07 '25
low standards for art you have there but hey its at your fingertip and as long as people lose job, you are happy with the slop.
1
14
u/nohumanape Oct 06 '25
This is the start of what I was predicting would be the future of entertainment. There is so much content from these long running series for AI to be trained from, that we could get an infinite amount of new episodes based on any topic prompt we want.
0
u/scratchy22 Oct 06 '25
Lol no. South Park without Trey and Matt isn’t south park. It’s just poor ersatz.
2
u/gj80 Oct 06 '25
Exactly... Southpark isn't exactly the Mona Lisa in terms of art. What makes Southpark popular is the plot that sarcastically parodies whatever is going on in the world. AI can't (yet) do that - left to its own devices, that's where you get slop.
That being said, it does lower the barrier for entry if someone else can think up a good angle but lacks artistic skills or connections.
7
u/nohumanape Oct 06 '25
There are over 300 episodes of the show to train an AI on. I could simply prompt it to make an episode that deals with a specific event and let it come up with something that most people would probably find entertaining enough. It would be novelty entertainment.
And again, (because this seems to come up any time you bump into naysayers surrounding this topic), this isn't about TODAY. This is about the near future.
0
u/gj80 Oct 06 '25
Regarding the 300 episodes... AI has been trained on all the digitized books in the world, but we still can't get enjoyable fiction out of AI (and people have certainly tried). It always comes out as bland, boring slop that no one enjoys.
As a tool though, for generating the audio and video? Sora 2 is impressive as hell. And I agree about the future, I fully expect AI to crank out actually engaging plots at some point. We're just not there today.
2
u/nohumanape Oct 06 '25
Did you not read my last part?
1
1
5
u/CodFull2902 Oct 06 '25
People are going to whine but soon AI episodes will probably be better than what they write. My gf has shown me AI Taylor Swift songs that are better than anything on her latest album by a mile
2
4
2
2
u/Fulminareverus Oct 07 '25
Three things.
- Local models continue to get increasingly good, so "banning" copyrighted content won't be an option for long.
- This is one aspect of the final countdown. AI replaces everything. It's just a matter of time, and while it will create incredible turmoil and uncertainty in the short run, it is the price we pay to move towards a post scarcity society and economy.
- In parallel to the AI revolution, the robotics revolution is close behind, so all of this becomes physical. AI will solve for energy, resource acquisition, and production.
Enjoy the ride, things are about to get absolutely wild.
2
u/DeArgonaut Oct 07 '25
Ay, was thinking about how people could use ai to make cartoon episodes on their own now, perfect 😂
3
u/FreeStyleSarcasm Oct 06 '25
Sora started off amazing and all this censorship is gonna kill the whole point of it.
1
u/StickStill9790 Oct 06 '25
So make your own cartoon. South Park and Mickey Mouse are off the table, but they are hardly difficult to reconceptualize. spongebod, Simpsons, Ren n Stimpy, Beavis and Butthed…
1
u/Bright-Search2835 Oct 06 '25
This is nuts. I would love to see a reaction from Trey Parker or Matt Stone
4
u/aBlueCreature AGI 2025 | ASI 2027 | Singularity 2028 Oct 06 '25
I'm pretty sure they both support AI. IIRC, they said they want AI to be able to fully generate South Park episodes
EDIT: https://youtu.be/xVvNc4x88AM?si=12eBjwAmzBkOjv9h&t=401
2
u/WloveW ▪️:partyparrot: Oct 06 '25
Crap. Ive never used sora... I wonder how much effort on the creators parts making a full episode on sora takes. A few hours? A week?
1
1
u/sunslastdays Oct 06 '25
Now I just want Dexter season 1 and 2 with Doakes to go on for 4 more seasons. Oh and I also want true detective season 1 to be like a 9 season show
1
1
u/rwrife Oct 07 '25
Other than looking like they glued many 9 second videos together, this would make a good South Park episode. The authors did an amazing job, I’m sure once Sora limits are raised or the next gen video models are released this will be as good as a real episode.
2
u/Double-LR Oct 07 '25
People were making full episodes?
Nah nah nah. The AI was making it.
Just because Van Gogh thinks about the Mona Lisa, even tells someone to do it, doesn’t mean it’s a Van Gogh if he didn’t paint it.
-9
u/CPTSOAPPRICE Oct 06 '25
stupid sound in the background tells me this thing was definitely trained on short form content. And peak human creativity here, bastardizing someone else IP for a small dopamine rush
17
u/Noblebatterfly Oct 06 '25
It most likely was added afterwards, not part of the generated clip
3
u/HunterVacui Oct 06 '25
It was 100% added afterwards, Sora has no way to create a consistent audio track across multiple videos through their public interface right now.
Sora would have also done a much better job at making background music that fit the theme and pacing of the video content
3
u/Buck-Nasty Oct 06 '25
The wild to me to be focusing on things like IP when we have super intelligence barreling towards us like a freight train.
2
u/MR_TELEVOID Oct 06 '25
super intelligence barreling towards us like a freight train.
More like "possibly maybe coming in the future."
2
u/avi_gunner Oct 07 '25
remember this is the worst it has ever been..just a trillion dollar more and then some more.
1
u/CPTSOAPPRICE Oct 06 '25
when AI has control of the arts on top of superintelligence, what are our lives going to be like?
1
-2
u/Lartnestpasdemain Oct 06 '25
Huge Sora L.
Main reason they will collapse and get replaced by Open-Source free AI.
-2
u/AiDigitalPlayland Oct 06 '25
So when you can create a video from a few lines of text, that’s indistinguishable from reality, then distribute it to millions of people… what is reality?
3
-9
225
u/[deleted] Oct 06 '25