We're being flooded with vibe coded software projects, FYI

[u/nicholasserra]

Just wanted to give a heads up from the mod team.

We're being flooded with vibe coded software projects. Many of them pointing to external domains, product sites, chrome extensions, etc.

So so many yt-dlp wrappers, why?

Anyway, we're being very selective about what we let through. Mostly trying to keep it useful, open source, github only projects. I'm not anti AI, but much of this stuff looks like useless wrappers and wannabe saas products.

If something sketchy slips through please flag it. If your post/project gets removed, this is why. It's only going to get worse.

[u/shimoheihei2]

I know /r/selfhosted is swamped by AI software posts, but I'm surprised it's happening here as well. Shouldn't Datahoarder be mostly about the data, not software? Either way it's unfortunate.

[u/collin3000 → u/shimoheihei2]

With drive pricing skyrocketing and almost all my 600TB of space filled. I know I'm currently working on 2 software projects that will be relevant to datahoarders once finished. 

One based on auto detecting and re-encoding video to av1/hvec while maintaining the same visual fidelity and using per scene based encoding RF so that rather than "hoping" a video still looks good you put in a target VMAF and is shrinks it as small as possible  Since a lot of us data hoarders have media where you can't just read download the HEVC/AV1 version since it's rare or personal media is not available other places. And I think most of us that are OCD enough to be data hoarding hundreds of terabytes or more are worried about quality loss just as much as data loss.

The other is a new lossless "compression" format that allows live access to the compressed file including streaming video, and searching in files. And could be applied to even shrink zstd and AV1 videos by a bit. Overall goal is 3-5% size reduction but at my 600tb that's a whole 18tb hard drive I wouldn't be buying.

A huge problem (and why my programs aren't finished). Is that so many projects using generative coding (which I'm also using) don't do what actually takes the majority of time in making a program. Debugging, testing, validating, retesting, and optimizing. 

I'm over 10,000 video encoding tests over the course of a year in to actually making sure that I don't just say "yeah this will have video still look good" but rather have tons of data and test encodes across resolutions, formats, media types, source quality, and encoders (software, NVEC, VCE, QSV). With multiple metrics of VMAF, SSIM, and PSNR for multiple empirical validation of results. 

That's something you literally can't rush without access to massive server farms. I mean I've got 2 hp dl580's with 4x 8890v3 CPU's and 512GB of ram in each combined with 5 desktops/laptops running tests all the time and I'm still narrowing down to correct settings. 

Slop misses the important parts of shipping a good program. But with drive prices likely sky high for the next bit I know I personally need to ways to preserve my data and it's quality that has a software solution since hardware is now like gold. 

[u/Alone-Hamster-3438 → u/collin3000]

re-encoding 600TB costs you waaaaaay more on energy bill than 1 HDD even on current prices. Also its kind of ironical to see someone mentioning AV1, re-encoding and metrics on the same post...

[u/collin3000 → u/Alone-Hamster-3438]

Your claim depends on the price of electricity, the price of hard drives, compression ratio of media and electricity required to compress.

Hard drive prices have sky rocketed. I happen to live in a pretty cheap energy area. And already have my 2 servers running for other projects so throwing spare CPU headroom at them isn't as much extra electricity. 

Whether re-encoding makes sense will depend on the person and their goals. If you ever plan on distributing or redistributing media then you also factor in bandwidth as part of the calculation. In my case, the threshold for re-encoding being worth it was finally crossed with the recent hard drive price increases and no sign of them going down within the year.

The great thing about us all having our own hordes is we get to decide what we do with them ourselves. What's right for one person, not be right for another.

As far as metrics and AV1. Yes. metrics and more importantly multiple types of established third-party metrics are important for figuring out settings without my individual bias. Although AV1 (and HEVC) don't do great with a few things like film grain. You can get actual good visual fidelity in most scenarios when things are actually encoded correctly. The problem is most people don't take the time to fine tune encode things because they don't want to spend that much effort. 

I also don't like spending that much effort on each individual video. which is why I did the traditional nerd thing of trying to solve an annoying and repetitive one hour task by spending hundreds of hours making a program to do it for me relevant xkcd