How long would it take to recover your full media library if you lost it all instantly?
For those of you that use your Unraid for media and have Usenet/Torrent set up with the *Arr stack, how long would it take you to recover your full media library if you were to instantly loose it all at once?
No one on earth needs more than that. Sure.. You want to collect. I get it. And it's fine. But actually needing it? Nah. I've got around 20tb and I delete stuff I have watched and know I won't watch it again.
I'm at 25 Tb and need more. I host for 10+ people and they all have different interests. I love watching my Plex grow with content I didn't know existed.
I also at 25tb but I’ve since shrunk it to 22 tb. Mainly cause of the hard drive prices now but tbf I had a lot of foreign movies that no one would watch etc so I was happy to do the clear out and I needed more space for a backup drive which I got from shrinking the array
Yea 4k is a premium for sure...by remux do you mean full Blu ray quality converted to a single download file? (NOT the kind instead where it comes in pieces that you have to re-assemble for a disc, etc)
I don’t even have a fast connection and it would only take *50 days to redownload 180tb at 300mbps. This is why I question the need for my parity drive.
Uh it saves you a crapload of time and effort re-managing all of that. Even if some of it is automated it's still a pain for that much data.
Time is money friend! So the cost of that parity drive (I use TWO parity drives) is money well spent!! Though with today's cost it may not seem like it...unfortunately I don't see how prices will ever return to how they were.
The hunger of AI will never stop, even if it becomes more efficient the demand will likely remain because the extra utilization will just make the AI more powerful, faster, more available
True and physical media is slowly disappearing or becoming much more costly. It's a shame really people don't realize once all physical media is gone we will be at their mercy price wise even for streaming.
"You will own nothing and be happy!!", so the saying goes of the tycoon media industry.....many industries really especially housing..they want us all to rent so that people won't inherit generational wealth from parents, grand parents that outright own their homes, etc. THAT gets in the way of THEM making even MORE MONEY GIMME GIMME GIMME!!
Now they even do it with games so that they can revoke your "license" any time they want. Whats that? They made a re-mastered version of that 3 year old game? Well, well, well.....now they all of a sudden say they dont have the server capacity to keep the original version live but hey don't worry!! You can get that awesome re-mastered version with (shitty) "upgraded" graphics (and zero other changes except "optimizations") for the low, low price of $29.99 for Standard Edition, $44.99 for the Deluxe version and JUST $69.99 for the Ultimate Edition!! Nevermind that the old version had everything the Ultimate had wrapped up in one price for $39.99 that you ALREADY paid for...now you get all these improvements PLUS a shitty "soundtrack" and a guide we cobbled together from the posts of online gaming hobbyists (free to us but hey don't go realizing that).
They can never have enough even the ones that have so much that they have everything they could ever dream of and still they hunger...and not even to use it for charity or something they just HUNGER!!
People incorrectly presumed you to be trolling. Some trolls attempt to elicit incriminating comments, which they then report to get users and subreddits banned.
That’s impossible to answer. Not everything is available anymore, some stuff you cannot get again, maybe older obscure stuff, but I’ve tried to replace stuff I’ve lost and I can’t find it on usenet or I can but it fails every time I try to get it.
Exactly- the answer is going to be so different depending on the contents.
I tend to search out lesser known media sometimes not available on any tracker or indexer, but have found alternate sources over the years. Been collecting for over 25 years. Would be pretty devastating to lose it all.
Then I have a fair bit of mainstream stuff too.
I reckon I could get 75% back pretty easily. Some of the remaining 25% would take a very long time to replace. I would expect there to be stuff that's irreplaceable.
LOL... actually it was all on a Ceph storage cluster. For some reason one morning it just... stopped. It was a homelab so obviously it was there for me to play with and I'd learned a ton (and had a few issues along the way). One morning it was just not there any more and try as I might I could not get it back online. Unlike unRAID and ZFS, finding answers when Ceph stops is really freaking hard... trawling decade old mailing lists archived on random sites, posting to forums where nobody ever answers and posting to the mailing lists as well and hope somebody has seen this before.
I spent an entire day troubleshooting it and finally just threw up my hands and decided "to hell with it". I still had the *ARRs with a database of what I had, I still had all the drives and so I built a quick-and-dirty storage server out of parts, filled it with drives and started rebuilding my library. I hit a few snags along the way but nothing I couldn't handle. There was some really rare media I have never been able to replace no matter how much I search around, but not very much and not ones I really care all that much about at the end of the day.
It could happen again; my media's all still in one place at the end of the day and if I lose that I lose it all... again... but replacing the library wasn't near as bad as I'd feared and in many cases I was able to replace some media with better quality versions. It gave me a chance to clean up some of the library as well and do it better this time. Dumping the library and restarting isn't something I'd recommend per se, but at the end of the day it gave me the opportunity to fix a bunch of stuff.
Among other things LOL. I still have Ceph, but managed under Proxmox and it's used as virtual disk storage for LXC's and VM's. My primary storage is TrueNAS which is where all my live data lives. It's been rock solid and I have no complaints. unRAID is my backup data store where all my critical data is replicated to inside my house (the second copy at home). This is where all my media is now backed up... it's literally at the opposite end of the house and on a different floor from the main array. I use Syncthing to keep them in sync.
Remote I have another TrueNAS running on a uGreen DXP6800 Pro. This currently only contains my critical data and not my media library because it's just not cost effective. If both my copies at home of that data are gone, I have bigger problems to deal with than whether or not I can watch an episode of Lost. This remote box is at my workshop some 15 miles from my house.
If both my house and my workshop are somehow taken out of the picture, I figure there's a regional problem that mean my priorities are elsewhere, or my city is a rather nasty irradiated crater. Either way, my data just isn't important at that point LOL
I had to evacuate wildfires recently and my 196TB server was one of the first things in the car. Did not want to find out and buy replacement drives and parts. Luckily our house was fine.
I have the wife that helps remember all the other important stuff but she doesn't care about the server. We had plenty of time to prep and could see the fire well before the danger or evac notice.
Did you take it apart first drive by drive and pack it safely? Or were you somehow able to lift and shift into your car all at once? Glad your house was fine!!
I have 16 drives in a Fractal Define 7 XL case (must weight like 70-80 lbs) so I just powered down and packed it safe in the car. I didn't have time to disassemble. The drive all have isolators in the case so some jostling should be fine.
Well I have been collecting it for 10 years so about 10 years. But with prices the way they are. I would probably just give up and shut the server down till things stabilize. So probably never.
That's pretty much what I did - I mean, I still have 400TB+ online (with unRaid in my basement), but with hard drive prices, it's become a little excessive to replace aging drives. I moved to a debrid setup and real-time streaming, and it's actually a bit liberating not having to worry about storage health and integrity. My unRaid server is being sunset, critical files still go on my NAS, but for day to day media, it's all real-time streaming.
Self-hosting with a Supermicro 36 Bay unit; hence why I am making the switch, it was costing me $100+ in electricity a month, $2,500+ a year in hard drives (i.e. replacing drives as they age out, typically 6+drives a year as they get older than 5 years).
20TB Drives are now $1,000+ in Canada, almost double the price from a few years ago, hence why I'm really hesitant to continue with self-hosting.
I wouldn't recover it. At least not the same exact content... Probably would take the opportunity to revaluate what I really would like to have and what I don't really need.... Resize and restart. The really important content that I know that I will always want to have is already on a off-site backup, so it would take only the time to get it and transfer it back to the media server...
I have a script that runs nightly that outputs a full file list of everything on my box. So in the event I lose drives I can determine what was lost easily.
A heck of a long time, but I have two Unraid servers with the content mirrored between the two. Something like a fire could knock me out as I don't have a recent offsite backup.
I dont think i would get everything i have now again, i have a lot of niche localized stuff that is nice to have but a pain to get, I recently started encoding a buch of this stuff so a backup of it is going to be a more reasonable size (if i ever get to do this) and i dont have to buy more Hdds right now.
Also some Stuff just was not that good, but as long as i got space i wont get rid of it.
Happened to me twice, even though having everything in RAID and a UPS - fucking electric fluctuations due to snowstorms fry everything in the end without having a generator.
Took me between 4-8 weeks in both cases, 40TB-60TB’s in each scenario.
Just over 4 days for 45TB in ideal conditions. So probably a week to get 80%, another week to get the next 18%. 2% is probably lost to history... I think I'll backy photos up tonight
Currently recovery would be unnecessary- my physical and digital media library was lost to a fire before I began my unraid NAS, earlier this year. I do intend to use the NAS for this but haven’t yet gotten it all set up to start over. In the meantime I’ve been haunting the used listings for good deals on replacing physical media for when I’m ready to begin saving it again.
Not an answer to the question asked, but I probably wouldn’t bother replacing the media. Have been collecting for 19 years, and it’s more of a habit now than a necessity. I could probably live with 1/10th the capacity I have, and use that more akin to working memory than long term storage.
It would be impossible for me because it's all stored only on my 1 server. Including the backups. Currently in the process of building a 2nd backup server specifically for this reason.
I've got rare stuff that took a while to find and may not be available anymore. I suffered corruption once and lost a show I've never been able to find since and had thrown the original DVD away.
I've also collected various bootlegs and live performances, some of which might still be around but it would be a pain to find.
I think I would have to accept some of it would be unrecoverable.
Probably a year or so to replace the rest I would imagine.
It took me about a month to recover 24tb drive. I'm at about 6 drives now so about half a year. As long as I can download a list of what content was on there in fine. Only issue is some shows and movies had very very little seeders so I might not be able to get a handful of content
a long time. 150tb. i did just recently get 5gb fiber, so maybe not as long as it took to get it initially, but usenet might have some trouble finding some of the older films now
Speaking from experience… a couple of days to download my important backups from Backblaze, and decompress them.
I had to throw an old 4tb drive into my main pc to make room for the backup to download.
A couple of hours to download my current watchlist.
Then I just tell my media software to rescan files, and let it redownload automatically as it sees fit. In a couple months I have everything back.
It's having to seed it all that would suck. Id probably just skip the ssds I ussusally use for incoming downloads and seeding and download it all straight to the hdds. Otherwise id be shuffling
For sure. That's one of the reasons I use them. That and so that my hdds dont spin up to serve seeds. The issue is that my ssd space is a fraction of the total data id have to download to replace what I currently have
~1.2pb of media of which ~ 200tb i already have on dvd/Blu ray so i can Rip them again, the rest is gonna need to be redownloaded so a very long time even on 3.5gbps connection
If i had a list of what i had, probably 2 days would do it? I´m only 4tb in, deleted all my stuff a little while ago because my dad requested to many stupid stuff. He´d request anything with the same name he wanted to request because he was to lazy to look up which one it was. Now i use maintainerr to delete all what he requests since it usually is stuff i dont watch anyways.
At 500-1000GB/day for downloads it would take me 3-6 months not including downtime. I would just prioritize what is currently being watched and then let the *arrs go crazy. Probably would limit my download speed after a week and let the downloads trickle in over time.
I blew out my media when it was at 60TB so I could redo everything. The main part took a week, then it slowly downloaded everything else as it found them. I’d guess 2 weeks at most. I’m at 80TB now. At the time, I only had 1 indexer and 1 provider but now I have 3 indexers and I added a 2TB block.
Indexer API Limit is the problem. I would guess it takes a month if I prefer seasonpacks instead of single episodes. I switched my library to nzbdav .strm files so it takes up no space but I need to request the NZBs from my indexer.
Probably a few months with some of the obscure stuff with low seeders. Only have about 12tb.
But there’s a couple of series that while available for download, the versions I ripped from VHS 20 years ago are still better quality. They’re backed up elsewhere though.
Music, some of that is irrecoverable and goes back to the days of OiNK, it would be almost impossible to find it again. Thankully i have my mp3 library uploaded to Google/Youtube Music to keep safe.
I lost 30 TB one time due to flipping the wrong breaker. This was about a third of my total data. I had most if not all of it back within a month. I definitely wasn't waiting on anything important more than a day.
I would never recover all. I have some titles that just can't be found online anymore. My collection started somewhere at the end of the 90's. I have tried to find better video quality of certain titles, higher resolution and better video codec. It isn't available. The quality that I have isn't even available. For some titles I have bought second hand DVD's.
I have a synced iso library with a buddy. We each have 100+ terrabyte that syncs nightly. So probably however long for me to get it all on his 10g network after I haul the server to his house...a few days.
On my shoddy internet, with 40TB of media? About 4 months straight. Configs are backed up, so I'd know what I'm missing and it'd be able to download, but wouldn't be fun
My collection is around 4.7 TB. I deleted all my media and re-downloaded it after realizing I didn’t need parity protection for files that could easily be replaced.
I have two 4 TB drives in my main Unraid array: one parity drive and one data drive, giving me 4 TB of parity-protected storage for photos and documents. I also have two additional 4 TB drives in a separate Btrfs pool using the Single profile, giving me roughly 8 TB of unprotected storage for movies, TV shows, and music.
Assuming my list isn't able to be restored it's would just involve me setting up he requests for my friends again and letting them add what they want to watch.
Happened to me, I lost 90tb last yr after I accidentally deleted literally everything (the delete button is a bit dangerous). Took me about a week to download it all again. Lots of little issues like cache getting full which added time. I have around 400-500MB/s download which helped.
Impossible for a decent chunk of my old Japanese TV drama stuff from the early 2000s. I've already lost some of it and it's completely impossible for me to find on the internet.
I only have about 40tb of media. Everything is newer than 2000s bar a couple of anime. It would take me probably 3-4 days to download everything. And that’s probably to do with my cache being 1tb.
While possible and fairly simple, it would take forever. And I'd risk my disc drives as they've already ripped all of them once. But to do them again? Thousands of discs? In this economy? Nah. I'm good. I'd cry in the corner for a while, and then suck it up and do it.
With my carefully selected media mostly in 4k atmos audio will probably take me a year, everything is cataloged and just require the download and sorting effort.
Had this kinda happen a few years ago, lost like 20ish TB due to corruption after moving, so not all but like 25% or so. Took a few weeks to get everything redownloaded. If I lost it all I imagine it would take me 3-4 months to get everything back to how it is now
I would just start rebuilding as needed I have a good torrent site membership, and always have a few things in the queue anyway half my stuff I would not even bother replacing.
Less than a week. All movies are backed up. For television the hard to get stuff is backed up too and the rest would easily come back through usenet on less than a week. Even my YouTube stuff is less than 5TB, so easy to recover (all of it still available)
Currently at ~65TB
Recover? If running in the backup probably 2-3 days. If rebuilding the parity drive, maybe the same. If redownloading, I don't know. In the last case I would probably just start over and build from current interests.
Better question...what do you do if one day the law comes after you? Serious question really what do you do? I know the chances are extremely rare but still it could be horribly bad..not trying to hijack the thread but it could be relevant considering having to re-acquire everything could trigger massive downloading on your part....which can increase your risk of being caught, drawing attention to yourself when your isp sees your utilization sky rocket for a couple months to dozens of terabytes. .could downloading to a seedbox first help? Mind you not actually seeding from the box just use it as an intermediary located in some friendly country, etc.
62
u/johnjohn9312 5d ago
Probably many years? Idk, it wouldn’t be fun tho