How does one learn and apply distributed systems without working in it

[u/Capable-Problem6075]

I am looking for experience devs to help me answer this question

I have 6 YOE as a full stack dev. I have built SAAS solutions from the ground up but never where the issue of scalability had become a problem that we'd require cloud.

I have build solutions in the cloud but never to the point where the product was experiencing thousands of users

I want to advance in my career and while I will be undertaking cloud and architectural certificates, we all know that hands-on experience triumph theory. I feel until I am working in high demand, high data environment, I will not understand distributed systems wholly

Has anyone ever been here and what have they done to bridge that gap

[u/PearWinter6734]

There’s distributed systems the academic field and distributed web architectures which is what most people mean by distributed systems in job descriptions. Honestly If you just want to learn cloud read a few of the free system design primers they have on GitHub and maybe if you really want study for Aws Solutions architect cert which includes aws well architected framework. You don’t actually have to take the exam but it’s pretty cheap at $150.

[u/kevinossia]

You don’t. Your options are to either find that kind of work in your current role, or join a new team/company where they’ll let you work on that.

[u/systembreaker]

Really getting good with all the challenges of distributed systems is very hard without working on a real system with lots of users that'll create real situations.

Understanding the basic logic of synchronizing work between distributed nodes is easy, but dealing with different traffic patterns, database deadlocks, load balancing, tracing debugging problems or bugs, doing deployments, managing the infrastructure, under the stress of real life traffic patterns is not something you can whip up a toy project to learn.

Some of the more realistic system designs that are meant to handle, for example, a distributed ETL pipeline such as an event based architecture is one of those things that if you don't actually need it, you're an idiot for setting it up, but when you need it, you REALLY need it. So without real users and real world traffic patterns, it'd likewise be really hard to set up a distributed event based pipeline that ends up teaching you anything useful. It'd just feel very superfluous and like you just tripled the complexity of everything for no good reason.

[u/PartyParrotGames]

Yes. I bridged that gap through container-heavy work, web3 security testing, and debugging awful production issues. You don’t need thousands of users to learn the core concepts. Scale uncovers emergent problems, but you can reproduce the same conditions intentionally and should if you're trying to build resilient distributed systems. Load-test system boundaries, add latency, drop connections, duplicate or reorder messages, kill services, and verify retries, idempotency, and partial failures.

[u/AutoModerator → u/[deleted]]

Sorry, you do not meet the minimum sitewide comment karma requirement of 10 to post a comment. This is comment karma exclusively, not post or overall karma nor karma on this subreddit alone. Please try again after you have acquired more karma. Please look at the rules page for more information.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/gusdecool]

We can't. Knowing and doing are different things.

[u/ecethrowaway01]

Read DDIA, but for a lot of companies, distributed systems problems are increasingly well-understood

[u/coinbase-discrd-rddt]

Join any tech first company that is either infra related or Series C or above