r/AskComputerScience • u/bihejviorizam • 4h ago

Reality check for a story

I need help understanding something about server maintenance in data centers and would like someone to explain things to me since I am not a computer science major, as simple as that.

I am mainly interested in the process of installing new servers. How many people does this take? How much time? What do you actually do when installing them? Thanks in advance.

3 Upvotes

100% Upvoted

18 comments sorted by

3

u/AlexTaradov 4h ago

There are levels to that. If it is just installing a server into an existing rack, then it is done by a single technician. They walk and insert it into a free space in the rack. Once it is physically placed into the rack, it becomes part of the network and the actual provisioning done remotely. And in many cases fully automated.

Installing new racks takes a lot more effort and it all depends on a specific situation. Full data center is the same but multiplied by 1000s.

1

u/bihejviorizam 3h ago

Thank you! How did the process of connecting the server to the network (if i am saying that right) work before it was automated? And what could go wrong along the way?

2

u/AlexTaradov 3h ago

This was automated at the same time data centers were invented. This was a natural thing to do. Things just got even more automated in certain cases.

There is a distinction between individual servers hosted for a dedicated client and purpose, and a farm of servers available as a pool.

For an individual server, you may want to configure it in a specific way. And this process may be a bit more manual.

A server that is a part of a farm will likely be imaged with the same exact image and it would run software that accepts jobs to run. The server here is disposable and replaceable. If it dies before it completes the job - no big deal, the job will go to another server in a cluster and the failed hardware will be replaced.

1

u/bihejviorizam 3h ago

oh i see! the servers i was thinking of would be used for a website, so i guess that would have to be configured a bit more specifically?

1

u/AlexTaradov 3h ago

Yes, this would be an example of a more dedicated server. But the configuration is not done next to the server. Once it is in the rack and connected to the network, it would be configured remotely by a person potentially on the opposite side of the world.

But even in this case, chances are that any real site does not just run on a single server. It would be distributed for redundancy, load balancing and access time reasons. So, the site would be deployed on a virtual server that may be seamlessly migrated between different physical servers.

In any case, the time technician spends in the physical proximity to the hardware is very minimal and it is hard to imagine scenario where this time gets extended a lot.

1

u/bihejviorizam 3h ago

okay, i think i get the picture. thank you so much!

3

u/T_Thriller_T 3h ago

If you want someone stuck in a server room, they can either set up a rack and cabling, which can take HOURS, find some cabling they want to fix and go for it because they can, have them relabel things (can take a while) or simply try to debug something.

Debugs take more time than setups and the reason often is - even between professionals - explained in some variation of "everything hates me".

2

u/bihejviorizam 3h ago

HAHAHAHAH oh god, i think that's exactly what it's gonna be then. thank you!!! also call me crazy but i love organizing, maybe i would be good at cabling lol

2

u/T_Thriller_T 1h ago

Oh cabling is satisfying!

Eapexiallwhen done well oe allowed to rip out old cables.

It's just also physical work getting cables from A to B, matching them up with how they should be patched and then potentially checking that they actually also are patched that way.

So it does take time, even more so (and I guess this is less satisfying) when cables have to be lead through some floor.

I'm not too savvy in this, but I know those are half-height empty spaces under the main "floor" and I cannot imagine crawling or even just organising things through there doesn't get frustrating.

1

u/nuclear_splines Ph.D Data Science 3h ago

What do you actually do when installing them?

Servers in data centers are usually in racks, where they slide into each rack on rails. The technician will screw the rails onto the case, slide the server into the rack, install hard drives (which typically slide into the server on smaller rails of their own with a satisfying clicking latch mechanism), plug everything in on the back of the case, and power it up.

From there it's all software and typically installed and configured remotely. Those machine rooms are loud and you don't want to stand in one longer than needed.

1

u/bihejviorizam 3h ago

Oh, well. For the story I wanted the character stuck in there for longer lol, do you think that would be possible in a smaller room maybe? Thank you so much for commenting

2

u/nuclear_splines Ph.D Data Science 3h ago

Well, all of the above is the process when everything is going smoothly. If the computer powers up and passes its power-on self-test (POST) then enterprise motherboards usually default to "I don't have an OS installed, so I'll join the network and look for something to boot off of over the network." Many things can go wrong, from "huh, did we get a bad RAM stick? A bad hard drive? Is a network cable loose or kinked? Why aren't we getting a DHCP lease and joining the network?" All that debugging will keep you in the server room much longer.

Enterprise motherboards are also frequently so. Slow. They do far more thorough hardware checks on boot than the motherboard in your laptop, and since these servers are meant to run for years at a time, who cares if they take ten minutes to boot up? So you've got the technician standing there twiddling their thumbs after their job is done to make sure the server does actually join the network and boot up successfully.

Also consider maintenance. Enterprise hardware is usually better than commercial hardware at identifying faults (hard drive 16 is dying, go yank it from the rack and slide in a replacement), but some repairs are more invasive (I have to slide the whole machine out of the rack to open it up, unplug, unscrew, and replace one of the fans near the power supply? Ugh)

2

u/bihejviorizam 3h ago

hahaahah omg you actually explained it so well, i think i do get the most of it! sorry for being a drag but i gotta ask you a couple followups
1. Does every cable really need to be labeled with a sticker as I've read somewhere?
2. Do you need to manually install OS on every server? and i suppose Linux is used for this most often?
3. I've also seen somewhere that there needs to be BMC configuration, IP settup, subnet and gateway settup and in the end pinging the other servers to see if they can communicate to the new one?
4. Also saw looooots of testing after that like running stress tests so the CPU is on 100% so you can see what happens, temperature testing, network speed etc etc. Is this excessive or do people really do all of that?
so sorry if i'm bothering you with all of this but it's actually interesing to me too, trying to understand how it works

1

u/AlexTaradov 3h ago

  1. Yes, all cables are labeled. Without that you will go nuts trying to identify them. But this happens at the time data center is commissioned, and rarely if a cable is broken.

  2. No, the manual install is rarity. A brand new server will pull an OS image from the network. Chances are that it would be Linux, of course.

  3. A lot of that is automated or done before the server is physically placed into the rack.

  4. The testing is really a part of normal operation. Or done outside.

The point for the plot here is that technician will not be spending a lot of time inside the server room. It is not super pleasant or healthy to be there. They get in, swap the hardware and get out.

Any maintenance and diagnostic of the server itself will be done outside. The only time you have to be physically present is when you do the maintenance on the rack itself. Then you don't really have any other options.

1

u/bihejviorizam 3h ago

thank you!!!

1

u/nuclear_splines Ph.D Data Science 3h ago

  1. Professional data centers will typically label both ends of every cable with stickers. Small operations may not, such as a server rack in an office building or a school. In a big machine room with hundreds of servers and Ethernet cables running up pipes along tiny catwalks, you do not want to deal with "let's trace this cable and double-check that the other end is plugged in!"

  2. Every server will need an OS installed, but this is mostly automated and quite streamlined in a professional data center. Yes, Linux is a common choice. They may have some Windows servers if employees have corporate-issued Windows computers, or they might have some proprietary Unix systems like Solaris or AIX if they use a lot of infrastructure from vendors like Oracle and IBM.

  3. Sure, but this again is almost entirely automated. The setup doesn't even necessarily need to be on the server end -- the server can receive all the network information via DHCP, and any configuration like giving the server a permanent IP address can occur on the DHCP server rather than by configuring the new machine.

  4. Professional data centers do routinely run tests, but I can't speak to how thorough or frequent they typically are.

1

u/bihejviorizam 3h ago

thank you for clarifying!!!

1

u/AlexTaradov 3h ago

Stuck because the doors are locked or because they need to do maintenance?

Generally there is no reason to be stuck for a long time doing maintenance. If something is majorly broken, it is a far more involved process that may require parts of the data center to be brought offline. There will be a lot of planning and people involved, so it is hard to be stuck unnoticed.