Fefe had a short pointer to an article Patching is Hard. It is, but you can make it a lot easier by doing a few things right. I did s small writeup (in German) to explain this, which Fefe posted.
I do have an older talk on this, titled “8 rollouts a day” (more like 30 these days). There are slides and a recording. The Devops talk “Go away or I will replace you with a little shell script” addresses it, too, but from a different angle (slides, recording).
In 2010, between April, 15 and April, 23, Air Travel was disrupted because of the ash cloud generated by the icelandic Eyjafjallajökull eruption.
That meant a lot of travellers could not reach the beds they had booked, while approximately the same number of people could not leave their beds to return home. On the first day, the volume of calls making it into the call center (not counting dropped calls) was nine times the normal volume. We added a lot of personal (three times the normal staff, IIRC) to call centers, upgraded servers to handle the increased churn on the databases, and added licenses to the phone system to cope – all in all an extremely busy time.
Also, many hoteliers realized that it is actually ok for customers to cancel, as long as the beds are warm (and paid for). Travelling became a lot more flexible in the aftermath of this incident.
Are you an infrastructure developer? Do you think like a developer, do you understand developer tools, but think like an infrastructure person? Are you more interested into worst case behavior than new best cases? Do you shoot trouble before it happens?
Where I work we have regular round tables, in which you can talk and ask questions to middle management from other departments than your own. I had the opportunity to talk to a person who manages development priorities and staffs teams, and who of course has some insight into hiring and the interview process. That was very enlightening.
For example, finding people to hire in a large organisation is a hard job. Hiring rates are quite fixed, so in order to find people to hire you need to go through a relatively fixed, larger number of resume reviews, phone screens and face to face interviews. Assume that for each three people you would want to hire you need to sift through 100 resumes – that’s 10.000 resumes to look at for 300 people to hire. And it can not be automated.