Patch management is becoming a more debated topic nowadays. The proliferation of Intune’s simplified patch management across Windows environments has upset some. and made others (like me) quite pleased. Let’s talk…
Patch Management: Circa 2014
Let’s talk about how we used to managed patches with our sweet RAT RMM tools. At least, this will be how I took my first stab at managing patches.
Backstory
Let’s talk about why we felt the need to manage patches. Around this time and history (and really time leading up to it), patches were something we could sell value against. “Sure thing Mr. Customer, our team meticulously validates and tests patches to ensure they won’t cause a problem in your business.” 🙄 As a wise man once said, “cool story bro.” But, that’s exactly what I used to talk about. In hindsight, it was a total waste of time. Maybe I’ll talk about selling on business value in a future article.
Beyond that of course, we did know that some patches would cause issues. That weird HP chipset driver, or that update from <insert large copier manufacturer here>. Sure, some patches can be problematic. You know what else is problematic? Breaches.
The “Patch Management” Process
When it was time to “approve our patches,” we’d go into our trusty RMM tool. By the time I had about 250 endpoints under management, the process became stupidly painful. We’d have a list of like 700 patches that needed “approving.” So I did what any dude who doesn’t want to waste 4 hours of his life did. I Googled “recent Microsoft patch issues” and just blocked and KB numbers of worry before bulk approving the rest. It was wildly ineffective. That’s not to say some shops larger than what I was don’t have people that vet them out, or deploy them to a lab ring or power user ring or whatever. I didn’t. I didn’t have the time to pay someone to tinker with patches.
When I shared this news with others in the community, some were shocked. Now, one thing I did do decently well in my shop was categorize incidents. Every ticket was categorized, and there was one called “Patch Issue.” The “Patch Issue” ticket would be used for anything we had to fix as the result of a bad patch. It accounted for sub 2% of the support load. So I was happy with my method.
Here Comes Trouble
Fast-forward to 2021. The last 2 years of so have been absolutely nuts with critical vulnerabilities. I can tally at least 100 hours of lost sleep to words like “ProxyLogon.” ProxyLogon was interesting in particular. The question I got from the folks I advise for the most was “how do we delay this patch?” WHAT?! We just spent an hour talking about how this vulnerability allows unauthenticated access to a customer’s environment. Unauthenticated access I can use to do some nefarious shit, and you want to delay the patch?! The first thing I did after the third or fourth conversation about this was take to LinkedIn and write “Impacting the Business in the Name of Security.” The TL;DR of that one is, sometimes the threat we’re facing meets a certain threshold. Once we hit that clipping level, we have to throw convenience out the window. Being vulnerable to ProxyLogon and having exposed Exchange environments was pretty much a guaranteed breach situation. In this scenario, you can bet that I will pull your mail offline mid-day and fix it.
So what does patch management even do?
Don’t get me wrong. Properly executed patch management is still important for all scenarios. For on-premises infrastructure, we should validate our patches and do a routine schedule type of operation. However, the patch management policy should allow for emergency patching (even if it causes interruption) in the event of a critical bug.
However, let’s think about Modern Workplace. We have little-to-no server environment, we’re managing things in the cloud. Intune has excellent patch management built in that just “does its thing.” Why not use it? For many of us, when we hit a certain scale, we bulk approve patches anyways. We haven’t “managed” anything. Just let patching do what it does, improve the security of the environment. Are you gonna get an annoying patch from time to time? Yep, you are. But guess what? If you build things right, it doesn’t matter. We used to remote wipe Windows machines all the time when they started having issues. Within an hour, it was back up and running with everything the user needed. Stop wasting your valuable time Googling KB817237812838 and just… patch. Especially if your customers are on current-gen, Modern Work environments.
