Hacker News new | past | comments | ask | show | jobs | submit login

Except further up this thread another poster mentions that CrowdStrike took down their Debian servers back in April as well. As soon as you're injecting third party software into your critical path with self-triggered updates you're vulnerable to the quality (or lack of) that software despite platform.

Honestly your comment highlights one of the few defenses... don't sit all on one platform.




Sure, but note the sales pitch was to encourage resiliency through diversity. While that may not be helpful in cases where one vendor may push the same breaking change through to multiple platforms, it also may be helpful. I remember doing some work with a mathematics package under Solaris while in university, while my peers were using the same package under Windows. Both had the same issue, but the behaviour was different. Under Solaris, it was possible to diagnose since the application crashed with useful diagnostic information. Under Windows, it was impossible to diagnose since it took out the operating system and (because of that) it was unable to provide diagnostic information. (It's worth noting that I've seen the opposite happen as well, so this isn't meant to belittle Windows.)


Yes, I already heard one manager at my company today say they're getting a mac for their next computer. That's great, the whole management team shouldn't be on Windows. The engineering team is already pretty diversified between mac, windows, and linux. The next one might take down all 3 but at least we tried to diversify the risk.


Yep, these episodes are the banana monoculture [0] applied to IT. The solution isn't to use this vendor or avoid that vendor, it's to diversify your systems such that you can have partial operability even if one major component is down.

[0] https://en.m.wikipedia.org/wiki/Gros_Michel_banana


> don't sit all on one platform.

Debian has automatic updates but they can be manual as well. That's not the case in Windows.

The best practice for security critical infrastructure in which peoples lives are at stake, is to install some version of BSD stripped down to it's bare minimum. But then the company has to pay for much more expensive admins. Windows admins are much cheaper and plentiful.

Also as a user of Ubuntu and Debian for more than a decade, i have a hunch that this will not happen in India [1].

[1] https://news.itsfoss.com/indian-govt-linux-windows/


Windows updates can definitely be manual. And anyway, this was not a Windows update. It was a CrowdStrike update.


Oh, i thought it was tied to OS updates. So Windows is not to blame, if that's the case.


well, in another sense, Windows is certainly to blame partially. Several technical solutions have been put forward here and in other places, that would've at least limited the blast radius of a faulty update/driver/critical path. Windows didn't implement any of those. Presumably by choice and for good reasons: A tradeoff would be that software like crowdstrike is more limited in protecting you. So the Windows devs deliberately opted for this risk.

Or they never considered it, which is far worse.


Hopefully they won't botch the update for two operating systems at the same time. But yeah. Hope.


Yeah, I see a lot of noise on social media blaming this on Microsoft/Windows... but AFAIK if you install a bad kernel driver into any major OS the result would be the same.

The specific of this CrowdStrike kernel driver (which AFAIK is intended to intercept and log/deny syscalls depending on threat assessment?) means that this is badnewsbears no matter which platform you're on.

Like sure, if an OS is vulnerable to kernel panics from code in userland, that's on the OS vendor, but this level of danger is intrinsic to kernel drivers!


> AFAIK if you install a bad kernel driver into any major OS the result would be the same

Updates should not be destructive. Linux doesn't typically overwrite previous kernels, and bootloaders let users choose a kernel during startup.

Furthermore, an immutable OS makes rollback trivial for the entire system, not just the kernel (reboot, select previous configuration).

I hope organizations learn from this, and we move to that model for all major OSes.

Immutability is great, as we know from functional programming. Nix and Guix are pushing these ideas forward, and other OSes should borrow them.


It's interesting to me that lay people are asking the right questions, but many in the industry, such as the parent here, seem to just accept the status quo. If you want to be part of the solution, you have to admit there is a problem.


True; except here's what's baffling:

CloudStrike only uses a kernel level driver on Windows. It's not necessary for Mac, it's not necessary for Linux.

Why did they feel that they needed kernel level interventions on Windows devices specifically? Windows may have some blame there.


Apple deprecated kernel extensions with 10.15 in order to improve reliability and eventually added a requirement that end users must disable SIP in order to install kexts. Security vendors moved to leverage the endpoint security framework and related APIs.

On Linux, ebpf provides an alternative, and I assume, plenty of advantages over trying to maintain kernel level extensions.

I haven’t researched, but my guess is that Microsoft hasn’t produced a suitable alternative for Windows security vendors.


> Why did they feel that they needed kernel level interventions on Windows devices specifically?

Maybe because everyone else in "security" and DRM does it, so they figured this is how it's done and they should do it too?

My prior on competence of "cybersecurity" companies is very, very low.


> My prior on competence of "cybersecurity" companies is very, very low.

Dmitri Alperovitch agrees with you.[0] He went on record a few months back in a podcast, and said that some of the most atrocious code he has ever seen was in security products.

I am certain he was implicitly referring, at least in part, to some of the code seen inside his past company's own code base.

0: https://nationalsecurity.gmu.edu/dmitri-alperovitch/ ["Co-founder and former CTO of Crowdstrike"]


> Maybe because everyone else in "security" and DRM does it, so they figured this is how it's done and they should do it too?

What DRM uses kernel drivers? And how do you plan to prevent malware from usermode?


> CloudStrike ONLY uses a kernel level driver on Windows

Crowdstrike uses a kernel level driver ONLY on Windows.


CrowdStrike uses a kernel level driver on Windows ONLY.

Even better..

ONLY on Windows does CrowdStrike use a kernel level driver.


Yeah, I think your point is totally valid. Why does CrowdStrike need syscall access on Windows when it doesn't need it elsewhere?

I do think there's an argument to be made that CrowdStrike is more invasive on Windows because Windows is intrinsically less secure. If this is true then yeah, MSFT has blame to share here.


I don't know about MacOS, but at least as recently as a couple years ago crowdstrike did ship a Linux kernel module. People were always complaining about the fact that it advertised the licensing as GPL and refused to distribute source.

I imagine they've simply moved to eBPF if they're not shipping the kernel module anymore.


I haven't looked too deeply into how EDRs are implemented on Linux and macOS, but I'd wager that CrowdStrike goes the way of its own bit of code in kernel space to overcome shortcomings in how ETW telemetry works. It was never meant for security applications; ETW's purpose was to aid in software diagnostics.

In particular, while it looks like macOS's Endpoint Security API[0] and Linux 4.x's inclusion of eBPF are both reasonably robust (if the literature I'm skimming is to be believed), ETW is still pretty susceptible to blinding attacks.

(But what about PatchGuard? Well, as it turns out, that doesn't seem to keep someone from loading their own driver and monkey patching whatever WMI_LOGGER_CONTEXT structures they can find in order to call ControlTraceW() with ControlCode = EVENT_TRACE_CONTROL_STOP against them.)

0: https://developer.apple.com/documentation/endpointsecurity


Non hardware "drivers" which cause a BSOD should be disabled automatically on next boot.

Windows offers it's users nothing here.


You can also make rollback easy. Just load the config before the one where you took the bad update.

Of course that means putting the user in control of when they apply updates, but maybe that would be a good thing anyway.




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: