What advantages FreeBSD offers over Linux so that for the next installation on EC2 one would consider running it instead of Linux? (question is not rhetorical)
I don't think there's anything specific to EC2; so your question just reduces to "why FreeBSD rather than Linux?"
There's a lot of differences which are more matters of taste than anything else: FreeBSD's ports system, the rc.d startup system, the fact that the kernel and userland are developed in tandem rather than independently, the presence of a central SVN repository, et cetera. I like all of these, but I know there are Linux people who prefer the "Linux approach" for each of these.
Some things which I've heard even hardened Linuxites saying make them tempted to try FreeBSD, however: ZFS, DTrace, LLVM/clang (now building the entire FreeBSD base system), virtualized networking, Capsicum. Also new in 9.0, but not getting much attention yet, is journaled soft-updates -- essentially the "best of both worlds", using soft-updates for the 99% of operations which soft-updates can handle quickly and safely, and journaling the remaining 1% to ensure that a dirty fsck is fast.
I'd add that "jails" are a very cool, under appreciated feature of FreeBSD. It is especially useful for packaging up and delivering applications as a single unit. While it may seem that they are in competition to virtual machines, I think of them as being a complement.
Thank you for your work and persistence Colin! I love using FreeBSD for my servers because of the consistency and great minds behind it. I love it so much that I have written a few posts how to setup a FreeBSD server [1].
Currently there are some other options for hosting your "virtual" FreeBSD server. Some good ones are RootBSD [2] (US), Brightbox [3] (UK) and TransIP [4] (NL). Quality wise that are equally good. I'm currently using TransIP because I'm located in the Netherlands.
Interesting to note is that the above providers make use of KVM virtualization technology. Phoronix did an interesting benchmark comparing Xen, Virtualbox and KVM. It showed that KVM was the winner on compute and disk performance [5]. Would be interesting to find out how FreeBSD performance on Amazon's XEN layer.
Phoronix is infamous for its broken benchmarks. I'm sure the disk performance numbers would look completely different if they used paravirtualized disks (like EC2 does) rather than emulated disks. The compute numbers are just plain weird; on something as cache-resident as ripper it should be impossible for the virtualization system to cause any significant real effect, so I'm sure there's a problem with their benchmark or their test setup somewhere.
I've always been curious about running a pure BSD on the open net, but its very hard to find cheap VPSes which will do that (and meet my other needs). Hopefully this will pave the way for that.
I've used them before. You could only connect to the VNC console once, and then you were locked out (which is a joke), and they don't do automatic provisioning. Pass.
cperciva: I know that some time back you said you'd signed an NDA with Amazon which prevented you from discussing what made this possible. Has that passed now? Was defenestration of Windows AMIs the secret?
Thanks. I had figured there was an interesting story behind the reason for which FreeBSD only ran on t1.micro instances initially, and your mention of an NDA only further piqued my interest.
I guess i don't use FreeBSD much, but my primary use of the OS is for pfsense which runs both my home router and also the routers I use for VMs running on dedicated VMWare servers.
I do know that FreeBSD's handling of network activity appears to be second to none, but given Amazon's load balancing system, I'm curious what people are actively using FreeBSD for outside this space? Particularly in regards to EC2.
I don't know what you are biting or why, but your question makes no sense. I didn't say anything was wrong with FreeBSD, I simply pointed out that networking is one of OpenBSD's primary focuses, and as such it offers much more. OpenBSD's firewalling and routing support is miles ahead of FreeBSDs. FreeBSD isn't the best at everything, just as no other OS is.
OpenBSD wrote the best BGP daemon around, the best firewall around, both of which integrate nicely together and with routing domains to allow perfectly fine-grained control of complex routing scenarios, created CARP for address redundancy ala VRRP but sans patent mines, had the very first IPSec implementation, etc, etc, etc. OpenBSD is used almost exclusively for routers, firewalls, VPN gateways, etc. You shouldn't take it as a personal insult that it excels in those roles.
I agree that OpenBSD has written some nice code, but it's hardly "miles ahead" of FreeBSD considering that FreeBSD includes pf and carp in the base system and openbgpd in the ports tree. The great thing about the BSD license is that when one project does some great work, everybody else catches up quickly. :-)
And I don't take it as a "personal insult" -- I'm just amused by your comment given that FreeBSD's network stack has vastly superior performance and support for far more 10GbE interfaces.
Which indicates that your mentality is "anyone who says anything is better than freebsd for any purpose must be trolling". That sort of rabid fanboyism doesn't make for productive conversation.
FreeBSD includes a very old, partial port of PF. Even CARP which is very simple is missing functionality in FreeBSD. And as I already mentioned, bgpd is fully integrated with PF and routing domains in openbsd, it isn't in free (which doesn't have routing domains).
>I'm just amused by your comment given that FreeBSD's network stack has vastly superior performance and support for far more 10GbE interfaces.
Did you have some data to support that claim? I've seen no such benchmarks, and the only 10gbe driver freebsd has that net/open haven't pulled in that you can actually purchase hardware for is broadcom, which is broken and can't even do 1gb in reality. All 4 BSDs pull in hardware drivers from each other all the time. The great thing about the BSD license is that when one project does some great work, everybody else catches up quickly. :-)
I am a BSD fan - a big one in fact as I have been using BSD since the late 80's. I use AWS for some things where the pricing/instance sizes are a good match for the app needs.
I don't keep up with the latest greatest Xen infrastructure in use by Amazon but...
Wouldn't it be easier/cheaper to get NetBSD running as it has PV support for Xen3? vs using a windows HVM?
I've talked to NetBSD people about EC2 a bit, but I don't know their current status. IIRC at one point they were limited by a lack of SMP support in their Xen-PV code, but I don't know if that's still the case.
I have a couple of boxes at home right now running NetBSD as dom0 w/xen3 and linux/freebsd/netbsd running as PV guests if I get a chance I will move them to some more modern HW and see what happens with NetBSD and multiple cores... Think one of them is a dual core right now but gotta check...
In any case NetBSD PV Kern on a small AWS might be a great combo for cheap BSD thrills....
