Scores are as follows, all latest stable releases: (edit: incorporating other responses)
OSX stable builds:
Chrome: 288 +13
Opera: 258 +7
FireFox: 255 +9
Safari: 228 +7
OSX dev builds:
Windows 7 stable builds:
Opera: 258 +7
FifeFox: 240 +9
IE9: 130 +5
Windows 7 dev builds:
IE10 PP: 130 +5
Linux stable builds:
Chrome: 273 +13
FF 3.6: 155 +4
Linux dev builds:
I'm surprised that firefox scored as (relatively) poorly as it did, although I suspect lack of support for closed formats had something to do with that. Similarly, I'm a bit surprised at Safari's poor showing given Apple's insistence that "HTML5 is the Future".
I'd be interested to see similar tests on Windows/Linux, with ie9 in there. Also, it'd be interesting to see how the beta/dev builds rate.
edit: Can anyone familiar with Chrome development shed some light on how it is "ahead" of the WebKit nightly? Regressions? Is there rendering code in Chrome that doesn't get pushed back to WebKit?
For me, the most surprising results in that table were the ones for mobile browsers. Blackberry and Opera Mobile scored higher than all Android and Apple devices listed.
If you're a web developer, the site http://www.caniuse.com/ breaks down exactly which browser support which "HTML5" technologies.
You can also compare 2 browsers (e.g. IE10 and Firefox 3.6: http://www.caniuse.com/#compare=y&b1=ie+10&b2=firefo...) and it shows support by current browser market share (e.g. 53% of users have browsers with CSS3 border-radius).
The tests are not thorough (not everything is testable), and I'm worried that it encourages browser developers to do shoddy job.
Case in point: for a while form validation in WebKit (in "stable" Chrome and Safari!) was only implemented on DOM side and had absolutely no UI, which caused invalid/incomplete forms to "mysteriously" fail to submit, leaving users confused and preventing server from offering fallback.
In the interest of completeness, I've tested the following browsers:
Internet Explorer 5: no score
Mozilla Phoenix 0.1: 16
Mozilla Firefox 1.0: 12
Mozilla Firefox 2.0: 37
Netscape Communicator 4.5: crashes
Netscape Navigator 6.2: 12
Netscape Navigator 7.0: 16
I know this will inspire some backlash but I am sick and tired of all these different browsers and what they can and cannot support. I would actually pay good money every year for a browser if they would all just do the same job.
As a developer all this crap gets boring, day after day, endless checking broswers. And to top that off, Microsoft are thinking of releasing another. IE6 support ends in 2014, are Microsoft trying their best to have 10 active versions before then??
I think, just leave one of the companies to it and let them get on with it, I know competition inspires innovation but its getting stupid now. I think leave either Mozilla, Google or Opera to the job. Operating system supplier, get on with what you are doing and make operating systems!
Microsoft, if you are reading this, please give it up. I use your products every day but I have not actively used Internet Explorer for 6 years, I only use it when I want my blood to boil when I test a website in IE6, IE7, IE8 and IE9. Only to see that it works in every single one but IE7????!!?!?!?!?!
Tests like this shouldn't need to exist. It should just be a case of, "Oh there is a new standard out, {Browser Manufacturer} supports this already and we are good to go".
I wish I could say, "Rant over". But it's not, i'm going grey at 27 and it's all because of browser testing. Yes Microsoft, I'm looking at you.
Honestly, if you can't handle developing for different systems with minor different bugs that you have to work around, I think development is not the thing for you. It's like people expect things to be perfect and they whine and moan if they have to do a little extra work because it's not. Get out in the real world of non-computer engineering and product development and you have to deal with all sorts of non-ideal situations, and people just deal with it.
So you are happy to accept that there is a standard that people do not adhere to?
It's not like I have to do a "little extra work", it's a lot of extra work. We can't do transparent PNG's because IE6 doesn't support them. We can't use PNG fixes as it overlays buttons that are absolutely positioned as it gives an overlay on them.
I'm not asking for things to be perfect, I'm asking for a standard and for people to stick to it. Microsoft are useless when it comes to browsers. Yeah they more or less invented ajax with OWA but beyond that they are just a pain.
I think you have given it to the situation, it doesn't have to be like this. There are many browsers that don't have these issues. When I test my website in Mozilla, I don't find myself checking Safari, Chrome or Opera, I know they work.
If you looked around the real world you would spot there is a whole campaign to bring IE6 down, its not just a few people "Whining and moaning", its a whole world annoyed with a stupid browser that will have been active for 13 years when it departs. It's with Microsoft having the worst update policies and coming out with stupid comments like "let's make HTML 5 native to Windows 7". Native for gods sake. The guy should have been shot when he came out with that comment. Sacked and forgotten, that is the mindset that has got Microsoft where it is today with IE6.
Calling this an HTML5 test seems overstated. Of the 400 possible total points, 131 are from "related standards". Seems like you'd want to break those out to have 269 for HTML and 131 for related.
Even though the related specs are not part of the actual W3C HTML5 spec, they are considered by many people to be HTML5 in the broader sense.
That being said, many of those related specs were part of HTML 5 in the past, but spun of to there own spec. Others were proposed on the WhatWG working group, part of the WhatWG Living HTML spec or extensions to HTML 5 elements.
Despite Microsoft mentioning "native" HTML5 dozens of times when IE 10 PP 1 was released, there are actually no changes in HTML 5 support in IE 10 PP 1. The only changes were either CSS or Javascript. So having the same score as IE 9 would be correct.
Unfortunately, these HTML/CSS/Javascript tests tend not to be about correctness on the large. ACID3 was probably the best example of something that was meant to specifically show how IE was poor. It tested specifically those items that IE was deficient at. It didn't weigh all aspects of the standard equally.
An ideal test would go through and enumerate the standard and attempt to weigh each feature with expected usage and potential to workaround. I've yet to see anyone even attempt it.
The W3C HTML Working Group is working on a complete testsuite for HTML5. The WebApps working group are doing similar things for their specs. many of which are often loosely referred to as "HTML5". The goal of these testsuites is to help browsers achieve interoperability, so the emphasis is on getting high quality tests that cover the whole spec and include difficult cases.
If you want to help out, contributions are very much appreciated; see [1]. Indeed, contributing tests is probably the most effective way to reduce browser interoperability problems and hence pain for future web developers.
If you want to take the tests and start adding subjective weightings based on importance and ease of working around fails, the liberal (BSD-style) license will let you do that (although obviously you can't claim that your derived work is the official W3C testsuite anymore).
It looks like they're not very far along. Do you know if there is an ETA on when they think it will reach signifigant?
I find it surprising to see they only have 925 tests. It seems to me that a spec of this nature is the type that would really benefit from a very test-based approach (as a s section is being written, so are conformance tests).
I'd be interested to see similar tests on Windows/Linux, with ie9 in there. Also, it'd be interesting to see how the beta/dev builds rate.