Speedometer 2.0 browser benchmark

You’re not. It’s just a number that doesn’t really mean all that much.

It correlates directly with single-threaded performance which is the best indicator of overall performance on any device, particularly a mobile device.

You could argue we have “enough” performance, if you want to be one of the “640kb is enough for anybody” apologists, but… you don’t want to be one of those people. Definitely not on mobile, not today, anyway. Perhaps in a decade.

Except it doesn’t. The result varies by browser vendor and by browser version and by OS patch and by what other applications are installed and running and by whether you are doing anything else with your bandwidth. It is so far obfuscated from the actual silicon it makes me laugh. At best it is a measure a particular browsers single threaded JavaScript performance at a particular patch level on a particular OS on a particular device at a particular point in time and as rebutted ad nauseum, has little to do with real world use or experience.

That’s fine, but this, exactly what you said, correlates exactly and directly with single-threaded CPU performance. I’ve been studying this for years, as my entire company depends on it.

Does having a higher number here mean my games run faster? Because if it doesn’t I don’t really care how fast my browser loads, from “instant” to “more instant” when it does a web page request. The bottleneck has been ISP provided speed for ages, hasn’t it?

This benchmark isn’t capped by network capacity - it basically loads and switches between various implementations of a very basic Javascript client-side DOM application to see how fast your browser parses and executes Javascript and how long it takes to render/refresh the DOM.

As such, it isn’t affected by ISP speed, and it doesn’t give any indication of how your PC performs in games. It basically shows how good the browser is in rendering an application that does not resemble real-life applications all that much, in a miriad of unefficient (and ocasionally efficient) ways.

So yeah, as I said before, it doesn’t mean much. It’s too limited to actually mean much outside of “how well does the browser run a lot of implementations of a Javascript-based Todo List application, in a way that users would never really interact with it in any meaningful way”.

Ironically the very forum software you typed this message into resembles them exactly.

Anything that cares about CPU perf – and JavaScript cares a hell of a lot about CPU perf – will be reflected in your speedometer benchmark. It’s basically a benchmark of single threaded CPU throughput. Which, again, is 95% of typical performance on any modern computer, unless the software in question is severely limited by GPU, disk I/O, network, or memory bandwidth.

“Exactly” is a very strong word. A “hello world” application that draws several polygons using OpenGL calls resembles something like GTA V, but they are more different than they are similar. We’re both software (and web) developers. We both know that.

As real-world applications usually are, including web applications. The benchmark tests hundreds of insertions in the DOM per minute, which (almost) no real application is going to do. The benchmark has no I/O interaction, which (almost) no real application will lack. And so on and so forth.

I concede the point that it might correlate to single-thread CPU performance (and even then, browsers are now branching out into multiple CPUs in a way that rendering goes in one, script execution goes in another, and so forth which basically diminishes even further the correlation between the benchmark and actual application performance/responsiveness). I still stand by my point that the benchmark is not a good indicator of how a typical web application performs, unless the application in question is an automated Todo List implementation.

It is the correct word, in this case. Note that ember.js is literally a part of speedometer, and guess, go on go ahead guess what Discourse is built with? I dog dare you! I double dog dare you! I sextuple dog dare you even!

Correlates exactly with Discourse. Source: WE FUCKING WROTE THE DAMN THING.

43,4 on Firefox 58 now. I think Apple is throttling my CPU.

69.3 Firefox 58 over WIFI.

Of course I knew that. Yes, there is a ember.js based implementation of the reference todo list application. Also Backbone, React, Elm (and this is one is actually unoptimized, which is why I say it doesn’t resemble a real application) and a lot of other implementations, all geared to solve a very limited, specific problem that doesn’t really resemble real applications all that much except in single-threaded CPU use and limited DOM manipulation!

I know. You’re also not the only person here writing web software - server or client side - for a living. But you know what? This is going nowhere, so let’s leave it at that, shall we?

As I’ve said many times, and many benchmarks have shown, 95% of general computer performance is single threaded CPU performance. Unless you know (or can prove) the bottleneck is elsewhere, that’s the first place to look, the very first port of call.

A computer is first and foremost a CPU. Then you connect it to storage, memory, display, etc.

I don’t disagree. But let’s do an exercise, shall we?

For a typical user using Discourse in an usual fashion: what percentage of the time is spent on

  1. Javascript parsing;
  2. Javascript execution;
  3. Browser/DOM rendering;
  4. Waiting on server calls (AJAX, etc)

?

1 and 2 correlate heavily to single threaded CPU performance. Arguably 3 too, though modern browsers are starting to delegate parts of the rendering process (CSS “box” processing) to different cores (at different rates of CPU utilization) and even the GPU, so I don’t really know how well it correlates. And 4 is almost entirely I/O bound, and we know how I/O is orders of magnitude slower than modern CPUs.

That is my point. Modern, complex applications depend on a lot more than just single CPU performance. Multiple CPUs, GPUs, I/O, they all change the overall picture. It’s kind of similar to how old terminals worked, in that the “upstream” network (well, modem, really) speeds were much smaller than the “downstream” ones, since the user interacting with the terminal was so much slower than the terminal itself.

Reducing complex software to a single point of reference seems incorrect to me, unless that software is mostly CPU bound. In my experience, that isn’t necessarily true of typical web software, even if it applies in some areas like rendering and DOM manipulation (and virtual DOM comparison parsing, etc).

Anyway, I get your point. I was hoping you would get mine. My point is simply that reducing “browser performance” to a single number mostly dependant on CPU performance can be misleading, in much the same way GPU benchmark programs in the past could be somewhat limited in saying how well a given game would perform. That’s all.

I’m not sure what exactly you are taking about when you say “general computer performance” here.

To put things in context, the score of the iPhone X was on par with my laptop’s brand new i7.

But it’s nowhere near as capable a computing device.

I will just toss in that I entirely agree here so you don’t feel like you’re in crazy town.

I agree too, but at this point have given up trying to find a way to restate the point that will somehow penetrate @wumpus’s myopia.

What y’all don’t realize is that Discourse is actually the most sophisticated ToDo List engine in Christendom, generating, completing, and wiping over a dozen intricately calculated ToDo Lists between every refresh of your pitiful 120hz monitor, then still going out and fetching random text from bash.org and skinning it with some fancy CSS to resemble a forum.

Maybe it’s your point, but I honestly don’t think it is. Mostly I feel like you and a few compatriots have become Qualcomm apologists in the vein of “640 kilobytes of memory really should be enough for everyone!” because you have apparently fallen in love with Android and since the only widely commercially available hardware you can get to run Android is Qualcomm… well, here we are.

With all due respect: no, fuck that. I have no beef with Android in particular, but if you are going to set things up so your entire personal identity is tied to particular hardware, then that’s on you, not me. If you are sad because your shit is painfully slow – and it is – that’s a choice you made.

The entire arc of computing history is obscene, gratuitous, vulgar amounts of power over time. For ever cheaper, in tinier and tinier devices. That’s the goal. That’s how this works. That is the engine driving the entire industry. And that’s how it has worked… until Qualcomm.

Qualcomm is a huge painful aberration to this trend – a canker sore on the anus of computing. They will receive no quarter from me, until they fix their shit. It’s bad enough we have billions of absolutely shitty computing devices on the market now that will take 5 years to age out.

@wumpus, I have honestly no idea why you’d ever think that about me. It might be true of others perhaps, but me? I don’t care which CPU is in whichever phone or how one may be more powerful than another. I have an Android phone (a Motorola G4 Play) because it’s far cheaper then any iPhone here in Brazil (seriously, I bought mine for about 30% of the price of the cheapest iPhone available here), and I never even looked at its CPU power (or brand) when I bought it - I just checked how responsive it felt and how lean it was, if the battery life was adequate, and if the price was right. I have no horse in that CPU race and I don’t have any intention of ever having one.

Also, I don’t care about benchmarks by themselves. I don’t really care about knowing which thing is strongest or fastest or largest. I don’t even want the fastest computer. I want a stable one that will do what I need or require of it. The same is true of any computing device I own or think about owning. Period.

Now, what I do care about is making software that most people can use, regardless of how powerful (or not) their devices are. And I care about people realizing that software is a lot more than how powerful a certain CPU is. Some of my favorite games don’t even need a powerful GPU to run, because games are more than the number of polygons they can push. And software, for me, is more about the experience you have than about the number of FLOPS you can put through.

So, please, stop projecting that obsession of yours into others. You have a thing for benchmarks? Good for you. That’s fine. Really. And I’m glad you’re eager to share them with us. But please don’t push them into other people (or try to make them seem more meaningful than they really are) because not everyone is as interested in benchmarks as you seem to be.