The last round was published more than half a year ago. Time for an update.

The js-framework-benchmark tries to compare the performance of web frameworks by measuring the duration for different operations on a large table like the duration for creating the table, updating, selecting or removing a row and appending rows.

The first round included 9 implementations. In Round 8 there are now 101 implementations. The project grew to more than 300 pull requests from contributors and 2,000 github stars. Thanks to everyone who contributed so far. It’s clear by looking at the number of implementations that keeping them up to date wouldn’t be possible without your contributions.

Keyed and non-keyed

A pretty significant milestone was the separation of keyed and non-keyed frameworks in round 5 (thanks to Leon Sorokin, author of domvm, for the guidance). As it’s an important concept to understand I’ll explain it here again in simple terms.
A framework can implement an update for a list of data in two ways. Either keep an association between a list element and a dom node (which is the keyed mode) or rearrange and reassign the DOM nodes as it wants to (non-keyed).
If you’re using CSS transitions or third-party frameworks non-keyed might cause problems, because e.g. removing an element might instead remove the last element from the dom node list and patch all others to display the correct values. This can cause problems when e.g. CSS transitions styles depend on removing the specific dom node.
Most (maybe al) keyed framework can behave non-keyed if one assigns the list’s array index as a key.
From the framework perspective handling only non-keyed updates makes dom reconciliation much easier. If you want my opinion: Choose a framework that supports keyed updates.

What has changed since round 7?

• Added the first webassembly frameworks in this benchmark: stdweb and yew
• Lots of small and fast frameworks were added: domc, stage0, solid, attodom
• A webcomponent variant for vanilla-js
• A fast alternative renderer for Knockout: ko-jsx
• Significant Improvements for good ‘old’ Marionette.js
• Updates for many others including angular 6 and a preview for the new angular ivy renderer
• Update to the startup metrics. They now simulate a mobile device and use the latest lighthouse version.

Important results:

• The fastest frameworks are very fast. I have an unresolved issue to “make vanillajs great again”. I can’t beat domc and stage0 and solid, surplus and ivy are very, very close.
• Marionette made a significant jump. The first implementation for version 3.3.1 was about 248% slower than the then fastest pure javascript ‘vanillajs’ version. Version 4.0.0-beta.1 is only 24% slower than domc.
• The (low level) webassembly framework stdweb is only 13% behind. Might be interesting to see how they continue.

Here’s the complete table:

Discuss this article on reddit.

Some people asked for an updated – and there were some interesting updates and contributions, so it’s a good time for a new round of the js web framework benchmark.

What’s changed

• Angular moved to version 4.1
• Polymer reached version 2
• Mithril moved to 1.1
• Marko and Glimmer were added
• as well as a few smaller frameworks: ivi, surplus, rx-domh, Datum, pico-dom, hyperapp, svelte. stemjs and slimjs
• A new benchmark that measures startup duration
• A result table with filtering and sorting – the static table was getting bigger and bigger…

Continue reading →

It’s been a few months since I published Round 2 of my javascript web frameworks performance comparison. In Javascript land months translate to years in other ecosystems so it’s more than justified to introduce round 3.

Here’s what’s new:

• Added a pure javascript version to have a baseline for the benchmarks (“vanillajs”).
• Added cycle.js v6 and v7. What a difference the new version makes!
• Added inferno.js. This small framework made writing a faster vanillajs version challenging.
• Updated all frameworks to their current version.
• New benchmarks: Append 10,000 rows, Remove all Rows, Swap two rows.
• Added two benchmarks that measure memory consumption directly after loading the page and when 1,000 rows are added to the table.

The result table for chrome 51 on my MacBook Pro 15 reveals some surprising results (click to enlarge):

(Update: cycle.js v7 was reported with an average slowdown of 2, but I forgot a logging statement. Without it it improved further to a factor of 1,8. The table has been updated.)

• Vanillajs is fastest, but not by much and that only by hard work.
• Inferno.js is an incredibly fast framework and only 1.3 times slower than vanillajs, which is simply amazing.
• plastiq comes in as  the second fastest framework being just 1.5 times slower, followed by vidom.
• React.js appears to have a major performance regression for the clear rows benchmarks. (Clearing all rows for the third time is much faster. There are two benchmarks pointing to that regression which is a bit unfair for react.js)
• Cycle.js v6 was slowest by far, v7 changes that completely and leaves the group of the slow frameworks consisting of ember, mithril and ractive.

All source code can be found on my github repository. Contributions are always welcome (ember and aurelia are looking for some loving care, but I to admin that I have no feelings for them. And I’d like to see a version of cycle.js with isolates) .

Thanks a lot to Baptiste Augrain for contributing additional tests and frameworks!

Here’s a follow-up to my last blog post since there’s good reason for an update:

• It turned out that ember performs better with ‘key=”identifier”‘ in the #each helper.
• One of the vue.js creators contacted me and told me that they fixed vue.js in response to the my benchmark
• Two other react-like libraries were added: Preact and react-lite

Here’s the new measurement showing the duration in milliseconds:

(Click the image to see the graph in full size)

The duration is measured from the beginning of the dom click event to the end of repainting and thus includes javascript execution, rendering and layout duration. If you want to read more about it please check the according blog entry. All tests are run via a selenium test and multiple samples are taken and averaged (after removing the worst measurement). The full description for the benchmarks can be found in round 1.

And the new conclusions:

• Ember got much faster for “partial update” (from 164 msecs to 58 msecs), but is still quite slow for create or update 1000 rows.
• Vue.js improved a lot for the “update 1000 rows (hot)”  benchmark (from 435 msecs to 260 msecs)
• Preact leaves quite a good impression. It’s a lot faster than react for create 1000 rows and update 1000 rows and not much slower for the rest.
• Almost the same can be said about react-lite though the performance for “remove row” is rather weak.

All in all especially angular 2, vidom and preact impress with their performance. Still aurelia feels much faster in the browser than in the selenium tests (except for startup duration which might be my fault not using the bundler correctly). And yes, vue.js is now pretty close to the fastest frameworks.

You can view the benchmarks in the browser and find all source code on github.

I must admit that this post came a bit unexpected. I just wanted to compare the performance of a few Javascript frontend frameworks and that’s where all the mess started.

What to measure

Before running a benchmark one should be clear about what to measure. In this case I wanted to know which framework is faster for a few test cases. I knew which test cases, which frameworks, which left unclear what faster actutally means. Let’s take a look at a chrome timeline:

Continue reading →