JS web frameworks benchmark – Round 6

Some people asked for an updated – and there were some interesting updates and contributions, so it’s a good time for a new round of the js web framework benchmark.

What’s changed

About the benchmark

If this is the first time you look at the benchmarks or you’ve somehow managed to forget about them you can read here what the benchmarks measure.

Here’s an article about how we use webdriver to extract the duration from the timeline for the measurement, though in the mean time we’ve switched to a typescript implementation of the test driver. What’s important is that we measure the duration as reported in the chrome timeline and measure from the event start until repainting in the browser finished (i.e. this is much longer than the time the pure javascript code needs).

This link allows you to play with the benchmark in your browser. All benchmarks report some approximate duration on the console. But please note that this is just for fun. The results table’s measurement is as described above.

There are two categories for the measurements, “keyed” and “non-keyed”. Details can be read here, put shortly non-keyed implementations can be faster, because they are allowed to freely choose the fastest way to update the DOM. This may cause problems e.g. with css transitions.

Please note that as with all benchmarks the data should be interpreted carefully. Measurement has some inaccuracy, such that each run varies a bit. We’re reporting the average of 20 runs per benchmark and framework here.

Results


(Click here for a filterable and sortable version or  here a for static version if your browser doesn’t support es6 yet.)

Conclusions

  • Some of the “smaller” implementations are incredibly fast like surplus, domvm, ivi and dio and almost beat vanillajs. Still small doesn’t necessarily mean faster than the big ones as Datum, rx-domh, slim and stem show.
  • Non-Keyed implementations seem easier to optimise than keyed. In my personal opinion non-keyed updates are a nice addition but I’d only choose a framework if it also supports keyed updates. Please note that the table is not complete regarding the question which modes a framework supports.
  • Svelte has an interesting concept , which one could call a perfect ahead of time compile-approach.  There’s no runtime javascript that needs to be added to your app. It has currently an issue with swap rows in the keyed mode but I bet they will improve in that benchmark.
  • Glimmer improves on embers results, but is still slower than react, angular or vue.js.
  • Angular 4.1.2 is a bit faster than 2.4.9 though 4.0 was actually slower.
  • Marionette and, although not as bad, Knockout, perform worse than angular.js and may even justify a migration.
  • Of the react clones preact lives up to its promise of being a faster react implementation. React-lite fails to deliver that promise.  Inferno is extremely fast – at least when a few extra knobs are pushed.
  • The startup time row highlight the benefits of smaller frameworks. The cost of adding frameworks can be clearly seen for react. mobx or redux increase startup cost about 30% in comparison to react.
  • Vue.js is the fastest of the big three modern frameworks although not by a large margin. It starts faster than both angular and react and shows little weakness.

15 Replies to “JS web frameworks benchmark – Round 6”

  1. Thanks for your comment – you’re talking about vuejs.org which has the title “The Progressive JavaScript Framework”?

  2. Thank you for doing this Stefan, much appreciated!

    In principle, would you think the Svelte approach could yield the same as vanilla JS? Subject, of course to, hand-optimized code vs generated code difference …

    Regarding memory usage. Even in this relatively simple benchmark, frameworks consume impressively more memory compared to vanilla JS, and in a more complex app, I would assume memory usage of frameworks would be even higher.

    As web apps are left running in browser tabs for long intervals, memory usage becomes even more critical.

  3. Marionette performance is really subpar but with minor changes like using the new CollectionView ( https://medium.com/blog-marionettejs/next-collection-view-d335f5a9736a ) and avoiding triggers improves the time to render 1000 rows by 25%.

    By removing jquery dependency is possible to reduce the render time by half getting results similar to Ember.

    Another consideration is that the current Marionette implementation is creating a View per row each with its own delegated events. On the other hand Vue and Angular implementations, e.g., treats the whole table as one Component (the equivalent for Marionette View class). By changing how events are delegated and/or implementing the table as an unique view is possible to improve even further the performance.

    While Marionette is mostly used with jQuery plus string template rendering, it allows to customize the renderer to use fast approaches like virtual dom (snabbdom, inferno, virtual-dom) or incremental-dom.

  4. It seems Angular 4 is beating React in almost all parameters. Or am I reading something wrong?

  5. @Robin: For non-keyed angular seems a bit faster, but for (the more important) keyed benchmark I wouldn’t say so. For create rows, create many rows and startup the reported durations for react are a little bit shorter. Please keep in mind that the measurement can’t be precise and for sure neither of both is statistically significant faster in the keyed benchmark.

    No matter if keyed or non-keyed I’d conclude from the benchmarks that both have the same performance strengths and both are about equally fast and have an overhead of about 30% over vanillajs (wich should be ok for most applications, so you can use other criteria to decide if you favour angular or react, e.g. template driven vs JSX, reactive programming vs redux state management, HOC components vs dependency injection, build tools and so on).

  6. By the way, I’ve been taking a serious look on surplus. It seems one of the most functional frontend libraries that actually are practical and minimalistic. Your benchmark clearly demonstrates its simple superiority performance wise. I would even say that surplus just as good as an example of a ‘disappearing’ framework when compared to svelte! Reading the surplus source code, you can see how the bulk of it really is just the transpiler, compiling from surplus syntax into native document.createElement! It’s just that surplus doesn’t have that marketing, or advertise itself as ‘disappearing’ on the front page. Besides, first hearing of svelte, I wanted to give it a try too, but its api still leaves a measure of simplicity to be desired.

    Also, I’d like to see how does the new https://github.com/WebReflection/hyperHTML fare against all these. Personally, I enjoy the functional approach in surplus much more compared to hyperHTML, and hyperHTML really is quite confusing too, to begin to understand. But looking at the way hyperHTML handles the dom, it seems quite convincing that it will perform blazingly fast as well.

Leave a Reply

Your email address will not be published. Required fields are marked *

*