Here’s another update for the js web framework benchmark. This time the benchmark has seen lots of contributions:
- Dominic Gannaway updated and optimized inferno
- Boris Kaul added the kivi framework
- Chris Reeves contributed the edge version of ractive
- Michel Weststrate updated react-mobX
- Gianluca Guarini updated the riot benchmark
- Gyandeep Singh added mithril 1.0-alpha
- Leon Sorokin contributed domvm
- Boris Letocha added bobril
- Jeremy Danyow rewrote the aurelia benchmark
- Filatov Dmitry updated the vidom benchmark
- Dan Abramov, Mark Struck, Baptiste Augrain and many more…
Thanks to all of you for contributing!
One approach to measure the performance would be to use browser tools like the chrome timeline, which reveals exact timings, but has the disadvantage of being a manual and time consuming process and yielding only a single sample.
At first I tried automated benchmarking tools such as Benchpress or protractor-perf, but I didn’t really understand the results and thus decided to roll my own selenium webdriver benchmark. I wrote an additional blog entry to describe this approach. To put it short it measures the duration from the start of a dom click event to the end of the repainting of the dom by parsing the performance log. To reduce sampling artifacts it takes the average of runnig each benchmark 12 times ignoring the two worst results.
A single benchmark doesn’t prove anything, so why not add another well known benchmark. I took NBody (as I always did on this blog ;-)).
The results for NBody confirmed those results. For C I took the fastest plain C implementation from the Computer Language Benchmarks Game. Once again the y-axis shows the duration (this time in seconds).
- Slower and faster usually cause headaches in benchmarks (There a nice paper about that http://hal.inria.fr/docs/00/73/92/37/PDF/percentfaster-techreport.pdf). I sticked with the elapsed time, such that e.g. 42% slower means that the factor of the durations was 1.42.
- On the MacBook Pro C was compiled with clang using -O3 -fomit-frame-pointer -march=native -mfpmath=sse -msse3 for x64. Java was Oracle Hotspot 1.8.0-ea-b87 on 64 bit (thus C2 aka Server Hotspot). Chrome was 28.0.1493.0, but the 32 bit version. I tried to compile V8 myself, but both the x86 and x64 custom built V8 were significantly slower than Chrome so I stick with Chrome.
- On the iPhone I used a release configuration using clang with (among others) -O3 -arch armv7
- The Google Nexus 7 runs Android 4.2.2, Chrome 26.0.1410.58. C was compiled with -march=armv6 -marm -mfloat-abi=softfp -mfpu=vfp -O3.