February 15, 2016

Shiny benchmarks

Filed under: Computing, R — Tags: , — csgillespie @ 5:49 pm

A couple of months ago, the first version of benchmarkme was released. Around 140 machines have now been benchmarked.

From the fastest (an Apple i7) which ran the tests in around 10 seconds, to the slowest (an Atom(TM) CPU N450 @ 1.66GHz) which took 420 seconds! Other interesting statistics:

  • Around 6% of people ran BLAS optimised versions of R;
  • No-one (except for machines that I used) ran a byte compiled version of the package.

I intend to write to a blog post or two on BLAS and byte compiling, but for the meantime you can investigate the results via the new shiny interface. The package is still only available on github and can be installed via:

## Update the package
install.packages(c("drat", "httr", "Matrix", "shiny"))
install.packages("benchmarkme", type="source")

You then load the package in the usual way

## View past results
## shine() # Needs shiny
## get_datatable_past() # Needs DT

To benchmark your system, use

## This will take somewhere between 0.5 and 5 minutes
## Increase runs if you have a higher spec machine
res = benchmark_std(runs=3)

You can then compare your results other users

## shine(res)
## get_datatable(res)

and upload your results

## You can control exactly what is uploaded. See details below.

This function returns a unique identifier that will allow you to identify your results from the public data sets.


  1. So as you hint at, there are two factors that cause much better performance: optimised BLAS for matrix calculations, and byte-compiling for the programming benchmarks. I was under the impression that byte-compilation would be the default for installed packages, but you actually have to enable this:

    install.packages(“benchmarkme”, type=”source”, INSTALL_opts=”–byte-compile”)

    (See https://cran.r-project.org/doc/manuals/R-admin.html#Byte_002dcompilation — base and recommended packages are byte-compiled, but others are not as it may cause problems.)

    Comment by Michael Kuhn — February 17, 2016 @ 12:13 pm

    • That’s one way of implementing byte compiling, the other is to set `R_COMPILE_PKGS=3` in your Renviron. I’ve had this flag set for a year now, and I’ve never had any issues. I intend to do a blog post on it in the near future.

      Comment by csgillespie — February 17, 2016 @ 7:28 pm

RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Blog at WordPress.com.

%d bloggers like this: