Why?

December 1, 2015

Crowd sourced benchmarks

Filed under: Computing, R — Tags: , — csgillespie @ 10:25 am

When discussing how to speed up slow R code, my first question is what is your computer spec? It always surprises me when complex biological experiments, costing a significant amount of money, are analysed using a six year old laptop. A new desktop machine costs around £1000 and that money would be saved within a month in user time. Typically the more the RAM you have, the larger the dataset you can handle. However it’s not so obvious of the benefit of upgrading the processor.

To quantify the impact of the CPU on an analysis, I’ve create a simple benchmarking package. The aim of this package is to provide a set of benchmarks routines and data from past runs. You can then compare your machine, with other CPUs. The package currently isn’t on CRAN, but you can install it via my drat repository

install.packages(c("drat", "httr", "Matrix"))
drat::addRepo("csgillespie")
install.packages("benchmarkme", type="source")

You can load the package in the usual way, and view past results via


library("benchmarkme")
plot_past()

to get

Timings1

Currently around forty machines have been benchmarked. To benchmark and compare your own system just run


## On slower machines, reduce runs.
res = benchmark_std(runs=3)
plot(res)

gives

my_benchmark

The final step is to upload your benchmarks


## You can control exactly what is uploaded. See the help page
upload_results(res)

The current record is held by a Intel(R) Core(TM) i7-4712MQ CPU.

Advertisements

8 Comments »

  1. This is a fantastic idea. However I tried to upload my results and it seems that your package doesn’t like OSX computers very much.

    Comment by Simon Garnier (@sjmgarnier) — December 1, 2015 @ 5:45 pm

    • I’ve made a few changes. Would you mind trying again. If it still doesn’t still, would you raise an issue on the github page. Thanks

      Comment by csgillespie — December 1, 2015 @ 5:55 pm

  2. OK, but which machine holds the current “worst” record?

    Comment by Carl Witthoft — December 1, 2015 @ 6:16 pm

    • The fastest time is around 15 seconds. The slowest is now 420 secs (Intel(R) Atom(TM) CPU N450)!

      Comment by csgillespie — December 2, 2015 @ 10:45 am

  3. An optimized blas library like Openblas or MKL may help as much as improved hardware.

    Comment by AndresTrujillo-B (@aatrujillob) — December 2, 2015 @ 9:19 pm

    • That’s a good point. Currently I detect the BLAS library on Linux. But not on Windows and Apple systems.

      Comment by csgillespie — December 2, 2015 @ 9:57 pm


RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Create a free website or blog at WordPress.com.

%d bloggers like this: