For a recent project I needed to calculate the pairwise distances of a set of observations to a set of cluster centers. In MATLAB you can use the pdist function for this. As far as I know, there is no equivalent in the R standard packages. So I looked into writing a fast implementation for R. Turns out that vectorizing makes it about 40x faster. Using Rcpp is another 5-6x faster, ending up with a 225x speed-up over the naive implementation.
Pairwise distances in R
Using C libraries in R with rdyncall
One reason I like using R for data analysis is that R has a great collection of packages that let you easily apply state-of-the-art methods to your problems. But once in a while you find a library that you would like to use that does not have a R wrapper, yet. While the great Rcpp package provides a convenient way to write R extensions in C++, it obviously requires you to write C++ code and to have a compiler installed.
An alternative I found about only recently is the rdyncall package. rdyncall provides an improved Foreign Function Interface (FFI), which allows you to dynamically invoke C libraries.
In this blog post I want to give you an example on how to employ rdyncall to use
the LWPR libary for Locally Weighted Projection Regression.
Continue reading →
R BLAS: GotoBLAS2 vs OpenBLAS vs MKL
Short update to Speed up R by using a different BLAS implementation/:
- MKL is overall the fastest
- OpenBLAS is faster than its parent GotoBLAS and comes close to MKL
A = matrix(rnorm(n*n),n,n) A %*% A solve(A) svd(A)
Speed up R by using a different BLAS implementation
It is no news that R’s default BLAS is much slower that other available BLAS implementations. In A trick to speed up R matrix calculation/ Yu-Sung Su recommends using the ATLAS BLAS which is available on CRAN. When I learned about the possible speed-up a while ago I tried several BLAS libraries and I found that GotoBLAS2 was giving me the best performance among the open-source BLAS implementations. Today I decided to check once again how much it makes sense to replace R’s default BLAS library.
Here are some results from my Intel i7-620M laptop running Windows 7:
Better R support in pygments by monkey patching SLexer
I started using knitr with reStructuredText today and I found that the syntax highlighting with pygments (used by
rst2html.py) was not as nice as the output of pandoc. So I ended up doing some monkeypatching.
Try adding the following to
# SLexer is the lexer used for R from pygments.lexers.math import SLexer from pygments.token import Keyword, Name # monkey patching SLexer ... # add some builtin functions (TODO: add more) SLexer.tokens['keywords'].append( (r'(?<![A-Za-z0-9_-])(c|library)(?=\()', Name.Builtin)) # treat all names in front of a parenthesis as function names SLexer.tokens['keywords'].append( (r'[a-zA-Z][a-zA-Z_0-9]+(?=\s*\()', Name.Function)) # parameter names inside function calls/definitions SLexer.tokens['root'].insert(0, (r'(?<=[\(,])\s*[a-z]+\s*(?==)', Name.Attribute))
Note: I assume you already added pygments’ rst-directive.py to