The R documentation is bad

I have been using R for some time now and still can find it frustrating to work with. Over the years have come to the conclusion that it is primarily due to the documentation being bad. I offer no actual solutions here, but thought I would try and write down exactly what I dislike about it.

The docs are more or less premised on knowing which function you wish to use. Want to find out what some argument of some specific function does? Sorted.

But, if you are new to the language or just want to check out a few different ways of doing things, the built-in documentation is not going to help.

The “phone book” type reference that just lists all functions in a package alphabetically is useless for exploring the language. This in turn makes it very hard for people to adopt R. I know this from my own experience and helping others who are new to R.

Example

As an example, let’s look at creating an identity matrix. Pretend we are new to R and staring at the prompt.

First I look at help(matrix) and see nothing about it there. Following the links in the “See Also” section, we visit the page for data.matrix and array, neither of which give any hints.

Let’s try a search, typing ??identity at the prompt. A big list of functions pops up, but only two from base and one from stats. First is called ‘dontCheck’ which probably isn’t going to make an identity matrix so I skip that, but second up is identity. 

A-ha! Let’s take a look

“A trivial identity function returning its argument.”

Oh. 

All right, lets bring up the phone book listing for everything in base. I type help(base), navigate to the index and type identity in the search box. Nope, just the two functions I saw before. 

Now, I personally know the right function is diag, but how are people meant to find that out? If they have to search google for basic things, it’s kinda hard to conclude that the documentation is good.

The second example is left as an exercise to the reader: find in the R help how to get the last element of a vector. And no, despite the helpful and intuitive name, the documentation for “[“ doesn’t say.

Help

The actual help browser is I dislike as well. I have not yet figured out how to have more than one page open at a time, and I really don’t like the idea of having lots of windows popping up everywhere. Something like tabs would be a lot more useful. 

Most people are reading on the screen, which makes big blocks of text hard to read, and big blocks of text feature quite often in the help pages. 

This becomes especially important when there are so many intricate details being explained. It’s easy to miss things, and many pages require careful and close readings. 

Usually, once you wade through all the text, there are examples. Many of these are not clear, seemingly having been written for brevity over elucidation. It’s hard to work out what exactly is going on, again especially for people new to R.

None of the examples include the output inline, which also makes it very hard to get a feel for what they do. 

This is made worse by the fact that often I find myself on a “function safari.” I know there must be a function that does what I need but I have no idea what it is called or how to find it. 

It’s a huge time sink if I have to go through a bunch of help pages, manually run all the examples to figure out if they do what I need. 

The font is ugly as well. Just throwing it out there.

Groups

Having specific documentation for each function is necessary, but there should be higher level grouping of related functions. 

For example when I look at help(matrix) why does it not include links to diag, rbind, cbind, t etc. or show examples of them all that include the output when run?

This is a good start, but it is all on one big page and not actually referenced in the help page for matrix. (Note also it does not mention diag)

One of the problems is the use of generics and the general mish-mash that is OO in R. It makes it hard to find functions that are related to working with a particular data structure or the task at hand. 

For example, compare 

R vector with the Python or C++ equivalents.

I really feel all these things compound to be a huge weakness of R.

Outro

For me the most frustrating thing is that the documentation I am after is usually there, it’s just too hard to find and often poorly presented. There is much room for improvement of structure and presentation. As it is now it is hard to search, hard to navigate and hard to read.

I would love to see a nice HTML based reference with navigation menus and logical groupings. It is easy to provide logically structured information as well as a reference index.

By switching to something browser based, it makes it a lot easier to find relevant information, as well as allowing users to pick their own fonts (and font sizes), as well as have multiple pages open at the same time.

I am sure there are people out there thinking “but I like the way R does its docs!!” and I can’t argue with that. But I have seen enough videos on the Internet to know some people seem to like some very strange things indeed ...

I could go on and on with lots more examples/things that are just a bit silly. As I said at the start, I have no real solution to offer, but hopefully provide some useful specifics, or at least maybe help some people realize “it’s not you, it’s R.” I have gotten myself up to speed in a lot of languages over the years and by far, R took the longest to feel competent with.

[update]

Pat Burns has shared a useful map of information R Navigation Tools

Also I found the The R Cookbook a useful reference for answering those "how do I X a Y" type questions. 

Ritchie Cotton has shared his thoughts on the topic here.

Intro to Curl Noise

I gave a small talk at the recent Sydney Creative Coding meet on Curl Noise.

It was an introductory talk with the goal of explaining the basics, so those interested can go off and look at other peoples work and hopefully understand what is going on.

I have put the slides up here. Throughout the talk I took breaks for questions and showed pictures of curl and miscellaneous mathematical facts, which is why those appear.

I also wrote the following apps as demos

curlgrid - This is a basic app to explore noise/vectors/curl and particles in perlin noise
gpu curl 2D - This is a 2D version doing particle advection and curl generation on the GPU
gpu curl 3D - As above but a 3D version

All of them require openFrameworks and the ofxGui addon which should come with oF.

Some links to others work

Robert Bridson - Wrote the original paper on the topic, has some code available
Philip Rideout - More comprehensive examples and explanations
miaumiau - More discussion and cool examples in WebGL

I did only really discuss the basics so if you are interested I would strongly encourage you to go check them out.

It did take me some time to get my head around things, so if you didn't manage to pick all of it up in the ~25 minutes I was speaking for, don't worry about it. Even now I am not 100% sure the 2D case is right, appreciate any comments.

The GPU versions use a transform feedback vertex shader, you can find more info about this here.

If you are interested in the math, you can find more info here

Some other links:

Cosine approximation to the normal distribution: here
A derivation of the volume on an n-sphere: here

Questions/Comments email me or post here

You can also find me on tumblr, vimeo and twitter

Thanks!

 

Fitting a mixture of independent Poisson distributions

This is an example from Zucchini & MacDonald’s book on Hidden Markov Models for Time Series (exercise 1.3).

The data is annual counts of earthquakes of magnitude 7 or greater, which exhibits both overdispersion for a Poisson (where the mean should equal the variance) as well as serial dependence.

The aim is to fit a mixture of m independent Poisson distributions to this data, using the non-linear minimizer nlm to minimize the negative log-likelihood of the Poisson mixture.

Sounds easy right? I have not done much optimisation stuff so it took me longer than it probably should have. There’s little better for the ego then getting stuck on things by page 10.

Constrained to unconstrained 

There are two sets of parameters, the lambda parameters to the Poisson distributions and the delta mixing distribution, the latter giving the proportions of each distribution to mix.

The Poisson parameters must be greater than zero and there are m of them. The mixing distribution values must all sum to one, and the first mixing value is implied by the m-1 subsequent values.

These are the constraints (lambdas greater than zero, deltas sum to one), and there are 2m – 1 values (m lambdas, m-1 deltas). We need to transform these to unconstrained values for use with nlm (becoming eta and tau respectively).

The formulas for transformation are reproduced from the book here

These transformed values are combined in to one parameter vector and passed to nlm. We also pass a function that calculates the negative log likelihood.

In the function, we first convert the parameters from the unconstrained “working” form to the constrained “natural” form using the inverse transform, and then use these values to do the likelihood calculation.

You can see the code here.

Outro

Now that I look over the code and write this out it does seem all fairly straightforward and relatively trivial, but it was new to me and took a while, so I figured other people might find the example useful as well.

The code has three examples for the cases m = 2, 3, and 4. Obviously feel free to mess around with the initial parameter estimates to see what comes up. 

It does seem to match the fitted models from the book, which I have copied here:

For all the details, check out the book, chapter 1!

Does Trend Following Work?

I’m not sure how I came across it, but I have had Jez Liberty’s Au.Tra.Sy blog in my reader since around 2009.

Since then, he has tracked well-known trend following systems and reported monthly performance figures. These are things like moving average crossovers, Bollinger band breakouts and stuff like that.

The systems had a very good month in November, and a very good year, up ~45% YTD.

I thought I would take a look and how things have gone over the time it has been tracked. Helpfully, Jez posts annual summaries for each system and a composite average, which is what I used. You can find the links to specific posts in the script linked under the table below. N.B the first two years are the average of actual trend following funds not the generic strategies.

There are results for 6 years, from 2009 to 2014, using the YTD November figure for 2014. We end up with a series like this, starting from 100.

Source [R]

We can see that 4 of the 6 years ended under water. Over the same time there has been a huge surge in US equities. A simple buy and hold in 2009 would have more than doubled equity.  

The pain of losing

I have been following Jez posting results every month for 5 – 6 years now. When I think back to where I was in 2009 versus where I am today, five years is a really, really long time.

You can get historical data from 1920s and even further back in time. If you run a backtest over 80 – 90 years of data, a 3 – 5 year period of underperformance is barely noticeable on an equity curve.

Sometimes I see people get excited about their backtests even though they include these periods of poor performance. There is plenty of research and evidence that humans in general find it very hard to stick with a losing system.

How seriously would you take someone touting a system that is underwater after five years, but they assured you a 45% year was just around the corner?

Professional CTA

I believe the Au.Tra.Sy data is based on commodities and futures trading, so thought I would take a look at the Barclay CTA Index as a comparison. The results seem somewhat similar, it has struggled somewhat since 2010, though both the upside and downside returns are smaller. 

In the 29 years from 1980 to 2008 inclusive, there were only 3 down years in total, none of them consecutive.

In the period we are examining here, the 6 years from 2009 – 2014, four of them have been down years, including three down years in a row (2011 - 2013).

Proselytizing

In some ways, you can think of mechanical trend following as being long tail risk. Effectively it wants a big sustained move in either direction. If you think about the distribution of returns, the big moves that are “pay days” are out in the tails.

I don’t follow commodities vol at all, but apparently it has been low for several years, according to this article from September, 2014. It probably shouldn’t be a surprise that strategies dependent on big moves have not performed well in an environment of low volatility. 

I personally dislike these indicator-based trend following systems, even though I think they give entries as good as any. The problem is they are too slow to close positions and give back too much profit.

They only really make serious money when a megatrend eventuates, and these are relatively rare and IMO getting rarer. In the mean time they can get chopped around and experience significant drawdowns.

I don’t want to draw any final conclusions about classical trend following, but I know I would have a lot of trouble sticking to systems like these in practise, even though a long term backtest might look really nice.

However, if you understand the conditions under which your system does well or is likely to underperform, it is a lot easier to stick with it during periods of underperformance, which are inevitable over the long run.

It is worth spending the time thinking about the conditions in which your system does well and does poorly. 

Book Review: R Object-oriented Programming

Packt sent me a copy of R Object-oriented Programming by Kelly Black to review so here we go:

The aim of this book is to provide a basic understanding of R and programming in R. It is suggested that readers are already familiar programming, but the book does seem to start from square one and I feel is probably accessible to most people using R. 

The contents are roughly split 50/50 between general R stuff like working with data and the various data types R has and then OO. The first half provides good coverage of the basics, and I thought it was a good introduction to R. It includes a chapter on working with time variables which I think would be very useful to people new to R and working with time series data.   

The second half is about objected orientated programming. S3 and S4 classes are covered, and some decent examples are provided that go beyond the usual trivial hello world or Shape, Circle, Square type examples, which is nice to see.

Strangely though, no mention is made of reference classes. For a book titled R Object-oriented Programming, making no mention about one of the ways to do OO in R seems like a rather large oversight. 

Another nitpick I have is the indent style and formatting. It is mostly Whitesmiths style (which I personally dislike), but beyond that it is used inconsistently throughout the examples. This is generally considered a rather bad practice in programming circles. Pick one style and stick with it. 

It is challenging to present source code and text together, but some of it is laid out very poorly, with lines wrapping in strange places. It doesn’t really affect the content because most people will likely download the code to experiment with it, but it does detract from what is otherwise a good book. 

Arguing about whitespace may seem petty, but the reality is most programmers spend most of their day looking at source code, so it is important. There's plenty of projects that wont even look at your code unless it is properly formatted. Again its not the choice of style, its the inconsistent use and poor formatting that bothers me. There's a lot of terrible R code out there and a book about programming should do better.

Despite these issues I think it gives good coverage of relevant topics for those starting to get more into the programming side of R. If you are starting out in statistics and looking for a book on R to get going with programming (versus say how to do linear regression etc) it's a useful one.

You can see some other reviews here, here, and on Amazon.