Another cut at market randomness

I have some background in computer security and one day found myself tasked with assessing the quality of randomness for session id tokens generated by popular web frameworks (namely Java and .NET). As it turns out, NIST have developed a series of tests for just this purpose detailed here.

As a non-believer in the absolute randomness of markets, I thought I might take a look at the series of returns from SPX/GSPC to see how they held up.

About the tests

The NIST tests are designed to be run on bit streams from a random number generator, so some liberties had to be taken. I am mainly interested in the predictability of up days vs down days, so I first generated a series of returns using quantmod then encoded them using a 1 for an up day and -1 for a down day and run the tests on the encoded stream. The NIST tests use -1 in place of a 0.

The main test is called the monobit test, to quote from the document:

The focus of the test is the proportion of zeroes and ones for the entire sequence.  The purpose of this test is to determine whether the number of ones and zeros in a sequence are approximately the same as would be expected for a truly random sequence.  The test assesses the closeness of the fraction of ones to ½, that is, the number of ones and zeroes in a sequence should be about the same.  

The tests are done using a significance level of 0.01, so come with a good degree in confidence assuming the underlying method is sound. 

One caveat is the length of the runs compared and how it relates to the distributions used to model the results. For the monobit test, the suggested input size is 100, requiring 101 days of data to determine 100 up or down days. If we were looking at 32 bit integers, 100 bits would only be 3 "full" random numbers, so arguably we would want to look at shorter time periods (e.g. 3-5 days of data). Given the difficulties around distributions which require a large n, I thought I would vary the significance level instead of a lower n, as our requirements are not as stringent as those for cryptographic random numbers.   

Results

At a basic level, this series does appear to be random, at least the vast majority of the time with n = 100 and alpha = 0.01. My confirmation bias was very upset with this. 

However, if we plot the proportion of runs deemed random vs the significance level, we see the proportion rising as one might expect. One thing that remains unexplained is why this appears to rise in steps rather than something more linear, though I expect this to be a side affect of either methodology or the normalisation done by the tests. I also took a look at the weekly data, which tends to a greater proportion of non-random runs quicker than daily data.

I am interested in the applications of machine learning to financial markets. Close to close returns that we have been looking at here are not the only information we have available, nor are they what I trade on a personal level. Also this is only one price series, and one could argue that in practise it is not actually a tradable series. 

Close to close returns are very useful in lots of applications, but if we are trying to build some predictive model we might need to look for more predictable pastures. Machine learning algorithms are great, but can only do so much. Finding some better potential inputs is what I will take a look at next.

The test code is available on github here: R Monobit test. Would be very interested to hear if anyone else takes a look.

I also took a visual and binomial look at randomness of the series in this post: A visual look at market randomness.

Oh and in case you were wondering, the web session tokens all turned out to be very strong.

A visual look at market randomness

I recently did some statistical testing to see if markets were random (details in the post Another cut at market randomness). It turns out they were, at least close to close returns for SPX/GSPC. My confirmation bias wasn't going to stand for that, so I thought about taking a different look.

Two things interested me. Firstly, I am looking at up vs down (i.e a higher or lower close than previous), rather than trying to predict an exact price. If markets are random then up or down have a probability of 0.5 each and are independent. A run of 5 consecutive ups or downs has a probability of 0.03125 or roughly 3%. How would that pan out looking at historical data?

Secondly, how could one visualise seemingly random data without it ending up looking like noise?

I came up with the following chart:

Each square represents one week and each line represents one year. If the close was higher than the previous week, it is blue, otherwise it is red. As the count of successive higher or lower weeks rise, the boxes get deeper in colour, up to a maximum of 5. As a side effect of date calculations and the definition of "week" some years have 53 weeks, which is why some lines are longer than others.

In total there were 138 runs of 5 weeks in the same direction out of 2208 samples, or around 6%, roughly double what we might expect.

Looking at that, I wondered what it would look like comparing weeks across years, comparing week 1 of year n with week 1 of year n + 1. That lead to the second chart:

This time we had 175 runs of length 5 out of 2208, just under 8%, again quite a bit more than the 3% we were expecting.  

That is all well and good, but these charts only represent the direction of the week to week moves, not the magnitude of the moves which is probably more important. Finally I took a look at the return over 5 periods. 

Again if it is positive the squares are blue, negative they are red. The colours are scaled as a proportion of the largest positive and negative returns for blue and red squares respectively. The very pale squares are where the returns were proportionally so close to zero they would not otherwise be visible, so I set a minimum level to ensure they displayed.

We can see that positive returns tend to follow positive returns and vice versa, at least for this 5 week look back period. This is somewhat deceptive as a negative return, though negative, may still be higher than the previous one implying a loss. 

What does all this mean? Not too much in practise, as it is another thing to know in advance if a series will have consecutive up days or down days. In this case a tradable edge is not so easily won.

However, it does reflect my understanding of how prices move a little better, in that they trend for a while then range for a while and vice versa, and things may not be as random as we might expect. My confirmation bias somewhat sated. 

The charts were done in Processing using the free weekly data from Yahoo! finance for GSPC. If you would like a chart for a given ticker, let me know.

Diversion: Generative Work

Here is another generative thing I have been working on lately

 

Mebane Faber Tactical Asset Allocation in R

In 2006 Mebane Faber published a great piece of research detailing an asset allocation system that was both very easy to understand and implement, as well as carrying very respectable risk adjusted returns.

The details are available in his paper A Quantitative Approach to Tactical Asset Allocation and were further expanded on in his book The Ivy Portfolio both of which are must reads.

The short version is to use diversified asset classes, long only, and only long when the price is above the 10 month simple moving average (approx 200 day). The assets he tests are U.S. Stocks, International Stocks, U.S. Government Bonds, Commodities and Real Estate, accessible via ETFs.

A rotational extension can also be added by investing only in the top 1-3 asset classes showing some degree of relative strength, which is defined as the average of 3, 6 and 12 month returns. They must also be over the 10 month SMA to be candidates.

The system updates monthly at the end of the month, it is about as hands off as you can get for active management.

There is an ETF for those so inclined, GTAA, but I am experimenting with a put selling implementation, which I might start tracking here month to month. I wrote a small R script using quantmod to display the relevant information for given symbols, which should be available here: Tactical Asset Allocation R script

The output looks like this:


  Sym         R3m         R6m        R12m  Close     AvgRet OverMA
4 VNQ  0.09295631  0.22412597  0.08488552  63.65 0.13398927   TRUE
1 VTI  0.11671109  0.22466699  0.05037598  72.26 0.13058469   TRUE
2 VEU  0.10908623  0.13282091 -0.10915250  44.22 0.04425155   TRUE
5 DBC  0.07048208  0.11194076 -0.05767911  28.80 0.04158124   TRUE
3 IEF -0.02193049 -0.01718305  0.10473673 103.28 0.02187440   TRUE


Let me know if you have any comments or find any bugs. 

Machine Learning Examples in R

This is a post that has been a long time in the making. Following on from the excellent Stanford Machine Learning Course I have made examples of the main algorithms covered in R.

We have Linear Regression

Followed by Neural Networks

And Support Vector Machines

One remaining item is Logistic Regression, I am yet to find a library in R that behaves as I want, so that will come at some future date. I've been sitting on this post for ages and got sick of waiting. As an aside I find the documentation in R to be variable at best, which can make it somewhat of a pain to work with. When it is good, yes it can be very good but often it is quite poor ...

R is great for data analysis and exploration, but I have found myself moving back to python for many more mundane tasks.

Anyway for those interested in the code, I have put it on Github. The data is from an exercise in the Stanford course, and by tweaking the parameters I really got a good feel for how the various algorithms work in practise.

Once I finish my backtesting engine I will probably put it up on Github as well, and then I can start digging into the applications of ML techniques for trading systems.

A short diversion

I have been busy learning machine learning techniques, writing a market data replay/limit book backtesting framework in python, and messing around with the Processing graphics environment.

More to come on the first two later, but here is a sketch of something I made in Java/Processing

December Ramblings

It's been a little quiet here of late, I have relocated from Berlin to Sydney which has engaged my attention but I am starting to feel a little more settled. It's great to be back with familiar faces and places after a few years out in the wilderness.

I have enrolled in a postgraduate Math/Statistics degree which I am very much looking forward to. It starts next year so I have been revising my undergraduate math material and slowly making my way through the free Stanford Machine Learning course material which I have been finding excellent.

I have still been testing out ideas but most have been dead ends, so not much to report there. Instead I would like to get on my soapbox on two things, the strange contradiction in some of those who self describe as "fiscally conservative", and a general rumination on democracy and efficiency.

Not so conservative

I don't really get how one can say they are fiscally conservative yet in the same breath advocate lower taxes. If you sit down and think about it, surely the conservative view would be to gather as much tax revenue as one could. I don't think many private sector accountants would advocate limiting ones revenue sources.

I'm not really for super high taxes (say >= 55%), but very low taxes (<= 30%) seem like a bad idea. Public services cost money to run, and society as a whole benefits from good infrastructure, health care and education system. An exception might be Switzerland, but after living there for a while, yes on the face of it taxes are low, but everything else is expensive. There are a lot of compulsory things like insurance, licenses for this and that, so you end up paying anyway. This in turn stratifies society as only those relatively wealthy can take up the benefits of life in a modern western society.

There are a bunch of countries with very high taxes but they are mostly well managed and great places to live, better than the US by pretty much any quantifiable metric. I remain a great fan of Sweden which to my mind has struck a good balance between social policy and equality while remaining a fundamentally capitalist economy.

Democracy and the worst case

People discuss the inefficiencies and weaknesses of democracy, as if they would magically disappear under some other system, or that private enterprise is some beacon of expediency and should be a model aspired to by governments.

From what I have seen large private enterprises are very bureaucratic and stifling, and I think no worse than your favourite government department. The difference is that public institutions have the light of transparency shining down on them, whilst private institutions remain just that, private, so the waste and inefficiency is largely unexposed.

It is very hard to get things done in a large organisation, and to an extent I believe that to be by design. It is a safety mechanism against the worst case scenarios. For sure opportunities may be missed due to inefficient processes, but not all ideas are good, and long, drawn out approval processes also kill bad ideas that may result in taking a company down. There are exceptions but as a generalisation I think there is a degree of truth to it.

When people glorify private enterprises as a model of efficacy and efficiency, they presuppose the aim of an organisation is to reach some optimal maximum, i.e. attaining the best case. And though I agree it is important to strive for good results and outcomes, for large organisations I think it is more important to ensure worst case outcomes never eventuate (I'm sure we can all think of examples.)

While I was in Europe I had the opportunity to visit Auschwitz in Poland, one of the most infamous concentration camps of WW2 pictured above. There is a train line that runs right up to the entrance of the gas chambers used to murder of millions of people over the course of the war. The powers of the time clearly thought this was a great idea.

There are many problems with democratic political systems, and it is true they will never likely reach some optimal best case. We have to accept that most of the time we probably wont achieve the best case, but at the least we avoid the worst case. History shows us the worst case can get very nasty, so all in all I think it's a reasonable trade off.

Hope you and your families all enjoy Christmas and the holiday season.