Trading in a low vol world

I wanted to take a look at what works in low vol environments, such as we are currently experiencing. I am open to the idea we have entered a period of structurally low volatility due to increased regulatory burden and flow on effects from the decline of institutional FICC trading. Or it may just be a function of QE, and post-tapering we will see a return to higher levels.

The plan

The main idea is to compare mean reversion (MR) vs. follow through (FT). For simplicity I define mean reversion as an up day being followed by a down day, and a down day being followed by an up day. Conversely, follow through sees an up day followed by another up day, and a down day followed by a down day.

I took a look at the major US equity indices, SPX (GSPC), NDX and RUT.

For each series we calculate daily log returns for the current period and shift the forward to get the return for the next period. Then we calculate realized volatility (RV) and split the data set into "low volatility" and "high volatility", by looking at median realized vol for the whole series.

Then, for each series, we use bootstrapped samples to simulate a number of trajectories/equity curves for each strategy (MR/FT) under the two classes of RV. Finally we take an average of the total return of each trajectory to get a ballpark idea of how they went.

Results

The data is from the start of 1999 to the present, so roughly 15 years. Each run generates 1000 trajectories with a sample size of roughly 950.

For the low vol case, the results are unfortunately ambiguous. Follow through in a low vol environment seemed to do well for NDX and RUT, but the opposite was the case for SPX.

The TR column is the sum of the series over the whole period for the volatility class (i.e. a simple long only strategy), giving an idea of a directional bias that may be present in the sampling.

In the high vol environment, mean reversion was a clear winner, and consistent over the different underlyings.

The results seem relatively stable across trajectory size/sample size.

Outro

I'm not really sure what is going on SPX. My intuition was that FT would do well in low vol environments, but that doesn't seem to be the case, at least not for SPX.

I was actually getting consistent votes for FT in the low vol case, then restarted R to run with a clean environment and started getting the above instead. You can't spell argh without R it seems.

Source is up here. As always you can find me on twitter here. Thanks for stopping by.

Divergence on NDX

I generally take a dim view of old-timey technical indicators, perhaps they work for some people but I have found there are much better tool available. One exception is divergence, which in this case is when price makes a new high, but the MACD (or your favourite oscillator) does not.  

There is very nice looking divergence on NDX, and it also shows up in a weaker form on SPX and INDU. I never take it as a trade signal by itself, but it does make me look a little closer. I have marked off some previous occurrences as well. It does not give any indication about when a sell off may occur, or how much of a sell off will eventuate. Pretty useful isn't it?

Another form of divergence I take note of is the marked failure of RUT to make it back to its recent highs, which differs from NDX/SPX/INDU. 

You can also see in the charts above that volume has been declining, especially over the last 4-5 weeks. 

I do think we are in a long run bull market which still has a few more years to go. In the event of a shorter term sell off I would generally be looking to buy dips. 

A good sign of a bull market is shrugging off negative events. We've had some reasonably serious geopolitical happenings, the invasion in Ukraine, a coup in Thailand, and anti-Chinese riots in Vietnam that produced a number of fatalities. 

Struggling to think what a catalyst might be, perhaps some unpleasant surprise regarding QE tapering, or unconstrained collapse in the Chinese property market, both of which I think are pretty unlikely

There's a bunch of macro data out next week, and Apple is having its WWDC. Apple used to make up a very large amount of NDX, something like 24% of the index value was determined by AAPL prices. I know they rebalanced it and am not up to date with where it currently stands.

The case for data snooping

When we are backtesting automated trading systems, accidental data snooping or look forward errors are an easy mistake to make. The nature of the error in this context is making our predictions using the data we are trying to predict. Typically, it comes from a mistake with our calculations of time offsets somewhere.

However, it can be a useful tool. If we give our system perfect forward knowledge:

1) We establish an upper bound for performance.

2) We can get a quick read if something is worth pursuing further, and

3) It can help highlight other coding errors.

The first two are pretty closely related. If our wonderful model is built using the values it is trying to predict, and still performs no better than random guessing, it’s probably not worth the effort trying to salvage it.

The flip side is when it performs well, that will be as good as it will ever get.

There are two main ways it can help identifying errors. Firstly, if our subsequent testing on non-snooped data provides comparable performance, we probably have another look ahead bug lurking somewhere.

Secondly, things like having amazing accuracy yet still performing poorly is another sign of a bug lurking somewhere.

Example

I wanted to compare SVM models when trained with actual prices vs a series of log returns, using the rolling model code I put up earlier. As a baseline, I also added in a 200 day simple moving average model.

(S) Indicates snooped data

A few things strike me about this.

For the SMA system, peeking ahead by a day only provides a small increase in accuracy. Given the longer-term nature of the 200 day SMA this is probably to be expected.

For the SVM trained systems, the results are somewhat contradictory.

For the look forward models, training on price data had much lower accuracy than the log returns, and the log return model performed much better. Note that both could have achieved 100% accuracy by predicting its first column of training data.

However, when not snooping, the models trained on closing prices did much better than those trained on returns. I’m not 100% sure there isn’t still some bug lurking somewhere, but hey if the code was off it would’ve shown up in the forward tested results no?

Feel free to take a look and have a play around with the code, which is up here.

Building models over rolling time periods

Often I have some idea for a trading system that is of the form “does some particular aspect of the last n periods of data have any predictive use for subsequent periods.”

I generally like to work with nice units of time, such as 4 weeks or 6 months, rather than 30 or 126 days. It probably doesn’t make a meaningful difference in most cases, but it’s nice to have the option.

At first this seemed like something rollapply() from the zoo package could help with, but there are a number of preconditions that need to be met and frankly I find them to be a bit of a pain.

In a nutshell I have not been able to find a nice way for it to apply a function to a rolling subset of time series data nicely aligned to weekly or monthly boundaries.

All is not lost, there is a neat function in xts called endpoints(), which takes a time series and a period (e.g. “weeks”, “months”) and returns the indexes into that time series for the corresponding periods.

Using this information it becomes easy to subset the time set data using normal row subset operations.

The xts package also has period.apply but it runs on non-overlapping intervals, which is close but still not quite what I want.

In the script for this post there are 4 or so functions of note.

The main one is roll_model, which takes a set of data to be subsetted and passed to the model, the size of per model training and test sets and the period size to split things up, which is anything valid for use with endpoints().

A utility function is train_test_split which also uses endpoints() to split a subset of data into 2 sets, one for training the model, one for testing. In practice it needs to be the same period type as you expect to use with roll_model.

The function that actually builds the model and returns some results is run_model(), which calls train_test_split to get the training and test set, builds a model using ksvm in this example, and sees how it goes based on the test set.

Another utility function is called before that, data_prep which builds the main data object to be passed to roll_model. In this example it takes a set of log closes to close returns, sets Y to be the return at time t, X1 the log return at t-1, X2 at t-2 and so on.

The example model is not a particularly useful way of looking at things, which is not surprising given close to close returns are effectively random noise. But perhaps the script itself is useful for other ideas, and if anyone knows better/easier/alternate ways of doing the same thing I would love to hear about them.

The script is available here.

A volatility filter using historical vol

We have been looking at a way to improve risk adjusted returns by using a volatility filter. Although we could use VIX or equivalent, it turns out that historical volatility will work just as well, if not a little better.

You can see part 1 here Digging into the VIX, and part 2 here What can we use VIX for?

Although the mean return of how we slice things is zero, the distribution of returns is wider for higher readings of our relative measure of volatility. High volatility begets high volatility, at least for our purposes.

By staying out during periods of higher relative volatility, we aim to reduce drawdowns and the volatility of our returns, leading to better risk adjusted results.

A plus of using HV over some external measure like VIX is that it is readily available for any underlying. This means such a filtering technique can be applied to whatever it is we are trading.

Performance

Below is a table with two comparisons, the first compares the HV filter to buy and hold.  Although performance is generally better, we still get some pretty big drawdowns.

The second adds in a 200 period moving average, which is a reasonably strong way of protecting against downside. Again we can see lower volatility and smaller drawdowns with the addition of a vol filter.

I used a 3 month/63 day look back for our relative volatility measure. I haven’t really dug in to what happens when volatility remains elevated for extended periods of time.

I also did not experiment much with the threshold for where we draw the line on ‘high’ relative volatility. I use 0.6 as the cutoff because I originally split things into quintiles when making the first charts.

I also ran this for RUT and NDX over the same period

These results are all frictionless, don’t factor in dividends, return on cash, etc, etc. I don’t consider this viable as a standalone indicator, but something that can be used along side other factors like rotational strategies, or as a potential tool if you are looking for lower volatility.

The source is up here, feel free to have a play around with it and see how you go.

Thanks for reading, 'till next time.

What can we use the VIX for?

In part 1, we took a look at VIX and the relationship it had between historical volatility and realized volatility.

Continuing on, I thought I would take a look at next day returns and the VIX. There is a relationship between SPX and VIX in that when SPX drops, VIX typically rises. This leads the question: does a high VIX serve as a useful indicator of what might happen to SPX the next day?

A problem in answering this is quantifying what we mean by a "high" VIX. A VIX of 25 might be high if it has been under 20 for the last 3 months, but might be considered low if it has been around 30.

To deal with this I use a proportional measure based on its previous trading range. If it is at the top of its range, it gets a 1, and at the bottom it gets a 0. Anywhere in between it gets a number between 0 and 1.

I used data from 2000-2013 and a 252 period lookback, which is roughly 1 year, recording the next day return of SPX. I split this data into 5 groups, the first with relative VIX readings up to 0.2, the second between 0.2 and 0.4, and so on up to 1.0. Then I plotted the next day returns as a boxplot, giving the distribution of returns for a VIX reading in its relative quintile.

The box on the far left corresponds to the lowest relative VIX levels, while the box on the far right corresponds to the highest.

One thing that jumps out to me is that the mean of all these returns is more or less zero. Using this relative measure of VIX has no value as a predictor of next day returns, as whatever measure we use, a mean return of zero will see us end up flat (or in practice, probably down a bit). If there was a real edge, we would expect the mean return to be somewhere other than zero.

The second thing that I notice is the distribution of returns is a lot wider for the right most two boxes, which corresponds to relative VIX measure above 0.60, i.e. relatively higher volatility. Now this makes sense, as we would reasonably expect periods of higher volatility to have higher volatility.

It seems that high readings of VIX can serve as a useful predictor of higher periods of next day volatility, and for someone trying to implement a volatility filter, that could be quite useful.

Conclusion

To recap, we have seen that:

1) VIX is not very good at forecasting 30 day realized volatility.
2) Historical volatility can be a reasonable proxy for IV/VIX.
3) Higher periods of volatility do portend continued higher volatility, at least for the short term (1 day).

Note that 3 does contradict the mean reversion of volatility, because it is looking at the next day returns only. Over a longer period we would certainly expect it mean revert, otherwise we'd be experiencing some very, very bumpy rides ...

Also, I did similar analysis using RUT & RVX and NDX & VXN, both of which had very similar results. We will take a look at them in the next post where we use the findings from this post and the previous to implement a volatility filter.

Code for this post and the previous is up: here

Digging into the VIX

I wanted to revisit using some sort of volatility filter for systematic trading. In particular, if we are trading SPX, can we somehow use the VIX to produce better risk adjust returns? This is not about trading volatility, but more about using additional "out of band" data in our systems.

I thought I would take a look at the VIX, what is it, and can it help us?

In theory, the VIX is the market consensus of what future 30 day realized volatility of SPX will actually be, as derived from option prices of SPX. Anyone who watches or trades SPX has seen that a drop in SPX usually results a rise in VIX, or the IV in SPX option chains.

Occasionally this relationship breaks down, which can be a useful as an indicator in itself, but I had a sneaking suspicion that the VIX would have more in common with historical vol than what realized vol turns out to be over the next 30 calendar days. That is, the VIX is not particularly useful as a longer term forward looking indicator of volatility, at least no more than current historical volatility.

I did a bit of data gathering and calculated realized and historical volatility for SPX. I used 21 periods when calculating vols as the VIX covers 30 calendar days, which is roughly a month, and there are roughly 21 trading days in an average month.

First of all, we can take a look in hindsight at how well the VIX forecast of volatility matched the actual realized volatility over the next 21 days.

In this chart, the red line is what realized volatility turned out to be, going forward from a given day. It is impossible to know this in advance, so we are engaging in some serious data snooping, for educational purposes only. The teal line is the VIX. Below them is a plot of VIX - SPX realized vol, and flagrant abuse of gradients.

You can see that the VIX gives an estimate that is typically higher than what actually eventuated, a well-known phenomenon, which for convenience I will put in "The Volatility Risk Premium" basket.

There is one big divergence around the end of July, 2011, when the VIX was significantly underestimating what realized vol would actually turn out to be.  In August 2011 there was a large selloff and you can see the VIX jumped to a rather ebullient level.

If we moved the red line forward 21 periods, we would get historical volatility, that is, the actually volatility that occurred over the last 21 trading periods.

We can see the relationship is much closer. The VIX still generally reflects a premium over historical volatility, but big moves in the underlying SPX correlate to big moves in VIX/IV.

Relations

The correlations between these three measures, IV, RV and HV are worth taking a look at. This is using data from January 2000 up to about the end of January 2013:

There is a 91% correlation between historical volatility and VIX. This is higher than the correlation between HV and RV, and HV is really just RV shifted back 21 periods. It seems the relation between VIX and HV is something quite strong.

Now because these are all measures of very similar things, we might expect high correlation. Another metric we could use too look at the relationship is R-Squared.

We can see a relatively strong relationship between historical vol and IV/VIX, at 0.60. The relationship between IV and realized vol is looking a lot weaker.

Conclusion

From all of this, I concluded that the VIX is not a particularly useful indicator of 30 day future realized volatility. However, it is a reasonable estimate of what historical volatility was.

As it turns out, this is not particularly useful, as we can easily calculate historical volatility. It does mean that we can flip things around, and use historical volatility as a reasonable proxy for VIX or market expectations of future realized volatility.

This will turn out to be quite useful, as we may find ourselves trading instruments that do not have nice, readily available VIX equivalents, but that is a topic for a subsequent post.

So ends part 1. These posts turned out very long so I decided to split them up, part 2 should be available here.

The R code for this and the next post is up here.