Streaming OANDA with python and ZeroMQ

I have been looking at its REST API for OANDA, for potential use with an FX trading system I developed.

The API has two streaming endpoints, one for prices and one for account events such as trades opening and stuff like that.  

Asynchronous IO is always a bit fiddly, and I wanted separate processes for incoming tick data and events. This enables them to be managed separately, and generally makes handling disconnects or other errors a bit cleaner.

If you try and mash your various feeds, trade identification, trade management, logging, accounting etc all into one big process it gets a bit messy and convoluted. Sometimes you don’t have a choice due to the API available, but thankfully in this case it’s easy to separate everything out.

Details

I use the requests library for streaming from OANDA, and ZeroMQ endpoints with the pub/sub pattern for passing data to the client(s).

Conceptually, it looks like this:

The prices feed connects to the OANDA prices endpoint, and publishes what it receives to a ZeroMQ socket. Likewise, the events feed does the same, publishing to another zmq socket.

The client connects to the respective sockets and subscribes to its feeds, and can do whatever is required with the data it receives. You are not limited to one client subscriber either.

I use JSON as the serialization format, as it arrives from OANDA. No real point in deserializing JSON to an object, then immediately serializing that with pickle or something for transport with zeromq. Your needs may differ of course.  

All up, a pretty painless process.

Code

The single file of code (here) is the prices feed, events feed and client that just prints out what it receives. Just run it separately three times, passing either prices, events or client as a single argument. You will also need to fill in your OANDA details as well.

I use a 10 second timeout for the prices feed, and a 20 second timeout for the events feed, as per OANDAs recommendations. The feed streams will print heartbeats but not publish them to clients.

Thanks for reading and let me know how you go.

Equity Ranking Backtest with Python/Pandas

I have been look at equities a bit of late, I am particularly interested in ranking a universe of equities for “low frequency” manual trading on a weekly or monthly basis.

Every period I would rank each name on a bunch of different factors, then invest in the highest ranked ones for that month.

I was initially working in R but the code grew unwieldy, and I wanted a second opinion on my approach so took the time to re implement it in python using Pandas.

Setup

For each symbol in our universe, we load the raw data and generate the information used for ranking. If we have 5 names, we end up with 5 dataframes.

Then we combine those dataframes into one big dataframe, and iterate through month by month, selecting the symbols that meet our ranking criteria. From those selected, we equally weight and sum the next period returns.

One thing that is really cool about the pandas dataframe is that it allows multiple rows with the same index.

This makes it easy to get the data for the month under consideration. We just pass the month to index function and get the subset of data for that month, e.g.

>>> df.ix['2015-02']

                 cpr       npr       avg   over  sym

Date                                               

2015-02-28  0.043302 -0.062449 -0.038914  False  DBC

2015-02-28 -0.025028  0.008524  0.006130   True  IEF

2015-02-28  0.056838 -0.014239  0.005434   True  VEU

2015-02-28 -0.037434  0.017171  0.015900   True  VNQ

2015-02-28  0.055832 -0.011697  0.009236   True  VTI

[5 rows x 5 columns]

>>> 

In this example there are 5 symbols, and we see the ranking information for February 2015.

Another option would be to use hierarchical indexing, with a sub-index for each month, but this way worked for my needs and I think is quite clean and simple.

If anyone knows an equivalent in R that is as clean and easy to work with for multiple time series I would love to hear about it. 

Code Notes

The demo code does a simple back test of the GTAA/Relative Strength trend following system using ETFs.  

I have stripped it down to the basics so hopefully it is easy to understand. Load the data, generate the dataframe with the info we want, make a combined data frame, then go through month by month.

The ranking is done by filtering out names under their 10 month moving average, then selecting the top n based on average 3 month return.

The “cpr” column is the current period return, and the “npr” column is the next period return, which is the return realized if we select a given security for that month.

The data is just ETF data from Yahoo, which I have put up here. Code is here.

I found Python For Data Analysis a very useful book is when working with pandas.

Machine Learning Examples in R

This is a post that has been a long time in the making. Following on from the excellent Stanford Machine Learning Course I have made examples of the main algorithms covered in R.

We have Linear Regression

Followed by Neural Networks

And Support Vector Machines

One remaining item is Logistic Regression, I am yet to find a library in R that behaves as I want, so that will come at some future date. I've been sitting on this post for ages and got sick of waiting. As an aside I find the documentation in R to be variable at best, which can make it somewhat of a pain to work with. When it is good, yes it can be very good but often it is quite poor ...

R is great for data analysis and exploration, but I have found myself moving back to python for many more mundane tasks.

Anyway for those interested in the code, I have put it on Github. The data is from an exercise in the Stanford course, and by tweaking the parameters I really got a good feel for how the various algorithms work in practise.

Once I finish my backtesting engine I will probably put it up on Github as well, and then I can start digging into the applications of ML techniques for trading systems.