How Effective Are Trading Bots Really?
A Review of Gekko
Trading bots are widely available programs that connect to a user’s cryptocurrency exchange and make trades on their behalf. They work using a variety of indicators and signals, such as moving averages and indices. The idea is simple: to help users make money in the markets, while not wasting a lot of their time. The underlying assumption is that computers are much better at trading than humans could ever be, because trading is all about mathematics and complex calculations of probability.
While this may or may not be the case, it is very difficult to find actual performance reviews of trading software, and anecdotal evidence seems to suggest that most traders either lose money, or do not outperform the markets.This is particularly true in the crypto-space, where there are no easily accessible vehicles to short the markets, making price increases the only viable option.
The extent to which mutual fund managers, underperform is very well documented, and often underappreciated. In several performance reviews by Standard & Poor, it was concluded that between 92% and 94% of actively managed mutual funds generated lower returns over a 15-year timespan than the S&P 500 index. This does not factor in the funds that failed over that time, making the actual figure even worse. From what we have seen, trading probably follows a similar pattern in which very few experts make a lot of money, and most people lose out.
To the best of our knowledge, no satisfactory studies and analyses exist for trading bots in the crypto space, nor for the average performance of day traders that try their luck in the blockchain markets. This is exactly why we are starting this series on trading and performance reviews.
Trading Algorithms
The suspicious lack of actual data supporting one trading algorithm or another, or delivering substantial evidence that these automated trading bots are able to outperform the markets could be due to several reasons:
1) They generally do not outperform the market, and it is against the interest of the trading bot industry to disclose this information.
2) Trading bot makers fear legal reprisals if they make concrete claims
3) Trading bots do outperform the market, and it is in the interest of users to keep their exact performance a secret, such that they may keep their edge.
4) They generally do about as well as the market.
The likeliest answers, in our estimation, is that trading bots, in general, either do not outperform the market, or perform about as well as the market. This does not necessarily explain the frustrating lack of statistical data, or performance analysis, but we must consider that the cryptocurrency trading phenomenon is relatively new. Many crypto traders are either not professional, or lack the analytical toolset, or motivation to report on their findings. Since the crypto markets have been, on the whole, extremely profitable even for amateur traders, it is not too surprising that no one has scrutinized the exact numbers, because the field has been dominated by gamblers and not professionals.
This is not to say that algorithmic trading is, on the whole, suspect. Arbitrage trading is something that computers have a distinct advantage at, because they can quickly detect the smallest of differences in the price of trading pairs, and continually scan through thousands of such prices. We do not claim that there is no possible successful trading algorithm, but suggest caution and skeptical inquiry.
Gekko — “I’m going to make you rich, Bud Fox”
Named after Gordon Gekko, the antagonist of the 1987 Wall Street movie, Gekko is an open source, free trading bot that is compatible with many exchanges and is controlled via web interface. Users have a variety of trading strategies to pick from, such as MACD, a common strategy using moving average signals to decide on whether to buy or sell.
The setup is straightforward and the user interface simple, intuitive and to the point. After connecting to Bitfinex via API key, we can set up the out-of-the-box MACD strategy to trade USD/BTC using a relatively short candle time-frames. Starting with $150 USD in our account, the bot set about collecting market data for about two hours before starting to trade. In our enthusiasm we decided to drive right into the markets, where our optimism quickly turned into disappointment.
After only a few hours, the live experiment was stopped because something was clearly not working. Gekko was consistently losing money, because it bought and sold with horrible timing as can be seen in the below chart. Buying high and selling low, was the overall result:
In a span of only a few hours, we lost 2.72% of our money. In comparison, during that same time, the Bitcoin market price experienced a 0.15% growth.
Backtesting
At this point we agreed that serious revaluation was in order. Heading straight into the market with an out-of-the-box strategy was clearly not working very well. Thankfully, Gekko has a great back-testing feature, that lets users download historical data, and then run their strategies on it. After running every single ready-made strategy that Gekko offers on the same piece of market data, a clear pattern emerged.
We decided to test every available out-of-the-box Gekko strategy over a six months time frame, in which the market itself experienced both periods of growth and decline, and then compared our month-to-month results to the buy-and-hold strategy. We also incorporated a neural network based approach developed by the community and available on GitHub.
On one minute candles, Gekko performs extraordinarily badly, losing 99% of our money with almost every approach, regardless of whether markets are up or down. There is often a more than 100% difference between market result and trading result.
On an hourly interval the algorithms do much better, with some months of profits and some periods of outperforming the markets. However, on average, all of them are worse than the buy-and-hold strategy. In fact, the times in which they do well seem random and unrelated to up or down trends in the market. This confirms our hypothesis that trading bots generally either do not outperform the market, or do about as well.
In this monthly breakdown we can see that most of the algorithms sometimes outperform the markets, but none of them do so consistently. In fact the standard MACD approach we chose in the very beginning almost never beats buy-and-hold.
Verdict
Gekko has excellent design, and ease of use, but it is clear that it’s cookie-cutter strategies do not work. It does warn users with standard disclaimers, and it should be understood that the risk is certainly with the trader, and losses cannot be the responsibility of the Gekko developers.
There is a community of Gekko enthusiasts online which share custom built strategies and discuss the pros and cons of various indicators and settings, and there may be strategies out there that could genuinely be profitable. Some of these user builds employ neural networks and deep learning techniques in order to gain deeper insight into the markets, but of all the strategies we have tested (including neural network ones), none could consistently outperform the hodl approach, and all of them lost money on a regular basis. While some backtests, with certain settings on certain data were profitable, applying those same settings to other data, or different time frames would result in disappointment.
We therefore come to the conclusion that the Gekko trading bot is not a worthwhile tool to trade on your behalf. Even if there are sophisticated strategies out there that may or may not work, most people trying to generate passive income, and have entered the crypto markets in hopes of huge returns, neither have a full understanding of technical analysis, nor time to delve into the intricacies of custom strategy building and backtesting. If they did, they would hardly need Gekko for passive income generation, and would simply be full time traders.