This article explores six essential lessons from Dr. Ernest Chan's influential book, "Machine Trading," which has significantly impacted many traders' approaches to the markets. The book challenges the common belief that complexity is the key to profitable trading and offers a different perspective on how to succeed in the financial markets.
The Credibility of Dr. Ernest Chan
Before diving into the lessons, it's important to understand the author's background. Dr. Ernest Chan is a highly credible quantitative trader. He began his career at IBM Research, focusing on pattern recognition, and then moved to Wall Street, working for Morgan Stanley and Credit Suisse. There, he developed systematic trading strategies. He later founded QTS Capital Management and PredictNow.ai, where he helps investors make data-driven decisions using AI and machine learning. His books have influenced numerous traders, including the author of this article.
Lesson 1: Start Simple
Counterintuitive Simplicity
It seems counterintuitive to start with simple techniques when competing with large firms using cutting-edge technology. Many retail traders mistakenly believe they need complex indicators and overfitted models to gain an edge. However, Dr. Chan emphasizes the importance of starting with the simplest techniques before moving on to more complex ones.
Real-World Application
The author's own portfolio of automated strategies, running on futures and crypto markets, are based on incredibly simple techniques. These strategies primarily extract risk premiums and inefficiencies that larger players can't exploit due to illiquidity or insufficient size, making it a profitable niche for retail traders.
Ernest Chan's Perspective
Dr. Chan believes that a simple, basic model with five parameters or fewer is best. When models become too complex, traders lose intuition behind it.
Lesson 2: Each Parameter Must Fight for Its Place
Parameter Significance
Dr. Chan warns against using excessive parameters, as they can easily lead to overfitting. Each parameter should demonstrate statistical significance and serve a clear purpose, whether it's generating returns or controlling risk.
Parameter Sensitivity
A useful method for evaluating parameters is parameter sensitivity analysis. This involves checking parameter values against a performance metric, often visualized on a heat map or 3D surface graph. This analysis helps determine how changes in parameter values impact performance.
Identifying Overfitting
If a small change in a parameter value causes a drastic drop in performance, it's likely an overfitted strategy. A slow, gradual decay in performance suggests a more robust strategy less susceptible to noise.
Lesson 3: Diversification is Key
The Goal Isn't One Perfect Strategy
Instead of searching for the perfect strategy, Chan advocates for diversification. Maintaining multiple, independent strategies reduces the risk of all strategies failing simultaneously. Big firms are constantly researching new ideas. The same applies to retail traders, as well.
Risk-Adjusted Returns
Running multiple uncorrelated strategies improves overall risk-adjusted returns. This results in less drawdown and higher returns for a given level of risk. Low correlation or uncorrelated strategies are considered the “Holy Grail” of trading.
Trading with a Single Strategy
While ideal to have multiple strategies, it's acceptable to begin trading with a single strategy, but with lowered leverage. Diversification is the key to reducing risk of an underperforming strategy.
Lesson 4: Focus on Inefficient Markets
Opportunities in Inefficiency
Newer or less efficient markets offer better opportunities for profit. These markets may have other risks, such as less clear infrastructure and regulation, but they often contain more alpha.
Examples in Crypto
Cross-exchange arbitrage, once common in crypto, is a prime example. While less prevalent now due to market maturation, opportunities still exist with new tokens and market segments, albeit with the risk of less regulated or secure exchanges.
Lesson 5: Why Most Machine Learning Trading Fails
The Limitations of Machine Learning
While many developers are drawn to using machine learning in trading, it's important to understand its limitations. Financial data is often limited and non-stationary, leading to overfitting.
Overcoming Overfitting
Machine learning can be useful for risk allocation and portfolio optimization. It's important to avoid overfitting by being wary of optimizing parameter values using machine learning.
Conditional Portfolio Optimization
Machine learning can be used to determine the probability of profit for a strategy on a given day, and can be used to allocate capital based on the market and economic environment. Instead of equal allocation, the strategy enables updating allocation regularly based on which strategy is expected to perform best.
Lesson 6: Continuous Research
Strategies Have Finite Lives
Strategies have finite lifespans, and poorly backtested ones may not work at all. Rapid prototyping, thorough testing, and discarding failures quickly are essential.
Accept Failure and Adapt
Don't try to fix a failing strategy. Accept that trading is about continuous research, testing, and adaptation. Remember, no strategy works forever.