Understanding Regime Changes for Robustness in Backtesting

In backtests, understanding how your factor signals perform in various regimes is crucial. The reason is market conditions continually evolve, so a factor signal that appears to be robust at one point in time may prove to be less effective within a different regime. In this article, we analzye the positives and negatives of extending your backtesting period.

We selected five factor signals: value, size, market sensitivity, profitability, and momentum. Using constituents of the Russell 3000 Index, we then analyzed their performance over different periods within the past 20 years. The factor returns are represented by the spread returns of the top factor score bucket portfolio, less the bottom factor score bucket portfolio.

The backtest class offered within the FactSet Programmatic Environment (FPE) enables you to effortlessly analyze factor performance across differing time periods. You can examine and compare the effectiveness of factor signals during varying economic or market scenarios. Doing so can provide insights into potential performance improvements and strategies to fine tune your approach.

Performance Across All Periods

We start by examining the performance across all periods. The two charts below illustrate that the profitability factor has the strongest performance both in terms of cumulative returns and information coefficient.

From the data in the metrics table below, we observe that the profitability factor has the highest alpha and Sharpe ratio, while the sensitivity factor generates negative alpha.

Momentum was found to have the most volatility and the highest maximum drawdown, suggesting higher overall risk despite its positive alpha.

Subperiod Analysis

Next, we compare the factor performance during specific market downturns over the past 20 years, specifically during the Financial Crisis (2007-2009) and COVID-19 pandemic (2020-2021).

In the financial crisis period, the profitability factor once again displayed the highest performance. This is evident from positive alpha, high Sharpe ratios, and the highest average returns. The value factor also reveals robust performance.

During the COVID period, however, the profitability factor experienced a downturn. Conversely, the market sensitivity factor outperformed the other factors. That is evident from the highest alpha, average returns, and Sharpe ratio along with the lowest maximum drawdown, thereby signifying the least historical decline.

Risk-Level Analysis

We then shift our focus to examine how our factor signals perform in line with the level of VIX, in turn analyzing the factor returns across different regimes of market volatility. We began by regressing the factor returns on the level of VIX over the backtest period to identify factors that are most sensitive to changes in market volatility.

From this preliminary analysis, we revealed that market sensitivity and profitability factors demonstrated statistical significance. Therefore, we delve into metrics for those factors when the VIX is low, medium, and high.

Sensitivity

Low VIX environment: A lower Sharpe ratio (0.132) and higher max drawdown (0.293) indicate the market sensitivity factor tends to perform relatively poorly during this period with lower risk-adjusted returns and higher peak-to-trough declines. The negative alpha (-0.001), although not statistically significant, suggests underperformance relative to the market benchmark.

Medium VIX environment: During the periods of medium market volatility, the performance of the market factor deteriorated markedly, as indicated by the negative Sharpe ratio (-0.827) and the relatively high max drawdown (0.323). However, the positive, statistically significant alpha (0.013) suggests this factor generates excess returns over the benchmark, despite higher risk.

High VIX environment: During periods of high volatility, the market sensitivity factor improved with a Sharpe ratio of 0.210 and a relatively lower max drawdown at 0.261. The negative alpha showed signs of benchmark underperformance, but the corresponding t-stat is not statistically significant.

Profitability

Low VIX environment: The profitability factor performed well in low VIX environments, with a higher Sharpe ratio of 0.560 and a lower max drawdown (0.151). Positive, statistically insignificant alpha suggests the factor generates slightly better risk-adjusted returns than the benchmark.

Medium VIX environment: Financial performance across metrics such as Sharpe ratio and alpha improved when market volatility was medium. High, statistically significant positive alpha indicates the profitability factor outperforms the benchmark during this period.

High VIX environment: The factor still performed relatively well during high volatility, with a solid Sharpe ratio (0.451) and a lower drawn-down level (0.221). Positive, statistically significant alpha indicates the strategy could generate considerable excess returns in highly volatile market conditions.

You can repeat the same exercise for other risk indicators such as interest rates, inflation rates, or oil prices.

The following charts show the performance of factor returns at different CPI levels, specifically among factors that exhibited statistical significance (value and profitability).

Conclusion

In this article we explored how factor performance can vary in different market conditions. While extending the backtesting period can potentially enhance the strategy's robustness by increasing data points, beware that it may include various market regimes. Since each investing period has unique characteristics and challenges, it is essential to understand the specific time period you’re dealing with and be mindful of the regime shifts.

This blog post is for informational purposes only. The information contained in this blog post is not legal, tax, or investment advice. FactSet does not endorse or recommend any investments and assumes no liability for any consequence relating directly or indirectly to any action or inaction taken based on the information contained in this article.