Back-testing Studies on the FactSet Short-term Risk Model During the COVID-19 Outbreak

Ivan Mitov PhD, Viviana Vieli, and Velislav Bodurov also contributed to this article.

It became a common theme post-2008 that VaR is not able to capture extreme events. This is only true if VaR is assessed based on a model of “Category 1” as described in Part 1 and referred throughout this series, i.e., models based on the normality or historical assumptions that believe that the future can only repeat history and/or the probability of extreme events dies out to zero with exponential speed. VaR based on models that incorporate time-varying probability of disasters (“the Noah effect”) can provide a statistically sound characterization of potentially large losses that is meaningful for investment management [1], [2].

Statistical Relevance

Below is a comparison of the one-day 99% VaR exceedances for the Normal model (with volatility clustering via GARCH) and the FactSet Fat-Tail model since the start of 2018 and YTD.

The tables signify that the FactSet Fat-Tail model predicts short-term VaR in a statistically valid way and that the number of breaches (when real loss exceeds the predicted VaR) is in line with the confidence level of the selected VaR estimate (i.e., 99%). On the opposite, the more myopic approach significantly underestimates the risk, both since the start of the period and the COVID-19 crisis unraveling.

We have just about 60 observations since the beginning of this year. A 99% VaR exceedance should happen in 0.6 days on average (i.e., the mean expected value of the VaR exceedances is 60*1%). This is not possible to happen in a precise manner. We know that with some high level of confidence that the model provides a statistically correct representation of reality if the exceedances are anywhere between zero and three for that short period of time (since the beginning of this year). If the number of exceedances is higher than the upper bound of three, the model is underestimating the expected loss via its VaR predictions. The longer the backtest period becomes, the closer that bound tights-up around the mean expected value of the exceedances. When the period is long enough, the lower bound becomes strictly positive and we can judge if the model is overly pessimistic as well. We do not observe this here for either of the models.

Fat-Tail Indicator

Shortly after the 2008 crisis, the industry expanded on the concept of VaR and borrowed from academia the notion of Conditional VaR, also called Expected Shortfall or Expected Tail Risk (ETL)[1]. Speaking of extremes in the context of portfolio management we could similarly derive important insight on the upside—the Expected Tail Return (ETR)[2].

The Fat-Tail Indicator is a function of ETL and ETR and represents both tail-fatness and skewness:

99% VaR and Fat-Tail Indicator - United States

In Figure 1 below, we depict (from top to bottom subplot) the following for the period January 1, 2018, through March 6, 2020: the cumulative performance of the U.S. stock market; stock market returns together with the Normal and FactSet Fat-Tail model 99% VaR measures: stock market return standard deviation and the FactSet Fat-Tail Indicator for the U.S. equity market.

Three observations are worth noticing. First, the recent market downturn has (unsurprisingly) led to a spike in the realized return volatility while the Fat-Tail Indicator is sustained at its already elevated level. Second, as discussed above, the FactSet Fat-Tail VaR has statically strong predictive power with 11 exceedances YTD and one counted since January 1, 2018. Third, and more interesting, the Fat-Tail Indicator has sustained an elevated level (i.e., above 50 percent) for almost the entirety of the last two years. The model has therefore been gradually “pricing in” increasing turbulence via individual news (i.e. trade deal negotiations, political developments, etc.) and market and economic disturbances that have been contributing to the build-up of “turbulence pressure”—a phenomenon that is not apparent through the lens of traditional volatility metrics. This is most clear in the second half of 2019 when the Fat-Tail Indicator records a significant increase and then maintains that elevated level prior to the spike in realized volatility in the subsequent equity market downturn of early 2020. That way, the model was already operating at an elevated expectancy of extremes and it thus provided a significantly better real-time assessment of the underlying market risk compared to more traditional risk measures during the COVID-19 financial markets collapse. The period in the beginning of the year corresponds to False-safety period/Pre-stress period as defined in Table 2; the period of the first three weeks of March to Developing Crisis period; and the last week of March to Developed Crisis period where we see the Fat-Tail Indicator starting to go down despite the extreme values of Vol.

Figure 1: MSCI USA Performance, Volatility, and Fat-Tail Indicator

99% VaR and Fat-Tail Indicator - Eurozone: Germany, France, & Italy

In the charts that follow below, we depict the same analysis for three Eurozone countries that have unfortunately been significantly affected by the virus outbreak: Germany (Figure 2), France (Figure 3), and Italy (Figure 4). Even though the extent of the outbreak differs in its intensity in the different countries, the common patterns are clear: a significant equity market downturn in the most recent period, an associated spike in realized market volatility after a two-year period of relative volatility staying calm on average, and a Fat-Tail Indicator that has been gathering positive momentum since at least early 2019. The increasing profile of the Fat-Tail Indicator appears to have been particularly stable since the second half of 2019 with one potential hypothesis for this behavior related to the growing concerns of developing recessionary dynamics in Germany as the common bloc’s largest economy, and a more widespread domestic and global economic slowdown. Similar to dynamics observed in the U.S. stock market, the FactSet Fat-Tail Indicator seems to have provided a better starting point for assessing the underlying equity market dynamics in light of the subsequent significant market moves and realized volatility spike that followed. Its further rapid increase in March is a signal for a potential transition from extreme events environment to a structural break-type setting (see Table 1 in Part I - Introduction). The three economies react similarly as measured but the Fat-Tail indicator at its peak is different and potentially justified by the severity of the crisis impact. The start of the flattening of the Indicator over the last week signifies that the crisis has reached a local plateau.

Figure 2: MSCI Germany Performance, Volatility, and Fat-Tail Indicator

Figure 3: MSCI France Performance, Volatility, and Fat-Tail Indicator

Figure 4: MSCI Italy Performance, Volatility, and Fat-Tail Indicator

99% VaR and Fat-Tail Indicator - Japan

Japan has also been significantly affected by the virus outbreak in Asia. Unsurprisingly, the local stock market has followed the global trend and suffered a significant fall while realized market volatility spiked. It is worth noting that, unlike what is observed in other markets over the same time period, the Fat-Tail Indicator stayed relatively stable and only just above the sligthly elevated levels of 55%-60%. Interestingly enough, the period of declining realized equity market volatility in Japan in the second half of 2019 —at least to an extent related to the more accomodative stance of the Bank of Japan—was not mirrored in the behavior of the Fat-Tail Indicator that remained stable and in a moderate “surprise-anticipation” mode.

Figure 5: MSCI Japan Performance, Volatility, and Fat-Tail Indicator

Fat-Tail Indices and Vol Comparison

It is worthwhile observing that the Fat-Tail indicator has different dynamics from that of Vol and in certain instances, acts as a predictor for a soon-forthcoming increase in volatility.

Below are the scatterplots of the daily changes of Vol versus daily changes of the Fat-Tail Indicator and the respective classical and robust exponentially-smoothed correlations for each market for the full period under consideration.

Figure 6: Fat-Tail Indicator vs. Volatility Scatter Plot

Table 3: Classical and Robust Correlations Between Fat-Tail Indicator vs. Volatility Changes

The scatterplots show highly non-elliptic structure of dependence, further demonstrated by the significant differences between Classical and Robust correlations.

A deeper analysis would reveal four clusters with different correlation patterns between the two measures corresponding to (1) the “central” business-as-usual days (little correlation), (2) pre-stress period (mild correlation), (3) developing crisis (high correlation), and the (4) developed crisis (low to negative correlation).

Figure 7: MSCI USA Fat-Tail Indicator vs. Volatility Scatter Plot

This brief review underscores the fact that the Fat-Tail indicator is a characteristic adding an additional and independent insight when contrasted with the volatility of a given market.                              

FactSet Short-Term Model and Implications for Investment Management

That FactSet Short-Term Fat-Tail MAC model available in our Portfolio Analytics suite features the characteristics described above and provides its users with the ability to monitor the differences between Fat-Tail and Normal risk as measured by VaR, ET,L, and ETR daily, gaining some predictability for the market regime changes.

What is the applicability of these sets of risk analytics for investment management?

1. As demonstrated above, for regulatory purposes, the Fat-Tail model provides a solid approach capable of satisfying the VaR-exceedances requirement and thus, builds trust in the organization’s risk management processes and systems.
2. The short-term one-day VaR model would, in any event, be too volatile in such highly turbulent environments and thus, is not particularly useful for portfolio management. However, the implied trends of the Fat-Tail Indicator versus Vol can be a useful signal for possible asset-class reallocations or for the construction of hedges.
3. Finally, a certain degree of de-risking (albeit more limited) could happen within an asset class as well via optimization. Below is a comparison of a broad universe of U.S. stocks represented by a widely used market index and minimum volatility portfolio based on the same universe. We have constructed a universe closely mimicking the index and executed min ETL optimization with the same constraints. Details of the set-up are in the Appendix. The ETL type optimization consistently provides a higher cumulative return, lower drawdown, and lower standard deviation compared to minimum Vol approach and thus can be a useful approach for tactical rebalancing within a selected investment universe.

What follows is a comparison between a broad universe of U.S. stocks represented by a widely used market index (MSCI US) and a minimum volatility portfolio based on the same universe. We have constructed a universe closely mimicking the original index and executed minimum ETL optimization [3] with the same constraints[3] as in the minimum volatility portfolio. The important difference is that the min Expected Tails Loss optimization process is based on a model accounting for both the Joseph and Noah effects. The ETL type optimization consistently provides higher cumulative returns, lower drawdowns, and lower standard deviation compared to min Vol approach and thus can be a useful approach for tactical rebalancing within a selected investment universe.

Minimum Tail Risk Portfolios

The Minimum ETL Portfolio shown below is constructed by optimizing over a broad universe of U.S. stocks. The optimization objective is to minimize the Expected Tail Loss at the 99% confidence level. The portfolio weights are recalculated on the last business day of May and November each year, beginning on May 31, 2006.

At each step, the optimization is subject to the following constraints:

• The maximum weight of a single asset in the portfolio is constrained by the smaller of 2% and 20x the weight in the investment universe
• The maximum weight of a single asset in the portfolio is 0.05%
• The maximum one-way turnover for each portfolio recalculation is 10% (if the optimization problem is unfeasible, this constraint is relaxed in 5% increments up to a maximum of 30%)
• Sector weights in the portfolio are constrained within +/- 5% of the corresponding sector weight in the investment universe
• Country weights in the portfolio are constrained within +/- 5% of the corresponding country weight in the investment universe
• Factor exposures are constrained within a +/- 0.25 standard deviation relative to the investment universe, except for the Beta and Volatility factors that are not constrained (factor exposures are according to the FactSet Global Equity Model[4])
• Model is fit on daily data with monthly horizon and rebalancing is done once every six months

Figure 8: Cumulative Returns, Full Period

Figure 9: Cumulative Returns, YTD

Table 4 A-H: Performance Statistics

Previously in the Series:

Be Aware, Adapt, Innovate--or What We Learn from Mandelbrot, Noah, and COVID-19
By Dr. B. Racheva-Iotova

Next in the Series:

Part 2: Constructing Semi-Subjective Scenarios for Possible Recovery and Assessing the Scenarios’ Plausibility Based on Monte Carlo Risk Models
By Georgi Mitov PhD, Emil Margaritov PhD, Shamin Parikh, Dr. B. Racheva-Iotova

Part 3: Stress-testing and quantitative assessment of the probability of subjective stress scenarios
By Georgi Mitov PhD, Emil Margaritov PhD, Shamin Parikh, Dr. B. Racheva-Iotova

Part 4: Innovation: Multi-period optimization for long-term allocations incorporating stress-tests and investment objectives
By Georgi Mitov PhD, Ivo Stefanov, Chen Sui

Part 5: Minimum Tail Risk Smart-Beta
By Georgi Mitov PhD, Nikolay Radev, James Egginton

References

1. B. Racheva-Iotova Et Al. FactSet Fat-Tail Multi-Asset Class Model, Introducing Turbulence Adjusted Risk. White-paper, FactSet, 2018
2. B. Racheva-Iotova Et Al. FactSet Fat-Tail Multi-Asset Class Model, Theoretical Foundations. White-paper, FactSet, 2018
3. Todor Bilarev Et Al. Multi-Period Optimization, White-paper, FactSet, 2019

[1] ETL at a given confidence level is defined as the average loss if the loss is higher than the VaR value at the selected confidence level.

[2] ETR at a given confidence level is defined as the average gain if the gain is higher that the quantile at the selected confidence level.

[3] FactSet ETL Optimization module is expected to be officially launched June 2020

[4] FactSet Global Equity Model is expected to be officially launched September 2020