Navigating Investment Uncertainty with Stochastic Optimization

Uncertainty is a fundamental risk factor in every real-world process, even when you meticulously devise models under well-structured conditions. For instance, you could base a stock portfolio on historical performance with the intention to maximize returns and minimize cost. However, adding uncertainty to the mix dramatically increases the complexity of your portfolio model, such as in the following situations.

• What happens if you want to lower the chance of a drawdown exceeding 5%?

• What's the most effective way to model stock returns as probability distributions instead of using only expectation and variance?

• What if you want your optimization to consider market shock events that could happen with a certain probability?

• How do you account for the fact that macro forces in an industry or operating environment can impact security prices?

In practice, these issues are typically simplified with manageable assumptions, sidestepped with sound heuristics, or examined through Monte Carlo simulations. However, these methods often break the problem into independent sections rather than optimizing over all considerations simultaneously. The latter approach provably generates equal and typically better solutions because it considers the entire “family” of random variables.

Managing Uncertainty with Stochastic Optimization

Recent advancements in efficient solvers for uncertainty bring life back to a time-tested tool: stochastic optimization. What exactly is stochastic optimization, and why should you examine it?

Optimization enables discovery of solutions that maximize or minimize a specific objective and is already at the core of many investment, business strategy, and engineering challenges. At its most basic, optimization involves pinpointing the optimal value of one or more decision variables within a well-defined problem, typically characterized by an objective function and a set of constraints.

Objective functions assess the quality of possible solutions, while constraints delineate the feasible range of decisions. Historically, mathematical techniques such as linear programming, nonlinear programming, and integer programming have solved deterministic optimization problems in which all parameters are fixed and known in advance.

Let’s take a look at a simple deterministic formulation of such a problem. Here you have a collection of n stocks s, where each stock has a cost c and an expected profit p. Each stock s is a binary variable, which is set to 1 if the stock is included in the portfolio. Therefore, your objective is to maximize your expected profit while spending no more than $100.

While deterministic optimization offers a potent tool for decision-making, many real-world problems involve uncertainties or random variables. Stochastic optimization steps in here.

Because of its ability to account for random variables, probability distributions, and expected outcomes, stochastic optimization makes it possible to tackle a wide range of complex problems. By considering the probability distribution of uncertain parameters, stochastic optimization enables decision-makers to identify solutions for a range of scenarios—as well as to evaluate risk and adopt a proactive approach to problem-solving.

In the deterministic example above, the formulation would simply change the expected profit p to be a random variable with some probability function. The objective function would also need to be modified to specify whether the stochastic solver should optimize for the maximum possible profit, the expected profit, or a particular decile, for example. The underlying solver would then aim to run many simulations to identify the best subset of stocks to include in the portfolio.

The above is, of course, a trivial example. Due to the ever-expanding scale and complexity of real-world problems, modern solvers confront issues with scalability, convergence, noise, and computational complexity. Traditional algorithms often struggled with the exponential increase in computational requirements accompanying large-scale problems.

This required new methodologies and enhancements to existing approaches, all focused on scalability and efficient convergence to global optima. Researchers persist in refining techniques to counter these obstacles, drawing on insights from interdisciplinary fields such as optimization, machine learning, data analytics, and artificial intelligence. And due to modern advances, stochastic solvers have come a long way.

As investment businesses grapple with uncertainties inherent to a constantly changing market, it’s vital for decision-makers to address the challenges ahead. By leveraging advancements in stochastic optimization and interdisciplinary collaboration, firms can ensure they remain agile and competitive, paving the way to success in an increasingly uncertain world.

Stochastic Optimization at FactSet

At FactSet, we continually explore state-of-the-art techniques for world-class solutions and performance. This includes stochastic optimization, which we’ve tested to design optimal product bundles. We also consider established solvers such as Gurobi and Cplex as well as promising newcomers like InsideOpt.

Areas like Generative AI and Large Language Models have justifiably been garnering their share of media coverage, but the recent developments in efficient and easy-to-use solvers for stochastic optimization should also be considered.

This blog post is for informational purposes only. The information contained in this blog post is not legal, tax, or investment advice. FactSet does not endorse or recommend any investments and assumes no liability for any consequence relating directly or indirectly to any action or inaction taken based on the information contained in this article.