Risk Management (CVaR)

mpi-sppy supports minimizing (or maximizing) a risk-averse objective that blends the expected cost with the Conditional Value-at-Risk (CVaR, also called expected shortfall) of the cost:

\[\lambda \cdot \mathbb{E}[\text{Cost}] \;+\; \beta \cdot \text{CVaR}_\alpha(\text{Cost})\]

This is the same “weighted CVaR” formulation used by PySP, where \(\alpha \in (0,1)\) is the confidence level, \(\beta \ge 0\) is the weight on CVaR, and \(\lambda \ge 0\) is the weight on the expectation.

How it works

CVaR is added as a per-scenario model transform using the Rockafellar–Uryasev linearization. For each scenario \(s\) a non-negative excess variable \(\delta_s\) is added together with a single shared Value-at-Risk variable \(\eta\):

\[\text{minimize} \quad \lambda \, \text{Cost}_s + \beta\, \eta + \frac{\beta}{1-\alpha}\, \delta_s \qquad \text{s.t.} \quad \delta_s \ge \text{Cost}_s - \eta,\;\; \delta_s \ge 0 .\]

Because \(\sum_s p_s = 1\) and \(\eta\) is a single first-stage (non-anticipative) variable, the risk measure distributes over scenarios and \(\eta\) becomes “just another first-stage variable.” The original risk-neutral objective is deactivated (but left on the model, so it remains available for separate \(\mathbb{E}[\text{Cost}]\) reporting) and replaced by a new active objective. As a result the extensive form (EF) and every cylinder (PH/APH hub, Lagrangian and subgradient outer-bound spokes, xhat inner-bound spokes, FWPH, …) inherit risk aversion with no algorithm changes.

Using it from the command line

With generic_cylinders (or any driver that calls cvar_args), add --cvar and, optionally, the weights:

# EF solve of the risk-averse farmer
python ../../mpisppy/generic_cylinders.py --module-name farmer --num-scens 3 \
    --EF --EF-solver-name gurobi --cvar --cvar-weight 2.0 --cvar-alpha 0.8

The flags are:

--cvar – apply the CVaR transform to every scenario (default off).
--cvar-weight – \(\beta\), the weight on CVaR (default 1.0).
--cvar-alpha – the confidence level \(\alpha\), 0 < alpha < 1 (default 0.95).
--cvar-mean-weight – \(\lambda\), the weight on \(\mathbb{E}[\text{Cost}]\) (default 1.0); use 0 for pure CVaR.

Using it programmatically

Wrap any scenario_creator with cvar_scenario_creator (or call add_cvar on an already-built scenario):

import mpisppy.utils.cvar as cvar

risk_creator = cvar.cvar_scenario_creator(
    my_scenario_creator, cvar_weight=2.0, cvar_alpha=0.8)
# risk_creator now has the same signature as my_scenario_creator and can be
# passed to create_EF, the PH constructor, the cfg_vanilla factories, etc.

Setting rho with CVaR (important)

Warning

The Value-at-Risk variable \(\eta\) typically has a very different cost profile from the model’s other first-stage variables: it lives on the scale of the objective (often orders of magnitude larger than, say, an acreage or a production quantity). A single uniform proximal \(\rho\) that is reasonable for the original variables is usually far too small for \(\eta\), so Progressive Hedging barely moves \(\eta\) and converges extremely slowly (or not at all within an iteration budget).

The robust fix is to let mpi-sppy choose a per-variable \(\rho\) from the cost coefficients. We recommend gradient-based rho (--grad-rho), and --sep-rho is another good cost-aware option. For example, on the three-scenario farmer with --cvar --cvar-weight 2.0 --cvar-alpha 0.8 (whose EF-CVaR optimum is -220700):

rho strategy	result after up to 100 PH iterations
`--default-rho 1`	40% gap; the bounds never close because \(\eta\) is stuck
`--grad-rho --grad-order-stat 0.5`	converges to the optimum (0% gap)
`--sep-rho`	converges to the optimum (0% gap)

So a risk-averse PH run on the farmer looks like:

mpiexec -np 3 python -m mpi4py ../../mpisppy/generic_cylinders.py \
    --module-name farmer --num-scens 3 --solver-name gurobi \
    --max-iterations 100 --default-rho 1 --grad-rho --grad-order-stat 0.5 \
    --lagrangian --xhatshuffle --rel-gap 1e-6 \
    --cvar --cvar-weight 2.0 --cvar-alpha 0.8

See Rho Setting for the full menu of rho strategies.

Maximization

Maximization objectives are supported as well. Risk aversion then applies to the lower (reward) tail: the excess variable becomes non-positive and the excess constraint mirrors (\(\delta_s \le \text{Cost}_s - \eta\), \(\delta_s \le 0\)), so the same --cvar flags maximize \(\lambda\,\mathbb{E}[\text{Reward}] + \beta\,\text{CVaR}_\alpha\) of the worst-case (lowest) rewards.

Confidence intervals

The zhat4xhat program (see zhat and Background for Confidence Intervals) evaluates whichever objective is active in each scenario. Because the CVaR transform leaves the risk-averse objective active, zhat4xhat automatically evaluates the risk-averse objective for a candidate xhat – no extra flags are needed – so the resulting interval reflects the risk-averse value rather than the risk-neutral cost.

Scope and limitations

Scope: single root-stage CVaR, not time-consistent

mpi-sppy’s CVaR support uses a single Value-at-Risk variable \(\eta\) at the root node and applies CVaR to the total (end-of-horizon) scenario cost. This is the same formulation as PySP. It is a single-period risk measure on the total cost and is not a nested / time-consistent multistage risk measure (there is no per-stage \(\eta\) and no recursive composition across stages). For two-stage problems this is the usual CVaR; for multistage problems it measures the risk of the total cost only. If you need a time-consistent (nested) risk measure, please contact the mpi-sppy developers.