quantlab.portfolio.predefined.mean_variance.MeanVarianceOptimizer#

class quantlab.portfolio.predefined.mean_variance.MeanVarianceOptimizer(config)[source]#

Bases: PortfolioConstructor

Mean-variance weights with a turnover penalty, long-only or long-short.

On each rebalance bar the candidates are the symbols the context marks tradable, covered by the covariance estimator (enough recent return history), and either with a finite prediction of expected_return_label or held; a held candidate without a prediction has an expected return of 0.0, so its turnover cost decides whether it is closed. A locked position (held, not tradable) keeps its current weight; a held, tradable symbol the risk model does not cover is closed, and the row reports it in attrs["events"]["closed_without_risk"]; every other symbol gets 0.0. Over the candidates the optimiser solves

maximise w @ mu - risk_aversion / 2 * w @ Sigma @ w

turnover_penalty * |w - w_current|_1

subject to, with direction="long_only",

sum(w) = 1, 0 <= w <= weight_cap

and with direction="long_short",

sum(w) = 0, |w|_1 <= 1, |w| <= weight_cap.

w in the risk term also holds the locked weights, so their covariance with the candidates counts, while the turnover and the cap apply to the candidates only; the budget is what the locked positions leave: the candidates sum to 1 - L long-only (nothing when L >= 1) and to -L long-short with |w|_1 <= 1 - |L|_1, L the locked weights. A locked position the covariance estimator does not cover adds no variance. The long-short gross exposure is a ceiling, not an equality: when the expected returns do not pay for the risk and the turnover, part of the book stays uninvested, down to no position at all.

Sigma is the covariance estimator’s one-bar covariance times the span n of expected_return_label (variance is linear in time). When the risk model’s estimate has a factor form (B F B' + diag(D)), the risk term is built as |F^(1/2) B' w|^2 + w' diag(D) w and the dense matrix is never formed. mu is the expected_return model’s (an ExpectedReturnModel, ADR 0046) on the prediction of the predicted candidates, handed sigma, the square root of Sigma’s diagonal, when the model needs it: GrinoldExpectedReturn gives ic * sigma * z, z the candidates’ cross-sectional z-score (ddof=1), so the prediction only ranks and any model’s output can feed the optimiser; ZScoreExpectedReturn drops the tilt to risky names, RawExpectedReturn takes a prediction in return units and PredictionExpectedReturn one in any units. w_current is the context’s current weights.

With volatility_label set (a Volatility label, say), the volatilities come from a model: the label’s prediction at the bar, divided by sqrt(n), is handed to the covariance estimator as the one-bar volatilities, so Sigma is the predicted volatilities around the risk model’s historical correlations, its diagonal the squared predictions, and the sigma the expected return model reads is the prediction itself. A symbol without a finite positive volatility prediction has no risk estimate, like one without enough history. bind requires the label to have the span of expected_return_label and a "raw" scale.

With candidate_top_k set, only a pool is optimised: the candidate_top_k candidates with the largest mu (largest |mu| long-short) plus every candidate currently held, so a held symbol that fell out of the top can still be closed at its turnover cost. The z-score is taken over every candidate before the pool is cut.

With max_names set, at most that many symbols hold a weight, long and short together, locked positions included: each bar is solved exactly as a mixed-integer program (one boolean per candidate, |w| <= weight_cap * z, sum(z) <= max_names less the locked names), names it leaves out get exactly 0, and a held one is closed whatever min_trade. A bar whose locked positions exceed the cap (long-only, or fill it while budget is left), or whose solve is not optimal, raises PortfolioConstructionError.

The solver’s solution is cleaned of its round-off: projected onto the feasible set long-only (the nearest weights summing to one within [0, weight_cap]), clipped and rescaled to dollar neutrality and a gross exposure of at most one long-short.

A bar whose candidates cannot hold their budget under weight_cap, a long-short bar whose locked positions exceed a gross exposure of one, a bar the solver fails on or leaves unsolved (bounds the candidates cannot reach among them), and a bar with a locked position lacking a bounded exposure raise PortfolioConstructionError: the backtest holds the current position there and records the bar.

Two extension points let a rule built on the optimiser change what risk means: reference_weights (the risk term prices w - b against a reference book, a benchmark say) and risk_constraints (constraints on the risk term, a tracking-error cap say). A third, posterior, lets it replace mu and the covariance estimate and change the candidates, after mu is computed and before candidate_top_k (ADR 0047). All are unused by default, and the problem is then exactly the one above. ActiveRiskOptimizer (quantlab.portfolio.predefined.active_risk) builds a benchmark-relative rule on the first two.

declared_inputs() is the covariance estimator’s declaration (its return window, its factor risk model) merged with the exposure_factors, whose outputs must not repeat a name. exposure_bounds may bound any of their outputs or any of the risk model’s exposure_names (read from context.risk_exposures); a bounded name must come from one of the two, not both. bind reads the span from the label’s LabelSpec, so a backtest binds the optimiser when it is built.

Parameters:

configMeanVarianceConfig: expected_return_label, covariance, risk_aversion, expected_return, turnover_penalty, weight_cap, direction, candidate_top_k, volatility_label, exposure_factors, exposure_bounds, max_names, mip_gap and solver.

Raises:

TypeError: If expected_return is not an ExpectedReturnModel.
ValueError: If direction is unknown, weight_cap is not in (0, 1], risk_aversion or turnover_penalty is negative, candidate_top_k is not a positive integer, volatility_label is set with a covariance estimator that does not take volatilities (FactorRiskStoreEstimator), solver is not an installed cvxpy solver, max_names is not a positive integer, is set long-only with weight_cap * max_names < 1, or with a solver that cannot solve mixed-integer conic programs, or mip_gap is outside (0, 1), set without max_names or for a solver it is not passed to.

Examples

specs holds the spec of a 5-bar ret_5 label (LabelSpec("ret_5", "raw", 1, 5)); context is a bar of four symbols AAA..``DDD``, all tradable and none held, with 60 bars of one-bar returns of volatility 1%, 1.5%, 2% and 2.5% and ret_5 predictions 0.8, -0.1, -0.3 and 0.2. With a risk aversion of 5 on so small an expected return the book leans toward the low-volatility symbols:

>>> from quantlab.portfolio.config import (
...     GrinoldExpectedReturnConfig, LedoitWolfEstimatorConfig, MeanVarianceConfig,
... )
>>> from quantlab.portfolio.predefined.expected_return import GrinoldExpectedReturn
>>> from quantlab.portfolio.predefined.ledoit_wolf import LedoitWolfEstimator
>>> optimizer = MeanVarianceOptimizer(MeanVarianceConfig(
...     expected_return_label="ret_5",
...     covariance=LedoitWolfEstimator(LedoitWolfEstimatorConfig(lookback_bars=60)),
...     expected_return=GrinoldExpectedReturn(GrinoldExpectedReturnConfig(ic=0.05)),
...     risk_aversion=5.0, weight_cap=0.4,
... ))
>>> optimizer.declared_inputs().lookback_bars
60
>>> optimizer.bind(specs)
>>> weights = optimizer.construct(context)
>>> float(weights.sum().round(6)), bool((weights >= 0).all()), bool((weights <= 0.4).all())
(1.0, True, True)

Long-short on the same bar the book is dollar-neutral, and its gross exposure (0.946) stays under the ceiling of one:

>>> long_short = MeanVarianceOptimizer(MeanVarianceConfig(
...     expected_return_label="ret_5",
...     covariance=LedoitWolfEstimator(LedoitWolfEstimatorConfig(lookback_bars=60)),
...     expected_return=GrinoldExpectedReturn(GrinoldExpectedReturnConfig(ic=0.05)),
...     risk_aversion=5.0, weight_cap=0.4, direction="long_short",
... ))
>>> long_short.bind(specs)
>>> weights = long_short.construct(context)
>>> weights.values.round(3)
array([ 0.4  , -0.212, -0.261,  0.073])

With DDD held at 0.3 and halted, it is a locked position: it keeps its weight and the candidates share the remaining 0.7:

>>> import dataclasses
>>> halted = dataclasses.replace(
...     context,
...     tradable=context.tradable.copy(data=[True, True, True, False]),
...     current_weights=context.current_weights.copy(data=[0.0, 0.0, 0.0, 0.3]),
... )
>>> optimizer.construct(halted).values.round(3)
array([0.4, 0.3, 0. , 0.3])

bind(labels)[source]#

Check the specs hold the optimiser’s labels and read the span.

Parameters:

labelsSequence[LabelSpec]: The specs of the predicted labels.

Raises:

ValueError: If expected_return_label or volatility_label is not one of the specs, either has no span (it is not a Forward label), their spans differ, volatility_label’s scale is not "raw", or the expected_return model refuses expected_return_label’s spec (RawExpectedReturn: a scale other than "raw").

Examples

>>> from quantlab.runs.prediction_panel import LabelSpec
>>> optimizer.bind([LabelSpec(name="ret_5", scale="raw", delay=1, span=5)])
>>> optimizer.span
5

config_cls#: alias of MeanVarianceConfig

construct(context)[source]#

Solve the bar’s mean-variance problem and return its weights.

Parameters:

contextPortfolioContext: The bar’s context.

Returns:

xr.DataArray: One finite weight per symbol of context.symbols, locked positions at their current weights: summing to one long-only (to the locked weights’ sum when that is already one or more); summing to zero with gross exposure at most one long-short. When a held symbol was closed for want of a risk estimate, attrs["events"]["closed_without_risk"] lists it.

Raises:

PortfolioConstructionError: If the bar is infeasible or the solver fails or leaves it unsolved.

Examples

>>> optimizer.construct(context).values.round(3)
array([0.4  , 0.4  , 0.009, 0.191])

declared_inputs()[source]#

The covariance estimator’s declaration, merged with the exposure_factors.

Examples

>>> optimizer.declared_inputs().history_bars  # Ledoit-Wolf: 60 + 1 + 5
66

keeps_every_symbol()[source]#

Whether the covariance estimator pools the symbols (CovarianceEstimator.pools_symbols).

Examples

>>> rule.keeps_every_symbol()  # a Ledoit-Wolf estimator
True

posterior(context, symbols, expected_return, estimate, candidates)[source]#

Return the expected return, the covariance estimate and the candidates the bar optimises.

An extension point, called by problem_inputs after mu is computed and before candidate_top_k cuts the pool: a rule may replace the expected return (a Bayes update toward a prior, say), replace the estimate (a posterior covariance) and widen or narrow the candidates among the selectable symbols. The default returns them unchanged, and the problem is then exactly the one in the class docstring.

Parameters:

contextPortfolioContext: The bar’s context.
symbolsnp.ndarray: The selectable symbols, in the context’s order: tradable, not locked, covered by the estimate and with every bounded exposure.
expected_returnnp.ndarray: mu per selectable symbol on the expected-return label’s span, NaN where there is no prediction.
estimateReturnCovariance: The covariance estimator’s estimate on the span, over the symbols it covers.
candidatesnp.ndarray: Booleans per selectable symbol: the ones the bar optimises (a prediction, or held).

Returns:

expected_returnnp.ndarray: mu per selectable symbol; a candidate left NaN gets 0.0.
estimateReturnCovariance: The estimate the risk term prices; it must cover every candidate (a ValueError otherwise), and the locked positions it covers enter the risk term.
candidatesnp.ndarray: Booleans per selectable symbol.

Examples

With optimizer and context as in the class example, the default changes nothing:

>>> symbols = context.symbols
>>> estimate = optimizer.config.covariance.estimate(context).scaled(5)
>>> mu, same, candidates = optimizer.posterior(
...     context, symbols, np.array([0.01, np.nan, -0.01, 0.0]), estimate,
...     np.array([True, False, True, True]),
... )
>>> mu.tolist(), same is estimate, candidates.tolist()
([0.01, nan, -0.01, 0.0], True, [True, False, True, True])

problem_inputs(context)[source]#

Return the candidates of the context’s bar and their mu, Sigma and weights.

See the class docstring for how each is computed.

Parameters:

contextPortfolioContext: The bar’s context.

Returns:

MeanVarianceInputs: The candidates in the context’s symbol order.

Raises:

RuntimeError: If the optimiser has not been bound to label specs.

Examples

>>> inputs = optimizer.problem_inputs(context)
>>> inputs.expected_return.round(4)
array([ 0.0017, -0.0008, -0.0022,  0.0003])

ranking_label(names)[source]#

Return expected_return_label: the scores the optimiser ranks by.

Examples

>>> optimizer.ranking_label(["vol_5", "ret_5"])
'ret_5'

reference_weights(context)[source]#

The book the risk term measures against at the bar, or None for total risk.

An extension point: with a reference book b (a benchmark, say) the risk term prices (w - b)' Sigma (w - b), the active risk, instead of w' Sigma w; the expected return, turnover, budget, cap and exposure bounds are unchanged. b is read on the bar’s symbols (NaN counts as 0); a symbol of it the covariance estimator does not cover is left out of it and reported in the row’s reference_without_risk event. None by default.

Parameters:

contextPortfolioContext: The bar’s context.

Returns:

xr.DataArray or None: Weights on symbol.

Examples

>>> optimizer.reference_weights(context) is None
True

risk_constraints(risk)[source]#

Constraints on the bar’s risk term, the forecast variance over the span.

An extension point: risk is the cvxpy expression of w' Sigma w (of w - b with a reference book), locked positions included, on the expected-return label’s span; a rule capping it returns, for example, [risk <= cap**2]. A cap the candidates cannot meet makes the bar infeasible, and the backtest holds it. Empty by default.

Parameters:

riskcvxpy.Expression: The risk term.

Returns:

list of cvxpy constraints

Examples

>>> optimizer.risk_constraints(None)
[]

property span#

The expected-return label’s span in bars, once bind has read it.

Examples

>>> optimizer.span
5