quantlab.dataset.constituent#

Point-in-time index-membership panels, one class per index and source.

Point-in-time membership records which securities belonged to an index on each past day, as known on that day. Using today’s member list for the past instead would cause survivorship bias: the backtest would only ever hold companies that later survived. A membership panel is an xarray.Dataset indexed by timestamp and symbol whose is_member variable is True on the days a symbol was a member.

Each class here plugs one membership source into the shared panel builder quantlab.dataset.raw.IndexConstituentDataset by implementing its two hooks, _pit_coverage_start and _build_intervals. Two sources cover the S&P 500 and the Nasdaq-100:

  • SP500ConstituentDataset and Nasdaq100ConstituentDataset replay a public Wikipedia change log and use tickers as the symbol axis.

  • CrspSP500ConstituentDataset and CompustatNasdaq100ConstituentDataset read locally downloaded CRSP reference tables and use the int64 PERMNO as the symbol axis.

CRSP (the Center for Research in Security Prices) is a US stock database sold through WRDS (Wharton Research Data Services). A PERMNO is CRSP’s permanent integer id for one security; unlike a ticker it never changes or gets reused, and it is also the symbol axis of the CRSP price panel, so the two line up column for column. CrspMarketConstituentDataset is the whole-market variant: every security CRSP lists, not an index.

config.cache_dir is the local directory the panel is built from: the cache of scraped web pages for the Wikipedia classes, and the CRSP reference directory for the CRSP classes. The CRSP classes only read parquet files on disk and need no WRDS login.

Classes

CompustatNasdaq100ConstituentDataset

Daily point-in-time Nasdaq-100 membership panel from Compustat.

CrspMarketConstituentDataset

Daily point-in-time whole-market listing panel from CRSP.

CrspSP500ConstituentDataset

Daily point-in-time S&P 500 membership panel from CRSP's membership records.

Nasdaq100ConstituentDataset

Daily point-in-time Nasdaq-100 membership panel from a public change log.

SP500ConstituentDataset

Daily point-in-time S&P 500 membership panel from a public change log.