quantlab.acquisition.wrds.crsp_reference#
Download of the CRSP, Compustat and CCM reference tables into _reference/.
CRSP (the Center for Research in Security Prices) publishes the standard database of US stock prices; Compustat publishes company fundamentals and index membership; CCM (the CRSP/Compustat Merged database) links the two by mapping Compustat company ids (gvkeys) to CRSP security ids (PERMNOs, the permanent integer CRSP gives each security). All three are served by WRDS (Wharton Research Data Services) through a PostgreSQL server.
Besides the daily prices, building a correct panel needs several lookup
tables: security names and tickers over time, delistings, distributions
(dividends and splits), and historical index membership. These are the
reference tables. quantlab.dataset.crsp.reference declares the six of
them (schema, column order and types) and reads them back without a
credential. CrspReferenceTables in this module does the download: it
copies each table whole through the shared WRDS session and writes one
parquet file per table, plus a manifest.json that records the CRSP
annual release (vintage) the tables came from.
This is deliberately not a SymbolBatchAcquisition. That base class downloads a
time series per symbol, in batches and pages, and records progress so it
can resume. These tables have no symbol axis and no date window; each one is
either present or not. They are written to .../wrds_crsp/_reference,
beside the raw directory and never inside it, because the dataset’s
conversion reads every file below the raw directory and a stray parquet or
manifest there would break it.
stkdelists (delisting events) is stored as event data only. The daily
table already includes the delisting return on its own daily row, so nothing
here adds delret to dlyret; doing so would count the delisting loss
twice.
Classes
Download the CRSP, Compustat and CCM reference tables into one directory. |