quantlab.model.predefined.transformer.LocalformerEncoder

class quantlab.model.predefined.transformer.LocalformerEncoder(encoder_layer, num_layers, d_model)

Bases: Module

Qlib’s Localformer encoder: each layer reads its input plus a convolution of it.

Before layer i, a width-3 convolution over time of the running output is added to it; the encoder returns the last layer’s output plus the last convolution.

Examples

>>> layer = nn.TransformerEncoderLayer(d_model=4, nhead=2, dropout=0.0)
>>> LocalformerEncoder(layer, num_layers=2, d_model=4)(torch.randn(5, 3, 4), None).shape
torch.Size([5, 3, 4])

forward(src, mask)

Map time-major [N, B, D] to [N, B, D].