dsipts.models.TFT module

class dsipts.models.TFT.TFT(d_model, out_channels, past_steps, future_steps, past_channels, future_channels, num_layers_RNN, embs, d_head, n_head, dropout_rate, persistence_weight=0.0, loss_type='l1', quantiles=[], optim=None, optim_config=None, scheduler_config=None, **kwargs)[source]

Bases: Base

TEMPORAL FUSION TRANSFORMER - Multi-Horizon TimeSeries Forecasting

  • Direct Model: predicting all future step at once..

  • Multi-Output Forecasting: predicting one or more variables.

  • Multi-Horizon Forecasting: predicting variables at multiple future time steps.

  • Attention based: Enhance selection of relevant time steps in the past and learn long-term dependencies. Weights of attention as importance magnitude for each head.

  • RNN Enrichment: Enpowering the initial autoregressive process. The RNN (here LSTM) provides an initial approximation of the target varible(s), then improved by the rest of th Net.

  • Gating Mechanisms: Minimize the contribution of irrelevant variables.

  • Prediction Intervals (Quantile Regression): Outputting percentiles at each timestep. [10th, 50th, 90th] usually.

TFT facilitates Interpretability identifying: - Global importance of variables for past and for future - Temporal patterns - Significant events

Parameters:

  • d_model (int) – general hidden dimension across the Net. Could be changed in subNets

  • out_channels (int) – number of variables to predict

  • past_steps (int) – steps of the look-back window

  • future_steps (int) – steps in the future to be predicted

  • past_channels (int) – total number of variables available in the past

  • future_channels (int) – total number of variables available in the future

  • num_layers_RNN (int) – number of layers for recurrent NN (here LSTM)

  • embs (list[int]) – embedding dimensions for added categorical variables (here for pos_seq, is_fut, pos_fut)

  • d_head (int) – attention head dimension

  • n_head (int) – number of attention heads

  • dropout_rate (float) – dropout. Common rate for all dropout layers used.

  • persistence_weight (float, optional) – ASK TO GOBBI. Defaults to 0.0.

  • loss_type (str, optional) – Type of loss for prediction. Defaults to ‘l1’.

  • quantiles (List[float], optional) – list of quantiles to predict. If empty, only the exact value. Only empty list or lisst of len 3 allowed. Defaults to [].

  • optim (Union[str,None], optional) – ASK TO GOBBI. Defaults to None.

  • optim_config (dict, optional) – ASK TO GOBBI. Defaults to None.

  • scheduler_config (dict, optional) – ASK TO GOBBI. Defaults to None.

handle_multivariate = True
handle_future_covariates = True
handle_categorical_variables = True
handle_quantile_loss = True
description = 'Can   handle multivariate output \nCan   handle future covariates\nCan   handle categorical covariates\nCan   handle Quantile loss function'
__init__(d_model, out_channels, past_steps, future_steps, past_channels, future_channels, num_layers_RNN, embs, d_head, n_head, dropout_rate, persistence_weight=0.0, loss_type='l1', quantiles=[], optim=None, optim_config=None, scheduler_config=None, **kwargs)[source]

TEMPORAL FUSION TRANSFORMER - Multi-Horizon TimeSeries Forecasting

  • Direct Model: predicting all future step at once..

  • Multi-Output Forecasting: predicting one or more variables.

  • Multi-Horizon Forecasting: predicting variables at multiple future time steps.

  • Attention based: Enhance selection of relevant time steps in the past and learn long-term dependencies. Weights of attention as importance magnitude for each head.

  • RNN Enrichment: Enpowering the initial autoregressive process. The RNN (here LSTM) provides an initial approximation of the target varible(s), then improved by the rest of th Net.

  • Gating Mechanisms: Minimize the contribution of irrelevant variables.

  • Prediction Intervals (Quantile Regression): Outputting percentiles at each timestep. [10th, 50th, 90th] usually.

TFT facilitates Interpretability identifying: - Global importance of variables for past and for future - Temporal patterns - Significant events

Parameters:

  • d_model (int) – general hidden dimension across the Net. Could be changed in subNets

  • out_channels (int) – number of variables to predict

  • past_steps (int) – steps of the look-back window

  • future_steps (int) – steps in the future to be predicted

  • past_channels (int) – total number of variables available in the past

  • future_channels (int) – total number of variables available in the future

  • num_layers_RNN (int) – number of layers for recurrent NN (here LSTM)

  • embs (list[int]) – embedding dimensions for added categorical variables (here for pos_seq, is_fut, pos_fut)

  • d_head (int) – attention head dimension

  • n_head (int) – number of attention heads

  • dropout_rate (float) – dropout. Common rate for all dropout layers used.

  • persistence_weight (float, optional) – ASK TO GOBBI. Defaults to 0.0.

  • loss_type (str, optional) – Type of loss for prediction. Defaults to ‘l1’.

  • quantiles (List[float], optional) – list of quantiles to predict. If empty, only the exact value. Only empty list or lisst of len 3 allowed. Defaults to [].

  • optim (Union[str,None], optional) – ASK TO GOBBI. Defaults to None.

  • optim_config (dict, optional) – ASK TO GOBBI. Defaults to None.

  • scheduler_config (dict, optional) – ASK TO GOBBI. Defaults to None.

forward(batch)[source]

Temporal Fusion Transformer

Collectiong Data - Extract the autoregressive variable(s) - Embedding and compute a first approximated prediction - ‘summary_past’ and ‘summary_fut’ collecting data about past and future Concatenating on the dimension 2 all different datas, which will be mixed through a MEAN over that imension Info get from other tensor of the batch taken as input

TFT actual computations - Residual Connection for y_past and summary_past - Residual Connection for y_fut and summary_fut - GRN1 for past and for fut - ATTENTION(summary_fut, summary_past, y_past) - Residual Connection for attention itself - GRN2 for attention - Residual Connection for attention and summary_fut - Linear for actual values and reshape

Parameters:

batch (dict) – Keys used are [‘x_num_past’, ‘idx_target’, ‘x_num_future’, ‘x_cat_past’, ‘x_cat_future’]

Returns:

shape [B, self.future_steps, self.out_channels, self.mul] or [B, self.future_steps, self.out_channels] according to quantiles

Return type:

torch.Tensor

remove_var(tensor, indexes_to_exclude, dimension)[source]

Function to remove variables from tensors in chosen dimension and position

Parameters:

  • tensor (torch.Tensor) – starting tensor

  • indexes_to_exclude (int) – index of the chosen dimension we want t oexclude

  • dimension (int) – dimension of the tensor on which we want to work

Returns:

new tensor without the chosen variables

Return type:

torch.Tensor