nnbma.learning package

Submodules

nnbma.learning.batch_scheduler module

class nnbma.learning.batch_scheduler.BatchScheduler

Bases: ABC

Abstract class for schedulers of the batch size \(b(t)\) with respect to the epoch attribute, denoted \(t\) in math equations.

abstract get_batch_size() → int

Returns the batch size \(b(t)\) for the next epoch depending on the number of already run epochs \(t\).

Returns:

batch size \(b(t)\).

Return type:

int

set_epoch(epoch: int) → None

Set the epoch attribute to a new value.

Parameters:

epoch (int) – new value for the epoch attribute.

step() → None

Increments the epoch attribute: \(t \leftarrow t+1\).

class nnbma.learning.batch_scheduler.ConstantBatch(batch_size: int)

Bases: object

Simplest scheduler, that always returns a predefined constant \(c\), i.e., \(b(t) = c\) for all \(t\)

Parameters:

batch_size (int) – batch size to consider during the training.

get_batch_size() → int

class nnbma.learning.batch_scheduler.ExponentialBatchScheduler(start: int, stop: int, n_epochs: int)

Bases: BatchScheduler

Scheduler based on an exponential interpolation between an initial batch size \(b_{i}\) and a final batch size \(b_{f}\) for a total number of epochs \(t_{f}\), i.e., for \(0 \leq t \leq t_{f}\),

\[b(t) = \left( \frac{b_{f}}{b_{i}} \right)^{t / t_{f}} b_{i}\]

Parameters:

• start (int) – starting value for the batch size \(b_{i}\).

• stop (int) – final value for the batch size \(b_{f}\).

• n_epochs (int) – total number of epochs considered for training \(t_{f}\).

get_batch_size() → int

Returns the batch size \(b(t)\) for the next epoch depending on the number of already run epochs \(t\).

Returns:

batch size \(b(t)\).

Return type:

int

class nnbma.learning.batch_scheduler.LinearBatchScheduler(start: int, stop: int, n_epochs: int)

Bases: BatchScheduler

Scheduler based on a linear interpolation between an initial batch size \(b_{i}\) and a final batch size \(b_{f}\) for a total number of epochs \(t_{f}\), i.e., for \(0 \leq t \leq t_{f}\),

\[b(t) = \frac{t}{t_{f}} b_{i} + \frac{t_{f} - t}{t_{f}} b_{f}\]

Parameters:

• start (int) – starting value for the batch size \(b_{i}\).

• stop (int) – final value for the batch size \(b_{f}\).

• n_epochs (int) – total number of epochs considered for training \(t_{f}\).

get_batch_size() → int

Returns the batch size \(b(t)\) for the next epoch depending on the number of already run epochs \(t\).

Returns:

batch size \(b(t)\).

Return type:

int

nnbma.learning.loss_functions module

class nnbma.learning.loss_functions.CauchyLoss

Bases: Module

Implements the Cauchy loss function, i.e.,

\[\mathrm{CL}(\widehat{y}, y) = \sum_{i} \log \left( 1 + \left( \widehat{y}_{i} - y_{i} \right)^2 \right)\]

Initialize internal Module state, shared by both nn.Module and ScriptModule.

forward(y_hat: Tensor, y: Tensor) → Tensor

Evaluates the Cauchy loss between a prediction y_hat and a reference y.

Parameters:

• y_hat (torch.Tensor) – network prediction.

• y (torch.Tensor) – true values. Must have the same shape as y_hat.

Returns:

evaluated loss, should be a float.

Return type:

torch.Tensor

training: bool

class nnbma.learning.loss_functions.MaskOverlay(loss)

Bases: MaskedLossFunction

Permits to use a MaskedLossFunction as a standard loss function.

Parameters:

loss (Callable) – loss function

forward(y_hat: Tensor, y: Tensor, _) → Tensor

Evaluate the loss function.

Parameters:

• y_hat (torch.Tensor) – network prediction.

• y (torch.Tensor) – true values. Must have the same shape as y_hat.

Returns:

evaluated loss, should be a float.

Return type:

torch.Tensor

training: bool

class nnbma.learning.loss_functions.MaskedLossFunction

Bases: Module, ABC

Implements a masked loss function which has a signature loss_fun(y_hat, y, mask).

abstract forward(y: Tensor, mask: Tensor) → Tensor

Evaluates the loss between a prediction y_hat and a reference y with some masked values indicated in mask.

Parameters:

• y_hat (torch.Tensor) – network prediction.

• y (torch.Tensor) – true values. Must have the same shape as y_hat.

• mask (torch.Tensor) – binary mask with values to disregard in the loss. Must have the same shape as y_hat.

Returns:

evaluated loss, should be a float.

Return type:

torch.Tensor

training: bool

class nnbma.learning.loss_functions.MaskedMSELoss

Bases: MaskedLossFunction

Implements the masked MSE loss function, i.e., for a binary mask \(m\), a prediction \(\widehat{y}\) and a true value \(y\),

\[\mathrm{MaskedMSE}(\widehat{y}, y, m) = \frac{1}{\sum_{i} m_{i}} \sum_{i} m_{i} \left( \widehat{y}_{i} - y_{i} \right)^2\]

forward(y_hat: Tensor, y: Tensor, mask: Tensor) → Tensor

Evaluates the masked MSE loss between a prediction y_hat and a reference y, with a binary mask m – where \(m_{i}=0\) corresponds to a masked value.

Parameters:

• y_hat (torch.Tensor) – network prediction.

• y (torch.Tensor) – true values. Must have the same shape as y_hat.

• mask (torch.Tensor) – binary mask. Must have the same shape as y_hat.

Returns:

evaluated loss, should be a float.

Return type:

torch.Tensor

training: bool

class nnbma.learning.loss_functions.SmoothL1Loss(beta: float)

Bases: Module

Implements the smooth L1 loss function, i.e.,

\[\begin{split}\mathrm{SmoothL1}(\widehat{y}, y) = \sum_{i} \begin{cases} \frac{1}{2\beta}\left( \widehat{y}_{i} - y_{i} \right)^2 \; \text{ if } \vert \widehat{y}_{i} - y_{i} \vert \leq \beta \\ \vert \widehat{y}_{i} - y_{i} \vert - 0.5 \beta \; \text{ otherwise} \end{cases}\end{split}\]

Parameters:

beta (float) – \(\beta\) parameter of the loss function.

forward(y_hat: Tensor, y: Tensor) → Tensor

Evaluates the smooth L1 loss between a prediction y_hat and a reference y.

Parameters:

• y_hat (torch.Tensor) – network prediction.

• y (torch.Tensor) – true values. Must have the same shape as y_hat.

Returns:

evaluated loss, should be a float.

Return type:

torch.Tensor

training: bool

nnbma.learning.network_learning module

class nnbma.learning.network_learning.LearningParameters(loss_fun: Callable[[Tensor, Tensor], Tensor] | MaskedLossFunction, epochs: int, batch_size: int | BatchScheduler | None, optimizer: Optimizer, scheduler: _LRScheduler | None = None)

Bases: object

Specifies the main parameters training, including the loss function to minimize and the stochastic gradient descent strategy.

Parameters:

• loss_fun (Union[ Callable[[torch.Tensor, torch.Tensor], torch.Tensor], MaskedLossFunction, ]) – loss function.

• epochs (int) – total number of epochs to perform.

• batch_size (Union[int, BatchScheduler, None]) – batch size value or scheduler to use during training.

• optimizer (Optimizer) – optimizer to use for training.

• scheduler (Optional[_LRScheduler], optional) – learning rate scheduler, by default None

nnbma.learning.network_learning.learning_procedure(model: NeuralNetwork, dataset: RegressionDataset | Tuple[RegressionDataset, RegressionDataset], learning_parameters: LearningParameters | List[LearningParameters], mask_dataset: MaskDataset | Tuple[MaskDataset, MaskDataset] | None | Tuple[None, None] = None, train_samples: Sequence | None = None, val_samples: Sequence | None = None, val_frac: float | None = None, additional_metrics: Dict[str, Callable[[Tensor, Tensor], Tensor]] | None = None, verbose_level: Literal[None, 0, 1, 2] = 1, seed: int | None = None, max_iter_no_improve: int | None = None) → Dict[str, object]

Performs the training of the neural network model to fit the provided training data.

Parameters:

• model (NeuralNetwork) – model to train.

• dataset (Union[RegressionDataset, Tuple[RegressionDataset, RegressionDataset]]) – dataset to use for training and validation. This argument is used with mask_dataset (to define the corresponding masked values) and val_frac (to define the proportion of entries to use in the validation set).

• learning_parameters (Union[LearningParameters, List[LearningParameters]]) – parameters of the stochastic gradient descent algorithm.

• mask_dataset (Union[ MaskDataset, Tuple[MaskDataset, MaskDataset], None, Tuple[None, None] ], optional) – _description_, by default None

• train_samples (Optional[Sequence], optional) – samples to use for training. When used, the arguments dataset, mask_dataset and val_frac are disregarded. By default None.

• val_samples (Optional[Sequence], optional) – samples to use for validation, by default None.

• val_frac (Optional[float], optional) – proportion of elements of the dataset to use in the validation set. If specified, should be between 0 and 1. By default None.

• additional_metrics (Optional[Dict[str, Callable[[torch.Tensor, torch.Tensor], float]]], optional) – metrics to track in addition to the loss function, by default None

• verbose_level (Literal[None, 0, 1, 2], optional) – amount of information provided during training. 0 or None: no display. 1: display of the epoch bar. 2: display of network description and also epoch and batch bars. Default: 1.

• seed (Optional[int], optional) – random seed for reproducibility, by default None.

• max_iter_no_improve (Optional[int], optional) – early stopping parameter. The training stops when the loss on the validation set does not decrease for max_iter_no_improve steps. By default None.

Returns:

This dictionary contains: - “train_loss”: the time series of the average train loss per epoch. - “val_loss”: the time series of the validation loss per epoch. - “train_metrics”: the time series of additional metrics on train set. - “val_metrics”: the time series of additional metrics on test set. - “train_set”: train_set. - “val_set”: val_set. - “lr”: the time series of the learning rate per epoch. - “batch_size”: the time series of the batch size per epoch. - “duration”: total duration of training.

Return type:

Dict[str, object]

Raises:

• TypeError – The dataset argument must be an instance of “RegressionDataset” or a tuple of two “RegressionDataset”.

• TypeError – The mask_dataset argument must be an instance of “MaskDataset” or a tuple of two “MaskDataset” or None.

• ValueError – The mask_dataset argument must not be a tuple when the dataset argument is a “RegressionDataset”.

• ValueError – The dataset argument must not be a tuple when mask_dataset is a “MaskDataset”.

• ValueError – The train dataset and validation dataset must not share samples.

• ValueError – The training dataset must not contain non finite values.

• ValueError – The learning_parameters argument must be an instance of “LearningParameters” or a list of “LearningParameters”.

• ValueError – The learning_parameter.loss_function must not be an instance of “MaskedLossFunction” when mask_dataset=None.

Module contents