bluemath_tk.deeplearning package

Submodules

bluemath_tk.deeplearning.autoencoders module

Autoencoders module.

This module is a pytorch translation from a tensorflow implementation developed by Sergio López Dubón.

This module contains the following autoencoders: - StandardAutoencoder - OrthogonalAutoencoder - LSTMAutoencoder - CNNAutoencoder - VisionTransformerAutoencoder - ConvLSTMAutoencoder - HybridConvLSTMTransformerAutoencoder

Each autoencoder is a subclass of BaseDeepLearningModel and implements the following methods: - fit(X, y=None, epochs=10, batch_size=32, verbose=1) - predict(X) - encode(X) - decode(X) - evaluate(X)

class bluemath_tk.deeplearning.autoencoders.CNNAutoencoder(k: int = 20, device: device | None = None, **kwargs)

Bases: BaseDeepLearningModel

Convolutional autoencoder for spatial grid data (images).

Uses 2D convolutions for encoding and transposed convolutions for decoding. Designed for 2D spatial data like images or gridded data.

Input Shape

Xnp.ndarray

Input data with shape (n_samples, C, H, W) - channels-first format. - n_samples: number of images - C: number of channels (e.g., 1 for grayscale, 3 for RGB) - H, W: height and width of the image Note: Only channels-first format is supported for consistency.

Examples

>>> # Single images (channels-first format required)
>>> X = np.random.randn(100, 3, 64, 64)  # 100 images, 3 channels, 64x64
>>> ae = CNNAutoencoder(k=20)
>>> history = ae.fit(X, epochs=10)
>>> X_recon = ae.predict(X)  # Shape: (100, 3, 64, 64)
>>> Z = ae.encode(X)  # Latent representations: (100, 20)

param k:

Number of latent dimensions. Default is 20.

type k:

int, optional

param device:

Device to run the model on. Default is None.

type device:

str or torch.device, optional

param **kwargs:

Additional keyword arguments passed to BaseDeepLearningModel.

class bluemath_tk.deeplearning.autoencoders.ConvLSTMAutoencoder(k: int = 20, device: device | None = None, **kwargs)

Bases: BaseDeepLearningModel

ConvLSTM autoencoder for spatiotemporal data (image sequences).

Combines convolutional and LSTM layers for spatiotemporal sequences. Designed for video-like data or time series of images.

Input Shape

Xnp.ndarray

Input data with shape (n_samples, seq_len, C, H, W). - n_samples: number of sequences - seq_len: number of frames in each sequence (automatically inferred from X.shape[1]) - C: number of channels (e.g., 1 for grayscale, 3 for RGB) - H, W: height and width of each frame

Examples

>>> # Video-like data (time series of images)
>>> X = np.random.randn(100, 10, 3, 64, 64)  # 100 sequences, 10 frames, 3 channels, 64x64
>>> ae = ConvLSTMAutoencoder(k=20)
>>> history = ae.fit(X, epochs=10)
>>> X_recon = ae.predict(X)  # Shape: (100, 3, 64, 64) - single frame reconstruction
>>> Z = ae.encode(X)  # Latent representations: (100, 20)

param k:

Number of latent dimensions, by default 20.

type k:

int, optional

param device:

Device to run the model on.

type device:

str or torch.device, optional

param **kwargs:

Additional keyword arguments passed to BaseDeepLearningModel.

class bluemath_tk.deeplearning.autoencoders.HybridConvLSTMTransformerAutoencoder(k: int = 20, d_model: int = 256, n_heads: int = 4, n_layers: int = 2, efficient_attention: str | None = 'linear', device: device | None = None, **kwargs)

Bases: BaseDeepLearningModel

Hybrid ConvLSTM + Transformer autoencoder for spatiotemporal data.

Combines ConvLSTM for spatiotemporal encoding with Transformer attention for temporal modeling. Designed for complex spatiotemporal patterns (video-like data or time series of images).

Input Shape

Xnp.ndarray

Input data with shape (n_samples, seq_len, C, H, W). - n_samples: number of sequences - seq_len: number of frames in each sequence (automatically inferred from X.shape[1]) - C: number of channels (e.g., 1 for grayscale, 3 for RGB) - H, W: height and width of each frame

Examples

>>> # Complex spatiotemporal data
>>> X = np.random.randn(100, 10, 3, 64, 64)  # 100 sequences, 10 frames, 3 channels, 64x64
>>> ae = HybridConvLSTMTransformerAutoencoder(k=20, d_model=256)
>>> history = ae.fit(X, epochs=10)
>>> X_recon = ae.predict(X)  # Shape: (100, 3, 64, 64) - single frame reconstruction
>>> Z = ae.encode(X)  # Latent representations: (100, 20)

param k:

Number of latent dimensions, by default 20.

type k:

int, optional

param d_model:

Model dimension, by default 256.

type d_model:

int, optional

param n_heads:

Number of attention heads, by default 4.

type n_heads:

int, optional

param n_layers:

Number of transformer layers, by default 2.

type n_layers:

int, optional

param efficient_attention:

Use ‘linear’ for efficient linear attention, None for standard MHA, by default ‘linear’.

type efficient_attention:

str, optional

param device:

Device to run the model on.

type device:

str or torch.device, optional

param **kwargs:

Additional keyword arguments passed to BaseDeepLearningModel.

class bluemath_tk.deeplearning.autoencoders.LSTMAutoencoder(k: int = 20, hidden: Tuple[int, int] = (256, 128), device: device | None = None, **kwargs)

Bases: BaseDeepLearningModel

LSTM-based autoencoder for sequential/temporal data.

Uses LSTM cells for encoding and decoding temporal sequences. Designed for time series data (not images or tabular data).

Input Shape

Xnp.ndarray

Input data with shape (n_samples, seq_len, n_features). - n_samples: number of sequences - seq_len: length of each sequence (automatically inferred from X.shape[1]) - n_features: number of features per timestep

Examples

>>> # Time series data (e.g., sensor readings over time)
>>> X = np.random.randn(100, 10, 5)  # 100 sequences, 10 timesteps, 5 features
>>> ae = LSTMAutoencoder(k=20, hidden=(256, 128))
>>> history = ae.fit(X, epochs=10)
>>> X_recon = ae.predict(X)  # Shape: (100, 10, 5)
>>> Z = ae.encode(X)  # Latent representations: (100, 20)

param k:

Number of latent dimensions, by default 20.

type k:

int, optional

param hidden:

Hidden layer dimensions for LSTM, by default (256, 128).

type hidden:

tuple, optional

param device:

Device to run the model on.

type device:

str or torch.device, optional

param **kwargs:

Additional keyword arguments passed to BaseDeepLearningModel.

class bluemath_tk.deeplearning.autoencoders.OrthogonalAutoencoder(k: int = 20, hidden_dims: list | None = None, lambda_W: float = 0.001, lambda_Z: float = 0.01, device: device | None = None, **kwargs)

Bases: BaseDeepLearningModel

Orthogonal autoencoder with orthogonal regularization.

Adds orthogonality constraints on encoder weights and latent decorrelation to encourage more interpretable latent representations. Designed for tabular/flattened data (not images or sequences).

Input Shape

Xnp.ndarray

Input data with shape (n_samples, n_features) or (n_samples,). - For 2D arrays: (n_samples, n_features) - each row is a sample - For 1D arrays: (n_features,) - single sample (will be reshaped) The model automatically flattens multi-dimensional inputs.

Examples

>>> # Tabular data with orthogonal constraints
>>> X = np.random.randn(1000, 784)  # 1000 samples, 784 features
>>> ae = OrthogonalAutoencoder(k=20, lambda_W=1e-3, lambda_Z=1e-2)
>>> history = ae.fit(X, epochs=10)
>>> Z = ae.encode(X)  # Decorrelated latent representations

param k:

Number of latent dimensions. Default is 20.

type k:

int, optional

param hidden_dims:

List of hidden layer dimensions. Default is [512, 256, 128, 64].

type hidden_dims:

list, optional

param lambda_W:

Weight orthogonality penalty strength. Default is 1e-3.

type lambda_W:

float, optional

param lambda_Z:

Latent decorrelation penalty strength. Default is 1e-2.

type lambda_Z:

float, optional

param device:

Device to run the model on. Default is None.

type device:

str or torch.device, optional

param **kwargs:

Additional keyword arguments passed to BaseDeepLearningModel.

fit(X: ndarray, y: ndarray | None = None, validation_split: float = 0.2, epochs: int = 500, batch_size: int = 64, learning_rate: float = 0.001, optimizer: Optimizer | None = None, criterion: Module | None = None, patience: int = 20, verbose: int = 1, **kwargs) → Dict[str, list]

Fit the orthogonal autoencoder with regularization losses.

This method overrides the base fit() to properly add orthogonality and decorrelation regularization losses during training.

class bluemath_tk.deeplearning.autoencoders.StandardAutoencoder(k: int = 20, hidden_dims: list | None = None, device: device | None = None, **kwargs)

Bases: BaseDeepLearningModel

Standard fully-connected autoencoder.

A simple feedforward autoencoder with symmetric encoder-decoder architecture. Designed for tabular/flattened data (not images or sequences).

Input Shape

Xnp.ndarray

Input data with shape (n_samples, n_features) or (n_samples,). - For 2D arrays: (n_samples, n_features) - each row is a sample - For 1D arrays: (n_features,) - single sample (will be reshaped) The model automatically flattens multi-dimensional inputs.

Examples

>>> # Tabular data (e.g., flattened features)
>>> X = np.random.randn(1000, 784)  # 1000 samples, 784 features
>>> ae = StandardAutoencoder(k=20, hidden_dims=[256, 128, 64])
>>> history = ae.fit(X, epochs=10)
>>> X_recon = ae.predict(X)
>>> Z = ae.encode(X)  # Get latent representations (1000, 20)

param k:

Number of latent dimensions. Default is 20.

type k:

int, optional

param hidden_dims:

List of hidden layer dimensions for encoder (decoder is symmetric). Default is [512, 256, 128, 64].

type hidden_dims:

list, optional

param device:

Device to run the model on. Default is None.

type device:

str or torch.device, optional

param **kwargs:

Additional keyword arguments passed to BaseDeepLearningModel.

class bluemath_tk.deeplearning.autoencoders.VisionTransformerAutoencoder(k: int = 20, patch_size: int = 8, d_model: int = 256, depth_enc: int = 4, depth_dec: int = 2, heads: int = 4, device: device | None = None, **kwargs)

Bases: BaseDeepLearningModel

Vision Transformer (ViT) autoencoder for spatial grid data (images).

Uses patch-based processing with transformer architecture. Designed for 2D spatial data like images or gridded data.

Input Shape

Xnp.ndarray

Input data with shape (n_samples, C, H, W) - channels-first format. - n_samples: number of images - C: number of channels (e.g., 1 for grayscale, 3 for RGB) - H, W: height and width of the image Note: Only channels-first format is supported.

Examples

>>> # Single images (channels-first format required)
>>> X = np.random.randn(100, 3, 64, 64)  # 100 images, 3 channels, 64x64
>>> ae = VisionTransformerAutoencoder(k=20, patch_size=8, d_model=256)
>>> history = ae.fit(X, epochs=10)
>>> X_recon = ae.predict(X)  # Shape: (100, 3, 64, 64)
>>> Z = ae.encode(X)  # Latent representations: (100, 20)

param k:

Number of latent dimensions, by default 20.

type k:

int, optional

param patch_size:

Size of each patch, by default 8.

type patch_size:

int, optional

param d_model:

Model dimension, by default 256.

type d_model:

int, optional

param depth_enc:

Number of encoder transformer blocks, by default 4.

type depth_enc:

int, optional

param depth_dec:

Number of decoder transformer blocks, by default 2.

type depth_dec:

int, optional

param heads:

Number of attention heads, by default 4.

type heads:

int, optional

param device:

Device to run the model on.

type device:

str or torch.device, optional

param **kwargs:

Additional keyword arguments passed to BaseDeepLearningModel.

bluemath_tk.deeplearning.gp_models module

Gaussian Process models module.

This module contains Gaussian Process Regression models using GPyTorch.

Classes: - BaseGPRModel: Base class for all GP models - ExactGPModel: Exact Gaussian Process Regression model

1. Wang, Z., Leung, M., Mukhopadhyay, S., et al. (2024). “A hybrid statistical–dynamical framework for compound coastal flooding analysis.” Environmental Research Letters, 20(1), 014005.

2. Wang, Z., Leung, M., Mukhopadhyay, S., et al. (2025). “Compound coastal flooding in San Francisco Bay under climate change.” npj Natural Hazards, 2(1), 3.

class bluemath_tk.deeplearning.gp_models.BaseGPRModel(device: str | device | None = None, **kwargs)

Bases: BlueMathModel

Base class for Gaussian Process Regression models.

This class provides common functionality for all GP models, including: - GP-specific training with marginal log likelihood - Prediction with uncertainty quantification - Model save/load with likelihood handling

GP models differ from standard deep learning models in several ways: - Use marginal log likelihood (MLL) instead of standard loss functions - Require explicit training data setting via set_train_data() - Return distributions (mean + variance) rather than point estimates - Typically train on full dataset (no batching during training)

GP models inherit directly from BlueMathModel (not BaseDeepLearningModel) because their training and prediction workflows are fundamentally different from standard neural networks.

model

The GPyTorch model.

Type:

gpytorch.models.GP

device

The device (CPU/GPU) the model is on.

Type:

torch.device

is_fitted

Whether the model has been fitted.

Type:

bool

likelihood

The GP likelihood module.

Type:

gpytorch.likelihoods.Likelihood

mll

The marginal log likelihood objective.

Type:

gpytorch.mlls.MarginalLogLikelihood

fit(X: ndarray, y: ndarray, epochs: int = 200, learning_rate: float = 0.1, optimizer: Optimizer | None = None, patience: int = 30, verbose: int = 1, **kwargs) → Dict[str, list]

Fit the Gaussian Process model.

GP models use marginal log likelihood (MLL) optimization, which is fundamentally different from standard deep learning training.

Parameters:

• X (np.ndarray) – Training input data with shape (n_samples, n_features).

• y (np.ndarray) – Training target data with shape (n_samples,) or (n_samples, 1).

• epochs (int, optional) – Maximum number of training epochs. Default is 200.

• learning_rate (float, optional) – Learning rate for optimizer. Default is 0.1.

• optimizer (torch.optim.Optimizer, optional) – Optimizer to use. If None, uses Adam. Default is None.

• patience (int, optional) – Early stopping patience. Default is 30.

• verbose (int, optional) – Verbosity level. Default is 1.

• **kwargs – Additional keyword arguments passed to _build_model.

Returns:

Training history with ‘train_loss’ key (negative MLL).

Return type:

Dict[str, list]

load_pytorch_model(model_path: str, **kwargs)

Load a GP model from a file.

Parameters:

• model_path (str) – Path to the file where the model is saved.

• **kwargs – Additional arguments for torch.load.

predict(X: ndarray, batch_size: int | None = None, return_std: bool = False, verbose: int = 1) → ndarray | Tuple[ndarray, ndarray]

Make predictions with the Gaussian Process model.

GP models return distributions, so predictions include uncertainty estimates (standard deviation) by default.

Parameters:

• X (np.ndarray) – Input data with shape (n_samples, n_features).

• batch_size (int, optional) – Batch size for prediction. If None, processes all at once. Default is None.

• return_std (bool, optional) – If True, returns both mean and standard deviation. Default is False.

• verbose (int, optional) – Verbosity level. Default is 1.

Returns:

If return_std=False: predictions (mean) with shape (n_samples,). If return_std=True: tuple of (mean, std) both with shape (n_samples,).

Return type:

np.ndarray or tuple

Raises:

ValueError – If model is not fitted.

save_pytorch_model(model_path: str, **kwargs)

Save the GP model to a file.

GP models require saving both the model and likelihood state dicts.

Parameters:

• model_path (str) – Path to the file where the model will be saved.

• **kwargs – Additional arguments for torch.save.

class bluemath_tk.deeplearning.gp_models.ExactGPModel(kernel: str = 'rbf+matern', ard_num_dims: int | None = None, device: device | None = None, **kwargs)

Bases: BaseGPRModel

Exact Gaussian Process Regression model using GPyTorch.

This model implements exact GP inference, suitable for datasets up to several thousand samples. For larger datasets, consider using approximate GP methods.

Parameters:

• kernel (str, optional) – Type of kernel to use. Options: ‘rbf’, ‘matern’, ‘rbf+matern’. Default is ‘rbf+matern’.

• ard_num_dims (int, optional) – Number of input dimensions for ARD (Automatic Relevance Determination). If None, will be inferred from data. Default is None.

• device (str or torch.device, optional) – Device to run the model on. Default is None (auto-detect).

• **kwargs – Additional keyword arguments passed to BaseGPRModel.

Examples

>>> import numpy as np
>>> from bluemath_tk.deeplearning import ExactGPModel
>>>
>>> # Generate sample data
>>> X = np.random.randn(100, 5)
>>> y = np.random.randn(100)
>>>
>>> # Create and fit model
>>> gp = ExactGPModel(kernel='rbf+matern')
>>> history = gp.fit(X, y, epochs=100, learning_rate=0.1)
>>>
>>> # Make predictions
>>> X_test = np.random.randn(50, 5)
>>> y_pred, y_std = gp.predict(X_test, return_std=True)

bluemath_tk.deeplearning.layers module

Project: BlueMath_tk Sub-Module: deeplearning.layers Author: GeoOcean Research Group, Universidad de Cantabria Repository: https://github.com/GeoOcean/BlueMath_tk.git Status: Under development (Working)

Custom PyTorch layers for deep learning models.

class bluemath_tk.deeplearning.layers.ConvLSTM(input_dim, hidden_dim, kernel_size=3, num_layers=1, batch_first=False, bias=True, return_all_layers=False)

Bases: Module

ConvLSTM module.

Parameters:

• input_dim (int) – Number of channels of input tensor.

• hidden_dim (int or list) – Number of channels of hidden state(s).

• kernel_size (int or tuple, optional) – Size of the convolutional kernel. Default is 3.

• num_layers (int, optional) – Number of ConvLSTM layers. Default is 1.

• batch_first (bool, optional) – If True, input and output tensors are provided as (batch, seq, channel, height, width). Default is False.

• bias (bool, optional) – Whether to add bias. Default is True.

• return_all_layers (bool, optional) – If True, returns all layers’ outputs. Default is False.

forward(input_tensor, hidden_state=None)

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class bluemath_tk.deeplearning.layers.ConvLSTMCell(input_dim, hidden_dim, kernel_size=3, bias=True)

Bases: Module

ConvLSTM Cell implementation.

Parameters:

• input_dim (int) – Number of channels of input tensor.

• hidden_dim (int) – Number of channels of hidden state.

• kernel_size (int or tuple, optional) – Size of the convolutional kernel. Default is 3.

• bias (bool, optional) – Whether to add bias. Default is True.

forward(input_tensor, cur_state)

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

init_hidden(batch_size, image_size)

class bluemath_tk.deeplearning.layers.DoubleConv(in_channels, out_channels, mid_channels=None, activation=None)

Bases: Module

Double convolution block: (convolution => [BN] => activation) * 2

Parameters:

• in_channels (int) – Number of input channels.

• out_channels (int) – Number of output channels.

• mid_channels (int, optional) – Number of intermediate channels. If None, uses out_channels.

• activation (nn.Module, optional) – Activation function. Default is SiLU.

forward(x)

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class bluemath_tk.deeplearning.layers.DoubleConv3D(in_channels, out_channels, mid_channels=None, activation=None)

Bases: Module

Double 3D convolution block: (convolution => [BN] => activation) * 2

Parameters:

• in_channels (int) – Number of input channels.

• out_channels (int) – Number of output channels.

• mid_channels (int, optional) – Number of intermediate channels. If None, uses out_channels.

• activation (nn.Module, optional) – Activation function. Default is SiLU.

forward(x)

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class bluemath_tk.deeplearning.layers.Down(in_channels, out_channels)

Bases: Module

Downscaling with maxpool then double conv.

Parameters:

• in_channels (int) – Number of input channels.

• out_channels (int) – Number of output channels.

forward(x)

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class bluemath_tk.deeplearning.layers.Down3D(in_channels, out_channels)

Bases: Module

Downscaling with maxpool then double 3D conv.

Parameters:

• in_channels (int) – Number of input channels.

• out_channels (int) – Number of output channels.

forward(x)

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class bluemath_tk.deeplearning.layers.LatentDecorr(strength: float = 0.01)

Bases: Module

Latent pass-through layer that adds covariance decorrelation loss.

This layer encourages the latent representations to be decorrelated by penalizing off-diagonal elements of the covariance matrix.

Parameters:

strength (float, optional) – Strength of the decorrelation penalty, by default 1e-2.

forward(z)

Forward pass with decorrelation loss.

Parameters:

z (torch.Tensor) – Latent representations, shape (batch, k).

Returns:

Unchanged latent representations.

Return type:

torch.Tensor

class bluemath_tk.deeplearning.layers.LinearSelfAttention(d_model: int, num_heads: int = 4)

Bases: Module

Softmax-free, Performer-style linear attention on the time axis.

Provides O(B * L * D * H) scaling, good for large sequence lengths.

Parameters:

• d_model (int) – Model dimension.

• num_heads (int, optional) – Number of attention heads, by default 4.

forward(x)

Forward pass.

Parameters:

x (torch.Tensor) – Input sequences, shape (B, L, D).

Returns:

Output sequences, shape (B, L, D).

Return type:

torch.Tensor

class bluemath_tk.deeplearning.layers.OutConv(in_channels, out_channels)

Bases: Module

Output convolution layer.

Parameters:

• in_channels (int) – Number of input channels.

• out_channels (int) – Number of output channels.

forward(x)

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class bluemath_tk.deeplearning.layers.OutConv3D(in_channels, out_channels)

Bases: Module

Output 3D convolution layer.

Parameters:

• in_channels (int) – Number of input channels.

• out_channels (int) – Number of output channels.

forward(x)

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class bluemath_tk.deeplearning.layers.Patchify(patch_size: int)

Bases: Module

Patchify layer that splits images into patches.

Parameters:

patch_size (int) – Size of each patch (patch_size x patch_size).

forward(x)

Forward pass.

Parameters:

x (torch.Tensor) – Input images, shape (B, C, H, W), where H and W are multiples of p.

Returns:

Patches, shape (B, N, p*p*C).

Return type:

torch.Tensor

class bluemath_tk.deeplearning.layers.PositionalEmbedding(n_tokens: int, d_model: int)

Bases: Module

Learnable positional embedding layer for transformer models.

Parameters:

• n_tokens (int) – Number of tokens/patches.

• d_model (int) – Model dimension.

forward(x)

Forward pass.

Parameters:

x (torch.Tensor) – Input tokens, shape (B, N, D).

Returns:

Tokens with positional embeddings added.

Return type:

torch.Tensor

class bluemath_tk.deeplearning.layers.TimePositionalEncoding(*args, **kwargs)

Bases: Module

Sinusoidal positional encoding for temporal sequences.

Adds 1D sinusoidal time positions to per-timestep embeddings.

forward(x)

Forward pass.

Parameters:

x (torch.Tensor) – Input sequences, shape (B, L, D).

Returns:

Sequences with positional encodings added.

Return type:

torch.Tensor

class bluemath_tk.deeplearning.layers.TripleConv(in_channels, out_channels, mid_channels=None, kernel_size=7)

Bases: Module

Triple convolution with separable spatial convolutions. Uses (1, kernel_size) and (kernel_size, 1) convolutions then combines.

Parameters:

• in_channels (int) – Number of input channels.

• out_channels (int) – Number of output channels.

• mid_channels (int, optional) – Number of intermediate channels. If None, uses out_channels.

• kernel_size (int, optional) – Kernel size for separable convolutions. Must be 3, 5, or 7. Default is 7.

forward(x)

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class bluemath_tk.deeplearning.layers.Unpatchify(patch_size: int, Hp: int, Wp: int, C: int)

Bases: Module

Unpatchify layer that reconstructs images from patches.

Parameters:

• patch_size (int) – Size of each patch.

• Hp (int) – Number of patches in height dimension.

• Wp (int) – Number of patches in width dimension.

• C (int) – Number of channels.

forward(tokens)

Forward pass.

Parameters:

tokens (torch.Tensor) – Patches, shape (B, N=Hp*Wp, p*p*C).

Returns:

Reconstructed images, shape (B, C, H, W).

Return type:

torch.Tensor

class bluemath_tk.deeplearning.layers.Up(in_channels, out_channels, bilinear=True)

Bases: Module

Upscaling then double conv.

Parameters:

• in_channels (int) – Number of input channels.

• out_channels (int) – Number of output channels.

• bilinear (bool, optional) – Whether to use bilinear upsampling. Default is True.

forward(x1, x2)

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class bluemath_tk.deeplearning.layers.Up3D(in_channels, out_channels, trilinear=True)

Bases: Module

Upscaling then double 3D conv.

Parameters:

• in_channels (int) – Number of input channels.

• out_channels (int) – Number of output channels.

• trilinear (bool, optional) – Whether to use trilinear upsampling. Default is True.

forward(x1, x2)

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

bluemath_tk.deeplearning.regularizers module

Project: BlueMath_tk Sub-Module: deeplearning.regularizers Author: GeoOcean Research Group, Universidad de Cantabria Repository: https://github.com/GeoOcean/BlueMath_tk.git Status: Under development (Working)

Regularization functions for PyTorch models.

bluemath_tk.deeplearning.regularizers.l1_regularizer(parameters, strength: float = 0.0001) → Tensor

L1 regularization (sparsity).

Parameters:

• parameters (iterable of torch.Tensor) – Model parameters to regularize.

• strength (float, optional) – Strength of the L1 penalty, by default 1e-4.

Returns:

Scalar penalty value.

Return type:

torch.Tensor

Examples

>>> import torch.nn as nn
>>> from bluemath_tk.deeplearning.regularizers import l1_regularizer
>>> model = nn.Linear(10, 5)
>>> penalty = l1_regularizer(model.parameters(), strength=1e-4)

bluemath_tk.deeplearning.regularizers.l2_regularizer(parameters, strength: float = 0.0001) → Tensor

L2 regularization (weight decay).

Parameters:

• parameters (iterable of torch.Tensor) – Model parameters to regularize.

• strength (float, optional) – Strength of the L2 penalty, by default 1e-4.

Returns:

Scalar penalty value.

Return type:

torch.Tensor

Examples

>>> import torch.nn as nn
>>> from bluemath_tk.deeplearning.regularizers import l2_regularizer
>>> model = nn.Linear(10, 5)
>>> penalty = l2_regularizer(model.parameters(), strength=1e-4)

bluemath_tk.deeplearning.regularizers.orthogonal_regularizer(W: Tensor, strength: float = 0.001) → Tensor

Weight orthogonality regularizer.

Encourages the weight matrix W to be orthogonal by penalizing deviations of W^T W from the identity matrix.

Parameters:

• W (torch.Tensor) – Weight matrix, shape (out_features, in_features).

• strength (float, optional) – Strength of the orthogonality penalty, by default 1e-3.

Returns:

Scalar penalty value.

Return type:

torch.Tensor

Examples

>>> import torch
>>> from bluemath_tk.deeplearning.regularizers import orthogonal_regularizer
>>> W = torch.randn(20, 128)
>>> penalty = orthogonal_regularizer(W, strength=1e-3)

bluemath_tk.deeplearning.unet module

Unet module.

This module is will try to generalize models like the ones in: https://github.com/oaeen/wind2iwp

class bluemath_tk.deeplearning.unet.UNet(n_channels=1, n_classes=1, base_channels=64, bilinear=True)

Bases: Module

U-Net architecture for 2D image segmentation/regression.

Parameters:

• n_channels (int) – Number of input channels.

• n_classes (int) – Number of output channels/classes.

• base_channels (int, optional) – Base number of channels. Default is 64.

• bilinear (bool, optional) – Whether to use bilinear upsampling. Default is True.

forward(x)

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class bluemath_tk.deeplearning.unet.UNet3D(n_channels=1, n_classes=1, base_channels=16, trilinear=True)

Bases: Module

U-Net architecture for 3D volumetric data.

Parameters:

• n_channels (int) – Number of input channels.

• n_classes (int) – Number of output channels/classes.

• base_channels (int, optional) – Base number of channels. Default is 16.

• trilinear (bool, optional) – Whether to use trilinear upsampling. Default is True.

forward(x)

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

Module contents

Project: BlueMath_tk Sub-Module: deeplearning Author: GeoOcean Research Group, Universidad de Cantabria Repository: https://github.com/GeoOcean/BlueMath_tk.git Status: Under development (Working)

PyTorch-based deep learning module for BlueMath_tk.