langml.transformer.encoder

Yet another transformer implementation.

Module Contents

Classes

TransformerEncoder
TransformerEncoderBlock

class langml.transformer.encoder.TransformerEncoder(attention_heads: int, hidden_dim: int, attention_activation: langml.tensor_typing.Activation = None, feed_forward_activation: langml.tensor_typing.Activation = gelu, dropout_rate: float = 0.0, trainable: bool = True, name: str = 'Transformer-Encoder')

__call__(self, inputs: langml.tensor_typing.Tensors) → langml.tensor_typing.Tensors

class langml.transformer.encoder.TransformerEncoderBlock(blocks: int, attention_heads: int, hidden_dim: int, attention_activation: langml.tensor_typing.Activation = None, feed_forward_activation: langml.tensor_typing.Activation = gelu, dropout_rate: float = 0.0, trainable: bool = False, name: str = 'TransformerEncoderBlock', share_weights: bool = False)

__call__(self, inputs: langml.tensor_typing.Tensors) → langml.tensor_typing.Tensors