langml.third_party.crf
Module Contents
Classes
Abstract object representing an RNN cell. |
|
Computes the forward decoding in a linear-chain CRF. |
Functions
|
|
|
Generate a zero filled tensor with shape [batch_size, state_size]. |
|
Constrains the inputs to filter out certain tags at each time step. |
|
Computes the unnormalized score for a tag sequence. |
|
Computes the unnormalized score of all tag sequences matching |
|
Computes the normalization for a CRF. |
|
Computes the log-likelihood of tag sequences in a CRF. |
|
Computes the unary scores of tag sequences. |
|
Computes the binary scores of tag sequences. |
|
Computes the alpha values in a linear-chain CRF. |
|
Computes forward decoding in a linear-chain CRF. |
|
Computes backward decoding in a linear-chain CRF. |
|
Decode the highest scoring sequence of tags. |
|
Decode the highest scoring sequence of tags under constraints. |
- langml.third_party.crf.viterbi_decode(score: langml.tensor_typing.Tensors, trans: langml.tensor_typing.Tensors) Tuple[langml.tensor_typing.Tensors, langml.tensor_typing.Tensors][source]
- Parameters
score – A [seq_len, num_tags] matrix of unary potentials.
trans – A [num_tags, num_tags] matrix of binary potentials.
- Returns
- A [seq_len] list of integers containing the highest scoring tag
indices.
viterbi_score: A float containing the score for the Viterbi sequence.
- Return type
viterbi
- langml.third_party.crf._generate_zero_filled_state_for_cell(cell, inputs, batch_size, dtype)[source]
Generate a zero filled tensor with shape [batch_size, state_size].
- langml.third_party.crf.crf_filtered_inputs(inputs: langml.tensor_typing.Tensors, tag_bitmap: langml.tensor_typing.Tensors) tensorflow.Tensor[source]
Constrains the inputs to filter out certain tags at each time step. tag_bitmap limits the allowed tags at each input time step. This is useful when an observed output at a given time step needs to be constrained to a selected set of tags. Args: inputs: A [batch_size, max_seq_len, num_tags] tensor of unary potentials
to use as input to the CRF layer.
- tag_bitmap: A [batch_size, max_seq_len, num_tags] boolean tensor
representing all active tags at each index for which to calculate the unnormalized score.
Returns: filtered_inputs: A [batch_size] vector of unnormalized sequence scores.
- langml.third_party.crf.crf_sequence_score(inputs: langml.tensor_typing.Tensors, tag_indices: langml.tensor_typing.Tensors, sequence_lengths: langml.tensor_typing.Tensors, transition_params: langml.tensor_typing.Tensors) tensorflow.Tensor[source]
Computes the unnormalized score for a tag sequence. :param inputs: A [batch_size, max_seq_len, num_tags] tensor of unary potentials
to use as input to the CRF layer.
- Parameters
tag_indices – A [batch_size, max_seq_len] matrix of tag indices for which we compute the unnormalized score.
sequence_lengths – A [batch_size] vector of true sequence lengths.
transition_params – A [num_tags, num_tags] transition matrix.
- Returns
A [batch_size] vector of unnormalized sequence scores.
- Return type
sequence_scores
- langml.third_party.crf.crf_multitag_sequence_score(inputs: langml.tensor_typing.Tensors, tag_bitmap: langml.tensor_typing.Tensors, sequence_lengths: langml.tensor_typing.Tensors, transition_params: langml.tensor_typing.Tensors) tensorflow.Tensor[source]
Computes the unnormalized score of all tag sequences matching tag_bitmap. tag_bitmap enables more than one tag to be considered correct at each time step. This is useful when an observed output at a given time step is consistent with more than one tag, and thus the log likelihood of that observation must take into account all possible consistent tags. Using one-hot vectors in tag_bitmap gives results identical to crf_sequence_score. :param inputs: A [batch_size, max_seq_len, num_tags] tensor of unary potentials
to use as input to the CRF layer.
- Parameters
tag_bitmap – A [batch_size, max_seq_len, num_tags] boolean tensor representing all active tags at each index for which to calculate the unnormalized score.
sequence_lengths – A [batch_size] vector of true sequence lengths.
transition_params – A [num_tags, num_tags] transition matrix.
- Returns
A [batch_size] vector of unnormalized sequence scores.
- Return type
sequence_scores
- langml.third_party.crf.crf_log_norm(inputs: langml.tensor_typing.Tensors, sequence_lengths: langml.tensor_typing.Tensors, transition_params: langml.tensor_typing.Tensors) tensorflow.Tensor[source]
Computes the normalization for a CRF. :param inputs: A [batch_size, max_seq_len, num_tags] tensor of unary potentials
to use as input to the CRF layer.
- Parameters
sequence_lengths – A [batch_size] vector of true sequence lengths.
transition_params – A [num_tags, num_tags] transition matrix.
- Returns
A [batch_size] vector of normalizers for a CRF.
- Return type
log_norm
- langml.third_party.crf.crf_log_likelihood(inputs: langml.tensor_typing.Tensors, tag_indices: langml.tensor_typing.Tensors, sequence_lengths: langml.tensor_typing.Tensors, transition_params: Optional[langml.tensor_typing.Tensors] = None) tensorflow.Tensor[source]
Computes the log-likelihood of tag sequences in a CRF. :param inputs: A [batch_size, max_seq_len, num_tags] tensor of unary potentials
to use as input to the CRF layer.
- Parameters
tag_indices – A [batch_size, max_seq_len] matrix of tag indices for which we compute the log-likelihood.
sequence_lengths – A [batch_size] vector of true sequence lengths.
transition_params – A [num_tags, num_tags] transition matrix, if available.
- Returns
- A [batch_size] Tensor containing the log-likelihood of
each example, given the sequence of tag indices.
- transition_params: A [num_tags, num_tags] transition matrix. This is
either provided by the caller or created in this function.
- Return type
log_likelihood
- langml.third_party.crf.crf_unary_score(tag_indices: langml.tensor_typing.Tensors, sequence_lengths: langml.tensor_typing.Tensors, inputs: langml.tensor_typing.Tensors) tensorflow.Tensor[source]
Computes the unary scores of tag sequences. :param tag_indices: A [batch_size, max_seq_len] matrix of tag indices. :param sequence_lengths: A [batch_size] vector of true sequence lengths. :param inputs: A [batch_size, max_seq_len, num_tags] tensor of unary potentials.
- Returns
A [batch_size] vector of unary scores.
- Return type
unary_scores
- langml.third_party.crf.crf_binary_score(tag_indices: langml.tensor_typing.Tensors, sequence_lengths: langml.tensor_typing.Tensors, transition_params: langml.tensor_typing.Tensors) tensorflow.Tensor[source]
Computes the binary scores of tag sequences. :param tag_indices: A [batch_size, max_seq_len] matrix of tag indices. :param sequence_lengths: A [batch_size] vector of true sequence lengths. :param transition_params: A [num_tags, num_tags] matrix of binary potentials.
- Returns
A [batch_size] vector of binary scores.
- Return type
binary_scores
- langml.third_party.crf.crf_forward(inputs: langml.tensor_typing.Tensors, state: langml.tensor_typing.Tensors, transition_params: langml.tensor_typing.Tensors, sequence_lengths: langml.tensor_typing.Tensors) tensorflow.Tensor[source]
Computes the alpha values in a linear-chain CRF. See http://www.cs.columbia.edu/~mcollins/fb.pdf for reference. :param inputs: A [batch_size, num_tags] matrix of unary potentials. :param state: A [batch_size, num_tags] matrix containing the previous alpha
values.
- Parameters
transition_params – A [num_tags, num_tags] matrix of binary potentials. This matrix is expanded into a [1, num_tags, num_tags] in preparation for the broadcast summation occurring within the cell.
sequence_lengths – A [batch_size] vector of true sequence lengths.
- Returns
- A [batch_size, num_tags] matrix containing the
new alpha values.
- Return type
new_alphas
- class langml.third_party.crf.AbstractRNNCell[source]
Bases:
tensorflow.keras.layers.LayerAbstract object representing an RNN cell. This is the base class for implementing RNN cells with custom behavior. Every RNNCell must have the properties below and implement call with the signature (output, next_state) = call(input, state). Examples: ```python
class MinimalRNNCell(AbstractRNNCell): def __init__(self, units, **kwargs):
self.units = units super(MinimalRNNCell, self).__init__(**kwargs)
@property def state_size(self):
return self.units
- def build(self, input_shape):
- self.kernel = self.add_weight(shape=(input_shape[-1], self.units),
initializer=’uniform’, name=’kernel’)
- self.recurrent_kernel = self.add_weight(
shape=(self.units, self.units), initializer=’uniform’, name=’recurrent_kernel’)
self.built = True
- def call(self, inputs, states):
prev_output = states[0] h = K.dot(inputs, self.kernel) output = h + K.dot(prev_output, self.recurrent_kernel) return output, output
``` This definition of cell differs from the definition used in the literature. In the literature, ‘cell’ refers to an object with a single scalar output. This definition refers to a horizontal array of such units. An RNN cell, in the most abstract setting, is anything that has a state and performs some operation that takes a matrix of inputs. This operation results in an output matrix with self.output_size columns. If self.state_size is an integer, this operation also results in a new state matrix with self.state_size columns. If self.state_size is a (possibly nested tuple of) TensorShape object(s), then it should return a matching structure of Tensors having shape [batch_size].concatenate(s) for each s in self.batch_size.
- abstract call(self, inputs, states)[source]
The function that contains the logic for one RNN step calculation. Args: inputs: the input tensor, which is a slide from the overall RNN input by
the time dimension (usually the second dimension).
- states: the state tensor from previous step, which has the same shape
as (batch, state_size). In the case of timestep 0, it will be the initial state user specified, or zero filled tensor otherwise.
Returns: A tuple of two tensors:
output tensor for the current timestep, with size output_size.
state tensor for next step, which has the shape of state_size.
- class langml.third_party.crf.CrfDecodeForwardRnnCell(transition_params: langml.tensor_typing.Tensors, **kwargs)[source]
Bases:
AbstractRNNCellComputes the forward decoding in a linear-chain CRF.
- property state_size(self)[source]
size(s) of state(s) used by this cell. It can be represented by an Integer, a TensorShape or a tuple of Integers or TensorShapes.
- compute_mask(self, inputs: langml.tensor_typing.Tensors, mask: Optional[langml.tensor_typing.Tensors] = None) Union[List[Union[langml.tensor_typing.Tensors, None]], langml.tensor_typing.Tensors][source]
- call(self, inputs: langml.tensor_typing.Tensors, state: langml.tensor_typing.Tensors, mask: Optional[langml.tensor_typing.Tensors] = None, **kwargs)[source]
Build the CrfDecodeForwardRnnCell. :param inputs: A [batch_size, num_tags] matrix of unary potentials. :param state: A [batch_size, num_tags] matrix containing the previous step’s
score values.
- Returns
A [batch_size, num_tags] matrix of backpointers. new_state: A [batch_size, num_tags] matrix of new score values.
- Return type
backpointers
- classmethod from_config(cls, config: dict) CrfDecodeForwardRnnCell[source]
- langml.third_party.crf.crf_decode_forward(inputs: langml.tensor_typing.Tensors, state: langml.tensor_typing.Tensors, transition_params: langml.tensor_typing.Tensors, sequence_lengths: langml.tensor_typing.Tensors) tensorflow.Tensor[source]
Computes forward decoding in a linear-chain CRF. :param inputs: A [batch_size, num_tags] matrix of unary potentials. :param state: A [batch_size, num_tags] matrix containing the previous step’s
score values.
- Parameters
transition_params – A [num_tags, num_tags] matrix of binary potentials.
sequence_lengths – A [batch_size] vector of true sequence lengths.
- Returns
A [batch_size, num_tags] matrix of backpointers. new_state: A [batch_size, num_tags] matrix of new score values.
- Return type
backpointers
- langml.third_party.crf.crf_decode_backward(inputs: langml.tensor_typing.Tensors, state: langml.tensor_typing.Tensors) tensorflow.Tensor[source]
Computes backward decoding in a linear-chain CRF. :param inputs: A [batch_size, num_tags] matrix of
backpointer of next step (in time order).
- Parameters
state – A [batch_size, 1] matrix of tag index of next step.
- Returns
- A [batch_size, num_tags]
tensor containing the new tag indices.
- Return type
new_tags
- langml.third_party.crf.crf_decode(potentials: langml.tensor_typing.Tensors, transition_params: langml.tensor_typing.Tensors, sequence_length: langml.tensor_typing.Tensors) tensorflow.Tensor[source]
Decode the highest scoring sequence of tags. :param potentials: A [batch_size, max_seq_len, num_tags] tensor of
unary potentials.
- Parameters
transition_params – A [num_tags, num_tags] matrix of binary potentials.
sequence_length – A [batch_size] vector of true sequence lengths.
- Returns
- A [batch_size, max_seq_len] matrix, with dtype tf.int32.
Contains the highest scoring tag indices.
best_score: A [batch_size] vector, containing the score of decode_tags.
- Return type
decode_tags
- langml.third_party.crf.crf_constrained_decode(potentials: langml.tensor_typing.Tensors, tag_bitmap: langml.tensor_typing.Tensors, transition_params: langml.tensor_typing.Tensors, sequence_length: langml.tensor_typing.Tensors) tensorflow.Tensor[source]
Decode the highest scoring sequence of tags under constraints. This is a function for tensor. :param potentials: A [batch_size, max_seq_len, num_tags] tensor of
unary potentials.
- Parameters
tag_bitmap – A [batch_size, max_seq_len, num_tags] boolean tensor representing all active tags at each index for which to calculate the unnormalized score.
transition_params – A [num_tags, num_tags] matrix of binary potentials.
sequence_length – A [batch_size] vector of true sequence lengths.
- Returns
- A [batch_size, max_seq_len] matrix, with dtype tf.int32.
Contains the highest scoring tag indices.
best_score: A [batch_size] vector, containing the score of decode_tags.
- Return type
decode_tags