`langml.utils`

Module Contents

`deprecated_warning`(msg='this function is deprecated! it might be removed in a future version.')
`bio_decode`(tags: List[str]) → List[Tuple[int, int, str]]	Decode BIO tags
`load_variables`(checkpoint_path: str) → Callable	load variables from chechkpoint
`auto_tokenizer`(vocab_path: str, lowercase: bool = False) → langml.tokenizer.Tokenizer

langml.utils.deprecated_warning(msg='this function is deprecated! it might be removed in a future version.')[source]

langml.utils.bio_decode(tags: List[str]) → List[Tuple[int, int, str]][source]

Decode BIO tags

Examples: >>> bio_decode([‘B-PER’, ‘I-PER’, ‘O’, ‘B-ORG’, ‘I-ORG’, ‘I-ORG’]) >>> [(0, 1, ‘PER’), (3, 5, ‘ORG’)]

langml.utils.load_variables(checkpoint_path: str) → Callable[source]: load variables from chechkpoint

langml.utils.auto_tokenizer(vocab_path: str, lowercase: bool = False) → langml.tokenizer.Tokenizer[source]