sima_utils.transformer.tokenizer
================================

.. py:module:: sima_utils.transformer.tokenizer


Submodules
----------

.. toctree::
   :maxdepth: 1

   /pages/api_reference/python-autoapi/sima_utils/transformer/tokenizer/sentencepiece_model_pb2/index
   /pages/api_reference/python-autoapi/sima_utils/transformer/tokenizer/whisper_tokenizer/index