sima_utils.transformer.model.language_model
===========================================

.. py:module:: sima_utils.transformer.model.language_model


Classes
-------

.. autoapisummary::

   sima_utils.transformer.model.language_model.LanguageModel


Module Contents
---------------

.. py:class:: LanguageModel



   Language model implementation.

   The language model consists of a stack of transformer layers and some layers after the last
   transformer layer to obtain the output probability or next token index.

   The implementation assumes a transformer is broken up into 3 parts.
   1. PreCacheModel: Pre cache model implements the transformer layer up to the batched
       matrix-multiply in the self-attention block.
   2. CacheModel: Cache model implements the self-attention block of a transformer layer without
       the qkv projection.
   3. PostCacheModel: Post cache model implements the transformer layer after the self-attention
       block, including the layers after the last transformer layer.


   .. py:method:: gen_files(gen_mode: sima_utils.transformer.model.base.FileGenMode, *, precision: sima_utils.transformer.model.base.FileGenPrecision | dict[str, sima_utils.transformer.model.base.FileGenPrecision] | None = None, log_level: int = logging.NOTSET, num_processes: int = 1, part: str | None = None, part_idx: int | None = None, resume: bool = False)

      Generates files based on the provided file generation mode.

      :param gen_mode: File generation mode.
      :param Precision: The precision to be used for Model SDK quantization mode.
      :param log_level: Logging level.
      :param part: Name of the part to be generated.
      :param part_idx: Specific index of the part to be generated. For pre/post model, the index is
                       the layer index; for cache model, the index is the token index.
      :param resume: Generate the files if missing.



   .. py:method:: run_model(eval_mode: sima_utils.transformer.model.base.EvalMode, ifms: list[numpy.ndarray]) -> list[numpy.ndarray]


   .. py:method:: calc_freq_real_imag(use_swa: bool) -> tuple[numpy.ndarray, numpy.ndarray]


   .. py:method:: get_embeddings_tensor() -> numpy.ndarray