afe.ir.serializer.yaml_codec ============================ .. py:module:: afe.ir.serializer.yaml_codec .. autoapi-nested-parse:: An encoder/decoder interface for mapping Python objects to data structures that are handled natively by pyyaml. See the documentation on Codec for a more detailed description of details about the interface. Attributes ---------- .. autoapisummary:: afe.ir.serializer.yaml_codec.SCALAR_TYPES afe.ir.serializer.yaml_codec.EncState afe.ir.serializer.yaml_codec.DecState afe.ir.serializer.yaml_codec.EncStateV afe.ir.serializer.yaml_codec.DecStateV afe.ir.serializer.yaml_codec.ScalarCodecType afe.ir.serializer.yaml_codec.scalar_codec afe.ir.serializer.yaml_codec.null_value_codec afe.ir.serializer.yaml_codec.Fld Exceptions ---------- .. autoapisummary:: afe.ir.serializer.yaml_codec.EncodeError afe.ir.serializer.yaml_codec.DecodeError Classes ------- .. autoapisummary:: afe.ir.serializer.yaml_codec.Codec afe.ir.serializer.yaml_codec.AliasCodec afe.ir.serializer.yaml_codec.FieldEncoding afe.ir.serializer.yaml_codec.ConstructorEncoding Functions --------- .. autoapisummary:: afe.ir.serializer.yaml_codec.expect_mapping afe.ir.serializer.yaml_codec.const_codec afe.ir.serializer.yaml_codec.map_codec afe.ir.serializer.yaml_codec.lazy_codec afe.ir.serializer.yaml_codec.enum_codec afe.ir.serializer.yaml_codec.list_codec afe.ir.serializer.yaml_codec.homogeneous_tuple_codec afe.ir.serializer.yaml_codec.dict_codec afe.ir.serializer.yaml_codec.optional_codec afe.ir.serializer.yaml_codec.named_dict_codec afe.ir.serializer.yaml_codec.singleton_dict_codec afe.ir.serializer.yaml_codec.unique_id_codec afe.ir.serializer.yaml_codec.unique_id_dec_table afe.ir.serializer.yaml_codec.aliasable_codec afe.ir.serializer.yaml_codec.aliasable_codec_enc_table afe.ir.serializer.yaml_codec.aliasable_codec_dec_table afe.ir.serializer.yaml_codec.intrusive_tagged_union_codec afe.ir.serializer.yaml_codec.type_tagged_codec afe.ir.serializer.yaml_codec.dataclass_style_codec afe.ir.serializer.yaml_codec.adt_style_codec Module Contents --------------- .. py:data:: SCALAR_TYPES .. py:data:: EncState .. py:data:: DecState .. py:data:: EncStateV .. py:data:: DecStateV .. py:exception:: EncodeError An error during YAML encoding due to invalid input or failure to handle the given input. .. py:exception:: DecodeError An error during YAML decoding due to invalid input or failure to handle the given input. .. py:function:: expect_mapping(permitted_keys: Set[str], required_keys: Set[str], doc: Any, context: str) -> None Verify that the document is a mapping with the given keys, and raise a DecodeError otherwise. :param permitted_keys: Keys that may be keys of the mapping :param required_keys: Keys that must be keys of the mapping. Must be a subset of permitted_keys. :param doc: Document to check :param context: Description of the document, used in error messages .. py:class:: Codec Protocol for an encoder/decoder that converts between a Python object and a data structure that pyyaml can natively dump or load. The encoded form of data must consist of data types that pyyaml handles natively: None, Bool, int, float, str, tuple, list, and dict. Object identity is significant, since Pyyaml uses anchors to de-duplicate multiple references to the same object in the output. A codec may read and write files or other persistent storage on the side. The encode function converts a Python object to YAML. The decode function does the reverse, and it should behave like the inverse of encode. Data that is used by encoding or decoding is passed in the 'state' parameter. In the intended usage, each type's codec uses its own mixin-style fields in the state (if it needs any), so that different instances do not interact. .. py:method:: encode(state: EncStateV, obj: _A) -> Any :abstractmethod: Encode an object to YAML. :param state: Encoder state used for encoding :param obj: Object to encode :return: Python object suitable for passing to yaml.dump .. py:method:: decode(state: DecStateV, doc: Any) -> _A :abstractmethod: Decode an object from YAML. :param state: Encoder state used for encoding :param doc: Python object received from yaml.load to decode :return: Decoded object .. py:function:: const_codec(decoded: _A, encoded: Any) -> Codec[EncState, DecState, _A] An encoder that ignores its input and returns a constant value when encoding or decoding. :param decoded: The constant value to return when decoding. :param encoded: The constant value to return when encoding. :return: Constant codec for the given values .. py:function:: map_codec(wrap: Callable[[_A], _B], unwrap: Callable[[_B], _A], codec: Codec[EncState, DecState, _A]) -> Codec[EncState, DecState, _B] Convert a codec of one type to a codec of another type by applying transformations to the codec's input and output. Encoding calls unwrap, then codec.encode. Decoding calls codec.decode, then wrap. :param wrap: Transform from the original type to the new one :param unwrap: Transform from the new type to the original one :param codec: Codec of the underlying type :return: Codec of the new type .. py:function:: lazy_codec(thunk: Callable[[], Codec[EncState, DecState, _A]]) -> Codec[EncState, DecState, _A] Lazily construct a codec. This is intended to be used for self-referential codecs that are involved in recursive data structures, like tree_codec = ... lazy_codec(lambda: tree_codec) ... :param thunk: Thunk that returns a codec :return: Codec that behaves like the thunk's return value. The thunk is evaluated the first time a method of the codec is called. .. py:data:: ScalarCodecType .. py:data:: scalar_codec :type: ScalarCodecType .. py:function:: enum_codec(values: Dict[_A, str], description: str) -> Codec[EncState, DecState, _A] Create a codec for an enum, representing enum values as strings. :param values: Mapping from enum values to strings. :param description: A noun phrase describing this data type. It is used in error messages. :return: Codec for enum values. .. py:function:: list_codec(item: Codec[EncState, DecState, _A]) -> Codec[EncState, DecState, List[_A]] Create a codec for lists. :param item: Codec for list elements. :return: Codec for list. .. py:function:: homogeneous_tuple_codec(item: Codec[EncState, DecState, _A]) -> Codec[EncState, DecState, Tuple[_A, Ellipsis]] Create a codec for homogeneous tuples. :param item: Codec for tuple elements. :return: Codec for tuple. .. py:function:: dict_codec(key: Codec[EncState, DecState, _A], value: Codec[EncState, DecState, _B]) -> Codec[EncState, DecState, Dict[_A, _B]] Create a codec for a dict. :param key: Codec for dict keys :param value: Codec for dict values :return: Codec for dict .. py:function:: optional_codec(item: Codec[EncState, DecState, _A]) -> Codec[EncState, DecState, Optional[_A]] Create a codec for optional values. Since this codec assigns an interpretation to None, the given codec should not interpret None. :param item: Codec for values. :return: Codec for optional values. .. py:data:: null_value_codec .. py:function:: named_dict_codec(index: Dict[str, Codec], description: str, *, defaults: Dict[str, Any] = {}) -> Codec[EncState, DecState, Dict[str, Any]] Make a codec that encodes a dict having a predetermined set of string keys as a YAML mapping. This is a typical way of encoding Python class instances. This codec encodes and decodes Python dictionaries having the same keys as 'index'. Each dict item is encoded and decoded using the codec for that key in 'index'. The 'defaults' dict provides default values for the encoding. When an item compares equal to a default value, it is omitted from YAML. Missing items in YAML are materialized using the default value. For example, consider the following codec. :: c = named_dict_codec({'name': scalar_codec, 'quantity': scalar_codec}, defaults = {'quantity': 1}) Calling c.encode({'name': 'x', 'quantity': 1}) returns {'name': 'x'}. Fields are optional only in the encoded form. All fields are required in the decoded form. :param index: A mapping from strings to codecs. Encoded dicts must have the same keys as this mapping, and their values are encoded or decoded using these codecs. :param description: A noun phrase describing this data type. It is used in error messages. :param defaults: Default values for items in the index. If a default value is given, items matching that value (when compared using __eq__) are omitted in the encoded form. :return: Codec for dicts .. py:function:: singleton_dict_codec(key: str, codec: Codec[EncState, DecState, _A], description: str) -> Codec[EncState, DecState, _A] Make a codec that puts its encoded value in a YAML mapping. It creates a YAML mapping containing a single item. This is useful when combined with other codecs. Suppose there is a codec c, and c.encode(x) returns y. Then, singleton_dict_codec('data', c).encode(x) returns {'data': y}. That contains the same result as the original codec, wrapped in a mapping. :param key: Key to use in the YAML mapping :param codec: Codec for the value :param description: A noun phrase describing this data type. It is used in error messages. :return: Codec that wraps the encoded value in a mapping .. py:function:: unique_id_codec(make_new_id: Callable[[], int], get_dec_table: Callable[[DecState], Dict[int, int]]) -> Codec[EncState, DecState, int] Make a codec that encodes unique integer IDs. IDs are encoded as YAML integers. IDs are converted to new values when they are decoded so that the result is unique after decoding. :param make_new_id: How to make a new unique ID. This is called during decoding for every ID that appears in the YAML document. :param get_dec_table: Get decoder state for this codec to use and modify. The state should be initialized by calling unique_id_dec_table. :return: codec for unique integer IDs. .. py:function:: unique_id_dec_table() -> Dict[int, int] Create a decoder table suitable for use by unique_id_codec. .. py:class:: AliasCodec(get_enc_table: Callable[[EncState], Dict[int, Any]], get_dec_table: Callable[[DecState], Dict[int, Any]], codec: Codec[EncState, DecState, _A]) Abstract base class for generic types. A generic type is typically declared by inheriting from this class parameterized with one or more type variables. For example, a generic mapping type might be defined as:: class Mapping(Generic[KT, VT]): def __getitem__(self, key: KT) -> VT: ... # Etc. This class can then be used as follows:: def lookup_name(mapping: Mapping[KT, VT], key: KT, default: VT) -> VT: try: return mapping[key] except KeyError: return default .. py:method:: encode(state: EncState, obj: _A) -> Any .. py:method:: decode(state: DecState, doc: Any) -> _A .. py:function:: aliasable_codec(get_enc_table: Callable[[EncState], Dict[int, Any]], get_dec_table: Callable[[DecState], Dict[int, Any]], codec: Codec[EncState, DecState, _A]) -> Codec[EncState, DecState, _A] Make a codec that applies YAML anchors to encoded data and uses anchors for repeated references to the same object. An object will be encoded at most once and will be recreated once during decoding, no matter how many times it appears. This is done by memoizing the encoded representation so that the yaml library will use anchors. Thus, it depends on the yaml library to use anchors and memoization when the yaml library encounters the same object in its input. :param get_enc_table: Get encoder state for this codec to use and modify. The state should be initialized by calling aliasable_codec_enc_table. :param get_dec_table: Get decoder state for this codec to use and modify. The state should be initialized by calling aliasable_codec_dec_table. :param codec: Codec to extend with aliasing :return: Codec that uses anchors for aliasing .. py:function:: aliasable_codec_enc_table() -> Dict[int, Any] Create an encoder table suitable for use by aliasable_codec. .. py:function:: aliasable_codec_dec_table() -> Dict[int, Any] Create a decoder table suitable for use by aliasable_codec. .. py:function:: intrusive_tagged_union_codec(tag_name: str, codecs: Dict[str, Codec[EncState, DecState, _A]], get_tag: Callable[[_A], str]) -> Codec[EncState, DecState, _A] Make a codec for a union-like type to be encoded as a tagged union. The encoding is a YAML mapping, where one key in the mapping has a string tag and the other keys are tag-dependent. For example, consider the following codec for Union[int, str]. :: value_codec = singleton_dict_codec('number', scalar_codec) info_codec = singleton_dict_codec('message', scalar_codec) def get_tag(x): return 'info' if isinstance(x, str) else 'value' c = intrusive_tagged_union_codec('type', {'value': value_codec, 'info': message_codec}, get_tag) Calling c.encode(3) returns {'type': 'value', 'number': 3}. Calling c.encode('Three') returns {'type': 'info', 'message': 'Three'}. The dict key 'type' is the tag indicating which union member it is. The other keys hold the rest of the data. :param tag_name: Name of the tag in encoded YAML mappings. :param codecs: Alternative codecs, indexed by tag. :param get_tag: Selects which codec to use for encoding a value. :return: Codec for the tagged union .. py:function:: type_tagged_codec(tag_name: str, codecs: Sequence[Tuple[str, type, Codec[EncState, DecState, _A]]]) -> Codec[EncState, DecState, _A] Make a tagged union codec where each tag stands for a distinct type. :param codecs: Tag, type, and codec of each alternative encoding. :return: Codec for the tagged union .. py:class:: FieldEncoding Describes the encoding of one class field as YAML data when encoding a class as a mapping. :param python_name: Name of the field in Python source code. :param yaml_name: Name of the field in a YAML mapping. :param codec: Encoding of the field's value. :param default: Field's default value, used if the field is omitted in the YAML mapping. .. py:attribute:: python_name :type: str .. py:attribute:: yaml_name :type: str .. py:attribute:: codec :type: Codec[EncState, DecState, Any] .. py:attribute:: default :type: Any .. py:data:: Fld .. py:function:: dataclass_style_codec(cls: type, fields: Sequence[FieldEncoding]) -> Codec[EncState, DecState, _A] Make a codec that translates a class to a mapping with named fields, for classes that resemble dataclasses. To use this codec, a class must have a dataclass-style constructor so that it's possible to clone with this pattern of field accesses and constructor call: :: copy_of_instance = MyClass(instance.field1, instance.field2, instance.field3) Python reflection is used to access fields by name. :param cls: The Python class to create the codec for :param fields: The class's fields :return: Codec for the class .. py:class:: ConstructorEncoding An encoding of one class instance as YAML data as part of an encoding scheme for algebraic data types. * If a sequence of one or more field encodings is given, a dataclass_style_codec encoding is used. * If an empty sequence is given, a const_codec encoding is used. * If a codec is given, the codec is used to encode an instance of the class. Its encoded form must be a mapping. The algebraic data type tag will be injected into the mapping. :param cls: The Python class that this constructor represents :param yaml_name: The string used to represent that class in YAML :param fields: Encoding of the class's fields. Either a list giving the encoding of each field, or a Codec giving the encoding of an entire class instance as a YAML mapping. .. py:attribute:: cls :type: type .. py:attribute:: yaml_name :type: str .. py:attribute:: fields :type: Union[Sequence[FieldEncoding], Codec[EncState, DecState, Any]] .. py:function:: adt_style_codec(tag_name: str, members: Sequence[ConstructorEncoding]) -> Codec[EncState, DecState, _A] Make a codec for data in the style of an algebraic data type. The decoded data is a union of the types given in members. The encoded data is a mapping constructed with intrusive_tagged_union_codec. It has a tag indicating which type is encoded there. :param tag_name: The name of the type tag in the encoded dict. :param members: The members of the data type. An instance of the type is an instance of one member. :return: Codec for the type.