I was originally going to make a big post on pydantic and how we could offer typed messages using that very very nice project despite there being a couple holdups for integration with msgpack.

However, it turns out just today an even faster and msgpack specific project was released: msgspec 🏄🏼

It claims to not only be faster then msgpack-python but also supports schema evolution and other niceties
It also has perf bumps when making multiple repeated encode/decode calls which is exactly how we're currently using msgpack inside our Channel.

Overall there looks to be no downside and we'll get typed message semantics fast and free 👍🏼

For reference, I'll leave a bunch of links I'd previously gathered regarding making pydantic work with msgpack:

• Customizing both decoding and encoding of a type pydantic/pydantic#951
• https://pydantic-docs.helpmanual.io/usage/dataclasses/
• Add generic functionality pydantic/pydantic#595
• What is the correct way to define custom pydantic types that also play nicely with JSON Schema? fastapi/fastapi#1285
• https://github.com/MolSSI/QCElemental/blob/master/qcelemental/models/basemodels.py#L121
  • this is just adding a BaseModel.serialize() effectively which looks up a serialize method by name (eg. json, msgpack) but isn't really adding any "native feeling" support nor speed gains afaict.

TODO

• support for a msgpack-python custom type serializer for pydantic.BaseModel such that we just implicitly render with .dict() as pack time and load via `Model(**message)`` at decode time?
• write ourselves a small bytes-length prefixed framing protocol for msgspec as per the comments in Try msgspec #212
  • example from a blog post on protobuf
  • consider how we might wrap trio.SocketStream using something like tricycle.BufferedReceiveStream; @oremanj was nice enough to provide usage:

    while header := await stream.receive_all_or_none(4):
        len, = struct.unpack("<I", header)
        # probably want to sanity-check len for not being unreasonably huge
        chunk = await stream.receive_exactly(len)
        # do something with chunk

• consider offering msgspec as an optional dependency if we end up liking it?