Advantages of Protobuf for Serialization in Python

<p>Would you like a way to send structured serialized data between different platforms and languages? What if the data was self-documenting, could automatically generate Python code, and would validate itself? This week on the show, Liran Haimovitch talks about protocol buffers and communicating with microservices through Remote Procedure Calls (RPC).</p> <p>Protocol buffers, aka protobuf, are a language-neutral, platform-neutral system for serializing structured data. Liran talks about how they go beyond text-based protocols like JSON, providing the benefits above, along with faster transmissions and a smaller footprint.</p> <p>Liran shares how his company uses protobuf to communicate between their tools. We also discuss using gRPC to communicate between microservices and scaling infrastructure in either direction.</p> <div class="alert alert-primary" role="alert"> <p><strong>Course Spotlight:</strong> <a href="https://realpython.com/courses/testing-your-code-with-pytest/">Testing Your Code With pytest</a></p> <p>In this video course, you&rsquo;ll learn how to take your testing to the next level with pytest. You&rsquo;ll cover intermediate and advanced pytest features such as fixtures, marks, parameters, and plugins. With pytest, you can make your test suites fast, effective, and less painful to maintain.</p> </div> <p>Topics:</p> <ul> <li>00:00:00 &ndash; Introduction</li> <li>00:01:59 &ndash; PyCon US 2022 Talk on protobuf</li> <li>00:04:46 &ndash; PyCon 2019 Talk on Understanding Python&rsquo;s Debugging Internals</li> <li>00:05:34 &ndash; The Production-First Mindset Podcast</li> <li>00:07:03 &ndash; Protobuf and serialization</li> <li>00:11:17 &ndash; Static vs dynamic serializers</li> <li>00:13:58 &ndash; Text vs binary serializers and metadata</li> <li>00:21:08 &ndash; How long have you been using protobuf?</li> <li>00:21:40 &ndash; What does it look like to set up?</li> <li>00:24:45 &ndash; Video Course Spotlight</li> <li>00:26:11 &ndash; Performance challenges and trade-offs</li> <li>00:34:29 &ndash; Remote procedure calls</li> <li>00:41:13 &ndash; Using RPC for microservices</li> <li>00:47:21 &ndash; Scaling your infrastructure up or down</li> <li>00:50:35 &ndash; Working across different languages</li> <li>00:54:02 &ndash; What is Rookout?</li> <li>00:55:11 &ndash; What are you excited about in the world of Python?</li> <li>00:55:59 &ndash; What do you want to learn next?</li> <li>00:56:57 &ndash; How can people learn more about what you do?</li> <li>00:57:31 &ndash; Thanks and goodbye</li> </ul> <p>Show Links:</p> <ul> <li><a href="https://www.rookout.com/">Rookout | Painless Cloud-Native Debugging</a></li> <li><a href="https://www.youtube.com/watch?v=QU158nGABxI">Liran Haimovitch - Understanding Python’s Debugging Internals - PyCon 2019 - YouTube</a></li> <li><a href="https://www.productionfirstmindset.com/">The Production-First Mindset Podcast</a></li> <li><a href="https://www.youtube.com/watch?v=Eb6y1WbF2is&amp;list=PL2Uw4_HvXqvYeXy8ab7iRHjA-9HiYhRQl&amp;index=31">Liran Haimovitch: Effective Protobuf: Everything You Wanted To Know, But Never Dared To Ask - PyCon 2022 - YouTube</a></li> <li><a href="https://developers.google.com/protocol-buffers">Protocol Buffers  |  Google Developers</a></li> <li><a href="https://developers.google.com/protocol-buffers/docs/faq">Frequently Asked Questions - Protocol Buffers  |  Google Developers</a></li> <li><a href="https://thrift-tutorial.readthedocs.io/en/latest/thrift-stack.html">Thrift protocol stack — Thrift Tutorial 1.0 documentation</a></li> <li><a href="https://realpython.com/python-microservices-grpc/">Python Microservices With gRPC – Real Python</a></li> <li><a href="https://grpc.io/">gRPC - Modern Open Source High Performance Remote Procedure Call Framework</a></li> <li><a href="https://cloud.google.com/blog/products/api-management/understanding-grpc-openapi-and-rest-and-when-to-use-them">gRPC vs REST: Understanding gRPC, OpenAPI and REST and when to use them in API design | Google Cloud Blog</a></li> <li><a href="https://us.pycon.org/2022/">Welcome to PyCon US 2022</a></li> <li><a href="https://www.rust-lang.org/">Rust Programming Language</a></li> </ul> <p>Level up your Python skills with our expert-led courses:</p> <ul> <li><a href="https://realpython.com/courses/working-json-data-python/">Working With JSON in Python</a></li> <li><a href="https://realpython.com/courses/testing-your-code-with-pytest/">Testing Your Code With pytest</a></li> <li><a href="https://realpython.com/courses/deploy-python-script-web-flask/">Deploy Your Python Script on the Web With Flask</a></li> </ul> <p><a rel="payment" href="https://realpython.com/join">Support the podcast &amp; join our community of Pythonistas</a></p>