Episode 194
Automate Tasks With Python & Building a Small Search Engine
The Real Python Podcast · Real Python
March 1, 202444m 36s
Audio is streamed directly from the publisher (dts.podtrac.com) as published in their RSS feed. Play Podcasts does not host this file. Rights-holders can request removal through the copyright & takedown page.
Show Notes
<p>What are the typical computer tasks you do manually every week? Could you automate those tasks with a Python script? Christopher Trudeau is back on the show this week, bringing another batch of PyCoder’s Weekly articles and projects.</p>
<p>We discuss a recent Hacker News thread about frequently used automation scripts. We share the kinds of tasks we’ve automated with Python in our work and personal lives. </p>
<p>Christopher shares a tutorial about building a micro-search engine from scratch using Python. The post takes you through coding the components of a crawler, index, and ranker. The finished engine is designed to search the posts of the blogs you follow.</p>
<p>We also share several other articles and projects from the Python community, including a news roundup, how a Polars query works under the hood, using Python for data analysis, understanding open-source licensing, summarizing the significant changes between Python versions, a robust TUI hex editor, and a lightweight dataframe library with a universal interface for data wrangling.</p>
<p>This week’s episode is brought to you by Intel.</p>
<div class="alert alert-primary" role="alert">
<p><strong>Course Spotlight:</strong> <a href="https://realpython.com/courses/python-argparse-command-line-interfaces/">Building Command Line Interfaces With argparse</a></p>
<p>In this step-by-step Python video course, you’ll learn how to take your command line Python scripts to the next level by adding a convenient command line interface that you can write with argparse.</p>
</div>
<p>Topics:</p>
<ul>
<li>00:00:00 – Introduction</li>
<li>00:02:23 – uv: Python Packaging in Rust</li>
<li>00:02:43 – Rye Grows With uv </li>
<li>00:03:20 – Python 3.13.0 Alpha 4 Is Now Available</li>
<li>00:03:45 – A Bird’s Eye View of Polars</li>
<li>00:07:28 – Polars: Why We Have Rewritten the String Data Type</li>
<li>00:09:33 – A Search Engine in 80 Lines of Python</li>
<li>00:13:14 – Using Python for Data Analysis</li>
<li>00:18:22 – Sponsor: Intel</li>
<li>00:18:53 – Understanding Open Source Licensing</li>
<li>00:21:54 – Summary of Major Changes Between Python Versions</li>
<li>00:23:19 – What Python automation scripts do you reuse frequently at work?</li>
<li>00:34:21 – Video Course Spotlight</li>
<li>00:35:52 – hexabyte: A modern, modular, and robust TUI hex editor</li>
<li>00:39:56 – ibis: The Flexibility of Python With the Scale of Modern SQL</li>
<li>00:43:31 – Thanks and goodbye</li>
</ul>
<p>News:</p>
<ul>
<li><a href="https://astral.sh/blog/uv"><code>uv</code>: Python Packaging in Rust</a> – uv is an extremely fast Python package installer and resolver, designed as a drop-in alternative to pip and pip-tools. This post introduces you to uv and shows some of its performance numbers.</li>
<li><a href="https://lucumr.pocoo.org/2024/2/15/rye-grows-with-uv/">Rye Grows With uv - Armin Ronacher’s Thoughts and Writings</a></li>
<li><a href="https://pythoninsider.blogspot.com/2024/02/python-3130-alpha-4-is-now-available.html">Python 3.13.0 Alpha 4 Is Now Available</a></li>
</ul>
<p>Show Links:</p>
<ul>
<li><a href="https://pola.rs/posts/polars_birds_eye_view/">A Bird’s Eye View of Polars</a> – This post on the Polars blog introduces you to how Polars works, showing the steps from queries, plans, optimizations, and then the final execution.</li>
<li><a href="https://pola.rs/posts/polars-string-type/">Polars: Why We Have Rewritten the String Data Type</a> – A large refactor on the string data type is underway in Polars. This deep dive explains why and what is changing.</li>
<li><a href="https://www.alexmolas.com/2024/02/05/a-search-engine-in-80-lines.html">A Search Engine in 80 Lines of Python</a> – In this post Alex explains how he built a micro-search engine from scratch using Python. The resulting search engine is used to search in the posts of the blogs he follows.</li>
<li><a href="https://realpython.com/python-for-data-analysis/">Using Python for Data Analysis</a> – In this tutorial, you’ll learn the importance of having a structured data analysis workflow, and you’ll get the opportunity to practice using Python for data analysis while following a common workflow process.</li>
<li><a href="https://semaphoreci.com/blog/open-source-licensing">Understanding Open Source Licensing</a> – This article discusses the importance of open-source licensing in software development and its implications for stakeholders.</li>
<li><a href="https://www.nicholashairs.com/posts/major-changes-between-python-versions/">Summary of Major Changes Between Python Versions</a> – This article is a quick reference covering the major changes introduced with each new version of Python. Can’t remember when the walrus operator was introduced? This is the place to look that up.</li>
</ul>
<p>Discussion:</p>
<ul>
<li><a href="https://news.ycombinator.com/item?id=39119424">What Python automation scripts do you reuse frequently at work? - Hacker News</a></li>
</ul>
<p>Projects</p>
<ul>
<li><a href="https://github.com/thetacom/hexabyte">hexabyte: A modern, modular, and robust TUI hex editor</a></li>
<li><a href="https://github.com/ibis-project/ibis">ibis: The Flexibility of Python With the Scale of Modern SQL</a></li>
</ul>
<p>Additional Links:</p>
<ul>
<li><a href="https://realpython.com/podcasts/rpp/193/">Episode #193: Wes McKinney on Improving the Data Stack & Composable Systems – The Real Python Podcast</a></li>
<li><a href="https://www.oreilly.com/library/view/open-source-licensing/0131487876/">Open Source Licensing: Software Freedom and Intellectual Property Law</a></li>
<li><a href="https://nedbatchelder.com/text/which-py.html">What’s in which Python - Ned Batchelder</a></li>
<li><a href="https://automatetheboringstuff.com/">Automate the Boring Stuff with Python - Al Sweigart</a></li>
<li><a href="https://realpython.com/working-with-files-in-python/">Working With Files in Python – Real Python</a></li>
<li><a href="https://realpython.com/command-line-interfaces-python-argparse/">Build Command-Line Interfaces With Python’s argparse – Real Python</a></li>
</ul>
<p>Level up your Python skills with our expert-led courses:</p>
<ul>
<li><a href="https://realpython.com/courses/defining-python-functions-with-optional-arguments/">Defining Python Functions With Optional Arguments</a></li>
<li><a href="https://realpython.com/courses/practical-recipes-files/">Practical Recipes for Working With Files in Python</a></li>
<li><a href="https://realpython.com/courses/python-argparse-command-line-interfaces/">Building Command Line Interfaces With argparse</a></li>
</ul> <p><a rel="payment" href="https://realpython.com/join">Support the podcast & join our community of Pythonistas</a></p>