
The Road to Airflow 2.0
This week, we linked up with Airflow release manager, core committer, and Astronomer platform engineer Ash Berlin-Taylor to discuss the Airflow 2.0 roadmap [1]. There is some great stuff in the works around performance, autoscaling, and usability t...
The Data Flowcast: Mastering Apache Airflow ® for Data Engineering and AI · The Data Flowcast
November 22, 201932m 28s
Audio is streamed directly from the publisher (audio-delivery.cohostpodcasting.com) as published in their RSS feed. Play Podcasts does not host this file. Rights-holders can request removal through the copyright & takedown page.
Show Notes
This week, we linked up with Airflow release manager, core committer, and Astronomer platform engineer Ash Berlin-Taylor to discuss the Airflow 2.0 roadmap [1]. There is some great stuff in the works around performance, autoscaling, and usability that we're excited about. In this episode, Ash lends his thoughts on the design, implementation, and value-add around all of the upcoming features, including:
- The Knative Executor
- A modern and real-time UI
- A production-grade API
- Improved scheduler and webserver performance
- An official production Docker image for Airflow
We hope you enjoy! Please email [email protected] if you have thoughts on topics you'd like to see covered in future episodes.
Separately, some good folks from the Airflow community are running a user survey that will help collect some useful information around the Airflow UX. If you have five minutes to spare, filling out the following form will help the core Airflow committers to shape the project roadmap: https://forms.gle/XAzR1pQBZiftvPQM7
[1] https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+2.0