Why you should try something else than Airflow for data pipeline orchestration

Let’s evaluate AWS step functions, Google workflows, Prefect next to Airflow.

mehdio
Towards Data Science
5 min readSep 20, 2021

--

Fan[Digital image] by rajat sarki, https://unsplash.com/photos/Gx2SU87s4WY

While Airflow has dominated the market in terms of usage and community size as a data orchestrator pipeline, it’s pretty old and wasn’t designed initially to meet some of the needs we have today. Airflow…

--

--