Airflow是Airbnb开源的一个用Python写就的工作流管理平台(workflow management platform)。在前一篇文章中,介绍了如何用Crontab管理数据流,但是缺点也是显而易见。针对于Crontab的缺点,灵活可扩展的Airflow具有以下特点: 工作流依赖关系的可视化; 日志追踪; (Python脚本)易于扩展 对比Java系的Oozie,Airflow奉行“Configura...
原文地址:Airflow: a workflow management platform 原文作者:Maxime Beauchemin 译文出自:掘金翻译计划 本文永久链接:github.com/xitu/gold-mi 译者:yqian1991 校对者:Park-ma DerekDick Airflow: 一个工作流程管理平台 出自Maxime Beauchemin Airbnb 是一个快速增长的、数据启示型的公司。我们的数据团队和数据量都在...
Airflow 是一个以编程方式编写、安排和监控工作流的平台。 Apache Airflow 是一个由 Airbnb 开源的工作流管理平台(workflow management platform),用于管理有向无环图 (DAG) 及其相关任务。默认情况下,Python 是用来确定管道任务的编程语言。 Airflow原理 Airflow中,创建和维护工作流的页面非常丰富,以下这些特点可以...
Airflow 是一个以编程方式编写、安排和监控工作流的平台。 Apache Airflow 是一个由 Airbnb 开源的工作流管理平台(workflow management platform),用于管理有向无环图 (DAG) 及其相关任务。默认情况下,Python 是用来确定管道任务的编程语言。 Airflow原理 Airflow中,创建和维护工作流的页面非常丰富,以下这些特点可以...
In this article, we will show you how all this can be done comfortably with the open-source workflow management platform Apache Airflow. 在本文中,我们将向您展示如何使用开源工作流管理平台Apache Airflow轻松完成所有这些操作。 Here you will find important functionalities, components and the most impor...
Secure and highly available managed workflow orchestration for Apache Airflow Get started with Amazon MWAA Deploy Apache Airflow Run Apache Airflow workloads Monitor environments Connect to AWS, cloud, or on-premises resources Why Amazon MWAA?
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows - apache/airflow
Apache airflow is a workflow (data-pipeline) management system developed by Airbnb. It is used by more than 200 companies such as Airbnb, Yahoo, PayPal, Intel, Stripe and many more. Apache Airflow 是由Airbnb开发的工作流程(数据管道)管理系统。它被200多家公司使用,如Airbnb,雅虎,PayPal,英特尔...
[5]https://medium.com/airbnb-engineering/airflow-a-workflow-management-platform-46318b977fd8 [6]https://medium.com/airbnb-engineering/experiment-reporting-framework-4e3fcd29e6c0 [7]https://medium.com/airbnb-engineering/https-medium-com-jonathan-parks-scaling-erf-23fd17c91166 ...
Apache Airflow is an open-source distributed workflow management platform for authoring, scheduling, and monitoring multi-stage workflows. It is designed to be extensible, and it’s compatible with several services like Amazon Elastic Kubernetes Service (Amazon EKS), Amazon Elastic Container Service (...