#apacheairflow — Public Fediverse posts
Live and recent posts from across the Fediverse tagged #apacheairflow, aggregated by home.social.
-
Learn how to install OpenMetadata on Ubuntu using Docker Compose. Deploy the complete stack with Airflow, PostgreSQL, and Elasticsearch.
Full guide here: https://ostechnix.com/install-openmetadata-ubuntu-docker-compose/
#Openmetadata #Metadata #DataGovernance #DataCatalog #Dataops #Devops #DataEngineering #Ubuntu #Postgresql #Docker #DockerCompose #Elasticsearch #ApacheAirflow
-
Learn how to install OpenMetadata on Ubuntu using Docker Compose. Deploy the complete stack with Airflow, PostgreSQL, and Elasticsearch.
Full guide here: https://ostechnix.com/install-openmetadata-ubuntu-docker-compose/
#Openmetadata #Metadata #DataGovernance #DataCatalog #Dataops #Devops #DataEngineering #Ubuntu #Postgresql #Docker #DockerCompose #Elasticsearch #ApacheAirflow
-
Whether you’re migrating to Airflow 3, scaling data pipelines, bringing AI and ML workloads into production, or defining your organization’s data strategy, you’ll leave with ideas you can put into practice.
📅 August 31–September 2, 2026
📍 Hyatt Regency AustinDon’t just watch the AI revolution unfold. Learn how to orchestrate it.
Secure your spot: https://airflowsummit.org/tickets/
#ApacheAirflow #DataEngineering #ArtificialIntelligence #MLOps #AirflowSummit
-
Whether you’re migrating to Airflow 3, scaling data pipelines, bringing AI and ML workloads into production, or defining your organization’s data strategy, you’ll leave with ideas you can put into practice.
📅 August 31–September 2, 2026
📍 Hyatt Regency AustinDon’t just watch the AI revolution unfold. Learn how to orchestrate it.
Secure your spot: https://airflowsummit.org/tickets/
#ApacheAirflow #DataEngineering #ArtificialIntelligence #MLOps #AirflowSummit
-
Оркестрация рабочих процессов: полное руководство
Оркестрация рабочих процессов (workflow orchestration) — это практика координации множества автоматизированных задач, сервисов и систем для обеспечения их согласованного...
#DST #DSTGlobal #ДСТ #ДСТГлобал #Оркестрация #GitHub #Kubernetes #ApacheAirflow #AWSStepFunctions #Temporal #Camunda #CICD #DevOps #Автоматизация #AIагенты #RBAC #SOAR #Обработкаданных #ETL
Источник: https://dstglobal.ru/club/1242-orkestracija-rabochih-processov-polnoe-rukovodstvo
-
Blog alert!
A new part in our series on the DP-700 exam. This time, we touch on Apache Airflow.
-
Blog alert!
A new part in our series on the DP-700 exam. This time, we touch on Apache Airflow.
-
🚩 CVE-2026-42359 (CRITICAL): Apache Airflow 3.2.0 XCom PATCH deserialization flaw enables authenticated users with XCom write to execute code remotely. Upgrade to 3.2.2+ to mitigate. No known exploits yet. https://radar.offseq.com/threat/cve-2026-42359-cwe-502-deserialization-of-untruste-34c6b2b1 #OffSeq #ApacheAirflow #RCE #Security
-
💡 Apache Airflow 2025 Recap
2026 has arrived which marks a great opportunity to review and assess the changes the last year has brought to the ever evolving landscape of open source data tools.
In our new #blog post we look at Apache Airflow and how the leading open source orchestration platform has changed in the last year with the bit v3 major release.
#apacheairflow #airflow #opensource #dataengineering #datascience
-
💡 Apache Airflow 2025 Recap
2026 has arrived which marks a great opportunity to review and assess the changes the last year has brought to the ever evolving landscape of open source data tools.
In our new #blog post we look at Apache Airflow and how the leading open source orchestration platform has changed in the last year with the bit v3 major release.
#apacheairflow #airflow #opensource #dataengineering #datascience
-
Building data pipelines with Java and open source
-
"The release served as a crucial turning point for the project. Downloads from its GitHub repository increased, and more enterprises adopted the software. Encouraged by this growth, the team envisioned the next generation of Airflow: a modular architecture, a more modern user interface, and a “run anywhere, anytime” feature, enabling it to operate on premises, in the cloud, or on edge devices and handle event-driven and ad hoc scenarios in addition to scheduled tasks. The team delivered on this vision with the launch of Airflow 3.0 last April.
“It was amazing that we managed to ‘rebuild the plane while flying it’ when we worked on Airflow 3—even if we had some temporary issues and glitches,” says Jarek Potiuk, one of the foremost contributors to Airflow and now a member of its project-management committee. “We had to refactor and move a lot of pieces of the software while keeping Airflow 2 running and providing some bug fixes for it.”
Compared with Airflow’s second version, which Koka says had only a few hundred to a thousand downloads per month on GitHub, “now we’re averaging somewhere between 35 to 40 million downloads a month,” he says. The project’s community also soared, with more than 3,000 developers of all skill levels from around the world contributing to Airflow."
https://spectrum.ieee.org/apache-airflow-3-programmatic-workflows
#AirFlow #ApacheAirflow #AirBnB #OpenSource #FLOSS #WorkflowOrchestratror #Python #DataPipelines
-
"The release served as a crucial turning point for the project. Downloads from its GitHub repository increased, and more enterprises adopted the software. Encouraged by this growth, the team envisioned the next generation of Airflow: a modular architecture, a more modern user interface, and a “run anywhere, anytime” feature, enabling it to operate on premises, in the cloud, or on edge devices and handle event-driven and ad hoc scenarios in addition to scheduled tasks. The team delivered on this vision with the launch of Airflow 3.0 last April.
“It was amazing that we managed to ‘rebuild the plane while flying it’ when we worked on Airflow 3—even if we had some temporary issues and glitches,” says Jarek Potiuk, one of the foremost contributors to Airflow and now a member of its project-management committee. “We had to refactor and move a lot of pieces of the software while keeping Airflow 2 running and providing some bug fixes for it.”
Compared with Airflow’s second version, which Koka says had only a few hundred to a thousand downloads per month on GitHub, “now we’re averaging somewhere between 35 to 40 million downloads a month,” he says. The project’s community also soared, with more than 3,000 developers of all skill levels from around the world contributing to Airflow."
https://spectrum.ieee.org/apache-airflow-3-programmatic-workflows
#AirFlow #ApacheAirflow #AirBnB #OpenSource #FLOSS #WorkflowOrchestratror #Python #DataPipelines
-
🚨 Apache Airflow 3.0.3 flaw (CVE-2025-54831) exposes sensitive connection info, including passwords, tokens & API keys, to read-only users.
🔧 Fix: upgrade to v3.0.4 or later.
💬 Should open-source software handling sensitive data undergo mandatory independent audits before release?👉 Follow @technadu for more cybersecurity updates.
#CyberSecurity #ApacheAirflow #CVE202554831 #OpenSourceSecurity #DevSecOps #InfoSec #Vulnerability #DataSecurity
-
New to #ApacheAirflow? Build your first workflow!
Learn how this #opensource orchestration tool helps you schedule, monitor & manage data pipelines in #Python. Watch Ricardo Sueiras guide beginners through hands-on steps.
Click here: https://youtu.be/sdBimSGxAcs
-
New to #ApacheAirflow? Build your first workflow!
Learn how this #opensource orchestration tool helps you schedule, monitor & manage data pipelines in #Python. Watch Ricardo Sueiras guide beginners through hands-on steps.
Click here: https://youtu.be/sdBimSGxAcs
-
🎤Ne manquez pas notre conférence aux #RPLL2025 :
Manipulation de la #GED avec Apache Airflow🗓️jeudi 15 mai à 12:00
📍Hôtel de la Métropole, Lyon
🙋Présentée par @jeremielesage👉https://jeci.fr/fr/evenements/rpll-atelier-2025/
#LogicielLibre #floss #ApacheAirflow -
🤖🚀 Ah yes, because nothing says "simple" like melding Apache Airflow and LLM workflows with a side of AI Agents. Just what every developer dreams of: wrestling with Pydantic AI while pretending to control the automation beast. 🛠️✨
https://github.com/astronomer/airflow-ai-sdk #ApacheAirflow #LLMworkflows #AIAgents #PydanticAutomation #DeveloperDreams #HackerNews #ngated -
🤖🚀 Ah yes, because nothing says "simple" like melding Apache Airflow and LLM workflows with a side of AI Agents. Just what every developer dreams of: wrestling with Pydantic AI while pretending to control the automation beast. 🛠️✨
https://github.com/astronomer/airflow-ai-sdk #ApacheAirflow #LLMworkflows #AIAgents #PydanticAutomation #DeveloperDreams #HackerNews #ngated -
LLM Workflows then Agents: Getting Started with Apache Airflow
https://github.com/astronomer/airflow-ai-sdk
#HackerNews #LLM #Workflows #Agents #ApacheAirflow #DataEngineering #AIWorkflow
-
LLM Workflows then Agents: Getting Started with Apache Airflow
https://github.com/astronomer/airflow-ai-sdk
#HackerNews #LLM #Workflows #Agents #ApacheAirflow #DataEngineering #AIWorkflow
-
🍹 C'est l'heure du cocktail de bienvenue proposé par Satya !
- une dose de #ModernDataStack,
- un trait de géo,
- un zeste d'Open Source,
- et beaucoup d'amour 💗.Cette recette vous est servie dans cet article qui détaille comment le Gard valorise ses géo-données.
:geotribu: https://geotribu.fr/articles/2025/2025-02-25_stack_data_gard/
Relecture 🧐 : @geojulien & Michaël Galien
#PostgreSQL #PostGIS #GDAL #OGR #DBT #Metabase #ApacheAirflow -
🍹 C'est l'heure du cocktail de bienvenue proposé par Satya !
- une dose de #ModernDataStack,
- un trait de géo,
- un zeste d'Open Source,
- et beaucoup d'amour 💗.Cette recette vous est servie dans cet article qui détaille comment le Gard valorise ses géo-données.
:geotribu: https://geotribu.fr/articles/2025/2025-02-25_stack_data_gard/
Relecture 🧐 : @geojulien & Michaël Galien
#PostgreSQL #PostGIS #GDAL #OGR #DBT #Metabase #ApacheAirflow -
Just caught up with the recent Delta Lake webinar,
> Revolutionizing Delta Lake workflows on AWS Lambda with Polars, DuckDB, Daft & Rust
Some interesting hints there regarding lightweight processing of big-ish data. Easy to relate to any other framework instead of Lambda, e.g. #ApacheAirflow tasks
#dataengineering #datascience #duckdb #daft #polars #pandas #python #spark #deltalake #databricks #airflow #bigdata #smalldata
-
Just caught up with the recent Delta Lake webinar,
> Revolutionizing Delta Lake workflows on AWS Lambda with Polars, DuckDB, Daft & Rust
Some interesting hints there regarding lightweight processing of big-ish data. Easy to relate to any other framework instead of Lambda, e.g. #ApacheAirflow tasks
#dataengineering #datascience #duckdb #daft #polars #pandas #python #spark #deltalake #databricks #airflow #bigdata #smalldata
-
Azure Data Factory And Apache Airflow Integration Flaws Let Attackers Gain Write Access https://cybersecuritynews.com/azure-airflow-security-flaw/ #InformationSecurityNews #SecurityVulnerabilities #CyberSecurityNews #VulnerabilityNews #AzureDataFactory #ApacheAirflow #vulnerability #cloud
-
Azure Data Factory And Apache Airflow Integration Flaws Let Attackers Gain Write Access https://cybersecuritynews.com/azure-airflow-security-flaw/ #InformationSecurityNews #SecurityVulnerabilities #CyberSecurityNews #VulnerabilityNews #AzureDataFactory #ApacheAirflow #vulnerability #cloud
-
If you're using #ApacheAirflow, I'm interested in hearing your use cases for it.
-
If you're using #ApacheAirflow, I'm interested in hearing your use cases for it.
-
📝 "How to build a data extraction pipeline with Apache Airflow"
👤 Déborah Mesquita
#pyladies #python #oldiebutgoodie #apacheairflow #nlp #handsontutorials #editorspick #datascience
-
📝 "How to build a data extraction pipeline with Apache Airflow"
👤 Déborah Mesquita
#pyladies #python #oldiebutgoodie #apacheairflow #nlp #handsontutorials #editorspick #datascience
-
Still, #dagster has less dependencies, and after some battling and downgrading version, I managed to start the dev environment...
Building a #Apacheairflow container, is pure chaos. Installing via pip, is another kind of hell, with broken builds all over the place (google-re2)... A tool that has 9 years in the market, being so overwhelming its installation process
At least I can make #dagster run.
I guess we are all ill served with workflow orchestration tools anyway, on the open source world
-
Still, #dagster has less dependencies, and after some battling and downgrading version, I managed to start the dev environment...
Building a #Apacheairflow container, is pure chaos. Installing via pip, is another kind of hell, with broken builds all over the place (google-re2)... A tool that has 9 years in the market, being so overwhelming its installation process
At least I can make #dagster run.
I guess we are all ill served with workflow orchestration tools anyway, on the open source world
-
I take back what I said yesterday... #dagster installation process is quite clumsy. And I detected some issues in terms of documentation/instructions.
I manage to install and run #Apacheairflow in 2 clicks... -
I take back what I said yesterday... #dagster installation process is quite clumsy. And I detected some issues in terms of documentation/instructions.
I manage to install and run #Apacheairflow in 2 clicks... -
How we orchestrate 2000+ #DBT models in #ApacheAirflow
https://medium.com/@aleexmagno/how-we-orchestrate-2000-dbt-models-in-apache-airflow-90901504032d
-
How we orchestrate 2000+ #DBT models in #ApacheAirflow
https://medium.com/@aleexmagno/how-we-orchestrate-2000-dbt-models-in-apache-airflow-90901504032d
-
Going to #PyConUS for the first time. Looking forward to actually meet the community folks. Just in time to gear up for development streamlining discussions for the upcoming
#apacheAirflow 3.0. -
Tenable Research discovered a one-click account takeover vulnerability in the AWS Managed Workflows Apache Airflow service, and that could have resulted in remote code execution (RCE) on the underlying instance, and in lateral movement to other services. Additional research revealed that numerous shared-parent service domains in AWS, Azure and GCP were misconfigured, putting cloud customers at considerable risk. No CVE ID associated. 🔗 https://www.tenable.com/blog/flowfixation-aws-apache-airflow-service-takeover-vulnerability-and-why-neglecting-guardrails
-
Tenable Research discovered a one-click account takeover vulnerability in the AWS Managed Workflows Apache Airflow service, and that could have resulted in remote code execution (RCE) on the underlying instance, and in lateral movement to other services. Additional research revealed that numerous shared-parent service domains in AWS, Azure and GCP were misconfigured, putting cloud customers at considerable risk. No CVE ID associated. 🔗 https://www.tenable.com/blog/flowfixation-aws-apache-airflow-service-takeover-vulnerability-and-why-neglecting-guardrails
-
FlowFixation: AWS Apache Airflow Service Takeover Vulnerability
Date: March 21, 2024
CVE: Not specified
Sources: Tenable BlogIssue Summary
Tenable Research discovered a vulnerability, named FlowFixation, in AWS Managed Workflows for Apache Airflow (MWAA) that could allow session hijacking leading to a full takeover of the victim's web management panel.
Technical Key findings
FlowFixation combines session fixation and XSS via Amazon AWS domain misconfiguration, enabling attackers to authenticate known sessions and gain control over victim's Apache Airflow management panels.
Vulnerable products
- AWS Managed Workflows for Apache Airflow (MWAA)
Impact assessment
Potential for remote code execution on underlying instances and lateral movement to other services.
Patches or workaround
AWS has addressed the vulnerability. Users should ensure they are using updated services.
Tags
#AWS #ApacheAirflow #CloudSecurity #SessionHijacking #Vulnerability
-
FlowFixation: AWS Apache Airflow Service Takeover Vulnerability
Date: March 21, 2024
CVE: Not specified
Sources: Tenable BlogIssue Summary
Tenable Research discovered a vulnerability, named FlowFixation, in AWS Managed Workflows for Apache Airflow (MWAA) that could allow session hijacking leading to a full takeover of the victim's web management panel.
Technical Key findings
FlowFixation combines session fixation and XSS via Amazon AWS domain misconfiguration, enabling attackers to authenticate known sessions and gain control over victim's Apache Airflow management panels.
Vulnerable products
- AWS Managed Workflows for Apache Airflow (MWAA)
Impact assessment
Potential for remote code execution on underlying instances and lateral movement to other services.
Patches or workaround
AWS has addressed the vulnerability. Users should ensure they are using updated services.
Tags
#AWS #ApacheAirflow #CloudSecurity #SessionHijacking #Vulnerability
-
LLMs are good coders, useless writers
https://samfirke.com/2023/12/22/llms-are-good-coders-useless-writers/
-
🌪️ Just published my first blog post on Apache Airflow: a zero-to-one introduction to this tool. It details how to get it up and running using Docker 🐳, guides you through creating and scheduling simple pipelines 📊, and shows how to keep an eye on them with the Airflow UI.
https://www.franciscoyira.com/post/data-pipelines-cloud-intro-airflow-docker/
#ApacheAirflow #DataScience -
I find #scopecreep to be a real problem with #selfGuidedLearning, particularly online. Do others find the same and, if so, how do you deal with it?
As an example, I sat down to dig into #ApacheSuperset but that led to #ApacheAirflow, which lead to the Astronomer.io service, which led to #ApacheFlink. By the end of the day I’d had a great overview of some really neat stuff but was not better positioned to be productive in Superset than I was at the start of the day.
-
Tech news digest from Alex, July’23
#apacheairflow #dbt #aws #snowflake
https://agordienko.medium.com/tech-news-digest-from-alex-july23-19950d00b22a
-
Do you know how to harness the power of Apache Airflow with CrateDB?
Learn it step-by-step in this first article of its series by Niklas Schmidtmer and Marija Selaković from CrateDB's Customer Engineering team 👩🏻💻👨🏼💻
Read it here 👇🏼
https://hubs.ly/Q01QFZcL0#apacheairflow #queries #astronomer #cratedb #cloud #db #database
-
➡️ Hear about #ApacheAirflow, what it is & how it helps with orchestrating data pipelines.
➡️ Learn what are some of the tradeoffs of using Apache Airflow, and what are some new capabilities that teams can use.
#InfoQ video by Ricardo Sueiras: http://bit.ly/3ZqmdJd