Apache Airflow common.compat 提供者演进全解从 1.0.0 到 1.19.0 的 Airflow 2→3 兼容层 Changelog 解读【免费下载链接】airflowApache Airflow - A platform to programmatically author, schedule, and monitor workflows项目地址: https://gitcode.com/GitHub_Trending/ai/airflowapache-airflow-providers-common-compat是 Apache Airflow 生态中承担版本兼容职责的特殊提供者包其 changelog.rst 记录了从 1.0.0 到 1.19.0 共 30 余个版本的全部演进条目。本文以该 Changelog 为主体逐版本完整梳理其功能特性、缺陷修复与杂项变更并结合 sdk.py、pyproject.toml 等仓库源码解析兼容导入层、异步连接、血缘lineage与 OpenLineage 注入等关键能力背后的实现机制帮助你在跨 Airflow 2/3 版本维护 DAG 与提供者代码时做出准确的版本选型。一、common.compat 定位与 Changelog 维护机制common.compat提供者在 provider.yaml 中的官方描述为Common Compatibility Provider - providing compatibility code for previous Airflow versions为旧版 Airflow 提供兼容性代码状态为state: ready、生命周期为production。它不是面向某个云厂商或数据库的集成包而是让 DAG 代码与提供者代码一份代码同时跑在 Airflow 2.x 与 3.x的底层基础设施——docs/index.rst 同样给出这一定位。阅读这份 Changelog 前需要理解文件头部写给贡献者的维护规则原文第 18–21 行的 NOTE只在存在破坏性变更breaking changes时才在 Changelog 标题正下方手工添加面向用户的说明条目解释用户应如何应对整份 Changelog 由发布管理员release manager半自动维护与更新。这也是为什么每个版本块的底部都带有一段以.. Below changes are excluded from the changelog...开头的注释区——那里是被工具排除在正式 Changelog 之外的提交依赖升级、文档调整、发布准备等并明确要求Do not delete the lines(!)以便下次生成时对比。下文按版本完整展开各条目并在此机制基础上补充实现层面的解读。二、版本支持策略速览Changelog 中多次出现.. note::标注标明某些版本仅适用于特定最低 Airflow 版本对应 Apache Airflow 社区提供者的最低支持版本策略。汇总如下provider 版本最低 Apache Airflow 版本Changelog 注记1.2.0Airflow 2.8.01.3.0Airflow 2.9.01.7.0Airflow 2.10.01.10.0Airflow 2.11.01.19.0当前apache-airflow2.11.0见 pyproject.toml 的dependencies当前仓库中 pyproject.toml 还声明了requires-python 3.10并对 Python 3.14 做了条件依赖asgiref3.11.1; python_version 3.14否则asgiref2.3.0——这与 1.14.2 中Add Python 3.14 Support (#63520)条目以及 docs/index.rst 的依赖表完全一致。provider.yaml 中列出的 30 个版本号1.19.0 → 1.0.0也与 Changelog 的版本一一对应两者共同构成该包的发布基线。三、版本演进详解1.0.0初始版本Initial version of the provider. (#40374)common.compat于 1.0.0 建立从此承担 Airflow 版本间的兼容垫片角色。1.1.0Hook 血缘支持起步FeaturesAdd method to common.compat to not force hooks to try/except every 2.10 hook lineage call (#40812)——此前每个 Hook 调用 2.10 的血缘接口时都要自行try/except此版本提供了统一方法免除这一负担。MiscMigrate OpenLineage provider to V2 facets. (#39530)Add support for hook lineage for S3Hook (#40819)1.2.xDataset → Asset 重命名的铺垫1.2.0 起最低 Airflow 版本升至 2.8.0Bump minimum Airflow version in providers to Airflow 2.8.0 (#41396)见 Misc 条目。1.2.1 MiscRename dataset related python variable names to asset (#41348)1.2.2 Bug Fixesserialize asset/dataset timetable conditions in OpenLineage info also for Airflow 2 (#43434)Move python operator to Standard provider (#42081)从源码结构看Dataset→Asset 的重命名在此后贯穿整个兼容层sdk.py 中的_RENAME_MAP将Asset/AssetAlias/AssetAll/AssetAny映射到 Airflow 2.x 的Dataset*旧路径airflow.datasets1.8.0 版本正是这套综合兼容导入落地的起点。1.3.0版本检查统一化最低 Airflow 2.9.0Bug Fixesfix(providers/common/compat): add back add_input_dataset and add_output_dataset to NoOpCollector (#44681)Fix name of private function in compat provider (#44680)MiscBump minimum Airflow version in providers to Airflow 2.9.0 (#44956)Remove references to AIRFLOW_V_2_9_PLUS (#44987)Consistent way of checking Airflow version in providers (#44686)Remove unnecessary compatibility code in S3 asset import (#44714)Move Asset user facing components to task_sdk (#43773)Make AssetAliasEvent a class context.py (#44709)Move triggers to standard provider (#43608)Consistent way of checking Airflow version in providers 对应到源码即 version_compat.pyget_base_airflow_version_tuple()解析airflow.__version__并派生出AIRFLOW_V_3_0_PLUS、AIRFLOW_V_3_1_PLUS等布尔常量。该文件头部注释特别说明它被刻意手工复制到其它提供者中以避免提供者之间产生不必要的依赖。1.4.0OpenLineage 传输信息注入Featuresfeat: automatically inject OL transport info into spark jobs (#45326)feat: Add OpenLineage support for some SQL to GCS operators (#45242)Bug Fixesfix: OpenLineage sql parsing add try-except for sqlalchemy engine (#46366)MiscRemove old lineage stuff (#45260)OpenLineage 相关实现位于 openlineage/ 子包含utils/spark.py、utils/sql.py等1.14.3 的 EMR Serverless/Glue 注入条目是这条线的延续。1.5.0 / 1.5.1重新发布与路径迁移1.5.0Changelog 明确注记——This version contains no code changes. It was released to replace a previous version that was yanked due to a packaging issue.该版本无代码变更是为替换因打包问题被撤回的上一版本而发布。1.5.1MiscRelocate airflow.auth to airflow.api_fastapi.auth (#47492)、Upgrade flit to 3.11.0 (#46938)1.6.0 / 1.6.1版本检查助手与基类迁移1.6.0 Featuresfeat: Add helper for any provider version check (#47909)feat: Add helper for OpenLineage version check (#47897)1.6.0 Misc 还包含 AIP-84 授权系列AIP-84 Add Auth for DAG Versioning (#47553)、AIP-84 Add Auth for backfill (#47482)、AIP 84 Add auth for asset alias (#47241)以及Move BaseNotifier to Task SDK (#48008)。1.6.1 Bug FixesMove bases classes to airflow.sdk.bases (#48487)——基类在 Task SDK 中的新位置为 1.14.1 的SkipMixin/BranchMixIn迁移铺平了道路。1.7.0–1.7.4最低版本与 Python 支持调整1.7.0最低 Airflow 2.10.0MiscRemove AIRFLOW_2_10_PLUS conditions (#49877)、Bump min Airflow version in providers to 2.10 (#49843)1.7.1Miscnit: Remove unreachable code (#51110)1.7.2MiscReplace models.BaseOperator to Task SDK one for Common Providers (#52443)、Drop support for Python 3.9 (#52072)1.7.3Miscfix unreachable mypy warnings (#53575)、Add Python 3.13 support for Airflow. (#46891)、Remove type ignore across codebase after mypy upgrade (#53243)、Remove upper-binding for python-requires (#52980)、Temporarily switch to use , pattern instead of ~ (#52967)1.7.4MiscBump mypy to 1.18.1 (#55596)1.8.0兼容导入层的成型1.8.0 是本包的分水岭FeaturesSimplify version-specific imports in the Standard provider (#56867)Add SQLAs mapped_column to common-compat (#56880)Add comprehensive compatibility imports for Airflow 2 to 3 migration (#56790)MiscCommon.Compat: Extract reusable compat utilities and rename to sdk (#56884)Simplify version-specific imports in the Google provider (#56793)Migrate Apache providers Elasticsearch to common.compat (#57016)Doc-onlyRemove placeholder Release Date in changelog and index files (#56056)这一版本建立了今天仍在使用的两套机制可在源码中直接印证sdk.py 的兼容导入映射_IMPORT_MAP第 148 行起以名称 → 模块路径或按新到旧排序的路径元组登记了BaseHook、BaseOperator、DAG、chain、各异常类、Stats、redact、conf等上百个符号优先尝试 Airflow 3 的airflow.sdk路径、失败后回退 Airflow 2 路径_MODULE_MAP第 330–333 行支持整模块级别的回退导入timezone、io_AIRFLOW_3_ONLY_EXCEPTIONS第 313–316 行则保证DownstreamTasksSkipped、DagRunTriggerException等仅 3.x 存在的异常只在AIRFLOW_V_3_0_PLUS时注册。_compat_utils.py 的create_module_getattr模块级__getattr__懒加载工厂先查rename_map新名→旧路径旧名如Asset→airflow.datasets.Dataset再查module_map、import_map全部失败才抛出带完整候选路径信息的ImportError/AttributeError。sdk.py 末尾即以三张映射表调用该工厂生成__getattr__。1.9.0向后兼容的异步连接Featuresfeat: backwards comp get async conn (#57143)MiscConvert all airflow distributions to be compliant with ASF requirements (#58138)对应实现即 connection/init.py 中的get_async_connection(conn_id, hook)若 Hook 类提供aget_connection则直接await否则通过asgiref.sync.sync_to_async包装同步的get_connection。这也解释了 pyproject.toml 中为何引入asgiref作为运行时依赖。1.10.0 / 1.10.1最低 Airflow 2.11.0 与异常回迁1.10.0FeaturesAdjust compat HookLevelLineage for new add_extra method (#58057)MiscMove out some exceptions to TaskSDK (#54505)、Bump minimum Airflow version in providers to Airflow 2.11.0 (#58612)1.10.1MiscAdd backcompat for exceptions in providers (#58727)、Move the traces and metrics code under a common observability package (#56187)、Bump minimum prek version to 0.2.0 (#58952)1.10.0 是最后一次提高最低 Airflow 版本当前 pyproject.toml 的apache-airflow2.11.0正是这一决策的产物。1.11.0共享 module_loading 与循环导入修复FeaturesExtract shared module_loading distribution (#59139)Bug FixesFix circular import errors when CeleryExecutor is used with sentry (#59745)MiscPrevent client secrets and proxy credentials from being logged in Microsoft Graph hook logs (#59688)共享库抽取体现在仓库顶层的 shared/module_loading/ 目录。1.12.0从 compat SDK 消费配置与异常FeaturesAllow providers to consume AirflowConfigException from compat sdk (#60044)Source conf from airflow.sdk.configuration.conf for apache providers (#59966)Consume AirflowOptionalProviderFeatureException from compat sdk in providers (#60335)MiscNew year means updated Copyright notices (#60344)、Move listeners module to shared library for client server separation (#59883)、Move over plugins_manager to a shared library (#59956)其中从airflow.sdk.configuration.conf取 conf在 sdk.py 中体现为conf: (airflow.sdk.configuration, airflow.configuration)的双路径映射即 Airflow 3 下从 Task SDK 读取配置、Airflow 2 下回退到airflow.configuration。listeners/plugins_manager 的迁移分别对应顶层 shared/listeners/ 与 shared/plugins_manager/ 目录。1.13.0 / 1.13.1PythonOperator 异步 callable 与血缘迁移1.13.0带有一条重要注记新功能Add support for async callables in PythonOperator仅在 Airflow 3.2 上生效。FeaturesAdd support for async callables in PythonOperator (#60268)Bug FixesFix BaseAsyncOperator in common-compat provider for Airflow 2.x (#60663)MiscOption to disable exporting of legacy metric names (#53722)、Define TaskInstanceKey in task-sdk to support client server separation (#60776)1.13.1FeaturesMove lineage from airflow core to task sdk (#61157)——血缘模块从 Airflow 核心移入 Task SDKChangelog 排除区显示该迁移曾因 revert/重做经历 #60968→#61151→#61157 的过程。此后 Hook 血缘的实现以 lineage/hook.py 与 lineage/entities.py 为准测试见 tests/unit/common/compat/lineage/。1.14.0–1.14.3Stats 路由、SDK 化重构与 2.x 兼容修复1.14.0FeaturesRoute providers to consume Stats from common compat provider (#61812)——各提供者改为经由兼容层消费Stats对应sdk.py中Stats: (airflow.sdk.observability.stats, airflow.observability.stats, airflow.stats)的三级回退该三级路径正是 1.10.1 将 traces/metrics 归入 common observability 包的结果。1.14.1MiscConsolidate SkipMixin imports through common-compat layer (#62776)、Move SkipMixin and BranchMixIn to Task SDK (#62749)、Move determine_kwargs and KeywordParameters to SDK DecoratedOperator (#62746)。1.14.2MiscAdd Python 3.14 Support (#63520)、Remove usage of deprecated module airflow.security.permissions (#63335)、Remove remaining session query usages (#62758)。1.14.3MiscAdd OpenLineage parent and transport info injection to EmrServerlessStartJobOperator (#64807)、Add OpenLineage parent info injection to GlueJobOperator (#64513)、Fix RESOURCE_ASSET compatibility with Airflow 2.x in common-compat (#64933)。1.15.0–1.17.0DAG 版本化持久化与异步 Hook 能力1.15.0FeaturesAdd BundleVersion dataclass and version_data persistence to DagVersion (#66491)1.16.0FeaturesAdded get_async_hook in common.compat provider (#69208)Bug FixesFix RecursionError in common.compat hook lineage add_extra polyfill (#68735)——修复了 Hook 血缘add_extra兼容垫片在 Airflow 2.x 上的递归错误可视为 1.10.0HookLevelLineage调整的后续修正。1.17.0FeaturesHonor async hook subclass overrides in get_async_connection (#69140)——get_async_connection 新增hook参数使子类对aget_connection/get_connection的覆写得以生效而非固定落到BaseHook。1.18.0审计日志与权限细化FeaturesGate audit log rows not tied to a Dag on a dedicated AccessView (#70759)Add IMPORT_ERRORS_ALL permission for import errors of files with no registered Dag (#69790)权限与访问视图的兼容实现位于 security/ 子包permissions.py、access_view.py测试见 tests/unit/common/compat/security/。1.19.0当前版本BaseEventTrigger 别名与血缘实体修复FeaturesAdd common.compat alias for BaseEventTrigger (#72140)Bug FixesFix lineage entities sharing mutable defaults across instances (#72509)——修复血缘实体在不同实例间共享可变默认值的隐患。排除区还记录了Add missing unit tests for common.compat lineage entities (#72506)、Adopt flit 4 as the provider distribution build backend (#71186)对应 pyproject.toml 的flit_core4.0.2构建后端等未入正式条目的变更。四、主题纵览Changelog 背后的四条演进主线逐版本通读后可以提炼出四条主线每条都有明确的源码落点兼容导入层1.8.0 起成型sdk.py的三张映射表 _compat_utils.create_module_getattr懒加载覆盖 Hooks、Operators、Sensors、装饰器、模型、异常、监听器、配置等全部跨版本符号version_compat.py提供统一的版本常量。异步能力1.9.0 → 1.17.0get_async_connection起步get_async_hook1.16.0补齐 Hook 实例的异步获取1.17.0 再解决子类覆写语义1.13.0 为PythonOperator引入 async callable限 Airflow 3.2。血缘与 OpenLineage1.1.0 → 1.19.0Hook 血缘免 try/except1.1.0、NoOpCollector 接口补齐1.3.0、OL transport 注入 Spark 任务1.4.0、add_extra签名适配1.10.0、血缘迁入 Task SDK1.13.1、递归与可变默认值缺陷修复1.16.0/1.19.0。共享化与 SDK 化1.11.0 → 1.14.1module_loading、listeners、plugins_manager抽为共享库对应仓库顶层 shared/ 目录SkipMixin/BranchMixIn、异常、基类陆续迁入 Task SDK 并经由兼容层再导出。五、安装与版本选择建议按 docs/index.rst 的说明安装方式pip install apache-airflow-providers-common-compat # 需要 OpenLineage 功能时连同跨提供者依赖一起安装 pip install apache-airflow-providers-common-compat[openlineage]pyproject.toml 定义了两个可选 extraopenlineage依赖apache-airflow-providers-openlineage与standard依赖apache-airflow-providers-standard。版本选择上若你的 Airflow 低于 2.8/2.9/2.10需分别停留在 1.1.x/1.2.x/1.6.x 之前的对应版本参见第二节的最低版本表当前 1.19.0 要求apache-airflow2.11.0、Python ≥3.10支持至 3.14是跨 Airflow 2.11 与 Airflow 3.x 部署的默认选择1.13.0 的 asyncPythonOperatorcallable 特性仅在 Airflow 3.2 上可用升级评估时需注意此前提。正式发布的 sdist/wheel 及其 sha512、asc 校验文件可从 Apache 官方下载站获取docs/index.rst 的 Downloading official packages 一节给出了 1.19.0 的完整下载与校验项清单从源码安装可参考 installing-providers-from-sources.rst。六、结语common.compat的 Changelog 表面上是 30 余个版本的条目流水实际勾勒出 Airflow 从 2.x 迈向 3.x 期间兼容基础设施的完整建设史1.8.0 的综合兼容导入与sdk重命名是架构基石异步连接/Hook 能力线解决了跨版本执行差异血缘与 OpenLineage 线保障了可观测性语义一致而共享库与 Task SDK 迁移线则持续降低版本分支代码量。对于同时维护双版本环境的团队这份 Changelog 加上 sdk.py 的导入映射表就是判断某个符号/行为在我选定的 provider 版本中是否可用的两份权威依据。【免费下载链接】airflowApache Airflow - A platform to programmatically author, schedule, and monitor workflows项目地址: https://gitcode.com/GitHub_Trending/ai/airflow创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考 Apache Airflow common.compat 提供者演进全解：从 1.0.0 到 1.19.0 的 Airflow 2→3 兼容层 Changelog 解读
Apache Airflow common.compat 提供者演进全解从 1.0.0 到 1.19.0 的 Airflow 2→3 兼容层 Changelog 解读【免费下载链接】airflowApache Airflow - A platform to programmatically author, schedule, and monitor workflows项目地址: https://gitcode.com/GitHub_Trending/ai/airflowapache-airflow-providers-common-compat是 Apache Airflow 生态中承担版本兼容职责的特殊提供者包其 changelog.rst 记录了从 1.0.0 到 1.19.0 共 30 余个版本的全部演进条目。本文以该 Changelog 为主体逐版本完整梳理其功能特性、缺陷修复与杂项变更并结合 sdk.py、pyproject.toml 等仓库源码解析兼容导入层、异步连接、血缘lineage与 OpenLineage 注入等关键能力背后的实现机制帮助你在跨 Airflow 2/3 版本维护 DAG 与提供者代码时做出准确的版本选型。一、common.compat 定位与 Changelog 维护机制common.compat提供者在 provider.yaml 中的官方描述为Common Compatibility Provider - providing compatibility code for previous Airflow versions为旧版 Airflow 提供兼容性代码状态为state: ready、生命周期为production。它不是面向某个云厂商或数据库的集成包而是让 DAG 代码与提供者代码一份代码同时跑在 Airflow 2.x 与 3.x的底层基础设施——docs/index.rst 同样给出这一定位。阅读这份 Changelog 前需要理解文件头部写给贡献者的维护规则原文第 18–21 行的 NOTE只在存在破坏性变更breaking changes时才在 Changelog 标题正下方手工添加面向用户的说明条目解释用户应如何应对整份 Changelog 由发布管理员release manager半自动维护与更新。这也是为什么每个版本块的底部都带有一段以.. Below changes are excluded from the changelog...开头的注释区——那里是被工具排除在正式 Changelog 之外的提交依赖升级、文档调整、发布准备等并明确要求Do not delete the lines(!)以便下次生成时对比。下文按版本完整展开各条目并在此机制基础上补充实现层面的解读。二、版本支持策略速览Changelog 中多次出现.. note::标注标明某些版本仅适用于特定最低 Airflow 版本对应 Apache Airflow 社区提供者的最低支持版本策略。汇总如下provider 版本最低 Apache Airflow 版本Changelog 注记1.2.0Airflow 2.8.01.3.0Airflow 2.9.01.7.0Airflow 2.10.01.10.0Airflow 2.11.01.19.0当前apache-airflow2.11.0见 pyproject.toml 的dependencies当前仓库中 pyproject.toml 还声明了requires-python 3.10并对 Python 3.14 做了条件依赖asgiref3.11.1; python_version 3.14否则asgiref2.3.0——这与 1.14.2 中Add Python 3.14 Support (#63520)条目以及 docs/index.rst 的依赖表完全一致。provider.yaml 中列出的 30 个版本号1.19.0 → 1.0.0也与 Changelog 的版本一一对应两者共同构成该包的发布基线。三、版本演进详解1.0.0初始版本Initial version of the provider. (#40374)common.compat于 1.0.0 建立从此承担 Airflow 版本间的兼容垫片角色。1.1.0Hook 血缘支持起步FeaturesAdd method to common.compat to not force hooks to try/except every 2.10 hook lineage call (#40812)——此前每个 Hook 调用 2.10 的血缘接口时都要自行try/except此版本提供了统一方法免除这一负担。MiscMigrate OpenLineage provider to V2 facets. (#39530)Add support for hook lineage for S3Hook (#40819)1.2.xDataset → Asset 重命名的铺垫1.2.0 起最低 Airflow 版本升至 2.8.0Bump minimum Airflow version in providers to Airflow 2.8.0 (#41396)见 Misc 条目。1.2.1 MiscRename dataset related python variable names to asset (#41348)1.2.2 Bug Fixesserialize asset/dataset timetable conditions in OpenLineage info also for Airflow 2 (#43434)Move python operator to Standard provider (#42081)从源码结构看Dataset→Asset 的重命名在此后贯穿整个兼容层sdk.py 中的_RENAME_MAP将Asset/AssetAlias/AssetAll/AssetAny映射到 Airflow 2.x 的Dataset*旧路径airflow.datasets1.8.0 版本正是这套综合兼容导入落地的起点。1.3.0版本检查统一化最低 Airflow 2.9.0Bug Fixesfix(providers/common/compat): add back add_input_dataset and add_output_dataset to NoOpCollector (#44681)Fix name of private function in compat provider (#44680)MiscBump minimum Airflow version in providers to Airflow 2.9.0 (#44956)Remove references to AIRFLOW_V_2_9_PLUS (#44987)Consistent way of checking Airflow version in providers (#44686)Remove unnecessary compatibility code in S3 asset import (#44714)Move Asset user facing components to task_sdk (#43773)Make AssetAliasEvent a class context.py (#44709)Move triggers to standard provider (#43608)Consistent way of checking Airflow version in providers 对应到源码即 version_compat.pyget_base_airflow_version_tuple()解析airflow.__version__并派生出AIRFLOW_V_3_0_PLUS、AIRFLOW_V_3_1_PLUS等布尔常量。该文件头部注释特别说明它被刻意手工复制到其它提供者中以避免提供者之间产生不必要的依赖。1.4.0OpenLineage 传输信息注入Featuresfeat: automatically inject OL transport info into spark jobs (#45326)feat: Add OpenLineage support for some SQL to GCS operators (#45242)Bug Fixesfix: OpenLineage sql parsing add try-except for sqlalchemy engine (#46366)MiscRemove old lineage stuff (#45260)OpenLineage 相关实现位于 openlineage/ 子包含utils/spark.py、utils/sql.py等1.14.3 的 EMR Serverless/Glue 注入条目是这条线的延续。1.5.0 / 1.5.1重新发布与路径迁移1.5.0Changelog 明确注记——This version contains no code changes. It was released to replace a previous version that was yanked due to a packaging issue.该版本无代码变更是为替换因打包问题被撤回的上一版本而发布。1.5.1MiscRelocate airflow.auth to airflow.api_fastapi.auth (#47492)、Upgrade flit to 3.11.0 (#46938)1.6.0 / 1.6.1版本检查助手与基类迁移1.6.0 Featuresfeat: Add helper for any provider version check (#47909)feat: Add helper for OpenLineage version check (#47897)1.6.0 Misc 还包含 AIP-84 授权系列AIP-84 Add Auth for DAG Versioning (#47553)、AIP-84 Add Auth for backfill (#47482)、AIP 84 Add auth for asset alias (#47241)以及Move BaseNotifier to Task SDK (#48008)。1.6.1 Bug FixesMove bases classes to airflow.sdk.bases (#48487)——基类在 Task SDK 中的新位置为 1.14.1 的SkipMixin/BranchMixIn迁移铺平了道路。1.7.0–1.7.4最低版本与 Python 支持调整1.7.0最低 Airflow 2.10.0MiscRemove AIRFLOW_2_10_PLUS conditions (#49877)、Bump min Airflow version in providers to 2.10 (#49843)1.7.1Miscnit: Remove unreachable code (#51110)1.7.2MiscReplace models.BaseOperator to Task SDK one for Common Providers (#52443)、Drop support for Python 3.9 (#52072)1.7.3Miscfix unreachable mypy warnings (#53575)、Add Python 3.13 support for Airflow. (#46891)、Remove type ignore across codebase after mypy upgrade (#53243)、Remove upper-binding for python-requires (#52980)、Temporarily switch to use , pattern instead of ~ (#52967)1.7.4MiscBump mypy to 1.18.1 (#55596)1.8.0兼容导入层的成型1.8.0 是本包的分水岭FeaturesSimplify version-specific imports in the Standard provider (#56867)Add SQLAs mapped_column to common-compat (#56880)Add comprehensive compatibility imports for Airflow 2 to 3 migration (#56790)MiscCommon.Compat: Extract reusable compat utilities and rename to sdk (#56884)Simplify version-specific imports in the Google provider (#56793)Migrate Apache providers Elasticsearch to common.compat (#57016)Doc-onlyRemove placeholder Release Date in changelog and index files (#56056)这一版本建立了今天仍在使用的两套机制可在源码中直接印证sdk.py 的兼容导入映射_IMPORT_MAP第 148 行起以名称 → 模块路径或按新到旧排序的路径元组登记了BaseHook、BaseOperator、DAG、chain、各异常类、Stats、redact、conf等上百个符号优先尝试 Airflow 3 的airflow.sdk路径、失败后回退 Airflow 2 路径_MODULE_MAP第 330–333 行支持整模块级别的回退导入timezone、io_AIRFLOW_3_ONLY_EXCEPTIONS第 313–316 行则保证DownstreamTasksSkipped、DagRunTriggerException等仅 3.x 存在的异常只在AIRFLOW_V_3_0_PLUS时注册。_compat_utils.py 的create_module_getattr模块级__getattr__懒加载工厂先查rename_map新名→旧路径旧名如Asset→airflow.datasets.Dataset再查module_map、import_map全部失败才抛出带完整候选路径信息的ImportError/AttributeError。sdk.py 末尾即以三张映射表调用该工厂生成__getattr__。1.9.0向后兼容的异步连接Featuresfeat: backwards comp get async conn (#57143)MiscConvert all airflow distributions to be compliant with ASF requirements (#58138)对应实现即 connection/init.py 中的get_async_connection(conn_id, hook)若 Hook 类提供aget_connection则直接await否则通过asgiref.sync.sync_to_async包装同步的get_connection。这也解释了 pyproject.toml 中为何引入asgiref作为运行时依赖。1.10.0 / 1.10.1最低 Airflow 2.11.0 与异常回迁1.10.0FeaturesAdjust compat HookLevelLineage for new add_extra method (#58057)MiscMove out some exceptions to TaskSDK (#54505)、Bump minimum Airflow version in providers to Airflow 2.11.0 (#58612)1.10.1MiscAdd backcompat for exceptions in providers (#58727)、Move the traces and metrics code under a common observability package (#56187)、Bump minimum prek version to 0.2.0 (#58952)1.10.0 是最后一次提高最低 Airflow 版本当前 pyproject.toml 的apache-airflow2.11.0正是这一决策的产物。1.11.0共享 module_loading 与循环导入修复FeaturesExtract shared module_loading distribution (#59139)Bug FixesFix circular import errors when CeleryExecutor is used with sentry (#59745)MiscPrevent client secrets and proxy credentials from being logged in Microsoft Graph hook logs (#59688)共享库抽取体现在仓库顶层的 shared/module_loading/ 目录。1.12.0从 compat SDK 消费配置与异常FeaturesAllow providers to consume AirflowConfigException from compat sdk (#60044)Source conf from airflow.sdk.configuration.conf for apache providers (#59966)Consume AirflowOptionalProviderFeatureException from compat sdk in providers (#60335)MiscNew year means updated Copyright notices (#60344)、Move listeners module to shared library for client server separation (#59883)、Move over plugins_manager to a shared library (#59956)其中从airflow.sdk.configuration.conf取 conf在 sdk.py 中体现为conf: (airflow.sdk.configuration, airflow.configuration)的双路径映射即 Airflow 3 下从 Task SDK 读取配置、Airflow 2 下回退到airflow.configuration。listeners/plugins_manager 的迁移分别对应顶层 shared/listeners/ 与 shared/plugins_manager/ 目录。1.13.0 / 1.13.1PythonOperator 异步 callable 与血缘迁移1.13.0带有一条重要注记新功能Add support for async callables in PythonOperator仅在 Airflow 3.2 上生效。FeaturesAdd support for async callables in PythonOperator (#60268)Bug FixesFix BaseAsyncOperator in common-compat provider for Airflow 2.x (#60663)MiscOption to disable exporting of legacy metric names (#53722)、Define TaskInstanceKey in task-sdk to support client server separation (#60776)1.13.1FeaturesMove lineage from airflow core to task sdk (#61157)——血缘模块从 Airflow 核心移入 Task SDKChangelog 排除区显示该迁移曾因 revert/重做经历 #60968→#61151→#61157 的过程。此后 Hook 血缘的实现以 lineage/hook.py 与 lineage/entities.py 为准测试见 tests/unit/common/compat/lineage/。1.14.0–1.14.3Stats 路由、SDK 化重构与 2.x 兼容修复1.14.0FeaturesRoute providers to consume Stats from common compat provider (#61812)——各提供者改为经由兼容层消费Stats对应sdk.py中Stats: (airflow.sdk.observability.stats, airflow.observability.stats, airflow.stats)的三级回退该三级路径正是 1.10.1 将 traces/metrics 归入 common observability 包的结果。1.14.1MiscConsolidate SkipMixin imports through common-compat layer (#62776)、Move SkipMixin and BranchMixIn to Task SDK (#62749)、Move determine_kwargs and KeywordParameters to SDK DecoratedOperator (#62746)。1.14.2MiscAdd Python 3.14 Support (#63520)、Remove usage of deprecated module airflow.security.permissions (#63335)、Remove remaining session query usages (#62758)。1.14.3MiscAdd OpenLineage parent and transport info injection to EmrServerlessStartJobOperator (#64807)、Add OpenLineage parent info injection to GlueJobOperator (#64513)、Fix RESOURCE_ASSET compatibility with Airflow 2.x in common-compat (#64933)。1.15.0–1.17.0DAG 版本化持久化与异步 Hook 能力1.15.0FeaturesAdd BundleVersion dataclass and version_data persistence to DagVersion (#66491)1.16.0FeaturesAdded get_async_hook in common.compat provider (#69208)Bug FixesFix RecursionError in common.compat hook lineage add_extra polyfill (#68735)——修复了 Hook 血缘add_extra兼容垫片在 Airflow 2.x 上的递归错误可视为 1.10.0HookLevelLineage调整的后续修正。1.17.0FeaturesHonor async hook subclass overrides in get_async_connection (#69140)——get_async_connection 新增hook参数使子类对aget_connection/get_connection的覆写得以生效而非固定落到BaseHook。1.18.0审计日志与权限细化FeaturesGate audit log rows not tied to a Dag on a dedicated AccessView (#70759)Add IMPORT_ERRORS_ALL permission for import errors of files with no registered Dag (#69790)权限与访问视图的兼容实现位于 security/ 子包permissions.py、access_view.py测试见 tests/unit/common/compat/security/。1.19.0当前版本BaseEventTrigger 别名与血缘实体修复FeaturesAdd common.compat alias for BaseEventTrigger (#72140)Bug FixesFix lineage entities sharing mutable defaults across instances (#72509)——修复血缘实体在不同实例间共享可变默认值的隐患。排除区还记录了Add missing unit tests for common.compat lineage entities (#72506)、Adopt flit 4 as the provider distribution build backend (#71186)对应 pyproject.toml 的flit_core4.0.2构建后端等未入正式条目的变更。四、主题纵览Changelog 背后的四条演进主线逐版本通读后可以提炼出四条主线每条都有明确的源码落点兼容导入层1.8.0 起成型sdk.py的三张映射表 _compat_utils.create_module_getattr懒加载覆盖 Hooks、Operators、Sensors、装饰器、模型、异常、监听器、配置等全部跨版本符号version_compat.py提供统一的版本常量。异步能力1.9.0 → 1.17.0get_async_connection起步get_async_hook1.16.0补齐 Hook 实例的异步获取1.17.0 再解决子类覆写语义1.13.0 为PythonOperator引入 async callable限 Airflow 3.2。血缘与 OpenLineage1.1.0 → 1.19.0Hook 血缘免 try/except1.1.0、NoOpCollector 接口补齐1.3.0、OL transport 注入 Spark 任务1.4.0、add_extra签名适配1.10.0、血缘迁入 Task SDK1.13.1、递归与可变默认值缺陷修复1.16.0/1.19.0。共享化与 SDK 化1.11.0 → 1.14.1module_loading、listeners、plugins_manager抽为共享库对应仓库顶层 shared/ 目录SkipMixin/BranchMixIn、异常、基类陆续迁入 Task SDK 并经由兼容层再导出。五、安装与版本选择建议按 docs/index.rst 的说明安装方式pip install apache-airflow-providers-common-compat # 需要 OpenLineage 功能时连同跨提供者依赖一起安装 pip install apache-airflow-providers-common-compat[openlineage]pyproject.toml 定义了两个可选 extraopenlineage依赖apache-airflow-providers-openlineage与standard依赖apache-airflow-providers-standard。版本选择上若你的 Airflow 低于 2.8/2.9/2.10需分别停留在 1.1.x/1.2.x/1.6.x 之前的对应版本参见第二节的最低版本表当前 1.19.0 要求apache-airflow2.11.0、Python ≥3.10支持至 3.14是跨 Airflow 2.11 与 Airflow 3.x 部署的默认选择1.13.0 的 asyncPythonOperatorcallable 特性仅在 Airflow 3.2 上可用升级评估时需注意此前提。正式发布的 sdist/wheel 及其 sha512、asc 校验文件可从 Apache 官方下载站获取docs/index.rst 的 Downloading official packages 一节给出了 1.19.0 的完整下载与校验项清单从源码安装可参考 installing-providers-from-sources.rst。六、结语common.compat的 Changelog 表面上是 30 余个版本的条目流水实际勾勒出 Airflow 从 2.x 迈向 3.x 期间兼容基础设施的完整建设史1.8.0 的综合兼容导入与sdk重命名是架构基石异步连接/Hook 能力线解决了跨版本执行差异血缘与 OpenLineage 线保障了可观测性语义一致而共享库与 Task SDK 迁移线则持续降低版本分支代码量。对于同时维护双版本环境的团队这份 Changelog 加上 sdk.py 的导入映射表就是判断某个符号/行为在我选定的 provider 版本中是否可用的两份权威依据。【免费下载链接】airflowApache Airflow - A platform to programmatically author, schedule, and monitor workflows项目地址: https://gitcode.com/GitHub_Trending/ai/airflow创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考