C Fusion Pass Development Guide【免费下载链接】geGEGraph Engine是面向昇腾的图编译器和执行器提供了计算图优化、多流并行、内存复用和模型下沉等技术手段加速模型执行效率减少模型内存占用。 GE 提供对 PyTorch、TensorFlow 前端的友好接入能力并同时支持 onnx、pb 等主流模型格式的解析与编译。项目地址: https://gitcode.com/cann/geThis guide is for developers who want to write GE fusion passes in C. It is recommended to first read the language-independent mechanism description: Fusion Pattern Pass Mechanism.C passes are delivered as dynamic libraries. Developers implement a pass class, register it with GE, and compile it into a.so. When GE compiles a model, it loads the.soand executes the pass at a specified stage.If you are still exploring patterns, it is recommended to use the Python Fusion Pass Development Guide for quick validation; migrate to C once the pattern is stable.1. Which Pass to ChooseGoalRecommended InterfaceMatch a fixed topology and replace it with another topologyPatternFusionPassMatch a specific operator type and decompose it into multiple operatorsDecomposePassThis guide coversPatternFusionPassfirst, thenDecomposePass.2. Minimal Example: Delete Add(x, 0)Goal:x ----\ Add ---- out x ---- out 0 ----/The core C pass code consists of four parts:InheritPatternFusionPass.Patterns()defines the structure to match.MeetRequirements()checks if the constant is 0.Replacement()returns the replacement structure.#include cmath #include cstdint #include iostream #include es_all_ops.h #include ge/fusion/pass/pattern_fusion_pass.h using namespace ge; using namespace ge::fusion; class AddZeroPass : public PatternFusionPass { protected: std::vectorPatternUniqPtr Patterns() override { std::vectorPatternUniqPtr patterns; auto builder es::EsGraphBuilder(add_zero_pattern); auto x builder.CreateInput(0); auto zero es::Const(builder); auto add es::Add(x, zero); auto graph builder.BuildAndReset({add}); patterns.emplace_back(std::make_uniquePattern(std::move(*graph))); return patterns; } bool MeetRequirements(const std::unique_ptrMatchResult match_result) override { for (const auto node : match_result-GetMatchedNodes()) { AscendString type; node.GetType(type); if (type ! Const) { continue; } Tensor value; if (node.GetAttr(value, value) ! GRAPH_SUCCESS) { return false; } return IsZero(value); } return false; } GraphUniqPtr Replacement(const std::unique_ptrMatchResult match_result) override { auto builder es::EsGraphBuilder(add_zero_replacement); auto x builder.CreateInput(0); return builder.BuildAndReset({x}); } private: bool IsZero(const Tensor tensor) const { switch (tensor.GetTensorDesc().GetDataType()) { case DT_FLOAT: return std::fabs(*reinterpret_castconst float *(tensor.GetData())) 1e-6; case DT_DOUBLE: return std::fabs(*reinterpret_castconst double *(tensor.GetData())) 1e-15; case DT_INT32: return *reinterpret_castconst int32_t *(tensor.GetData()) 0; default: return false; } } }; REG_FUSION_PASS(AddZeroPass).Stage(CustomPassStage::kBeforeInferShape);A complete runnable example is available at AddZeroPass C Example.3. Patterns: Define What to FindPatterns()returns one or more patterns. Each pattern is a small graph.std::vectorPatternUniqPtr Patterns() override { std::vectorPatternUniqPtr patterns; auto builder es::EsGraphBuilder(pattern); auto a builder.CreateInput(0); auto b builder.CreateInput(1); auto c builder.CreateInput(2); auto matmul es::MatMul(a, b); auto add es::Add(matmul, c); auto graph builder.BuildAndReset({add}); patterns.emplace_back(std::make_uniquePattern(std::move(*graph))); return patterns; }This pattern represents:a ----\ MatMul ----\ b ----/ Add ---- pattern output c ----------------/To support bothMatMul AddandBatchMatMulV2 Add, create two patterns and add both topatterns.When writing patterns, note:External inputs are declared withCreateInput.Tensors that will still be used externally after replacement must be outputs of the pattern.Input count for normal operators must match the real graph.Do not use control edges, subgraphs, or nodes with dynamic input/output counts in patterns.4. MeetRequirements: Determine Whether to ReplacePatterns()only handles topology matching. If additional checks are needed after topology matching, writeMeetRequirements().For example, after matchingAdd(x, Const), verify that Const equals 0:bool MeetRequirements(const std::unique_ptrMatchResult match_result) override { for (const auto node : match_result-GetMatchedNodes()) { AscendString type; node.GetType(type); if (type ! Const) { continue; } Tensor value; if (node.GetAttr(value, value) ! GRAPH_SUCCESS) { return false; } return IsZero(value); } return false; }If no filtering is needed, this method can be omitted; it returnstrueby default.5. Replacement: Define What to Replace WithReplacement()returns the replacement graph.When deletingAdd(x, 0), the replacement graph has only one external input:GraphUniqPtr Replacement(const std::unique_ptrMatchResult match_result) override { auto builder es::EsGraphBuilder(replacement); auto x builder.CreateInput(0); return builder.BuildAndReset({x}); }When fusingMatMul AddintoGEMM:GraphUniqPtr Replacement(const std::unique_ptrMatchResult match_result) override { auto builder es::EsGraphBuilder(replacement); auto a builder.CreateInput(0); auto b builder.CreateInput(1); auto c builder.CreateInput(2); auto alpha builder.CreateScalar(1); auto beta builder.CreateScalar(1); auto gemm es::GEMM(a, b, c, alpha, beta); return builder.BuildAndReset({gemm}); }If the pass is registered after InferShape, shape information for new nodes in the replacement needs to be handled manually. Refer to existing examples for usingGeUtils::InferShapewhen shape inference is needed for replacement.6. CaptureTensor: Read Key Tensors in PatternWhenMeetRequirements()orReplacement()needs to know which real node corresponds to a intermediate tensor, capture it in the pattern.auto matmul es::MatMul(a, b); auto add es::Add(matmul, c); auto graph builder.BuildAndReset({add}); auto pattern std::make_uniquePattern(std::move(*graph)); pattern-CaptureTensor({*matmul.GetProducer(), 0}); patterns.emplace_back(std::move(pattern));After successful matching, retrieve frommatch_result:NodeIo matmul_output; if (match_result-GetCapturedTensor(0, matmul_output) ! GRAPH_SUCCESS) { return false; }Refer to capture tensor C example.7. PatternMatcherConfig: Put Simple Conditions in MatcherIf you want the matcher to directly check Const values or IR attributes, pass configuration to thePatternFusionPassconstructor.class MatmulAddFusionPass : public PatternFusionPass { public: MatmulAddFusionPass() : PatternFusionPass(PatternMatcherConfigBuilder() .EnableConstValueMatch() .EnableIrAttrMatch() .Build()) {} };Common configurations:ConfigurationEffectEnableConstValueMatch()Const values in pattern must match Const values in real graphEnableIrAttrMatch()IR attributes and values in pattern must match real graphIf judgment requires floating-point tolerance, dtype normalization, or more complex logic, it is still recommended to put it inMeetRequirements().Refer to PatternMatcherConfig C example.8. Register Execution StageUseREG_FUSION_PASSto registerPatternFusionPass:REG_FUSION_PASS(AddZeroPass).Stage(CustomPassStage::kBeforeInferShape);Common stages:C EnumerationUsage RecommendationCustomPassStage::kBeforeInferShapeMost commonly used. Replacement will go through unified shape inferenceCustomPassStage::kAfterInferShapeUse when dependent on inferred shape; replacement must ensure shape informationCustomPassStage::kAfterBuiltinFusionPassExecute after GE built-in fusionCustomPassStage::kAfterOriginGraphOptimizeExecute after original graph optimizationFor initial development, usekBeforeInferShape.9. Writing DecomposePassIf you want to decompose one node into multiple nodes, useDecomposePass.Skeleton is as follows:#include ge/fusion/pass/decompose_pass.h #include es_all_ops.h using namespace ge; using namespace ge::fusion; class MyDecomposePass : public DecomposePass { public: explicit MyDecomposePass(const std::vectorAscendString op_types) : DecomposePass(op_types) {} protected: bool MeetRequirements(const GNode matched_node) override { // Read matched_node attributes to determine if decomposition is needed return true; } GraphUniqPtr Replacement(const GNode matched_node) override { auto builder es::EsGraphBuilder(replacement); // Construct subgraph for replacing matched_node ... return builder.BuildAndReset({output}); } }; REG_DECOMPOSE_PASS(MyDecomposePass, {Conv2D}).Stage(CustomPassStage::kAfterInferShape);The second parameter ofREG_DECOMPOSE_PASSis the list of operator types to match. GE will pass real nodes of these types to the pass, thenMeetRequirements()makes further judgment.Complete example see DecomposePass C example.10. Compilation and RunningEach example directory comes withCMakeLists.txt. General process is as follows.Set CANN environment variables:source ${ASCEND_PATH}/set_env.shCompile and install pass dynamic library:mkdir build cd build cmake .. make -j$(nproc) target_name make installCMake configuration not expanded in this document, use examples as template during development:AddZeroPass CMakeLists.txtMatMulAdd CMakeLists.txtIf need to add new header file paths or link libraries, append in corresponding positions of exampleCMakeLists.txt, do not delete original configuration.Offline compilation can useatcto trigger:atc --model./model.onnx --framework5 --soc_versionxxx --output./modelOnline scenario usually triggers GE compilation throughtorch_forward.pyin examples.11. Verification and TroubleshootingRecommend enabling graph dump:export DUMP_GE_GRAPH1Compare graphs before and after pass:PreRunBegin: Before pass execution.RunCustomPass...: After custom pass execution.Common problems:PhenomenonPossible CauseCheck Methodpass not executed.sonot installed to directory GE will load, or registration stage incorrectCheck installation path and registration macropattern not matchedOperator type, input count, output boundary inconsistentCompare dump graph andPatterns()matched but not replacedMeetRequirements()returnedfalsePrint matched node attributesGraph abnormal after replacementreplacement output did not cover Tensor needed by external consumersGo back to mechanism document boundary rulesWhen more logs needed, can set:export ASCEND_SLOG_PRINT_TO_STDOUT1 export ASCEND_GLOBAL_LOG_LEVEL0When usingatc, can add--logdebug.12. Recommended Reading OrderFusion Pattern Pass MechanismAddZeroPass C exampleMatMulAdd C examplecapture tensor C examplePatternMatcherConfig C exampleDecomposePass C example【免费下载链接】geGEGraph Engine是面向昇腾的图编译器和执行器提供了计算图优化、多流并行、内存复用和模型下沉等技术手段加速模型执行效率减少模型内存占用。 GE 提供对 PyTorch、TensorFlow 前端的友好接入能力并同时支持 onnx、pb 等主流模型格式的解析与编译。项目地址: https://gitcode.com/cann/ge创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考