Example Usage Guide【免费下载链接】geGEGraph Engine是面向昇腾的图编译器和执行器提供了计算图优化、多流并行、内存复用和模型下沉等技术手段加速模型执行效率减少模型内存占用。 GE 提供对 PyTorch、TensorFlow 前端的友好接入能力并同时支持 onnx、pb 等主流模型格式的解析与编译。项目地址: https://gitcode.com/cann/geFunction DescriptionThis example is a custom pass for moving ReLU before Concat. When fusion pass scenarios involveoperators with dynamic input/output counts, refer to this example. The example provides both online inference and atc tool offline model compilation to demonstrate how the framework calls custom pass to complete graph optimization, using eager style api and fusion interfaces.Directory Structure├── src │ ├──move_relu_before_concat_pass.cpp // pass implementation file ├── CMakeLists.txt // build script ├── data | ├──es_gen_air.py // export air | ├──torch_forward.py // torch script for online inference |—— gen_es_api | |——CMakeLists.txt // build script for generating eager style apiEnvironment RequirementsCompiler: GCC 7.3.xPython and dependency versions: python3.9, pytorch2.1Completed environment preparation.Implementation StepsDefineMoveReluBeforeConcatPassclass inheritingFusionBasePass.Override base classFusionBasePassRunmethod, implement custom pass logic.DefineFindConcatNodesMeetRequirementsto traverse nodes in graph, get Concat nodes meeting conditions.DefineMoveReluBeforeConcatto implement graph modification:Replacementbuilds replacement structure based on concat nodeGetSubgraphBoundarybuilds boundary of subgraph to be replacedFinally callSubgraphRewriterReplacemethod to implement replacementProgram CompilationAssume CANN package installation directory is INSTALL_PATH, e.g.,/home/HwHiAiUser/Ascend/.Configure environment variables.Run environment variable script from the package:source ${ASCEND_PATH}/set_env.sh${ASCEND_PATH}is the cann path under CANN package installation directory. Replace with actual installation path, e.g.,${INSTALL_PATH}/cann.Modify the following information inCMakeLists.txtas needed.ASCEND_PATH: Can set default package path. If$ASCEND_HOME_PATHis set via set_env.sh, no modification needed.PASS_SO_DIR: Can set custom fusion pass dynamic library installation directory name, default ispass_so_dir.target_include_directories: Header files to include. For this example, no modification needed. For user-developed code, when adding headers, add lines below the example, do not delete existing items. If network has custom operators, add custom operator prototype definition header files.target_link_libraries: Libraries to link. For this example, no modification needed. For user-developed code, when adding link libraries, add lines below the example, do not delete existing items.Do not link other so from the package, otherwise may cause compatibility issues during future upgrades.Execute sequentially:mkdir build cd build cmake ..Execute make command to compile custom pass so. After successful compilation, use make install to install dynamic library file libmove_relu_before_concat_pass.so to custom fusion pass directory. Can add optional parameter-j$(nproc)after make for parallel build,$(nproc)dynamically gets CPU core count.make -j$(nproc) move_relu_before_concat_pass make installAfter example validation completes, execute the following command to clean custom pass so installed under CANN package, to avoid affecting subsequent UT/ST:make clean_custom_passProgram ExecutionConfigure environment variables (if already done, skip).Run environment variable script from the package:source ${ASCEND_PATH}/set_env.shReplace${ASCEND_PATH}with actual package installation path.Use ATC offline inference.Set environment variable to dump model graph during compilation:export DUMP_GE_GRAPH1Install es_all.whlpip install --force-reinstall --upgrade --target ${ASCEND_PATH}/python/site-packages/ ${BUILD_PATH}/es_output/whl/es_all-*****.whlReplace${BUILD_PATH}with actual build directory path.Set environment variable to add es_all.so pathLD_LIBRARY_PATH${BUILD_PATH}/es_output/lib64:${LD_LIBRARY_PATH}Enter data directory and execute .py file to export air:python es_gen_air.pyAfter execution, .air format model file named graph.air is generated in data directory.Execute ATC tool command (for detailed ATC tool instructions, visit Ascend Documentation and search ATC Offline Model Compilation Tool), modifysoc_versionper actual environment:atc --model./graph.air --framework1 --soc_versionxxx --output./modelFollowing log appears:MoveReluBeforeConcatPass Define Replacement for MoveReluBeforeConcatPass Replacement of MoveReluBeforeConcatPass succeededOnline inferenceSet environment variable to dump model graph during compilation:export DUMP_GE_GRAPH1Enter data directory and execute .py file for online inference (ensure torch_npu plugin is installed for online inference):python torch_forward.pyFollowing log appears:MoveReluBeforeConcatPass Define Replacement for MoveReluBeforeConcatPass Replacement of MoveReluBeforeConcatPass succeededView execution resultsAfter ATC tool command completes, a series of .pbtxt files are generated in the directory. Compare the following dump graphs:ge_onnx_xxxxx_PreRunBegin.pbtxtdump graph before executionge_onnx_xxxxx_RunCustomPassBeforeInferShape.pbtxtcustom pass dump graph before InferShape executionFind model optimized as expected, i.e., ReLU moved before Concat.If expected result is not obtained, can set the following environment variables (if using atc command, also add parameter--logdebug) to print logs to screen for troubleshooting:export ASCEND_SLOG_PRINT_TO_STDOUT1 # print logs to screen export ASCEND_GLOBAL_LOG_LEVEL0 # log level debug【免费下载链接】geGEGraph Engine是面向昇腾的图编译器和执行器提供了计算图优化、多流并行、内存复用和模型下沉等技术手段加速模型执行效率减少模型内存占用。 GE 提供对 PyTorch、TensorFlow 前端的友好接入能力并同时支持 onnx、pb 等主流模型格式的解析与编译。项目地址: https://gitcode.com/cann/ge创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考