CANN/asc-devkit Mmad性能测试示例

Mmad (Cube Matrix Multiplication) Performance Test Example【免费下载链接】asc-devkit本项目是CANN 推出的昇腾AI处理器专用的算子程序开发语言原生支持C和C标准规范主要由类库和语言扩展层构成提供多层级API满足多维场景算子开发诉求。项目地址: https://gitcode.com/cann/asc-devkitOverviewThis example tests the performance of matrix multiply-add operations using Mmad series interfaces. It covers the computation path where matrices A and B are multiplied by the Cube computation unit from the L0A Buffer/L0B Buffer and the result is written to the L0C Buffer. This is a non-functional performance test. It does not verify computation results and only collects Cube computation latency.Supported Products and CANN VersionsProductArchitecture CodeCANN VersionAscend 950PR/Ascend 950DTdav-3510 CANN 9.1.0Atlas A3 Training/Inference Seriesdav-2201 CANN 9.0.0Atlas A2 Training/Inference Seriesdav-2201 CANN 9.0.0Directory Structure├── mmad_perf │ ├── CMakeLists.txt // Build configuration file │ ├── mmad_perf.asc // Mmad performance test implementation and entry point │ ├── perf.sh // Performance test script │ ├── generate_roofline.py // Roofline generation script │ ├── README.md // Example documentationExample DescriptionThis example uses the runtime parameterSCENARIO_NUMto select different data types and computation interfaces. Matrix dimensions are passed at runtime through./demo SCENARIO_NUM M K N. The kernel function name ismmad_perf_custom.The supported test scenarios vary by platform architecture:Platform ArchitectureSupported ScenariosDescriptionAtlas A3/A2 Training/Inference Platform (dav-2201)1-4Standard Mmad for b8, b16, b32, and structured sparse b8Ascend 950PR/950DT Platform (dav-3510)11-15Standard Mmad for b8, b16, b32, and MX quantization mxfp8, mxfp4Atlas A3/A2 Training/Inference Platform ScenariosSCENARIO_NUMInterfaceLeft Matrix ARight Matrix BResult Matrix CDescription1Mmadint8_tint8_tint32_tb8 standard matrix multiplication2Mmadhalfhalffloatb16 standard matrix multiplication3Mmadfloatfloatfloatb32 standard matrix multiplication4MmadWithSparseint8_tint8_tint32_tb8 4:2 structured sparse matrix multiplicationAscend 950PR/950DT Platform ScenariosSCENARIO_NUMInterfaceLeft Matrix ARight Matrix BScaleResult Matrix CDescription11Mmadint8_tint8_t-int32_tb8 standard matrix multiplication12Mmadhalfhalf-floatb16 standard matrix multiplication13Mmadfloatfloat-floatb32 standard matrix multiplication14MmadMxfp8_e4m3fn_tfp8_e4m3fn_tfp8_e8m0_tfloatmxfp8 quantized matrix multiplication15MmadMxfp4x2_e2m1_tfp4x2_e2m1_tfp8_e8m0_tfloatmxfp4 quantized matrix multiplicationNone of the scenarios include bias.Build and RunRun the following steps from the root directory of this example to build and run it.Configure Environment VariablesConfigure environment variables based on the installation method of the CANN development kit on your system.source ${install_path}/cann/set_env.sh${install_path}is the CANN package installation directory. If no installation directory is specified, the default installation path is/usr/local/Ascend. After runningsource, verify that the CANN package path is ready by runningecho ${ASCEND_HOME_PATH}. Duringperf.shbuild,ASC_DIRis derived from this path by default.To run this example in a simulation environment, configure the simulator dynamic library path. Different architectures correspond to different simulator models:# dav-2201 (Atlas A3/A2 Training/Inference Platform) export LD_LIBRARY_PATH${ASCEND_HOME_PATH}/tools/simulator/Ascend910B3/lib:$LD_LIBRARY_PATH # dav-3510 (Ascend 950PR/950DT Platform) export LD_LIBRARY_PATH${ASCEND_HOME_PATH}/tools/simulator/Ascend950PR_9589/lib:$LD_LIBRARY_PATHperf.shautomatically selects the corresponding model and configures this path based on the scenario. When runningmsprof op simulatormanually, export this path yourself.Build the ExampleBuild for Atlas A3/A2 Training/Inference Platform (dav-2201):mkdir -p build cd build cmake -DCMAKE_ASC_ARCHITECTURESdav-2201 -DCMAKE_ASC_RUN_MODEsim .. make -j cd ..Build for Ascend 950PR/950DT Platform (dav-3510):mkdir -p build cd build cmake -DCMAKE_ASC_ARCHITECTURESdav-3510 -DCMAKE_ASC_RUN_MODEsim .. make -j cd ..Run the ExampleThe runtime parameter order isSCENARIO_NUM M K N:# Atlas A3/A2 Training/Inference Platform examples (scenarios 1-4) ./build/demo 1 128 128 128 ./build/demo 4 128 256 128 # Ascend 950PR/950DT Platform examples (scenarios 11-15) ./build/demo 11 128 128 128 ./build/demo 14 128 512 128ParameterDescriptionSCENARIO_NUMTest scenario number. Use 1-4 for Atlas A3/A2 Training/Inference Platform; use 11-15 for Ascend 950PR/950DT PlatformMNumber of rows in left matrix AKNumber of columns in left matrix A, which equals the number of rows in right matrix BNNumber of columns in right matrix BMatrix dimensions must meet alignment requirements: M and N must be multiples of 16, and K must be a multiple of 64. For MX scenarios, K must be a multiple of 64. For consistency, K should be a multiple of 64 in all scenarios.Collecting Performance DataUse themsprof op simulatortool to collect detailed performance data in the simulation environment:msprof op simulator build/demo 1 128 128 128 Themsproftool requires CANN Commercial or Community Edition. For details, refer to the msprof Tool Installation Guide.After the command completes, a folder namedOPPROF_{timestamp}_XXXis generated in the default directory. The simulation performance data folder structure is as follows:├── dump # Raw performance data └── simulator ├── trace.json # Overall timing trace ├── visualize_data.bin # MindStudio Insight presentation file └── core0.cubecore0 ├── core0.cubecore0_instr_exe.csv # Cube core instruction-by-instruction execution details (including cycles and running_time) ├── core0.cubecore0_code_exe.csv # Execution details aggregated by code line └── trace.json # Cube core instruction-by-instruction timing trace (including high-precision dur)This example focuses on MMAD instruction performance data. View the specific performance data results as follows:cat ./OPPROF_*/simulator/core0.cubecore0/core0.cubecore0_instr_exe.csvKey metrics to monitor:MetricSourceDescriptiondur(us)durfield ofMMADevents incore0.cubecore0/trace.jsonMMAD instruction duration (high precision)cyclescyclescolumn of theMMADrow incore0.cubecore0_instr_exe.csvMMAD instruction cycle countPerformance Test Scriptperf.shperforms batch simulation builds, runsmsprof op simulator, extracts the duration and cycle count of MMAD instructions, and generates a CSV summary.# View help ./perf.sh --help # Test scenario 1, using dav-2201 by default ./perf.sh 1 # Test scenario 11, using dav-3510 by default ./perf.sh 11 # Explicitly specify the platform. The platform must match the scenario; otherwise, an error is reported ./perf.sh 1 dav-2201 ./perf.sh 14 dav-3510Built-in default shape sequences:ScenarioData TypeDefault Shape Sequence (M K N)1b8 (dav-2201)(32,32,32) (64,64,64) (128,128,128) (128,256,128) (128,512,128)2b16 (dav-2201)(32,32,32) (64,64,64) (64,128,64) (128,128,128) (128,256,128)3b32 (dav-2201)(32,32,32) (64,64,64) (64,128,64) (128,128,128)4sparse b8 (dav-2201)(64,64,64) (64,128,64) (128,128,128) (128,256,128) (128,512,128)11b8 (dav-3510)(32,32,32) (64,64,64) (128,128,128) (128,256,128) (256,256,256)12b16 (dav-3510)(32,32,32) (64,64,64) (64,128,64) (128,128,128) (256,128,256)13b32 (dav-3510)(32,32,32) (64,64,64) (64,128,64) (128,128,128)14mxfp8 (dav-3510)(64,64,64) (64,128,64) (128,128,128) (128,256,128) (256,256,256)15mxfp4 (dav-3510)(64,64,64) (128,128,128) (128,256,128) (128,512,128) (256,512,256)To test specific shapes, run./build/demo SCENARIO_NUM M K Ndirectly.After testing, results are saved toperf_data_${timestamp}_scenario${SCENARIO}/perf_result_scenario${SCENARIO}.csv. Rawmsprofdata is saved in thetest_${id}_${M}_${K}_${N}subdirectory under the same directory.Performance Metricsperf.shextracts the duration and cycle count of MMAD instructions from the simulation outputcore0.cubecore0/and writes them to a CSV. The columns in the CSV are as follows:ColumnSourceDescriptionMMAD_Dur(us)durfield ofMMADevents incore0.cubecore0/trace.jsonMMAD instruction duration (high precision, measured)Cyclescyclescolumn of theMMADrow incore0.cubecore0_instr_exe.csvMMAD instruction cycle count (measured)Performance Metric Calculation MethodsIn the simulation output,Cyclesis the measured cycle count for MMAD instructions, andMMAD_Dur(us)is the corresponding duration.perf.shreads these two columns directly. Measured performance and compute utilization are derived bygenerate_roofline.pyfromCyclesbased on hardware parallelism. No clock frequency conversion is needed.Mmad Performance CalculationMmad performance is expressed inMAC/cycle. The computation volume is counted as MAC operationsM×N×K(one multiply-add counts as 1 MAC). The hardware parallelism (element blockcube_m×cube_n×cube_kprocessed per cycle) and equivalent peak compute for each data type are as follows:ArchitectureData Typecube_mcube_ncube_kk_divisorEquivalent Peak Compute (MAC/cycle)dav-2201b816163218192dav-2201b1616161614096dav-2201b321616411024dav-2201sparse b8161632216384dav-3510b816163218192dav-3510b1616161614096dav-3510b32161611256dav-3510mxfp816163218192dav-3510mxfp4161664116384k_divisoris the K-dimension effective reduction factor. In sparse 4:2 structured sparsity, the right matrix is densified to[K/2, N], and the hardware traverses onlyK/2fractals in the K direction. Therefore, the computation cycle is halved and the equivalent peak compute is doubled (16×16×32×216384). For standard scenarios,k_divisor1.Theoretical and measured calculation formulas (first-instruction overhead is treated as 0):Computation cycles ceil(M/cube_m) × ceil(N/cube_n) × ceil(K / (cube_k × k_divisor)) Theoretical total cycles Computation cycles (first-instruction overhead is 0) Theoretical time (us) Theoretical total cycles / Frequency(MHz) Equivalent peak cube_m × cube_n × cube_k × k_divisor (MAC/cycle) Theoretical performance M×N×K / Theoretical total cycles (MAC/cycle, equals equivalent peak) Measured cycles Cycles (from MMAD in simulation instr_exe.csv, read directly) Measured performance M×N×K / Measured cycles (MAC/cycle) Compute utilization Measured performance / Equivalent peak × 100%For example, scenario 1 (dav-2201 b8, parallelism 16×16×32, k_divisor1) with shape[128, 128, 128]:Computation cycles ceil(128/16) × ceil(128/16) × ceil(128/32) 8 × 8 × 4 256 Theoretical total cycles 256 (first-instruction overhead is 0) Theoretical time 256 / 1800 0.1422 us Theoretical performance 128×128×128 / 256 8192 MAC/cycle (equals peak 8192)The platform clock frequency is set automatically byperf.shbased on the scenario:PlatformArchitecture CodeClock FrequencyApplicable ScenariosAtlas A3/A2 Training/Inference Platformdav-22011800 MHz1-4Ascend 950PR/950DT Platformdav-35101650 MHz11-15Roofline AnalysisThis example providesgenerate_roofline.py, which generates ASCII reports and charts from the CSV output produced byperf.sh.Python Package Dependenciesgenerate_roofline.pyuses Python standard libraries to read CSV and generate ASCII reports. To generate PNG/PDF charts, installmatplotlibandnumpy.python3 -m pip install --user matplotlib numpyIf these dependencies are not installed, the script still generates.txtASCII analysis reports but skips chart generation.# Automatically find the latest perf_data directory results python3 generate_roofline.py # Specify a CSV file python3 generate_roofline.py --csv perf_data_xxx_scenario1/perf_result_scenario1.csvThe default output file prefix is${perf_data directory name}_cube_roofline. The generated.txtfile contains detailed ASCII analysis. After installingmatplotlib, a.pngchart is also generated.Chart ExampleThe following is a Roofline chart example generated for scenario 1:NotesScenario numbers must match the platform: use scenarios 1-4 for dav-2201 and scenarios 11-15 for dav-3510.perf.shvalidates the match and reports errors for mismatches.Scenario 4 (MmadWithSparse) is supported only on dav-2201. Scenarios 14 and 15 (MmadMx) are supported only on dav-3510.MmadMx constraints: K must be a multiple of 64. The start addresses of A and B must be aligned to 1024 bytes for fp8 scenarios and 512 bytes for fp4 scenarios.This is a pure performance test and does not verify computation results. The kernel function does not initialize data in the L0 Buffer. The execution and latency of Mmad instructions depend only on the shape and architecture, not on data content.【免费下载链接】asc-devkit本项目是CANN 推出的昇腾AI处理器专用的算子程序开发语言原生支持C和C标准规范主要由类库和语言扩展层构成提供多层级API满足多维场景算子开发诉求。项目地址: https://gitcode.com/cann/asc-devkit创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考