Atlas 300I duo mindie 部署qwen3-vl-30b-a3b笔记

这里写自定义目录标题Atlas 300I duo mindie 部署qwen3-vl-30b-a3b笔记第一步 mindie镜像下载安装第二步第三步第四步第五步第六步第七步 启动服务其他容器简单推理测化命令参考连接Atlas 300I duo mindie 部署qwen3-vl-30b-a3b笔记第一步 mindie镜像下载安装镜像地址https://www.hiascend.com/developer/ascendhub/detail/af85b724a7e5469ebd7ea13c3439d48fdocker pull --platformarm64 swr.cn-south-1.myhuaweicloud.com/ascendhub/mindie:3.0.0b2-300I-Duo-py311-openeuler24.03-lts第二步dockerrun-it-d\--shm-size64g\--namemindie_310\--device/dev/davinci0\--device/dev/davinci1\--device/dev/davinci2\--device/dev/davinci3\--device/dev/davinci4\--device/dev/davinci5\--device/dev/davinci_manager\--device/dev/hisi_hdc\-v/usr/local/Ascend/driver:/usr/local/Ascend/driver:ro\-v/usr/local/dcmi:/usr/local/dcmi:ro\-v/usr/local/bin/npu-smi:/usr/local/bin/npu-smi:ro\-v/usr/local/Ascend/firmware:/usr/local/Ascend/firmware:ro\-v/usr/local/sbin:/usr/local/sbin:ro\-v/data/alg:/root/alg\-p29511:29511\-p29512:29512\-p29513:29513\-p1025:1025\-p1026:1026\-p1027:1027\-p1121:1121\swr.cn-south-1.myhuaweicloud.com/ascendhub/mindie:3.0.0-300I-Duo-py311-openeuler24.03-lts\bash第三步dockerexec-itmindie_310 /bin/bashcd/usr/local/Ascend/atb-models/requirements/models pipinstall-rrequirements_qwen3vl.txt第四步cd/usr/local/Ascend/mindie/latestchmod750mindie-servicechmod-R550mindie-service/binchmod550mindie-service/libchmod440mindie-service/lib/*chmod550mindie-service/lib/grpcchmod440mindie-service/lib/grpc/*chmod-R550mindie-service/includechmod-R550mindie-service/scriptschmod750mindie-service/logschmod750mindie-service/confchmod640mindie-service/conf/config.jsonchmod700mindie-service/securitychmod-R700mindie-service/security/*第五步# 配置CANN环境默认安装在/usr/local目录下source/usr/local/Ascend/ascend-toolkit/set_env.sh# 配置加速库环境source/usr/local/Ascend/nnal/atb/set_env.sh# 配置模型仓环境变量source/usr/local/Ascend/atb-models/set_env.sh# MindIEsource/usr/local/Ascend/mindie/latest/mindie-llm/set_env.shsource/usr/local/Ascend/mindie/latest/mindie-service/set_env.sh# 开启MindIE日志打印exportMINDIE_LOG_TO_STDOUTtrue第六步cd/usr/local/Ascend/mindie/latest/mindie-servicecdmindie-service/confvimconfig.json 修改httpsEnabled:false修改modelName:qwen2-7b,modelWeightPath:/home/weight,其余参数视情况修改注意maxSeqLen:1280000,#2560倍数maxInputTokenLen:1024000,1024倍数maxPrefillTokens:1024000,需大于maxInputTokenLen第七步 启动服务nohup./bin/mindieservice_daemonoutput.log21或 ./bin/mindieservice_daemon 回显 Daemon start success!表明启动成功测试importtimeimportrequestsimportjsondeftest_generation_speed(prompt输出工地安全准则,max_tokens1204000,runs3):urlhttp://127.0.0.1:1025/v1/chat/completionstotal_tokens0total_time0foriinrange(runs):print(f\n--- Run{i1}---)start_timetime.time()generated_tokens0responserequests.post(url,headers{Content-Type:application/json},json{model:qwen3-vl-qwen3-vl-a3b,messages:[{role:user,content:prompt}],max_tokens:max_tokens,# 增加生成长度temperature:0.7,stream:True},timeout3600,# 增加超时时间streamTrue)ifresponse.status_code200:print(生成内容: ,end,flushTrue)forlineinresponse.iter_lines():ifline:lineline.decode(utf-8)ifline.startswith(data: ):ifline.strip()data: [DONE]:breaktry:chunkjson.loads(line[6:])contentchunk[choices][0][delta].get(content,)ifcontent:print(content,end,flushTrue)generated_tokenslen(content.split())except:passprint()# 换行else:print(fRun{i1}: Error{response.status_code}-{response.text})continuedurationtime.time()-start_time speedgenerated_tokens/durationifduration0else0print(fRun{i1}: ~{generated_tokens}tokens in{duration:.2f}s →{speed:.2f}tokens/s)total_tokensgenerated_tokens total_timedurationiftotal_time0:print(f\nAverage:{total_tokens/total_time:.2f}tokens/s over{runs}runs)if__name____main__:test_generation_speed()输出其他容器简单推理测化命令torchrun--standalone--nnodes1--nproc_per_node1\examples/run_pa.py\--model_path/root/alg/qwen3-vl-8b\--input_texts什么是深度学习深度学习和机器学习有什么区别请举例说明深度学习的应用场景。\--max_output_length512\--max_batch_size3\--trust_remote_code参考连接https://www.hiascend.com/document/detail/zh/mindie/300/envdeployment/instg/docs/zh/user_guide/install/source/image_usage_guide.mdhttps://gitcode.com/Ascend/MindIE-LLM/blob/v3.0.0/docs/zh/user_guide/quick_start/quick_start.md