1目的。 使用多个相机采集图像经计算处理获得感兴趣区域的映射图将映射图拼接为一幅图像以达到扩大视野范围的目的。示意图变换后:2原理。 计算C1世界坐标系中感兴趣区域的右上角C1A的世界坐标将此坐标设为创建ImageMap所需的PoseNewOrigin1的原点。 计算出C1世界坐标系中感兴趣区域的右上角C1B基于世界坐标系默认原点C1Origin的偏移矩阵结合C1世界坐标系的默认原点C1Origin与C2世界坐标系的默认原点C2Origin存在偏移矩阵关系两个坐标系的原点存在固定的距离可计算出C1B相对C2世界坐标系默认原点C2Origin的变化矩阵。 因C1B与C2世界坐标系中感兴趣区域的左上角C2A重叠所以C1B相对C2世界坐标系默认原点C2Origin的偏移旋转矩阵可视为C2A相对C2世界坐标系默认原点C2Origin的偏移旋转矩阵。结合C2相机的位姿可计算出C2A在C2相机坐标系的位姿PoseNewOrigin2。 通过PoseNewOrigin1PoseNewOrigin2获得采集图像与感兴趣区域位置的映射关系map再通过map获得裁剪后的兴趣图像兴趣图像拼接后即为所需的大视野图像。在像素坐标系中原点位于图像的左上角。所以这里将原点重置到左上角便于后续生成原点位于左上角的映射图。注意该案例的标定模型使用的是两个相机一个标定板不是两个标定板不是两个标定板左相机的标定板水平向右移动DistancePlates成为右相机的标定板。2注意事项。进行变换时尽量使用矩阵而不是位姿位姿在变换时误差较大达不到预期效果。标定完成后默认标定板的几何中心或第一个特征点如棋盘格左上角为对应世界坐标系的原点。3主要的算子。3.1elliptic_axis1. 功能定义计算输入区域Region的等效椭圆参数包括Ra等效椭圆长轴半径主半径满足Ra ≥ Rb ≥ 0Rb等效椭圆短轴半径次半径Phi长轴与水平轴X轴的夹角弧度制范围-π/2 Phi ≤ π/22. 核心特性等效性生成的椭圆与输入区域具有相同的二阶矩惯性矩特性包括方向、长宽比和质心位置。计算基础基于区域的几何矩moments_region_2nd算子计算的M20、M02、M11推导出Ra、Rb和Phi特殊输入处理空区域或单点区域返回Ra Rb 0直线状区域如线段返回Rb 0Phi为线段方向角3. 应用场景方向检测通过Phi判断工业零件如PCB焊盘、金属条的旋转角度形状筛选结合select_shape算子通过anisometry即Ra/Rb过滤细长或圆形区域椭圆拟合验证与gen_ellipse联用生成等效椭圆可视化验证计算结果4. 注意事项角度一致性对于非规则形状如L型区域Phi与smallest_rectangle2算子的角度结果一致但等效椭圆面积通常大于原区域面积多区域输入支持区域数组输入输出参数为对应长度的数组XLD扩展针对轮廓点云XLD需使用elliptic_axis_points_xld算子其计算逻辑类似但基于点集矩3.2gen_image_to_world_plane_map图像映射算子用于建立图像平面与世界坐标系中z0平面之间的投影关系。通过相机内参和世界坐标系的3D位姿Pose计算图像像素到世界平面的单应性变换矩阵消除透视畸变和径向畸变模拟无畸变相机垂直拍摄的效果。1. 输入参数. CameraParam (相机内参矩阵)定义包含焦距(f)、主点坐标(cx,cy)、畸变系数(k1,k2,k3,p1,p2)等标定信息格式通常为数组形式例焦距,kappa,Sx,Sy,Cx,Cy,焦距,kappa,Sx,Sy,Cx,Cy,图像宽,图像高作用提供镜头畸变矫正所需的物理参数直接影响像素坐标系到世界坐标系的转换精度.WorldPose (世界坐标系位姿)定义描述目标平面(z0平面)在相机坐标系中的3D位置和方向构成通常包含旋转分量(Rx,Ry,Rz)和平移分量(Tx,Ty,Tz)使用四元数或旋转向量表示姿态作用确定世界平面与成像平面的空间几何关系用于消除透视畸变. WidthIn/HeightIn (输入图像尺寸)单位像素作用限定原始图像的物理分辨率范围确保映射矩阵与输入图像尺寸匹配2. 输出参数. Map (映射图像)数据类型多通道图像int4或int8格式内容存储每个输出像素对应的原始图像坐标偏移量用于后续map_image算子转换通道数单通道直接索引映射无插值双通道支持双线性插值.WidthMapped/HeightMapped (输出图像尺寸)单位像素设置依据根据目标平面实际物理尺寸和Scale参数计算得出. Scale (像素-世界单位比例)定义输出图像中单个像素对应的物理尺寸毫米/英寸等典型值当Scale1时1像素1世界单位常用于等比例映射场景3. 辅助参数.MapType (映射类型)可选值定义插值方式如’bilinear’或’nearest_neighbor’影响决定map_image算子执行时的采样策略和图像质量.参数关联性参数组关联性说明CameraParam ↔ Scale内参矩阵的焦距参数直接影响Scale的物理意义需保证单位统一性WorldPose ↔ Map位姿参数决定映射矩阵的几何变换类型如纯旋转、平移或复合变换Map ↔ map_image映射图像作为中间数据通过map_image实现实际图像转换4. 注意事项. CameraParam的完整性需确保包含所有必要畸变参数如径向畸变k1/k2/k3和切向畸变p1/p26.WorldPose的精度要求姿态误差会导致校正后的图像产生倾斜或比例失真推荐使用高精度标定方法26.Scale的合理性过大的Scale值可能导致输出图像分辨率不足过小则可能超出内存限制3.3tile_images是HALCON中用于平铺多图像的核心算子其参数体系直接影响图像拼接效果。以下为参数详解及关联分析输入参数Images (输入图像组)类型图像对象数组要求所有输入图像需具有相同通道数支持不同尺寸但会统一处理作用提供待拼接的原始图像数据图像顺序影响最终排列结构NumColumns (列数)类型整数定义指定输出图像中每行排列的子图列数控制横向排列密度动态调整若输入图像总数非列数整数倍末行可能留空或填充未定义区域TileOrder (排列顺序)可选值horizontal从左到右横向优先填充vertical从上到下纵向优先填充影响决定子图在输出画布上的布局方向适用于不同对比需求. 输出参数TiledImage (拼接图像)类型图像对象尺寸规则宽度NumColumns × 最大子图宽度高度ceil(总子图数/NumColumns) × 最大子图高度内容特性小尺寸子图居中显示边缘填充背景色默认黑色通道数与输入一致支持多通道拼接.参数耦合关系参数关联影响NumColumns ↔ 输出尺寸列数直接影响输出图像的宽度较大的列数会减少行数TileOrder ↔ 排列逻辑横向优先时按行填充纵向优先时按列填充影响视觉对比效果输入尺寸 ↔ 输出块尺寸所有子图统一为最大宽高可能导致小图边缘留空或大图裁剪. 注意事项通道一致性输入图像通道数必须相同否则触发运行时错误内存限制拼接大尺寸图像组时需评估输出画布尺寸避免内存溢出填充策略未占满区域显示为黑色若需自定义背景需结合其他算子预处理. 与变体算子对比相较于tile_images_offset的可控布局tile_images更适用于自动化排列场景典型差异包括tile_images_offset支持自定义子图位置和输出画布尺寸灵活性更高tile_images通过列数和排列顺序简化参数配置适合快速拼接对比4程序详解。4.1相机标定* 参考案例库two_camera_calibration.hdev * 基于相机标定的高精度拼接示例程序 dev_update_off () ImgPath : 3d_machine_vision/multiple_cameras/ * * * Window of left image and right image * dev_close_window () read_image (Image1, ImgPath camera1_ref) get_image_size (Image1, Width, Height) WindowScale : 0.66 dev_open_window (0, 0, Width * WindowScale, Height * WindowScale, black, WindowHandle1) dev_set_color (green) dev_set_draw (margin) dev_set_line_width (2) dev_open_window (0, Width * WindowScale 6, Width * WindowScale, Height * WindowScale, black, WindowHandle2) dev_set_color (green) dev_set_draw (margin) dev_set_line_width (2) dev_set_window (WindowHandle1) dev_set_part (0, 0, Height - 1, Width - 1) set_display_font (WindowHandle1, 16, mono, true, false) dev_set_window (WindowHandle2) dev_set_part (0, 0, Height - 1, Width - 1) set_display_font (WindowHandle2, 16, mono, true, false) * * * * * Calibration * * * ----------------------------- * Two caltab images * ----------------------------- * Assume that the two cameras are already calibrated (internal camera parameters) * 相机1标定后内参 CamParam1 : [0.01619,-734.789,7.402e-006,7.4e-006,324.911,256.894,640,480] * 相机2标定后的内参 CamParam2 : [0.0162584,-763.35,7.39842e-006,7.4e-006,324.176,245.371,640,480] * read_image (Image1, ImgPath camera1_ref) read_image (Image2, ImgPath camera2_ref) dev_set_window (WindowHandle1) dev_display (Image1) dev_set_window (WindowHandle2) dev_display (Image2) * CaltabName : caltab_30mm.descr create_calib_data (calibration_object, 2, 1, CalibDataID) set_calib_data_calib_object (CalibDataID, 0, CaltabName) * dev_set_window (WindowHandle1) set_calib_data_cam_param (CalibDataID, 0, area_scan_division, CamParam1) find_calib_object (Image1, CalibDataID, 0, 0, 0, [], []) get_calib_data_observ_points (CalibDataID, 0, 0, 0, RCoord1, CCoord1, Index1, Pose1) get_calib_data_observ_contours (Caltab, CalibDataID, caltab, 0, 0, 0) dev_set_color (green) dev_display (Caltab) dev_set_window (WindowHandle2) set_calib_data_cam_param (CalibDataID, 1, area_scan_division, CamParam2) find_calib_object (Image2, CalibDataID, 1, 0, 0, [], []) get_calib_data_observ_points (CalibDataID, 1, 0, 0, RCoord2, CCoord2, Index2, Pose2) get_calib_data_observ_contours (Caltab, CalibDataID, caltab, 1, 0, 0) dev_set_color (green) dev_display (Caltab) * * 显示camera1世界坐标系 dev_set_colored (12) disp_3d_coord_system (WindowHandle1, CamParam1, Pose1, 0.01) * 显示camera2的世界坐标系 disp_3d_coord_system (WindowHandle2, CamParam2, Pose2, 0.01) disp_message (WindowHandle1, Calibration successful, window, 12, 12, black, true) disp_continue_message (WindowHandle1, black, true) stop ()真实物理世界中物体的三维位置原点通常定义在标定板的几何中心或第一个特征点如棋盘格左上角如下所示标定板使用过程首先找到标定板在左相机的姿态。其次向右水平移动DistancePlates距离至右相机视野。最后查找标定板在右相机坐标系的位姿。可知标定板的中心即为世界坐标系的默认原点camera01的世界坐标系原点水平偏移DistancePlates即可变换为camera02的世界坐标系原点如下所示4.2 修改poses 创建 map_image* * Modify poses such that map_image creates images, * which can be merged in order to form the image mosaic * * * Determine the offset between the calibration plate surface and the object surface * 两个标定板坐标系统原点之间的距离单位m,如4.1中的插图所示 DistancePlates : 0.06488 ThicknessCaliper : 2.9 / 1000.0 ThicknessPlate : 5.65 / 1000.0 * 标定板厚度 DiffHeight : ThicknessPlate - ThicknessCaliper * * 分辨率确定映射后图像的大小 * PixelSize : 0.0001 * * Define the upper left corner of the mosaic image and the size of the two mapped images * 视野边框所占比例 BorderInPercent : 7 * 左右相机视野重叠比例 OverlapInPercent : 20 get_image_size (Image1, WidthImage1, HeightImage1) * ROI左上角坐标 ULRow : HeightImage1 * BorderInPercent / 100.0 ULCol : WidthImage1 * BorderInPercent / 100.0 * Camera01视野ROI左上角坐标世界坐标系 image_points_to_world_plane (CamParam1, Pose1, ULRow, ULCol, m, ULX, ULY) LowerRow : HeightImage1 * (100 - BorderInPercent) / 100.0 RightCol : WidthImage1 * (100 - OverlapInPercent / 2.0) / 100.0 * Camera01视野ROI左下角世界坐标系 image_points_to_world_plane (CamParam1, Pose1, LowerRow, ULCol, m, X1, LowerY) * Camera01视野ROI右上角世界坐标系 image_points_to_world_plane (CamParam1, Pose1, ULRow, RightCol, m, RightX, Y1) * Camera01视野ROI有效范围 image_points_to_world_plane (CamParam1, Pose1, LowerRow, RightCol, m, X, Y) HeightRect : int((LowerY - ULY) / PixelSize) WidthRect : int((RightX - ULX) / PixelSize) * -------------------------------------------------------------------- * Translate pose of left camera into plane of measuerement * -------------------------------------------------------------------- * 设置Camera01世界坐标系的原点 set_origin_pose (Pose1, ULX, ULY, DiffHeight, PoseNewOrigin1) *设置 PoseNewOrigin1的第二种方法 * create_pose (ULX, ULY, DiffHeight, 0, 0, 0, RpT, gba, point, newOriginInWorldPose) * pose_compose (Pose1, newOriginInWorldPose, PoseNewOrigin1) * * 图像到世界平面转换 - 生成一个投影图 * 该图描述了图像平面与世界坐标系中 z0 平面之间的映射关系。 gen_image_to_world_plane_map (MapSingle1, CamParam1, PoseNewOrigin1, Width, Height, WidthRect, HeightRect, PixelSize, bilinear) * * ---------------------------------------------------------------- * Merge right image in coordinate system of left image * ---------------------------------------------------------------- *第二幅图像必须校正使其正好适合第一幅校正图像的右侧。 *这意味着第二校正图像的左上角必须与第一校正图像的右上角相同。 *因此我们需要知道第一个校正图像的右上角在由第二个图像中的校准板定义的坐标系中的坐标。 *首先我们在世界坐标系中表示第一幅校正图像的右上角该坐标系由第一幅图像中的校准板定义。 *它可以通过从原点到第一校正图像的左上角的变换平移以及沿着第一校正图像的上边界的平移来确定。 *与补偿校准板厚度的位移一起该变换由齐次变换矩阵表示 hom_mat3d_identity (HomMat3DIdentity) * camera01视野右上角的偏移矩阵 * 注意与该偏移矩阵配合使用的是原点在中心的pose1而不是原点在视野有效区域左上角的PoseNewOrigin1 hom_mat3d_translate_local (HomMat3DIdentity, ULX PixelSize * WidthRect, ULY, DiffHeight, cp1Hur1) * camera01世界坐标系与camera02的世界坐标系关系矩阵 * DistancePlatescamera01 世界坐标系原点到 camera02 世界坐标系原点的距离 * 注意这里的原点是基于pose1原点在中心而不是视野有效区域左上角的PoseNewOrigin1 hom_mat3d_translate_local (HomMat3DIdentity, DistancePlates, 0, 0, cp1Hcp2) * camera01 世界坐标系在 camera02 世界坐标系的位姿 hom_mat3d_invert (cp1Hcp2, cp2Hcp1) * 因为camera01 视野的有效区域的右上角与camera02视野有效区域的左上角重合 * 所以矩阵合并 可获得camera01 视野的有效区域的右上角在camera02 世界坐标系的矩阵 * 即camera02视野有效区域的左上角在camera02 世界坐标系的矩阵 hom_mat3d_compose (cp2Hcp1, cp1Hur1, cp2Hul2) * camera2世界坐标系在camera2相机坐标系的矩阵 pose_to_hom_mat3d (Pose2, cam2Hcp2) hom_mat3d_compose (cam2Hcp2, cp2Hul2, cam2Hul2) * camera02视野有效区域的左上角即camer02世界坐标系原点在相机坐标系的位姿 hom_mat3d_to_pose (cam2Hul2, PoseNewOrigin2) *获取PoseNewOrigin2的第二种方法 *hom_mat3d_to_pose (cp1Hur1, Hur1Incp1Pose) *create_pose (-DistancePlates,0, 0, 0, 0, 0, RpT, gba, point, cp1Incp2Pose) *pose_compose ( cp1Incp2Pose, Hur1Incp1Pose, Hur1InCp2Pose) *pose_compose (Pose2, Hur1InCp2Pose, PoseNewOrigin2) * * * 该图描述了图像平面与世界坐标系中 z0 平面之间的映射关系。 gen_image_to_world_plane_map (MapSingle2, CamParam2, PoseNewOrigin2, Width, Height, WidthRect, HeightRect, PixelSize, bilinear) *示意图变换后PoseNewOrigin1、PoseNewOrigin2对应的世界坐标系4.3映射、拼接图像并显示。 * Window of merged image * dev_open_window (Height * WindowScale, 0, Width * 2 * WindowScale, Height * WindowScale, black, WindowHandleCombined) set_display_font (WindowHandleCombined, 16, mono, true, false) dev_set_color (green) dev_set_draw (margin) ScalePlot : 200 RowPlot : 400 Coord : [0:2000] * * * * * Start Measurement * * for I : 1 to 3 by 1 dev_set_window (WindowHandle1) read_image (Image1, ImgPath camera1_ I$02d) get_image_size (Image1, WidthImage1, HeightImage1) dev_set_part (0, 0, HeightImage1 - 1, WidthImage1 - 1) dev_display (Image1) dev_set_window (WindowHandle2) read_image (Image2, ImgPath camera2_ I$02d) get_image_size (Image2, WidthImage2, HeightImage2) dev_set_part (0, 0, HeightImage2 - 1, WidthImage2 - 1) dev_display (Image2) * Time measurement count_seconds (TimeStart1) * 对图像进行变换截取感兴趣的区域 map_image (Image1, MapSingle1, RectifiedImage1) map_image (Image2, MapSingle2, RectifiedImage2) concat_obj (RectifiedImage1, RectifiedImage2, Concat) count_seconds (TimeEnd1) Time1 : TimeEnd1 - TimeStart1 * dev_set_window (WindowHandleCombined) * Time measurement count_seconds (TimeStart2) * 图像拼接 tile_images (Concat, Combined, 2, vertical) count_seconds (TimeEnd2) Time2 : TimeEnd2 - TimeStart2 get_image_size (Combined, WidthComb, HeightComb) dev_set_part (0, 0, HeightComb - 1, WidthComb - 1) dev_display (Combined) disp_message (WindowHandle1, Merge cameras: (1000 * (Time1 Time2))$.3 ms, window, 12, 12, black, true) * plot_mosaicking_accuracy (Combined, WidthRect, HeightRect, WindowHandleCombined, Coord, ScalePlot, RowPlot) if (I 3) disp_continue_message (WindowHandleCombined, black, true) stop () endif endfor拼接效果如下5完整程序。* Example program for high precision mosaicking based on camera calibration dev_update_off () ImgPath : 3d_machine_vision/multiple_cameras/ * * * Window of left image and right image * dev_close_window () read_image (Image1, ImgPath camera1_ref) get_image_size (Image1, Width, Height) WindowScale : 0.66 dev_open_window (0, 0, Width * WindowScale, Height * WindowScale, black, WindowHandle1) dev_set_color (green) dev_set_draw (margin) dev_set_line_width (2) dev_open_window (0, Width * WindowScale 6, Width * WindowScale, Height * WindowScale, black, WindowHandle2) dev_set_color (green) dev_set_draw (margin) dev_set_line_width (2) dev_set_window (WindowHandle1) dev_set_part (0, 0, Height - 1, Width - 1) set_display_font (WindowHandle1, 16, mono, true, false) dev_set_window (WindowHandle2) dev_set_part (0, 0, Height - 1, Width - 1) set_display_font (WindowHandle2, 16, mono, true, false) * * * * * Calibration * * * ----------------------------- * Two caltab images * ----------------------------- * Assume that the two cameras are already calibrated (internal camera parameters) CamParam1 : [0.01619,-734.789,7.402e-006,7.4e-006,324.911,256.894,640,480] CamParam2 : [0.0162584,-763.35,7.39842e-006,7.4e-006,324.176,245.371,640,480] * read_image (Image1, ImgPath camera1_ref) read_image (Image2, ImgPath camera2_ref) dev_set_window (WindowHandle1) dev_display (Image1) dev_set_window (WindowHandle2) dev_display (Image2) * CaltabName : caltab_30mm.descr * 两个相机一个标定板 create_calib_data (calibration_object, 2, 1, CalibDataID) set_calib_data_calib_object (CalibDataID, 0, CaltabName) * dev_set_window (WindowHandle1) set_calib_data_cam_param (CalibDataID, 0, area_scan_division, CamParam1) find_calib_object (Image1, CalibDataID, 0, 0, 0, [], []) get_calib_data_observ_points (CalibDataID, 0, 0, 0, RCoord1, CCoord1, Index1, Pose1) get_calib_data_observ_contours (Caltab, CalibDataID, caltab, 0, 0, 0) dev_set_color (green) dev_display (Caltab) dev_set_colored (12) disp_3d_coord_system (WindowHandle1, CamParam1, Pose1, 0.01) * dev_set_window (WindowHandle2) dev_set_color (green) set_calib_data_cam_param (CalibDataID, 1, area_scan_division, CamParam2) find_calib_object (Image2, CalibDataID, 1, 0, 0, [], []) get_calib_data_observ_points (CalibDataID, 1, 0, 0, RCoord2, CCoord2, Index2, Pose2) get_calib_data_observ_contours (Caltab, CalibDataID, caltab, 1, 0, 0) dev_set_color (green) dev_display (Caltab) * dev_set_colored (12) disp_3d_coord_system (WindowHandle2, CamParam2, Pose2, 0.01) dev_set_color (green) disp_message (WindowHandle1, Calibration successful, window, 12, 12, black, true) disp_continue_message (WindowHandle1, black, true) stop () * * * Modify poses such that map_image creates images, * which can be merged in order to form the image mosaic * * * Determine the offset between the calibration plate surface and the object surface * 标定板从左相机视野水平移动右相机视野距离 DistancePlates : 0.06488 * 卡尺厚度 ThicknessCaliper : 2.9 / 1000.0 * 标定板厚度 ThicknessPlate : 5.65 / 1000.0 * 标定板与卡尺的厚度差异 DiffHeight : ThicknessPlate - ThicknessCaliper * * Define the pixel size of the mosaic image * 定义映射图的放大比例 PixelSize : 0.0001 * * Define the upper left corner of the mosaic image and the size of the two mapped images * BorderInPercent : 7 OverlapInPercent : 20 get_image_size (Image1, WidthImage1, HeightImage1) * 左图ROI左上角坐标 ULRow : HeightImage1 * BorderInPercent / 100.0 ULCol : WidthImage1 * BorderInPercent / 100.0 image_points_to_world_plane (CamParam1, Pose1, ULRow, ULCol, m, ULX, ULY) LowerRow : HeightImage1 * (100 - BorderInPercent) / 100.0 RightCol : WidthImage1 * (100 - OverlapInPercent / 2.0) / 100.0 * 左图ROI右下角映射 image_points_to_world_plane (CamParam1, Pose1, LowerRow, ULCol, m, X1, LowerY) * 左图ROI右上角映射 image_points_to_world_plane (CamParam1, Pose1, ULRow, RightCol, m, RightX, Y1) * 左图ROI大小PixelSize光学放大率决定了下面拼接图的映射分辨率 HeightRect : int((LowerY - ULY) / PixelSize) WidthRect : int((RightX - ULX) / PixelSize) * -------------------------------------------------------------------- * Translate pose of left camera into plane of measuerement * -------------------------------------------------------------------- * 设置原点DiffHeight标定板厚度-卡尺厚度意思将坐标系原点移至卡尺上平面 * 由前标定板标定的坐标系可知Z轴正向朝下 * DiffHeight : ThicknessPlate - ThicknessCaliper * ThicknessPlate:将原点沿标定板厚度移至标定板底面即平面 * - ThicknessCaliper移到平面后在上升移动到卡尺上面标定Z轴正向朝下 * 方法1 set_origin_pose (Pose1, ULX, ULY, DiffHeight, PoseNewOrigin1) * 方法2 create_pose (ULX, ULY, DiffHeight, 0, 0, 0, RpT, gba, point, OriginInWordPose) pose_compose (Pose1, OriginInWordPose, OriginInCamPose) gen_image_to_world_plane_map (MapSingle1, CamParam1, PoseNewOrigin1, Width, Height, WidthRect, HeightRect, PixelSize, bilinear) * * ---------------------------------------------------------------- * Merge right image in coordinate system of left image * ---------------------------------------------------------------- * 方法1计算计算右图ROI左上点在右相机的位姿 hom_mat3d_identity (HomMat3DIdentity) * 变换到左图ROI右上点世界长度 hom_mat3d_translate_local (HomMat3DIdentity, ULX PixelSize * WidthRect, ULY, DiffHeight, cp1Hur1) * 左图单位坐标系变换到右图单位坐标系cpCalibration Plate hom_mat3d_translate_local (HomMat3DIdentity, DistancePlates, 0, 0, cp1Hcp2) hom_mat3d_invert (cp1Hcp2, cp2Hcp1) * hom_mat3d_compose (cp2Hcp1, cp1Hur1, cp2Hul2) pose_to_hom_mat3d (Pose2, cam2Hcp2) * 左图右上点与右图左上点重合在右相机世界坐标系的位姿 hom_mat3d_compose (cam2Hcp2, cp2Hul2, cam2Hul2) hom_mat3d_to_pose (cam2Hul2, PoseNewOrigin2) * 方法2 计算右图ROI左上点在右相机的位姿 create_pose ( ULX PixelSize * WidthRect, ULY, DiffHeight, 0, 0, 0, RpT, gba, point, URInLeftRefPose) create_pose (-DistancePlates, 0, 0, 0, 0, 0, RpT, gba, point, LeftRefInRightRefPose) * 左图ROI的右上点与右图的ROI的左上点重合所以两者可认为是同一点 pose_compose (LeftRefInRightRefPose, URInLeftRefPose,ULInRightRefPose ) pose_compose (Pose2, ULInRightRefPose, ULInRightCamPose) gen_image_to_world_plane_map (MapSingle2, CamParam2, PoseNewOrigin2, Width, Height, WidthRect, HeightRect, PixelSize, bilinear) * * * * Window of merged image * dev_open_window (Height * WindowScale, 0, Width * 2 * WindowScale, Height * WindowScale, black, WindowHandleCombined) set_display_font (WindowHandleCombined, 16, mono, true, false) dev_set_color (green) dev_set_draw (margin) ScalePlot : 200 RowPlot : 400 Coord : [0:2000] * * * * * Start Measurement * * for I : 1 to 3 by 1 dev_set_window (WindowHandle1) read_image (Image1, ImgPath camera1_ I$02d) get_image_size (Image1, WidthImage1, HeightImage1) dev_set_part (0, 0, HeightImage1 - 1, WidthImage1 - 1) dev_display (Image1) dev_set_window (WindowHandle2) read_image (Image2, ImgPath camera2_ I$02d) get_image_size (Image2, WidthImage2, HeightImage2) dev_set_part (0, 0, HeightImage2 - 1, WidthImage2 - 1) dev_display (Image2) * Time measurement count_seconds (TimeStart1) map_image (Image1, MapSingle1, RectifiedImage1) map_image (Image2, MapSingle2, RectifiedImage2) concat_obj (RectifiedImage1, RectifiedImage2, Concat) count_seconds (TimeEnd1) Time1 : TimeEnd1 - TimeStart1 * dev_set_window (WindowHandleCombined) * Time measurement count_seconds (TimeStart2) tile_images (Concat, Combined, 2, vertical) count_seconds (TimeEnd2) Time2 : TimeEnd2 - TimeStart2 get_image_size (Combined, WidthComb, HeightComb) dev_set_part (0, 0, HeightComb - 1, WidthComb - 1) dev_display (Combined) disp_message (WindowHandle1, Merge cameras: (1000 * (Time1 Time2))$.3 ms, window, 12, 12, black, true) * plot_mosaicking_accuracy (Combined, WidthRect, HeightRect, WindowHandleCombined, Coord, ScalePlot, RowPlot) if (I 3) disp_continue_message (WindowHandleCombined, black, true) stop () endif endfor 双目视野高精度拼接
1目的。 使用多个相机采集图像经计算处理获得感兴趣区域的映射图将映射图拼接为一幅图像以达到扩大视野范围的目的。示意图变换后:2原理。 计算C1世界坐标系中感兴趣区域的右上角C1A的世界坐标将此坐标设为创建ImageMap所需的PoseNewOrigin1的原点。 计算出C1世界坐标系中感兴趣区域的右上角C1B基于世界坐标系默认原点C1Origin的偏移矩阵结合C1世界坐标系的默认原点C1Origin与C2世界坐标系的默认原点C2Origin存在偏移矩阵关系两个坐标系的原点存在固定的距离可计算出C1B相对C2世界坐标系默认原点C2Origin的变化矩阵。 因C1B与C2世界坐标系中感兴趣区域的左上角C2A重叠所以C1B相对C2世界坐标系默认原点C2Origin的偏移旋转矩阵可视为C2A相对C2世界坐标系默认原点C2Origin的偏移旋转矩阵。结合C2相机的位姿可计算出C2A在C2相机坐标系的位姿PoseNewOrigin2。 通过PoseNewOrigin1PoseNewOrigin2获得采集图像与感兴趣区域位置的映射关系map再通过map获得裁剪后的兴趣图像兴趣图像拼接后即为所需的大视野图像。在像素坐标系中原点位于图像的左上角。所以这里将原点重置到左上角便于后续生成原点位于左上角的映射图。注意该案例的标定模型使用的是两个相机一个标定板不是两个标定板不是两个标定板左相机的标定板水平向右移动DistancePlates成为右相机的标定板。2注意事项。进行变换时尽量使用矩阵而不是位姿位姿在变换时误差较大达不到预期效果。标定完成后默认标定板的几何中心或第一个特征点如棋盘格左上角为对应世界坐标系的原点。3主要的算子。3.1elliptic_axis1. 功能定义计算输入区域Region的等效椭圆参数包括Ra等效椭圆长轴半径主半径满足Ra ≥ Rb ≥ 0Rb等效椭圆短轴半径次半径Phi长轴与水平轴X轴的夹角弧度制范围-π/2 Phi ≤ π/22. 核心特性等效性生成的椭圆与输入区域具有相同的二阶矩惯性矩特性包括方向、长宽比和质心位置。计算基础基于区域的几何矩moments_region_2nd算子计算的M20、M02、M11推导出Ra、Rb和Phi特殊输入处理空区域或单点区域返回Ra Rb 0直线状区域如线段返回Rb 0Phi为线段方向角3. 应用场景方向检测通过Phi判断工业零件如PCB焊盘、金属条的旋转角度形状筛选结合select_shape算子通过anisometry即Ra/Rb过滤细长或圆形区域椭圆拟合验证与gen_ellipse联用生成等效椭圆可视化验证计算结果4. 注意事项角度一致性对于非规则形状如L型区域Phi与smallest_rectangle2算子的角度结果一致但等效椭圆面积通常大于原区域面积多区域输入支持区域数组输入输出参数为对应长度的数组XLD扩展针对轮廓点云XLD需使用elliptic_axis_points_xld算子其计算逻辑类似但基于点集矩3.2gen_image_to_world_plane_map图像映射算子用于建立图像平面与世界坐标系中z0平面之间的投影关系。通过相机内参和世界坐标系的3D位姿Pose计算图像像素到世界平面的单应性变换矩阵消除透视畸变和径向畸变模拟无畸变相机垂直拍摄的效果。1. 输入参数. CameraParam (相机内参矩阵)定义包含焦距(f)、主点坐标(cx,cy)、畸变系数(k1,k2,k3,p1,p2)等标定信息格式通常为数组形式例焦距,kappa,Sx,Sy,Cx,Cy,焦距,kappa,Sx,Sy,Cx,Cy,图像宽,图像高作用提供镜头畸变矫正所需的物理参数直接影响像素坐标系到世界坐标系的转换精度.WorldPose (世界坐标系位姿)定义描述目标平面(z0平面)在相机坐标系中的3D位置和方向构成通常包含旋转分量(Rx,Ry,Rz)和平移分量(Tx,Ty,Tz)使用四元数或旋转向量表示姿态作用确定世界平面与成像平面的空间几何关系用于消除透视畸变. WidthIn/HeightIn (输入图像尺寸)单位像素作用限定原始图像的物理分辨率范围确保映射矩阵与输入图像尺寸匹配2. 输出参数. Map (映射图像)数据类型多通道图像int4或int8格式内容存储每个输出像素对应的原始图像坐标偏移量用于后续map_image算子转换通道数单通道直接索引映射无插值双通道支持双线性插值.WidthMapped/HeightMapped (输出图像尺寸)单位像素设置依据根据目标平面实际物理尺寸和Scale参数计算得出. Scale (像素-世界单位比例)定义输出图像中单个像素对应的物理尺寸毫米/英寸等典型值当Scale1时1像素1世界单位常用于等比例映射场景3. 辅助参数.MapType (映射类型)可选值定义插值方式如’bilinear’或’nearest_neighbor’影响决定map_image算子执行时的采样策略和图像质量.参数关联性参数组关联性说明CameraParam ↔ Scale内参矩阵的焦距参数直接影响Scale的物理意义需保证单位统一性WorldPose ↔ Map位姿参数决定映射矩阵的几何变换类型如纯旋转、平移或复合变换Map ↔ map_image映射图像作为中间数据通过map_image实现实际图像转换4. 注意事项. CameraParam的完整性需确保包含所有必要畸变参数如径向畸变k1/k2/k3和切向畸变p1/p26.WorldPose的精度要求姿态误差会导致校正后的图像产生倾斜或比例失真推荐使用高精度标定方法26.Scale的合理性过大的Scale值可能导致输出图像分辨率不足过小则可能超出内存限制3.3tile_images是HALCON中用于平铺多图像的核心算子其参数体系直接影响图像拼接效果。以下为参数详解及关联分析输入参数Images (输入图像组)类型图像对象数组要求所有输入图像需具有相同通道数支持不同尺寸但会统一处理作用提供待拼接的原始图像数据图像顺序影响最终排列结构NumColumns (列数)类型整数定义指定输出图像中每行排列的子图列数控制横向排列密度动态调整若输入图像总数非列数整数倍末行可能留空或填充未定义区域TileOrder (排列顺序)可选值horizontal从左到右横向优先填充vertical从上到下纵向优先填充影响决定子图在输出画布上的布局方向适用于不同对比需求. 输出参数TiledImage (拼接图像)类型图像对象尺寸规则宽度NumColumns × 最大子图宽度高度ceil(总子图数/NumColumns) × 最大子图高度内容特性小尺寸子图居中显示边缘填充背景色默认黑色通道数与输入一致支持多通道拼接.参数耦合关系参数关联影响NumColumns ↔ 输出尺寸列数直接影响输出图像的宽度较大的列数会减少行数TileOrder ↔ 排列逻辑横向优先时按行填充纵向优先时按列填充影响视觉对比效果输入尺寸 ↔ 输出块尺寸所有子图统一为最大宽高可能导致小图边缘留空或大图裁剪. 注意事项通道一致性输入图像通道数必须相同否则触发运行时错误内存限制拼接大尺寸图像组时需评估输出画布尺寸避免内存溢出填充策略未占满区域显示为黑色若需自定义背景需结合其他算子预处理. 与变体算子对比相较于tile_images_offset的可控布局tile_images更适用于自动化排列场景典型差异包括tile_images_offset支持自定义子图位置和输出画布尺寸灵活性更高tile_images通过列数和排列顺序简化参数配置适合快速拼接对比4程序详解。4.1相机标定* 参考案例库two_camera_calibration.hdev * 基于相机标定的高精度拼接示例程序 dev_update_off () ImgPath : 3d_machine_vision/multiple_cameras/ * * * Window of left image and right image * dev_close_window () read_image (Image1, ImgPath camera1_ref) get_image_size (Image1, Width, Height) WindowScale : 0.66 dev_open_window (0, 0, Width * WindowScale, Height * WindowScale, black, WindowHandle1) dev_set_color (green) dev_set_draw (margin) dev_set_line_width (2) dev_open_window (0, Width * WindowScale 6, Width * WindowScale, Height * WindowScale, black, WindowHandle2) dev_set_color (green) dev_set_draw (margin) dev_set_line_width (2) dev_set_window (WindowHandle1) dev_set_part (0, 0, Height - 1, Width - 1) set_display_font (WindowHandle1, 16, mono, true, false) dev_set_window (WindowHandle2) dev_set_part (0, 0, Height - 1, Width - 1) set_display_font (WindowHandle2, 16, mono, true, false) * * * * * Calibration * * * ----------------------------- * Two caltab images * ----------------------------- * Assume that the two cameras are already calibrated (internal camera parameters) * 相机1标定后内参 CamParam1 : [0.01619,-734.789,7.402e-006,7.4e-006,324.911,256.894,640,480] * 相机2标定后的内参 CamParam2 : [0.0162584,-763.35,7.39842e-006,7.4e-006,324.176,245.371,640,480] * read_image (Image1, ImgPath camera1_ref) read_image (Image2, ImgPath camera2_ref) dev_set_window (WindowHandle1) dev_display (Image1) dev_set_window (WindowHandle2) dev_display (Image2) * CaltabName : caltab_30mm.descr create_calib_data (calibration_object, 2, 1, CalibDataID) set_calib_data_calib_object (CalibDataID, 0, CaltabName) * dev_set_window (WindowHandle1) set_calib_data_cam_param (CalibDataID, 0, area_scan_division, CamParam1) find_calib_object (Image1, CalibDataID, 0, 0, 0, [], []) get_calib_data_observ_points (CalibDataID, 0, 0, 0, RCoord1, CCoord1, Index1, Pose1) get_calib_data_observ_contours (Caltab, CalibDataID, caltab, 0, 0, 0) dev_set_color (green) dev_display (Caltab) dev_set_window (WindowHandle2) set_calib_data_cam_param (CalibDataID, 1, area_scan_division, CamParam2) find_calib_object (Image2, CalibDataID, 1, 0, 0, [], []) get_calib_data_observ_points (CalibDataID, 1, 0, 0, RCoord2, CCoord2, Index2, Pose2) get_calib_data_observ_contours (Caltab, CalibDataID, caltab, 1, 0, 0) dev_set_color (green) dev_display (Caltab) * * 显示camera1世界坐标系 dev_set_colored (12) disp_3d_coord_system (WindowHandle1, CamParam1, Pose1, 0.01) * 显示camera2的世界坐标系 disp_3d_coord_system (WindowHandle2, CamParam2, Pose2, 0.01) disp_message (WindowHandle1, Calibration successful, window, 12, 12, black, true) disp_continue_message (WindowHandle1, black, true) stop ()真实物理世界中物体的三维位置原点通常定义在标定板的几何中心或第一个特征点如棋盘格左上角如下所示标定板使用过程首先找到标定板在左相机的姿态。其次向右水平移动DistancePlates距离至右相机视野。最后查找标定板在右相机坐标系的位姿。可知标定板的中心即为世界坐标系的默认原点camera01的世界坐标系原点水平偏移DistancePlates即可变换为camera02的世界坐标系原点如下所示4.2 修改poses 创建 map_image* * Modify poses such that map_image creates images, * which can be merged in order to form the image mosaic * * * Determine the offset between the calibration plate surface and the object surface * 两个标定板坐标系统原点之间的距离单位m,如4.1中的插图所示 DistancePlates : 0.06488 ThicknessCaliper : 2.9 / 1000.0 ThicknessPlate : 5.65 / 1000.0 * 标定板厚度 DiffHeight : ThicknessPlate - ThicknessCaliper * * 分辨率确定映射后图像的大小 * PixelSize : 0.0001 * * Define the upper left corner of the mosaic image and the size of the two mapped images * 视野边框所占比例 BorderInPercent : 7 * 左右相机视野重叠比例 OverlapInPercent : 20 get_image_size (Image1, WidthImage1, HeightImage1) * ROI左上角坐标 ULRow : HeightImage1 * BorderInPercent / 100.0 ULCol : WidthImage1 * BorderInPercent / 100.0 * Camera01视野ROI左上角坐标世界坐标系 image_points_to_world_plane (CamParam1, Pose1, ULRow, ULCol, m, ULX, ULY) LowerRow : HeightImage1 * (100 - BorderInPercent) / 100.0 RightCol : WidthImage1 * (100 - OverlapInPercent / 2.0) / 100.0 * Camera01视野ROI左下角世界坐标系 image_points_to_world_plane (CamParam1, Pose1, LowerRow, ULCol, m, X1, LowerY) * Camera01视野ROI右上角世界坐标系 image_points_to_world_plane (CamParam1, Pose1, ULRow, RightCol, m, RightX, Y1) * Camera01视野ROI有效范围 image_points_to_world_plane (CamParam1, Pose1, LowerRow, RightCol, m, X, Y) HeightRect : int((LowerY - ULY) / PixelSize) WidthRect : int((RightX - ULX) / PixelSize) * -------------------------------------------------------------------- * Translate pose of left camera into plane of measuerement * -------------------------------------------------------------------- * 设置Camera01世界坐标系的原点 set_origin_pose (Pose1, ULX, ULY, DiffHeight, PoseNewOrigin1) *设置 PoseNewOrigin1的第二种方法 * create_pose (ULX, ULY, DiffHeight, 0, 0, 0, RpT, gba, point, newOriginInWorldPose) * pose_compose (Pose1, newOriginInWorldPose, PoseNewOrigin1) * * 图像到世界平面转换 - 生成一个投影图 * 该图描述了图像平面与世界坐标系中 z0 平面之间的映射关系。 gen_image_to_world_plane_map (MapSingle1, CamParam1, PoseNewOrigin1, Width, Height, WidthRect, HeightRect, PixelSize, bilinear) * * ---------------------------------------------------------------- * Merge right image in coordinate system of left image * ---------------------------------------------------------------- *第二幅图像必须校正使其正好适合第一幅校正图像的右侧。 *这意味着第二校正图像的左上角必须与第一校正图像的右上角相同。 *因此我们需要知道第一个校正图像的右上角在由第二个图像中的校准板定义的坐标系中的坐标。 *首先我们在世界坐标系中表示第一幅校正图像的右上角该坐标系由第一幅图像中的校准板定义。 *它可以通过从原点到第一校正图像的左上角的变换平移以及沿着第一校正图像的上边界的平移来确定。 *与补偿校准板厚度的位移一起该变换由齐次变换矩阵表示 hom_mat3d_identity (HomMat3DIdentity) * camera01视野右上角的偏移矩阵 * 注意与该偏移矩阵配合使用的是原点在中心的pose1而不是原点在视野有效区域左上角的PoseNewOrigin1 hom_mat3d_translate_local (HomMat3DIdentity, ULX PixelSize * WidthRect, ULY, DiffHeight, cp1Hur1) * camera01世界坐标系与camera02的世界坐标系关系矩阵 * DistancePlatescamera01 世界坐标系原点到 camera02 世界坐标系原点的距离 * 注意这里的原点是基于pose1原点在中心而不是视野有效区域左上角的PoseNewOrigin1 hom_mat3d_translate_local (HomMat3DIdentity, DistancePlates, 0, 0, cp1Hcp2) * camera01 世界坐标系在 camera02 世界坐标系的位姿 hom_mat3d_invert (cp1Hcp2, cp2Hcp1) * 因为camera01 视野的有效区域的右上角与camera02视野有效区域的左上角重合 * 所以矩阵合并 可获得camera01 视野的有效区域的右上角在camera02 世界坐标系的矩阵 * 即camera02视野有效区域的左上角在camera02 世界坐标系的矩阵 hom_mat3d_compose (cp2Hcp1, cp1Hur1, cp2Hul2) * camera2世界坐标系在camera2相机坐标系的矩阵 pose_to_hom_mat3d (Pose2, cam2Hcp2) hom_mat3d_compose (cam2Hcp2, cp2Hul2, cam2Hul2) * camera02视野有效区域的左上角即camer02世界坐标系原点在相机坐标系的位姿 hom_mat3d_to_pose (cam2Hul2, PoseNewOrigin2) *获取PoseNewOrigin2的第二种方法 *hom_mat3d_to_pose (cp1Hur1, Hur1Incp1Pose) *create_pose (-DistancePlates,0, 0, 0, 0, 0, RpT, gba, point, cp1Incp2Pose) *pose_compose ( cp1Incp2Pose, Hur1Incp1Pose, Hur1InCp2Pose) *pose_compose (Pose2, Hur1InCp2Pose, PoseNewOrigin2) * * * 该图描述了图像平面与世界坐标系中 z0 平面之间的映射关系。 gen_image_to_world_plane_map (MapSingle2, CamParam2, PoseNewOrigin2, Width, Height, WidthRect, HeightRect, PixelSize, bilinear) *示意图变换后PoseNewOrigin1、PoseNewOrigin2对应的世界坐标系4.3映射、拼接图像并显示。 * Window of merged image * dev_open_window (Height * WindowScale, 0, Width * 2 * WindowScale, Height * WindowScale, black, WindowHandleCombined) set_display_font (WindowHandleCombined, 16, mono, true, false) dev_set_color (green) dev_set_draw (margin) ScalePlot : 200 RowPlot : 400 Coord : [0:2000] * * * * * Start Measurement * * for I : 1 to 3 by 1 dev_set_window (WindowHandle1) read_image (Image1, ImgPath camera1_ I$02d) get_image_size (Image1, WidthImage1, HeightImage1) dev_set_part (0, 0, HeightImage1 - 1, WidthImage1 - 1) dev_display (Image1) dev_set_window (WindowHandle2) read_image (Image2, ImgPath camera2_ I$02d) get_image_size (Image2, WidthImage2, HeightImage2) dev_set_part (0, 0, HeightImage2 - 1, WidthImage2 - 1) dev_display (Image2) * Time measurement count_seconds (TimeStart1) * 对图像进行变换截取感兴趣的区域 map_image (Image1, MapSingle1, RectifiedImage1) map_image (Image2, MapSingle2, RectifiedImage2) concat_obj (RectifiedImage1, RectifiedImage2, Concat) count_seconds (TimeEnd1) Time1 : TimeEnd1 - TimeStart1 * dev_set_window (WindowHandleCombined) * Time measurement count_seconds (TimeStart2) * 图像拼接 tile_images (Concat, Combined, 2, vertical) count_seconds (TimeEnd2) Time2 : TimeEnd2 - TimeStart2 get_image_size (Combined, WidthComb, HeightComb) dev_set_part (0, 0, HeightComb - 1, WidthComb - 1) dev_display (Combined) disp_message (WindowHandle1, Merge cameras: (1000 * (Time1 Time2))$.3 ms, window, 12, 12, black, true) * plot_mosaicking_accuracy (Combined, WidthRect, HeightRect, WindowHandleCombined, Coord, ScalePlot, RowPlot) if (I 3) disp_continue_message (WindowHandleCombined, black, true) stop () endif endfor拼接效果如下5完整程序。* Example program for high precision mosaicking based on camera calibration dev_update_off () ImgPath : 3d_machine_vision/multiple_cameras/ * * * Window of left image and right image * dev_close_window () read_image (Image1, ImgPath camera1_ref) get_image_size (Image1, Width, Height) WindowScale : 0.66 dev_open_window (0, 0, Width * WindowScale, Height * WindowScale, black, WindowHandle1) dev_set_color (green) dev_set_draw (margin) dev_set_line_width (2) dev_open_window (0, Width * WindowScale 6, Width * WindowScale, Height * WindowScale, black, WindowHandle2) dev_set_color (green) dev_set_draw (margin) dev_set_line_width (2) dev_set_window (WindowHandle1) dev_set_part (0, 0, Height - 1, Width - 1) set_display_font (WindowHandle1, 16, mono, true, false) dev_set_window (WindowHandle2) dev_set_part (0, 0, Height - 1, Width - 1) set_display_font (WindowHandle2, 16, mono, true, false) * * * * * Calibration * * * ----------------------------- * Two caltab images * ----------------------------- * Assume that the two cameras are already calibrated (internal camera parameters) CamParam1 : [0.01619,-734.789,7.402e-006,7.4e-006,324.911,256.894,640,480] CamParam2 : [0.0162584,-763.35,7.39842e-006,7.4e-006,324.176,245.371,640,480] * read_image (Image1, ImgPath camera1_ref) read_image (Image2, ImgPath camera2_ref) dev_set_window (WindowHandle1) dev_display (Image1) dev_set_window (WindowHandle2) dev_display (Image2) * CaltabName : caltab_30mm.descr * 两个相机一个标定板 create_calib_data (calibration_object, 2, 1, CalibDataID) set_calib_data_calib_object (CalibDataID, 0, CaltabName) * dev_set_window (WindowHandle1) set_calib_data_cam_param (CalibDataID, 0, area_scan_division, CamParam1) find_calib_object (Image1, CalibDataID, 0, 0, 0, [], []) get_calib_data_observ_points (CalibDataID, 0, 0, 0, RCoord1, CCoord1, Index1, Pose1) get_calib_data_observ_contours (Caltab, CalibDataID, caltab, 0, 0, 0) dev_set_color (green) dev_display (Caltab) dev_set_colored (12) disp_3d_coord_system (WindowHandle1, CamParam1, Pose1, 0.01) * dev_set_window (WindowHandle2) dev_set_color (green) set_calib_data_cam_param (CalibDataID, 1, area_scan_division, CamParam2) find_calib_object (Image2, CalibDataID, 1, 0, 0, [], []) get_calib_data_observ_points (CalibDataID, 1, 0, 0, RCoord2, CCoord2, Index2, Pose2) get_calib_data_observ_contours (Caltab, CalibDataID, caltab, 1, 0, 0) dev_set_color (green) dev_display (Caltab) * dev_set_colored (12) disp_3d_coord_system (WindowHandle2, CamParam2, Pose2, 0.01) dev_set_color (green) disp_message (WindowHandle1, Calibration successful, window, 12, 12, black, true) disp_continue_message (WindowHandle1, black, true) stop () * * * Modify poses such that map_image creates images, * which can be merged in order to form the image mosaic * * * Determine the offset between the calibration plate surface and the object surface * 标定板从左相机视野水平移动右相机视野距离 DistancePlates : 0.06488 * 卡尺厚度 ThicknessCaliper : 2.9 / 1000.0 * 标定板厚度 ThicknessPlate : 5.65 / 1000.0 * 标定板与卡尺的厚度差异 DiffHeight : ThicknessPlate - ThicknessCaliper * * Define the pixel size of the mosaic image * 定义映射图的放大比例 PixelSize : 0.0001 * * Define the upper left corner of the mosaic image and the size of the two mapped images * BorderInPercent : 7 OverlapInPercent : 20 get_image_size (Image1, WidthImage1, HeightImage1) * 左图ROI左上角坐标 ULRow : HeightImage1 * BorderInPercent / 100.0 ULCol : WidthImage1 * BorderInPercent / 100.0 image_points_to_world_plane (CamParam1, Pose1, ULRow, ULCol, m, ULX, ULY) LowerRow : HeightImage1 * (100 - BorderInPercent) / 100.0 RightCol : WidthImage1 * (100 - OverlapInPercent / 2.0) / 100.0 * 左图ROI右下角映射 image_points_to_world_plane (CamParam1, Pose1, LowerRow, ULCol, m, X1, LowerY) * 左图ROI右上角映射 image_points_to_world_plane (CamParam1, Pose1, ULRow, RightCol, m, RightX, Y1) * 左图ROI大小PixelSize光学放大率决定了下面拼接图的映射分辨率 HeightRect : int((LowerY - ULY) / PixelSize) WidthRect : int((RightX - ULX) / PixelSize) * -------------------------------------------------------------------- * Translate pose of left camera into plane of measuerement * -------------------------------------------------------------------- * 设置原点DiffHeight标定板厚度-卡尺厚度意思将坐标系原点移至卡尺上平面 * 由前标定板标定的坐标系可知Z轴正向朝下 * DiffHeight : ThicknessPlate - ThicknessCaliper * ThicknessPlate:将原点沿标定板厚度移至标定板底面即平面 * - ThicknessCaliper移到平面后在上升移动到卡尺上面标定Z轴正向朝下 * 方法1 set_origin_pose (Pose1, ULX, ULY, DiffHeight, PoseNewOrigin1) * 方法2 create_pose (ULX, ULY, DiffHeight, 0, 0, 0, RpT, gba, point, OriginInWordPose) pose_compose (Pose1, OriginInWordPose, OriginInCamPose) gen_image_to_world_plane_map (MapSingle1, CamParam1, PoseNewOrigin1, Width, Height, WidthRect, HeightRect, PixelSize, bilinear) * * ---------------------------------------------------------------- * Merge right image in coordinate system of left image * ---------------------------------------------------------------- * 方法1计算计算右图ROI左上点在右相机的位姿 hom_mat3d_identity (HomMat3DIdentity) * 变换到左图ROI右上点世界长度 hom_mat3d_translate_local (HomMat3DIdentity, ULX PixelSize * WidthRect, ULY, DiffHeight, cp1Hur1) * 左图单位坐标系变换到右图单位坐标系cpCalibration Plate hom_mat3d_translate_local (HomMat3DIdentity, DistancePlates, 0, 0, cp1Hcp2) hom_mat3d_invert (cp1Hcp2, cp2Hcp1) * hom_mat3d_compose (cp2Hcp1, cp1Hur1, cp2Hul2) pose_to_hom_mat3d (Pose2, cam2Hcp2) * 左图右上点与右图左上点重合在右相机世界坐标系的位姿 hom_mat3d_compose (cam2Hcp2, cp2Hul2, cam2Hul2) hom_mat3d_to_pose (cam2Hul2, PoseNewOrigin2) * 方法2 计算右图ROI左上点在右相机的位姿 create_pose ( ULX PixelSize * WidthRect, ULY, DiffHeight, 0, 0, 0, RpT, gba, point, URInLeftRefPose) create_pose (-DistancePlates, 0, 0, 0, 0, 0, RpT, gba, point, LeftRefInRightRefPose) * 左图ROI的右上点与右图的ROI的左上点重合所以两者可认为是同一点 pose_compose (LeftRefInRightRefPose, URInLeftRefPose,ULInRightRefPose ) pose_compose (Pose2, ULInRightRefPose, ULInRightCamPose) gen_image_to_world_plane_map (MapSingle2, CamParam2, PoseNewOrigin2, Width, Height, WidthRect, HeightRect, PixelSize, bilinear) * * * * Window of merged image * dev_open_window (Height * WindowScale, 0, Width * 2 * WindowScale, Height * WindowScale, black, WindowHandleCombined) set_display_font (WindowHandleCombined, 16, mono, true, false) dev_set_color (green) dev_set_draw (margin) ScalePlot : 200 RowPlot : 400 Coord : [0:2000] * * * * * Start Measurement * * for I : 1 to 3 by 1 dev_set_window (WindowHandle1) read_image (Image1, ImgPath camera1_ I$02d) get_image_size (Image1, WidthImage1, HeightImage1) dev_set_part (0, 0, HeightImage1 - 1, WidthImage1 - 1) dev_display (Image1) dev_set_window (WindowHandle2) read_image (Image2, ImgPath camera2_ I$02d) get_image_size (Image2, WidthImage2, HeightImage2) dev_set_part (0, 0, HeightImage2 - 1, WidthImage2 - 1) dev_display (Image2) * Time measurement count_seconds (TimeStart1) map_image (Image1, MapSingle1, RectifiedImage1) map_image (Image2, MapSingle2, RectifiedImage2) concat_obj (RectifiedImage1, RectifiedImage2, Concat) count_seconds (TimeEnd1) Time1 : TimeEnd1 - TimeStart1 * dev_set_window (WindowHandleCombined) * Time measurement count_seconds (TimeStart2) tile_images (Concat, Combined, 2, vertical) count_seconds (TimeEnd2) Time2 : TimeEnd2 - TimeStart2 get_image_size (Combined, WidthComb, HeightComb) dev_set_part (0, 0, HeightComb - 1, WidthComb - 1) dev_display (Combined) disp_message (WindowHandle1, Merge cameras: (1000 * (Time1 Time2))$.3 ms, window, 12, 12, black, true) * plot_mosaicking_accuracy (Combined, WidthRect, HeightRect, WindowHandleCombined, Coord, ScalePlot, RowPlot) if (I 3) disp_continue_message (WindowHandleCombined, black, true) stop () endif endfor