导入模型文件到robosuite的Demo场景，形成自己的场景

找到需要的模型文件格式如YCB_Dataset/ycb/lemon at main · elpis-lab/YCB_Dataset可使用https://github.com/elpis-lab/YCB_Dataset/tree/main/ycb/lemon复制网址下载github单个文件夹。xml存放路径大致如下/home/用户名/anaconda3/envs/robosuite/lib/python3.10/site-packages/robosuite/models/assets/objects将下载的文件夹解压并放入上述文件夹中并新建一个.xml文本xml文件如下自行替换路径这里需要根据文件夹中的textured.obj文件来自行替换下面的原点、重心等细节信息出现模型一碰就飞或者一碰就翻面等情况大都考虑重心设置问题。?xml version1.0 ? mujoco modelb_cup asset texture namecup_tex type2d fileb_cup/texture_map.png/ material namecup_mat texturecup_tex specular0.3 shininess0.2/ mesh namecup_visual_mesh fileb_cup/textured.obj/ mesh namecup_col_mesh_0 fileb_cup/textured_coacd_0.stl/ mesh namecup_col_mesh_1 fileb_cup/textured_coacd_1.stl/ mesh namecup_col_mesh_2 fileb_cup/textured_coacd_2.stl/ mesh namecup_col_mesh_3 fileb_cup/textured_coacd_3.stl/ mesh namecup_col_mesh_4 fileb_cup/textured_coacd_4.stl/ /asset worldbody body body nameobject inertial pos0 0 0.0 mass0.15 diaginertia0.0001 0.0001 0.0001/ geom typemesh meshcup_visual_mesh materialcup_mat contype0 conaffinity0 group1/ geom typemesh meshcup_col_mesh_0 rgba0 1 0 0 contype1 conaffinity1 group0/ geom typemesh meshcup_col_mesh_1 rgba0 1 0 0 contype1 conaffinity1 group0/ geom typemesh meshcup_col_mesh_2 rgba0 1 0 0 contype1 conaffinity1 group0/ geom typemesh meshcup_col_mesh_3 rgba0 1 0 0 contype1 conaffinity1 group0/ geom typemesh meshcup_col_mesh_4 rgba0 1 0 0 contype1 conaffinity1 group0/ /body site nameobject_site pos0 0 0.05 size0.005 rgba1 0 0 1 group2/ site rgba0 0 0 0 size0.005 pos0 0 0.0 namebottom_site/ site rgba0 0 0 0 size0.005 pos0 0 0.1 nametop_site/ site rgba0 0 0 0 size0.005 pos0.04 0.04 0.05 namehorizontal_radius_site/ /body /worldbody /mujoco场景路径大致如下/home/用户名/anaconda3/envs/robosuite/lib/python3.10/site-packages/robosuite/environments/manipulation以Lift场景为例复制一份并修改后的Lift2.py代码如下主要添加开头的类、装饰器、def _load_model和def _setup_references里的定义加载和打包打包后self.objects对原先参数的替换以及def _setup_observables中新增传感器和夹爪距离。from collections import OrderedDict import numpy as np from robosuite.environments.manipulation.manipulation_env import ManipulationEnv from robosuite.models.arenas import TableArena from robosuite.models.objects import BoxObject, MujocoXMLObject from robosuite.models.tasks import ManipulationTask from robosuite.utils.mjcf_utils import CustomMaterial, xml_path_completion from robosuite.utils.observables import Observable, sensor from robosuite.utils.placement_samplers import UniformRandomSampler from robosuite.utils.transform_utils import convert_quat from robosuite.environments.base import register_env # ----------------- 【步骤 A声明 YCB 杯子的 Python 类】 ----------------- class YCBCupObject(MujocoXMLObject): def __init__(self, name): super().__init__( xml_path_completion(objects/b_cup.xml), # 指向你的 cup.xml namename, joints[{type: free, name: object_joint}], obj_typeall, ) register_env # 关键加上这个装饰器系统会自动把它吃进 REGISTERED_ENVS 字典中 class Lift2(ManipulationEnv): This class corresponds to the lifting task for a single robot arm. Args: robots (str or list of str): Specification for specific robot arm(s) to be instantiated within this env (e.g: Sawyer would generate one arm; [Panda, Panda, Sawyer] would generate three robot arms) Note: Must be a single single-arm robot! env_configuration (str): Specifies how to position the robots within the environment (default is default). For most single arm environments, this argument has no impact on the robot setup. controller_configs (str or list of dict): If set, contains relevant controller parameters for creating a custom controller. Else, uses the default controller for this specific task. Should either be single dict if same controller is to be used for all robots or else it should be a list of the same length as robots param gripper_types (str or list of str): type of gripper, used to instantiate gripper models from gripper factory. Default is default, which is the default grippers(s) associated with the robot(s) the robots specification. None removes the gripper, and any other (valid) model overrides the default gripper. Should either be single str if same gripper type is to be used for all robots or else it should be a list of the same length as robots param base_types (None or str or list of str): type of base, used to instantiate base models from base factory. Default is default, which is the default base associated with the robot(s) the robots specification. None results in no base, and any other (valid) model overrides the default base. Should either be single str if same base type is to be used for all robots or else it should be a list of the same length as robots param initialization_noise (dict or list of dict): Dict containing the initialization noise parameters. The expected keys and corresponding value types are specified below: :magnitude: The scale factor of uni-variate random noise applied to each of a robots given initial joint positions. Setting this value to None or 0.0 results in no noise being applied. If gaussian type of noise is applied then this magnitude scales the standard deviation applied, If uniform type of noise is applied then this magnitude sets the bounds of the sampling range :type: Type of noise to apply. Can either specify gaussian or uniform Should either be single dict if same noise value is to be used for all robots or else it should be a list of the same length as robots param :Note: Specifying default will automatically use the default noise settings. Specifying None will automatically create the required dict with magnitude set to 0.0. table_full_size (3-tuple): x, y, and z dimensions of the table. table_friction (3-tuple): the three mujoco friction parameters for the table. use_camera_obs (bool): if True, every observation includes rendered image(s) use_object_obs (bool): if True, include object (cube) information in the observation. reward_scale (None or float): Scales the normalized reward function by the amount specified. If None, environment reward remains unnormalized reward_shaping (bool): if True, use dense rewards. placement_initializer (ObjectPositionSampler): if provided, will be used to place objects on every reset, else a UniformRandomSampler is used by default. has_renderer (bool): If true, render the simulation state in a viewer instead of headless mode. has_offscreen_renderer (bool): True if using off-screen rendering render_camera (str): Name of camera to render if has_renderer is True. Setting this value to None will result in the default angle being applied, which is useful as it can be dragged / panned by the user using the mouse render_collision_mesh (bool): True if rendering collision meshes in camera. False otherwise. render_visual_mesh (bool): True if rendering visual meshes in camera. False otherwise. render_gpu_device_id (int): corresponds to the GPU device id to use for offscreen rendering. Defaults to -1, in which case the device will be inferred from environment variables (GPUS or CUDA_VISIBLE_DEVICES). control_freq (float): how many control signals to receive in every second. This sets the amount of simulation time that passes between every action input. lite_physics (bool): Whether to optimize for mujoco forward and step calls to reduce total simulation overhead. Set to False to preserve backward compatibility with datasets collected in robosuite 1.4.1. horizon (int): Every episode lasts for exactly horizon timesteps. ignore_done (bool): True if never terminating the environment (ignore horizon). hard_reset (bool): If True, re-loads model, sim, and render object upon a reset call, else, only calls sim.reset and resets all robosuite-internal variables camera_names (str or list of str): name of camera to be rendered. Should either be single str if same name is to be used for all cameras rendering or else it should be a list of cameras to render. :Note: At least one camera must be specified if use_camera_obs is True. :Note: To render all robots cameras of a certain type (e.g.: robotview or eye_in_hand), use the convention all-{name} (e.g.: all-robotview) to automatically render all camera images from each robots camera list). camera_heights (int or list of int): height of camera frame. Should either be single int if same height is to be used for all cameras frames or else it should be a list of the same length as camera names param. camera_widths (int or list of int): width of camera frame. Should either be single int if same width is to be used for all cameras frames or else it should be a list of the same length as camera names param. camera_depths (bool or list of bool): True if rendering RGB-D, and RGB otherwise. Should either be single bool if same depth setting is to be used for all cameras or else it should be a list of the same length as camera names param. camera_segmentations (None or str or list of str or list of list of str): Camera segmentation(s) to use for each camera. Valid options are: None: no segmentation sensor used instance: segmentation at the class-instance level class: segmentation at the class level element: segmentation at the per-geom level If not None, multiple types of segmentations can be specified. A [list of str / str or None] specifies [multiple / a single] segmentation(s) to use for all cameras. A list of list of str specifies per-camera segmentation setting(s) to use. Raises: AssertionError: [Invalid number of robots specified] def __init__( self, robots, env_configurationdefault, controller_configsNone, gripper_typesdefault, base_typesdefault, initialization_noisedefault, table_full_size(0.8, 0.8, 0.05), table_friction(1.0, 5e-3, 1e-4), use_camera_obsTrue, use_object_obsTrue, reward_scale1.0, reward_shapingFalse, placement_initializerNone, has_rendererFalse, has_offscreen_rendererTrue, render_camerafrontview, render_collision_meshFalse, render_visual_meshTrue, render_gpu_device_id-1, control_freq20, lite_physicsTrue, horizon1000, ignore_doneFalse, hard_resetTrue, camera_namesagentview, camera_heights256, camera_widths256, camera_depthsFalse, camera_segmentationsNone, # {None, instance, class, element} renderermjviewer, renderer_configNone, seedNone, ): # settings for table top self.table_full_size table_full_size self.table_friction table_friction self.table_offset np.array((0, 0, 0.8)) # reward configuration self.reward_scale reward_scale self.reward_shaping reward_shaping # whether to use ground-truth object states self.use_object_obs use_object_obs # object placement initializer self.placement_initializer placement_initializer super().__init__( robotsrobots, env_configurationenv_configuration, controller_configscontroller_configs, base_typesdefault, gripper_typesgripper_types, initialization_noiseinitialization_noise, use_camera_obsuse_camera_obs, has_rendererhas_renderer, has_offscreen_rendererhas_offscreen_renderer, render_camerarender_camera, render_collision_meshrender_collision_mesh, render_visual_meshrender_visual_mesh, render_gpu_device_idrender_gpu_device_id, control_freqcontrol_freq, lite_physicslite_physics, horizonhorizon, ignore_doneignore_done, hard_resethard_reset, camera_namescamera_names, camera_heightscamera_heights, camera_widthscamera_widths, camera_depthscamera_depths, camera_segmentationscamera_segmentations, rendererrenderer, renderer_configrenderer_config, seedseed, ) def reward(self, actionNone): Reward function for the task. Sparse un-normalized reward: - a discrete reward of 2.25 is provided if the cube is lifted Un-normalized summed components if using reward shaping: - Reaching: in [0, 1], to encourage the arm to reach the cube - Grasping: in {0, 0.25}, non-zero if arm is grasping the cube - Lifting: in {0, 1}, non-zero if arm has lifted the cube The sparse reward only consists of the lifting component. Note that the final reward is normalized and scaled by reward_scale / 2.25 as well so that the max score is equal to reward_scale Args: action (np array): [NOT USED] Returns: float: reward value reward 0.0 # sparse completion reward if self._check_success(): reward 2.25 # use a shaping reward elif self.reward_shaping: # reaching reward dist self._gripper_to_target( gripperself.robots[0].gripper, targetself.cube.root_body, target_typebody, return_distanceTrue ) reaching_reward 1 - np.tanh(10.0 * dist) reward reaching_reward # grasping reward if self._check_grasp(gripperself.robots[0].gripper, object_geomsself.cube): reward 0.25 # Scale reward if requested if self.reward_scale is not None: reward * self.reward_scale / 2.25 return reward def _load_model(self): Loads an xml model, puts it in self.model super()._load_model() # Adjust base pose accordingly xpos self.robots[0].robot_model.base_xpos_offset[table](self.table_full_size[0]) self.robots[0].robot_model.set_base_xpos(xpos) # load model for table top workspace mujoco_arena TableArena( table_full_sizeself.table_full_size, table_frictionself.table_friction, table_offsetself.table_offset, ) # Arena always gets set to zero origin mujoco_arena.set_origin([0, 0, 0]) # initialize objects of interest tex_attrib { type: cube, } mat_attrib { texrepeat: 1 1, specular: 0.4, shininess: 0.1, } redwood CustomMaterial( textureWoodRed, tex_nameredwood, mat_nameredwood_mat, tex_attribtex_attrib, mat_attribmat_attrib, ) self.cube BoxObject( namecube, size_min[0.020, 0.020, 0.020], # [0.015, 0.015, 0.015], size_max[0.022, 0.022, 0.022], # [0.018, 0.018, 0.018]) rgba[1, 0, 0, 1], materialredwood, rngself.rng, ) # 2. 新增加载自定义的 YCB 杯子 self.cup YCBCupObject(nameb_cup) # 3. 将两个物体打包成一个列表放入采样器让它们在桌面上随机排布 self.objects [self.cube, self.cup] # Create placement initializer if self.placement_initializer is not None: self.placement_initializer.reset() self.placement_initializer.add_objects(self.objects) else: self.placement_initializer UniformRandomSampler( nameObjectSampler, mujoco_objectsself.objects, x_range[-0.10, 0.10], y_range[-0.10, 0.10], rotationNone, ensure_object_boundary_in_rangeTrue, ensure_valid_placementTrue, reference_posself.table_offset, z_offset0.15, rngself.rng, ) # task includes arena, robot, and objects of interest self.model ManipulationTask( mujoco_arenamujoco_arena, mujoco_robots[robot.robot_model for robot in self.robots], mujoco_objectsself.objects, ) def _setup_references(self): Sets up references to important components. A reference is typically an index or a list of indices that point to the corresponding elements in a flatten array, which is how MuJoCo stores physical simulation data. super()._setup_references() # Additional object references from this env self.cube_body_id self.sim.model.body_name2id(self.cube.root_body) self.cup_body_id self.sim.model.body_name2id(self.cup.root_body) def _setup_observables(self): Sets up observables to be used for this environment. Creates object-based observables if enabled Returns: OrderedDict: Dictionary mapping observable names to its corresponding Observable object observables super()._setup_observables() # low-level object information if self.use_object_obs: # define observables modality modality object # cube-related observables sensor(modalitymodality) def cube_pos(obs_cache): return np.array(self.sim.data.body_xpos[self.cube_body_id]) sensor(modalitymodality) def cube_quat(obs_cache): return convert_quat(np.array(self.sim.data.body_xquat[self.cube_body_id]), toxyzw) # ------- 2. 新增杯子的传感器群 ------- sensor(modalitymodality) def cup_pos(obs_cache): return np.array(self.sim.data.body_xpos[self.cup_body_id]) sensor(modalitymodality) def cup_quat(obs_cache): return convert_quat(np.array(self.sim.data.body_xquat[self.cup_body_id]), toxyzw) sensors [cube_pos, cube_quat, cup_pos, cup_quat] arm_prefixes self._get_arm_prefixes(self.robots[0], include_robot_nameFalse) full_prefixes self._get_arm_prefixes(self.robots[0]) # gripper to cube position sensor; one for each arm sensors [ self._get_obj_eef_sensor(full_pf, cube_pos, f{arm_pf}gripper_to_cube_pos, modality) for arm_pf, full_pf in zip(arm_prefixes, full_prefixes) ] # 新增夹爪到杯子的距离 sensors [ self._get_obj_eef_sensor(full_pf, cup_pos, f{arm_pf}gripper_to_cup_pos, modality) for arm_pf, full_pf in zip(arm_prefixes, full_prefixes) ] names [s.__name__ for s in sensors] # Create observables for name, s in zip(names, sensors): observables[name] Observable( namename, sensors, sampling_rateself.control_freq, ) return observables def _reset_internal(self): Resets simulation internal configurations. super()._reset_internal() # Reset all object positions using initializer sampler if were not directly loading from an xml if not self.deterministic_reset: # Sample from the placement initializer for all objects object_placements self.placement_initializer.sample() # Loop through all objects and reset their positions for obj_pos, obj_quat, obj in object_placements.values(): print(f {obj.name} ) print(fjoints: {obj.joints}) print(fjoints[0]: {obj.joints[0] if obj.joints else EMPTY!}) print(fplacement pos: {obj_pos}) if obj.joints: self.sim.data.set_joint_qpos(obj.joints[0], np.concatenate([np.array(obj_pos), np.array(obj_quat)])) else: print(fWARNING: {obj.name} has no joints!) def visualize(self, vis_settings): In addition to super call, visualize gripper site proportional to the distance to the cube. Args: vis_settings (dict): Visualization keywords mapped to T/F, determining whether that specific component should be visualized. Should have grippers keyword as well as any other relevant options specified. # Run superclass method first super().visualize(vis_settingsvis_settings) # Color the gripper visualization site according to its distance to the cube if vis_settings[grippers]: self._visualize_gripper_to_target(gripperself.robots[0].gripper, targetself.cube) def _check_success(self): Check if cube has been lifted. Returns: bool: True if cube has been lifted cube_height self.sim.data.body_xpos[self.cube_body_id][2] table_height self.model.mujoco_arena.table_offset[2] # cube is higher than the table top above a margin return cube_height table_height 0.04