Description
Bug Description
When using two tiled cameras in the scene, the last camera overrides all observations, resulting in both cameras capturing the same image.
Steps to reproduce
- Run the following script, which initializes the
CartpoleRGBCameraEnvCfg
environment with two environments and two tiled cameras.
from omni.isaac.lab.app import AppLauncher
app_launcher = AppLauncher(headless=False, enable_cameras=True)
simulation_app = app_launcher.app
"""Rest everything follows."""
import torch
from omni.isaac.lab.envs import ManagerBasedRLEnv
from omni.isaac.lab.sensors.camera.utils import save_images_to_file
from omni.isaac.lab_tasks.manager_based.classic.cartpole import CartpoleRGBCameraEnvCfg
def main():
env_cfg = CartpoleRGBCameraEnvCfg()
env_cfg.scene.num_envs = 2
env = ManagerBasedRLEnv(cfg=env_cfg)
while simulation_app.is_running():
with torch.inference_mode():
actions = torch.rand_like(env.action_manager.action)
obs, _, _, _, _ = env.step(actions)
save_images_to_file(obs["policy"]["image1"], "/tmp/image1.png")
save_images_to_file(obs["policy"]["image2"], "/tmp/image2.png")
env.close()
if __name__ == "__main__":
main()
simulation_app.close()
- Modify the
CartpoleRGBCameraSceneCfg
class in the following file:source/extensions/omni.isaac.lab_tasks/omni/isaac/lab_tasks/manager_based/classic/cartpole/cartpole_camera_env_cfg.py
, adding two tiled cameras as shown below:
@configclass
class CartpoleRGBCameraSceneCfg(CartpoleSceneCfg):
tiled_camera1: TiledCameraCfg = TiledCameraCfg(
prim_path="{ENV_REGEX_NS}/Camera1",
offset=TiledCameraCfg.OffsetCfg(pos=(-7.0, 0.0, 3.0), rot=(0.9945, 0.0, 0.1045, 0.0), convention="world"),
data_types=["rgb"],
spawn=sim_utils.PinholeCameraCfg(
focal_length=24.0, focus_distance=400.0, horizontal_aperture=20.955, clipping_range=(0.1, 20.0)
),
width=1280,
height=720,
)
tiled_camera2: TiledCameraCfg = TiledCameraCfg(
prim_path="{ENV_REGEX_NS}/Camera2",
offset=TiledCameraCfg.OffsetCfg(pos=(-7.0, 0.0, 3.0), convention="world"),
data_types=["rgb"],
spawn=sim_utils.PinholeCameraCfg(
focal_length=24.0, focus_distance=400.0, horizontal_aperture=20.955, clipping_range=(0.1, 20.0)
),
width=1280,
height=720,
)
- Modify
RGBObservationsCfg
to include observations for both cameras:
@configclass
class RGBObservationsCfg:
"""Observation specifications for the MDP."""
@configclass
class RGBCameraPolicyCfg(ObsGroup):
"""Observations for policy group."""
image1 = ObsTerm(func=grab_images, params={"sensor_cfg": SceneEntityCfg("tiled_camera1"), "data_type": "rgb", "normalize": True})
image2 = ObsTerm(func=grab_images, params={"sensor_cfg": SceneEntityCfg("tiled_camera2"), "data_type": "rgb", "normalize": True})
def __post_init__(self) -> None:
self.enable_corruption = False
self.concatenate_terms = False
policy: ObsGroup = RGBCameraPolicyCfg()
- Modify the scene lights in
source/extensions/omni.isaac.lab_tasks/omni/isaac/lab_tasks/manager_based/classic/cartpole/cartpole_env_cfg.py
for clearer image results:
@configclass
class CartpoleSceneCfg(InteractiveSceneCfg):
"""Configuration for a cart-pole scene."""
# ground plane
ground = AssetBaseCfg(
prim_path="/World/ground",
spawn=sim_utils.GroundPlaneCfg(size=(100.0, 100.0), color=(0.5, 0.5, 0.5)),
)
# cartpole
robot: ArticulationCfg = CARTPOLE_CFG.replace(prim_path="{ENV_REGEX_NS}/Robot")
# lights
dome_light = AssetBaseCfg(
prim_path="/World/DomeLight",
spawn=sim_utils.DomeLightCfg(color=(0.9, 0.9, 0.9), intensity=10.0),
)
distant_light = AssetBaseCfg(
prim_path="/World/DistantLight",
spawn=sim_utils.DistantLightCfg(color=(0.9, 0.9, 0.9), intensity=1.0),
init_state=AssetBaseCfg.InitialStateCfg(rot=(0.738, 0.477, 0.477, 0.0)),
)
sphere_light = AssetBaseCfg(
prim_path="{ENV_REGEX_NS}/SphereLight",
spawn=sim_utils.SphereLightCfg(
color=(0.1, 0.1, 0.1), enable_color_temperature=True, color_temperature=5500, intensity=100, radius=2.0, exposure=10.0,
),
init_state=AssetBaseCfg.InitialStateCfg(pos=(0, 0, 5)),
)
Additional context
Two identical tiled cameras with slightly different orientations (one capturing more of the ground, the other more of the sky) were used. Observation data was saved as images for inspection.
Running the following scenarios, only the first one reproduces the bug:
-
Scenario 1: Running two tiled cameras in two environments causes the bug—both image files show the second camera’s view.
-
Scenario 2: Running two tiled cameras in a single environment works correctly—each camera renders its own FOV.
-
Scenario 3: Running two non-tiled cameras in two environments works as expected.
Note: Although it may be unrelated, there is a noticeable decline in image quality in the first scenario compared to the other two.