Merge branch 'og-develop' into feat/doc-update-josiah

This commit is contained in:
Josiah Wong 2024-07-11 21:57:18 -07:00 committed by GitHub
commit 3c626d6fa1
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
17 changed files with 731 additions and 44 deletions

View File

@ -56,7 +56,7 @@ jobs:
- name: Run tests
working-directory: omnigibson-src
run: pytest tests/${{ matrix.test_file }}.py --junitxml=${{ matrix.test_file }}.xml && cp ${{ matrix.test_file }}.xml ${GITHUB_WORKSPACE}/
run: pytest -s tests/${{ matrix.test_file }}.py --junitxml=${{ matrix.test_file }}.xml && cp ${{ matrix.test_file }}.xml ${GITHUB_WORKSPACE}/
- name: Deploy artifact
uses: actions/upload-artifact@v3

View File

@ -1,11 +1,6 @@
#!/usr/bin/env bash
set -e -o pipefail
docker build \
-t stanfordvl/omnigibson-dev:latest \
-f docker/dev.Dockerfile \
.
docker build \
-t stanfordvl/omnigibson:latest \
-t stanfordvl/omnigibson:$(sed -ne "s/.*version= *['\"]\([^'\"]*\)['\"] *.*/\1/p" setup.py) \

View File

@ -1,4 +1,4 @@
FROM stanfordvl/omnigibson-dev:og-develop
FROM stanfordvl/omnigibson:og-develop
ARG DUMB_INIT_VERSION="1.2.2"
ARG GIT_CORE_PPA_KEY="A1715D88E1DF1F24"

View File

@ -1,4 +1,4 @@
FROM nvcr.io/nvidia/isaac-sim:2023.1.1
FROM nvcr.io/nvidia/isaac-sim:4.0.0
# Set up all the prerequisites.
RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y \
@ -10,9 +10,6 @@ RUN rm -rf /isaac-sim/exts/omni.isaac.ml_archive/pip_prebundle/gym*
RUN rm -rf /isaac-sim/kit/extscore/omni.kit.pip_archive/pip_prebundle/numpy*
RUN /isaac-sim/python.sh -m pip install click~=8.1.3
# Disable the livestream extension getting launched by default in Isaac Sim 2023.1.1
RUN sed -i 's/\"omni.kit.livestream.native.*//g' /isaac-sim/apps/omni.isaac.sim.python.kit
# Mount the data directory
VOLUME ["/data"]
ENV OMNIGIBSON_DATASET_PATH /data/og_dataset

View File

@ -34,6 +34,8 @@ Alternatively, a scene can be directly imported at runtime by first creating the
### Runtime
To import an object into a scene, call `scene.add_object(obj)`.
The scene keeps track of and organizes all imported objects via its owned `scene.object_registry`. Objects can quickly be queried by relevant property keys (1), such as `name`, `prim_path`, and `category`, from `env.scene.object_registry` as follows:
{ .annotate }

View File

@ -6,20 +6,20 @@ icon: material/repeat
## Description
**`OmniGibson`**'s [Simulator](../reference/simulator.html) class is the global singleton that serves as the interface with omniverse's low-level physx (physics) backend. It provides utility functions for modulating the ongoing simulation as well as the low-level interface for importing scenes and objects. For standard use-cases, interfacing with the simulation exclusively through a created [environment](./environments.md) should be sufficient, though for more advanced or prototyping use-cases it may be common to interface via this simulator class.
**`OmniGibson`**'s [Simulator](../reference/simulator.html) class is the global singleton that serves as the interface with omniverse's low-level physx (physics) backend. It provides utility functions for modulating the ongoing simulation as well as the low-level interface for importing scenes and objects. For standard use-cases, interfacing with the simulation exclusively through a created [environment](./environments.md) or [vector environment](./vector_environments.md) should be sufficient, though for more advanced or prototyping use-cases it may be common to interface via this simulator class.
## Usage
### Creating
Because this `Simulator` is a global singleton, it is instantiated every time the stage is cleared and always occurs at the _very beginning_ of **`OmniGibson`**'s launching. This either occurs when an environment instance is created (`env = Environment(...)`), when `og.launch()` is explicitly called, or whenever the stage is reset.
Because this `Simulator` is a global singleton, it is instantiated every time the stage is cleared and always occurs at the _very beginning_ of **`OmniGibson`**'s launching. This either occurs when an environment instance is created (`env = Environment(...)` or `vec_env = VectorEnvironment(...)`), when `og.launch()` is explicitly called, or whenever the stage is reset.
### Runtime
After **`OmniGibson`** is launched, the simulator interface can be accessed globally via `og.sim`. Below, we briefly describe multiple common usages of the simulation interface:
#### Importing and Removing Scenes / Objects
The simulator can directly import a scene via `sim.import_scene(scene)`, and an object can be imported into a scene via `scene.import_object(object)`. The imported scene and its corresponding objects can be directly accessed through `sim.scenes`. To remove a desired object, call `sim.remove_object(object)`. The simulator can also clear all the scenes via `og.clear()`.
The simulator can directly import a scene via `sim.import_scene(scene)`. The imported scene and its corresponding objects can be directly accessed through `sim.scenes`. To remove a desired object, call `sim.remove_object(object)`. The simulator can also clear all the scenes via `og.clear()`.
#### Propagating Physics
The simulator can be manually stepped, with or without physics / rendering (`sim.step()`, `sim.step_physics()`, `sim.render()`), and can be stopped (`sim.stop()`), paused (`sim.pause()`), or played (`sim.play()`). Note that physics only runs when the simulator is playing! The current sim mode can be checked via `sim.is_stopped()`, `sim.is_paused()`, and `sim.is_playing()`.

View File

@ -6,15 +6,15 @@ icon: material/water-outline
## Description
**`OmniGibson`**'s [`System`](../reference/systems/base_system.html) class represents global singletons that encapsulate a single particle type. These system classes provide functionality for generating, tracking, and removing any number of particles arbitrarily located throughout the current scene.
**`OmniGibson`**'s [`System`](../reference/systems/base_system.html)s represents scene singletons that encapsulate a single particle type. These systems provide functionality for generating, tracking, and removing any number of particles arbitrarily located throughout the current scene.
## Usage
### Creating
For efficiency reasons, systems are created dynamically on an as-needed basis. A system can be dynamically created (or referenced, if it already exists) via `get_system(name)`, where `name` defines the name of the system. For a list of all possible system names, see `REGISTERED_SYSTEMS`. Both of these utility functions can be directly imported from `omnigibson.systems`.
For efficiency reasons, systems are created dynamically on an as-needed basis. A system can be dynamically created (or referenced, if it already exists) via `scene.get_system(name)`, where `name` defines the name of the system. If you do not wish to initialize a system when refrencing it, e.g. for performance reasons, use the `force_init` flag: `scene.get_system(name, force_init=False)`. For a list of all possible system names, see `scene.system_registry.objects`.
### Runtime
A given system can be accessed globally at any time via `get_system(...)`. Systems can generate particles via `system.generate_particles(...)`, track their states via `system.get_particles_position_orientation()`, and remove them via `system.remove_particles(...)`. Please refer to the [`System`'s API Reference](../reference/systems/base_system.html) for specific information regarding arguments. Moreover, specific subclasses may implement more complex generation behavior, such as `VisualParticleSystem`s `generate_group_particles(...)` which spawn visual (non-collidable) particles that are attached to a specific object.
A given system can be accessed at any time via `scene.get_system(...)`. Systems can generate particles via `system.generate_particles(...)`, track their states via `system.get_particles_position_orientation()`, and remove them via `system.remove_particles(...)`. Please refer to the [`System`'s API Reference](../reference/systems/base_system.html) for specific information regarding arguments. Moreover, specific subclasses may implement more complex generation behavior, such as `VisualParticleSystem`s `generate_group_particles(...)` which spawn visual (non-collidable) particles that are attached to a specific object.
## Types

View File

@ -20,7 +20,7 @@ Transition rules are **`OmniGibson`**'s method for simulating complex physical p
Because `TransitionRule`s are monolithic classes, these should be defined _before_ **`OmniGibson`** is launched. A rule can be easily extended by subclassing the `BaseTransitionRule` class and implementing the necessary functions. For a simple example, please see the [`SlicingRule`](../reference/transition_rules.html#transition_rules.SlicingRule) class.
### Runtime
At runtime, the monolithic [`TransitionRuleAPI`](../reference/transition_rules.html#transition_rules.TransitionRuleAPI) automatically handles the stepping and processing of all defined transition rule classes. For efficiency reasons, rules are dynamically loaded and checked based on the object / system set currently active in the simulator. A rule will only be checked if there is at least one valid candidate combination amongst the current object / system set. For example, if there is no sliceable object present in the simulator, then `SlicingRule` will not be active. Every time an object / system is added / removed from the simulator, all rules are refreshed so that the current active transition rule set is always accurate.
At runtime, each scene owns a [`TransitionRuleAPI`](../reference/transition_rules.html#transition_rules.TransitionRuleAPI) instance, which automatically handles the stepping and processing of all defined transition rule classes. For efficiency reasons, rules are dynamically loaded and checked based on the object / system set currently active in the scene. A rule will only be checked if there is at least one valid candidate combination amongst the current object / system set. For example, if there is no sliceable object present in this scene, then `SlicingRule` will not be active. Every time an object / system is added / removed from the scene, all rules are refreshed so that the current active transition rule set is always accurate.
In general, you should not need to interface with the `TransitionRuleAPI` class at all -- if your rule implementation is correct, then the API will automatically handle the transition when the appropriate conditions are met!

View File

@ -0,0 +1,36 @@
---
icon: octicons/stack-16
---
# 🌌 **Vector Environments**
## Description
To support large-scale parallelization, we now support vector environments. Each environment is similar to our regular [environment](./environments.md), but our simulator now can keep track of multiple environments simultaneously. We have implemented many vectorized operations to optimize the performance of these parallel environments. We are also actively working on further enhancements to make them faster. Some use cases for this include reinforcement learning, parallelized training with domain randomization, and parallelized policy evaluation.
## Usage
Creating a minimal vector environment requires the definition of a config dictionary. This dictionary is copied across all environments:
??? code "vec_env_simple.py"
``` python linenums="1"
import omnigibson as og
cfg = {
"scene": {
"type": "InteractiveTraversableScene",
"scene_model": "Rs_int",
"load_object_categories": ["floors", "walls"],
},
"objects": [],
"robots": [
{
"type": "Fetch",
"obs_modalities": [],
}
]
}
vec_env = og.VectorEnvironment(num_envs=3, config=cfg)
actions = [...]
observations, rewards, terminates, truncates, infos = vec_env.step(actions)
```

View File

@ -99,6 +99,7 @@ nav:
- Transition Rules: modules/transition_rules.md
- Simulator: modules/simulator.md
- Environments: modules/environments.md
- Vector Environments: modules/vector_environments.md
- Tutorials:
- Demo Collection: tutorials/demo_collection.md
- Using Macros: tutorials/using_macros.md

View File

@ -135,7 +135,7 @@ class BaseController(Serializable, Registerable, Recreatable):
np.array(self._control_limits[self.control_type][0])[self.dof_idx],
np.array(self._control_limits[self.control_type][1])[self.dof_idx],
)
if type(command_input_limits) == str and command_input_limits == "default"
if type(command_output_limits) == str and command_output_limits == "default"
else command_output_limits
)
self._command_input_limits = (

View File

@ -15,6 +15,7 @@ from omnigibson.tasks import REGISTERED_TASKS
from omnigibson.utils.config_utils import parse_config
from omnigibson.utils.gym_utils import (
GymObservable,
maxdim,
recursively_generate_compatible_dict,
recursively_generate_flat_dict,
)
@ -337,12 +338,12 @@ class Environment(gym.Env, GymObservable, Recreatable):
for robot in self.robots:
# Load the observation space for the robot
robot_obs = robot.load_observation_space()
if gym.spaces.utils.flatdim(robot_obs) > 0:
if maxdim(robot_obs) > 0:
obs_space[robot.name] = robot_obs
# Also load the task obs space
task_space = self._task.load_observation_space()
if gym.spaces.utils.flatdim(task_space) > 0:
if maxdim(task_space) > 0:
obs_space["task"] = task_space
# Also load any external sensors
@ -470,11 +471,11 @@ class Environment(gym.Env, GymObservable, Recreatable):
# Grab all observations from each robot
for robot in self.robots:
if gym.spaces.utils.flatdim(robot.observation_space) > 0:
if maxdim(robot.observation_space) > 0:
obs[robot.name], info[robot.name] = robot.get_obs()
# Add task observations
if gym.spaces.utils.flatdim(self._task.observation_space) > 0:
if maxdim(self._task.observation_space) > 0:
obs["task"] = self._task.get_obs(env=self)
# Add external sensor observations if they exist

View File

@ -0,0 +1,630 @@
[package]
title = "OmniGibson"
description = "A platform for accelerating Embodied AI research"
version = "2023.1.1"
# That makes it browsable in UI with "experience" filter
keywords = ["experience", "app", "usd"]
[dependencies]
# The Main UI App
"omni.kit.uiapp" = {}
"omni.kit.renderer.core" = {}
# Livestream - OV Streaming Client
"omni.kit.streamsdk.plugins" = {version = "2.5.2", exact = true}
# Status Bar
"omni.kit.window.status_bar" = {}
"omni.stats" = {}
"omni.kit.telemetry" = {}
"omni.kit.menu.utils" = {}
"omni.kit.menu.file" = {}
"omni.kit.menu.edit" = {}
"omni.kit.menu.create" = {}
"omni.kit.menu.common" = {}
"omni.kit.menu.stage" = {}
"omni.kit.window.file" = {}
"omni.kit.context_menu" = {}
"omni.kit.selection" = {}
"omni.kit.stage_templates" = {}
# "omni.kit.stage.mdl_converter" = {}
# Animation
# "omni.anim.skelvis" = {}
# PhysX
"omni.physx.bundle" = {}
"omni.physx.tensors" = {}
# "omni.physx.fabric" = {}
# "omni.physx.zerogravity" = {}
# "omni.kit.search.service" = {}
"omni.kit.primitive.mesh" = {}
# Create Windows
"omni.kit.window.title" = {}
"omni.kit.widget.live" = {}
"omni.kit.window.stage" = {}
"omni.kit.widget.layers" = {}
"omni.kit.window.cursor" = {}
"omni.kit.window.toolbar" = {}
"omni.kit.window.commands" = {}
# New Viewport, load the default bundle of extensions
"omni.kit.viewport.bundle" = {}
"omni.kit.viewport.menubar.lighting" = {}
# Load the rendering extensions
# "omni.renderer" = { tag = "rtx" }
# Load the RTX rendering bundle
"omni.kit.viewport.rtx" = {}
# Load the Storm rendering bundle
"omni.kit.viewport.pxr" = {}
# Needed for Fabric delegate
"omni.resourcemonitor" = {}
# Additional Viewport features (legacy grid etc, HUD GPU stats)
"omni.kit.viewport.legacy_gizmos" = {}
"omni.kit.viewport.ready" = {}
"omni.hydra.engine.stats" = {}
"omni.rtx.settings.core" = {} # this is the new Render Settings 2.0
# "omni.kit.window.movie_capture" = { }
"omni.kit.profiler.window" = {}
"omni.kit.stage_column.variant" = {}
"omni.kit.stage_column.payload" = {}
# Viewport Widgets and Collaboration
# "omni.kit.viewport_widgets_manager" = {}
"omni.kit.collaboration.channel_manager" = {}
# "omni.kit.widgets.custom" = {}
# utils window
# "omni.kit.window.about" = {} # Isaac Sim: disable this and replace with our own
# "omni.kit.window.privacy" = {}
# "omni.kit.window.provide_feedback" = {} # Isaac Sim: disable this and replace with our own
# "omni.kit.material.library" = {}
# "omni.kit.window.imageviewer" = {}
"omni.kit.widget.filebrowser" = {}
"omni.kit.window.filepicker" = {}
"omni.kit.window.content_browser" = {}
"omni.kit.window.stats" = { order = 1000 }
"omni.kit.window.script_editor" = {}
"omni.kit.window.console" = {}
"omni.kit.window.extensions" = {}
# browsers
"omni.kit.browser.sample" = {}
# "omni.kit.browser.asset" = {}
# "omni.kit.browser.asset_store" = {}
# "omni.kit.browser.asset_provider.local" = {}
# "omni.kit.browser.asset_provider.sketchfab" = {}
# "omni.kit.browser.asset_provider.turbosquid" = {}
# "omni.kit.browser.asset_provider.actorcore" = {}
# "omni.kit.window.environment" = {} # potentially increases startup times
# Material
# "omni.kit.window.material" = { }
# "omni.kit.graph.delegate.default" = { }
# "omni.kit.window.material_graph" = { }
# "omni.kit.window.usd_paths" = {}
# "omni.kit.window.preferences" = { order = 1000 } # so the menu is in the correct place
# "omni.kit.renderer.capture" = {}
# "omni.kit.thumbnails.usd" = {}
# "omni.kit.thumbnails.images" = {}
# bring all the property Widgets and Window
"omni.kit.window.property" = {}
"omni.kit.property.bundle" = {}
"omni.kit.property.layer" = {}
# tool
# "omni.kit.asset_converter" = {}
# "omni.kit.tool.asset_importer" = {}
# "omni.kit.tool.asset_exporter" = {}
# "omni.kit.tool.collect" = {}
# "omni.kit.tool.remove_unused.core" = {}
# "omni.kit.tool.remove_unused.controller" = {}
# Iray
# "omni.iray.settings.core" = {}
# "omni.hydra.iray" = { order = -1000 }
#Particle/PointCloud FileFormat
# "omni.usd.fileformat.e57" = { }
# "omni.kit.pointclouds" = {}
# External Scene
# "omni.geo.streaming.bundle" = {}
# All QuickSearch
# "omni.kit.window.quicksearch" = {}
# "omni.kit.quicksearch.actions" = {}
# "omni.kit.quicksearch.settings" = {}
# "omni.kit.quicksearch.select" = {}
# "omni.kit.quicksearch.commands" = {}
# "omni.kit.quicksearch.menu" = {}
# "omni.kit.quicksearch.material" = {}
# "omni.kit.quicksearch.hdri" = {}
# "omni.kit.quicksearch.props" = {}
# "omni.kit.search.files" = {}
# Compatibility Checker
# "omni.kit.compatibility_checker" = {}
# VERSIONING
# "omni.kit.widget.versioning" = {}
# Paint Default now
# "omni.paint.system.bundle" = {}
# Manipulator
"omni.kit.manipulator.prim" = {}
"omni.kit.manipulator.transform" = {}
"omni.kit.manipulator.viewport" = {}
# "omni.kit.manipulator.tool.mesh_snap" = {}
# Destruction schema
# "omni.usd.schema.destruction" = {}
# Animation
# "omni.anim.skelJoint" = { }
# "omni.anim.curve" = { }
# "omni.kit.widget.timeline" = { }
# "omni.anim.curve_editor" = { }
# "omni.anim.window.timeline" = { }
# "omni.anim.shared.core" = {}
# "omni.anim.timeline" = { }
# "omni.anim.graph.bundle" = {}
# "omni.anim.graph.core" = {}
# "omni.anim.graph.ui" = {}
# "omni.anim.retarget.bundle" = {}
# "omni.anim.retarget.core" = {}
# "omni.anim.retarget.ui" = {}
#"omni.anim.camera_tool" = {}
# Needed to properly load navigation mesh
"omni.anim.graph.schema" = {}
"omni.anim.navigation.schema" = {}
# OmniGraph
"omni.graph.bundle.action" = {}
"omni.graph.window.action" = {}
"omni.graph.window.generic" = {}
"omni.graph.visualization.nodes" = {}
# Python Scripting Component
# "omni.kit.scripting" = {}
# kit-testing
# "omni.kit.tests.usd_stress" = {}
# Curves
# "omni.curve.manipulator" = {}
# General Proceduralism
# "omni.genproc.bundle" = {}
# Sequencer
# "omni.kit.window.sequencer" = {}
# "omni.services.usd" = {}
# SBSAR
# "omni.kit.property.sbsar" = {}
# "omni.usd.fileformat.sbsar" = {}
# Thumbnails
# "omni.kit.thumbnails.mdl" = {}
# Quicklayout
# "omni.kit.quicklayout" = {}
# AOV
# "omni.kit.menu.aov" = {}
# "omni.graph.examples.cpp" = {}
# Collections
# "omni.kit.window.collection" = {}
# "omni.kit.widget.collection" = {}
# "omni.kit.property.collection" = {}
# Extended Searchfield
# "omni.kit.widget.extended_searchfield" = {}
# Particle
# "omni.particle.system.bundle" = {}
# Scene Visualization
"omni.usd.schema.scene.visualization" = {}
# "omni.scene.visualization.bundle" = {}
#Section Tool
# "omni.kit.window.section" = {}
# startfleet auth enabled for cloud_share to work on the receiver
# "omni.services.starfleet.auth" = {}
# Array Tool
# "omni.tools.array" = {}
# "omni.tools.pivot" = {}
# Randomizer
# "omni.tools.randomizer" = {}
# Deepsearch
# "omni.kit.browser.deepsearch" = {}
# Actions
# "omni.kit.actions.window" = {}
# "omni.kit.viewport.actions" = {}
# Scene Optimizer (formerly Data Adapter)
# "omni.scene.optimizer.bundle" = {}
# Hotkeys
"omni.kit.hotkeys.window" = {}
# USDA
# "omni.kit.usda_edit" = {}
# "omni.rakis" = {}
"omni.warp" = {}
# needed for omni.kit.viewport.ready.viewport_ready
"omni.activity.profiler" = {}
"omni.activity.pump" = {}
"omni.kit.widget.cache_indicator" = {}
[settings]
renderer.active = "rtx"
exts."omni.kit.viewport.menubar.camera".expand = true # Expand the extra-camera settings by default
exts."omni.kit.window.file".useNewFilePicker = true
exts."omni.kit.tool.asset_importer".useNewFilePicker = true
exts."omni.kit.tool.collect".useNewFilePicker = true
exts."omni.kit.widget.layers".useNewFilePicker = true
exts."omni.kit.renderer.core".imgui.enableMips = true
exts."omni.kit.browser.material".enabled = false
exts."omni.kit.browser.asset".visible_after_startup = false
exts."omni.kit.window.material".load_after_startup = true
exts."omni.kit.widget.cloud_share".require_access_code = false
exts."omni.kit.pipapi".installCheckIgnoreVersion = true
exts."omni.kit.viewport.window".startup.windowName="Viewport" # Rename from Viewport Next
exts."omni.kit.menu.utils".logDeprecated = false
# app.content.emptyStageOnStart = false
app.file.ignoreUnsavedOnExit = true # prevents save dialog when exiting
# deprecate support for old kit.ui.menu
app.menu.legacy_mode = false
# use omni.ui.Menu for the MenuBar
app.menu.compatibility_mode = false
# Setting the port for the embedded http server
exts."omni.services.transport.server.http".port = 8211
# default viewport is fill
app.runLoops.rendering_0.fillResolution = false
exts."omni.kit.window.viewport".blockingGetViewportDrawable = false
exts."omni.kit.test".includeTests.1 = "*isaac*"
[settings.app.python]
# These disable the kit app from also printing out python output, which gets confusing
interceptSysStdOutput = false
logSysStdOutput = false
[settings.app.settings]
persistent = false
dev_build = false
fabricDefaultStageFrameHistoryCount = 3 # needed for omni.syntheticdata TODO105 Still True?
[settings.app.window]
title = "OmniGibson"
hideUi = false
_iconSize = 256
iconPath = "${app}/../exts/omni.isaac.app.setup/data/nvidia-omniverse-isaacsim.ico"
# width = 1700
# height = 900
# x = -1
# y = -1
# Fonts
[setting.app.font]
file = "${fonts}/OpenSans-SemiBold.ttf"
size = 16
# [setting.app.runLoops]
# main.rateLimitEnabled = false
# main.rateLimitFrequency = 60
# main.rateLimitUseBusyLoop = false
# rendering_0.rateLimitEnabled = false
[settings.exts.'omni.kit.window.extensions']
# List extensions here we want to show as featured when extension manager is opened
featuredExts = []
[settings]
# MGPU is always on, you can turn it from the settings, and force this off to save even more resource if you
# only want to use a single GPU on your MGPU system
# False for Isaac Sim
renderer.multiGpu.enabled = true
renderer.multiGpu.autoEnable = true
'rtx-transient'.resourcemanager.enableTextureStreaming = true
# app.hydra.aperture.conform = 4 # in 105.1 pixels are square by default
app.hydraEngine.waitIdle = false
rtx.newDenoiser.enabled = true
# Enable Iray and pxr by setting this to "rtx,iray,pxr"
renderer.enabled = "rtx"
physics.autoPopupSimulationOutputWindow=false
### async rendering settings
omni.replicator.asyncRendering = false
app.asyncRendering = false
app.asyncRenderingLowLatency = false
### Render thread settings
app.runLoops.main.rateLimitEnabled = false
app.runLoops.main.rateLimitFrequency = 120
app.runLoops.main.rateLimitUsePrecisionSleep = true
app.runLoops.main.syncToPresent = false
app.runLoops.present.rateLimitFrequency = 120
app.runLoops.present.rateLimitUsePrecisionSleep = true
app.runLoops.rendering_0.rateLimitFrequency = 120
app.runLoops.rendering_0.rateLimitUsePrecisionSleep = true
app.runLoops.rendering_0.syncToPresent = false
app.runLoops.rendering_1.rateLimitFrequency = 120
app.runLoops.rendering_1.rateLimitUsePrecisionSleep = true
app.runLoops.rendering_1.syncToPresent = false
app.runLoopsGlobal.syncToPresent = false
app.vsync = false
exts.omni.kit.renderer.core.present.enabled = false
exts.omni.kit.renderer.core.present.presentAfterRendering = false
persistent.app.viewport.defaults.tickRate = 120
rtx-transient.dlssg.enabled = false
privacy.externalBuild = true
# Basic Kit App
################################
app.versionFile = "${app}/../VERSION"
app.name = "Isaac-Sim"
app.version = "2023.1.1"
# hide NonToggleable Exts
exts."omni.kit.window.extensions".hideNonToggleableExts = true
exts."omni.kit.window.extensions".showFeatureOnly = false
# Hang Detector
################################
# app.hangDetector.enabled = false
# app.hangDetector.timeout = 120
# Browsers
exts."omni.kit.browser.material".folders = [
"Base::http://omniverse-content-production.s3-us-west-2.amazonaws.com/Materials/Base",
"vMaterials::http://omniverse-content-production.s3.us-west-2.amazonaws.com/Materials/vMaterials_2/",
"Twinbru Fabrics::https://twinbru.s3.eu-west-1.amazonaws.com/omniverse/Twinbru Fabrics/"
]
exts."omni.kit.window.environment".folders = [
"https://omniverse-content-production.s3.us-west-2.amazonaws.com/Assets/Skies/2022_1/Skies",
"http://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Scenes/Templates",
]
exts."omni.kit.browser.sample".folders = [ "http://omniverse-content-production.s3-us-west-2.amazonaws.com//Samples" ]
exts."omni.kit.browser.asset".folders = [
"http://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Vegetation",
"http://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/ArchVis/Commercial",
"http://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/ArchVis/Industrial",
"http://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/ArchVis/Residential",
"http://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/DigitalTwin/Assets/Warehouse/Equipment",
"http://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/DigitalTwin/Assets/Warehouse/Safety",
"http://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/DigitalTwin/Assets/Warehouse/Shipping",
"http://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/DigitalTwin/Assets/Warehouse/Storage",
]
exts."omni.kit.browser.texture".folders = [
"http://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Vegetation",
]
#RTX Settings
[settings.rtx]
translucency.worldEps = 0.005
# Content Browser
###############################
[settings.exts."omni.kit.window.content_browser"]
enable_thumbnail_generation_images = false # temp fix to avoid leaking python processes
# Extensions
###############################
[settings.exts."omni.kit.registry.nucleus"]
registries = [
{ name = "kit/default", url = "https://ovextensionsprod.blob.core.windows.net/exts/kit/prod/shared" },
{ name = "kit/sdk", url = "https://ovextensionsprod.blob.core.windows.net/exts/kit/prod/sdk/${kit_version_short}/${kit_git_hash}" },
{ name = "kit/community", url = "https://dw290v42wisod.cloudfront.net/exts/kit/community" },
]
[settings.app.extensions]
skipPublishVerification = false
registryEnabled = true
[settings.exts."omni.kit.window.modifier.titlebar"]
titleFormatString = " Isaac Sim {version:${app}/../SHORT_VERSION,font_color=0x909090,font_size=16} {separator} {file, board=true}"
showFileFullPath = true
icon.file = "${app}/../exts/omni.isaac.app.setup/data/nvidia-omniverse-isaacsim.ico"
icon.size = 256
defaultFont.name = "Arial"
defaultFont.size = 16
defaultFont.color = 0xD0D0D0
separator.color = 0x00B976
separator.width = 1
windowBorder.color = 0x0F0F0F
windowBorder.width = 2
colors.caption = 0x0F0F0F
colors.client = 0x0F0F0F
respondOnMouseUp = true
changeWindowRegion = true
# Register extension folder from this repo in kit
[settings.app.exts]
folders = ["${app}/../exts", "${app}/../extscache", "${app}/../extsPhysics"]
[settings.crashreporter.data]
experience = "Isaac Sim Python"
# Isaac Sim Settings
###############################
[settings.app.renderer]
skipWhileMinimized = false
sleepMsOnFocus = 0
sleepMsOutOfFocus = 0
resolution.width=1280
resolution.height=720
[settings.app.livestream]
proto = "ws"
allowResize = true
outDirectory = "${data}"
# default camera position in meters
[settings.app.viewport]
defaultCamPos.x = 5
defaultCamPos.y = 5
defaultCamPos.z = 5
[settings.rtx]
raytracing.fractionalCutoutOpacity = false
hydra.enableSemanticSchema = true
# descriptorSets=60000
# reservedDescriptors=500000
# sceneDb.maxInstances=1000000
# Enable this for static scenes, improves visual quality
# directLighting.sampledLighting.enabled = true
[settings.persistent]
app.file.recentFiles = []
app.stage.upAxis = "Z"
app.stage.movePrimInPlace = false
app.stage.instanceableOnCreatingReference = false
app.stage.materialStrength = "weakerThanDescendants"
app.transform.gizmoUseSRT = true
app.viewport.grid.scale = 1.0
app.viewport.pickingMode = "kind:model.ALL"
app.viewport.camMoveVelocity = 0.05 # 5 m/s
app.viewport.gizmo.scale = 0.01 # scaled to meters
app.viewport.previewOnPeek = false
app.viewport.snapToSurface = false
app.viewport.displayOptions = 31887 # Disable Frame Rate and Resolution by default
app.window.uiStyle = "NvidiaDark"
app.primCreation.DefaultXformOpType = "Scale, Orient, Translate"
app.primCreation.DefaultXformOpOrder="xformOp:translate, xformOp:orient, xformOp:scale"
app.primCreation.typedDefaults.camera.clippingRange = [0.01, 10000000.0]
simulation.minFrameRate = 15
simulation.defaultMetersPerUnit = 1.0
omnigraph.updateToUsd = false
omnigraph.useSchemaPrims = true
omnigraph.disablePrimNodes = true
physics.updateToUsd = true
physics.updateVelocitiesToUsd = true
physics.useFastCache = false
physics.visualizationDisplayJoints = false
physics.visualizationSimulationOutput = false
omni.replicator.captureOnPlay = true
exts."omni.anim.navigation.core".navMesh.viewNavMesh = false
renderer.startupMessageDisplayed = true # hides the IOMMU popup window
# Make Detail panel visible by default
app.omniverse.content_browser.options_menu.show_details = true
app.omniverse.filepicker.options_menu.show_details = true
[settings.ngx]
enabled=true # Enable this for DLSS
# Isaac Sim Extensions
###############################
[dependencies]
"omni.isaac.core_archive" = {}
"omni.pip.compute" = {}
"omni.pip.cloud" = {}
"omni.isaac.ml_archive" = {}
"omni.isaac.urdf" = {}
"omni.isaac.mjcf" = {}
"omni.isaac.utils" = {}
"omni.isaac.range_sensor" = {}
"omni.isaac.dynamic_control" = {}
"omni.isaac.kit" = {}
"omni.isaac.core" = {}
"omni.isaac.core_nodes" = {}
"omni.isaac.cloner" = {}
"omni.isaac.cortex" = {}
"omni.isaac.cortex.sample_behaviors" = {}
"omni.isaac.dofbot" = {}
"omni.isaac.surface_gripper" = {}
# "omni.kit.property.isaac" = {}
"omni.isaac.scene_blox" = {}
"omni.isaac.sensor" = {}
"omni.isaac.debug_draw" = {}
"omni.isaac.gym" = {}
"omni.isaac.franka" = {}
"omni.isaac.manipulators" = {}
"omni.isaac.quadruped" = {}
"omni.isaac.wheeled_robots" = {}
"omni.isaac.lula" = {}
"omni.isaac.motion_generation" = {}
"omni.isaac.universal_robots" = {}
"omni.isaac.occupancy_map" = {}
"omni.replicator.isaac" = {}
"omni.kit.loop-isaac" = {}
#linux only extensions
[dependencies."filter:platform"."linux-x86_64"]
# "omni.isaac.ocs2" = {}
# Non Isaac Sim Extensions
######################
[dependencies]
"omni.syntheticdata" = {}
"semantics.schema.editor" = {}
"semantics.schema.property" = {}
"omni.replicator.core" = {}
"omni.replicator.replicator_yaml" = {}
"omni.replicator.composer" = {}
"omni.importer.mjcf" = {}
"omni.importer.urdf" = {}

View File

@ -46,8 +46,8 @@ class VisionSensor(BaseSensor):
Args:
relative_prim_path (str): Scene-local prim path of the Sensor to encapsulate or create.
name (str): Name for the object. Names need to be unique per scene.
modalities (str or list of str): Modality(s) supported by this sensor. Default is "all", which corresponds
to all modalities being used. Otherwise, valid options should be part of cls.all_modalities.
modalities (str or list of str): Modality(s) supported by this sensor. Default is "rgb".
Otherwise, valid options should be part of cls.all_modalities.
For this vision sensor, this includes any of:
{rgb, depth, depth_linear, normal, seg_semantic, seg_instance, flow, bbox_2d_tight,
bbox_2d_loose, bbox_3d, camera}
@ -113,7 +113,7 @@ class VisionSensor(BaseSensor):
self,
relative_prim_path,
name,
modalities="all",
modalities=["rgb"],
enabled=True,
noise=None,
load_config=None,

View File

@ -35,7 +35,6 @@ from omnigibson.utils.constants import LightingMode
from omnigibson.utils.python_utils import Serializable
from omnigibson.utils.python_utils import clear as clear_python_utils
from omnigibson.utils.python_utils import create_object_from_init_info
from omnigibson.utils.sim_utils import meets_minimum_isaac_version
from omnigibson.utils.ui_utils import (
CameraMover,
create_module_logger,
@ -70,6 +69,11 @@ m.OBJECT_GRAVEYARD_POS = (100.0, 100.0, 100.0)
m.SCENE_MARGIN = 10.0
m.INITIAL_SCENE_PRIM_Z_OFFSET = -100.0
m.KIT_FILES = {
(4, 0, 0): "omnigibson_4_0_0.kit",
(2023, 1, 1): "omnigibson_2023_1_1.kit",
}
# Helper functions for starting omnigibson
def print_save_usd_warning(_):
@ -102,27 +106,28 @@ def _launch_app():
# sys.argv.append("--/log/outputStreamLevel=error")
warnings.simplefilter("ignore", category=NumbaPerformanceWarning)
# Copy the OmniGibson kit file to the Isaac Sim apps directory. This is necessary because the Isaac Sim app
# expects the extensions to be reachable in the parent directory of the kit file. We copy on every launch to
# ensure that the kit file is always up to date.
assert "EXP_PATH" in os.environ, "The EXP_PATH variable is not set. Are you in an Isaac Sim installed environment?"
kit_file = Path(__file__).parent / "omnigibson.kit"
kit_file_target = Path(os.environ["EXP_PATH"]) / "omnigibson.kit"
try:
shutil.copy(kit_file, kit_file_target)
except Exception as e:
raise e from ValueError("Failed to copy omnigibson.kit to Isaac Sim apps directory.")
launch_context = nullcontext if gm.DEBUG else suppress_omni_log
# First obtain the Isaac Sim version
version_file_path = os.path.join(os.environ["ISAAC_PATH"], "VERSION")
assert os.path.exists(version_file_path), f"Isaac Sim version file not found at {version_file_path}"
with open(version_file_path, "r") as file:
version_content = file.read().strip()
isaac_version = version_content.split("-")[0]
assert meets_minimum_isaac_version(
"2023.1.1", current_version=isaac_version
), "This version of OmniGibson supports Isaac Sim 2023.1.1 and above. Please update Isaac Sim."
isaac_version_str = version_content.split("-")[0]
isaac_version_tuple = tuple(map(int, isaac_version_str.split(".")[:3]))
assert isaac_version_tuple in m.KIT_FILES, f"Isaac Sim version must be one of {list(m.KIT_FILES.keys())}"
kit_file_name = m.KIT_FILES[isaac_version_tuple]
# Copy the OmniGibson kit file to the Isaac Sim apps directory. This is necessary because the Isaac Sim app
# expects the extensions to be reachable in the parent directory of the kit file. We copy on every launch to
# ensure that the kit file is always up to date.
assert "EXP_PATH" in os.environ, "The EXP_PATH variable is not set. Are you in an Isaac Sim installed environment?"
kit_file = Path(__file__).parent / kit_file_name
kit_file_target = Path(os.environ["EXP_PATH"]) / kit_file_name
try:
shutil.copy(kit_file, kit_file_target)
except Exception as e:
raise e from ValueError(f"Failed to copy {kit_file_name} to Isaac Sim apps directory.")
launch_context = nullcontext if gm.DEBUG else suppress_omni_log
with launch_context(None):
app = lazy.omni.isaac.kit.SimulationApp(config_kwargs, experience=str(kit_file_target.resolve(strict=True)))

View File

@ -132,3 +132,23 @@ class GymObservable(metaclass=ABCMeta):
log.debug(f"Loaded obs space dictionary for: {self.__class__.__name__}")
return self.observation_space
def maxdim(space):
"""
Helper function to get the maximum dimension of a gym space
Args:
space (gym.spaces.Space): Gym space to get the maximum dimension of
Returns:
int: Maximum dimension of the gym space
"""
if isinstance(space, (gym.spaces.Dict, gym.spaces.Tuple)):
return sum([maxdim(s) for s in space.spaces.values()])
elif isinstance(space, (gym.spaces.Box, gym.spaces.Discrete, gym.spaces.MultiDiscrete, gym.spaces.MultiBinary)):
return gym.spaces.utils.flatdim(space)
elif isinstance(space, (gym.spaces.Sequence, gym.spaces.Graph)):
return float("inf")
else:
raise ValueError(f"Unsupported gym space type: {type(space)}")