Merge GBuffer features. (#5960)

* Add Misc/GBuffer

* Add EnqueueRenderSceneImmediateWithGBuffer.

* Modified ASceneCaptureSensor to allow gbuffer recording.

* Minor GBuffer progress.

* More GBuffer changes.

* Removed unnecessary files.

* Add FRHITexture* functions to FPixelReader + temporarily disabled non-gbuffer rendering in ASceneCaptureSensor.

* Add missing FPixelReader functions.

* Minor changes (Switching to Windows).

* Remove FRHITexture functions from FPixelReader, added the file ImageUtil.

* Remove FRHITexture functions from FPixelReader, added the file ImageUtil. (2)

* Added API to listen GBuffer data.

* Added gbuffer serializer classes

* Temporarily remove ViewRect hack.

* Add USceneCaptureComponent* derived classes.

* Disable USceneCaptureComponent*_CARLA and add initial FRHIGPUTextureReadBack-based code.

* Fix and re-enable custom SceneCaptureComponents.

* Fully switch to FRHIGPUTextureReadback.

* Remove unnecessary call to FlushRenderingCommands.

* Minor API changes.

* Add support for PF_DepthStencil in ImageUtil.

* More API progress...

* More API progress... (2)

* Removed testing code.

* Minor changes for testing.

* GBuffer API fixes.

* Improve GBuffer capture code.

* Fixed SceneDepth transfer issues and added SceneStencil, CustomDepth and CustomStencil to the GBuffer capture.

* Fix compilation error due to the usage of C++17 features.

* Removed major memory leak and added manual_control_gbuffer.py.

* Fixed a silly mistake.

* Minor changes to manual_control_gbuffer and SceneCaptureSensor.

* Fix compilation error on some versions of Ubuntu.

* Disable TAA when reading GBuffers to avoid jitter.

* Improve memory usage.

* Progress towards automatically detecting when a GBuffer stream is unused.

* Fix includes in SceneCaptureSensor + minor change in manual_control_gbuffer.py

* Progress on automatically detecting which GBuffers aren't needed.

* Remove unneeded __declspec.

* Revert ASensor changes + fix tutorial_gbuffer.py

* Update CHANGELOG.md

* Apply requested changes for the PR, add gitignore for the file OptionalModules.ini and add a GBufferTextureID enum to the Python API.

* Remove OptionalModules.ini.

* Fix indentation.

* Fix indentation (2).

* Fix indentation (3).

* Add documentation and more indentation fixes.

* Remove commented includes.

* Add missing line break.

* Fix memory leak + remove unneeded files.

* Add .uproject again, fix EngineAssociation.

* Remove unneeded ENQUEUE_RENDER_COMMAND.

* Fix manual_control_gbuffer.py.

* Add `stop_gbuffer` to the Python API.

* Minor fixes.

* Fix performance bug.

Previously, when a client requested a gbuffer that is unused it would stay open, even after stopping it explicitly. This commit fixes this issue.

* Fix indentation.

* Add missing braces, more indentation fixes and simplify some of the code.

* Update sensor.yml docs.

* Update docs.

* Remove unnecessary UE_Log + changed one verbosity level.

Co-authored-by: Axel <axellopez92@outlook.com>
Co-authored-by: Axel1092 <35765780+Axel1092@users.noreply.github.com>
This commit is contained in:
Marcel Pi 2022-11-29 11:24:26 +01:00 committed by GitHub
parent dcda50140a
commit a152863b11
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
42 changed files with 3428 additions and 590 deletions

View File

@ -48,6 +48,8 @@
* Added post process effects for rainy and dusty weathers.
* Switched data type of the dust storm weather parameter from bool to float.
* Check for the version of the installed Clang compiler during build.
* Added API function to get direct access to the GBuffer textures of a sensor:
- `listen_to_gbuffer`: to set a callback for a specific GBuffer texture
## CARLA 0.9.13

View File

@ -83,6 +83,32 @@ Check out the [introduction to blueprints](core_actors.md).
- `toe` (_Float_) <sub>_- Modifiable_</sub>
- `use_log` (_Bool_) <sub>_- Modifiable_</sub>
- `white_clip` (_Float_) <sub>_- Modifiable_</sub>
- **<font color="#498efc">sensor.camera.instance_segmentation</font>**
- **Attributes:**
- `fov` (_Float_) <sub>_- Modifiable_</sub>
- `image_size_x` (_Int_) <sub>_- Modifiable_</sub>
- `image_size_y` (_Int_) <sub>_- Modifiable_</sub>
- `lens_circle_falloff` (_Float_) <sub>_- Modifiable_</sub>
- `lens_circle_multiplier` (_Float_) <sub>_- Modifiable_</sub>
- `lens_k` (_Float_) <sub>_- Modifiable_</sub>
- `lens_kcube` (_Float_) <sub>_- Modifiable_</sub>
- `lens_x_size` (_Float_) <sub>_- Modifiable_</sub>
- `lens_y_size` (_Float_) <sub>_- Modifiable_</sub>
- `role_name` (_String_) <sub>_- Modifiable_</sub>
- `sensor_tick` (_Float_) <sub>_- Modifiable_</sub>
- **<font color="#498efc">sensor.camera.normals</font>**
- **Attributes:**
- `fov` (_Float_) <sub>_- Modifiable_</sub>
- `image_size_x` (_Int_) <sub>_- Modifiable_</sub>
- `image_size_y` (_Int_) <sub>_- Modifiable_</sub>
- `lens_circle_falloff` (_Float_) <sub>_- Modifiable_</sub>
- `lens_circle_multiplier` (_Float_) <sub>_- Modifiable_</sub>
- `lens_k` (_Float_) <sub>_- Modifiable_</sub>
- `lens_kcube` (_Float_) <sub>_- Modifiable_</sub>
- `lens_x_size` (_Float_) <sub>_- Modifiable_</sub>
- `lens_y_size` (_Float_) <sub>_- Modifiable_</sub>
- `role_name` (_String_) <sub>_- Modifiable_</sub>
- `sensor_tick` (_Float_) <sub>_- Modifiable_</sub>
- **<font color="#498efc">sensor.camera.optical_flow</font>**
- **Attributes:**
- `fov` (_Float_) <sub>_- Modifiable_</sub>
@ -306,6 +332,10 @@ Check out the [introduction to blueprints](core_actors.md).
- **Attributes:**
- `role_name` (_String_) <sub>_- Modifiable_</sub>
- `size` (_String_)
- **<font color="#498efc">static.prop.busstoplb</font>**
- **Attributes:**
- `role_name` (_String_) <sub>_- Modifiable_</sub>
- `size` (_String_)
- **<font color="#498efc">static.prop.calibrator</font>**
- **Attributes:**
- `role_name` (_String_) <sub>_- Modifiable_</sub>
@ -366,6 +396,10 @@ Check out the [introduction to blueprints](core_actors.md).
- **Attributes:**
- `role_name` (_String_) <sub>_- Modifiable_</sub>
- `size` (_String_)
- **<font color="#498efc">static.prop.foodcart</font>**
- **Attributes:**
- `role_name` (_String_) <sub>_- Modifiable_</sub>
- `size` (_String_)
- **<font color="#498efc">static.prop.fountain</font>**
- **Attributes:**
- `role_name` (_String_) <sub>_- Modifiable_</sub>
@ -410,6 +444,14 @@ Check out the [introduction to blueprints](core_actors.md).
- **Attributes:**
- `role_name` (_String_) <sub>_- Modifiable_</sub>
- `size` (_String_)
- **<font color="#498efc">static.prop.haybale</font>**
- **Attributes:**
- `role_name` (_String_) <sub>_- Modifiable_</sub>
- `size` (_String_)
- **<font color="#498efc">static.prop.haybalelb</font>**
- **Attributes:**
- `role_name` (_String_) <sub>_- Modifiable_</sub>
- `size` (_String_)
- **<font color="#498efc">static.prop.ironplank</font>**
- **Attributes:**
- `role_name` (_String_) <sub>_- Modifiable_</sub>
@ -428,7 +470,10 @@ Check out the [introduction to blueprints](core_actors.md).
- `size` (_String_)
- **<font color="#498efc">static.prop.mesh</font>**
- **Attributes:**
- `mass` (_Float_) <sub>_- Modifiable_</sub>
- `mesh_path` (_String_) <sub>_- Modifiable_</sub>
- `role_name` (_String_) <sub>_- Modifiable_</sub>
- `scale` (_Float_) <sub>_- Modifiable_</sub>
- **<font color="#498efc">static.prop.mobile</font>**
- **Attributes:**
- `role_name` (_String_) <sub>_- Modifiable_</sub>
@ -437,14 +482,6 @@ Check out the [introduction to blueprints](core_actors.md).
- **Attributes:**
- `role_name` (_String_) <sub>_- Modifiable_</sub>
- `size` (_String_)
- **<font color="#498efc">static.prop.omri-0</font>**
- **Attributes:**
- `role_name` (_String_) <sub>_- Modifiable_</sub>
- `size` (_String_)
- **<font color="#498efc">static.prop.omri-1</font>**
- **Attributes:**
- `role_name` (_String_) <sub>_- Modifiable_</sub>
- `size` (_String_)
- **<font color="#498efc">static.prop.pergola</font>**
- **Attributes:**
- `role_name` (_String_) <sub>_- Modifiable_</sub>
@ -597,6 +634,14 @@ Check out the [introduction to blueprints](core_actors.md).
- **Attributes:**
- `role_name` (_String_) <sub>_- Modifiable_</sub>
- `size` (_String_)
- **<font color="#498efc">static.prop.warningaccident</font>**
- **Attributes:**
- `role_name` (_String_) <sub>_- Modifiable_</sub>
- `size` (_String_)
- **<font color="#498efc">static.prop.warningconstruction</font>**
- **Attributes:**
- `role_name` (_String_) <sub>_- Modifiable_</sub>
- `size` (_String_)
- **<font color="#498efc">static.prop.wateringcan</font>**
- **Attributes:**
- `role_name` (_String_) <sub>_- Modifiable_</sub>
@ -612,480 +657,518 @@ Check out the [introduction to blueprints](core_actors.md).
### vehicle
- **<font color="#498efc">vehicle.audi.a2</font>**
- **Attributes:**
- `base_type` (_String_)
- `color` (_RGBColor_) <sub>_- Modifiable_</sub>
- `generation` (_Int_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `base_type` (_String_)
- `special_type` (_String_)
- `has_dynamic_doors` (_Bool_)
- `has_lights` (_Bool_)
- `base_type` (_String_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `role_name` (_String_) <sub>_- Modifiable_</sub>
- `special_type` (_String_)
- `sticky_control` (_Bool_) <sub>_- Modifiable_</sub>
- `terramechanics` (_Bool_) <sub>_- Modifiable_</sub>
- **<font color="#498efc">vehicle.audi.etron</font>**
- **Attributes:**
- `base_type` (_String_)
- `color` (_RGBColor_) <sub>_- Modifiable_</sub>
- `generation` (_Int_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `base_type` (_String_)
- `special_type` (_String_)
- `has_dynamic_doors` (_Bool_)
- `has_lights` (_Bool_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `role_name` (_String_) <sub>_- Modifiable_</sub>
- `special_type` (_String_)
- `sticky_control` (_Bool_) <sub>_- Modifiable_</sub>
- `terramechanics` (_Bool_) <sub>_- Modifiable_</sub>
- **<font color="#498efc">vehicle.audi.tt</font>**
- **Attributes:**
- `base_type` (_String_)
- `color` (_RGBColor_) <sub>_- Modifiable_</sub>
- `generation` (_Int_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `base_type` (_String_)
- `special_type` (_String_)
- `has_dynamic_doors` (_Bool_)
- `has_lights` (_Bool_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `role_name` (_String_) <sub>_- Modifiable_</sub>
- `special_type` (_String_)
- `sticky_control` (_Bool_) <sub>_- Modifiable_</sub>
- `terramechanics` (_Bool_) <sub>_- Modifiable_</sub>
- **<font color="#498efc">vehicle.bh.crossbike</font>**
- **Attributes:**
- `base_type` (_String_)
- `color` (_RGBColor_) <sub>_- Modifiable_</sub>
- `driver_id` (_Int_) <sub>_- Modifiable_</sub>
- `generation` (_Int_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `base_type` (_String_)
- `special_type` (_String_)
- `has_dynamic_doors` (_Bool_)
- `has_lights` (_Bool_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `role_name` (_String_) <sub>_- Modifiable_</sub>
- `special_type` (_String_)
- `sticky_control` (_Bool_) <sub>_- Modifiable_</sub>
- `terramechanics` (_Bool_) <sub>_- Modifiable_</sub>
- **<font color="#498efc">vehicle.bmw.grandtourer</font>**
- **Attributes:**
- `base_type` (_String_)
- `color` (_RGBColor_) <sub>_- Modifiable_</sub>
- `generation` (_Int_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `base_type` (_String_)
- `special_type` (_String_)
- `has_dynamic_doors` (_Bool_)
- `has_lights` (_Bool_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `role_name` (_String_) <sub>_- Modifiable_</sub>
- `special_type` (_String_)
- `sticky_control` (_Bool_) <sub>_- Modifiable_</sub>
- `terramechanics` (_Bool_) <sub>_- Modifiable_</sub>
- **<font color="#498efc">vehicle.carlamotors.carlacola</font>**
- **Attributes:**
- `base_type` (_String_)
- `color` (_RGBColor_) <sub>_- Modifiable_</sub>
- `generation` (_Int_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `base_type` (_String_)
- `special_type` (_String_)
- `has_dynamic_doors` (_Bool_)
- `has_lights` (_Bool_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `role_name` (_String_) <sub>_- Modifiable_</sub>
- `special_type` (_String_)
- `sticky_control` (_Bool_) <sub>_- Modifiable_</sub>
- `terramechanics` (_Bool_) <sub>_- Modifiable_</sub>
- **<font color="#498efc">vehicle.carlamotors.firetruck</font>**
- **Attributes:**
- `base_type` (_String_)
- `color` (_RGBColor_) <sub>_- Modifiable_</sub>
- `generation` (_Int_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `base_type` (_String_)
- `special_type` (_String_)
- `has_dynamic_doors` (_Bool_)
- `has_lights` (_Bool_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `role_name` (_String_) <sub>_- Modifiable_</sub>
- `special_type` (_String_)
- `sticky_control` (_Bool_) <sub>_- Modifiable_</sub>
- `terramechanics` (_Bool_) <sub>_- Modifiable_</sub>
- **<font color="#498efc">vehicle.chevrolet.impala</font>**
- **Attributes:**
- `base_type` (_String_)
- `color` (_RGBColor_) <sub>_- Modifiable_</sub>
- `generation` (_Int_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `base_type` (_String_)
- `special_type` (_String_)
- `has_dynamic_doors` (_Bool_)
- `has_lights` (_Bool_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `role_name` (_String_) <sub>_- Modifiable_</sub>
- `special_type` (_String_)
- `sticky_control` (_Bool_) <sub>_- Modifiable_</sub>
- `terramechanics` (_Bool_) <sub>_- Modifiable_</sub>
- **<font color="#498efc">vehicle.citroen.c3</font>**
- **Attributes:**
- `base_type` (_String_)
- `color` (_RGBColor_) <sub>_- Modifiable_</sub>
- `generation` (_Int_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `base_type` (_String_)
- `special_type` (_String_)
- `has_dynamic_doors` (_Bool_)
- `has_lights` (_Bool_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `role_name` (_String_) <sub>_- Modifiable_</sub>
- `special_type` (_String_)
- `sticky_control` (_Bool_) <sub>_- Modifiable_</sub>
- `terramechanics` (_Bool_) <sub>_- Modifiable_</sub>
- **<font color="#498efc">vehicle.diamondback.century</font>**
- **Attributes:**
- `base_type` (_String_)
- `color` (_RGBColor_) <sub>_- Modifiable_</sub>
- `driver_id` (_Int_) <sub>_- Modifiable_</sub>
- `generation` (_Int_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `base_type` (_String_)
- `special_type` (_String_)
- `has_dynamic_doors` (_Bool_)
- `has_lights` (_Bool_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `role_name` (_String_) <sub>_- Modifiable_</sub>
- `special_type` (_String_)
- `sticky_control` (_Bool_) <sub>_- Modifiable_</sub>
- `terramechanics` (_Bool_) <sub>_- Modifiable_</sub>
- **<font color="#498efc">vehicle.dodge.charger_2020</font>**
- **Attributes:**
- `base_type` (_String_)
- `color` (_RGBColor_) <sub>_- Modifiable_</sub>
- `generation` (_Int_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `base_type` (_String_)
- `special_type` (_String_)
- `has_dynamic_doors` (_Bool_)
- `has_lights` (_Bool_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `role_name` (_String_) <sub>_- Modifiable_</sub>
- `special_type` (_String_)
- `sticky_control` (_Bool_) <sub>_- Modifiable_</sub>
- `terramechanics` (_Bool_) <sub>_- Modifiable_</sub>
- **<font color="#498efc">vehicle.dodge.charger_police</font>**
- **Attributes:**
- `base_type` (_String_)
- `color` (_RGBColor_) <sub>_- Modifiable_</sub>
- `generation` (_Int_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `base_type` (_String_)
- `special_type` (_String_)
- `has_dynamic_doors` (_Bool_)
- `has_lights` (_Bool_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `role_name` (_String_) <sub>_- Modifiable_</sub>
- `special_type` (_String_)
- `sticky_control` (_Bool_) <sub>_- Modifiable_</sub>
- `terramechanics` (_Bool_) <sub>_- Modifiable_</sub>
- **<font color="#498efc">vehicle.dodge.charger_police_2020</font>**
- **Attributes:**
- `base_type` (_String_)
- `color` (_RGBColor_) <sub>_- Modifiable_</sub>
- `generation` (_Int_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `base_type` (_String_)
- `special_type` (_String_)
- `has_dynamic_doors` (_Bool_)
- `has_lights` (_Bool_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `role_name` (_String_) <sub>_- Modifiable_</sub>
- `special_type` (_String_)
- `sticky_control` (_Bool_) <sub>_- Modifiable_</sub>
- `terramechanics` (_Bool_) <sub>_- Modifiable_</sub>
- **<font color="#498efc">vehicle.ford.ambulance</font>**
- **Attributes:**
- `base_type` (_String_)
- `color` (_RGBColor_) <sub>_- Modifiable_</sub>
- `generation` (_Int_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `base_type` (_String_)
- `special_type` (_String_)
- `has_dynamic_doors` (_Bool_)
- `has_lights` (_Bool_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `role_name` (_String_) <sub>_- Modifiable_</sub>
- `special_type` (_String_)
- `sticky_control` (_Bool_) <sub>_- Modifiable_</sub>
- `terramechanics` (_Bool_) <sub>_- Modifiable_</sub>
- **<font color="#498efc">vehicle.ford.crown</font>**
- **Attributes:**
- `base_type` (_String_)
- `color` (_RGBColor_) <sub>_- Modifiable_</sub>
- `generation` (_Int_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `base_type` (_String_)
- `special_type` (_String_)
- `has_dynamic_doors` (_Bool_)
- `has_lights` (_Bool_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `role_name` (_String_) <sub>_- Modifiable_</sub>
- `special_type` (_String_)
- `sticky_control` (_Bool_) <sub>_- Modifiable_</sub>
- `terramechanics` (_Bool_) <sub>_- Modifiable_</sub>
- **<font color="#498efc">vehicle.ford.mustang</font>**
- **Attributes:**
- `base_type` (_String_)
- `color` (_RGBColor_) <sub>_- Modifiable_</sub>
- `generation` (_Int_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `base_type` (_String_)
- `special_type` (_String_)
- `has_dynamic_doors` (_Bool_)
- `has_lights` (_Bool_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `role_name` (_String_) <sub>_- Modifiable_</sub>
- `special_type` (_String_)
- `sticky_control` (_Bool_) <sub>_- Modifiable_</sub>
- `terramechanics` (_Bool_) <sub>_- Modifiable_</sub>
- **<font color="#498efc">vehicle.gazelle.omafiets</font>**
- **Attributes:**
- `base_type` (_String_)
- `color` (_RGBColor_) <sub>_- Modifiable_</sub>
- `driver_id` (_Int_) <sub>_- Modifiable_</sub>
- `generation` (_Int_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `base_type` (_String_)
- `special_type` (_String_)
- `has_dynamic_doors` (_Bool_)
- `has_lights` (_Bool_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `role_name` (_String_) <sub>_- Modifiable_</sub>
- `special_type` (_String_)
- `sticky_control` (_Bool_) <sub>_- Modifiable_</sub>
- `terramechanics` (_Bool_) <sub>_- Modifiable_</sub>
- **<font color="#498efc">vehicle.harley-davidson.low_rider</font>**
- **Attributes:**
- `base_type` (_String_)
- `color` (_RGBColor_) <sub>_- Modifiable_</sub>
- `driver_id` (_Int_) <sub>_- Modifiable_</sub>
- `generation` (_Int_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `base_type` (_String_)
- `special_type` (_String_)
- `has_dynamic_doors` (_Bool_)
- `has_lights` (_Bool_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `role_name` (_String_) <sub>_- Modifiable_</sub>
- `special_type` (_String_)
- `sticky_control` (_Bool_) <sub>_- Modifiable_</sub>
- `terramechanics` (_Bool_) <sub>_- Modifiable_</sub>
- **<font color="#498efc">vehicle.jeep.wrangler_rubicon</font>**
- **Attributes:**
- `base_type` (_String_)
- `color` (_RGBColor_) <sub>_- Modifiable_</sub>
- `generation` (_Int_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `base_type` (_String_)
- `special_type` (_String_)
- `has_dynamic_doors` (_Bool_)
- `has_lights` (_Bool_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `role_name` (_String_) <sub>_- Modifiable_</sub>
- `special_type` (_String_)
- `sticky_control` (_Bool_) <sub>_- Modifiable_</sub>
- `terramechanics` (_Bool_) <sub>_- Modifiable_</sub>
- **<font color="#498efc">vehicle.kawasaki.ninja</font>**
- **Attributes:**
- `base_type` (_String_)
- `color` (_RGBColor_) <sub>_- Modifiable_</sub>
- `driver_id` (_Int_) <sub>_- Modifiable_</sub>
- `generation` (_Int_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `base_type` (_String_)
- `special_type` (_String_)
- `has_dynamic_doors` (_Bool_)
- `has_lights` (_Bool_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `role_name` (_String_) <sub>_- Modifiable_</sub>
- `special_type` (_String_)
- `sticky_control` (_Bool_) <sub>_- Modifiable_</sub>
- `terramechanics` (_Bool_) <sub>_- Modifiable_</sub>
- **<font color="#498efc">vehicle.lincoln.mkz_2017</font>**
- **Attributes:**
- `base_type` (_String_)
- `color` (_RGBColor_) <sub>_- Modifiable_</sub>
- `generation` (_Int_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `base_type` (_String_)
- `special_type` (_String_)
- `has_dynamic_doors` (_Bool_)
- `has_lights` (_Bool_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `role_name` (_String_) <sub>_- Modifiable_</sub>
- `special_type` (_String_)
- `sticky_control` (_Bool_) <sub>_- Modifiable_</sub>
- `terramechanics` (_Bool_) <sub>_- Modifiable_</sub>
- **<font color="#498efc">vehicle.lincoln.mkz_2020</font>**
- **Attributes:**
- `base_type` (_String_)
- `color` (_RGBColor_) <sub>_- Modifiable_</sub>
- `driver_id` (_Int_) <sub>_- Modifiable_</sub>
- `generation` (_Int_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `base_type` (_String_)
- `special_type` (_String_)
- `has_dynamic_doors` (_Bool_)
- `has_lights` (_Bool_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `role_name` (_String_) <sub>_- Modifiable_</sub>
- `special_type` (_String_)
- `sticky_control` (_Bool_) <sub>_- Modifiable_</sub>
- `terramechanics` (_Bool_) <sub>_- Modifiable_</sub>
- **<font color="#498efc">vehicle.mercedes.coupe</font>**
- **Attributes:**
- `base_type` (_String_)
- `color` (_RGBColor_) <sub>_- Modifiable_</sub>
- `generation` (_Int_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `base_type` (_String_)
- `special_type` (_String_)
- `has_dynamic_doors` (_Bool_)
- `has_lights` (_Bool_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `role_name` (_String_) <sub>_- Modifiable_</sub>
- `special_type` (_String_)
- `sticky_control` (_Bool_) <sub>_- Modifiable_</sub>
- `terramechanics` (_Bool_) <sub>_- Modifiable_</sub>
- **<font color="#498efc">vehicle.mercedes.coupe_2020</font>**
- **Attributes:**
- `base_type` (_String_)
- `color` (_RGBColor_) <sub>_- Modifiable_</sub>
- `generation` (_Int_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `base_type` (_String_)
- `special_type` (_String_)
- `has_dynamic_doors` (_Bool_)
- `has_lights` (_Bool_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `role_name` (_String_) <sub>_- Modifiable_</sub>
- `special_type` (_String_)
- `sticky_control` (_Bool_) <sub>_- Modifiable_</sub>
- `terramechanics` (_Bool_) <sub>_- Modifiable_</sub>
- **<font color="#498efc">vehicle.mercedes.sprinter</font>**
- **Attributes:**
- `base_type` (_String_)
- `color` (_RGBColor_) <sub>_- Modifiable_</sub>
- `generation` (_Int_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `base_type` (_String_)
- `special_type` (_String_)
- `has_dynamic_doors` (_Bool_)
- `has_lights` (_Bool_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `role_name` (_String_) <sub>_- Modifiable_</sub>
- `special_type` (_String_)
- `sticky_control` (_Bool_) <sub>_- Modifiable_</sub>
- `terramechanics` (_Bool_) <sub>_- Modifiable_</sub>
- **<font color="#498efc">vehicle.micro.microlino</font>**
- **Attributes:**
- `base_type` (_String_)
- `color` (_RGBColor_) <sub>_- Modifiable_</sub>
- `generation` (_Int_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `base_type` (_String_)
- `special_type` (_String_)
- `has_dynamic_doors` (_Bool_)
- `has_lights` (_Bool_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `role_name` (_String_) <sub>_- Modifiable_</sub>
- `special_type` (_String_)
- `sticky_control` (_Bool_) <sub>_- Modifiable_</sub>
- `terramechanics` (_Bool_) <sub>_- Modifiable_</sub>
- **<font color="#498efc">vehicle.mini.cooper_s</font>**
- **Attributes:**
- `base_type` (_String_)
- `color` (_RGBColor_) <sub>_- Modifiable_</sub>
- `generation` (_Int_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `base_type` (_String_)
- `special_type` (_String_)
- `has_dynamic_doors` (_Bool_)
- `has_lights` (_Bool_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `role_name` (_String_) <sub>_- Modifiable_</sub>
- `special_type` (_String_)
- `sticky_control` (_Bool_) <sub>_- Modifiable_</sub>
- `terramechanics` (_Bool_) <sub>_- Modifiable_</sub>
- **<font color="#498efc">vehicle.mini.cooper_s_2021</font>**
- **Attributes:**
- `base_type` (_String_)
- `color` (_RGBColor_) <sub>_- Modifiable_</sub>
- `generation` (_Int_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `base_type` (_String_)
- `special_type` (_String_)
- `has_dynamic_doors` (_Bool_)
- `has_lights` (_Bool_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `role_name` (_String_) <sub>_- Modifiable_</sub>
- `special_type` (_String_)
- `sticky_control` (_Bool_) <sub>_- Modifiable_</sub>
- `terramechanics` (_Bool_) <sub>_- Modifiable_</sub>
- **<font color="#498efc">vehicle.nissan.micra</font>**
- **Attributes:**
- `base_type` (_String_)
- `color` (_RGBColor_) <sub>_- Modifiable_</sub>
- `generation` (_Int_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `base_type` (_String_)
- `special_type` (_String_)
- `has_dynamic_doors` (_Bool_)
- `has_lights` (_Bool_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `role_name` (_String_) <sub>_- Modifiable_</sub>
- `special_type` (_String_)
- `sticky_control` (_Bool_) <sub>_- Modifiable_</sub>
- `terramechanics` (_Bool_) <sub>_- Modifiable_</sub>
- **<font color="#498efc">vehicle.nissan.patrol</font>**
- **Attributes:**
- `base_type` (_String_)
- `color` (_RGBColor_) <sub>_- Modifiable_</sub>
- `generation` (_Int_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `base_type` (_String_)
- `special_type` (_String_)
- `has_dynamic_doors` (_Bool_)
- `has_lights` (_Bool_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `role_name` (_String_) <sub>_- Modifiable_</sub>
- `special_type` (_String_)
- `sticky_control` (_Bool_) <sub>_- Modifiable_</sub>
- `terramechanics` (_Bool_) <sub>_- Modifiable_</sub>
- **<font color="#498efc">vehicle.nissan.patrol_2021</font>**
- **Attributes:**
- `base_type` (_String_)
- `color` (_RGBColor_) <sub>_- Modifiable_</sub>
- `generation` (_Int_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `base_type` (_String_)
- `special_type` (_String_)
- `has_dynamic_doors` (_Bool_)
- `has_lights` (_Bool_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `role_name` (_String_) <sub>_- Modifiable_</sub>
- `special_type` (_String_)
- `sticky_control` (_Bool_) <sub>_- Modifiable_</sub>
- `terramechanics` (_Bool_) <sub>_- Modifiable_</sub>
- **<font color="#498efc">vehicle.seat.leon</font>**
- **Attributes:**
- `base_type` (_String_)
- `color` (_RGBColor_) <sub>_- Modifiable_</sub>
- `generation` (_Int_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `base_type` (_String_)
- `special_type` (_String_)
- `has_dynamic_doors` (_Bool_)
- `has_lights` (_Bool_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `role_name` (_String_) <sub>_- Modifiable_</sub>
- `special_type` (_String_)
- `sticky_control` (_Bool_) <sub>_- Modifiable_</sub>
- `terramechanics` (_Bool_) <sub>_- Modifiable_</sub>
- **<font color="#498efc">vehicle.tesla.cybertruck</font>**
- **Attributes:**
- `generation` (_Int_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `base_type` (_String_)
- `special_type` (_String_)
- `generation` (_Int_)
- `has_dynamic_doors` (_Bool_)
- `has_lights` (_Bool_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `role_name` (_String_) <sub>_- Modifiable_</sub>
- `special_type` (_String_)
- `sticky_control` (_Bool_) <sub>_- Modifiable_</sub>
- `terramechanics` (_Bool_) <sub>_- Modifiable_</sub>
- **<font color="#498efc">vehicle.tesla.model3</font>**
- **Attributes:**
- `base_type` (_String_)
- `color` (_RGBColor_) <sub>_- Modifiable_</sub>
- `generation` (_Int_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `base_type` (_String_)
- `special_type` (_String_)
- `has_dynamic_doors` (_Bool_)
- `has_lights` (_Bool_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `role_name` (_String_) <sub>_- Modifiable_</sub>
- `special_type` (_String_)
- `sticky_control` (_Bool_) <sub>_- Modifiable_</sub>
- `terramechanics` (_Bool_) <sub>_- Modifiable_</sub>
- **<font color="#498efc">vehicle.toyota.prius</font>**
- **Attributes:**
- `base_type` (_String_)
- `color` (_RGBColor_) <sub>_- Modifiable_</sub>
- `generation` (_Int_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `base_type` (_String_)
- `special_type` (_String_)
- `has_dynamic_doors` (_Bool_)
- `has_lights` (_Bool_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `role_name` (_String_) <sub>_- Modifiable_</sub>
- `special_type` (_String_)
- `sticky_control` (_Bool_) <sub>_- Modifiable_</sub>
- `terramechanics` (_Bool_) <sub>_- Modifiable_</sub>
- **<font color="#498efc">vehicle.vespa.zx125</font>**
- **Attributes:**
- `base_type` (_String_)
- `color` (_RGBColor_) <sub>_- Modifiable_</sub>
- `driver_id` (_Int_) <sub>_- Modifiable_</sub>
- `generation` (_Int_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `base_type` (_String_)
- `special_type` (_String_)
- `has_dynamic_doors` (_Bool_)
- `has_lights` (_Bool_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `role_name` (_String_) <sub>_- Modifiable_</sub>
- `special_type` (_String_)
- `sticky_control` (_Bool_) <sub>_- Modifiable_</sub>
- `terramechanics` (_Bool_) <sub>_- Modifiable_</sub>
- **<font color="#498efc">vehicle.volkswagen.t2</font>**
- **Attributes:**
- `base_type` (_String_)
- `color` (_RGBColor_) <sub>_- Modifiable_</sub>
- `generation` (_Int_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `base_type` (_String_)
- `special_type` (_String_)
- `has_dynamic_doors` (_Bool_)
- `has_lights` (_Bool_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `role_name` (_String_) <sub>_- Modifiable_</sub>
- `special_type` (_String_)
- `sticky_control` (_Bool_) <sub>_- Modifiable_</sub>
- `terramechanics` (_Bool_) <sub>_- Modifiable_</sub>
- **<font color="#498efc">vehicle.volkswagen.t2_2021</font>**
- **Attributes:**
- `base_type` (_String_)
- `color` (_RGBColor_) <sub>_- Modifiable_</sub>
- `generation` (_Int_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `base_type` (_String_)
- `special_type` (_String_)
- `has_dynamic_doors` (_Bool_)
- `has_lights` (_Bool_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `role_name` (_String_) <sub>_- Modifiable_</sub>
- `special_type` (_String_)
- `sticky_control` (_Bool_) <sub>_- Modifiable_</sub>
- `terramechanics` (_Bool_) <sub>_- Modifiable_</sub>
- **<font color="#498efc">vehicle.yamaha.yzf</font>**
- **Attributes:**
- `base_type` (_String_)
- `color` (_RGBColor_) <sub>_- Modifiable_</sub>
- `driver_id` (_Int_) <sub>_- Modifiable_</sub>
- `generation` (_Int_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `base_type` (_String_)
- `special_type` (_String_)
- `has_dynamic_doors` (_Bool_)
- `has_lights` (_Bool_)
- `number_of_wheels` (_Int_)
- `object_type` (_String_)
- `role_name` (_String_) <sub>_- Modifiable_</sub>
- `special_type` (_String_)
- `sticky_control` (_Bool_) <sub>_- Modifiable_</sub>
- `terramechanics` (_Bool_) <sub>_- Modifiable_</sub>
### walker
- **<font color="#498efc">walker.pedestrian.0001</font>**
@ -1472,3 +1555,11 @@ Check out the [introduction to blueprints](core_actors.md).
- `is_invincible` (_Bool_) <sub>_- Modifiable_</sub>
- `role_name` (_String_) <sub>_- Modifiable_</sub>
- `speed` (_Float_) <sub>_- Modifiable_</sub>
- **<font color="#498efc">walker.pedestrian.0049</font>**
- **Attributes:**
- `age` (_String_)
- `gender` (_String_)
- `generation` (_Int_)
- `is_invincible` (_Bool_) <sub>_- Modifiable_</sub>
- `role_name` (_String_) <sub>_- Modifiable_</sub>
- `speed` (_Float_) <sub>_- Modifiable_</sub>

View File

@ -772,6 +772,57 @@ Initializes a color, black by default.
---
## carla.GBufferTextureID<a name="carla.GBufferTextureID"></a>
Defines the identifiers of each GBuffer texture (See the method `[carla.Sensor.listen_to_gbuffer](#carla.Sensor.listen_to_gbuffer)`).
### Instance Variables
- <a name="carla.GBufferTextureID.SceneColor"></a>**<font color="#f8805a">SceneColor</font>**
The texture "SceneColor" contains the final color of the image.
- <a name="carla.GBufferTextureID.SceneDepth"></a>**<font color="#f8805a">SceneDepth</font>**
The texture "SceneDepth" contains the depth buffer - linear in world units.
- <a name="carla.GBufferTextureID.SceneStencil"></a>**<font color="#f8805a">SceneStencil</font>**
The texture "SceneStencil" contains the stencil buffer.
- <a name="carla.GBufferTextureID.GBufferA"></a>**<font color="#f8805a">GBufferA</font>**
The texture "GBufferA" contains the world-space normal vectors in the RGB channels. The alpha channel contains "per-object data".
- <a name="carla.GBufferTextureID.GBufferB"></a>**<font color="#f8805a">GBufferB</font>**
The texture "GBufferB" contains the metallic, specular and roughness in the RGB channels, respectively. The alpha channel contains a mask where the lower 4 bits indicate the shading model and the upper 4 bits contain the selective output mask.
- <a name="carla.GBufferTextureID.GBufferC"></a>**<font color="#f8805a">GBufferC</font>**
The texture "GBufferC" contains the diffuse color in the RGB channels, with the indirect irradiance in the alpha channel.<br> If static lightning is not allowed, the alpha channel will contain the ambient occlusion instead.
- <a name="carla.GBufferTextureID.GBufferD"></a>**<font color="#f8805a">GBufferD</font>**
The contents of the "GBufferD" varies depending on the rendered object's material shading model (GBufferB):<br>
- MSM_Subsurface (2), MSM_PreintegratedSkin (3), MSM_TwoSidedFoliage (6):<br>
RGB: Subsurface color.<br>
A: Opacity.<br>
- MSM_ClearCoat (4):<br>
R: Clear coat.<br>
G: Roughness.<br>
- MSM_SubsurfaceProfile (5):<br>
RGB: Subsurface profile.<br>
- MSM_Hair (7):<br>
RG: World normal.<br>
B: Backlit value.<br>
- MSM_Cloth (8):<br>
RGB: Subsurface color.<br>
A: Cloth value.<br>
- MSM_Eye (9):<br>
RG: Eye tangent.<br>
B: Iris mask.<br>
A: Iris distance.
- <a name="carla.GBufferTextureID.GBufferE"></a>**<font color="#f8805a">GBufferE</font>**
The texture "GBufferE" contains the precomputed shadow factors in the RGBA channels. This texture is unavailable if the selective output mask (GBufferB) does not have its 4th bit set.
- <a name="carla.GBufferTextureID.GBufferF"></a>**<font color="#f8805a">GBufferF</font>**
The texture "GBufferF" contains the world-space tangent in the RGB channels and the anisotropy in the alpha channel. This texture is unavailable if the selective output mask (GBufferB) does not have its 5th bit set.
- <a name="carla.GBufferTextureID.Velocity"></a>**<font color="#f8805a">Velocity</font>**
The texture "Velocity" contains the screen-space velocity of the scene objects.
- <a name="carla.GBufferTextureID.SSAO"></a>**<font color="#f8805a">SSAO</font>**
The texture "SSAO" contains the screen-space ambient occlusion texture.
- <a name="carla.GBufferTextureID.CustomDepth"></a>**<font color="#f8805a">CustomDepth</font>**
The texture "CustomDepth" contains the Unreal Engine custom depth data.
- <a name="carla.GBufferTextureID.CustomStencil"></a>**<font color="#f8805a">CustomStencil</font>**
The texture "CustomStencil" contains the Unreal Engine custom stencil data.
---
## carla.GearPhysicsControl<a name="carla.GearPhysicsControl"></a>
Class that provides access to vehicle transmission details by defining a gear and when to run on it. This will be later used by [carla.VehiclePhysicsControl](#carla.VehiclePhysicsControl) to help simulate physics.
@ -2114,8 +2165,17 @@ When <b>True</b> the sensor will be waiting for data.
The function the sensor will be calling to every time a new measurement is received. This function needs for an argument containing an object type [carla.SensorData](#carla.SensorData) to work with.
- **Parameters:**
- `callback` (_function_) - The called function with one argument containing the sensor data.
- <a name="carla.Sensor.listen_to_gbuffer"></a>**<font color="#7fb800">listen_to_gbuffer</font>**(<font color="#00a6ed">**self**</font>, <font color="#00a6ed">**gbuffer_id**</font>, <font color="#00a6ed">**callback**</font>)
The function the sensor will be calling to every time the desired GBuffer texture is received.<br> This function needs for an argument containing an object type [carla.SensorData](#carla.SensorData) to work with.
- **Parameters:**
- `gbuffer_id` (_[carla.GBufferTextureID](#carla.GBufferTextureID)_) - The ID of the desired Unreal Engine GBuffer texture.
- `callback` (_function_) - The called function with one argument containing the received GBuffer texture.
- <a name="carla.Sensor.stop"></a>**<font color="#7fb800">stop</font>**(<font color="#00a6ed">**self**</font>)
Commands the sensor to stop listening for data.
- <a name="carla.Sensor.stop_gbuffer"></a>**<font color="#7fb800">stop_gbuffer</font>**(<font color="#00a6ed">**self**</font>, <font color="#00a6ed">**gbuffer_id**</font>)
Commands the sensor to stop listening for the specified GBuffer texture.
- **Parameters:**
- `gbuffer_id` (_[carla.GBufferTextureID](#carla.GBufferTextureID)_) - The ID of the Unreal Engine GBuffer texture.
##### Dunder methods
- <a name="carla.Sensor.__str__"></a>**<font color="#7fb800">\__str__</font>**(<font color="#00a6ed">**self**</font>)
@ -3977,384 +4037,6 @@ document.getElementById("snipets-container").innerHTML = null;
}
</script>
<div id ="carla.World.enable_environment_objects-snipet" style="display: none;">
<p class="SnipetFont">
Snippet for carla.World.enable_environment_objects
</p>
<div id="carla.World.enable_environment_objects-code" class="SnipetContent">
```py
# This recipe turn visibility off and on for two specifc buildings on the map
# Get the buildings in the world
world = client.get_world()
env_objs = world.get_environment_objects(carla.CityObjectLabel.Buildings)
# Access individual building IDs and save in a set
building_01 = env_objs[0]
building_02 = env_objs[1]
objects_to_toggle = {building_01.id, building_02.id}
# Toggle buildings off
world.enable_environment_objects(objects_to_toggle, False)
# Toggle buildings on
world.enable_environment_objects(objects_to_toggle, True)
```
<button id="button1" class="CopyScript" onclick="CopyToClipboard('carla.World.enable_environment_objects-code')">Copy snippet</button>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<button id="button1" class="CloseSnipet" onclick="CloseSnipet()">Close snippet</button><br><br>
</div>
<div id ="carla.DebugHelper.draw_string-snipet" style="display: none;">
<p class="SnipetFont">
Snippet for carla.DebugHelper.draw_string
</p>
<div id="carla.DebugHelper.draw_string-code" class="SnipetContent">
```py
# This recipe is a modification of lane_explorer.py example.
# It draws the path of an actor through the world, printing information at each waypoint.
# ...
current_w = map.get_waypoint(vehicle.get_location())
while True:
next_w = map.get_waypoint(vehicle.get_location(), lane_type=carla.LaneType.Driving | carla.LaneType.Shoulder | carla.LaneType.Sidewalk )
# Check if the vehicle is moving
if next_w.id != current_w.id:
vector = vehicle.get_velocity()
# Check if the vehicle is on a sidewalk
if current_w.lane_type == carla.LaneType.Sidewalk:
draw_waypoint_union(debug, current_w, next_w, cyan if current_w.is_junction else red, 60)
else:
draw_waypoint_union(debug, current_w, next_w, cyan if current_w.is_junction else green, 60)
debug.draw_string(current_w.transform.location, str('%15.0f km/h' % (3.6 * math.sqrt(vector.x**2 + vector.y**2 + vector.z**2))), False, orange, 60)
draw_transform(debug, current_w.transform, white, 60)
# Update the current waypoint and sleep for some time
current_w = next_w
time.sleep(args.tick_time)
# ...
```
<button id="button1" class="CopyScript" onclick="CopyToClipboard('carla.DebugHelper.draw_string-code')">Copy snippet</button>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<button id="button1" class="CloseSnipet" onclick="CloseSnipet()">Close snippet</button><br><br>
</div>
<div id ="carla.World.unload_map_layer-snipet" style="display: none;">
<p class="SnipetFont">
Snippet for carla.World.unload_map_layer
</p>
<div id="carla.World.unload_map_layer-code" class="SnipetContent">
```py
# This recipe toggles off several layers in our "_Opt" maps
# Load town one with minimum layout (roads, sidewalks, traffic lights and traffic signs)
# as well as buildings and parked vehicles
world = client.load_world('Town01_Opt', carla.MapLayer.Buildings | carla.MapLayer.ParkedVehicles)
# Toggle all buildings off
world.unload_map_layer(carla.MapLayer.Buildings)
# Toggle all parked vehicles off
world.unload_map_layer(carla.MapLayer.ParkedVehicles)
```
<button id="button1" class="CopyScript" onclick="CopyToClipboard('carla.World.unload_map_layer-code')">Copy snippet</button>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<button id="button1" class="CloseSnipet" onclick="CloseSnipet()">Close snippet</button><br><br>
</div>
<div id ="carla.Vehicle.set_wheel_steer_direction-snipet" style="display: none;">
<p class="SnipetFont">
Snippet for carla.Vehicle.set_wheel_steer_direction
</p>
<div id="carla.Vehicle.set_wheel_steer_direction-code" class="SnipetContent">
```py
# Sets the appearance of the vehicles front wheels to 40°. Vehicle physics will not be affected.
vehicle.set_wheel_steer_direction(carla.VehicleWheelLocation.FR_Wheel, 40.0)
vehicle.set_wheel_steer_direction(carla.VehicleWheelLocation.FL_Wheel, 40.0)
```
<button id="button1" class="CopyScript" onclick="CopyToClipboard('carla.Vehicle.set_wheel_steer_direction-code')">Copy snippet</button>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<button id="button1" class="CloseSnipet" onclick="CloseSnipet()">Close snippet</button><br><br>
</div>
<div id ="carla.Client.__init__-snipet" style="display: none;">
<p class="SnipetFont">
Snippet for carla.Client.__init__
</p>
<div id="carla.Client.__init__-code" class="SnipetContent">
```py
# This recipe shows in every script provided in PythonAPI/Examples
# and it is used to parse the client creation arguments when running the script.
argparser = argparse.ArgumentParser(
description=__doc__)
argparser.add_argument(
'--host',
metavar='H',
default='127.0.0.1',
help='IP of the host server (default: 127.0.0.1)')
argparser.add_argument(
'-p', '--port',
metavar='P',
default=2000,
type=int,
help='TCP port to listen to (default: 2000)')
argparser.add_argument(
'-s', '--speed',
metavar='FACTOR',
default=1.0,
type=float,
help='rate at which the weather changes (default: 1.0)')
args = argparser.parse_args()
speed_factor = args.speed
update_freq = 0.1 / speed_factor
client = carla.Client(args.host, args.port)
```
<button id="button1" class="CopyScript" onclick="CopyToClipboard('carla.Client.__init__-code')">Copy snippet</button>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<button id="button1" class="CloseSnipet" onclick="CloseSnipet()">Close snippet</button><br><br>
</div>
<div id ="carla.Map.get_waypoint-snipet" style="display: none;">
<p class="SnipetFont">
Snippet for carla.Map.get_waypoint
</p>
<div id="carla.Map.get_waypoint-code" class="SnipetContent">
```py
# This recipe shows the current traffic rules affecting the vehicle.
# Shows the current lane type and if a lane change can be done in the actual lane or the surrounding ones.
# ...
waypoint = world.get_map().get_waypoint(vehicle.get_location(),project_to_road=True, lane_type=(carla.LaneType.Driving | carla.LaneType.Shoulder | carla.LaneType.Sidewalk))
print("Current lane type: " + str(waypoint.lane_type))
# Check current lane change allowed
print("Current Lane change: " + str(waypoint.lane_change))
# Left and Right lane markings
print("L lane marking type: " + str(waypoint.left_lane_marking.type))
print("L lane marking change: " + str(waypoint.left_lane_marking.lane_change))
print("R lane marking type: " + str(waypoint.right_lane_marking.type))
print("R lane marking change: " + str(waypoint.right_lane_marking.lane_change))
# ...
```
<button id="button1" class="CopyScript" onclick="CopyToClipboard('carla.Map.get_waypoint-code')">Copy snippet</button>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<button id="button1" class="CloseSnipet" onclick="CloseSnipet()">Close snippet</button><br><br>
<img src="/img/snipets_images/carla.Map.get_waypoint.jpg">
</div>
<div id ="carla.World.spawn_actor-snipet" style="display: none;">
<p class="SnipetFont">
Snippet for carla.World.spawn_actor
</p>
<div id="carla.World.spawn_actor-code" class="SnipetContent">
```py
# This recipe attaches different camera / sensors to a vehicle with different attachments.
# ...
camera = world.spawn_actor(rgb_camera_bp, transform, attach_to=vehicle, attachment_type=Attachment.Rigid)
# Default attachment: Attachment.Rigid
gnss_sensor = world.spawn_actor(sensor_gnss_bp, transform, attach_to=vehicle)
collision_sensor = world.spawn_actor(sensor_collision_bp, transform, attach_to=vehicle)
lane_invasion_sensor = world.spawn_actor(sensor_lane_invasion_bp, transform, attach_to=vehicle)
# ...
```
<button id="button1" class="CopyScript" onclick="CopyToClipboard('carla.World.spawn_actor-code')">Copy snippet</button>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<button id="button1" class="CloseSnipet" onclick="CloseSnipet()">Close snippet</button><br><br>
</div>
<div id ="carla.WalkerAIController.stop-snipet" style="display: none;">
<p class="SnipetFont">
Snippet for carla.WalkerAIController.stop
</p>
<div id="carla.WalkerAIController.stop-code" class="SnipetContent">
```py
#To destroy the pedestrians, stop them from the navigation, and then destroy the objects (actor and controller).
# stop pedestrians (list is [controller, actor, controller, actor ...])
for i in range(0, len(all_id), 2):
all_actors[i].stop()
# destroy pedestrian (actor and controller)
client.apply_batch([carla.command.DestroyActor(x) for x in all_id])
```
<button id="button1" class="CopyScript" onclick="CopyToClipboard('carla.WalkerAIController.stop-code')">Copy snippet</button>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<button id="button1" class="CloseSnipet" onclick="CloseSnipet()">Close snippet</button><br><br>
</div>
<div id ="carla.DebugHelper.draw_box-snipet" style="display: none;">
<p class="SnipetFont">
Snippet for carla.DebugHelper.draw_box
</p>
<div id="carla.DebugHelper.draw_box-code" class="SnipetContent">
```py
# This recipe shows how to draw traffic light actor bounding boxes from a world snapshot.
# ....
debug = world.debug
world_snapshot = world.get_snapshot()
for actor_snapshot in world_snapshot:
actual_actor = world.get_actor(actor_snapshot.id)
if actual_actor.type_id == 'traffic.traffic_light':
debug.draw_box(carla.BoundingBox(actor_snapshot.get_transform().location,carla.Vector3D(0.5,0.5,2)),actor_snapshot.get_transform().rotation, 0.05, carla.Color(255,0,0,0),0)
# ...
```
<button id="button1" class="CopyScript" onclick="CopyToClipboard('carla.DebugHelper.draw_box-code')">Copy snippet</button>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<button id="button1" class="CloseSnipet" onclick="CloseSnipet()">Close snippet</button><br><br>
<img src="/img/snipets_images/carla.DebugHelper.draw_box.jpg">
</div>
<div id ="carla.World.get_spectator-snipet" style="display: none;">
<p class="SnipetFont">
Snippet for carla.World.get_spectator
</p>
<div id="carla.World.get_spectator-code" class="SnipetContent">
```py
# This recipe spawns an actor and the spectator camera at the actor's location.
# ...
world = client.get_world()
spectator = world.get_spectator()
vehicle_bp = random.choice(world.get_blueprint_library().filter('vehicle.bmw.*'))
transform = random.choice(world.get_map().get_spawn_points())
vehicle = world.try_spawn_actor(vehicle_bp, transform)
# Wait for world to get the vehicle actor
world.tick()
world_snapshot = world.wait_for_tick()
actor_snapshot = world_snapshot.find(vehicle.id)
# Set spectator at given transform (vehicle transform)
spectator.set_transform(actor_snapshot.get_transform())
# ...
```
<button id="button1" class="CopyScript" onclick="CopyToClipboard('carla.World.get_spectator-code')">Copy snippet</button>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<button id="button1" class="CloseSnipet" onclick="CloseSnipet()">Close snippet</button><br><br>
</div>
<div id ="carla.Sensor.listen-snipet" style="display: none;">
<p class="SnipetFont">
Snippet for carla.Sensor.listen
</p>
<div id="carla.Sensor.listen-code" class="SnipetContent">
```py
# This recipe applies a color conversion to the image taken by a camera sensor,
# so it is converted to a semantic segmentation image.
# ...
camera_bp = world.get_blueprint_library().filter('sensor.camera.semantic_segmentation')
# ...
cc = carla.ColorConverter.CityScapesPalette
camera.listen(lambda image: image.save_to_disk('output/%06d.png' % image.frame, cc))
# ...
```
<button id="button1" class="CopyScript" onclick="CopyToClipboard('carla.Sensor.listen-code')">Copy snippet</button>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<button id="button1" class="CloseSnipet" onclick="CloseSnipet()">Close snippet</button><br><br>
</div>
<div id ="carla.TrafficLight.set_state-snipet" style="display: none;">
<p class="SnipetFont">
Snippet for carla.TrafficLight.set_state
</p>
<div id="carla.TrafficLight.set_state-code" class="SnipetContent">
```py
# This recipe changes from red to green the traffic light that affects the vehicle.
# This is done by detecting if the vehicle actor is at a traffic light.
# ...
world = client.get_world()
spectator = world.get_spectator()
vehicle_bp = random.choice(world.get_blueprint_library().filter('vehicle.bmw.*'))
transform = random.choice(world.get_map().get_spawn_points())
vehicle = world.try_spawn_actor(vehicle_bp, transform)
# Wait for world to get the vehicle actor
world.tick()
world_snapshot = world.wait_for_tick()
actor_snapshot = world_snapshot.find(vehicle.id)
# Set spectator at given transform (vehicle transform)
spectator.set_transform(actor_snapshot.get_transform())
# ...# ...
if vehicle_actor.is_at_traffic_light():
traffic_light = vehicle_actor.get_traffic_light()
if traffic_light.get_state() == carla.TrafficLightState.Red:
# world.hud.notification("Traffic light changed! Good to go!")
traffic_light.set_state(carla.TrafficLightState.Green)
# ...
```
<button id="button1" class="CopyScript" onclick="CopyToClipboard('carla.TrafficLight.set_state-code')">Copy snippet</button>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<button id="button1" class="CloseSnipet" onclick="CloseSnipet()">Close snippet</button><br><br>
<img src="/img/snipets_images/carla.TrafficLight.set_state.gif">
</div>
<div id ="carla.ActorBlueprint.set_attribute-snipet" style="display: none;">
<p class="SnipetFont">
Snippet for carla.ActorBlueprint.set_attribute
@ -4462,6 +4144,333 @@ for i in range(0, len(all_actors), 2):
</div>
<div id ="carla.Client.__init__-snipet" style="display: none;">
<p class="SnipetFont">
Snippet for carla.Client.__init__
</p>
<div id="carla.Client.__init__-code" class="SnipetContent">
```py
# This recipe shows in every script provided in PythonAPI/Examples
# and it is used to parse the client creation arguments when running the script.
argparser = argparse.ArgumentParser(
description=__doc__)
argparser.add_argument(
'--host',
metavar='H',
default='127.0.0.1',
help='IP of the host server (default: 127.0.0.1)')
argparser.add_argument(
'-p', '--port',
metavar='P',
default=2000,
type=int,
help='TCP port to listen to (default: 2000)')
argparser.add_argument(
'-s', '--speed',
metavar='FACTOR',
default=1.0,
type=float,
help='rate at which the weather changes (default: 1.0)')
args = argparser.parse_args()
speed_factor = args.speed
update_freq = 0.1 / speed_factor
client = carla.Client(args.host, args.port)
```
<button id="button1" class="CopyScript" onclick="CopyToClipboard('carla.Client.__init__-code')">Copy snippet</button>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<button id="button1" class="CloseSnipet" onclick="CloseSnipet()">Close snippet</button><br><br>
</div>
<div id ="carla.DebugHelper.draw_box-snipet" style="display: none;">
<p class="SnipetFont">
Snippet for carla.DebugHelper.draw_box
</p>
<div id="carla.DebugHelper.draw_box-code" class="SnipetContent">
```py
# This recipe shows how to draw traffic light actor bounding boxes from a world snapshot.
# ....
debug = world.debug
world_snapshot = world.get_snapshot()
for actor_snapshot in world_snapshot:
actual_actor = world.get_actor(actor_snapshot.id)
if actual_actor.type_id == 'traffic.traffic_light':
debug.draw_box(carla.BoundingBox(actor_snapshot.get_transform().location,carla.Vector3D(0.5,0.5,2)),actor_snapshot.get_transform().rotation, 0.05, carla.Color(255,0,0,0),0)
# ...
```
<button id="button1" class="CopyScript" onclick="CopyToClipboard('carla.DebugHelper.draw_box-code')">Copy snippet</button>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<button id="button1" class="CloseSnipet" onclick="CloseSnipet()">Close snippet</button><br><br>
<img src="/img/snipets_images/carla.DebugHelper.draw_box.jpg">
</div>
<div id ="carla.DebugHelper.draw_string-snipet" style="display: none;">
<p class="SnipetFont">
Snippet for carla.DebugHelper.draw_string
</p>
<div id="carla.DebugHelper.draw_string-code" class="SnipetContent">
```py
# This recipe is a modification of lane_explorer.py example.
# It draws the path of an actor through the world, printing information at each waypoint.
# ...
current_w = map.get_waypoint(vehicle.get_location())
while True:
next_w = map.get_waypoint(vehicle.get_location(), lane_type=carla.LaneType.Driving | carla.LaneType.Shoulder | carla.LaneType.Sidewalk )
# Check if the vehicle is moving
if next_w.id != current_w.id:
vector = vehicle.get_velocity()
# Check if the vehicle is on a sidewalk
if current_w.lane_type == carla.LaneType.Sidewalk:
draw_waypoint_union(debug, current_w, next_w, cyan if current_w.is_junction else red, 60)
else:
draw_waypoint_union(debug, current_w, next_w, cyan if current_w.is_junction else green, 60)
debug.draw_string(current_w.transform.location, str('%15.0f km/h' % (3.6 * math.sqrt(vector.x**2 + vector.y**2 + vector.z**2))), False, orange, 60)
draw_transform(debug, current_w.transform, white, 60)
# Update the current waypoint and sleep for some time
current_w = next_w
time.sleep(args.tick_time)
# ...
```
<button id="button1" class="CopyScript" onclick="CopyToClipboard('carla.DebugHelper.draw_string-code')">Copy snippet</button>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<button id="button1" class="CloseSnipet" onclick="CloseSnipet()">Close snippet</button><br><br>
</div>
<div id ="carla.Map.get_waypoint-snipet" style="display: none;">
<p class="SnipetFont">
Snippet for carla.Map.get_waypoint
</p>
<div id="carla.Map.get_waypoint-code" class="SnipetContent">
```py
# This recipe shows the current traffic rules affecting the vehicle.
# Shows the current lane type and if a lane change can be done in the actual lane or the surrounding ones.
# ...
waypoint = world.get_map().get_waypoint(vehicle.get_location(),project_to_road=True, lane_type=(carla.LaneType.Driving | carla.LaneType.Shoulder | carla.LaneType.Sidewalk))
print("Current lane type: " + str(waypoint.lane_type))
# Check current lane change allowed
print("Current Lane change: " + str(waypoint.lane_change))
# Left and Right lane markings
print("L lane marking type: " + str(waypoint.left_lane_marking.type))
print("L lane marking change: " + str(waypoint.left_lane_marking.lane_change))
print("R lane marking type: " + str(waypoint.right_lane_marking.type))
print("R lane marking change: " + str(waypoint.right_lane_marking.lane_change))
# ...
```
<button id="button1" class="CopyScript" onclick="CopyToClipboard('carla.Map.get_waypoint-code')">Copy snippet</button>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<button id="button1" class="CloseSnipet" onclick="CloseSnipet()">Close snippet</button><br><br>
<img src="/img/snipets_images/carla.Map.get_waypoint.jpg">
</div>
<div id ="carla.Sensor.listen-snipet" style="display: none;">
<p class="SnipetFont">
Snippet for carla.Sensor.listen
</p>
<div id="carla.Sensor.listen-code" class="SnipetContent">
```py
# This recipe applies a color conversion to the image taken by a camera sensor,
# so it is converted to a semantic segmentation image.
# ...
camera_bp = world.get_blueprint_library().filter('sensor.camera.semantic_segmentation')
# ...
cc = carla.ColorConverter.CityScapesPalette
camera.listen(lambda image: image.save_to_disk('output/%06d.png' % image.frame, cc))
# ...
```
<button id="button1" class="CopyScript" onclick="CopyToClipboard('carla.Sensor.listen-code')">Copy snippet</button>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<button id="button1" class="CloseSnipet" onclick="CloseSnipet()">Close snippet</button><br><br>
</div>
<div id ="carla.TrafficLight.set_state-snipet" style="display: none;">
<p class="SnipetFont">
Snippet for carla.TrafficLight.set_state
</p>
<div id="carla.TrafficLight.set_state-code" class="SnipetContent">
```py
# This recipe changes from red to green the traffic light that affects the vehicle.
# This is done by detecting if the vehicle actor is at a traffic light.
# ...
world = client.get_world()
spectator = world.get_spectator()
vehicle_bp = random.choice(world.get_blueprint_library().filter('vehicle.bmw.*'))
transform = random.choice(world.get_map().get_spawn_points())
vehicle = world.try_spawn_actor(vehicle_bp, transform)
# Wait for world to get the vehicle actor
world.tick()
world_snapshot = world.wait_for_tick()
actor_snapshot = world_snapshot.find(vehicle.id)
# Set spectator at given transform (vehicle transform)
spectator.set_transform(actor_snapshot.get_transform())
# ...# ...
if vehicle_actor.is_at_traffic_light():
traffic_light = vehicle_actor.get_traffic_light()
if traffic_light.get_state() == carla.TrafficLightState.Red:
# world.hud.notification("Traffic light changed! Good to go!")
traffic_light.set_state(carla.TrafficLightState.Green)
# ...
```
<button id="button1" class="CopyScript" onclick="CopyToClipboard('carla.TrafficLight.set_state-code')">Copy snippet</button>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<button id="button1" class="CloseSnipet" onclick="CloseSnipet()">Close snippet</button><br><br>
<img src="/img/snipets_images/carla.TrafficLight.set_state.gif">
</div>
<div id ="carla.Vehicle.set_wheel_steer_direction-snipet" style="display: none;">
<p class="SnipetFont">
Snippet for carla.Vehicle.set_wheel_steer_direction
</p>
<div id="carla.Vehicle.set_wheel_steer_direction-code" class="SnipetContent">
```py
# Sets the appearance of the vehicles front wheels to 40°. Vehicle physics will not be affected.
vehicle.set_wheel_steer_direction(carla.VehicleWheelLocation.FR_Wheel, 40.0)
vehicle.set_wheel_steer_direction(carla.VehicleWheelLocation.FL_Wheel, 40.0)
```
<button id="button1" class="CopyScript" onclick="CopyToClipboard('carla.Vehicle.set_wheel_steer_direction-code')">Copy snippet</button>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<button id="button1" class="CloseSnipet" onclick="CloseSnipet()">Close snippet</button><br><br>
</div>
<div id ="carla.WalkerAIController.stop-snipet" style="display: none;">
<p class="SnipetFont">
Snippet for carla.WalkerAIController.stop
</p>
<div id="carla.WalkerAIController.stop-code" class="SnipetContent">
```py
#To destroy the pedestrians, stop them from the navigation, and then destroy the objects (actor and controller).
# stop pedestrians (list is [controller, actor, controller, actor ...])
for i in range(0, len(all_id), 2):
all_actors[i].stop()
# destroy pedestrian (actor and controller)
client.apply_batch([carla.command.DestroyActor(x) for x in all_id])
```
<button id="button1" class="CopyScript" onclick="CopyToClipboard('carla.WalkerAIController.stop-code')">Copy snippet</button>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<button id="button1" class="CloseSnipet" onclick="CloseSnipet()">Close snippet</button><br><br>
</div>
<div id ="carla.World.enable_environment_objects-snipet" style="display: none;">
<p class="SnipetFont">
Snippet for carla.World.enable_environment_objects
</p>
<div id="carla.World.enable_environment_objects-code" class="SnipetContent">
```py
# This recipe turn visibility off and on for two specifc buildings on the map
# Get the buildings in the world
world = client.get_world()
env_objs = world.get_environment_objects(carla.CityObjectLabel.Buildings)
# Access individual building IDs and save in a set
building_01 = env_objs[0]
building_02 = env_objs[1]
objects_to_toggle = {building_01.id, building_02.id}
# Toggle buildings off
world.enable_environment_objects(objects_to_toggle, False)
# Toggle buildings on
world.enable_environment_objects(objects_to_toggle, True)
```
<button id="button1" class="CopyScript" onclick="CopyToClipboard('carla.World.enable_environment_objects-code')">Copy snippet</button>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<button id="button1" class="CloseSnipet" onclick="CloseSnipet()">Close snippet</button><br><br>
</div>
<div id ="carla.World.get_spectator-snipet" style="display: none;">
<p class="SnipetFont">
Snippet for carla.World.get_spectator
</p>
<div id="carla.World.get_spectator-code" class="SnipetContent">
```py
# This recipe spawns an actor and the spectator camera at the actor's location.
# ...
world = client.get_world()
spectator = world.get_spectator()
vehicle_bp = random.choice(world.get_blueprint_library().filter('vehicle.bmw.*'))
transform = random.choice(world.get_map().get_spawn_points())
vehicle = world.try_spawn_actor(vehicle_bp, transform)
# Wait for world to get the vehicle actor
world.tick()
world_snapshot = world.wait_for_tick()
actor_snapshot = world_snapshot.find(vehicle.id)
# Set spectator at given transform (vehicle transform)
spectator.set_transform(actor_snapshot.get_transform())
# ...
```
<button id="button1" class="CopyScript" onclick="CopyToClipboard('carla.World.get_spectator-code')">Copy snippet</button>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<button id="button1" class="CloseSnipet" onclick="CloseSnipet()">Close snippet</button><br><br>
</div>
<div id ="carla.World.load_map_layer-snipet" style="display: none;">
<p class="SnipetFont">
Snippet for carla.World.load_map_layer
@ -4490,6 +4499,57 @@ world.load_map_layer(carla.MapLayer.ParkedVehicles)
</div>
<div id ="carla.World.spawn_actor-snipet" style="display: none;">
<p class="SnipetFont">
Snippet for carla.World.spawn_actor
</p>
<div id="carla.World.spawn_actor-code" class="SnipetContent">
```py
# This recipe attaches different camera / sensors to a vehicle with different attachments.
# ...
camera = world.spawn_actor(rgb_camera_bp, transform, attach_to=vehicle, attachment_type=Attachment.Rigid)
# Default attachment: Attachment.Rigid
gnss_sensor = world.spawn_actor(sensor_gnss_bp, transform, attach_to=vehicle)
collision_sensor = world.spawn_actor(sensor_collision_bp, transform, attach_to=vehicle)
lane_invasion_sensor = world.spawn_actor(sensor_lane_invasion_bp, transform, attach_to=vehicle)
# ...
```
<button id="button1" class="CopyScript" onclick="CopyToClipboard('carla.World.spawn_actor-code')">Copy snippet</button>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<button id="button1" class="CloseSnipet" onclick="CloseSnipet()">Close snippet</button><br><br>
</div>
<div id ="carla.World.unload_map_layer-snipet" style="display: none;">
<p class="SnipetFont">
Snippet for carla.World.unload_map_layer
</p>
<div id="carla.World.unload_map_layer-code" class="SnipetContent">
```py
# This recipe toggles off several layers in our "_Opt" maps
# Load town one with minimum layout (roads, sidewalks, traffic lights and traffic signs)
# as well as buildings and parked vehicles
world = client.load_world('Town01_Opt', carla.MapLayer.Buildings | carla.MapLayer.ParkedVehicles)
# Toggle all buildings off
world.unload_map_layer(carla.MapLayer.Buildings)
# Toggle all parked vehicles off
world.unload_map_layer(carla.MapLayer.ParkedVehicles)
```
<button id="button1" class="CopyScript" onclick="CopyToClipboard('carla.World.unload_map_layer-code')">Copy snippet</button>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<button id="button1" class="CloseSnipet" onclick="CloseSnipet()">Close snippet</button><br><br>
</div>
</div>

View File

@ -24,6 +24,10 @@ namespace client {
if (IsListening() && GetEpisode().IsValid()) {
try {
Stop();
for (uint32_t i = 1; i != 16; ++i) {
if (listening_mask.test(i))
StopGBuffer(i);
}
} catch (const std::exception &e) {
log_error("exception trying to stop sensor:", GetDisplayId(), ':', e.what());
}
@ -34,26 +38,48 @@ namespace client {
log_debug("calling sensor Listen() ", GetDisplayId());
log_debug(GetDisplayId(), ": subscribing to stream");
GetEpisode().Lock()->SubscribeToSensor(*this, std::move(callback));
_is_listening = true;
listening_mask.set(0);
}
void ServerSideSensor::Stop() {
log_debug("calling sensor Stop() ", GetDisplayId());
if (!_is_listening) {
if (!IsListening()) {
log_warning(
"attempting to unsubscribe from stream but sensor wasn't listening:",
GetDisplayId());
return;
}
GetEpisode().Lock()->UnSubscribeFromSensor(*this);
_is_listening = false;
listening_mask.reset(0);
}
void ServerSideSensor::ListenToGBuffer(uint32_t GBufferId, CallbackFunctionType callback) {
log_debug(GetDisplayId(), ": subscribing to gbuffer stream");
GetEpisode().Lock()->SubscribeToGBuffer(*this, GBufferId, std::move(callback));
listening_mask.set(0);
listening_mask.set(GBufferId + 1);
}
void ServerSideSensor::StopGBuffer(uint32_t GBufferId) {
log_debug(GetDisplayId(), ": unsubscribing from gbuffer stream");
GetEpisode().Lock()->UnSubscribeFromGBuffer(*this, GBufferId);
listening_mask.reset(GBufferId + 1);
if (listening_mask.count() == 1) {
listening_mask.reset(0);
}
}
bool ServerSideSensor::Destroy() {
log_debug("calling sensor Destroy() ", GetDisplayId());
if (IsListening()) {
for (uint32_t i = 1; i != 16; ++i) {
if (listening_mask.test(i)) {
StopGBuffer(i);
}
}
Stop();
}
listening_mask = {};
return Actor::Destroy();
}

View File

@ -7,6 +7,7 @@
#pragma once
#include "carla/client/Sensor.h"
#include <bitset>
namespace carla {
namespace client {
@ -33,7 +34,17 @@ namespace client {
/// Return whether this Sensor instance is currently listening to the
/// associated sensor in the simulator.
bool IsListening() const override {
return _is_listening;
return listening_mask.test(0);
}
/// Listen fr
void ListenToGBuffer(uint32_t GBufferId, CallbackFunctionType callback);
/// Stop listening for a specific gbuffer stream.
void StopGBuffer(uint32_t GBufferId);
inline bool IsListeningGBuffer(uint32_t id) const {
return listening_mask.test(id + 1);
}
/// @copydoc Actor::Destroy()
@ -43,7 +54,8 @@ namespace client {
private:
bool _is_listening = false;
std::bitset<32> listening_mask;
};
} // namespace client

View File

@ -586,6 +586,27 @@ namespace detail {
_pimpl->streaming_client.UnSubscribe(token);
}
void Client::SubscribeToGBuffer(
rpc::ActorId ActorId,
uint32_t GBufferId,
std::function<void(Buffer)> callback)
{
std::vector<unsigned char> token_data = _pimpl->CallAndWait<std::vector<unsigned char>>("get_gbuffer_token", ActorId, GBufferId);
streaming::Token token;
std::memcpy(&token.data[0u], token_data.data(), token_data.size());
_pimpl->streaming_client.Subscribe(token, std::move(callback));
}
void Client::UnSubscribeFromGBuffer(
rpc::ActorId ActorId,
uint32_t GBufferId)
{
std::vector<unsigned char> token_data = _pimpl->CallAndWait<std::vector<unsigned char>>("get_gbuffer_token", ActorId, GBufferId);
streaming::Token token;
std::memcpy(&token.data[0u], token_data.data(), token_data.size());
_pimpl->streaming_client.UnSubscribe(token);
}
void Client::DrawDebugShape(const rpc::DebugShape &shape) {
_pimpl->AsyncCall("draw_debug_shape", shape);
}

View File

@ -368,8 +368,17 @@ namespace detail {
const streaming::Token &token,
std::function<void(Buffer)> callback);
void SubscribeToGBuffer(
rpc::ActorId ActorId,
uint32_t GBufferId,
std::function<void(Buffer)> callback);
void UnSubscribeFromStream(const streaming::Token &token);
void UnSubscribeFromGBuffer(
rpc::ActorId ActorId,
uint32_t GBufferId);
void DrawDebugShape(const rpc::DebugShape &shape);
void ApplyBatch(

View File

@ -390,8 +390,25 @@ namespace detail {
});
}
void Simulator::UnSubscribeFromSensor(const Sensor &sensor) {
void Simulator::UnSubscribeFromSensor(Actor &sensor) {
_client.UnSubscribeFromStream(sensor.GetActorDescription().GetStreamToken());
// If in the future we need to unsubscribe from each gbuffer individually, it should be done here.
}
void Simulator::SubscribeToGBuffer(
Actor &actor,
uint32_t gbuffer_id,
std::function<void(SharedPtr<sensor::SensorData>)> callback) {
_client.SubscribeToGBuffer(actor.GetId(), gbuffer_id,
[cb=std::move(callback), ep=WeakEpisodeProxy{shared_from_this()}](auto buffer) {
auto data = sensor::Deserializer::Deserialize(std::move(buffer));
data->_episode = ep.TryLock();
cb(std::move(data));
});
}
void Simulator::UnSubscribeFromGBuffer(Actor &actor, uint32_t gbuffer_id) {
_client.UnSubscribeFromGBuffer(actor.GetId(), gbuffer_id);
}
void Simulator::FreezeAllTrafficLights(bool frozen) {

View File

@ -590,7 +590,16 @@ namespace detail {
const Sensor &sensor,
std::function<void(SharedPtr<sensor::SensorData>)> callback);
void UnSubscribeFromSensor(const Sensor &sensor);
void UnSubscribeFromSensor(Actor &sensor);
void SubscribeToGBuffer(
Actor & sensor,
uint32_t gbuffer_id,
std::function<void(SharedPtr<sensor::SensorData>)> callback);
void UnSubscribeFromGBuffer(
Actor & sensor,
uint32_t gbuffer_id);
/// @}
// =========================================================================

View File

@ -27,6 +27,8 @@
#include "carla/sensor/s11n/ObstacleDetectionEventSerializer.h"
#include "carla/sensor/s11n/RadarSerializer.h"
#include "carla/sensor/s11n/SemanticLidarSerializer.h"
#include "carla/sensor/s11n/GBufferUint8Serializer.h"
#include "carla/sensor/s11n/GBufferFloatSerializer.h"
// 2. Add a forward-declaration of the sensor here.
class ACollisionSensor;
@ -46,6 +48,8 @@ class ASemanticSegmentationCamera;
class AInstanceSegmentationCamera;
class ARssSensor;
class FWorldObserver;
struct FCameraGBufferUint8;
struct FCameraGBufferFloat;
namespace carla {
namespace sensor {
@ -74,7 +78,9 @@ namespace sensor {
std::pair<ASceneCaptureCamera *, s11n::ImageSerializer>,
std::pair<ASemanticSegmentationCamera *, s11n::ImageSerializer>,
std::pair<AInstanceSegmentationCamera *, s11n::ImageSerializer>,
std::pair<FWorldObserver *, s11n::EpisodeStateSerializer>
std::pair<FWorldObserver *, s11n::EpisodeStateSerializer>,
std::pair<FCameraGBufferUint8 *, s11n::GBufferUint8Serializer>,
std::pair<FCameraGBufferFloat *, s11n::GBufferFloatSerializer>
>;
} // namespace sensor

View File

@ -16,6 +16,9 @@ namespace data {
/// An image of 32-bit BGRA colors (8-bit channels, 4 bytes)
using Image = ImageTmpl<Color>;
/// An image of float BGRA colors (32-bit channels)
using FloatImage = ImageTmpl<rpc::FloatColor>;
/// An image of 64-bit BGRA colors (16-bit channels, 2 floats)
using OpticalFlowImage = ImageTmpl<OpticalFlowPixel>;

View File

@ -10,6 +10,8 @@
#include "carla/sensor/data/Array.h"
#include "carla/sensor/s11n/ImageSerializer.h"
#include "carla/sensor/s11n/OpticalFlowImageSerializer.h"
#include "carla/sensor/s11n/GBufferUint8Serializer.h"
#include "carla/sensor/s11n/GBufferFloatSerializer.h"
#include "carla/sensor/s11n/NormalsImageSerializer.h"
namespace carla {
@ -28,6 +30,8 @@ namespace data {
friend Serializer;
friend SerializerOpticalFlow;
friend s11n::GBufferUint8Serializer;
friend s11n::GBufferFloatSerializer;
friend SerializerNormals;
explicit ImageTmpl(RawData &&data)

View File

@ -0,0 +1,22 @@
// Copyright (c) 2022 Computer Vision Center (CVC) at the Universitat Autonoma
// de Barcelona (UAB).
//
// This work is licensed under the terms of the MIT license.
// For a copy, see <https://opensource.org/licenses/MIT>.
#include "carla/sensor/s11n/GBufferFloatSerializer.h"
#include "carla/sensor/data/Image.h"
namespace carla {
namespace sensor {
namespace s11n {
SharedPtr<SensorData> GBufferFloatSerializer::Deserialize(RawData &&data) {
auto image = SharedPtr<data::FloatImage>(new data::FloatImage{std::move(data)});
return image;
}
} // namespace s11n
} // namespace sensor
} // namespace carla

View File

@ -0,0 +1,62 @@
// Copyright (c) 2022 Computer Vision Center (CVC) at the Universitat Autonoma
// de Barcelona (UAB).
//
// This work is licensed under the terms of the MIT license.
// For a copy, see <https://opensource.org/licenses/MIT>.
#pragma once
#include "carla/Memory.h"
#include "carla/sensor/RawData.h"
#include <cstdint>
#include <cstring>
namespace carla {
namespace sensor {
class SensorData;
namespace s11n {
/// Serializes image buffers generated by camera sensors.
class GBufferFloatSerializer {
public:
#pragma pack(push, 1)
struct ImageHeader {
uint32_t width;
uint32_t height;
float fov_angle;
};
#pragma pack(pop)
constexpr static auto header_offset = sizeof(ImageHeader);
static const ImageHeader &DeserializeHeader(const RawData &data) {
return *reinterpret_cast<const ImageHeader *>(data.begin());
}
template <typename Sensor>
static Buffer Serialize(const Sensor &sensor, Buffer &&bitmap,
uint32_t ImageWidth, uint32_t ImageHeight, float FovAngle);
static SharedPtr<SensorData> Deserialize(RawData &&data);
};
template <typename Sensor>
inline Buffer GBufferFloatSerializer::Serialize(const Sensor &/*sensor*/, Buffer &&bitmap,
uint32_t ImageWidth, uint32_t ImageHeight, float FovAngle) {
DEBUG_ASSERT(bitmap.size() > sizeof(ImageHeader));
ImageHeader header = {
ImageWidth,
ImageHeight,
FovAngle
};
std::memcpy(bitmap.data(), reinterpret_cast<const void *>(&header), sizeof(header));
return std::move(bitmap);
}
} // namespace s11n
} // namespace sensor
} // namespace carla

View File

@ -0,0 +1,22 @@
// Copyright (c) 2022 Computer Vision Center (CVC) at the Universitat Autonoma
// de Barcelona (UAB).
//
// This work is licensed under the terms of the MIT license.
// For a copy, see <https://opensource.org/licenses/MIT>.
#include "carla/sensor/s11n/GBufferUint8Serializer.h"
#include "carla/sensor/data/Image.h"
namespace carla {
namespace sensor {
namespace s11n {
SharedPtr<SensorData> GBufferUint8Serializer::Deserialize(RawData &&data) {
auto image = SharedPtr<data::Image>(new data::Image{std::move(data)});
return image;
}
} // namespace s11n
} // namespace sensor
} // namespace carla

View File

@ -0,0 +1,62 @@
// Copyright (c) 2022 Computer Vision Center (CVC) at the Universitat Autonoma
// de Barcelona (UAB).
//
// This work is licensed under the terms of the MIT license.
// For a copy, see <https://opensource.org/licenses/MIT>.
#pragma once
#include "carla/Memory.h"
#include "carla/sensor/RawData.h"
#include <cstdint>
#include <cstring>
namespace carla {
namespace sensor {
class SensorData;
namespace s11n {
/// Serializes image buffers generated by camera sensors.
class GBufferUint8Serializer {
public:
#pragma pack(push, 1)
struct ImageHeader {
uint32_t width;
uint32_t height;
float fov_angle;
};
#pragma pack(pop)
constexpr static auto header_offset = sizeof(ImageHeader);
static const ImageHeader &DeserializeHeader(const RawData &data) {
return *reinterpret_cast<const ImageHeader *>(data.begin());
}
template <typename Sensor>
static Buffer Serialize(const Sensor &sensor, Buffer &&bitmap,
uint32_t ImageWidth, uint32_t ImageHeight, float FovAngle);
static SharedPtr<SensorData> Deserialize(RawData &&data);
};
template <typename Sensor>
inline Buffer GBufferUint8Serializer::Serialize(const Sensor &/*sensor*/, Buffer &&bitmap,
uint32_t ImageWidth, uint32_t ImageHeight, float FovAngle) {
DEBUG_ASSERT(bitmap.size() > sizeof(ImageHeader));
ImageHeader header = {
ImageWidth,
ImageHeight,
FovAngle
};
std::memcpy(bitmap.data(), reinterpret_cast<const void *>(&header), sizeof(header));
return std::move(bitmap);
}
} // namespace s11n
} // namespace sensor
} // namespace carla

View File

@ -14,6 +14,13 @@ static void SubscribeToStream(carla::client::Sensor &self, boost::python::object
self.Listen(MakeCallback(std::move(callback)));
}
static void SubscribeToGBuffer(
carla::client::ServerSideSensor &self,
uint32_t GBufferId,
boost::python::object callback) {
self.ListenToGBuffer(GBufferId, MakeCallback(std::move(callback)));
}
void export_sensor() {
using namespace boost::python;
namespace cc = carla::client;
@ -27,6 +34,8 @@ void export_sensor() {
class_<cc::ServerSideSensor, bases<cc::Sensor>, boost::noncopyable, boost::shared_ptr<cc::ServerSideSensor>>
("ServerSideSensor", no_init)
.def("listen_to_gbuffer", &SubscribeToGBuffer, (arg("gbuffer_id"), arg("callback")))
.def("stop_gbuffer", &cc::ServerSideSensor::StopGBuffer, (arg("gbuffer_id")))
.def(self_ns::str(self_ns::self))
;

View File

@ -387,6 +387,24 @@ void export_sensor_data() {
.value("CityScapesPalette", EColorConverter::CityScapesPalette)
;
// The values here should match the ones in the enum EGBufferTextureID,
// from the CARLA fork of Unreal Engine (Renderer/Public/GBufferView.h).
enum_<int>("GBufferTextureID")
.value("SceneColor", 0)
.value("SceneDepth", 1)
.value("SceneStencil", 2)
.value("GBufferA", 3)
.value("GBufferB", 4)
.value("GBufferC", 5)
.value("GBufferD", 6)
.value("GBufferE", 7)
.value("GBufferF", 8)
.value("Velocity", 9)
.value("SSAO", 10)
.value("CustomDepth", 11)
.value("CustomStencil", 12)
;
class_<csd::Image, bases<cs::SensorData>, boost::noncopyable, boost::shared_ptr<csd::Image>>("Image", no_init)
.add_property("width", &csd::Image::GetWidth)
.add_property("height", &csd::Image::GetHeight)

View File

@ -45,6 +45,29 @@
doc: >
Commands the sensor to stop listening for data.
# --------------------------------------
- def_name: listen_to_gbuffer
params:
- param_name: gbuffer_id
type: carla.GBufferTextureID
doc: >
The ID of the desired Unreal Engine GBuffer texture.
- param_name: callback
type: function
doc: >
The called function with one argument containing the received GBuffer texture.
doc: >
The function the sensor will be calling to every time the desired GBuffer texture is received.<br>
This function needs for an argument containing an object type carla.SensorData to work with.
# --------------------------------------
- def_name: stop_gbuffer
params:
- param_name: gbuffer_id
type: carla.GBufferTextureID
doc: >
The ID of the Unreal Engine GBuffer texture.
doc: >
Commands the sensor to stop listening for the specified GBuffer texture.
# --------------------------------------
- def_name: __str__
# --------------------------------------

View File

@ -863,4 +863,71 @@
# --------------------------------------
- def_name: __str__
# --------------------------------------
- class_name: GBufferTextureID
# - DESCRIPTION ------------------------
doc: >
Defines the identifiers of each GBuffer texture (See the method `carla.Sensor.listen_to_gbuffer`).
# - PROPERTIES -------------------------
instance_variables:
- var_name: SceneColor
doc: >
The texture "SceneColor" contains the final color of the image.
- var_name: SceneDepth
doc: >
The texture "SceneDepth" contains the depth buffer - linear in world units.
- var_name: SceneStencil
doc: >
The texture "SceneStencil" contains the stencil buffer.
- var_name: GBufferA
doc: >
The texture "GBufferA" contains the world-space normal vectors in the RGB channels. The alpha channel contains "per-object data".
- var_name: GBufferB
doc: >
The texture "GBufferB" contains the metallic, specular and roughness in the RGB channels, respectively. The alpha channel contains a mask where the lower 4 bits indicate the shading model and the upper 4 bits contain the selective output mask.
- var_name: GBufferC
doc: >
The texture "GBufferC" contains the diffuse color in the RGB channels, with the indirect irradiance in the alpha channel.<br>
If static lightning is not allowed, the alpha channel will contain the ambient occlusion instead.
- var_name: GBufferD
doc: >
The contents of the "GBufferD" varies depending on the rendered object's material shading model (GBufferB):<br>
- MSM_Subsurface (2), MSM_PreintegratedSkin (3), MSM_TwoSidedFoliage (6):<br>
RGB: Subsurface color.<br>
A: Opacity.<br>
- MSM_ClearCoat (4):<br>
R: Clear coat.<br>
G: Roughness.<br>
- MSM_SubsurfaceProfile (5):<br>
RGB: Subsurface profile.<br>
- MSM_Hair (7):<br>
RG: World normal.<br>
B: Backlit value.<br>
- MSM_Cloth (8):<br>
RGB: Subsurface color.<br>
A: Cloth value.<br>
- MSM_Eye (9):<br>
RG: Eye tangent.<br>
B: Iris mask.<br>
A: Iris distance.
- var_name: GBufferE
doc: >
The texture "GBufferE" contains the precomputed shadow factors in the RGBA channels. This texture is unavailable if the selective output mask (GBufferB) does not have its 4th bit set.
- var_name: GBufferF
doc: >
The texture "GBufferF" contains the world-space tangent in the RGB channels and the anisotropy in the alpha channel. This texture is unavailable if the selective output mask (GBufferB) does not have its 5th bit set.
- var_name: Velocity
doc: >
The texture "Velocity" contains the screen-space velocity of the scene objects.
- var_name: SSAO
doc: >
The texture "SSAO" contains the screen-space ambient occlusion texture.
- var_name: CustomDepth
doc: >
The texture "CustomDepth" contains the Unreal Engine custom depth data.
- var_name: CustomStencil
doc: >
The texture "CustomStencil" contains the Unreal Engine custom stencil data.
# --------------------------------------
...

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,127 @@
#!/usr/bin/env python
# Copyright (c) 2019 Computer Vision Center (CVC) at the Universitat Autonoma de
# Barcelona (UAB).
#
# This work is licensed under the terms of the MIT license.
# For a copy, see <https://opensource.org/licenses/MIT>.
import glob
import os
import sys
try:
sys.path.append(glob.glob('../carla/dist/carla-*%d.%d-%s.egg' % (
sys.version_info.major,
sys.version_info.minor,
'win-amd64' if os.name == 'nt' else 'linux-x86_64'))[0])
except IndexError:
pass
import carla
import random
import time
def main():
actor_list = []
# In this tutorial script, we are going to add a vehicle to the simulation
# and let it drive in autopilot. We will also create a camera attached to
# that vehicle, and save all the images generated by the camera to disk.
# Additionally, we will save all of the gbuffer textures for each frame.
try:
# First of all, we need to create the client that will send the requests
# to the simulator. Here we'll assume the simulator is accepting
# requests in the localhost at port 2000.
client = carla.Client('127.0.0.1', 2000)
client.set_timeout(2.0)
# Once we have a client we can retrieve the world that is currently
# running.
world = client.get_world()
# The world contains the list blueprints that we can use for adding new
# actors into the simulation.
blueprint_library = world.get_blueprint_library()
# Now let's filter all the blueprints of type 'vehicle' and choose one
# at random.
bp = random.choice(blueprint_library.filter('vehicle'))
# A blueprint contains the list of attributes that define a vehicle's
# instance, we can read them and modify some of them. For instance,
# let's randomize its color.
if bp.has_attribute('color'):
color = random.choice(bp.get_attribute('color').recommended_values)
bp.set_attribute('color', color)
# Now we need to give an initial transform to the vehicle. We choose a
# random transform from the list of recommended spawn points of the map.
transform = world.get_map().get_spawn_points()[0]
# So let's tell the world to spawn the vehicle.
vehicle = world.spawn_actor(bp, transform)
# It is important to note that the actors we create won't be destroyed
# unless we call their "destroy" function. If we fail to call "destroy"
# they will stay in the simulation even after we quit the Python script.
# For that reason, we are storing all the actors we create so we can
# destroy them afterwards.
actor_list.append(vehicle)
print('created %s' % vehicle.type_id)
# Let's put the vehicle to drive around.
vehicle.set_autopilot(True)
# Let's add now a "rgb" camera attached to the vehicle. Note that the
# transform we give here is now relative to the vehicle.
camera_bp = blueprint_library.find('sensor.camera.rgb')
camera_bp.set_attribute('image_size_x', '1920')
camera_bp.set_attribute('image_size_y', '1080')
camera_transform = carla.Transform(carla.Location(x=1.5, z=2.4))
camera = world.spawn_actor(camera_bp, camera_transform, attach_to=vehicle)
actor_list.append(camera)
print('created %s' % camera.type_id)
# Register a callback for whenever a new frame is available. This step is
# currently required to correctly receive the gbuffer textures, as it is
# used to determine whether the sensor is active.
camera.listen(lambda image: image.save_to_disk('_out/FinalColor-%06d.png' % image.frame))
# Here we will register the callbacks for each gbuffer texture.
# The function "listen_to_gbuffer" behaves like the regular listen function,
# but you must first pass it the ID of the desired gbuffer texture.
camera.listen_to_gbuffer(carla.GBufferTextureID.SceneColor, lambda image: image.save_to_disk('_out/SceneColor-%06d.png' % image.frame))
camera.listen_to_gbuffer(carla.GBufferTextureID.SceneDepth, lambda image: image.save_to_disk('_out/SceneDepth-%06d.png' % image.frame))
camera.listen_to_gbuffer(carla.GBufferTextureID.SceneStencil, lambda image: image.save_to_disk('_out/SceneStencil-%06d.png' % image.frame))
camera.listen_to_gbuffer(carla.GBufferTextureID.GBufferA, lambda image: image.save_to_disk('_out/GBufferA-%06d.png' % image.frame))
camera.listen_to_gbuffer(carla.GBufferTextureID.GBufferB, lambda image: image.save_to_disk('_out/GBufferB-%06d.png' % image.frame))
camera.listen_to_gbuffer(carla.GBufferTextureID.GBufferC, lambda image: image.save_to_disk('_out/GBufferC-%06d.png' % image.frame))
camera.listen_to_gbuffer(carla.GBufferTextureID.GBufferD, lambda image: image.save_to_disk('_out/GBufferD-%06d.png' % image.frame))
# Note that some gbuffer textures may not be available for a particular scene.
# For example, the textures E and F are likely unavailable in this example,
# which will result in them being sent as black images.
camera.listen_to_gbuffer(carla.GBufferTextureID.GBufferE, lambda image: image.save_to_disk('_out/GBufferE-%06d.png' % image.frame))
camera.listen_to_gbuffer(carla.GBufferTextureID.GBufferF, lambda image: image.save_to_disk('_out/GBufferF-%06d.png' % image.frame))
camera.listen_to_gbuffer(carla.GBufferTextureID.Velocity, lambda image: image.save_to_disk('_out/Velocity-%06d.png' % image.frame))
camera.listen_to_gbuffer(carla.GBufferTextureID.SSAO, lambda image: image.save_to_disk('_out/SSAO-%06d.png' % image.frame))
camera.listen_to_gbuffer(carla.GBufferTextureID.CustomDepth, lambda image: image.save_to_disk('_out/CustomDepth-%06d.png' % image.frame))
camera.listen_to_gbuffer(carla.GBufferTextureID.CustomStencil, lambda image: image.save_to_disk('_out/CustomStencil-%06d.png' % image.frame))
time.sleep(10)
finally:
print('destroying actors')
camera.destroy()
client.apply_batch([carla.command.DestroyActor(x) for x in actor_list])
print('done.')
if __name__ == '__main__':
main()

1
Unreal/CarlaUE4/Config/.gitignore vendored Normal file
View File

@ -0,0 +1 @@
OptionalModules.ini

View File

@ -71,6 +71,7 @@ public class Carla : ModuleRules
"Core",
"RenderCore",
"RHI",
"Renderer",
"ProceduralMeshComponent"
// ... add other public dependencies that you statically link with here ...
}

View File

@ -0,0 +1,146 @@
// Copyright (c) 2022 Computer Vision Center (CVC) at the Universitat Autonoma
// de Barcelona (UAB).
//
// This work is licensed under the terms of the MIT license.
// For a copy, see <https://opensource.org/licenses/MIT>.
#include "Carla/Sensor/ImageUtil.h"
#include "Runtime/RHI/Public/RHISurfaceDataConversion.h"
namespace ImageUtil
{
void DecodePixelsByFormat(
void* PixelData,
int32 SourcePitch,
FIntPoint SourceExtent,
FIntPoint DestinationExtent,
EPixelFormat Format,
FReadSurfaceDataFlags Flags,
TArrayView<FLinearColor> Out)
{
SourcePitch *= GPixelFormats[Format].BlockBytes;
auto OutPixelCount = DestinationExtent.X * DestinationExtent.Y;
switch (Format)
{
case PF_G16:
case PF_R16_UINT:
case PF_R16_SINT:
// Shadow maps
ConvertRawR16DataToFLinearColor(DestinationExtent.X, DestinationExtent.Y, (uint8*)PixelData, SourcePitch, Out.GetData());
break;
case PF_R8G8B8A8:
ConvertRawR8G8B8A8DataToFLinearColor(DestinationExtent.X, DestinationExtent.Y, (uint8*)PixelData, SourcePitch, Out.GetData());
break;
case PF_B8G8R8A8:
ConvertRawB8G8R8A8DataToFLinearColor(DestinationExtent.X, DestinationExtent.Y, (uint8*)PixelData, SourcePitch, Out.GetData());
break;
case PF_A2B10G10R10:
ConvertRawA2B10G10R10DataToFLinearColor(DestinationExtent.X, DestinationExtent.Y, (uint8*)PixelData, SourcePitch, Out.GetData());
break;
case PF_FloatRGBA:
case PF_R16G16B16A16_UNORM:
case PF_R16G16B16A16_SNORM:
ConvertRawR16G16B16A16FDataToFLinearColor(DestinationExtent.X, DestinationExtent.Y, (uint8*)PixelData, SourcePitch, Out.GetData(), Flags);
break;
case PF_FloatR11G11B10:
ConvertRawRR11G11B10DataToFLinearColor(DestinationExtent.X, DestinationExtent.Y, (uint8*)PixelData, SourcePitch, Out.GetData());
break;
case PF_A32B32G32R32F:
ConvertRawR32G32B32A32DataToFLinearColor(DestinationExtent.X, DestinationExtent.Y, (uint8*)PixelData, SourcePitch, Out.GetData(), Flags);
break;
case PF_A16B16G16R16:
ConvertRawR16G16B16A16DataToFLinearColor(DestinationExtent.X, DestinationExtent.Y, (uint8*)PixelData, SourcePitch, Out.GetData());
break;
case PF_G16R16:
ConvertRawR16G16DataToFLinearColor(DestinationExtent.X, DestinationExtent.Y, (uint8*)PixelData, SourcePitch, Out.GetData());
break;
case PF_DepthStencil: // Depth / Stencil
ConvertRawD32S8DataToFLinearColor(DestinationExtent.X, DestinationExtent.Y, (uint8*)PixelData, SourcePitch, Out.GetData(), Flags);
break;
case PF_X24_G8: // Depth Stencil
ConvertRawR24G8DataToFLinearColor(DestinationExtent.X, DestinationExtent.Y, (uint8*)PixelData, SourcePitch, Out.GetData(), Flags);
break;
case PF_R32_FLOAT: // Depth Stencil
ConvertRawR32DataToFLinearColor(DestinationExtent.X, DestinationExtent.Y, (uint8*)PixelData, SourcePitch, Out.GetData(), Flags);
break;
case PF_R16G16B16A16_UINT:
case PF_R16G16B16A16_SINT:
ConvertRawR16G16B16A16DataToFLinearColor(DestinationExtent.X, DestinationExtent.Y, (uint8*)PixelData, SourcePitch, Out.GetData());
break;
default:
UE_LOG(LogCarla, Warning, TEXT("Unsupported format %llu"), (unsigned long long)Format);
check(false);
break;
}
}
void DecodePixelsByFormat(
void* PixelData,
int32 SourcePitch,
FIntPoint SourceExtent,
FIntPoint DestinationExtent,
EPixelFormat Format,
FReadSurfaceDataFlags Flags,
TArrayView<FColor> Out)
{
SourcePitch *= GPixelFormats[Format].BlockBytes;
auto OutPixelCount = DestinationExtent.X * DestinationExtent.Y;
switch (Format)
{
case PF_G16:
case PF_R16_UINT:
case PF_R16_SINT:
// Shadow maps
ConvertRawR16DataToFColor(DestinationExtent.X, DestinationExtent.Y, (uint8*)PixelData, SourcePitch, Out.GetData());
break;
case PF_R8G8B8A8:
ConvertRawR8G8B8A8DataToFColor(DestinationExtent.X, DestinationExtent.Y, (uint8*)PixelData, SourcePitch, Out.GetData());
break;
case PF_B8G8R8A8:
ConvertRawB8G8R8A8DataToFColor(DestinationExtent.X, DestinationExtent.Y, (uint8*)PixelData, SourcePitch, Out.GetData());
break;
case PF_A2B10G10R10:
ConvertRawR10G10B10A2DataToFColor(DestinationExtent.X, DestinationExtent.Y, (uint8*)PixelData, SourcePitch, Out.GetData());
break;
case PF_FloatRGBA:
case PF_R16G16B16A16_UNORM:
case PF_R16G16B16A16_SNORM:
ConvertRawR16G16B16A16FDataToFColor(DestinationExtent.X, DestinationExtent.Y, (uint8*)PixelData, SourcePitch, Out.GetData(), false);
break;
case PF_FloatR11G11B10:
ConvertRawR11G11B10DataToFColor(DestinationExtent.X, DestinationExtent.Y, (uint8*)PixelData, SourcePitch, Out.GetData(), false);
break;
case PF_A32B32G32R32F:
ConvertRawR32G32B32A32DataToFColor(DestinationExtent.X, DestinationExtent.Y, (uint8*)PixelData, SourcePitch, Out.GetData(), false);
break;
case PF_A16B16G16R16:
ConvertRawR16G16B16A16DataToFColor(DestinationExtent.X, DestinationExtent.Y, (uint8*)PixelData, SourcePitch, Out.GetData());
break;
case PF_G16R16:
ConvertRawR16G16DataToFColor(DestinationExtent.X, DestinationExtent.Y, (uint8*)PixelData, SourcePitch, Out.GetData());
break;
case PF_DepthStencil: // Depth / Stencil
ConvertRawD32S8DataToFColor(DestinationExtent.X, DestinationExtent.Y, (uint8*)PixelData, SourcePitch, Out.GetData(), Flags);
break;
case PF_X24_G8: // Depth / Stencil
ConvertRawR24G8DataToFColor(DestinationExtent.X, DestinationExtent.Y, (uint8*)PixelData, SourcePitch, Out.GetData(), Flags);
break;
case PF_R32_FLOAT: // Depth
ConvertRawR32DataToFColor(DestinationExtent.X, DestinationExtent.Y, (uint8*)PixelData, SourcePitch, Out.GetData(), Flags);
break;
case PF_R16G16B16A16_UINT:
case PF_R16G16B16A16_SINT:
ConvertRawR16G16B16A16DataToFColor(DestinationExtent.X, DestinationExtent.Y, (uint8*)PixelData, SourcePitch, Out.GetData());
break;
case PF_G8:
ConvertRawR8DataToFColor(DestinationExtent.X, DestinationExtent.Y, (uint8*)PixelData, SourcePitch, Out.GetData());
break;
default:
UE_LOG(LogCarla, Warning, TEXT("Unsupported format %llu"), (unsigned long long)Format);
check(false);
break;
}
}
}

View File

@ -0,0 +1,33 @@
// Copyright (c) 2022 Computer Vision Center (CVC) at the Universitat Autonoma
// de Barcelona (UAB).
//
// This work is licensed under the terms of the MIT license.
// For a copy, see <https://opensource.org/licenses/MIT>.
#pragma once
#include "CoreMinimal.h"
class FRHIGPUTextureReadback;
namespace ImageUtil
{
void DecodePixelsByFormat(
void* PixelData,
int32 SourcePitch,
FIntPoint SourceExtent,
FIntPoint DestinationExtent,
EPixelFormat Format,
FReadSurfaceDataFlags Flags,
TArrayView<FColor> Out);
void DecodePixelsByFormat(
void* PixelData,
int32 SourcePitch,
FIntPoint SourceExtent,
FIntPoint DestinationExtent,
EPixelFormat Format,
FReadSurfaceDataFlags Flags,
TArrayView<FLinearColor> Out);
}

View File

@ -69,8 +69,6 @@ public:
template <typename TSensor, typename TPixel>
static void SendPixelsInRenderThread(TSensor &Sensor, bool use16BitFormat = false, std::function<TArray<TPixel>(void *, uint32)> Conversor = {});
private:
/// Copy the pixels in @a RenderTarget into @a Buffer.
///
/// @pre To be called from render-thread.

View File

@ -13,17 +13,17 @@
FActorDefinition ASceneCaptureCamera::GetSensorDefinition()
{
constexpr bool bEnableModifyingPostProcessEffects = true;
return UActorBlueprintFunctionLibrary::MakeCameraDefinition(
TEXT("rgb"),
bEnableModifyingPostProcessEffects);
constexpr bool bEnableModifyingPostProcessEffects = true;
return UActorBlueprintFunctionLibrary::MakeCameraDefinition(
TEXT("rgb"),
bEnableModifyingPostProcessEffects);
}
ASceneCaptureCamera::ASceneCaptureCamera(const FObjectInitializer &ObjectInitializer)
: Super(ObjectInitializer)
ASceneCaptureCamera::ASceneCaptureCamera(const FObjectInitializer& ObjectInitializer)
: Super(ObjectInitializer)
{
AddPostProcessingMaterial(
TEXT("Material'/Carla/PostProcessingMaterials/PhysicLensDistortion.PhysicLensDistortion'"));
AddPostProcessingMaterial(
TEXT("Material'/Carla/PostProcessingMaterials/PhysicLensDistortion.PhysicLensDistortion'"));
}
void ASceneCaptureCamera::BeginPlay()
@ -62,3 +62,8 @@ void ASceneCaptureCamera::PostPhysTick(UWorld *World, ELevelTick TickType, float
);
FPixelReader::SendPixelsInRenderThread<ASceneCaptureCamera, FColor>(*this);
}
void ASceneCaptureCamera::SendGBufferTextures(FGBufferRequest& GBuffer)
{
SendGBufferTexturesInternal(*this, GBuffer);
}

View File

@ -27,6 +27,8 @@ public:
protected:
virtual void SendGBufferTextures(FGBufferRequest& GBuffer) override;
void BeginPlay() override;
void EndPlay(const EEndPlayReason::Type EndPlayReason) override;

View File

@ -6,20 +6,11 @@
#include "Carla.h"
#include "Carla/Sensor/SceneCaptureSensor.h"
#include "Carla/Game/CarlaStatics.h"
#include "Async/Async.h"
#include "Components/DrawFrustumComponent.h"
#include "Components/SceneCaptureComponent2D.h"
#include "Components/StaticMeshComponent.h"
#include "ContentStreaming.h"
#include "Engine/Classes/Engine/Scene.h"
#include "Engine/TextureRenderTarget2D.h"
#include "HAL/UnrealMemory.h"
#include "HighResScreenshot.h"
#include "Misc/CoreDelegates.h"
#include "RHICommandList.h"
#include <mutex>
#include <atomic>
#include <thread>
static auto SCENE_CAPTURE_COUNTER = 0u;
@ -62,11 +53,12 @@ ASceneCaptureSensor::ASceneCaptureSensor(const FObjectInitializer &ObjectInitial
CaptureRenderTarget->AddressX = TextureAddress::TA_Clamp;
CaptureRenderTarget->AddressY = TextureAddress::TA_Clamp;
CaptureComponent2D = CreateDefaultSubobject<USceneCaptureComponent2D>(
FName(*FString::Printf(TEXT("SceneCaptureComponent2D_%d"), SCENE_CAPTURE_COUNTER)));
CaptureComponent2D = CreateDefaultSubobject<USceneCaptureComponent2D_CARLA>(
FName(*FString::Printf(TEXT("USceneCaptureComponent2D_CARLA_%d"), SCENE_CAPTURE_COUNTER)));
check(CaptureComponent2D != nullptr);
CaptureComponent2D->ViewActor = this;
CaptureComponent2D->SetupAttachment(RootComponent);
CaptureComponent2D->PrimitiveRenderMode = ESceneCapturePrimitiveRenderMode::PRM_RenderScenePrimitives;
CaptureComponent2D->bCaptureOnMovement = false;
CaptureComponent2D->bCaptureEveryFrame = false;
CaptureComponent2D->bAlwaysPersistRenderingState = true;
@ -456,8 +448,96 @@ float ASceneCaptureSensor::GetChromAberrOffset() const
void ASceneCaptureSensor::EnqueueRenderSceneImmediate() {
TRACE_CPUPROFILER_EVENT_SCOPE(ASceneCaptureSensor::EnqueueRenderSceneImmediate);
// Creates an snapshot of the scene, requieres bCaptureEveryFrame = false.
GetCaptureComponent2D()->CaptureScene();
// Equivalent to "CaptureComponent2D->CaptureScene" + (optional) GBuffer extraction.
CaptureSceneExtended();
}
constexpr const TCHAR* GBufferNames[] =
{
TEXT("SceneColor"),
TEXT("SceneDepth"),
TEXT("SceneStencil"),
TEXT("GBufferA"),
TEXT("GBufferB"),
TEXT("GBufferC"),
TEXT("GBufferD"),
TEXT("GBufferE"),
TEXT("GBufferF"),
TEXT("Velocity"),
TEXT("SSAO"),
TEXT("CustomDepth"),
TEXT("CustomStencil"),
};
template <EGBufferTextureID ID, typename T>
static void CheckGBufferStream(T& GBufferStream, FGBufferRequest& GBuffer)
{
GBufferStream.bIsUsed = GBufferStream.Stream.AreClientsListening();
if (GBufferStream.bIsUsed)
GBuffer.MarkAsRequested(ID);
}
static uint64 Prior = 0;
void ASceneCaptureSensor::CaptureSceneExtended()
{
auto GBufferPtr = MakeUnique<FGBufferRequest>();
auto& GBuffer = *GBufferPtr;
CheckGBufferStream<EGBufferTextureID::SceneColor>(CameraGBuffers.SceneColor, GBuffer);
CheckGBufferStream<EGBufferTextureID::SceneDepth>(CameraGBuffers.SceneDepth, GBuffer);
CheckGBufferStream<EGBufferTextureID::SceneStencil>(CameraGBuffers.SceneStencil, GBuffer);
CheckGBufferStream<EGBufferTextureID::GBufferA>(CameraGBuffers.GBufferA, GBuffer);
CheckGBufferStream<EGBufferTextureID::GBufferB>(CameraGBuffers.GBufferB, GBuffer);
CheckGBufferStream<EGBufferTextureID::GBufferC>(CameraGBuffers.GBufferC, GBuffer);
CheckGBufferStream<EGBufferTextureID::GBufferD>(CameraGBuffers.GBufferD, GBuffer);
CheckGBufferStream<EGBufferTextureID::GBufferE>(CameraGBuffers.GBufferE, GBuffer);
CheckGBufferStream<EGBufferTextureID::GBufferF>(CameraGBuffers.GBufferF, GBuffer);
CheckGBufferStream<EGBufferTextureID::Velocity>(CameraGBuffers.Velocity, GBuffer);
CheckGBufferStream<EGBufferTextureID::SSAO>(CameraGBuffers.SSAO, GBuffer);
CheckGBufferStream<EGBufferTextureID::CustomDepth>(CameraGBuffers.CustomDepth, GBuffer);
CheckGBufferStream<EGBufferTextureID::CustomStencil>(CameraGBuffers.CustomStencil, GBuffer);
if (GBufferPtr->DesiredTexturesMask == 0)
{
// Creates an snapshot of the scene, requieres bCaptureEveryFrame = false.
CaptureComponent2D->CaptureScene();
return;
}
if (Prior != GBufferPtr->DesiredTexturesMask)
UE_LOG(LogCarla, Verbose, TEXT("GBuffer selection changed (%llu)."), GBufferPtr->DesiredTexturesMask);
Prior = GBufferPtr->DesiredTexturesMask;
GBufferPtr->OwningActor = CaptureComponent2D->GetViewOwner();
#define CARLA_GBUFFER_DISABLE_TAA // Temporarily disable TAA to avoid jitter.
#ifdef CARLA_GBUFFER_DISABLE_TAA
bool bTAA = CaptureComponent2D->ShowFlags.TemporalAA;
if (bTAA) {
CaptureComponent2D->ShowFlags.TemporalAA = false;
}
#endif
CaptureComponent2D->CaptureSceneWithGBuffer(GBuffer);
#ifdef CARLA_GBUFFER_DISABLE_TAA
if (bTAA) {
CaptureComponent2D->ShowFlags.TemporalAA = true;
}
#undef CARLA_GBUFFER_DISABLE_TAA
#endif
AsyncTask(ENamedThreads::AnyHiPriThreadNormalTask, [this, GBuffer = MoveTemp(GBufferPtr)]() mutable
{
SendGBufferTextures(*GBuffer);
});
}
void ASceneCaptureSensor::SendGBufferTextures(FGBufferRequest& GBuffer)
{
SendGBufferTexturesInternal(*this, GBuffer);
}
void ASceneCaptureSensor::BeginPlay()

View File

@ -8,15 +8,112 @@
#include "Carla/Sensor/PixelReader.h"
#include "Carla/Sensor/Sensor.h"
#include "Carla/Sensor/UE4_Overridden/SceneCaptureComponent2D_CARLA.h"
#include "Carla/Sensor/ImageUtil.h"
#include "Async/Async.h"
#include "Renderer/Public/GBufferView.h"
#include <type_traits>
#include "Runtime/RenderCore/Public/RenderCommandFence.h"
#include "SceneCaptureSensor.generated.h"
class UDrawFrustumComponent;
class USceneCaptureComponent2D;
class UStaticMeshComponent;
class UTextureRenderTarget2D;
struct FCameraGBufferUint8
{
/// Prevent this sensor to be spawned by users.
using not_spawnable = void;
void SetDataStream(FDataStream InStream)
{
Stream = std::move(InStream);
}
/// Replace the Stream associated with this sensor.
void SetStream(FDataMultiStream InStream)
{
Stream = std::move(InStream);
}
/// Return the token that allows subscribing to this sensor's stream.
auto GetToken() const
{
bIsUsed = true;
return Stream.GetToken();
}
/// Dummy. Required for compatibility with other sensors only.
FTransform GetActorTransform() const
{
return {};
}
/// Return the FDataStream associated with this sensor.
///
/// You need to provide a reference to self, this is necessary for template
/// deduction.
template <typename SensorT>
FAsyncDataStream GetDataStream(const SensorT &Self)
{
return Stream.MakeAsyncDataStream(Self, Self.GetEpisode().GetElapsedGameTime());
}
mutable bool bIsUsed = false;
FDataStream Stream;
};
struct FCameraGBufferFloat
{
/// Prevent this sensor to be spawned by users.
using not_spawnable = void;
void SetDataStream(FDataStream InStream)
{
Stream = std::move(InStream);
}
/// Replace the Stream associated with this sensor.
void SetStream(FDataMultiStream InStream)
{
Stream = std::move(InStream);
}
/// Return the token that allows subscribing to this sensor's stream.
auto GetToken() const
{
bIsUsed = true;
return Stream.GetToken();
}
/// Dummy. Required for compatibility with other sensors only.
FTransform GetActorTransform() const
{
return {};
}
/// Return the FDataStream associated with this sensor.
///
/// You need to provide a reference to self, this is necessary for template
/// deduction.
template <typename SensorT>
FAsyncDataStream GetDataStream(const SensorT &Self)
{
return Stream.MakeAsyncDataStream(Self, Self.GetEpisode().GetElapsedGameTime());
}
mutable bool bIsUsed = false;
FDataStream Stream;
};
/// Base class for sensors using a USceneCaptureComponent2D for rendering the
/// scene. This class does not capture data, use
/// `FPixelReader::SendPixelsInRenderThread(*this)` in derived classes.
@ -32,6 +129,7 @@ class CARLA_API ASceneCaptureSensor : public ASensor
friend class ACarlaGameModeBase;
friend class FPixelReader;
friend class FPixelReader2;
public:
@ -304,8 +402,29 @@ public:
// FlushRenderingCommands();
}
struct
{
FCameraGBufferUint8 SceneColor;
FCameraGBufferUint8 SceneDepth;
FCameraGBufferUint8 SceneStencil;
FCameraGBufferUint8 GBufferA;
FCameraGBufferUint8 GBufferB;
FCameraGBufferUint8 GBufferC;
FCameraGBufferUint8 GBufferD;
FCameraGBufferUint8 GBufferE;
FCameraGBufferUint8 GBufferF;
FCameraGBufferUint8 Velocity;
FCameraGBufferUint8 SSAO;
FCameraGBufferUint8 CustomDepth;
FCameraGBufferUint8 CustomStencil;
} CameraGBuffers;
protected:
void CaptureSceneExtended();
virtual void SendGBufferTextures(FGBufferRequest& GBuffer);
virtual void BeginPlay() override;
virtual void PrePhysTick(float DeltaSeconds) override;
@ -321,7 +440,7 @@ protected:
/// Scene capture component.
UPROPERTY(EditAnywhere)
USceneCaptureComponent2D *CaptureComponent2D = nullptr;
USceneCaptureComponent2D_CARLA *CaptureComponent2D = nullptr;
UPROPERTY(EditAnywhere)
float TargetGamma = 2.4f;
@ -342,6 +461,109 @@ protected:
UPROPERTY(EditAnywhere)
bool bEnable16BitFormat = false;
FRenderCommandFence RenderFence;
private:
template <
typename SensorT,
typename CameraGBufferT>
static void SendGBuffer(
SensorT& Self,
CameraGBufferT& CameraGBuffer,
FGBufferRequest& GBufferData,
EGBufferTextureID TextureID)
{
using PixelType = typename std::conditional<
std::is_same<std::remove_reference_t<CameraGBufferT>, FCameraGBufferUint8>::value,
FColor,
FLinearColor>::type;
FIntPoint ViewSize = {};
TArray<PixelType> Pixels;
if (GBufferData.WaitForTextureTransfer(TextureID))
{
TRACE_CPUPROFILER_EVENT_SCOPE_STR("GBuffer Decode");
void* PixelData;
int32 SourcePitch;
FIntPoint SourceExtent;
GBufferData.MapTextureData(TextureID, PixelData, SourcePitch, SourceExtent);
ViewSize = GBufferData.ViewRect.Size();
auto Format = GBufferData.Readbacks[(size_t)TextureID]->GetFormat();
Pixels.AddUninitialized(ViewSize.X * ViewSize.Y);
FReadSurfaceDataFlags Flags = {};
Flags.SetLinearToGamma(true);
ImageUtil::DecodePixelsByFormat(PixelData, SourcePitch, SourceExtent, ViewSize, Format, Flags, Pixels);
GBufferData.UnmapTextureData(TextureID);
}
auto GBufferStream = CameraGBuffer.GetDataStream(Self);
auto Buffer = GBufferStream.PopBufferFromPool();
Buffer.copy_from(carla::sensor::SensorRegistry::get<CameraGBufferT*>::type::header_offset, Pixels);
if (Buffer.empty()) {
return;
}
SCOPE_CYCLE_COUNTER(STAT_CarlaSensorStreamSend);
TRACE_CPUPROFILER_EVENT_SCOPE_STR("Stream Send");
GBufferStream.Send(
CameraGBuffer, std::move(Buffer),
ViewSize.X, ViewSize.Y,
Self.GetFOVAngle());
}
protected:
template <typename T>
void SendGBufferTexturesInternal(T& Self, FGBufferRequest& GBufferData)
{
for (size_t i = 0; i != FGBufferRequest::TextureCount; ++i)
{
if ((GBufferData.DesiredTexturesMask & (UINT64_C(1) << i)) == 0) {
continue;
}
auto& C = CameraGBuffers;
EGBufferTextureID ID = (EGBufferTextureID)i;
switch (ID)
{
case EGBufferTextureID::SceneColor:
SendGBuffer(Self, C.SceneColor, GBufferData, ID);
break;
case EGBufferTextureID::SceneDepth:
SendGBuffer(Self, C.SceneDepth, GBufferData, ID);
break;
case EGBufferTextureID::SceneStencil:
SendGBuffer(Self, C.SceneStencil, GBufferData, ID);
break;
case EGBufferTextureID::GBufferA:
SendGBuffer(Self, C.GBufferA, GBufferData, ID);
break;
case EGBufferTextureID::GBufferB:
SendGBuffer(Self, C.GBufferB, GBufferData, ID);
break;
case EGBufferTextureID::GBufferC:
SendGBuffer(Self, C.GBufferC, GBufferData, ID);
break;
case EGBufferTextureID::GBufferD:
SendGBuffer(Self, C.GBufferD, GBufferData, ID);
break;
case EGBufferTextureID::GBufferE:
SendGBuffer(Self, C.GBufferE, GBufferData, ID);
break;
case EGBufferTextureID::GBufferF:
SendGBuffer(Self, C.GBufferF, GBufferData, ID);
break;
case EGBufferTextureID::Velocity:
SendGBuffer(Self, C.Velocity, GBufferData, ID);
break;
case EGBufferTextureID::SSAO:
SendGBuffer(Self, C.SSAO, GBufferData, ID);
break;
case EGBufferTextureID::CustomDepth:
SendGBuffer(Self, C.CustomDepth, GBufferData, ID);
break;
case EGBufferTextureID::CustomStencil:
SendGBuffer(Self, C.CustomStencil, GBufferData, ID);
break;
default:
abort();
}
}
}
};

View File

@ -81,18 +81,18 @@ public:
UFUNCTION(BlueprintCallable)
void SetSeed(int32 InSeed);
protected:
void PostActorCreated() override;
void EndPlay(EEndPlayReason::Type EndPlayReason) override;
const UCarlaEpisode &GetEpisode() const
{
check(Episode != nullptr);
return *Episode;
}
protected:
void PostActorCreated() override;
void EndPlay(EEndPlayReason::Type EndPlayReason) override;
/// Return the FDataStream associated with this sensor.
///
/// You need to provide a reference to self, this is necessary for template

View File

@ -10,6 +10,7 @@
#include "Carla/Game/CarlaGameInstance.h"
#include "Carla/Game/CarlaStatics.h"
#include "Carla/Sensor/Sensor.h"
#include "Carla/Sensor/SceneCaptureSensor.h"
#include <compiler/disable-ue4-macros.h>
#include <carla/sensor/SensorRegistry.h>
@ -137,6 +138,23 @@ FActorSpawnResult ASensorFactory::SpawnActor(
Sensor->SetEpisode(*Episode);
Sensor->Set(Description);
Sensor->SetDataStream(GameInstance->GetServer().OpenStream());
ASceneCaptureSensor * SceneCaptureSensor = Cast<ASceneCaptureSensor>(Sensor);
if(SceneCaptureSensor)
{
SceneCaptureSensor->CameraGBuffers.SceneColor.SetDataStream(GameInstance->GetServer().OpenStream());
SceneCaptureSensor->CameraGBuffers.SceneDepth.SetDataStream(GameInstance->GetServer().OpenStream());
SceneCaptureSensor->CameraGBuffers.SceneStencil.SetDataStream(GameInstance->GetServer().OpenStream());
SceneCaptureSensor->CameraGBuffers.GBufferA.SetDataStream(GameInstance->GetServer().OpenStream());
SceneCaptureSensor->CameraGBuffers.GBufferB.SetDataStream(GameInstance->GetServer().OpenStream());
SceneCaptureSensor->CameraGBuffers.GBufferC.SetDataStream(GameInstance->GetServer().OpenStream());
SceneCaptureSensor->CameraGBuffers.GBufferD.SetDataStream(GameInstance->GetServer().OpenStream());
SceneCaptureSensor->CameraGBuffers.GBufferE.SetDataStream(GameInstance->GetServer().OpenStream());
SceneCaptureSensor->CameraGBuffers.GBufferF.SetDataStream(GameInstance->GetServer().OpenStream());
SceneCaptureSensor->CameraGBuffers.Velocity.SetDataStream(GameInstance->GetServer().OpenStream());
SceneCaptureSensor->CameraGBuffers.SSAO.SetDataStream(GameInstance->GetServer().OpenStream());
SceneCaptureSensor->CameraGBuffers.CustomDepth.SetDataStream(GameInstance->GetServer().OpenStream());
SceneCaptureSensor->CameraGBuffers.CustomStencil.SetDataStream(GameInstance->GetServer().OpenStream());
}
}
UGameplayStatics::FinishSpawningActor(Sensor, Transform);
return FActorSpawnResult{Sensor};

View File

@ -0,0 +1,19 @@
// Copyright (c) 2022 Computer Vision Center (CVC) at the Universitat Autonoma
// de Barcelona (UAB).
//
// This work is licensed under the terms of the MIT license.
// For a copy, see <https://opensource.org/licenses/MIT>.
#include "SceneCaptureComponent2D_CARLA.h"
USceneCaptureComponent2D_CARLA::USceneCaptureComponent2D_CARLA(const FObjectInitializer& ObjectInitializer) :
Super(ObjectInitializer)
{
}
const AActor* USceneCaptureComponent2D_CARLA::GetViewOwner() const
{
return ViewActor;
}

View File

@ -0,0 +1,26 @@
// Copyright (c) 2022 Computer Vision Center (CVC) at the Universitat Autonoma
// de Barcelona (UAB).
//
// This work is licensed under the terms of the MIT license.
// For a copy, see <https://opensource.org/licenses/MIT>.
#pragma once
#include "CoreMinimal.h"
#include "Components/SceneCaptureComponent2D.h"
#include "SceneCaptureComponent2D_CARLA.generated.h"
UCLASS(hidecategories=(Collision, Object, Physics, SceneComponent, Mobility))
class CARLA_API USceneCaptureComponent2D_CARLA : public USceneCaptureComponent2D
{
GENERATED_BODY()
public:
USceneCaptureComponent2D_CARLA(const FObjectInitializer& = FObjectInitializer::Get());
const AActor* ViewActor;
virtual const AActor* GetViewOwner() const override;
};

View File

@ -0,0 +1,19 @@
// Copyright (c) 2022 Computer Vision Center (CVC) at the Universitat Autonoma
// de Barcelona (UAB).
//
// This work is licensed under the terms of the MIT license.
// For a copy, see <https://opensource.org/licenses/MIT>.
#include "SceneCaptureComponentCube_CARLA.h"
USceneCaptureComponentCube_CARLA::USceneCaptureComponentCube_CARLA(const FObjectInitializer& ObjectInitializer) :
Super(ObjectInitializer)
{
}
const AActor* USceneCaptureComponentCube_CARLA::GetViewOwner() const
{
return ViewActor;
}

View File

@ -0,0 +1,26 @@
// Copyright (c) 2022 Computer Vision Center (CVC) at the Universitat Autonoma
// de Barcelona (UAB).
//
// This work is licensed under the terms of the MIT license.
// For a copy, see <https://opensource.org/licenses/MIT>.
#pragma once
#include "CoreMinimal.h"
#include "Components/SceneCaptureComponentCube.h"
#include "SceneCaptureComponentCube_CARLA.generated.h"
UCLASS(hidecategories=(Collision, Object, Physics, SceneComponent, Mobility))
class CARLA_API USceneCaptureComponentCube_CARLA : public USceneCaptureComponentCube
{
GENERATED_BODY()
public:
USceneCaptureComponentCube_CARLA(const FObjectInitializer& = FObjectInitializer::Get());
const AActor* ViewActor;
virtual const AActor* GetViewOwner() const override;
};

View File

@ -0,0 +1,19 @@
// Copyright (c) 2022 Computer Vision Center (CVC) at the Universitat Autonoma
// de Barcelona (UAB).
//
// This work is licensed under the terms of the MIT license.
// For a copy, see <https://opensource.org/licenses/MIT>.
#include "SceneCaptureComponent_CARLA.h"
USceneCaptureComponent_CARLA::USceneCaptureComponent_CARLA(const FObjectInitializer& ObjectInitializer) :
Super(ObjectInitializer)
{
}
const AActor* USceneCaptureComponent_CARLA::GetViewOwner() const
{
return ViewActor;
}

View File

@ -0,0 +1,26 @@
// Copyright (c) 2022 Computer Vision Center (CVC) at the Universitat Autonoma
// de Barcelona (UAB).
//
// This work is licensed under the terms of the MIT license.
// For a copy, see <https://opensource.org/licenses/MIT>.
#pragma once
#include "CoreMinimal.h"
#include "Components/SceneCaptureComponent.h"
#include "SceneCaptureComponent_CARLA.generated.h"
UCLASS(hidecategories=(Collision, Object, Physics, SceneComponent, Mobility))
class CARLA_API USceneCaptureComponent_CARLA : public USceneCaptureComponent
{
GENERATED_BODY()
public:
USceneCaptureComponent_CARLA(const FObjectInitializer& = FObjectInitializer::Get());
const AActor* ViewActor;
virtual const AActor* GetViewOwner() const override;
};

View File

@ -2001,7 +2001,7 @@ void FCarlaServer::FPimpl::BindActions()
{
REQUIRE_CARLA_EPISODE();
FCarlaActor* CarlaActor = Episode->FindCarlaActor(ActorId);
if (CarlaActor)
if (!CarlaActor)
{
return RespondError(
"get_light_boxes",
@ -2034,6 +2034,107 @@ void FCarlaServer::FPimpl::BindActions()
}
};
// ~~ GBuffer tokens ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
BIND_SYNC(get_gbuffer_token) << [this](const cr::ActorId ActorId, uint32_t GBufferId) -> R<std::vector<unsigned char>>
{
REQUIRE_CARLA_EPISODE();
FCarlaActor* CarlaActor = Episode->FindCarlaActor(ActorId);
if(!CarlaActor)
{
return RespondError(
"get_gbuffer_token",
ECarlaServerResponse::ActorNotFound,
" Actor Id: " + FString::FromInt(ActorId));
}
if (CarlaActor->IsDormant())
{
return RespondError(
"get_gbuffer_token",
ECarlaServerResponse::FunctionNotAvailiableWhenDormant,
" Actor Id: " + FString::FromInt(ActorId));
}
ASceneCaptureSensor* Sensor = Cast<ASceneCaptureSensor>(CarlaActor->GetActor());
if (!Sensor)
{
return RespondError(
"get_gbuffer_token",
ECarlaServerResponse::ActorTypeMismatch,
" Actor Id: " + FString::FromInt(ActorId));
}
switch (GBufferId)
{
case 0:
{
const auto &Token = Sensor->CameraGBuffers.SceneColor.GetToken();
return std::vector<unsigned char>(std::begin(Token.data), std::end(Token.data));
}
case 1:
{
const auto &Token = Sensor->CameraGBuffers.SceneDepth.GetToken();
return std::vector<unsigned char>(std::begin(Token.data), std::end(Token.data));
}
case 2:
{
const auto& Token = Sensor->CameraGBuffers.SceneStencil.GetToken();
return std::vector<unsigned char>(std::begin(Token.data), std::end(Token.data));
}
case 3:
{
const auto &Token = Sensor->CameraGBuffers.GBufferA.GetToken();
return std::vector<unsigned char>(std::begin(Token.data), std::end(Token.data));
}
case 4:
{
const auto &Token = Sensor->CameraGBuffers.GBufferB.GetToken();
return std::vector<unsigned char>(std::begin(Token.data), std::end(Token.data));
}
case 5:
{
const auto &Token = Sensor->CameraGBuffers.GBufferC.GetToken();
return std::vector<unsigned char>(std::begin(Token.data), std::end(Token.data));
}
case 6:
{
const auto &Token = Sensor->CameraGBuffers.GBufferD.GetToken();
return std::vector<unsigned char>(std::begin(Token.data), std::end(Token.data));
}
case 7:
{
const auto &Token = Sensor->CameraGBuffers.GBufferE.GetToken();
return std::vector<unsigned char>(std::begin(Token.data), std::end(Token.data));
}
case 8:
{
const auto &Token = Sensor->CameraGBuffers.GBufferF.GetToken();
return std::vector<unsigned char>(std::begin(Token.data), std::end(Token.data));
}
case 9:
{
const auto &Token = Sensor->CameraGBuffers.Velocity.GetToken();
return std::vector<unsigned char>(std::begin(Token.data), std::end(Token.data));
}
case 10:
{
const auto &Token = Sensor->CameraGBuffers.SSAO.GetToken();
return std::vector<unsigned char>(std::begin(Token.data), std::end(Token.data));
}
case 11:
{
const auto& Token = Sensor->CameraGBuffers.CustomDepth.GetToken();
return std::vector<unsigned char>(std::begin(Token.data), std::end(Token.data));
}
case 12:
{
const auto& Token = Sensor->CameraGBuffers.CustomStencil.GetToken();
return std::vector<unsigned char>(std::begin(Token.data), std::end(Token.data));
}
default:
UE_LOG(LogCarla, Error, TEXT("Requested invalid GBuffer ID %u"), GBufferId);
return {};
}
};
// ~~ Logging and playback ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
BIND_SYNC(start_recorder) << [this](std::string name, bool AdditionalData) -> R<std::string>

View File

@ -10,6 +10,7 @@
#include "YieldSignComponent.h"
#include "SpeedLimitComponent.h"
#include "Components/BoxComponent.h"
#include "Runtime/CoreUObject/Public/UObject/ConstructorHelpers.h"
#include "UObject/ConstructorHelpers.h"

View File

@ -10,6 +10,7 @@
#include "TrafficLightGroup.h"
#include "TrafficSignBase.h"
#include "Carla/OpenDrive/OpenDrive.h"
#include "TrafficLightManager.generated.h"
/// Class In charge of creating and assigning traffic