diff --git a/Docs/core_sensors.md b/Docs/core_sensors.md index 1b97e4ba3..8cb595fc6 100644 --- a/Docs/core_sensors.md +++ b/Docs/core_sensors.md @@ -128,22 +128,19 @@ __Retrieve data:__ every simulation step. --------------- That is a wrap on sensors and how do these retrieve simulation data. -There is yet a lot to learn about CARLA, but this has been the last of the first steps. Now it is time to really discover the possibilities of CARLA. -However, here is a brief guidance for some of the different paths that are opened right now: +Thus, this was the last step of this introduction to CARLA. However there is yet a lot to learn about CARLA. Here are some different paths opened at the moment: -* __For those who want to gain some practise__: -> Python Cookbook +* __Gain some practise__: if diving alone in CARLA is still frightening, it may be a good idea to try some of the code recipes provided in this documentation and combine them with the example scripts or some ideas of your own.

- -CARLA forum + +Code recipes

-* __For those who want to continue learning__: -> Advanced step +* __Continue learning__: there are other more advanced features in CARLA such as rendering options, traffic manager, the recorder, and some more. Now that some fundaments on CARLA have been provided, it is a good moment to learn about these.
@@ -153,20 +150,18 @@ Configuring the simulation

-* __For those who want to experiment freely__: -> References +* __Experiment freely__: but don't forget to take a look at the __References__ section of this documentation. They contain detailed information on the classes in the Python API, sensors and their outputs, and much more.

- + Python API reference

-* __For those who have something to say__: -> Forum +* __Give your two cents__: speak your mind whatever it is going around. Any doubts, suggestions and ideas about CARLA are welcome in the forum.
diff --git a/Docs/ref_sensors.md b/Docs/ref_sensors.md index 6893374d5..41a2e5748 100644 --- a/Docs/ref_sensors.md +++ b/Docs/ref_sensors.md @@ -1,4 +1,4 @@ -

Sensor references

+

Sensors' documentation

* [__Collision detector__](#collision-detector) * [__Depth camera__](#depth-camera) @@ -16,314 +16,297 @@ ##Collision detector * __Blueprint:__ sensor.other.collision -* __Output:__ [carla.CollisionEvent](python_api.md#carla.CollisionEvent) +* __Output:__ [carla.CollisionEvent](python_api.md#carla.CollisionEvent) per collision. -This sensor, when attached to an actor, it registers an event each time the -actor collisions against something in the world. This sensor does not have any -configurable attribute. +This sensor registers an event each time its parent actor collisions against something in the world. Several collisions may be detected during a single simulation step. +Collision detectors do not have any configurable attribute. !!! note This sensor creates "fake" actors when it collides with something that is not an actor, this is so we can retrieve the semantic tags of the object we hit. -

Output attributes

+

Output attributes:

-This sensor produces a -[`carla.CollisionEvent`](python_api.md#carla.CollisionEvent) -object for each collision registered - -| Sensor data attribute | Type | Description | -| ---------------------- | ----------- | ----------- | -| `frame` | int | Frame number when the measurement took place | -| `timestamp` | double | Timestamp of the measurement in simulation seconds since the beginning of the episode | -| `transform` | carla.Transform | Transform in world coordinates of the sensor at the time of the measurement | -| `actor` | carla.Actor | Actor that measured the collision ("self" actor) | -| `other_actor` | carla.Actor | Actor against whom we collide | -| `normal_impulse` | carla.Vector3D | Normal impulse result of the collision | - -Note that several collision events might be registered during a single -simulation update. +| Sensor data attribute | Type | Description | +| ---------------------- | ----------- | ----------- | +| `frame` | int | Frame number when the measurement took place. | +| `timestamp` | double | Simulation time of the measurement in seconds since the beginning of the episode. | +| `transform` | [carla.Transform](python_api.md#carla.Transform) | Location and rotation in world coordinates of the sensor at the time of the measurement. | +| `actor` | [carla.Actor](python_api.md#carla.Actor) | Actor that measured the collision (sensor's parent). | +| `other_actor` | [carla.Actor](python_api.md#carla.Actor) | Actor against whom the parent collided. | +| `normal_impulse` | [carla.Vector3D](python_api.md#carla.Vector3D) | Normal impulse result of the collision. | --------------- ##Depth camera * __Blueprint:__ sensor.camera.depth -* __Output:__ [carla.Image](python_api.md#carla.Image) +* __Output:__ [carla.Image](python_api.md#carla.Image) per step. +Provides a view over the scene codifying the distance of each pixel to the camera (also known as **depth buffer** or **z-buffer**) to create a depth map of the elements. [carla.colorConverter](python_api.md#carla.ColorConverter) -![ImageDepth](img/capture_depth.png) -The "Depth" camera provides a view over the scene codifying the distance of each -pixel to the camera (also known as **depth buffer** or **z-buffer**). - -

Basic camera attributes

- -| Blueprint attribute | Type | Default | Description | -| ------------------- | ---- | ------- | ----------- | -| `image_size_x` | int | 800 | Image width in pixels | -| `image_size_y` | int | 600 | Image height in pixels | -| `fov` | float | 90.0 | Horizontal field of view in degrees | -| `sensor_tick` | float | 0.0 | Seconds between sensor captures (ticks) | - -

Camera lens distortion attributes

- -| Blueprint attribute | Type | Default | Description | -|---------------------|------|---------|-------------| -| `lens_circle_falloff` | float | 5.0 | Range: [0.0, 10.0] | -| `lens_circle_multiplier` | float | 0.0 | Range: [0.0, 10.0] | -| `lens_k` | float | -1.0 | Range: [-inf, inf] | -| `lens_kcube` | float | 0.0 | Range: [-inf, inf] | -| `lens_x_size` | float | 0.08 | Range: [0.0, 1.0] | -| `lens_y_size` | float | 0.08 | Range: [0.0, 1.0] | - -

Output attributes

- -This sensor produces [`carla.Image`](python_api.md#carla.Image) -objects. - -| Sensor data attribute | Type | Description | -| --------------------- | ---- | ----------- | -| `frame` | int | Frame number when the measurement took place | -| `timestamp` | double | Timestamp of the measurement in simulation seconds since the beginning of the episode | -| `transform` | carla.Transform | Transform in world coordinates of the sensor at the time of the measurement | -| `width` | int | Image width in pixels | -| `height` | int | Image height in pixels | -| `fov` | float | Horizontal field of view in degrees | -| `raw_data` | bytes | Array of BGRA 32-bit pixels | - - -The image codifies the depth in 3 channels of the RGB color space, from less to -more significant bytes: R -> G -> B. The actual distance in meters can be -decoded with +The image codifies depth value per pixel using 3 channels of the RGB color space, from less to more significant bytes: R -> G -> B. The actual distance in meters can be +decoded with: ``` normalized = (R + G * 256 + B * 256 * 256) / (256 * 256 * 256 - 1) in_meters = 1000 * normalized ``` +![ImageDepth](img/capture_depth.png) + + + +

Basic camera attributes

+ +| Blueprint attribute | Type | Default | Description | +| ------------------- | ---- | ------- | ----------- | +| `image_size_x` | int | 800 | Image width in pixels. | +| `image_size_y` | int | 600 | Image height in pixels. | +| `fov` | float | 90.0 | Horizontal field of view in degrees. | +| `sensor_tick` | float | 0.0 | Seconds between sensor captures (ticks). | + +

Camera lens distortion attributes

+ +| Blueprint attribute | Type | Default | Description | +|--------------------------|-------|---------|-------------| +| `lens_circle_falloff` | float | 5.0 | Range: [0.0, 10.0] | +| `lens_circle_multiplier` | float | 0.0 | Range: [0.0, 10.0] | +| `lens_k` | float | -1.0 | Range: [-inf, inf] | +| `lens_kcube` | float | 0.0 | Range: [-inf, inf] | +| `lens_x_size` | float | 0.08 | Range: [0.0, 1.0] | +| `lens_y_size` | float | 0.08 | Range: [0.0, 1.0] | + +

Output attributes

+ +| Sensor data attribute | Type | Description | +| --------------------- | ------------------------------------------------ | ----------- | +| `frame` | int | Frame number when the measurement took place. | +| `timestamp` | double | Simulation time of the measurement in seconds since the beginning of the episode. | +| `transform` | [carla.Transform](python_api.md#carla.Transform) | Location and rotation in world coordinates of the sensor at the time of the measurement. | +| `width` | int | Image width in pixels. | +| `height` | int | Image height in pixels. | +| `fov` | float | Horizontal field of view in degrees. | +| `raw_data` | bytes | Array of BGRA 32-bit pixels. | + --------------- ##GNSS sensor * __Blueprint:__ sensor.other.gnss -* __Output:__ [carla.GNSSMeasurement](python_api.md#carla.GNSSMeasurement) +* __Output:__ [carla.GNSSMeasurement](python_api.md#carla.GNSSMeasurement) per step. -This sensor, when attached to an actor, reports its current gnss position. -The gnss position is internally calculated by adding the metric position to -an initial geo reference location defined within the OpenDRIVE map definition. +Reports current [gnss position](https://www.gsa.europa.eu/european-gnss/what-gnss) of its parent object. This is calculated by adding the metric position to an initial geo reference location defined within the OpenDRIVE map definition.

Output attributes

-This sensor produces -[`carla.GnssMeasurement`](python_api.md#carla.GnssMeasurement) -objects. - -| Sensor data attribute | Type | Description | -| ---------------------- | ----------- | ----------- | -| `frame` | int | Frame number when the measurement took place | -| `timestamp` | double | Timestamp of the measurement in simulation seconds since the beginning of the episode | -| `transform` | carla.Transform | Transform in world coordinates of the sensor at the time of the measurement | -| `latitude` | double | Latitude position of the actor | -| `longitude` | double | Longitude position of the actor | -| `altitude` | double | Altitude of the actor | +| Sensor data attribute | Type | Description | +| ---------------------- | ------------------------------------------------ | ----------- | +| `frame` | int | Frame number when the measurement took place. | +| `timestamp` | double | Simulation time of the measurement in seconds since the beginning of the episode. | +| `transform` | [carla.Transform](python_api.md#carla.Transform) | Location and rotation in world coordinates of the sensor at the time of the measurement. | +| `latitude` | double | Latitude of the actor. | +| `longitude` | double | Longitude of the actor. | +| `altitude` | double | Altitude of the actor. | --------------- ##IMU sensor * __Blueprint:__ sensor.other.imu -* __Output:__ [carla.IMUMeasurement](python_api.md#carla.IMUMeasurement) +* __Output:__ [carla.IMUMeasurement](python_api.md#carla.IMUMeasurement) per step. -This sensor, when attached to an actor, the user can access to it's accelerometer, gyroscope and compass. +Provides measures that accelerometer, gyroscope and compass would retrieve for the parent object. The data is collected from the object's current state.

Output attributes

-This sensor produces -[`carla.IMUMeasurement`](python_api.md#carla.IMUMeasurement) -objects. - -| Sensor data attribute | Type | Description | -| --------------------- | --------------- | ----------- | -| `frame` | int | Frame number when the measurement took place | -| `timestamp` | double | Timestamp of the measurement in simulation seconds since the beginning of the episode | -| `transform` | carla.Transform | Transform in world | -| `accelerometer` | carla.Vector3D | Measures linear acceleration in `m/s^2` | -| `gyroscope` | carla.Vector3D | Measures angular velocity in `rad/sec` | -| `compass` | float | Orientation with respect to the North (`(0.0, -1.0, 0.0)` in Unreal) in radians | +| Sensor data attribute | Type | Description | +| --------------------- | ------------------------------------------------ | ----------- | +| `frame` | int | Frame number when the measurement took place. | +| `timestamp` | double | Simulation time of the measurement in seconds since the beginning of the episode. | +| `transform` | [carla.Transform](python_api.md#carla.Transform) | Transform in world. | +| `accelerometer` | [carla.Vector3D](python_api.md#carla.Vector3D) | Measures linear acceleration in `m/s^2`. | +| `gyroscope` | [carla.Vector3D](python_api.md#carla.Vector3D) | Measures angular velocity in `rad/sec`. | +| `compass` | float | Orientation in radians. North is `(0.0, -1.0, 0.0)` in UE. | --------------- ##Lane invasion detector * __Blueprint:__ sensor.other.lane_invasion -* __Output:__ [carla.LaneInvasionEvent](python_api.md#carla.LaneInvasionEvent) +* __Output:__ [carla.LaneInvasionEvent](python_api.md#carla.LaneInvasionEvent) per lane marking crossed. > _This sensor is a work in progress, currently very limited._ -This sensor, when attached to an actor, it registers an event each time the -actor crosses a lane marking. This sensor is somehow special as it works fully -on the client-side. The lane invasion uses the road data of the active map to -determine whether a vehicle is invading another lane. This information is based -on the OpenDrive file provided by the map, therefore it is subject to the -fidelity of the OpenDrive description. In some places there might be -discrepancies between the lanes visible by the cameras and the lanes registered -by this sensor. +Registers an event each time its parent crosses a lane marking. +The sensor uses road data provided by the OpenDRIVE description of the map to determine whether the parent vehicle is invading another lane. Thus, there may be discrepancies between the lanes visible by the cameras and the lanes registered by this sensor, as information does not come directly from the simulation. This sensor does not have any configurable attribute. +!!! Important + This sensor works fully on the client-side. +

Output attributes

-This sensor produces a -[`carla.LaneInvasionEvent`](python_api.md#carla.LaneInvasionEvent) -object for each lane marking crossed by the actor - -| Sensor data attribute | Type | Description | -| ----------------------- | ----------- | ----------- | -| `frame` | int | Frame number when the measurement took place | -| `timestamp` | double | Timestamp of the measurement in simulation seconds since the beginning of the episode | -| `transform` | carla.Transform | Transform in world coordinates of the sensor at the time of the measurement | -| `actor` | carla.Actor | Actor that invaded another lane ("self" actor) | -| `crossed_lane_markings` | carla.LaneMarking list | List of lane markings that have been crossed | +| Sensor data attribute | Type | Description | +| ----------------------- | ---------------------------------------------------------- | ----------- | +| `frame` | int | Frame number when the measurement took place. | +| `timestamp` | double | Simulation time of the measurement in seconds since the beginning of the episode. | +| `transform` | [carla.Transform](python_api.md#carla.Transform) | Location and rotation in world coordinates of the sensor at the time of the measurement. | +| `actor` | [carla.Actor](python_api.md#carla.Actor) | Vehicle that invaded another lane (parent actor). | +| `crossed_lane_markings` | list([carla.LaneMarking](python_api.md#carla.LaneMarking)) | List of lane markings that have been crossed. | --------------- ##Lidar raycast sensor * __Blueprint:__ sensor.lidar.ray_cast -* __Output:__ [carla.LidarMeasurement](python_api.md#carla.LidarMeasurement) +* __Output:__ [carla.LidarMeasurement](python_api.md#carla.LidarMeasurement) per step. Each one contains [] -![LidarPointCloud](img/lidar_point_cloud.gif) +This sensor simulates a rotating Lidar implemented using ray-casting. +The points are computed by adding a laser for each channel distributed in the vertical FOV. The rotation is simulated computing the horizontal angle that the Lidar rotated in a frame. The point cloud is calculated by doing a ray-cast for each laser in every frame: +`points_per_second / (FPS * channels)` -This sensor simulates a rotating Lidar implemented using ray-casting. The points -are computed by adding a laser for each channel distributed in the vertical FOV, -then the rotation is simulated computing the horizontal angle that the Lidar -rotated this frame, and doing a ray-cast for each point that each laser was -supposed to generate this frame; `points_per_second / (FPS * channels)`. - -

Lidar attributes

- -| Blueprint attribute | Type | Default | Description | -| -------------------- | ---- | ------- | ----------- | -| `channels` | int | 32 | Number of lasers | -| `range` | float | 10.0 | Maximum measurement distance in meters _(<=0.9.6: is in centimeters)_ | -| `points_per_second` | int | 56000 | Points generated by all lasers per second | -| `rotation_frequency` | float | 10.0 | Lidar rotation frequency | -| `upper_fov` | float | 10.0 | Angle in degrees of the upper most laser | -| `lower_fov` | float | -30.0 | Angle in degrees of the lower most laser | -| `sensor_tick` | float | 0.0 | Seconds between sensor captures (ticks) | - -

Output attributes

- -This sensor produces -[`carla.LidarMeasurement`](python_api.md#carla.LidarMeasurement) -objects. - -| Sensor data attribute | Type | Description | -| -------------------------- | ---------- | ----------- | -| `frame` | int | Frame number when the measurement took place | -| `timestamp` | double | Timestamp of the measurement in simulation seconds since the beginning of the episode | -| `transform` | carla.Transform | Transform in world coordinates of the sensor at the time of the measurement | -| `horizontal_angle` | float | Angle in XY plane of the lidar this frame (in radians) | -| `channels` | int | Number of channels (lasers) of the lidar | -| `get_point_count(channel)` | int | Number of points per channel captured this frame | -| `raw_data` | bytes | Array of 32-bits floats (XYZ of each point) | - -The object also acts as a Python list of [`carla.Location`](python_api.md#carla.Location) +A Lidar measurement contains a packet with all the points generated during a `1/FPS` interval. During this interval the physics is not updated so all the points in a measurement reflect the same "static picture" of the scene. +This output also acts as a list of the [`carla.Location`](python_api.md#carla.Location) for every point: ```py for location in lidar_measurement: print(location) ``` -A Lidar measurement contains a packet with all the points generated during a -`1/FPS` interval. During this interval the physics is not updated so all the -points in a measurement reflect the same "static picture" of the scene. +!!! Tip + Running the simulator at [fixed time-step](configuring_the_simulation.md#fixed-time-step) it is possible to tune the rotation for each measurement. Adjust the + step and the rotation frequency to get, for instance, a 360 view each measurement. -!!! tip - Running the simulator at - [fixed time-step](configuring_the_simulation.md#fixed-time-step) it is - possible to tune the horizontal angle of each measurement. By adjusting the - frame rate and the rotation frequency is possible, for instance, to get a - 360 view each measurement. +![LidarPointCloud](img/lidar_point_cloud.gif) +

Lidar attributes

+ +| Blueprint attribute | Type | Default | Description | +| -------------------- | ---- | ------- | ----------- | +| `channels` | int | 32 | Number of lasers. | +| `range` | float | 10.0 | Maximum distance to measure/raycast in meters (centimeters for CARLA 0.9.6 or previous). | +| `points_per_second` | int | 56000 | Points generated by all lasers per second. | +| `rotation_frequency` | float | 10.0 | Lidar rotation frequency. | +| `upper_fov` | float | 10.0 | Angle in degrees of the highest laser. | +| `lower_fov` | float | -30.0 | Angle in degrees of the lowest laser. | +| `sensor_tick` | float | 0.0 | Seconds between sensor captures (ticks). | + +

Output attributes

+ +| Sensor data attribute | Type | Description | +| -------------------------- | ------------------------------------------------ | ----------- | +| `frame` | int | Frame number when the measurement took place. | +| `timestamp` | double | Simulation time of the measurement in seconds since the beginning of the episode. | +| `transform` | [carla.Transform](python_api.md#carla.Transform) | Location and rotation in world coordinates of the sensor at the time of the measurement. | +| `horizontal_angle` | float | Angle (radians) in the XY plane of the lidar this frame. | +| `channels` | int | Number of channels (lasers) of the lidar. | +| `get_point_count(channel)` | int | Number of points per channel captured this frame. | +| `raw_data` | bytes | Array of 32-bits floats (XYZ of each point). | --------------- ##Obstacle detector * __Blueprint:__ sensor.other.obstacle -* __Output:__ [carla.ObstacleDetectionEvent](python_api.md#carla.ObstacleDetectionEvent) +* __Output:__ [carla.ObstacleDetectionEvent](python_api.md#carla.ObstacleDetectionEvent) per obstacle detected. -This sensor, when attached to an actor, reports if there is obstacles ahead. +Registers an event every time the parent actor has an obstacle ahead. -!!! note - This sensor creates "fake" actors when it detects obstacles with something that is not an actor, - this is so we can retrieve the semantic tags of the object we hit. +!!! Note + This sensor creates "fake" actors when it detects obstacles with something that is not an actor, this is so we can retrieve the semantic tags of the object we hit. | Blueprint attribute | Type | Default | Description | | -------------------- | ---- | ------- | ----------- | -| `distance` | float | 5 | Distance to throw the trace to | -| `hit_radius` | float | 0.5 | Radius of the trace | -| `only_dynamics` | bool | false | If true, the trace will only look for dynamic objects | -| `debug_linetrace` | bool | false | If true, the trace will be visible | -| `sensor_tick` | float | 0.0 | Seconds between sensor captures (ticks) | +| `distance` | float | 5 | Distance to trace. | +| `hit_radius` | float | 0.5 | Radius of the trace. | +| `only_dynamics` | bool | false | If true, the trace will only consider dynamic objects. | +| `debug_linetrace` | bool | false | If true, the trace will be visible. | +| `sensor_tick` | float | 0.0 | Seconds between sensor captures (ticks). |

Output attributes

-This sensor produces -[`carla.ObstacleDetectionEvent`](python_api.md#carla.ObstacleDetectionEvent) -objects. - -| Sensor data attribute | Type | Description | -| ---------------------- | ----------- | ----------- | -| `frame` | int | Frame number when the measurement took place | -| `timestamp` | double | Timestamp of the measurement in simulation seconds since the beginning of the episode | -| `transform` | carla.Transform | Transform in world | -| `actor` | carla.Actor | Actor that detected the obstacle ("self" actor) | -| `other_actor` | carla.Actor | Actor detected as obstacle | -| `distance` | float | Distance from actor to other_actor | +| Sensor data attribute | Type | Description | +| ---------------------- | ------------------------------------------------ | ----------- | +| `frame` | int | Frame number when the measurement took place. | +| `timestamp` | double | Simulation time of the measurement in seconds since the beginning of the episode. | +| `transform` | [carla.Transform](python_api.md#carla.Transform) | Location and rotation in world coordinates of the sensor at the time of the measurement. | +| `actor` | [carla.Actor](python_api.md#carla.Actor) | Actor that detected the obstacle (parent actor). | +| `other_actor` | [carla.Actor](python_api.md#carla.Actor) | Actor detected as an obstacle. | +| `distance` | float | Distance from `actor` to `other_actor`. | --------------- ##Radar sensor * __Blueprint:__ sensor.other.radar -* __Output:__ [carla.RadarMeasurement](python_api.md#carla.RadarMeasurement) +* __Output:__ [carla.RadarMeasurement](python_api.md#carla.RadarMeasurement) per step. + +Creates a 2D point map that models elements in sight and their speed regarding the sensor. This can be used to shape elements and evaluate their movement and direction. +The points measured are contained in [carla.RadarMeasurement](python_api.md#carla.RadarMeasurement) as an array of [carla.RadarDetection](python_api.md#carla.RadarDetection). + +| Blueprint attribute | Type | Default | Description | +| --------------------- | ---- | ------- | ----------- | +| `horizontal_fov` | float | | Horizontal field of view in degrees. | +| `points_per_second` | int | | Points generated by all lasers per second. | +| `range` | float | | Maximum distance to measure/raycast in meters. | +| `sensor_tick` | float | | Seconds between sensor captures (ticks). | +| `vertical_fov` | float | | Vertical field of view in degrees. | + +

Output attributes

+ +| Sensor data attribute | Type | Description | +| ---------------------- | ------------------------------------------------ | ----------- | +| | | | +| | | | --------------- ##RGB camera * __Blueprint:__ sensor.camera.rgb -* __Output:__ [carla.Image](python_api.md#carla.Image) +* __Output:__ [carla.Image](python_api.md#carla.Image) per step. +The "RGB" camera acts as a regular camera capturing images from the scene. [carla.colorConverter](python_api.md#carla.ColorConverter) +If `enable_postprocess_effects` is enabled, a set of post-process effects is applied to the image for the sake of realism: + +* __Vignette:__ darkens the border of the screen. +* __Grain jitter:__ adds some noise to the render. +* __Bloom:__ intense lights burn the area around them. +* __Auto exposure:__ modifies the image gamma to simulate the eye adaptation to darker or brighter areas. +* __Lens flares:__ simulates the reflection of bright objects on the lens. +* __Depth of field:__ blurs objects near or very far away of the camera. + + +The `sensor_tick` tells how fast we want the sensor to capture the data. +A value of 1.5 means that we want the sensor to capture data each second and a half. By default a value of 0.0 means as fast as possible. + ![ImageRGB](img/capture_scenefinal.png) -The "RGB" camera acts as a regular camera capturing images from the scene. -

Basic camera attributes

-| Blueprint attribute | Type | Default | Description | -|---------------------|------|---------|-------------| -| `sensor_tick` | float | 0.0 | Seconds between sensor captures (ticks) | -| `image_size_x` | int | 800 | Image width in pixels | -| `image_size_y` | int | 600 | Image height in pixels | -| `gamma` | float | 2.2 | Target gamma value of the camera | -| `fov` | float | 90.0 | Horizontal field of view in degrees | -| `shutter_speed` | float | 60.0 | The camera shutter speed in seconds (1.0 / s) | -| `iso` | float | 1200.0 | The camera sensor sensitivity | -| `fstop` | float | 1.4 | Defines the opening of the camera lens. Aperture is `1 / fstop` with typical lens going down to f / 1.2 (larger opening). Larger numbers will reduce the Depth of Field effect | +| Blueprint attribute | Type | Default | Description | +|---------------------|-------|---------|-------------| +| `fov` | float | 90.0 | Horizontal field of view in degrees. | +| `fstop` | float | 1.4 | Opening of the camera lens. Aperture is `1 / fstop` with typical lens going down to f / 1.2 (larger opening). Larger numbers will reduce the Depth of Field effect. | +| `image_size_x` | int | 800 | Image width in pixels. | +| `image_size_y` | int | 600 | Image height in pixels. | +| `iso` | float | 1200.0 | The camera sensor sensitivity. | +| `gamma` | float | 2.2 | Target gamma value of the camera. | +| `sensor_tick` | float | 0.0 | Seconds between sensor captures (ticks). | +| `shutter_speed` | float | 60.0 | The camera shutter speed in seconds (1.0 / s). |

Camera lens distortion attributes

-| Blueprint attribute | Type | Default | Description | -|---------------------|------|---------|-------------| -| `lens_circle_falloff` | float | 5.0 | Range: [0.0, 10.0] | -| `lens_circle_multiplier` | float | 0.0 | Range: [0.0, 10.0] | -| `lens_k` | float | -1.0 | Range: [-inf, inf] | -| `lens_kcube` | float | 0.0 | Range: [-inf, inf] | -| `lens_x_size` | float | 0.08 | Range: [0.0, 1.0] | -| `lens_y_size` | float | 0.08 | Range: [0.0, 1.0] | +| Blueprint attribute | Type | Default | Description | +|--------------------------|-------|---------|-------------| +| `lens_circle_falloff` | float | 5.0 | Range: [0.0, 10.0] | +| `lens_circle_multiplier` | float | 0.0 | Range: [0.0, 10.0] | +| `lens_k` | float | -1.0 | Range: [-inf, inf] | +| `lens_kcube` | float | 0.0 | Range: [-inf, inf] | +| `lens_x_size` | float | 0.08 | Range: [0.0, 1.0] | +| `lens_y_size` | float | 0.08 | Range: [0.0, 1.0] |

Advanced camera attributes

-Since these effects are provided from Unreal Engine 4, please make sure to check their documentation on how they work and the relation between them: +Since these effects are provided by UE, please make sure to check their documentation: * [Automatic Exposure][AutomaticExposure.Docs] * [Cinematic Depth of Field Method][CinematicDOFMethod.Docs] @@ -333,120 +316,59 @@ Since these effects are provided from Unreal Engine 4, please make sure to check [CinematicDOFMethod.Docs]: https://docs.unrealengine.com/en-US/Engine/Rendering/PostProcessEffects/DepthOfField/CinematicDOFMethods/index.html [ColorGrading.Docs]: https://docs.unrealengine.com/en-US/Engine/Rendering/PostProcessEffects/ColorGrading/index.html -| Blueprint attribute | Type | Default | Description | -|---------------------|------|---------|-------------| -| `min_fstop` | float | 1.2 | Maximum Aperture | -| `blade_count` | int | 5 | The number of blades that make up the diaphragm mechanism | -| `exposure_mode` | str | `"manual"` | Can be `"manual"` or `"histogram"`. More info in [UE4 official docs][AutomaticExposure.gamesetting] | -| `exposure_compensation` | float | 3.0 | Logarithmic adjustment for the exposure. 0: no adjustment, -1:2x darker, -2:4 darker, 1:2x brighter, 2:4x brighter | -| `exposure_min_bright` | float | 0.1 | Used when `exposure_mode`:`"histogram"` The minimum brightness for auto exposure that limits the lower brightness the eye can adapt within. Values must be greater than 0 and should be less than or equal to `exposure_max_bright` | -| `exposure_max_bright` | float | 2.0 | Used when `exposure_mode`:`"histogram"` The maximum brightness for auto exposure that limits the upper brightness the eye can adapt within. Values must be greater 0 and should be greater than or equal to `exposure_min_bright` | -| `exposure_speed_up` | float | 3.0 | Used when `exposure_mode`:`"histogram"` The speed at which the adaptation occurs from a dark environment to a bright environment | -| `exposure_speed_down` | float | 1.0 | Used when `exposure_mode`:`"histogram"` The speed at which the adaptation occurs from a bright environment to a dark environment | -| `calibration_constant` | float | 16.0 | Calibration constant for 18% Albedo | -| `focal_distance` | float | 1000.0 | The distance in which the depth of field effect should be sharp. This value is measured in Unreal Units (cm) | -| `blur_amount` | float | 1.0 | Strength/intensity of motion blur | -| `blur_radius` | float | 0.0 | Radius in pixels at 1080p resolution to apply according to distance from camera to emulate atmospheric scattering | -| `motion_blur_intensity` | float | 0.45 | Strength of motion blur. 1 is max and 0 is off | -| `motion_blur_max_distortion` | float | 0.35 | Max distortion caused by motion blur, in percent of the screen width, 0 is off | -| `motion_blur_min_object_screen_size` | float | 0.1 | Percentage of screen width objects must have for motion blur, lower value means less draw calls -| `slope` | float | 0.88 | This will adjust the steepness of the S-curve used for the tonemapper, where larger values will make the slope steeper (darker) and lower values will make the slope less steep (lighter). Range: [0.0, 1.0] | -| `toe` | float | 0.55 | This will adjust the dark color in the tonemapper. Range: [0.0, 1.0] | -| `shoulder` | float | 0.26 | This will adjust the bright color in the tonemapper. Range: [0.0, 1.0] | -| `black_clip` | float | 0.0 | This will set where the crossover happens when black tones start to cut off their value. In general, this value should NOT be adjusted. Range: [0.0, 1.0] | -| `white_clip` | float | 0.04 | This will set where the crossover happens when white tones start to cut off their value. This will appear as a subtle change in most cases. Range: [0.0, 1.0] | -| `temp` | float | 6500.0 | This will adjust the white balance in relation to the temperature of the light in the scene. When the light temperature and this one match, the light will appear white. When a value is used that is higher than the light in the scene it will yield a "warm" or yellow color, and, conversely, if the value is lower, it would yield a "cool" or blue color | -| `tint` | float | 0.0 | This will adjust the white balance temperature tint for the scene by adjusting the cyan and magenta color ranges. Ideally, this setting should be used once you've adjusted the white balance Temp property to get accurate colors. Under some light temperatures, the colors may appear to be more yellow or blue. This can be used to balance the resulting color to look more natural | -| `chromatic_aberration_intensity` | float | 0.0 | Scaling factor that controls how much color shifting occurs, more noticable on the screen borders | -| `chromatic_aberration_offset` | float | 0.0 | Normalized distance to the center of the image where the effect takes place | -| `enable_postprocess_effects` | bool | True | Whether the post-process effect in the scene affects the image | +| Blueprint attribute | Type | Default | Description | +|--------------------------------------|------|-------------|-------------| +| `min_fstop` | float | 1.2 | Maximum aperture. | +| `blade_count` | int | 5 | Number of blades that make up the diaphragm mechanism. | +| `exposure_mode` | str | `"manual"` | Can be `"manual"` or `"histogram"`. More in [UE4 docs][AutomaticExposure.gamesetting]. | +| `exposure_compensation` | float | 3.0 | Logarithmic adjustment for the exposure. 0: no adjustment, -1:2x darker, -2:4 darker, 1:2x brighter, 2:4x brighter. | +| `exposure_min_bright` | float | 0.1 | In `exposure_mode: "histogram"`. Minimum brightness for auto exposure. The lowest the eye can adapt within. Must be greater than 0 and less than or equal to `exposure_max_bright`. | +| `exposure_max_bright` | float | 2.0 | In `exposure_mode: "histogram"`. Maximum brightness for auto exposure. The highestthe eye can adapt within. Must be greater than 0 and greater than or equal to `exposure_min_bright`. | +| `exposure_speed_up` | float | 3.0 | In `exposure_mode: "histogram"`. Speed at which the adaptation occurs from dark to bright environment. | +| `exposure_speed_down` | float | 1.0 | In `exposure_mode: "histogram"`. Speed at which the adaptation occurs from bright to dark environment. | +| `calibration_constant` | float | 16.0 | Calibration constant for 18% albedo. | +| `focal_distance` | float | 1000.0 | Distance at which the depth of field effect should be sharp. Measured in cm (UE units). | +| `blur_amount` | float | 1.0 | Strength/intensity of motion blur. | +| `blur_radius` | float | 0.0 | Radius in pixels at 1080p resolution to emulate atmospheric scattering according to distance from camera. | +| `motion_blur_intensity` | float | 0.45 | Strength of motion blur [0,1]. | +| `motion_blur_max_distortion` | float | 0.35 | Max distortion caused by motion blur. Percentage of screen width. | +| `motion_blur_min_object_screen_size` | float | 0.1 | Percentage of screen width objects must have for motion blur, lower value means less draw calls. | +| `slope` | float | 0.88 | Steepness of the S-curve for the tonemapper. Larger values make the slope steeper (darker) [0.0, 1.0]. | +| `toe` | float | 0.55 | Adjusts dark color in the tonemapper [0.0, 1.0] | +| `shoulder` | float | 0.26 | Adjusts bright color in the tonemapper [0.0, 1.0] | +| `black_clip` | float | 0.0 | This should NOT be adjusted. Sets where the crossover happens and black tones start to cut off their value [0.0, 1.0]. | +| `white_clip` | float | 0.04 | Set where the crossover happens and white tones start to cut off their value. Subtle change in most cases [0.0, 1.0]. | +| `temp` | float | 6500.0 | White balance in relation to the temperature of the light in the scene. __White light:__ when this matches light temperature. __Warm light:__ When higher than the light in the scene, it is a yellowish color. __Cool light:__ When lower than the light. Blueish color. | +| `tint` | float | 0.0 | White balance temperature tint. Adjusts cyan and magenta color ranges. This should be used along with the white balance Temp property to get accurate colors. Under some light temperatures, the colors may appear to be more yellow or blue. This can be used to balance the resulting color to look more natural. | +| `chromatic_aberration_intensity` | float | 0.0 | Scaling factor to control color shifting, more noticeable on the screen borders. | +| `chromatic_aberration_offset` | float | 0.0 | Normalized distance to the center of the image where the effect takes place. | +| `enable_postprocess_effects` | bool | True | Post-process effects activation. | [AutomaticExposure.gamesetting]: https://docs.unrealengine.com/en-US/Engine/Rendering/PostProcessEffects/AutomaticExposure/index.html#gamesetting -The `sensor_tick` tells how fast we want the sensor to capture the data. -A value of 1.5 means that we want the sensor to capture data each second and a half. -By default a value of 0.0 means as fast as possible. - - - -

Output attributes

-This sensor produces [`carla.Image`](python_api.md#carla.Image) -objects. - -| Sensor data attribute | Type | Description | -| --------------------- | ---- | ----------- | -| `frame` | int | Frame number when the measurement took place | -| `timestamp` | double | Timestamp of the measurement in simulation seconds since the beginning of the episode | -| `transform` | carla.Transform | Transform in world coordinates of the sensor at the time of the measurement | -| `width` | int | Image width in pixels | -| `height` | int | Image height in pixels | -| `fov` | float | Horizontal field of view in degrees | -| `raw_data` | bytes | Array of BGRA 32-bit pixels | +| Sensor data attribute | Type | Description | +| --------------------- | ------------------------------------------------ | ----------- | +| `frame` | int | Frame number when the measurement took place. | +| `timestamp` | double | Simulation time of the measurement in seconds since the beginning of the episode. | +| `transform` | [carla.Transform](python_api.md#carla.Transform) | Location and rotation in world coordinates of the sensor at the time of the measurement. | +| `width` | int | Image width in pixels. | +| `height` | int | Image height in pixels. | +| `fov` | float | Horizontal field of view in degrees. | +| `raw_data` | bytes | Array of BGRA 32-bit pixels. | --------------- ##Semantic segmentation camera * __Blueprint:__ sensor.camera.semantic_segmentation -* __Output:__ [carla.Image](python_api.md#carla.Image) +* __Output:__ [carla.Image](python_api.md#carla.Image) per step. +This camera classifies every object in sight by displaying it in a different color according to its class (e.g., pedestrians in a different color than vehicles). This is implemented by tagging every object in the scene before hand (either at begin play or on spawn). The objects are classified by their relative file path in the project. E.g., every mesh stored in the _"Unreal/CarlaUE4/Content/Static/Pedestrians"_ folder is tagged as `Pedestrian`. [carla.colorConverter](python_api.md#carla.ColorConverter) -![ImageSemanticSegmentation](img/capture_semseg.png) - -The "Semantic Segmentation" camera classifies every object in the view by -displaying it in a different color according to the object class. E.g., -pedestrians appear in a different color than vehicles. - -

Basic camera attributes

- -| Blueprint attribute | Type | Default | Description | -| ------------------- | ---- | ------- | ----------- | -| `image_size_x` | int | 800 | Image width in pixels | -| `image_size_y` | int | 600 | Image height in pixels | -| `fov` | float | 90.0 | Horizontal field of view in degrees | -| `sensor_tick` | float | 0.0 | Seconds between sensor captures (ticks) | - -

Camera lens distortion attributes

- -| Blueprint attribute | Type | Default | Description | -|---------------------|------|---------|-------------| -| `lens_circle_falloff` | float | 5.0 | Range: [0.0, 10.0] | -| `lens_circle_multiplier` | float | 0.0 | Range: [0.0, 10.0] | -| `lens_k` | float | -1.0 | Range: [-inf, inf] | -| `lens_kcube` | float | 0.0 | Range: [-inf, inf] | -| `lens_x_size` | float | 0.08 | Range: [0.0, 1.0] | -| `lens_y_size` | float | 0.08 | Range: [0.0, 1.0] | - -

Output attributes

- -This sensor produces [`carla.Image`](python_api.md#carla.Image) -objects. - -| Sensor data attribute | Type | Description | -| --------------------- | ---- | ----------- | -| `frame` | int | Frame number when the measurement took place | -| `timestamp` | double | Timestamp of the measurement in simulation seconds since the beginning of the episode | -| `transform` | carla.Transform | Transform in world coordinates of the sensor at the time of the measurement | -| `width` | int | Image width in pixels | -| `height` | int | Image height in pixels | -| `fov` | float | Horizontal field of view in degrees | -| `raw_data` | bytes | Array of BGRA 32-bit pixels | - -The server provides an image with the tag information **encoded in the red -channel**. A pixel with a red value of x displays an object with tag x. The -following tags are currently available - +The server provides an image with the tag information **encoded in the red channel**: A pixel with a red value of `x` belongs to an object with tag `x`. +The following tags are currently available: | Value | Tag | Converted color | | -----:|:------------ | --------------- | @@ -464,14 +386,40 @@ following tags are currently available | 11 | Wall | (102, 102, 156) | | 12 | Traffic sign | (220, 220, 0) | -This is implemented by tagging every object in the scene before hand (either at -begin play or on spawn). The objects are classified by their relative file path -in the project. E.g., every mesh stored in the -_"Unreal/CarlaUE4/Content/Static/Pedestrians"_ folder it's tagged as pedestrian. - -!!! note +!!! Note **Adding new tags**: - At the moment adding new tags is not very flexible and requires to modify - the C++ code. Add a new label to the `ECityObjectLabel` enum in "Tagger.h", - and its corresponding filepath check inside `GetLabelByFolderName()` - function in "Tagger.cpp". + It requires some C++ coding. Add a new label to the `ECityObjectLabel` enum in "Tagger.h", and its corresponding filepath check inside `GetLabelByFolderName()` function in "Tagger.cpp". + +![ImageSemanticSegmentation](img/capture_semseg.png) + +

Basic camera attributes

+ +| Blueprint attribute | Type | Default | Description | +| ------------------- | ---- | ------- | ----------- | +| `fov` | float | 90.0 | Horizontal field of view in degrees. | +| `image_size_x` | int | 800 | Image width in pixels. | +| `image_size_y` | int | 600 | Image height in pixels. | +| `sensor_tick` | float | 0.0 | Seconds between sensor captures (ticks). | + +

Camera lens distortion attributes

+ +| Blueprint attribute | Type | Default | Description | +|------------------------- |------ |---------|-------------| +| `lens_circle_falloff` | float | 5.0 | Range: [0.0, 10.0] | +| `lens_circle_multiplier` | float | 0.0 | Range: [0.0, 10.0] | +| `lens_k` | float | -1.0 | Range: [-inf, inf] | +| `lens_kcube` | float | 0.0 | Range: [-inf, inf] | +| `lens_x_size` | float | 0.08 | Range: [0.0, 1.0] | +| `lens_y_size` | float | 0.08 | Range: [0.0, 1.0] | + +

Output attributes

+ +| Sensor data attribute | Type | Description | +| --------------------- | ------------------------------------------------ | ----------- | +| `fov` | float | Horizontal field of view in degrees. | +| `frame` | int | Frame number when the measurement took place. | +| `height` | int | Image height in pixels. | +| `raw_data` | bytes | Array of BGRA 32-bit pixels. | +| `timestamp` | double | Simulation time of the measurement in seconds since the beginning of the episode. | +| `transform` | [carla.Transform](python_api.md#carla.Transform) | Location and rotation in world coordinates of the sensor at the time of the measurement. | +| `width` | int | Image width in pixels. | diff --git a/mkdocs.yml b/mkdocs.yml index 2be8801e5..329e5273b 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -33,6 +33,7 @@ nav: - 'Blueprint Library': 'bp_library.md' - 'C++ reference' : 'cpp_reference.md' - 'Recorder binary file format': 'recorder_binary_file_format.md' + - "Sensors' documentation": 'ref_sensors.md' - How to... (general): - 'Add a new sensor': 'dev/how_to_add_a_new_sensor.md' - 'Add friction triggers': "how_to_add_friction_triggers.md"