DVS docs reviewed

2020-05-08 16:57:44 +02:00 · 2020-05-08 16:57:44 +02:00 · d305afa2fa
parent cfee8bdfaa
commit d305afa2fa
3 changed files with 176 additions and 98 deletions
--- a/Docs/python_api.md
+++ b/Docs/python_api.md
@ -504,6 +504,61 @@ No changes applied to the image.
 ---
 ## carla.DVSEvent<a name="carla.DVSEvent"></a>
 Class that defines a DVS event. An event is a quadruple, so a tuple of 4 elements, with `x`, `y` pixel coordinate location, timestamp `t` and polarity `pol` of the event. Learn more about them [here](ref_sensors.md).  
 <h3>Instance Variables</h3>
 - <a name="carla.DVSEvent.x"></a>**<font color="#f8805a">x</font>** (_int_)  
 X pixel coordinate.  
 - <a name="carla.DVSEvent.y"></a>**<font color="#f8805a">y</font>** (_int_)  
 Y pixel coordinate.  
 - <a name="carla.DVSEvent.t"></a>**<font color="#f8805a">t</font>** (_int_)  
 Timestamp of the moment the event happened.  
 - <a name="carla.DVSEvent.pol"></a>**<font color="#f8805a">pol</font>** (_bool_)  
 Polarity of the event. __True__ for positive and __False__ for negative.  
 <h3>Methods</h3>
 <h3>Dunder methods</h3>
 - <a name="carla.DVSEvent.__str__"></a>**<font color="#7fb800">\__str__</font>**(<font color="#00a6ed">**self**</font>)  
 ---
 ## carla.DVSEventArray<a name="carla.DVSEventArray"></a>
 Class that defines a stream of events in [[carla.DVSEvent](#carla.DVSEvent)](#[carla.DVSEvent](#carla.DVSEvent)). Such stream is an array of arbitrary size depending on the number of events. This class also stores the field of view, the height and width of the image and the timestamp from convenience. Learn more about them [here](ref_sensors.md).  
 <h3>Instance Variables</h3>
 - <a name="carla.DVSEventArray.fov"></a>**<font color="#f8805a">fov</font>** (_float_)  
 Horizontal field of view of the image in degrees.  
 - <a name="carla.DVSEventArray.height"></a>**<font color="#f8805a">height</font>** (_int_)  
 Image height in pixels.  
 - <a name="carla.DVSEventArray.width"></a>**<font color="#f8805a">width</font>** (_int_)  
 Image width in pixels.  
 - <a name="carla.DVSEventArray.raw_data"></a>**<font color="#f8805a">raw_data</font>** (_bytes_)  
 <h3>Methods</h3>
 - <a name="carla.DVSEventArray.to_image"></a>**<font color="#7fb800">to_image</font>**(<font color="#00a6ed">**self**</font>)  
 Converts the image following this pattern: blue indicates positive events, red indicates negative events.  
 - <a name="carla.DVSEventArray.to_array"></a>**<font color="#7fb800">to_array</font>**(<font color="#00a6ed">**self**</font>)  
 Converts the stream of events to an array of int values in the following order <code>[x, y, t, pol]</code>.  
 - <a name="carla.DVSEventArray.to_array_x"></a>**<font color="#7fb800">to_array_x</font>**(<font color="#00a6ed">**self**</font>)  
 Returns an array with X pixel coordinate of all the events in the stream.  
 - <a name="carla.DVSEventArray.to_array_y"></a>**<font color="#7fb800">to_array_y</font>**(<font color="#00a6ed">**self**</font>)  
 Returns an array with Y pixel coordinate of all the events in the stream.  
 - <a name="carla.DVSEventArray.to_array_t"></a>**<font color="#7fb800">to_array_t</font>**(<font color="#00a6ed">**self**</font>)  
 Returns an array with the timestamp of all the events in the stream.  
 - <a name="carla.DVSEventArray.to_array_pol"></a>**<font color="#7fb800">to_array_pol</font>**(<font color="#00a6ed">**self**</font>)  
 Returns an array with the polarity of all the events in the stream.  
 <h3>Dunder methods</h3>
 - <a name="carla.DVSEventArray.__getitem__"></a>**<font color="#7fb800">\__getitem__</font>**(<font color="#00a6ed">**self**</font>, <font color="#00a6ed">**pos**=int</font>)  
 - <a name="carla.DVSEventArray.__iter__"></a>**<font color="#7fb800">\__iter__</font>**(<font color="#00a6ed">**self**</font>)  
 - <a name="carla.DVSEventArray.__len__"></a>**<font color="#7fb800">\__len__</font>**(<font color="#00a6ed">**self**</font>)  
 - <a name="carla.DVSEventArray.__setitem__"></a>**<font color="#7fb800">\__setitem__</font>**(<font color="#00a6ed">**self**</font>, <font color="#00a6ed">**pos**=int</font>, <font color="#00a6ed">**color**=[carla.Color](#carla.Color)</font>)  
 - <a name="carla.DVSEventArray.__str__"></a>**<font color="#7fb800">\__str__</font>**(<font color="#00a6ed">**self**</font>)  
 ---
 ## carla.DebugHelper<a name="carla.DebugHelper"></a>
 Helper class part of [carla.World](#carla.World) that defines methods for creating debug shapes. By default, shapes last one second. They can be permanent, but take into account the resources needed to do so. Check out this [recipe](ref_code_recipes.md#debug-bounding-box-recipe) where the user takes a snapshot of the world and then proceeds to draw bounding boxes for traffic lights.  
@ -2769,58 +2824,4 @@ Links another command to be executed right after. It allows to ease very common
    - **Parameters:**
        - `command` (_any carla Command_) – a Carla command.  
 ---
 ## carla.DVSEvent<a name="carla.DVSEvent"></a>
 Class that defines a DVS event. An event is a quadruple, so a tuple of 4 elements, with x, y pixel coordinate location, timestamp t and polarity of the event. Learn more about them [here](ref_sensors.md).  
 <h3>Instance Variables</h3>
 - <a name="carla.DVSEvent.x"></a>**<font color="#f8805a">x</font>** (_int_)  
 X pixel coordinate.
 - <a name="carla.DVSEvent.y"></a>**<font color="#f8805a">y</font>** (_int_)  
 Y pixel coordinate.  
 - <a name="carla.DVSEvent.t"></a>**<font color="#f8805a">t</font>** (_int_)  
 Timestamp.
 - <a name="carla.DVSEvent.pol"></a>**<font color="#f8805a">pol</font>** (_bool_)  
 Polarity of the event in boolean. True for positive and False for negative.
 <h3>Dunder methods</h3>
 - <a name="carla.DVSEvent.__str__"></a>**<font color="#7fb800">\__str__</font>**(<font color="#00a6ed">**self**</font>) 
 ---
 ## carla.DVSEventArray<a name="carla.DVSEventArray"></a>
 Class that defines a stream of events in [carla.DVSEvent](#carla.DVSEvent). Such stream is an array of arbitrary size depending on the number of events. This class also stores the field of view, the height and width of the image and the timestamp from convenience. Learn more about them [here](ref_sensors.md).  
 <h3>Instance Variables</h3>
 - <a name="carla.DVSEventArray.fov"></a>**<font color="#f8805a">fov</font>** (_float_)  
 Horizontal field of view of the image in degrees.  
 - <a name="carla.DVSEventArray.height"></a>**<font color="#f8805a">height</font>** (_int_)  
 Image height in pixels.  
 - <a name="carla.DVSEventArray.width"></a>**<font color="#f8805a">width</font>** (_int_)  
 Image width in pixels.  
 - <a name="carla.DVSEventArray.raw_data"></a>**<font color="#f8805a">raw_data</font>** (_bytes_)  
 <h3>Methods</h3>
 - <a name="carla.DVSEventArray.to_image"></a>**<font color="#7fb800">to_image</font>**(<font color="#00a6ed">**self**</font>) 
 Converts the image following this pattern: blue indicates positive events, red indicates negative events.
 - <a name="carla.DVSEventArray.to_array"></a>**<font color="#7fb800">to_array</font>**(<font color="#00a6ed">**self**</font>)
 Convert the stream of events to an array of integer values in the following order [x, y, t, pol].
 - <a name="carla.DVSEventArray.to_array_x"></a>**<font color="#7fb800">to_array_x</font>**(<font color="#00a6ed">**self**</font>)
 Return an array with X pixel coordinate of all the events in the stream.
 - <a name="carla.DVSEventArray.to_array_y"></a>**<font color="#7fb800">to_array_y</font>**(<font color="#00a6ed">**self**</font>)
 Return an array with Y pixel coordinate of all the events in the stream.
 - <a name="carla.DVSEventArray.to_array_t"></a>**<font color="#7fb800">to_array_t</font>**(<font color="#00a6ed">**self**</font>)
 Return an array with the timestamp of all the events in the stream.
 - <a name="carla.DVSEventArray.to_array_pol"></a>**<font color="#7fb800">to_array_pol</font>**(<font color="#00a6ed">**self**</font>)
 Return an array with the polarity of all the events in the stream.
 <h3>Dunder methods</h3>
 - <a name="carla.DVSEventArray.__getitem__"></a>**<font color="#7fb800">\__getitem__</font>**(<font color="#00a6ed">**self**</font>, <font color="#00a6ed">**pos**=int</font>)  
 - <a name="carla.DVSEventArray.__iter__"></a>**<font color="#7fb800">\__iter__</font>**(<font color="#00a6ed">**self**</font>)  
 - <a name="carla.DVSEventArray.__len__"></a>**<font color="#7fb800">\__len__</font>**(<font color="#00a6ed">**self**</font>)  
 - <a name="carla.DVSEventArray.__setitem__"></a>**<font color="#7fb800">\__setitem__</font>**(<font color="#00a6ed">**self**</font>, <font color="#00a6ed">**pos**=int</font>, <font color="#00a6ed">**color**=[carla.Color](#carla.Color)</font>)  
 - <a name="carla.DVSEventArray.__str__"></a>**<font color="#7fb800">\__str__</font>**(<font color="#00a6ed">**self**</font>)  
 ---
--- a/Docs/ref_sensors.md
+++ b/Docs/ref_sensors.md
@ -1419,59 +1419,39 @@ The following tags are currently available:
 *   __Output:__ [carla.DVSEventArray](python_api.md#carla.DVSEventArray) per step (unless `sensor_tick` says otherwise).  
-A Dynamic Vision Sensor (DVS) or Event camera is a sensor that works
+A Dynamic Vision Sensor (DVS) or Event camera is a sensor that works radically differently from a conventional camera. Instead of capturing
-radically differently from a conventional camera. Instead of capturing
+intensity images at a fixed rate, event cameras measure changes of intensity asynchronously, in the form of a stream of events, which encode per-pixel
-intensity images at a fixed rate, event cameras measure changes of intensity
+brightness changes. Event cameras possess outstanding properties when compared to standard cameras. They have a very high dynamic range (140 dB
-asynchronously, in the form of a stream of events, which encode per-pixel
+versus 60 dB), no motion blur, and high temporal resolution (in the order of microseconds). Event cameras are thus sensors that can provide high-quality
-brightness changes. Event cameras possess outstanding properties when
+visual information even in challenging high-speed scenarios and high dynamic range environments, enabling new application domains for vision-based
 compared to standard cameras. They have a very high dynamic range (140 dB
 versus 60 dB), no motion blur, and high temporal resolution (in the order of
 microseconds). Event cameras are thus sensors that can provide high-quality
 visual information even in challenging high-speed scenarios and high dynamic
 range environments, enabling new application domains for vision-based
 algorithms.
-The DVS camera outputs a stream of events. An event $e=(x,y,t,pol)$ is
+The DVS camera outputs a stream of events. An event `e=(x,y,t,pol)` is triggered at a pixel `x`, `y` at a timestamp `t` when the change in
-triggered at a pixel $x$, $y$ at a timestamp $t$ when the change in
+logarithmic intensity `L` reaches a predefined constant threshold `C` (typically between 15% and 30%).
 logarithmic intensity $L$ reaches a predefined constant threshold $C$
 (typically between 15% and 30%),
-$$
+``
 L(x,y,t) - L(x,y,t-\delta t) = pol C
-$$
+``
-where $t-\delta t$ is the time when the last event at that pixel was
+`t-\delta t` is the time when the last event at that pixel was triggered and `pol` is the polarity of the event according to the sign of the
-triggered and $pol$ is the polarity of the event according to the sign of the
+brightness change. The polarity is positive `+1` when there is increment in brightness and negative `-1` when a decrement in brightness occurs. The
-brightness change. The polarity is positive $+1$ when there is increment in
+working principles depicted in the following figure. The standard camera outputs frames at a fixed rate, thus sending redundant information
-brightness and negative $-1$ when a decrement in brightness occurs. The
+when no motion is present in the scene. In contrast, event cameras are data-driven sensors that respond to brightness changes with microsecond
-working principles depicted in the following figure. The standard
+latency. At the plot, a positive (resp. negative) event (blue dot, resp. red dot) is generated whenever the (signed) brightness change exceeds the
-camera outputs frames at a fixed rate, thus sending redundant information
+contrast threshold `C` for one dimension `x` over time `t`. Observe how the event rate grows when the signal changes rapidly.
 when no motion is present in the scene. In contrast, event cameras are
 data-driven sensors that respond to brightness changes with microsecond
 latency. At the plot, a positive (resp. negative) event (blue dot, resp. red
 dot) is generated whenever the (signed) brightness change exceeds the
 contrast threshold $C$ for one dimension $x$ over time $t$. Observe how the
 event rate grows when the signal changes rapidly.
 ![DVSCameraWorkingPrinciple](img/sensor_dvs_scheme.jpg)
-The current implementation of the DVS camera works in a uniform sampling
+The current implementation of the DVS camera works in a uniform sampling manner between two consecutive synchronous frames. Therefore, in order to
-manner between two consecutive synchronous frames. Therefore, in order to
+emulate the high temporal resolution (order of microseconds) of a real event camera, the sensor requires to execute at a high frequency (much higher
-emulate the high temporal resolution (order of microseconds) of a real event
+frequency than a conventional camera). Effectively, the number of events increases as faster a CARLA car drives. Therefore, the sensor frequency
-camera, the sensor requires to execute at a high frequency (much higher
+should increase accordingly with the dynamic of the scene. The user should find their balance between time accuracy and computational cost.
 frequency than a conventional camera). Effectively, the number of events
 increases as faster a Carla car drives. Therefore, the sensor frequency
 should increase accordingly with the dynamic of the scene. The user should find
 their balance between time accuracy and computational cost.
-The provided script `manual_control.py` uses the DVS camera in order to show
+The provided script `manual_control.py` uses the DVS camera in order to show how to configure the sensor, how to get the stream of events and how to depict such events in an image format, usually called event frame.
 how to configure the sensor, how to get the stream of events and how to depict such
 events in an image format, usually called event frame.
 ![DVSCameraWorkingPrinciple](img/sensor_dvs.gif)
-DVS is a camera and therefore has all the attributes available in the RGB camera. Nevertheless, there are few attributes
+DVS is a camera and therefore has all the attributes available in the RGB camera. Nevertheless, there are few attributes exclusive to the working principle of an Event camera.
 exclusive to the working principle of an Event camera.
 #### DVS camera attributes
@ -1497,12 +1477,12 @@ exclusive to the working principle of an Event camera.
 <td><code>sigma_positive_threshold</code></td>
 <td>float</td>
 <td>0</td>
-<td>White noise standard deviation for positive events (0-1)</td>
+<td>White noise standard deviation for positive events (0-1).</td>
 <tr>
 <td><code>sigma_negative_threshold</code></td>
 <td>float</td>
 <td>0</td>
-<td>White noise standard deviation for negative events (0-1)</td>
+<td>White noise standard deviation for negative events (0-1).</td>
 <tr>
 <td><code>refractory_period_ns</code></td>
 <td>int</td>
@ -1517,7 +1497,7 @@ exclusive to the working principle of an Event camera.
 <td><code>log_eps</code></td>
 <td>float</td>
 <td>0.001</td>
-<td>Epsilon value used to convert images to log: L = log(eps + I / 255.0). Where I is the grayscale value of the RGB image I = 0.2989*R +  0.5870*G + 0.1140*B.</td>
+<td>Epsilon value used to convert images to log: <code>L = log(eps + I / 255.0)</code>.<br> Where <code>I</code> is the grayscale value of the RGB image: <br><code>I = 0.2989*R +  0.5870*G + 0.1140*B</code>.</td>
 </tbody>
 </table>
 <br>
--- a/PythonAPI/docs/sensor_data.yml
+++ b/PythonAPI/docs/sensor_data.yml
@ -469,4 +469,101 @@
    - def_name: __str__
    # --------------------------------------
  - class_name: DVSEvent
    # - DESCRIPTION ------------------------
    doc: >
      Class that defines a DVS event. An event is a quadruple, so a tuple of 4 elements, with `x`, `y` pixel coordinate location, timestamp `t` and polarity `pol` of the event. Learn more about them [here](ref_sensors.md).  
    # - PROPERTIES -------------------------
    instance_variables:
    - var_name: x
      type: int
      doc: >
        X pixel coordinate.
    # --------------------------------------
    - var_name: 'y'
      type: int
      doc: >
        Y pixel coordinate.
    # --------------------------------------
    - var_name: t
      type: int
      doc: >
        Timestamp of the moment the event happened. 
    # --------------------------------------
    - var_name: pol
      type: bool
      doc: >
        Polarity of the event. __True__ for positive and __False__ for negative.
    # - METHODS ----------------------------
    methods:
    - def_name: __str__
    # --------------------------------------
  - class_name: DVSEventArray
    # - DESCRIPTION ------------------------
    doc: >
      Class that defines a stream of events in [carla.DVSEvent](#carla.DVSEvent). Such stream is an array of arbitrary size depending on the number of events. This class also stores the field of view, the height and width of the image and the timestamp from convenience. Learn more about them [here](ref_sensors.md).  
    # - PROPERTIES -------------------------
    instance_variables:
    - var_name: fov
      type: float
      doc: >
        Horizontal field of view of the image in degrees. 
    # --------------------------------------
    - var_name: height
      type: int
      doc: >
        Image height in pixels. 
    # --------------------------------------
    - var_name: width
      type: int
      doc: >
        Image width in pixels. 
    # --------------------------------------
    - var_name: raw_data
      type: bytes
    # - METHODS ----------------------------
    methods:
    - def_name: to_image
      doc: >
        Converts the image following this pattern: blue indicates positive events, red indicates negative events.
    # --------------------------------------
    - def_name: to_array
      doc: >
        Converts the stream of events to an array of int values in the following order <code>[x, y, t, pol]</code>.
    # --------------------------------------
    - def_name: to_array_x
      doc: >
        Returns an array with X pixel coordinate of all the events in the stream.
    # --------------------------------------
    - def_name: to_array_y
      doc: >
        Returns an array with Y pixel coordinate of all the events in the stream.
    # --------------------------------------
    - def_name: to_array_t
      doc: >
        Returns an array with the timestamp of all the events in the stream.
    # --------------------------------------
    - def_name: to_array_pol
      doc: >
        Returns an array with the polarity of all the events in the stream.
    # --------------------------------------
    - def_name: __getitem__
      params:
      - param_name: pos
        type: int
    # --------------------------------------
    - def_name: __iter__
    # --------------------------------------
    - def_name: __len__
    # --------------------------------------
    - def_name: __setitem__
      params:
      - param_name: pos
        type: int
      - param_name: color
        type: carla.Color
    # --------------------------------------
    - def_name: __str__
    # --------------------------------------
 ...