p71924506
/
carla
mirror of https://github.com/carla-simulator/carla.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Fixing details on the documentation

This commit is contained in:
felipecode 2018-04-17 17:43:49 +02:00
parent a74c8c2727
commit ac35e26d88
13 changed files with 276 additions and 80 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

195
Docs/benchmark_basic_results_town01.md
View File

@ -3,6 +3,9 @@ We show the results for test and train weathers when
The following result should print on the screen after running the
example.
----- Printing results for training weathers (Seen in Training) -----
Percentage of Successful Episodes
Weather: Clear Noon
@ -19,72 +22,168 @@ example.
Average Percentage of Distance to Goal Travelled
Weather: Clear Noon
Task: 0 -> 0.9642745353103807
Task: 1 -> 0.6793899335407552
Task: 2 -> 0.6593335145871936
Task: 3 -> 0.6696988482193544
Task: 0 -> 0.8118712628185436
Task: 1 -> 0.10663609976621982
Task: 2 -> -0.20449201041073187
Task: 3 -> -0.20447665032772155
Average Between Weathers
Task 0 -> 0.9642745353103807
Task 1 -> 0.6793899335407552
Task 2 -> 0.6593335145871936
Task 3 -> 0.6696988482193544
Task 0 -> 0.8118712628185436
Task 1 -> 0.10663609976621982
Task 2 -> -0.20449201041073187
Task 3 -> -0.20447665032772155
Avg. Kilometers driven before a collision to a PEDESTRIAN
Weather: Clear Noon
Task 0 -> more than 0.043162963265066225
Task 1 -> more than 0.1235917529074286
Task 2 -> more than 0.24004519587214496
Task 3 -> more than 0.19999741925565595
Task 0 -> more than 0.007091656691537298
Task 1 -> more than 0.038524663402911156
Task 2 -> more than 0.039421759075344356
Task 3 -> more than 0.03941748888225085
Average Between Weathers
More than 0 -> 0.043162963265066225
More than 1 -> 0.1235917529074286
More than 2 -> 0.24004519587214496
More than 3 -> 0.19999741925565595
Task 0 -> more than 0.007091656691537298
Task 1 -> more than 0.038524663402911156
Task 2 -> more than 0.039421759075344356
Task 3 -> more than 0.03941748888225085
Avg. Kilometers driven before a collision to a VEHICLE
Weather: Clear Noon
Task 0 -> more than 0.043162963265066225
Task 1 -> more than 0.1235917529074286
Task 2 -> more than 0.24004519587214496
Task 3 -> 0.03999948385113119
Task 0 -> more than 0.007091656691537298
Task 1 -> more than 0.038524663402911156
Task 2 -> more than 0.039421759075344356
Task 3 -> more than 0.03941748888225085
Average Between Weathers
More than 0 -> 0.043162963265066225
More than 1 -> 0.1235917529074286
More than 2 -> 0.24004519587214496
Task 3 -> 0.03999948385113119
Task 0 -> more than 0.007091656691537298
Task 1 -> more than 0.038524663402911156
Task 2 -> more than 0.039421759075344356
Task 3 -> more than 0.03941748888225085
Avg. Kilometers driven before a collision to a STATIC OBSTACLE
Weather: Clear Noon
Task 0 -> more than 0.043162963265066225
Task 1 -> more than 0.1235917529074286
Task 2 -> more than 0.24004519587214496
Task 3 -> 0.19999741925565595
Task 0 -> more than 0.007091656691537298
Task 1 -> more than 0.038524663402911156
Task 2 -> more than 0.039421759075344356
Task 3 -> 0.019708744441125425
Average Between Weathers
More than 0 -> 0.043162963265066225
More than 1 -> 0.1235917529074286
More than 2 -> 0.24004519587214496
Task 3 -> 0.19999741925565595
Task 0 -> more than 0.007091656691537298
Task 1 -> more than 0.038524663402911156
Task 2 -> more than 0.039421759075344356
Task 3 -> 0.019708744441125425
Avg. Kilometers driven before going OUTSIDE OF THE ROAD
Weather: Clear Noon
Task 0 -> more than 0.043162963265066225
Task 1 -> more than 0.1235917529074286
Task 2 -> more than 0.24004519587214496
Task 3 -> more than 0.19999741925565595
Task 0 -> more than 0.007091656691537298
Task 1 -> 0.038524663402911156
Task 2 -> 0.039421759075344356
Task 3 -> 0.03941748888225085
Average Between Weathers
More than 0 -> 0.043162963265066225
More than 1 -> 0.1235917529074286
More than 2 -> 0.24004519587214496
More than 3 -> 0.19999741925565595
Task 0 -> more than 0.007091656691537298
Task 1 -> 0.038524663402911156
Task 2 -> 0.039421759075344356
Task 3 -> 0.03941748888225085
Avg. Kilometers driven before invading the OPPOSITE LANE
Weather: Clear Noon
Task 0 -> more than 0.043162963265066225
Task 1 -> 0.1235917529074286
Task 2 -> 0.008890562810079442
Task 3 -> 0.19999741925565595
Task 0 -> more than 0.007091656691537298
Task 1 -> 0.038524663402911156
Task 2 -> 0.039421759075344356
Task 3 -> 0.03941748888225085
Average Between Weathers
More than 0 -> 0.043162963265066225
Task 1 -> 0.1235917529074286
Task 2 -> 0.008890562810079442
Task 3 -> 0.19999741925565595
Task 0 -> more than 0.007091656691537298
Task 1 -> 0.038524663402911156
Task 2 -> 0.039421759075344356
Task 3 -> 0.03941748888225085
----- Printing results for test weathers (Unseen in Training) -----
Percentage of Successful Episodes
Weather: Clear Noon
Task: 0 -> 1.0
Task: 1 -> 0.0
Task: 2 -> 0.0
Task: 3 -> 0.0
Average Between Weathers
Task 0 -> 1.0
Task 1 -> 0.0
Task 2 -> 0.0
Task 3 -> 0.0
Average Percentage of Distance to Goal Travelled
Weather: Clear Noon
Task: 0 -> 0.8118712628185436
Task: 1 -> 0.10663609976621982
Task: 2 -> -0.20449201041073187
Task: 3 -> -0.20447665032772155
Average Between Weathers
Task 0 -> 0.8118712628185436
Task 1 -> 0.10663609976621982
Task 2 -> -0.20449201041073187
Task 3 -> -0.20447665032772155
Avg. Kilometers driven before a collision to a PEDESTRIAN
Weather: Clear Noon
Task 0 -> more than 0.007091656691537298
Task 1 -> more than 0.038524663402911156
Task 2 -> more than 0.039421759075344356
Task 3 -> more than 0.03941748888225085
Average Between Weathers
Task 0 -> more than 0.007091656691537298
Task 1 -> more than 0.038524663402911156
Task 2 -> more than 0.039421759075344356
Task 3 -> more than 0.03941748888225085
Avg. Kilometers driven before a collision to a VEHICLE
Weather: Clear Noon
Task 0 -> more than 0.007091656691537298
Task 1 -> more than 0.038524663402911156
Task 2 -> more than 0.039421759075344356
Task 3 -> more than 0.03941748888225085
Average Between Weathers
Task 0 -> more than 0.007091656691537298
Task 1 -> more than 0.038524663402911156
Task 2 -> more than 0.039421759075344356
Task 3 -> more than 0.03941748888225085
Avg. Kilometers driven before a collision to a STATIC OBSTACLE
Weather: Clear Noon
Task 0 -> more than 0.007091656691537298
Task 1 -> more than 0.038524663402911156
Task 2 -> more than 0.039421759075344356
Task 3 -> 0.019708744441125425
Average Between Weathers
Task 0 -> more than 0.007091656691537298
Task 1 -> more than 0.038524663402911156
Task 2 -> more than 0.039421759075344356
Task 3 -> 0.019708744441125425
Avg. Kilometers driven before going OUTSIDE OF THE ROAD
Weather: Clear Noon
Task 0 -> more than 0.007091656691537298
Task 1 -> 0.038524663402911156
Task 2 -> 0.039421759075344356
Task 3 -> 0.03941748888225085
Average Between Weathers
Task 0 -> more than 0.007091656691537298
Task 1 -> 0.038524663402911156
Task 2 -> 0.039421759075344356
Task 3 -> 0.03941748888225085
Avg. Kilometers driven before invading the OPPOSITE LANE
Weather: Clear Noon
Task 0 -> more than 0.007091656691537298
Task 1 -> 0.038524663402911156
Task 2 -> 0.039421759075344356
Task 3 -> 0.03941748888225085
Average Between Weathers
Task 0 -> more than 0.007091656691537298
Task 1 -> 0.038524663402911156
Task 2 -> 0.039421759075344356
Task 3 -> 0.03941748888225085

120
Docs/benchmark_basic_results_town02.md
View File

@ -4,6 +4,9 @@ The following result should print on the screen after running the
example.
----- Printing results for training weathers (Seen in Training) -----
Percentage of Successful Episodes
Weather: Clear Noon
@ -23,70 +26,163 @@ example.
Task: 0 -> 0.8118712628185436
Task: 1 -> 0.10663609976621982
Task: 2 -> -0.20449201041073187
Task: 3 -> -0.20447574648836767
Task: 3 -> -0.20447646890991833
Average Between Weathers
Task 0 -> 0.8118712628185436
Task 1 -> 0.10663609976621982
Task 2 -> -0.20449201041073187
Task 3 -> -0.20447574648836767
Task 3 -> -0.20447646890991833
Avg. Kilometers driven before a collision to a PEDESTRIAN
Weather: Clear Noon
Task 0 -> more than 0.007091656691537298
Task 1 -> more than 0.038524663402911156
Task 2 -> more than 0.039421759075344356
Task 3 -> more than 0.03941413920457633
Task 3 -> more than 0.03941736123307525
Average Between Weathers
More than 0 -> 0.007091656691537298
More than 1 -> 0.038524663402911156
More than 2 -> 0.039421759075344356
More than 3 -> 0.03941413920457633
More than 3 -> 0.03941736123307525
Avg. Kilometers driven before a collision to a VEHICLE
Weather: Clear Noon
Task 0 -> more than 0.007091656691537298
Task 1 -> more than 0.038524663402911156
Task 2 -> more than 0.039421759075344356
Task 3 -> more than 0.03941413920457633
Task 3 -> more than 0.03941736123307525
Average Between Weathers
More than 0 -> 0.007091656691537298
More than 1 -> 0.038524663402911156
More than 2 -> 0.039421759075344356
More than 3 -> 0.03941413920457633
More than 3 -> 0.03941736123307525
Avg. Kilometers driven before a collision to a STATIC OBSTACLE
Weather: Clear Noon
Task 0 -> more than 0.007091656691537298
Task 1 -> more than 0.038524663402911156
Task 2 -> more than 0.039421759075344356
Task 3 -> 0.019707069602288164
Task 3 -> 0.019708680616537626
Average Between Weathers
More than 0 -> 0.007091656691537298
More than 1 -> 0.038524663402911156
More than 2 -> 0.039421759075344356
Task 3 -> 0.019707069602288164
Task 3 -> 0.019708680616537626
Avg. Kilometers driven before going OUTSIDE OF THE ROAD
Weather: Clear Noon
Task 0 -> more than 0.007091656691537298
Task 1 -> 0.038524663402911156
Task 2 -> 0.039421759075344356
Task 3 -> 0.03941413920457633
Task 3 -> 0.03941736123307525
Average Between Weathers
More than 0 -> 0.007091656691537298
Task 1 -> 0.038524663402911156
Task 2 -> 0.039421759075344356
Task 3 -> 0.03941413920457633
Task 3 -> 0.03941736123307525
Avg. Kilometers driven before invading the OPPOSITE LANE
Weather: Clear Noon
Task 0 -> more than 0.007091656691537298
Task 1 -> 0.038524663402911156
Task 2 -> 0.039421759075344356
Task 3 -> 0.03941413920457633
Task 3 -> 0.03941736123307525
Average Between Weathers
More than 0 -> 0.007091656691537298
Task 1 -> 0.038524663402911156
Task 2 -> 0.039421759075344356
Task 3 -> 0.03941413920457633
Task 3 -> 0.03941736123307525
----- Printing results for test weathers (Unseen in Training) -----
Percentage of Successful Episodes
Weather: Clear Noon
Task: 0 -> 1.0
Task: 1 -> 0.0
Task: 2 -> 0.0
Task: 3 -> 0.0
Average Between Weathers
Task 0 -> 1.0
Task 1 -> 0.0
Task 2 -> 0.0
Task 3 -> 0.0
Average Percentage of Distance to Goal Travelled
Weather: Clear Noon
Task: 0 -> 0.8118712628185436
Task: 1 -> 0.10663609976621982
Task: 2 -> -0.20449201041073187
Task: 3 -> -0.20447646890991833
Average Between Weathers
Task 0 -> 0.8118712628185436
Task 1 -> 0.10663609976621982
Task 2 -> -0.20449201041073187
Task 3 -> -0.20447646890991833
Avg. Kilometers driven before a collision to a PEDESTRIAN
Weather: Clear Noon
Task 0 -> more than 0.007091656691537298
Task 1 -> more than 0.038524663402911156
Task 2 -> more than 0.039421759075344356
Task 3 -> more than 0.03941736123307525
Average Between Weathers
More than 0 -> 0.007091656691537298
More than 1 -> 0.038524663402911156
More than 2 -> 0.039421759075344356
More than 3 -> 0.03941736123307525
Avg. Kilometers driven before a collision to a VEHICLE
Weather: Clear Noon
Task 0 -> more than 0.007091656691537298
Task 1 -> more than 0.038524663402911156
Task 2 -> more than 0.039421759075344356
Task 3 -> more than 0.03941736123307525
Average Between Weathers
More than 0 -> 0.007091656691537298
More than 1 -> 0.038524663402911156
More than 2 -> 0.039421759075344356
More than 3 -> 0.03941736123307525
Avg. Kilometers driven before a collision to a STATIC OBSTACLE
Weather: Clear Noon
Task 0 -> more than 0.007091656691537298
Task 1 -> more than 0.038524663402911156
Task 2 -> more than 0.039421759075344356
Task 3 -> 0.019708680616537626
Average Between Weathers
More than 0 -> 0.007091656691537298
More than 1 -> 0.038524663402911156
More than 2 -> 0.039421759075344356
Task 3 -> 0.019708680616537626
Avg. Kilometers driven before going OUTSIDE OF THE ROAD
Weather: Clear Noon
Task 0 -> more than 0.007091656691537298
Task 1 -> 0.038524663402911156
Task 2 -> 0.039421759075344356
Task 3 -> 0.03941736123307525
Average Between Weathers
More than 0 -> 0.007091656691537298
Task 1 -> 0.038524663402911156
Task 2 -> 0.039421759075344356
Task 3 -> 0.03941736123307525
Avg. Kilometers driven before invading the OPPOSITE LANE
Weather: Clear Noon
Task 0 -> more than 0.007091656691537298
Task 1 -> 0.038524663402911156
Task 2 -> 0.039421759075344356
Task 3 -> 0.03941736123307525
Average Between Weathers
More than 0 -> 0.007091656691537298
Task 1 -> 0.038524663402911156
Task 2 -> 0.039421759075344356
Task 3 -> 0.03941736123307525

BIN
Docs/img/town01_109_29.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 92 KiB

BIN
Docs/img/town01_140_134.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 93 KiB

0
PythonClient/carla/driving_benchmark/experiment_suite/__init__.py → PythonClient/carla/driving_benchmark/experiment_suites/__init__.py
View File

0
PythonClient/carla/driving_benchmark/experiment_suite/basic_experiment_suite.py → PythonClient/carla/driving_benchmark/experiment_suites/basic_experiment_suite.py
View File

2
PythonClient/carla/driving_benchmark/experiment_suite/corl_2017.py → PythonClient/carla/driving_benchmark/experiment_suites/corl_2017.py
View File

@ -12,7 +12,7 @@ from __future__ import print_function
from carla.driving_benchmark.experiment import Experiment
from carla.sensor import Camera
from carla.settings import CarlaSettings
from carla.driving_benchmark.experiment_suite.experiment_suite import ExperimentSuite
from carla.driving_benchmark.experiment_suites.experiment_suite import ExperimentSuite

0
PythonClient/carla/driving_benchmark/experiment_suite/experiment_suite.py → PythonClient/carla/driving_benchmark/experiment_suites/experiment_suite.py
View File

2
PythonClient/carla/driving_benchmark/results_printer.py
View File

@ -114,7 +114,7 @@ def print_summary(metrics_summary, weathers, path):
print (' Average Between Weathers')
for i in range(len(metric_sum_values)):
if metric_sum_values[i] == 0:
print(' More than ', i, ' -> ', summed_driven_kilometers[i])
print(' Task ', i, ' -> more than ', summed_driven_kilometers[i])
else:
print(' Task ', i, ' -> ', summed_driven_kilometers[i] / metric_sum_values[i])
print ("")

9
PythonClient/driving_benchmark_example.py
View File

@ -10,8 +10,8 @@ import argparse
import logging
from carla.driving_benchmark import run_driving_benchmark
from carla.driving_benchmark.experiment_suite import CoRL2017
from carla.driving_benchmark.experiment_suite import BasicExperimentSuite
from carla.driving_benchmark.experiment_suites import CoRL2017
from carla.driving_benchmark.experiment_suites import BasicExperimentSuite
from carla.agent import ForwardAgent
@ -83,8 +83,9 @@ if __name__ == '__main__':
if args.corl_2017:
experiment_suite = CoRL2017(args.city_name)
else:
print (' WARNING: running the basic driving benchmark, to run the CORL 2017, you should run' \
' python driving_benchmark_example.py --corld-2017')
print (' WARNING: running the basic driving benchmark, to run for CoRL 2017'
' experiment suites, you should run'
' python driving_benchmark_example.py --corl-2017')
experiment_suite = BasicExperimentSuite(args.city_name)
# Now actually run the driving_benchmark

2
PythonClient/setup.py
View File

@ -6,7 +6,7 @@ setup(
name='carla_client',
version='0.8.1',
packages=['carla', 'carla.driving_benchmark', 'carla.agent',
'carla.driving_benchmark.experiment_suite', 'carla.planner'],
'carla.driving_benchmark.experiment_suites', 'carla.planner'],
license='MIT License',
description='Python API for communicating with the CARLA server.',
url='https://github.com/carla-simulator/carla',

2
PythonClient/test/acceptance_tests/test_benchmark_module.py
View File

@ -18,7 +18,7 @@ import unittest
from carla.agent.agent import Agent
from carla.driving_benchmark.experiment_suite.basic_experiment_suite import BasicExperimentSuite
from carla.driving_benchmark.experiment_suites.basic_experiment_suite import BasicExperimentSuite
from carla.client import make_carla_client, VehicleControl
from carla.tcp import TCPConnectionError

6
PythonClient/test/unit_tests/test_experiment_suite.py
View File

@ -1,9 +1,9 @@
import unittest
from carla.driving_benchmark.experiment_suite.experiment_suite import ExperimentSuite
from carla.driving_benchmark.experiment_suites.experiment_suite import ExperimentSuite
from carla.driving_benchmark.experiment_suite.basic_experiment_suite import BasicExperimentSuite
from carla.driving_benchmark.experiment_suites.basic_experiment_suite import BasicExperimentSuite
from carla.driving_benchmark.experiment_suite.corl_2017 import CoRL2017
from carla.driving_benchmark.experiment_suites.corl_2017 import CoRL2017
class testExperimentSuite(unittest.TestCase):