Go to file
Michal Sojka 2de8ad017d
PythonAPI: Fix segfaults due to incorrect GIL locking (#6718)
* PythonAPI: Fix segfault in GetAvailableMaps

When using CARLA with Python 3.10, I'm getting a segfault in
GetAvailableMaps. The problem disappears when PyList manipulation does
not happen with GIL unlocked, as done in this commit.

The initial part of crash backtrace (from GDB) is below:

    Program terminated with signal SIGSEGV, Segmentation fault.

    warning: Section `.reg-xstate/49253' in core file too small.
    #0  _PyInterpreterState_GET () at ./Include/internal/pycore_pystate.h:117
    117	   return tstate->interp;
    [Current thread is 1 (Thread 0x7fe6fe48f740 (LWP 49253))]
    (gdb) bt
    #0  _PyInterpreterState_GET () at ./Include/internal/pycore_pystate.h:117
    #1  get_list_state () at Objects/listobject.c:26
    #2  PyList_New (size=0) at Objects/listobject.c:159
    #3  0x00007fe6fdc0dab0 in boost::python::detail::list_base::list_base() () from /nix/store/c95f3nrkz3sflvycihyw1c8q4nk47p4m-boost-1.79.0/lib/libboost_python310.so.1.79.0
    #4  0x00007fe6ef9ecfc4 in boost::python::list::list (this=0x7ffd8a8aae28) at include/boost/python/list.hpp:61
    #5  GetAvailableMaps (self=...) at source/libcarla/Client.cpp:26
    #6  0x00007fe6efb6a8fe in boost::python::detail::invoke<boost::python::to_python_value<boost::python::list const&>, boost::python::list (*)(carla::client::Client const&), boost::python::arg_from_python<carla::client::Client const&> > (rc=..., f=<optimized out>, ac0=...)
        at include/boost/python/detail/invoke.hpp:73
    #7  boost::python::detail::caller_arity<1u>::impl<boost::python::list (*)(carla::client::Client const&), boost::python::default_call_policies, boost::mpl::vector2<boost::python::list, carla::client::Client const&> >::operator() (args_=<optimized out>, this=<optimized out>)
        at include/boost/python/detail/caller.hpp:233
    #8  boost::python::objects::caller_py_function_impl<boost::python::detail::caller<boost::python::list (*)(carla::client::Client const&), boost::python::default_call_policies, boost::mpl::vector2<boost::python::list, carla::client::Client const&> > >::operator() (
        this=<optimized out>, args=<optimized out>, kw=<optimized out>) at include/boost/python/object/py_function.hpp:38
    #9  0x00007fe6fdc1b4dd in boost::python::objects::function::call(_object*, _object*) const () from /nix/store/c95f3nrkz3sflvycihyw1c8q4nk47p4m-boost-1.79.0/lib/libboost_python310.so.1.79.0
    #10 0x00007fe6fdc1b6a8 in boost::detail::function::void_function_ref_invoker0<boost::python::objects::(anonymous namespace)::bind_return, void>::invoke(boost::detail::function::function_buffer&) ()
       from /nix/store/c95f3nrkz3sflvycihyw1c8q4nk47p4m-boost-1.79.0/lib/libboost_python310.so.1.79.0
    ...

* PythonAPI: Fix segfault in get_random_location_from_navigation()

When I run generate_traffic.py under Python 3.10, I get a segfault at
line:

    loc = world.get_random_location_from_navigation()

The backtrace from gdb looks like this:

    #0  0x00007f04552ad7e7 in new_threadstate () from /nix/store/zqj9irpw63pal9r4671p1gjd9jiw5sid-ros-env/lib/libpython3.10.so.1.0
    #1  0x00007f04552adaa1 in PyGILState_Ensure () from /nix/store/zqj9irpw63pal9r4671p1gjd9jiw5sid-ros-env/lib/libpython3.10.so.1.0
    #2  0x00007f040afd4f32 in std::_Function_handler<void (carla::client::WorldSnapshot), MakeCallback(boost::python::api::object)::{lambda(auto:1)#1}>::_M_invoke(std::_Any_data const&, carla::client::WorldSnapshot&&) ()
       from /nix/store/zqj9irpw63pal9r4671p1gjd9jiw5sid-ros-env/lib/python3.10/site-packages/carla/libcarla.cpython-310-x86_64-linux-gnu.so
    #3  0x00007f040b1d4ab1 in carla::client::detail::CallbackList<carla::client::WorldSnapshot>::Call(carla::client::WorldSnapshot) const () from /nix/store/zqj9irpw63pal9r4671p1gjd9jiw5sid-ros-env/lib/python3.10/site-packages/carla/libcarla.cpython-310-x86_64-linux-gnu.so
    #4  0x00007f040b1d424a in std::_Function_handler<void (carla::Buffer), carla::client::detail::Episode::Listen()::{lambda(auto:1)#1}>::_M_invoke(std::_Any_data const&, carla::Buffer&&) ()
       from /nix/store/zqj9irpw63pal9r4671p1gjd9jiw5sid-ros-env/lib/python3.10/site-packages/carla/libcarla.cpython-310-x86_64-linux-gnu.so
    #5  0x00007f040b23fc41 in boost::asio::detail::completion_handler<boost::asio::detail::binder0<carla::streaming::detail::tcp::Client::ReadData()::{lambda()#1}::operator()() const::{lambda(boost::system::error_code, unsigned long)#1}::operator()(boost::system::error_code, unsigned long) const::{lambda()#1}>, boost::asio::io_context::basic_executor_type<std::allocator<void>, 0ul> >::do_complete(void*, boost::asio::detail::scheduler_operation*, boost::system::error_code const&, unsigned long) ()
       from /nix/store/zqj9irpw63pal9r4671p1gjd9jiw5sid-ros-env/lib/python3.10/site-packages/carla/libcarla.cpython-310-x86_64-linux-gnu.so
    #6  0x00007f040b24ae85 in boost::asio::detail::strand_service::do_complete(void*, boost::asio::detail::scheduler_operation*, boost::system::error_code const&, unsigned long) ()
       from /nix/store/zqj9irpw63pal9r4671p1gjd9jiw5sid-ros-env/lib/python3.10/site-packages/carla/libcarla.cpython-310-x86_64-linux-gnu.so
    #7  0x00007f040b1a94f5 in boost::asio::detail::scheduler::do_run_one(boost::asio::detail::conditionally_enabled_mutex::scoped_lock&, boost::asio::detail::scheduler_thread_info&, boost::system::error_code const&) ()
       from /nix/store/zqj9irpw63pal9r4671p1gjd9jiw5sid-ros-env/lib/python3.10/site-packages/carla/libcarla.cpython-310-x86_64-linux-gnu.so
    #8  0x00007f040b199351 in boost::asio::detail::scheduler::run(boost::system::error_code&) [clone .isra.0] () from /nix/store/zqj9irpw63pal9r4671p1gjd9jiw5sid-ros-env/lib/python3.10/site-packages/carla/libcarla.cpython-310-x86_64-linux-gnu.so
    #9  0x00007f040b1ac1cb in std:🧵:_State_impl<std:🧵:_Invoker<std::tuple<carla::ThreadPool::AsyncRun(unsigned long)::{lambda()#1}> > >::_M_run() ()
       from /nix/store/zqj9irpw63pal9r4671p1gjd9jiw5sid-ros-env/lib/python3.10/site-packages/carla/libcarla.cpython-310-x86_64-linux-gnu.so
    #10 0x00007f040bce05c3 in execute_native_thread_routine () from /nix/store/2fpmbk0g0ggm9zq89af7phvvvv8dnm7n-gcc-12.3.0-lib/lib/libstdc++.so.6
    #11 0x00007f045509fdd4 in start_thread () from /nix/store/1x4ijm9r1a88qk7zcmbbfza324gx1aac-glibc-2.37-8/lib/libc.so.6
    #12 0x00007f04551219b0 in clone3 () from /nix/store/1x4ijm9r1a88qk7zcmbbfza324gx1aac-glibc-2.37-8/lib/libc.so.6

It turns out that its caused by releasing GIL for too long. We fix it
by releasing the GIL only for the actual libcarla call and
constructing Python objects with GIL locked.

---------

Co-authored-by: bernat <bernatx@gmail.com>
2023-10-27 14:37:46 +02:00
.github Merge branch 'dev' 2021-08-02 18:06:44 +02:00
Co-Simulation fixed bug checking safe blueprints 2021-08-02 16:36:28 +02:00
Docs Remove duplicated instance variable `is_junction` 2023-09-27 13:56:45 +02:00
Examples/CppClient change tool chain for examples 2023-02-23 18:32:29 +01:00
Import Fixed bugs of importing assets (#2068) 2019-09-17 14:57:22 +02:00
LibCarla Adding a new function that loads a map only if it is different (#6651) 2023-09-06 12:34:59 +02:00
PythonAPI PythonAPI: Fix segfaults due to incorrect GIL locking (#6718) 2023-10-27 14:37:46 +02:00
Unreal/CarlaUE4 CarlaExporter tool now exports the box collider and convex collider of the object if it has one, otherwise the mesh (fixed problem with big meshes) 2023-10-19 10:37:06 +02:00
Util Remove Recast hash version 2023-10-23 16:32:29 +02:00
osm-world-renderer Digital Twins Update 26//05 (#6502) 2023-06-06 11:19:19 +02:00
.gitattributes Remove git lfs 2017-07-25 17:07:35 +02:00
.gitignore CarlaExporter tool now exports the box collider and convex collider of the object if it has one, otherwise the mesh (fixed problem with big meshes) 2023-10-19 10:37:06 +02:00
.pep8 Add PythonAPI module 2018-07-04 11:13:43 +02:00
.readthedocs.yml Fix readthedocs navigation and page order 2019-05-08 16:14:25 +02:00
.travis.yml Revert "Removed more compiler/linker version requirements." 2022-04-26 16:17:00 +02:00
CHANGELOG.md PythonAPI: Fix segfaults due to incorrect GIL locking (#6718) 2023-10-27 14:37:46 +02:00
CMakeLists.txt Change cmake version from 3.9 to 3.5 2018-11-20 14:26:48 +01:00
Doxyfile Fdomf/jenkins-wa (#1963) 2019-08-02 14:36:06 +02:00
Jenkinsfile Adding Town13 as AdditionalMap 2023-10-24 20:44:26 +02:00
LICENSE Update license 2018-01-22 16:42:10 +01:00
Makefile Replace Linux build system 2018-07-04 19:08:42 +02:00
README.md Saving AdditionalMaps in nightly build 2023-04-14 08:41:52 +02:00
Update.bat Fix retrieve of content 2020-03-19 17:34:07 +01:00
Update.sh Rain particles and Update.sh fixes (#2814) 2020-05-05 16:46:41 +02:00
mkdocs.yml changed left menu 2023-03-01 11:20:02 +01:00
wheel_config.ini added steering wheel control example 2019-02-22 11:10:27 +01:00

README.md

CARLA Simulator

Build Status Documentation

carla.org download documentation forum discord

CARLA is an open-source simulator for autonomous driving research. CARLA has been developed from the ground up to support development, training, and validation of autonomous driving systems. In addition to open-source code and protocols, CARLA provides open digital assets (urban layouts, buildings, vehicles) that were created for this purpose and can be used freely. The simulation platform supports flexible specification of sensor suites and environmental conditions.

CARLA Video

If you want to benchmark your model in the same conditions as in our CoRL17 paper, check out Benchmarking.

Linux:

Windows:

  • Intel i7 gen 9th - 11th / Intel i9 gen 9th - 11th / AMD ryzen 7 / AMD ryzen 9
  • +16 GB RAM memory
  • NVIDIA RTX 2070 / NVIDIA RTX 2080 / NVIDIA RTX 3070, NVIDIA RTX 3080
  • Ubuntu 18.04

CARLA Ecosystem

Repositories associated to the CARLA simulation platform:

Like what you see? Star us on GitHub to support the project!

Paper

If you use CARLA, please cite our CoRL17 paper.

CARLA: An Open Urban Driving Simulator
Alexey Dosovitskiy, German Ros, Felipe Codevilla, Antonio Lopez, Vladlen Koltun; PMLR 78:1-16 [PDF] [talk]

@inproceedings{Dosovitskiy17,
  title = {{CARLA}: {An} Open Urban Driving Simulator},
  author = {Alexey Dosovitskiy and German Ros and Felipe Codevilla and Antonio Lopez and Vladlen Koltun},
  booktitle = {Proceedings of the 1st Annual Conference on Robot Learning},
  pages = {1--16},
  year = {2017}
}

Building CARLA

Use git clone or download the project from this page. Note that the master branch contains the most recent release of CARLA with the latest fixes and features.

Then follow the instruction at How to build on Linux or How to build on Windows.
The Linux build needs for an UE patch to solve some visualization issues regarding Vulkan. Those already working with a Linux build should install the patch and make the UE build again using the following commands.

# Download and install the UE patch  
cd ~/UnrealEngine_4.24
wget https://carla-releases.s3.eu-west-3.amazonaws.com/Linux/UE_Patch/430667-13636743-patch.txt ~/430667-13636743-patch.txt
patch --strip=4 < ~/430667-13636743-patch.txt
# Build UE
./Setup.sh && ./GenerateProjectFiles.sh && make

Unfortunately we don't have official instructions to build on Mac yet, please check the progress at issue #150.

Contributing

Please take a look at our Contribution guidelines.

F.A.Q.

If you run into problems, check our FAQ.

CARLA Talks

The team creates some additional content for users, besides the docs. This is a great way to cover different subjects such as detailed explanations for a specific module, latest improvements in a feature, future work and much more.

CARLA Talks 2020 (May):

  • General
  • Modules
  • Features
    • Co-Simulations with SUMO and PTV-Vissim — video | slides
    • Integration of RSS-lib — slides
    • The External Sensor Interface (ESI) — video | slides
    • The OpenDRIVE Standalone Mode — video | slides

Licenses

CARLA licenses

CARLA specific code is distributed under MIT License.

CARLA specific assets are distributed under CC-BY License.

CARLA Dependency and Integration licenses

The ad-rss-lib library compiled and linked by the RSS Integration build variant introduces LGPL-2.1-only License.

Unreal Engine 4 follows its own license terms.

CARLA uses three dependencies as part of the SUMO integration:

CARLA uses one dependency as part of the Chrono integration:

CARLA uses the Autodesk FBX SDK for converting FBX to OBJ in the import process of maps. This step is optional, and the SDK is located here

This software contains Autodesk® FBX® code developed by Autodesk, Inc. Copyright 2020 Autodesk, Inc. All rights, reserved. Such code is provided "as is" and Autodesk, Inc. disclaims any and all warranties, whether express or implied, including without limitation the implied warranties of merchantability, fitness for a particular purpose or non-infringement of third party rights. In no event shall Autodesk, Inc. be liable for any direct, indirect, incidental, special, exemplary, or consequential damages (including, but not limited to, procurement of substitute goods or services; loss of use, data, or profits; or business interruption) however caused and on any theory of liability, whether in contract, strict liability, or tort (including negligence or otherwise) arising in any way out of such code."