* PythonAPI: Fix segfault in GetAvailableMaps
When using CARLA with Python 3.10, I'm getting a segfault in
GetAvailableMaps. The problem disappears when PyList manipulation does
not happen with GIL unlocked, as done in this commit.
The initial part of crash backtrace (from GDB) is below:
Program terminated with signal SIGSEGV, Segmentation fault.
warning: Section `.reg-xstate/49253' in core file too small.
#0 _PyInterpreterState_GET () at ./Include/internal/pycore_pystate.h:117
117 return tstate->interp;
[Current thread is 1 (Thread 0x7fe6fe48f740 (LWP 49253))]
(gdb) bt
#0 _PyInterpreterState_GET () at ./Include/internal/pycore_pystate.h:117
#1 get_list_state () at Objects/listobject.c:26
#2 PyList_New (size=0) at Objects/listobject.c:159
#3 0x00007fe6fdc0dab0 in boost::python::detail::list_base::list_base() () from /nix/store/c95f3nrkz3sflvycihyw1c8q4nk47p4m-boost-1.79.0/lib/libboost_python310.so.1.79.0
#4 0x00007fe6ef9ecfc4 in boost::python::list::list (this=0x7ffd8a8aae28) at include/boost/python/list.hpp:61
#5 GetAvailableMaps (self=...) at source/libcarla/Client.cpp:26
#6 0x00007fe6efb6a8fe in boost::python::detail::invoke<boost::python::to_python_value<boost::python::list const&>, boost::python::list (*)(carla::client::Client const&), boost::python::arg_from_python<carla::client::Client const&> > (rc=..., f=<optimized out>, ac0=...)
at include/boost/python/detail/invoke.hpp:73
#7 boost::python::detail::caller_arity<1u>::impl<boost::python::list (*)(carla::client::Client const&), boost::python::default_call_policies, boost::mpl::vector2<boost::python::list, carla::client::Client const&> >::operator() (args_=<optimized out>, this=<optimized out>)
at include/boost/python/detail/caller.hpp:233
#8 boost::python::objects::caller_py_function_impl<boost::python::detail::caller<boost::python::list (*)(carla::client::Client const&), boost::python::default_call_policies, boost::mpl::vector2<boost::python::list, carla::client::Client const&> > >::operator() (
this=<optimized out>, args=<optimized out>, kw=<optimized out>) at include/boost/python/object/py_function.hpp:38
#9 0x00007fe6fdc1b4dd in boost::python::objects::function::call(_object*, _object*) const () from /nix/store/c95f3nrkz3sflvycihyw1c8q4nk47p4m-boost-1.79.0/lib/libboost_python310.so.1.79.0
#10 0x00007fe6fdc1b6a8 in boost::detail::function::void_function_ref_invoker0<boost::python::objects::(anonymous namespace)::bind_return, void>::invoke(boost::detail::function::function_buffer&) ()
from /nix/store/c95f3nrkz3sflvycihyw1c8q4nk47p4m-boost-1.79.0/lib/libboost_python310.so.1.79.0
...
* PythonAPI: Fix segfault in get_random_location_from_navigation()
When I run generate_traffic.py under Python 3.10, I get a segfault at
line:
loc = world.get_random_location_from_navigation()
The backtrace from gdb looks like this:
#0 0x00007f04552ad7e7 in new_threadstate () from /nix/store/zqj9irpw63pal9r4671p1gjd9jiw5sid-ros-env/lib/libpython3.10.so.1.0
#1 0x00007f04552adaa1 in PyGILState_Ensure () from /nix/store/zqj9irpw63pal9r4671p1gjd9jiw5sid-ros-env/lib/libpython3.10.so.1.0
#2 0x00007f040afd4f32 in std::_Function_handler<void (carla::client::WorldSnapshot), MakeCallback(boost::python::api::object)::{lambda(auto:1)#1}>::_M_invoke(std::_Any_data const&, carla::client::WorldSnapshot&&) ()
from /nix/store/zqj9irpw63pal9r4671p1gjd9jiw5sid-ros-env/lib/python3.10/site-packages/carla/libcarla.cpython-310-x86_64-linux-gnu.so
#3 0x00007f040b1d4ab1 in carla::client::detail::CallbackList<carla::client::WorldSnapshot>::Call(carla::client::WorldSnapshot) const () from /nix/store/zqj9irpw63pal9r4671p1gjd9jiw5sid-ros-env/lib/python3.10/site-packages/carla/libcarla.cpython-310-x86_64-linux-gnu.so
#4 0x00007f040b1d424a in std::_Function_handler<void (carla::Buffer), carla::client::detail::Episode::Listen()::{lambda(auto:1)#1}>::_M_invoke(std::_Any_data const&, carla::Buffer&&) ()
from /nix/store/zqj9irpw63pal9r4671p1gjd9jiw5sid-ros-env/lib/python3.10/site-packages/carla/libcarla.cpython-310-x86_64-linux-gnu.so
#5 0x00007f040b23fc41 in boost::asio::detail::completion_handler<boost::asio::detail::binder0<carla::streaming::detail::tcp::Client::ReadData()::{lambda()#1}::operator()() const::{lambda(boost::system::error_code, unsigned long)#1}::operator()(boost::system::error_code, unsigned long) const::{lambda()#1}>, boost::asio::io_context::basic_executor_type<std::allocator<void>, 0ul> >::do_complete(void*, boost::asio::detail::scheduler_operation*, boost::system::error_code const&, unsigned long) ()
from /nix/store/zqj9irpw63pal9r4671p1gjd9jiw5sid-ros-env/lib/python3.10/site-packages/carla/libcarla.cpython-310-x86_64-linux-gnu.so
#6 0x00007f040b24ae85 in boost::asio::detail::strand_service::do_complete(void*, boost::asio::detail::scheduler_operation*, boost::system::error_code const&, unsigned long) ()
from /nix/store/zqj9irpw63pal9r4671p1gjd9jiw5sid-ros-env/lib/python3.10/site-packages/carla/libcarla.cpython-310-x86_64-linux-gnu.so
#7 0x00007f040b1a94f5 in boost::asio::detail::scheduler::do_run_one(boost::asio::detail::conditionally_enabled_mutex::scoped_lock&, boost::asio::detail::scheduler_thread_info&, boost::system::error_code const&) ()
from /nix/store/zqj9irpw63pal9r4671p1gjd9jiw5sid-ros-env/lib/python3.10/site-packages/carla/libcarla.cpython-310-x86_64-linux-gnu.so
#8 0x00007f040b199351 in boost::asio::detail::scheduler::run(boost::system::error_code&) [clone .isra.0] () from /nix/store/zqj9irpw63pal9r4671p1gjd9jiw5sid-ros-env/lib/python3.10/site-packages/carla/libcarla.cpython-310-x86_64-linux-gnu.so
#9 0x00007f040b1ac1cb in std:🧵:_State_impl<std:🧵:_Invoker<std::tuple<carla::ThreadPool::AsyncRun(unsigned long)::{lambda()#1}> > >::_M_run() ()
from /nix/store/zqj9irpw63pal9r4671p1gjd9jiw5sid-ros-env/lib/python3.10/site-packages/carla/libcarla.cpython-310-x86_64-linux-gnu.so
#10 0x00007f040bce05c3 in execute_native_thread_routine () from /nix/store/2fpmbk0g0ggm9zq89af7phvvvv8dnm7n-gcc-12.3.0-lib/lib/libstdc++.so.6
#11 0x00007f045509fdd4 in start_thread () from /nix/store/1x4ijm9r1a88qk7zcmbbfza324gx1aac-glibc-2.37-8/lib/libc.so.6
#12 0x00007f04551219b0 in clone3 () from /nix/store/1x4ijm9r1a88qk7zcmbbfza324gx1aac-glibc-2.37-8/lib/libc.so.6
It turns out that its caused by releasing GIL for too long. We fix it
by releasing the GIL only for the actual libcarla call and
constructing Python objects with GIL locked.
---------
Co-authored-by: bernat <bernatx@gmail.com>
* Adding a new function to load a map only if it is different
* Removing commented code
* Updated AddVehicleWheelsAnimation function
* Revert "Updated AddVehicleWheelsAnimation function"
This reverts commit 55405a3de7.
* Small correction: removing blank space
* Changelog.md update
---------
Co-authored-by: Adri Ollero <adriollero@gmail.com>
Co-authored-by: Axel1092 <35765780+Axel1092@users.noreply.github.com>
* Updated changelog and content versions
* Added vehicle.sixwheeltruck.sixwheeltruck to smoke test exclusion list
* Fix old town tests
* fixed vehicle id in smoke test
* Update ContentVersions.txt
---------
Co-authored-by: bernat <bernatx@gmail.com>
The supported compiler is clang, but that is not the default on Ubuntu.
Unreal Editor itself ships clang-10 for use on linux (along with the
libc++ that Unreal itself uses) so just use those in CARLA.
* Change handling of empty gbuffers.
Now, instead of sending an empty image, a black one with the size of the viewport is sent if the target gbuffer is unavailable.
* Add more GBufferID checks + improve empty gbuffer handling.
* Fix manual_control_gbuffer.py error on repeated listen_to_gbuffer.
* Expose is_listening and is_listening_gbuffer to the PythonAPI.
* Fix data race + autoremove unused gbuffers in manual_control_gbuffer.
* Update PythonAPI docs.
* Remove magic number.
* Switch from error to warning on invalid sensor type when requesting gbuffers.
* Add Misc/GBuffer
* Add EnqueueRenderSceneImmediateWithGBuffer.
* Modified ASceneCaptureSensor to allow gbuffer recording.
* Minor GBuffer progress.
* More GBuffer changes.
* Removed unnecessary files.
* Add FRHITexture* functions to FPixelReader + temporarily disabled non-gbuffer rendering in ASceneCaptureSensor.
* Add missing FPixelReader functions.
* Minor changes (Switching to Windows).
* Remove FRHITexture functions from FPixelReader, added the file ImageUtil.
* Remove FRHITexture functions from FPixelReader, added the file ImageUtil. (2)
* Added API to listen GBuffer data.
* Added gbuffer serializer classes
* Temporarily remove ViewRect hack.
* Add USceneCaptureComponent* derived classes.
* Disable USceneCaptureComponent*_CARLA and add initial FRHIGPUTextureReadBack-based code.
* Fix and re-enable custom SceneCaptureComponents.
* Fully switch to FRHIGPUTextureReadback.
* Remove unnecessary call to FlushRenderingCommands.
* Minor API changes.
* Add support for PF_DepthStencil in ImageUtil.
* More API progress...
* More API progress... (2)
* Removed testing code.
* Minor changes for testing.
* GBuffer API fixes.
* Improve GBuffer capture code.
* Fixed SceneDepth transfer issues and added SceneStencil, CustomDepth and CustomStencil to the GBuffer capture.
* Fix compilation error due to the usage of C++17 features.
* Removed major memory leak and added manual_control_gbuffer.py.
* Fixed a silly mistake.
* Minor changes to manual_control_gbuffer and SceneCaptureSensor.
* Fix compilation error on some versions of Ubuntu.
* Disable TAA when reading GBuffers to avoid jitter.
* Improve memory usage.
* Progress towards automatically detecting when a GBuffer stream is unused.
* Fix includes in SceneCaptureSensor + minor change in manual_control_gbuffer.py
* Progress on automatically detecting which GBuffers aren't needed.
* Remove unneeded __declspec.
* Revert ASensor changes + fix tutorial_gbuffer.py
* Update CHANGELOG.md
* Apply requested changes for the PR, add gitignore for the file OptionalModules.ini and add a GBufferTextureID enum to the Python API.
* Remove OptionalModules.ini.
* Fix indentation.
* Fix indentation (2).
* Fix indentation (3).
* Add documentation and more indentation fixes.
* Remove commented includes.
* Add missing line break.
* Fix memory leak + remove unneeded files.
* Add .uproject again, fix EngineAssociation.
* Remove unneeded ENQUEUE_RENDER_COMMAND.
* Fix manual_control_gbuffer.py.
* Add `stop_gbuffer` to the Python API.
* Minor fixes.
* Fix performance bug.
Previously, when a client requested a gbuffer that is unused it would stay open, even after stopping it explicitly. This commit fixes this issue.
* Fix indentation.
* Add missing braces, more indentation fixes and simplify some of the code.
* Update sensor.yml docs.
* Update docs.
* Remove unnecessary UE_Log + changed one verbosity level.
Co-authored-by: Axel <axellopez92@outlook.com>
Co-authored-by: Axel1092 <35765780+Axel1092@users.noreply.github.com>
* Added sparse map reprsentation
* Added pytorch for terramechanics modules
* Added saving loading particles in independent thread
* Added blank space at the EOF
* Added texture creation lambda
* Moving function to component
* Added input/output architecture for neural network
* Filling heightmap with particles data. Adding Logging to debug
* Updating Texture data at realtime
* Connected 3 stages, using MPC Position to Update and Texture
* Added square particle sampling. Added second model input/output scheme. Fixed start up crash.
* Added new flags and fixes
* Position to update not updating in material but yes in MPC
* Fixed coordinate frame issues and crashes
* Prepared to follow position in the 1st vehicle in map
* Fixed height map alignment
* Fixed large map and terrain tiles alignment
* Fixed inputs for new model
* Added scale factor
* Fixed slow read write operations
* Bug fixes
* Removed debug output
* Removed nvidia profinling marks
* Preparing merge debug code removed, runtime working by rounding the loading data properly
* Made pytorch optional module
* Added TRACE_CPUPROFILER_EVENT_SCOPE to Update and UpdateTexture
* Added optional pytorch conditions and macros
* Removed static Path
* Fixed #ifdef clause. Added missing include.
* Deformation on vehicle working. Ready to start optimisation
* First optimsation done. Heightmap per Tile
* Updated tiles'heightmap to make their size relative to texture and tilesize
* Fixed slow frames. Added functionality
* Removed unecessary library links
* Fixed pytorch link
* Limited search to relevant tiles
* Added multithreaded particle search. Fixed cuda architectures compilation
* Deformation plane added
* Added missing resources
* Updated particle movement update methods
* Spawn on runtime deformation plane
* Added cachemap lock when initializing a region
* Added UHeightMapDataAsset to hold heightmap data
* Fixing some settings
Co-authored-by: Aaron <samaniegoaaron112@gmail.com>
Co-authored-by: bernatx <bernatx@gmail.com>
Michal Sojka