p71924506
/
carla
mirror of https://github.com/carla-simulator/carla.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Removed unused dockerfiles

This commit is contained in:
Joel Moriana 2024-12-18 15:07:01 +01:00 committed by Blyron
parent a1c34f4a92
commit c664199a64
6 changed files with 0 additions and 416 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

19
Util/Docker/Carla.Dockerfile
View File

@ -1,19 +0,0 @@
FROM carla-prerequisites:latest
ARG GIT_BRANCH
USER carla
WORKDIR /home/carla
RUN cd /home/carla/ && \
if [ -z ${GIT_BRANCH+x} ]; then git clone --depth 1 https://github.com/carla-simulator/carla.git; \
else git clone --depth 1 --branch $GIT_BRANCH https://github.com/carla-simulator/carla.git; fi && \
cd /home/carla/carla && \
./Update.sh && \
make CarlaUnrealEditor && \
make PythonAPI && \
make build.utils && \
make package && \
rm -r /home/carla/carla/Dist
WORKDIR /home/carla/carla

57
Util/Docker/Prerequisites.Dockerfile
View File

@ -1,57 +0,0 @@
FROM ubuntu:18.04
USER root
ARG EPIC_USER=user
ARG EPIC_PASS=pass
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update ; \
apt-get install -y wget software-properties-common && \
add-apt-repository ppa:ubuntu-toolchain-r/test && \
wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key|apt-key add - && \
apt-add-repository "deb http://apt.llvm.org/xenial/ llvm-toolchain-xenial-8 main" && \
apt-get update ; \
apt-get install -y build-essential \
clang-8 \
lld-8 \
g++-7 \
cmake \
ninja-build \
libvulkan1 \
python \
python-pip \
python-dev \
python3-dev \
python3-pip \
libpng-dev \
libtiff5-dev \
libjpeg-dev \
tzdata \
sed \
curl \
unzip \
autoconf \
libtool \
rsync \
libxml2-dev \
git \
aria2 && \
pip3 install -Iv setuptools==47.3.1 && \
pip3 install distro && \
update-alternatives --install /usr/bin/clang++ clang++ /usr/lib/llvm-8/bin/clang++ 180 && \
update-alternatives --install /usr/bin/clang clang /usr/lib/llvm-8/bin/clang 180
RUN useradd -m carla
COPY --chown=carla:carla . /home/carla
USER carla
WORKDIR /home/carla
ENV UE4_ROOT /home/carla/UE4.26
RUN git clone --depth 1 -b carla "https://${EPIC_USER}:${EPIC_PASS}@github.com/CarlaUnreal/UnrealEngine.git" ${UE4_ROOT}
RUN cd $UE4_ROOT && \
./Setup.sh && \
./GenerateProjectFiles.sh && \
make
WORKDIR /home/carla/

104
Util/Docker/README.md
View File

@ -1,104 +0,0 @@
# Building Carla in a Docker
_These instructions have been tested in **Ubuntu 16.04**._
This file is intended to explain how to build a Docker image that uses **Ubuntu 18.04** to compile Carla.
Since this building process is based on [**ue4-docker**](https://github.com/adamrehn/ue4-docker) project, it is recommended to take a look at their [documentation](https://adamrehn.com/docs/ue4-docker/read-these-first/introduction-to-ue4-docker).
## Important information
**Building these Docker images can be high time and disk space consuming (~4 hours and up to 300GB on Linux and 500GB on Windows in the build process).**
More information about large containers can be found [here](https://adamrehn.com/docs/ue4-docker/read-these-first/large-container-images-primer).
**The Docker images produced by the ue4-docker Python package contain the UE4 Engine Tools in both source code and object code form. As per Section 1A of the [Unreal Engine EULA](https://www.unrealengine.com/en-US/eula), Engine Licensees are prohibited from public distribution of the Engine Tools unless such distribution takes place via the Unreal Marketplace or a fork of the Epic Games UE4 GitHub repository. Public distribution of the built images via an openly accessible Docker Registry (e.g. Docker Hub) is a direct violation of the license terms. It is your responsibility to ensure that any private distribution to other Engine Licensees (such as via an organization's internal Docker Registry) complies with the terms of the Unreal Engine EULA.**
For more details, see the [Unreal Engine EULA Restrictions](https://unrealcontainers.com/docs/obtaining-images/eula-restrictions) page on the [Unreal Containers community hub](https://unrealcontainers.com/).
## Requirements
```
- 64-bit version of Docker in Ubuntu 16.04+.
- Minimum 8GB of RAM
- Minimum 300GB available disk space for building container images
```
---
## Prerequisites
You need Docker installed and configured so your Docker images can access to the Internet during the build process.
Make sure you have installed **Python 3.6 or newer**, check that is in the path, and is callable using `python3` in your terminal. One possible way to achieve so is by using [virtualenvwrapper](https://virtualenvwrapper.readthedocs.io/en/latest/).
## Dependencies
As mentioned before, in order to use Unreal Engine inside Docker we use [ue4-docker](https://github.com/adamrehn/ue4-docker). You can install it using `pip3`.
Further information on installing ue4-docker on Linux can be found [here](https://adamrehn.com/docs/ue4-docker/configuration/configuring-linux).
```
pip3 install ue4-docker
```
You can use ue4-docker to automatically configure your Linux firewall:
```
ue4-docker setup
```
## Building the Docker images
Navigate to `carla/Util/Docker` and use the following commands, each one will take a long time.
First, let's create a Docker image containing a compiled version of Unreal Engine 4 version `24.3`. Change the version if needed.
```
ue4-docker build 4.24.3 --no-engine --no-minimal
```
Next, this will build the image with all the necessary requisites to build Carla in a **Ubuntu 18.04**
```
docker build -t carla-prerequisites -f Prerequisites.Dockerfile .
```
Finally create the actual Carla image, it will search for `carla-prerequisites:latest`:
```
docker build -t carla -f Carla.Dockerfile .
```
---
## Other useful information
You can use a specific repository **branch** or **tag** from our repository, using:
```
docker build -t carla -f Carla.Dockerfile . --build-arg GIT_BRANCH=branch_name
```
Clean up the intermediate images from the build (keep the ue4-source image so you can use it for full rebuilds)
```
ue4-docker clean
```
## Using the Docker tools
The `docker_tools.py` (in `/carla/Util/Docker`) is an example of how you can take advantages of these Docker images. It uses [docker-py](https://github.com/docker/docker-py) whose documentation can be found [here](https://docker-py.readthedocs.io/en/stable/).
The code is really simple and can be easily expanded to interact with docker in other ways.
You can create a Carla package (distribution) from the Docker image using:
```
./docker_tools.py --output /output/path
```
Or you can use it to cook assets (like new maps and meshes), ready to be consumed by a Carla package (distribution):
```
./docker_tools.py --input /assets/to/import/path --output /output/path --packages PkgeName1,PkgeName2
```
The needed files and hierarchy to import assets is explained [here](https://carla.readthedocs.io/en/latest/export_import_dist/).

148
Util/Docker/docker_tools.py
View File

@ -1,148 +0,0 @@
#!/usr/bin/env python
# Copyright (c) 2024 Computer Vision Center (CVC) at the Universitat Autonoma
# de Barcelona (UAB).
#
# This work is licensed under the terms of the MIT license.
# For a copy, see <https://opensource.org/licenses/MIT>.
"""
Helper script to generate consumables for Carla in Docker
"""
from __future__ import print_function
import argparse
import docker
import docker_utils
import os
def print_formated_dict(dic):
for k, v in dic.items():
print(' - "' + str(k) + '"' + ": " + str(v))
print()
def bold(text):
return ''.join([docker_utils.BOLD, text, docker_utils.ENDC])
def bold_underline(text):
return ''.join([docker_utils.UNDERLINE, bold(text)])
def parse_args():
argparser = argparse.ArgumentParser(
description=__doc__)
argparser.add_argument(
'-i', '--input',
type=str,
help='Path of all the assets to convert')
argparser.add_argument(
'-o', '--output',
type=str,
help='Path where all the assets will be after conversion. Default: current directory.')
argparser.add_argument(
'--packages',
type=str,
help='(Optional) Packages to generate. Usage: "--packages=PkgeName1,PkgeName2"')
argparser.add_argument(
'-v', '--verbose',
action='store_true',
default=False,
help='Prints extra information')
argparser.add_argument(
'--image',
type=str,
help='Use a specific Carla image. Default: "carla:latest"',
default='carla:latest',
)
args = argparser.parse_args()
if not args.output:
args.output = os.getcwd()
if args.packages is not None and args.packages and not args.input:
print(
docker_utils.RED +
"[Error] The Input Path [-i|--input] must be specified "
"if you are processing individual packages." + docker_utils.ENDC)
exit(1)
print()
print(bold("- ") + bold_underline("Params:"))
print(" - Output path: " + str(args.output))
print(" - Packages: " + str(args.packages))
print(" - Input path: " + str(args.input))
print(" - Verbose: " + str(args.verbose))
print()
return args
def main():
args = parse_args()
carla_image_name = args.image
inbox_assets_path = '/home/carla/carla/Import'
client = docker.from_env()
# All possible Docker arguments are here:
# https://docker-py.readthedocs.io/en/stable/containers.html
container_args = {
"image": carla_image_name,
"user": 'carla',
"auto_remove": True,
"stdin_open": True,
"tty": True,
"detach": True}
if args.packages:
container_args["volumes"] = {
args.input: {'bind': inbox_assets_path, 'mode': 'rw'}}
print(bold("- ") + bold_underline("Docker arguments:"))
print_formated_dict(container_args)
try:
print("Running Docker...")
carla_container = client.containers.run(**container_args)
if args.packages:
# If there is packages, import them first and package them
docker_utils.exec_command(
carla_container,
'make import',
user='carla', verbose=args.verbose, ignore_error=False)
docker_utils.exec_command(
carla_container,
'make package ARGS="--packages=' + str(args.packages) + '"',
user='carla', verbose=args.verbose, ignore_error=False)
else:
# Just create a package of the whole project
docker_utils.exec_command(
carla_container,
'make package',
user='carla', verbose=args.verbose, ignore_error=False)
# Get the files routes to export
files_to_copy = docker_utils.get_file_paths(
carla_container,
'/home/carla/carla/Dist/*.tar.gz',
user='carla', verbose=args.verbose)
# Copy these files to the output folder
docker_utils.extract_files(carla_container, files_to_copy, args.output)
finally:
print("Closing container " + carla_image_name)
carla_container.stop()
if __name__ == '__main__':
main()

87
Util/Docker/docker_utils.py
View File

@ -1,87 +0,0 @@
#!/usr/bin/env python
# Copyright (c) 2024 Computer Vision Center (CVC) at the Universitat Autonoma
# de Barcelona (UAB).
#
# This work is licensed under the terms of the MIT license.
# For a copy, see <https://opensource.org/licenses/MIT>.
import tarfile
import os
BLUE = '\033[94m'
GREEN = '\033[92m'
RED = '\033[91m'
ENDC = '\033[0m'
BOLD = '\033[1m'
UNDERLINE = '\033[4m'
class ReadableStream():
def __init__(self, generator):
self._generator = generator
def read(self):
return next(self._generator)
def get_container_name(container):
return str(container.attrs['Config']['Image'])
def exec_command(container, command, user="root",
silent=False, verbose=False, ignore_error=True):
command_prefix = "bash -c '"
if not silent:
print(''.join([BOLD, BLUE,
user, '@', get_container_name(container),
ENDC, '$ ', str(command)]))
command_result = container.exec_run(
command_prefix + command + "'",
user=user)
if not silent and verbose and command_result.exit_code:
print(''.join([RED, 'Command: "', command,
'" exited with error: ', str(command_result.exit_code),
ENDC]))
print('Error:')
if not silent:
out = command_result.output.decode().strip()
if out:
print(out)
if not ignore_error and command_result.exit_code:
exit(1)
return command_result
def get_file_paths(container, path, user="root",
absolute_path=True, hidden_files=True, verbose=False):
# command = "bash -c 'ls -ad $PWD/* "
command = "ls "
if hidden_files:
command += "-a "
if absolute_path:
command += "-d "
result = exec_command(container, command + path, user=user, silent=True)
if result.exit_code:
if verbose:
print(RED + "No files found in " + path + ENDC)
return []
file_list = [x for x in result.output.decode('utf-8').split('\n') if x]
if verbose:
print("Found files: " + str(file_list))
return file_list
def extract_files(container, file_list, out_path):
for file in file_list:
print('Copying "' + file + '" to ' + out_path)
strm, _ = container.get_archive(file)
f = open("%s/result.tar.gz" % out_path, "wb")
for d in strm:
f.write(d)
f.close()
pw_tar = tarfile.TarFile("%s/result.tar.gz" % out_path)
pw_tar.extractall(out_path)
os.remove("%s/result.tar.gz" % out_path)

1
Util/Docker/requirements.txt
View File

@ -1 +0,0 @@
docker