openkylin
/
qemu
mirror of https://gitee.com/openkylin/qemu.git
9
0
Fork 0
Code Issues Projects Releases Wiki Activity
qemu/.gitlab-ci.d/containers.yml

254 lines
6.0 KiB
YAML
Raw Normal View History

gitlab: build all container images during CI We have a number of container images in tests/docker/dockerfiles that are intended to provide well defined environments for doing test builds. We want our CI system to use these containers too. This introduces builds of all of them as the first stage in the CI, so that the built containers are available for later build jobs. The containers are setup to use the GitLab container registry as the cache, so we only pay the penalty of the full build when the dockerfiles change. The main qemu-project/qemu repo is used as a second cache, so that users forking QEMU will see a fast turnaround time on their CI jobs. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-Id: <20200622153318.751107-3-berrange@redhat.com> [AJB: tweak the tag format] Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Acked-by: Thomas Huth <thuth@redhat.com> Message-Id: <20200701135652.1366-22-alex.bennee@linaro.org>
2020-07-01 21:56:33 +08:00
.container_job_template: &container_job_definition
image: docker:stable
stage: containers
services:
- docker:dind
before_script:
- export TAG="$CI_REGISTRY_IMAGE/qemu/$NAME:latest"
- export COMMON_TAG="$CI_REGISTRY/qemu-project/qemu/$NAME:latest"
containers.yml: build with docker.py tooling Instead of building the docker files directly use the same docker.py scripting as we do for building locally. This should help ensure we use the exact same steps and allow us to cache properly when building locally. To get this working you have to have a fairly recent docker binary otherwise you will see the error message: => ERROR importing cache manifest from registry.gitlab.... So far docker 19.03.12 works (from the docker apt repos) but 18.09.1, build 4c52b90 which is packaged in Debian Buster fails. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20200701135652.1366-39-alex.bennee@linaro.org>
2020-07-01 21:56:50 +08:00
- apk add python3
gitlab: build all container images during CI We have a number of container images in tests/docker/dockerfiles that are intended to provide well defined environments for doing test builds. We want our CI system to use these containers too. This introduces builds of all of them as the first stage in the CI, so that the built containers are available for later build jobs. The containers are setup to use the GitLab container registry as the cache, so we only pay the penalty of the full build when the dockerfiles change. The main qemu-project/qemu repo is used as a second cache, so that users forking QEMU will see a fast turnaround time on their CI jobs. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-Id: <20200622153318.751107-3-berrange@redhat.com> [AJB: tweak the tag format] Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Acked-by: Thomas Huth <thuth@redhat.com> Message-Id: <20200701135652.1366-22-alex.bennee@linaro.org>
2020-07-01 21:56:33 +08:00
- docker info
gitlab-ci: Use $CI_REGISTRY instead of hard-coding registry.gitlab.com Update containers.yml to use the $CI_REGISTRY variable as other files such as edk2.yml do. Signed-off-by: Rebecca Cran <rebecca@nuviainc.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-Id: <20201113172519.31056-1-rebecca@nuviainc.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-11-14 01:25:19 +08:00
- docker login $CI_REGISTRY -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD"
gitlab: build all container images during CI We have a number of container images in tests/docker/dockerfiles that are intended to provide well defined environments for doing test builds. We want our CI system to use these containers too. This introduces builds of all of them as the first stage in the CI, so that the built containers are available for later build jobs. The containers are setup to use the GitLab container registry as the cache, so we only pay the penalty of the full build when the dockerfiles change. The main qemu-project/qemu repo is used as a second cache, so that users forking QEMU will see a fast turnaround time on their CI jobs. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-Id: <20200622153318.751107-3-berrange@redhat.com> [AJB: tweak the tag format] Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Acked-by: Thomas Huth <thuth@redhat.com> Message-Id: <20200701135652.1366-22-alex.bennee@linaro.org>
2020-07-01 21:56:33 +08:00
script:
containers.yml: build with docker.py tooling Instead of building the docker files directly use the same docker.py scripting as we do for building locally. This should help ensure we use the exact same steps and allow us to cache properly when building locally. To get this working you have to have a fairly recent docker binary otherwise you will see the error message: => ERROR importing cache manifest from registry.gitlab.... So far docker 19.03.12 works (from the docker apt repos) but 18.09.1, build 4c52b90 which is packaged in Debian Buster fails. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20200701135652.1366-39-alex.bennee@linaro.org>
2020-07-01 21:56:50 +08:00
- echo "TAG:$TAG"
- echo "COMMON_TAG:$COMMON_TAG"
gitlab: build all container images during CI We have a number of container images in tests/docker/dockerfiles that are intended to provide well defined environments for doing test builds. We want our CI system to use these containers too. This introduces builds of all of them as the first stage in the CI, so that the built containers are available for later build jobs. The containers are setup to use the GitLab container registry as the cache, so we only pay the penalty of the full build when the dockerfiles change. The main qemu-project/qemu repo is used as a second cache, so that users forking QEMU will see a fast turnaround time on their CI jobs. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-Id: <20200622153318.751107-3-berrange@redhat.com> [AJB: tweak the tag format] Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Acked-by: Thomas Huth <thuth@redhat.com> Message-Id: <20200701135652.1366-22-alex.bennee@linaro.org>
2020-07-01 21:56:33 +08:00
- docker pull "$TAG" || docker pull "$COMMON_TAG" || true
containers.yml: build with docker.py tooling Instead of building the docker files directly use the same docker.py scripting as we do for building locally. This should help ensure we use the exact same steps and allow us to cache properly when building locally. To get this working you have to have a fairly recent docker binary otherwise you will see the error message: => ERROR importing cache manifest from registry.gitlab.... So far docker 19.03.12 works (from the docker apt repos) but 18.09.1, build 4c52b90 which is packaged in Debian Buster fails. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20200701135652.1366-39-alex.bennee@linaro.org>
2020-07-01 21:56:50 +08:00
- ./tests/docker/docker.py --engine docker build
-t "qemu/$NAME" -f "tests/docker/dockerfiles/$NAME.docker"
-r $CI_REGISTRY_IMAGE
- docker tag "qemu/$NAME" "$TAG"
gitlab: build all container images during CI We have a number of container images in tests/docker/dockerfiles that are intended to provide well defined environments for doing test builds. We want our CI system to use these containers too. This introduces builds of all of them as the first stage in the CI, so that the built containers are available for later build jobs. The containers are setup to use the GitLab container registry as the cache, so we only pay the penalty of the full build when the dockerfiles change. The main qemu-project/qemu repo is used as a second cache, so that users forking QEMU will see a fast turnaround time on their CI jobs. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-Id: <20200622153318.751107-3-berrange@redhat.com> [AJB: tweak the tag format] Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Acked-by: Thomas Huth <thuth@redhat.com> Message-Id: <20200701135652.1366-22-alex.bennee@linaro.org>
2020-07-01 21:56:33 +08:00
- docker push "$TAG"
after_script:
- docker logout
gitlab: limit re-builds of the containers Most of the time we are just rebuilding the same things. We can skip this although currently there is no mechanism for picking up new distro releases. Rather than try to be too fine grained allow any change to trigger all the images being rebuilt. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-Id: <20200701135652.1366-38-alex.bennee@linaro.org>
2020-07-01 21:56:49 +08:00
rules:
- changes:
- .gitlab-ci.d/containers.yml
- tests/docker/*
gitlab-ci/containers: Add missing wildcard where we should look for changes The tests/docker/* wildcard seems to only match the files that are directly in the tests/docker folder - but changes to the files in the directory tests/docker/dockerfiles are currently ignored. Seems like we need a separate entry to match the files in that folder. With this wildcard added, the stages now get re-run successfully when something in the dockerfiles has been changed. Signed-off-by: Thomas Huth <thuth@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20200713182235.30379-1-thuth@redhat.com>
2020-07-14 02:22:35 +08:00
- tests/docker/dockerfiles/*
gitlab: limit re-builds of the containers Most of the time we are just rebuilding the same things. We can skip this although currently there is no mechanism for picking up new distro releases. Rather than try to be too fine grained allow any change to trigger all the images being rebuilt. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-Id: <20200701135652.1366-38-alex.bennee@linaro.org>
2020-07-01 21:56:49 +08:00
- if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'
- if: '$CI_COMMIT_REF_NAME == "testing/next"'
gitlab: build all container images during CI We have a number of container images in tests/docker/dockerfiles that are intended to provide well defined environments for doing test builds. We want our CI system to use these containers too. This introduces builds of all of them as the first stage in the CI, so that the built containers are available for later build jobs. The containers are setup to use the GitLab container registry as the cache, so we only pay the penalty of the full build when the dockerfiles change. The main qemu-project/qemu repo is used as a second cache, so that users forking QEMU will see a fast turnaround time on their CI jobs. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-Id: <20200622153318.751107-3-berrange@redhat.com> [AJB: tweak the tag format] Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Acked-by: Thomas Huth <thuth@redhat.com> Message-Id: <20200701135652.1366-22-alex.bennee@linaro.org>
2020-07-01 21:56:33 +08:00
amd64-centos7-container:
<<: *container_job_definition
variables:
NAME: centos7
amd64-centos8-container:
<<: *container_job_definition
variables:
NAME: centos8
amd64-debian10-container:
<<: *container_job_definition
variables:
NAME: debian10
amd64-debian11-container:
<<: *container_job_definition
variables:
NAME: debian11
alpha-debian-cross-container:
<<: *container_job_definition
stage: containers-layer2
needs: ['amd64-debian10-container']
variables:
NAME: debian-alpha-cross
amd64-debian-cross-container:
<<: *container_job_definition
stage: containers-layer2
needs: ['amd64-debian10-container']
variables:
NAME: debian-amd64-cross
tests/docker: add a linux-user testing focused image We happily use all the cross images for both cross-building QEMU as well as building the linux-user tests. However calling docker from within docker seems not to work. As we can build in Debian anyway why not include an image that has all the compilers available for non-docker invocation. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-Id: <20200701135652.1366-33-alex.bennee@linaro.org>
2020-07-01 21:56:44 +08:00
amd64-debian-user-cross-container:
<<: *container_job_definition
stage: containers-layer2
needs: ['amd64-debian10-container']
variables:
NAME: debian-all-test-cross
gitlab: build all container images during CI We have a number of container images in tests/docker/dockerfiles that are intended to provide well defined environments for doing test builds. We want our CI system to use these containers too. This introduces builds of all of them as the first stage in the CI, so that the built containers are available for later build jobs. The containers are setup to use the GitLab container registry as the cache, so we only pay the penalty of the full build when the dockerfiles change. The main qemu-project/qemu repo is used as a second cache, so that users forking QEMU will see a fast turnaround time on their CI jobs. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-Id: <20200622153318.751107-3-berrange@redhat.com> [AJB: tweak the tag format] Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Acked-by: Thomas Huth <thuth@redhat.com> Message-Id: <20200701135652.1366-22-alex.bennee@linaro.org>
2020-07-01 21:56:33 +08:00
amd64-debian-container:
<<: *container_job_definition
stage: containers-layer2
needs: ['amd64-debian10-container']
variables:
NAME: debian-amd64
arm64-debian-cross-container:
<<: *container_job_definition
stage: containers-layer2
needs: ['amd64-debian10-container']
variables:
NAME: debian-arm64-cross
arm64-test-debian-cross-container:
<<: *container_job_definition
stage: containers-layer2
needs: ['amd64-debian11-container']
variables:
NAME: debian-arm64-test-cross
armel-debian-cross-container:
<<: *container_job_definition
stage: containers-layer2
needs: ['amd64-debian10-container']
variables:
NAME: debian-armel-cross
armhf-debian-cross-container:
<<: *container_job_definition
stage: containers-layer2
needs: ['amd64-debian10-container']
variables:
NAME: debian-armhf-cross
hppa-debian-cross-container:
<<: *container_job_definition
stage: containers-layer2
needs: ['amd64-debian10-container']
variables:
NAME: debian-hppa-cross
m68k-debian-cross-container:
<<: *container_job_definition
stage: containers-layer2
needs: ['amd64-debian10-container']
variables:
NAME: debian-m68k-cross
mips64-debian-cross-container:
<<: *container_job_definition
stage: containers-layer2
needs: ['amd64-debian10-container']
variables:
NAME: debian-mips64-cross
mips64el-debian-cross-container:
<<: *container_job_definition
stage: containers-layer2
needs: ['amd64-debian10-container']
variables:
NAME: debian-mips64el-cross
mips-debian-cross-container:
<<: *container_job_definition
stage: containers-layer2
needs: ['amd64-debian10-container']
variables:
NAME: debian-mips-cross
mipsel-debian-cross-container:
<<: *container_job_definition
stage: containers-layer2
needs: ['amd64-debian10-container']
variables:
NAME: debian-mipsel-cross
powerpc-debian-cross-container:
<<: *container_job_definition
stage: containers-layer2
needs: ['amd64-debian10-container']
variables:
NAME: debian-powerpc-cross
ppc64-debian-cross-container:
<<: *container_job_definition
stage: containers-layer2
needs: ['amd64-debian10-container']
variables:
NAME: debian-ppc64-cross
ppc64el-debian-cross-container:
<<: *container_job_definition
stage: containers-layer2
needs: ['amd64-debian10-container']
variables:
NAME: debian-ppc64el-cross
riscv64-debian-cross-container:
<<: *container_job_definition
stage: containers-layer2
needs: ['amd64-debian10-container']
variables:
NAME: debian-riscv64-cross
s390x-debian-cross-container:
<<: *container_job_definition
stage: containers-layer2
needs: ['amd64-debian10-container']
variables:
NAME: debian-s390x-cross
sh4-debian-cross-container:
<<: *container_job_definition
stage: containers-layer2
needs: ['amd64-debian10-container']
variables:
NAME: debian-sh4-cross
sparc64-debian-cross-container:
<<: *container_job_definition
stage: containers-layer2
needs: ['amd64-debian10-container']
variables:
NAME: debian-sparc64-cross
tricore-debian-cross-container:
<<: *container_job_definition
stage: containers-layer2
tests/docker: Update the tricore container to debian 10 We do not support Debian 9 anymore, thus update the Tricore container to Debian 10 now. Signed-off-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Message-Id: <20200921174320.46062-4-thuth@redhat.com> Message-Id: <20200925154027.12672-12-alex.bennee@linaro.org>
2020-09-25 23:40:23 +08:00
needs: ['amd64-debian10-container']
gitlab: build all container images during CI We have a number of container images in tests/docker/dockerfiles that are intended to provide well defined environments for doing test builds. We want our CI system to use these containers too. This introduces builds of all of them as the first stage in the CI, so that the built containers are available for later build jobs. The containers are setup to use the GitLab container registry as the cache, so we only pay the penalty of the full build when the dockerfiles change. The main qemu-project/qemu repo is used as a second cache, so that users forking QEMU will see a fast turnaround time on their CI jobs. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-Id: <20200622153318.751107-3-berrange@redhat.com> [AJB: tweak the tag format] Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Acked-by: Thomas Huth <thuth@redhat.com> Message-Id: <20200701135652.1366-22-alex.bennee@linaro.org>
2020-07-01 21:56:33 +08:00
variables:
NAME: debian-tricore-cross
xtensa-debian-cross-container:
<<: *container_job_definition
variables:
NAME: debian-xtensa-cross
cris-fedora-cross-container:
<<: *container_job_definition
variables:
NAME: fedora-cris-cross
amd64-fedora-container:
<<: *container_job_definition
variables:
NAME: fedora
i386-fedora-cross-container:
<<: *container_job_definition
variables:
NAME: fedora-i386-cross
tests/docker: Use Fedora containers for MinGW cross-builds in the gitlab-CI According to our support policy, we do not support Debian 9 in QEMU anymore, and we only support building the Windows binaries with a very recent version of the MinGW toolchain. So we should not test the MinGW cross-compilation with Debian 9 anymore, but switch to something newer like Fedora. To do this, we need a separate Fedora container for each build that provides the QEMU_CONFIGURE_OPTS environment variable. Unfortunately, the MinGW 64-bit compiler seems to be a little bit slow, so we also have to disable some features like "capstone" in the build here to make sure that the CI pipelines still finish within a reasonable amount of time. Signed-off-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20200921174320.46062-2-thuth@redhat.com> Message-Id: <20200925154027.12672-10-alex.bennee@linaro.org>
2020-09-25 23:40:21 +08:00
win32-fedora-cross-container:
<<: *container_job_definition
variables:
NAME: fedora-win32-cross
win64-fedora-cross-container:
<<: *container_job_definition
variables:
NAME: fedora-win64-cross
gitlab: build all container images during CI We have a number of container images in tests/docker/dockerfiles that are intended to provide well defined environments for doing test builds. We want our CI system to use these containers too. This introduces builds of all of them as the first stage in the CI, so that the built containers are available for later build jobs. The containers are setup to use the GitLab container registry as the cache, so we only pay the penalty of the full build when the dockerfiles change. The main qemu-project/qemu repo is used as a second cache, so that users forking QEMU will see a fast turnaround time on their CI jobs. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-Id: <20200622153318.751107-3-berrange@redhat.com> [AJB: tweak the tag format] Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Acked-by: Thomas Huth <thuth@redhat.com> Message-Id: <20200701135652.1366-22-alex.bennee@linaro.org>
2020-07-01 21:56:33 +08:00
amd64-ubuntu1804-container:
<<: *container_job_definition
variables:
NAME: ubuntu1804
amd64-ubuntu2004-container:
<<: *container_job_definition
variables:
NAME: ubuntu2004
amd64-ubuntu-container:
<<: *container_job_definition
variables:
NAME: ubuntu
gitlab-ci.yml: Add openSUSE Leap 15.2 for gitlab CI/CD Add build-system-opensuse jobs and opensuse-leap.docker dockerfile. Use openSUSE Leap 15.2 container image in the gitlab-CI. Signed-off-by: Cho, Yu-Chen <acho@suse.com> Tested-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Message-Id: <20201229085046.8536-1-acho@suse.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-12-29 16:50:46 +08:00
amd64-opensuse-leap-container:
<<: *container_job_definition
variables:
NAME: opensuse-leap