act/pkg/runner/job_executor.go

200 lines
5.5 KiB
Go
Raw Permalink Normal View History

package runner
import (
"context"
"fmt"
"time"
"github.com/nektos/act/pkg/common"
"github.com/nektos/act/pkg/model"
)
type jobInfo interface {
matrix() map[string]interface{}
steps() []*model.Step
startContainer() common.Executor
stopContainer() common.Executor
closeContainer() common.Executor
interpolateOutputs() common.Executor
result(result string)
}
Add support for service containers (#1949) * Support services (#42) Removed createSimpleContainerName and AutoRemove flag Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com> Co-authored-by: Jason Song <i@wolfogre.com> Reviewed-on: https://gitea.com/gitea/act/pulls/42 Reviewed-by: Jason Song <i@wolfogre.com> Co-authored-by: Zettat123 <zettat123@gmail.com> Co-committed-by: Zettat123 <zettat123@gmail.com> * Support services options (#45) Reviewed-on: https://gitea.com/gitea/act/pulls/45 Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com> Co-authored-by: Zettat123 <zettat123@gmail.com> Co-committed-by: Zettat123 <zettat123@gmail.com> * Support intepolation for `env` of `services` (#47) Reviewed-on: https://gitea.com/gitea/act/pulls/47 Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com> Co-authored-by: Zettat123 <zettat123@gmail.com> Co-committed-by: Zettat123 <zettat123@gmail.com> * Support services `credentials` (#51) If a service's image is from a container registry requires authentication, `act_runner` will need `credentials` to pull the image, see [documentation](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idservicesservice_idcredentials). Currently, `act_runner` incorrectly uses the `credentials` of `containers` to pull services' images and the `credentials` of services won't be used, see the related code: https://gitea.com/gitea/act/src/commit/0c1f2edb996a87ee17dcf3cfa7259c04be02abd7/pkg/runner/run_context.go#L228-L269 Co-authored-by: Jason Song <i@wolfogre.com> Reviewed-on: https://gitea.com/gitea/act/pulls/51 Reviewed-by: Jason Song <i@wolfogre.com> Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com> Co-authored-by: Zettat123 <zettat123@gmail.com> Co-committed-by: Zettat123 <zettat123@gmail.com> * Add ContainerMaxLifetime and ContainerNetworkMode options from: https://gitea.com/gitea/act/commit/b9c20dcaa43899cb3bb327619d447248303170e0 * Fix container network issue (#56) Follow: https://gitea.com/gitea/act_runner/pulls/184 Close https://gitea.com/gitea/act_runner/issues/177 - `act` create new networks only if the value of `NeedCreateNetwork` is true, and remove these networks at last. `NeedCreateNetwork` is passed by `act_runner`. 'NeedCreateNetwork' is true only if `container.network` in the configuration file of the `act_runner` is empty. - In the `docker create` phase, specify the network to which containers will connect. Because, if not specify , container will connect to `bridge` network which is created automatically by Docker. - If the network is user defined network ( the value of `container.network` is empty or `<custom-network>`. Because, the network created by `act` is also user defined network.), will also specify alias by `--network-alias`. The alias of service is `<service-id>`. So we can be access service container by `<service-id>:<port>` in the steps of job. - Won't try to `docker network connect ` network after `docker start` any more. - Because on the one hand, `docker network connect` applies only to user defined networks, if try to `docker network connect host <container-name>` will return error. - On the other hand, we just specify network in the stage of `docker create`, the same effect can be achieved. - Won't try to remove containers and networks berfore the stage of `docker start`, because the name of these containers and netwoks won't be repeat. Co-authored-by: Jason Song <i@wolfogre.com> Reviewed-on: https://gitea.com/gitea/act/pulls/56 Reviewed-by: Jason Song <i@wolfogre.com> Co-authored-by: sillyguodong <gedong_1994@163.com> Co-committed-by: sillyguodong <gedong_1994@163.com> * Check volumes (#60) This PR adds a `ValidVolumes` config. Users can specify the volumes (including bind mounts) that can be mounted to containers by this config. Options related to volumes: - [jobs.<job_id>.container.volumes](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idcontainervolumes) - [jobs.<job_id>.services.<service_id>.volumes](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idservicesservice_idvolumes) In addition, volumes specified by `options` will also be checked. Currently, the following default volumes (see https://gitea.com/gitea/act/src/commit/a72822b3f83d3e68ffc697101b713b7badf57e2f/pkg/runner/run_context.go#L116-L166) will be added to `ValidVolumes`: - `act-toolcache` - `<container-name>` and `<container-name>-env` - `/var/run/docker.sock` (We need to add a new configuration to control whether the docker daemon can be mounted) Co-authored-by: Jason Song <i@wolfogre.com> Reviewed-on: https://gitea.com/gitea/act/pulls/60 Reviewed-by: Jason Song <i@wolfogre.com> Co-authored-by: Zettat123 <zettat123@gmail.com> Co-committed-by: Zettat123 <zettat123@gmail.com> * Remove ContainerMaxLifetime; fix lint * Remove unused ValidVolumes * Remove ConnectToNetwork * Add docker stubs * Close docker clients to prevent file descriptor leaks * Fix the error when removing network in self-hosted mode (#69) Fixes https://gitea.com/gitea/act_runner/issues/255 Reviewed-on: https://gitea.com/gitea/act/pulls/69 Co-authored-by: Zettat123 <zettat123@gmail.com> Co-committed-by: Zettat123 <zettat123@gmail.com> * Move service container and network cleanup to rc.cleanUpJobContainer * Add --network flag; default to host if not using service containers or set explicitly * Correctly close executor to prevent fd leak * Revert to tail instead of full path * fix network duplication * backport networkingConfig for aliaes * don't hardcode netMode host * Convert services test to table driven tests * Add failing tests for services * Expose service container ports onto the host * Set container network mode in artifacts server test to host mode * Log container network mode when creating/starting a container * fix: Correctly handle ContainerNetworkMode * fix: missing service container network * Always remove service containers Although we usually keep containers running if the workflow errored (unless `--rm` is given) in order to facilitate debugging and we have a flag (`--reuse`) to always keep containers running in order to speed up repeated `act` invocations, I believe that these should only apply to job containers and not service containers, because changing the network settings on a service container requires re-creating it anyway. * Remove networks only if no active endpoints exist * Ensure job containers are stopped before starting a new job * fix: go build -tags WITHOUT_DOCKER --------- Co-authored-by: Zettat123 <zettat123@gmail.com> Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com> Co-authored-by: Jason Song <i@wolfogre.com> Co-authored-by: sillyguodong <gedong_1994@163.com> Co-authored-by: ChristopherHX <christopher.homberger@web.de> Co-authored-by: ZauberNerd <zaubernerd@zaubernerd.de>
2023-10-19 17:24:52 +08:00
//nolint:contextcheck,gocyclo
feat: split job steps into its own files/structs (#1004) * refactor: split step_context into separate files This commit moves functions from the step_context.go file into different files, but does otherwise not change anything. This is done to make it easier to review the changes made to these functions in the next commit, where we introduce a step factory to facilitate better unit testing of steps. Co-authored-by: Marcus Noll <marcus.noll@new-work.se> Co-authored-by: Björn Brauer <bjoern.brauer@new-work.se> Co-authored-by: Robert Kowalski <robert.kowalski@new-work.se> Co-authored-by: Philipp Hinrichsen <philipp.hinrichsen@new-work.se> Co-authored-by: Jonas Holland <jonas.holland@new-work.se> * refactor: introduce step factory and make steps testable With this commit we're introducing the `stepFactory` and interfaces and implementations for each different kind of step (run, docker, local and remote actions). Separating each step kind into its own interface and implementation makes it easier to reason about and to change behaviour of the step. By introducing interfaces we enable better unit testability as now each step implementation, the step factory and the job executor can be tested on their own by mocking out parts that are irrelevant. This commits prepares us for implementing pre/post actions in a later PR. Co-authored-by: Marcus Noll <marcus.noll@new-work.se> Co-authored-by: Björn Brauer <bjoern.brauer@new-work.se> Co-authored-by: Robert Kowalski <robert.kowalski@new-work.se> Co-authored-by: Philipp Hinrichsen <philipp.hinrichsen@new-work.se> Co-authored-by: Jonas Holland <jonas.holland@new-work.se> * fix: run post steps in reverse order * test: add missing asserts for mocks * refactor: use local reference instead of function This may make code more easy to follow. * refactor: correct typo in function name * test: use named structs * test: only expected valid calls There are mocks which are only called on certain conditions. * refactor: use step-model to get step name Using the step-logger we have to get the logger name from the step model. * test: only mock stopContainer if required Co-authored-by: Björn Brauer <bjoern.brauer@new-work.se> Co-authored-by: Marcus Noll <marcus.noll@new-work.se> Co-authored-by: Robert Kowalski <robert.kowalski@new-work.se> Co-authored-by: Philipp Hinrichsen <philipp.hinrichsen@new-work.se> Co-authored-by: Jonas Holland <jonas.holland@new-work.se> Co-authored-by: Casey Lee <cplee@nektos.com> Co-authored-by: Christopher Homberger <christopher.homberger@web.de>
2022-03-23 05:13:00 +08:00
func newJobExecutor(info jobInfo, sf stepFactory, rc *RunContext) common.Executor {
steps := make([]common.Executor, 0)
feat: split job steps into its own files/structs (#1004) * refactor: split step_context into separate files This commit moves functions from the step_context.go file into different files, but does otherwise not change anything. This is done to make it easier to review the changes made to these functions in the next commit, where we introduce a step factory to facilitate better unit testing of steps. Co-authored-by: Marcus Noll <marcus.noll@new-work.se> Co-authored-by: Björn Brauer <bjoern.brauer@new-work.se> Co-authored-by: Robert Kowalski <robert.kowalski@new-work.se> Co-authored-by: Philipp Hinrichsen <philipp.hinrichsen@new-work.se> Co-authored-by: Jonas Holland <jonas.holland@new-work.se> * refactor: introduce step factory and make steps testable With this commit we're introducing the `stepFactory` and interfaces and implementations for each different kind of step (run, docker, local and remote actions). Separating each step kind into its own interface and implementation makes it easier to reason about and to change behaviour of the step. By introducing interfaces we enable better unit testability as now each step implementation, the step factory and the job executor can be tested on their own by mocking out parts that are irrelevant. This commits prepares us for implementing pre/post actions in a later PR. Co-authored-by: Marcus Noll <marcus.noll@new-work.se> Co-authored-by: Björn Brauer <bjoern.brauer@new-work.se> Co-authored-by: Robert Kowalski <robert.kowalski@new-work.se> Co-authored-by: Philipp Hinrichsen <philipp.hinrichsen@new-work.se> Co-authored-by: Jonas Holland <jonas.holland@new-work.se> * fix: run post steps in reverse order * test: add missing asserts for mocks * refactor: use local reference instead of function This may make code more easy to follow. * refactor: correct typo in function name * test: use named structs * test: only expected valid calls There are mocks which are only called on certain conditions. * refactor: use step-model to get step name Using the step-logger we have to get the logger name from the step model. * test: only mock stopContainer if required Co-authored-by: Björn Brauer <bjoern.brauer@new-work.se> Co-authored-by: Marcus Noll <marcus.noll@new-work.se> Co-authored-by: Robert Kowalski <robert.kowalski@new-work.se> Co-authored-by: Philipp Hinrichsen <philipp.hinrichsen@new-work.se> Co-authored-by: Jonas Holland <jonas.holland@new-work.se> Co-authored-by: Casey Lee <cplee@nektos.com> Co-authored-by: Christopher Homberger <christopher.homberger@web.de>
2022-03-23 05:13:00 +08:00
preSteps := make([]common.Executor, 0)
implement pre and post steps (#1089) * feat: add post step to actions and add state command This commit includes requried changes for running post steps for local and remote actions. This allows general cleanup work to be done after executing an action. Communication is allowed between this steps, by using the action state. * feat: collect pre and post steps for composite actions * refactor: move composite action logic into own file * refactor: restructure composite handling * feat: run composite post steps during post step lifecycle * refactor: remove duplicate log output * feat: run all composite post actions in a step Since composite actions could have multiple pre/post steps inside, we need to run all of them in a single top-level pre/post step. This PR includes a test case for this and the correct order of steps to be executed. * refactor: remove unused lines of code * refactor: simplify test expression * fix: use composite job logger * fix: make step output more readable * fix: enforce running all post executor To make sure every post executor/step is executed, it is chained with it's own Finally executor. * fix: do not run post step if no step result is available Having no step result means we do not run any step (neither pre nor main) and we do not need to run post. * fix: setup defaults If no pre-if or post-if is given, it should default to 'always()'. This could be set even if there is no pre or post step. In fact this is required for composite actions and included post steps to run. * fix: output step related if expression * test: update expectation * feat: run pre step from actions (#1110) This PR implements running pre steps for remote actions. This includes remote actions using inside local composite actions. * fix: set correct expr default status checks For post-if conditions the default status check should be always(), while for all other if expression the default status check is success() References: https://docs.github.com/en/actions/learn-github-actions/expressions#status-check-functions https://docs.github.com/en/actions/creating-actions/metadata-syntax-for-github-actions#runspost-if * fix: remove code added during rebase
2022-05-24 21:36:06 +08:00
var postExecutor common.Executor
steps = append(steps, func(ctx context.Context) error {
logger := common.Logger(ctx)
if len(info.matrix()) > 0 {
logger.Infof("\U0001F9EA Matrix: %v", info.matrix())
}
return nil
})
feat: split job steps into its own files/structs (#1004) * refactor: split step_context into separate files This commit moves functions from the step_context.go file into different files, but does otherwise not change anything. This is done to make it easier to review the changes made to these functions in the next commit, where we introduce a step factory to facilitate better unit testing of steps. Co-authored-by: Marcus Noll <marcus.noll@new-work.se> Co-authored-by: Björn Brauer <bjoern.brauer@new-work.se> Co-authored-by: Robert Kowalski <robert.kowalski@new-work.se> Co-authored-by: Philipp Hinrichsen <philipp.hinrichsen@new-work.se> Co-authored-by: Jonas Holland <jonas.holland@new-work.se> * refactor: introduce step factory and make steps testable With this commit we're introducing the `stepFactory` and interfaces and implementations for each different kind of step (run, docker, local and remote actions). Separating each step kind into its own interface and implementation makes it easier to reason about and to change behaviour of the step. By introducing interfaces we enable better unit testability as now each step implementation, the step factory and the job executor can be tested on their own by mocking out parts that are irrelevant. This commits prepares us for implementing pre/post actions in a later PR. Co-authored-by: Marcus Noll <marcus.noll@new-work.se> Co-authored-by: Björn Brauer <bjoern.brauer@new-work.se> Co-authored-by: Robert Kowalski <robert.kowalski@new-work.se> Co-authored-by: Philipp Hinrichsen <philipp.hinrichsen@new-work.se> Co-authored-by: Jonas Holland <jonas.holland@new-work.se> * fix: run post steps in reverse order * test: add missing asserts for mocks * refactor: use local reference instead of function This may make code more easy to follow. * refactor: correct typo in function name * test: use named structs * test: only expected valid calls There are mocks which are only called on certain conditions. * refactor: use step-model to get step name Using the step-logger we have to get the logger name from the step model. * test: only mock stopContainer if required Co-authored-by: Björn Brauer <bjoern.brauer@new-work.se> Co-authored-by: Marcus Noll <marcus.noll@new-work.se> Co-authored-by: Robert Kowalski <robert.kowalski@new-work.se> Co-authored-by: Philipp Hinrichsen <philipp.hinrichsen@new-work.se> Co-authored-by: Jonas Holland <jonas.holland@new-work.se> Co-authored-by: Casey Lee <cplee@nektos.com> Co-authored-by: Christopher Homberger <christopher.homberger@web.de>
2022-03-23 05:13:00 +08:00
infoSteps := info.steps()
feat: split job steps into its own files/structs (#1004) * refactor: split step_context into separate files This commit moves functions from the step_context.go file into different files, but does otherwise not change anything. This is done to make it easier to review the changes made to these functions in the next commit, where we introduce a step factory to facilitate better unit testing of steps. Co-authored-by: Marcus Noll <marcus.noll@new-work.se> Co-authored-by: Björn Brauer <bjoern.brauer@new-work.se> Co-authored-by: Robert Kowalski <robert.kowalski@new-work.se> Co-authored-by: Philipp Hinrichsen <philipp.hinrichsen@new-work.se> Co-authored-by: Jonas Holland <jonas.holland@new-work.se> * refactor: introduce step factory and make steps testable With this commit we're introducing the `stepFactory` and interfaces and implementations for each different kind of step (run, docker, local and remote actions). Separating each step kind into its own interface and implementation makes it easier to reason about and to change behaviour of the step. By introducing interfaces we enable better unit testability as now each step implementation, the step factory and the job executor can be tested on their own by mocking out parts that are irrelevant. This commits prepares us for implementing pre/post actions in a later PR. Co-authored-by: Marcus Noll <marcus.noll@new-work.se> Co-authored-by: Björn Brauer <bjoern.brauer@new-work.se> Co-authored-by: Robert Kowalski <robert.kowalski@new-work.se> Co-authored-by: Philipp Hinrichsen <philipp.hinrichsen@new-work.se> Co-authored-by: Jonas Holland <jonas.holland@new-work.se> * fix: run post steps in reverse order * test: add missing asserts for mocks * refactor: use local reference instead of function This may make code more easy to follow. * refactor: correct typo in function name * test: use named structs * test: only expected valid calls There are mocks which are only called on certain conditions. * refactor: use step-model to get step name Using the step-logger we have to get the logger name from the step model. * test: only mock stopContainer if required Co-authored-by: Björn Brauer <bjoern.brauer@new-work.se> Co-authored-by: Marcus Noll <marcus.noll@new-work.se> Co-authored-by: Robert Kowalski <robert.kowalski@new-work.se> Co-authored-by: Philipp Hinrichsen <philipp.hinrichsen@new-work.se> Co-authored-by: Jonas Holland <jonas.holland@new-work.se> Co-authored-by: Casey Lee <cplee@nektos.com> Co-authored-by: Christopher Homberger <christopher.homberger@web.de>
2022-03-23 05:13:00 +08:00
if len(infoSteps) == 0 {
return common.NewDebugExecutor("No steps found")
}
2022-11-17 05:29:45 +08:00
preSteps = append(preSteps, func(ctx context.Context) error {
// Have to be skipped for some Tests
if rc.Run == nil {
return nil
}
rc.ExprEval = rc.NewExpressionEvaluator(ctx)
// evaluate environment variables since they can contain
// GitHub's special environment variables.
for k, v := range rc.GetEnv() {
rc.Env[k] = rc.ExprEval.Interpolate(ctx, v)
}
return nil
})
feat: split job steps into its own files/structs (#1004) * refactor: split step_context into separate files This commit moves functions from the step_context.go file into different files, but does otherwise not change anything. This is done to make it easier to review the changes made to these functions in the next commit, where we introduce a step factory to facilitate better unit testing of steps. Co-authored-by: Marcus Noll <marcus.noll@new-work.se> Co-authored-by: Björn Brauer <bjoern.brauer@new-work.se> Co-authored-by: Robert Kowalski <robert.kowalski@new-work.se> Co-authored-by: Philipp Hinrichsen <philipp.hinrichsen@new-work.se> Co-authored-by: Jonas Holland <jonas.holland@new-work.se> * refactor: introduce step factory and make steps testable With this commit we're introducing the `stepFactory` and interfaces and implementations for each different kind of step (run, docker, local and remote actions). Separating each step kind into its own interface and implementation makes it easier to reason about and to change behaviour of the step. By introducing interfaces we enable better unit testability as now each step implementation, the step factory and the job executor can be tested on their own by mocking out parts that are irrelevant. This commits prepares us for implementing pre/post actions in a later PR. Co-authored-by: Marcus Noll <marcus.noll@new-work.se> Co-authored-by: Björn Brauer <bjoern.brauer@new-work.se> Co-authored-by: Robert Kowalski <robert.kowalski@new-work.se> Co-authored-by: Philipp Hinrichsen <philipp.hinrichsen@new-work.se> Co-authored-by: Jonas Holland <jonas.holland@new-work.se> * fix: run post steps in reverse order * test: add missing asserts for mocks * refactor: use local reference instead of function This may make code more easy to follow. * refactor: correct typo in function name * test: use named structs * test: only expected valid calls There are mocks which are only called on certain conditions. * refactor: use step-model to get step name Using the step-logger we have to get the logger name from the step model. * test: only mock stopContainer if required Co-authored-by: Björn Brauer <bjoern.brauer@new-work.se> Co-authored-by: Marcus Noll <marcus.noll@new-work.se> Co-authored-by: Robert Kowalski <robert.kowalski@new-work.se> Co-authored-by: Philipp Hinrichsen <philipp.hinrichsen@new-work.se> Co-authored-by: Jonas Holland <jonas.holland@new-work.se> Co-authored-by: Casey Lee <cplee@nektos.com> Co-authored-by: Christopher Homberger <christopher.homberger@web.de>
2022-03-23 05:13:00 +08:00
for i, stepModel := range infoSteps {
stepModel := stepModel
if stepModel == nil {
return func(ctx context.Context) error {
return fmt.Errorf("invalid Step %v: missing run or uses key", i)
}
}
feat: split job steps into its own files/structs (#1004) * refactor: split step_context into separate files This commit moves functions from the step_context.go file into different files, but does otherwise not change anything. This is done to make it easier to review the changes made to these functions in the next commit, where we introduce a step factory to facilitate better unit testing of steps. Co-authored-by: Marcus Noll <marcus.noll@new-work.se> Co-authored-by: Björn Brauer <bjoern.brauer@new-work.se> Co-authored-by: Robert Kowalski <robert.kowalski@new-work.se> Co-authored-by: Philipp Hinrichsen <philipp.hinrichsen@new-work.se> Co-authored-by: Jonas Holland <jonas.holland@new-work.se> * refactor: introduce step factory and make steps testable With this commit we're introducing the `stepFactory` and interfaces and implementations for each different kind of step (run, docker, local and remote actions). Separating each step kind into its own interface and implementation makes it easier to reason about and to change behaviour of the step. By introducing interfaces we enable better unit testability as now each step implementation, the step factory and the job executor can be tested on their own by mocking out parts that are irrelevant. This commits prepares us for implementing pre/post actions in a later PR. Co-authored-by: Marcus Noll <marcus.noll@new-work.se> Co-authored-by: Björn Brauer <bjoern.brauer@new-work.se> Co-authored-by: Robert Kowalski <robert.kowalski@new-work.se> Co-authored-by: Philipp Hinrichsen <philipp.hinrichsen@new-work.se> Co-authored-by: Jonas Holland <jonas.holland@new-work.se> * fix: run post steps in reverse order * test: add missing asserts for mocks * refactor: use local reference instead of function This may make code more easy to follow. * refactor: correct typo in function name * test: use named structs * test: only expected valid calls There are mocks which are only called on certain conditions. * refactor: use step-model to get step name Using the step-logger we have to get the logger name from the step model. * test: only mock stopContainer if required Co-authored-by: Björn Brauer <bjoern.brauer@new-work.se> Co-authored-by: Marcus Noll <marcus.noll@new-work.se> Co-authored-by: Robert Kowalski <robert.kowalski@new-work.se> Co-authored-by: Philipp Hinrichsen <philipp.hinrichsen@new-work.se> Co-authored-by: Jonas Holland <jonas.holland@new-work.se> Co-authored-by: Casey Lee <cplee@nektos.com> Co-authored-by: Christopher Homberger <christopher.homberger@web.de>
2022-03-23 05:13:00 +08:00
if stepModel.ID == "" {
stepModel.ID = fmt.Sprintf("%d", i)
}
step, err := sf.newStep(stepModel, rc)
if err != nil {
return common.NewErrorExecutor(err)
}
feat: split job steps into its own files/structs (#1004) * refactor: split step_context into separate files This commit moves functions from the step_context.go file into different files, but does otherwise not change anything. This is done to make it easier to review the changes made to these functions in the next commit, where we introduce a step factory to facilitate better unit testing of steps. Co-authored-by: Marcus Noll <marcus.noll@new-work.se> Co-authored-by: Björn Brauer <bjoern.brauer@new-work.se> Co-authored-by: Robert Kowalski <robert.kowalski@new-work.se> Co-authored-by: Philipp Hinrichsen <philipp.hinrichsen@new-work.se> Co-authored-by: Jonas Holland <jonas.holland@new-work.se> * refactor: introduce step factory and make steps testable With this commit we're introducing the `stepFactory` and interfaces and implementations for each different kind of step (run, docker, local and remote actions). Separating each step kind into its own interface and implementation makes it easier to reason about and to change behaviour of the step. By introducing interfaces we enable better unit testability as now each step implementation, the step factory and the job executor can be tested on their own by mocking out parts that are irrelevant. This commits prepares us for implementing pre/post actions in a later PR. Co-authored-by: Marcus Noll <marcus.noll@new-work.se> Co-authored-by: Björn Brauer <bjoern.brauer@new-work.se> Co-authored-by: Robert Kowalski <robert.kowalski@new-work.se> Co-authored-by: Philipp Hinrichsen <philipp.hinrichsen@new-work.se> Co-authored-by: Jonas Holland <jonas.holland@new-work.se> * fix: run post steps in reverse order * test: add missing asserts for mocks * refactor: use local reference instead of function This may make code more easy to follow. * refactor: correct typo in function name * test: use named structs * test: only expected valid calls There are mocks which are only called on certain conditions. * refactor: use step-model to get step name Using the step-logger we have to get the logger name from the step model. * test: only mock stopContainer if required Co-authored-by: Björn Brauer <bjoern.brauer@new-work.se> Co-authored-by: Marcus Noll <marcus.noll@new-work.se> Co-authored-by: Robert Kowalski <robert.kowalski@new-work.se> Co-authored-by: Philipp Hinrichsen <philipp.hinrichsen@new-work.se> Co-authored-by: Jonas Holland <jonas.holland@new-work.se> Co-authored-by: Casey Lee <cplee@nektos.com> Co-authored-by: Christopher Homberger <christopher.homberger@web.de>
2022-03-23 05:13:00 +08:00
preSteps = append(preSteps, useStepLogger(rc, stepModel, stepStagePre, step.pre()))
feat: split job steps into its own files/structs (#1004) * refactor: split step_context into separate files This commit moves functions from the step_context.go file into different files, but does otherwise not change anything. This is done to make it easier to review the changes made to these functions in the next commit, where we introduce a step factory to facilitate better unit testing of steps. Co-authored-by: Marcus Noll <marcus.noll@new-work.se> Co-authored-by: Björn Brauer <bjoern.brauer@new-work.se> Co-authored-by: Robert Kowalski <robert.kowalski@new-work.se> Co-authored-by: Philipp Hinrichsen <philipp.hinrichsen@new-work.se> Co-authored-by: Jonas Holland <jonas.holland@new-work.se> * refactor: introduce step factory and make steps testable With this commit we're introducing the `stepFactory` and interfaces and implementations for each different kind of step (run, docker, local and remote actions). Separating each step kind into its own interface and implementation makes it easier to reason about and to change behaviour of the step. By introducing interfaces we enable better unit testability as now each step implementation, the step factory and the job executor can be tested on their own by mocking out parts that are irrelevant. This commits prepares us for implementing pre/post actions in a later PR. Co-authored-by: Marcus Noll <marcus.noll@new-work.se> Co-authored-by: Björn Brauer <bjoern.brauer@new-work.se> Co-authored-by: Robert Kowalski <robert.kowalski@new-work.se> Co-authored-by: Philipp Hinrichsen <philipp.hinrichsen@new-work.se> Co-authored-by: Jonas Holland <jonas.holland@new-work.se> * fix: run post steps in reverse order * test: add missing asserts for mocks * refactor: use local reference instead of function This may make code more easy to follow. * refactor: correct typo in function name * test: use named structs * test: only expected valid calls There are mocks which are only called on certain conditions. * refactor: use step-model to get step name Using the step-logger we have to get the logger name from the step model. * test: only mock stopContainer if required Co-authored-by: Björn Brauer <bjoern.brauer@new-work.se> Co-authored-by: Marcus Noll <marcus.noll@new-work.se> Co-authored-by: Robert Kowalski <robert.kowalski@new-work.se> Co-authored-by: Philipp Hinrichsen <philipp.hinrichsen@new-work.se> Co-authored-by: Jonas Holland <jonas.holland@new-work.se> Co-authored-by: Casey Lee <cplee@nektos.com> Co-authored-by: Christopher Homberger <christopher.homberger@web.de>
2022-03-23 05:13:00 +08:00
stepExec := step.main()
steps = append(steps, useStepLogger(rc, stepModel, stepStageMain, func(ctx context.Context) error {
logger := common.Logger(ctx)
err := stepExec(ctx)
if err != nil {
logger.Errorf("%v", err)
common.SetJobError(ctx, err)
} else if ctx.Err() != nil {
logger.Errorf("%v", ctx.Err())
common.SetJobError(ctx, ctx.Err())
}
return nil
}))
feat: split job steps into its own files/structs (#1004) * refactor: split step_context into separate files This commit moves functions from the step_context.go file into different files, but does otherwise not change anything. This is done to make it easier to review the changes made to these functions in the next commit, where we introduce a step factory to facilitate better unit testing of steps. Co-authored-by: Marcus Noll <marcus.noll@new-work.se> Co-authored-by: Björn Brauer <bjoern.brauer@new-work.se> Co-authored-by: Robert Kowalski <robert.kowalski@new-work.se> Co-authored-by: Philipp Hinrichsen <philipp.hinrichsen@new-work.se> Co-authored-by: Jonas Holland <jonas.holland@new-work.se> * refactor: introduce step factory and make steps testable With this commit we're introducing the `stepFactory` and interfaces and implementations for each different kind of step (run, docker, local and remote actions). Separating each step kind into its own interface and implementation makes it easier to reason about and to change behaviour of the step. By introducing interfaces we enable better unit testability as now each step implementation, the step factory and the job executor can be tested on their own by mocking out parts that are irrelevant. This commits prepares us for implementing pre/post actions in a later PR. Co-authored-by: Marcus Noll <marcus.noll@new-work.se> Co-authored-by: Björn Brauer <bjoern.brauer@new-work.se> Co-authored-by: Robert Kowalski <robert.kowalski@new-work.se> Co-authored-by: Philipp Hinrichsen <philipp.hinrichsen@new-work.se> Co-authored-by: Jonas Holland <jonas.holland@new-work.se> * fix: run post steps in reverse order * test: add missing asserts for mocks * refactor: use local reference instead of function This may make code more easy to follow. * refactor: correct typo in function name * test: use named structs * test: only expected valid calls There are mocks which are only called on certain conditions. * refactor: use step-model to get step name Using the step-logger we have to get the logger name from the step model. * test: only mock stopContainer if required Co-authored-by: Björn Brauer <bjoern.brauer@new-work.se> Co-authored-by: Marcus Noll <marcus.noll@new-work.se> Co-authored-by: Robert Kowalski <robert.kowalski@new-work.se> Co-authored-by: Philipp Hinrichsen <philipp.hinrichsen@new-work.se> Co-authored-by: Jonas Holland <jonas.holland@new-work.se> Co-authored-by: Casey Lee <cplee@nektos.com> Co-authored-by: Christopher Homberger <christopher.homberger@web.de>
2022-03-23 05:13:00 +08:00
postExec := useStepLogger(rc, stepModel, stepStagePost, step.post())
implement pre and post steps (#1089) * feat: add post step to actions and add state command This commit includes requried changes for running post steps for local and remote actions. This allows general cleanup work to be done after executing an action. Communication is allowed between this steps, by using the action state. * feat: collect pre and post steps for composite actions * refactor: move composite action logic into own file * refactor: restructure composite handling * feat: run composite post steps during post step lifecycle * refactor: remove duplicate log output * feat: run all composite post actions in a step Since composite actions could have multiple pre/post steps inside, we need to run all of them in a single top-level pre/post step. This PR includes a test case for this and the correct order of steps to be executed. * refactor: remove unused lines of code * refactor: simplify test expression * fix: use composite job logger * fix: make step output more readable * fix: enforce running all post executor To make sure every post executor/step is executed, it is chained with it's own Finally executor. * fix: do not run post step if no step result is available Having no step result means we do not run any step (neither pre nor main) and we do not need to run post. * fix: setup defaults If no pre-if or post-if is given, it should default to 'always()'. This could be set even if there is no pre or post step. In fact this is required for composite actions and included post steps to run. * fix: output step related if expression * test: update expectation * feat: run pre step from actions (#1110) This PR implements running pre steps for remote actions. This includes remote actions using inside local composite actions. * fix: set correct expr default status checks For post-if conditions the default status check should be always(), while for all other if expression the default status check is success() References: https://docs.github.com/en/actions/learn-github-actions/expressions#status-check-functions https://docs.github.com/en/actions/creating-actions/metadata-syntax-for-github-actions#runspost-if * fix: remove code added during rebase
2022-05-24 21:36:06 +08:00
if postExecutor != nil {
Add support for service containers (#1949) * Support services (#42) Removed createSimpleContainerName and AutoRemove flag Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com> Co-authored-by: Jason Song <i@wolfogre.com> Reviewed-on: https://gitea.com/gitea/act/pulls/42 Reviewed-by: Jason Song <i@wolfogre.com> Co-authored-by: Zettat123 <zettat123@gmail.com> Co-committed-by: Zettat123 <zettat123@gmail.com> * Support services options (#45) Reviewed-on: https://gitea.com/gitea/act/pulls/45 Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com> Co-authored-by: Zettat123 <zettat123@gmail.com> Co-committed-by: Zettat123 <zettat123@gmail.com> * Support intepolation for `env` of `services` (#47) Reviewed-on: https://gitea.com/gitea/act/pulls/47 Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com> Co-authored-by: Zettat123 <zettat123@gmail.com> Co-committed-by: Zettat123 <zettat123@gmail.com> * Support services `credentials` (#51) If a service's image is from a container registry requires authentication, `act_runner` will need `credentials` to pull the image, see [documentation](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idservicesservice_idcredentials). Currently, `act_runner` incorrectly uses the `credentials` of `containers` to pull services' images and the `credentials` of services won't be used, see the related code: https://gitea.com/gitea/act/src/commit/0c1f2edb996a87ee17dcf3cfa7259c04be02abd7/pkg/runner/run_context.go#L228-L269 Co-authored-by: Jason Song <i@wolfogre.com> Reviewed-on: https://gitea.com/gitea/act/pulls/51 Reviewed-by: Jason Song <i@wolfogre.com> Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com> Co-authored-by: Zettat123 <zettat123@gmail.com> Co-committed-by: Zettat123 <zettat123@gmail.com> * Add ContainerMaxLifetime and ContainerNetworkMode options from: https://gitea.com/gitea/act/commit/b9c20dcaa43899cb3bb327619d447248303170e0 * Fix container network issue (#56) Follow: https://gitea.com/gitea/act_runner/pulls/184 Close https://gitea.com/gitea/act_runner/issues/177 - `act` create new networks only if the value of `NeedCreateNetwork` is true, and remove these networks at last. `NeedCreateNetwork` is passed by `act_runner`. 'NeedCreateNetwork' is true only if `container.network` in the configuration file of the `act_runner` is empty. - In the `docker create` phase, specify the network to which containers will connect. Because, if not specify , container will connect to `bridge` network which is created automatically by Docker. - If the network is user defined network ( the value of `container.network` is empty or `<custom-network>`. Because, the network created by `act` is also user defined network.), will also specify alias by `--network-alias`. The alias of service is `<service-id>`. So we can be access service container by `<service-id>:<port>` in the steps of job. - Won't try to `docker network connect ` network after `docker start` any more. - Because on the one hand, `docker network connect` applies only to user defined networks, if try to `docker network connect host <container-name>` will return error. - On the other hand, we just specify network in the stage of `docker create`, the same effect can be achieved. - Won't try to remove containers and networks berfore the stage of `docker start`, because the name of these containers and netwoks won't be repeat. Co-authored-by: Jason Song <i@wolfogre.com> Reviewed-on: https://gitea.com/gitea/act/pulls/56 Reviewed-by: Jason Song <i@wolfogre.com> Co-authored-by: sillyguodong <gedong_1994@163.com> Co-committed-by: sillyguodong <gedong_1994@163.com> * Check volumes (#60) This PR adds a `ValidVolumes` config. Users can specify the volumes (including bind mounts) that can be mounted to containers by this config. Options related to volumes: - [jobs.<job_id>.container.volumes](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idcontainervolumes) - [jobs.<job_id>.services.<service_id>.volumes](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idservicesservice_idvolumes) In addition, volumes specified by `options` will also be checked. Currently, the following default volumes (see https://gitea.com/gitea/act/src/commit/a72822b3f83d3e68ffc697101b713b7badf57e2f/pkg/runner/run_context.go#L116-L166) will be added to `ValidVolumes`: - `act-toolcache` - `<container-name>` and `<container-name>-env` - `/var/run/docker.sock` (We need to add a new configuration to control whether the docker daemon can be mounted) Co-authored-by: Jason Song <i@wolfogre.com> Reviewed-on: https://gitea.com/gitea/act/pulls/60 Reviewed-by: Jason Song <i@wolfogre.com> Co-authored-by: Zettat123 <zettat123@gmail.com> Co-committed-by: Zettat123 <zettat123@gmail.com> * Remove ContainerMaxLifetime; fix lint * Remove unused ValidVolumes * Remove ConnectToNetwork * Add docker stubs * Close docker clients to prevent file descriptor leaks * Fix the error when removing network in self-hosted mode (#69) Fixes https://gitea.com/gitea/act_runner/issues/255 Reviewed-on: https://gitea.com/gitea/act/pulls/69 Co-authored-by: Zettat123 <zettat123@gmail.com> Co-committed-by: Zettat123 <zettat123@gmail.com> * Move service container and network cleanup to rc.cleanUpJobContainer * Add --network flag; default to host if not using service containers or set explicitly * Correctly close executor to prevent fd leak * Revert to tail instead of full path * fix network duplication * backport networkingConfig for aliaes * don't hardcode netMode host * Convert services test to table driven tests * Add failing tests for services * Expose service container ports onto the host * Set container network mode in artifacts server test to host mode * Log container network mode when creating/starting a container * fix: Correctly handle ContainerNetworkMode * fix: missing service container network * Always remove service containers Although we usually keep containers running if the workflow errored (unless `--rm` is given) in order to facilitate debugging and we have a flag (`--reuse`) to always keep containers running in order to speed up repeated `act` invocations, I believe that these should only apply to job containers and not service containers, because changing the network settings on a service container requires re-creating it anyway. * Remove networks only if no active endpoints exist * Ensure job containers are stopped before starting a new job * fix: go build -tags WITHOUT_DOCKER --------- Co-authored-by: Zettat123 <zettat123@gmail.com> Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com> Co-authored-by: Jason Song <i@wolfogre.com> Co-authored-by: sillyguodong <gedong_1994@163.com> Co-authored-by: ChristopherHX <christopher.homberger@web.de> Co-authored-by: ZauberNerd <zaubernerd@zaubernerd.de>
2023-10-19 17:24:52 +08:00
// run the post executor in reverse order
postExecutor = postExec.Finally(postExecutor)
implement pre and post steps (#1089) * feat: add post step to actions and add state command This commit includes requried changes for running post steps for local and remote actions. This allows general cleanup work to be done after executing an action. Communication is allowed between this steps, by using the action state. * feat: collect pre and post steps for composite actions * refactor: move composite action logic into own file * refactor: restructure composite handling * feat: run composite post steps during post step lifecycle * refactor: remove duplicate log output * feat: run all composite post actions in a step Since composite actions could have multiple pre/post steps inside, we need to run all of them in a single top-level pre/post step. This PR includes a test case for this and the correct order of steps to be executed. * refactor: remove unused lines of code * refactor: simplify test expression * fix: use composite job logger * fix: make step output more readable * fix: enforce running all post executor To make sure every post executor/step is executed, it is chained with it's own Finally executor. * fix: do not run post step if no step result is available Having no step result means we do not run any step (neither pre nor main) and we do not need to run post. * fix: setup defaults If no pre-if or post-if is given, it should default to 'always()'. This could be set even if there is no pre or post step. In fact this is required for composite actions and included post steps to run. * fix: output step related if expression * test: update expectation * feat: run pre step from actions (#1110) This PR implements running pre steps for remote actions. This includes remote actions using inside local composite actions. * fix: set correct expr default status checks For post-if conditions the default status check should be always(), while for all other if expression the default status check is success() References: https://docs.github.com/en/actions/learn-github-actions/expressions#status-check-functions https://docs.github.com/en/actions/creating-actions/metadata-syntax-for-github-actions#runspost-if * fix: remove code added during rebase
2022-05-24 21:36:06 +08:00
} else {
postExecutor = postExec
implement pre and post steps (#1089) * feat: add post step to actions and add state command This commit includes requried changes for running post steps for local and remote actions. This allows general cleanup work to be done after executing an action. Communication is allowed between this steps, by using the action state. * feat: collect pre and post steps for composite actions * refactor: move composite action logic into own file * refactor: restructure composite handling * feat: run composite post steps during post step lifecycle * refactor: remove duplicate log output * feat: run all composite post actions in a step Since composite actions could have multiple pre/post steps inside, we need to run all of them in a single top-level pre/post step. This PR includes a test case for this and the correct order of steps to be executed. * refactor: remove unused lines of code * refactor: simplify test expression * fix: use composite job logger * fix: make step output more readable * fix: enforce running all post executor To make sure every post executor/step is executed, it is chained with it's own Finally executor. * fix: do not run post step if no step result is available Having no step result means we do not run any step (neither pre nor main) and we do not need to run post. * fix: setup defaults If no pre-if or post-if is given, it should default to 'always()'. This could be set even if there is no pre or post step. In fact this is required for composite actions and included post steps to run. * fix: output step related if expression * test: update expectation * feat: run pre step from actions (#1110) This PR implements running pre steps for remote actions. This includes remote actions using inside local composite actions. * fix: set correct expr default status checks For post-if conditions the default status check should be always(), while for all other if expression the default status check is success() References: https://docs.github.com/en/actions/learn-github-actions/expressions#status-check-functions https://docs.github.com/en/actions/creating-actions/metadata-syntax-for-github-actions#runspost-if * fix: remove code added during rebase
2022-05-24 21:36:06 +08:00
}
}
implement pre and post steps (#1089) * feat: add post step to actions and add state command This commit includes requried changes for running post steps for local and remote actions. This allows general cleanup work to be done after executing an action. Communication is allowed between this steps, by using the action state. * feat: collect pre and post steps for composite actions * refactor: move composite action logic into own file * refactor: restructure composite handling * feat: run composite post steps during post step lifecycle * refactor: remove duplicate log output * feat: run all composite post actions in a step Since composite actions could have multiple pre/post steps inside, we need to run all of them in a single top-level pre/post step. This PR includes a test case for this and the correct order of steps to be executed. * refactor: remove unused lines of code * refactor: simplify test expression * fix: use composite job logger * fix: make step output more readable * fix: enforce running all post executor To make sure every post executor/step is executed, it is chained with it's own Finally executor. * fix: do not run post step if no step result is available Having no step result means we do not run any step (neither pre nor main) and we do not need to run post. * fix: setup defaults If no pre-if or post-if is given, it should default to 'always()'. This could be set even if there is no pre or post step. In fact this is required for composite actions and included post steps to run. * fix: output step related if expression * test: update expectation * feat: run pre step from actions (#1110) This PR implements running pre steps for remote actions. This includes remote actions using inside local composite actions. * fix: set correct expr default status checks For post-if conditions the default status check should be always(), while for all other if expression the default status check is success() References: https://docs.github.com/en/actions/learn-github-actions/expressions#status-check-functions https://docs.github.com/en/actions/creating-actions/metadata-syntax-for-github-actions#runspost-if * fix: remove code added during rebase
2022-05-24 21:36:06 +08:00
postExecutor = postExecutor.Finally(func(ctx context.Context) error {
jobError := common.JobError(ctx)
var err error
if rc.Config.AutoRemove || jobError == nil {
// always allow 1 min for stopping and removing the runner, even if we were cancelled
ctx, cancel := context.WithTimeout(common.WithLogger(context.Background(), common.Logger(ctx)), time.Minute)
defer cancel()
Add support for service containers (#1949) * Support services (#42) Removed createSimpleContainerName and AutoRemove flag Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com> Co-authored-by: Jason Song <i@wolfogre.com> Reviewed-on: https://gitea.com/gitea/act/pulls/42 Reviewed-by: Jason Song <i@wolfogre.com> Co-authored-by: Zettat123 <zettat123@gmail.com> Co-committed-by: Zettat123 <zettat123@gmail.com> * Support services options (#45) Reviewed-on: https://gitea.com/gitea/act/pulls/45 Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com> Co-authored-by: Zettat123 <zettat123@gmail.com> Co-committed-by: Zettat123 <zettat123@gmail.com> * Support intepolation for `env` of `services` (#47) Reviewed-on: https://gitea.com/gitea/act/pulls/47 Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com> Co-authored-by: Zettat123 <zettat123@gmail.com> Co-committed-by: Zettat123 <zettat123@gmail.com> * Support services `credentials` (#51) If a service's image is from a container registry requires authentication, `act_runner` will need `credentials` to pull the image, see [documentation](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idservicesservice_idcredentials). Currently, `act_runner` incorrectly uses the `credentials` of `containers` to pull services' images and the `credentials` of services won't be used, see the related code: https://gitea.com/gitea/act/src/commit/0c1f2edb996a87ee17dcf3cfa7259c04be02abd7/pkg/runner/run_context.go#L228-L269 Co-authored-by: Jason Song <i@wolfogre.com> Reviewed-on: https://gitea.com/gitea/act/pulls/51 Reviewed-by: Jason Song <i@wolfogre.com> Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com> Co-authored-by: Zettat123 <zettat123@gmail.com> Co-committed-by: Zettat123 <zettat123@gmail.com> * Add ContainerMaxLifetime and ContainerNetworkMode options from: https://gitea.com/gitea/act/commit/b9c20dcaa43899cb3bb327619d447248303170e0 * Fix container network issue (#56) Follow: https://gitea.com/gitea/act_runner/pulls/184 Close https://gitea.com/gitea/act_runner/issues/177 - `act` create new networks only if the value of `NeedCreateNetwork` is true, and remove these networks at last. `NeedCreateNetwork` is passed by `act_runner`. 'NeedCreateNetwork' is true only if `container.network` in the configuration file of the `act_runner` is empty. - In the `docker create` phase, specify the network to which containers will connect. Because, if not specify , container will connect to `bridge` network which is created automatically by Docker. - If the network is user defined network ( the value of `container.network` is empty or `<custom-network>`. Because, the network created by `act` is also user defined network.), will also specify alias by `--network-alias`. The alias of service is `<service-id>`. So we can be access service container by `<service-id>:<port>` in the steps of job. - Won't try to `docker network connect ` network after `docker start` any more. - Because on the one hand, `docker network connect` applies only to user defined networks, if try to `docker network connect host <container-name>` will return error. - On the other hand, we just specify network in the stage of `docker create`, the same effect can be achieved. - Won't try to remove containers and networks berfore the stage of `docker start`, because the name of these containers and netwoks won't be repeat. Co-authored-by: Jason Song <i@wolfogre.com> Reviewed-on: https://gitea.com/gitea/act/pulls/56 Reviewed-by: Jason Song <i@wolfogre.com> Co-authored-by: sillyguodong <gedong_1994@163.com> Co-committed-by: sillyguodong <gedong_1994@163.com> * Check volumes (#60) This PR adds a `ValidVolumes` config. Users can specify the volumes (including bind mounts) that can be mounted to containers by this config. Options related to volumes: - [jobs.<job_id>.container.volumes](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idcontainervolumes) - [jobs.<job_id>.services.<service_id>.volumes](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idservicesservice_idvolumes) In addition, volumes specified by `options` will also be checked. Currently, the following default volumes (see https://gitea.com/gitea/act/src/commit/a72822b3f83d3e68ffc697101b713b7badf57e2f/pkg/runner/run_context.go#L116-L166) will be added to `ValidVolumes`: - `act-toolcache` - `<container-name>` and `<container-name>-env` - `/var/run/docker.sock` (We need to add a new configuration to control whether the docker daemon can be mounted) Co-authored-by: Jason Song <i@wolfogre.com> Reviewed-on: https://gitea.com/gitea/act/pulls/60 Reviewed-by: Jason Song <i@wolfogre.com> Co-authored-by: Zettat123 <zettat123@gmail.com> Co-committed-by: Zettat123 <zettat123@gmail.com> * Remove ContainerMaxLifetime; fix lint * Remove unused ValidVolumes * Remove ConnectToNetwork * Add docker stubs * Close docker clients to prevent file descriptor leaks * Fix the error when removing network in self-hosted mode (#69) Fixes https://gitea.com/gitea/act_runner/issues/255 Reviewed-on: https://gitea.com/gitea/act/pulls/69 Co-authored-by: Zettat123 <zettat123@gmail.com> Co-committed-by: Zettat123 <zettat123@gmail.com> * Move service container and network cleanup to rc.cleanUpJobContainer * Add --network flag; default to host if not using service containers or set explicitly * Correctly close executor to prevent fd leak * Revert to tail instead of full path * fix network duplication * backport networkingConfig for aliaes * don't hardcode netMode host * Convert services test to table driven tests * Add failing tests for services * Expose service container ports onto the host * Set container network mode in artifacts server test to host mode * Log container network mode when creating/starting a container * fix: Correctly handle ContainerNetworkMode * fix: missing service container network * Always remove service containers Although we usually keep containers running if the workflow errored (unless `--rm` is given) in order to facilitate debugging and we have a flag (`--reuse`) to always keep containers running in order to speed up repeated `act` invocations, I believe that these should only apply to job containers and not service containers, because changing the network settings on a service container requires re-creating it anyway. * Remove networks only if no active endpoints exist * Ensure job containers are stopped before starting a new job * fix: go build -tags WITHOUT_DOCKER --------- Co-authored-by: Zettat123 <zettat123@gmail.com> Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com> Co-authored-by: Jason Song <i@wolfogre.com> Co-authored-by: sillyguodong <gedong_1994@163.com> Co-authored-by: ChristopherHX <christopher.homberger@web.de> Co-authored-by: ZauberNerd <zaubernerd@zaubernerd.de>
2023-10-19 17:24:52 +08:00
logger := common.Logger(ctx)
logger.Infof("Cleaning up container for job %s", rc.JobName)
if err = info.stopContainer()(ctx); err != nil {
logger.Errorf("Error while stop job container: %v", err)
}
}
setJobResult(ctx, info, rc, jobError == nil)
setJobOutputs(ctx, rc)
return err
})
feat: split job steps into its own files/structs (#1004) * refactor: split step_context into separate files This commit moves functions from the step_context.go file into different files, but does otherwise not change anything. This is done to make it easier to review the changes made to these functions in the next commit, where we introduce a step factory to facilitate better unit testing of steps. Co-authored-by: Marcus Noll <marcus.noll@new-work.se> Co-authored-by: Björn Brauer <bjoern.brauer@new-work.se> Co-authored-by: Robert Kowalski <robert.kowalski@new-work.se> Co-authored-by: Philipp Hinrichsen <philipp.hinrichsen@new-work.se> Co-authored-by: Jonas Holland <jonas.holland@new-work.se> * refactor: introduce step factory and make steps testable With this commit we're introducing the `stepFactory` and interfaces and implementations for each different kind of step (run, docker, local and remote actions). Separating each step kind into its own interface and implementation makes it easier to reason about and to change behaviour of the step. By introducing interfaces we enable better unit testability as now each step implementation, the step factory and the job executor can be tested on their own by mocking out parts that are irrelevant. This commits prepares us for implementing pre/post actions in a later PR. Co-authored-by: Marcus Noll <marcus.noll@new-work.se> Co-authored-by: Björn Brauer <bjoern.brauer@new-work.se> Co-authored-by: Robert Kowalski <robert.kowalski@new-work.se> Co-authored-by: Philipp Hinrichsen <philipp.hinrichsen@new-work.se> Co-authored-by: Jonas Holland <jonas.holland@new-work.se> * fix: run post steps in reverse order * test: add missing asserts for mocks * refactor: use local reference instead of function This may make code more easy to follow. * refactor: correct typo in function name * test: use named structs * test: only expected valid calls There are mocks which are only called on certain conditions. * refactor: use step-model to get step name Using the step-logger we have to get the logger name from the step model. * test: only mock stopContainer if required Co-authored-by: Björn Brauer <bjoern.brauer@new-work.se> Co-authored-by: Marcus Noll <marcus.noll@new-work.se> Co-authored-by: Robert Kowalski <robert.kowalski@new-work.se> Co-authored-by: Philipp Hinrichsen <philipp.hinrichsen@new-work.se> Co-authored-by: Jonas Holland <jonas.holland@new-work.se> Co-authored-by: Casey Lee <cplee@nektos.com> Co-authored-by: Christopher Homberger <christopher.homberger@web.de>
2022-03-23 05:13:00 +08:00
pipeline := make([]common.Executor, 0)
pipeline = append(pipeline, preSteps...)
pipeline = append(pipeline, steps...)
return common.NewPipelineExecutor(info.startContainer(), common.NewPipelineExecutor(pipeline...).
Finally(func(ctx context.Context) error { //nolint:contextcheck
var cancel context.CancelFunc
if ctx.Err() == context.Canceled {
// in case of an aborted run, we still should execute the
// post steps to allow cleanup.
ctx, cancel = context.WithTimeout(common.WithLogger(context.Background(), common.Logger(ctx)), 5*time.Minute)
defer cancel()
}
return postExecutor(ctx)
}).
implement pre and post steps (#1089) * feat: add post step to actions and add state command This commit includes requried changes for running post steps for local and remote actions. This allows general cleanup work to be done after executing an action. Communication is allowed between this steps, by using the action state. * feat: collect pre and post steps for composite actions * refactor: move composite action logic into own file * refactor: restructure composite handling * feat: run composite post steps during post step lifecycle * refactor: remove duplicate log output * feat: run all composite post actions in a step Since composite actions could have multiple pre/post steps inside, we need to run all of them in a single top-level pre/post step. This PR includes a test case for this and the correct order of steps to be executed. * refactor: remove unused lines of code * refactor: simplify test expression * fix: use composite job logger * fix: make step output more readable * fix: enforce running all post executor To make sure every post executor/step is executed, it is chained with it's own Finally executor. * fix: do not run post step if no step result is available Having no step result means we do not run any step (neither pre nor main) and we do not need to run post. * fix: setup defaults If no pre-if or post-if is given, it should default to 'always()'. This could be set even if there is no pre or post step. In fact this is required for composite actions and included post steps to run. * fix: output step related if expression * test: update expectation * feat: run pre step from actions (#1110) This PR implements running pre steps for remote actions. This includes remote actions using inside local composite actions. * fix: set correct expr default status checks For post-if conditions the default status check should be always(), while for all other if expression the default status check is success() References: https://docs.github.com/en/actions/learn-github-actions/expressions#status-check-functions https://docs.github.com/en/actions/creating-actions/metadata-syntax-for-github-actions#runspost-if * fix: remove code added during rebase
2022-05-24 21:36:06 +08:00
Finally(info.interpolateOutputs()).
Finally(info.closeContainer()))
}
func setJobResult(ctx context.Context, info jobInfo, rc *RunContext, success bool) {
logger := common.Logger(ctx)
jobResult := "success"
// we have only one result for a whole matrix build, so we need
// to keep an existing result state if we run a matrix
if len(info.matrix()) > 0 && rc.Run.Job().Result != "" {
jobResult = rc.Run.Job().Result
}
if !success {
jobResult = "failure"
}
info.result(jobResult)
if rc.caller != nil {
// set reusable workflow job result
rc.caller.runContext.result(jobResult)
}
jobResultMessage := "succeeded"
if jobResult != "success" {
jobResultMessage = "failed"
}
logger.WithField("jobResult", jobResult).Infof("\U0001F3C1 Job %s", jobResultMessage)
}
func setJobOutputs(ctx context.Context, rc *RunContext) {
if rc.caller != nil {
// map outputs for reusable workflows
callerOutputs := make(map[string]string)
ee := rc.NewExpressionEvaluator(ctx)
for k, v := range rc.Run.Workflow.WorkflowCallConfig().Outputs {
callerOutputs[k] = ee.Interpolate(ctx, ee.Interpolate(ctx, v.Value))
}
rc.caller.runContext.Run.Job().Outputs = callerOutputs
}
}
func useStepLogger(rc *RunContext, stepModel *model.Step, stage stepStage, executor common.Executor) common.Executor {
return func(ctx context.Context) error {
ctx = withStepLogger(ctx, stepModel.ID, rc.ExprEval.Interpolate(ctx, stepModel.String()), stage.String())
rawLogger := common.Logger(ctx).WithField("raw_output", true)
logWriter := common.NewLineWriter(rc.commandHandler(ctx), func(s string) bool {
if rc.Config.LogOutput {
rawLogger.Infof("%s", s)
} else {
rawLogger.Debugf("%s", s)
}
return true
})
oldout, olderr := rc.JobContainer.ReplaceLogWriter(logWriter, logWriter)
defer rc.JobContainer.ReplaceLogWriter(oldout, olderr)
return executor(ctx)
}
}