mirror/act_runner
1
0
Fork 0
mirror of https://gitea.com/gitea/act_runner.git synced 2025-02-06 18:04:45 +01:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'main' into feature/fetch_task_with_index

Browse Source
This commit is contained in:
sillyguodong 2023-06-07 14:02:44 +08:00
commit 333d82258a
14 changed files with 337 additions and 91 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

30
README.md
View File

@ -88,32 +88,6 @@ You can specify the configuration file path with `-c`/`--config` argument.
./act_runner -c config.yaml daemon # run with config file
```
### Run a docker container
### Example Deployments
```sh
docker run -e GITEA_INSTANCE_URL=http://192.168.8.18:3000 -e GITEA_RUNNER_REGISTRATION_TOKEN=<runner_token> -v /var/run/docker.sock:/var/run/docker.sock -v $PWD/data:/data --name my_runner gitea/act_runner:nightly
```
The `/data` directory inside the docker container contains the runner API keys after registration.
It must be persisted, otherwise the runner would try to register again, using the same, now defunct registration token.
### Running in docker-compose
```yml
...
gitea:
image: gitea/gitea
...
runner:
image: gitea/act_runner
restart: always
depends_on:
- gitea
volumes:
- ./data/act_runner:/data
- /var/run/docker.sock:/var/run/docker.sock
environment:
- GITEA_INSTANCE_URL=<instance url>
- GITEA_RUNNER_REGISTRATION_TOKEN=<registration token>
```
Check out the [examples](examples) directory for sample deployment types.

12
examples/README.md Normal file
View File

@ -0,0 +1,12 @@
# Usage Examples for `act_runner`
Welcome to our collection of usage and deployment examples specifically designed for Gitea setups. Whether you're a beginner or an experienced user, you'll find practical resources here that you can directly apply to enhance your Gitea experience. We encourage you to contribute your own insights and knowledge to make this collection even more comprehensive and valuable.
| Section | Description |
|-----------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [`docker`](docker) | This section provides you with scripts and instructions tailored for running containers on a workstation or server where Docker is installed. It simplifies the process of setting up and managing your Gitea deployment using Docker. |
| [`docker-compose`](docker-compose) | In this section, you'll discover examples demonstrating how to utilize docker-compose to efficiently handle your Gitea deployments. It offers a straightforward approach to managing multiple containerized components of your Gitea setup. |
| [`kubernetes`](kubernetes) | If you're utilizing Kubernetes clusters for your infrastructure, this section is specifically designed for you. It presents examples and guidelines for configuring Gitea deployments within Kubernetes clusters, enabling you to leverage the scalability and flexibility of Kubernetes. |
| [`vm`](vm) | This section is dedicated to examples that assist you in setting up Gitea on virtual or physical servers. Whether you're working with virtual machines or physical hardware, you'll find helpful resources to guide you through the deployment process. |
We hope these resources provide you with valuable insights and solutions for your Gitea setup. Feel free to explore, contribute, and adapt these examples to suit your specific requirements.

20
examples/docker-compose/README.md Normal file
View File

@ -0,0 +1,20 @@
### Running `act_runner` using `docker-compose`
```yml
...
gitea:
image: gitea/gitea
...
runner:
image: gitea/act_runner
restart: always
depends_on:
- gitea
volumes:
- ./data/act_runner:/data
- /var/run/docker.sock:/var/run/docker.sock
environment:
- GITEA_INSTANCE_URL=<instance url>
- GITEA_RUNNER_REGISTRATION_TOKEN=<registration token>
```

8
examples/docker/README.md Normal file
View File

@ -0,0 +1,8 @@
### Run `act_runner` in a Docker Container
```sh
docker run -e GITEA_INSTANCE_URL=http://192.168.8.18:3000 -e GITEA_RUNNER_REGISTRATION_TOKEN=<runner_token> -v /var/run/docker.sock:/var/run/docker.sock -v $PWD/data:/data --name my_runner gitea/act_runner:nightly
```
The `/data` directory inside the docker container contains the runner API keys after registration.
It must be persisted, otherwise the runner would try to register again, using the same, now defunct registration token.

8
examples/kubernetes/README.md Normal file
View File

@ -0,0 +1,8 @@
## Kubernetes Docker in Docker Deployment with `act_runner`
NOTE: Docker in Docker (dind) requires elevated privileges on Kubernetes. The current way to achieve this is to set the pod `SecurityContext` to `privileged`. Keep in mind that this is a potential security issue that has the potential for a malicious application to break out of the container context.
Files in this directory:
- [`dind-docker.yaml`](dind-docker.yaml)
How to create a Deployment and Persistent Volume for Kubernetes to act as a runner. The Docker credentials are re-generated each time the pod connects and does not need to be persisted.

78
examples/kubernetes/dind-docker.yaml Normal file
View File

@ -0,0 +1,78 @@
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: act-runner-vol
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
storageClassName: standard
---
apiVersion: v1
data:
token: << base64 encoded registration token >>
kind: Secret
metadata:
name: runner-secret
type: Opaque
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: act-runner
name: act-runner
spec:
replicas: 1
selector:
matchLabels:
app: act-runner
strategy: {}
template:
metadata:
creationTimestamp: null
labels:
app: act-runner
spec:
restartPolicy: Always
volumes:
- name: docker-certs
emptyDir: {}
- name: runner-data
persistentVolumeClaim:
claimName: act-runner-vol
containers:
- name: runner
image: gitea/act_runner:nightly
command: ["sh", "-c", "while ! nc -z localhost 2376 </dev/null; do echo 'waiting for docker daemon...'; sleep 5; done; /sbin/tini -- /opt/act/run.sh"]
env:
- name: DOCKER_HOST
value: tcp://localhost:2376
- name: DOCKER_CERT_PATH
value: /certs/client
- name: DOCKER_TLS_VERIFY
value: "1"
- name: GITEA_INSTANCE_URL
value: http://gitea-http.gitea.svc.cluster.local:3000
- name: GITEA_RUNNER_REGISTRATION_TOKEN
valueFrom:
secretKeyRef:
name: runner-secret
key: token
volumeMounts:
- name: docker-certs
mountPath: /certs
- name: runner-data
mountPath: /data
- name: daemon
image: docker:23.0.6-dind
env:
- name: DOCKER_TLS_CERTDIR
value: /certs
securityContext:
privileged: true
volumeMounts:
- name: docker-certs
mountPath: /certs

6
examples/vm/README.md Normal file
View File

@ -0,0 +1,6 @@
## `act_runner` on Virtual or Physical Servers
Files in this directory:
- [`rootless-docker.md`](rootless-docker.md)
How to set up a rootless docker implementation of the runner.

87
examples/vm/rootless-docker.md Normal file
View File

@ -0,0 +1,87 @@
## Using Rootless Docker with`act_runner`
Here is a simple example of how to set up `act_runner` with rootless Docker. It has been created with Debian, but other Linux should work the same way.
Note: This procedure needs a real login shell -- using `sudo su` or other method of accessing the account will fail some of the steps below.
As `root`:
- Create a user to run both `docker` and `act_runner`. In this example, we use a non-privileged account called `rootless`.
```bash
useradd -m rootless
passwd rootless
```
- Install [`docker-ce`](https://docs.docker.com/engine/install/)
- (Recommended) Disable the system-wide Docker daemon
``systemctl disable --now docker.service docker.socket``
As the `rootless` user:
- Follow the instructions for [enabling rootless mode](https://docs.docker.com/engine/security/rootless/)
- Add the following lines to the `/home/rootless/.bashrc`:
```bash
export XDG_RUNTIME_DIR=/home/rootless/.docker/run
export PATH=/home/rootless/bin:$PATH
export DOCKER_HOST=unix:///run/user/1001/docker.sock
```
- Reboot. Ensure that the Docker process is working.
- Create a directory for saving `act_runner` data between restarts
`mkdir /home/rootless/act_runner`
- Register the runner from the data directory
```bash
cd /home/rootless/act_runner
act_runner register
```
- Generate a `act_runner` configuration file in the data directory. Edit the file to adjust for the system.
```bash
act_runner generate-config >/home/rootless/act_runner/config
```
- Create a new user-level`systemd` unit file as `/home/rootless/.config/systemd/user/act_runner.service` with the following contents:
```bash
Description=Gitea Actions runner
Documentation=https://gitea.com/gitea/act_runner
After=docker.service
[Service]
Environment=PATH=/home/rootless/bin:/sbin:/usr/sbin:/home/rootless/bin:/home/rootless/bin:/home/rootless/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
Environment=DOCKER_HOST=unix:///run/user/1001/docker.sock
ExecStart=/usr/bin/act_runner daemon -c /home/rootless/act_runner/config
ExecReload=/bin/kill -s HUP $MAINPID
WorkingDirectory=/home/rootless/act_runner
TimeoutSec=0
RestartSec=2
Restart=always
StartLimitBurst=3
StartLimitInterval=60s
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
TasksMax=infinity
Delegate=yes
Type=notify
NotifyAccess=all
KillMode=mixed
[Install]
WantedBy=default.target
```
- Reboot
After the system restarts, check that the`act_runner` is working and that the runner is connected to Gitea.
````bash
systemctl --user status act_runner
journalctl --user -xeu act_runner

2
go.mod
View File

@ -87,4 +87,4 @@ require (
gopkg.in/yaml.v2 v2.4.0 // indirect
)
replace github.com/nektos/act => gitea.com/gitea/act v0.245.2-0.20230516060355-9283cfc9b166
replace github.com/nektos/act => gitea.com/gitea/act v0.245.2-0.20230606002131-6ce5c93cc815

4
go.sum
View File

@ -2,8 +2,8 @@ code.gitea.io/actions-proto-go v0.2.1 h1:ToMN/8thz2q10TuCq8dL2d8mI+/pWpJcHCvG+TE
code.gitea.io/actions-proto-go v0.2.1/go.mod h1:00ys5QDo1iHN1tHNvvddAcy2W/g+425hQya1cCSvq9A=
code.gitea.io/gitea-vet v0.2.3-0.20230113022436-2b1561217fa5 h1:daBEK2GQeqGikJESctP5Cu1i33z5ztAD4kyQWiw185M=
code.gitea.io/gitea-vet v0.2.3-0.20230113022436-2b1561217fa5/go.mod h1:zcNbT/aJEmivCAhfmkHOlT645KNOf9W2KnkLgFjGGfE=
gitea.com/gitea/act v0.245.2-0.20230516060355-9283cfc9b166 h1:hvyzFmxDmdSZBd8S2+r8VqPSK9eihTD2SrTBAvwgYsA=
gitea.com/gitea/act v0.245.2-0.20230516060355-9283cfc9b166/go.mod h1:1ffiGQZAZCLuk9QEBDdbRuQj1GL4uAQk6GNNtcEnPmI=
gitea.com/gitea/act v0.245.2-0.20230606002131-6ce5c93cc815 h1:u4rHwJLJnH6mej1BjEc4iubwknVeJmRVq9xQP9cAMeQ=
gitea.com/gitea/act v0.245.2-0.20230606002131-6ce5c93cc815/go.mod h1:1ffiGQZAZCLuk9QEBDdbRuQj1GL4uAQk6GNNtcEnPmI=
github.com/Azure/go-ansiterm v0.0.0-20170929234023-d6e3b3328b78 h1:w+iIsaOQNcT7OZ575w+acHgRric5iCyQh+xv+KJ4HB8=
github.com/Azure/go-ansiterm v0.0.0-20170929234023-d6e3b3328b78/go.mod h1:LmzpDX56iTiv29bbRTIsUNlaFfuhWRQBWjQdVyAevI8=
github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=

28
internal/app/cmd/daemon.go
View File

@ -7,6 +7,10 @@ import (
"context"
"fmt"
"os"
"path"
"runtime"
"strconv"
"strings"
"github.com/mattn/go-isatty"
log "github.com/sirupsen/logrus"
@ -23,14 +27,13 @@ import (
func runDaemon(ctx context.Context, configFile *string) func(cmd *cobra.Command, args []string) error {
return func(cmd *cobra.Command, args []string) error {
log.Infoln("Starting runner daemon")
cfg, err := config.LoadDefault(*configFile)
if err != nil {
return fmt.Errorf("invalid configuration: %w", err)
}
initLogging(cfg)
log.Infoln("Starting runner daemon")
reg, err := config.LoadRegistration(cfg.Runner.File)
if os.IsNotExist(err) {
@ -79,10 +82,11 @@ func runDaemon(ctx context.Context, configFile *string) func(cmd *cobra.Command,
// initLogging setup the global logrus logger.
func initLogging(cfg *config.Config) {
isTerm := isatty.IsTerminal(os.Stdout.Fd())
log.SetFormatter(&log.TextFormatter{
format := &log.TextFormatter{
DisableColors: !isTerm,
FullTimestamp: true,
})
}
log.SetFormatter(format)
if l := cfg.Log.Level; l != "" {
level, err := log.ParseLevel(l)
@ -90,6 +94,22 @@ func initLogging(cfg *config.Config) {
log.WithError(err).
Errorf("invalid log level: %q", l)
}
// debug level
if level == log.DebugLevel {
log.SetReportCaller(true)
format.CallerPrettyfier = func(f *runtime.Frame) (string, string) {
// get function name
s := strings.Split(f.Function, ".")
funcname := "[" + s[len(s)-1] + "]"
// get file name and line number
_, filename := path.Split(f.File)
filename = "[" + filename + ":" + strconv.Itoa(f.Line) + "]"
return funcname, filename
}
log.SetFormatter(format)
}
if log.GetLevel() != level {
log.Infof("log level changed to %v", level)
log.SetLevel(level)

30
internal/app/cmd/exec.go
View File

@ -357,6 +357,24 @@ func runExec(ctx context.Context, execArgs *executeArgs) func(cmd *cobra.Command
log.Infof("cache handler listens on: %v", handler.ExternalURL())
execArgs.cacheHandler = handler
if len(execArgs.artifactServerAddr) == 0 {
if ip := common.GetOutboundIP(); ip == nil {
return fmt.Errorf("unable to determine outbound IP address")
} else {
execArgs.artifactServerAddr = ip.String()
}
}
if len(execArgs.artifactServerPath) == 0 {
tempDir, err := os.MkdirTemp("", "gitea-act-")
if err != nil {
fmt.Println(err)
}
defer os.RemoveAll(tempDir)
execArgs.artifactServerPath = tempDir
}
// run the plan
config := &runner.Config{
Workdir: execArgs.Workdir(),
@ -381,6 +399,7 @@ func runExec(ctx context.Context, execArgs *executeArgs) func(cmd *cobra.Command
AutoRemove: true,
ArtifactServerPath: execArgs.artifactServerPath,
ArtifactServerPort: execArgs.artifactServerPort,
ArtifactServerAddr: execArgs.artifactServerAddr,
NoSkipCheckout: execArgs.noSkipCheckout,
// PresetGitHubContext: preset,
// EventJSON: string(eventJSON),
@ -403,16 +422,6 @@ func runExec(ctx context.Context, execArgs *executeArgs) func(cmd *cobra.Command
return err
}
if len(execArgs.artifactServerPath) == 0 {
tempDir, err := os.MkdirTemp("", "gitea-act-")
if err != nil {
fmt.Println(err)
}
defer os.RemoveAll(tempDir)
execArgs.artifactServerPath = tempDir
}
artifactCancel := artifacts.Serve(ctx, execArgs.artifactServerPath, execArgs.artifactServerAddr, execArgs.artifactServerPort)
log.Debugf("artifacts server started at %s:%s", execArgs.artifactServerPath, execArgs.artifactServerPort)
@ -459,6 +468,7 @@ func loadExecCmd(ctx context.Context) *cobra.Command {
execCmd.Flags().StringArrayVarP(&execArg.containerCapDrop, "container-cap-drop", "", []string{}, "kernel capabilities to remove from the workflow containers (e.g. --container-cap-drop SYS_PTRACE)")
execCmd.Flags().StringVarP(&execArg.containerOptions, "container-opts", "", "", "container options")
execCmd.PersistentFlags().StringVarP(&execArg.artifactServerPath, "artifact-server-path", "", ".", "Defines the path where the artifact server stores uploads and retrieves downloads from. If not specified the artifact server will not start.")
execCmd.PersistentFlags().StringVarP(&execArg.artifactServerAddr, "artifact-server-addr", "", "", "Defines the address where the artifact server listens")
execCmd.PersistentFlags().StringVarP(&execArg.artifactServerPort, "artifact-server-port", "", "34567", "Defines the port where the artifact server listens (will only bind to localhost).")
execCmd.PersistentFlags().StringVarP(&execArg.defaultActionsUrl, "default-actions-url", "", "https://gitea.com", "Defines the default url of action instance.")
execCmd.PersistentFlags().BoolVarP(&execArg.noSkipCheckout, "no-skip-checkout", "", false, "Do not skip actions/checkout")

50
internal/app/run/runner.go
View File

@ -177,26 +177,26 @@ func (r *Runner) run(ctx context.Context, task *runnerv1.Task, reporter *report.
Workdir: filepath.FromSlash(fmt.Sprintf("/%s/%s", r.cfg.Container.WorkdirParent, preset.Repository)),
BindWorkdir: false,
ReuseContainers: false,
ForcePull: false,
ForceRebuild: false,
LogOutput: true,
JSONLogger: false,
Env: r.envs,
Secrets: task.Secrets,
GitHubInstance: r.client.Address(),
AutoRemove: true,
NoSkipCheckout: true,
PresetGitHubContext: preset,
EventJSON: string(eventJSON),
ContainerNamePrefix: fmt.Sprintf("GITEA-ACTIONS-TASK-%d", task.Id),
ContainerMaxLifetime: maxLifetime,
ContainerNetworkMode: container.NetworkMode(r.cfg.Container.Network),
ContainerOptions: r.cfg.Container.Options,
Privileged: r.cfg.Container.Privileged,
DefaultActionInstance: taskContext["gitea_default_actions_url"].GetStringValue(),
PlatformPicker: r.labels.PickPlatform,
Vars: task.Vars,
ReuseContainers: false,
ForcePull: false,
ForceRebuild: false,
LogOutput: true,
JSONLogger: false,
Env: r.envs,
Secrets: task.Secrets,
GitHubInstance: strings.TrimSuffix(r.client.Address(), "/"),
AutoRemove: true,
NoSkipCheckout: true,
PresetGitHubContext: preset,
EventJSON: string(eventJSON),
ContainerNamePrefix: fmt.Sprintf("GITEA-ACTIONS-TASK-%d", task.Id),
ContainerMaxLifetime: maxLifetime,
ContainerNetworkMode: container.NetworkMode(r.cfg.Container.Network),
ContainerOptions: r.cfg.Container.Options,
Privileged: r.cfg.Container.Privileged,
DefaultActionsURLs: parseDefaultActionsURLs(taskContext["gitea_default_actions_url"].GetStringValue()),
PlatformPicker: r.labels.PickPlatform,
Vars: task.Vars,
}
rr, err := runner.New(runnerConfig)
@ -214,3 +214,13 @@ func (r *Runner) run(ctx context.Context, task *runnerv1.Task, reporter *report.
reporter.SetOutputs(job.Outputs)
return execErr
}
func parseDefaultActionsURLs(s string) []string {
urls := strings.Split(s, ",")
trimmed := make([]string, 0, len(urls))
for _, u := range urls {
t := strings.TrimRight(strings.TrimSpace(u), "/")
trimmed = append(trimmed, t)
}
return trimmed
}

65
internal/pkg/config/config.go
View File

@ -14,33 +14,46 @@ import (
"gopkg.in/yaml.v3"
)
// Log represents the configuration for logging.
type Log struct {
Level string `yaml:"level"` // Level indicates the logging level.
}
// Runner represents the configuration for the runner.
type Runner struct {
File string `yaml:"file"` // File specifies the file path for the runner.
Capacity int `yaml:"capacity"` // Capacity specifies the capacity of the runner.
Envs map[string]string `yaml:"envs"` // Envs stores environment variables for the runner.
EnvFile string `yaml:"env_file"` // EnvFile specifies the path to the file containing environment variables for the runner.
Timeout time.Duration `yaml:"timeout"` // Timeout specifies the duration for runner timeout.
Insecure bool `yaml:"insecure"` // Insecure indicates whether the runner operates in an insecure mode.
FetchTimeout time.Duration `yaml:"fetch_timeout"` // FetchTimeout specifies the timeout duration for fetching resources.
FetchInterval time.Duration `yaml:"fetch_interval"` // FetchInterval specifies the interval duration for fetching resources.
}
// Cache represents the configuration for caching.
type Cache struct {
Enabled *bool `yaml:"enabled"` // Enabled indicates whether caching is enabled. It is a pointer to distinguish between false and not set. If not set, it will be true.
Dir string `yaml:"dir"` // Dir specifies the directory path for caching.
Host string `yaml:"host"` // Host specifies the caching host.
Port uint16 `yaml:"port"` // Port specifies the caching port.
}
// Container represents the configuration for the container.
type Container struct {
Network string `yaml:"network"` // Network specifies the network for the container.
NetworkMode string `yaml:"network_mode"` // Deprecated: use Network instead. Could be removed after Gitea 1.20
Privileged bool `yaml:"privileged"` // Privileged indicates whether the container runs in privileged mode.
Options string `yaml:"options"` // Options specifies additional options for the container.
WorkdirParent string `yaml:"workdir_parent"` // WorkdirParent specifies the parent directory for the container's working directory.
}
// Config represents the overall configuration.
type Config struct {
Log struct {
Level string `yaml:"level"`
} `yaml:"log"`
Runner struct {
File string `yaml:"file"`
Capacity int `yaml:"capacity"`
Envs map[string]string `yaml:"envs"`
EnvFile string `yaml:"env_file"`
Timeout time.Duration `yaml:"timeout"`
Insecure bool `yaml:"insecure"`
FetchTimeout time.Duration `yaml:"fetch_timeout"`
FetchInterval time.Duration `yaml:"fetch_interval"`
} `yaml:"runner"`
Cache struct {
Enabled *bool `yaml:"enabled"` // pointer to distinguish between false and not set, and it will be true if not set
Dir string `yaml:"dir"`
Host string `yaml:"host"`
Port uint16 `yaml:"port"`
} `yaml:"cache"`
Container struct {
Network string `yaml:"network"`
NetworkMode string `yaml:"network_mode"` // Deprecated: use Network instead. Could be removed after Gitea 1.20
Privileged bool `yaml:"privileged"`
Options string `yaml:"options"`
WorkdirParent string `yaml:"workdir_parent"`
} `yaml:"container"`
Log Log `yaml:"log"` // Log represents the configuration for logging.
Runner Runner `yaml:"runner"` // Runner represents the configuration for the runner.
Cache Cache `yaml:"cache"` // Cache represents the configuration for caching.
Container Container `yaml:"container"` // Container represents the configuration for the container.
}
// LoadDefault returns the default configuration.