mirror of
https://github.com/criyle/go-judge.git
synced 2025-09-26 22:39:12 +08:00

Some checks failed
Build / Goreleaser (push) Has been cancelled
Build / Upload artifacts-${{ matrix.os }}-${{ matrix.arch }} (amd64_v3, darwin) (push) Has been cancelled
Build / Upload artifacts-${{ matrix.os }}-${{ matrix.arch }} (amd64_v3, linux) (push) Has been cancelled
Build / Upload artifacts-${{ matrix.os }}-${{ matrix.arch }} (amd64_v3, windows) (push) Has been cancelled
Build / Upload artifacts-${{ matrix.os }}-${{ matrix.arch }} (arm64_v8.0, darwin) (push) Has been cancelled
Build / Upload artifacts-${{ matrix.os }}-${{ matrix.arch }} (arm64_v8.0, linux) (push) Has been cancelled
Build / Upload artifacts-${{ matrix.os }}-${{ matrix.arch }} (arm64_v8.0, windows) (push) Has been cancelled
163 lines
8.7 KiB
Markdown
163 lines
8.7 KiB
Markdown
# go-judge
|
||
|
||
[](https://pkg.go.dev/github.com/criyle/go-judge) [](https://goreportcard.com/report/github.com/criyle/go-judge) [](https://github.com/criyle/go-judge/releases/latest) 
|
||
|
||
[中文文档](README.cn.md) | [Documentation](https://docs.goj.ac)
|
||
|
||
Fast, Simple, Secure
|
||
|
||
## Quick Start
|
||
|
||
### Install & Run
|
||
|
||
Download compiled executable `go-judge` for your platform from [Release](https://github.com/criyle/go-judge/releases) and run.
|
||
|
||
Or, by docker
|
||
|
||
```bash
|
||
docker run -it --rm --privileged --shm-size=256m -p 5050:5050 --name=go-judge criyle/go-judge
|
||
```
|
||
|
||
### REST API
|
||
|
||
A REST service to run program in restricted environment (Listening on `localhost:5050` by default).
|
||
|
||
- **POST /run execute program in the restricted environment**
|
||
- GET /file list all cached file id to original name map
|
||
- POST /file prepare a file in the go judge (in memory), returns fileId (can be referenced in /run parameter)
|
||
- GET /file/:fileId downloads file from go judge (in memory), returns file content
|
||
- DELETE /file/:fileId delete file specified by fileId
|
||
- /ws WebSocket version for /run
|
||
- /stream WebSocket for stream run. Supports streaming interface
|
||
- GET /version gets build git version (e.g. `v1.9.0`) together with runtime information (go version, os, platform)
|
||
- GET /config gets some configuration (e.g. `fileStorePath`, `runnerConfig`) together with some supported features
|
||
|
||
### REST API Interface
|
||
|
||
[API Interface Structure Definition](https://docs.goj.ac/api#rest-api-interface)
|
||
|
||
### Example Request & Response
|
||
|
||
[Example Request & Response](https://docs.goj.ac/example)
|
||
|
||
## Documentation
|
||
|
||
### Prerequisite
|
||
|
||
- Linux Kernel Version >= 3.10
|
||
- Cgroup file system mounted at /sys/fs/cgroup. Usually done by systemd
|
||
|
||
### Architecture
|
||
|
||
```text
|
||
+----------------------------------------------------------------------------------+
|
||
| Transport Layer (HTTP / WebSocket / FFI / ...) |
|
||
+----------------------------------------------------------------------------------+
|
||
| Sandbox Worker (Environment Pool w/ Environment Builder ) |
|
||
+-----------------------------------------------------------+----------------------+
|
||
| EnvExec | File Store |
|
||
+--------------------+----------------+---------------------+---------------+------+
|
||
| Linux (go-sandbox) | Windows (winc) | macOS (app sandbox) | Shared Memory | Disk |
|
||
+--------------------+----------------+---------------------+---------------+------+
|
||
```
|
||
|
||
### Configurations
|
||
|
||
Server:
|
||
|
||
- The default binding address for the go judge is `localhost:5050`. Can be specified with `-http-addr` flag.
|
||
- By default gRPC endpoint is disabled, to enable gRPC endpoint, add `-enable-grpc` flag.
|
||
- The default binding address for the gRPC go judge is `localhost:5051`. Can be specified with `-grpc-addr` flag.
|
||
- The default log level is info, use `-silent` to disable logs or use `-release` to enable release logger (auto turn on if in docker).
|
||
- `-auth-token` to add token-based authentication to REST / gRPC
|
||
- By default, the GO debug endpoints (`localhost:5052/debug`) are disabled, to enable, specifies `-enable-debug`, and it also enables debug log
|
||
- By default, the prometheus metrics endpoints (`localhost:5052/metrics`) are disabled, to enable, specifies `-enable-metrics`
|
||
- Monitoring HTTP endpoint is enabled if metrics / debug is enabled, the default addr is `localhost:5052` and can be specified by `-monitor-addr`
|
||
|
||
Sandbox:
|
||
|
||
- The default concurrency equal to number of CPU, Can be specified with `-parallelism` flag.
|
||
- `-mount-conf` specifies detailed mount configuration, please refer [File System Mount](https://docs.goj.ac/mount) as a reference (Linux only)
|
||
- `-file-timeout` specifies maximum TTL for file created in file store (e.g. `30m`)
|
||
- The default file store is in memory(`/dev/shm/`), local cache can be specified with `-dir` flag.
|
||
- `-output-limit` specifies size limit of POSIX rlimit of output (default 256MiB)
|
||
- `-copy-out-limit` specifies the default file copy out max (default 64MiB)
|
||
|
||
You can find [more available configuration here](https://docs.goj.ac/configuration).
|
||
|
||
### Run Terminal in the Container
|
||
|
||
Download `go-judge-shell` from [Release](https://github.com/criyle/go-judge/releases) and run. It will connect local `go-judge`, and open an interactive shell in the container for debugging purpose.
|
||
|
||
### Return Status
|
||
|
||
- Accepted: Program exited with status code 0 within time & memory limits
|
||
- Memory Limit Exceeded: Program uses more memory than memory limits
|
||
- Time Limit Exceeded: (`exitStatus` usually have value `9` as killed by `SIGKILL` after timeout)
|
||
- Program uses more CPU time than cpuLimit
|
||
- Or, program uses more clock time than clockLimit
|
||
- Output Limit Exceeded:
|
||
- Program output more than pipeCollector limits
|
||
- Or, program output more than output-limit
|
||
- File Error:
|
||
- CopyIn file is not existed
|
||
- Or, CopyIn file too large for container file system
|
||
- Or, CopyOut file is not existed after program exited
|
||
- Non Zero Exit Status: Program exited with non 0 status code within time & memory limits
|
||
- Signalled: Program exited with signal (e.g. `SIGSEGV`)
|
||
- Dangerous Syscall: Program killed by seccomp filter (not enabled by default)
|
||
- Internal Error:
|
||
- Program is not exist
|
||
- Or, container create not successful (e.g. not privileged docker)
|
||
- Or, other errors
|
||
|
||
### Container Root Filesystem
|
||
|
||
For linux platform, the default mounts points are bind mounting host's `/lib`, `/lib64`, `/usr`, `/bin`, `/etc/ld.so.cache`, `/etc/alternatives`, `/etc/fpc.cfg`, `/dev/null`, `/dev/urandom`, `/dev/random`, `/dev/zero`, `/dev/full` and mounts tmpfs at `/w`, `/tmp` and creates `/proc`.
|
||
|
||
To [customize mount points](https://docs.goj.ac/mount#customization), please look at example `mount.yaml` file.
|
||
|
||
If `mount.yaml` is not specified, the size of `tmpfs` for `/w` and `/tmp` is configured through `-tmp-fs-param` with default value `size=128m,nr_inodes=4k`
|
||
|
||
If a file named `/.env` exists in the container rootfs, the container will load the file as environment variable line by line.
|
||
|
||
If a bind mount is specifying a target within the previous mounted one, please ensure the target exists in the previous mount point.
|
||
|
||
### Metrics Monitoring Endpoint
|
||
|
||
[Prometheus Metrics Monitoring Endpoint](https://docs.goj.ac/api#prometheus-monitor-api)
|
||
|
||
### Notice
|
||
|
||
> [!WARNING]
|
||
> Window and macOS support are experimental and should not be used in production environments
|
||
|
||
#### cgroup usage
|
||
|
||
For cgroup v1, the `go-judge` need root privilege to create `cgroup`. Either creates sub-directory `/sys/fs/cgroup/cpuacct/gojudge`, `/sys/fs/cgroup/memory/gojudge`, `/sys/fs/cgroup/pids/gojudge` and make execution user readable or use `sudo` to run it.
|
||
|
||
For cgroup v2, systemd dbus will be used to create a transient scope for cgroup integration.
|
||
|
||
If no permission to create cgroup, the cgroup related limit will not be effective.
|
||
|
||
#### cgroup v2 support
|
||
|
||
The cgroup v2 is supported by `go-judge` now when running as root since more Linux distribution are enabling cgroup v2 by default (e.g. Ubuntu 21.10+, Fedora 31+). However, for kernel < 5.19, due to missing `memory.max_usage_in_bytes` in `memory` controller, the memory usage is now accounted by `maxrss` returned by `wait4` syscall. Thus, the memory usage appears higher than those who uses cgroup v1. For kernel >= 5.19, `memory.peak` is being used.
|
||
|
||
When running in containers, the `go-judge` will migrate all processed into `/api` hierarchy to enable nesting support.
|
||
|
||
When running in Linux distributions powered by `systemd`, the `go-judge` will contact `systemd` via `dbus` to create a transient scope as cgroup root.
|
||
|
||
When running with kernel >= 5.7, the `go-judge` will try faster `clone3(CLONE_INTO_CGROUP)` and `vfork` method.
|
||
|
||
#### Memory Usage
|
||
|
||
The controller usually consumes `20M` memory and each container usually consumes `20M` + size of tmpfs `2 * 128M`. For each request, it consumes as much as user program limit + extra limit (`16k`) + total copy out max. Notice that the cached file stores in the shared memory (`/dev/shm`) of the host, so please ensure enough size allocated.
|
||
|
||
For example, when concurrency = 4, the container itself can consume as much as `60 + (20+32) * 4M = 268M` + 4 * total copy out + total max memory of requests.
|
||
|
||
Due to limitation of GO runtime, the memory will not return to OS automatically, which could lead to OOM killer. A background worker was introduced to checks heap usage and invokes GC when necessary.
|
||
|
||
- `-force-gc-target` default `20m`, the minimal size to trigger GC
|
||
- `-force-gc-interval` default `5s`, the interval to check memory usage
|