From 6234433bcde6767ba7adc79c0495a03a4b932158 Mon Sep 17 00:00:00 2001 From: criyle Date: Tue, 25 Feb 2025 00:47:49 +0000 Subject: [PATCH] doc: reduce length & move more to docs.goj.ac --- README.cn.md | 69 ++++++++++++++---------------------------------- README.md | 74 ++++++++++++++++------------------------------------ 2 files changed, 42 insertions(+), 101 deletions(-) diff --git a/README.cn.md b/README.cn.md index 0752893..f90071f 100644 --- a/README.cn.md +++ b/README.cn.md @@ -22,15 +22,15 @@ docker run -it --rm --privileged --shm-size=256m -p 5050:5050 --name=go-judge cr 沙箱服务提供 REST API 接口来在受限制的环境中运行程序(默认监听于 `localhost:5050`)。 -- **/run POST 在受限制的环境中运行程序(下面有例子)** -- /file GET 得到所有在文件存储中的文件 ID 到原始命名映射 -- /file POST 上传一个文件到文件存储,返回一个文件 ID 用于提供给 /run 接口 -- /file/:fileId GET 下载文件 ID 指定的文件 -- /file/:fileId DELETE 删除文件 ID 指定的文件 +- **POST /run 在受限制的环境中运行程序** +- GET /file 得到所有在文件存储中的文件 ID 到原始命名映射 + - POST /file 上传一个文件到文件存储,返回一个文件 ID 用于提供给 /run 接口 + - GET /file/:fileId 下载文件 ID 指定的文件 + - DELETE /file/:fileId 删除文件 ID 指定的文件 - /ws /run 接口的 WebSocket 版 -- /stream 运行交互式命令 +- /stream 运行交互式命令。支持流式 api - /version 得到本程序编译版本和 go 语言运行时版本 -- /config 得到本程序部分运行参数,包括沙箱详细参数 + - /config 得到本程序部分运行参数,包括沙箱详细参数 ### REST API 接口定义 @@ -63,13 +63,15 @@ docker run -it --rm --privileged --shm-size=256m -p 5050:5050 --name=go-judge cr +--------------------+----------------+---------------------+----------+-----+ ``` -### 命令行参数 +### 配置 + +[详细配置文档](https://docs.goj.ac/cn/configuration) 服务相关: - 默认监听地址是 `localhost:5050`,使用 `-http-addr` 指定 - 默认 gRPC 接口处于关闭状态,使用 `-enable-grpc` 开启 -- 默认 gRPC 监听地址是 `localhost:5051` ,使用 `-grpc-addr` 指定 + - 默认 gRPC 监听地址是 `localhost:5051` ,使用 `-grpc-addr` 指定 - 默认日志等级是 info ,使用 `-silent` 关闭 或 使用 `-release` 开启 release 级别日志 - 默认没有开启鉴权,使用 `-auth-token` 指定令牌鉴权 - 默认没有开启 go 语言调试接口(`localhost:5052/debug`),使用 `-enable-debug` 开启,同时将日志层级设为 Debug @@ -79,35 +81,25 @@ docker run -it --rm --privileged --shm-size=256m -p 5050:5050 --name=go-judge cr 沙箱相关: - 默认同时运行任务数为和 CPU 数量相同,使用 `-parallelism` 指定 -- 默认文件存储在内存里,使用 `-dir` 指定本地目录为文件存储 -- 默认 cgroup 的前缀为 `gojudge` ,使用 `-cgroup-prefix` 指定 -- 默认没有磁盘文件复制限制,使用 `-src-prefix` 限制 copyIn 操作文件目录前缀,使用逗号 `,` 分隔(需要绝对路径)(例如:`/bin,/usr`) +- 使用 `-mount-conf` 指定沙箱文件系统挂载细节,详细请参见 [文件系统挂载](https://docs.goj.ac/cn/mount) (仅 Linux) +- 使用 `-file-timeout` 指定文件存储文件最大时间。超出时间的文件将会删除。(举例 `30m`) +- 默认文件存储在内存里(`/dev/shm/`),使用 `-dir` 指定本地目录为文件存储 - 默认时间和内存使用检查周期为 100 毫秒(`100ms`),使用 `-time-limit-checker-interval` 指定 - 默认最大输出限制为 `256MiB`,使用 `-output-limit` 指定 - 默认最大打开文件描述符为 `256`,使用 `-open-file-limit` 指定 - 默认最大额外内存使用为 `16KiB` ,使用 `-extra-memory-limit` 指定 - 默认最大 `copyOut` 文件大小为 `64MiB` ,使用 `-copy-out-limit` 指定 -- 使用 `-cpuset` 指定 `cpuset.cpus` (仅 Linux) - 默认容器用户开始区间为 0(不启用) 使用 `-container-cred-start` 指定(仅 Linux) - 举例,默认情况下第 0 个容器使用 10001 作为容器用户。第 1 个容器使用 10002 作为容器用户,以此类推 +- 使用 `-pre-fork` 指定启动时创建的容器数量 +- 默认 cgroup 的前缀为 `gojudge` ,使用 `-cgroup-prefix` 指定 +- 使用 `-cpuset` 指定 `cpuset.cpus` (仅 Linux) - 使用 `-enable-cpu-rate` 开启 `cpu` cgroup 来启用 `cpuRate` 控制(仅 Linux) - 使用 `-cpu-cfs-period` 指定 cfs_period if cpu rate is enabled (default 100ms) (valid value: \[1ms, 1s\]) -- 使用 `-seccomp-conf` 指定 `seecomp` 过滤器(需要编译标志 `seccomp`,默认不开启)(仅 Linux) -- 使用 `-pre-fork` 指定启动时创建的容器数量 -- 使用 `-tmp-fs-param` 指定容器内 `tmpfs` 的挂载参数(仅 Linux) -- 使用 `-file-timeout` 指定文件存储文件最大时间。超出时间的文件将会删除。(举例 `30m`) -- 使用 `-mount-conf` 指定沙箱文件系统挂载细节,详细请参见 `mount.yaml` (仅 Linux) -- 使用 `-container-init-path` 指定 `cinit` 路径 (请不要使用,仅 debug) (仅 Linux) -### 环境变量 +### 沙箱终端 -所有命令行参数都可以通过环境变量的形式来指定,(类似 `ES_HTTP_ADDR` 来指定 `-http-addr`)。使用 `go-judge --help` 查看所有环境变量 - -### 编译沙箱终端 - -编译 `go build ./cmd/go-judge-shell` - -运行 `./go-judge-shell`,需要打开 gRPC 接口来使用。提供一个沙箱内的终端环境。 +从 [Release](https://github.com/criyle/go-judge/releases) 下载 `go-judge-shell` 。运行将连接本地 `go-judge` 沙箱服务并开启一个容器内的终端用于调试。 ### /run 接口返回状态 @@ -183,26 +175,3 @@ docker run -it --rm --privileged --shm-size=256m -p 5050:5050 --name=go-judge cr - `-force-gc-target` 默认 `20m`, 堆内存使用超过该值是强制垃圾收集和归还内存 - `-force-gc-interval` 默认 `5s`, 为后台线程检查的频繁程度 - -### WebSocket 流接口 - -WebSocket 流接口是用于运行一个程序,同时和它的输入输出进行交互。所有的消息都应该使用 WebSocket 的 binary 格式来发送来避免兼容性问题。 - -```text -+--------+--------+---... -| 类型 | 载荷 ... -+--------|--------+---... -请求: -请求类型 = - 1 - 运行请求 (载荷 = JSON 编码的请求体) - 2 - 设置终端窗口大小 (载荷 = JSON 编码的请求体) - 3 - 输入 (载荷 = 1 字节 (4 位的 命令下标 + 4 位的 文件描述符) + 输入内容) - 4 - 取消 (没有载荷) - -响应: -响应类型 = - 1 - 运行结果 (载荷 = JSON 编码的运行结果) - 2 - 输出 (载荷 = 1 字节 (4 位的 命令下标 + 4 位的 文件描述符) + 输入内容) -``` - -任何的不完整,或者不合法的消息会被认为是错误,并终止运行。 diff --git a/README.md b/README.md index d8b452b..ce2ce87 100644 --- a/README.md +++ b/README.md @@ -22,15 +22,15 @@ docker run -it --rm --privileged --shm-size=256m -p 5050:5050 --name=go-judge cr A REST service to run program in restricted environment (Listening on `localhost:5050` by default). -- **/run POST execute program in the restricted environment (examples below)** -- /file GET list all cached file id to original name map -- /file POST prepare a file in the go judge (in memory), returns fileId (can be referenced in /run parameter) -- /file/:fileId GET downloads file from go judge (in memory), returns file content -- /file/:fileId DELETE delete file specified by fileId -- /ws WebSocket for /run -- /stream WebSocket for stream run -- /version gets build git version (e.g. `v1.4.0`) together with runtime information (go version, os, platform) -- /config gets some configuration (e.g. `fileStorePath`, `runnerConfig`) together with some supported features +- **POST /run execute program in the restricted environment** +- GET /file list all cached file id to original name map + - POST /file prepare a file in the go judge (in memory), returns fileId (can be referenced in /run parameter) + - GET /file/:fileId downloads file from go judge (in memory), returns file content + - DELETE /file/:fileId delete file specified by fileId +- /ws WebSocket version for /run +- /stream WebSocket for stream run. Supports streaming interface +- GET /version gets build git version (e.g. `v1.9.0`) together with runtime information (go version, os, platform) + - GET /config gets some configuration (e.g. `fileStorePath`, `runnerConfig`) together with some supported features ### REST API Interface @@ -61,13 +61,15 @@ A REST service to run program in restricted environment (Listening on `localhost +--------------------+----------------+---------------------+---------------+------+ ``` -### Command Line Arguments +### Configurations + +[Configurations](https://docs.goj.ac/configuration) Server: - The default binding address for the go judge is `localhost:5050`. Can be specified with `-http-addr` flag. - By default gRPC endpoint is disabled, to enable gRPC endpoint, add `-enable-grpc` flag. -- The default binding address for the gRPC go judge is `localhost:5051`. Can be specified with `-grpc-addr` flag. + - The default binding address for the gRPC go judge is `localhost:5051`. Can be specified with `-grpc-addr` flag. - The default log level is info, use `-silent` to disable logs or use `-release` to enable release logger (auto turn on if in docker). - `-auth-token` to add token-based authentication to REST / gRPC - By default, the GO debug endpoints (`localhost:5052/debug`) are disabled, to enable, specifies `-enable-debug`, and it also enables debug log @@ -77,32 +79,25 @@ Server: Sandbox: - The default concurrency equal to number of CPU, Can be specified with `-parallelism` flag. -- The default file store is in memory, local cache can be specified with `-dir` flag. -- The default CGroup prefix is `gojudge`, Can be specified with `-cgroup-prefix` flag. -- `-src-prefix` to restrict `src` copyIn path split by comma (need to be absolute path) (example: `/bin,/usr`) -- `-time-limit-checker-interval` specifies time limit checker interval (default 100ms) (valid value: \[1ms, 1s\]) +- `-mount-conf` specifies detailed mount configuration, please refer [File System Mount](https://docs.goj.ac/mount) as a reference (Linux only) +- `-file-timeout` specifies maximum TTL for file created in file store (e.g. `30m`) +- The default file store is in memory(`/dev/shm/`), local cache can be specified with `-dir` flag. +- `-time-limit-checker-interval` specifies time limit checker interval (default 100ms) - `-output-limit` specifies size limit of POSIX rlimit of output (default 256MiB) - `-extra-memory-limit` specifies the additional memory limit to check memory limit exceeded (default 16KiB) - `-copy-out-limit` specifies the default file copy out max (default 64MiB) - `-open-file-limit` specifies the max number of open files (default 256) -- `-cpuset` specifies `cpuset.cpus` cgroup for each container (Linux only) - `-container-cred-start` specifies container `setuid` / `setgid` credential start point (default: 0 (disabled)) (Linux only) - for example, by default container 0 will run with 10001 uid & gid and container 1 will run with 10002 uid & gid... +- `-pre-fork` specifies number of container to create when server starts +- The default CGroup prefix is `gojudge`, Can be specified with `-cgroup-prefix` flag. +- `-cpuset` specifies `cpuset.cpus` cgroup for each container (Linux only) - `-enable-cpu-rate` enabled `cpu` cgroup to control cpu rate using cfs_quota & cfs_period control (Linux only) - `-cpu-cfs-period` specifies cfs_period if cpu rate is enabled (default 100ms) (valid value: \[1ms, 1s\]) -- `-seccomp-conf` specifies `seccomp` filter setting to load when running program (need build tag `seccomp`) (Linux only) - - for example, by `strace -c prog` to get all `syscall` needed and restrict to that sub set - - however, the `syscall` count in one platform(e.g. x86_64) is not suitable for all platform, so this option is not recommended - - the program killed by seccomp filter will have status `Dangerous Syscall` -- `-pre-fork` specifies number of container to create when server starts -- `-tmp-fs-param` specifies the tmpfs parameter for `/w` and `/tmp` when using default mounting (Linux only) -- `-file-timeout` specifies maximum TTL for file created in file store (e.g. `30m`) -- `-mount-conf` specifies detailed mount configuration, please refer `mount.yaml` as a reference (Linux only) -- `-container-init-path` specifies path to `cinit` (do not use, debug only) (Linux only) -### Environment Variables +### Terminal in Container -Environment variable will be override by command line arguments if they both present and all command line arguments have its correspond environment variable (e.g. `ES_HTTP_ADDR`). Run `go-judge --help` to see all the environment variable configurations. +Download `go-judge-shell` from [Release](https://github.com/criyle/go-judge/releases) and run. It will connect local `go-judge`, and open an interactive shell in the container for debugging purpose. ### Return Status @@ -146,7 +141,7 @@ If a bind mount is specifying a target within the previous mounted one, please e ### Notice > [!WARNING] -> Window and macOS support are expriental and should not be used in production environments +> Window and macOS support are experimental and should not be used in production environments #### cgroup usage @@ -176,26 +171,3 @@ Due to limitation of GO runtime, the memory will not return to OS automatically, - `-force-gc-target` default `20m`, the minimal size to trigger GC - `-force-gc-interval` default `5s`, the interval to check memory usage - -### WebSocket Stream Interface - -Websocket stream interface is used to run command interactively with inputs and outputs pumping from the command. All message is transmitted in binary format for maximum compatibility. - -```text -+--------+--------+---... -| type | payload ... -+--------|--------+---... -request: -type = - 1 - request (payload = JSON encoded request) - 2 - resize (payload = JSON encoded resize request) - 3 - input (payload = 1 byte (4-bit index + 4-bit fd), followed by content) - 4 - cancel (no payload) - -response: -type = - 1 - response (payload = JSON encoded response) - 2 - output (payload = 1 byte (4-bit index + 4-bit fd), followed by content) -``` - -Any incomplete / invalid message will be treated as error.