事象
KubernetesにてPodの起動を試みるもステータスがContainerCreating/CrashLoopBackOffで起動に失敗し、Event欄を見ると以下のようなエラーが出ている。
Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 118s default-scheduler Successfully assigned test/nginx-xx-8mpmp to .. Warning FailedCreatePodSandBox 68s (x25 over 116s) kubelet Failed to create pod sandbox: rpc error: code = Unknown desc = failed to create containerd task: failed to create shim: OCI runtime create failed: container_linux.go:380: starting container process caused: container init was OOM-killed (memory limit too low?): unknown
環境情報
Kubernetes 1.21.4
原因/解決策
コンテナのリソース制限にて以下のようにメモリ使用量にlimitが記載されている。
resources: limits: memory: 3Mi requests: memory: 3Mi
同limits部分を取り払う、またはより大きな値を記載すると正常に起動した。
以下、補足です。
補足
コンテナの起動時にメモリが不足すると以下のようにメインプロセスが立ち上がらずエラーになります。
Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 105s default-scheduler Successfully assigned test/nginx-xx-2wzs5 to .. Warning FailedCreatePodSandBox 75s kubelet Failed to create pod sandbox: rpc error: code = Unknown desc = failed to create containerd task: failed to create shim: OCI runtime create failed: runc did not terminate successfully: exit status 2: unknown Normal Created 72s kubelet Created container nginx Warning Failed 72s kubelet Error: failed to create containerd task: failed to create shim: OCI runtime create failed: container_linux.go:364: creating new parent process caused: container_linux.go:2005: running lstat on namespace path "/proc/96417/ns/ipc" caused: lstat /proc/96417/ns/ipc: no such file or directory: unknown Warning FailedCreatePodSandBox 69s kubelet Failed to create pod sandbox: rpc error: code = Unknown desc = failed to create containerd task: failed to create shim: OCI runtime create failed: container_linux.go:380: starting container process caused: process_linux.go:545: container init caused: rootfs_linux.go:76: mounting "tmpfs" to rootfs at "/dev" caused: mkdir /run/containerd/io.containerd.runtime.v2.task/k8s.io/f533e79053d075549c699d00993fe620f40a1f245eb6307dce58fa634140d3a7/rootfs/dev: cannot allocate memory: unknown Normal SandboxChanged 66s (x3 over 71s) kubelet Pod sandbox changed, it will be killed and re-created. Warning FailedCreatePodSandBox 66s (x16 over 104s) kubelet Failed to create pod sandbox: rpc error: code = Unknown desc = failed to create containerd task: failed to create shim: OCI runtime create failed: container_linux.go:380: starting container process caused: container init was OOM-killed (memory limit too low?): unknown Normal Pulled 65s (x2 over 72s) kubelet Container image "..." already present on machine
動作しているPodのリソースは以下コマンドで確認ができますが
$ kubectl top pod .. NAME CPU(cores) MEMORY(bytes) nginx-xx-z88f6 41m 3Mi
その値を参考にlimitに値を入れてしまうと起動時にはそのメモリ以上が必要であれば起動に失敗してしまいます。
以上です。
