(O+P)ut

アウトプット



(O+P)ut

エンジニアのアウトプット

【Kubernetes】FailedCreatePodSandBoxでPodが起動しない事象

スポンサーリンク

事象

KubernetesにてPodの起動を試みるもステータスがContainerCreating/CrashLoopBackOffで起動に失敗し、Event欄を見ると以下のようなエラーが出ている。

Events:
  Type     Reason                  Age                  From               Message
  ----     ------                  ----                 ----               -------
  Normal   Scheduled               118s                 default-scheduler  Successfully assigned test/nginx-xx-8mpmp to ..
  Warning  FailedCreatePodSandBox  68s (x25 over 116s)  kubelet            Failed to create pod sandbox: rpc error: code = Unknown desc = failed to create containerd task: failed to create shim: OCI runtime create failed: container_linux.go:380: starting container process caused: container init was OOM-killed (memory limit too low?): unknown
環境情報
Kubernetes 1.21.4

原因/解決策

コンテナのリソース制限にて以下のようにメモリ使用量にlimitが記載されている。

        resources:
          limits:
            memory: 3Mi
          requests:
            memory: 3Mi

同limits部分を取り払う、またはより大きな値を記載すると正常に起動した。

以下、補足です。

補足

コンテナの起動時にメモリが不足すると以下のようにメインプロセスが立ち上がらずエラーになります。

Events:
  Type     Reason                  Age                  From               Message
  ----     ------                  ----                 ----               -------
  Normal   Scheduled               105s                 default-scheduler  Successfully assigned test/nginx-xx-2wzs5 to ..
  Warning  FailedCreatePodSandBox  75s                  kubelet            Failed to create pod sandbox: rpc error: code = Unknown desc = failed to create containerd task: failed to create shim: OCI runtime create failed: runc did not terminate successfully: exit status 2: unknown
  Normal   Created                 72s                  kubelet            Created container nginx
  Warning  Failed                  72s                  kubelet            Error: failed to create containerd task: failed to create shim: OCI runtime create failed: container_linux.go:364: creating new parent process caused: container_linux.go:2005: running lstat on namespace path "/proc/96417/ns/ipc" caused: lstat /proc/96417/ns/ipc: no such file or directory: unknown  Warning  FailedCreatePodSandBox  69s                  kubelet            Failed to create pod sandbox: rpc error: code = Unknown desc = failed to create containerd task: failed to create shim: OCI runtime create failed: container_linux.go:380: starting container process caused: process_linux.go:545: container init caused: rootfs_linux.go:76: mounting "tmpfs" to rootfs at "/dev" caused: mkdir /run/containerd/io.containerd.runtime.v2.task/k8s.io/f533e79053d075549c699d00993fe620f40a1f245eb6307dce58fa634140d3a7/rootfs/dev: cannot allocate memory: unknown
  Normal   SandboxChanged          66s (x3 over 71s)    kubelet            Pod sandbox changed, it will be killed and re-created.
  Warning  FailedCreatePodSandBox  66s (x16 over 104s)  kubelet            Failed to create pod sandbox: rpc error: code = Unknown desc = failed to create containerd task: failed to create shim: OCI runtime create failed: container_linux.go:380: starting container process caused: container init was OOM-killed (memory limit too low?): unknown
  Normal   Pulled                  65s (x2 over 72s)    kubelet            Container image "..." already present on machine

動作しているPodのリソースは以下コマンドで確認ができますが

$ kubectl top pod
..
NAME                     CPU(cores)   MEMORY(bytes)
nginx-xx-z88f6   41m          3Mi

その値を参考にlimitに値を入れてしまうと起動時にはそのメモリ以上が必要であれば起動に失敗してしまいます。

以上です。


他の記事を読む