更新最新版V0.8.41后，docker中调用nvidia失效的一些分享

老谈仙森

[i=s] 本帖最后由 玉尺书生 于 2025-3-28 01:34 编辑 [/i]<br /> <br />

系统背景

首先根据《在飞牛OS使用libnvidia-container让docker容器支持NVIDIA GPU加速》文章中 https://club.fnnas.com/forum.php?mod=viewthread&tid=14106描述的方式成功运行nvidia-container，如果没有完成，请尝试按照教程中的方法执行命令，有一些问题，会在后面提到。

你可能遇到的问题

1. 安装nvidia-container-toolkit时，出现了这样类似的输出

   ldconfig: /lib/x86_64-linux-gnu/libnvidia-fbc.so.1 is not a symbolic link

   ldconfig: /lib/x86_64-linux-gnu/libnvidia-opticalflow.so.1 is not a symbolic link

   ldconfig: /lib/x86_64-linux-gnu/libGLESv1_CM_nvidia.so.1 is not a symbolic link

   ldconfig: /lib/x86_64-linux-gnu/libnvidia-egl-gbm.so.1 is not a symbolic link

   先不要慌，请把这些打印出来的信息复制到一个文本文件中

2. 运行命令

   docker run --rm --runtime=nvidia --gpus all ubuntu nvidia-smi

   时，会出现错误,类似这样的错误 docker: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error running hook #0: error running hook: exit status 1, stdout: , stderr: Auto-detected mode as 'legacy' nvidia-container-cli: initialization error: load library failed: libnvidia-ml.so.1: cannot open shared object file: no such file or directory: unknown.

   ---

   还有这样的错误 docker: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error running hook #0: error running hook: exit status 1, stdout: , stderr: Auto-detected mode as 'legacy' nvidia-container-cli: detection error: open failed: /usr/lib/x86_64-linux-gnu/libvdpau_nvidia.so: permission denied: unknown.

你需要做的事情

1. 在应用商店中卸载原来安装的驱动，同时进入文件管理页面--》管理员视角，找到@app开头的文件夹，每一个都进入检查以下，删除Nvidia-Driver文件夹

2. 重新安装应用商店的驱动

3. 将之前保存的文本中，所有出现过的每一行路径，类似/lib/x86_64-linux-gnu/libnvidia-fbc.so.1，都执行一遍

   sudo ln -sf /lib/x86_64-linux-gnu/libnvidia-fbc.so.560.28.03 /lib/x86_64-linux-gnu/libnvidia-fbc.so.1

   记得替换文件名称，以下是我自己的，可能和你有所不一样，你可以对比一下再执行

   sudo ln -sf /lib/x86_64-linux-gnu/libnvidia-fbc.so.560.28.03 /lib/x86_64-linux-gnu/libnvidia-fbc.so.1
   sudo ln -sf /lib/x86_64-linux-gnu/libnvidia-opticalflow.so.560.28.03 /lib/x86_64-linux-gnu/libnvidia-opticalflow.so.1
   sudo ln -sf /lib/x86_64-linux-gnu/libGLESv1_CM_nvidia.so.560.28.03 /lib/x86_64-linux-gnu/libGLESv1_CM_nvidia.so.1
   sudo ln -sf /lib/x86_64-linux-gnu/libnvidia-egl-gbm.so.560.28.03 /lib/x86_64-linux-gnu/libnvidia-egl-gbm.so.1
   sudo ln -sf /lib/x86_64-linux-gnu/libnvidia-nvvm.so.560.28.03 /lib/x86_64-linux-gnu/libnvidia-nvvm.so.4
   sudo ln -sf /lib/x86_64-linux-gnu/libnvidia-ngx.so.560.28.03 /lib/x86_64-linux-gnu/libnvidia-ngx.so.1
   sudo ln -sf /lib/x86_64-linux-gnu/libOpenCL.so.560.28.03 /lib/x86_64-linux-gnu/libOpenCL.so.1
   sudo ln -sf /lib/x86_64-linux-gnu/libnvidia-cfg.so.560.28.03 /lib/x86_64-linux-gnu/libnvidia-cfg.so.1
   sudo ln -sf /lib/x86_64-linux-gnu/libnvcuvid.so.560.28.03 /lib/x86_64-linux-gnu/libnvcuvid.so.1
   sudo ln -sf /lib/x86_64-linux-gnu/libnvidia-vksc-core.so.560.28.03 /lib/x86_64-linux-gnu/libnvidia-vksc-core.so.1
   sudo ln -sf /lib/x86_64-linux-gnu/libcudadebugger.so.560.28.03 /lib/x86_64-linux-gnu/libcudadebugger.so.1
   sudo ln -sf /lib/x86_64-linux-gnu/libnvidia-egl-wayland.so.560.28.03 /lib/x86_64-linux-gnu/libnvidia-egl-wayland.so.1
   sudo ln -sf /lib/x86_64-linux-gnu/libnvidia-allocator.so.560.28.03 /lib/x86_64-linux-gnu/libnvidia-allocator.so.1
   sudo ln -sf /lib/x86_64-linux-gnu/libnvidia-ptxjitcompiler.so.560.28.03 /lib/x86_64-linux-gnu/libnvidia-ptxjitcompiler.so.1
   sudo ln -sf /lib/x86_64-linux-gnu/libGLESv2_nvidia.so.560.28.03 /lib/x86_64-linux-gnu/libGLESv2_nvidia.so.2
   sudo ln -sf /lib/x86_64-linux-gnu/libnvidia-encode.so.560.28.03 /lib/x86_64-linux-gnu/libnvidia-encode.so.1
   sudo ln -sf /lib/x86_64-linux-gnu/libnvoptix.so.560.28.03 /lib/x86_64-linux-gnu/libnvoptix.so.1

4. 针对open failed: /usr/lib/x86_64-linux-gnu/libvdpau_nvidia.so: permission denied这种错误，执行

   sudo chown root:root /usr/lib/x86\_64-linux-gnu/libvdpau\_nvidia.so

   然后再次执行

   docker run --rm --runtime=nvidia --gpus all ubuntu nvidia-smi

   直到不再出现错误;有其他问题，下面留言，我能解决的，就帮忙解决

H357753

ldconfig: /usr/local/lib/libzmq.so.5 is not a symbolic link


ldconfig: /lib/x86_64-linux-gnu/libnvidia-encode.so.1 is not a symbolic link


ldconfig: /lib/x86_64-linux-gnu/libnvidia-cfg.so.1 is not a symbolic link


ldconfig: /lib/x86_64-linux-gnu/libGLESv1_CM_nvidia.so.1 is not a symbolic link


ldconfig: /lib/x86_64-linux-gnu/libnvidia-egl-gbm.so.1 is not a symbolic link


ldconfig: /lib/x86_64-linux-gnu/libnvcuvid.so.1 is not a symbolic link


ldconfig: /lib/x86_64-linux-gnu/libnvidia-egl-wayland.so.1 is not a symbolic link


ldconfig: /lib/x86_64-linux-gnu/libcuda.so.1 is not a symbolic link


ldconfig: /lib/x86_64-linux-gnu/libnvidia-opencl.so.1 is not a symbolic link


ldconfig: /lib/x86_64-linux-gnu/libnvidia-vksc-core.so.1 is not a symbolic link


ldconfig: /lib/x86_64-linux-gnu/libnvoptix.so.1 is not a symbolic link


ldconfig: /lib/x86_64-linux-gnu/libcudadebugger.so.1 is not a symbolic link


ldconfig: /lib/x86_64-linux-gnu/libnvidia-opticalflow.so.1 is not a symbolic link


ldconfig: /lib/x86_64-linux-gnu/libEGL_nvidia.so.0 is not a symbolic link


ldconfig: /lib/x86_64-linux-gnu/libnvidia-ptxjitcompiler.so.1 is not a symbolic link


ldconfig: /lib/x86_64-linux-gnu/libGLX_nvidia.so.0 is not a symbolic link


ldconfig: /lib/x86_64-linux-gnu/libnvidia-allocator.so.1 is not a symbolic link


ldconfig: /lib/x86_64-linux-gnu/libnvidia-fbc.so.1 is not a symbolic link


ldconfig: /lib/x86_64-linux-gnu/libnvidia-nvvm.so.4 is not a symbolic link


ldconfig: /lib/x86_64-linux-gnu/libnvidia-ml.so.1 is not a symbolic link


ldconfig: /lib/x86_64-linux-gnu/libnvidia-ngx.so.1 is not a symbolic link


ldconfig: /lib/x86_64-linux-gnu/libOpenCL.so.1 is not a symbolic link


ldconfig: /lib/x86_64-linux-gnu/libGLESv2_nvidia.so.2 is not a symbolic link


执行命令后还是没用, 卸载nvidia-container-toolkit提示ldconfig: /usr/local/lib/libzmq.so.5 is not a symbolic link 这个库应该链接哪一个啊,大佬

警卫一点

大佬帮忙看看  我的按照教程 设置 没有效果  我的系统是全新安装的


ldconfig: /usr/local/lib/libzmq.so.5 is not a symbolic link

ldconfig: /lib/x86_64-linux-gnu/libnvidia-ml.so.1 is not a symbolic link

ldconfig: /lib/x86_64-linux-gnu/libnvidia-nvvm.so.4 is not a symbolic link

ldconfig: /lib/x86_64-linux-gnu/libnvidia-egl-gbm.so.1 is not a symbolic link

ldconfig: /lib/x86_64-linux-gnu/libOpenCL.so.1 is not a symbolic link

ldconfig: /lib/x86_64-linux-gnu/libEGL_nvidia.so.0 is not a symbolic link

ldconfig: /lib/x86_64-linux-gnu/libnvidia-opencl.so.1 is not a symbolic link

ldconfig: /lib/x86_64-linux-gnu/libcudadebugger.so.1 is not a symbolic link

ldconfig: /lib/x86_64-linux-gnu/libnvidia-egl-wayland.so.1 is not a symbolic link

ldconfig: /lib/x86_64-linux-gnu/libnvidia-allocator.so.1 is not a symbolic link

ldconfig: /lib/x86_64-linux-gnu/libnvidia-cfg.so.1 is not a symbolic link

ldconfig: /lib/x86_64-linux-gnu/libGLX_nvidia.so.0 is not a symbolic link

ldconfig: /lib/x86_64-linux-gnu/libnvoptix.so.1 is not a symbolic link

ldconfig: /lib/x86_64-linux-gnu/libGLESv1_CM_nvidia.so.1 is not a symbolic link

ldconfig: /lib/x86_64-linux-gnu/libnvcuvid.so.1 is not a symbolic link

ldconfig: /lib/x86_64-linux-gnu/libcuda.so.1 is not a symbolic link

ldconfig: /lib/x86_64-linux-gnu/libnvidia-opticalflow.so.1 is not a symbolic link

ldconfig: /lib/x86_64-linux-gnu/libnvidia-fbc.so.1 is not a symbolic link

ldconfig: /lib/x86_64-linux-gnu/libnvidia-ptxjitcompiler.so.1 is not a symbolic link

ldconfig: /lib/x86_64-linux-gnu/libnvidia-ngx.so.1 is not a symbolic link

ldconfig: /lib/x86_64-linux-gnu/libnvidia-encode.so.1 is not a symbolic link

ldconfig: /lib/x86_64-linux-gnu/libGLESv2_nvidia.so.2 is not a symbolic link

ldconfig: /lib/x86_64-linux-gnu/libnvidia-vksc-core.so.1 is not a symbolic link

酷玩x

root@fnOS:/# ls -lh /usr/lib/x86_64-linux-gnu/libnvidia-ml.so*
lrwxrwxrwx 1 root  root    51 Mar 25 10:33 /usr/lib/x86_64-linux-gnu/libnvidia-ml.so -> /usr/lib/x86_64-linux-gnu/libnvidia-ml.so.560.28.03
lrwxrwxrwx 1 root  root    51 Mar 25 10:33 /usr/lib/x86_64-linux-gnu/libnvidia-ml.so.1 -> /usr/lib/x86_64-linux-gnu/libnvidia-ml.so.560.28.03
-rw-rw---- 1 admin Users 2.1M Mar  4 23:51 /usr/lib/x86_64-linux-gnu/libnvidia-ml.so.1.bak
-rw-rw---- 1 admin Users 2.1M Mar  4 23:51 /usr/lib/x86_64-linux-gnu/libnvidia-ml.so.560.28.03
-rw-rw---- 1 admin Users 2.1M Mar  4 23:51 /usr/lib/x86_64-linux-gnu/libnvidia-ml.so.bak
root@fnOS:/# ls -lh /usr/lib/x86_64-linux-gnu/libcuda.so*
lrwxrwxrwx 1 root  root   46 Mar 25 10:33 /usr/lib/x86_64-linux-gnu/libcuda.so -> /usr/lib/x86_64-linux-gnu/libcuda.so.560.28.03
lrwxrwxrwx 1 root  root   46 Mar 25 10:33 /usr/lib/x86_64-linux-gnu/libcuda.so.1 -> /usr/lib/x86_64-linux-gnu/libcuda.so.560.28.03
-rw-rw---- 1 admin Users 34M Mar  4 23:51 /usr/lib/x86_64-linux-gnu/libcuda.so.1.bak
-rw-rw---- 1 admin Users 34M Mar  4 23:51 /usr/lib/x86_64-linux-gnu/libcuda.so.560.28.03
-rw-rw---- 1 admin Users 34M Mar  4 23:51 /usr/lib/x86_64-linux-gnu/libcuda.so.bak
root@fnOS:/# docker run --rm --runtime=nvidia --gpus all ubuntu nvidia-smi
docker: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error running prestart hook #0: exit status 1, stdout: , stderr: Auto-detected mode as 'legacy'
nvidia-container-cli: initialization error: load library failed: libnvidia-ml.so.1: cannot open shared object file: no such file or directory: unknown
卸载驱动，删除了nvidia的文件夹，重装驱动，还是报错。

酷玩x

看了github上nvidia-container-toolkit的相同的issue，感觉就是飞牛这个驱动有问题，或者是docker本身有问题。

警卫一点

那位大佬帮忙研究下

老谈仙森

H357753 发表于 2025-3-22 12:29
ldconfig: /usr/local/lib/libzmq.so.5 is not a symbolic link

ldconfig: /usr/local/lib/libzmq.so.5
这个链接库不需要关注，还有，你指的命令执行后没有用是执行的哪个命令？

老谈仙森

酷玩x 发表于 2025-3-25 11:15
看了github上nvidia-container-toolkit的相同的issue，感觉就是飞牛这个驱动有问题，或者是docker本身有问 ...

不应该是nvidia-container的问题，我在生产环境上运行了很多这样的容器，似乎还没有遇到，一般应该都是系统本身的问题

老谈仙森

警卫一点 发表于 2025-3-24 14:22
大佬帮忙看看  我的按照教程 设置 没有效果  我的系统是全新安装的

执行了还是这样？你先不要复制我的命令。单独执行某一条，然后再查看一下，看看错误列表中是否已经少了这条错误。

老谈仙森

酷玩x 发表于 2025-3-25 10:36
root@fnOS:/# ls -lh /usr/lib/x86_64-linux-gnu/libnvidia-ml.so*
lrwxrwxrwx 1 root  root    51 Mar 25 ...

你现在连接了/usr/lib/x86_64-linux-gnu目录下的链接库文件，你需要把/lib/x86_64-linux-gnu/下的也链接一遍

老谈仙森

警卫一点 发表于 2025-3-25 13:16
那位大佬帮忙研究下

你的错误和我遇到的一样么？

H357753

大佬帮忙看看呢

老谈仙森

<br /> <br />

<p>我传不了图片了，，你分别检查一下/usr/lib/x86_64-linux-gnu以及/lib/x86_64-linux-gnu这两个目录，尽量都保证nvidia相关的so.1文件都链接上</p>

老谈仙森

sudo chown -R root:root /usr/lib/x86_64-linux-gnu
修复权限试试