Cross post from stackoverflow: https://stackoverflow.com/questions/54373156/using-the-gpu-backend-in-h2o-xgboost-in-a-rocker-based-docker-container
I have installed CUDA + nvidia-docker on the host machine and CUDA (9.0 - 9.2) in the container. I’m running the container with the following,
nvidia-docker run -d -p 8787:8787 -e USER=tidyverse-gpu -e PASSWORD=tidyverse-gpu --name tidyverse-gpu seabbs/tidyverse-gpu
Base Xgboost works with GPU support in both R and Python (and
nvidia-smi returns usage stats etc when run inside the container). When the GPU backend is enabled in
h2o.xgboost the following error is returned.
Illegal argument(s) for XGBoost model: XGBoost_model_R_1548450637489_3. Details: ERRR on field: _backend: GPU backend (gpu_id: 0) is not functional. Check CUDA_PATH and/or GPU installation.
Initially I had not added the
CUDA_PATH in the Dockerfile but testing adding this has had no effect.
Sys.getenv("CUDA_PATH")  "/usr/local/cuda"
h2o startup logs show no issue with the
xgboost module (that I can see). I’ve tried rolling back to CUDA 8.0 but this errors in the latest rocker containers as the gcc version being used is not supported by
Any help would be much appreciated as I don’t have a clue