CUDA error : 2 : out of Memory

Answered

Comments

14 comments

  • Avatar
    Carmel

    got same problem today   454 pictures  only  imported form video  and when trying to generate  high model i 'm getting this error message     

    and i'm using    gtx 970 + gtx 660ti in my pc    with i7-3820  and 64 gb ram 

    0
    Comment actions Permalink
  • Avatar
    Erik Kubiňan CR

    Hello dear users

    We we recommend to only use 1 GPU while working with RealityCapture currently. Try to do so and report back if this helped you.

     

    0
    Comment actions Permalink
  • Avatar
    Scott Kreutzkamp

    I have tried with one card and that did not change the result. Also want to be clear, do you mean take the graphics card out of the machine or make sure the settings are set to only use one? I have changed the settings and got that same result.

    0
    Comment actions Permalink
  • Avatar
    Scott Kreutzkamp

    I've run a few more tests and i'm now getting a crash when running normal reconstruction. it jumps between this and the CUDA error

    0
    Comment actions Permalink
  • Avatar
    ikt

    Got the same problem today after updating to a version fixing an issue with 2D view of laserscan pointclouds. No difference using just one or both GPUs. 

    2x GTX 1080

    3000 images and about 100 laserscans

    1
    Comment actions Permalink
  • Avatar
    Carmel

    Exactly there is no difference if you will use one or two GPU units and support only advice o use one graphic card. Besides all my scans was done with high detail on two GPU units until installation of new version of reality capture :( so definitely it's a software issue in new version

    0
    Comment actions Permalink
  • Avatar
    Scott Kreutzkamp

    @Erik Kubiňan CR any word on how we should be proceeding?

    0
    Comment actions Permalink
  • Avatar
    Tomáš

    Hello,

    This is our official statement for this issue.

     

    Two immediate temporary solutions:

    1. Physically remove all GPU cards, except one - single card configuration.

    or

    2. Install old GPU driver. The version 419.67 is working for us.

    Reason:

    The issue is with the CUDA memory de-allocation function, that has stopped working properly with latest NVIDIA GPU drivers.

    More specifically the function CUDAFreeHost() resulted with success code, but the memory was not de-allocated and therefore after some time, the GPU pinned memory was filled up and the SW ended up with the message "CUDA error : 2 : Out of memory".

    It seems that this bug causes problems also to another CUDA developers:

    https://devtalk.nvidia.com/…/cudafreehost-not-clearing-all…/

    We have contacted NVIDIA and we hope that there will be a new driver with fix very soon.

    Meanwhile, we are also trying to fix the problem by some workaround in our code and we will try to release hotfix ASAP.

    0
    Comment actions Permalink
  • Avatar
    Scott Kreutzkamp

    thank you, Going to the 419. 67 version of Nvidia drivers worked for me I guess I wasn't going back far enough in the archive.

    0
    Comment actions Permalink
  • Avatar
    Erik Kubiňan CR

    That's great news Mr. Keutzkamp. Good luck in your projects and enjoy RC.

    0
    Comment actions Permalink
  • Avatar
    jblessing

    Is 419.67 still the recommended driver version for the latest release of RC?

    0
    Comment actions Permalink
  • Avatar
    Erik Kubiňan CR

    With the latest update, you should be able to run with the newest drivers. If you anyways experience any of such issues try rolling back to the 419.67s, I do believe you will not have to though.

    0
    Comment actions Permalink
  • Avatar
    Vratislav Zika

    Hi, I have this CUDA error popping up once in a while without any clear logic - is sometimes appears, sometimes not -no matter if the project has 40 or 900 images - but when it does, it is usually during the texturing process and it keeps reappearing repeatedly even after canceling the error window many times - so as the result the texturing process could not be finished...

    I have tried the newest Nvidia drivers, and the older one recommended above (419.67) but it still keeps appearing, can you help me solve this, please???

    PS I do not have SLI - just one older GeForce GTX 750 card...

    0
    Comment actions Permalink
  • Avatar
    Erik Kubiňan CR

    Dear user,
    this error is not RC related and aims directly towards Nvidia as your GPU, it probably has some issues with memory leaking or use. Sorry to see that, but I don't think I can help you here. 

    0
    Comment actions Permalink

Please sign in to leave a comment.