Notice: This page will be redirected to the Epic Developer Community. Spend a few minutes to get familiar with the new community-led support today.

CUDA error : 2 : out of Memory

Answered

Comments

15 comments

  • Avatar
    Carmel

    got same problem today   454 pictures  only  imported form video  and when trying to generate  high model i 'm getting this error message     

    and i'm using    gtx 970 + gtx 660ti in my pc    with i7-3820  and 64 gb ram 

  • Avatar
    Erik Kubiňan CR

    Hello dear users

    We we recommend to only use 1 GPU while working with RealityCapture currently. Try to do so and report back if this helped you.

     

  • Avatar
    Sek19howell

    I have tried with one card and that did not change the result. Also want to be clear, do you mean take the graphics card out of the machine or make sure the settings are set to only use one? I have changed the settings and got that same result.

  • Avatar
    Sek19howell

    I've run a few more tests and i'm now getting a crash when running normal reconstruction. it jumps between this and the CUDA error

  • Avatar
    IKTNIKU

    Got the same problem today after updating to a version fixing an issue with 2D view of laserscan pointclouds. No difference using just one or both GPUs. 

    2x GTX 1080

    3000 images and about 100 laserscans

  • Avatar
    Carmel

    Exactly there is no difference if you will use one or two GPU units and support only advice o use one graphic card. Besides all my scans was done with high detail on two GPU units until installation of new version of reality capture :( so definitely it's a software issue in new version

  • Avatar
    Sek19howell

    @Erik Kubiňan CR any word on how we should be proceeding?

  • Avatar
    Tomáš

    Hello,

    This is our official statement for this issue.

     

    Two immediate temporary solutions:

    1. Physically remove all GPU cards, except one - single card configuration.

    or

    2. Install old GPU driver. The version 419.67 is working for us.

    Reason:

    The issue is with the CUDA memory de-allocation function, that has stopped working properly with latest NVIDIA GPU drivers.

    More specifically the function CUDAFreeHost() resulted with success code, but the memory was not de-allocated and therefore after some time, the GPU pinned memory was filled up and the SW ended up with the message "CUDA error : 2 : Out of memory".

    It seems that this bug causes problems also to another CUDA developers:

    https://devtalk.nvidia.com/…/cudafreehost-not-clearing-all…/

    We have contacted NVIDIA and we hope that there will be a new driver with fix very soon.

    Meanwhile, we are also trying to fix the problem by some workaround in our code and we will try to release hotfix ASAP.

  • Avatar
    Sek19howell

    thank you, Going to the 419. 67 version of Nvidia drivers worked for me I guess I wasn't going back far enough in the archive.

  • Avatar
    Erik Kubiňan CR

    That's great news Mr. Keutzkamp. Good luck in your projects and enjoy RC.

  • Avatar
    jblessing

    Is 419.67 still the recommended driver version for the latest release of RC?

  • Avatar
    Erik Kubiňan CR

    With the latest update, you should be able to run with the newest drivers. If you anyways experience any of such issues try rolling back to the 419.67s, I do believe you will not have to though.

  • Avatar
    Vratislav Zika 1

    Hi, I have this CUDA error popping up once in a while without any clear logic - is sometimes appears, sometimes not -no matter if the project has 40 or 900 images - but when it does, it is usually during the texturing process and it keeps reappearing repeatedly even after canceling the error window many times - so as the result the texturing process could not be finished...

    I have tried the newest Nvidia drivers, and the older one recommended above (419.67) but it still keeps appearing, can you help me solve this, please???

    PS I do not have SLI - just one older GeForce GTX 750 card...

  • Avatar
    Erik Kubiňan CR

    Dear user,
    this error is not RC related and aims directly towards Nvidia as your GPU, it probably has some issues with memory leaking or use. Sorry to see that, but I don't think I can help you here. 

  • Avatar
    Simutech21

    Sorry for posting in this old thread, I just came across the same memory problem and didn't want to open a new one just to post what helped me fix it:

    I've got a GTX 1050Ti and a GT 710 that I only use for extra monitors. I've reconstructed parts of the same mesh multiple times before but when I started the full reconstruction in high detail I got the same "low device ... memory" error message. It looks like RC ran out of memory because it was using the 710 instead of the 1050 (which should be the main one).

    You can choose the GPU RC uses in:

    Reconstruction - Process / Settings - Image depth map calculation / GPUs to use

    I set the 710 to "False" and it looks like it's working fine now. :)

Please sign in to leave a comment.