Amazon EC2 GPU instances for RC processing
I have recently tried using a G3.16xlarge instance in Amazon AWS EC2 to process a job (4x Tesla M60 GPU's, 64vCPU and 488GB RAM) and it was no quicker that a 3x GPU card workstation taking approximately 20 hours. Interestingly in the cloud watch monitoring tool I could see it maxing the CPU usage. It's a shame there's no simply way of monitoring the GPU performance. I find it hard to believe that this isn't any quicker than a local workstation.
Has anyone else seen similar results, or perhaps some suggestions to optimise the GPU's?
"It's a shame there's no simply way of monitoring the GPU performance" (wot no quote facility in the new forum?)
I think he meant in the cloud...
Aha - now I know what 'G3.16xlarge instance in Amazon AWS EC2' means!
http://www.worldcadaccess.com/blog/2017/08/after-cloud-computing-comes-edge-computing.html is interesting -
"Cloud computing is soon going to take a back seat to edge computing, and we will very quickly see the majority of processing taking place at the device level."
Did you try installing GPU Shark or HWInfo on the cloud service?
rc only uses 3 x gpu, and 32 x threads. with one license
you will need to arrange with them, if you want it to work with more.
its also expected that you wouldn't get the same performance over aws that the same hardware would locally. there is a bit of a performance hit with how their setup works.
So what you are saying is that it will only give an advantage if the setup in the cloud service is at least 50% better than your own setup? Considering the additional costs of the service and for many people also upping their internet connection, I would say it should be more like 100% better. Then again, it can free processing time on your own system - one could set up new projects while another one is processing somewhere else. That's also an advantage...
well it might be useful if you need lots of ram.
but cpu power and gpu power not so much. since you can't take full advantage of the more powerful instances with lots of cpu cores and lots of gpu's.
also if your using anything other than steam version. then you'll need to wait a while until you get your license back.
then you need to worry about internet speeds... which in Australia are pretty bad when trying to deal with something like.
Ah, I forgot about RAM. That could make a difference indeed. Although I think mostly for gigantic models and then only the simplification step. Or is there another memory intense process? I vaguely remember some limitation of total images, but it's way beyond what my setup could only imagine to handle... The display issue is marginal in my view (pun intended).
Hmm, that could be a point in favor of steam. How does that work then, you would log in via the instance on the cloud service?
Other than that, one could just invest in another license. A PROMO can handle a lot in spite of the limitations.
Well, the internet connection can be bad pretty much anywhere. I also would need to upgrade seriously from my breath-taking 10 mbit plan. lol
Is it possible to use two promo licenses on steam to use more than 32 cores and 3GPU? I'm doubtful.
I am trying to figure out a way to have a cloud connected storage that will allow me to do the alignment and manual control points locally and then move the project to the cloud for the main rendering.
When it's done, I move the project back to intermediate storage, shut the expensive processing VM and pull the data back home for filtering and post work.
Thats the idea anyway :)
You mean you would keep all your images on the cloud right away so you don't have to upload them again when you use the cloud service? I would be interested to hear how it worked out. I personally would be worried that uploading 10s to 100s of GBs of images would take way too long to make it worthwhile.
You must activate the GRID license...
Please sign in to leave a comment.