tricky mesh per frame performance scanning
Answered
Hi Guys,
I'm new to this forum so sorry if this has been discussed already - I tried to look at some tutorials but since it's a fairly specific issue some of you might have a solution more fitted to this.
I need to reconstruct an actors facial performance, I'd say around 35-40 minutes of footage split into shots. It's captured by an 8 camera rig, which is fixed, 4 camera pairs about 3 meters from the actor, not sure of the exact focal lenght ( I didn't get any exif data, only png's to make it more fun I guess ) The footage was provided by an outside vendor so I dont know exact dimensions and such.
Long story short, I can't change the cameras, I managed to do fairly decent reconstructions given the circumstances.
The question is, is there any way of inputing chunks of lets say 8*1000 photos and batch process them so it would output 1 mesh for every frame of the actors performance? Without it trying to match the photos of the next frame of course. I have proper naming/folder structure, like "camera_1_frame_0001.png etc"
I recreated a sample frame with a free scan rendered where the actor would be (intentionally crappy placement), maybe that makes it more clear -
https://drive.google.com/open?id=1nXY7-K_Ys-8LGjznZFrpE4a5zm_hF_-U
Any ideas are much appreciated.
Cheers,
Dan
-
Hello Dan,
our users usually use RealityCapture CLI version for this purpose (batch processing): www.capturingreality.com/Product#Buy,
you can try the CLI commands even in the Demo: www.capturingreality.com/Product#Download
-
Hi Lubenko,
so just to double check, I'd be able do batch process camera clusters in CLI (which is what we have at the company I believe)? Wasn't sure if there's a way of telling RC that it should align pairs of 8 images from a selection of frames. For example if I had a sequence for each of the 8 cameras in seperate folders, or named accordingly, would there be a way of aligning only the matching 8 frames from each sequence?
I've seen infinite realities doing some tests last year, also with RC, which turned out great (they were using a 16 camera setup though) -
http://ir-ltd.net/introducing-the-aeon-motion-scanning-system/
I believe it was frame by frame reconstruction though - so if there was a way to automate this for a given set of cameras, that'd be a huge plus.
Cheers,
Dan
-
Dear Dan,
you can process each frame as independent individual reconstruction from 8 images. The important think is to preserve coordinate system from frame to frame.Please read the application help for detailed information on how to preserve coordinate system from scan to scan :
"Export, Import and Re-use Computations:" -> "Re-use Alignment" -> "Full-body Scans and Coordinate System Preservation" -> ""In your case I assume that you will have fixed rig so you can use XMPs workflow. In the help just navigate to the "Approaches ( 1.a.i ) and ( 1.a.ii ) are described step by step in the video tutorial Coordinate system preservation with XMPs." ... there is step by step tutorial including sample data and scripts.
Finally you can also use new commands ( avalilable in next version that will be released in few days ) for selection and filtering of large triangles etc. -
Thanks Michal,
The XMP workflow sounds striaght forward - I'll make some tests and see where it takes me.
The tricky part where I'm having issues was automating processing to run them as individual frames, which in my case would be 72.000 frames (8 fixed cameras each) for the initial project.
The triangle filtering sounds super usefull, I had to do that externally before for large datasets based with additional masking regions.
Thanks again for your tips,
Dan
-
Hi Dan/Michal,
This is amazing I came across your post.
I am actually looking to do something very similar as you are - I have a 4 camera setup and looking to recreate facial expressions of pet animals.
From my research so far it seems that Reality Capture doesn't have native functionality to stitch the 3D models together into a stop motion video sequence.
How are you doing this on your side?
Is there a particular software you would recommend? -
Hi Hornlessu,
for presentation purposes I have a script that would set the visibility of the scans (after they have been batch processed with a general cleanup/filter pass) according to frame numbering in maya. As far as the general pipeline integration goes I use optical flow wrap to align and fit the production ready UV'd topology onto the scans, extract a mesh per frame which would be blendable, so depending on the general pipeline could be used as a series of in-betweens (handy if you want to tweak the animation or apply it onto an existing rig) or just as an alembic bake which can be shotsculpted, finetuned.
I have experience with stereo pairs, 4, 8 and 16 cam setups, and I'd go with at least an 8 cam setup especially for animals/furry things.
Cheers,
Dan
-
Hi Dan,
Many thanks, tons of great info in your response - you're clearly a master in the field.
I'd love to connect with you over email - my address is hornlessu@gmail.comPlease do reach out :)
Cheers
Please sign in to leave a comment.
Comments
7 comments