Workflow Process - Interior Image Alignment
Answered
Hi Everyone,
I thought I would put together some experience I had with trying to align photos of interior buildings. Most of the information is available in other posts but it seems to me like it needs to be pieced together a bit.
Keep in mind the projects I am working on are to bring real house interiors into VR. Valve has an amazing guide here but there were a few details that still need to be addressed.
First thing are pictures, for a room you should be able to get away with under 500 pictures but for full floors and houses, you will need around 1500-2000 per floor depending on the house. Now this is where RAM becomes a factor if you want to align them all at once. (And have a good amount of features for the calculations)
** You can run the alignment multiple times to refine the alignment, this is necessary for these types of projects **
For best results you want to use the following settings:
Image overlap = low (Wish mentions why in other posts, but the software will then use the whole image for alignment points)
Max Features per image = 80000 - 120000 (This is important if you are using low in image overlap but this has a big impact on RAM requirements, 32GB of RAM will be required if you are using 80k and have 2000 images, please use that as your calculation, if you have 500 images you should be fine with 120k and have 16GB RAM)
Detector sensitivity = High
Next dont try to match components first
Run the alignment.
This should give you the best result without re-matching anything yet.
If you find you have too many components you can now try "Merge Components Only", First without any control points and if you do not have success start adding control points to line up your components then retry the alignment with "Merge Components Only" again.
That should give you the best alignment with the images you have. If you still are not satisfied you may need to take more pictures.
Hope this helps anyone who is working on interior models!
Here is an example of two floors aligned: 
Next steps for me are running a high detail reconstruction and hopefully working on the workflow process for unwraping, texture, exporting, reconstructing in something like Modo/Maya/meshlab, reimporting a low-poly mesh, retexture and finalize a model for a game engine like unreal. If anyone has a good process here please let me know. Thx!
-
Yes, the model is very simple. It could be made more detailed by adding the white trim boards, etc. There are 10 cameras in the scene. They were all used to create the texture. I also tested texturing the model with Maximal textures count, one texture at 4096x4096 and it looked good.
Here is an image of the low poly model with the RC high poly reference mesh.
Götz, if you download the model and look at the texture of the second shorter board leaning against the shed you can see that the texture appears distorted since I didn't create geometry for that board. Likewise the areas of ivy that are furthest from the low poly model look a little odd, but not bad.
These were photos I had taken to test a camera, but never processed in RC until I put together this example. The whole process of making the high poly in RC to making the low poly model in Blender, and texturing the model in RC was in the neighborhood of a couple of hours (with some fiddling).
Here's a link to the RC project complete with photos, simplified mesh, and low poly textured mesh. File size around 150 MB.
http://www.digital-mapping.net/forums/CapturingReality/2018/RC_high-low_poly.zip
In the 3D view the low poly is best viewed as solid or sweet.
-
Hi Bill, Thanks for the replies.
Really nice work on the quality of the texture.
My problem is ensuring RC or whichever 3d tool I am using utilizes the UV and material textures. Here is an example of what RC exported and my confusion on how I optimize the UV map.
This is what RC created for UV maps and the white UV would be my low poly UV map I created in Modo. What I want to do is utilize 5 texture maps. RC created 4 as you can see on the right and 1 more for my new LowPoly UV.Utilizing each 8k texture is key to ensure I am getting the best quality texture for my model and if I have 5 8k textures I want to use them all up. But why isnt RC exporting them properly? or creating them properly?
Also if I want to reimport new UV's using Modo or any other tool how to I create a proper UV map using 5 UV sections to optimize 5 8k textures. (If that makes sense).
Thanks!
-
Hi Mike,
With the majority of my projects I use targets with Ground Control Points so that the model is in world dimensions. Since I know the actual dimensions of the object, I know about what dimension I want a pixel to be. Lets say my units are meters and I want a pixel dimension of about 1 millimeter. I will either use the Unwrap style Fixed texel size with the Texel size set to 0.001 and Unwrap, or I will use Unwrap style Maximal texture count and change the Maximal textures count and unwrap until the Texel size displayed in the Selected model(s) dialog is about 0.001. Often times the Optimal texel size will be a lot smaller than 1 millimeter and would be overkill.
The Unrwrap Style Maximal texture count is the one I use most often when exporting simplified meshes. Here's an image showing the dialogs for Maximal texture count and the Selected model(s) for one of my projects. Hopefully it will make the previous discussion clearer.
When you export your model what Tile type are you using? I believe modo supports UDIM, you might want to try that if you haven't already.
Another thing I do in my workflow is build the initial model in Normal Detail without a Reconstruction Region. I then use Set Reconstruction Region and use the Filtering Tool to remove the unwanted outlying area. This removes large polygons (and the closed box). I then use the Simplify and Unwrap tools on the resulting model.
-
Thanks Bill that was extremely helpful, I feel like im super close to getting this working but my current issue now is after I create a new UV and material. When I try to reimport I get the error "texture coordinate u or v is bigger than 1"
Is there something I need to identify before I reimport it back to RC?Thank you so much!
Mike
-
Hi Mike,
I have modo version 9 and played around a bit with importing multiple textures from modo into RC. As far as I can tell the best way to do this is to use the 0 to 1 UV space. The way to have multiple textures utilize this space is to use multiple materials, each with a texture. I have attached a modified model of my shed front that I made six blank textures for in Photoshop. In modo I created six materials and assigned one of the textures to each of the materials. I then selected different elements of the shed and assigned different materials to them. When I imported the model into RC all six textures were there.
Here is an image showing the six textures I made in Photoshop.
Here is an image showing the six textures exported from RC after texturing.
.This image indicates the Tile type that exports multiple textures that utilize the 0 to 1 UV space.
Here is a link to the .obj file I modified in modo along with the 6 textures. It also includes a folder named test that contains the model and textures exported from RC after it was textured. If you download the RC project from one of my previous post, you can import modo.obj and texture it to see how it works.
http://www.digital-mapping.net/forums/CapturingReality/2018/modo.zip
Here's a link to the model on Sketchfab: https://skfb.ly/6wyxV
You can use the Model Inspector on Sketchfab to checkout the wireframe, textures, and UV's.
-
Hi Mike,
I played around some more with using UDIM's with RC and got it to work. I renamed my materials and textures to the UDIM naming convention.
Then I moved my 6 UV's form 0 to 1 UV space to UDIM's.
I imported it in to RC, textured, and exported the results. It worked as hoped.
I tried combining all the textures into one material and importing into RC, but got the message "A texture coordinate u or v is bigger than 1", so it appears each UDIM has to have it's own material in order to import into RC.
Here's a link to the modo UDIM .obj:
http://www.digital-mapping.net/forums/CapturingReality/2018/modo_UDIM.zip
-
Hi Bill,
Appreciate all the help dealing with these pesky UV's, I think I have the handle on assigning UV's to a individual mesh.
Im only running into some minor issues now when I have a lowpoly mesh with UV's and a highpoly mesh with UV's. Selecting both layers and importing back into RC doesnt align properly and where I should have 6 UV's it might create 12. I may have to merge both meshed first before I export back for RC.
I played around a lot this weekend exporting and importing different variations of UV's and how I can import them back. The problem now is dealing with 2 different meshes on export but that should be a lot more straight forward now.
Thanks again for all your help trying to understand things better.
-
Hi Bill,
Just wanted to thank you for the help, its really progressed my knowledge in this space on how to handle working in 3rd party modeling software.
Understanding UV's and how to utilize them for low poly models has been extremely useful.
Right now I am focusing on remodeling every flat surface in my interior model and leaving the rounded contours to the RC high poly mesh. Its a slow process but the results are very satisfying.
I should note for anyone trying this, lining up your new low poly model with the construction from RC is very important in order to get clean textures. There probably will be a lot of playing around for the final model to get the best texture quality but now that I can create my own UV's, I can really get out the detail.
Here is an example of the low poly walls and floors (In currently 1 8k texture), note the distortion was due to the missing high poly meshes of the furniture. The next plan is to blend the two models to really clean things up.
-
Hi Mike,
I bet getting the contours right for RC is quite a job!
Have you considered to use a different software where you can select a camera by yourself and paint the texture wherever you want? That might ultimately be quicker because you don't get the blurriness that RC creates due to false geometry...
-
Haha =P
I believe Modo should be able to do that as well, but texture cleanup is the very last step here.
Here is how Modo can clean it up.
-
You might want to check this series of modo videos on vimeo:
I think Projection Shaders and Projection Ink are the most relevant, but the others are probably worth a look as well.
-
Hey Mike,
Yes, Modo is also possible I believe, but I thought you might prefer less pricy options...
So yo do that stuff in Modo already? Seems quite practical to do texture and geometry at the same time.
Why would you re-import it in RC though? I thought the whole point of the Modo workflow is to get a clean deometry with manually selected texture projection?
-
Hi everyone.
I'm using a very similar workflow to the one shared by ValerioPaolux but I'm struggling with a couple of issues that seem very specific to dealing with UDIMs/multi tile uv's:
a) hole filling and unwraping of new geometry
Tried with Re**p Photo, which does it automatically but unfortunately keeps crashing (with 18 loaded textures at least). Mudb*x barely manages it with a few crashes now and then. Also tried Blender but multi tile support is half baked and it becomes extremely slow.
b) texture clean-ups
For 3D painting across tiles I guess we only have 3 options: Mari, Mudbox and 3D-Coat. 3D-Coat doesn't seem to handle these large datasets, as it loads them into RAM and even with 128 Gb it's not workable. Mudbox struggles a lot once you load more than a couple of tiles and uses all available VRAM (I have 2x1080Ti). I'm trying to avoid Mari because of it's price/uselfuness ratio.
Has anyone found a better way to do these things?
Please sign in to leave a comment.
Comments
47 comments