Control Point Auto Alignment

Completed

Comments

11 comments

  • Avatar
    Tom Foster

    An amazing discovery! Few people would have suspected, or had the patience for such pixel-scale investigation.

    0
    Comment actions Permalink
  • Avatar
    Götz Echtenacher

    Hi Alice,

    as far as I can tell there is no solution for your request! You are basically criticizing the original concept of CPs.

    The sole reason for placing CPs is to help RC along with an alignment. You only need to do that if RC is wrong in placing its own tie points (a CP is just a manual tie point). So if you place the CPs in a way that all the errors are close to 0px, then you are just confirming RC and either there was no problem or you are reinforcing a misalignment. If the errors are big and you are certain you placed your CPs right, then you should be glad because you found a problem spot!

    You need to run another alignment and the errors should go down. If they don't, you can delete all components and start from scratch. If there are still errors, then you need to place more CPs because the misalignment has just moved to the next area. It's a time trap that you can only significantly speed up by taking better images (if it is possible).

    1
    Comment actions Permalink
  • Avatar
    Tom Foster

    Interesting - manually created Control Points (CPs) 'a few pixels off' sounds like RC should be able to take the assistance offered, however imperfect.

    But come to think, if RC gives those rather approximate CPs the same status as its automatically generated Tie Points (TPs), that's giving authority to some wildly error-full points, by TP standards. Especially if those CPs are given a high weighting number.

    When Mean and Median reprojection error are typically in the 0.3 to 0.5 pixel range, and Max Reprojection error is set to 1 or 2 pixels, then CPs 'a few pixels off' are seriously error-full. Looks like CPs need to be zoomed-in enough to be placed with exact 1-pixel accuracy - and even that is way over the 0.3 to 0.5 pixel range.

    Furthermore, if those CPs (if they're in fact not really 'points' but artificial 2D Features) are projected (just as normal automatically detected 2D Features) to 3D near-intersection to create an artificial TP, which is then REprojected to the 2D picture planes where REprojection error is measured ...

    ... maybe a geometric multiplier something like 1/sin30 is applied, so the REprojected error resulting from 'a few pixels' CP-placement error is actually twice (or more) greater.

    This all would make reliance on CPs into a dangerous process, inherently injecting lots of error (rather than, you'd have thought, confident human-visual accuracy) into the system. Especially if there's lots of CPs and/or they're given high weighting.

    Maybe CPs just can't be placed with anything like high enough accuracy, compared to most TPs.

    0
    Comment actions Permalink
  • Avatar
    Götz Echtenacher

    There are two aspects to this:

    1. To lower the errors, one needs to run another alignment.

    2. RC is not perfect. It uses mathematical propabilities (just a layman's expression) to fit all infos into a (distortion-) model that produces the least errors according to a certain definition. This is very often quite correct (especially with good image sets) but can also go terribly wrong (especially with weak image sets). As I stated elswhere today, RC does not know what's on the images and how the 3D model should look like. It can solely rely on it's maths.

    Also, I am pretty certain it is correct to adress CPs as manual tie points. They exist as well in 2D (on the images) as calculated from that in 3D. If you look closely at a troublesome CP with high errors, you might notice that it is not on the surface indicated by the sparse pointcloud but rather above or below. That again illustrates the difference between RC's maths and the users input, which has the advantage of having advances content recognition...  ;-)

    0
    Comment actions Permalink
  • Avatar
    Tom Foster
     
    Götz Echtenacher said: "The sole reason for placing CPs is to help RC along with an alignment. You only need to do that if RC is wrong in placing its own tie points"
     
    First sentence true - but the second?
     
    Isn't it more often 'You only need to do that if RC can't place certain tie point(s) at all'? meaning it doesn't recognise that something that exists in one pic is the same something that exists in another pic. When our eyes/minds can tell that, but RCs algorithms can't.
     
    And I'm suggesting that the 1 pixel CP placement resolution which is the best that we can achieve, is certain to be very error-full compared to the way-sub-1px resolution with which RC locates its own Features on a 2D pic.
     
    So it's an error-introducing way, both of
    filling in 'sameness' info that RC's algorithms can't, and
    putting right 2D Feature locations (hence 3D Tie point placements) that RC's got wrong.
     
    It's gd to know, as everyone keeps saying, that the real answer is to get better photo sets!
    0
    Comment actions Permalink
  • Avatar
    Tom Foster

    Götz Echtenacher said: " I am pretty certain it is correct to adress CPs as manual tie points. They exist as well in 2D (on the images) as calculated from that in 3D."

    Do you mean that CPs are treated

    not as artificial Features - which as usual are projected (the 'beamer, you called it) back to near-intersection in 3D space (the 3D Tie point) then REprojected forward again to the two (or more) 2D picture planes (as 2D REprojected features?)

    but as the 2D REprojected features themselves - from which an artificial Tie point is reverse-engineered?

     

    0
    Comment actions Permalink
  • Avatar
    Götz Echtenacher

    Whoow, Tom, I am not sure I can still follow you!  :-D

    In my universe (which is partly fictional), 2D tie points are the ones you mark on an image. 3D tie points are the ones calculated by the projections. Does that help anything?

    I do think that they are artificial tie points that can help convince RC that certain images go together. I wouldn't use the term "features" because a feature, in my understanding, is only a "candidate" for a tie point. I think that it only becomes a tie point when RC matches two or more of them and uses them for the further processes.

    However I do not think that they can entirely replace RCs "natural" tie points. If RC has nothing to go on in certain areas, you will not be able to force it to by adding CPs. RC will also not create any mesh in such places.

    0
    Comment actions Permalink
  • Avatar
    Tom Foster

    Now I get it Gotz, after re-reading end parts of brilliant thread https://support.capturingreality.com/hc/en-us/community/posts/115002427611-Pixels-Points-Features-Projections

    So when you say "I am pretty certain it is correct to adress CPs as manual [2D] tie points", the associated 3D tie point must be kinda reverse-engineered from same - and there's no Mean/Median reprojection error figure attached?

    Hmmm I'm not sure - what wierdness RC would have to assume to make that true.

    0
    Comment actions Permalink
  • Avatar
    Götz Echtenacher

    Not sure what you mean by reverse-engineered, Tom. I think that the 2D CPs are projected according to the active alignment. If the alignment is slightly wrong, this leads to certain reprojection errors of the (hopefully correctly placed) Control Points. Ideally, with a new alignment RC is convinced our CPs are right and hence the errors smaller.

    BTW, it is absolutely possible to get CPs with 0.0 px error. It's a matter of practice and a bit luck...  :-)

    0
    Comment actions Permalink
  • Avatar
    Tom Foster
     
    Götz Echtenacher said: "I think that the 2D CPs are projected according to the active alignment"
     
    Yes, I'd agree - and that means that when manually placed they are (or have same status as) artificial Features, which are then, as you say, "projected according to the active alignment" to define a 3D Tie point, which is then Reprojected to form 2D Tie points on the picture planes.
     
    This means that CPs as placed are not quite same as the resultant 2D Tie points - close beside but with Reprojection error of so many pixels distant, as counted on the picture plane.
     
    Götz Echtenacher said: "it is absolutely possible to get CPs with 0.0 px error". Truer to say that you can place a CP with a resolution of 1px - but I'm sure that RC's geometry engine is working at resolution very much finer than 1px.
     
    On recognising a Feature, which must be a pattern spread over several pixels, RC's algorithm must decide on a (centre?) point of that pattern, to use - which needn't be the centre of any pixel. Hopefully RC decides on exactly the 'same' centre point for all of the 2 or more slightly differing views of that pattern, that it can see on 2 or more photos. Unless it achieves that perfectly, that's straight away a source of eventual REprojection error.

    RC must be locating those centre points at resolution far finer than 1px, or even 0.3px, if after round-trip projections back (at 5o to 30o convergence) to a 3D Tie point and forward again to 2D Tie point, it's achieving typical Mean and Median Reprojection error figures typically in the 0.3 to 0.5px range.

    0
    Comment actions Permalink
  • Avatar
    Lucia CR

    @Alice: RealityCapture can suggest measurements since this release: https://support.capturingreality.com/hc/en-us/articles/360000994532-Version-1-0-3-3909-RC-PRE-RELEASE-

    More in the Help section: Control Points / Creating and Placing CPs / Suggestion of Control Points

    0
    Comment actions Permalink

Please sign in to leave a comment.