In this tutorial we will show you how to geo-reference your scene faster when using ground control points (GCPs). We will use a model of a gravel pit, where 11 ground control points have been marked and measured by land surveyors. The coordinates will be loaded from a .txt file and we will show you how to perform more than a hundred of measurements of 11 ground control points within several minutes using the control point measurements' suggestions. In order to use automatic suggestions, you need to align the images first.
Add GCPs to your project
Once you load the GCPs into a project, you will be able to see their position on the world map in the Map view.
In our case, we have a .txt file with a list of all ground control points with coordinates in the projected coordinate system epsg: 3016 SWEREF99 20 15. The coordinates are in this order: name, Y coordinate, X coordinate and altitude separated by the tabulator.
To load the file with the GCP coordinates, go to WORKFLOW tab -> Import&Metadata -> Ground Control.
After choosing the file, the Import Ground Control dialog shows up. Choose the correct format of the imported coordinates in the File format line...
... and the corresponding coordinate system in the Coordinate system line. You can either scroll down the list of coordinate systems in the database or search for a proper coordinate system by entering its epsg number or key-word.
If the project coordinate system is different from the coordinate system of the imported GCPs, a warning dialog shows up (see the picture below). To prevent the loss of precision during conversion between coordinate systems, we recommend setting the coordinate system of GCPs as the project coordinate system. You can also set the output coordinate system in this step.
After loading the file with ground control points' coordinates, you can see them in the 1Ds view under the Control points. After selecting one control point, the Selected control point(s) table with the details on the selected control point shows up. The marker in the red box in the picture below indicates that the control point type is Ground control.
You can also create a new control point by clicking on Create (see the picture above), change the name of a control point, change the type of the control point to Ground control, and enter the measured coordinates.
Using Map view to check the position of ground control points
Once your ground control points are loaded, you are able to see their position in the Map view even before the alignment in a clear project. This is how you can easily check whether the coordinate system and coordinates are correct and also use the satellite images to find the position of your ground control points on the ground and use it for better orientation in the sparse point cloud when placing the GCPs.
Switch one of the views to 'map'. In the Map view context menu, change the Map provider to Mapbox satellite (see the picture below).
When you use the Mapbox maps for the first time in RealityCapture, the Map credentials dialog will appear and the application will ask you for the credentials from your Mapbox account. There is a need to insert the so-called Access token into the field User credentials.
To get the access token for Mapbox maps, you need to register at https://www.mapbox.com/ first. The registration is free of charge.
Once you are successfully registered to Mapbox, go to the main page of your account and search for the section Access tokens. In this section, you can see the generated Default public token. Copy the Default public token and insert it into RC into the User token field and press OK. The satellite images from Mapbox will be loaded into the map view.
You can use the button Frame control points in the Map view context windows to fit the area of control points into the window.
Now you will be able to see the position of your ground control points based on satellite images.
You can always change the User credentials in the 'map 1D' view.
You can see the illustration how to load ground control points into the project and display them in Map view in the short video below:
Uploading images to a project and the alignment
You can upload images to the project via WORKFLOW tab –> 1. Add imagery –> Inputs, or upload the whole folder of images via the WORKFLOW tab –> 1. Add imagery –> Folder. You can also simply drag and drop images from Windows file explorer to your RealityCapture project.
Once you have successfully uploaded your images into the project, run the alignment by pressing WORKFLOW tab -> 2. Process -> Align Images or ALIGNMENT tab -> Align Images.
Once the alignment is finished, you can see that the created component (or more of them) is (are) added into the 1Ds view. If more components have been created, you can find more information how to put them together here: How to put together more components.
In our case, 294 images out of 294 have been aligned into a single component.
Control points' measurement
In this part we can use the implemented function for automatic suggestions of control points' measurements.
First of all, we recommend you to change the application layout in WORKFLOW tab -> Application -> Layout -> 1 + 2 + 2 Layout to see the 1Ds view, 3D view, Map view and two 2D views.
RealityCapture offers the possibility to place control points in a 3D view. To start placing the control points, go to ALIGNMENT tab -> Optional -> Add Control Points (or press F3) and select one of the control points in the 1Ds view.
RC will show two different suggested images in the separate 2D views while moving the cursor in the 3D view and you will be able to zoom in and zoom out both in the 3D and the 2D view simultaneously during the control point placement and editing. Images in the 2D are automatically rotated, zoomed and scaled in 2D when placing CP in the 3D view to match the 3D view and the cursor.
You can use the 'map' view for better orientation and localization of the selected control point.
If none of the loaded control points is selected, marking the position with the Control Points function turned on will create a new control point.
Now you can move your cursor into the 3D view and find the location for the selected ground control point.
Once you click with the left mouse button, the control point will be placed. If you want to move your model in the 3D view, hold the left mouse button and move the model. When you zoom in to find the precise location of the control point, both 3D and 2D views will be zoomed in simultaneously.
When placing control points, you can see the suggestion lines in the 3D view (grey lines in the bottom right corner of the picture above). The suggestion lines are control point-to-camera lines which appear when selecting the control point in 3D view. After the control point placement, suggested measurements appear as grey lines, confirmed measurements appear as orange lines.
When you have localized the specific control point in the 3D view, along with the displayed images in the 2D views, click the left mouse button to place the control point. A list of suggested measurements will appear in the 1Ds view.
You can see the illustration how to create measurements' suggestions for one ground control point in this video:
When you move the mouse cursor on the suggested measurement in the 1Ds view, the red cross will show up and by clicking it you can erase this suggestion from the list of measurements. When you click on the green plus sign, the suggested measurement will be validated as it is. When you select a specific suggestion, the corresponding image shows up in the 2D view (must be marked blue) and you can adjust the suggested measurement. You can also drag and drop the control point measurement to any 2D view, the associated image will be displayed.
! IMPORTANT !
To change the position of a control point in the image, move the mouse cursor over the point in the image in the 2D view and hold the left mouse button (do not release the button). The image will be automatically zoomed in and, by moving the mouse along with holding the left mouse button, you can adjust the position of the control point.
Do not release the left mouse button and press the down arrow key. The adjusted measurement for the first image will be automatically confirmed and the second suggestion will be shown in the 2D view. You do not need to zoom in again, the next image is zoomed precisely as the previous one. While you still hold the left mouse button, move the mouse cursor to adjust the position of the measurement in this image and press the down arrow key to confirm the position in this image and to skip to another suggestion.
With this workflow you can easily make tens of measurements for the same control point within seconds.
After adjusting the suggested measurements, you can see the images in which a GCP has been identified and the per-image-measurement deviations in pixels (the red box in the picture below).
The yellow triangle next to the measurement means that the projection error of that measurement is over the limit. The limit is set to 2px. You can adjust that measurement again or delete it from the list of measurements.
TIP! We recommend you to first place the suggestions in the 3D view for all control points and subsequently adjust their position with the function ALIGNMENT tab -> Optional -> Add Control Points turned off. This will prevent an accidental placement of a new control point. However, if you accidentally place a new undesired control point, you can erase it by holding Crtl + Z, or manually remove it in the 1Ds view by clicking the red cross right next to it.
The illustration of the proposed workflow for adjusting the suggested measurements can be found in this video (with subtitles):
Placing 4 GCPs and suggesting the position of others
For considerable time savings, the function Suggest Measurements can be used. If you place at least 4 (strongly recommended) ground control points and Update the scene, you can use the implemented function for suggesting measurements for all other control points.
In this case, for more precise suggestions, we recommend to manually locate ground control points from the corners of your area, or from the corners and the middle of the area, and not the ones close to each other.
Once you have placed at least 4 GCPs, press ALIGNMENT tab -> Registration -> Update to geo-reference your project. We chose the ground control points in the corners and in the middle of covered area for more precise suggestions.
After the scene update, you will be able to see the total deviations in position for already placed control point next to the control points' name in the GCP coordinate system units.
Function SCENE 1D tab -> TOOLS -> Constraints -> Suggest Measurements will suggest control point measurements for selected control points. If none of the control points is selected, or all of them are selected, the function will suggest measurements for all control points in the control points list. To select all control points, simply press Ctrl + A, or click on the first control point, hold the Shift key and select the last control point.
Click on the button SCENE 1D tab -> TOOLS -> Constraints -> Suggest Measurements to automatically suggest measurements for all other loaded ground control points.
In this case there is no need for placing the other ground control points manually, all you need to do is adjust the suggested measurements.
You can see the illustration how to use the Suggest measurements tool in the short video below:
Geo-referencing a scene
Once you are finished with creating and placing control points, click on the ALIGNMENT tab -> Registration -> Align Images. You can also use the ALIGNMET tab -> Registration -> Update button to apply changes made in ground control points but this will just find a non-rigid transformation of the placed GCPs in the component to the imported 3D positions of GCPs. However, it does not change relative positions of cameras in a scene.
In the picture below, after clicking the Update button, we can see the total deviations in position for each control point in the GCP coordinate system units.
Alignment is always more precise, but since the camera poses might change too, all previously created models will not match with the cameras. Therefore, a new component is created and models are left in the previous component. After you click ALIGNMENT tab -> Registration -> Align images, a new geo-referenced component will be created.
Once you click on the newly created component in the 1Ds view, the Selected component(s) table shows up and the status in the Alignment Report –> Geo-referenced and Metric will be set to 'Yes'. Further, you can see the recalculated total deviations for each ground control point.
By using the implemented function for suggesting control points' measurements, we have been able to identify 193 measurements for 11 control points and geo-reference the project in less than 15 minutes!