This program calculates the relative positions of a set of cameras filming a scene with non-overlapping frames. It uses an panoramic frame taken from an omnidirectional camera as a reference.
It has been implemented by Guillaume BEA and Florian LE VERN during their internship at KEIO University in the Hyper Vision Research Laboratory of Professor Hideo SAITO [link] from June to August 2016.
In order to use the program, some dependencies must be installed:
-
RT Estimation between omnidirectional and flat cameras
Copyright © 2016 Hyper Vision Research Laboratory, Shogo MIYATA, Guillaume BEA, Florian LE VERN. -
Spherical Pose Solver
Copyright © 2012-2013 Pierre MOULON. Licensed under the MPL2 license. -
Open Source Computer Vision Library
Copyright © OpenCV authors. Licensed under the 3-Clause BSD License. -
OpenMVG - open Multiple View Geometry
Copyright © OpenMVG authors. Licensed under the MPL2 license.
First of all, build the program into a new folder:
$ mkdir build
$ (cd build && cmake ..)
$ (cd build && make)
The main action of this program is to take a set of pictures following a specific architecture in the file system and to calculate the Rotation and Translation matrices indicating the pose of these frames in the scene in the system of coordinates of the panoramic frame.
Thus, the first thing to do is to prepare your set of data.
In a new folder that we will call experiment, create the following architecture:
- experiment
- cam
> Insert in this folder all the images
from the different cameras filming the scene
that you want their relative positions.
- omni
> Insert in this folder the panoramic image
of the scene. If there is more than one picture
in this folder, only the first one will be used.
Then, the program can be launched with the following command (see the option section a bit farther below for more possibilities):
$ ./build/rt_estimation <path_to_experiment_folder>
The calculated Rotation and Translation matrices will be saved into a res.txt file where the program has been launched.
The RT Estimation program provides some options to allow several kind of experiments.
-help or -h or -usage or -?
Print the help in the standard output.
-manual or -m
Use the keypoints stored in the experiment folder (keypoints.yml) images instead of using SIFT. This file is generated by executing point_marker on an experiment (see next section).
-reference=<image_name> or -ref=<image_name> or -r=<image_name>
Name of the image to use as the reference for coordinates instead of the panoramic image.
-output= or -o=
Name of the output file containing (R,T) in the reference system of coordinates.
Another executable is generated after building the program. It can be used to manually select the matched keypoints, wich are stored in the experiment folder.
$ ./build/point_marker <path_to_experiment_folder>
A python 3D viewer as been made in order to see the positions of the different cameras. Its input is the file generated by rt_estimation. Use the following command :
$ python plot/plot.py < res.txt
Here is a brief description of the theory that this program uses:
In order to calculate the rotation and translation matrices, we use the panoramic image that sees all the scene as a reference. The issue is therefore to find the (R,T) couple between the omnidirectional image and the flat image.
The idea is to use feature detection and feature matching to match some singular points between the two images. With this data, we are able to write epipolar equations which solution will provide an Essential matrix.
Then, this matrix can be decomposed into four possible (R,T) pairs. The selection of the unique solution uses the fact that only one pair shows all the singular 3D points in front of both cameras (in reality we take the maximum).