基于运动结构的三维重建

Question

我想用运动结构算法进行三维重建。我正在使用opencv在python中做这件事。但是如何将获得的点云一分为二。我的输入图像是：Image 1 Image 2 Image 3.我每2个图像就匹配一次，比如image1与image2，image2与图像3。我尝试了不同的特征检测器，如SIFT，KAZE和SURF。使用所获得的点，我计算基本矩阵。我从Opencv的摄像机校准中获得了摄像机内部特性，并存储在下面代码中的变量'mtx‘和'dist’中。

```file = os.listdir('Path_to _images')

file.sort(key=lambda f: int(''.join(filter(str.isdigit，f)

path = os.path.join(os.getcwd()，'Path_to_images/')

对于范围(0，len(文件)-1)中的i：

if(i == len(file) - 1):

    break

path1 = cv2.imread(path + file[i], 0)

path1 = cv2.equalizeHist(path1)

path2 = cv2.imread(path + file[i+1], 0)

path2 = cv2.equalizeHist(path2)

特征检测

sift = cv2.xfeatures2d.SIFT_create()

kp1, des1 = sift.detectAndCompute(path1,None)

kp2, des2 = sift.detectAndCompute(path2,None)

特征匹配

FLANN_INDEX_KDTREE = 0              

index_params = dict(algorithm = FLANN_INDEX_KDTREE, trees = 5)

search_params = dict(checks=50)   

flann = cv2.FlannBasedMatcher(index_params,search_params)

matches = flann.knnMatch(des1,des2,k=2)

good = []

pts1 = []

pts2 = []

for j, (m,n) in enumerate(matches):

    if m.distance < 0.8*n.distance:

        good.append(m)

        pts2.append(kp2[m.trainIdx].pt)

        pts1.append(kp1[m.queryIdx].pt)

pts1 = np.int32(pts1)

pts2 = np.int32(pts2)

pts1 = np.array([pts1],dtype=np.float32)

pts2 = np.array([pts2],dtype=np.float32)

未组织的点

pts1_norm = cv2.undistortPoints(pts1, mtx, dist)

pts2_norm = cv2.undistortPoints(pts2, mtx, dist)

计算基础矩阵

F, mask = cv2.findFundamentalMat(pts1_norm,pts2_norm,cv2.FM_LMEDS)

计算本质矩阵

E, mask = cv2.findEssentialMat(pts1_norm, pts2_norm, focal=55.474, pp=(33.516, 16.630), method=cv2.FM_LMEDS, prob=0.999, threshold=3.0)

姿势恢复

points, R, t, mask = cv2.recoverPose(E, pts1_norm, pts2_norm)

anglesBetweenImages = rotationMatrixToEulerAngles(R)

sys.stdout = open('path_to_folder/angles.txt', 'a')

print(anglesBetweenImages)

构造R，t的投影阵

matrix_1 = np.hstack((R, t))

matrix_2 = np.hstack((np.eye(3, 3), np.zeros((3, 1))))

projMat_1 = np.dot(mtx, matrix_1)

projMat_2 = np.dot(mtx, matrix_2)

三角化点

point_4d_hom = cv2.triangulatePoints(projMat_1[:3], projMat_2[:3], pts1[:2].T, pts2[:2].T)

将4D结果均匀化为3D

point_4d = point_4d_hom

point_3d = point_4d[:3, :].T                # Obtains 3D points

np.savetxt('/path_to_folder/'+ file[i] +'.txt', point_3d)

在cv2.triangulatePoints之后，我希望获得一个点云。但我得到的结果是两个表面，如下图所示。

Result 1.如果有人能告诉我我的算法出了什么问题，我真的很感激。谢谢!

Answer 1

您需要以交互的方式完成此操作

如下所示：

cv::Mat pointsMat1(2, 1, CV_64F);
cv::Mat pointsMat2(2, 1, CV_64F);

int size0 = m_history.getHistorySize();

for(int i = 0; i < size0; i++){
  cv::Point pt1 = m_history.getOriginalPoint(0, i);
  cv::Point pt2 = m_history.getOriginalPoint(1, i);

  pointsMat1.at<double>(0,0) = pt1.x;
  pointsMat1.at<double>(1,0) = pt1.y;
  pointsMat2.at<double>(0,0) = pt2.x;
  pointsMat2.at<double>(1,0) = pt2.y;

  cv::Mat pnts3D(4, 1, CV_64F);

  cv::triangulatePoints(m_projectionMat1, m_projectionMat2, pointsMat1, pointsMat2, pnts3D);
}