Python，OpenCV - 一個接一個地排列和疊加多個圖像

Question

我的項目是對齊航拍照片以製作一張馬賽克地圖。我的計劃是從兩張照片開始，將第二張與第一張對齊，然後從兩張對齊的圖像中創建「初始鑲嵌」。一旦完成，我將第三張照片與最初的馬賽克對齊，然後將第四張照片與其結果等對齊，從而逐步構建地圖。

我有兩種技術可以做到這一點，但使用calcOpticalFlowPyrLK()的更精確的技術僅適用於雙圖像階段，因爲兩個輸入圖像必須具有相同的尺寸。正因爲如此，我嘗試了一種新的解決方案，但它不太準確，並且在每一步都引入了錯誤，最終產生了無意義的結果。

我的問題是雙重的，但如果你知道一個答案，你不必回答這兩個問題，除非你想。首先，有沒有辦法使用類似於calcOpticalFlowPyrLK()的東西，但有兩個不同大小的圖像（這包括任何潛在的解決方法）？其次，是否有辦法修改檢測器/描述符解決方案以使其更加準確？

這裏只適用於兩個圖像的準確版本：

# load images 
base = cv2.imread("images/1.jpg") 
curr = cv2.imread("images/2.jpg") 

# convert to grayscale 
base_gray = cv2.cvtColor(base, cv2.COLOR_BGR2GRAY) 

# find the coordinates of good features to track in base 
base_features = cv2.goodFeaturesToTrack(base_gray, 3000, .01, 10) 

# find corresponding features in current photo 
curr_features = np.array([]) 
curr_features, pyr_stati, _ = cv2.calcOpticalFlowPyrLK(base, curr, base_features, curr_features, flags=1) 

# only add features for which a match was found to the pruned arrays 
base_features_pruned = [] 
curr_features_pruned = [] 
for index, status in enumerate(pyr_stati): 
    if status == 1: 
     base_features_pruned.append(base_features[index]) 
     curr_features_pruned.append(curr_features[index]) 

# convert lists to numpy arrays so they can be passed to opencv function 
bf_final = np.asarray(base_features_pruned) 
cf_final = np.asarray(curr_features_pruned) 

# find perspective transformation using the arrays of corresponding points 
transformation, hom_stati = cv2.findHomography(cf_final, bf_final, method=cv2.RANSAC, ransacReprojThreshold=1) 

# transform the images and overlay them to see if they align properly 
# not what I do in the actual program, just for use in the example code 
# so that you can see how they align, if you decide to run it 
height, width = curr.shape[:2] 
mod_photo = cv2.warpPerspective(curr, transformation, (width, height)) 
new_image = cv2.addWeighted(mod_photo, .5, base, .5, 1) 

下面是對多個圖像作品（直到誤差變得太大）不準確的一項：

# load images 
base = cv2.imread("images/1.jpg") 
curr = cv2.imread("images/2.jpg") 


# convert to grayscale 
base_gray = cv2.cvtColor(self.base, cv2.COLOR_BGR2GRAY) 

# DIFFERENCES START 
curr_gray = cv2.cvtColor(self.curr_photo, cv2.COLOR_BGR2GRAY) 

# create detector, get keypoints and descriptors 
detector = cv2.ORB_create() 
base_keys, base_desc = detector.detectAndCompute(base_gray, None) 
curr_keys, curr_desc = detector.detectAndCompute(curr_gray, None) 

matcher = cv2.DescriptorMatcher_create("BruteForce-Hamming") 

max_dist = 0.0 
min_dist = 100.0 

for match in matches: 
    dist = match.distance 
    min_dist = dist if dist < min_dist else min_dist 
    max_dist = dist if dist > max_dist else max_dist 

good_matches = [match for match in matches if match.distance <= 3 * min_dist ] 

base_matches = [] 
curr_matches = [] 
for match in good_matches: 
    base_matches.append(base_keys[match.queryIdx].pt) 
    curr_matches.append(curr_keys[match.trainIdx].pt) 

bf_final = np.asarray(base_matches) 
cf_final = np.asarray(curr_matches) 

# SAME AS BEFORE 

# find perspective transformation using the arrays of corresponding points 
transformation, hom_stati = cv2.findHomography(cf_final, bf_final, method=cv2.RANSAC, ransacReprojThreshold=1) 

# transform the images and overlay them to see if they align properly 
# not what I do in the actual program, just for use in the example code 
# so that you can see how they align, if you decide to run it 
height, width = curr.shape[:2] 
mod_photo = cv2.warpPerspective(curr, transformation, (width, height)) 
new_image = cv2.addWeighted(mod_photo, .5, base, .5, 1) 

最後，在這裏是我使用的一些圖片：

個

Answer 1

單應組成，所以如果你有img1和img2之間和img2和img3之間那麼這兩個單應組成的單應給予img1和img3之間的單應。

當然因爲您想將img3與包含img1和img2的拼接圖像匹配，所以您的尺碼已關閉。但你不需要那樣做。不要縫製它們，直到你在每一對連續的圖像之間有所有的單應性。然後你可以用兩種方法之一進行;從後面工作或從前面工作。我將用於例如h31指代將img3彎曲成img1的座標的單應性。

從前面（僞）：

warp img2 into coordinates of img1 with h21 
warp img3 into coordinates of img1 with h31 = h32.dot(h21) 
warp img4 into coordinates of img1 with h41 = h43.dot(h31) 
... 
stitch/blend images together 

從背面（僞）：

... 
warp prev stitched img into coordinates of img3 with h43 
stitch warped stitched img with img3 
warp prev stitched img into coordinates of img2 with h32 
stitch warped stitched img with img2 
warp prev stitched img into coordinates of img1 with h21 
stitch warped stitched img with img1 

的想法是要麼你從前面開始，經編一切都變成第一圖像座標框架，或從後面開始，翹曲到先前的圖像和針跡，然後將該拼接的圖像扭曲到先前的圖像中，然後重複。我認爲第一種方法可能更容易。無論哪種情況，您都必須擔心單應性估計中的錯誤傳播，因爲它們將建立在多個組合單應性之上。

這是一種簡單的方法來混合多個圖像在一起，只有單應性。更復雜的方法是使用束調整，它考慮了所有圖像上的特徵點。然後爲了良好的混合，這些步驟是增益補償以消除相機增益調整和漸暈，然後進行多波段混合以防止模糊。請參閱Brown and Lowe的開創性論文here以及一個精彩的示例和免費演示軟件here。