图像处理.比较2幅图像和排序相似性

Question

我的代码比较了任意形状/尺寸的2幅图像，并将它们按相似顺序排列。它从读取包含绝对路径的image1、image2列的CSV文件开始，然后输出到包含image1、image2、相似性、time_elapsed列的CSV文件。

我更关心格式化建议或简化，以使代码更容易阅读

import cv2
import numpy as np
import csv
import time

#read from CSV file
def read_data():
    print("starting to read data")
    with open('input_test_data.csv') as input_data:
        print("opened data file")
        global line_count
        csv_reader = csv.reader(input_data, delimiter=',')
        set_of_images=[row for idx, row in enumerate(csv_reader) if idx == line_count]
        print(set_of_images)
        original_image = set_of_images[0][0]
        image_to_compare = set_of_images[0][1]
        line_count += 1
        return [original_image, image_to_compare]

# wrote column titles for result CSV file
def write_header():
    global line_count
    with open('result_data.csv', mode='w') as result:
        result_writer = csv.writer(result, delimiter=',', quotechar='"', quoting=csv.QUOTE_MINIMAL)
        result_writer.writerow(['image1', 'image2', 'similarity_ratio', 'time_elapsed'])
    print('wrote result header')
    line_count += 1

# count total number of entries in CSV file
def count_images():
    with open('input_test_data.csv') as input_data:
        print("counting number of rows in CSV file")
        csv_reader = csv.reader(input_data, delimiter=',')
        return sum(1 for row in csv_reader)


# reading images
def read_images(original_img: str, image_to_compare: str):
    original_image = cv2.imread(original_img)
    compared_image = cv2.imread(image_to_compare)
    print("Read images")
    return [original_image, compared_image]


# preliminary check of RGB values are same 
def check_identical(original_image , compared_image , start_time: float, image_names):
    # if images are the same shape and size - could be identical
    if original_image.shape == compared_image.shape:
        print("Images have the same size and channel")
        difference = cv2.subtract(original_image, compared_image)   # subtract RGB values between images to see difference between image organization
        b,g, r = cv2.split(difference)

        # if RGB value difference is 0 - same image
        if cv2.countNonZero(b) == 0 and cv2.countNonZero(g) == 0 and cv2.countNonZero(r) == 0:
            publish_results(image_names[0], image_names[1], 0.0,  start_time)
            print("The images are completely Equal")
            return True

        else:
            print("The images are NOT completely Equal")
            return False

# if images are not identical:
# use OpenCV's SIFT Algorithm to find key points and features between images
def generate_keypoints(original_image: str, compared_image: str):
    sift = cv2.xfeatures2d.SIFT_create()
    key_point_image1, descriptor_image1 = sift.detectAndCompute(original_image, None)
    key_point_image2, descriptor_image2 = sift.detectAndCompute(compared_image, None)
    print("Generated keypoints for both images")
    return [key_point_image1, descriptor_image1, key_point_image2, descriptor_image2]

# FLANN - Fast Library for Approximate Neighbors
# uses Euclidean distance between common key points to judge similarity  
def compare_images(key_point_image1, descriptor_image1, key_point_image2, descriptor_image2  ):
    index_params = dict(algorithm=0, trees=5)
    search_params = dict()
    flann = cv2.FlannBasedMatcher(index_params, search_params)
    matches = flann.knnMatch(descriptor_image1, descriptor_image2, k=2)
    print("found common points in images")
    return matches

# SIFT algorithm has high recall, but low precision
# so a threshold ratio needs to be set to filter inacurate results
def filter_results(matches: enumerate):
    good_points = 0
    ratio = 0.6 # approximated through trial and error
    print("Matches:" + str(len(matches)))
    for m, n in matches:
            if m.distance < ratio*n.distance:
                    good_points += 1
    print("Good Matches: " + str(good_points))
    return [good_points, len(matches)]


# Generate Similarity Score
def generate_similarity_score(correct_matches: int, total_matches: int ):
    return 1.0 - correct_matches/total_matches   # as defined in requirements, values close to 0 is most similar, so more correct matches mean similar pictures

# Publish to CSV
def publish_results(original_img: str, image_to_compare: str, similarity_ratio , start_time):
    print("writing to CSV file...")
    with open('result_data.csv', mode='a', newline='') as result:
        result_writer = csv.writer(result, delimiter=',', quotechar='"', quoting=csv.QUOTE_MINIMAL)
        result_writer.writerow([original_img, image_to_compare, str(similarity_ratio), str(round(time.time() - start_time, 2))] )
        print("Write successful")

# Main Function
# reading data from CSV
print("starting program")
line_count = 0

# find number of items in CSV file to compare
total_images_to_compare = count_images()

# prepare column titles for result CSV file
write_header()

for img in range(line_count, total_images_to_compare):

    # start recording execution time
    start_time = time.time()

    # read set of images
    image_names = read_data()

    # read original image and image to compare into memory
    images = read_images(image_names[0], image_names[1])


    # determine if identical images
    if (check_identical(images[0], images[1], start_time, image_names)):
        print("continue")
       # continue

    print("images NOT EQUAL - starting SIFT")

    # if images are not equal - start using SIFT to generate keypoints and features of both images to measure similarity
    keypoints_descriptors = generate_keypoints(images[0], images[1])

    # Attempt at finding common points in both images
    matches = compare_images(keypoints_descriptors[0], keypoints_descriptors[1], keypoints_descriptors[2], keypoints_descriptors[3] )

    print("filtering results now...")
    # filter out outliers and mismatches
    acurate_points = filter_results(matches)

    print("generating a score...")
    # find a ratio for closely matched images are
    score = generate_similarity_score(acurate_points[0], acurate_points[1])

    # write data to CSV file
    publish_results(image_names[0], image_names[1], score, start_time)

    # increment next set of images to be compared
    img += 1

这是input_test_data.csv：

image1  image2
D:\Programming\test\images\original_golden_bridge.jpg   D:\Programming\test\images\black_and_white.jpg
D:\Programming\test\images\original_golden_bridge.jpg   D:\Programming\test\images\blue_filter.jpg
D:\Programming\test\images\original_golden_bridge.jpg   D:\Programming\test\images\blurred.jpg
D:\Programming\test\images\original_golden_bridge.jpg   D:\Programming\test\images\cartoonized.jpg
D:\Programming\test\images\original_golden_bridge.jpg   D:\Programming\test\images\exposured.jpg
D:\Programming\test\images\original_golden_bridge.jpg   D:\Programming\test\images\george-washington-bridge.jpg
D:\Programming\test\images\original_golden_bridge.jpg   D:\Programming\test\images\old_photo.jpg
D:\Programming\test\images\original_golden_bridge.jpg   D:\Programming\test\images\original_golden_bridge.jpg
D:\Programming\test\images\original_golden_bridge.jpg   D:\Programming\test\images\overlay.jpg
D:\Programming\test\images\original_golden_bridge.jpg   D:\Programming\test\images\resized.jpg
D:\Programming\test\images\original_golden_bridge.jpg   D:\Programming\test\images\rotated.jpg
D:\Programming\test\images\original_golden_bridge.jpg   D:\Programming\test\images\sharpened.jpg
D:\Programming\test\images\original_golden_bridge.jpg   D:\Programming\test\images\sunburst.jpg
D:\Programming\test\images\original_golden_bridge.jpg   D:\Programming\test\images\textured.jpg

result_data.csv：

image1  image2  similarity_ratio    time_elapsed

D:\Programming\test\images\original_golden_bridge.jpg   D:\Programming\test\images\black_and_white.jpg  0.308852067 3.55
D:\Programming\test\images\original_golden_bridge.jpg   D:\Programming\test\images\blue_filter.jpg  0.13136289  3.45
D:\Programming\test\images\original_golden_bridge.jpg   D:\Programming\test\images\blurred.jpg  0.944469324 1.46
D:\Programming\test\images\original_golden_bridge.jpg   D:\Programming\test\images\cartoonized.jpg  0.601731602 3.17
D:\Programming\test\images\original_golden_bridge.jpg   D:\Programming\test\images\exposured.jpg    0.736229288 2.91
D:\Programming\test\images\original_golden_bridge.jpg   D:\Programming\test\images\george-washington-bridge.jpg 1   6.25
D:\Programming\test\images\original_golden_bridge.jpg   D:\Programming\test\images\old_photo.jpg    0.637557844 4.07
D:\Programming\test\images\original_golden_bridge.jpg   D:\Programming\test\images\original_golden_bridge.jpg   0   0.14
D:\Programming\test\images\original_golden_bridge.jpg   D:\Programming\test\images\duplicate.jpg    0   3.55
D:\Programming\test\images\original_golden_bridge.jpg   D:\Programming\test\images\overlay.jpg  0.824600687 2.11
D:\Programming\test\images\original_golden_bridge.jpg   D:\Programming\test\images\resized.jpg  0.907598149 1.2
D:\Programming\test\images\original_golden_bridge.jpg   D:\Programming\test\images\rotated.jpg  0.150619495 3.51
D:\Programming\test\images\original_golden_bridge.jpg   D:\Programming\test\images\sharpened.jpg    0.522316764 5.07
D:\Programming\test\images\original_golden_bridge.jpg   D:\Programming\test\images\sunburst.jpg 0.922675026 8.92
D:\Programming\test\images\original_golden_bridge.jpg   D:\Programming\test\images\textured.jpg 0.494103598 4.84

我为results.csv的格式表示歉意

我还打算将所有print语句替换为日志语句，以使将来的开发人员更容易维护，同时更专业。

Answer 1

免责声明:下面的代码是(部分)未经测试的，所以可能有一些粗糙的边缘需要抛光。

风格

首先，一些与风格有关的笔记。Python有一个正式的风格指南，通常被称为PEP8。与代码IMHO最相关的部分是文档字符串。本质上，您应该遵循官方的建议，将您的函数文档放入函数体中的"""triple quotes"""中。从您的代码中改编的示例：

def compare_images(key_point_image1, descriptor_image1,
                   key_point_image2, descriptor_image2):
    """
    uses Euclidean distance between common key points to judge similarity
    in FLANN (Fast Library for Approximate Neighbors) matching
    """
    ...

通常使用显式列表从函数返回多个值。这是不寻常的，也可能是不必要的。大多数情况下，当返回多个值时不存在显式类型。

def swap(a, b):
    return b, a

在这个框架下，Python从函数返回一个元组，如果查看type(swap(1, 2))，它将很高兴地在交互式Python解释器中打印tuple。

避免全局变量

只要有可能，就应该尽量避免全局变量。在您的例子中，全局变量是绝对不必要的。我们马上就会讨论这个问题。

read_data

read_data写得太浪费了。你总是读完整的文件，只是为了得到一行。通常的方法是逐行读取文件，然后进行相应的处理。或者作为一个整体读取文件并逐行处理其内容。这还允许您摆脱line_count，以跟踪下一步需要查看的行。要读取的文件的名称可能也应该是一个参数，而不是硬编码。新版本的read_data可能如下所示：

def read_data(input_file):
    """read from CSV file"""
    with open(input_file) as input_data:
        csv_reader = csv.reader(input_data, delimiter=',')
        return [row for row in csv_reader]

此实现使用列表理解将csv文件的所有行读入内存。有了这个，count_images()可以简单地替换为

images_list = read_data(...)
total_images_to_compare = len(images_list)

您还可以使用一个生成器表达式，它基本上是一个列表理解，但并不是所有的元素都必须同时保存在内存中。请注意，对于生成器表达式，由于没有为生成器表达式定义len(...)，上面的内容将无法工作。

这两个版本都可以在脚本的主要部分中使用如下：

for image_names in images_list:
    # read original image and image to compare into memory
    images = read_images(image_names[0], image_names[1])

    ...

check_identical

名单上的下一个是check_identical。IMHO这里的执行也比必须要复杂得多。在这里，numpy可以极大地帮助简化代码：

def check_identical(original_image, compared_image):
    """preliminary check of RGB values are same"""
    if original_image.shape == compared_image.shape:
        # if images are the same shape and size - could be identical
        return np.count_nonzero(original_image - compared_image) == 0

    return False

在它的核心，功能仍然做你所做的，但对整个图像，而不是单独为每个颜色通道。这里的主要区别是start_time和image_names已经从参数列表中消失了，因为为什么check_indentical必须关心将任何东西写入输出文件是有原因的。

元组解包

在您的代码中有几个部分可以执行foobar(images[0], images[1])。如果images具有与foobar(...)预期的相同数量的元素，则只需使用foobar(*images)即可。这被称为元组拆装，但也适用于其他序列，如lists。

另外，有时将多个返回值赋值给命名变量以更好地查看传递的内容更容易理解。

correct_matches, total_matches = filter_results(matches)

score = generate_similarity_score(correct_matches, total_matches)

Sidenote:也可以：

score = generate_similarity_score(*filter_results(matches))

硬编码值

代码中也有很多硬编码的值。下面是一些例子：

'input_test_data.csv'
'result_data.csv'
index_params = dict(algorithm=0, trees=5)
matches = flann.knnMatch(..., k=2)
ratio = 0.6 # approximated through trial and error

IMHO，它们都应该是函数参数，可以在不接触实际代码的情况下进行更改。您还可以为它们分配与当前默认值相匹配的默认值，因此程序中的任何内容都不需要更改。在当前状态下，如果您想要更改其中的一些值，则必须跳转到函数实现的位置，并在那里直接更改它。虽然这对于一些值来说可能是可行的，但如果您曾经想过在一次运行中尝试几个值(可能是为了优化一个参数)，那么麻烦就开始了。

main函数

虽然您的代码最后大胆地声明了# Main Function，但是这里没有一个实际的main函数。但绝对应该有一个！而且，由于我们是这样做的，所以最好确保脚本的main函数只有在文件实际用作脚本时才被调用，而不是导入到其他代码中。进入顶层脚本环境！

# ... all the import and other functions here ...

def main():
    """Read a list of image pairs from disk and check their similarity"""
    ...

if __name__ == "__main__":
    main()

有关这是如此的帖子的扩展说明，请参阅__name__。