创建用于存储数据集的h5文件以训练超级分辨率GAN

Question

我正在尝试创建一个h5文件，用于存储用于训练超级分辨率GAN的数据集。每个训练对都是低分辨率和高分辨率的图像。数据集将以下列方式包含数据：[LR1、HR1、LR2、HR2、...LRn、HRn]。我有256 x 256 RGB图像用于HR，128x128 RGB用于LR。我对将其存储在h5文件中的最佳方法有些怀疑，在将它们存储到h5文件之前，我是否应该将图像缩放到255个？

为此，我编写了以下代码。如有任何帮助/建议，将不胜感激。

import h5py
import numpy as np
import os
import cv2
import glob



def store_super_resolution_dataset_in_h5_file(path_to_LR,path_to_HR):
    '''This function takes the files with the same name from LR and HR folders and stores the new dataset in h5 format'''
    #create LR and HR image lists
    LR_images = glob.glob(path_to_LR+'*.jpg')
    HR_images = glob.glob(path_to_HR+'*.jpg')
    #sort the lists
    LR_images.sort()
    HR_images.sort()
    print('LR_images: ',LR_images)
    print('HR_images: ',HR_images)
    #create a h5 file
    h5_file = h5py.File('super_resolution_dataset.h5','w')
    #create a dataset in the h5 file
    dataset = h5_file.create_dataset('super_resolution_dataset',(len(LR_images),2,256,256),dtype='f')
    #store the images in the dataset
    for i in range(len(LR_images)):
        LR_image = cv2.imread(LR_images[i])
        HR_image = cv2.imread(HR_images[i])
        dataset[i,0,:,:] = LR_image
        dataset[i,1,:,:] = HR_image
    #close the h5 file
    h5_file.close()

Answer 1

下面有两个代码段。第一个代码段显示了我推荐的方法:将Hi和低分辨率图像加载到单独的数据集中，以减少HDF5文件大小。第二个简单地纠正代码中的错误(修改为使用with/as:上下文管理器)。两个代码段都在#create a h5 file注释之后开始。

我对43个图像进行了测试，以比较产生的文件大小。结果如下：

• 1数据集大小=66.0MB
• 2数据集大小=41.3MB(减少37%)

推荐使用两个数据集的方法：

# get image dtypes and create a h5 file
LR_dt = cv2.imread(LR_images[0]).dtype
HR_dt = cv2.imread(HR_images[0]).dtype
with h5py.File('low_hi_resolution_dataset.h5','w') as h5_file:
    #create 2 datasets for LR and HR images in the h5 file
    lr_ds = h5_file.create_dataset('low_res_dataset',(len(LR_images),128,128,3),dtype=LR_dt)
    hr_ds = h5_file.create_dataset('hi_res_dataset',(len(LR_images),256,256,3),dtype=HR_dt)
    #store the images in the dataset
    for i in range(len(LR_images)):
        LR_image = cv2.imread(LR_images[i])
        HR_image = cv2.imread(HR_images[i])
        lr_ds[i] = LR_image
        hr_ds[i] = HR_image

对方法的修改：

# get LR image dtype and create a h5 file
LR_dt = cv2.imread(LR_images[0]).dtype
with h5py.File('super_resolution_dataset.h5','w') as h5_file:
    #create a dataset in the h5 file
    dataset = h5_file.create_dataset('super_resolution_dataset',(len(LR_images),2,256,256,3),dtype=LR_dt)
    #store the images in the dataset
    for i in range(len(LR_images)):
        LR_image = cv2.imread(LR_images[i])
        HR_image = cv2.imread(HR_images[i])
        dataset[i,0,0:128,0:128,:] = LR_image
        dataset[i,1,:,:,:] = HR_image