用包含灰度图像的数据集传输学习MobileNet

Question

我有个问题。我想用MobileNevV2训练一个网络，但据我所知，你只能把彩色图像传递给它。

我的数据集只包含黑白图像。如果我试图将图像的形状传递为(224,224，1)，很明显，错误会返回我：

Traceback (most recent call last):
  File "/home/andrea/Scrivania/COMPUTER-VISION/MobileNet_train.py", line 40, in <module>
    mobielNetV2 = tensorflow.keras.applications.MobileNetV2(input_shape=IMG_SHAPE, include_top=False, weights='imagenet')
  File "/home/andrea/.local/lib/python3.9/site-packages/keras/applications/mobilenet_v2.py", line 279, in MobileNetV2
    input_shape = imagenet_utils.obtain_input_shape(
  File "/home/andrea/.local/lib/python3.9/site-packages/keras/applications/imagenet_utils.py", line 372, in obtain_input_shape
    raise ValueError('The input must have 3 channels; Received '
ValueError: The input must have 3 channels; Received `input_shape=(224, 224, 1)`  

如何训练黑白图像模型？下面是我创建的代码：

import matplotlib.pyplot as plt
import tensorflow
from tensorflow.keras.layers import Dense, Flatten
from tensorflow.keras.models import Model
from tensorflow.keras.preprocessing.image import ImageDataGenerator

IMAGE_SIZE = (224, 224)
IMG_SHAPE = IMAGE_SIZE + (1,)
DATASET_DIR = "/home/andrea/Scrivania/COMPUTER-VISION/DATASET/KAGGLE/new_train"
BATCH_SIZE = 32
EPOCHS = 5

datagen = ImageDataGenerator(
    validation_split=0.2,
    rescale=1. / 255,  # per processare più velocemente i dati
    brightness_range=[1, 2]
)

train_generator = datagen.flow_from_directory(
    DATASET_DIR,
    target_size=IMAGE_SIZE,
    batch_size=BATCH_SIZE,
    class_mode="categorical",
    subset="training"
)

test_generator = datagen.flow_from_directory(
    DATASET_DIR,
    target_size=IMAGE_SIZE,
    batch_size=BATCH_SIZE,
    class_mode="categorical",
    subset="validation"
)

mobielNetV2 = tensorflow.keras.applications.MobileNetV2(input_shape=IMG_SHAPE, include_top=False, weights='imagenet')

for layer in mobielNetV2.layers:
    layer.trainable = False

x = Flatten()(mobielNetV2.output)
prediction = Dense(6, activation='softmax')(x)

model = Model(inputs=mobielNetV2.input, outputs=prediction)
# tell the model what cost and optimization method to use
model.summary()

model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])

r = model.fit(train_generator, validation_data=test_generator, epochs=EPOCHS, steps_per_epoch=len(train_generator),
              validation_steps=len(test_generator))
model.save("MobileNet_Hand.h5")

Answer 1

让我们假设您已经拥有灰度图像，并且希望将它们转换为rgb图像来训练您的模型：

import tensorflow as tf
import matplotlib.pyplot as plt

flowers = tf.keras.utils.get_file(
    'flower_photos',
    'https://storage.googleapis.com/download.tensorflow.org/example_images/flower_photos.tgz',
    untar=True)

img_gen = tf.keras.preprocessing.image.ImageDataGenerator(rescale=1./255)

ds = tf.data.Dataset.from_generator(
    lambda: img_gen.flow_from_directory(flowers, batch_size=32, shuffle=False),
    output_types=(tf.float32, tf.float32))

ds = ds.map(lambda x, y: (tf.image.rgb_to_grayscale(x), y))
images, _ = next(iter(ds.take(1)))

image = images[0]
print(image.shape)
plt.imshow(image.numpy()[:,:,0], cmap='gray')

(256, 256, 1)

灰度：

​

​

RGB：

ds = ds.map(lambda x, y: (tf.image.grayscale_to_rgb(x), y))
images, _ = next(iter(ds.take(1)))

image = images[0]
print(image.shape)
plt.imshow(image.numpy())

(256, 256, 3)

​

​

所以，只需将tf.image.grayscale_to_rgb与dataset.map结合使用，就可以了。

更新1

datagen = ImageDataGenerator(
    validation_split=0.2,
    rescale=1. / 255,  # per processare più velocemente i dati
    brightness_range=[1, 2]
)

train_generator = datagen.flow_from_directory(
    DATASET_DIR,
    target_size=IMAGE_SIZE,
    batch_size=BATCH_SIZE,
    class_mode="categorical",
    subset="training"
)
ds = tf.data.Dataset.from_generator(
    lambda: train_generator,
    output_types=(tf.float32, tf.float32))

Answer 2

您收到的错误很可能是因为您正在传递一个灰度输入，但是您的数据存储程序默认需要color_model = rgb。查看颜色模式这里的可能选项-(问题1)。此外，您还将一个灰度输入(h, w, 1)传递给一个具有ImageNet权重的模型，该模型在RGB输入(h, w, 3) -(问题2)上进行了训练。

IMAGE_SIZE = (224, 224)
IMG_SHAPE = IMAGE_SIZE + (1,)

.. .flow_from_directory(
    DATASET_DIR,
    color_mode='rgb', # < ------ ISSUE 1 
    target_size=IMAGE_SIZE,
    batch_size=BATCH_SIZE,
    class_mode="categorical",
    subset="training"
)

mobielNetV2 = ..applications.MobileNetV2(
                    input_shape=IMG_SHAPE, # < ------ ISSUE 2 
                    include_top=False, 
                    weights='imagenet')
...
...
ValueError: The input must have 3 channels; Received `input_shape=(224, 224, 1)` 

解决方案1

如果不关心ImageNet的预训练权重，那么您需要在设置中进行如下更改：

.. .flow_from_directory(
        DATASET_DIR,
        color_mode='grayscale',  # < ------ SOVLED ISSUE 1 
        ...
    )

mobielNetV2 = ...applications.MobileNetV2(
                 input_shape=IMG_SHAPE, 
                 include_top=False, 
                 weights=None) # < ------ SOVLED ISSUE 2

解决方案2

如果您想要ImageNet权重，而也希望您的灰度同时作为输入，那么有两个选项可供选择。

• 选项a：(推荐)

使用可训练层将灰度输入映射到RGB，然后使用ImageNet模型。

.. .flow_from_directory(
      DATASET_DIR,
      color_mode='grayscale', # < ------ SOVLED ISSUE 1 
       ...
   )
    
# rgb-imagenet-model
mobilenet = keras.applications.MobileNetV2(include_top=False, 
                                           weights='imagenet', 
                                           input_shape=(100, 100, 3))

# mapping grayscale to RGB 
input_tensor = keras.Input(shape=(100, 100, 1))
x = keras.layers.Conv2D(3, (3, 3), padding='same')(input_tensor)

# followed by image-net model 
x = mobilenet(x)

y = keras.layers.GlobalAveragePooling2D()(x)
y = ....
model = keras.Model(input_tensor, y)

• 选项b

使用不可训练的tensorflow操作将灰度输入映射到RGB，然后使用ImageNet模型。

 .. .flow_from_directory(
           DATASET_DIR,
           color_mode='grayscale', # < ------ SOVLED ISSUE 1 
           ....
         )

# rgb-imagenet-model
mobilenet = keras.applications.MobileNetV2(include_top=False, 
                                           weights='imagenet', 
                                           input_shape=(100, 100, 3))

# mapping grayscale to RGB 
input_tensor = keras.Input(shape=(100, 100, 1))
gray_to_rgb = keras.layers.Lambda(lambda x:tf.image.grayscale_to_rgb(x),
                                  output_shape=lambda x:x)
x = gray_to_rgb(input_tensor)

# followed by image-net model 
x = mobilenet(x)

y = keras.layers.GlobalAveragePooling2D()(x)
model = keras.Model(input_tensor, y)
``