Keras: rescale=1./255 vs preprocessing_function=preprocess_input - which one to use?

imagedatagenerator
keras preprocess
directoryiterator keras
keras resize image
keras data generator
keras applications
keras preprocess_input
image augmentation
Background

I find quite a lot of code examples where people are preprocessing their image-data with either using rescale=1./255 or they are using they preprocessing_function setting it to the preprocess_input of the respective model they are using within the ImageDataGenerator. First I thought using rescale=1./255 only works when dealing with a pretrained vgg16 model, but I keep seeing examples where it is being used with pre-trained resetnet50, inception etc. as well.

While the keras-blog (https://blog.keras.io/building-powerful-image-classification-models-using-very-little-data.html) uses this approach...

ImageDataGenerator(rescale=1./255, ...

... the Keras docs (https://keras.io/applications/) uses this approach:

from keras.applications.vgg19 import preprocess_input
ImageDataGenerator(preprocessing_function=preprocess_input, ...

I thought using the respective preprocess_input of the respective model I want to train is always superior to using the rescale=1./255 approach, since it will 100% reflect the preprocessing that has been used during training of the pretrained model.

Question

I need some clarification on when to use rescale=1./255 vs keras build-in preprocess_input of the respective model I want to train on when preprocessing images for transfer-learning. Does this only make a difference when using pretrained-models i.e. with loaded weights vs training from scratch?


First I thought using rescale=1./255 only works when dealing with a pretrained vgg16 model, but I keep seeing examples where it is being used with pre-trained resetnet50, inception etc. as well.

The reason that is done is because you need to NORMALIZE your input. Normally the formula for min-max normalization is

Which is the equivalent of doing

1./255

Since the pixel values of the image will be between 0 and 1

The reason for normalizing the input has to do with numerical stability and convergence (technically you do not need it, but with it, the neural network has a higher chance of converging and the gradient descent/adam algorithm is way more likely to be stable)

As per Does this only make a difference when using pretrained-models i.e. with loaded weights vs training from scratch? No, it is not linked to pretrained models only, it is a common technique when using certain algorithms in machine learning (neural networks being one of them).

If you are interested on REALLY understanding what goes on behind all this and why it is so important to normalize, I strongly recommend you to take the Andrew Ng course on machine learning

Image data preprocessing, I have a new thinking on this. I think it maybe ok to use a different but reasonable preprocessing (such as 1/255.) in the context of transfer learning (pre-training),  Obviously when you directly used a pre trained model for inference, you have to use preprocess_input: for example I tried to use resnet50 on the kaggle dogs-vs-cats dataset, with rescale=1./255 it returns the index 111 (nematode, nematode worm, roundworm) as the most likely class for all images, whereas with preprocess_input it mostly returns indices corresponding to dogs and cats as expected.


I had similar questions, and after running the small experiments below, I think you need to always use preprocess_input when using pre trained models, and use rescale when training from scratch.

Obviously when you directly used a pre trained model for inference, you have to use preprocess_input: for example I tried to use resnet50 on the kaggle dogs-vs-cats dataset, with rescale=1./255 it returns the index 111 (nematode, nematode worm, roundworm) as the most likely class for all images, whereas with preprocess_input it mostly returns indices corresponding to dogs and cats as expected.

Then I tried to use resnet50 with include_top=False, frozen weights from imagenet, one GlobalAveragePooling2D layer and one final dense sigmoid layer. I trained it with Adam on 2000 images of the kaggle dogs-vs-cats dataset (and I used 1000 images as validation). Using rescale it does not manage to fit anything after 5 epochs, it always predict the first class (though strangely the training accuracy is 97%, but when I run evaluate_generator`` on the training images, the accuracy is **50%**). But withpreprocess_input, it achieves **98%** accuracy on the validation set. Also note that you do not really need the images to be of the same dimensions as the trained models, for example if I use 150 instead of 224, I still get a **97.5%** accuracy. Without any rescaling orpreprocess_input`, I got a 95% validation accuracy.

I tried the same thing with vgg16, with rescaling it does manage to fit, but to 87% vs 97% using preprocess_input, and 95% without anything.

Then I trained a small conv network from scratch with 10 epochs, without anything or using resnet50 preprocess_input, it does not fit at all, but with rescaling I got a 70% validation accuracy.

Why is input preprocessing in VGG16 in Keras not 1/255.0, I saw this process in Keras blog, and your implementation. But I don't understand why? def preprocess_input(x, dim_ordering='default'): if  datagen = ImageDataGenerator(rescale=1./255) I'm getting this error: TypeError: __init__() got an unexpected keyword argument 'rescale' Provided that I'm using Keras v.1.0.3 with a Tensorflow backend. I'm using TensorFlow v.1.0.0. Using the latest version of Keras will cause me the issue below, which is why I'm avoiding using the latest version:


If you are using Transfer Learning in a way where you only leveraging the network structure but retraining the entire network (may be starting with leveraged weights), you may choose to setup your own preprocessing strategy. Which means you can scale by diving by 255.0 or use preprocess_input or even a custom implementation of preprocess.

If you are using Transfer Learning where you are not retraining the entire network but replacing the last layer with a few fully connected dense layers, then it is strongly recommended to use the preprocess_input associated with the network you are training. This is because the weights associated with layers which you are not training are accustomed to a specific preprocessing step. For example, if you look at preprocess_input for InceptionResNetV2 and follow the codepath to _preprocess_numpy_input, it doesn't normalize the image in every case, only when mode is "tf" or "pytorch". So if you trained an InceptionResNetV2 and normalized the image by diving by 0, it might not train the classifier the way you intended to.

Why we have to rescale by 1. / 255 · Issue #1 · Arsey/keras-transfer , It appears that after this pull: #9273 Is it now impossible to use the ImageDataGenerator with from keras.applications.xception import preprocess_input 19 color_mode = 'rgb', ---> 20 batch_size = 1024)) # one big batch Preprocessing, ImageDataGenerator, recent changes to it and cropping #9706. # Returns Two data generators, one for training data and one for test data """ train_datagen = ImageDataGenerator(dtype='float32', preprocessing_function=preprocess_input) train_generator = train_datagen.flow_from_directory( args.train_path, target_size=(224, 224), color_mode='rgb', batch_size=args.batch_size, class_mode='categorical', shuffle=True, interpolation='custom_imagenet') val_datagen = ImageDataGenerator(dtype='float32', preprocessing_function=preprocess_input) val_generator = val


ImageDataGenerator now longer allows preprocessing_functions , Overview · decode_predictions · preprocess_input vertical_flip=False, rescale=​None, preprocessing_function=None, If None or 0, no rescaling is applied, otherwise we multiply the data by the value provided (after The function should take one argument: one image (Numpy tensor with rank 3), and should output a  Thanks for reporting this! Fix is here: c37ba69 Note however that this fix will only work if you are running the latest version of the core Keras library (>= 2.1.2).


tf.keras.preprocessing.image.ImageDataGenerator, ImageDataGenerator( rescale=1/255., shear_range=0.1, zoom_range=0.1, horizontal_flip=True, preprocessing_function=preprocess_input ) 6.3.6 イテレータを  It defaults to the image_data_format value found in your Keras config file at ~/.keras/keras.json. If you never set it, then it will be "channels_last". validation_split: Float. Fraction of images reserved for validation (strictly between 0 and 1). dtype: Dtype to use for the generated arrays. Examples. Example of using .flow(x, y):


keras.preprocessing.image.ImageDataGenerator Python Example, To load the training and test images we therefore use a image data generator. Returns Two data generators, one for training data and one for test data preprocessing_function=preprocess_input) train_generator 1 datagen = ImageDataGenerator(rescale=1. / 255) # build the VGG16 network model = applications. Hello, I am using Python 3.6.9 - Anaconda (Windows), keras 2.2.4 and tensorflow 1.10.0, trying to make a binary classifier using partial data from celebA dataset. My files are divided as following: