HW 11.

• Running the models may take minutes. This HW takes ~30 min to complete in computational time, so make sure you don't start it 1 hour before it is due.

• Tasks 2-4. should be done using the sklearn library, the last is a pure TensorFlow (Keras is part of TensorFlow) example.

  • Use tf.keras instead of the standalone keras package

• The example notebook was run in Google COLAB without any package installation. I advise you to use Google COLAB with a GPU instance for the last task.

• Where not asked otherwise, use the default settings for the model.

• You may try running the models using more CPU cores to speed the training (sklearn supports for most of the models with a parameter, usually n_jobs).

1. Load the CIFAR 10 dataset from the tf.keras.datasets API and train a LogisticRegression model on the dataset and predict all test outcomes with the sklearn API

• Create an image grid visualization of randomly selected images (9, 16) with labels.
• Preprocess the dataset for sklearn, scale the pixels [0-1], and also flatten each example to a vector.
• Use the multi_class='multinomial' option, describe what it means.
• Plot the ROC curves and AUC scores on the same figure for each class.
• Calculate the accuracy of the classifier on the test set.

Hint:

• from sklearn.preprocessing import LabelBinarizer might be useful for you.

2. Train an SGDClassifier regression model on the dataset and predict all the test outcomes with the sklearn API.

• Select an appropiate loss for this task, explain what this means.
• Time is precious, run the classifier paralell on many jobs.
• Plot the ROC curves and AUC scores on the same figure for the test set.
• Calculate the accuracy of the classifier.
• Describe the above model with your own words, how is it different than the logistic regression model?

3. Train a RandomForest classifier

• Plot the ROC curve with AUC scores on the test set.
• Calculate accuracy of the classifier on the test set.

4. Train an multi layer perceptron classifier

• use the MLPClassifier from sklearn
• Set its parameter to max_iter = 30 or if you have time, set it for at least 100. After 30 iterations the model does not converge but gives reasonable predictions (with default parameters).
• Plot the ROC curves with AUC scores for the test set.
• Calculate the accuracy of the model on the test set.

5. Train a ResNet50 CNN model on the dataset, utilize ImageNet pre-trained weights and fine-tune for at least 3 epochs:

• training for 3 epochs should be enough to prove that this model is superior compared to others, train longer to explore the possibilities of the model

Convert the dataset:

train_dataset = tf.data.Dataset.from_tensor_slices((x_train, y_train))
train_dataset = train_dataset.shuffle(buffer_size=1024).batch(32)

test_dataset = tf.data.Dataset.from_tensor_slices((x_test, y_test))
test_dataset = test_dataset.batch(32)

Hints:

• loading a pretrained model and letting its parameters be tunable

backbone = tf.keras.applications.YOUR_MODEL_OF_CHOICE # set include_top = False to get rid of the dense layers
backbone.trainable = True # set if you want to fine-tune the pretrained weights too, otherwise set to False

• defining your custom model with the pretrained backbone

# YOUR_MODEL_OF_CHOICE here is ResNet50 as per the task description.

# Functional TensorFlow API
def my_own_model():
  inp = tf.keras.layers.Input(shape=(32, 32, 3))
  x = tf.keras.applications.YOUR_MODEL_OF_CHOISE.preprocess_input(inp)

  x = backbone(x)
  # Here comes some more layers
  # and flattening where needed!
  out = # layer outputting the specified number of classes
        # with or without a softmax activation, later on
        # the choice of the loss depends on this
  model = tf.keras.models.Model(inputs=inp, outputs=out)
  return model