Model explainability

The growing availability of big data has increased the benefits of using complex models, forcing data scientists to choose between accuracy and interpretability of a model’s output. Adding explainability methods to models that are not easily understood helps:

Ensure transparency algorithmic decision-making
Identify potential bias in the training data
Give a deeper understanding of the process being modeled

These methods are able to give insight into why your model generates specific outputs; explainability algorithms are especially useful in highly regulated industries (ie, pinpoint the attributes that caused someone to be denied/approved a home loan). We'll demonstrate an anchor explainer in this notebook to better understand why a generic image detection model is creating certain outputs using the library seldon-core.

[ ]

#Note: if running in Operate First environment, you must first run the command `pipenv install --dev`

[ ]

import os
import tensorflow as tf
import matplotlib.pyplot as plt
import numpy as np

from tensorflow.keras.applications.inception_v3 import InceptionV3, decode_predictions
from tensorflow.keras.preprocessing.image import load_img, img_to_array
from alibi.explainers import AnchorImage
from seldon_core.seldon_client import SeldonClient

To begin, we will set up a SeldonClient, which is how we will communicate to the classifier model that was deployed to our cluster via the Seldon operator. This model, named image-default-classifier is deployed into the image-classifier namespace and uses gRPC transport via port 9001.

[2]

# gateway endpoint is model-name.namespace-name:port

sc = SeldonClient(gateway_endpoint="image-default-classifier.image-classifier:9001",\
                  grpc_max_send_message_length= 27 * 1024 * 1024,\
                  grpc_max_receive_message_length= 27 * 1024 * 1024)

[5]

# [WIP] serving model via gateway, current workaround is local model

#cat_predict = sc.predict(transport="grpc",\
#                         data=image_batch,\
#                         payload_type="tftensor",\
#                         gateway="ambassador",\
#                         client_return_type="dict")

[6]

# # [WIP] Reformat results, for served model
#cat_values = np.asarray(cat_predict.response['data']['tftensor']['floatVal'])
#shape = np.asarray([1,1000])
#cat_preds = cat_values.reshape(shape)

Next, we load and format the photo into a machine readable array.

[3]

def image_pipeline(image_name):  

    #Format custom picture
    resized_image = load_img(image_name, target_size=(299, 299))
    numpy_image = img_to_array(resized_image)
    image_batch = np.expand_dims(numpy_image, axis=0)
    
    model = InceptionV3(weights='imagenet')
    preds = model.predict(image_batch)
    
    #Display results
    label = decode_predictions(preds, top=3)
    plt.title(label[0])
    plt.imshow(resized_image)

[4]

image_name = '../data/cat.jpg'
image_pipeline(image_name)

The predict function returns the predictions via Softmax, which mean that the prediction can be translated as the probability that the image falls into one of the categories given in training. Even though we see this as a cat, the model gives a 100% probability that the image shows a pitcher. There are a few things to note here.

This is a photo I took of my own cat, so I can confidently know that the model has never seen the picture before. To a human eye, it is (hopefully) pretty obvious that the image is of a cat. However, oftentimes training data, especially images, does not accurately reflect photos that are taken by you or I. For example, the training photos may all have centered images in good lighting with nothing in the background, which is unrealistic for everyday images.
This is a Softmax output. The model only gives us the names of its best guesses, not how strong the prediction is. This is a subtle differentiation. The model believes that pitcher is definitely the best guess, but it could only be 1% certain in this prediction. To solve this, we could add new outcome metrics, or tweak the model.

Before we do any of that though, it could be useful for us to better understand how the model is making this prediction. We'll use something called an explainer to dive deeper.

Explainability

We will use a local version of the model to build an anchor explainer. This will show us what parts of the photo the model used in order to give the "pitcher" prediction.

[7]

def explainability_pipeline(image_name): 
    
    # format custom picture
    resized_image = load_img(image_name, target_size=(299, 299))
    numpy_image = img_to_array(resized_image)
    image_batch = np.expand_dims(numpy_image, axis=0)
    
    # set hyperparameters
    image_shape = (299, 299, 3)
    segmentation_fn = 'slic'
    kwargs = {'n_segments': 15, 'compactness': 20, 'sigma': .5}

    # load model
    model = InceptionV3(weights='imagenet')
    
    # explainer, from alibi library
    explainer = AnchorImage(model, image_shape, segmentation_fn=segmentation_fn, 
                            segmentation_kwargs=kwargs, images_background=None)
    
    cat_image = image_batch[0]
    explanation = explainer.explain(cat_image, threshold=.95, \
                                    p_sample=.5, tau=0.25)

    plt.imshow(explanation.anchor)

[8]

explainability_pipeline(image_name)

skimage.measure.label's indexing starts from 0. In future version it will start from 1. To disable this warning, explicitely set the `start_label` parameter to 1.

It looks as though the model uses primarily the background to create this prediction, so it comes as no surprise that the classification is wildly incorrect. With this information in mind, data scientists may decide to go back and create a more robust exploration of the data and model. It may be the case that the training data of cats has only solid backgrounds, different lighting, different color/hair length cat, or some other feature that caused this image to not be identified correctly.

Explainers will not fix the model itself. However, they are useful tools for data scientists to build well-architected models by exposing bias in training data and giving transparency to black-box models.

Try your own pets!

Import your own photo to the data folder. Change your-image-name in the cell below to match your image's name, and press run!

[9]

custom_image_name = '../data/your-image-name.jpg'
image_pipeline(custom_image_name)
explainability_pipeline(custom_image_name)

---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
<ipython-input-9-ad971adb8d54> in <module>
      1 image_name = '../data/your-image-name.jpg'
----> 2 image_pipeline(image_name)
      3 explainability_pipeline(image_name)

<ipython-input-3-f42d480a86fe> in image_pipeline(image_name)
      2 
      3     #Format custom picture
----> 4     resized_image = load_img(image_name, target_size=(299, 299))
      5     numpy_image = img_to_array(resized_image)
      6     image_batch = np.expand_dims(numpy_image, axis=0)

/opt/app-root/lib64/python3.8/site-packages/tensorflow/python/keras/preprocessing/image.py in load_img(path, grayscale, color_mode, target_size, interpolation)
    297       ValueError: if interpolation method is not supported.
    298   """
--> 299   return image.load_img(path, grayscale=grayscale, color_mode=color_mode,
    300                         target_size=target_size, interpolation=interpolation)
    301 

/opt/app-root/lib64/python3.8/site-packages/keras_preprocessing/image/utils.py in load_img(path, grayscale, color_mode, target_size, interpolation)
    111         raise ImportError('Could not import PIL.Image. '
    112                           'The use of `load_img` requires PIL.')
--> 113     with open(path, 'rb') as f:
    114         img = pil_image.open(io.BytesIO(f.read()))
    115         if color_mode == 'grayscale':

FileNotFoundError: [Errno 2] No such file or directory: '../data/your-image-name.jpg'

Contribute to this page