Using Ivy ResNet

Use the Ivy ResNet model for image classification.

Installation

Since we want the packages to be available after installing, after running the first cell, the notebook will automatically restart.

You can then do Runtime -> Run all after the notebook has restarted, to run all of the cells.

Make sure you run this demo with GPU enabled!

!pip install -q ivy
!git clone https://github.com/unifyai/models.git

# Installing models package from cloned repository! 😄
!cd models/ && pip install .

!python3 -m pip install torchvision

exit()

Imports

import ivy
import torch

Data Preparation

Prepare the set of labels

To show the predicted category, we download the labels associated with the pretrained weights. The labels are then loaded into a Python list.

!wget https://raw.githubusercontent.com/pytorch/hub/master/imagenet_classes.txt
with open("imagenet_classes.txt", "r") as f:
    categories = [s.strip() for s in f.readlines()]

Load the image example 🖼️

!wget https://raw.githubusercontent.com/unifyai/models/master/images/cat.jpg
filename = "cat.jpg"
# Preprocess torch image
from torchvision import transforms
from PIL import Image

preprocess = transforms.Compose([
    transforms.Resize(256),
    transforms.CenterCrop(224),
    transforms.ToTensor(),
    transforms.Normalize(
    mean=[0.485, 0.456, 0.406],
    std=[0.229, 0.224, 0.225]
)])
torch_img = Image.open(filename)
torch_img = preprocess(torch_img)
torch_img = torch.unsqueeze(torch_img, 0)

Visualise image

from IPython.display import Image, display
display(Image(filename))

Model Inference ResNet34

Initializing Native Torch ResNet34

from torchvision.models import resnet34, ResNet34_Weights

torch_resnet_34 = resnet34(weights=ResNet34_Weights.IMAGENET1K_V1).to("cuda")
torch_resnet_34.eval()

Initializing Ivy ResNet34 with Pretrained Weights ⬇️

The model is then initialized with the Pretrained Weights when pretrained=True 🔗.

from ivy_models.resnet import resnet_34

ivy_resnet_34 = resnet_34(pretrained=True)

Compile the forward pass for efficiency.

ivy.set_backend('torch')

img = ivy.asarray(torch_img.permute((0, 2, 3, 1)), dtype="float32", device="gpu:0")
ivy_resnet_34.compile(args=(img,))

Use the model to classify your images 🚀

For comparison, both results from Torch ResNet34 and Ivy ResNet34 are shown below.

  1. Torch ResNet34
torch_output = torch.softmax(torch_resnet_34(torch_img.cuda()), dim=1)
torch_classes = torch.argsort(torch_output[0], descending=True)[:3]
torch_logits = torch.take(torch_output[0], torch_classes)

print("Indices of the top 3 classes are:", torch_classes)
print("Logits of the top 3 classes are:", torch_logits)
print("Categories of the top 3 classes are:", [categories[i] for i in torch_classes])
Indices of the top 3 classes are: tensor([282, 281, 285], device='cuda:0')
Logits of the top 3 classes are: tensor([0.8507, 0.1351, 0.0069], device='cuda:0', grad_fn=<TakeBackward0>)
Categories of the top 3 classes are: ['tiger cat', 'tabby', 'Egyptian cat']
  1. Ivy ResNet34
output = ivy.softmax(ivy_resnet_34(img))  # pass the image to the model
classes = ivy.argsort(output[0], descending=True)[:3]  # get the top 3 classes
logits = ivy.gather(output[0], classes)  # get the logits

print("Indices of the top 3 classes are:", classes)
print("Logits of the top 3 classes are:", logits)
print("Categories of the top 3 classes are:", [categories[i] for i in classes.to_list()])
Indices of the top 3 classes are: ivy.array([282, 281, 285], dev=gpu:0)
Logits of the top 3 classes are: ivy.array([0.85072654, 0.13506058, 0.00688287], dev=gpu:0)
Categories of the top 3 classes are: ['tiger cat', 'tabby', 'Egyptian cat']

Model Inference ResNet50

Initializing Native Torch ResNet50

from torchvision.models import resnet50, ResNet50_Weights

torch_resnet_50 = resnet50(weights=ResNet50_Weights.IMAGENET1K_V2).to("cuda")
torch_resnet_50.eval()

Initializing Ivy ResNet50 with Pretrained Weights ⬇️

The model is then initialized with the Pretrained Weights when pretrained=True 🔗.

from ivy_models.resnet import resnet_50

ivy_resnet_50 = resnet_50(pretrained=True)

Compile the forward pass for efficiency.

ivy_resnet_50.compile(args=(img,))

Use the model to classify your images 🚀

For comparison, both results from Torch ResNet50 and Ivy ResNet50 are shown below.

  1. Torch ResNet50
torch_output = torch.softmax(torch_resnet_50(torch_img.cuda()), dim=1)
torch_classes = torch.argsort(torch_output[0], descending=True)[:3]
torch_logits = torch.take(torch_output[0], torch_classes)

print("Indices of the top 3 classes are:", torch_classes)
print("Logits of the top 3 classes are:", torch_logits)
print("Categories of the top 3 classes are:", [categories[i] for i in torch_classes])
Indices of the top 3 classes are: tensor([282, 281, 285], device='cuda:0')
Logits of the top 3 classes are: tensor([0.3429, 0.0408, 0.0121], device='cuda:0', grad_fn=<TakeBackward0>)
Categories of the top 3 classes are: ['tiger cat', 'tabby', 'Egyptian cat']
  1. Ivy ResNet50
output = ivy.softmax(ivy_resnet_50(ivy.asarray(img)))  # pass the image to the model
classes = ivy.argsort(output[0], descending=True)[:3]  # get the top 3 classes
logits = ivy.gather(output[0], classes)  # get the logits

print("Indices of the top 3 classes are:", classes)
print("Logits of the top 3 classes are:", logits)
print("Categories of the top 3 classes are:", [categories[i] for i in classes.to_list()])
Indices of the top 3 classes are: ivy.array([282, 281, 285], dev=gpu:0)
Logits of the top 3 classes are: ivy.array([0.34288213, 0.04077019, 0.0121203 ], dev=gpu:0)
Categories of the top 3 classes are: ['tiger cat', 'tabby', 'Egyptian cat']