Building a Logistic Regression Classifier in PyTorch

by

Logistic regression is a widely used statistical method for predicting the probability of an event, making it a valuable tool for classification tasks in machine learning, artificial intelligence, and data mining.

At its core, logistic regression applies a sigmoid function to the output of a linear function. In this article, you’ll learn how to build a logistic regression classifier using the popular MNIST dataset to train and test your model.

After reading this article, you will understand:

  • How to implement logistic regression in PyTorch and its real-world applications.
  • How to load and analyze datasets from torchvision.
  • How to construct and train a logistic regression classifier for image datasets.

Let’s get started!

Overview

This tutorial is divided into four key parts:

  1. Introduction to the MNIST Dataset
  2. Loading the Dataset into DataLoader
  3. Building the Model with nn.Module
  4. Training the Classifier

Introduction to the MNIST Dataset

In this project, you will train and test a logistic regression model using the MNIST dataset, which contains 60,000 training images and 10,000 testing images of handwritten digits. It is one of the most referenced datasets in the machine learning field.

The MNIST dataset is integrated with PyTorch, making it easy to load with the following code:

import torch
import torchvision.transforms as transforms
from torchvision import datasets

# Loading training data
train_dataset = datasets.MNIST(root='./data', 
                                 train=True, 
                                 transform=transforms.ToTensor(),
                                 download=True)

# Loading test data
test_dataset = datasets.MNIST(root='./data', 
                               train=False, 
                               transform=transforms.ToTensor())

This will download and extract the dataset to the specified directory.

To verify the number of training and testing samples, use the following snippet:

print("Number of training samples: ", len(train_dataset))
print("Number of testing samples: ", len(test_dataset))

You should see something like:

Number of training samples: 60000
Number of testing samples: 10000

Each sample consists of an image and a label. To inspect the first sample’s data type and size, use:

print("Data type of the first training sample: ", train_dataset[0][0].type())
print("Size of the first training sample: ", train_dataset[0][0].size())

This will output:

Data type of the first training sample: torch.FloatTensor
Size of the first training sample: torch.Size([1, 28, 28])

This means the first image is a 28×28 pixel grayscale image, represented as a tensor.

You can also check the labels of the first two samples:

print("Label of the first training sample: ", train_dataset[0][1])
print("Label of the second training sample: ", train_dataset[1][1])

The output should be:

Label of the first training sample: 5
Label of the second training sample: 0

Loading the Dataset into DataLoader

Typically, you won’t train the model directly with the dataset but rather through a DataLoader class, which facilitates data loading in batches. Here’s how you can set it up for the MNIST dataset:

from torch.utils.data import DataLoader

batch_size = 32
train_loader = DataLoader(dataset=train_dataset, batch_size=batch_size, shuffle=True) 
test_loader = DataLoader(dataset=test_dataset, batch_size=batch_size, shuffle=False)

Building the Model with nn.Module

Next, you will define your logistic regression model using PyTorch’s nn.Module. Here’s an example structure for your model:

import torch.nn as nn

class LogisticRegression(nn.Module):
    def __init__(self, n_inputs, n_outputs):
        super(LogisticRegression, self).__init__()
        self.linear = nn.Linear(n_inputs, n_outputs)

    def forward(self, x):
        y_pred = torch.sigmoid(self.linear(x))
        return y_pred

This model takes a 784-dimensional input vector (flattened from a 28×28 image) and outputs a value between 0 and 1 for each of the 10 classes (digits 0-9).

Training the Classifier

You will train this logistic regression model using the stochastic gradient descent optimizer with a learning rate of 0.001 and the cross-entropy loss function.

The code for the training loop is as follows:

import numpy as np
import torch.optim as optim

# Define loss function and optimizer
loss_fn = nn.CrossEntropyLoss()
optimizer = optim.SGD(log_regr.parameters(), lr=0.001)

epochs = 50
Loss = []
acc = []
for epoch in range(epochs):
    for i, (images, labels) in enumerate(train_loader):
        optimizer.zero_grad()
        outputs = log_regr(images.view(-1, 28*28))
        loss = loss_fn(outputs, labels)
        loss.backward()
        optimizer.step()
    Loss.append(loss.item())

    correct = 0
    for images, labels in test_loader:
        outputs = log_regr(images.view(-1, 28*28))
        _, predicted = torch.max(outputs.data, 1)
        correct += (predicted == labels).sum().item()

    accuracy = 100 * correct / len(test_dataset)
    acc.append(accuracy)
    print(f'Epoch: {epoch}. Loss: {loss.item():.4f}. Accuracy: {accuracy:.2f}%')

During training, you should see output reflecting the loss and accuracy for each epoch.

Visualizing Results

You can visualize the loss and accuracy using Matplotlib with the following code:

import matplotlib.pyplot as plt

# Plot the training loss
plt.plot(Loss)
plt.xlabel("Number of Epochs")
plt.ylabel("Total Loss")
plt.title("Loss Over Epochs")
plt.show()

# Plot the accuracy
plt.plot(acc)
plt.xlabel("Number of Epochs")
plt.ylabel("Total Accuracy")
plt.title("Accuracy Over Epochs")
plt.show()

Summary

In this tutorial, you learned how to build a logistic regression classifier using PyTorch. Specifically, you covered:

  • The setup of the MNIST dataset and its loading into DataLoader.
  • The implementation of a logistic regression model using nn.Module.
  • The training process and the evaluation of the model.

These concepts provide a solid foundation for applying logistic regression and other deep learning techniques using PyTorch. With continued practice and experimentation, you can effectively implement deep learning models for various tasks in your projects.

Leave a Comment