# CS461 Recitation 12 - Homework 5 (Image Blending and Feature Spaces)

_Daize Dong, Rutgers University, Dec 1, 2025_

## Overview

This [homework](https://firner.com/presentations/cs461/homework5/) won't ask you to implement anything serious, just run some tests and make some graphs. The goal is to get you to think about feature extraction and feature spaces by interacting with them. For lecture 24, there were a large number of trained models. Let's take the 50 epoch ResNet model and the RBF kernel SVM trained with its feature vector and do some experiments.

## Setup

You still need to download the following files from the course website:

1. **Pretrained model**: [Download](https://firner.com/presentations/cs461/homework5/resnet_digits_50epochs.pyt) and place it under `models/`
2. **Trained SVM pickle file**: [Download](https://firner.com/presentations/cs461/homework5/svm_resnet_digits_rbf_gamma_0_01_C_100.pkl) and place it under `models/`
3. **Example digit images**:
    - First twenty [test](https://firner.com/presentations/cs461/homework5/20_test_digits.tar) images: Place under `data/test/`
    - First twenty [training](https://firner.com/presentations/cs461/homework5/20_train_digits.tar) images: Place under `data/train/`

## Usage

The main script `hw5_template.py` blends two digit images and runs both the ResNet DNN and SVM classifiers on the blended sequence. Here's the basic usage:

```bash
python hw5.py \
    --first data/test/example_test_digit_1.png \
    --second data/test/example_test_digit_2.png \
    --num-steps 50 \
    --load-dnn models/resnet_digits_50epochs.pyt \
    --load-svm models/svm_resnet_digits_rbf_gamma_0_01_C_100.pkl
```

To plot scores, use the `--plot-svm-classes` and `--plot-dnn-classes` flags. The script will automatically identify the two input digit classes and plot their scores:

```bash
python hw5.py \
    ...
    --plot-svm-classes \
    --plot-dnn-classes
```

To save numpy arrays, blended images, and figures, use the `--save-blended-dir` argument:

```bash
python hw5.py \
    ...
    --save-blended-dir results/blend
```
```

## Deliverables

> **IMPORTANT NOTE**
> - This homework requires creating and submitting multiple plots/figures. Make sure to save generated plots for your submission.
> - For Questions 4 and 5, you need to automate the search through **ALL** possible pairs of digits to find classification failures. Do not manually test just a few pairs.

This homework asks for answers to two open-ended questions and for several figures.

### Question 1 (20 points)

> The first question should be done before loading any numbers.

Before loading any numbers (or reading ahead), how do you think the SVM and residual network will classify the blended digits? Will they guess wildly when presented an image that is an equal blend of two different digits? Explain, in a few sentences.

### Question 2 (20 points)

Load a few pairs of digits to see if your intuition is correct or incorrect. If your supposition was correct, how could you verify your thought process? If your supposition was incorrect, where do you think you erred?

### Question 3 (20 points)

Choose two different digits in either the training or the testing set and get their blended results. **Plot the scores from the SVM and the DNN for the two correct classes** (with score on the y axis and blend step on the x axis). You will need to modify the code to see all of the correct values.

**Deliverable**: Two plots (one for SVM scores, one for DNN scores) showing the scores for the two correct classes across all blend steps.

### Question 4 (20 points)

Search for two digits where either the DNN or the SVM (or both) predict a third, nonpresent character. Provide the two digits (testing or training set and their index) and save an image of the blended figure where the SVM and/or DNN hallucinated a new digit. **Also plot the scores vs blend step of the classifiers for the two correct classes and the incorrect third class.**

**Deliverables**:

- Two plots (SVM and DNN) showing scores for the two correct classes and the incorrect third class
- Saved image of the blended digit where hallucination occurs
- Documentation of which digits (set and index) were used

### Question 5 (20 points)

> You should write a script to automate this search.

Are there any pairs of digits belonging to the same class where either the DNN or SVM (or both) predict a different character while they are being blended? If you find such a case, save the image where the SVM and/or DNN hallucinated a new digit.

**Deliverable**: Saved image(s) of blended digits where hallucination occurs for same-class pairs (if found).

### Extra Credit (20 points)

Write your own dataset of 10 digits and test classification with the SVM and DNN. You will need to convert your images to white on black, and scale to 28x28 pixels. Are any misclassified?
