Assignment 02

• Objectives
• Part 1: Modifying your image loader
• Part 2: Implementing Filters
• Part 3: Written Questions
• Submission
• Grading

You will extend your basic image loading program to be able to load images, process them with certain image processing filters, and then save the filtered image to a file.

Please click here to create your repository.

Objectives

This assignment is designed to teach you techniques that relate to:

• Color spaces and representations of the image range space.
• Processing color spaces to provide adjustments common to how images are displayed.
• Implementing these adjustments through rescaling filters.
• Processing images related to a signal processing framework.
• Implementing a basic resizing filter that relies on signal processing of regions of data to control for sampling artifacts.

Part 1: Modifying your image loader

Note that this repository does not include any default code to start with. I have distributed a few new image files to test with (although you can and should test with some of the previous images as well).

Starting with the code from your previous assignment, you should modify your code so that you can both load and also save an image. To do so, you should implement the basic file I/O to write a PPM file. You should also modify your main.cpp to give the user the option to save an image to a filename that they specify (specifically, your executable should accept parameters for both a filename to read from and a filename to write to).

Part 2: Implementing Filters

Next, you will modify your code to support two types of image processing operations:

• Rescaling, by adjusting the displayed colors on a per pixel basis. In particular, the user must be able to adjust the gain, bias, and gamma of the displayed image.

• Resizing, by producing and displaying the image at a different resolution and using signal processing concepts to reconstruct the input image and control for artifacts before saving the resulting output image.

Both of these filters will take a collection of parameters, and the user should be able to adjust these parameters while the program is executing, to dynamically set them before applying the filter. You are encouraged to use whatever interface you like for this (e.g. keyboard and mouse commands), but please make sure your README documents both how to run and how to use your program.

After adjusting the image using a combination of the above filters, the user should then be able to save the modified image to the aforementioned filename.

Details on Rescaling

Your rescaling filter should modify the resulting RGB values of the input data. It is most straightforward to think about these filters processing data in the range \([0,1]\), so you may want to convert how your image class from Assignment 01 stores the underlying data. After specifying the gain, bias, and gamma, the user should be able to scale all color channels.

The easiest method to do this is compute a scale value that you will multiply each channel with separately. This scale value should be computed based on the luminance, \(L\), of the pixel. There are a variety of equations that one could use to go from \(RGB\) to \(L\), but for this assignment we will use one of the simplest:

This exact equation is somewhat different weights from the Y channel in YUV color or the B in HSB, but provides a good approximation to how humans perceive intensity from color.

After computing the luminance, you can use the gain, bias, and gamma to compute an updated luminance, \(L'\) (in particular \(L' = (\texttt{gain}*L + \texttt{bias})^\texttt{gamma}\). You can then compute your scale value by \(\texttt{scale} = L'/L\).

After computing the scale, you can update the RGB values by multiplying each channel by it. Note, this might produce values outside of the range \([0,1]\). In these situations you should clamp your values back into the appropriate range. If you fail to clamp the data, you will produce a variety of visual artifacts (which might be fun to test with, but you will be penalized if you do not correct them!).

Details on Resizing

Resizing is a bit more complicated because resizing the image on the fly requires reallocating an SDL_Texture. Instead, to satisfy the minimum requirements for the assignment, you can specify a combination of width and/or height for the output image as a command line parameter. For example, I ran my sample program using:

$ ./prog02 [NC] [NR] [input] [output]

Where [NC] and [NR] are the number of columns and rows in the output image, [input] is the input filename, and [output] is the output filename.

To resize, I first load the input image from file. I next initialize a second image that I will display. I then populate the second image using the “inverse” approach described in class, computing the color for each output pixel using a reconstruction filter. I treated this filter as discrete-to-continuous convolution filter, where I let the user adjust the radius of the smoothing kernel. When filtering, it should be applied to each of the \(R\), \(G\), and \(B\) channels separately.

Typically, when decreasing the size of the image, you will want to do some amount of smoothing to account for high frequency features being impossible to represent with fewer samples. For increasing the size of the image, some amount of smoothing can be used to reduce the box-like artifacts from nearest neighbor interpolation. You are encouraged to experiment with different types of filters and mechanisms for allowing the user to minimize artifacts, for example, you may want to let the user adjust the size of the smoothing kernel and recomputing the output image while the program is running. To receive full credit for artifact mitigation, you must consider both the case of enlarging and shrinking the image and document your solution to artifact removal in the README.

You will run into an implementation edge case that you must also handle and document precisely how you handled it. This case occurs when you are working with a pixel near the boundary of the image. In the cases where pixels are close to the boundary, a convolution kernel centered at that pixel will extend beyond the image extents. To address this, you must implement a boundary condition. There were three different ways discussed in class, and you are welcome to pick any of the three.

Part 3: Written Questions

Please answer the following written questions. You are not required to typeset these questions in any particular format, but you may want to take the opportunity to include images (either photographed hand-drawings or produced using an image editing tool).

These questions are both intended to provide you additional material to consider the conceptual aspects of the course as well as to provide sample questions in a similar format to the questions on the midterm and final exam. Most questions should able to be answered in 100 words or less of text.

Please create a commit a separate directory in your repo called written and post all files (text answers and written) to this directory.

1. What is a pixel? How big is a pixel? Both of these questions have multiple answers, briefly explain yours.

2. 3 × 3 convolution kernels can create a variety of effects. Consider the following three kernels. Briefly describe the output image that is produced as a result of convolution with each kernel (you may assume each are scaled as necessary):

   a. \(H_a = \begin{bmatrix} 0 & 1 & 0 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \end{bmatrix}\)

   b. \(H_b = \begin{bmatrix} 1 & 2 & 1 \\ 1 & 2 & 1 \\ 1 & 2 & 1 \end{bmatrix}\)

   c. \(H_c = \begin{bmatrix} 0 & -1 & 0 \\ -1 & 4 & -1 \\ 0 & -1 & 0 \end{bmatrix}\)

3. Given an image \(I\) of \(200 \times 50\), and a kernel \(K\) of size \(5 \times 5\), how many multiplications are required to compute \(K \star I\)? Be sure to state your boundary condition.

4. Draw and label a diagram of the HSV color space. Include a brief description of each variable, its role in the final color, and a possible numeric range.

5. In rescaling images, individually adjusting only gain or bias typically is not sufficient to improve the image. Explain why we need both. In particular, discuss the resulting effects on the image when adjusting gain vs. bias.

Submission

• You should use git to submit all source code files and a working CMakeLists.txt. Do not submit any build files, binary files, or otherwise. The expectation is that your code will be graded by cloning your repo and then executing:

$ mkdir build
$ cd build
$ cmake ..
$ make

Your code must compile on the CS lab machines, in particular we will test on cambridge. Code that does not compile in this way will be given an automatic zero. You will be given one “warning” for the first instance during the semester that it does not compile, but after a zero will occur. If you are working on a different environment, please log into cambridge and test your code before submitting.

• Make sure that this build process produces an executable named prog02. You will need to edit CMakeLists.txt accordingly.

• Please provide a README.md file that provides a text description of how to run your program and any command line parameters that you used. Also document any idiosyncrasies, behaviors, or bugs of note that you want us to be aware of.

• To summarize, my expectation is that your repo will contain:

  1. A README.md file
  2. Answers to the written questions in a separate directory named written
  3. A CMakeLists.txt file
  4. All .cpp and .h files that you authored and are necessary to compile your code.
  5. (Optionally) any other test images that you want.

Grading

Deductions

Point Breakdown of Features

Extra Credit

Implementing features above and beyond the specification may result in extra credit, please document these in your README.