ELECENG 4061/7060 Assignment 1 2020

4061/7060 Image Sensors Processing

Assignment 1 [REVISED], March 2020

(Assignment Value: 17%)

Overview

This assignment is intended to give you some hands-on experience with manipulating image

data in MATLAB and the basic image manipulation concepts covered in the course up to the end

of Week 3.

In this assignment work you will be required to modify 4 functions which implement basic Bayer

image reconstruction, local contrast enhancement and two noise filtering steps.

Assignment Submission

Assignments are to be submitted as a ZIP archive file via the link on MyUni, The ZIP file should

contain the key MATLAB functions and a word or PDF document summarising your results with

comments on the performance of the processing steps employed (in 1A,1B-1 and 1B-2) and

written answers to questions (1C).

No routines from the image processing toolbox (such as histeq, medfilt2 conv2) etc may be used

in your assignment solutions. Using such routines will result in zero marks for that component of

the assignment work. You are also reminded that all assignment work, unless clearly

acknowledged and referenced, must be your own work.

Source Materials

All required materials can be found on MyUni via the Assignment-1 link.

You will also require a copy of MATLAB version 2015 or later to complete this assignment. All

students, including those affected by travel restrictions should be able to install a student

version of MATLAB for study purposes (see https://www.adelaide.edu.au/technology/your-

services/software/software-for-students). If you are unable to do so, please advise me via email

and try using the open source package OCTAVE instead (please note I cannot guarantee that the

supplied code will work without modification).

Do you need help or advice on your assignment?

If you need help in getting your code working or you have any questions about this assignment,

then try the following:

1. Ask a question at a tutorial or via the online discussion board

2. Get help at one of the advertised consultation times with either the tutor or me

3. Talk to me after lectures

4. contact me via email (but please try the other options first)

The first part of the tutorials session each week will be dedicated to discussing any questions

you may have on the assignment work.

ELECENG 4061/7060 Assignment 1 2020

Exercise 1A – ( 4% ) - Simple Bayer Image reconstruction

Your company is working on a suite of imaging modules for an upcoming product range

including some basic processing to eventually go on to an ultra-cheap digital camera. One of the

required modules, which you have been asked to look at, will need to convert an NxM Bayer

image into a full colour NxMx3 RGB image.

To help you get started you have been supplied with a test function make_bayer.m which

converts an NxMx3 image into an MxN monochrome Bayer image using the 2x2 Bayer pattern:

RED GREEN

GREEN BLUE

In effect, this function mimics the image capture process that occurs on a typical CCD array

(NOTE: the ordering of the pixels in this Bayer pattern may not be the same as used in the

example in Tutorial-1)

A second supplied function show_bayer.m allows you to display the red, green and blue

samples of an image generated by make_bayer.m.

STEP 1: Modify the function bayer_reconstruct.m such that given an MxN Bayer pattern image

(eg. created by make_bayer.m) the function returns a full colour NxNx3 estimate of the original

image data using the method presented in the slides in the Lecture 1 notes.

In short there are 4 situations for which you need to compute red or blue samples shown in

(a),(b),(c) and (d) below:

In case (a) the red middle sample is the average of the left and right samples and the blue the

average of the upper and lower samples, case (b) is very similar to (a) with the reds and blues

reversed. In case (c) we have the red sample and the blue can be calculated from the diagonal

average of the blues. Case (d) is very similar to (c) with the reds and blues reversed.

For the purposes of this assignment to reconstruct the missing green samples for cases (c) and

(d) above simply take the median of the above, below, left and right green samples (you may

use the inbuilt MATLAB function median() to do this). Do NOT use the far more complicated

conditional expression presented in the week 1 notes.

ELECENG 4061/7060 Assignment 1 2020

The reconstruction algorithm can thus be summarised as follows:

1. Extract the red, green and blue samples from the Bayer image and place them into their

correct positons in the NxMx3 array representing the full colour image (see the function

show_bayer.m for clues as to how to do this)

2. Reconstruct the red and blue channels for the four cases (a), (b), (c) and (d) shown

earlier. Note that the process for reconstructing the red and blue channels is basically

the same except one is shifted by one pixel across and down.

3. Reconstruct the green using the median of the four nearby samples.

Do not worry about correctly reconstructing points near the boundary (this will take too much

time). You may use either ‘for’ loops or MATLAB’s a:b:c indexing notation to implement your

solutions (recall the notes in tutorial 1). The important detail here is ensuring that you are

referring to the correct locations and colours when calculating the averages. If you get the

indexing wrong, then you will either see black holes in your solution or patches of incorrect

colour (eg. pink skies and orange trees).

You can test your reconstruction function using the supplied test script bayer_test.m

Note that in the supplied code fragments, the 3 colour channels are converted to doubles such

that pixel values are in the range 0.0 to 1.0. You should not need to deal with uint8 values.

STEP 2: Test out your solution using the supplied colour imagery (eg. lighthouse.jpg, lilly.jpg and

lift_colour.jpg) OR on colour imagery of your own choice. Examine the details of the resulting

image and compare it to the original image data. Make a special note of how the reconstruction

works near the edges of shapes in the imagery. Write up your results and observations and

include these in your short assignment report.

Comment: the main difficulty you’re likely to encounter with coding this Bayer reconstruction

function is in getting the indexing right (remember MATLAB uses row,column order indexing of

arrays) . I suggest you work on the blue channel first and then use this as the template for the

reconstructing the red channel. If in doubt you may find it useful to draw yourself a diagram and

use that to check your logic. A tell-tale sign that you have accidentally used the wrong indexing

somewhere is the presence of black holes or clear differences in colour in the reconstruction.

(if you happen to be one of the 8% of men or 0.5% of women who are colour blind then get

someone else to double check your output)

ELECENG 4061/7060 Assignment 1 2020

Input Image

20 40 60 80 100 120

10

20

30

40

50

60

70

80

90

BAYER Image

20 40 60 80 100 120

10

20

30

40

50

60

70

80

90

BAYER Image (in color)

20 40 60 80 100 120

10

20

30

40

50

60

70

80

90

Reconstructed Image

20 40 60 80 100 120

10

20

30

40

50

60

70

80

90

Above: An example of passing a small version of ‘peppers.png’ through a working version of

bayer_reconstruct.m. The results you will obtain will not be as good as shown in class as we are

using a very simple approach to estimate the missing colour values.

ELECENG 4061/7060 Assignment 1 2020

Exercise 1B – A Simple Image Enhancement Tool (total 10%)

You have been asked to complete work on another part of the simple image processing tool

which your company is developing but which is running behind schedule. Several of the key

processing steps still need to be implemented. These are contained within functions

local_histeq.m, alpha_trimmed.m and adap_med_filt.m.

Exercise 1B-1 – (3%) – Contrast Enhancement using Local Histogram

Equalisation

The supplied function local_histeq.m is intended to apply local histogram equalisation to a

supplied MxN monochrome image (values in range 0 to 255).

From the week 2 lecture notes histogram equalisation mapping m(k) between the old pixel

values and the new pixel values is defined as follows for pixels in the range 0 to 255:

?

?

?

?

?

?

?

?

?

−?

?

?

?

?

?

−?

?

?

?

?

?

=

?

?

=

=

H

HH

HiH

HiH

H

km

o

o

i

o

k

i

ofentry zero-nonfirst

theis and histogram theis whereotherwise,

)(

)(

of

entry zero-nonfirst theincluding and below for values

0

)(

255

0

0

where?

=

k

i

iH

1

)( is the cumulative sum of the histogram H (of the whole image) up to the kth and

oH is the number of elements in the first non-zero entry of H. element.

For local histogram equalisation we notionally perform the same operation at each point I(r,c)

using the histogram of its local neighbourhood. Taken at face value, this requires us to compute

many thousands of histograms per image to compute m(k) at each location and hence the new

mapping for each pixel I(r,c).

However, for local histogram equalisation we only use the mapping m(k) once at each pixel I(r,c)

so much of this calculation is unnecessary. For local histogram equalisation the above

expression can be rewritten as:

o

i

cr

o

crI

i

cr

eq

HiH

HiH

crI

−?

?

?

?

?

?

−??

?

?

??

?

?

=

?

?

=

=

255

0

,

),(

0

,

)(

)(

),(

Ignoring the oH term and noting that for some value k the expression ?

=

k

i

cr iH

1

, )( is simply the

number of elements in the local neighbourhood with intensity values k? , then for an n by n

pixel neighbourhood crN , about pixel I(r,c) this expression for local histogram equalisation can

be reduced to:

ELECENG 4061/7060 Assignment 1 2020

( )

2

),( ,

),(),(

),(

n

crIjiI

crI

N

Nji

eq

cr

?

?

?

?

That is, for each local neighbourhood we simply find the number of pixels of lesser or equal

value to the pixel I(r,c) and use this to compute the new value for I(r,c). There is no need to

compute any histograms or local mapping table m().

STEP 1: Using the above explanation, complete the function localhisteq(I,n) such that given an

image (I) and a size (n) for the n x n pixel neighbourhood it computes the simple estimate of the

local histogram equalisation shown above. Note that for pixels near the boundary the n x n

neighbourhood does not have to be centred on the pixel being processed.

STEP 2: Test your solution on the image data provided (this may require writing your own test

script based on localhisteq_test.m) and write up your results and any observations in your

assignment report. Note that even though this is a simpler calculation, your developed solution

may take some time to run.

Above: Example output from the test script ‘localhisteq_test.m’.

Exercise 1B-2 – (7%) – Spatial Filtering

STEP 1: (3%) Complete the implementation of the function alpha_trimmed(I,n,d1,d2) such that

it implements an n by n alpha trimmed mean filter similar to that described in class. You may

assume n is odd (ie. ,3,5,7 etc). Unlike the version given in the notes, this version should employ

ELECENG 4061/7060 Assignment 1 2020

two values d1 and d2 which are the number of pixels to remove from the start and the end of

the sorted list of elements before you calculate the average. In this way an image with a

different proportion of salt and pepper samples can be dealt with. The function sort() can be

used to greatly simplify your solution. Do not worry about pixels close to the boundary in your

solution.

valueslocal oflist sorted theis () where)(

)(

1

),('

2

2

1 1

,

21

2

siS

ddn

yxI

dni

di

yx?

−=

+=+−

=

Above: an example output from the test script for the Alpha trimmed filter.

STEP 2: (4%) Modify the function adap_med_filter(I) such that it implements a version of the

adaptive median filter described in lectures using 3x3 max/min filters and 3x3, 5x5 and 7x7

median filters. The pseudo-code for this is:

Calculate local max/min estimates over 3x3 neighbourhood.

Precompute the 3x3,5x5 and 7x7 median filtered images.

At each location (i,j) in the image:

If value at (I,j) is same as the local max/min ( and max > min ) then

If the 3x3 median is the same as local max min then

If 5x5 median same as local max min then

…etc up to a 7x7 filter…

else replace with 5x5 median value.

else replace with 3x3 median value.

else use the original pixel value.

To do this you will need to modify the functions med_filter(), max_filter() and min_filter()

contained in adap_med_filter.m . The max and min filters should be minor modifications of

your median filter solution. You may use the MATLAB commands sort(), min(), max() and

median() to assist in implementing the med, max and min and median filtering steps. It is

ELECENG 4061/7060 Assignment 1 2020

suggested you get your solution working for a 3x3 median first before adding the layers required

to use the 5x5 and 7x7 filters if required.

STEP 3: Test your solutions using the greyscale imagery and the following provided functions:

alpha_trimmed_test() - tests your solutions for step 1

adap_med_test() - tests your adaptive median solution

You can modify the given test files to use other imagery out of the examples given.

STEP 4: Write up your results and include various examples (not just the one in the test script).

Try modifying the tests to increase the level of noise introduced into the imagery and comment

on the performance your solutions.

Above: An example outputs from the Adaptive median filter test script.

ELECENG 4061/7060 Assignment 1 2020

Exercise 1C – (3%) – Written Questions

Write up written answers (in your own words) to the following questions and include them in

your assignment report (1-2 paragraphs for each question plus working):

1. (0.5%) A colour photograph has been under exposed and as a result contains very little

contrast. Suggest a way the contrast might be enhanced without noticeably affecting

the colour tones of the image. Is there more than one way this can be achieved?

2. (1%) A people tracking system is to be used to detect and count people passing through

the door of a supermarket to determine their busiest shopping times. The image

processing software you are using is only capable of detecting objects bigger than 8

pixels in height in the image data, and the camera uses a 512x512 array of pixels and a

lens with a field of view of 45 degrees. Assuming a person is, on average, 1.5 metres tall,

what is the furthest distance away from the camera a person might be detected?

3. (1%) A spatial filter is to be used to clean up an image containing both white noise and

salt and pepper noise. a) Comment on the effects of using either a mean or median filter

to do this and suggest an alternative filter which might work better. b) If a median filter

had been used to process this image, what proportion of the points in the NxN

neighbourhood could contain either salt or pepper noise before the filter fails?

4. (0.5%) A scanned image of a photograph contains a series of pixel thick light and dark

vertical artefacts similar to those shown on slide 19 of lecture 1, where the odd columns

of the image are slightly brighter than the even ones. Suggest a simple filter which could

significantly reduce the visibility of these stripes without significantly distorting the

picture.

NOTES / HINTS

• Test routines have been provided to you wherever possible. Please use them.

• No Image Processing (IP) toolbox functions are to be used in your solutions. If you are

unsure about whether a given function is from this toolbox use the ‘which’ command

(eg. ‘which im2double’). If the pathname includes the word \images\ it probably comes

from the IP toolbox.

• As stated earlier the MATLAB functions min() max() hist() linspace() sort() mean(), and

median() may be useful in your solutions. The function reshape() can be used to convert

to and from arrays and vectors with the same number of elements, and don’t forget and

2D array X can be made into a vector using X(:).

• MATLAB uses (row,column) ordering of arrays as opposed to (x,y) ordering. All

references to arrays in this assignment assume (row,column) ordering.

• Your written reports should be around 6-8 pages and form a key part of the assessment.

Include example results and comments for exercises 1A and 1B based on images from

the supplied set or on other images of your choice as well as 1C.