CS671. Computer Vision. Segmentation. Lecture 10

1.

Dr. Juan Shan
Department of Computer Science
Seidenberg School of CSIS
Pace University
©CS671, Juan Shan
1

2.

Image segmentation is the operation of partitioning an
image into a collection of connected sets of pixels.
©CS671, Juan Shan
2

3.

©CS671, Juan Shan
3

4.

Image segmentation is the operation of partitioning
an image into a collection of connected sets of
pixels
Segmentation criteria: a segmentation is a partition
of an image I into a set of regions S satisfying:
1.
2.
3.
4.
Si = S
Si Sj = , i j
Si, P(Si) = true
P(Si Sj) = false,
i j, Si adjacent Sj
©CS671, Juan Shan
Partition covers the whole image.
No regions intersect.
Homogeneity predicate is
satisfied by each region.
Union of adjacent regions
does not satisfy it.
4

5.

What defines an object?
Subjective problem, but has been well-studied.
“What is interesting and what is not” depends on
the application.
There is no single solution to this problem.
©CS671, Juan Shan
5

6.

All we have to do is to define and implement the
similarity predicate.
What do we want to be similar in each region?
Is there any property that will cause the regions to
be meaningful objects?
Example approaches:
Histogram-based
Clustering-based
Region growing
Morphological
Contour-based
Deep convolutional neural networks
©CS671, Juan Shan
6

7.

How many pixels belong to the eagle
in this image?
This type of question can be answered
by looking at the histogram.
What about color images?
©CS671, Juan Shan
7

8.

How many modes are there?
Solve this by reducing the number of intensities to K
and mapping each pixel to the closest intensity.
Here’s what it looks like if we use two intensities.
©CS671, Juan Shan
8

9.

Besides histogram, clustering method can help to choose
the representative intensities or colors
K-means algorithm:
Partition all samples into K clusters
Randomly select the initial centers, and then update the
centers iteratively to minimize the within-cluster variation
Unsupervised
©CS671, Juan Shan
9

10.

K-means clustering of color.
©CS671, Juan Shan
10

11.

K-means clustering of color.
©CS671, Juan Shan
11

12.

Clustering can also be used with other features (e.g.,
texture) in addition to color.
Original Images
©CS671, Juan Shan
Color Regions
Texture Regions
12

13.

Pros:
Simple, fast to compute
Unsupervised learning,
converges to local
minimum of withincluster squared error
Cons:
Setting K?
Sensitive to initial centers
Sensitive to outliers
Detects spherical clusters
©CS671, Juan Shan
Adapted from Kristen Grauman
13

14.

Problem: histogram-based and clustering-based
segmentation using intensity/color/texture can
produce messy noisy regions. (Why?)
No clean separation between objects
How can these be improved?
©CS671, Juan Shan
14

15.

Region growing algorithm starts with one pixel of a
potential region and try to grow it by adding
adjacent pixels till the pixels being compared are
too dissimilar.
The first pixel selected can be just the first
unlabeled pixel in the image or a set of seed pixels
can be chosen (manually or automatically) from the
image.
We need to define a measure of similarity between a
pixel and a set of pixels as well as a rule that makes a
decision for growing based on this measure.
©CS671, Juan Shan
15

16.

Usually a statistical test is used to decide which
pixels can be added to a region.
Let R be the region with N pixels so far and p be a
neighboring pixel with gray level y.
ത and scatter S2 (sample variance)
Define the mean