Course Syllabus
1.Color
2.Camera models, camera calibration
3.Advanced image pre-processing
•Line detection
•Corner detection
•Maximally stable extremal regions
4.Mathematical Morphology
•binary
•gray-scale
•skeletonization
•granulometry
•morphological segmentation
•Scale in image processing
5.Wavelet theory in image processing
6.Image Compression
7.Texture
8.Image Registration
•rigid
•non-rigid
•RANSAC
References
•Books:
•Chapter 11, Image Processing, Analysis, and MachineVision, Sonka et al
•Chapter 9, Digital Image Processing, Gonzalez &Woods
Topics
1.Basic Morphological concepts
2.Binary Morphological operations
•Dilation & erosion
•Hit-or-miss transformation
•Opening & closing
3.Gray scale morphological operations
4.Some basic morphological operations
•Boundary extraction
•Region filling
•Extraction of connected component
•Convex hull
5.Skeletonization
6.Granularity
7.Morphological segmentation and watersheds
Introduction
1.Morphological operators often take a binary image and astructuring element as input and combine them using a setoperator (intersection, union, inclusion, complement).
2.The structuring element is shifted over the image and at eachpixel of the image its elements are compared with the set ofthe underlying pixels.
3.If the two sets of elements match the condition defined by theset operator (e.g. if set of pixels in the structuring element is asubset of the underlying image pixels), the pixel underneaththe origin of the structuring element is set to a pre-definedvalue (0 or 1 for binary images).
4.A morphological operator is therefore defined by itsstructuring element and the applied set operator.
5.Image pre-processing (noise filtering, shape simplification)
6.Enhancing object structures (skeletonization, thinning, convexhull, object marking)
7.Segmentation of the object from background
8.Quantitative descriptors of objects (area, perimeter, projection,Euler-Poincaré characteristics)
binaryimage
structuringelement
Example: Morphological Operation Let ‘⊕’ denote a morphological operator 𝑋⊕𝐵={𝑝∈ 𝑍 2 |𝑝=𝑥+𝑏,𝑥∈𝑋,𝑏∈𝐵
Principles of Mathematical Morphology Compatibility with translation Translation-dependent operators: 𝛹 𝑂 ( 𝑋 ℎ )= 𝛹 −ℎ (𝑋) ℎ Translation-independent operators: 𝛹( 𝑋 ℎ )= 𝛹(𝑋) ℎ Compatibility with scale change Scale-dependent operators: 𝛹 𝜆 (𝑋)=𝜆𝛹 1 𝜆 𝑋 Scale-independent operators: 𝛹(𝜆𝑋)=𝜆𝛹 𝑋 Local knowledge: For any bounded point set Z´ in the transformation Ψ(X), there exits a bounded set Z, knowledge of which is sufficient to predict Ψ(X) over Z´. 𝛹(𝑋∩𝑍)]∩ 𝑍 ′ =𝛹(𝑋)∩ 𝑍 ′ Upper semi-continuity: Changes incurred by a morphological operation are incremental in nature, i.e., its effect has an upper bound.
Dilation Morphological dilation ‘⊕’ combines two sets using vector of set elements 𝑋⊕𝐵={𝑝∈ 𝑍 2 |𝑝=𝑥+𝑏,𝑥∈𝑋,𝑏∈𝐵 Commutative: 𝑋⊕𝐵=𝐵⊕𝑋 Associative: 𝑋⊕ 𝐵⊕𝐷 = 𝑋⊕𝐵 ⊕𝐷 Invariant of translation: 𝑋 ℎ ⊕𝐵= 𝑋⊕𝐵 ℎ If 𝑋⊆𝑌 then 𝑋⊕𝐵⊆𝑌⊕𝐵
Erosion Morphological erosion ‘⊖’ combines two sets using vector subtraction of set elements and is a dual operator of dilation 𝑋⊖𝐵={𝑝∈ 𝑍 2 |∀𝑏∈𝐵,𝑝+𝑏∈𝑋 Not Commutative: 𝑋⊖𝐵≠𝐵⊖𝑋 Not associative: 𝑋⊖ 𝐵⊖𝐷 = 𝑋⊖𝐵 ⊖𝐷 Invariant of translation: 𝑋 ℎ ⊖𝐵= 𝑋⊖𝐵 ℎ and 𝑋⊖ 𝐵 ℎ = 𝑋⊖𝐵 −ℎ If 𝑋⊆𝑌 then 𝑋⊖𝐵⊆𝑌⊖𝐵
Duality: Dilation and Erosion Transpose Ă of a structuring element A is defined as follows 𝐴 ={−𝑎|𝑎∈𝐴 Duality between morphological dilation and erosion operators 𝑋⊖𝐵 𝑐 = 𝑋 𝑐 ⊕ 𝐵
Hit-Or-Miss transformation Hit-or-miss is a morphological operators for finding local patterns of pixels. Unlike dilation and erosion, this operation is defined using a composite structuring element 𝐵= 𝐵 1 , 𝐵 2 . The hit-or-miss operator is defined as follows 𝑋⊗𝐵={𝑥| 𝐵 1 ⊂𝑋 and 𝐵 2 ⊂ 𝑋 𝒄
Hit-Or-Miss transformation: another example Relation with erosion: 𝑋⊗𝐵= 𝑋⊖ 𝐵 1 ∩ 𝑋 𝑐 ⊖ 𝐵 2
Hit-Or-Miss transformation: yet another example
Hit-Or-Miss transformation
Opening Erosion and dilation are not inverse transforms. An erosion followed by a dilation leads to an interesting morphological operation called opening 𝑋∘𝐵= 𝑋⊖𝐵)⊕𝐵
Opening Erosion and dilation are not inverse transforms. An erosion followed by a dilation leads to an interesting morphological operation called opening 𝑋∘𝐵= 𝑋⊖𝐵)⊕𝐵
Opening Erosion and dilation are not inverse transforms. An erosion followed by a dilation leads to an interesting morphological operation called opening 𝑋∘𝐵= 𝑋⊖𝐵)⊕𝐵
Opening Erosion and dilation are not inverse transforms. An erosion followed by a dilation leads to an interesting morphological operation called opening 𝑋∘𝐵= 𝑋⊖𝐵)⊕𝐵
Closing A dilation followed by an erosion leads to the interesting morphological operation called closing 𝑋•𝐵=(𝑋⊕𝐵)⊖𝐵
Closing A dilation followed by an erosion leads to the interesting morphological operation called closing 𝑋•𝐵=(𝑋⊕𝐵)⊖𝐵
Closing A dilation followed by an erosion leads to the interesting morphological operation called closing 𝑋•𝐵=(𝑋⊕𝐵)⊖𝐵
Gray Scale Morphological Operation
•Basic Morphological concepts
•Four Morphological principles
•Binary Morphological operations
•Dilation & erosion
•Hit-or-miss transformation
•Opening & closing
•Gray scale morphological operations
•Some basic morphological operations
•Boundary extraction
•Region filling
•Extraction of connected component
•Convex hull
•Skeletonization
Gray Scale Morphological Operation
Support F
top surfaceT[A]
Set A
Gray Scale Morphological Operation
•A: a subset of n-dimensional Euclidean space, A  Rn
•F: support of A
•Top hat or surface
•A top surface is essentially a gray scale image f : F  R
•An umbra U(f) of a gray scale image f : F  R is the wholesubspace below the top surface representing the gray scaleimage f. Thus,
Gray Scale Morphological Operation
top surfaceT[A]
umbra
Support F
Gray Scale Morphological Operation
top surfaceT[A]
Gray Scale Morphological Operation
•The gray scale dilation between two functions may be defined as thetop surface of the dilation of their umbras
•More computation-friendly definitions
•Commonly, we consider the structure element k as a binary set. Thenthe definitions of gray-scale morphological operations simplifies to
Morphological Boundary Extraction
•The boundary of an object A denoted by δ(A) can be obtained by firsteroding the object and then subtracting the eroded image from theoriginal image.
Quiz
•How to extract edges along a given orientation using morphologicaloperations?
Morphological noise filtering
•An opening followed by a closing
•Or, a closing followed by an opening
Morphological noise filtering
MATLAB DEMO
Morphological Region Filling
•Task: Given a binary image X and a (seed) point p, fill the regionsurrounded by the pixels of X and contains p.
•A: An image where only the boundary pixels are labeled 1 and othersare labeled 0
•Ac: The Complement of A
•We start with an image X0 where only the seed point p is 1 and othersare 0. Then we repeat the following steps until it converges
Morphological Region Filling
A
Ac
Morphological Region Filling
•The boundary of an object A denoted by δ(A) can be obtained by firsteroding the object and then subtracting the eroded image from theoriginal image.
A
Morphological Region Filling
Morphological Region Filling
Homotopic Transformation
•Homotopic tree
r1
r2
h1
h2
Quitz: Homotopic Transformation
•What is the relation between an element in the ith and i+1th levels?
Skeletonization
•Skeleton by maximal balls: locii of the centers of maximal ballscompletely included by the object
Skeletonization
•Matlab Demo
•HW: Write an algorithm using morphologic operators to retrieve backthe portions of the GOOD curves lost during pruning
Skeletonization and Pruning
•Skeletonization preserves both
•End points
•Topology
•Pruning preserves only
•Topology
Outlined diamondOutlined diamond
Outlined diamond
Outlined diamond
Outlined diamondOutlined diamond
Outlined diamondOutlined diamond
Outlined diamond
Outlined diamond
Outlined diamond
Outlined diamond
Outlined diamond
Outlined diamond
Outlined diamond
Outlined diamond
Outlined diamond
afterskeletonization
after pruning
after retrieval
Quench function
•Every location p on the skeleton S(X) of a shape X has an associatedradius qX(p) of maximal ball; this function is termed as quenchfunction
•The set X is recoverable from its skeleton and its quench function
Ultimate Erosion
•The ultimate erosion of a set X, denoted by Ult(X), is the set ofregional maxima of the quench functions
•Morphological reconstruction: Assume two sets A, B such that B  A.The reconstruction σA(B) of the set A is the unions of all connectedcomponents of A with nonempty intersection with B.
B
A
Ultimate Erosion
•The ultimate erosion of a set X, denoted by Ult(X), is the set ofregional maxima of the quench functions
•Morphological reconstruction: Assume two sets A, B such that B  A.The reconstruction σA(B) of the set A is the unions of all connectedcomponents of A with nonempty intersection with B.
Convex Hull
•A set A is said to be convex if the straight line joining any two pointswithin A lies in A.
•Q: Is an empty set convex?
•Q: What ar4e the topological properties of a convex set?
•A convex hull H of a set X is the minimum convex set containing X.
•The set difference H – X is called the convex deficiency of X.
Outlined diamond
Outlined diamondOutlined diamond
Outlined diamondOutlined diamond
Outlined diamond
Outlined diamondOutlined diamondOutlined diamondOutlined diamondOutlined diamondOutlined diamond
Outlined diamondOutlined diamond
Outlined diamondOutlined diamond
Outlined diamondOutlined diamond
Outlined diamondOutlined diamondOutlined diamondOutlined diamondOutlined diamondOutlined diamond
Geodesic Morphological Operations
•The geodesic distance DX(x,y) between two points x and y w.r.t. a setX is the length of the shortest path between x and y that entirely lieswithin X.
??
Geodesic Balls
•The geodesic ball BX(p,n) of center p and radius n w.r.t. a set X is aball constrained by X.
Geodesic Operations
•The geodesic dilation δX(n)(Y) of the set Y  by a geodesic ball of radiusn w.r.t. a set X is :
•The geodesic erosion εX(n)(Y) of the set Y  by a geodesic ball of radiusn w.r.t. a set X is :
An example
•What happens if we apply geodesic erosion on X – {p}where p is a point in X?
Implementation Issue
•An efficient solution: select a ball of radius ‘1’ and thendefine
Morphological Reconstruction
•Assume that we want to reconstruct objects of a given shape from abinary image that was originally obtained by thresholding. Allconnected components in the input image constitute the set X.However, we are interested only a few connected components markedby a marker set Y.
How?
•Successive geodesic dilations of the set Y inside the bigger set X leadsto the reconstruction of connected components of X marked by Y.
•The geodesic dilation terminates when all connected components of Xmarked by Y are filled, i.e., an idempotency is reached :
•This operation is called reconstruction and is denoted by ρX(Y).
Geodesic Influence Zone
•Let Y, Y1, Y2, ..Ym denote m marker sets on a bigger set X such that eachof Y and Yis is a subset of X.
Reconstruction to Gray-Scale Images
•This requires the extension of geodesy to gray-scale images.
•Any increasing transformation defined for binary images can be extendedto gray-level images
•A gray level image I is viewed as a stack of binary images obtained bysuccessive thresholding – this process is called threshold decomposition
•Threshold decomposition principle
Reconstruction to Gray-Scale Images
•Returning to the reconstruction transformation, binary geodesicreconstruction ρ is an increasing transformation
•Gray-scale reconstruction: Let J, I be two gray-scale images both overthe domain D such that J  I, the gray-scale reconstruction ρI(J) of theimage I from J is defined as
Reconstruction to Gray-Scale Images