PowerPoint Presentation

Multiscale Geometric SignalProcessing in High Dimensions

Hyeokho Choi Richard Baraniuk

Wai Lam Chan Mike Wakin

Geometric Image Features

Edges:

“location”

“orientation”

Goal: multiscale feature analysis

Location Information in 1-D

• Fourier phase analysis?

$C:\Documents and Settings\choi\Desktop\txp_fig.png$

• Linear phase change as signal shifts

• No time localization

Local Fourier Analysis

• Local Fourier analysis for “local” location

• Short time Fourier transform (Gabor analysis)

1.Local bandpass filtering

2.Uniform time-frequency tiling

t

f

Wavelet Analysis

1.“Multiscale” analysis

2.Sparse representation of piecewise smoothsignals

3.Orthonormal basis / tight frame

4.Fast computation by filter banks

HP

LP

1-D Wavelet Transform

$C:\Documents and Settings\choi\Desktop\txp_fig.png$

$C:\Documents and Settings\choi\Desktop\txp_fig.png$

HP

LP

HP

LP

1-D Wavelet Transform

HP

LP

HP

LP

2

2

2

2

1-D Wavelet Transform

$C:\Documents and Settings\choi\Desktop\txp_fig.png$

$C:\Documents and Settings\choi\Desktop\txp_fig.png$

Short Time Fourier vs. Wavelet

Short time Fourier

Wavelet

$C:\Documents and Settings\choi\Desktop\txp_fig.png$

t

f

$C:\Documents and Settings\wailam\Desktop\icip2004_talk\wavelet_bases.png$

f

t

$C:\Documents and Settings\wailam\Desktop\icip2004_talk\stft1.gif$

$C:\Documents and Settings\wailam\Desktop\icip2004_talk\stft2.gif$

$C:\Documents and Settings\wailam\Desktop\icip2004_talk\stft3.gif$

$C:\Documents and Settings\wailam\Desktop\icip2004_talk\stft2_sin.gif$

$C:\Documents and Settings\wailam\Desktop\icip2004_talk\stft1_sin.gif$

$C:\Documents and Settings\wailam\Desktop\icip2004_talk\stft3_sin.gif$

“Real”

Phase for Wavelets ?

$C:\Documents and Settings\choi\Desktop\txp_fig.png$

• Need to have quadrature component

• phase shift of

“Hilbert Transform”

$C:\Documents and Settings\choi\Desktop\txp_fig.png$

$C:\Documents and Settings\choi\Desktop\txp_fig.png$

Complex Wavelet

$C:\Documents and Settings\choi\Desktop\txp_fig.png$

• Complex wavelet transform (CWT)

[Kingsbury,Selesnik,Lina,…]

• Phase linear to local signal shift

+ j ×

$D:\research\Spring2004\arkansas\images\cplx_wavelet1d.jpg$

wavelet

Hilbert Transform

1-D Complex Wavelet Transform (CWT)

-j

+j

+1

+1

Complex (analytic) wavelet

+ j*

=

+2

$C:\Documents and Settings\choi\Desktop\txp_fig.png$

$C:\Documents and Settings\choi\Desktop\txp_fig.png$

$C:\Documents and Settings\choi\Desktop\txp_fig.png$

$D:\home\choi\conference\AFOSR-review-workshop-2005\txp_fig.png$

$D:\home\choi\conference\AFOSR-review-workshop-2005\txp_fig.png$

$D:\home\choi\conference\AFOSR-review-workshop-2005\txp_fig.png$

2-D Fourier Analysis

• Phase ambiguity

$C:\Documents and Settings\choi\Desktop\txp_fig.png$

• cannot obtain from phase shift

$C:\Documents and Settings\choi\Desktop\txp_fig.png$

$D:\home\choi\conference\AFOSR-review-workshop-2005\txp_fig.png$

Quaternion Fourier Transform (QFT)

• Separate 4 “quadrature” components

• Organize as quaternion

• Quaternions:

• Multiplication rules: and

[Bulow et al.]

$D:\home\choi\conference\AFOSR-review-workshop-2005\txp_fig.png$

• Quaternion phase angles: Shift theorem

$C:\Documents and Settings\choi\Desktop\txp_fig.png$

QFT Phase

• QFT shift theorem:

1. invariant to signal shift

2. linear to signal shift

$C:\Documents and Settings\choi\Desktop\txp_fig.png$

$C:\Documents and Settings\choi\Desktop\txp_fig.png$

$C:\Documents and Settings\choi\Desktop\txp_fig.png$

Quaternion Wavelet!

“Real” 2-D Wavelet Transform

“Real” 2-D Wavelet Transform

“Real” 2-D Wavelet Transform

HH

LL

LH

HL

$C:\Documents and Settings\choi\Desktop\txp_fig.png$

$C:\Documents and Settings\choi\Desktop\txp_fig.png$

$C:\Documents and Settings\choi\Desktop\txp_fig.png$

$C:\Documents and Settings\choi\Desktop\txp_fig.png$

2-D Hilbert Transform

u

v

u

v

u

v

u

v

$C:\Documents and Settings\choi\Desktop\txp_fig.png$

$C:\Documents and Settings\choi\Desktop\txp_fig.png$

$C:\Documents and Settings\choi\Desktop\txp_fig.png$

$C:\Documents and Settings\choi\Desktop\txp_fig.png$

$C:\Documents and Settings\choi\Desktop\txp_fig.png$

$C:\Documents and Settings\choi\Desktop\txp_fig.png$

$C:\Documents and Settings\choi\Desktop\txp_fig.png$

$C:\Documents and Settings\choi\Desktop\txp_fig.png$

$C:\Documents and Settings\choi\Desktop\txp_fig.png$

$C:\Documents and Settings\choi\Desktop\txp_fig.png$

$C:\Documents and Settings\choi\Desktop\txp_fig.png$

$C:\Documents and Settings\choi\Desktop\txp_fig.png$

HT in u

HT in v

HT in both

2-D Hilbert Transform

u

v

u

v

u

v

u

v

$C:\Documents and Settings\choi\Desktop\txp_fig.png$

$C:\Documents and Settings\choi\Desktop\txp_fig.png$

$C:\Documents and Settings\choi\Desktop\txp_fig.png$

$C:\Documents and Settings\choi\Desktop\txp_fig.png$

$C:\Documents and Settings\choi\Desktop\txp_fig.png$

$C:\Documents and Settings\choi\Desktop\txp_fig.png$

$C:\Documents and Settings\choi\Desktop\txp_fig.png$

$C:\Documents and Settings\choi\Desktop\txp_fig.png$

$C:\Documents and Settings\choi\Desktop\txp_fig.png$

$C:\Documents and Settings\choi\Desktop\txp_fig.png$

$C:\Documents and Settings\choi\Desktop\txp_fig.png$

$C:\Documents and Settings\choi\Desktop\txp_fig.png$

$C:\Documents and Settings\choi\Desktop\txp_fig.png$

$C:\Documents and Settings\choi\Desktop\txp_fig.png$

$C:\Documents and Settings\choi\Desktop\txp_fig.png$

$C:\Documents and Settings\choi\Desktop\txp_fig.png$

Quaternion Wavelet

60%

60%

60%

60%

+1

+1

+1

+1

$C:\Documents and Settings\choi\Desktop\txp_fig.png$

Quaternion Wavelet

60%

60%

60%

60%

+1

+1

+1

+1

60%

60%

60%

60%

+1

+1

+1

+1

+j

+j

-j

-j

HTx

HTy

60%

+1

60%

+1

60%

-1

60%

-1

60%

60%

60%

60%

+j

-j

+j

-j

HTy

60%

60%

60%

60%

+j

+j

-j

-j

HTx

$C:\Documents and Settings\choi\Desktop\txp_fig.png$

$D:\home\choi\conference\AFOSR-review-workshop-2005\txp_fig.png$

$D:\home\choi\conference\AFOSR-review-workshop-2005\txp_fig.png$

$D:\home\choi\conference\AFOSR-review-workshop-2005\txp_fig.png$

Quaternion Wavelet Transform (QWT)

• Quaternion basis function

• 3 subbands (HH, HL, LH)

v

u

HH subband

HL subband

LH subband

• single-quadrant spectrum (QFT domain)

$D:\home\choi\conference\AFOSR-review-workshop-2005\txp_fig.png$

$D:\home\choi\conference\AFOSR-review-workshop-2005\txp_fig.png$

QWT Applications

•Edge parameter (offset/orientation) estimation

–

 edge offset

–QWT magnitude edge orientation

$C:\Documents and Settings\wailam\Desktop\icip2004_talk\est_result4.gif$

$C:\Documents and Settings\wailam\Desktop\icip2004_talk\est_result3.gif$

$C:\Documents and Settings\wailam\Desktop\icip2004_talk\est_result2.gif$

$C:\Documents and Settings\wailam\Desktop\icip2004_talk\est_result1.gif$

QWT phase-based flow estimation

$D:\home\choi\conference\AFRL-kickoff-3may2005\checker.jpg$

$D:\home\choi\conference\AFRL-kickoff-3may2005\checker_rotate.jpg$

QWT

QWT

Phase difference

Local image shift

Image 2

Image 1

Test image

Rotated image

$D:\home\choi\conference\AFRL-kickoff-3may2005\checker_flow_J4.jpg$

$D:\home\choi\conference\AFRL-kickoff-3may2005\checker.jpg$

$D:\home\choi\conference\AFRL-kickoff-3may2005\checker_rotate.jpg$

•Local estimation of image shift : motion field

•Phase as local image features

QWT phase-based flow estimation

$D:\home\choi\conference\AFOSR-review-workshop-2005\rubik1.jpg$

$D:\home\choi\conference\AFOSR-review-workshop-2005\rubik2.jpg$

Flow Estimation Example

$D:\home\choi\conference\AFOSR-review-workshop-2005\rubik_top_est_r.jpg$

Summary: Quaternion Wavelets

• Quaternion wavelets for 2-D image processing

•To date:

- 2-D quaternion wavelet implementation

- flow analysis using quaternion phase

• Current and future work:

- theoretical analysis of quaternion phase

- statistical geometry modeling in QWT domain

- extension to higher dimensions

Part II : Multiscale Image Manifold

Multiple View Images

$D:\home\choi\job-interview\tank.gif$

“Light Field”

Multiple View Images

$D:\home\choi\job-interview\tank.gif$

$D:\home\choi\job-interview\tank.gif$

RN2

“Light field Manifold”

Manifold Signal Processing

•Often data can be interpreted as living along alow-dimensional manifold in ahigh-dimensional ambient space

•Ex:each NxN image is a point in RNxNbut most points in RNxN look like “noise”

•Samples on manifold: point clouds

$C:\HomeRichB\Talks\ImageProcessing\pptTalks\new-manifold-figs\tp_denoise.tiff$

$C:\HomeRichB\Talks\ImageProcessing\pptTalks\new-manifold-figs\tp_org.tiff$

$C:\Documents and Settings\wakin\Desktop\Matlab\Donoho\demoReg0.tiff$

3-D pose estimation

Manifold Navigation

$C:\Documents and Settings\wakin\Desktop\Matlab\Donoho\demoReg0.tiff$

Navigation guided by multiscale tangents

$C:\Documents and Settings\wakin\Desktop\Matlab\Donoho\demoReg0.tiff$

3-D pose estimation

$C:\Documents and Settings\wakin\Desktop\Matlab\Donoho\demoReg1.tiff$

s = 1/2

$C:\Documents and Settings\wakin\Desktop\Matlab\Donoho\demoReg2.tiff$

s = 1/4

$C:\Documents and Settings\wakin\Desktop\Matlab\Donoho\demoReg3.tiff$

s = 1/16

$C:\Documents and Settings\wakin\Desktop\Matlab\Donoho\demoReg4.tiff$

s = 1/256

Summary: Multiscale Manifolds

• Multiscale manifold analysis in high dimensions

•To date:

- multiscale tangents and navigation

- image articulation manifold

- imaging parameter estimation

• Current and future work:

- geometric properties of articulation manifold

- application to ATR problem

www.dsp.rice.edu