|
Preface |
6 |
|
|
Table of Contents |
7 |
|
|
1 Introduction |
11 |
|
|
1.1 Context |
11 |
|
|
1.2 Applications |
13 |
|
|
2 Preprocessing |
19 |
|
|
2.1 Nonlinear filters |
21 |
|
|
2.1.1 Median filters and rank-order filters |
21 |
|
|
2.1.2 Morphological filters |
25 |
|
|
2.1.3 Polynomial filters |
29 |
|
|
2.2 Amplitude-value transformations |
30 |
|
|
2.2.1 Amplitude mapping characteristics |
31 |
|
|
2.2.2 Probability distribution modification and equalization |
32 |
|
|
2.3 Interpolation |
34 |
|
|
2.3.1 Zero and first order interpolation basis functions |
35 |
|
|
2.3.2 LTI systems as interpolators |
37 |
|
|
2.3.3 Spline, Lagrangian and polynomial interpolation |
38 |
|
|
2.3.4 Interpolation on 2D grids |
43 |
|
|
2.4 Multi-resolution representation |
47 |
|
|
2.5 Locally adaptive filters |
53 |
|
|
2.5.1 Steerable smoothing filters |
53 |
|
|
2.5.2 Iterative smoothing (diffusion filters) |
55 |
|
|
2.6 Problems |
58 |
|
|
3 Signal and Parameter Estimation |
61 |
|
|
3.1 Expected values and probability description |
61 |
|
|
3.2 Observation and degradation models |
66 |
|
|
3.3 Estimation based on linear filters |
67 |
|
|
3.3.1 Inverse filters |
67 |
|
|
3.3.2 Wiener filters |
68 |
|
|
3.4 Least-squares estimation |
70 |
|
|
3.5 Singular value decomposition |
75 |
|
|
3.6 ML and MAP estimation |
77 |
|
|
3.7 Parameter estimation and fitting |
79 |
|
|
3.8 Outlier rejection |
81 |
|
|
3.9 Correspondence analysis |
84 |
|
|
3.10 State modeling and estimation |
87 |
|
|
3.10.1 Markov processes and random fields |
87 |
|
|
3.10.2 Hidden Markov models |
90 |
|
|
3.10.3 Kalman filters |
91 |
|
|
3.10.4 Particle filters |
94 |
|
|
3.11 Problems |
94 |
|
|
4 Features of Multimedia Signals |
97 |
|
|
4.1 Color |
97 |
|
|
4.1.1 Color space transformations |
98 |
|
|
4.1.2 Representation of color features |
107 |
|
|
4.2 Texture |
112 |
|
|
4.2.1 Texture analysis based on occurrence counts |
114 |
|
|
4.2.2 Texture analysis based on statistical models |
117 |
|
|
4.2.3 Spectral features of texture |
120 |
|
|
4.2.4 Inhomogeneous texture analysis |
124 |
|
|
4.3 Edge analysis |
125 |
|
|
4.3.1 Edge detection by gradient operators |
125 |
|
|
4.3.2 Edge characterization by second derivative |
129 |
|
|
4.3.3 Edge finding and consistency analysis |
131 |
|
|
4.3.4 Edge model fitting |
134 |
|
|
4.3.5 Description and analysis of edge properties |
135 |
|
|
4.4 Salient feature detection |
137 |
|
|
4.5 Contour and shape analysis |
142 |
|
|
4.5.1 Contour fitting |
142 |
|
|
4.5.2 Contour description by orientation and curvature |
146 |
|
|
4.5.3 Geometric features and binary shape features |
150 |
|
|
4.5.4 Projection and geometric mapping |
154 |
|
|
4.5.5 Moment analysis of region shapes |
164 |
|
|
4.5.6 Region shape analysis by basis functions |
168 |
|
|
4.6 Motion analysis |
169 |
|
|
4.6.1 Projection of 3D motion into the image plane |
169 |
|
|
4.6.2 Motion estimation by the optical flow principle |
173 |
|
|
4.6.3 Motion estimation by matching |
178 |
|
|
4.6.4 Estimation of non-translational motion parameters |
188 |
|
|
4.6.5 Estimation of motion vector fields at object boundaries |
190 |
|
|
4.7 Disparity and depth analysis |
193 |
|
|
4.7.1 Coplanar stereoscopy |
193 |
|
|
4.7.2 Epipolar geometry |
196 |
|
|
4.7.3 Camera calibration |
199 |
|
|
4.8 Audio signal features |
203 |
|
|
4.8.1 Audio feature extraction on the timeline |
204 |
|
|
4.8.2 Time domain features |
206 |
|
|
4.8.3 Spectral domain features |
212 |
|
|
4.8.4 Cepstral domain features |
216 |
|
|
4.8.5 Harmonic features |
217 |
|
|
4.8.6 Multi-channel features |
222 |
|
|
4.8.7 Perceptual features |
223 |
|
|
4.8.8 Semantic features |
225 |
|
|
4.9 Problems |
227 |
|
|
5 Feature Transforms and Classification |
233 |
|
|
5.1 Feature value normalization and transforms |
233 |
|
|
5.2 Distance metrics |
248 |
|
|
5.3 Compressed representation of feature data |
261 |
|
|
5.4 Feature-based comparison |
263 |
|
|
5.5 Reliability |
267 |
|
|
5.6 Classification methods |
274 |
|
|
5.7 Belief, plausibility and evidence |
299 |
|
|
5.8 Problems |
302 |
|
|
6 Signal Decomposition |
305 |
|
|
6.1 Spatial segmentation of pictures |
306 |
|
|
6.1.1 Segmentation based on sample classification |
307 |
|
|
6.1.2 Region-based methods |
312 |
|
|
6.1.3 Contour-based methods |
314 |
|
|
6.1.4 Segmentation based on ‘energy minimization’ |
315 |
|
|
6.2 Segmentation of video signals |
321 |
|
|
6.2.1 Key picture and shot transition detection |
322 |
|
|
6.2.2 Segmentation by background differencing |
323 |
|
|
6.2.3 Object tracking and spatio-temporal segmentation |
324 |
|
|
6.2.4 Combined segmentation and motion estimation |
330 |
|
|
6.3 3D surface and volume reconstruction |
331 |
|
|
6.3.1 3D point cloud generation |
332 |
|
|
6.3.2 3D surface reconstruction |
333 |
|
|
6.3.3 3D volume reconstruction |
335 |
|
|
6.3.4 Projection based description of 3D shapes |
336 |
|
|
6.4 Decomposition of audio signals |
339 |
|
|
6.4.1 Temporal segmentation of audio |
339 |
|
|
6.4.2 Audio source separation |
339 |
|
|
6.5 Problems |
341 |
|
|
7 Signal Composition, Rendering and Presentation |
343 |
|
|
7.1 Composition and mixing of multimedia signals |
343 |
|
|
7.2 Mosaicking and stitching |
348 |
|
|
7.3 Synthesis of picture content |
351 |
|
|
7.4 Warping and morphing |
355 |
|
|
7.5 Virtual view synthesis |
357 |
|
|
7.6 Frame rate conversion |
362 |
|
|
7.7 View-adaptive and stereoscopic rendering of image and video signals |
366 |
|
|
7.8 Composition and rendering of audio signals |
369 |
|
|
7.8.1 Sound effects |
371 |
|
|
7.8.2 Spatial (room) features |
374 |
|
|
A Fundamentals and definitions |
377 |
|
|
A.1 Fundamentals of signal processing and signal analysis |
377 |
|
|
A.2 Fundamentals of stochastic analysis and description |
386 |
|
|
A.3 Vector and matrix algebra |
395 |
|
|
B Symbols and Variables |
401 |
|
|
C Glossary and Acronyms |
406 |
|
|
D References |
408 |
|
|
E Index |
421 |
|