Preface	6
	Table of Contents	7
	1 Introduction	11
	1.1 Context	11
	1.2 Applications	13
	2 Preprocessing	19
	2.1 Nonlinear filters	21
	2.1.1 Median filters and rank-order filters	21
	2.1.2 Morphological filters	25
	2.1.3 Polynomial filters	29
	2.2 Amplitude-value transformations	30
	2.2.1 Amplitude mapping characteristics	31
	2.2.2 Probability distribution modification and equalization	32
	2.3 Interpolation	34
	2.3.1 Zero and first order interpolation basis functions	35
	2.3.2 LTI systems as interpolators	37
	2.3.3 Spline, Lagrangian and polynomial interpolation	38
	2.3.4 Interpolation on 2D grids	43
	2.4 Multi-resolution representation	47
	2.5 Locally adaptive filters	53
	2.5.1 Steerable smoothing filters	53
	2.5.2 Iterative smoothing (diffusion filters)	55
	2.6 Problems	58
	3 Signal and Parameter Estimation	61
	3.1 Expected values and probability description	61
	3.2 Observation and degradation models	66
	3.3 Estimation based on linear filters	67
	3.3.1 Inverse filters	67
	3.3.2 Wiener filters	68
	3.4 Least-squares estimation	70
	3.5 Singular value decomposition	75
	3.6 ML and MAP estimation	77
	3.7 Parameter estimation and fitting	79
	3.8 Outlier rejection	81
	3.9 Correspondence analysis	84
	3.10 State modeling and estimation	87
	3.10.1 Markov processes and random fields	87
	3.10.2 Hidden Markov models	90
	3.10.3 Kalman filters	91
	3.10.4 Particle filters	94
	3.11 Problems	94
	4 Features of Multimedia Signals	97
	4.1 Color	97
	4.1.1 Color space transformations	98
	4.1.2 Representation of color features	107
	4.2 Texture	112
	4.2.1 Texture analysis based on occurrence counts	114
	4.2.2 Texture analysis based on statistical models	117
	4.2.3 Spectral features of texture	120
	4.2.4 Inhomogeneous texture analysis	124
	4.3 Edge analysis	125
	4.3.1 Edge detection by gradient operators	125
	4.3.2 Edge characterization by second derivative	129
	4.3.3 Edge finding and consistency analysis	131
	4.3.4 Edge model fitting	134
	4.3.5 Description and analysis of edge properties	135
	4.4 Salient feature detection	137
	4.5 Contour and shape analysis	142
	4.5.1 Contour fitting	142
	4.5.2 Contour description by orientation and curvature	146
	4.5.3 Geometric features and binary shape features	150
	4.5.4 Projection and geometric mapping	154
	4.5.5 Moment analysis of region shapes	164
	4.5.6 Region shape analysis by basis functions	168
	4.6 Motion analysis	169
	4.6.1 Projection of 3D motion into the image plane	169
	4.6.2 Motion estimation by the optical flow principle	173
	4.6.3 Motion estimation by matching	178
	4.6.4 Estimation of non-translational motion parameters	188
	4.6.5 Estimation of motion vector fields at object boundaries	190
	4.7 Disparity and depth analysis	193
	4.7.1 Coplanar stereoscopy	193
	4.7.2 Epipolar geometry	196
	4.7.3 Camera calibration	199
	4.8 Audio signal features	203
	4.8.1 Audio feature extraction on the timeline	204
	4.8.2 Time domain features	206
	4.8.3 Spectral domain features	212
	4.8.4 Cepstral domain features	216
	4.8.5 Harmonic features	217
	4.8.6 Multi-channel features	222
	4.8.7 Perceptual features	223
	4.8.8 Semantic features	225
	4.9 Problems	227
	5 Feature Transforms and Classification	233
	5.1 Feature value normalization and transforms	233
	5.2 Distance metrics	248
	5.3 Compressed representation of feature data	261
	5.4 Feature-based comparison	263
	5.5 Reliability	267
	5.6 Classification methods	274
	5.7 Belief, plausibility and evidence	299
	5.8 Problems	302
	6 Signal Decomposition	305
	6.1 Spatial segmentation of pictures	306
	6.1.1 Segmentation based on sample classification	307
	6.1.2 Region-based methods	312
	6.1.3 Contour-based methods	314
	6.1.4 Segmentation based on ‘energy minimization’	315
	6.2 Segmentation of video signals	321
	6.2.1 Key picture and shot transition detection	322
	6.2.2 Segmentation by background differencing	323
	6.2.3 Object tracking and spatio-temporal segmentation	324
	6.2.4 Combined segmentation and motion estimation	330
	6.3 3D surface and volume reconstruction	331
	6.3.1 3D point cloud generation	332
	6.3.2 3D surface reconstruction	333
	6.3.3 3D volume reconstruction	335
	6.3.4 Projection based description of 3D shapes	336
	6.4 Decomposition of audio signals	339
	6.4.1 Temporal segmentation of audio	339
	6.4.2 Audio source separation	339
	6.5 Problems	341
	7 Signal Composition, Rendering and Presentation	343
	7.1 Composition and mixing of multimedia signals	343
	7.2 Mosaicking and stitching	348
	7.3 Synthesis of picture content	351
	7.4 Warping and morphing	355
	7.5 Virtual view synthesis	357
	7.6 Frame rate conversion	362
	7.7 View-adaptive and stereoscopic rendering of image and video signals	366
	7.8 Composition and rendering of audio signals	369
	7.8.1 Sound effects	371
	7.8.2 Spatial (room) features	374
	A Fundamentals and definitions	377
	A.1 Fundamentals of signal processing and signal analysis	377
	A.2 Fundamentals of stochastic analysis and description	386
	A.3 Vector and matrix algebra	395
	B Symbols and Variables	401
	C Glossary and Acronyms	406
	D References	408
	E Index	421