Science: Photons to Phenomenology 1st Edition – Ebook PDF Version
Visit to download the full and correct content document: https://ebookmass.com/product/vision-science-photons-to-phenomenology-1st-edition -ebook-pdf-version/
More products digital (pdf, epub, mobi) instant download maybe you interests ...
Introduction to 80×86 Assembly Language and Computer Architecture – Ebook PDF Version
https://ebookmass.com/product/introduction-to-8086-assemblylanguage-and-computer-architecture-ebook-pdf-version/
Groundwater Science – Ebook PDF Version
https://ebookmass.com/product/groundwater-science-ebook-pdfversion/
Introduction to Political Science 1st Edition, (Ebook PDF)
https://ebookmass.com/product/introduction-to-politicalscience-1st-edition-ebook-pdf/
Introduction to Counseling: An Art and Science Perspective – Ebook PDF Version
https://ebookmass.com/product/introduction-to-counseling-an-artand-science-perspective-ebook-pdf-version/
An Introduction to Data Science 1st Edition, (Ebook PDF)
https://ebookmass.com/product/an-introduction-to-datascience-1st-edition-ebook-pdf/
Medical Laboratory Science Review – Ebook PDF Version
https://ebookmass.com/product/medical-laboratory-science-reviewebook-pdf-version/
Political Science Research Methods 8th Edition – Ebook
PDF Version
https://ebookmass.com/product/political-science-researchmethods-8th-edition-ebook-pdf-version/
Principles of Environmental Science 8th Edition – Ebook
PDF Version
https://ebookmass.com/product/principles-of-environmentalscience-8th-edition-ebook-pdf-version/
Foundations of Binocular Vision: A Clinical Perspective 1st Edition, (Ebook PDF)
https://ebookmass.com/product/foundations-of-binocular-vision-aclinical-perspective-1st-edition-ebook-pdf/
mi © 1999 Massachusetts Institute of Technology
All rights reserved. No part of this book may be reproduced in any form by any electronic or mechanical means (including photocopying, record¬ ing, or information storage and retrieval) without permission in writing from the publisher.
This book was set in Baskerville by Asco Typesetters, Hong Kong, and was printed and bound in the United States of America.
Library of Congress Cataloging-in-Publication Data
Palmer, Stephen E.
Vision science—photons to phenomenology / Stephen E. Palmer, p. cm.
Includes bibliographical references and index.
ISBN 0-262-16183-4
1. Vision. 2. Visual perception. 3. Cognitive science.
I. Title.
QP475.P24 1999
612.8'4—dc21 99-11785
In loving memory of my mentor, colleague, and friend, Irvin Rock (1922—1995), who taught me more about visual perception than everyone else combined and who showed me by example what it means to be a scientist.
Glossary 701
References 737
Name Index 771
Subject Index 780
An Introduction to Vision Science 3 1.1 Visual Perception 5 1.1.1 Defining Visual Perception 5
1.1.2 The Evolutionary Utility of Vision 6
1.1.3 Perception as a Constructive Act 7 Adaptation and Aftereffects 7 Reality and Illusion 7 Ambiguous Figures 9
1.1.4 Perception as Modeling the Environment 10 Visual Completion 10
Objects 11
the Future 12
1.1.5 Perception as Apprehension of Meaning 13
13
and Consciousness 13 1.2 Optical Information 15 1.2.1 The Behavior of Light 15
15
with Surfaces 16 The Ambient Optic Array 18 1.2.2 The Formation of Images 19 Optical Images 20 Projective Geometry 20 Perspective and Orthographic Projection 21
1.2.3 Vision as an “Inverse” Problem 23 1.3 Visual Systems 24
1.3.1 The Human Eye 24 Eye and Brain 24
Anatomy of the Eye 25
Physiological Optics 26
1.3.2 The Retina 28 Neurons 28
Photoreceptors 29 Peculiarities of Retinal Design 33 Pathways to the Brain 35
1.3.3 Visual Cortex 35 Localization of Function 35 Occipital Cortex 37 Parietal and Temporal Cortex 38 Mapping Visual Cortex 39
The Physiological Pathways Hypothesis 42
2 Theoretical Approaches to Vision 45
2.1 Classical Theories of Vision 47
2.1.1 Structuralism 48
2.1.2 Gestaltism 50
Holism 50
Psychophysiological Isomorphism 51
2.1.3 Ecological Optics 53
Analyzing Stimulus Structure 53
Direct Perception 54
2.1.4 Constructivism 55
Unconscious Inference 56
Heuristic Interpretation 57
2.2 A Brief History of Information Processing 59
2.2.1 Computer Vision 59
The Invention of Computers 59
Blocks World 60
Computational Approaches to Ecological Optics 61
Connectionism and Neural Networks 62
2.2.2 Information Processing Psychology 63
2.2.3 Biological Information Processing 64
Early Developments 64
Single-Cell Recording 64
Autoradiography 66
Brain Imaging Techniques 66
2.3 Information Processing Theory 70
2.3.1 The Computer Metaphor 71
2.3.2 Three Levels of Information Processing 71
The Computational Level 72
The Algorithmic Level 72
The Implementational Level 73
2.3.3 Three Assumptions of Information Processing 73
Informational Description 73
Recursive Decomposition 74
Physical Embodiment 77
2.3.4 Representation 77
2.3.5 Processes 80
Implicit versus Explicit Information 80
Processing as Inference 80
Hidden Assumptions 81
Heuristic Processes 83
Hidden Assumptions versus Ecological Validity 83
Top-Down versus Bottom-Up Processes 84
2.4 Four Stages of Visual Perception 85
2.4.1 The Retinal Image 85
2.4.2 The Image-Based Stage 87
2.4.3 The Surface-Based Stage 88
2.4.4 The Object-Based Stage 90
2.4.5 The Category-Based Stage 91
3 Color Vision: A Microcosm of Vision Science 94
3.1 The Computational Description of Color Perception 96
3.1.1 The Physical Description of Light 96
3.1.2 The Psychological Description of Color 97 Color Space 97
Hue 98
Saturation 98
Lightness 98
Lightness versus Brightness 99
3.1.3 The Psychophysical Correspondence 99
3.2 Image-Based Color Processing 101
3.2.1 Basic Phenomena 101
Light Mixture 101
Color Blindness 104
Color Afterimages 105
Simultaneous Color Contrast 106
Chromatic Adaptation 107
3.2.2 Theories of Color Vision 107
Trichromatic Theory 107
Opponent Process Theory 108
Dual Process Theory 110
3.2.3 Physiological Mechanisms 112
Three Cone Systems 112
Color Opponent Cells 113
Reparameterization in Color Processing 114
Lateral Inhibition 115
Adaptation and Aftereffects 119
Double Opponent Cells 119 Higher Cortical Mechanisms 120
3.2.4 Development of Color Vision 121 3.3 Surface-Based Color Processing 122
3.3.1 Lightness Constancy 125
Adaptation Theories 125
Unconscious Inference versus Relational Theories 126
The Importance of Edges 128 Retinex Theory 128
The Scaling Problem 129
Illumination versus Reflectance Edges 130
Distinguishing Illumination from Reflectance Edges 132
3.3.2 Chromatic Color Constancy 133
Constraining the Problem 133
Illumination versus Reflectance Edges Revisited 134
Development of Color Constancy 136
3.4 The Category-Based Stage 137
3.4.1 Color Naming 137
3.4.2 Focal Colors and Prototypes 139
3.4.3 A Fuzzy-Logical Model of Color Naming 140
Fuzzy Set Theory 140
Primary, Derived, and Composite Color Categories 141
143
4 Processing Image Structure 145
4.1 Physiological Mechanisms 146
4.1.1 Retinal and Geniculate Cells 147
Ganglion Cells 147
Bipolar Cells 148
Lateral Geniculate Nucleus 148
4.1.2 Striate Cortex 151
Hubei and Wiesel’s Discovery 151
Simple Cells 151
Complex Cells 153
Hypercomplex Cells 153
4.1.3 Striate Architecture 154
The Retinotopic Map 155
Ocular Dominance Slabs 155
Columnar Structure 156
4.1.4 Development of Receptive Fields 157
4.2 Psychophysical Channels 158
4.2.1 Spatial Frequency Theory 159
Fourier Analysis 160
Spatial Frequency Channels 162
Contrast Sensitivity Functions 163
Selective Adaptation of Channels 165
Spatial Frequency Aftereffects 166
Thresholds for Sine Wave versus Square
Wave Gratings 167
Development of Spatial Frequency Channels 168
4.2.2 Physiology of Spatial Frequency Channels 169
4.3 Computational Approaches 171
4.3.1 Marr’s Primal Sketches 172
4.3.2 Edge Detection 172
Edge Operators and Convolution 173
The Marr-Hildreth Zero-Crossing Algorithm 175
Neural Implementation 179
Scale Integration 180
The Raw Primal Sketch 180
4.3.3 Alternative Computational Theories 182
Texture Analysis 184
Structure from Shading 184
4.3.4 A Theoretical Synthesis 186
Local Spatial Frequency Filters 186
Exploiting the Structure of Natural Images 188
4.4 Visual Pathways 193
4.4.1 Physiological Evidence 193
4.4.2 Perceptual Evidence 195
5 Perceiving Surfaces Oriented in Depth 199
5.1 The Problem of Depth Perception 201
5.1.1 Heuristic Assumptions 202
5.1.2 Marr’s 2.5-D Sketch 202
5.2 Ocular Information 203
5.2.1 Accommodation 203
5.2.2 Convergence 205
5.3 Stereoscopic Information 206
5.3.1 Binocular Disparity 206
Corresponding Retinal Positions 207
The Horopter 208 Stereograms 210
5.3.2 The Correspondence Problem 211
Random Dot Stereograms 212
Autostereograms 214
Binocular Rivalry 216
5.3.3 Computational Theories 216
The First Marr-Poggio Algorithm 217
Edge-Based Algorithms 220 Filtering Algorithms 221
5.3.4 Physiological Mechanisms 222
5.3.5 Vertical Disparity 224
5.3.6 Da Vinci Stereopsis 224
5.4 Dynamic Information 225
5.4.1 Motion Parallax 225
5.4.2 Optic Flow Caused by a Moving Observer 226
5.4.3 Optic Flow Caused by Moving Objects 228
5.4.4 Accretion/Deletion of Texture 229
5.5 Pictorial Information 229
5.5.1 Perspective Projection 230
5.5.2 Convergence of Parallel Lines 231
5.5.3 Position Relative to the Horizon of a Surface 231
5.5.4 Relative Size 232
5.5.5 Familiar Size 234
5.5.6 Texture Gradients 234
5.5.7 Edge Interpretation 236
Vertex Classification 237
Four Types of Edges 237
Edge Labels 238
Physical Constraints 239
Extensions and Generalizations 241
5.5.8 Shading Information 243
Perceiving Surface Orientation from Shading 243
Horn’s Computational Analysis 245
Cast Shadows 246
5.5.9 Aerial Perspective 246
5.5.10 Integrating Information Sources 247
Dominance 247
Compromise 248 Interaction 249
5.6 Development of Depth Perception 249
5.6.1 Ocular Information 250
5.6.2 Stereoscopic Information 251
5.6.3 Dynamic Information 252
5.6.4 Pictorial Information 252
6 Organizing Objects and Scenes 254
The Problem of Perceptual Organization 255
The Experience Error 257
6.1 Perceptual Grouping 257
6.1.1 The Classical Principles of Grouping 257
6.1.2 New Principles of Grouping 259
6.1.3 Measuring Grouping Effects
Quantitatively 261
6.1.4 Is Grouping an Early or Late Process? 263
6.1.5 Past Experience 266
6.2 Region Analysis 266
6.2.1 Uniform Connectedness 268
6.2.2 Region Segmentation 269
Boundary-Based Approaches 270
Region-Based Approaches 271
Evidence from Stabilized Images 273
Parts and Parsing 274
6.2.3 Texture Segregation 275
Discovering the Features of Texture 276
Texture Segregation as a Parallel Process 276
A Theory of Texture Segregation 277
6.3 Figure/Ground Organization 280
6.3.1 Principles of Figure/Ground Organization 281
6.3.2 Ecological Considerations 283
6.3.3 Effects of Meaningfulness 284
6.3.4 The Problem of Holes 285
6.4 Visual Interpolation 287
6.4.1 Visual Completion 288
Figural Familiarity Theories 289
Figural Simplicity Theories 289
Ecological Constraint Theories 290
6.4.2 Illusory Contours 292
Relation to Visual Completion 293
Physiological Basis of Illusory Contours 294
6.4.3 Perceived Transparency 296
6.4.4 Figural Scission 298
6.4.5 The Principle of Nonaccidentalness 299
6.5 Multistability 300
6.5.1 Connectionist Network Models 301
6.5.2 Neural Fatigue 302
6.5.3 Eye Fixations 304
6.5.4 The Role of Instructions 304
6.6 Development of Perceptual Organization 305
6.6.1 The Habituation Paradigm 306
6.6.2 The Development of Grouping 306
7 Perceiving Object Properties and Parts 311
Constancy and Illusion 312
Modes of Perception: Proximal and Distal 313
Size 314
7.1.1 Size Constancy 315
The Size-Distance Relation 315
Demonstrations of Size Constancy 315
Departures from Constancy 317
Taking Account of Distance 317
Texture Occlusion 318
Relative Size 319
The Horizon Ratio 321
Development of Size Constancy 321
7.1.2 Size Illusions 322
The Moon Illusion 322
The Ponzo Illusion 324 Illusions of Relative Size 325
Occlusion Illusions 326
Shape 327
7.2.1 Shape Constancy 327 Perspective Changes 327
Two-Dimensional Figures 328
Three-Dimensional Objects 329
Development of Shape Constancy 331
7.2.2 Shape Illusions 332 Orientation 333
7.3.1 Orientation Constancy 333
7.3.2 Orientation Illusions 336 Frames of Reference 336
Geometric Illusions 337 Position 338
7.4.1 Perception of Direction 338
7.4.2 Position Constancy 339
Indirect Theories of Position Constancy 340
Direct Theories of Position Constancy 341
7.4.3 Position Illusions 342
Perceptual Adaptation 343 Parts 348
7.6.1 Evidence for Perception of Parts 348 Linguistic Evidence 348
Phenomenological Demonstrations 349
Perceptual Experiments 350
7.6.2 Part Segmentation 351
Shape Primitives 351
Boundary Rules 353
7.6.3 Global and Local Processing 354
Global Precedence 355
Configural Orientation Effects 357
Word, Object, and Configural Superiority Effects 359
Representing Shape and Structure 362
8.1 Shape Equivalence 363
8.1.1 Defining Objective Shape 364
8.1.2 Invariant Features 365
8.1.3 Transformational Alignment 367
8.1.4 Object-Centered Reference Frames 368 Geometric Coordinate Systems 369 Perceptual Reference Frames 370 Accounting for Failures of Shape Equivalence 371
Orientation and Shape 373 Heuristics in Reference Frame Selection 374
8.2 Theories of Shape Representation 377
8.2.1 Templates 377 Strengths 378 Weaknesses 379
8.2.2 Fourier Spectra 383 Strengths 384 Weaknesses 384
8.2.3 Features and Dimensions 385 Multidimensional Representations 387
Multifeatural Representations 390 Strengths 391 Weaknesses 392
8.2.4 Structural Descriptions 394
Shape Primitives 396 Strengths 397 Weaknesses 397
8.3 Figural Goodness and Pragnanz 398
8.3.1 Theories of Figural Goodness 399 Classical Information Theory 399 Rotation and Reflection Subsets 400 Symmetry Subgroups 401
8.3.2 Structural Information Theory 402 Primitive Codes 403 Removing Redundancies 403 Information Load 404 Applications to Perceptual Organization 405 Strengths 405 Weaknesses 405
Perceiving Function and Category 408
9.1 The Perception of Function 409
9.1.1 Direct Perception of Affordances 410
9.1.2 Indirect Perception of Function by Categorization 413
Four Components of Categorization 413
Comparison Processes 414 Decision Processes 414
9.2 Phenomena of Perceptual Categorization 416
9.2.1 Categorical Hierarchies 416
Prototypes 417
Basic-Level Categories 418
Entry-Level Categories 419
9.2.2 Perspective Viewing Conditions 420
Canonical Perspective 421 Priming Effects 424 Orientation Effects 426
9.2.3 Part Structure 427
9.2.4 Contextual Effects 428
9.2.5 Visual Agnosia 431
9.3 Theories of Object Categorization 433
9.3.1 Recognition by Components Theory 434 Geons 434
Nonaccidental Features 435 Geon Relations 436 Stages of Object Categorization in RBC 437 A Neural Network Implementation 438
9.3.2 Accounting for Empirical Phenomena 440 Typicality Effects 440
Entry-Level Categories 440
Viewing Conditions 441
Part Structures 442
Contextual Effects 442
Visual Agnosia 443
Weaknesses 443
9.3.3 Viewpoint-Specific Theories 444
The Case for Multiple Views 444 Aspect Graphs 445
Alignment with 3-D Models 448 Alignment with 2-D View Combinations 448
Weaknesses 451
9.4 Identifying Letters and Words 453
9.4.1 Identifying Letters 453
9.4.2 Identifying Words and Letters Within Words 455
9.4.3 The Interactive Activation Model 458 Feature Level 458
Letter Level 458
Word Level 459
Word-to-Letter Feedback 460 Problems 460
III Visual Dynamics 463
10 Perceiving Motion and Events 465
10.1 Image Motion 466
10.1.1 The Computational Problem of Motion 466
10.1.2 Continuous Motion 469
Adaptation and Aftereffects 470
Simultaneous Motion Contrast 470
The Autokinetic Effect 471
10.1.3 Apparent Motion 471
Early Gestalt Investigations 472
Motion Picture Technology 473
The Correspondence Problem of Apparent Motion 474
Short-Range versus Long-Range Apparent Motion 477
The Aperture Problem 479
10.1.4 Physiological Mechanisms 481
The Magno and Parvo Systems 481
Cortical Analysis of Motion 482
Neuropsychology of Motion Perception 483
10.1.5 Computational Theories 484
Delay-and-Compare Networks 484
Edge-Based Models 485
Spatial-Frequency-Based Models 485
Integrating Local Motion 486
10.2 Object Motion 487
10.2.1 Perceiving Object Velocity 487
10.2.2 Depth and Motion 488
Rigid Motion in Depth 489
The Kinetic Depth Effect 489
The Rigidity Heuristic and the Correspondence Problem 490
The Stereo-Kinetic Effect 491
Perception of Nonrigid Motion 492
10.2.3 Long-Range Apparent Motion 493
Apparent Rotation 493
Curved Apparent Motion 495
Conditions for Long-Range Apparent Motion 497
10.2.4 Dynamic Perceptual Organization 498
Grouping by Movement 498
Configural Motion 499
Induced Motion 501
Kinetic Completion and Illusory Figures 502
Anorthoscopic Perception 502
10.3 Self-Motion and Optic Flow 504
10.3.1 Induced Motion of the Self 504
Position and Orientation 504
Balance and Posture 506
10.3.2 Perceiving Self-Motion 506
Direction of Self-Motion 506
Speed of Self-Motion 509
Virtual Reality and Ecological Perception 510
10.4 Understanding Events 511
10.4.1 Biological Motion 511
10.4.2 Perceiving Causation 513
Launching, Triggering, and Entraining Events 513
Perceiving Mass Relations 514
10.4.3 Intuitive Physics 515
Recognizing versus Generating Answers 515
Particle versus Extended Body Motion 517
11 Visual Selection: Eye Movements and Attention
519
11.1 Eye Movements 520
11.1.1 Types of Eye Movements 521
Physiological Nystagmus 521
Saccadic Movements 523
Smooth Pursuit Movements 524
Vergence Movements 525
Vestibular Movements 525
Optokinetic Movements 526
11.1.2 The Physiology of the Oculomotor System 527
11.1.3 Saccadic Exploration of the Visual Environment 528
Patterns of Fixations 528
Transsaccadic Integration 531
11.2 Visual Attention 531
11.2.1 Early versus Late Selection 533
Auditory Attention 533
The Inattention Paradigm 534
The Attentional Blink 537
Change Blindness 538
Intentionally Ignored Information 539
11.2.2 Costs and Benefits of Attention 541
The Attentional Cuing Paradigm 542
Voluntary versus Involuntary Shifts of Attention 543
Three Components of Shifting Attention 544
11.2.3 Theories of Spatial Attention 544
The Spotlight Metaphor 545
The Zoom Lens Metaphor 546
Space-Based versus Object-Based Approaches 547
11.2.4 Selective Attention to Properties 549
The Stroop Effect 549
Integral versus Separable Dimensions 550
11.2.5 Distributed versus Focused Attention 554
Visual Pop-Out 554
Search Asymmetry 556
11.2.6 Feature Integration Theory 556
Conjunction Search 557
Texture Segregation 558
Illusory Conjunctions 558
Problems with Feature Integration Theory 559
Object Files 561
11.2.7 The Physiology of Attention 563
Unilateral Neglect 563
Balint’s Syndrome 565
Brain Imaging Studies 566
Electrophysiological Studies 567
11.2.8 Attention and Eye Movements 568
12 Visual Memory and Imagery 572
12.1 Visual Memory 573
12.1.1 Three Memory Systems 573
12.1.2 Iconic Memory 575
The Partial Report Procedure 575
Duration 576
Content 576
Maintenance 577
Loss 577
Masking 578
Persistence versus Processing 579
12.1.3 Visual Short-Term Memory 580
Visual STM versus Iconic Memory 581
Visual STM versus Visual LTM 582
The Visuo-Spatial Scratch Pad 584
Transsaccadic Memory 585
Conceptual Short-Term Memory 586
12.1.4 Visual Long-Term Memory 588
Three Types of LTM 588
Visual Routines 589
Recall versus Recognition 589
How Good Is Episodic Visual LTM? 590
Visual Imagery as a Mnemonic Device 591
Dual Coding Theory 592
Photographic Memory 593
Mnemonists 594
Neuropsychology of Visual Memory 594
12.1.5 Memory Dynamics 596
Tendencies toward Goodness 596
Effects of Verbal Labels 597
The Misinformation Effect 597
Representational Momentum 601
12.2 Visual Imagery 602
12.2.1 The Analog/Propositional Debate 603
The Analog Position 603
The Propositional Position 604
12.2.2 Mental Transformations 605
Mental Rotation 605
Other Transformations 606
12.2.3 Image Inspection 607
Image Scanning 607
Image Size Effects 607
Mental Psychophysics 608
Reinterpreting Images 608
12.2.4 Kosslyn’s Model of Imagery 609
12.2.5 The Relation of Imagery to Perception 611
Behavioral Evidence 611
Neuropsychological Evidence 612
Brain Imaging Studies 613
13 Visual Awareness 615
13.1 Philosophical Foundations 618
13.1.1 The Mind-Body Problem 618
Dualism 618
Idealism 620
Materialism 621
Behaviorism 621
Functionalism 623
Supervenience 624
13.1.2 The Problem of Other Minds 624
Criteria for Consciousness 624
The Inverted Spectrum Argument 625
Phenomenological Criteria 627
Behavioral Criteria 628
Physiological Criteria 629
Correlational versus Causal Theories 630
13.2 Neuropsychology of Visual Awareness 630
13.2.1 Split-Brain Patients 631
13.2.2 Blindsight 633
The Case History of D.B. 633
Accurate Guessing without Visual Experience 634
The Two Visual Systems Hypothesis 635
Methodological Challenges 635
13.2.3 Unconscious Processing in Neglect and Balint’s Syndrome 636
13.2.4 Unconscious Face Recognition in Prosopagnosia 637
13.3 Visual Awareness in Normal Observers 638
13.3.1 Perceptual Defense 638
13.3.2 Subliminal Perception 639
Marcel’s Experiments 639
Objective versus Subjective Thresholds of Awareness 641
Functional Correlates of Consciousness 642
13.3.3 Inattentional Blindsight 643
13.4 Theories of Consciousness 644
13.4.1 Functional Architecture Theories 645
The STM Hypothesis 645
An Activation-Based Conception of STM 646
The Attention Hypothesis 647
Working Memory Theories 648
The 2.5-D Sketch Theory of Consciousness 649
13.4.2 Biological Theories 649
Activation Thresholds 650
Duration Thresholds 651
The Cortical Hypothesis 651
The Crick/Koch Conjectures 652
ERTAS: The Extended ReticularThalamic Activating System 654
Causal Theories of Consciousness: An Analogy 655
13.4.3 Consciousness and the Limits of Science 656
Relational Structure 657
The Isomorphism Constraint 658
Relation to Functionalism 659
Biology to the Rescue? 661
Appendix A: Psychophysical Methods 665
A. 1 Measuring Thresholds 665
A. 1 1 Method of Adjustment 666
A. 1 2 Method of Limits 666
A. 1.3 Method of Constant Stimuli 666
A. 1 4 The Theoretical Status of Thresholds 667
A.2 Signal Detection Theory 668
A.2.1 Response Bias 668
A.2.2 The Signal Detection Paradigm 668
A.2.3 The Theory of Signal Detectability 669
A.3 Difference Thresholds 671
A.3.1 Just Noticeable Differences 671
A.3.2 Weber’s Law 671
A. 4 Psychophysical Scaling 672
A.4.1 Fechner’s Law 672
A. 4.2 Stevens’s Law 673
Appendix B: Connectionist
B. l Network Behavior 676
Modeling 675
B. 1 1 Unit Behavior 677
Combining Input Activation 677
Determining Output Activation 678
B. 1 2 System Architecture 678
Feedforward Networks 678
Feedback Networks 678
Symmetric Networks 679
Winner-Take-All Networks 679
B. 1 3 Systemic Behavior 679
Graceful Degradation 679
Settling into a Stable State 680
Soft Constraint Satisfaction 680
Pattern Completion 680
B.2 Connectionist Learning Algorithms 681
B.2.1 Back Propagation 681
The Delta Rule 682
The Generalized Delta Rule 683
B. 2 2 Gradient Descent 683
Input Vector Space 683
Partitioning the Input Vector Space 684
State Space 684
Weight Space 685
Weight-Error Space 686
Gradient Descent 686
Local versus Global Minima 686
Appendix C: Color Technology 689
C.l Additive versus Subtractive Color Mixture 690
C. 1.1 Adding versus Multiplying Spectra 691
C.l.2 Maxwell’s Color Triangle 691
C. 1 3 C.I.E. Color Space 692
C.1.4 Subtractive Color Mixture Space? 693
C.2 Color Television 694
C.3 Paints and Dyes 696
C.3.1 Subtractive Combination of Paints 696
C.3.2 Additive Combination of Paints 697
C.4 Color Photography 697
C.5 Color Printing 699
Glossary 701 References 737
Name Index 771
Subject Index 780
Preface
Writing this book has been a long and difficult under¬ taking. Because several good textbooks are available that present the basic facts about vision in a clear and readable fashion, the reader may wonder why I em¬ barked on this journey. Indeed, I often wonder myself! It was not that I thought I could do a better job at what these other books do. Truthfully, I doubt I could. It was that I felt the need for a different kind of textbook, one that accurately reflects the way most modern research scientists think about vision. In fact, the scientific under¬ standing of visual perception has changed profoundly over the past 25 years, and almost all the current text¬ books are still in the “old” mold in both structure and content. New results are included, of course, but the new approach to vision is not.
So what is this new approach? The change in the na¬ ture of visual research began in the 1970s, resulting from the gradual emergence of an interdisciplinary field that I will call vision science. It arose at the intersection of several existing disciplines in which scientists were concerned with image understanding: how the structure of optical images was (or could be) processed to extract useful information about the environment. Perceptual psychologists, psychophysicists, computer scientists, neu¬ rophysiologists, and neuropsychologists who study vision started talking and listening to each other at this time because they began to recognize that they were working on the same problem from different but compatible and complementary perspectives. Vision science is a branch of a larger interdisciplinary endeavor known as cogni¬ tive science that began at about the same time. Cog¬ nitive science is the study of all mental states and processes—not just visual ones—from an even greater variety of methodologically distinct fields, including not only psychology, computer science, and neuroscience, but also linguistics, philosophy, anthropology, sociology, and others. In my own view, vision science is not just one branch of cognitive science, but the single most co-
herent, integrated, and successful branch of cognitive science.
Central to this new approach is the idea that vision is a kind of computation. In living organisms, it occurs in eyes and brains through complex neural information processing, but it can, at least in theory, also take place when information from video cameras is fed to properly programmed digital computers. This idea has had an important unifying effect on the study of vision, en¬ abling psychologists, computer scientists, and physiolo¬ gists to relate their findings to each other in the common language of computation. Vision researchers from dis¬ parate fields now read and cite each other’s work regu¬ larly, participate in interdisciplinary conferences, and collaborate on joint research projects. Indeed, the study of vision is rapidly becoming a unified field in which the boundaries between the component disciplines have become largely transparent.
This interdisciplinary convergence has dominated the cutting edge of vision research for more than two dec¬ ades, but it is curiously underrepresented or even absent in most modern textbooks about perception. One reason is that most textbooks that cover vision also include hearing, taste, touch, and smell. With the exception of hearing, the computational approach has not yet gained a firm foothold in these other sensory modalities. The attempt to provide a consistent framework for research in all modalities thus precludes using the computational approach so dominant in vision research.
Another reason the computational approach to vision has not been well represented in textbooks is that its essential core is theoretical, and introductory textbook authors tend to shy away from theory. The reasons are several, having to do partly with many authors’ lack of computational background, partly with the difficulty of presenting complex quantitative theories clearly without overwhelming the reader, and partly with students’ de¬ sire to learn only things that are “right.” In the final analysis, all phenomena are “right,” and all theories (except one) are presumably “wrong”—although some are “wronger” than others. Students are understandably wary of expending much effort on learning a theory that is surely flawed in some way or other. Such consid¬ erations have led to a generation of textbooks that are as theoretically neutral as possible, usually by being as atheoretical as possible. But the importance of theories in science lies not so much in their ultimate truth or
falsity as in the crucial role they play in understanding known phenomena and in predicting new ones. Given that we have few, if any, truly adequate theories in vision science yet, virtually every insight we have into known phenomena and every predicted new one have been generated by incorrect theories! They are, quite simply, an essential component of vision science.
In this book I have therefore taken the position that it is just as important for students of vision to understand theories as to know about phenomena. Most chapters include a healthy dose of theory, and some (e.g., Chap¬ ters 2 and 8) are almost entirely theoretical. But I have tried to do more than simply catalog bits and pieces of existing theory; I have tried to present a theoretical syn¬ thesis that is internally consistent and globally coherent. This is a tall order, to be sure, for the classical theories of visual perception seem so different as to be diametri¬ cally opposed. Structuralist theory, for example, claimed that wholes are nothing but associations of elementary parts, whereas Gestalt theory championed the primacy of wholes over parts. Helmholtz’s theory of unconscious inference claimed that vision is mediated by thoughtlike deductions, whereas Gibson’s ecological theory coun¬ tered that perception is direct and unmediated. How can a theoretically coherent position be fashioned from such diverse and contradictory components? I do not claim to have succeeded completely in this synthesis, for I do have to deny some important tenets of certain posi¬ tions. But not many. Much has been made of differences that are more apparent than real, and I believe that the computational approach presented in this book can span the vast majority of them without strain. The strong form of Gibson’s claim for direct perception is an ex¬ ception, but weaker forms of this view are quite com¬ patible with the computational view taken in this book, as I explain in Chapter 2.
The unified theoretical viewpoint I present is not so much my own theory as my construction of what I think of as the current “modal theory.” Experts on vision will naturally find aspects of it to which they take exception, but I believe the vast majority will find it consistent with most of their firmly held beliefs. The theoretical frame¬ work I advocate owes much to the influential proposals of the late David Marr and his colleagues at MIT, but this is true of the field in general. In many cases, I have generalized Marr’s specific proposals to make clear how his own detailed theories were examples of a more gen-
eral framework into which a variety of other specific theories fit quite comfortably. Even so, I do not consider the view I describe as exclusively or even primarily Marr’s; it owes just as much to classical perceptual theo¬ rists such as Helmholtz, Wertheimer, Gibson, and Rock. The interweaving of such diverse theoretical ideas is not difficult to achieve, provided one avoids divisive dogma and instead concentrates on the positive contributions of each view.
Because the book is much more theoretical and inter¬ disciplinary than most perception textbooks, it is corre¬ spondingly longer and more difficult. It is designed for an upper division undergraduate course or an entry-level graduate course on vision, most likely as part of a pro¬ gram of study in psychology, cognitive science, or op¬ tometry. I have tried to explain both theories and phenomena clearly enough to be understood by intelli¬ gent, motivated students with no prior background in the field of vision. I do presume that readers have some basic understanding of behavioral experiments, com¬ puter programming, and neurobiology. Those who are unfamiliar with this material may find certain portions of the text more difficult and have to work harder as a result, but the technical prerequisites are intended to be relatively few and low-level, mainly high school geome¬ try and algebra.
Despite the strongly interdisciplinary nature of this book, it is written primarily from a psychological per¬ spective. The reason is simply that I am a psychologist by training, and no matter how seriously I have read the literature in computer vision and visual neuroscience, the core of my viewpoint is still psychological. In keep¬ ing with this perspective, I have avoided presenting the complex mathematical details that would be central to a computer scientist’s presentation of the same topics and the biological details that would figure prominently in a neuroscientist’s presentation. By the same token, I have included details of experimental methods and results that they might well have omitted by nonpsychologists. Vision science may have made the boundaries between disciplines more transparent, but it has not eliminated them. Psychologists still perform experiments on sighted organisms, computer scientists still write programs that extract and transform optical information, and neuro¬ scientists still study the structure and function of the visual nervous system. Such methodological differences will not disappear. Indeed, they should not disappear,
because they are precisely what makes an interdisciplin¬ ary approach desirable. What is needed is a group of vision scientists who are well versed in all these dis¬ ciplines. It is my sincere hope that this book will help create such a community of scientists.
In addition to being used as a textbook, I hope that this book will be useful as a reference text for members of the expanding vision science community. Although the sections describing one’s own field of specialization may seem elementary, the rest of the book can provide useful background material and relatively sophisticated introductions to other areas of vision research. The cov¬ erage is not intended to be at the same level as a profes¬ sional handbook, in which each chapter is expected to be a definitive treatment of a specific topic written by a world-class expert for an audience of other experts, but it is also more accessible and internally consistent than any handbook I have ever seen. It is therefore particu¬ larly useful for someone who wants to get a global view of vision science—the “lay of the land,” if you will— within which the focused chapters that one finds in pro¬ fessional handbooks will fit comfortably and make more sense.
Organization of the Book
Because the aim of this book is to integrate material across disciplines, each chapter includes findings from many different approaches. There is no “physiology chapter,” no “psychophysics chapter,” no “devel¬ opmental chapter,” no “neuropsychology chapter,” and no “computational chapter” in which the separate and often conflicting mini-views within each of these dis¬ ciplines can be conveniently described in isolation. I have avoided this approach because it compartmental¬ izes knowledge, blocking the kind of synthesis that I am trying to achieve and that I view as essential for progress in the field. Rather, the topic of each chapter is discussed from the perspectives of all relevant disciplines, some¬ times including those that writers of textbooks on vision traditionally ignore, such as computer science, philoso¬ phy, and linguistic anthropology. Even within the more standard visual disciplines, the coverage is not uniform because the distribution of knowledge is not uniform. We know a great deal more about the physiology of early image processing, for example, than we do about the physiology of categorization and visual imagery.
This unevenness is merely a reflection of the current state of understanding.
The overall organization of the book is defined by its three parts: Foundations, Spatial Vision, and Visual Dynamics.
Foundations. The Foundations section covers a basic introduction to the interdisciplinary science of vision. Chapter 1 introduces the problem of visual perception and sets forth an interdisciplinary framework for ap¬ proaching it. It covers many of the most important perceptual, optical, and physiological facts on which vision is based. Chapter 2 then discusses theoretical approaches to vision from an historical perspective. It covers the classical theories of vision as well as the infor¬ mation processing (or computational) approach, includ¬ ing several important proposals from the work of the late David Marr (1982) that play a large role in defining the superstructure of the rest of the book. The key idea is that visual perception can be analyzed into a sequence of four basic stages: one that deals with extracting image structure (Marr’s “primal sketch”), one that deals with recovering surfaces in depth (Marr’s “2.5-D sketch”), one that deals with describing 3-D objects (Marr’s “vol¬ umetric descriptions”), and one that deals with identify¬ ing objects in terms of known categories. This sequence of processes—which I call image-based, surface-based, objectbased, and category-based—is then traced for each of the major topics covered in the book: color, space, and mo¬ tion perception. The final chapter of the Foundations section, Chapter 3, is a long but important one. It tells “the color story,” which spans vision science from the physiology of retinal receptors to the linguistic analysis of color names in different cultures of the world. Its importance derives from the fact that the current under¬ standing of color processing illustrates better than any other single example in all of cognitive science why an integrated, interdisciplinary approach is necessary for a complete understanding of a perceptual domain.
Spatial Vision. Chapters 4 through 9 cover spatial perception as a sequence of processes: extracting image structure (Chapter 4), recovering oriented surfaces in depth (Chapter 5), organizing perception into coherent objects (Chapter 6), perceiving object properties and parts (Chapter 7), representing shape (Chapter 8), and identifying objects as members of known categories
(Chapter 9). This material on spatial processing of im¬ ages is the heart and soul of classical visual perception. Because it is much more complex than color processing, we understand it much less well. It is hard at times not to be overwhelmed by the mountains of facts and frus¬ trated at the lack of good theory, but I believe we are beginning to get some clearer notion of how this all fits together.
Visual Dynamics. The final section concerns percep¬ tual dynamics: how visual perception and its aftereffects change over time. Perception of motion and events is the first topic considered (Chapter 10), being essentially an extension of spatial perception to the domain of space-time. Then we discuss ways in which the visual system selects different information over time by makingovert eye movements and covert attentional adjustments (Chapter 11). Next we consider memory for visual infor¬ mation within a multistore framework—iconic memory, short-term visual memory, and long-term visual mem¬ ory—and examine how such stored information can be reconstructed and transformed in visual imagery (Chap¬ ter 12). Finally, Chapter 13 takes up what is perhaps the most fascinating of all topics: the nature of visual awareness (and its absence in certain neurological syn¬ dromes) and various attempts at explaining it. This topic is very much on the cutting edge of modern vision sci¬ ence and is finally getting the attention that it deserves.
Tailoring the Book to Different Needs
Because the book contains more topics and material than can comfortably fit into any single-term under¬ graduate course, instructors are encouraged to be selec¬ tive in using it. I have included too much rather than too little because I find it easier to skip what I do not want to cover in a single unified textbook than to find external readings that cover the desired material at an appropri¬ ate level and in a framework that is compatible with the main textbook—a nearly impossible task, I have found. There are several ways of tailoring the present book to different needs. Most obviously, certain chapters can be skipped in their entirety. For example, if color is not a high priority, Chapter 3 can be omitted with only minor ramifications for later chapters. Chapter 10 on motion perception is likewise reasonably independent of the rest of the book. For courses that are restricted to
classical visual perception, Chapter 11 on eye move¬ ments and attention and Chapter 12 on memory and imagery are probably the least relevant. A course em¬ phasizing high-level vision can reasonably omit Chapter 4 on image-based processing.
Another approach to selective coverage is omitting subsections within chapters. For traditional courses on the psychology of vision, the sections on computational theory and other technical material may be eliminated or assigned as optional. (One effective approach I have used is to teach an honors section of the course for addi¬ tional credit in which the more difficult material is required and other sections for which it is not.) Elimi¬ nating this material has the advantages of making the book substantially shorter and easier to understand for students with less technical backgrounds. The devel¬ opmental sections can also generally be omitted without much affecting the book’s continuity and cohesion.
For students with strong scientific backgrounds who are highly motivated to learn about modern vision science, I encourage instructors to use as much of the book as possible. It is perfectly reasonable, for example, to cover the entire book in a graduate course on vision that lasts a full semester.
Acknowledgments
There are many people I wish to thank for helping me in various phases of writing this book. First and foremost, I gratefully acknowledge my debt to my late colleague and friend, Irvin Rock, to whom this book is dedicated. Irv not only taught me about perception in his own gen¬ tle, probing, inimitable way, but he also read and com¬ mented on earlier drafts of the first nine chapters before his death in 1995. Moreover, his 1975 textbook An Intro¬ duction to Perception served as a model for this one in cer¬ tain important ways. In that book, Irv tried to present the phenomena of visual perception at an introductory level yet within a coherent and principled theoretical view of perception as a problem solving process. While it was still in print, it was my favorite perception text, and I know that some instructors continue to use it in photocopied readers to this day.
Irv’s influence on this book has been substantial, as careful readers will surely discover. Had he lived, I be¬ lieve his continued contributions would have improved it further and kept me from making some mistakes I
doubtless have made in his absence. After Irv’s death, Arien Mack, one of Irv’s most distinguished students and collaborators, became my primary reviewer for the remaining chapters of the book. One or the other of them has read and commented on every chapter. Many other experts in vision science have also read more limited portions of the book, either at my own request or at that of MIT Press, and provided valuable comments on material in their specialty areas. I wish to thank the following scholars, plus several anonymous re¬ viewers, for the time and effort they spent in evaluating portions of the manuscript:
Chapter 1: Irvin Rock, Jack Gallant, Paul Kube
Chapter 2: Irvin Rock, James Cutting, Ulric Neisser, Paul Kube, Jitendra Malik, and an anonymous re¬ viewer
Chapter 3: Irvin Rock, Karen DeValois, Alan Gilchrist, C. Lawrence Hardin, Paul Kay, and an anonymous reviewer
Chapter 4: Irvin Rock, Jitendra Malik, Jack Gallant, Ken Nakayama, and an anonymous reviewer
Chapter 5: Irvin Rock, Jitendra Malik, Ken Nakayama, and an anonymous reviewer
Chapter 6: Irvin Rock, Jitendra Malik, and Michael Kubovy
Chapter 7: Irvin Rock, Arien Mack, and an anonymous reviewer
Chapter 8: Irvin Rock, John Hummel, and an anony¬ mous reviewer
Chapter 9: Irvin Rock, John Hummel, and an anony¬ mous reviewer
Chapter 10: Arien Mack, James Cutting, Dennis Prof¬ fitt, and an anonymous reviewer
Chapter 11: Arien Mack, Michael Posner, Anne Treisman, and William Prinzmetal
Chapter 12: Arien Mack and Martha Farah
Chapter 13: Arien Mack, Alison Gopnik, John Watson, Bruce Mangan, Bernard Baars, and C. Lawrence Hardin
Appendix A: Ken Nakayama and Ervin Hafter
Appendix B: John Kruschke and Jerome Leldman
Appendix C: Alan Gilchrist
Several students, postdoctoral fellows, and visitors in my lab have also taken the time to comment on various portions of the book. Without differentiating among chapters, I wish to thank Daniel Levitin, Elisabeth Pa-
chiere, Joel Norman, Akira Shimaya, Diane Beck, Justin Beck, Sheryl Ehrlich, Craig Fox, Jonathan Neff, Charles Schreiber, and Christopher Stecker for their helpful comments. In addition, I would like to thank Christo¬ pher Linnett, Sheryl Ehrlich, Diane Beck, Thomas Leung, William Prinzmetal, Gregory Larson for doing some of the more complex and technical illustra¬ tions, Lisa Hamilton for working on design issues, and Richard Powers for improving my work environment. For their help in copy editing and preparing the final manuscript for production, I would like to thank Bar¬ bara Willette and Peggy Gordon, respectively. Last, but not least, I must thank Edward Hubbard for his tireless help in tracking down references, obtaining permission to reprint figures, checking page proofs, and generally overseeing the final stages of preparing the manuscript for publication.
This book took a long time to write—certainly a good deal longer than I had planned or than I would like to admit—and its writing put a significant strain on all other aspects of my life. During this time, many people have contributed emotional support and understanding, for which they are due both thanks for their help and apologies for the time this project has stolen from them. They include Paul Harris, Stephen Forsling, David Shiver, and Andy Utiger, as well as Linda, Emily, and Nathan Palmer.
Foundations
