In seeking to understand how the listener makes sense of music, should psychologists treat it as a set of arbitrary conventions (as David Hilbert views mathematics), or as a description of reality (as Plato sees both mathematics and music), or as a property of mind (as Chomsky regards language)? All three approaches are essential, since all three characteristics — pattern-structural, acoustic, and grammatical — play their part; moreover they function interdependently in musical perception. Though a musical grammar may have been culturally evolved, acoustic characteristics and the constraints of performance will have contributed to its formation. That we perceive music phrase by phrase is not only because of repeated practice in hearing conventional design, but also because phrase length, as in sentence structure, has been historically constrained by what can be managed in one breath. The human auditory system evolved not only to be receptive (as are those of other species) to the sounds of nature, but also apparently to be 'wired for speech'. Vowel perception, like that of musical pitch and timbre, depends on the spectral characteristics of the complex waveform. Even so, perception phoneme by phoneme is not totally an acoustic process, for it also depends on knowledge of phonological constraints. In both speech and music, this 'wiring' for sequential processing of meaningful acoustic events must form the basis for the subsequent, more specialized expansion of perceptual skill through experience.
1. Music as pattern
2. Music and psychoacoustics
3. Music as language
4. Music and performance
5. Music and aesthetics
1. Music as pattern
The Hilbertian approach is relevant here, since the brain is biased towards detecting regularities, the non-random organization of pattern, irrespective of sense modality. Experiments on auditory temporal pattern (Garner 1974, Jones 1978) treat music as arbitrary design, evoking perception of features such as symmetry, repetition, and imitation, which occur also in other non-musical symbolic or visual design. The span of short-term memory sets a limit to the perception of auditory pattern; musical palindromes are difficult to perceive for this reason, whereas visual perception of bilateral symmetry is immediate. Listeners tend to impose structure and to discover or unconsciously apply the rule by which a pattern is defined. Formal transformations of melodies used in serial music are perceptible (depending on their length) in an ascending order of difficulty from exact repetition to inversion, retrograde, and retrograde inversion. Perceiving retrograde tunes is a particular tax upon short-term memory, and the strong salience of forward temporal order is apparent if one plays backwards the tape of a well-known piece, even one of equal note values, where rhythm plays no part. In experiments where subjects are asked to identify the starting point and the segmentation pattern of a sequence which is continually recurring, like auditory wallpaper, their perception tends to conform to Gestalt principles of organization. Patterns of eight or ten recurring events, composed of only two pitch elements, seem to evoke a figure-ground perception, sometimes reversible, as in an Escher picture. A slow tempo of presentation affords active coding into memory, whereas a fast one yields a more passively received Gestalt. The formal devices of composition are amenable to this approach, often presented by psychologists in terms of information theory. But such an approach can give only an inadequate account of musical perception where it disregards the listener's use of an implicit tonal grammar in perceiving even quite simple melodies.2. Music and psychoacoustics
The second approach, the acoustic one, is the study of constraints imposed upon the perceptibility of musical phrases by psychoacoustic factors. Throughout musical history these constraints have been intuitively respected or exploited by composers, even when not yet scientifically defined. For instance, tonal proximity in melodic steps is an important perceptual organizing principle. The ear requires some milliseconds of extra processing time to monitor large pitch intervals in melody. This need has been explained as due either to the 'critical band' (the theory that the ear functions neurally as a series of band-pass filters), or to the brain's momentary conflict between integrating the frequency change as one of pitch or of timbre (this also is a neurological theory). Palestrina and other 16th-century composers observed the rule that, after a melodic leap in polyphonic music, voice must return by stepwise motion within the compass of the leap — an intuitive recognition of the ear's need of extra steadying time. Even a well-known tune, when transposed note by note into disparate octave registers, becomes unrecognizable. Yet this very widely spaced layout is a characteristic device of 20th-century serial music, and either of the two psychoacoustic theories mentioned would suggest why it is notoriously difficult to perceive. The salience of tonal proximity in perceptual organization is also apparent when two tunes, each consisting of large ascending and descending leaps, are presented one to each ear (Deutsch 1975). They are heard as consecutive by the tonal proximity of the notes arriving alternately at each ear, and not, as might be expected, as a left ear tune and a right ear tune. A similar perception by tonal proximity which overrides that by separate sources occurs with the layout between violin parts in Tchaikovsky's Sixth Symphony (see Fig. 1).J. S. Bach, in his solo violin partitas, exploits a related phenomenon now called 'auditory stream segregation' (McAdams and Bregman 1979), where alternating notes will appear to separate into two coexistent tunes, depending on the tempo and on the pitch separation between the alternate notes. An eight-note tune, continually recurring, will perceptually separate into more than two streams of ever more restricted frequency ranges: the faster the tempo, the more streams are heard. Prestissimo gives the experience of eight coexistent streams forming a continuous chord or an inharmonic timbre (see Fig. 2). However, streaming does depend on what is being listened for, whether streaming itself or coherence, so it cannot be entirely attributed to neural factors. In some sequences, one of the eight notes may be 'captured' by either of two adjacent streams; the brain makes the best bet on the basis either of tonal proximity or of harmonic pleasingness. This latter criterion, however unconsciously it is used, reflects the listener's internalized musical grammar, derived from past experience, which will affect his performance in all experimental tasks, however musically neutral they may be. In general, the validity of explaining perception as due to acoustic rather than attentional processes rests on the experimenter's selection of subjects, how rigorously he distinguishes among non-musicians, musicians, those with absolute pitch, and acoustic engineers, for experience endows each group with demonstrably different coding processes.
The drawback of many psychoacoustic experiments hitherto has been their endeavour to regard perception as context free. Furthermore, the functions — amplitude, duration, frequency, and spectral complexity of the waveform — of the four main attributes of music — loudness, rhythm, pitch, and timbre — have often been treated independently of each other. This has perpetuated the philosophical notion of 'raw sense data'. For music the notion originated in the Pythagorean naive realism concerning sensory coding, which assumed that sensory processes exactly matched physical events in the world. It survived in the assumptions of classical music theory (e.g. that of Rameau and Tartini) and in Helmholtz's model of the ear as a frequency analyser which coded each discriminable pitch by its specific nerve. Combination or difference tones were thought to be manufactured in the waveform in the ear, rather than neurally. But the four attributes are by no means independent; loudness also depends on frequency, pitch is affected by amplitude, and timbre by frequency. Paul Divenyi (1971) shows that perception of time intervals is affected by the frequency separation between notes, and this has obvious importance for rhythm. Then also, there is no clear distinction between pitch and timbre, but rather a continuous dimension from the unequivocal pitch of pure tones to the spectral inharmonicity of church bells (though their fundamental pitch is distinguishable), to the vague 'pitchiness' of noise within a narrow band of wavelength (Erickson 1975). Timbre is of course multidimensional, being also affected by fluctuations of harmonics or of pitch (in vibrato), and by the non-pitched starting noises of instruments, or 'transients'. A similar continuous dimension can be traced from the single complex tone to the chord, for if a single harmonic of a complex tone is sufficiently amplified it separates from the fundamental, and a chord is heard. This property is exploited in the chant of a Tibetan monk when he sings alone in two-part harmony (Smith et al. 1965).
Fig. 1. A section of the violin parts of Tchaikovsky's Sixth Symphony, showing a the music as it is actually played, and b how it is perceived by the listener.
The rise of computer and electronically synthesized music has greatly expanded the vocabulary of music so that, as the American composer Milton Babbitt has remarked, the limits are no longer those of sound production, but rather those of the human auditory system. Rapid changes of timbre are difficult to follow, just as speech recorded syllable by syllable in different voices is incomprehensible. Richard Warren and his colleagues (Warren and Obusek 1972, Warren 1974) have shown that subjects have great difficulty in identifying the order of recurring sequences of sounds which are unrelated in timbre (for instance, a high tone, a hiss, a lower tone, and a buzz) although they are presented at a slower speed than that necessary for auditory temporal resolution. Yet if the two tones are placed adjacently within the four-event sequence, performance is improved. That continuity of timbre is important to perceptual organization is also apparent where a single, pure tone is interrupted by noise, the cessation of the tone being exactly synchronized with the start of the noise, and vice versa (see Fig. 3). The pure tone is heard as continuing through the noise, much as in vision one object is seen as existing behind another by which it is occluded. (See illusions.) Computer music is easily perceived in the degree to which the sounds resemble or are systematic near distortions of natural sounds. Composers of computer music not only are able to use these parameters of sound from experience but, in collaboration with psychologists, are also able to assess the reasons for the limits to the listeners' perceptual skills (IRCAM Reports).
Fig. 2. The decomposition of an acoustic sequence into smaller and smaller perceptual streams as the frequency separation between the tones or the tempo of the sequence increases. In the latter case, a point is ultimately reached where one can no longer perceive individual tonal events; a texture or timbre is heard instead.
Fig. 3. The stimuli used by Bregman and Dannenbring (1977) consisted of a pure tone interrupted by noise. The level of the tone leading into and coming out of the noise burst was varied as shown. The greatest degree of continuity was found in the centre case where there was no change in level.
3. Music as language
The third approach, the grammatical one, treats music as analogous to language, since music also is hierarchically organized and makes selective use of the same neurally close-knit system of voice, ear, and brain, though without denotative meaning, music is both syntactic and communicative (of subjective states). Its two linguistic elements, melodic phrasing, which reflects the intonations of speech, and the formal, syntactic devices of composition, imply that the listener understands both the expressiveness of contour and the 'argument' of musical form. Even the musically untutored listener acquires a musical grammar, a system of rules unconsciously applied, by which he makes sense of music and would detect 'wrong notes' in an unfamiliar work in the tonal idiom, though unable to name the notes or to state the rules. This grammar implies that the single note is characterized not only by its tone height, and its chroma (all B flats sound alike), but also by its grammatic function (e.g. whether it is, at a particular point, functioning as a tonic, leading note, or unaccented passing note). As a word in a sentence is defined by its context, even more so is a note in music.Formal theories of music and artificial intelligence models seek to define how the listener correctly perceives rhythm, key, tonal modulations, and thematic organization. Some models are based on traditional music theory and linguistics (Winograd 1968, Longuet-Higgins 1971), or on Gestalt principles (Tenney and Polansky 1980). A more recent theory (Lerdahl and Jackendoff 1981) covers formal, acoustic, and quasi-linguistic aspects, and accommodates the fact that a musical phrase might admit of more than one correct parsing by distinguishing 'well-formedness' rules from those 'preference' rules which can account for conscious and unconscious organizing principles of musical perception. Its emphasis on the interactions between rhythmic and tonal aspects provides a useful antidote to some research where such interactions have hitherto been disregarded.
Experiments show that, as with language, salience of originally perceived or imposed rhythmic groupings is very strong in subsequent recognition memory, and tunes ending in cadence are easier to remember than those which are harmonically inconclusive. Most musicians would agree that tonal grammar is dominant, that it is not easy to eradicate entirely one's tonal habits of listening when hearing non-tonal idioms.
However, the swiftly moving innovations in the history of tonal music, particularly of the last two centuries, undermine the analogy of its grammar with that of language, which is anchored by semantic meaning. Any ancient Greek who conversed with Socrates could equally well have done so with Wittgenstein, but having heard only the musical grammar of the Delphic hymns he would not naturally make sense of Boulez, Beethoven, or even Byrd. Musical style and comprehensibility have increasingly depended upon the listener's developing tolerance of protracted dissonance through his grammatical understanding of its possible resolution in consonance, implicit in his familiarity with past and present idioms. That his long-term memory store, upon which present comprehension depends, is more idiosyncratic than it could ever be for language presents a difficulty to the psychologist in testing general theories of musical grammar.
In psycholinguistics the experimenter tests a linguistic theory which treats grammaticality as paramount (i.e. whether a string of words constitutes a sentence), whereas ambiguity plays a much lesser role in the whole theory (i.e. whether a sentence is susceptible of more than one meaning and consequently of more than one structural description). But phrases in music frequently admit of differing interpretations and may be ambiguous. Indeed, the performing musician's enduring task is to choose, between the various possibilities of phrasing, the one that to him seems to represent the composer's message most faithfully and eloquently, and his interpretation is to a large extent judged by its internal coherence. Thus the psychologist of music must work with a grammar which can accommodate the latitude of perhaps many equally 'correct' parsings of a musical structure.
The three approaches outlined here, each concerned with an essential factor operative in musical cognition, remain at present more experimentally distinct than is warranted by their interrelation in the perception of musical structure. In making sense of music, its attributes of contour, interval, tonality, modulation, rhythm, symmetry, repetition, inversion, imitation, and the like must in turn and at different moments assume different shares of the total cognitive process, and must influence the nature of perceptual grouping or coding processes. Exactly how this fluctuating process operates remains the fundamental question.
4. Music and performance
Musical perception and performance are both dependent on short-term memory and expectancy. 'The practically cognised present is no knife-edge but a saddleback, with a certain breadth of its own, on which we sit perched, and from which we look in two directions into time' (William James, 1890). From this perch the performer monitors and constantly corrects the flow, shape, and tone quality as it occurs, by a highly integrated system of coexistent codes: auditory, visual, and kinaesthetic. Implicit vocalizing may underlie musical perception, much as Liberman holds that we perceive speech by 'internal generation' at some neural level below that of overt response. But the acquired codes of the musician instrumentalist are more complex, including visual ones for musical notation and instrumental fingering, and kinaesthetic ones for fingering and more general muscular movement. Kinaesthetic imagery for one's own instrument can also underlie one's perception while only listening. Karl Lashley (1951) cites the musician's performance when he emphasizes that all human serial behaviour (speech, gesture, and perceptual motor skill) is grammatically structured, on hierarchically ordered levels of organization. Well-practised sequences occurring too fast for monitoring at the level of individual events are perceived and executed at a higher level of pre-planned groups. In music (unlike, for instance, Morse code) the flow of 'information' to be monitored or executed is not constant. Not only the number of consecutive events but also the denseness of harmonic texture may vary from moment to moment, yet the performer maintains a steady flow of action and perception through these variations in information load. The basis here is principally the intrinsic feedback of the nervous system, amplified by the extrinsic feedback of sight and sound, e.g. of the conductor's beat and the sound of other instruments.Although the appropriate model for research on performance may well be the cybernetic one, it is not surprising that little has yet been done to tease out the components and interactions in this very complex skill. Seashore's research (1938) on intonation and vibrato anticipated the present experimental technique of wiring up musical instruments to computers which record the infinitesimal variations in tone and rhythm that distinguish live performance from computer deadpan accuracy, and one artist from another. Analysis of these recordings shows the variation to be consistent in respect of some general principles of rhythmic displacement, or differences of intonation according to the function of a particular note within a tonality. Some experiments on sight-reading indicate that good sight-readers take in and hold in short-term memory (the eye–hand span) phrase-length chunks (see chunking), rather than a steady bar-by-bar succession of chunks at the rate of their performance.
A promising source of evidence as to how the components of musical skill may be integrated is the clinical evidence on amusia, the breakdown of a previously established capability for music due to brain pathology. Although there are strong indications that in right-handed people music is stored in the non-dominant hemisphere of the brain, this is by no means invariably so, nor are musical and linguistic functions entirely lateralized. There are many mental abilities common to both speech and language, for instance perception of rhythm and of temporal order, and these seem to be processed in the dominant hemisphere. In left-handers, either hemisphere can mediate many musical functions, and in some professional musicians a dominant life work may be stored in the dominant hemisphere. The distinctions between the roles of each hemisphere have been oversimplified in investigations hitherto, although broadly they are apparent. Nevertheless, research which correlates a patient's performance with damage to a particular brain structure is gradually building up a picture which illuminates our understanding of the nature of musical skill.
An equally promising area is that of developmental psychology, an area that has been largely centred on aptitude tests. The early acquisition of the unconscious ability to operate the rules of musical grammar can be as rich a source of psychological theory as that of the child's acquisition of language. Teplov (1966) finds that children will complete half-finished musical phrases presented to them, with a fine sense of their tonality, rhythmic character, and implied harmonic cadence. The special aptitude of absolute pitch, the possession of which correlates with the early age at which the names of the notes were learned, illustrates the power of category systems in expanding musical intelligence. If more parents were equipped to teach note names as easily as colour names, it is possible that music might more easily resemble an artistic lingua franca.
5. Music and aesthetics
Most musicians would agree that, as well as the sensory and cognitive aspects we have discussed, aesthetic and emotional factors also play a part in perceiving and remembering music, but it is less easy to see how these can be approached experimentally. The belief that memory for melodic contour and implicit harmony is allied to its affective character is well described by Deryck Cooke (1958), whose theory gives general principles why, for instance, we all experience, with Browning, 'those minor thirds so plaintive, Sixths diminished sigh on sigh'. This characterization is, of course, dependent not on pitch intervals alone, but also on rhythm, phrasing, and tempo (for instance, descending couplet phrasing is a particularly plaintive use of minor thirds). The music of non-Western cultures, based on quite different scale systems, no doubt has different aesthetic and emotional significances which are nevertheless experienced to similar degrees in those cultures. In Western music, the factor of musical imagery is partly understood universally by musical people, and is partly idiosyncratic to each composer and listener. Composers have self-consistent mood associations with key colour and timbre. Bach's use of the brazen timbre of the D major trumpet associated that key, for him, with jubilance, and the D major open string basis of tone colour enhances this characterization for most composers and hence for most listeners.Hitherto experiments on aesthetic aspects have centred largely on subjects' ratings for a certain attribute, or for the more general 'pleasingness'; they were often directed to whole works, or to very large segments of them, rather than to phrases (Schoen 1927). There are also correlations of measures of autonomic arousal with hearing or performing certain music, including one such study with the conductor Herbert von Karajan as subject (Harrer and Harrer 1977). Sometimes responses are related to a measure of the information content of a work (Berlyne 1974). Psychologists may well feel that judgements of whole works do not yield data that is specific enough, and they may be daunted by the great number of confounding variables, such as social, cultural, or fashion determinants of taste.
As yet no systematic experimental study of Cooke's theory has emerged relating it to musical education, though a pilot study (Gabriel 1978) showed that, for 22 non-musician students, Cooke's characterizations of musical phrases were not experienced. This evoked a music theorist's objection to the theory itself (Cazden 1979), and another to the validity of using 'deadpan' sine wave sequences as an experimental test of a theory concerning real live music (Nettheim 1979). To these one must add an objection to the restricted choice of subjects. A study using real live music, and musician subjects, to test Leonard Meyer's theory of the perception of certain melodic characteristics which he calls 'archetypes' attributes the variability of response to the degree of complexity in the underlying hierarchic phrase structure (Rosner and Meyer 1981). It would seem that experimental musical aesthetics may have to await further progress in the rapprochement between experimental psychology and the theories of musical grammar discussed earlier.
Is it possible for the analytic methods of science to contribute a comprehensive account of the flexible, living performance and enjoyment of music? At the sensory and cognitive levels there has been substantial and illuminating progress. On the other hand, at present the aesthetic level of the musician's use of imagery, particularly with contour, key colour, tone quality, and emotional association, seems, like the painter's idiosyncratic choice of palette, to be outside the realm of experimental psychology.
(Published 1987)
— Lady Natasha Spender
- Bibliography
- Berlyne, D. (1974). Studies in the New Experimental Aesthetics.
- Bregman, A., and Dannenbring, G. (1977). 'Auditory continuity and amplitude edges'. Canadian Journal of Psychology, 31.
- Burns, E. (1974). 'Octave adjustment by non-Western musicians'. Journal of the Acoustical Society of America, 56.
- Cazden, N. (1979). 'Can verbal meanings inhere in fragments of music?' Psychology of Music, 7/2.
- Cooke, D. (1958). The Language of Music.
- Deutsch, D. (1975). 'Musical illusions'. Scientific American, 233/4.
- Divenyi, P. (1971). 'The rhythmic perception of micro-melodies'. In Gordon, R. (ed.), Research in the Psychology of Music.
- Erickson, R. (1975). Sound Structure in Music.
- Gabriel, C. (1978). 'An experimental study of Deryck Cooke's theory of music and meaning'. Psychology of Music, 16/1.
- Garner, W. (1974). The Processing of Information and Structure.
- Harrer, G., and Harrer, H. (1977). 'Music, emotion and autonomic function'. In Critchley, M., and Henson, J. (eds.), Music and the Brain.
- Helmholtz, H. (1863). On the Sensations of Tone (trans. 1954).
- IRCAM Reports, Pompidou Centre, Paris.
- James, W. (1890). Principles of Psychology.
- Jones, M. Reiss (1978). 'Auditory patterns: studies in the perception of structure'. In Cartarette and Friedman, Handbook of Perception, 8.
- Lashley, K. (1951). 'The problem of serial order in behavior'. In Jeffries, L. (ed.), Cerebral Mechanisms in Behavior.
- Lerdahl, F., and Jackendoff, R. (1981). A Generative Theory of Tonal Music.
- Longuet-Higgins, C. (1971). 'On interpreting Bach'. Machine Intelligence, 16.
- McAdams, S., and Bregman, A. (1979). 'Hearing musical streams'. Computer Music Journal, 3/4.
- Nettheim, N. (1979). 'Comment on a paper by Gabriel on Cooke's theory'. Psychology of Music, 17/2.
- Rameau, J. P. (1750). Treatise on Harmony (trans. 1971).
- Rosner, B., and Meyer, L. (1981). Melodic Processes and the Perception of Music.
- Schoen, M. (1927). The Effects of Music.
- Seashore, C. (1938). The Psychology of Music (repr. 1967).
- Smith, H., Stevens, K., and Tomlinson, R. (1965). 'On an unusual mode of chanting by certain Tibetan lamas'. Journal of the Acoustical Society of America, 61/5.
- Sundberg, J., and Lindquist, J. (1973). 'Musical octaves and pitch'. Journal of the Acoustical Society of America, 54.
- Tenney, J., and Polansky, L. (1980). 'Temporal Gestalt perception in music'. Journal of Music Theory.
- Teplov, B. (1966). Psychologie des aptitudes musicales.
- Tobias, J. (1970, 1972). Foundations of Modern Auditory Theory, 2 vols.
- Ward, W. D. (1954). 'Subjective musical pitch'. Journal of the Acoustical Society of America, 26.
- Warren, R. (1974). 'Auditory temporal discrimination by trained listeners'. Cognitive Psychology, 6.
- — — and Obusek, C. (1972). 'Identification of temporal order within auditory sequences'. Perception and Psychophysics, 12.
- Winograd, T. (1968). 'Linguistics and the computer analysis of tonal harmony'. Journal of Music Theory, 12.
