Tip! Click on references as they appear in the text in order to see the full bibliographic records.

JMM 5, Summer 2007, section 6 — A JMM Essay

Nigel Nettheim
Composer-Specific Computer Animations

6.1. Introduction

People sensitive to music may experience, whether consciously or not, a pulse-like sensation running throughout it. In some types of music that response is commonly observable in an external manifestation, such as foot-tapping in jazz, where it may be referred to as the groove. In other types, however, including Western art music it is scarcely visible: audiences at symphonic concerts, for instance, normally retain their response to the music within themselves. Any physical response is likely to be internal or, if external, of such subtlety as to be difficult of access for further study.

The lack of an accessible physical response to Western art music poses a problem for research into the nature and effect of the music. Several attempts have been made to overcome that problem; a historical survey of those attempts is sketched in, for instance, Shove and Repp (1995) and Nettheim (1996). Here I will describe (but not necessarily advocate) just one attempt, that of Becking (1928).

Becking allowed a small hand-held baton to move along with the music, thus manifesting sympathetic motions (Mitbewegungen). He graphed his observed shapes in diagrams which became known as Becking Curves specific to each of a number of composers; examples will shortly be seen. The purpose of these motions was not to control a performance but instead the opposite: to respond to a performance, an important distinction which I have endeavoured to convey through the term armchair conducting. Thus armchair conducting differs essentially from the conducting of a live orchestra. Orchestral conductors carry out practical functions including the control of ensemble and tempo, and indicate the beats of the bar by means of strokes designed to be seen clearly at a distance. Further, live conducting may slightly anticipate the musical events, whereas armchair conducting slightly follows them or swims along with them; although the time difference involved in that distinction is small, it is conceptually vital. Live conducting, therefore, does not readily lend itself to the study of those qualities of the music heard which accord with our present purpose, now to be explained more fully, whereas armchair conducting does.

Sympathetic motions, according to Becking, can reveal the particular world view, or attitude to the given, of each composer. The given is represented by gravity, the universal force. Because it has so pervasive an influence on humans, gravity is often taken for granted and overlooked in discussions of the meaning of music, whereas it might better occupy a primary place. A specific composer may take attitudes such as are indicated briefly as follows (Becking, 1928, pp.21-22, my translation):

Gravity is simply given to the composer; he cannot create it. He approaches it as a force of nature that one can put to work for oneself. He brings it under control, shapes and manipulates it, and merges it into the streaming rhythm. He can respect it willingly or try to subdue it; he can act in far-reaching idealism as if he were creating it; he can behave as if it were not there, as the late Romantics did—but in fact he cannot create or abolish it, for it always remains there and remains itself, always basically the same. Where it holds sway over the musical events we beat [p. 22] downward, whether we do so with joy or fraught with doubt, fervently or reticently—but we must go along with it; we cannot escape from its influence. So every creator or interpreter, having a rhythm to be shaped, is confronted by gravity as something simply given, as a thing-in-itself, with which he has to come to terms. But the manner in which he fulfils this task is connected most intimately with his attitude to what is given in general, that is, to the world. Man's attitude in the face of the thing-in-itself is reflected in the rhythm of all his actions. The philosopher puts it into ideas, the artist represents it in plastic form, and the "common man" reveals it in the tasks of everyday life. Personalities, nations and times are differentiated according to the fundamental statements they make.

Becking has discussed such attitudes at length, referring to many examples from musical compositions. Those attitudes constitute only one strand of the meaning of music, however, for the emotional and story-telling aspects of a particular work are not thereby taken into account. Any composer can express joy, for instance; but Beethoven’s joy and Mozart’s joy are two different things, and it is that difference which is relevant for the present purpose. Further, the composers’ attitudes may be grouped and studied according to their nationality and historical time; but again my purpose here is rather to concentrate attention on the composer-specific attitudes and to present the corresponding motions in an effective way.

Conventional musical analysis cannot fully explain the phenomena studied here. Thus instead of (discrete) rhythmical analysis we have the (continuous) flow of the response to the music, which has sometimes been referred to by the Greek term ‘rhythmos’.[1] Style analysis will also not yield the desired result, for music in a given form or having a given harmonic or melodic progression, for instance, may be filled with content representing the different attitudes of different composers (Becking (1928), pp. 17, 45).

Becking dealt with many composers, from which I select here Bach, Mozart, Beethoven and Schubert; the corresponding curves are shown in Figures 1-4 (as seen by a right-handed person executing them). The use of a baton is of some importance in that it provides work to be done, and an attitude to be taken, in relation to the pervasive force of gravity, which is represented as downward on the page. The varying thickness of the curve represents the varying strength applied (taking into account both agonist and antagonist muscles), but the varying velocity is not shown graphically. Becking added verbal indications of the quality of the beating in each case and these are shown in the captions, as are also my own brief descriptions of the motions. The animations presented later will suggest more explicitly how each curve may be executed. It should be acknowledged, however, that no representation of this phenomenon can be a fully adequate substitute for experiencing the phenomenon itself.

Figure 1 Becking curve for Bach, two versions. Becking: Arm! Die Abstriche barock höhlend (Use the arm. Downstrokes are hollowed out in the baroque manner.) Nettheim: The downbeat motion starts at or near the upper right. Move to the left and back, somewhat like a ticking clock. Source: Becking (1928): end-plate (left), p.55 (right).
Figure 2

Becking curve for Mozart. Becking: Selbstverständlich abwärts. Sorgfältig getönt (Naturally down. Carefully shaded.) Nettheim: The downbeat motion starts at or near the upper right. Move briskly down, swing around and return via the notch. Source: Becking (1928), end-plate.
Figure 3

Becking curve for Beethoven. Becking: Tief abwärts zwingen (Force deep downwards.) Nettheim: The downbeat motion starts at or near the upper right. Move down, swing around to the left, and return similarly around the lower right. Source: Becking (1928), end-plate.
Figure 4
Becking curve for Schubert. Becking: Führen und Schwingen (Guide in and swing around.) Nettheim: The downbeat motion starts at or near the upper right. Draw to the left along the lower path, then loop around for the return. Source: Becking (1928), end-plate.

The existence of some such curves underlying music and representing a pulsation is taken as an axiom for the present investigation. The degree of authenticity of the particular curves seen in these sketches may, however, be questioned: those curves might be Becking’s idea alone, or they might have more general value. Their reception in German-speaking countries has generally been favourable, whereas so far they are little-known in English-speaking countries. The authenticity of these particular curves will be taken as a hypothesis, capable of being rejected if the audio-video animations to be created prove not to be convincing. For any composer, there might conceivably be more than one convincing curve; I am not aware of alternative curves having been proposed but, if they are proposed, the method to be used here could be used to investigate their implications too.

The aim of the present work is one of representation: to construct computer animations of the four given curves, finding a suitable time shaping, or pattern of velocities, for each curve. This is seen as of value in making the curves – which are static and silent on paper – dynamic and synchronised to a sounding performance; they thus become much more explicit, and so more suited to further study as well as to application in music education. A second aim is analytical: to find properties of the curves in relation to the musical performances. Performance nuances, especially of timing, vary from bar to bar, and principles are sought by which a given curve shape may be modified in its time shaping (or even to a certain extent in its contour) to adapt to such varied bars. The present composers are suited to relatively modest timing nuances, whereas some other composers, notably Chopin and Schumann, are suited to occasionally more extreme nuances, those of rubato. The present study of the former composers, while seen as important in itself, is also intended to provide a firm basis for the future study of the latter ones.

In earlier work I had provided animations for Mozart and Schubert, the time shaping for each varying bar being adjusted manually (Nettheim, 2003b, 2003c). What is new here, beyond the inclusion of Bach and Beethoven, is the attempt to automate the adjustment of a basic curve animation to each of the varied bars. This automated procedure, if successful, throws light on the sensitivity of different regions of the curves to changes in their velocity. It also reduces the considerable time otherwise required for a manual procedure, an essential saving if animations of longer excerpts or of complete movements are to be attempted later.

6.2. Method

6.2.1. Selection of Score Excerpts

The first task was to select suitable score excerpts. In principle, any excerpt from a given composer could be used, but I wished to select typical excerpts for these first implementations in order that the results may be grasped as readily as possible.[2] For Mozart and Schubert I had carried out an extensive study of the scores with the aid of fingerprints (Nettheim, 1998, 2003), aiming to acquire an understanding of what is typical of those composers’ works, and I have applied the results of those studies here; I have not yet completed the corresponding task for Bach and Beethoven. For the present purpose it was considered an advantage that the excerpts be rather well-known. The desire to reveal each composer’s musical personality with a minimum of confounding features also implied that it was best to avoid movements or excerpts having special rhythmical character of their own.[3] The scores of the chosen excerpts are shown in Figures 5, 7, 9, 11.[4] Despite my attempt to take the above-mentioned desiderata into account, I make no claim that the chosen excerpts have significance beyond their being useful for the purpose of demonstrating the animations.

6.2.2. Selection of Performances

The next task was to select, for each chosen score excerpt, a recorded performance suited to the purpose of matching the sound to the curve. The present study deals with the musical personalities of composers, not of performers; thus performers may interfere with this purpose or they may play a valuable auxiliary role.[5] I selected, for the respective composers, recordings by Münchinger (n.d.), Gieseking (1954), Schnabel (1934), and Schnabel (1937).[6]

6.2.3. Measurement of Performances

The onset times of each required tone in the selected performances were measured by slow-motion replay (1/7 speed), using the computer programs Musicians CD Player© and Goldwave©, noting the time when the sound was first heard on playback at a fixed sound level. These measurements were needed not to construct the animations but to allow the subsequent locating of the tone onsets along the animated curves. It may be objected that the beginning of a sound heard in slow motion might not correspond to the time when the sound is felt to “catch on”, and that the slowing down process might alter some properties of the sound; but both these effects could be assumed fairly constant throughout the excerpt, thus causing no appreciable error in inter-onset intervals. Resolution of about 20 ms was obtained for the piano excerpts.[7] Considering that the video resolution is limited to 25 frames per sec, or 40 ms, in the PAL format, this resolution seemed adequate. In the case of the Bach excerpt, which has a string orchestra and leading violinist, the sound onset for most tones could not be assigned as clearly as in the piano excerpts, for a wide variety of amplitude envelopes was found. I tried two methods (again at 1/7 speed and fixed sound level): (a) measuring the time when the sound could first be heard, and (b) visually seeking the peak amplitude of the spectrum for the given tone. Method (a) could be executed more objectively but its significance is less clear in the case of varying sound behaviour in the further course of the tones; method (b) was harder to execute objectively, but its results might be more closely correlated with the time when the tone is heard to “catch on”. Both methods could be used in combination, and further research in this area would be welcome when higher resolution is required, but I used just method (a) in the present work, and the results seem justified by the matching obtained in the resulting videos.

6.2.4. Scope of the Conducting Curve

To implement a conducting curve, and to grasp how the music “goes”, it is necessary to decide upon its scope, that is, the number N/C of notated bars per conducting bar. This is normally taken to be constant over whole movements, and may in general be 1, ½ or 2 (or perhaps rarely another value). This question was discussed at length in Nettheim (1998), where several methods were suggested which may help to determine the value, though in some cases it may remain impossible to determine with certainty. Two of the present excerpts were selected with an attempt to make sure that N/C was especially clear, and here I simply state my determination of N/C=1 for the Mozart excerpt and 2 for the Schubert excerpt; thus each conducting bar of the Schubert excerpt comprises two notated bars (starting with the first notated bar).

The Bach excerpt seems to require N/C=½; factors favouring this determination include: (i) strong cadences in the score may give an inappropriately weak effect if performed in less than one curve, whereas here many of these have the scope of only half a notated bar (bars 8, 18, 28, 41, 226 and others), (ii) some such cadences occur in the first half of a notated bar, rather than the second half (bars 31, 39, 58, 139 and others), again suggesting that half a notated bar is the operating unit, and (iii) the solo harpsichord passages have a large number of notes per notated bar – often 32, and bar 202 even has 44 notes – making it seem unlikely to perform or think the piece with N/C=1. Further support for this determination is seen in Becking (1928, pp. 41-42, my translation):

Beethoven showed a strong predilection for the bar-line, particularly in his middle creative period, and used it as a means to draw attention to the individual beats and to the small-scale rhythmical element in general. His tendency to level out the weight gradations led him to indicate as many values as possible as being equally heavy by the use of frequent bar-lines. Other times and other composers are very restrained in the use of the bar-line, by [p. 42] comparison with Beethoven. Bach and Handel, for example, avoid the forced highlighting of details, and Mozart also treats the demarcation of the bars with caution, which [demarcation] is particularly important for him because of the weight gradations associated with it.

The Beethoven excerpt could arguably have N/C = 1 or ½, and from a preliminary survey it became clear that some performers understood one and some the other possibility. I consider N/C=1 to be more likely, especially considering bar 8 where the appoggiatura and resolution seem to need to belong to the same curve, rather than that they be separated by executing one downbeat for each note (bb, ab). Indeed, it seemed that Schnabel, in the chosen performance, played according to N/C=1.[8]

The durations of the conducting bars are shown in Table 1.[9]

Table 1
Bar durations in seconds for four excerpts. Percentage differences from the row averages are shown in bold.
Conducting bar12345678
Note: The durations are shown for conducting bars rather than notated bars. No timing is available from the sound recording for the incomplete bar 0 of Schubert (a value of 3.37, not shown in the Table, was estimated).

6.2.5. Construction of the Animations: Preferred, Deliberately Faulty, and Mismatched Versions

I approached the task of constructing an animation by using the Matlab® computer program to produce the video track frame by frame, to which I added the audio track using the Multiquence© program.

The first step was to place points at approximately equal distances along each curve, having diameter equal to the curve thickness at each point. Sometimes it is not clear what the thickness of a Becking curve is, at points traversed more than once during the motion. In the case of Bach, Becking (1928, p.55) had indicated his intentions in another diagram of that curve, showing that it retraces itself exactly on the return journey, the thickness varying between the two sections as his arrows indicate (see Figure 1). It is also instructive to note that Becking’s two curves for Bach are not identical: a curve should be understood not as absolutely fixed in all details but fixed in its general features only, subject to which it may vary to a certain fairly limited extent during the course of a movement. In the curve for Mozart there is no such overlapping; in the curves for Beethoven and Schubert it was assumed that the overlapping portions are divided into thinner sections. Further, in all the animations the dot diameter was proportionately exaggerated, compared with the curve thickness, in order that the varying strength applied in carrying out the motions might be better conveyed to a person viewing an animation rather than viewing a static curve.

The next step, the one of greatest interest here, was the specification of the velocity of the animation as it varies over the course of a curve. I proceeded by successive approximation to the appearance of my conducting beat. Since each excerpt spans several curves of slightly different durations and since the performance nuances also differ a little between curves, I began by seeking to optimize the video for just one typical curve, usually the first; I then adjusted that animation to accommodate the duration and character of each of the other curves in turn. The adjustment procedure in previous work (Nettheim, 2003b, 2003c) was manual rather than automated. In the present work I sought an automated procedure for carrying out the required adjustments. A first attempt, the curve duration procedure, involved adjusting the animation of each successive curve so as to have the duration indicated by the measurements from the sound file, while retaining the same time shaping as in the optimized curve. A second attempt, the sensitive points procedure, involved requiring the downbeat note of each conducting bar to occur at a particular point on the curve (specifically in the first video frame of the chosen point), again utilising the same time shaping as in the optimized curve.[10]

Deliberately faulty animations were also prepared for the Bach excerpt, as a control to check one’s sensitivity in this situation. In the first of these, the velocity pattern was rotated by 3/16 of the curve’s length (causing the downbeat to arrive at curve point 8 instead of 15 in the preferred version – for the point numbers see Figure 6). In the second, the velocity pattern producing the effect of a flinging back and forth between resting places was replaced by a uniform velocity pattern.[11]

Deliberate mismatches (or more accurately cross-matches) between the composers used for the audio and video tracks were also prepared. In these, the music of one composer was combined with the curve shape, velocity pattern and match-point thought to be appropriate to the other composer.[12] The purpose of constructing mismatches was to establish that, even if acceptable animations for the given composers need not closely resemble those proposed in Videos 1-4, there are nevertheless certain composer-specific limits which need to be respected if intolerable results are to be avoided.

In attempting to construct mismatches, a fundamental problem immediately arises: the idea of feeling the conducting shape of one composer matched to the music of another composer is unnatural to a musician sensitive to the two composers’ musical personalities. Let us consider the example with Mozart’s music and Bach’s curve, velocity pattern and match-points: here one is required to imagine that the heard music, actually composed by Mozart, belongs to Bach’s output, an unnatural task. It might be preferable to have the performer, too, assume that the composer was Bach, replacing the sound track accordingly, and I have tried such mismatched playing myself (shown in a video in Nettheim, 2000) – also not a natural task.

Further, it is not clear what value of N/C should be assigned, nor on what basis one should attempt to assign it; when one ignores the composer’s name at the top of the score and substitutes the wrong composer’s name, one should presumably abandon reference to the correct composer’s collected works, and assign N/C on the basis of the wrong composer’s works, which is likely to be an impossible task. Thus it may be difficult or impossible to know even where to place the curves within the score, quite apart from their shape, velocity pattern and match-points.

Despite such basic objections, I carried out the mismatching exercise. With the present four excerpts twelve mismatches were possible; it was convenient to match the tempos as closely as possible, so I chose the Mozart-Bach and the Schubert-Beethoven pairings in both directions. Because of the false basis of these implementations, there was no sense in which I could attempt to optimize them; instead the audio and video tracks from the previous animations were re-combined, with the requirement that the same velocity pattern and downbeat match-point be applied as had been done when using the music written by the composer to whom the curve belongs.

6.3. Results

The animations for the various composers' excerpts are presented as a set of videos. In all these, the curve numbers or conducting bar numbers (C0, C1...) are shown for reference, C0 indicating an upbeat bar or preparatory section.

6.3.1. Preferred Versions

The curve duration method was used first, and inspection of its results suggested the specification for the downbeat point to be matched in the sensitive points method. These two methods produced quite similar results in the present cases (though they might not do so in other cases), the sensitive points result seeming a little preferable, so I present just the results for the latter method (Videos 1-4).[13]

Video 1 Bach excerpt, preferred animation.


Video 2 Mozart excerpt, preferred animation.


Video 3 Beethoven excerpt, preferred animation.


Video 4 Schubert excerpt, preferred animation.

Details of the results obtained from the videos are shown in Figures 5-12 and Tables 2-5. Scores and nominal tone onsets of the excerpts are shown there, followed by performed tone onsets mapped to curve locations according to the sensitive points method. Finally, the points placed around the curves are shown, the points being numbered for reference.

Figure 5 Bach Brandenburg Concerto No. 5 in D major, BWV 1050, I, bars 1-2. (N/C = notated bars per conducting bar = ½).
Table 2
Bach excerpt. Tone onsets in each conducting bar: nominal and performed per mille, and curve points for animation.
Nominal: 0001252503755006257508751000
 4:000 271 536    
Curve points:1:1518181933112 
 4:15 18 33    
Figure 6 Bach conducting curve with numbered point centres and arrow indicating the downbeat onset point.

Figure 7 Mozart Piano Sonata in D major, K 576, III, bars 1-4. (N/C = notated bars per conducting bar = 1).
Table 3
Mozart excerpt. Tone onsets in each conducting bar: nominal and performed per mille, and curve points for animation.
Nominal: 0002503754385007501000
 2:000252  5117711000
 4:000277  5157771000
Curve points:1:182931323646 
 2:1829  3650 
 4:1829  3652 
Figure 8
Mozart conducting curve with numbered point centres and arrow indicating the downbeat onset point.

Figure 9 Beethoven Piano Sonata in c minor, II, bars 1-8. (N/C = notated bars per conducting bar = 1).
Table 4
Beethoven excerpt. Tone onsets in each conducting bar: nominal and performed per mille, and curve points for animation.
Nominal: 0001252503755006257508759381000
Performed:1:000193302412522649766888 1000
 2:000139254369489628741880 1000
 3:000139236357457589702841 1000
 4:000149260376505624732866 1000
 6:000138242366495622751850 1000
 7:000129267355499645754864 1000
 8:000125235358523    1000
Curve points:1:142631384452585  
Figure 10 Beethoven conducting curve with numbered point centres and arrow indicating the downbeat onset point.

Figure 11 Schubert Moment musical D780 No. 6, bars 1-16. (N/C = notated bars per conducting bar = 2).
Table 5
Schubert excerpt. Tone onsets in each conducting bar: nominal and performed per mille, and curve points for animation.
Nominal: 0003335007508339171000
Performed:0:    ?  
 1:000 514 831 1000
 2:000 5457838549161000
 3:000300475 807 1000
 4:000 497 822 1000
 5:000 482 817 1000
 6:000 490 845 1000
 7:000314480 8148921000
 8:000 618   1000
Curve points:0:    1  
 1:36 102 1  
 2:36 106118 
 3:3685102 1  
 4:36 100 1  
 5:36 100 1  
 6:36 97 1  
 7:3684100 16 
 8:36 119    
Figure 12 Schubert conducting curve with numbered point centres and arrow indicating the downbeat onset point.

6.3.2. Deliberately Faulty Versions

The deliberately faulty animations are shown in Videos 5-6.

Video 5 Bach excerpt animation, deliberately wrong: rotated by 3/16 of curve’s length.


Video 6 Bach excerpt animation, deliberately wrong: uniform velocity.

6.3.3. Mismatched Versions

The mismatched versions are shown in Videos 7-10.

Video 7 Animation of Mozart’s music mismatched with Bach conducting curve.


Video 8 Animation of Bach’s music mismatched with Mozart conducting curve.


Video 9 Animation of Schubert’s music mismatched with Beethoven conducting curve.


Video 10 Animation of Beethoven’s music mismatched with Schubert conducting curve.

6.4. Discussion

In judging the animations it is not sufficient to contemplate them in an uninvolved manner; it is instead necessary to execute the corresponding gesture in reality or in imagination and to conceive of that gesture in relation to the given composer’s musical personality, which was the motivation for this work.

6.4.1. Preferred Versions: Bach, Mozart, Beethoven and Schubert

I refer here to Videos 1-4 and the corresponding Figures 5-12 and Tables 2-5.

Although the visual shape of the Bach curve on paper may suggest the motion of a pendulum, the velocity pattern and the strength pattern together produce a motion between the two extreme points that is very different from that of a pendulum and more like that of a ticking clock. The preliminary spring, which I have added before Becking’s curve for the first conducting bar only (shown in Video 1 but not in Figure 6) and which does not belong to the curve itself, allows the motion to begin satisfactorily; it is needed because the motion of the first curve cannot properly be begun suddenly from a resting position when a note begins sounding. A feature of Bach’s musical personality is that his beats tick on throughout a movement with relatively little modification, their character becoming evident in the long run more readily than in the short run (Becking, 1928, 135-136); here I nevertheless sought to extract something of its essence from a short excerpt.

The exaggeration of the range of point sizes compared with those of the static diagram seems to allow the effect of varying strength to be conveyed to the viewer fairly well. The velocity pattern had first been optimized and the point sizes were then exaggerated, producing perhaps slightly too sharp an impression as a result of the combined action of both ingredients. Further improvement in representing the varying strength seems possible, whether a better method turns out to be again a visual one or instead the provision of an additional sound representing the making of an effort.

The duration of the bars was rather uniform (see Table 1), allowing the sensitive points method to work fairly well. The downbeat arrives at the same point, chosen to be point 15, in all conducting bars, in accordance with that method. From the relative performed timings it is observed that the first note was considerably prolonged (163‰ compared to the nominal 125‰, thus by 30%), as is not uncommon at the beginning of a movement of music. The points visited at the onset of the notes varied little over the four conducting bars, as was to be expected in such a uniform performance (see Table 2).

In the case of Mozart Becking had drawn in the preliminary spring (Becking, 1928, p.24). The uniformity of timing observed between bars in the Bach performance applies here to only a slightly lesser extent. The visual notch, which appears in the figure corresponding approximately to the two 16th-notes in the score in bars 1 and 3, is specially characteristic of Mozart (Nettheim, 2003a), and the appearance of those two 16th-notes provided one reason for the selection of the present score excerpt; no justification appears for such a feature in any of the excerpts chosen for the other composers. Not surprisingly, the automated procedure used here gave slightly different results from those obtained in earlier manual work (Nettheim, 2003b, 2003c); which is to be preferred seems uncertain.

For Beethoven, Becking again indicated the preliminary motion leading to the downbeat (Becking, 1928, p.27). Here the bar durations are less uniform than in the previous cases, and in particular the last bar (bar 8) is about 16% slower than the average. That bar also has a feminine ending, in that the final cadence tone arrives after the downbeat. To obtain a fair result in such cases by the present method it is necessary to take into account also the following downbeat, which has indeed been done here. It is clear that the great strength exerted shortly before the downbeat and again before the middle beat is an important feature for Beethoven, much emphasized by Becking (1928, e.g. p.27). The velocity pattern within each curve is more uniform than in the other composers treated here.[14]

More animation points were used in the case of Schubert, because the motion is slower so that the number of frames can be greater (at a fixed number of frames per second), the resulting animation appearing smoother. The incomplete upbeat bar is given as a full bar pre-padded with silence (bar 0); this method will suit any upbeat bars, even almost complete ones, and in any case a preparatory gesture is always needed and should not be confused with an upbeat. The duration of this bar can be estimated but not measured (this problem is the converse to that of an incomplete bar at the end of a piece). I have added a preliminary spring for bar 0 (Becking did not draw one for Schubert). The bar durations are somewhat variable: conducting bar 6 (notated bars 11 and 12 united) is longer than average, reflecting the gripping harmonic event in that bar. The method seems to succeed well in accommodating that slower bar, but the last curve seems not quite appropriately handled by the method of downbeat matching, the last note, a resolution, arriving a little too late on the curve; this demonstrates the need for more control than is provided by the matching of the downbeat only, and here the matching of the mid-point in addition would seem warranted, an experiment not so far carried out.

6.4.2. Deliberately Faulty Versions

Deliberately faulty versions were prepared only for the Bach excerpt. In Video 5 (faulty version with rotation) the ludicrous result seems patent. In Video 6 (faulty version with uniform velocity) the motion seems lacking in energy, although this version seems more tolerable than the rotated one. A version intermediate between the uniform and the preferred could fit the conception of a particularly detached impersonality in the Bach beat – that would reflect a possible criticism that Munchinger’s conception and mine are too energetic and personally involved (cf. Becking, 1928, p.55, footnote 1). The reader should be the judge; in any case, the preferred (non-faulty) version given here may fairly be claimed to be a possible one.

In my work seeking an optimum result I naturally came upon very many versions considered faulty to various degrees – too crisp a flinging at the beginning of the gesture, too sluggish a movement, and so on. It became clear that a change of even one frame at 25 frames per second could produce a noticeable difference in the result and its effectiveness; that is not surprising, as the corresponding time difference is 40 ms which is large in terms of perceived sound timing differences. The visual resolution as the animation proceeds around the curve is limited not only by the number of points used for the animation (which can be set at will) but also by the number of frames per second; it nevertheless seemed satisfactory in all the cases considered here.

6.4.3. Mismatched Versions

Despite the objections raised earlier to the method of mismatches, some value might be obtained from the results. Focussing attention on the middle 16th-notes in bars 1 and 3 of Mozart’s excerpt, the effect of the manner in which they seem to help the motion around the secondary peak in the Mozart curve is lost when the Bach curve is applied (Video 7); such observations could have educational value. In the contrary case, Bach’s music with Mozart’s curve, the crispness at the midpoint of each bar is lost in the rounded middle section of the Mozart curve (Video 8).[15]

With Schubert’s music matched to Beethoven’s curve (Video 9), the resolving of the dissonances in conducting bars 1, 2, 4, 5, 6, and 8 is matched to an inappropriate heavy and muscular downward motion, for resolving requires an effect of tapering-off or relaxation. Conversely, with Beethoven’s music matched to Schubert’s curve (Video 10), the middle phase of each bar seems too gentle. Other features of these mismatches readily lead to other opportunities for instructive comparisons.

6.4.4. Further Discussion

Timing inflection, or nuance, is present in all human musical performance, but for the music of the composers treated here it is normally relatively discreet, as in the present performances, whereas in the music of Chopin, Schumann and some other composers it will sometimes be found to be overt, and may then be referred to as rubato. In the latter cases the behaviour of the animated curves will take on special significance: the regions of the curve where the rubato can effectively take place, and those where it can not, will be of interest, and a knowledge of those regions may have value in music education. Thus it might prove desirable to search for an explicit mapping from performance timings to curve positions and velocities. The present work attempts to establish a sound basis for such future exploration.

One would like to know the degree of sensitivity of the motions, that is, to what extent a given small alteration in a detail of an animation would bring about a significant difference in the character or acceptability of the result. It is hard to judge this from the limited amount of evidence so far available. For instance, are the two halves of the Bach motion equally fast or is there a significant difference in their speed without which the animation would be less effective? In this connection Becking mentioned that “tempo and dynamics [in the two directions of the Bach motion] are almost exactly the same” (1928, p.55, my translation), but one might wish to quantify that word “almost”. Future experience with animations may assist in answering such questions.

To test the animations by means of ratings provided by suitable subjects seems difficult. Given the axiom that a fairly regular shaped pulse operates internally in Western art music, three hypotheses operate together: (1) Becking’s versions of the curves are valid (2) the selected performances are appropriate (3) I have carried out the animations of Becking’s curves appropriately. If the animations were rated satisfactory, none of these hypotheses would thereby be rejected; but for lower ratings it would not be clear which one or more of the hypotheses should be rejected. Formal testing has not so far been carried out, but in this situation informal responses may be preferable, and these have so far been generally favourable.[16]

Dynamics (loudness) has not been taken into account explicitly in this work. It might well be needed for a fuller treatment, but on the other hand it might turn out to be largely implied by the variables already present.

Finally, for the purpose of systematic study of the underlying phenomena the computer animations might be well suited, but for purposes of music education the question might be raised whether real-life motions, or a video of them, would be better suited. Given an understanding teacher, the real-life motions might well be preferred. There is some evidence, however, that an abstract moving point provides a more effective representation for imitation by humans – see the literature referred to in Scully & Carnegie (1998, p.474). Paradoxically, the more impersonal approach to displaying the motions might be preferable even in this ultimately personal matter.

6.5. Conclusions and Future Work

Computer animations of sympathetic motions associated with music, that is, of armchair conducting, have been shown. The preferred versions seem reasonably effective, allowing for differences in the taste of viewers and for the undoubted possibilities remaining for improvement, while the deliberately faulty versions and mismatches seem quite unsatisfactory. Any animation can represent the whole phenomenon only in part, for the quality of feeling belonging to the gestures is an essential part that is not represented. The most that can be hoped for from the animations, therefore, is a representation of the underlying phenomenon that is acceptable and more explicit than was previously available.

The algorithmic adjustment of a basic animation time-shape for a given curve to the requirements of other portions of the same musical excerpt via the sensitive points method seems to have been generally successful. That adjustment avoids the need for time-consuming manual implementation of entire excerpts and at the same time highlights the downbeat as a sensitive point to be maintained throughout the excerpt. The notion of the differential sensitivity of various regions of the curve is expected to come to the fore in future studies of the music of Chopin and others in which rubato plays a more prominent role than it does in the music of the composers treated so far.

An offshoot of the present work might arise in the area of speech, most readily of poetry which follows nominal timings (analogous to a musical score), so that one could study the rubato or expressiveness in its delivery by relating performance timings to nominal timings within each conducting bar. Indeed, written and spoken language was studied by E. Sievers and others in the early work that lead to Becking’s (Becking, 1928, throughout).

Another offshoot might arise in the area of multi-modal perception in psychology. In matching the curves to the music, ear-to-eye coordination is involved, and one might ask subjects to estimate, from listening and looking, at what point on the curve a given sound starts. With slow-motion playback the task may be easy, while at original playback speed it is often more difficult. It seems that at moderate speeds one tends to estimate a later point than the objectively measured one, even allowing for the measurement here having been taken from the sound onset (results might be modified if the sound peak or the moment when the sound is heard to “catch on” were measured, for example). The relative speed of the brain’s processing of the visual and auditory streams, and their coordination, is relevant in this connection. In any case, the attempt to relate the visual domain, here representing a motion corresponding with an internal feeling, to the auditory domain, here a musical performance, seems worth making for its own sake, and the present approach appears to be novel.

But those remain offshoots, and my main purpose has been to contribute to the characterization of the musical personalities of the great composers.





To refer to this article: click in the target section

JMM: The Journal of Music and Meaning

ISSN: 1603-7170
© JMM and the contributing authors