Karlheinz Essl
Lexikon-Sonate
An Interactive Realtime Composition for Computer-Controlled Piano1995
Lexikon-Sonate is a work in progress which was started in 1992. Instead of being a composition in which the structure is fixed by notation, it manifests itself as a computer program that composes the piece - or, more precisely: an excerpt of a virtually endless piano piece - in real time. Lexikon-Sonate lacks two characteristics of a traditional piano piece: 1) there is no pre-composed text to be interpreted, and 2) there is no need for an interpreter. Instead, the instructions for playing the piano - the indication "which key should be pressed how quickly and held down for how long" - are directly generated by a computer program and transmitted immediately to a player piano which executes them. In this paper I will describe from where I started and how I arrived at the concept of an infinite interactive realtime composition.
Navigation map of ELEX - the electronic "Lexikon-Roman" (excerpt)
© by Libraries of the Mind (1994)
More than twenty years later an interdisciplinary group of artists and media designers called "Libraries of the Mind" started to create an electronic version of this book using HyperCard as a programming environment. Now the navigation through the text was easily achieved by clicking onto the reference arrows, the "links". This electronic implementation - the so-called ELEX (electronic Lexikon-Roman) which manifest itself as a CD-ROM - also provides new features that were impossible with a printed book: an electronic logfile which keeps track of the ways and deviations of the reader, search for keywords, the possibility of making annotations etc.
Andreas Okopenko, who himself belongs to the "Libraries", suggested adding other media like pictures, photos, spoken language, music and sound. And so other artists joined the group: a graphic artist, a photographer, and at last myself, a composer.
After reading the book three demands for the music became obvious:
1) Music for the "Lexikon-Roman" cannot merely consist of "jingles" which are played whenever a certain text particle has been selected. With music the problem of time emerges: music - unlike a static pictorial object or even a text - is always related to time: it takes place "in time", whereas beholding a picture or reading a text happens "out of time". One can meditate over a poem for a long time, or just read over it. But music is always linked to a certain time span, reflecting time. So it became clear that the music cannot consist of pre-recorded pieces that are simply recalled. It should reflect the reading behavior of the reader: if she spends a long time on a chapter, the music should stay in the same "mood" or character, and if she starts zapping nervously between the textural links, this should also be reflected by the music, resulting in quick changes of character.
2) The complex structure of the novel challenged me to achieve something related in musical composition: a complex network of musical meanings, an infinite maze of sounds.
3) The lexical principle of references - starting at a certain point and arriving somewhere else by reference arrows - gave me an idea of the formal aspect of the composition. If the music changes, this change should not be abrupt, but taking some aspects of its former state and perpetuate it, while something new is added. Consider you are making a transition from A to B to C - for instance, when you are reading an encyclopedia starting with the keyword "A" which leads you to "B" by a link, and from there to "C". There is a semantical relationship between A and B, but to a lesser extent between A and C. When you are in the B state, you will still remember A which provided the reference; and when you approach C, A will still be present, but only to a lesser extent. If you dare move towards D, you will probably forget about A. Indeed, this lexical concept of links is the underlying formal principle of Lexikon-Sonate.
On the other hand, the piano has a big advantage: as a polyphonic instrument it allows different voices to be played at the same time. Due to its equal timbral characteristic it is predestinated to represent the structure of traditional music. In this respect it was utilized during the last 250 years. Before the development of radio and records, piano transcriptions were used to obtain an acoustical impression of a symphony or opera.
The decision to write a piano piece can also be seen in another light: taking revenge for the piano lessons I had to take since my early childhood. Using a piano also implies awareness of its incorporated history: its role in the bourgeois salon, as an inspirational tool for a composer, and as a handy instrument to unload emotional energies. At last: writing for the piano means to reflect on the whole history of this instrument, its repertory, its highly developed virtuoso techniques and its typical compositional subjects. Writing a solo piece for this beloved and hated instrument must result in a "hyper"-piano piece which increases its historical, social and compositional implications; a music beyond the scope, virtually never-ending, which exceeds the facilities of a human player. A composition, that can only be executed by a computer-controlled piano.
Partikel-Bewegungen (1991 ff.)
for flute, bass clarinet & saxophone
page from one of the instrumental parts (duration: 20 ")
When I started to work on a commission at IRCAM in 1992 (Entsagung for flute, bass clarinet, prepared piano, percussion and the IRCAM Musical Workstation), I came across MAX, a "graphical development environment for multimedia and music" (IRCAM/ Opcode). Immediately I realised that this was the very programing language I was looking for since a long time - a powerful tool which allows to experiment with compositional strategies and to listen to the result immediately.
Lecture on Realtime Composition
Karlheinz Essl talking to music students of the University of Montana
Essl Museum, Klosterneuburg/Vienna (10 Jun 2010)
The result was so compelling and encouraging that I began to implement some models for algorithmic composition, which later became the starting point for Lexikon-Sonate. As a side-effect a whole library of compositional tools - the Real Time Composition Library (RTC-lib) for MAX - evolved, which is available from several ftp-sites (see appendix).
This library offers the possibility to experiment with a number of compositional techniques, such as serial procedures, permutations and controlled randomness. Most of these objects are geared towards straightforward processing of data. By using these specialized objects together in a patch, programming becomes much clearer and easier. Many functions that are often useful in algorithmic composition are provided with this library - allowing the composer to concentrate on the composition rather than the programming aspects.
Regardless of the fact that this library was conceived for a particular project it became more and more obvious that its functionalities are open and generic enough to be used by other composers in different compositional contexts.
Although the theoretical point of view of the library is based on paradigms which have been extracted from serialism and its further developments until today, it does not force a single aesthetic, but provides a programming environment for testing and developing musical strategies. "Serialism" here refers to a certain method of musical thinking rather than orthodox dodecaphonic techniques which has been abandoned by serial theory itself (cf. Stockhausen, 1957 and Koenig, 1965).
The library is composed of two main categories of objects: basic programming tools (like toolbox functions, chance and list operations) and specific musical functions (harmony, rhythm, envelopes).
Content of RTC-lib (click-able map)
Clicking on one of the boxes (like "Toolbox") will show you its content and provides you with further information.
As an example I will discuss the "group-rhythm"-object which generates a rhythmic structure according to Stockhausen's "Gruppen"-theory (Stockhausen, 1957) and takes into account the concept of "periodicity" as it was formulated by Gottfried Michael Koenig (Koenig, 1965). These concepts indicated the end of the orthodox "punctual" serialism and finally led to the abolition of row permutation techniques. Instead of a permutation program which was derived from a given basic row, Koenig introduced the method of random selection as it manifestates itself first in his Streichquartett 1959 and later in his composition program Project 1 (Koenig, 1970 and Essl, 1989).
The "group-rhythm" object
As its basic material this rhythm generator uses a supply of entry delays (ED) which form a geometrical row between a minimum and a maximum ED in a certain number of values. In our example the min.ED is 100 ms, the max.ED is 1000 ms - between these boundaries a geometrical row is constructed:
row index: 0 1 2 3 4 5 6 entry delay: 100 464 215 316 464 681 1000When "group-rhythm" is switched on, it chooses an ED-value by chance. Now the "periodicity factor" determines how often this value will be repeated, before another one is chosen. When the factor is 1, the resulting rhythm will be completely periodic - an even pulsation. If the periodicity factor becomes 0, a completely aperiodic rhythm with no repetitions of a chosen ED is generated. In between these boundaries of pure "periodic" or "aperiodic" rhythms a broad field of interesting intermediate steps is situated. By gradually changing the periodicity parameter over the time, transitions between different grades of (a)periodicity can be achieved easily.
These specialized generators of the RTC-lib are functional implementations of a certain algorithmic model whose "behavior" can be changed by the model's parameters. In this way an infinite variety of variants can be produced, which are always linked to the central idea of the model, even when the results are very different. Combining different RTC-generators in one patch, is a convenient way to implement specific algorithmic compositional models, as will be shown in the following chapter.
By using different random generators which are controlling each other (which - serially thought - form a scale between a completely deterministic and a completely chaotic behavior) new variants of the same model are generated. Variants may differ dramatically from each other, although they are always perceptible as "instances" of the given structural model.
One of the simplest (but nevertheless most compelling) structure generators of Lexikon-Sonate can be found in ESPRIT:
Structure generator ESPRIT
This module generates melodies with a pronounced "espressivo" character. Before investigating what "espressivo" means and how it is achieved let us first look at the flow diagram of this patch; a basic structure, that appears in nearly all structure generators of Lexikon-Sonate.
From top to bottom we notice the following object boxes which are connected by lines:
weight-factor = 1: short AD, long PD ---> background
weight-factor = 2: medium AD, medium PD ---> middle ground
weight-factor = 3: long AD, short PD ---> foreground
In other words: the weight factor determines the perceptual level of a structure generator - whether it is dominantly playing in the foreground, being modest in the middle ground, or hiding itself in the background.
Parameter setting of ESPRIT
brown factor = 0: always repeats the same value
brown factor = 1: each value between min. and max. can be chosen
In order to filter out tone repetitions, octaves, and oscillating pitches, the resulting stream of pitches is evaluated by two objects, anti-octave&prime and anti-bis&osc. If such an undesireable event were about to take place the pitch is suppressed and brownian is asked to supply another one that fits into the constraints. This method avoids disturbing musical effects of a not-so-smart harmonical algorithm.
Harmony generator of ESPRIT
Although the music generation is achieved by random operations, it will never sound like this. In rhythm the repetitions of values create "islands of periodicity" within a complex (chaotic) situation; whereas in harmony, dynamic and duration the randomness is controlled by the "brown factor" leading to the emergences of melodic cells, envelope shapes, articulated phrases etc. By this the desired "espressivo" character - with its association to Viennese music since Mozart until Schoenberg, Webern and Berg - emerges; not at will, but as a consequence of a particular compositional strategy.
In the following all structure generators are listed, together with a short description; furthermore their relationship to the 5 types of musical structures and references ("cf.") to other modules with similar properties are given.
When you click on the "play" button (and you browser is configured to play MIDI files) you will hear an example of this structure generator:
ARPEGGIO: chord / texture (cf. GLISSANDO)
Arpeggios of 4 to 11 notes, ascending or descending, which are built up of 2 - 4 different intervals.
BROWNCHORDS: chords (cf. PULSCHORDS)
1-6part chords on a brownian rhythm. The harmonic structure is built of intervalic rows.
CLOUDS: texture / melody; (cf. TRILLER)
"Clouds" of fast moving notes ("rubato rhythm") within a certain pitch range. Some notes are
highlighted by dynamic accents.
ESPRIT: melody
Espressivo-melody of some complexity, referring to the so-called "Viennese Espressivo".
FERMATA: pause
Inserts global pauses into the whole musical stream and sustains the notes at these
"cutting points" as resonances with the sustain pedal.
FIGUREN: melody / texture
Grace-note figures with crescendo towards the main note.
GLISSANDI: texture; (cf. ARPEGGIO)
Glissandi, composed of minor and major seconds which are sustained with the pedal.
GENERALPAUSE: pause
Entirely stops the stream of music. By this the whole infinite process of music generation will be organized
in sections.
GRUPPEN: melody, repetition
Parameters organized in "Gruppen" according to serial theory (see above). Each of the structural
parameters rhythm, harmony, dynamic range and duration factor obtain their own, individual periodicity
factor which determines whether the respective parameter value is being kept for a longer period
("periodicity") or is changing immediately ("aperiodicity").
HACKER: pause; (cf. FERMATA)
Interrupts the global stream of music by inserting short breaks (statistically shorter
than FERMATA).
HOQUETUS: melody / texture
Periodic rubato-rhythm with constantly changing registers and generally loud dynamics. The harmony is
built up from a supply of two intervals.
JOYCE: melody; (cf. MOTIV)
Music obtained from a radical choice approach: for each section values from predefined parameter supplies are chosen (by the selection principles ALEA, SERIES or SEQUENCE) to build up a musical structure.
MELOCHORD: melody / chord
Structural transition between
melody and chord. Depending on the duration of the entry delay (ED) the
repetition rate of the chosen ED and the chord size are determined,
according to the following relation:
MOTIV: melody; (cf. JOYCE)
For each phrase different sequences of parameter values (for rhythm, harmony, dynamic and duration) are calculated which are "looped" for the duration of this very phrase. By this method the concept of "motives" (as it traditionally appears in rhythmical-harmonical contexts) is extended to other structural
parameters.
ORGELPUNKT: repetition; (cf. REPLAY)
A repeated note which is dynamically increasing and decreasing.
PAUSE: pause
Simply does nothing at all. Like GENERALPAUSE, HACKER and
FERMATE it allows that only two different structure generators are combined, instead
of three.
POINTILIST: melody
Parameters changing each note ("Punktuelle Musik"). Parameter ranges and row sizes may change.
PULSCHORDS: chord (cf. BROWNCHORD)
Up to 6-part chords on a
constant rhythmical pulsation of different speed. The harmony is built
of intervalic rows of different sizes.
REPLAY: repetition / texture; (cf. ORGELPUNKT)
Layers of repeated notes of different speeds which are dynamically increasing and decreasing. The harmonic structure is composed of two or three different intervals.
RêVERIE: melody
Melodic line of complex rubati
with moving harmonic constellations.
RICOCHET: repetition; (cf. ORGELPUNKT)
Repetitions of a single note with increasing or decreasing speed with crescendo or decrescendo.
SLEEP: melody / pause
Most of the time it does nothing (like PAUSE), but sometimes it plays a short melodic phrase.
SUSPCHORDS: chord / repetition
1 - 4part legato-chords on a constant pulse which is structured by rests. The harmonic structure consists of intervalic rows where several note may occur in the next chord (harmonic "suspension").
TRILLER: texture; (cf. CLOUDS)
Thrills of 2 - 6 notes,
dynamically increasing and decreasing, mostly together with
accelerando or ritardando. NB: A six-note thrill consists
of rapid permutations of six notes within a single octave register.
As stated before these 24 structure generator are forming a sort of musical HyperText. At the "boundaries" of its algorithmic model a structure generator can obtain characteristics of another one, giving a reference ("link") to it. Some examples:
This shows that those structure generators are not closed entities with an exclusive behavior - they are linked to each other in a very complex way by references. When modules are being combined during the piece, they are acting completely independent of each other. They don't "know" what the others are doing. Hence, by the process of perception the listener will relate some structural aspects of different modules to each other, composing her own "version" of the piece. This approach, as it is viewed by "Radical Constructivism" (Essl, 1992), entitles the listener to become a "composer" - constructing the piece in her mind by finding an individual way through a polyvalent maze. A way, that is determined rather by personal criteria of the observer than by objective structures.
When combining structure generators in Lexikon-Sonate, there will always be one in the "foreground" (weight = 3), one in the "middle ground" (weight = 2), and one in the "background" (weight = 1). The weight for each module will change whenever a new one is brought into the game: e.g. a "foreground" structure could turn into a "background" or the "middle ground" into the "foreground" etc.
The combination of structure generators takes place in a chain of three boxes, which is filled by the chosen modules, from top to bottom. On the right side of each box the number refers to the weight of the position. In the example shown below, the first box always occupies weight 3 (= foreground), the middle 2 (= middle ground), and the last 1 (background).
For example, imagine ESPRIT has been chosen. It is being put into the first box to which the weight=3 is associated. Hence, ESPRIT will be played as a foreground structure: long melodic phrases, interrupted by short pauses (see above).
After a certain time the next structure generator FIGUREN is selected. Now ESPRIT will be transferred to the second position, which holds the weight 2, whereas FIGUREN will be played as a foreground structure with weight 3. The "influence" of ESPRIT becomes weaker, being displaced by the recently entered FIGUREN.
Finally, BROWNCHORDS enters the scene, taking over the foreground. ESPRIT is turned into the background, and the weight of FIGUREN is decreased.
When ARPEGGIO is put into the chain, ESPRIT is abandoned. We have seen that this structure generator - which started as foreground - gradually lost its power, becoming weaker and weaker until it was dropped completely. We also notice a formal transition: in the beginning ESPRIT alone, then in counterpoint with FIGUREN, and at last a polyphony of three different structure generators. Since the occurence of ARPEGGIO, however, a situation is established, where two modules are kept (BROWNCHORDS and FIGUREN) as a "memory" of the previous situation.
With the entrance of JOYCE, FIGUREN is cancelled. Again we notice a formal shift, where two compounds of the recent musical situation are maintained, while a new aspect is brought into the game.
The different structure generators are chosen by random - an already picked module is being blocked until all others are selected. Whenever GENERALPAUSE occurs, all active structure generators will be switched off, resulting in silence. By this the infinite process of Lexikon-Sonate will be articulated in "movements". Afterwards the sequence of weights will be mixed again (now - instead of 3-2-1 perhaps 2-3-1 in) which will serve as a formal principle for the next movement.
User interface of Lexikon-Sonate 3.2
click-able map: clicking on one of the boxes (like "Esprit") will supply you
with more information of this structure generator and a sounding example.
On the top one can find a box named control with some buttons attached to it:
The "control" box serves as a sort of conductor which gives cues to the structure generators, that are placed below. Clicking on the auto-toggle starts the simplest performance mode: the built-in conductor will entirely take control, selecting the structure generators in time and switching them on and off. If one wants to influence the behavior of the machine, one can click on the "add module" button. Whenever this is done, a new structure generator will be chosen by random and combined with the two others that are still active. Clicking on the "change weight" button will change the weight factors as they are applied to the module, determining whether a chosen module serves as a foreground, middle ground or background structure. This can be seen at the bottom of the display - the three boxes forming the "chain" that was discussed in the previous chapter.
However, there are also more advanced levels of interaction. Instead of merely requesting a change to take place (by clicking on the "add module" button), one can decide at will, which modules shall be combined. This is achieved by choosing a module from the "select a module" pop-up menu, which will be sent into the combination chain. Furthermore, the sequence of weight factors can be determined, by selecting it from the other pop-up menu "select a sequence of weights".
But one can also by-pass the combination chain which allows only three modules at a time to be played, and with different "weights". By opening a module itself (by double-clicking on its box), one can directly access its parameters, choosing various settings and all the possibilities of combinations.
At a lecture I once asked two persons from the audience to come on stage and sit there, back to back, so that they could not see each other. By giving signs with their hands, they indicated when they desired a change in music. Although these persons could not see each other, they could hear when the other had required a change - this led to a situation where the both started to "play" with each other, resulting in a wonderful and energetic performance.
Recently, Lexikon-Sonate has undergone another transformation: since version 2.0 (released 3 March 1998) it appears also as a meta-instrument which allows one to play the piece by using certain keys of the computer keyboard. By hitting those specified keys, certain structur generators can either be triggered (like evoking one glissando gesture) or being switched on or off (like starting resp. stoping an ESPRIT melody). Here the borders between composition, performance, and instrument are absolutely blurred.
Essl, Karlheinz (1991). Computer Aided Composition; in: Distel, "Mensch Maschine", 46/47. Bozen: Distel Verlag.
Essl, Karlheinz (1992). Kompositorische Konsequenzen des Radikalen Konstruktivismus; in: Positionen. Beiträge zur neuen Musik, 11.
Henck, Herbert (1980). Karlheinz Stockhausens Klavierstück X. Ein Beitrag zum Verständnis der seriellen Kompositionstechnik. Cologne: Neuland Musikverlag, 19-23.
Koenig, Gottfried Michael (1965). Serielle und aleatorische Verfahren in der elektronischen Musik; in: Die Sonde, 5 (1).
Koenig, Gottfried Michael (1970). Project 1. Electronic Music Reports, 2. Utrecht: Instituut voor Sonology.
Okopenko, Andreaas (1970). Lexikon-Roman einer sentimentalen Reise zum Exporteurtreffen in Druden. Salzburg: Residenz Verlag
Stockhausen, Karlheinz (1957). ...wie die Zeit vergeht...; in: die Reihe. Informationen über serielle Musik, 3. Vienna: Universal Edition.
Information about a recording of Lexikon-Sonate as a Disklavier disk (which can be played on a Yamaha Disklavier) can be obtained from Bob Willey's website (CRCA Studio at the University of California, San Diego).
#1 (10 kB SMF): 3 min 10 sec
#2 (15 kB SMF): 3 min 39 sec
#3 (15 kB SMF): 3 min 22 sec
#4 (11 kB SMF): 2 min 52 sec
#5 (14 kB SMF): 2 min 47 sec
#6 (12 kB SMF): 2 min 54 sec
A preliminary version was first published in the Proceedings of the "II Brazilian Symposium on Computer Music" (1995). - This current extended version was printed in: Musik im virtuellen Raum. KlangArt-Kongreß 1997, ed. by Bernd Enders and Joachim Stange-Elbe (= Musik und Neue Technologie, vol. 3), p. 311-328 (Osnabrück 2000) ISBN 3-934005-64-0
| Home | Works | Sounds | Bibliography | Concerts |
Updated: 30 May 2019