DJ playing with a slider and force sensor on the turntables 

Context in source publication

Context 1

... However, one guitarist decided to detect strumming rhythm using an infrared distance sensor, which was mounted on the body of the guitar, under the strings. This sensor was set up so that when the guitarist strummed the strings their hand would pass over it. It was then configured as an on/off switch which trig- gered whenever the guitarist’s hand passed over it. This switch between on and off then provided a measurement of the strumming rhythm. One other possibility that guitarists examined for control gestures was the use of head and body movements. Sug- gestions included the use of head mounted accelerometers to detect head tilting and the use of accelerometers on the body to detect weight shifting. However, these were found to be too cumbersome and/or restrictive for use when playing. For each of the guitarists, the control gestures just described were mapped to a number of audio effects in Apple’s Logic Pro. The choice of gestures, effects and mappings were left to the individual guitarists. Logic Pro was chosen as it is a software package that many of the participants were familiar with and also offers a large number of possible effects to control. All of the guitarists chose to use effects that they were already somewhat familiar with and that are commonly used by guitarists playing electric guitar. These effects included distortion, delay, chorus, flanger, and master volume. Example mappings included the control of delay using the tilt of the guitar, controlling distortion using the picking position and mapping strumming rhythm to master volume. Two of the participants in the design process were bassists. The bass guitar is interesting due to its similarity in form factor to the electric guitar, while having a different playing style. This allows us to look at any similarities between the instruments developed by the guitarists and bassists. The sensing of the gestures for the bass was similar to the guitar. In particular, both bassists also used the tilt of the bass guitar as a control parameter, again detected using an accelerometer mounted to the headstock or body of the instrument. One of the bassists also tried to use body movements as a control parameter. As with the guitarists, he attempted to use head tilt (detected with an accelerometer on the head) as a control. While finding this somewhat difficult to control, he also found it extremely enjoyable and kept it as part of his instrument. The effect that the bassists had the most fun with was the wah effect mapped to the accelerometer measuring the tilt of the neck. This is essentially a bi-pass cutoff where the cutoff frequency is set by the sensor. The wah has existed for many years as a foot pedal for guitarists and bassists alike but transferring this concept to the angle of the neck proved quite difficult for one bassist who had little experience with effects. Instead he ended up playing the bass as normal, with a few slight body movements in time with the music. This created very subtle changes to the ambience of the bass as the wah moved in time with the music. Other effects tested by the bassists included distortion and filter effects. These were often mapped to the tilt of the instrument, allowing a subtle, graduated control of the effect. Two DJs took part in the development sessions. One of the DJs primarily played Techno, while the other was a Drum and Bass DJ. These two styles of music make use of different beat styles, which may be suited to different audio effects. Although a DJ tends to have their hands full much of the time, we found that the DJs preferred to use their hands to control the sensors, rather than finding some unused performance gesture. This meant that they were often simply utilising the properties of the sensor directly, rather than attempting to use the sensor to sense a gesture. As such the sensors often became extra controls for their mixer. In testing, one DJ who used the system made extensive use of 3 sensors: an accelerometer, a slider and a force sensor. The accelerometer was attached to the performer’s hand and used as a tilt sensor. This allowed them to control effects by tilting their hand in 2 axes. This was the only sensor which the DJ used to sense movement, rather than as a direct control. Sliders are extremely common in DJ equipment and are used for volume, turntable speed, as well as many effects. The DJ quickly picked up on the advantage of the slider. Retaining its position and it’s location next to the pitch control on the turntable allowed for quick and easy adjust- ments whilst mixing. When a DJ mixes, a large proportion of his time is spent focusing on the pitch control which is located on the turntable next to the tone arm and so the DJ was able to quickly switch to this slider to control effects. The force sensor was attached to the opposite turntable in the same place. The DJ activates the force sensor by pressing on it. The force of this pressing is then measured by the sensor. Figure 2 shows the system in use. The second DJ made use entirely of sliders, using them as additional effect controls on top of their turntable decks. DJs can make use of a large number of effects during a performance, switching effects during a track or when changing tracks. As one of our DJs made use only of sliders as additional effect controls, the mapping is not particularly interesting. As such, we will concentrate on the other DJ who made use of a number of sensors and effects. This DJ who tested the system tried a large number of effects with each sensor, before settling on several options for each sensor. The result is that for each sensor the DJ has a number of effects, which can be controlled one at a time or even several at once. For the accelerometer, the effects chosen were a bandpass filter effect and a beat repeating effect called Beatmasher. While the bandpass filter resulted in controlled, predictable effects, controlling the Beatmasher resulted in interesting but more random results. Interestingly the DJs found that the Beatmasher effect was more useful when used on Techno music than Drum and Bass. For the slider, the DJ chose the distortion drive level and the Transpose Stretch effect, which pitch shifts and time stretches the audio. Finally, for the force sensor the DJ chose to control a number of effect mix levels, including reverb mix, delay mix, flanger mix and phaser mix. The nature of the force sensor, which returns to a zero value output as soon as the performer stops pressing on it, allowed the DJ to add and effect by pressing the sensor, increasing the effect by increasing pressure and then instantly stop the effect by releasing the sensor. A single vocalist also took part in the design process for the Augmentalist. Vocalists offer an interesting contrast to the other performers as their hands are generally free when performing (except when holding a microphone or mic stand). This allows them to use a great range of finely controlled mappings utilising the dexterity of the hands, as well as other parts of the body. Note that in the case of the vocalist, the instrument could be considered to be their own body, or the microphone and/or mic stand. The vocalist (an MC who ’rapped’ rather than sang) made use of hand gestures to control effects. This included sens- ing of the tilting of his hand in two dimensions. This was accomplished through the use an accelerometer strapped to the back of his hand. As with the other participants, the vocalist also considered the use of head movements, again sensing head tilt using an accelerometer. However, these movements were found to be too disconcerting to use in performance. The vocalist also examined the augmentation of the microphone. Gestures used included the sliding of the hand along the microphone (measured using a slider attached to the microphone body) and grip pressure on the microphone, detected using an FSR attached to the microphone body. Most MCs hold the microphone to perform, instead of using a stand and so to put controls on the microphone itself proved to be intuitive. Furthermore, by mapping the tight- ness of the grip on the microphone to an effect mix, the mapping was a natural extension of emotive performance as with the accelerometer on the guitar. This particular vocalist was not as well versed on all the various effects and their parameters as, for example, the guitarist. This meant that often effects were discovered by accident as more experimentation took place, rather than attempting to achieve a specific sound. Something that the vocalist was keen to try straight away was a pitch shifter. This effect simply changes the pitch of the input by an amount specified on a discrete bidirectional scale. After trying with the accelerometer and struggling to maintain a steady hand (i.e. keep the pitch normal) he requested that we be able to limit the MIDI output at half so the he could keep the sensor at 0 more easily. After this he found it very intuitive to map a drop in pitch to the downwards movement of his hand and keep the pitch at 0 with his hand up. The pressure sensor with its ’return to zero’ style of op- eration worked really well with effects that made the sound messy as when released the effect would return to normal. Delay mix and reverb mix as well a flanger intensity worked well to accent and in some cases twist quite dramatically the sound before snapping back to a dry signal when released. The final participant in the development process for the Augmentalist was a saxophonist. In general, we would not expect saxophonists to be overly familiar with the use of electronic effects and audio processing. However, the interview stage revealed that this saxophonist was also a keen producer of electronic music and as such was very well versed and comfortable with standard studio style controls. As will be seen, this may have influenced his choice of sensing and control gestures. In a similar way to the DJs that ...