No full-text available
To read the full-text of this research,
you can request a copy directly from the author.
Algorithmic Spatialization Using Object-Based Audio and Indoor Positioning System
Abstract
The author presents a novel compositional framework to guide designing interplay between moving listeners and sound objects in space. Demonstrated by a case study of interactive octophonic installation, the presented framework offers new ways to articulate and analyze artistic interplay using real-world location context as a spatial composition canvas.
ResearchGate has not been able to resolve any citations for this publication.
Sound field synthesis methods like Wave Field Synthesis (WFS) and Near-Field Compensated Higher Order Ambisonics synthesize a sound field in an extended area surrounded by loudspeakers. Because of the limited number of applicable loudspeakers the synthesized sound field includes artifacts. This paper investigates the influence of these artifacts on the accuracy with which a listener can localize a synthesized source. This was performed with listening tests using dynamic binaural synthesis to simulate different sound field synthesis methods and incorporated several listening positions. The results show that WFS is able to provide good localization accuracy in the whole listening area even for a low number of loudspeakers. For Near-Field Compensated Higher Order Ambisonics the achievable localization accuracy of the listener depends highly on the Ambisonics order and shows large localization deviations for low orders, where splitting of the perceived sound source was sometimes reported.
This paper takes a systemic perspective on interactive signal processing and introduces the author's Audible
Eco-Systemic
Interface (AESI) project. It starts with a discussion of the paradigm of ‘interaction’ in existing computer music and live electronics approaches, and develops following bio-cybernetic principles such as ‘system/ambience coupling’, ‘noise’, and ‘self-organisation’. Central to the paper is an understanding of ‘interaction’ as a network of interdependencies among system components, and as a means for dynamical behaviour to emerge upon the contact of an autonomous system (e.g. a DSP unit) with the external environment (room or else hosting the performance). The author describes the design philosophy in his current work with the AESI (whose DSP component was implemented as a signal patch in KYMA5.2), touching on compositional implications (not only live electronics situations, but also sound installations).
In the context of current developments towards an exchange format for spatial audio, it is important to consider the interactions and tensions between spatial composition techniques and spatial audio technologies. This paper gives an overview of common compositional techniques and audio technologies used for spatial compositions, and discusses various forms of hybrid uses of audio technologies. A spatial composition created using certain audio technologies may lose integrity when transferred to another audio technology, when the compositional intent is not taken into account. Tools that are flexible with regard to the spatial audio reproduction technology applied are useful for the comparison of various spatialisation methods during the compositional process, but are also essential to enable a hybrid use of technologies within one composition.
Mixed Reality (MR) visual displays, a particular subset of Virtual Reality (VR) related technologies, involve the merging of real and virtual worlds somewhere along the 'virtuality continuum' which connects completely real environments to completely virtual ones. Augmented Reality (AR), probably the best known of these, refers to all cases in which the display of an otherwise real environment is augmented by means of virtual (computer graphic) objects. The converse case on the virtuality continuum is therefore Augmented Virtuality (AV). Six classes of hybrid MR display environments are identified. However quite different groupings are possible and this demonstrates the need for an efficient taxonomy, or classification framework, according to which essential differences can be identified. An approximately three-dimensional taxonomy is proposed comprising the following dimensions: extent of world knowledge, reproduction fidelity, and extent of presence metaphor.
Ecological approaches to psychology suggest succinct accounts ofeasily-used artifacts. Affordances are properties of the world that are compatible with and relevant for people's interactions. When affordances are perceptible, they offer a direct link between perception and action; hidden and false affordances lead to mistakes. Complex actions can be understood in terms of groups of affordances that are sequential in time or nested in space, and in terms of the abilities of different media to reveal them. I illustrate this discussion with several examples of interface techniques, and suggest that the concept of affordances can provide a useful tool for user-centered analyses of technologies.
Electroacoustic music lacks a definitive vocabulary for describing its spatiality. Not only does it lack a vocabulary for
describing the spatial attributes of individual sound sources, it lacks a vocabulary for describing how these attributes participate
in artistic expression. Following work by Rumsey, the definition of spatial attributes is examined in the broader context
of auditory scene analysis. A limited number of spatial attributes are found to be adequate to characterize the individual
levels of organization nested within the auditory scene. These levels are then viewed in relationship to auditory spatial
schemata, the recurrent patterns by which listeners understand the behavior of sound in space. In electroacoustic music the
interrelationship of spatial attributes and spatial schemata is often engaged in a play of perceptual grouping that blurs
and confounds distinctions like source and ensemble. Our ability to describe and categorize these complex interactions depends
on having clear concepts and terminology.
This paper presents the concepts and techniques used in a family of location based multimedia works. The paper has three main sections: 1.) to describe the architecture of an audio-visual hardware/software framework we have developed for the realization of a series of locative media artworks, 2.) to discuss the theoretical and conceptual underpinnings motivating the design of the technical framework, and 3.) to elicit from this, fundamental issues and questions that can be generalized and applicable to the growing practice of locative media.
Abstract We present the design of the Allosphere and initial experiences from its ongoing implementation. The UCSB Allosphere is a novel large-scale instrument for immersive visualization and simulation, which in its full realization will be one of the world’s largest immer- sive environments. The three-story high cubical space comprises an anechoic chamber with a spherical display screen, ten meters in di- ameter, surrounding from one to thirty users standing on a bridge structure. The Allosphere is differentiated from conventional vir- tual reality environments,by its size and focus on collaborative ex- periences, its seamless surround-view capabilities and its focus on multiple sensory modalities and interaction. The Allosphere is be- ing equipped with high-resolution active stereo projectors, a com- plete 3D sound system with hundreds of speakers, and interaction technology. In this paper we will give an overview of the purpose of the instrument as well as the systems that are being put into place in order to equip it. We also review the first results and experiences in developing and using the Allosphere in several prototype projects.
The results of the qualitative and quantitative analysis undertaken to understand the use of current technologies and compositional practices for spatialization are presented. The survey, consisting of multiple-choice and comment-form questions in English, was divided into two parts including 13 compositional and 11 technical questions. More than 90 percent of the respondents present spatial music in a concert situation and the second most frequent presentation form is sound installations. New media forms, represented through the categories 'Web application' and 'Film, video' are the least common forms for respondents with more than 10 years of experience. Most frequently (58 percent), composers intentionally use spatialization to enhance the listening experience. The two-channel audio CD is the most common medium (80 percent), and all other formats are used by less than 40 percent of the respondents.
"Bregman has written a major book, a unique and important contribution to the rapidly expanding field of complex auditory perception. This is a big, rich, and fulfilling piece of work that deserves the wide audience it is sure to attract." -- Stewart H. Hulse, Science
Auditory Scene Analysis addresses the problem of hearing complex auditory environments, using a series of creative analogies to describe the process required of the human auditory system as it analyzes mixtures of sounds to recover descriptions of individual sounds. In a unified and comprehensive way, Bregman establishes a theoretical framework that integrates his findings with an unusually wide range of previous research in psychoacoustics, speech perception, music theory and composition, and computer modeling.
A new television audio system based on the MPEG-H Audio standard is being designed and tested to offer interactive and immersive sound, employing the standard's audio objects, height channels, and higher order Ambisonics features. Object-based interactive audio offers users the ability to personalize their listening experience, setting their preferred language and dialogue level, or selecting elements to hear their home team or listen to their favorite race driver's radio. A fourstage process is introduced for implementing the complete system in TV networks. Additionally, the plant design, creative, and operational implications of producing content are discussed, based on the design and field testing of the system. Consumer reproduction implications are also presented, such as a 3D Soundbar prototype, the control of loudness in the system, and rendering for playback on both traditional and new media devices.
The authors describe an artistic exploration of sound in space enabled by real-time computer vision algorithms that provide hand shape and 3D hand-tracking information.
The authors present their musical smartphone app that uses GPS data to trigger specific sound events relative to spots within an area predetermined by the artists. Moving through the area, users listen to these events via headphones, completing the soundscape composition. The article outlines the effects and the workings of the app, which combines elements of composition and installation, and which the artists have so far adapted to a number of different areas.
In this article we describe our ongoing research and development efforts towards integrating the control of sound spatialisation in computer-aided composition. Most commonly, the process of sound spatialisation is separated from the world of symbolic computation. We propose a model in which spatial sound rendering is regarded as a subset of sound synthesis, and spatial parameters are treated as abstract musical materials within a global compositional framework. The library
OMPrisma is presented, which implements a generic system for the control of spatial sound synthesis in the computer-aided composition environment OpenMusic.
Glimmer, a composition for chamber orchestra and audience, uses novelty light sticks, video cameras, computer software, multi-colored stand lights, and projected video animation to create a continuous feedback loop in which audience activities, software algorithms, and orchestral performance together create the music. This paper establishes the aesthetic background and motivations behind Glimmer, describes the conceptual and technical design of the piece, and evaluates Glimmer's successes and failures in meeting its design goals.
Spatial quality in reproduced sound is a subset of the broad topic of sound quality. In the past it has been studied less rigorously than other aspects of reproduced sound quality, leading to a lack of clarity in standard definitions of subjective attributes. Rigor in the physical measurement of sound signals should be matched by equal rigor in semantics relating to subjective evaluation. A scene-based paradigm for the description and assessment of spatial quality is described, which enables clear distinctions to be made between elements of a reproduced sound scene and will assist in the search for related physical parameters.
Spatial elements in acousmatic music are inherent to the art form, in composition and in the projection of the music to the listener. But is it possible for spatial elements to be as important carriers of musical structure as the other aspects of sound? For a parameter to serve the requirements of musical development, it is necessary for that parameter to cover a range of perceptually different states. For ‘space’ to be more than a setting within which the main active elements in the structure unfold, it needs to satisfy these requirements. This paper explains a number of important spatial composition strategies available to the acousmatic composer in light of current technology and sound reproduction situations. The analysis takes an aesthetical rather than a technical standpoint.
The analytical discussion of acousmatic music can benefit from being based on spatial concepts, and this article aims to provide a framework for investigation. A personal experience of soundscape listening is the starting point, and uncovers basic ideas relating to the disposition and behaviour of sounding content, and listening strategy. This enables the opening out of the discussion to include source-bonded sounds in general, giving particular consideration to how experience of sense modes other than the aural are implicated in our understanding of space, and in acousmatic listening. Attention then shifts to a source-bonded spatial model based on the production of space by the gestural activity of music performance, prior to focusing in more detail on acousmatic music, initially by delving into spectral space, where ideas about gravitation and diagonal forces are germane. This leads to concepts central to the structuring of perspectival space in relation to the vantage point of the listener. The final section considers a methodology for space-form investigation.
Spatial Sounds (100dB at 100km/h) is an interactive installation that focuses on man-machine interaction and plays with the
question whether we control machines or machines control us. This paper gives a description of the installation and creates
a context around the work from the perspective of human robot personal relationships. The used examples and comparisons are
made from a personal perspective and meant to stimulate the current debate in the field.
Researchers at the Massachusetts Institute of Technology have developed a motion sensor that is cheap enough to be given away to crowds at interactive entertainment events. Radio-frequency pulses from the sensors are converted to MIDI and analyzed by a Max patch to detect rhythmic features and overall activity level, which are then mapped to musical attributes such as tempo, timbre abd style. In tests, groups of 15 to 100 participants collectively controlled the music to which they danced. The sensors worked well and the participants enjoyed the interaction. Although many participants felt the music was responsive to their movements, the majority desired more control.
Since telecommunication can never equal the richness of face-to-face interaction on its own terms, the most interesting examples of networked music go beyond the paradigm of musicians playing together in a virtual room. The Open Sound Control protocol has facilitated dozens of such innovative networked music projects. First the protocol itself is described, followed by some theoretical limits on communication latency and what they mean for music making. Then there is a representative list of some of the projects that take advantage of the protocol, describing each project in terms of the paradigm of musical interaction that it provides.
Wireless indoor positioning systems have become very popular in recent years. These systems have been successfully used in many applications such as asset tracking and inventory management. This paper provides an overview of the existing wireless indoor positioning solutions and attempts to classify different techniques and systems. Three typical location estimation schemes of triangulation, scene analysis, and proximity are analyzed. We also discuss location fingerprinting in detail since it is used in most current system or solutions. We then examine a set of properties by which location systems are evaluated, and apply this evaluation method to survey a number of existing systems. Comprehensive performance comparisons including accuracy, precision, complexity, scalability, robustness, and cost are presented.
Sound Art and Spatial Practices: Situating Sound Installation Art Since 1958
- Gascia Ouzounian
Gascia Ouzounian, "Sound Art and Spatial Practices: Situating
Sound Installation Art Since 1958, " Diss. UC San Diego, 2008.
Creative Symbolic Interaction
- Gérard Assayag
Gérard Assayag, "Creative Symbolic Interaction, " 40th International
Computational Musical Conference and 11th Sound and Music Computer Conference (ICMC/SMC joint conf.), ICMA, SMC, National
and Kapodistrian University of Athens, IRMA, 2014.
Spatial Audio Reproduction: from Theory to Production
- Frank Melchior
- Sascha Spors
Frank Melchior and Sascha Spors, "Spatial Audio Reproduction:
from Theory to Production, " tutorial, 129th Convention of the AES,
San Francisco, CA, U.S.A., 2010.
Introducing the Zirkonium MK2 System for Spatial Composition
- David Wagner
David Wagner et al., "Introducing the Zirkonium MK2 System for
Spatial Composition, " Proceedings ICMC/SMC/2014 (Athens: International Computer Music Association, 2014) pp. 823-829.
MidiNet: A Convo-Lutional Generative Adversarial Network for Symbolic-Domain Music Generation
- Li-Chia Yang
- Szu-Yu Chou
- Yi-Hsuan Yang
Li-Chia Yang, Szu-Yu Chou and Yi-Hsuan Yang, "MidiNet: A
Convo-Lutional Generative Adversarial Network for Symbolic-Domain Music Generation, " arXiv preprint, arXiv:1703.10847 (2017).
SVG to OSC Transcoding: Towards a Platform for Notational Praxis and Electronic Performance
- Rama Gottfried
Rama Gottfried, "SVG to OSC Transcoding: Towards a Platform
for Notational Praxis and Electronic Performance, " Proceedings of
the First International Conference on Technologies for Notation and
Representation (Paris: TENOR, 2015).