Music and Media Technologies
I have a long standing interest in music and media technologies. In particular I have had an interest in how we can use novel computation techniques to create intuitive interfaces to such systems.
Interfaces to Synthesis Systems
An early example of this was the use of interactive genetic algorithms as an interface to a sound synthesis system. We observed that there were basically two ways in which complex synthesis algorithms are used: naively, e.g. by using factory presets; and in a sophisticated way, but only by a small number of experienced users. The aim was to provide an intermediate-level system, where users were able to make a guided move around the search space by giving feedback on sounds heard. This feedback became the fitness function in a genetic algorithm, which manipulated synthesis parameters and provided new sounds for the next sequence of explorations.
In more recent work with my research student Alex Gounaropolous, we have been looking at how we can use machine learning techniques such as neural networks and evolutionary search to modify timbres of synthesised sound. Experienced players of acoustic instruments are able to access a wide range of timbral subtlety, however synthesised instruments often lack this, despite the availability of a wide variety of timbral colour within the synthesis algorithm. We have built a system where a neural network is trained to identify timbral characteristics of sound, and then the parameters of a synthesis algorithm are optimized against user-specified timbral characteristic with the measure of the error being given by the network.
Computer-aided Music Analysis
A recent interest is in using computers to analyse music, not just to create it. Traditionally, music analysis has been primarily about deductive processes, where we take a score and make a sequence of substitutions (either guided by formal theories or in an attempt to elucidate percepts) which results in a new sequence of symbols which explain some of the semantics of the music. This process is somewhat reminiscent of the process known as refinement in logic.
By contrast to this I am attempting to create systems which are grounded in inductive reasoning. The aim of this is both to provide ways of extending traditional analysis methods, and to provide analysis methods which can be applied to domains where traditional analysis techniques are weak, such as the analysis of timbre. This approach to analysis also opens up ways for using many other computational techniques, such as data mining technologies.
Robotics, Puppetry and Technology-Facilitated Theatre Spaces
I have been working with colleagues here in drama and electronics in investigating the use of computer technology in the theatre. This has included the use of computer graphic technologies in collaboration with live actors, the development of ideas for using robot control systems in puppets and the capability of unusual virtual spaces (such as video-conferencing) as spaces for theatre performance.
Colin Johnson University of Kent at Canterbury