The Bucky: A Handheld Three Dimensional Interface for Sound Manipulation

Abstract

The Bucky is a grip sized interface that can be used either free floating or in a desktop environment.  With multiple simultaneous axis of continuous data it is suited for a wide range of musical applications.  This paper will discuss one application that it is particularly suited for; the live placement of sound sources within a three dimensional ambisonic soundfield.

1.Introduction

The Bucky is named in honor of the architect and theorist Buckminster Fuller, and it’s initial design was inspired by the sculptures of Kenneth Snelson that Fuller named ‘tensegrity’.  The principle of Snelson’s sculptures is summed up in the Fuller’s phrase “Islands of compression in a sea of of tension.”  Following this, the Bucky has a cylinder with a rounded end as the ‘island’ of compression which is suspended within the housing by eight springs representing the ‘sea’ of compression.  The outside of the housing is a proportionally large ring which allows the interface to be tilted and rolled along its horizontal axis.

The motions of the Bucky are captured with two separate sensors, a proximity sensor and an inertial tracking unit.  These two combine to give information in a three dimensional framework; the x and y supplied by the ITU and the z given by the proximity sensor as it tracks the position of the ‘island’ in relation to the housing.

2.Design

2.1.Interactive Considerations

Following principles discussed extensively in related literature like Dan Overholts Musical Interface Technology Design Space (2009), the Bucky strives to combine some of the most important attributes of traditional musical instruments with the flexibility of an electronic interface.  In the traditional instrument paradigm, the Bucky, especially in the application as a spatialization controller, maps one to one in the relationship between gesture and result.  Also in the traditional paradigm, the Bucky presents progressively more resistance to compression of the ‘island’ into the housing, thus emulating the similar interaction to bending a string on a guitar, biting into a string with a bow, pinching a read with the lips, etc.  The progressive physical difficulty allows performance space for exploration of the resistance.  It allows the performer to have a certain amount of learning and opportunity to practice in order to master the relationship between gesture and result.  In the act of practicing, it invests the performer emotionally in the performance and encourages further practice, thus setting up a positive feedback loop.

2.2 Aesthetic Considerations

In designing the Bucky, it was important for the object itself to have an intriguing appearance that encourages and invites a user to reach out and grasp the item with curiosity and without apprehension.  In further iterations the Bucky will have a wood exterior on the housing in order to emulate the feel and appearance of a traditional musical instrument.  It was also important for it to be hand sized so as to take advantage of the wide range of motion and control inherent in the human hand, wrist, elbow and shoulder.  When not compressing the ‘island’ the interaction is fluidly in the wrist.  As compression is applied it engages the elbow and shoulder in order to maintain the rotational position while changing the compression position.  Finally, the combination of springs and rings give the Bucky a singular appearance.  A similar effect could have been achieved with a single compression spring hidden in the housing, but the web of springs promotes curiosity, and shows a user without any instruction that there is some sort of interaction involving the compression of the ‘island’.

2.3 Technical Considerations

In order to encourage initial as well as continuing engagement with the Bucky, the technical and electronic underpinnings must be as transparent as possible.  There should be no significant calibration process, and it should work in the same manner relative to the space that it is being used in (be it desktop or handheld) no matter which orientation it was in when it was picked up.  The tech should be hidden as much as possible, including the tether.  In future iterations Bluetooth communications will be added so there will be no tether at all, thus eliminating any notion of ‘front’ or ‘back’ of the item.

2.4 Design Process

The design process started as a hand drawn sketch, but as the completed prototype was to be rendered with a 3D printer the process was quickly moved into Autodesk’s Fusion360 CAD program.  This allowed for quick iterations combined with virtual modeling to come up with a finished working physical prototype in just under two weeks.  The initial design (fig1) was much more ergonomically pleasing, but turned out to be unsuited for 3D printing, as well as not allowing much room for the electronics.  A compromise housing was made for prototype that still achieves a similar aesthetic effect while being much more suited for 3D printing.(fig 2)

With all of the parts including the ‘island’, a shield for mounting electronics and a cap completing the design, the final shape of the prototype was ready to send to the 3D printer.  Here is a video documenting the whole mechanism:

And the finished product:

3.Interaction Implementation

3.1  Sensors

There are two basic interactions with the Bucky, rolling around the ring and compression of the island.  While it might have been possible to get all of this information with one sensor, for more accuracy in the island depth two different sensors were implemented; a 9 Degree of Freedom Intertial Tracking Unit (ITU) and a Proximity Sensor.  Both sensors were acquired from Adafruit Industries and implement some of their prepared libraries.  The ITU is the Bosch BNO005 on an Adafruit breakout board (Fig.4)implementing an i2c bus.  The proximity sensor is Adafruits VCNL4010 (Fig.5), also implementing the i2c bus.  The i2c bus was critical in meeting one of the important design considerations, that the tech be as unobtrusive as possible.  The i2c bus is a venerable serial communication protocol using on/off variations around a clock signal to send and receive messages very quickly to many individually addressed slave units.  The wiring requires a data (SDA) pin, a clock (SCL) pin, 5v power and ground; this allowed the use of a small four wire cable with shielding as an unobtrusive tether to the micro-controller.(Fig.6)

3.2 Data Interpretation

The data is collected by an Arduino Uno running the i2c bus, which forwards the information over USB to a MaxMSP patch.  The ITU generates data for orientation, velocity, acceleration (angular, linear and gravitational), compass data and temperature.  For this project I only used the orientation and compass data.  For the Island, the proximity sensor was mounted inside the tube of the housing at the top.  The orientation data came in the form of Euler angles which are a set of three angles in degrees representing the BNO005’s orientation in reference to magnetic north and the horizon.  The angles are mathematically known as x,y and z, but are also called pitch, roll and yaw. The method of interpretation of the y and z data, which is what is affect when the Bucky is tipped or rolled, takes a bit of a visualization to describe.  Imagine standing in a cylinder with the Bucky in the center.  Out of the center of the top of the Bucky, extend a straight line to the top of the cylinder making a point on the circle. As you tip the Bucky around, the point moves in a way that could be described as Cartesian, with an x and y position on a flat plane.  In practice, when resting on it’s ring, the readings range from -45 degrees to 45 degrees on both the y and z axis.  Map the [-45,45] range to [-1,1], with Euler z becoming the Cartesian y and the Euler y becoming the Cartesian x and you are now describing a point confined with a unit circle, which can then be used for all sorts of interpretation and conversion in with trigonometric functions.

One problem remains which is stated in the Technical Considerations, especially in the framework of a Ambisonic sound field; the device should map the same to the room no matter what orientation it was started or picked up, it should adhere to the compass so no matter how it is being held, when it is pointed north the sound should come from the north.  This is were the Euler x data, which is the compass data, comes into play.  The first step is to convert the new Cartesian x/y data (From the Euler y/z) into polar coordinates.  Then the compass data, which is in degrees, must be converted into radians which is the unit used for the angle in polar coordinates.  Then, because 0 degrees on a compass is North, and zero angle in polar is to the left, π/2 must be subtracted from the compass data.  In practice this took some trial and error and in fact, because of other translation problems between compass and polar, the best results came from multiplying the compass by -π/180.  Lastly, an additional tuning feature was added whereby one can add fractions of π to the results in order to tune the Bucky to the room if (as is the case in the University of Michigans Davis Theatre) the compass signal is somewhat weak.

Finally, the Island data is gathered by the proximity sensor, which gives a reading between 0 and 65535, which when installed ranges between 16000 and 65535, which is also mapped to [0,1] for ease of use later.  This image shows the MaxMSP logic used to interpret the data.

4.Implementation

4.1 HOA Library

This use of the Bucky is for positional information in an Ambisonic sound field.  For generation of the soundfield MaxMSP was used, implementing the Higher Order Ambisonic Library by Julien Colafrancesco, Pierre Guillot, and Eliott Paris from the Centre de recherche Informatique et Création  Musicale in Paris. Description of the mechanics of Ambisonics is outside the scope of this paper, but it is a method  for reproducing a three dimensional sound field with an array of loud speakers.   Through a variety of useful gui objects, as well as an in depth array of Ambisonic Encoders, Decoders and effects processors, it makes synthesizing a 3D sound field much more accessible for someone lacking the necessary mathematics in order to do a ground up implementation.

The position data from the Bucky gets fed into the hoa.map~ object, which gives a visual interpretation of the data, as well as returning the data in a format that can be read by the Ambisonics encoder.  Some Ambisonic reverb is then applied to the signal to tie together the various sound sources, a few more optimizations are applied and then the sound is decoded for the loudspeaker array.

For further control of the sound, a Novation Launchpad is used to trigger sound samples and to change the parameters affected by the compression of the Island; in this patch that includes volume of sound sources and binaural seperation of two channel audio sources.  Here is a video of the Bucky and the patch working together.  The video was done with the binaural decoder in the HOA library, so the spatial effects are best perceived in this instance using headphones.

The Arduino code and Max patch can be found in a Google Drive folder at this location.