WO2006050577A1

WO2006050577A1 - Motion sensors in a hand-held button-field musical instrument

Info

Publication number: WO2006050577A1
Application number: PCT/AU2005/001735
Authority: WO
Inventors: James Lee Plamondon; Matthew James Darke
Original assignee: Thumtronics Ltd
Priority date: 2004-11-15
Filing date: 2005-11-15
Publication date: 2006-05-18

Abstract

A hand held musical instrument including a button field, the button field including a plurality of buttons each forming a note controlling input device, and the instrument further including at least one motion sensor, at least one motion sensor forming a musical effects input device; the musical effects input device modifying music produced by the note controlling input device.

Description

MOTION SENSORS IN A HAND-HELD BUTTON-FIELD MUSICAL

INSTRUMENT FIELD OF THE INVENTION

The present invention relates to musical instruments and to electronic musical instruments in particular.

BACKGROUND OF THE INVENTION

A note-controlling device or surface can be called a 'button," while a bounded two-dimensional array of at least three such buttons (not all in the same line) can be called a "button-field." The specific spatial pattern of buttons within a button-field can be called its "button-arrangement" or simply "arrangement," and a pattern of association between musical notes and a button-field's buttons can be called a "layout." A musical instrument including at least one such button-field can be called a "button-field instrument," just as a musical instrument including at least one piano-style keyboard is can be called a "keyboard instrument." A button-field's arrangement can be "static" or "dynamic." If the button- arrangement can be changed at the user's discretion it is dynamic, whereas if it is fixed at the time of manufacture then it is static. Expressive Potential The emergence of electronic musical synthesis allows musical instrument designers to give greater consideration to maximizing their instruments' expressive potential, range, and polyphony, while minimizing their instruments' size and weight.

One possible source of musical expressive potential in an electronic instrument is the use of motion sensors. Such sensors can be either internal to the instrument (e.g., sensing linear acceleration or rotational velocity) or external to the instrument (e.g., via lasers, radar, video analysis, joint-angle sensors, etc.). The use of internal motion sensors is known from Downes (US #4,776,253) and Yamaha (US #5,541 ,358), specifically covering the use of internal electronic motion sensors in electronic musical instruments. However the use of motion sensors in musical instruments has not proven to be commercially successful. The control of pitch with motion sensors is less practical than it might at first appear to be, in part because of the human ear's extremely fine discrimination of pitch. Further, such motion-sensors have usually been combined with instruments that were monophonic (producing no more than one note at a time) rather than polyphonic (producing one or more notes at a time), decreasing their commercial appeal.

Polyphony Hand-held musical instruments with a high degree of polyphony e also known. The concertina was a very popular polyphonic musical instrument in the period 1830-1900. It has since fallen into almost complete obscurity, largely due to its harsh timbre and lack of expressive power. Larger variations include the bandoneon, melodeon, chromatic button accordion, and the relatively more familiar piano accordion, of which the latter two are usually supported by a harness suspended from the player's shoulders rather than being hand-held per se. All are characterized by a button-field (for the bass notes at least), in which each button controls either a single note or an entire chord.

Hand-Held The concertina is considered to be a hand-held instrument, although its weight is sometimes supported in part by a neck strap or an instrument stand, because in its "normal" playing mode, for which its design is optimized, its weight is supported by the player's fingers/hands/arms. Similarly, the trumpet is a hand¬ held instrument, whereas a guitar and piano — both of which require a support system other than the player's fingers/hands/arms — are not.

Electronic Keyboards

The most common user interface for electronic musical instruments is the traditional piano keyboard. This keyboard, familiar to all those skilled in the musical arts, is well over a meter wide in its full 88-key embodiment. Electronic keyboards often weigh many kilograms, such that they almost invariably rest on tables or stands during performance.

Such electronic keyboards have been slung from shoulder straps, although this arrangement has proven to be only marginally popular, in large part because it is impossible to play such a shoulder-slung keyboard with both hands as is required in most piano-playing styles.

Attempts to incorporate motion sensors with electronic piano keyboards have been made, but only through recognition of the piano keyboard as a stationary object, over which something else might move. In the Roland D-Beam system, for example, a light source and light detector are embedded in the surface of the keyboard and detect motion within a limited cone of space above the sensor. That is, the stationary keyboard instrument detects movement of an external object. This is quite different from having the instrument itself move.

Object of this Invention

The object of this invention is to maximize or at least improve the expressive potential, range, and polyphony of hand-held electronic musical instruments. SUMMARY OF THE INVENTION

With the above object in mind the present invention is a hand-held electronic musical instrument including at least one button-field and at least one motion sensor.

In one aspect the present invention provides a hand held musical instrument including a button field, said button field including a plurality of buttons each forming a note controlling input device, and said instrument further including at least one motion sensor, said at least one motion sensor forming a musical effects input device; said musical effects input device modifying music produced by said note controlling input device.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention will now be described, by way of example only, with reference to the accompanying drawings, in which:

Figure 1 shows a rear view of an embodiment of the present invention. Figure 2 shows a front view of the same instrument shown in Figure 1 , showing thumb controls on its front face.

Figure 3 shows an embodiment of the present invention being held in front of the body with a back-of-the-hand brace, with the player's other arm held out of the way. Note the labelling of the x, y, and z axes. Figure 4 shows an embodiment of the present invention, having been translated directly upwards (+z), without rotation, from the position shown in Figure 3. Figure 5 shows an embodiment of the present invention, having been translated directly to the player's left (-x), without rotation, from the position shown in Figure 3.

Figure 6 shows an embodiment of the present invention, having been translated directly frontwards of the player (+y), without rotation, from the position shown in Figure 3.

Figure 7 shows an embodiment of the present invention, having been rotated +60 degrees around its x axis (+60 degrees pitch), without translation, from the position shown in Figure 3. Figure 8 shows an embodiment of the present invention, having been rotated +60 degrees around its y axis (+60 degrees roll), without translation, from the position shown in Figure 3.

Figure 9 shows an embodiment of the present invention, having been rotated +60 degrees around its z axis (+60 degrees yaw), without translation, from the position shown in Figure 3.

Figure 10 shows an embodiment of the present invention, having been rotated upwards around the player's elbow towards the player's face from the initial position shown in Figure 3, resulting in a positive translation along the Z axis (+z), a negative translation along the Y axis (-y), and a positive rotation around its x axis (+pitch).

Figure 11 shows two views of an embodiment of the present invention being held in front of the body by a single forearm brace.

Figure 12 shows a closer view of an embodiment of the present invention with a forearm brace. Figure 13 shows a neck strap including a hook to/from which the forearm brace can rapidly attach/detach, with the strap detached from the brace.

Figure 14 shows the neck strap and brace shown in Figure 13, attached.

Figure 15 shows user-interface electronics in the forearm brace.

Figure 16 shows a battery pack and music-synthesis electronics in the forearm brace.

Figure 17 shows a block diagram of the present invention's electronics. DESCRIPTION OF PREFERRED EMBODIMENT

The preferred embodiment of the present invention is a hand-held electronic musical instrument including at least one button-field and at least one motion sensor. Button-Fields

The present invention's size is determined mostly by the area covered by its button-field(s). In one embodiment, each button-field is roughly the size of a single hand's span. Given a sufficiently-dense button-field arrangement and note-layout, some embodiments of the present invention could place three octaves of buttons under a single hand's span. Two such button-fields — one for each hand — could therefore place six octaves of note-controlling buttons on an instrument that is little more than a single hand-span tall. This is almost as many octaves as the traditional piano keyboard's seven octaves, in an instrument that is smaller and therefore potentially lighter. , Figure 1 shows one possible embodiment of such a hand-held button-field instrument, presenting a separate button-field to each hand. Instrument Support Systems

The small size and weight makes the preferred embodiment of the present invention easy to lift while playing, but nonetheless some means of support and stabilization may be necessary. Using the player's digits to support the instrument interferes with their ability to interact with the instrument's controls. Therefore, a means of supporting the instrument that does not involve the player's digits would be preferred.

Back-of-the-hand straps as used in concertinas are certainly possible embodiments of such support means. Such back-of-the-hand straps or braces leave the wrist free to rotate, but restrict movement of the fingers/hand over the button-field. Neck or shoulder straps are also possible embodiments. While such neck or shoulder straps effectively shift the weight of the instrument away from the fingers/hands/arms, they do not stabilize the instrument firmly. An accordion- style harness would both support and stabilise the instrument, at the cost of immobilizing it. The present invention could be supported and stabilized by affixing it to a stand, such as a microphone stand, although this would also tend to limit its range of motion. In another embodiment, the present invention could be supported by a single rigid forearm brace, as shown in Figure 11 and Figure 12. Using such a brace would both support and stabilize the instrument, freeing the player's hand/wrist to flex through a wider range of movement over the button-field than would a back-of-the-hand strap.

In the preferred embodiment, a single forearm brace would be used, coupled with a neck strap as shown in Figure 13 and Figure 14 or instrument stand (not shown) at the player's discretion, to and from which the instrument or brace could be attached or detached rapidly through various mechanical means. The forearm brace could house some of the instrument's electronics, as shown in Figure 15 and Figure 16. The brace's centre of mass is closer to the weight-bearing elbow than the instrument's button-fields. Any electronics, tone generators, batteries, displays, touch-screens, user interface devices, or other elements of the instrument's functionality that could be placed in the forearm brace rather than the button-field-containing portion, would make it easier to lift the instrument. The upper and inner surfaces of such a forearm brace are generally visible to the player while playing and are certainly both visible and accessible in-between songs.

Another advantage of the forearm brace is that it can stabilise the instrument in a fixed position right in front of the digits of the arm to which it is attached. The exact geometry of the instrument to the digits could be adjustable by mechanical means.

With a rigid brace, this position relative to the arms' digits would remain constant no matter how the instrument were translated or rotated in space. Further, the player's other arm/hand would not be required to stabilise the instrument, treeing it for other purposes.

An alternative embodiment of the present invention could provide only a single button-field, devoting some or all of the digits of one hand to holding the instrument, the other digits of said one hand potentially operating musical effects controls such as but not limited to joysticks, dials, sliders, etc.

Motion Sensors

The instrument would include at least one motion-sensing electronic component, such as a solid-state linear accelerometer or gyroscope. Such sensors are often combined into a single solid-state component including two, three, or even more such sensors.

Three-dimensional space is commonly defined using a Cartesian coordinate system of three orthogonal axes, conventionally named X, Y, and Z as shown in Figure 3. Linear accelerometers measure acceleration along a single spatial axis, whereas gyroscopes measure rotational velocity around a single spatial axis. As is known to all those versed in the engineering or mathematical arts, given acceleration or velocity, integration or differentiation can yield position, velocity, or acceleration as desired. Each of the three spatial axes thus has two components — linear acceleration and rotational velocity — yielding six independent variables, also called six "degrees of freedom." Each of the six independent variables thus has two other dependent variables. Any or all of these eighteen possible motion- derived variables can be made available from a six degree-of-freedom set of motion sensors.

One or more of these motion-derived variables could be processed to control musical effects such as timbre, pitch bend, vibrato, and so on. In the preferred embodiment, translation along all three orthogonal axes of space, and the rotation around each, would be measured. Measuring translation alone is difficult, due to the need to compensate for the acceleration of gravity, If measurement of only a subset of the six degrees of freedom is desired — perhaps to reduce component cost — then measuring only rotation around all three axes without measuring translation is preferred.

While motion can be an excellent means of controlling continuous musical effects, it is a very poor method of selecting pitches, which are discrete and need to be precisely in tune. Motion sensors may reasonably control "pitch bend" (that being a variation in pitch for expressive effect), however.

Keyboard instruments excel at making it easy to select arbitrary combinations of a large number of notes simultaneously. Keyboard synthesizers may offer many knobs, dials, switches, etc., for controlling musical synthesis variables, but relatively few of these variables can be conveniently adjusted while actually playing. For example, most keyboard synthesizers include a pitch bend control and a modulation (vibrato) control, allowing the control of those two variables — but one must stop playing accompaniment with the left hand, move that hand to the control wheels, and devote them to controlling those variables (singly or together) to add musical expression during a performance.

Piano-style keyboards are large and heavy, and are hence impractical to support, translate, and/or rotate through space.

A hand-held button-field instrument including motion sensors, on the other hand, can be quite small, light, and easily supported. It can provide all of the polyphonic benefits of a piano keyboard while also being easy to translate and rotate through space. By using motion sensors within the body of (or affixed to the exterior of) the present invention, up to eighteen motion-derived variables can be controlled simultaneously simply by translating and/or rotating the instrument through space.

This is only possible because the use of button-fields makes the present invention so small and light that it can be moved and rotated through space easily while still providing polyphony — unlike conventional piano keyboards.

Motion Examples

Figure 3 shows a player holding an embodiment of the present invention supported by a back-of-the-hand brace. The player's right hand is held out of the way to simplify the drawing, although it could be placed to access the instrument's right-hand button-field (if any), with or without a right-hand back-of- the-hand or forearm brace. The instrument shown in this and all subsequent figures is presumed to contain a full set of linear accelerometers and gyroscopes, such that movement along and around all three spatial axes is measured.

In Figure 4, the player has lifted the instrument directly upwards (+z) from its initial position as shown in Figure 3. This movement would be detected and reported by the instrument's Z-axis accelerometer.

In Figure 5, the player has moved the instrument directly leftwards (-x) from its initial position as shown in Figure 3. This movement would be detected and reported by the instrument's X-axis accelerometer. In Figure 6, the player has pushed the instrument directly away (+y) from its initial position as shown in Figure 3. This movement would be detected and reported by the instrument's Y-axis accelerometer. In Figure 7, the player has rotated the instrument positively around the X axis, away from its initial position as shown in Figure 3. This movement would be detected and reported by the instrument's X-axis gyroscope.

In Figure 8, the player has rotated the instrument positively around the Y axis, away from its initial position as shown in Figure 3. This movement would be detected and reported by the instrument's Y-axis gyroscope.

In Figure 9, the player has rotated the instrument positively around the Z axis, away from its initial position as shown in Figure 3. This movement would be detected and reported by the instrument's Z-axis gyroscope. Figure 10 shows a more complex motion, combining (a) translation upwards along the Z axis (+z), (b) translation towards the player's body along the Y axis (-y), and (c) rotation around the X axis (+pitch). These movements would be detected by the instrument's Z-axis accelerometer, Y axis accelerometer, and X-axis gyroscope, respectively. Thumb Controls

Because the instrument in the preferred embodiment is stabilized and positioned by its forearm brace, the player's thumbs need not stabilise or position the present invention, as they must do for concertinas — they are free for other duties. One or more of the player's thumbs can be devoted to the manipulation of controls affecting musical effects, such as pitch bend, vibrato, overall volume, timbre, etc. In one embodiment, the controls could take the form of physical devices like joysticks, buttons, sliders, dials, and the like, some of which are shown in Figure 2. In another possible embodiment they could take the form of virtual controls on a touch-sensitive display. Yet another embodiment could contain some physical devices and some virtual controls. Muting and Resetting the Motion Controls

It can be inconvenient to have the motion sensors active at all times. Sometimes, the player will want to move while playing without having the expression of the instrument change. Hence one embodiment of the present invention can be equipped with user-operable devices which "mute" and/or "un- mute" one, some, or all motion sensors. Muting a motion sensor suppresses its output, allowing the player to move freely along or around that sensor's axis without affecting musical expression. The preferred embodiment of the present invention has a user-operable "reset button." Resetting the motion sensor module establishes its current position as being the origin from which all subsequent motions are measured as deviations. Motion sensors may tend to "drift" in a time scale that may affect a musical performance. Just as a trumpet has a spit valve, the preferred embodiment's reset button may be used to "drain the drift" from its motion sensors, resetting their origin to the instrument's position when reset.

In the preferred embodiment, in addition to thumb-operated controls such as but not limited to those above, thumb controls can be included to selectively mute/unmute and/or reset individual motion sensors or combinations thereof.

Block Diagram

There are a number of music-control data protocols, of which the most familiar to those versed in the art of electronic music is MIDI (Musical Instrument Device Interface). There are alternative protocols, such as Open Sound Control

(OSC), and other protocols are known or may be developed in future. These will all be referred to below as MIDI.

Figure 17 shows a simple block diagram of the present invention's electronics. Data regarding the state of the finger controls, thumb controls, and motion sensors all goes to a central processing unit. This central processing unit may process this data through various algorithmic means familiar to those versed in the art of electronic music to emit MIDI. This MIDI data could then be routed through standard cabling or wireless means to an external tone-generating unit or units (not shown) which would produce audio data based on the MIDI data stream.

Alternatively or additionally the central processing unit may include a tone- generating unit and emit audio data without requiring an external tone-generating unit.

The generality and simplicity of the block diagram shown in Figure 17 is a testament to the power of standardized music-control data protocols such as

MIDI. So long as the present invention's central processing unit emits MIDI, it can be used to "drive" any MIDI-compatible tone-generating unit. Generally speaking, the tone module would treat changes to a given MIDI variable the same way whether the changes were being controlled by a finger-operated button, a thumb-operated joystick, a motion sensor, or any other MIDI-compatible controller. This optional separation of controller and tone-generator facilitates the development of a new controller such as an embodiment of the present invention, by allowing its adopters to leverage their existing investment in tone-generating equipment.

Heading

All currently-available motion-sensing technologies are subject to small errors which accumulate over time, leading to drift. The effects of drift can be minimized through the use of an external reference point. Such an external reference point can be Magnetic North, detected through use of a magnetic sensor. However, large magnets are often used in musical speakers, making it very difficult to identify a consistent and accurate reading for Magnetic North using such magnetic sensors in a typical live musical performance venue. Alternatively, or in addition, an external device can be used to emit or reflect a signal which the present invention's sensors can use as a reference point. Such external devices could include emitters of signals such as radio, infra-red, laser light, or other such signals known to those versed in the electronic arts. Such an external device could be stand-alone, or integrated into a musical instrument's carrying-case, or attached to a microphone stand, or provided in a variety of other form-factors.

Complete Music-Making System

A complete music-making system, of which the present invention can be a part, comprises four logical units: a) a hand unit, including thumb controls (if any), at least one button- field, and at least one motion sensor either internally or externally affixed; b) a tone-generation unit, which may be incorporated into the hand unit or reside in a separate unit, the two being connected by wired or wireless means; c) a speaker or headphone unit, which may include a tone-generation unit, an amplifier, user-interface elements, etc., d) possibly incorporated into a carrying-case for the other portions; e) optionally, an instrument support system such as a forearm brace or back-of-the-hand brace, which may include the tone-generation unit and/or other elements of the system's functionality; f) optionally, a heading-indicating signal emitter.

Claims

THE CLAIMS DEFINING THE INVENTION ARE AS FOLLOWS:

1. A hand held musical instrument including a button field, said button field including a plurality of buttons each forming a note controlling input device, and said instrument further including at least one motion sensor, said at least one motion sensor forming a musical effects input device; said musical effects input device modifying music produced by said note controlling input device.

2. An instrument as claimed in claim 1 wherein said at least one motion sensor includes at least one linear accelerometer to detect movement along at least one spatial axis.

3. An instrument as claimed in claim 1 or claim 2 wherein said at least one motion sensor includes at least one gyroscope to detect rotational velocity about at least one spatial axis.

4. An instrument as claimed in any one of claims 1 to 3 further including a forearm brace attached to said instrument, said forearm brace enabling said instrument to be supported by a user's forearm.

5. An instrument as claimed in claim 4 wherein said brace is detachable from said instrument.

6. An instrument as claimed in claim 4 or 5 further including a connector to receive a neck strap.

7. An instrument as claimed in any one of claims 4 to 6 wherein said brace for further including a housing to house electronic compounds and/or motion sensors of said instrument.

8. An instrument as claimed in any preceding claim further including a drift sensor, said drift sensor receiving a signal from an external source to provide a reference point for said motion sensor.

9. An instrument substantially as hereinbefore described with reference to the accompanying figures.