SuperMouse Rolls the Waves   

I am working on a interface device that brings optical sensor and routing of sensor data in one hand: SuperMouse. The idea is to select one or two parameters with a keypress, and have instantaneous control over the value(s) by moving the device around. The hardware part is a matter of soldering and screwing a commercial available mouse and numerical keypad together. Here is the result so far:

The c-shaped strip is mounted to provide extended grip when moving the thing around with one of the lower keys pressed. It could have been any shape or material, but the strip was at hand and it makes for a transparant appearance.

The bottom side of SuperMouse has the regular optical sensor. I selected the smallest mouse that I could find, it's brand is still visible. The numerical keypad (of same brand) conveniently has a usb hub function built in, and I wired the connection between mouse and keypad internally.

The hardware aspect of SuperMouse was, well, not really piece of cake in this first experiment, but still the easiest part. 

Now the question is: how to get access to mouse-events, and how to convert them into data for sound-processing purposes. I decided to translate mouse-steps into midi messages. At the receiving side, these steps are interpreted as increments and decrements on a ten-bit parameter value, running from 0 till 1023. The steps proper are 7 bit midi values. This is one way to use midi for a parameter resolution other than 7 bits.

On the OS level, I know nothing (yet) of mouse, keyboard and midi devices. Fortunately, software like PureData, Max/Msp and Steim's junXion offer functionality to experiment with such processing, while requiring no knowledge of the underlying protocols.

Using Max/Msp's human interface object, keyboard and mouse events can be read from individual devices. When I press digit 1 on the numerical keypad, the response reads 89. That is not ASCII or Unicode, but probably 'scancode'. Computer keyboards have their connections laid out as a matrix, which is constantly scanned by a microprocessor.

Digit 2 generates code 90, digit 3 does 91 and so on. But pressing digit 1 and 2 simultaneously gives 23129!. That is because each key has a one-byte code, and up to three bytes at a time can be present. 89 is binary 01011001, 90 is 01011010, and 21329 is 0101101001011001. When no key is pressed, a zero is sent.

By reading the bytes individually, it is possible to keep track of the keys on/off status. These messages are also available as one/zero on separate elements, 106 till 115 for the digits on a numerical keypad. The patch on the left shows digits 1 and 8 pressed at the moment of capture.

In a similar way, mouse events can be reported. The point of reference is top-left. Moving the mouse rightward gives positive displacement values, leftward is negative. Moving downward gives positive values, upward is negative. The polling time interval is regular and the displacement value is variable.

In my mouse-to-midi conversion, the mouse-events are sent out as continuous controller messages. Key presses on the numerical pad determine the cc number, and mouse steps become the cc values. An offset is added because midi can not represent negative values. A mouse event is now restricted from 1 to 63 in positive or negative direction, horizontal or vertical. These steps are scanned every 10 milliseconds but that could be done at a different speed if needed.

At the receiving side, where some sound processing must be controlled, the midified mouse steps are collected into clipping integrators with values restricted inbetween 0 and 1023. These are arbitrary limits, but I wanted to set a certain standard for my application, and this resolution seems to serve well in practice. Notice that absolute values can not be received with this method, it is designed for incremental information.

In a first test, six such clipping integrators were assigned to control a parametric logarithmic Fourier filter, another homebrew project.

Now imagine what you can do: morph two parameters while pressing two keys, morph two other parameters while pressing different keys. Select and morph without worrying about the actual parameter values. It just resumes from where you left it. This could almost be done blindfolded, once you can memorize the key assignment. Running SuperMouse on the Fourier filter gave me good confidence for it's future role as my cute and faithful assistant.

Actually, the processing is slightly more complicated than sketched so far. A method is required to assign horizontal and vertical mouse events to cc numbers according to first and second key pressed. Further, I wanted multiple keymodes, like momentary action (during keydown), permanent action, action at key-up. And I wish to control more than one patch or process. To copy and annotate all these routines and subroutines here would make this page quite boring. Instead, I will illustrate some facilities, referencing the graphical interface as it evolved after a while:

Program number field. Set a program number 1 - 9 to which the sound processing patch will respond. On SuperMouse, the active program number can be selected after pressing zero. So you can quickly switch between nine different patches to control.

Active mouse indicator. Two SuperMouse devices can send their messages via midi channels 1 and 2 respectively. The active mouse icon indicates which mouse is active on a patch.

Parameter sliders 1 - 9. These appear in the 3x3 configuration as on the numerical keypad. Therefore I did not feel the need to number them. The parameter values are shown, and the white bar confirms the information in vertical slider style. An icon tells the parameter function.

Discrete step bar. Sometimes you do not want a continuous parameter increment, but go with one big step. Pressing the plus-key on SuperMouse sends a cc code to close an output gate at the receiving side till key-up. The number and red slider indicate the integrated value, while the white slider and parameter output will retain the old value till key-up, and then jump to the integrated value.

All together, handling and graphical interface of SuperMouse is quite simplistic. You can choose one or two of nine patches to control at a time, and of these, one or two of nine parameters at a time. The first key-press routes vertical mouse events to a parameter, and an optional second key-press routes horizontal mouse events to another parameter. So the basic orientation is vertical. The restriction of nine parameters, nine program numbers and two SuperMouse devices was done on purpose. Of course, the system could be expanded to address two-digit or three-digit cc and program numbers. But I want to use it for realtime sound processing, therefore I need the speed and ease of simplicity.

SuperMouse's physical appearance was refactored according to the style of it's graphical interface. The keys surrounding the green 3x3 block have sandpaper in different grades at their surface. This helps in finding the functions by mere touch.