Week 2 of my Senior Design Project has been pretty exciting. The progress I've made so far has been mostly planning / scoping / meeting / contacting, and I've decided to make some changes to my initial concept as a result. Last week I wrote a lot about the different isomorphisms that were possible on a hexagonal tiling, and after experimenting with a huge variety of layouts in the Musix app I've come to the conclusion that the Park layout (named after / invented by the creator of the Rainboard) will work best for this project with the popular Harmonic Table layout as a close second and backup. The Park layout is defined as a vertical gap of +1 semitone and a horizontal gap of +5 semitones (a perfect fourth).
Working with this design and without thinking about technical constraints, I came up with a layout that would allow 4 octaves of playing room starting from any of the 12 notes of the chromatic scale. It ranges from C2 to C7 (there are 5 octaves for C because it happens to be conveniently located), though the rest of the range could be accessible by shifting the note mapping in software. It looks like this:
It may look daunting with that color scheme but I wanted to present it without any coloring bias at first. The irregular shape could be smoothed out with the addition of extra keys on the corners (though they would be redundant). This is just the keys that are required for 4 octaves starting from any key. If the notes were colored to show the key of C major, it would look like this:
Here the root note, C, is highlighted in green, with the in-key notes in blue and the out-of-key notes colored black. The blue Bs at the top are redundant, but necessary for other keys. The C Major scale is C-D-E-F-G-A-B-C, and a quick inspection of the keyboard above reveals that the 'direction' of the instrument is on the diagonal down to the right. The complete series of Major scale notes from C2 to C7 can be played by moving one's hand steadily from the left to right along the blue strip, using only minor finger movements to play each slanted column of notes. The notes are all contiguous, easy to reach, and clearly grouped (visually). The various musical structures can be seen here:
The A-flat Major scale is show in orange. The F Minor scale is shown in purple. The G Major and A-flat Minor chords are shown in green and red respectively. Note that the major and minor chords are the same geometric pattern rotated 180 degrees. All of these patterns are constant across the board, and in this mapping they're fairly convenient to play. An important consideration is that this diagram only points to the instrument as a flat board. This could of course be warped across a curved surface, rolled around a tube, or otherwise distorted for comfort and ease of use. These variations could provide ergonomic improvements.
There are, unfortunately, some technical drawbacks here. One of the goals for the project is to make the instrument really expressive, specifically, that the user should be able to control the loudness, pitch and timbre of each note continuously over time. This would require that each of the 90 notes above have three independent input channels, making a total of 270 analog inputs to be sampled and processed. I met recently with my advisor, Rahul Mangharam, to ask some questions about the technical barriers. He explained to me some common methods for handling that much analog input (like muxing), and said that it's a non-trivial problem, suggesting I should scale it back somehow. I had anticipated this and worked out the minimum viable layout using the above structure:
If I were to exclude all the red-colored notes, there would be 36 keys left (the minor scale grouping is included for reference/scale). The complexity of the hardware and input channels could be reduced if each note were given loudness and pitch controls, with the timbre control being universal for the whole instrument - this would only require that each note be represented by, say, an X/Y touchpad.
Two dimensions of control over 36 keys would make 72 analog input channels, which is still large but much more manageable. I had a thought recently that each key could be represented as a thumbstick, like those found on game controllers. The springy action would allow both fast attack, fast decay plucking actions (like plucking a violin string) and slow, controlled notes (like a drawn out note on a clarinet). Since the thumbsticks are essentially a pair of potentiometers measuring X and Y angle, the two parameters could be mapped to loudness and pitch. They can also be purchased online for about $2.50 each.
The instrument can be simplified further by removing polyphony, the ability to play multiple notes at once. In fact, many orchestra instruments are monophonic - all the wind instruments and most string instruments. While this could be done to the existing layout to reduce the amount of processing and data transmission necessary, the hex grid seems to naturally imply that the user can press several controls at once. It would be counter-intuitive if that weren't the case. I think if the instrument were made monophonic, it would also have to be redesigned in such a way that it visually appears monophonic - it may have to resemble an abstraction of existing wind instruments. Specifically, the act of playing a note could be separated into the selection of the note (with finger positions), and the activation of the note (with an object like a bow, or a control like a breath pipe). I'm currently toying with this design challenge, but haven't come up with a compelling idea yet.
Rahul also said to not dismiss the idea of pivoting to another kind of music-related project. We tossed around some alternate ideas briefly - everything from installation-level interactive public instruments, to a giant Monome controller using the second-hand Monome components that had been donated to the lab.
So where are we now?
Of the six goals I had from last week, I completed four. I hammered out most of a design concept for the instrument, I spent some time identifying sensors (the thumbsticks in particular, though I'm also looking into force-sensitive resistors) and I met with Rahul. I also determined that the RaspberryPi could be useful for this project, but it depends on the number of input channels. I contacted Andrew McPherson (a Penn grad, now doing musical interface research) to ask some questions about the tech he uses in his projects. He mentioned that sampling that many input channels is possible with additional hardware (a multiplexer), and that certain types of sensors can require additional processing hardware (especially touchpads). I'm beginning to think there are two options. The first is to construct a scaled back version of the Park layout with simpler control interfaces like the thumbsticks - this would provide a proof of concept for the larger instrument and a very direct avenue for continued work. The second is to make the design constraints tighter and quickly generate a new concept for a compressed-range monophonic instrument. This would likely permit a greater opportunity to focus on expressivity at the expense of a really flexible interface.
Goals for the next week:
- Finalize and sketch out instrument concept
- Build paper / cardboard prototype to evaluate ergonomics
- Research multiplexing and determine the feasibility of using 36 thumbstick controllers
- Meet with Rahul again to get an idea of components available in the lab
- MIDI Hello World on the RaspberryPi (rolled over from last week)
Random side note, I found a video about the Terpstra keyboard, which solves the losing-your-place-on-an-isomorphic-keyboard problem by offsetting the keys vertically so that as the notes ascend, the keys get slightly taller. This gives the instrument an extra physicality that helps when playing.