Toward a new vision of the sequencer piano roll

John S. Allen

The piano roll view is one of the essential elements of a computer-based music sequencer. The design of the piano roll views of commercial sequencers is very similar. Is this because the fundamental design is so highly perfected that little improvement is possible? Or rather, because too little attention is being given to possible improvements?

Like other articles in this series, this article will use examples from Cakewalk Pro Audio; but more than most of the other articles, this one applies to most other sequencer packages.

Let's start with the basic questions of which way is up, down, right and left.

Which way is up?

I don't know of any language whose writing runs from bottom to top. Some run from right to left, like English, left to right, like Hebrew and Arabic, or back and forth, like old Greek Boustrophedon ("plowing with the ox") inscriptions. some Chinese calligraphy runs from top to bottom.

Bottom-to-top is absent from that list because of the need not to smear what has just been written. Bottom-to-top is used in pavement markings, as in

which makes sense, because you reach the lower word first as you drive along the street.

The other common use of bottom-to-top is the traditional paper player piano roll. I don't know why piano rolls make you read from bottom to top -- probably, because nobody thought of including the lyrics until after piano rolls had become standardized. The lyrics on a piano roll look like the following illustration.You can simulate the scrolling of a piano roll by going to the bottom of the illustration, then pushing up the vertical scroll bar of your browser window as you read the lyrics:

MAY...

  THAT BLOOM IN

FLOWERS

 THEY BRING THE

WAY...

  COME YOUR

SHOWERS

 WHEN APRIL

There is probably no deep psychological reason that reading from bottom to top should be difficult. After all, that is how you look at the ground as you walk or drive or ride a bicycle, and that is why the pavement markings read from bottom to top. But you are not used to reading this way. Your eyes want to move down to the next line of text. The lyrics on a piano roll are right-justified to keep them out of the way of the notes as much as possible, and that makes the lyrics even harder to read, because you don't know where to find the start of each new line of print.

Are the design choices in the sequencer piano roll any better than those of the player piano's paper roll? How can we improve the sequencer piano roll to make it more effective as a tool for music composition and editing?

Let's first look at the coordinates of the graphics screen layout: low frequencies are at the bottom and high frequencies at the top, earlier events at the left and later ones at the right.

There are strong precedents for this use of graphic coordinates. It is the one used in music notation. Not surprisingly, since the first sequencers were built by engineers, the same orientation is used in a conventional engineering graph of frequency against time.

But is this frame of reference the best one for a piano roll?

Consider that the virtual keyboard on your computer screen runs up the left side of the computer screen, at a right angle to the actual keyboard under the musician's fingers. You are looking down from overhead at the imaginary musician playing the screen keyboard. This imaginary musician is facing the left side of the computer screen.

As music begins to play, new notes move into the keyboard from the right side of the screen, the opposite side from the virtual keyboard.. With the sequencers I've seen, this motion is not smooth. Instead, a "now" line moves to the right across the screen until it approaches the right edge of the screen, and than a screen-width of notes abruptly jerks moves to the left and disappears under the keyboard, while a new screen-width appears from the right. The effect is similar to having a rug pulled out from under you repeatedly, not to the smooth, continous motion of a paper piano roll or Teleprompter screen .

Before the sequencer piano roll view can feel natural and normal, the musician needs to adapt to the peculiarities I've described -- the odd orientation of the screen keyboard, and the jumpy motion of the notes into, rather than away from, the keyboard. Even for a musician who has adapted, a certain amount of psychological energy has to be dissipated in the adaptation.

Suppose we try a different layout? Let's place the screen keyboard horizontally on the screen, so it is in the same orientation as the physical keyboard. Let's put the keyboard at the bottom of the screen, and let's have the notes move up and away from the back of the keyboard. (The now line would be somewhat above the back of the keyboard, so you could see the old notes before they play). As time progresses, new notes appear from the backs of the keys that sound them. This orientation is, by the way, used in science and engineering for a "waterfall" spectral plot: higher frequencies are to the right; newer events appear at the bottom of the screen and move upward as they age.

What have we accomplished?

• The screen keyboard is now in the same orientation as the physical keyboard, so the musician need no longer translate from one spatial frame of reference into another.

• Since the rears of the keys face the music, all keys, including black keys, meet the corresponding lines of the piano roll.

• The onscreen keyboard is no longer just a reference scale which allows us to identify the notes on the screen. Since the music moves away from rather than toward the keyboard, the screen keyboard conveys the message that the keyboard is creating the music, which is what actually happens when you play music.

• We can place lyrics or text comments in a vertical column at one side like the lyrics of the traditional (hardware) piano roll. They then read from top to bottom, the normal way.

• Today's sequencers, including Cakewalk, have a separate (and horizontally-oriented) virtual keyboard for note entry. (This is one of many examples of window proliferation which is rife in Cakewalk). We can dispense with the virtual keyboard and use the keyboard in piano roll view instead, since this keyboard is now properly oriented. Shrinking the piano roll view so it displays only the keyboard, and expanding it back out, should be simple, one-click or one-key functions.,

• A well-designed sequencer will have several options for scrolling (to be discussed later). One of them, on a computer powerful enough to support it, should be smooth scrolling with a stationary "now" line, to convey the message of constantly flowing time as notes are recorded.

A piano roll that scrolls vertically probably requires more computataional resources than one that scrolls horizontally, because of the structure of the scanned image with its horizontal lines. But this is a "legacy" problem. Modern graphics controllers have plenty of power to manage vertical scrolling.

The piano roll view with a horizontal keyboard is a logical one for a keyboardist, though this should ideally not be the only option. Other arrangements may make better sense in some cases:

• A percussionist may find it easier to work with the conventional read-toward-the-right layout, since this allows the names of the instruments to be spelled out horizontally on the screen (though we might put the instrument names at the right side of the screen so the notes move away from them...or at both sides, to make it easier to associate notes with them). On the other hand, a virtual drum kit arrayed across the bottom of the screen looks more like what the drummer actually sees while playing, and works well with the waterfall plot orientation I have proposed for the keyboard. Options of use to a percussionist include the abilities to restrict the virtual drum kit to instruments which the drummer actually has in the physical drum kit, and to choose which line each instrument appears on in the piano roll. The standard MIDI piano roll display offers many more percussion instruments than a typical piece of music is likely to use, making the display hard to read. Standard drum notation does not waste space on unused instruments like the MIDI display, but also does not map nicely to the physical layout of the drum kit.

• Pitched controllers other than the keyboard may align most logically to a piano roll-like display in different ways. For example, an orchestral harp, with 7 strings in each octave, maps well to a piano roll with only 7 degrees in each octave. If the layout of the piano roll matches the actual layout of the harp, it becomes possible to use a graphics interface to draw idiomatic harp glissandi directly on the display. The altered scale degrees which the harpist controls with the foot pedals could be displayed in the piano roll using different colors.

• If a piano roll view is aligned across the screen with notation, then the piano roll view also must use the conventional alignment, like the notation. This is a moot issue in most sequencers, which provide no way to align notation to a piano roll. Aligning notation to a piano roll results in strange note spacing in the notation; also, the functions of the piano roll and of notation are similar enough that the advantage of an ergonomically oriented piano roll probably outweighs that of aligning notation to it in most cases. Still, some functions -- for example, entry of tuplets -- are better visualized in notation than in a piano roll, and some musicians may find the ability to align the to views quite useful.

Scrolling choices

As already mentioned, Cakewalk offers only automated jump scrolling, displaying a new section of music to move the "now" line from the right edge of the screen to the left edge. The smooth scrolling I've described earlier would be nice for recording or playback in real time -- and would always place the "now" line at the same location on the screen when playback stops. Another important option is manual scrolling. This option is especially useful when playing a short section of music over and over ("looping"). Not all of the section of music being played would be visible, but the part being worked on would not repeatedly appear and disappear as it does with the automated jump scrolling.