Digital Electronics Lab 2018 – Final Project Report

1. Simple Explanation

Essentially, my box is a bunch of knobs that allow me to communicate with a computer in whatever way I want. I can set the knobs to control different things in different pieces of software, such as those for making music and visuals. The idea has always been to provide tangible feedback to the user in real time. With a simple USB connection, the entire device can power itself and exchange information with a computer. The layout has been kept simple so I do not have to think very much about what my hands are doing while I’m using it; this frees me up to consider other creative aspects of my projects.

2. Video

3. Detailed Explanation

Teensy Code (C++)

Let’s start with the hardware. 20 potentiometers in a 4×5 grid are directly mounted to a flat sheet of acrylic. LEDs above each row indicate which step (column) of the grid is being powered and measured. All the wires on the back of the panel are grouped by type (signal, power, LED), and connect via pin headers to the Teensy below. The Teensy rests in a custom-soldered bed of pin headers atop a PCB, for easy troubleshooting. For now, to reduce the strain on any wires, an acrylic box suspends the panel 4″ above the Teensy on the PCB.

The acrylic panel was laser-cut to the exact dimensions necessary to accommodate the potentiometers. The shafts of the pots protrude through the front of the panel, and are fastened by a washer and nut. Knurled knobs atop the shafts add aesthetic appeal.

Every pot is connected to ground through a network of wires that serve as the grounding bus. The power for each row is supplied by a digital-on signal from one of the digital output pins on the teensy. (Each column/step of the power rails are electrically connected to each pot, but are separate from the others.) This configuration requires fewer wires and used pins on the Teensy. Four ribbons of five wires each attach to the middle lugs of each potentiometer, per column. On the Teensy side, detachable male pin headers allow these ribbons to be disconnected for repairs.

As will be explained in a moment, the project has two modes: one where it responds to MIDI coming from a different source (like a daw), and another where the Teensy spits out MIDI to be used by a physical or software instrument. A SPST switch on the board next to the Teensy allows for switching between these modes (however, I suggest stopping whatever relevant activities are happening in the computer before switching modes).

Now for the software. The Teensy source code accesses the Serial and MIDI libraries. All the input and output digital pins are configured in the setup() function.

Let’s begin with declaration of variables. stepLength is set to the default time value for which a single step will hold, in milliseconds. ledPins[] is an array that holds the references for the output pins for the LEDs. currentStep holds the value for the step in the sequence, and is updated after each stepTime (for outputMode()) or when the Teensy receives an incoming MIDI signal (for inputMode()). lastStep is used for comparisons in order to properly trigger later events (like sending Serial data). switchPin holds the reference for the center pin on the SPST switch that’s used to go between modes. potMatrixPins[][] holds all the references for the analog pins. Meanwhile, potMatrix[][] holds the values read on each step. rowPowerPins[] holds the values for the digital output pins that light up the LEDs on the panel. currentMIDI[] and lastMIDI[] are used by outputMode() to send out MIDI, convert between MIDI and serial (byte) values, and compare between two adjacent steps.

I intentionally kept the loop() function pretty minimal, and broke out everything into a bunch of other functions. Depending on which way the mode switch is thrown, loop() will continuously call either inputMode() or outputMode(). These respective modes will call their own sub-functions.

First, inputMode(). It simply reads incoming MIDI and calls the event handler. The actual event handler (myNoteOn()) powers up the current step, invokes readPotValues() to read the values, and transmitSerial() to transmit bytes of Serial. Then myNoteOff() cuts the power to the pots and increments stepCount.

outputMode() is not broken out like the structure for the input mode, because the Teensy is directly in control of whatever’s happening; it’s not waiting to read incoming values from elsewhere. Its code is nearly identical to that of myNoteOn() and myNoteOff() combined. The only differences are that it calls the function txMIDI() to transmit MIDI values to a connected device and that it delays the program function for previously read-and-mapped stepLength values.

Processing Code (Java)

Now, let’s discuss the code in Processing, a visual scripting language coded in Java. Late in the project, I migrated my code from the traditional Processing IDE to Eclipse, which is much better for more complicated software development. The default IDE is what’s known as a PApplet, which extends the functionality of Java with new classes and methods, while simplifying some of the syntax. In order for Eclipse to run Processing sketches, I had to import essential libraries that are buried within the “package contents” of the Processing app. These .jar files are self-contained with the functionality I need to work with Serial and to make anything display on the screen.

One of the tradeoffs of working in Eclipse is that the syntax becomes more complicated. Java really cares about the cope of variables and methods, so I have to designate them as public or private, in addition to their return types. I also have to specify that my class inherits the functionality of the PApplet class. Inheritance is an enormously important concept in higher-level programming. Essentially, one class is able to take on the methods and variables of the “parent” class, and can even extend the functionality. This makes the “child” or subclass a subtype of the parent’s class type. Writing my class as inheriting the PApplet’s methods allows me to call some of its methods for my own uses. If you know how to do this, it’s definitely worth using Eclipse.

The main() method is analogous to the loop() function in the C++ code. It’s what runs while the program is operating, cooperating with all the other methods to make everything work. PApplet has its own overriden main method, which I use inside my class’ own main method to have the PApplet handle everything I write in my class.

One extra method I have to create is settings() at the top of the class. This is where I set the size of the canvas on which I will draw the visuals. setup() takes care of instantiating Serial communication and a PImage object called fader, which will be used to loop delayed projections of previous frames atop one another. incomingSerial, mappedSerial, lastStep, and currentStep are exactly what they sound like; they store the recorded incoming serial values. current is an array that holds all the bytes of the pot values from all steps in the cycle.

Now for the other methods. getSerial() recieves the incoming Serial that’s sent out by the Teensy. interpolate() creates smooth transisitions among the frames for the objects on screen. It calls linInt(), which compares the target and current values, and then advances the values slightly towards their goal. This is actually a recreation of the lerp() function that Processing already has. The problem with lerp(), however, is that it is not friendly to the standard double-type values that Eclipse interprets decimal numbers as (Processing likes floats). So, I figured out the logic behind lerp() and rewrote the method to take doubles, just to make my life a little easier.

All these methods serve the primary method called draw(), which is analogous to the main() method in Java and the loop() method in the Arduino C++ code. Every frame, it first clears the screen, takes in the serial values from the Teensy (via getSerial()), linearly interpolates them between the last stored value for the previous step and the current incoming serial, and then draws ellipses to the screen.

The PImage object “fader” captures what’s on the screen in the previous frame and displays it again in the following, before the new ellipses are drawn. When this is continuously repeated, the result is an echoing or trailing effect of multicolored ellipses dancing on screen, either to the timing of the Teensy (in outputMode()) or from the incoming MIDI from a DAW (in inputMode()).

In a nutshell, that’s all the code that I’ve written for this project! The Teensy’s compiled C++ sends information out, while the processing Java code receives all this and makes something appealing out of it.

4. Source Code

Arduino (Teensy) Code (C++):

#include <MIDI.h> // invoking MIDI library.
// Serial included by default, as well.

int stepLength = 250; // default, in milliseconds.
int ledPins[4] = {0, 1, 2, 3}; // not including ground.
int currentStep; // 0-3 (4 steps)
int lastStep; // for comparison purposes

int switchPin = 28; // for changing modes

// specific analog reading row pins
int potMatrixPins[4][5] =
{
  {A0, A1, A2, A3, A4},
  {A5, A6, A7, A8, A9},
  {A13, A14, A15, A16, A17},
  {A18, A19, A20, A21, A22},
}; // all the attachments of pots to the analog pins.

int potMatrix[4][5]; // stored values from analogRead()s

int rowPowerPins[4] = {12, 11, 10, 9}; // not including ground

int currentMIDI[5]; // outputed/read MIDI per step
int lastMIDI[5]; // for comparison purposes

void setup() // configuring the Teensy for proper functionality.
{

  pinMode(switchPin, INPUT); // Sets up pin that changes between inputMode() and outputMode()

  for (int i = 0; i < 4; i++) // Setting up pins for lighting up LEDs
  {
    pinMode(ledPins[i], OUTPUT);
  }

  for (int i = 0; i < 4; i++) // Setting up pins for powering columns (steps) of pots.
  {
    pinMode(rowPowerPins[i], OUTPUT);
  }

  Serial.begin(115200); // Establishing baud rate for Serial
  usbMIDI.setHandleNoteOn(myNoteOn); // Establishing which function is called (myNoteOn()) when a note-on MIDI signal is detected
  usbMIDI.setHandleNoteOff(myNoteOff); // Establishing which function is called (myNoteOff()) when a note-off MIDI signal is detected.
}

void loop() // continuously runs during program execution
{
  if (digitalRead(switchPin)) // If the switch is set to on/HIGH, go into input mode.
  {
    inputMode();
  }

  else // If the switch is set to off/LOW, go into output mode.
  {
    outputMode();
  }
}


// six values: step #, pot values 1-5.
// The first should correspond to the step (in this case, the row of the array).
// The others should correspond to the analog read values for the 5 respective pots.
void transmitSerial()
{
  Serial.write(currentStep); // Transmission of first value
  for (int i = 0; i < 5; i++) // it's 5, not 6!!!
  {
    Serial.write(potMatrix[currentStep][i]); // Transmitting pot values.
  }
}


// mapping range of Serial values to MIDI note values.
void calculateMIDI()
{
  for (int j = 0; j < 5; j++)
  {
    lastMIDI[j] = currentMIDI[j]; // Priming for later comparisons.
    currentMIDI[j] = map(potMatrix[currentStep][j], 0, 255, 24, 108); // Conversion from Serial to MIDI.
  }
}

void txMIDI()
{
  // We need to send out usable MIDI values to other places,
  // so we first have to "convert" from Serial values to MIDI values.
  calculateMIDI();

  if (lastMIDI[0] != currentMIDI[0]) // detection of change in MIDI note
  {
    // Turn off previous note
    usbMIDI.sendNoteOff(lastMIDI[0], 0, 1);

    // Control change
    usbMIDI.sendControlChange(10, currentMIDI[2], 1);

    // Turn on next note
    usbMIDI.sendNoteOn(currentMIDI[0], 100, 1);
  }
}

// Event handler for when incoming MIDI note detected
void myNoteOn(byte channel, byte note, byte velocity)
{
  // turn on both LED indicator and power to column of pots.
  digitalWrite(rowPowerPins[currentStep], HIGH);
  digitalWrite(ledPins[currentStep], HIGH);

  // Once pots are powered, read analog values.
  readPotValues();
  // Send them out (to processing or elsewhere).
  transmitSerial();
}

// Event handler for when incoming MIDI off signal detected
void myNoteOff(byte channel, byte note, byte velocity)
{
  // Cut power to LED indicator and respective column/step of pots.
  digitalWrite(rowPowerPins[currentStep], LOW);
  digitalWrite(ledPins[currentStep], LOW);

  // increment currentStep value (remain within 0-3)
  currentStep ++;
  currentStep %= 4;
}

// Not yet used; for later implementation.
// Void myControlChange(byte channel, byte control, byte value)

// Use external (incoming) MIDI values for control
void inputMode()
{
  usbMIDI.read(); // call incoming MIDI-on event handler (myNoteOn())

  // Not yet used; for later implementation.
  // usbMIDI.setHandleControlChange(myControlChange);
}

// Use internal pot values for MIDI control.
void outputMode()
{

  // Turning on power. to LEDs and pots.
  digitalWrite(rowPowerPins[currentStep], HIGH);
  digitalWrite(ledPins[currentStep], HIGH);

  // Read pot values for the row.
  // If you make this a double for-loop,
  // you will always be reading zero values on the other rows!
  readPotValues();

  //Serial data to be sent to Processing (or elsewhere).
  // Transmit MIDI to DAW or other compatible instrument.
  //******** Where the magic happens.
  transmitSerial();
  txMIDI();

  stepLength = map(potMatrix[currentStep][4], 20, 255, 1, 750); // 20 seems to be as low as they go.
  delay(stepLength);

  // Turning off power to LEDs and step/column of pots.
  digitalWrite(rowPowerPins[currentStep], LOW);
  digitalWrite(ledPins[currentStep], LOW);

  // Iterate step (within range 0-3).
  currentStep ++;
  currentStep %= 4;
}

// Analog-read the pot values for the current powered-up step.
void readPotValues()
{
  for (int j = 0; j < 5; j++)
  {
    potMatrix[currentStep][j] = 1023 - analogRead(potMatrixPins[currentStep][j]);
    // map them from 10-bit values to 8-bit values (bytes).
    potMatrix[currentStep][j] = map(potMatrix[currentStep][j], 0, 1023, 0, 255);
  }
}


Processing (Java) Code:


import processing.core.*; // Underlying code to make the Processing PApplet run.
import processing.serial.*; // Getting Processing to communicate with the Teensy.

//Testing01 is a subclass of the type PApplet, so it inherits all of the parent's functions.
public class Testing01 extends PApplet
{
	// The main() method required to make a Java program run (at least in
	// Eclipse).
	public static void main(String[] args)
	{
		// Calling the overridden main() method and passing it the argument of
		// the current class.
		PApplet.main("Testing01");
	}

	// Will capture the incoming Serial data from the Teensy.
	// Create serial object from Serial library.
	Serial mySerial;

	// Required method to create canvas.
	public void settings()
	{
		size(1080, 720); // Setting size of canvas.
	}

	// The standard function in Processing to establish the fundamentals of the
	// program.
	// Similar to in C++ for Arduino.
	public void setup()
	{
		background(0); // Clear frame; make completely black.

		// constructor (this, the serial port, the baud rate).
		mySerial = new Serial(this, Serial.list()[1], 115200);
		fader = get(0, 0, width, height);
	}

	// serial-related declarations//
	int[] incomingSerial = new int[6]; // Stores incoming Serial bytes for
										// current frame.
	int[][] mappedSerial = new int[4][6]; // Stores Serial bytes that have been
											// converted to pixel values.
	int lastStep; // Same as in C++
	int currentStep; // Same as in C++

	// Contains current (mapped) values, for comparison with interpolated
	// values.
	int[][] current = new int[4][5]; // does not include step count

	/// Visual-related declarations.
	PImage fader; // Image that will be scaled for delayed-feedback effect.

	public void draw() // Analogous to loop().
	{
		// Clears frame.
		background(0);

		// Reduces alpha (transparency) for incoming projection of image of
		// previous frame.
		tint(255, 255, 255, incomingSerial[5]);
		image(fader, 0, 0); // Projection of image of previous frame.
		// Disables transparency settings for projection of elements in current
		// frame.
		noTint();

		rectMode(CORNER); // Establishes top left corner as pixel (0,0).
		// Retrieval of Serial from Teensy.
		getSerial();
		// Interpolates (blends) values of current ellipse dimensions with those
		// just read over Serial.
		interpolate();

		// For reference/comparison.
		lastStep = currentStep;
		currentStep = incomingSerial[0];

		// Drawing ellipses of current frame to canvas.
		for (int i = 0; i < 4; i++) 		{ 			// Setting ellipse color. 			fill(incomingSerial[1], incomingSerial[2], incomingSerial[3]); 			// Draws the actual ellipse (per iteration of loop). 			ellipse(current[i][0], current[i][1], current[i][2], current[i][3]); 		} 		// Overwrites previously captured image with updated capture of the most recent frame. 		fader = get(0, 0, width, height); 	} 	// Retrieve Serial from Teensy. 	void getSerial() 	{ 		// Ensuring all 6 Serial values have been transmitted at a time. 		if (mySerial.available() >= 6)
		{
			for (int i = 0; i < 6; i++)
			{
				// The int() casting is apparently unnecessary
				// Commit new Serial values to memory.
				incomingSerial[i] = (mySerial.read());

				// Mapped values are arbitrary.
				// In this case, I'm using the dimensions of the canvas as the limits of the mapping.
				mappedSerial[currentStep][i] = (int) (map((float) (incomingSerial[i]), (float) (0.0), (float) (255.0),
						(float) (0.0), (float) (height)));
			}

			// Debug
			// Prints Serial values to console.
			for (int i = 0; i < 6; i++)
			{
				print(incomingSerial[i] + " ");
			}
			println(); // New line.
		}
	}

	// Smoothly iterates between current positions/dimensions of objects on screen and their eventual targets,
	// as defined by the incoming Serial data.
	public void interpolate() 
	{
		for (int i = 0; i < 4; i++)
		{
			for (int j = 0; j < 5; j++)
			{
				// Don't interpolate if the difference between the current position and target position 
				// is less than 3 pixels.
				if (abs(current[i][j] - mappedSerial[i][j + 1]) < 3) current[i][j] = mappedSerial[i][j + 1];
				// If the difference is greater than that, linearly interpolate.
				else current[i][j] = linInt(current[i][j], mappedSerial[i][j + 1], 0.1);
			}
		}

	}

	// Linear interpolation. 
	// Method that combines map() and lerp() functionality 
	// into something easier for me to use in the Eclipse environment.
	public int linInt(int a, int b, double c)
	{
		double delta = b - a; // Find difference between two points.
		delta *= c; // Multiply difference by some factor.
		return a + (int) (delta); // Add this scaled difference to first point.
	}
}
Advertisements

Digital Electronics Lecture 2018 – Assignment for Week 11: Final Project Writeup (for Lecture, Week 1)

As Mark suggested, I’ve spend most of my focus this week on the actual hardware for my project.

This past Friday afternoon, I travelled with Jack to pick up a 1/8″X18″X24″ sheet of clear acrylic at Canal Plastics.  (They have quite an interesting selection of different plastics there, who would have figured.)

In addition, my potentiometers (knurled shafts) and knobs arrived in the mail. It turns out that the knobs were poorly labelled – they are meant for pots with smooth shafts, but the description mislead me by saying “knurled.” That was annoying, so I then ordered 20 different knobs, which turned out to be correctly labelled. Also, as Mark suggested, I bought a pair of digital calipers. These would prove very useful in the coming days.

The latter shipment arrived on Monday. I booked a time to use the prototyping lab (in the basement of the Leslie eLab) for Tuesday during the weekend, indicating that I would need someone to stop by and assist me in the design of the Adobe Illustrator file.

IMG_7201Franklin, a guardian at the eLab, read my request, and set aside his time to help. I read out the measurements I acquired with my digital calipers of various important dimensions, including the diameter of the potentiometer shafts, the length and width of the stabilizing metallic protrusion on the sides of the pots, and the distance between these two items. I then determined how far apart I wanted the pots to be placed in relation to one another. Here is a list of the important measurements we found:

  1. Diameter of shaft: 0.27″ (radius = 0.135″)
  2. Distance between edge of shaft and stabilizing protrusion: 0.155″
  3. Width of protrusion: 0.05″
  4. Height of protrusion: 0.083″
  5. Distance (in both x and y-axes) between adjacent pot shaft centers (decided upon by me): 0.65″
  6. Sheet width (decided upon by me): 5.255″
  7. Sheet height (decided upon by me): 6.17″

We first cut a test piece to ensure the measurements were right before we went ahead and duplicated the holes for the other pots.

Carefully inputting these measurements one at a time into Illustrator, we came up with the following file:

img_7200.jpg

Franklin doubled the pass of the laser, just to ensure that all the unwanted pieces would separate. The result was impressive: very clean and even. Thank you Franklin!

 

Popping out some of the small pieces proved difficult, but I eventually managed to wiggle them free with a very small box cutter:

IMG_7210

The whole boxcutter process was reminiscent of wiggling out my baby teeth as a kid: some fell out easily, while others were rather stubborn.

To remove residue, I used cleaning pads containing isopropyl alcohol. This actually took a long time to do thoroughly.

IMG_7214

When I arrived home in the evening, I began placing the 20 potentiometers in their spots. Strangely, it was easier to mount them on one face than the other. I suspect this is due to the fact that the laser may have done a slightly better job on one – the one that was directly facing the beam. Like before, some went in with an audible snap, while others required quite a bit of force on my part. (I just had to be careful not to break the acrylic.)

IMG_7217

Finally, I placed on the knobs — the ones that had the knurled insides. However, they ended up being really long, and I’m afraid that they may fall off too easily. Therefore, I might try to use a hand power tool (like a Dremel) to shorten them.

So, here’s what I’m left with as of now! I’m quite happy and excited with how it’s turning out.

IMG_7221

 

Time Spent: 3.5 hours.

What I have left to do

  • Solder the power and grounding wires.
  • solder wires to a board onto which the Teensy will be mounted.
  • Create a serial communication protocol
    • testing this protocol
  • Interfacing with Arduino and Processing
  • Building outwards: making commit functions and adding functionality
    • [simply dependent on time]

What left to buy:

  • Perhaps mounts or a box? Not essential at the moment.