To expand the versatility of my final project for Digital Audio Programming I decided to explor adding Vocal removal.

The standard technique is relatively simple to implement but can be hit-or-miss. It relies on the assumption that vocals will be mixed ‘center’ meaning equal energy in each channel, left and right. By negating one channel of audio and adding it to the other the vocals in theory will cancel out.

This works near perfectly with some songs and not at all with others. Some songs completely cancel out, indicating that they were not very stereophonic to begin with.

Another issue to deal with is that MP3 is a ‘lossy’ encoding scheme so the audio itself is altered when it is saved. This means that even if a certain segment of the song is supposed to be the same in both channels there may be slight differences. The result is that ‘echoes’ of the vocals can be heard. Sometimes it is barely noticeable, other times it is completely counterproductive and renders the method unusable.

The vocal removal effect is now integrated with the Juce Pitcher, before the pitch shifting. It is important to do it before pitch shifting because the pitch shifting would render any chance of effective vocal removal essentially zero.

An interesting thing to note is that the observable/desired effects of pitch-shift are focused more on the vocals. Removing the vocals and then trying to pitch shift does not always result in a largely discernable change. Nevertheless, it is possible to do both at once.

The CPU usage meter in Ableton Live hits a max of about 30% when using both vocal removal and pitch shifting which is higher than desired but still workable.

I also added enable/disable buttons for the Low-pass and high-pass filters in case it is desired to pitch shift on the entire frequency range.

For MAT200A, Arts & Technology we are making websites for our final projects. The projects are based on the intersection of arts and technology and we are paired with a partner to collectively develop and present a concept.

Our concept is ‘Vivari: Representative Avatar for Online Shopping & Gaming’. We propose placing ‘measurement sites’ in retailers that measure a user’s body shape and creates a virtual avatar in their image. In addition to simply representing them online it can be used to shop for clothing.

By measuring the user’s body the goal is to create a model that can ‘try on’ clothing and have a realistic depiction of how those clothes fit. This will make online shopping a more enjoyable and accurate process.

In addition to online shopping, we propose creating ‘Virtual Shopping Kiosks’ that allow the user to shop for clothes from any number of participating retailers and have their choices be projected on their avatar in front of them. It can be located in a mall or other public space and allows for shopping at stores that may not have a physical location nearby.

In preparation for our website I have been creating 3D models of the various assets involved, starting with the measurement and shopping kiosks. I used Google Sketchup which is a very simple program for basic 3D modeling.

Avatar Creation Station

Virtual Shopping Kiosk

I am developing a Virtual Studio (VST) Plugin for MAT240A, Digital Audio Programming. VST plugins are used in professional audio software such as Cubase, Sony ACID and Ableton Live. Development is simpler than making a standalone audio application since the details of manipulating sound files are taken care of by the plugin host.

The host gives you a buffer of values corresponding to the audio stream and the plugin operates on the buffer and returns it so it can be passed to the next plugin in the host, or the output.

I am using the JUCE framework to develop my plugin. Though JUCE has many features and benefits I am using it because it greatly simplifies the Graphical User Interface development through a tool called the Jucer.

My project goal is to create a plugin that will shift the pitch of the audio while keeping duration constant (since it is used in a real-time playback situation). To make things more interesting I want to have multiple bands that let me pitch-shift certain segments of the frequency spectrum independently.

Pitch shifting can be accomplished in the time-domain or the frequency-domain. Time-domain involves modifying the sampling rate and discarding samples while frequency-based shifting uses the Fast Fourier Transform (FFT).

I started by reading about the various techniques and decided it would be interesting to try it with the Fourier transform. This article has very good information about the method as well as the FFT in general. The FFT method is effective but computationally intensive.

Many people have written open-source libraries of code that are extremely optimized for this task. As the goal of this class is to explore the creation of audio-processing applications rather than the effects algorithms themselves I decided to incorporate one such library into my code.

SoundTouch is an open-source library that provides pitch-shift, time-stretch and playback-rate effects. It is object-oriented and very easy to use.  It is used in many open source applications and audio plugins.

The JUCE framework has built-in classes for audio filters as part of the audio plugin code which will make splitting the signal into different bands much easier.

Below is the curent user interface for the plugin as it is begin developed:

MAT594O,  “Sensors and Interfaces for Multimedia Art” taught by Professor Stephen Pope

The goal of the class is to use electronic sensors and other components to make a project that can be used for Multimedia Art. Beyond that there are not many requirements.

For my project I am using a Heartrate Monitor strap from Polar along with the Heartrate Monitor Interface (HRMI) from Sparkfun Electronics.

The Heartrate Monitor is a plastic strap with electrode patches and a transmitter that the user wears around their chest. In typical fitness training situations the user also wears a special Watch that receives the signals from the monitor and displays the heartrate.

I ordered the monitor from Amazon Marketplace (used) and was fortunate to also receive the watch which is useful to verify the software is receiving an accurate heartrate value.

The HRMI provides a simple USB interface to get the numerical heartrate in beats-per-minute over a USB-Serial port.

(more…)