## Monday, February 4, 2013

### Sonification of Surfaces in MATLAB

The problem: A common problem in data visualization is the representation of higher dimensions. A common technique to deal with this problem is Principal Components Analysis (PCA), in which the original data is rotated and then projected along the directions of greatest variation, thus creating a shadow of the original data set in lower dimensions. The problem is that with PCA we can at most represent the 3 largest dimensions of variation in a 3-D plot.

The Solution: Add two additional dimensions via color and sound. First the deceivingly simple "color" option. If you use MATLAB, you know that you have several options for color maps that simply represent the Z axis... but what if your color is not related to the Z axis, but instead to other dimension. It turns out to be a non-trivial programming tasks in MATLAB (that I will comment in other entry). The second option is sonification of the image, that is to say, each point in your 3-D plot has a related sound that presents the 5th dimension of your data.

The result is easily observable in the following video:

The source code of the main function (file sonification.m) is rather simple:

function Sonification()
% By: Juan B Gutierrez - jgutierr at uga dot edu
% Source: http://www.math.uga.edu/~juan

% First, create a dummy surface with a color map representing an
x = 1:.1:10; y = 1:.1:10; z = sin(x)'*cos(x);
fig = figure(1);
surf(x,y,z);
colormap([1,0,0; 1,1,1]);

% Now, add the handler to the cursor event in the plot
h = datacursormode(fig);
set(h,'UpdateFcn',@DataCursorSound)
end

And the auxiliary functions are simple too  (both can be placed in a single file called  DataCursorSound.m); note that I am using a function written by Ryan McGee (who has developed some sophisticated sonification):

function txt = DataCursorSound(empt,event_obj)
% By: Juan B Gutierrez - jgutierr at uga dot edu
% Source: http://www.math.uga.edu/~juan

try
% Retrieve position in the plot
pos = get(event_obj,'Position');

% This is an artificial way of generating
tonic = floor(pos(1));
if bitor(tonic==0,tonic==9), tonic = 4; end

key = 220 + tonic*100/abs(pos(3));
fs = 44100;
tonic = 1;
seconds = 0.5;
Triad(key, tonic, 'min', 'per', fs, seconds);

sFeature1 = num2str(pos(1));
sFeature2 = num2str(pos(2));
sFeature3 = num2str(pos(3));
fFeature4 = sign(pos(3));
if fFeature4==1, sFeature4 = 'White'; else sFeature4 = 'Red'; end
sFeature5 = num2str(key);

% Customizes text of data tips
txt = {
['Dimension 1: ',sFeature1],...
['Dimension 2: ',sFeature2],...
['Dimension 3: ',sFeature3],...
['Dimension 4: ',sFeature4],...
['Dimension 5: ',sFeature5],...
};
catch err
txt = '';
end
end

function y = Triad(key, tonic, third, fifth, fs, seconds)
% By: Ryan McGee
% Source: http://www.lifeorange.com/MATLAB/MATLAB_music.htm

%establish tonic
if tonic == 1
ton = 0;
elseif tonic == 2
ton = 2;
elseif tonic == 3
ton = 4;
elseif tonic == 4
ton = 5;
elseif tonic == 5
ton = 7;
elseif tonic == 6
ton = 9;
elseif tonic == 7
ton = 11;
elseif tonic == 8
ton = 12;
end

%establish third
if third == 'min'
c1 = 3;
elseif third == 'maj'
c1 = 4;
end

%establish fifth
if fifth == 'aug'
c2 = 8;
elseif fifth == 'dim'
c2 = 6;
elseif fifth == 'per'
c2 = 7;
end

%Set up discrete-time vector
n = [1:seconds*fs];

f1 = key/fs;
f1 = f1*2^(ton/12);
f2 = f1*2^(c1/12);
f3 = f1*2^(c2/12);

A1 = 1;
A2 = 1;
A3 = 1;