X

Week 8: Audio analysis, writing another plugin, and more added to todo list!

May 01, 2021

Hello everyone, welcome to week 8 of my senior project. After finishing my
plugin, including the code, using it, and the documentation, I am on a new
route for audio visualizing and analysis, while I am writing another plugin
for more advanced features. In this post, we will first cover some basic
introductions on how to use the audio visualizer software, Sonic Visualizer,
and what I wrote for another plugin.

Sonic Visualizer revisit

Sonic Visualiser is an application for viewing and analysing the contents of
music audio files. It is available for most of the operating systems. After
downloading it, we can load audios into by simply clicking ‘open’ in the
interface. In order to include more audio files, we need to do ‘adding another
audio’ in the interface.

The key point in our analysis is to compare different audios together: the
audios are the processed ones under different color filters or the ones
produced under different plugin features. The case for this week is to compare
two audios processed under different filters. In order to add both in the same
place, we need to first import the two audios together. Then, for one of the
audio, we add another layer: go to the layer option in the software and click
‘add spectrogram’: now we can choose the audio we want to add upon the other.
The result will be like this (the purple curve and green curve represent two audios):

 

Result Diagram

 

There are many other details such as adding a threshold and dbs, or shifting
the unit of axis. More
discussion will be written for next weeks.

Writing yet another plugin

After adding different options in the CombineTrackWeighted, there is another
issue worth tacking: how to identify the state of one point in the track as
the position in the whole track? To think about this, we can think about the
track as a curve, and to express each point’s state, we can use the slope of
the point in the curve, or as known as derivative. In my plugin here, I try to
write the slope of each point in the track and try to find the outlier: if one
point’s motion, expressed by the derivative, is way too different, then I will
try to ignore it.

The code is here, the derivative is calculated by approximating the slope
of the curve at the point using a hyper-parameter step:

else
{
int step = 1;
for (int i=0; i < track.Count; i++)
{
double sum = 0;
for (int j = 2; j < track[i].Count; j++)
{
sum += (track[i][j] – track[(i+step)%track.Count][j])/step;
}
double average = sum / track[i].Count;
for (int j = 2; j < track[i].Count; j++)
{
if((track[i][j]-track[i][(j+step)%track[i].Count])/step<average)
{
if (j == 2)
track[i][j] = 0;
else
track[i][j] = track[i][j – 1];
}
}
}
int num = track[0].Count – 1;
for (int i = 0; i < track.Count; i++)
{
track[i][1] = (track[i].Sum() – track[i][0]) / num;
track[i].RemoveRange(2, num – 1);
}
}
return 0;
}

However, there are more things needed to do to improve this plugin: the way of
using the average of slopes isn’t perfect, and better ways need to be
introduced to the plugin. Besides, after talking to some other programmers
working on weaver, I realized I can use smoothing to improve the quality of
the sound (using the points around a point to get a better value of the track value). More content will come next week.

Next Week

For the next week, I am first going to redo the trackings for some of the
audio images and process some other images. Besides, I will try to improve my
derivative plugin feature. I will also try to go deeper into the audio
analysis and start writing my paper. Stay tuned!

One Reply to “Week 8: Audio analysis, writing another plugin, and more added to todo list!”

  1. Eric M. says:

    Glad to hear all the developments! Still can’t claim to follow, but I’m sure you’re getting real work done!

Leave a Reply