# Python Challenge 2 - How do computers recognise songs?

## Python Challenge 2 - How do computers recognise songs?

How can a computer recognise a song? Everyone has seen it: a snippet of a song is recorded on a phone, then after a few seconds it recognises the song that is playing. This week, we consider how humans (and machines!) can recognise songs using a Spectrogram.

Sound waves are recorded by a microphone as a series of numbers. These signals can therefore be plotted. However, it is actually quite challenging for a human (and even a computer) to analyse all of the information in one go just by using that information.

A common approach is to look at the frequency content of a signal (the `spectrum'). This is done by chopping the signal into sections, and then calculating the Fourier Transform of the chopped up signal. Putting all of the chopped up frequency content together creates a Spectrogram of a song. This week's program will produce a spectrogram of a song, as below. ## Target:

Write one page of Python that will:

1. Load a song, and then have Python play the song back to you.
2. Plot the waveform of the song.
3. Plot the logarithm of the spectrogram of your song, against time and frequency.

## Extensions:

• When you calculate and then plot the frequency data, experiment by increasing the time resolution. What do you notice about the `sharpness' of the frequency if you reduce the time step?
• What you observe is related mathematically to a well-known physics phenomenon. Can you identify which phenomenon it is related to?
• A paper detailing a popular song-search algorithm is freely available online. Can you find this paper? How does it use a spectrogram to find songs?

## Python Challenge 2 Hints and Tips

### Hints for Python Challenge 2

The easiest way to manipulate audio data is as a '.wav' file.

#### Packages

In this solution to this problem, pygame can be used to play sound. The scipy packages has modules for loading '.wav' sound files within scipy.io, and scipy.signal has a method to create a spectrogram.

Within matplotlib the contourf function is used to plot the spectrogram above. Try experimenting with different colourmaps (using colormap within matplotlib)

#### Hints

• It can be quite tricky to find the path to a file sometimes. On a first run, the easiest way to open a file may be to specify the path directly, but it is best practise to append paths to sys.path.
• Often a wav file will be in `stereo', so the sound is in two channels (left and right). A single signal can be found for analysis either by finding the average of these two signals (for example using numpy.mean) or each channel could be chosen individually.
• The data that is read in from scipy.io is for a whole song. What time does each entry of the array count for? It is helpful to only select a few seconds of the song to analyse, because each file is large.
• The output of the spectrogram is in the form of 'complex numbers'. To convert to find the 'magnitude' of the response, which we are plotting, use the 'abs' function within numpy.
• When plotting the spectrogram, sometimes the range is too large to get a nice picture. Adding 1 to all points in the logarithm (for example, Z = np.log10(np.abs(spect) + 1)) sometimes makes figures more clear by ignoring very small responses.

#### Notes

Sound is perceived on a logarithmic scale with frequency. That is, doubling the frequency increases the pitch of a note by one octave. Plotting the spectrogram with a logarithmic frequency scale (using, for example, plt.gca().set_yscale('log') may therefore make the spectrogram more intelligible, and also highlights low notes better than if a linear scale from 0 to 22 kHz is used (human hearing is approximately 0 to 20 kHz).