At this step, it is crucial to analyse some random audio samples from the collected data. This procedure, will help us to better understand our data by visualizing the audio files with various techniques. Furthermore, this analysis aims to extract audio features that will be used later on in the building process of our model.
Audio feature extraction
For audio feature extraction we will implement some methods and functions from the torchaudio (opens in a new tab) python framework. We will try to explore the relationship between common audio features, such as the waveform, with advanced features the torchaudio's API offers For presentation purposes we will present two different examples from the collected data, not yet labeled with aprox 4 seconds length each.
Audio
Jan23_21-00-33-hip-hop-228614956.wav
Stats
- File size: 1008080 bytes
- AudioMetaData(sample_rate=44100, num_frames=168000, num_channels=2, bits_per_sample=24, encoding=PCM_S) Sample Rate: 44100 Shape: (2, 168000) Dtype: torch.float32
- Max: 0.330
- Min: -0.258
- Mean: -0.000
- Std Dev: 0.049
Waveform
![Hello](/_next/static/media/Jan23_21-00-33-waveform.a6c9c97a.png)
def waveform(waveform, sample_rate, title=None, save=False):
waveform = waveform.numpy()
num_channels, num_frames = waveform.shape
time_axis = torch.arange(0, num_frames) / sample_rate
figure, axes = plt.subplots(num_channels, 1)
if num_channels == 1:
axes = [axes]
for c in range(num_channels):
axes[c].plot(time_axis, waveform[c], linewidth=1)
axes[c].grid(True)
if num_channels > 1:
axes[c].set_ylabel(f"Channel {c + 1}")
figure.suptitle("waveform")
plt.show(block=False)
if save:
figure.savefig(Util.create_plot_path(title=title))
import torchaudio
waveform, sample_rate = torchaudio.load(AUDIO_SAMPLE_FILE)
Visualization.waveform(waveform, sample_rate=sample_rate, title=AUDIO_FILE_TITLE, save=True)
Spectogram
![Spectogram](/_next/static/media/Jan23_21-00-33-hip-hop-228614956-2.2b49ccc0.png)
Spectogram of 2 channels with 44100 sample rate
Torchaudio feature extraction
Mel Spectogram
MelSpectrogram for a raw audio signal
![Spectogram](/_next/static/media/Jan23_21-00-33-Spectogram.a20a52b9.png)
![Mel Spectogram](/_next/static/media/Jan23_21-00-33-MelSpectrogram.7db349fa.png)
mel spectogram parameters
n_fft = 1024
win_length = None
hop_length = 512
n_mels = 128
MFCC
Mel-frequency cepstrum
![MFCC Spectogram](/_next/static/media/Jan23_21-00-33-MFCC = Spectogram.c16b2b35.png)
MFCC spectogram parameters
n_fft = 2048
win_length = None
hop_length = 512
n_mels = 256
n_mfcc = 256
MFCC is the widely used technique for extracting the features from the audio signal, and is often used for speach recognition.
LFCC
![MFCC Spectogram](/_next/static/media/Jan23_21-00-33-LFCC = Spectogram.fda59815.png)
LFCC spectogram parameters
n_fft = 2048
win_length = None
hop_length = 512
n_lfcc = 256
Pitch
![Pitch waveform](/_next/static/media/Jan23_21-00-33-Spectogram-pitch.4a969b97.png)
Kaldi pitch with NFCC
![Pitch waveform](/_next/static/media/Jan23_22-10-03--.c50903ac.png)
Pitch feature [1] is a pitch detection mechanism tuned for automatic speech recognition (ASR) applications. This is a beta feature in torchaudio, and it is available as torchaudio.functional.compute_kaldi_pitch().
Sample #2
![Pitch waveform](/_next/static/media/Jan23_21-00-33-Spectogram-pitch-kaldi.83e212f2.png)