Polyphonic Piano Transcription With a Note Based Music Language
Automatic music transcription (AMT) is the process of converting raw performance audio signals into digital representations of symbolic music for possible computational musicology analysis. Due to the strong overlapping in both time and frequency domain of musical signals, it is challenging to detect and extract each individual note separately in polyphonic piano music. In addition, the octave relationship among notes makes it difficult to identify a particular note from a set of isolated notes. Hence, it is important to model the high-dimensional outputs of a multi-note piano using a note based music language model.
Recent advances in AMT with neural networks have mainly focused on the development of an acoustic model for detecting different note onsets or sustains, which are then combined into a complete note sequence. A number of approaches have been proposed for this task, including frame-level acoustic models and acoustic/music language models. However, these methods often use separate neural network architectures for each state, resulting in multiple loss functions and complex inter-neural connections. Furthermore, the temporal evolution of the note states is usually handled by either a post-processing module or by learning an auxiliary model from the data.
A music language model (MLM) is a model that represents the structure of a musical sequence by encoding it as a hidden Markov chain with different state probability distributions. It can be superimposed on the outputs of a frame-level acoustic model in order to improve the performance of AMT. This type of model can capture the structural regularity in a music sequence, which cannot be captured by an acoustic model alone. MLMs are especially useful for polyphonic AMT since it can better handle concurrent notes.
https://tartalover.net/nicole-pesce
Automating Polyphonic Piano Transcription With a Note Based Music Language Model
The aim of this paper is to develop a note-based MLM for predicting a complete piano sequence from a set of corresponding notes. The MLM is trained by a restricted Boltzmann machine (RBM), which is a variant of the RNN that uses only a limited number of weights and biases. Then, it is augmented with a recurrent structure in order to model the sequences of corresponding note regions from onsets to offsets. The resulting model outperforms the original MLM and the acoustic/music language model alone in terms of note-wise F-measure, proving that it can adequately model the high-dimensional outputs of recurrent piano sequences.
https://m.youtube.com/@tartalover2682/videos
The proposed MLM is divided into two stages, namely, feature learning and label learning. In the first stage, a pair of convolutional neural networks is used to recognize the onsets of notes preliminarily, and in the second stage, attack templates for each individual piano are factorized to estimate probabilities of note combinations in an NMF based framework.
This approach achieves better recall and precision than the acoustic/music language models alone on public polyphonic piano datasets. In addition, it outperforms other adapted CNNs in note recognition and note verification, illustrating the necessity of the note-based MLM to address the problems caused by the dimension explosion in a multi-note piano sequence. Note-wise F-measure of 82% is obtained, outperforming the state-of-the-art.
