How to detect how similar a speech recording is to another speech recording?

voice comparison python
compare audio files
voice analysis python
split different voices in an audio recording python
audio sentiment analysis python
musicg
word similarity algorithm

I would like to build a program to detect how close a user's audio recording is to another recording in order to correct the user's pronunciation. For example:

  1. I record myself saying "Good morning"
  2. I let a foreign student record "Good morning"
  3. Compare his recording to mine to see if his pronunciation was good enough.

I've seen this in some language learning tools (I believe Rosetta Stone does this), but how is it done? Note we're only dealing with speech (and not, say, music). What are some algorithms or libraries I should look into?


Chronset: An automated tool for detecting speech onset, This kind of problem is usually solved using machine learning techniques. Break down the signal into a sequence of 20ms or 50ms frames. Then, using a training set of audio you have manually labeled as being a speech / not speech, train a classifier (Gaussian mixture models, SVM) on the frames features. This will permit you to classify unlabelled frames into speech/non-speech categories.


Idea: The way biotechnologists align two protein sequences is as follows: Each sequence is represented as a string on an alphabet as(A/C/G/T - these are different types of proteins, irrelevant for us), where each letter (here, an entry) represents a particular amino acid. The quality of an alignment (its score) is calculated from the similarity of each pair of corresponding entries, and the number and length of the blank entries that need to be inserted to produce that alignment.

Same algorithm (http://en.wikipedia.org/wiki/Needleman-Wunsch_algorithm) can be used for pronunciation, from substitution frequencies in a set of alternate pronunciations. Then you can calculate alignment scores to measure the similarity between the two pronunciations in a way that is sensitive to the differences between phonemes. Measures of similarity that can be used here are Levenshtein distance, phoneme error rate, and word error rate.

Algorithms The minimum number of insertions, deletions and substitutions required for transformation of one sequence into another is the Levenshtein distance. More info at http://php.net/manual/en/function.levenshtein.php Phoneme error rate (PER) is the Levenshtein distance between a predicted pronunciation and the reference pronunciation, divided by the number of phonemes in the reference pronunciation. Word error rate (WER) is the proportion of predicted pronunciations with at least one phoneme error to the total number of pronunciations.

Source: Did an Internship on this at UW-Madison

What needs to be measured to compare two speech signals?, I guess you need to know more about speech recognition. be relatively higher than two roughly similar sentences recorded in different environment, i want use Mfcc feature extraction technique to identify important components of audio  When you listen to a recording of your voice, is that actually how you sound to others or does the act of recording your voice change it someh


The musicg api https://code.google.com/p/musicg/ has a audio fingerprint generator and scorer along with source code to show how its done.

I think it looks for the most similar point in each track, then scores based on how far it can match.

It might look something like

import com.musicg.wave.Wave
   com.musicg.fingerprint.FingerprintSimilarity
   com.musicg.fingerprint.FingerprintSimilarityComputer
   com.musicg.fingerprint.FingerprintManager

double score =
new FingerprintsSimilarity(
    new Wave("voice1.wav").getFingerprint(),
    new Wave("voice2.wav").getFingerprint() ).getSimilarity();

Determining how similar audio is to human speech, One method is to use speech recognition software to obtain words from an audio segment. However, this method is unable to come up with how "  Noise in speech and text : Noise is any unwanted signal distorting the original signal. Adding noise to speech vs adding noise to text is very different. Given a speech signal with amplitude s[n], where n is the sample index, noise is any other signal, w[n] which interferes with the speech.


A carefully configured Levenshtein distance should do the trick.

Separating different speakers in an audio recording, A transcription result can include numbers up to as many speakers as Speech-to-​Text can uniquely identify in the audio sample. When you use speaker diarization​  The analysis of speech onset times has a longstanding tradition in experimental psychology as a measure of how a stimulus influences a spoken response. Yet the lack of accurate automatic methods to measure such effects forces researchers to rely on time-intensive manual or semiautomatic techniques. Here we present Chronset, a fully automated tool that estimates speech onset on the basis of


you can use Musicg https://code.google.com/p/musicg/ as roy zhang suggested. In android, just include musicg jar file in your android project and use it. A tested example:

import com.musicg.wave.Wave;
import com.musicg.fingerprint.FingerprintSimilarity;


        //somewhere in your code add
        String file1 = Environment.getExternalStorageDirectory().getAbsolutePath();
        file1 += "/test.wav";

        String file2 = Environment.getExternalStorageDirectory().getAbsolutePath();
        file2 += "/test.wav";

        Wave w1 = new Wave(file1);
        Wave w2 = new Wave(file2);


        FingerprintSimilarity fps = w1.getFingerprintSimilarity(w2);
        float score = fps.getScore();
        float sim = fps.getSimilarity();



        Log.d("score", score+"");
        Log.d("similarities", sim+"");

Good luck

[PDF] Detection and Analysis of Emotion From Speech Signals, of speaker or speech recognition becomes relatively an easier one when compared with that occurs when uttering the same thing under different emotional situations. The behavior of the AUC is similar to the recognition accuracy. Adobe says it is aware of the potential for misuse with Project VoCo, so is already working on technologies that will make it possible to detect if a recording has been tampered with – such as embedding hidden audio watermarks, which could potentially trigger voice security features used in systems like digital banking.


Advances in Nonlinear Speech Processing: 6th International , For continuous speech, it is already demonstrated that it contains more of the speech recordings in the MEEI database and perform the automatic detection of the in running speech signals by means of different implementations of the jitter​,  3. the human ear is capable of listening to speed up sounds to about 4x the original speed. You could simply listen to the recordings and detect the voices yourself. It will take less time then normally and will be more certain that you get complete results. using RX4 or another program could possible skip important parts of the recording.


Speech and Computer: 18th International Conference, SPECOM 2016, , The solution to the calibration problem is to determine parameters of score mapping In [5,13] the investigation of different types of score calibration is presented. This paper investigates the effect of speech duration in trial recordings on the a corresponding approximation function, and compare our results with a similar  located somewhere at around 1,000 Hz. There is another, much smaller peak running at around 2,500 Hz. Between 1050 and 1250 ms, we see a rise in the second peak. In a clean speech recording, we would have to conclude that we see two formants, f1 and f2, of a fronting and raising diphthong-like sound; however, this certainly is not what


Robust Speech Recognition of Uncertain or Missing Data: Theory and , We have found that an MDT recognizer that is trained with speech from the much lower accuracies than previously obtained on similar recognition tasks (​such as With real-world recordings, however, the speech in the other recording in the SPEECON and SpeechDat-Cardatabases makes it possible to detect strong  To effectively mimic someone's voice, an original recording is needed of the person you want to imitate. Then you need to find and record someone reading the exact same text in a similar voice