Better than Speech Recognition for Language Learning?

Speech recognition has become a popular feature in online language learning programs and apps. You've probably come across speech recognition as a language-learning tool if you've used programs such as Duolingo, Rosetta Stone, Babbel, Mondly, Busuu, Rocket Languages, etc.

As a starter, it may be necessary to distinguish between speech recognition and voice recognition.

Totalvoicetech explains the difference as follows:
"Voice recognition and speech recognition are terms that are interchangeably used. However, they both refer to completely different things. The purpose behind speech recognition is to arrive at the words that are being spoken. Therefore, speech recognition programs strip away personal idiosyncrasies such as accents to detect words. Voice recognition aims to recognize the person speaking the words, rather than the words themselves. Therefore, voice recognition software disregards language. Voice recognition can also be called speaker recognition."

Speech Recognition for Language Learners

Ideally, speech recognition provides an immersive language learning experience. Using a speech recognition feature, will help you improve your pronunciation and make you more fluent.

The idea seems compelling:
• Have the language program - with the stored and correct pronunciation of a native speaker - judge a learner's pronunciation of a sentence.

• Let the learner repeat the sentence until the program determines that there is a match with the stored and correct sentence.

What's wrong with that? Well, different from a teacher, who can explain what you do wrong (or right), a speech recognition tool can give you at best limited feedback. It can't really "grade" your pronunciation, (although some reviewers seem to suggest that some programs do.) Your pronunciation is either accepted or rejected.

Let's take Duolingo as an example. (In my experience, Rosetta Stone and Babbel, etc. are similar.) Duolingo lets you speak foreign sentences here and there and judges whether you did it correctly.

However, you have no clue whether you actually said the words right. I've actually recorded myself adding a different language but similar intonation and found my answer accepted.
Surprise: - For the Italian "lui ha vinta le elezioni", I said "Er hat die elezioni gewonnen." So, no way did my words match those of the native speaker! Maybe the only match was that the length of both sentences was nearly the same.

Or sometimes, I'll try several times and none of my tries are accepted. I know I have a trace of a German accent in all my languages and certain sound or word combinations just never make it.
Rocket Language has a somewhat improved method: The program gives you a percentage rating of your recording and transcribes the sentence while highlighting the mistakes, as can be seen on this screenshot. Still, it still took me several tries until my purposely wrong pronunciation of “Je saute” was flagged.

(However, you can also replay your own recording - with the arrow beside the microphone – to compare yourself against the native speaker – the upper arrow, which, we believe and discuss below, is a more effective way.)

For both of us, speech recognition has always proved very frustrating. But there are other issues.

Can a Beginner sound like a Native Speaker?

In fact, during the early phases of language learning, it's nearly impossible to sound "like a native speaker". And, having a perfect pronunciation – while obviously a desirable goal for most - is certainly not essential for the beginner.

Focusing initially more on listening and hearing the melody of the new language will pay greater dividends later on, when you work on your speaking skills. Having a beginning learner worry about correct pronunciation is a little bit like getting a new student driver on a highway with fast moving traffic. He or she could get easily scared and discouraged.

But even for the advanced learner, learning with speech recognition may not always be useful. Just having my spoken translation accepted as “correct” may feel like a pat on the back, but can I really trust it? What was correct? What do I need to improve?

Still, if the idea of speech recognition prompts a learner to speak aloud and imitate the native speaker - that's good! Indeed, anything that gets you to start talking in your new language is a good thing.

The Mondly app has an interesting approach: it uses augmented reality create a language learning environment. Animals and objects come to life and you're encouraged to participate in a conversation.

On the whole, though, we've always found that there is a better way for improving your pronunciation with language apps or online language programs:

Record Your Voice and Compare it to that of the Native Speaker

When you are learning a new language, a teacher or tutor will at times point out your mistakes and correct your pronunciation. He or she will often not only encourage you to imitate their pronunciation, but also explain to you the mouth mechanics that the particular language requires.

Online language programs can also give you such advice or suggestions, but then it's mostly up to you to figure out how to produce the new sounds of your language. For most languages there are also Youtube videos that explain the mouth mechanic specifics that you can then practice on your own.

And it's here that recording your own voice and comparing it to the native speaker does three important things for you:
1. It lets you become aware of the pronunciation differences between you and the native speaker.
2. It lets you try as many times as you want to get closer to the native speaker.
3. It lets you yourself become aware of the progress you are making.

Now, I also know that for many beginners hearing themselves in a foreign language can be frustrating and even discouraging. You may ask yourself: Will I ever be able to speak like the native speaker? The honest answer for me and many other adults is: Probably not.

Unless you heard the new language as a child, chances are that you may not HEAR certain sounds any longer as an adult. You'll therefore also have trouble reproducing them.
(This is due to our “categorical perception”, which we discussed in an earlier post: Beyond “Learning a Language Like a child”)

However, hearing yourself and imitating the native speaker both in pronunciation as well as in language melody, is an excellent way to practice and improve your pronunciation.
And, as a precursor to speaking, listening to as much of your new language as you can, is the obvious thing to do.

uTalk

We recently re-discovered the app uTalk, (available for Android, iPhone/iPad, Windows 10. Mac and Kindle Fire HD) with 140 languages. It has an excellent self-recording/native speaker comparison system and two of its six units for each topic lets you listen to phrases and then record yourself saying them.

It's not free, but we recently picked up a life-time subscription for 6 languages for $24.99. So look around for a deal, if you are interested. We are planning a review later on.

The Goal of Language Learning

For some language learners passive activities like listening and reading are enough. But for most others, communicating with friends, family, business associates or during travel adventures is the real goal of language learning. That means speaking practice is essential.

If speak recognition features of an app or online program encourage you to do that, great! We find self-recording/native speaker comparison systems more effective than speech recognition.

What is  your experience? What works best for you? Please share with us your thoughts below.

PS: Previous users of our online GamesforLanguage courses will have noticed that we turned off the Flash Player based recording several months ago. Flash Player recordings were not supported on most phones and mobile devices and had other problems. We are still looking for a replacement.

Bio: Ulrike & Peter Rettig are co-founders of Gamesforlanguage.com. They are lifelong language learners, growing up in several European countries before moving to Canada and the United States. You can follow them on Facebook, Twitter and Instagram, and leave any comments with contact.