#mycroft — Public Fediverse posts

je me relance dans mes investigations Speech-to-text et Text-to-speech. Bizarrement c'est quelque chose qui revient régulièrement. Est ce que des gens savent ce qu'est devenu #DeepSpeech et si #MozillaVoice est toujours maintenu?
De mon côté je me base sur #mycroft mais j'ai le sentiment que le projet est un peu à l'arrêt mais je me trompe peut-être.

It's good that other people are also bringing up the elephant in the room: why do you need to pay money for one more electronic gadget that listens to you 24/7, when voice assistants aren't supposed to be rocket science in 2023 anymore? https://news.ycombinator.com/item?id=35857631

I wrote two articles on how to build custom #VoiceAssistants using just a Raspberry Pi and a microphone, one in 2019 https://blog.platypush.tech/article/Build-your-customizable-voice-assistant-with-Platypush and one in 2020 https://blog.platypush.tech/article/Build-custom-voice-assistants.
It's definitely doable and I still have my own custom assistants in the house. However, I had to get around with a #Snowboy model for hotword detection (and Snowboy is now basically abandoned), Mozilla #DeepSpeech model for speech-to-text (and that's quite heavy), and #Mycroft's mimic3 text-to-speech model (and Mycroft is now basically bankrupt). Then writing the integration is relatively easy - I used #Platypush, but it can definitely be done with Home Assistant and OpenHAB too.

Compared to 3-4 years ago, I think we're now in a state where the content is no longer the issue (just plug into a LLM, and all of your text requests will get an answer), nor integrations are a problem (just write a Platypush event hook on speech detected, and you can connect it to everything, no need for "Works with Google/Alexa" labels). Text-to-speech synthesis has also become cheap and ubiquitous.

But the hotword detection and speech-to-text models are still IMHO the bottleneck. Hotword detection is a field where you need a very small and lightweight model that only detects a specific word or phrase in a very reliable way. Snowboy was an amazing FOSS project - which also came with this cool idea of "crowd-funded models", where in order to download a model for a certain hotword you were first supposed to provide three audio tracks where you say that word in order to improve the model. But it's now discontinued because it cost the volunteers too much to run the infra.

And Mozilla DeepSpeech is a relatively good choice for general-purpose speech-to-text, but it's heavy (it takes 100% of the CPU when it runs on a Raspberry Pi) and it's mostly optimized for English - even support for other Western languages is patchy. OpenAI's recent Whisper model seems like a solid alternative, but it's also plagued by the 100% CPU issue - also, I no longer trust anything that comes from OpenAI, no matter how noble some of their efforts may look.

If there are other open-source alternatives that solve these problems, I'd be very happy to learn about them. Once these blockers are removed, there should be really no reason for anyone to feed their audio streams to Google or Amazon.

In the meantime, I'm planning to spend some time playing with some self-hosted LLM model to see if I can replace the Google Assistant library on the last Raspberry Pi that runs it in my home.

I'm sad about the Mycroft announcement, but on the other hand, it's already essentially obsolete.

With advances like the Whisper voice recognition system, the Deepspeech TTS project, and now the OpenAssistant project using language models, we're on the cusp of a much more fluid, complete assistant that is privacy respecting, realistic, and much more powerful.

#DeepSpeech #OpenAssistant #Mycroft #Whisper #AI #FLOSS

#AI #SpeechDetection

#OpenAI has just released #Whisper https://github.com/openai/whisper, a new open-source model for speech detection.

While after a couple of tries I'm impressed by its accuracy (you need to use the small model or a larger one if you want enough precision though), I'm also still unimpressed by its resource usage and performance.

The small model took ~30 seconds to process an audio file with 2 seconds of speech on my 6-year-old laptop with an i7 CPU, and in the meantime it used up more than 4 GB of RAM.

Mozilla's #DeepSpeech model was also heavy when I last used it ~1 year ago, but not THIS slow (although it was also slightly less accurate).

For now I definitely see the use-case for using OpenAI's new model for offline transcriptions, but they are still very far from being used for real-time applications such as voice assistants.

I'm still looking for a good open-source model that can be run on a RaspberryPi as a stable voice assistant. Ideally, it needs a small and simple model that can be used for hotword detection (I used to use Snowboy, but that project is now dead), and a more complex model to be used once the hotword is detected in order to transcribe the speech. And the audio transcription needs to be done within max 5 seconds in order to be compatible with the real-time expectations from a voice assistant.

Ideally, it needs to only include the model, not a lot of bloat around it that makes it harder to embed it - so #Mycroft is excluded.

So far, I haven't found any such model. My RPi still run the Google Assistant's push-to-talk script that I adapted into Platypush years ago, and a Snowboy hotword detection model that I managed to train before the project was shut down. If anybody knows of better solutions that could cut this last dependency on Google, I'd be happy to try them out.

Awesome: #Mycroft will default to #Mozilla's #DeepSpeech speech-to-text engine starting March 31st. The option for a user to change to a less desirable SST engine will be maintained.

https://mycroft.ai/blog/mycroft-speech-to-text-and-balance/