Cloud-based voice recognition assistants have grown massively over the past few years with Apples Siri, Amazons Alexa, Microsoft’s Cortana and the open source Mycroft pushing the expansion. Voice Assistants, however, do not seem to have easy access towards the Linux desktop but are possible using software development kits released by the producers.
In this tutorial, we will show you how to set up Google Assistant on ubuntu 18.04 and control it with a Gnome shell extension.
What you will need;
- A PC with Ubuntu 18.04 installed and Gnome Shell Extensions Installed (Details Below)
- A Google Account
- An Android Phone to allow personal responses
- A working microphone.
- A working speaker system.
Many people, me included have struggled with getting the Gnome shell extension to work. This is down to conflicting advice from both Google and the developer. Not intentionally but it’s all down to where the python modules are placed. Google recommends a virtual environment (Sensible) but the Extension can’t seem to interact with it. Google also give information about installing for python 2 & 3, which can get a bit confusing but anyway:
The complete steps to install:
Configure a developer account:
Go to https://console.actions.google.com/, and log in to your google account.
Click on Add/Import Project
Enter a name for your project. i.e. linuxassistant, gsassistant etc
Choose your country from the list underneath.
On the next page click skip
click on the gear icon and select project settings.
make a note of your project ID, this is needed later for testing.
open a new browser window and go to https://console.developers.google.com/apis/api/embeddedassistant.googleapis.com/overview
Search for google assistant API select and click Enable
We now need to create the Oauth credentials page. Click on the configure consent screen button on the left-hand side.
Now enter your email address and a product name (optional) and click save.
Now open a new window or tab go to https://myaccount.google.com/activitycontrols , and toggle all switches to on.
Switch Back to the actions console https://console.actions.google.com/ and click on Device Registration.
Click on Register Model and enter a product name and manufacturer name, Note down the Device Model Id as this is needed for testing. Clicking Register Model will enable you to download a clients_secrets.json file to register your PC with your account.
If you ever need to download the client_secret.json again simply go back to Register Device, select the entry and click on the 3 dots menu to give an option to download the Oauth2 file.
Install the Google Assistant SDK
open a terminal and install the dependencies:
sudo apt-get install python3-dev python3-pip
sudo apt-get install portaudio19-dev libffi-dev libssl-dev libmpg123-dev python3-openssl
Now we need to get the Google SDK python libraries:
pip3 install --user google-assistant-library
pip3 install --user google-assistant-sdk[samples]
pip3 install --user google-auth-oauthlib[tool]
Authorise our Google Assistant
./.local/bin/google-oauthlib-tool --scope https://www.googleapis.com/auth/assistant-sdk-prototype \ --scope https://www.googleapis.com/auth/gcm \ --save --headless --client-secrets /path/to/client_secret_client-id.json
Make sure you give the full path to the client_secret.json (Quick tip, select the file in a file manager, right click to select rename and copy and paste then add .json to the end, this helps against typos!)
Right click on the link produced in the terminal and a new browser window will open, log in to your google account and copy and paste the authorisation code back into the terminal.
Testing the Assistant
Run the following code with the Project ID and Device ID saved from setting up a Developer account:
./.local/bin/googlesamples-assistant-hotword --project_id my-dev-project --device_model_id my-model
when loaded say “ok google” and ask a question, if Google recognises the speech it will give an appropriate response.
You may need to turn up the microphone and speaker volumes using the sliders in the Gnome Shell user menu. If speech responses from Google are still quiet, say “OK Google set volume to 10” to boost the assistant volume level.
Close the terminal window (or ctrl+c) to end the assistant when done.
Installing the Gnome Shell Extension
Please see our quick tip about enabling gnome shell extensions, if not already enabled on your system.
Go to https://extensions.gnome.org/extension/1441/gassistant/ in your browser and toggle the on/off slider to enable the extension. Click on Install and wait for it to load onto the Gnome Shell panel. Right click and select ‘Restart Google Assistant’. If the ZZZ icon is showing it means the extension has found the Assistant installation and is waiting to be triggered with the hot word (OK Google).
The gnome extension will give the option to enable, restart or turn off the service (for those paranoid that Google is always listening) it will also give an audible beep when google assistant is invoked to show it is ready for voice input.
Personalise with Android
You can control some features of the Assistant through an android phone (connected to the same account)
go to settings>google> search> Settings the device should now show up in the Devices list, enabling the options gives it access to personal accounts. It’s also possible to give it a location for things like weather, travel updates and to set up news providers.
Expanding your Assistant
Google assistant SDK gives access to nearly all the features of Google assistant on Android, but most notably missing is access to music services. It is possible by setting up local servers solutions exposed to the internet and constructing actions to expand on the skills available through the SDK, but it’s not a simple method to achieve. It is possible to control a Linux PC using skills such as the TriggerCMD service.
Do you use voice assistants? Do you have some of them controlling some amazing devices or functions if so we would love to hear about them in the comments?