Voice control Linux using Alexa or Google Assistant.

The growth of voice controlled devices know no bounds and extendable with skills and services such as IFTTT (If This Then That). Many unique and ingenious ways of voice controlling daily life are emerging from makers all over, mainly from projects with raspberry pi and voice assistant SDKs.

So why not extend this function onto a normal desktop? Voice Controlling actions have many advantages and possibilities to expand applications, from simply running a program or script to helping with accessibility barriers.

Triggercmd

Triggercmd is a service that runs on your machine and connects with your selected voice assistant via a skill. When triggercmd receives a valid command it will pass it on to your desktop and run a bash command of your choosing. TriggerCMD is available as both a free and paid service. The free service has limitations of only using 1 trigger per minute and only authorizing one machine. The paid service has no limitation on triggers and multiple machines can be registered.

How it works

  1. Voice command is given to assistant (i.e Alexa, Google Assistant)
  2. Command recognized and transcribed by the assistant
  3. Passed on to the Triggercmd Skill.
  4. Triggercmd sends the request to your specified and authorized computer(s)
  5. TriggerCMD then executes the corresponding bash command.

Want to try?

You will need a device that is either Alexa or Google Assistant enabled such as:

Alexa

  • An echo device like a Dot or Echo Show.
  • An android based device like a Kindle Fire
  • Access to Alexa Home app via an Android smartphone.
  • A device such as a raspberry pi running the Alexa SDK

If you don’t have one, access is still possible via the echosim.io site used for testing Alexa skills which simulates an echo device in a web browser.

Google Assistant

  • An android (5+) device, with access to the assistant.
  • A google home device such as a google mini.
  • If you want to add google assistant to the desktop you are controlling check out the post on integrating Google Assistant into Gnome Shell. This can also be run from the terminal on any Linux machine.

Create an account with TriggerCMD

First go to TriggerCMD and either log in via Facebook, Twitter or Google Authentication, alternatively sign up for a TriggerCMD account. Once verified download the client from here, Available as a .rpm and .deb package. It is also available with no graphical interface suitable for use on a raspberry pi. The GUI version, however, makes it easy to add and edit triggers from the desktop, whilst running in the system tray.

Install the required package by double-clicking in the file manager and installing with your default package manager.

Set the program to autostart

Screenshot from 2019-02-25 10-28-00

Open the startup applications app from the application menu and click add. Enter Triggercmd for the name and triggercmdagent in the command box. Click Add to enter it into the list of startup applications.

Authorize triggerecmd

Screenshot from 2019-02-25 10-56-49

Open triggercmd from the applications menu, and you will be asked to enter your authorization token. This is available by logging into your account created on Triggercmd on the instructions option in the side menu. Copy and paste the token to create the link between the computer and triggercmd.

Set up a command to try

Screenshot from 2019-02-25 11-12-19

Click on the triggercmd icon in the status area and select GUI Command editor. This will open a window where you can add and edit commands to run. Click the add command option to set up a new command to run. For example, opening a browser window. Set the following options

Trigger: Internet (The name of the command to run)

Command: Enter either firefox, google-chrome-stable, chromium-browser or opera depending on your browser. If you use another enter the relevant command you would use to launch it from the command line.

Ground: Select foreground. Trigger CMD can be run as a daemon in the background with an option to install the service from the system tray icon. However, running the task in the foreground gives easier access to editing commands and stopping and starting the service through a graphical interface.

Voice: internet ( the trigger word you will use to launch the command)

Voice Reply (Optional): What you want triggercmd to respond with when the command is executed.

Allow Parameters: Set to false. Setting this option to true gives you the option to pass additional variables to TriggerCMD within a single invocation.

Click Save the new command will be added to the list and update immediately with TriggerCMD. Hovering the mouse over the command in the GUI Editor will give the option to run it with the play button. The command can also be tested by running it through the triggercmd website or the app available through the Amazon and Google app stores

Link to a voice assistant.

With triggercmd set up and functioning correctly. Now it’s time to link to a voice assistant to trigger the command.

Alexa

Amazon’s Alexa has two skills, either of which can be used, one called ‘TriggerCMD‘ where the user has to say “Alexa ask Trigger C M D to run” and another called “Trigger Command” where the user has to say “Alexa ask Trigger Command to run”. Both skills perform the same function, however, it is upto the user to decide which would be easier to say. Simply select the skill and click enable, next link to your account which will redirect you to the triggercmd website to login.

Give it a try with your Alexa enabled device

“Alexa ask trigger c m d to run internet” or “Alexa ask trigger command to run internet” (one registered device)

“Alexa ask trigger c m d to run internet on (device name)” or “Alexa ask trigger command to run internet on (device name)” for multiple registered devices.

Alternatively, you can say “Alexa open trigger c m d / command” wait for the response and reply “run internet” or “run internet on (device name)”

Providing everything has been set up correctly you should be greeted by a new browser window.

Google Assistant

Simply visit here to say “trigger C M D” and log in with your Google account. Alternatively, enable the “trigger command” skill to say “trigger command” to open the skill. Select the link button and you will be redirected to the triggercmd website to link your accounts. Log into triggercmd and you should be good to go. Try it with

“Ok/hey Google ask trigger c m d to run internet” or “Ok /hey Google ask trigger c m d to run internet on (device name)”

Or

“Ok/hey Google ask trigger command to run internet” or “Ok /hey Google ask trigger command to run internet on (device name)”

Like before you can also say “Ok/hey Google open trigger c m d/command” wait for the response and reply “run internet” or “run internet on (device name)”

As before you should be greeted with a new browser window.

Think about security

Although this is a very cool and hugely expandable tool, both Alexa/Google when registered can control your pc with the commands allowed from any device registered to you from any location. That’s why it’s always a good idea to consider the security of what commands you are allowing to be run as well as who has access to the voice services for example through an echo speaker anyone can use it, whereas through an android phone a single user is more likely.

Expand with some more commands.

If it can be run via a bash terminal then it can be triggered with a voice command, this can be emptying the recycle bin, launching an application(s) as shown in the example or controlling windows and issuing key combination. It’s also possible to power off and reboot the PC. Turning on is possible but will require another device (PC, Raspberry Pi or Android device for example) to send a Wake on Lan Packet.

Applications

Here’s a list of simple commands to add to open applications, these are only some recommendations, remember any application can be opened just by setting the command option to the command you would use to open the application from the terminal.

TriggerCommandGroundVoiceVoice ReplyAllow Parameters
Calculatorgnome-calculatorforegroundcalculator(Optional)false
File managernautilusforegroundfile manager(Optional)false
nemoforeground(Optional)false
cajaforeground(Optional)false
Text Editorxedforegroundtext editor(Optional)false
geditforeground(Optional)false
Steamsteamforegroundgaming(Optional)false
GIMPgimp-2.8foregroundimage editor(Optional)false
Twittercorebirdforegroundtwitter(Optional)false
Emailthunderbirdforegroundemail(Optional)false
Rsslifereaforegroundnews feeds(Optional)false
Messagespidginforegroundmessenger(Optional)false
Skype/usr/bin/skypeforlinuxforegroundskype(Optional)false
Spreadsheetlibreoffice –calcforegroundspreadsheet(Optional)false
Writerlibreoffice –writerforegroundwriter(Optional)false
Presentationlibreoffice –impressforegroundpresentation(Optional)false
VSCode/usr/share/code/code –unity-launch %Fforegroundcode(Optional)false
Webcamcheeseforegroundwebcam(Optional)false
Kodikodiforegroundkodi(Optional)false
Terminalgnome-terminalforegroundterminal(Optional)false
Musicrhythmboxforegroundmusic(Optional)false

Controlling the window manager and simulating key inputs.

Any X window system is controllable from bash using the xdotool, xautomation and wmctrl programs. These applications simulate key and mouse input and can select and interact with windows. It can be installed through your package manager or by opening a terminal and typing:

sudo apt-get install xdotool xautomation wmctrl

Test it out by opening a terminal window and entering:

xdotool windowkill `xdotool getactivewindow`

This should close the active terminal window. We can now enter commands into triggerCMD to control the X window system. Below is a list of commands to try:

TriggerCommandGroundVoiceVoice ReplyAllow Parameters
Close Windowxdotool windowkill `xdotool getactivewindow`foregroundclose window(Optional)false
Lock Screenxdotool key Ctrl+alt+llock screen
App Menuxdotool key superapps
Switch Windowsxte “keydown Alt_L” “key Tab” “sleep 1” “keyup Alt_L”switch window
Logoutgnome-session-quit logout

With the Logout command, we can also append –no-prompt to prevent any logout dialog and timer.

While xdotool can access windows and issue simple key combinations, xautomation can automate longer tasks for example in the Switch windows command it effectively equals pressing alt-tab on the keyboard to switch windows. Holding the alt key down, tapping the tab key to activate the window switcher and giving it time to activate before pressing tab again to switch to the next window. Finally releasing the alt key to place the focus on the next window.

To run the above commands like before we simply say:

“(Alexa/(OK/Hey) google) ask (triggercmd/trigger command) to run (command name) (on device) eg.

“OK google ask trigger command to run logout on desktop.”

Switch Workspaces

We can use the xdotool to switch between workspaces by issuing commands like

xdotool set_desktop 1
xdotool set_desktop 0

to switch to desktop 2 and then back to desktop 1. Using the following command we can send the current active window to desktop 2 for example

xdotool getactivewindow set_desktop_for_window 1

Below is a command table giving options for working with 2 workspaces.

TriggerCommandGroundVoiceVoice ReplyAllow Parameters
workspace 1xdotool set_desktop 0foregroundworkspace left(Optional)false
workspace 2xdotool set_desktop 1foregroundworkspace right(Optional)false
send workspace 1xdotool getactivewindow set_desktop_for_window 0foregroundsend workspace left(Optional)false
send workspace 2xdotool getactivewindow set_desktop_for_window 1foregroundsend workspace right(Optional)false

Shutting down the PC

We can also control power functions for the computer by using the systemctl function for example:

TriggerCommandGroundVoiceVoice ReplyAllow Parameters
shutdownsystemctl poweroffforegroundshutdown(Optional)false
rebootsystemctl rebootforegroundreboot(Optionalfalse
Boot Biossystemctl reboot –firmware-setupforegroundboot bios(Optional)false

Empty the recycle bin

Automate the task of emptying your recycle bin using the trash-cli tool. Simply install by opening a terminal and typing:

sudo apt-get install trash-cli

We can then add it to triggercmd through the GUI Editor, with the following parameters

TriggerCommandGroundVoiceVoice ReplyAllow Parameters
Empty Bintrash-emptyforegroundempty bin(Optional)false

To run as before simply say OK/Hey Google/Alexa ask trigger command to run empty bin on (device)

Take a screenshot

We can take screenshots by using the scrot application. install with

sudo apt-get install scrot

Screenshots can be taken by running the command:

scrot -b -d 5 '%Y:%m:%d:%H:%M:%S.jpg' -e 'mv $f ~/Pictures/'

This will initiate scrot with a 5-second delay and label the file with the date and time before moving the file to the Pictures folder.

Install a pre-made command list

A premade command list tested for use with Cinnamon and Gnome Shell can be downloaded from here. It contains the following commands, which can be changed to suit your needs.

Hardware Control: Bluetooth on/off, Wifi On/Off/Reconnect Brightness Up/Down, Play/Pause/Next/Previous Mute & Unmute, Screen On/Off, Suspend/Hibernate/Reboot/Reboot to Bios and Power Off.

System Actions: Empty Recycle Bin, Check Updates, Install Updates, lock screen, take a screenshot.

Window Controls: Open app menu, Maximise/Restore/Minimise/Close the current window, Switch current window, Send current window to workspace 1/2, Switch to workspace 1/2, Enter/Exit fullscreen, Show/Restore Desktop.

Open Applications: Calculator, Character Map, Chrome/Chromium, Stacer, VScode, Disks, Gedit, Xed, Thunderbird (and new email window), Lifrea, file manager, Firefox, Font Viewer, Steam, gimp, google-play-music desktop player, Kodi, pidgin, gnome-system-monitor, opera, synaptic, Libre Office (Writer,Calc or Impress), Retroarch, Skype, Software Center, Terminal, Corebird, Virtualbox, Cheese.

To Install copy the code to /home/user/.TRIGGERcmdData/commands.json.backup and then /home/user/.TRIGGERcmdData/commands.json. The triggercmd agent should then automatically update on change.

Expanding Even further

TriggerCMD is expandable even further, You can experiment with using voice variables to pass on to commands, Set routines through Google to trigger multiple smart home tasks such as turning lights off when you turn the computer off or even shortening the phrase to say from “ok google ask trigger command to run command on device” to “ok google open command”. Accessibility through the command line means you can use it to run any program or script, triggering commands is even possible remotely by using curl.

IFTTT

IFTTT (If THIS then THAT) can be integrated into triggerCMD using webhooks see this post. Linking actions from IFTTT when events occur to a webhook to trigger an action on your PC. A simple example of this is when someone activates a connected smart home device like a light switch, using webhooks to fire a triggercmd command to run libnotify and display a notification that the device has turned on.

Do you automate your Linux system by voice? Let us know your experiences in the comments below.

 

 

 

 

%d bloggers like this: