Filter results by

Google Assistant

The Google Assistant cloud service provides a means of recognizing spoken phrases and returning both text and spoken responses. Here we use Python to interact with Google Assistant, as well as two Web page consoles to set everything up.

Google Actions
console
Google API
console

The procedure in this article has been verified using an ARTIK 530s module.

The Google Assistant Library is not supported for ARTIK 710 (64-bit) architecture at this time.

Prerequisites

This procedure presumes you have an account with Google. Log in on your development PC before proceeding.

You'll need to set some activity controls for your account to allow the operations in this article. Open the Activity Controls page of your Google Account and enable each of the options below.

  • Web & App Activity – also selecting the 
    Include Chrome browsing history and activity from websites and apps that use Google services
    checkbox.

  • Device Information

  • Voice & Audio Activity

Preparation

Test audio subsystem

Start by verifying that recording and playback work on the ARTIK board.

  1. Plug in a powered speaker or earphones to your ARTIK board and play a test sound. From your terminal window, run
    speaker-test -t wav

    Press Ctrl + C when done.

  2. Adjust the volume to your liking – try using alsamixer. Press the up or down arrow key to set the volume. Press ESC when done.

  3. Record a short audio clip, and then play it back.

    arecord --format=S16_LE --duration=5 --rate=16000 --file-type=raw out.raw

    aplay --format=S16_LE --rate=16000 out.raw

Prepare for file transfers

You'll need to copy files from your development PC to your ARTIK board.

First, you'll find it convenient to know your ARTIK board IP address for transferring files.

hostname -I

To actually copy files, you'll use scp. For Mac users, or Windows users with Git and MinGW32/64 installed, you can simply run the scp command from your terminal window. For example, to copy a downloaded credentials file:

scp credentials.json root@ARTIK-IP-ADDRESS:/home

where you replace ARTIK-IP-ADDRESS with the IP address of your ARTIK board, and root with your login name if different. Windows users can instead use an scp client like WinSCP or FileZilla.

Setting Up New Project

  1. Open the Actions Console: https://console.actions.google.com/u/1/

  2. Click on Add/import project.

  3. Create a new project by entering a Project name like the one shown below, and clicking CREATE PROJECT

    You will see the page shown. Keep the browser window open to this page.

Enabling Google Assistant API

Open a new browser window to the Google API Console to enable the API for your project.

  1. From the pull-down on the top bar, select the project you just created.
    Then click ENABLE APIS AND SERVICES.

  2. Search for "assistant" and select the one that comes up.

  3. Enable the Google Assistant API for the project.

  4. Go to Credentials and select the "OAuth consent screen" tab.

  5. Make sure at least your e-mail address is selected; update other information as you choose.

Registering Device Model

You'll be registering a device model – that is, giving basic name and manufacturer information – to Google Actions so that it can associate a product-unique "credentials" file. For the purposes of this demonstration, you can use any information you choose.

  1. Go to your Actions browser page.

  2. Scroll to the bottom, and you should see a “Device registration” button under “More options”. Click this button.

  3. Click the REGISTER MODEL button.

  4. Create Model. Fill out Product Name, Manufacturer name and select Device type, then click REGISTER MODEL.

  5. Download credentials file.

  6. Specify traits. Enable all 7 traits and SAVE TRAITS.

  7. Copy your credentials file over to your ARTIK /home directory.

    scp client_secret....json root@ARTIK-IP-ADDRESS:/home

Installing SDK and Sample Code

Install Google Assistant tools on ARTIK board

Make sure you have followed all Ubuntu update procedures before continuing.

  1. Install a Python virtual environment to isolate the SDK and its dependencies from the system Python.

    apt install python-dev python-virtualenv virtualenv
    virtualenv env --no-site-packages
    env/bin/python -m pip install --upgrade pip setuptools wheel

  2. Enter the virtual environment.

    source env/bin/activate

    Your prompt should look something like this:

    (env) [root@artik home]#

    Whenever you need to, use deactivate to exit the virtual environment.

  3. Install package dependencies.

    apt install portaudio19-dev libffi-dev libssl-dev

  4. Install the latest version of the Python Google Assistant package.

    python -m pip install --upgrade google-assistant-library
    python -m pip install --upgrade google-assistant-sdk[samples]

  5. Install the Google credentials tool.
    python -m pip install --upgrade google-auth-oauthlib[tool]

Generate ARTIK board credentials

Run the credentials tool to obtain credentials for your ARTIK board.

google-oauthlib-tool --scope https://www.googleapis.com/auth/assistant-sdk-prototype --save --headless --client-secrets /home/client_secret[Tab]
  (hit Tab key to auto-complete)

You should get this response.

Please visit this URL to authorize this application: https://accounts.google.com/o/oauth2/auth?response_type=code&client_id=..

Open the displayed URL in a browser, then copy and paste the authorization code to your terminal.

Enter the authorization code: 4/XXXX

If authorization was successful, you will see a response like this:

credentials saved: /root/.config/google-oauthlib-tool/credentials.json

Getting an OAuth error message? Make sure you followed the last step here.

Running Applications

If not already there, don't forget to start by entering the Python virtual environment:

source env/bin/activate

Run the sample app

You can run a sample Google Assistant application to demonstrate how it interacts with your board. Use the command line

googlesamples-assistant-pushtotalk --project-id PROJECT_ID --device-model-id MODEL_ID
where you'll replace PROJECT_ID and MODEL_ID with your own.

Go to the
Google Actions console
and click » » » »
PROJECT ID

MODEL_ID

Run the sample application, substituting your project ID and model ID in the command line, and watch the results. For example:

googlesamples-assistant-pushtotalk --project-id voiceenablementonartik-83975 --device-model-id voiceenablementonartik-83975-voicegateway-rtxxqm

INFO:root:Connecting to embeddedassistant.googleapis.com
Press Enter to send a new request...

Press Enter when you're ready to start. Maybe you're going to be flying through Chicago. Ask "What's the weather in Chicago?" and then hit Enter again when finished.

INFO:root:Recording audio request.
INFO:root:Transcript of user request: "what".
INFO:root:Transcript of user request: "what's".
INFO:root:Transcript of user request: "what's the".
INFO:root:Transcript of user request: "what's the weather".
INFO:root:Transcript of user request: "what's  the weather".
INFO:root:Transcript of user request: "what's the  weather".
INFO:root:Transcript of user request: "what's the  weather in".
INFO:root:Transcript of user request: "what's the weather  in".
INFO:root:Transcript of user request: "what's the weather  in Key".
INFO:root:Transcript of user request: "what's the weather  in Chico".
INFO:root:Transcript of user request: "what's the weather  in Chicago".
INFO:root:Transcript of user request: "what's the weather in  Chicago".
INFO:root:Transcript of user request: "what's the weather in Chicago".
INFO:root:End of audio request detected.
INFO:root:Stopping recording.
INFO:root:Transcript of user request: "what's the weather in Chicago".
INFO:root:Playing assistant response.
INFO:root:Finished playing assistant response.

Getting an error message? Make sure you followed the prerequisites.

Customizing the responses

Let's enable Google Assistant to recognize and respond to queries specific to ARTIK. We'll provide a JSON file with the action responses to get through some details.

  1. Make sure you're in the virtual Python environment.

    (env)$

  2. Download the file bundle that contains the JSON file and a Google Actions CLI program.

  3. Use scp to copy the files to your ARTIK board /home directory. You can either extract the two files while on your PC before copying them over, or can copy the .zip file and install/use unzip on your ARTIK board to extract the files.

  4. Make gactions executable.

    chmod 777 gactions

  5. On the ARTIK board, remove any existing credentials from the same directory as the gactions tool.

    rm creds.data

  6. Save your Action Package to Google by using the gactions CLI. Replace project_id with your Actions Console project ID.

    ./gactions update --action_package actions.json --project project_id

    The first time you run this command you will be given a URL. Use the same procedure to enter the authorization code as your did in the credentials section.

    If authorization was successful, you will see a response similar to the following:

    Your app for the Assistant for project my-devices-project was successfully updated with your actions.
  7. Deploy your action package into test mode by using the gactions CLI. You must have saved your Action Package to Google at least once before running this command. Test mode enables the action package on your user account only. For example:

    ./gactions test --action_package actions.json --project voiceenablementonartik-f03d2

     Pushing the app for the Assistant for testing...
     Your app for the Assistant for project voiceenablementonartik-f03d2 is now ready for testing on Actions on Google enabled devices or the Actions Web Simulator at https://console.actions.google.com/project/voiceenablementonartik-f03d2/simulator/
     

    Now, run the sample app again with questions like “What is Samsung ARTIK?”

Developing your own applications

To start developing your own applications, you'll need to install the Google Assistant SDK on your development PC. Then you can copy over your created JSON file and the gactions tool.

Start by cloning the Google Assistant SDK to your development PC.

git clone https://github.com/googlesamples/assistant-sdk-python

Now, when you create new actions, you can copy them to your ARTIK board.

  1. From the cloned files on your PC, scp the downloaded actions.js and gactions command line tool to the ARTIK /home directory, replacing ARTIK-IP-ADDRESS with your own.

    scp actions.js gactions root@ARTIK-IP-ADDRESS:~/assistant-sdk-python/google-assistant-sdk/actions.json

  2. On the ARTIK board, remove any existing credentials from the same directory as the gactions tool.

    rm creds.data

  3. Make gactions executable.

    chmod 777 gactions

  4. Save your Action Package to Google by using the gactions CLI, and then push the app for testing.

    ./gactions update --action_package actions.json --project project_id

    ./gactions test --action_package actions.json --project project_id

You can now run the sample app to test.

Extra Credit

You enabled seven traits on the Google Assistant device model created. As the next step, try using these traits to control peripheral devices.

Last updated on: