Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.
  2. We have updated the language to the Editor Terms based on feedback from our employees and community. Learn more.
    Dismiss Notice

Speech semantics example?

Discussion in 'Documentation' started by mikewarren, Feb 4, 2019.

  1. mikewarren

    mikewarren

    Joined:
    Apr 21, 2014
    Posts:
    109
    Anyone have a sample grammar that includes semantics and works in Unity (2018.3)?
    I haven't been able to cobble together a working example.

    Mike
     
  2. mikewarren

    mikewarren

    Joined:
    Apr 21, 2014
    Posts:
    109
    I'm stumped. Here's my grammar (example from web) and script. I don't get any recognition events unless the semantic tags are removed. Status of the recognizer is "running" either way.

    Could use a good reference for creating semantic tags as well.


    <?xml version="1.0" encoding="UTF-8" ?>
    <grammar xmlns="http://www.w3.org/2001/06/grammar"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://www.w3.org/2001/06/grammar
    http://www.w3.org/TR/speech-grammar/grammar.xsd"
    xml:lang="en"
    version="1.0" mode="voice" root="main">

    <rule id="main">
    <one-of>
    <item><ruleref uri="#rule1" /></item>
    </one-of>
    </rule>

    <rule id="rule1" scope="public">
    <one-of>
    <item>1<tag>out.start="1";</tag></item>
    <item>2<tag>out.start="2";</tag></item>
    <item>3<tag>out.start="3";</tag></item>
    </one-of>
    </rule>

    </grammar>


    Code (CSharp):
    1. using System.Collections;
    2. using System.Collections.Generic;
    3. using System.Linq;
    4. using System.IO;
    5.  
    6. using UnityEngine;
    7. using UnityEngine.Windows.Speech;
    8.  
    9.  
    10. public class SpeechGrammarTest : MonoBehaviour
    11. {
    12.     public string grammarPath;
    13.     [Header("Read Only")]
    14.     public string dateTime;
    15.     public string fullPath;
    16.     public bool running;
    17.     public string utterance;
    18.     public List<string> semantics;
    19.  
    20.     GrammarRecognizer recognizer;
    21.  
    22.     // Start is called before the first frame update
    23.     void Start()
    24.     {
    25.         fullPath = Path.Combine(Application.streamingAssetsPath, grammarPath);
    26.         recognizer = new GrammarRecognizer(fullPath, ConfidenceLevel.Low);
    27.         recognizer.OnPhraseRecognized += PhraseRecognized;
    28.         recognizer.Start();
    29.         running = recognizer.IsRunning;
    30.     }
    31.  
    32.     public bool hasSemantics = false;
    33.  
    34.     private void PhraseRecognized(PhraseRecognizedEventArgs args)
    35.     {
    36.         utterance = args.text;
    37.  
    38.         hasSemantics = args.semanticMeanings != null;
    39.         semantics.Clear();
    40.         if (args.semanticMeanings != null)
    41.         {
    42.             foreach (SemanticMeaning sm in args.semanticMeanings)
    43.             {
    44.                 string newSm = sm.key;
    45.                 foreach (string val in sm.values)
    46.                     newSm = newSm + " " + val;
    47.                 semantics.Add(newSm);
    48.             }
    49.         }
    50.     }
    51.  
    52.     // Update is called once per frame
    53.     void Update()
    54.     {
    55.         dateTime = System.DateTime.Now.ToString();
    56.     }
    57. }
    58.  
     
  3. mikewarren

    mikewarren

    Joined:
    Apr 21, 2014
    Posts:
    109
    I think I've figured it out.
    The grammar header must have a tag-format declaration if you're doing semantic interpretation. So, the last line of the grammar node should be

    version="1.0" mode="voice" tag-format="semantics/1.0" root="main">

    Found the issue when I tried to load the grammar into a regular MS speech grammar (non-Unity) and it threw an error.
    Hopefully, this can help someone else.

    Good resources here.
    https://www.w3.org/TR/speech-grammar/
    https://www.w3.org/TR/semantic-interpretation/
     
    PECoq and Shuyin76 like this.