Search Unity

Speech semantics example?

Discussion in 'Documentation' started by mikewarren, Feb 4, 2019.

  1. mikewarren

    mikewarren

    Joined:
    Apr 21, 2014
    Posts:
    109
    Anyone have a sample grammar that includes semantics and works in Unity (2018.3)?
    I haven't been able to cobble together a working example.

    Mike
     
  2. mikewarren

    mikewarren

    Joined:
    Apr 21, 2014
    Posts:
    109
    I'm stumped. Here's my grammar (example from web) and script. I don't get any recognition events unless the semantic tags are removed. Status of the recognizer is "running" either way.

    Could use a good reference for creating semantic tags as well.


    <?xml version="1.0" encoding="UTF-8" ?>
    <grammar xmlns="http://www.w3.org/2001/06/grammar"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://www.w3.org/2001/06/grammar
    http://www.w3.org/TR/speech-grammar/grammar.xsd"
    xml:lang="en"
    version="1.0" mode="voice" root="main">

    <rule id="main">
    <one-of>
    <item><ruleref uri="#rule1" /></item>
    </one-of>
    </rule>

    <rule id="rule1" scope="public">
    <one-of>
    <item>1<tag>out.start="1";</tag></item>
    <item>2<tag>out.start="2";</tag></item>
    <item>3<tag>out.start="3";</tag></item>
    </one-of>
    </rule>

    </grammar>


    Code (CSharp):
    1. using System.Collections;
    2. using System.Collections.Generic;
    3. using System.Linq;
    4. using System.IO;
    5.  
    6. using UnityEngine;
    7. using UnityEngine.Windows.Speech;
    8.  
    9.  
    10. public class SpeechGrammarTest : MonoBehaviour
    11. {
    12.     public string grammarPath;
    13.     [Header("Read Only")]
    14.     public string dateTime;
    15.     public string fullPath;
    16.     public bool running;
    17.     public string utterance;
    18.     public List<string> semantics;
    19.  
    20.     GrammarRecognizer recognizer;
    21.  
    22.     // Start is called before the first frame update
    23.     void Start()
    24.     {
    25.         fullPath = Path.Combine(Application.streamingAssetsPath, grammarPath);
    26.         recognizer = new GrammarRecognizer(fullPath, ConfidenceLevel.Low);
    27.         recognizer.OnPhraseRecognized += PhraseRecognized;
    28.         recognizer.Start();
    29.         running = recognizer.IsRunning;
    30.     }
    31.  
    32.     public bool hasSemantics = false;
    33.  
    34.     private void PhraseRecognized(PhraseRecognizedEventArgs args)
    35.     {
    36.         utterance = args.text;
    37.  
    38.         hasSemantics = args.semanticMeanings != null;
    39.         semantics.Clear();
    40.         if (args.semanticMeanings != null)
    41.         {
    42.             foreach (SemanticMeaning sm in args.semanticMeanings)
    43.             {
    44.                 string newSm = sm.key;
    45.                 foreach (string val in sm.values)
    46.                     newSm = newSm + " " + val;
    47.                 semantics.Add(newSm);
    48.             }
    49.         }
    50.     }
    51.  
    52.     // Update is called once per frame
    53.     void Update()
    54.     {
    55.         dateTime = System.DateTime.Now.ToString();
    56.     }
    57. }
    58.  
     
  3. mikewarren

    mikewarren

    Joined:
    Apr 21, 2014
    Posts:
    109
    I think I've figured it out.
    The grammar header must have a tag-format declaration if you're doing semantic interpretation. So, the last line of the grammar node should be

    version="1.0" mode="voice" tag-format="semantics/1.0" root="main">

    Found the issue when I tried to load the grammar into a regular MS speech grammar (non-Unity) and it threw an error.
    Hopefully, this can help someone else.

    Good resources here.
    https://www.w3.org/TR/speech-grammar/
    https://www.w3.org/TR/semantic-interpretation/
     
    PECoq and Shuyin76 like this.