Skip to content

M5 Atom Echo - Wake Sound #75

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
MorganMLGman opened this issue Feb 25, 2025 · 8 comments
Open

M5 Atom Echo - Wake Sound #75

MorganMLGman opened this issue Feb 25, 2025 · 8 comments
Labels

Comments

@MorganMLGman
Copy link

Hi,

Can wake sound be added like it is in VoicePE?

I tried to add it myself, but I can't get the right sequence and wake word is played after LISTENING phase.

switch:
  ...
  - platform: template
    name: Use wake sound
    id: use_wake_sound
    optimistic: true
    restore_mode: RESTORE_DEFAULT_ON
    entity_category: config

media_player:
  - platform: speaker
    ...
    files:
     ....
      - id: wake_word_triggered_wave_file
        file: wake_word_triggered1.wav

micro_wake_word:
  ...
  on_wake_word_detected:
    - if:
        condition:
          switch.is_on: use_wake_sound
        then:
          - media_player.speaker.play_on_device_media_file:
              media_file: wake_word_triggered_wave_file
              announcement: true
          - wait_until:
              and:  
                - media_player.is_idle:
                - speaker.is_stopped:
    - voice_assistant.start:
        wake_word: !lambda return wake_word;

In current form I get:

  1. Wake word ("Ok nabu")
  2. Listening phase (I say something like "what's the weather?")
  3. WAKE SOUND
  4. Response
@jesserockz
Copy link
Member

Try add a delay: 10ms action before the wait_until. The media_player.speaker.play_on_device_media_file is "asynchronous" and might not have put the media player out of idle state right away.

@arilence
Copy link

arilence commented Feb 27, 2025

I ran into the same issue of the wake word audio playing after the LISTENING phase.

Looking at the logs, even with a delay: 1000ms before the wait_until, it outputs the error Error writing to I2S: ESP_ERR_INVALID_STATE looping infinitely until the voice pipeline finished.

[03:09:13][W][i2s_audio.speaker:143]: Error writing to I2S: ESP_ERR_INVALID_STATE
[03:09:13][D][ring_buffer:034][speaker_task]: Created ring buffer with size 1920
[03:09:13][D][i2s_audio.speaker:111]: Starting Speaker
[03:09:13][D][i2s_audio.speaker:129]: Stopped Speaker
....
(loops error until voice pipeline finishes)

That's when I noticed the logs would output [D][voice_assistant:645]: Assist Pipeline running before the i2s_audio error regardless of any amount of delay (10-5000ms) put before it.

I managed to fix this "out of order" sequence by putting everything inside the then: key

micro_wake_word:
  on_wake_word_detected:
    then:
      ...

Now the wake word audio plays at the beginning before the voice_assistant.start runs. However, it's now crashing once it starts waiting for the response.

My Atom Echo Config
substitutions:
  name: m5stack-atom-echo
  friendly_name: Bedroom Atom Echo
packages:
  m5stack.atom-echo-wake-word-voice-assistant: github://esphome/wake-word-voice-assistants/m5stack-atom-echo/m5stack-atom-echo.yaml@main
esphome:
  name: ${name}
  name_add_mac_suffix: false
  friendly_name: ${friendly_name}
esp32:
  board: m5stack-atom
  framework:
    type: esp-idf
    version: 4.4.8
    platform_version: 5.4.0
api:
  encryption:
    key: <snip>
wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password
micro_wake_word:
  on_wake_word_detected:
    then:
      - media_player.speaker.play_on_device_media_file:
          media_file: timer_finished_wave_file
          announcement: true
      - delay: 100ms
      - wait_until:
          - media_player.is_idle:
          - speaker.is_stopped:
      - voice_assistant.start:
          wake_word: !lambda return wake_word;
Crash Log
[D][api.connection:1446]: Home Assistant 2025.2.5 (10.0.20.10): Connected successfully
[W][micro_wake_word:183]: Wake word is already stopped
[D][light:036]: 'Bedroom Atom Echo' Setting:
[D][ring_buffer:034]: Created ring buffer with size 2048
[D][micro_wake_word:399]: Resetting buffers and probabilities
[D][micro_wake_word:195]: State changed from IDLE to START_MICROPHONE
[W][component:237]: Component script took a long time for an operation (59 ms).
[W][component:238]: Components should block for at most 30 ms.
[D][micro_wake_word:107]: Starting Microphone
[D][micro_wake_word:195]: State changed from START_MICROPHONE to STARTING_MICROPHONE
[D][esp-idf:000]: I (14084) I2S: DMA Malloc info, datalen=blocksize=1024, dma_buf_count=4

[D][micro_wake_word:195]: State changed from STARTING_MICROPHONE to DETECTING_WAKE_WORD
[I][safe_mode:041]: Boot seems successful; resetting boot loop counter
[D][esp32.preferences:114]: Saving 1 preferences to flash...
[D][esp32.preferences:143]: Saving 1 preferences to flash: 0 cached, 1 written, 0 failed
[D][micro_wake_word:162]: The 'Okay Nabu' model sliding average probability is 0.993 and most recent probability is 1.000
[D][micro_wake_word:123]: Wake Word 'Okay Nabu' Detected
[D][micro_wake_word:195]: State changed from DETECTING_WAKE_WORD to STOP_MICROPHONE
[D][micro_wake_word:129]: Stopping Microphone
[D][micro_wake_word:195]: State changed from STOP_MICROPHONE to STOPPING_MICROPHONE
[D][esp-idf:000]: I (62123) I2S: DMA queue destroyed

[D][micro_wake_word:195]: State changed from STOPPING_MICROPHONE to IDLE
[D][ring_buffer:034][ann_read]: Created ring buffer with size 6000
[D][light:036]: 'Bedroom Atom Echo' Setting:
[D][light:047]:   State: ON
[D][light:051]:   Brightness: 100%
[D][light:059]:   Red: 0%, Green: 0%, Blue: 100%
[D][speaker_media_player:420]: State changed to ANNOUNCING
[D][speaker_media_player.pipeline:114]: Reading WAV file type
[D][ring_buffer:034][speaker_task]: Created ring buffer with size 1920
[D][esp-idf:000][speaker_task]: I (62186) I2S: queue free spaces: 5

[D][esp-idf:000][speaker_task]: I (62194) I2S: DMA Malloc info, datalen=blocksize=960, dma_buf_count=4

[D][i2s_audio.speaker:116]: Started Speaker
[D][speaker_media_player:420]: State changed to IDLE
[D][esp-idf:000][speaker_task]: I (63120) I2S: DMA queue destroyed

[D][i2s_audio.speaker:123]: Stopping Speaker
[D][i2s_audio.speaker:129]: Stopped Speaker
[W][micro_wake_word:183]: Wake word is already stopped
[D][voice_assistant:506]: State changed from IDLE to START_MICROPHONE
[D][voice_assistant:512]: Desired state set to START_PIPELINE
[D][voice_assistant:223]: Starting Microphone
[D][ring_buffer:034]: Created ring buffer with size 16384
[D][voice_assistant:506]: State changed from START_MICROPHONE to STARTING_MICROPHONE
[D][esp-idf:000]: I (63150) I2S: DMA Malloc info, datalen=blocksize=1024, dma_buf_count=4

[D][voice_assistant:506]: State changed from STARTING_MICROPHONE to START_PIPELINE
[D][voice_assistant:277]: Requesting start...
[D][voice_assistant:506]: State changed from START_PIPELINE to STARTING_PIPELINE
[D][voice_assistant:527]: Client started, streaming microphone
[D][voice_assistant:506]: State changed from STARTING_PIPELINE to STREAMING_MICROPHONE
[D][voice_assistant:512]: Desired state set to STREAMING_MICROPHONE
[D][voice_assistant:642]: Event Type: 1
[D][voice_assistant:645]: Assist Pipeline running
[D][voice_assistant:642]: Event Type: 3
[D][voice_assistant:656]: STT started
[D][light:036]: 'Bedroom Atom Echo' Setting:
[D][light:059]:   Red: 0%, Green: 0%, Blue: 100%
[D][light:109]:   Effect: 'Slow Pulse'
[D][light:036]: 'Bedroom Atom Echo' Setting:
[D][light:047]:   State: OFF
[D][light:109]:   Effect: 'None'
[D][micro_wake_word:399]: Resetting buffers and probabilities
[D][micro_wake_word:195]: State changed from IDLE to START_MICROPHONE
[W][component:237]: Component script took a long time for an operation (62 ms).
[W][component:238]: Components should block for at most 30 ms.
[D][micro_wake_word:107]: Starting Microphone
[D][micro_wake_word:195]: State changed from START_MICROPHONE to STARTING_MICROPHONE
[D][micro_wake_word:195]: State changed from STARTING_MICROPHONE to DETECTING_WAKE_WORD
[D][voice_assistant:642]: Event Type: 11
[D][voice_assistant:805]: Starting STT by VAD
[D][voice_assistant:642]: Event Type: 12
[D][voice_assistant:809]: STT by VAD end
[D][voice_assistant:506]: State changed from STREAMING_MICROPHONE to STOP_MICROPHONE
[D][voice_assistant:512]: Desired state set to AWAITING_RESPONSE
[D][voice_assistant:506]: State changed from STOP_MICROPHONE to STOPPING_MICROPHONE
[D][light:036]: 'Bedroom Atom Echo' Setting:
[D][light:047]:   State: ON
[D][light:059]:   Red: 0%, Green: 0%, BlueGuru Meditation Error: Core  1 panic'ed (LoadProhibited). Exception was unhandled.
[18:49:22]
[18:49:22]Core  1 register dump:
[18:49:22]PC      : 0x4012c686  PS      : 0x00060230  A0      : 0x800dd88b  A1      : 0x3ffbfc80  
[18:49:22]A2      : 0x00000000  A3      : 0x3ffedb6c  A4      : 0x00000200  A5      : 0x3ffbfcc0  
[18:49:22]A6      : 0x00000064  A7      : 0x3ffb3fa8  A8      : 0x00000000  A9      : 0x3ffbfc70  
[18:49:22]A10     : 0x3ffb4208  A11     : 0x3ffb4208  A12     : 0x3ffb420c  A13     : 0x00060223  
[18:49:22]A14     : 0x00060220  A15     : 0x00000001  SAR     : 0x0000000a  EXCCAUSE: 0x0000001c  
[18:49:22]EXCVADDR: 0x0000001c  LBEG    : 0x40083739  LEND    : 0x40083741  LCOUNT  : 0x00000027  
[18:49:22]
[18:49:22]
[18:49:22]Backtrace: 0x4012c683:0x3ffbfc80 0x400dd888:0x3ffbfcc0 0x400e2c9e:0x3ffbfcf0 0x400e30d5:0x3ffbfd10 0x401b8de9:0x3ffbfd90 0x401b8e8d:0x3ffbfdb0 0x400ed5d4:0x3ffbfdd0 0x400f1706:0x3ffbfe00 0x400dbc02:0x3ffbfe20

@arilence
Copy link

arilence commented Feb 27, 2025

The crash I was experiencing is due to the media player calling start_wake_word after any audio is finished playing, including the wake word audio. Shown here:

on_idle:
- script.execute: start_wake_word

As a quick hack, overwriting the script.execute: start_wake_word from the media player's on_idle "fixes" the issue, but it's not a real solution:

media_player:
  - id: !extend echo_media_player
    on_idle:
      then:
        - delay: 1ms

I'm new to ESPHome and Voice assistant stuff, what would be the proper way to restart the wake word on idle if the audio that just finished is not from the wake word?

Full atom echo config
substitutions:
  name: m5stack-atom-echo
  friendly_name: Bedroom Atom Echo
packages:
  m5stack.atom-echo-wake-word-voice-assistant: github://esphome/wake-word-voice-assistants/m5stack-atom-echo/m5stack-atom-echo.yaml@main
esphome:
  name: ${name}
  name_add_mac_suffix: false
  friendly_name: ${friendly_name}
esp32:
  board: m5stack-atom
  framework:
    type: esp-idf
    version: 4.4.8
    platform_version: 5.4.0
api:
  encryption:
    key: <snip>
wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password
micro_wake_word:
  on_wake_word_detected:
    then:
      - media_player.speaker.play_on_device_media_file:
          media_file: timer_finished_wave_file
          announcement: true
      - delay: 10ms
      - wait_until:
          - media_player.is_idle:
          - speaker.is_stopped:
      - voice_assistant.start:
          wake_word: !lambda return wake_word;
media_player:
  - id: !extend echo_media_player
    on_idle:
      then:
        - delay: 1ms

@ptbsare
Copy link

ptbsare commented Mar 4, 2025

hi @arilence , here is my solution:

globals:
  - id: not_playing_wake_word
    type: bool
    restore_value: false
    initial_value: "true"

...

micro_wake_word:  
      ...
  on_wake_word_detected:
    then:
      - lambda: id(not_playing_wake_word) = false;
      - media_player.speaker.play_on_device_media_file:
          media_file: sounds_awake
          announcement: true
      - delay: 100ms
      - wait_until:
          - media_player.is_idle:
          - speaker.is_stopped:
      - lambda: id(not_playing_wake_word) = true;
      - voice_assistant.start: 
          wake_word: !lambda return wake_word;

media_player:
  - platform: speaker
      ...
    on_idle:
      - if:
          condition:
            lambda: return id(not_playing_wake_word);
          then:
            - micro_wake_word.start:

@MorganMLGman
Copy link
Author

@ptbsare nice solution, I added switch "use wake sound" in my setup. Works great.

@distante
Copy link

@ptbsare for this, I have to use the hole config yaml, correct? Because I am currently using

packages:
  m5stack.atom-echo-voice-assistant: github://esphome/wake-word-voice-assistants/m5stack-atom-echo/m5stack-atom-echo.yaml@main

But +1 to be able to define this directly on the default configuration

@arilence
Copy link

arilence commented Mar 19, 2025

@distante Using the package as a base works too. I'm currently doing that and then extending the media player to add the additional functionality.

I've attached my full config below. Thanks to MorganMLGman and ptbsare for their solutions.

Note I added an additional sound file sounds/sounds_wake_word_triggered.wav which is not included in esphome by default. Alternatively change media_file: to media_file: timer_finished_wave_file to use a sound built-in to the atom echo config.

Expand to see full config
substitutions:
  name: m5stack-atom-echo
  friendly_name: Bedroom Atom Echo
packages:
  m5stack.atom-echo-wake-word-voice-assistant: github://esphome/wake-word-voice-assistants/m5stack-atom-echo/m5stack-atom-echo.yaml@main
esphome:
  name: ${name}
  name_add_mac_suffix: false
  friendly_name: ${friendly_name}
esp32:
  board: m5stack-atom
api:
  encryption:
    key: <snip>
wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password
globals:
  - id: playing_wake_word
    type: bool
    restore_value: false
    initial_value: "false"
switch:
  - platform: template
    name: Use wake sound
    id: use_wake_sound
    optimistic: true
    restore_mode: RESTORE_DEFAULT_ON
    entity_category: config
micro_wake_word:
  on_wake_word_detected:
    then:
      - if:
          condition:
            switch.is_on: use_wake_sound
          then:
            - lambda: id(playing_wake_word) = true;
            - media_player.speaker.play_on_device_media_file:
                media_file: wake_word_triggered_sound
                announcement: true
            - delay: 10ms
            - wait_until:
                - media_player.is_idle:
                - speaker.is_stopped:
            - lambda: id(playing_wake_word) = false;
      - voice_assistant.start:
          wake_word: !lambda return wake_word;
media_player:
  - id: !extend echo_media_player
    files:
      - id: wake_word_triggered_sound
        file: sounds/sounds_wake_word_triggered.wav
    on_idle:
      then:
        - if:
            condition:
              not:
                lambda: return id(playing_wake_word);
            then:
              - micro_wake_word.start:

Copy link

github-actions bot commented May 1, 2025

As there has been no activity on this issue for 30 days, I am marking it as stale. If you think this is a mistake, please comment below and I will remove the stale label.

@github-actions github-actions bot added the stale label May 1, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

5 participants