Howto for TTS with Emacs-Reveal

Jens Lechtenbörger

Press “play” once in audio-controls below (or type “a”) to start the presentation, which advances automatically afterwards.

1. General thoughts

All of this builds on emacs-reveal (Lechtenbörger 2019a, 2019b)
- Check out its howto first
Text-To-Speech (TTS) should read notes (#+begin_notes ... #+end_notes)
- Controlled by option reveal-with-tts
  - Use customization for available speakers
- Audio is played with the audio slideshow plugin for Reveal.js
If slides with audio advance automatically, this is a video mode
- Then, notes are required for every slides
- Reveal.js “fragments” (animations) are still possible

1.1. Technical Idea

Implement TTS as two-stage process
- First, extract notes from presentation
  - Generate a text file for each note
    - Its name is a hash value of the contents
  - Generate one index file that stores names (and other information) for all text files
  - This happens during export/publication of Org files into reveal.js presentations
- Second, run TTS software on index file to generate audio
  - Implemented in Docker image emacs-reveal/tts
  - Generated audio shares hash value of its text as part of its name, enabling caching of unchanged audio
Use audio slideshow plugin to play audio

Speech is generated from text in a process with two stages. First, usual presentation notes serve as text input. These notes may embed SSML break elements to specify a break with a given duration in seconds between sentences. See the source code of this slide for examples.

While processing the org source code to generate a presentation, each note is extracted into a text file (with some preprocessing). The name of such a text file is the hash value of its contents. Thus, changing contents lead to changing names.

In addition, another text file serves as index, collecting the names and positions of texts in a presentation. Besides, this index file also records configuration information, such as the speaker to be used. Second, the index serves as input for the text to speech implementation, which is available as Docker image. Here, names of generated audio again embed the hash values of their input texts, enabling caching of unchanged audio.

In presentations, audio is played with the audio slideshow plugin.

1.2. Docker image `emacs-reveal/tts`

Contains two free and open TTS implementations
- SpeechBrain
- Microsoft SpeechT5
For size reasons, without GPU support
Small wrapper package tts.py
- Sample invocation shown in .gitlab-ci.yml of this presentation

2. Slide with notes and fragments

These notes are supposed to be transformed to audio by tea tea ess and read by the audio plugin (if it is enabled). Org-re-reveal converts text to have each sentence on a single line, which is converted to audio by Docker image tts of emacs-reveal.

Note that hyphenated words and abbreviations are usually not pronounced correctly, e.g., CPUs. We might rewrite this to use multiple see pea use?

Notes can contain Org markup, such as hyperlinks, bold, emphasis, code, verbatim.

Such markup is removed for tea tea ess in org-re-reveal.

Lists can be used in notes as well:

This is a first item in a list.
Second item.

Note that numbers are currently skipped in text to speech. Thus, the text of enumerations should clarify which point is currently read.

As we aim for text to speech, notes should consist of full sentences, including full stops, question marks etc. Warnings are shown upon export if the code detects this not to be the case.

Notes on this slide clarify some aspects of the text generated by org-re-reveal as basis for TTS.

Besides, for demonstration purposes, this slide contains fragments with separate notes:

First appearing point, with notes

Each fragment has its own notes. These ones are meant for the first bullet point.
Second appearing point

Explanations continue with this second bullet point.

3. Slides with notes, no fragments

First point
- Notes only for overall slide
Second point

3.1. Slide on second level of nesting

3.1.1. Third level of nesting

3.1.2. Third level of nesting, again

3.2. Another test case

4. The End

Person taking steps to top

The road ahead …

“Figure” under CC0 1.0; converted from Pixabay

https://gitlab.com/oer/

4.1. Final slide (silent)

Here, the notes specify the names for the text and audio file as another-slide-audio
- Thus, the audio plugin will not play this properly with the current configuration

4.2. Bibliography

Lechtenbörger, Jens. 2019a. “Emacs-reveal: A software bundle to create OER presentations.” Journal of Open Source Education (Jose) 2 (18). https://doi.org/10.21105/jose.00050.

———. 2019b. “Simplifying license attribution for OER with emacs-reveal.” In 17. Fachtagung Bildungstechnologien (DELFI 2019), edited by Niels Pinkwart and Johannes Konert, 205–16. Bonn: Gesellschaft für Informatik e.V. https://doi.org/10.18420/delfi2019_280.

License Information

No warranties are given. The license may not give you all of the permissions necessary for your intended use.

In particular, trademark rights are not licensed under this license. Thus, rights concerning third party logos (e.g., on the title slide) and other (trade-) marks (e.g., “Creative Commons” itself) remain with their respective holders.