<ed-speech> component

A webcomponent for speech to text answers, transcribed in the browser with Whisper.

import "@ed-components/ed-components/ed-speech";

Support

ed-speech transcribes what the learner says with a Whisper model run in the browser by transformers.js: it works in any recent browser, and the voice never leaves the device.

The model is downloaded the first time the microphone is used, while the learner speaks (about 100 MB with the default model), then cached by the browser. Recordings are stopped after 30 seconds.

Usage

Click on the microphone and speak: the recording stops after a silence of 2 seconds, or when clicking on the microphone again. The transcription is shown, and the learner can submit it or try again.

<ed-speech lang="en" answer="Hello">
  Say «Hello».
</ed-speech>

The expected answer can also be written in the content, as in an ed-pb problem: after :say: in a list.

<ed-speech lang="en">
  Say «Hello».

  - :say: Hello
</ed-speech>

As in every ed-component, you can use Markdown as content.

<ed-speech lang="en" answer="heat">
  Say **heat** and not _it_.
</ed-speech>
<ed-speech lang="es" answer="jamón">
  Pronuncia «jamón» en español.
</ed-speech>

A blockquote after the answer is the feedback of the question (at the end of the question without :say: list), shown once the learner submitted the answer, with the score when there is an expected answer.

<ed-speech lang="en">
  Say **heat**.

  - :say: heat

  > _heat_ has a long vowel, unlike _hit_.
</ed-speech>

Attributes

  • lang: language of the answer (fr, en, es…), defaults to the page language.
  • answer: expected answer, instead of the :say: list. The edEvent then gets a score from 0 to 100, the similarity between the transcription and the answer, ignoring case, accents and punctuation.
  • silence: seconds of silence after speech stopping the recording, 2 by default. With 0, the recording is stopped with the microphone button only.
  • model: Whisper model from the Hugging Face hub, onnx-community/whisper-base by default. onnx-community/whisper-small is more accurate but larger (about 270 MB) and slower; onnx-community/whisper-tiny is lighter (about 60 MB) but not reliable in French.

Self-hosting

transformers.js is loaded from jsDelivr, the model from the Hugging Face hub and the WebAssembly runtime from jsDelivr. To serve them yourself, set the module URL before the first transcription, then configure its env:

import { EdSpeechElement } from "@ed-components/ed-components";

EdSpeechElement.transformersUrl = "/vendor/transformers.js";
const { env } = await import("/vendor/transformers.js");
env.localModelPath = "/models/";
env.allowRemoteModels = false;
env.backends.onnx.wasm.wasmPaths = "/vendor/onnxruntime/";

Credits