shakuhachi - code explanation with voice.

[JSerZANP]

Motivation

Think about recording a video explaining a piece of code

1. need screen recording
2. need editing
3. upload somewhere
4. embed as video widget

The problem is Video is too heavy a tool for such use case, what we actually want is

1. a voice companion for the code snippet
2. a cursor to indicate where we are.

This actually could be achieved by recording two stream(voice stream, action stream) and put them together.

Design

1. recording

1. voice stream

Just use MediaStream api

https://developer.mozilla.org/en-US/docs/Web/API/MediaStream_Recording_API/Using_the_MediaStream_Recording_API

2. action stream

We only need to record the important actions, including

1. mouse click
2. selection
3. Shaku-specfic events

Shaku-specific events are events that triggers visual changes on code snippets, e.g.

1. highlight a line
2. toggle @ fold

Events must have location information attached to specific lines & characters.

2. Edit (low priority)

It should be feasible to cut the spaces between actions and voice.
If not able to do it automatically, we can at least edit manually.

3. Generating

Voice data and the event streams are bundled together, with time info on one single timeline.

3. Rendering

Just play the voice and events along the timeline.

For fast forward, we just increase the play speed.
For rewind, we need to start from the very start.

We need controls on the UI.

1. play
2. pause
3. progress bar