Skip to content

Commit

Permalink
chore: update demo url in the README
Browse files Browse the repository at this point in the history
  • Loading branch information
@hughfenghen
hughfenghen committed Dec 8, 2023
1 parent 5d447e6 commit d5c500e
Show file tree
Hide file tree
Showing 3 changed files with 20 additions and 231 deletions.
60 changes: 5 additions & 55 deletions packages/av-canvas/README.md
View file
Original file line number Diff line number Diff line change
@@ -1,64 +1,14 @@
# AVCanvas

Combine Text, Image, Video, Audio, UserMedia, DisplayMedia to generate MediaStream.
With [AVRcorder](../av-recorder/README.md) you can output MP4 streams and save them as local files or push them to the server.
With [AVRcorder](../av-recorder/README.md) you can output MP4 streams and save them as local files or push them to the server.

使用文字、图片、音视频文件、摄像头&麦克风、分享屏幕来生成 MediaStream。
配合 [AVRcorder](../av-recorder/README.md) 可以输出 MP4 流，然后保存为本地文件或推送至服务器。
配合 [AVRcorder](../av-recorder/README.md) 可以输出 MP4 流，然后保存为本地文件或推送至服务器。

## Example
Record a video tutorial, add camera, micphone and screen to the canvas, then export mp4 and save as mp4 file.
录制视频教程，在画布中添加 camera、micphone 和屏幕，然后导出mp4保存为mp4文件。

```ts
import { AVCanvas, VideoSprite } from '@webav/av-canvas'
import { AVRecorder } from '@webav/av-recorder'

const avCvs = new AVCanvas(document.querySelector('#app') as HTMLElement, {
bgColor: '#333',
resolution: {
width: 1920,
height: 1080
}
})

const userMediaStream = await navigator.mediaDevices.getUserMedia({
video: true,
audio: true
})
const userSprite = new VideoSprite('userMedia', userMediaStream, {
audioCtx: avCvs.spriteManager.audioCtx
})
await avCvs.spriteManager.addSprite(userSprite)

const screenStream = await navigator.mediaDevices.getDisplayMedia({
video: true,
audio: true
})
const screenSprite = new VideoSprite('screen', screenStream, {
audioCtx: avCvs.spriteManager.audioCtx
})
await avCvs.spriteManager.addSprite(screenSprite)

Record a video tutorial, add camera, micphone and screen to the canvas, then export mp4 and save as mp4 file.
录制视频教程，在画布中添加 camera、micphone 和屏幕，然后导出 mp4 保存为 mp4 文件。

// start event
const recorder = new AVRecorder(avCvs.captureStream(), {
width: 1280,
height: 720,
audioCodec: 'aac'
})
await recorder.start()

const fileHandle = await window.showSaveFilePicker({
suggestedName: 'tutorial.mp4'
})
recorder.outputStream
.pipeTo(fileHandle.createWritable())

// stop event
recorder.stop()
```

## Demo
[Record AVCanvas](https://hughfenghen.github.io/WebAV/demo/record-avcanvas.html)
[Demo code](demo/record-avcanvas.ts)
<https://hughfenghen.github.io/WebAV/demo/4_2-recorder-avcanvas>
151 changes: 11 additions & 140 deletions packages/av-cliper/README.md
View file
Original file line number Diff line number Diff line change
@@ -1,153 +1,24 @@
# AVCliper

WebCodecs-based, combine video, audio, images, text, with animation support
基于 WebCodecs 合成 视频、音频、图片、文字，支持动画

## Demo
Source code: `./demo/*.ts`

[concat media](https://hughfenghen.github.io/WebAV/demo/concat-media.html)
[decode media](https://hughfenghen.github.io/WebAV/demo/decode-media.html)
[fast concat mp4](https://hughfenghen.github.io/WebAV/demo/fast-concat-mp4.html), without re-encoding video track
基于 WebCodecs 合成 视频、音频、图片、文字，支持动画

## Basic Concepts 基础概念

- `Combinator`: add `OffscreenSprite`, output video stream
- Currently only supports outputting binary streams for MP4,AAC format
- Currently only supports outputting binary streams for MP4,AAC format
- `OffscreenSprite`: wraps the resource (Clip), attaching Rect (position, width, height, animation) information to the resource so that it can be drawn to the Canvas
- `IClip`: Abstract encapsulation of resources, reading (`IClip.tick(time)`) raw data of resources by time slice, has been implemented for the `IClip` interface with:
- `MP4Clip, AudioClip, ImgClip, EmbedSubtitlesClip` (embedded SRT subtitles), these are the types of resources supported by `Combinator`
- `IClip`: Abstract encapsulation of resources, reading (`IClip.tick(time)`) raw data of resources by time slice, has been implemented for the `IClip` interface with:
- `MP4Clip, AudioClip, ImgClip, EmbedSubtitlesClip` (embedded SRT subtitles), these are the types of resources supported by `Combinator`

<hr />

- `Combinator`： 视频合成器， add `OffscreenSprite`， output 视频流
- `Combinator`： 视频合成器， add `OffscreenSprite`， output 视频流
- 目前仅支持输出 MP4,AAC 格式的二进制流
- `OffscreenSprite`： 包装资源（Clip），给资源附加 Rect（位置、宽高、动画）信息，使得资源可被绘制到 Canvas 上
- `IClip`： 资源的抽象封装，按时间片段读取（`IClip.tick(time)`）资源的原始数据，已实现 `IClip` 接口的有：
- `MP4Clip, AudioClip, ImgClip, EmbedSubtitlesClip`（内嵌SRT字幕），这些是合成视频所支持的资源类型


## Usage
**Two video overlays**
Re-encoding, output video formats are MP4(H264), AAC

**两个视频叠加**
重新编码，输出音视频格式为 MP4(H264), AAC

```ts
import { Combinator, OffscreenSprite, MP4Clip } from '@webav/av-cliper'

const com = new Combinator({
// 画布大小，视频分辨率
width: 1280,
height: 720
})
const spr1 = new OffscreenSprite(
'spr1',
new MP4Clip((await fetch('<mp4 url>')).body!)
)
const spr2 = new OffscreenSprite(
'spr1',
new MP4Clip((await fetch('<mp4 url>')).body!)
)
// 合并成长度 10s 的视频，丢弃 10s 以后的部分
await com.add(spr1, { duration: 10, main: true })
// 第二个视频从第 3s 开始出现，叠加在第一个视频上方
await com.add(spr2, { offset: 3 })
// com.output() return a ReadableStream, upload or save to disk
```

**Quickly concatenate two videos**
To concatenate two videos, the video attributes (resolution, audio and video encoding format, etc.) must be consistent, without re-encoding.
- `OffscreenSprite`： 包装资源（Clip），给资源附加 Rect（位置、宽高、动画）信息，使得资源可被绘制到 Canvas 上
- `IClip`： 资源的抽象封装，按时间片段读取（`IClip.tick(time)`）资源的原始数据，已实现 `IClip` 接口的有：
- `MP4Clip, AudioClip, ImgClip, EmbedSubtitlesClip`（内嵌 SRT 字幕），这些是合成视频所支持的资源类型

**快速串联两个视频**
两个视频前后衔接，视频属性（分辨率、音视频编码格式等）必须一致
```ts
import { fastConcatMP4 } from '@webav/av-cliper'

// return a ReadableStream, upload or save to disk
fastConcatMP4(
await Promise.all([
'<mp4 url 1>',
'<mp4 url 2>',
].map(async url => (await fetch(url)).body!))
)
```

**Add animation to video**
**给视频添加动画**
```ts
import { Combinator, OffscreenSprite, MP4Clip } from '@webav/av-cliper'

const com = new Combinator({
width: 1280,
height: 720
})
const spr = new OffscreenSprite(
'spr',
new MP4Clip((await fetch('<mp4 url>')).body!)
)
await spr.ready
// 初始旋转 180°
spr.rect.angle = Math.PI
// 参考 css animation
spr.setAnimation(
{
from: { angle: Math.PI, x: 0, y: 0, opacity: 1 },
to: { angle: Math.PI * 2, x: 300, y: 300, opacity: 0 }
},
{ duration: 3 }
)
await com.add(spr)
// com.output() return a ReadableStream, upload or save to disk
```

**Dubbing the video**
**视频配音**
```ts
import { Combinator, OffscreenSprite, MP4Clip, AudioClip } from '@webav/av-cliper'

const com = new Combinator({
// 画布大小，视频分辨率
width: 1280,
height: 720
})
const spr1 = new OffscreenSprite(
'spr1',
new MP4Clip((await fetch('<mp4 url>')).body!, {
// 原视频消音
audio: false
})
)
const spr2 = new OffscreenSprite(
'spr1',
new AudioClip((await fetch('<mp3 url>')).body!, {
loop: true
})
)

await com.add(spr1, { main: true })
await com.add(spr2)
```

**video frame interceptor(eg: chromakey)**
**视频帧拦截器(示例：绿幕抠图)**
```ts
import { createChromakey } from '@webav/av-cliper'

const chromakey = createChromakey({
similarity: 0.35,
smoothness: 0.05,
spill: 0.05,
})
const clip = new MP4Clip((await fetch('./public/video/chromakey-test.mp4')).body!)
clip.tickInterceptor = async (_, tickRet) => {
if (tickRet.video == null) return tickRet
return {
...tickRet,
video: await chromakey(tickRet.video)
}
}
```
## Demo

*images, subtitles are relatively simple, for examples please see the demo code, `. /demo/*.ts`*
*图片、字幕相对简单，示例请查看demo代码，`./demo/*.ts`*
<https://hughfenghen.github.io/WebAV/demo/1_1-decode-video>
40 changes: 4 additions & 36 deletions packages/av-recorder/README.md
View file
Original file line number Diff line number Diff line change
@@ -1,43 +1,11 @@
# AVRecorder

Record MediaStream to MP4, Use Webcodecs API encode VideoFrame and AudioData, [mp4box.js](https://github.com/gpac/mp4box.js) as muxer.
录制 MediaStream 到 MP4，使用 Webcodecs API 编码 VideoFrame、AudioData，mp4box.js 封装。
录制 MediaStream 到 MP4，使用 Webcodecs API 编码 VideoFrame、AudioData，mp4box.js 封装。

## Example
Record camera & microphone, save data to MP4 file.
录制摄像头和麦克风，保存为 MP4 文件。

```ts
import { AVRecorder } from '@webav/av-recorder'

// MediaStream from: getUserMedia, displayMedia, VideoHTMLElement.captureStream, VideoCanvasElement.captureStream, AVCanvas.captureStream etc...
const mediaStream = await navigator.mediaDevices.getUserMedia({
video: true,
audio: true
})

const recorder = new AVRecorder(mediaStream, {
width: 1280,
height: 720,
})
await recorder.start()

// pause recording
recorder.pause()
// resume recording
recorder.resume()

const fileHandle = await window.showSaveFilePicker({
suggestedName: `av-recorder.mp4`
})

recorder.outputStream
.pipeTo(await fileHandle.createWritable())


// await recorder.stop()
```
Record camera & microphone, save data to MP4 file.
录制摄像头和麦克风，保存为 MP4 文件。

## Demo
[Record camera & microphone](https://hughfenghen.github.io/WebAV/demo/record-usermedia.html)
[Demo code](./demo/record-usermedia.ts)
<https://hughfenghen.github.io/WebAV/demo/4_1-recorder-usermedia>

0 comments on commit d5c500e

Please sign in to comment.