This repository has no description
📝 Document video submodule
author
Gwenn Le Bihan
committer
GitHub
date
(Feb 27, 2025, 11:33 PM +0100)
commit
b83b3b55
b83b3b553d5d06fe738b6d742a487abd73a3ea2b
parent
d73631ac
d73631acb57fe3c67d97bdafe007ff4336083e55
+20
+20
src/video/README.md
+20
src/video/README.md
···
1
+
# Shapemaker Videos
2
+
3
+
Creation of shapemaker videos is implemented through a video engine, that allows library users to hook into moments of the song (see `../synchronization`) and modify the video's `Canvas` (see `../graphics`) to change it for the frames to come, until it is changed again.
4
+
5
+
The definition of types and methods to register these hooks, is in `engine.rs`.
6
+
7
+
Some hooks are named "later hooks" because they are triggered manually while running another hook. These are manily used to implement animations, see `animation.rs`
8
+
9
+
## Rendering logic
10
+
11
+
The rendering and encoding is done in `encoding.rs`:
12
+
13
+
The rendering loop goes as follows:
14
+
15
+
1. Iterate over every millisecond value for the duration of the song
16
+
2. For every hook (that were attached to the struct by other method calls, see `engine.rs`), determine if the hook needs to run, and if it does, run it, passing it a mutable reference to the canvas as well as a "Context" struct instance, that gives useful information such as the curent timestamp, current beat, etc.
17
+
3. If the current millisecond value corresponds to a new video frame (calculated by taking into account the target FPS), render the canvas to SVG.(see `../rendering`)
18
+
4. That string is then rendered to matrix of pixels by the [resvg](https://crates.io/crates/resvg) crate
19
+
3. That matrix is a flat 2D array of RGBA values, which we convert to the representation ffmpeg needs (a "Height-Width-Channels" 3-dimensionnal array)
20
+
4. These HWC frames are given to the encoder (libx264, used through the [video-rs](https://crates.io/crates/video-rs) crate) to create the final video file