Made with the help of Lilien 
A pack of custom nodes that enable audio reactivity within ComfyUI, allowing you to generate AI-driven animations that sync with music
- Create Audio Reactive AI videos, enable controls over AI generations styles, content and composition with any audio
- Simple: Just Drop one of our Workflows in ComfyUI and specify your audio and visuals input
- Flexible: Works with existing ComfyUI AI tech and nodes (eg: IPAdapter, AnimateDiff, ControlNet, etc.)
Yvann.mp4 |
@idgrafix.mp4 |
@idgrafix2.mp4 |
Yvann2.mp4 |
@idgrafix3.mp4 |
@NebSh.mp4 |
Lilien.mp4 |
@VisualFrisson.mp4 |
- Install ComfyUI and ComfyUI-Manager
-
Images → Video
- Takes a set of images plus an audio track.
- Watch Tutorial:
-
Video → Video
- Takes a source video plus an audio track.
- Watch Tutorial:
-
Download the
.jsonfile for the workflow you picked: -
Drop the
.jsonfile into the ComfyUI window. -
Open the “🧩 Manager” → “Install Missing Custom Nodes”
- Install each pack of nodes that appears.
- Restart ComfyUI if prompted.
-
Set Your Inputs & Generate
- Provide the inputs needed (everything explained here
- Click Queue button to produce your audio-reactive animation!
That’s it! Have fun playing with the differents settings now !! (if you have any questions or problems, check my Youtube Tutorials
Click to Expand: Node-by-Node Reference
Analyzes audio to generate reactive weights for each frame.
Node Parameters
- audio_sep_model: Model from "Load Audio Separation Model"
- audio: Input audio file
- batch_size: Frames to associate with audio weights
- fps: Frame rate for the analysis
Parameters:
- analysis_mode: e.g., Drums Only, Vocals, Full Audio
- threshold: Minimum weight pass-through
- multiply: Amplification factor
Outputs:
- graph_audio (image preview),
- processed_audio, original_audio,
- audio_weights (list of values).
Loads or downloads an audio separation model (e.g., HybridDemucs, OpenUnmix).
Node Parameters
- model: Choose between HybridDemucs / OpenUnmix.
- Outputs: audio_sep_model (connect to Audio Analysis or Remixer).
Identifies peaks in the audio weights to trigger transitions or events.
Node Parameters
- peaks_threshold: Sensitivity.
- min_peaks_distance: Minimum gap in frames between peaks.
- Outputs: Binary peak list, alternate list, peak indices/count, graph.
Manages transitions between images based on peaks. Great for stable or style transitions.
Node Parameters
- images: Batch of images.
- peaks_weights: From “Audio Peaks Detection”.
- blend_mode, transitions_length, min_IPA_weight, etc.
Links text prompts to peak indices.
Node Parameters
- peaks_index: Indices from peaks detection.
- prompts: multiline string.
- Outputs: mapped schedule string.
Adjusts volume levels (drums, vocals, bass, others) in a track.
Node Parameters
- drums_volume, vocals_volume, bass_volume, others_volume
- Outputs: single merged audio track.
Repeats a set of images N times.
Node Parameters
- mask: Mask input.
- Outputs: Repeated images.
Flips sign of float values.
Node Parameters
- floats: list of floats.
- Outputs: inverted list.
Plots float values as a graph.
Node Parameters
- floats (and optional second/third).
- Outputs: visual graph image.
Converts a mask into a single float value.
Node Parameters
- mask: input.
- Outputs: float.
Transforms float lists into an IPAdapter “weight strategy.”
Node Parameters
- floats: list of floats.
- Outputs: dictionary with strategy info.
Please give a ⭐ on GitHub it helps us enhance our Tool and it's Free for you !! (:
