Made with the help of Lilien 
With this pack of nodes, you can analyze audio, extract drums, bass, vocal tracks, and use the scheduled masks and weights to create AI-generated audio-reactive animations in ComfyUI
Workflow File (click on the link, then on "download raw file" button at the right and drop the file into ComfyUI) 📜
AudioReactive_ImagesToVideo_Yvann.json
ImgToVid.mov
AudioReactive VideoToVideo
Workflow File (click on the link, then on "download raw file" button at the right and drop the file into ComfyUI) 📜
AudioReactive_VideoToVideo_Yvann.json
Demo_Lilien.mp4
Analyzes audio to generate reactive weights and graph. Can extract specific elements like drums, vocals, bass. Parameters allow manual control over audio weights
Node Parameters
- audio_sep_model: Loaded model from "Load Audio Separation Model"
- audio: Input audio file
- batch_size: Number of frames to associate with audio weights
- fps: Frames per second for processing audio weights
Parameters:
- analysis_mode: Select audio component to analyze
- threshold: Minimum weight value to pass through
- multiply: Amplification factor for weights before normalization
Outputs:
- graph_audio: Graph image of audio weights over frames
- processed_audio: Separated or processed audio (e.g., drums vocals)
- original_audio: Original unmodified audio input
- audio_weights: List of audio-reactive weights based on processed audio
Load an audio separation model, If unavailable downloads to `ComfyUI/models/audio_separation_model/
Node Parameters
- model: Audio separation model to load
- HybridDemucs: Most accurate fastest and lightweight
- OpenUnmix: Alternative model
Outputs:
- audio_sep_model: Loaded audio separation model
Connect it to "Audio Analysis" or "Audio Remixer"
Detects peaks in audio weights based on a threshold and minimum distance. Identifies significant audio events to trigger visual changes or actions.
Node Parameters
- peaks_threshold: Threshold for peak detection
- min_peaks_distance: Minimum frames between consecutive peaks help remove close unwanted peaks around big peaks
Outputs:
- peaks_weights: Binary list indicating peak presence (1 for peak 0 otherwise)
- peaks_alternate_weights: Alternating binary list based on detected peaks
- peaks_index: String of peak indices
- peaks_count: Total number of detected peaks
- graph_peaks: Visualization image of detected peaks over audio weights
Uses "peaks_weights" from "Audio Peaks Detection" to control image transitions based on audio peaks. Outputs images and weights for two IPAdapter batches, logic from "IPAdapter Weights", IPAdapter_Plus
Node Parameters
- **images**: Batch of images for transitions, Loops images to match peak count - **peaks_weights**: List of audio peaks from "Audio Peaks Detection"Parameters:
- blend_mode: transition method applied to weights
- transitions_length: Frames used to blend between images
- min_IPA_weight: Minimum weight applied by IPAdapter per frame
- max_IPA_weight: Maximum weight applied by IPAdapter per frame
Outputs:
- image_1: Starting image for transition Connect to first IPAdapter batch "image"
- weights: Blending weights for transitions Connect to first IPAdapter batch "weight"
- image_2: Ending image for transition Connect to second IPAdapter batch "image"
- weights_invert: Inversed weights Connect to second IPAdapter batch "weight"
- graph_transitions: Visualization of weight transitions over frames
Associates "prompts" with "peaks_index" into a scheduled format. Connect output to "batch prompt schedule" of Fizz Nodes add an empty line between each individual prompts
Node Parameters
- **peaks_index**: frames where peaks occurs from "Audio Peaks Detections" - **prompts**: Multiline string of prompts for each indexOutputs:
- prompt_schedule: String mapping each audio index to a prompt
Modify input audio by adjusting the intensity of drums bass vocals or others elements
Node Parameters
- **audio_sep_model**: Loaded model from "Load Audio Separation Model" - **audio**: Input audio fileParameters:
- bass_volume: Adjusts bass volume
- drums_volume: Adjusts drums volume
- others_volume: Adjusts others elements' volume
- vocals_volume: Adjusts vocals volume
Outputs:
- merged_audio: Composition of separated tracks with applied modifications
Repeats images N times, Cycles inputs if N > images
Node Parameters
- **mask**: Mask input to convertOutputs:
- float: Float value
Inverts each value in a list of floats
Node Parameters
- floats: List of float values to invert.
Outputs:
- inverted_floats: Inverted list of float values.
Generates a graph from floats for visual data comparison
Useful to compare audio weights
Node Parameters
- floats: Primary list of floats to visualize
- floats_optional1: (Optional) Second list of floats
- floats_optional2: (Optional) Third list of floats
Parameters:
- title: Graph title
- x_label: Label for the x-axis
- y_label: Label for the y-axis
Outputs:
- visual_graph: Visual graph of provided floats
Converts mask into float works with batch of mask
Node Parameters
- mask: Mask input to convert.
Outputs:
- float: Float value representing the average value of the mask.
Converts a list of floats into an IPAdapter weights strategy format. Use with "IPAdapter Weights From Strategy" or "Prompt Schedule From Weights Strategy" to integrate output into IPAdapter pipeline
Node Parameters
Inputs:
- floats: List of float values to convert
Outputs:
- WEIGHTS_STRATEGY: Dictionary of the weights strategy
- Install ComfyUI & ComfyUI-Manager
- Launch ComfyUI
- Click on "🧩 Manager" -> "Custom Nodes Manager"
- Search for
ComfyUI_Yvann-Nodesin the manager and install it