Unable to download some AV1 VODs

[Demon1342]

Checklist

• I have checked the issue page for duplicates
• I am running the latest version (download here)

Edition

Windows GUI Application

Describe your issue here

I get this error just as the VOD gets to finalization : ERROR: Failed to finalize video (code -1094995529), retrying in 5 seconds...

Possibly because there isn't a 1440p download option.

Add any related files or extra information here

https://www.twitch.tv/videos/2255954672

[Demon1342]

Seems like the only way to download these VODs is by getting the m3u8 link with VodRecovery and rendering it through VLC

[ScrubN]

Seems like the only way to download these VODs is by getting the m3u8 link with VodRecovery and rendering it through VLC

Strange, VLC 3.0.21 refuses to play the individual video files for me. Are you using VLC nightly?

[lay295]

When pasting the M3U8 URL into VLC via the Media > Open Network Stream it plays back the stream correctly, but doesn't when I try and play individual segments

#EXTM3U
#EXT-X-VERSION:6
#EXT-X-TARGETDURATION:12
#ID3-EQUIV-TDTG:2024-09-20T22:55:31
#EXT-X-PLAYLIST-TYPE:EVENT
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-TWITCH-ELAPSED-SECS:0.000
#EXT-X-TWITCH-TOTAL-SECS:20396.233
#EXT-X-MAP:URI="init-0.mp4"
#EXTINF:10.000,
0.mp4

When I prepend the contents of init-0.mp4 into the individual segments I'm able to play them in VLC, although when I breakpoint before the video contact finalization and manually prepend the contents it still fails so maybe something else needs to be changed too.

Simplifying the contact.txt to the following allowed the finalization to finish. I think I remember the stream information that was supposed to be in here was a fix where concatenating muted segments would end up with a muted final video.

ffconcat version 1.0
file '0.mp4'
duration 10.000
file '1.mp4'
duration 10.000

9-20-24.Twitch.-.TwitchCon.San.Diego.2024.Day.1.Opening.Ceremony.w.@djclancy.@merrykish.@maya.@vanessa_lopez_official.and.more.mp4

Looks like these new segments don't have specific IDs for their streams like the older .ts files did

new codec ffprobe stream output

"streams": [
      {
          "index": 0,
          "codec_name": "aac",
          "codec_long_name": "AAC (Advanced Audio Coding)",
          "profile": "LC",
          "codec_type": "audio",
          "codec_time_base": "1/48000",
          "codec_tag_string": "mp4a",
          "codec_tag": "0x6134706d",
          "sample_fmt": "fltp",
          "sample_rate": "48000",
          "channels": 2,
          "channel_layout": "stereo",
          "bits_per_sample": 0,
          "r_frame_rate": "0/0",
          "avg_frame_rate": "0/0",
          "time_base": "1/48000",
          "start_pts": 2976960,
          "start_time": "62.020000",
          "duration_ts": 3461312,
          "duration": "72.110667",
          "bit_rate": "27189",
          "disposition": {
              "default": 1,
              "dub": 0,
              "original": 0,
              "comment": 0,
              "lyrics": 0,
              "karaoke": 0,
              "forced": 0,
              "hearing_impaired": 0,
              "visual_impaired": 0,
              "clean_effects": 0,
              "attached_pic": 0,
              "timed_thumbnails": 0
          },
          "tags": {
              "language": "und",
              "handler_name": "SoundHandler"
          }
      },
      {
          "index": 1,
          "codec_name": "h264",
          "codec_long_name": "H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10",
          "profile": "High",
          "codec_type": "video",
          "codec_time_base": "1/120",
          "codec_tag_string": "avc1",
          "codec_tag": "0x31637661",
          "width": 1920,
          "height": 1080,
          "coded_width": 1920,
          "coded_height": 1088,
          "closed_captions": 0,
          "has_b_frames": 1,
          "sample_aspect_ratio": "1:1",
          "display_aspect_ratio": "16:9",
          "pix_fmt": "yuv420p",
          "level": 42,
          "color_range": "tv",
          "color_space": "bt709",
          "color_transfer": "bt709",
          "color_primaries": "bt709",
          "chroma_location": "left",
          "refs": 1,
          "is_avc": "true",
          "nal_length_size": "4",
          "r_frame_rate": "60/1",
          "avg_frame_rate": "60/1",
          "time_base": "1/1000000",
          "start_pts": 62100000,
          "start_time": "62.100000",
          "duration_ts": 72000000,
          "duration": "72.000000",
          "bit_rate": "838275",
          "bits_per_raw_sample": "8",
          "disposition": {
              "default": 1,
              "dub": 0,
              "original": 0,
              "comment": 0,
              "lyrics": 0,
              "karaoke": 0,
              "forced": 0,
              "hearing_impaired": 0,
              "visual_impaired": 0,
              "clean_effects": 0,
              "attached_pic": 0,
              "timed_thumbnails": 0
          },
          "tags": {
              "language": "und",
              "handler_name": "VideoHandler"
          }
      }
  ]

old codec ffprobe stream output

"streams": [
      {
          "index": 0,
          "codec_name": "aac",
          "codec_long_name": "AAC (Advanced Audio Coding)",
          "profile": "LC",
          "codec_type": "audio",
          "codec_time_base": "1/48000",
          "codec_tag_string": "[15][0][0][0]",
          "codec_tag": "0x000f",
          "sample_fmt": "fltp",
          "sample_rate": "48000",
          "channels": 2,
          "channel_layout": "stereo",
          "bits_per_sample": 0,
          "id": "0x100",
          "r_frame_rate": "0/0",
          "avg_frame_rate": "0/0",
          "time_base": "1/90000",
          "start_pts": 9183450,
          "start_time": "102.038333",
          "duration_ts": 896640,
          "duration": "9.962667",
          "bit_rate": "164208",
          "disposition": {
              "default": 0,
              "dub": 0,
              "original": 0,
              "comment": 0,
              "lyrics": 0,
              "karaoke": 0,
              "forced": 0,
              "hearing_impaired": 0,
              "visual_impaired": 0,
              "clean_effects": 0,
              "attached_pic": 0,
              "timed_thumbnails": 0
          }
      },
      {
          "index": 1,
          "codec_name": "h264",
          "codec_long_name": "H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10",
          "profile": "Main",
          "codec_type": "video",
          "codec_time_base": "1/120",
          "codec_tag_string": "[27][0][0][0]",
          "codec_tag": "0x001b",
          "width": 1280,
          "height": 720,
          "coded_width": 1280,
          "coded_height": 720,
          "closed_captions": 0,
          "has_b_frames": 2,
          "pix_fmt": "yuv420p",
          "level": 31,
          "color_range": "tv",
          "color_space": "bt709",
          "color_transfer": "bt709",
          "color_primaries": "bt709",
          "chroma_location": "left",
          "field_order": "progressive",
          "refs": 1,
          "is_avc": "false",
          "nal_length_size": "0",
          "id": "0x101",
          "r_frame_rate": "60/1",
          "avg_frame_rate": "60/1",
          "time_base": "1/90000",
          "start_pts": 9182880,
          "start_time": "102.032000",
          "duration_ts": 900060,
          "duration": "10.000667",
          "bits_per_raw_sample": "8",
          "disposition": {
              "default": 0,
              "dub": 0,
              "original": 0,
              "comment": 0,
              "lyrics": 0,
              "karaoke": 0,
              "forced": 0,
              "hearing_impaired": 0,
              "visual_impaired": 0,
              "clean_effects": 0,
              "attached_pic": 0,
              "timed_thumbnails": 0
          }
      },
      {
          "index": 2,
          "codec_name": "timed_id3",
          "codec_long_name": "timed ID3 metadata",
          "codec_type": "data",
          "codec_tag_string": "ID3 ",
          "codec_tag": "0x20334449",
          "id": "0x102",
          "r_frame_rate": "0/0",
          "avg_frame_rate": "0/0",
          "time_base": "1/90000",
          "start_pts": 9182880,
          "start_time": "102.032000",
          "disposition": {
              "default": 0,
              "dub": 0,
              "original": 0,
              "comment": 0,
              "lyrics": 0,
              "karaoke": 0,
              "forced": 0,
              "hearing_impaired": 0,
              "visual_impaired": 0,
              "clean_effects": 0,
              "attached_pic": 0,
              "timed_thumbnails": 0
          }
      }
  ]

Example stream with new format and muted audio at beginning
https://www.twitch.tv/videos/2252059715

[ScrubN]

Looks like these new segments don't have specific IDs for their streams like the older .ts files did

Good find, and also very unfortunate. Maybe we can try using codecs instead of stream IDs?

Edit: It does not work

[ScrubN]

I have a very rough proof of concept working, I just need to change a few more things and clean it up. Hoping for it to be ready tomorrow.

[ScrubN]

I've run into a small issue. The finalized VOD works when viewer in FFmpeg based applications like MPV and Blender, but not VLC or Windows Photos. For whatever reason, they don't like the way that FFmpeg creates the no-copy concatenated AV1 video stream, causing it to be black.

Doing any kind of re-encoding (FFmpeg, Handbrake) seems to have the possibility of video corruption.

• When tested on the first 3 minutes of 2321073288 playback is completely normal in MPV/Blender, but VLC/Win Photos do not show anything.
• Re-encoding the video stream with FFmpeg/Handbrake allows these other applications to decode the video stream, however around 1:38 the video stream just stops decoding and all applications refuse to manually seek beyond this point.

[ScrubN]

Downloading HEVC VODs works and AV1 VODs kinda work for some applications. I can't be bothered to figure it all out right now, and I think getting experimental support out is better than the current behavior. I'll just adding a warning for AV1 VODs with some recommendations on what to do if there are any playback issues until we can figure it out.

[ScrubN]

Going to keep this open because some AV1 VODs aren't finalized correctly.

[sourKiwi0]

Is there a way to know if a VOD will have "Invalid data found when processing input" / "Failed to finalize video" error before downloading?

[ScrubN]

Is there a way to know if a VOD will have "Invalid data found when processing input" / "Failed to finalize video" error before downloading?

Please provide your VOD ID

[sourKiwi0]

Please provide your VOD ID

https://www.twitch.tv/videos/2342020739
OS: Rocky Linux 8.10

[<USER>@localhost ~]$ TwitchDownloaderCLI videodownload -u https://www.twitch.tv/videos/2342020739 -q 720p60 -o test_2342020739_full.mp4 --temp-path "TDTemp/" --log-level Status,Verbose,Info,Warning,Error,Ffmpeg
TwitchDownloaderCLI 1.55.1 Copyright (c) lay295 and contributors
[STATUS] - Fetching Video Info [1/4]
[STATUS] - Downloading 100% [2/4]
[STATUS] - Verifying Parts 100% [3/4]
[STATUS] - Finalizing Video 0% [4/4]
[VERBOSE] - Running "ffmpeg" in "/home/<USER>/TDTemp/TwitchDownloader/2342020739_638714109964353124" with args: -stats -y -avoid_negative_ts make_zero -analyzeduration 2147483647 -probesize 2147483647 -f concat -max_streams 2147483647 -i /home/<USER>/TDTemp/TwitchDownloader/2342020739_638714109964353124/concat.txt -i /home/<USER>/TDTemp/TwitchDownloader/2342020739_638714109964353124/metadata.txt -map_metadata 1 -c copy /home/<USER>/test_2342020739_full.mp4
[ERROR] - Failed to finalize video (code 1), retrying in 5 seconds...
[VERBOSE] - Running with audio copy disabled.
[VERBOSE] - Running "ffmpeg" in "/home/<USER>/TDTemp/TwitchDownloader/2342020739_638714109964353124" with args: -stats -y -avoid_negative_ts make_zero -analyzeduration 2147483647 -probesize 2147483647 -f concat -max_streams 2147483647 -i /home/<USER>/TDTemp/TwitchDownloader/2342020739_638714109964353124/concat.txt -i /home/<USER>/TDTemp/TwitchDownloader/2342020739_638714109964353124/metadata.txt -map_metadata 1 -c:v copy /home/<USER>/test_2342020739_full.mp4
[ERROR] - Failed to finalize video (code 1), retrying in 5 seconds...
Unhandled exception. System.AggregateException: One or more errors occurred. (Failed to finalize video. The download cache has not been cleared and can be found at /home/<USER>/TDTemp/TwitchDownloader/2342020739_638714109964353124 along with a log file.)
 ---> System.Exception: Failed to finalize video. The download cache has not been cleared and can be found at /home/<USER>/TDTemp/TwitchDownloader/2342020739_638714109964353124 along with a log file.
   at TwitchDownloaderCore.VideoDownloader.DownloadAsyncImpl(FileInfo outputFileInfo, FileStream outputFs, CancellationToken cancellationToken) in /home/runner/work/TwitchDownloader/TwitchDownloader/TwitchDownloaderCore/VideoDownloader.cs:line 144
   at TwitchDownloaderCore.VideoDownloader.DownloadAsyncImpl(FileInfo outputFileInfo, FileStream outputFs, CancellationToken cancellationToken) in /home/runner/work/TwitchDownloader/TwitchDownloader/TwitchDownloaderCore/VideoDownloader.cs:line 157
   at TwitchDownloaderCore.VideoDownloader.DownloadAsync(CancellationToken cancellationToken) in /home/runner/work/TwitchDownloader/TwitchDownloader/TwitchDownloaderCore/VideoDownloader.cs:line 52
   at TwitchDownloaderCore.VideoDownloader.DownloadAsync(CancellationToken cancellationToken) in /home/runner/work/TwitchDownloader/TwitchDownloader/TwitchDownloaderCore/VideoDownloader.cs:line 60
   at TwitchDownloaderCore.VideoDownloader.DownloadAsync(CancellationToken cancellationToken) in /home/runner/work/TwitchDownloader/TwitchDownloader/TwitchDownloaderCore/VideoDownloader.cs:line 61
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean )
   at System.Threading.Tasks.Task.Wait(Int32 , CancellationToken )
   at System.Threading.Tasks.Task.Wait()
   at TwitchDownloaderCLI.Modes.DownloadVideo.Download(VideoDownloadArgs inputOptions) in /home/runner/work/TwitchDownloader/TwitchDownloader/TwitchDownloaderCLI/Modes/DownloadVideo.cs:line 27
   at CommandLine.ParserResultExtensions.WithParsed[T](ParserResult`1 result, Action`1 action)
   at TwitchDownloaderCLI.Program.Main(String[] args) in /home/runner/work/TwitchDownloader/TwitchDownloader/TwitchDownloaderCLI/Program.cs:line 34
Aborted (core dumped)
[<USER>@localhost ~]$

ffmpegLog.txt
metadata.txt
concat.txt