Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor VideoDecoder C++ initialization #435

Merged
merged 1 commit into from
Dec 16, 2024
Merged

Refactor VideoDecoder C++ initialization #435

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 27 additions & 14 deletions src/torchcodec/decoders/_core/VideoDecoder.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -234,59 +234,72 @@ VideoDecoder::VideoDecoder(const void* buffer, size_t length) {
}

void VideoDecoder::initializeDecoder() {
// Some formats don't store enough info in the header so we read/decode a few
// frames to grab that. This is needed for the filter graph. Note: If this
// takes a long time, consider initializing the filter graph after the first
// frame decode.
Comment on lines -237 to -240
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Curious if we should keep that comment, or what was the reason to remove it? It might be helpful to keep (especially the "Note" part), particularly so that the reader knows that some frame decoding does happen within avformat_find_stream_info()?

I'm finding this from the FFmpeg docs

Some formats do not have a header or do not store enough information there, so it is recommended that you call the avformat_find_stream_info() function which tries to read and decode a few frames to find missing information.

Copy link
Contributor Author

@scotts scotts Jan 9, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. I removed the comment because of the latter part about lazy initialization - we're doing that now. But the other stuff is both relevant and informative. I'll submit a PR which revives the relevant stuff.

TORCH_CHECK(!initialized_, "Attempted double initialization.");

int ffmpegStatus = avformat_find_stream_info(formatContext_.get(), nullptr);
if (ffmpegStatus < 0) {
throw std::runtime_error(
"Failed to find stream info: " +
getFFMPEGErrorStringFromErrorCode(ffmpegStatus));
}
containerMetadata_.streams.resize(0);

for (int i = 0; i < formatContext_->nb_streams; i++) {
AVStream* stream = formatContext_->streams[i];
containerMetadata_.streams.resize(containerMetadata_.streams.size() + 1);
auto& curr = containerMetadata_.streams.back();
curr.streamIndex = i;
curr.mediaType = stream->codecpar->codec_type;
curr.codecName = avcodec_get_name(stream->codecpar->codec_id);
curr.bitRate = stream->codecpar->bit_rate;
StreamMetadata meta;

TORCH_CHECK(
i == stream->index,
"Our stream index, " + std::to_string(i) +
", does not match AVStream's index, " +
std::to_string(stream->index) + ".");
meta.streamIndex = i;
meta.mediaType = stream->codecpar->codec_type;
meta.codecName = avcodec_get_name(stream->codecpar->codec_id);
meta.bitRate = stream->codecpar->bit_rate;

int64_t frameCount = stream->nb_frames;
if (frameCount > 0) {
curr.numFrames = frameCount;
meta.numFrames = frameCount;
}

if (stream->duration > 0 && stream->time_base.den > 0) {
curr.durationSeconds = av_q2d(stream->time_base) * stream->duration;
meta.durationSeconds = av_q2d(stream->time_base) * stream->duration;
}

double fps = av_q2d(stream->r_frame_rate);
if (fps > 0) {
curr.averageFps = fps;
meta.averageFps = fps;
}

if (stream->codecpar->codec_type == AVMEDIA_TYPE_VIDEO) {
containerMetadata_.numVideoStreams++;
} else if (stream->codecpar->codec_type == AVMEDIA_TYPE_AUDIO) {
containerMetadata_.numAudioStreams++;
}

containerMetadata_.streams.push_back(meta);
}

if (formatContext_->duration > 0) {
containerMetadata_.durationSeconds =
ptsToSeconds(formatContext_->duration, AV_TIME_BASE);
}

if (formatContext_->bit_rate > 0) {
containerMetadata_.bitRate = formatContext_->bit_rate;
}

int bestVideoStream = getBestStreamIndex(AVMEDIA_TYPE_VIDEO);
if (bestVideoStream >= 0) {
containerMetadata_.bestVideoStreamIndex = bestVideoStream;
}

int bestAudioStream = getBestStreamIndex(AVMEDIA_TYPE_AUDIO);
if (bestAudioStream >= 0) {
containerMetadata_.bestAudioStreamIndex = bestAudioStream;
}

initialized_ = true;
}

std::unique_ptr<VideoDecoder> VideoDecoder::createFromFilePath(
Expand Down
2 changes: 2 additions & 0 deletions src/torchcodec/decoders/_core/VideoDecoder.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -420,6 +420,8 @@ class VideoDecoder {
std::unique_ptr<AVIOBytesContext> ioBytesContext_;
// Whether or not we have already scanned all streams to update the metadata.
bool scanned_all_streams_ = false;
// Tracks that we've already been initialized.
bool initialized_ = false;
};

// --------------------------------------------------------------------------
Expand Down
Loading