Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow scaling factor of 2 or 3 for REAL-ESRGAN #1313

Open
DanielPodo opened this issue Jan 29, 2025 · 7 comments
Open

Allow scaling factor of 2 or 3 for REAL-ESRGAN #1313

DanielPodo opened this issue Jan 29, 2025 · 7 comments
Labels
state:Backlog This issue will be worked on in the future type:Enhancement New feature or request

Comments

@DanielPodo
Copy link

In previous versions of Video2X, a lot drivers allowed a smaller scaling factor. With the new version, these drivers have been replaced with a choice of only 3, and the one dubbed REAL-ESRGAN, recommended for real life videos, forces a scaling factor of 4. This is a bit excessive, and smaller scaling factors would be appreciated.

@DanielPodo DanielPodo added the type:Enhancement New feature or request label Jan 29, 2025
@github-actions github-actions bot added the state:Backlog This issue will be worked on in the future label Jan 29, 2025
@k4yt3x
Copy link
Owner

k4yt3x commented Jan 29, 2025

When you're saying "previous versions" you mean <6.0.0? The reason why Real-ESRGAN supports the scales it does it limited by the models available:

Image

As you can see, we only have a 4x model for realesrgan-plus. I tried to add the new 2x model at #1289, but the model conversion doesn't work with pnnx.

@Pete4K
Copy link

Pete4K commented Jan 29, 2025

There is a Anime Plus Model which is 2x, trained by Sudo based on Anime Plus 4x. Maybe WE could use this for the Anime Plus 2x scale.

@DanielPodo
Copy link
Author

When you're saying "previous versions" you mean <6.0.0? The reason why Real-ESRGAN supports the scales it does it limited by the models available:

Image

As you can see, we only have a 4x model for realesrgan-plus. I tried to add the new 2x model at #1289, but the model conversion doesn't work with pnnx.

Thanks for the response. I see your point, and yes, it was before 6 (I have 2.8.1 on my laptop and a newer one on my main PC but I can't recall the exact version, like 3 or 4?). In any case, I've no idea whether the anime ones will be good for real life videos though, but perhaps the suggestion by Pete4k may work?

@k4yt3x
Copy link
Owner

k4yt3x commented Jan 29, 2025

There's an official 2x model here: https://github.com/xinntao/Real-ESRGAN/releases/tag/v0.2.1

The issue is that it's in PyTorch format. I tried to convert it into ncnn which we are using, and pnnx fails to convert it (produces black frames). Someone else also tried to do the conversion with chaiNNer and it also failed. I'm not quite sure why.

@Pete4K
Copy link

Pete4K commented Jan 29, 2025

Did not someone try to send a pull request for this last Saturday?

@k4yt3x
Copy link
Owner

k4yt3x commented Jan 29, 2025

I don't think so. There was only a PR for RIFE

@Pete4K
Copy link

Pete4K commented Jan 29, 2025

Oh,ok

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
state:Backlog This issue will be worked on in the future type:Enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants