-
Notifications
You must be signed in to change notification settings - Fork 5.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Multi-image masking for single IP Adapter #7499
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Changes looking nice ❤️ Great work as always. Let's add a fast test and see how it goes?
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
thank you @sayakpaul ready |
@@ -544,3 +544,33 @@ def test_ip_adapter_multiple_masks(self): | |||
|
|||
max_diff = numpy_cosine_similarity_distance(image_slice, expected_slice) | |||
assert max_diff < 5e-4 | |||
|
|||
def test_ip_adapter_multiple_masks_one_adapter(self): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is slow test. I was wondering if it's possible to add a fast test like
def test_ip_adapter_single(self, expected_max_diff: float = 1e-4, expected_pipe_slice=None): |
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see, I missed that
I'll add it now
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thank you for working on this!
looks good. I left some feedbacks. let's make very clear what's the expected shape for ip_adapter_mask
and scale
Co-authored-by: YiYi Xu <yixu310@gmail.com>
@yiyixuxu @sayakpaul now it's ready |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
looks good to me!
I will run slow tests to make sure it does not break anything before merge
thank you YiYi! I fixed the fast test for the panorama pipeline that was failing and the slow test I ran all the slow tests related to image masking locally and they didn't break :) |
diffusers commit a0cf607 Multi-image masking for single IP Adapter (huggingface/diffusers#7499)
What does this PR do?
Image masking is currently supported by passing a mask and an image for each IP Adapter in the pipeline.
This PR extends this usage by allowing multiple masks and images to be passed to a single IP Adapter. Scales must be passed accordingly.
Discussed in #7419
@sayakpaul @yiyixuxu