Issue with IQueryable Projection in Combination with Sieve/QueryKit

[Seszele]

Hi, I was trying to filter with Sieve and QueryKit mapping my model to Dto with IQueryable projection like this:

var offers = _dbContext.ExchangeOffers.AsNoTracking().ToDto().ApplyQueryKitFilter(filterInput);

where ToDto is a projection method:

public static partial IQueryable<ExchangeOfferDto> ToDto(this IQueryable<ExchangeOffer> exchangeOffer);

But If I use the code generated by Mapperly I get:
Failed to fetch and filter exchange offers. Reason: The LINQ expression 'DbSet<ExchangeOffer>().Where(........' could not be translated. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly by inserting a call to 'AsEnumerable', 'AsAsyncEnumerable', 'ToList', or 'ToListAsync'.

Upon further investigation, I found that the issue can be traced back to the null-check for the collection property, specifically:

x.WantedPublishedBooks != null ? ... :default
and
x.OfferedBooks != null ? ... : default

Mapperly checks for null by default and i can't find a way to change this behavior. When I manually write the mapping to the DTO without this null-check, everything functions correctly.

So to summarize this manual projection works and i can filter it correctly:

 public static IQueryable<ExchangeOfferDto> ToDto(this IQueryable<ExchangeOffer> exchangeOffer)
     {
         return Queryable.Select(exchangeOffer, x => new ExchangeOfferDto()
         {
             Id = x.Id,
             Points = x.Points,
             DateCreated = x.DateCreated,
             Status = x.Status,
             OfferedBooks = Enumerable.ToList(Enumerable.Select(x.OfferedBooks, x1 => new UserBookDto()
             {
                 Id = x1.Id,
                 UserId = x1.UserId,
                 PublishedBookId = x1.PublishedBookId,
                 IsAvailable = x1.IsAvailable,
             })),
             WantedPublishedBooks =  Enumerable.ToList(Enumerable.Select(x.WantedPublishedBooks, x1 => new DetailedPublishedBookDto()
             {
                 Id = x1.Id,
                 ISBN = x1.ISBN,
                 Title = x1.GeneralBook.Title,
                 CoverImageURL = x1.CoverImageURL,
                 PublicationYear = x1.PublicationYear,
                 Publisher = x1.Publisher,
                 Author = x1.GeneralBook.Author,
             }))
         });
     }

And this one causes the error - the only difference is the x.WantedPublishedBooks != null and x.OfferedBooks != null which are inserted by mapperly

 public static IQueryable<ExchangeOfferDto> ToDto(this IQueryable<ExchangeOffer> exchangeOffer)
{
return Queryable.Select(exchangeOffer, x => new ExchangeOfferDto()
         {
             Id = x.Id,
             Points = x.Points,
             DateCreated = x.DateCreated,
             Status = x.Status,
             OfferedBooks = x.OfferedBooks != null ? Enumerable.ToList(Enumerable.Select(x.OfferedBooks, x1 => new UserBookDto()
             {
                 Id = x1.Id,
                 UserId = x1.UserId,
                 PublishedBookId = x1.PublishedBookId,
                 IsAvailable = x1.IsAvailable,
             })) : default,
             WantedPublishedBooks = x.WantedPublishedBooks != null ? Enumerable.ToList(Enumerable.Select(x.WantedPublishedBooks, x1 => new DetailedPublishedBookDto()
             {
                 Id = x1.Id,
                 ISBN = x1.ISBN,
                 Title = x1.GeneralBook.Title,
                 CoverImageURL = x1.CoverImageURL,
                 PublicationYear = x1.PublicationYear,
                 Publisher = x1.Publisher,
                 Author = x1.GeneralBook.Author,
             })) : default
         });
}

Any ideas why that might be causing issues? Is it possible for Mapperly to avoid generating such null-checks for collection properties ?

My Dto:

public class ExchangeOfferDto
{
    public int Id { get; set; }
    public int Points { get; set; }
    public DateTime DateCreated { get; set; }
    public ExchangeOfferStatus Status { get; set; }
    public UserDto? User { get; set; }
    public ICollection<UserBookDto>? OfferedBooks { get; set; }
    public ICollection<DetailedPublishedBookDto>? WantedPublishedBooks { get; set; }
}

[latonz]

Are your objects in a disabled nullable context?

[Seszele]

Yes I have both ExchangeOfferDto and ExchangeOffer in #nullable disable as well as all the classes that I directly reference like UserBookDto etc.
Do I have to make all classes that are in some way related to the class that i want to map not nullable?
(for example ExchangeOffer <- User <- UserBook <- ExchangeRequest <- .... do I have to make all those classes be in disabled nullable context?)

I also have ef core generated join table ExchangeOfferUserBook but i dont see how that would cause this issue.

full exception message

Failed to fetch and filter exchange offers. Reason: The LINQ expression 'DbSet() .Where(e => MaterializeCollectionNavigation( Navigation: ExchangeOffer.OfferedBooks, subquery: DbSet>("ExchangeOfferUserBook") .Where(e0 => EF.Property(e, "Id") != null && object.Equals( objA: (object)EF.Property(e, "Id"), objB: (object)EF.Property(e0, "ExchangeOffersId"))) .Join( inner: DbSet(), outerKeySelector: e0 => (object)EF.Property(e0, "OfferedBooksId"), innerKeySelector: u => (object)EF.Property(u, "Id"), resultSelector: (e0, u) => new TransparentIdentifier, UserBook>( Outer = e0, Inner = u )) .Select(ti => ti.Inner)) != null ? DbSet>("ExchangeOfferUserBook") .Where(e1 => EF.Property(e, "Id") != null && object.Equals( objA: (object)EF.Property(e, "Id"), objB: (object)EF.Property(e1, "ExchangeOffersId"))) .Join( inner: DbSet(), outerKeySelector: e1 => (object)EF.Property(e1, "OfferedBooksId"), innerKeySelector: u0 => (object)EF.Property(u0, "Id"), resultSelector: (e1, u0) => new TransparentIdentifier, UserBook>( Outer = e1, Inner = u0 )) .Join( inner: DbSet(), outerKeySelector: ti0 => EF.Property(ti0.Inner, "PublishedBookId"), innerKeySelector: p => EF.Property(p, "Id"), resultSelector: (o, i) => new TransparentIdentifier, UserBook>, PublishedBook>( Outer = o, Inner = i )) .LeftJoin( inner: DbSet(), outerKeySelector: ti0 => EF.Property(ti0.Inner, "GeneralBookId"), innerKeySelector: g => EF.Property(g, "Id"), resultSelector: (o, i) => new TransparentIdentifier, UserBook>, PublishedBook>, GeneralBook>( Outer = o, Inner = i )) .Select(ti0 => ti0.Outer.Outer.Inner == null ? null : new UserBookDto{ Id = ti0.Outer.Outer.Inner.Id, UserId = ti0.Outer.Outer.Inner.UserId, PublishedBookId = ti0.Outer.Outer.Inner.PublishedBookId, IsAvailable = ti0.Outer.Outer.Inner.IsAvailable, BookCondition = ti0.Outer.Outer.Inner.BookCondition, PublishedBook = ti0.Outer.Inner != null ? new DetailedPublishedBookDto{ Id = ti0.Outer.Inner.Id, ISBN = ti0.Outer.Inner.ISBN, Title = ti0.Inner.Title, CoverImageURL = ti0.Outer.Inner.CoverImageURL, PublicationYear = ti0.Outer.Inner.PublicationYear, Publisher = ti0.Outer.Inner.Publisher, Author = ti0.Inner.Author } : null } ) .ToList() : null .Select(y => y.Id) .Any(z => z == 1))' could not be translated. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly by inserting a call to 'AsEnumerable', 'AsAsyncEnumerable', 'ToList', or 'ToListAsync'. See https://go.microsoft.com/fwlink/?linkid=2101038 for more information.

[latonz]

Nah I think you are good, the problem here is that since these properties are in a nullable disabled context, Mapperly assumes they are nullable and generates the not-null condition. EF core seems to fail to translate non-null conditions on collections (which don‘t make sense from an EF core point of view). As a workaround you can probably enable nullability for the collection properties and set them to non-nullable. I‘ll create an issue that Mapperly should not generate this non-null condition in projection mappings for collections.

[Seszele]

Thanks for help.
Setting both destination and source collection properties as not-nullable worked.

[latonz]

Created #867