Support size-optimal representations of enum's

[michaeljklein]

Problem

An enum such as:

enum Foo {
    VariantA(Field),
    VariantB(Field),
    VariantC(bool),
}

Is currently approximately represented as:

(FooDiscriminant, Field, Field, bool)

While the optimal representation, without knowing anything else about the usage of the type, is the following:

(FooDiscriminant, Field)

Happy Case

Represent enums by approximately:

(FooDiscriminant, [Field; max(num_fields_to_represent(t) | t <- variants(Foo))])

Where a slice [Field] is used instead of an array when any of the enum's variants include a slice or other unsized type.

Workaround

Yes

Workaround Description

Define this kind of enum manually, as a struct, without using any of the enum features, e.g. match.

Additional Context

No response

Project Impact

Nice-to-have

Blocker Context

No response

Would you like to submit a PR for this Issue?

None

Support Needs

No response

[TomAFrench]

One hurdle to this that I see is that SSA won't see that VariantC is a bool rather than a Field. I can't point to anything in particular that will break but I'm sure that something will (or be very suboptimal)

[michaeljklein]

@TomAFrench For this representation, I was thinking that it would work at the granularity of Field's, so bool would have size 1 Field.

[jfecher]

@TomAFrench yeah, I'd be worried about potentially "violating" type information there too. It could probably still work with some additional handling though, e.g. dynamically constraining to a bool then casting where appropriate where cast is just a no-op.

If that proves difficult we could start with a lesser version of this and only combine identical types. Or we could delegate to the user-directed #[repr(..)] approach mentioned in the original enum issue.