Write-through for tables

[cristi16]

Why is write-through only supported for structs and not tables?

Is this something that would be considered for a future release?

Thanks,
Cris

[jamescourtney]

It's something that I have thought about. The short version is: write through for tables has some weird edge cases that structs don't, which can lead to surprising and nonobvious outcomes. You probably know this, but the primary difference between structs and tables is that all struct fields are mandatory and are always written, while table fields (even primitives) are skipped if they match their default value.

There is no technical reason that it won't work. However, one of the things I really think about when adding something is: "Would this surprise me if I didn't know how this worked under the hood?" and "Can I imagine a way to use this in which I might be surprised?"

Here's an example of something I consider surprising:

// Table members have implicit default values even when not specified. In this case, the default Value
// of 'Value' is 0.
table FancyTable { Value : int; }

FancyTable three = new() { Value = 3 };
FancyTable zero = new() { Value = 0 };

byte[] serializedThree = FlatBufferSerializer.Default.Serialize<FancyTable>(three);
FancyTable parsedThree = FlatBufferSerializer.Parse<FancyTable>(serializedThree);

byte[] serializedZero = FlatBufferSerializer.Default.Serialize<FancyTable>(zero);
FancyTable parsedZero = FlatBufferSerializer.Parse<FancyTable>(serializedZero);

// This works, because 3 is not the default value of 0, so there is space in the buffer for it to be written.
parsedThree.Value = 10;

// This fails because FlatBuffers implemenations simply skip a table field when it matches its default value.
// So there's no space for the value to go. FlatSharp would be forced to throw an exception here.
parsedZero.Value = 10; 

Since FlatSharp does everything with property overrides and inheritance, there's no good mechanism I have to add "TryWriteThrough" methods, since the only thing you as a developer see is the base class, and the subclass is the one doing all the interesting work.

The way I've personally worked with this is to just define some simple wrapper structs:

struct MutableInt (fs_writeThrough) { Value : int32; }
table FancyTable
{
      Value : MutableInt;
}

The serialized output of this is identical to the above, though in this case if the MutableInt struct is not null, then you are guaranteed that your write through will succeed, and the library behaves predictably. The main penalty you pay in this case is an extra object allocation for MutableInt (which I agree isn't great). There is an open issue with value-type structs not supporting writethrough (#201), but it's still in the brainstorming phase of work at the moment. I haven't found a design for that one that I'm on board with quite yet.

Hope this helps.

[jamescourtney]

FlatSharp 6.0 (upcoming release) will support write through in some additional scenarios:

• Vectors of Structs (both reference and value structs).
• Struct table members that are marked as required (ie, guaranteed to exist in the underlying buffer). Both table and value structs.

[jamescourtney]

FlatSharp 6.0.0-beta2 is published with support for value-type struct writethrough. The supported scenarios are:

• Value structs (fs_valueStruct) support write through within tables and within vectors (declare fs_writeThrough on the table field or vector).
• Reference structs support "internal writethrough", but not through their parent object (declare fs_writeThrough inside a struct).

Full details are here: #231
Changelog for 6.0.0 is here: #216