How to conveniently work with binaries as vectors of bytes?

[radrow]

In my project I extensively use various byte arrays (Vec<u8>, [u8]). I have quite a number of types which comprise those as fields.

My problem is, ruslter converts those to Erlang lists, and I don't like for obvious reasons. This leaves me to these workarounds:

1. Implement almost all decoders/encoders by hand
2. Wrap all byte arrays with a newtype
3. Cry

The problem with 1 is that it's quite a tedious work. It also adds maintenance burden. Normally I would use a derive macro, but it automatically picks lists. I know the current way to go is 2, but I don't want to pollute entire code with a custom wrapper for a byte array for this one sole purpose. My project is not Erlang-first oriented. Regarding 3, I am out of tears already.

Is there any better way around it? If not, could this be a feature request?

[filmor]

Use Binary and OwnedBinary.

[NinoScript]

We're having the same issue. It would be great to be able to give a hint to the derive macro.

Something like:

struct Tree {
  #[encode_as(Binary)]
  data: Vec<u8>,
  
  nodes: Vec<Tree>
}

(We ended up implementing the encoder by hand)

[hansihe]

Since Binary/OwnedBinary implements Encode/Decode, you can do this:

struct Tree<'a> {
  data: Binary<'a>,
  nodes: Vec<Tree>
}

This has the advantage of being zero copy. The data doesn't have to be copied into a Vec<u8>, the Binary simply references the memory the BEAM owns.

If you don't want to deal with lifetimes, you can do:

struct Tree {
  data: OwnedBinary,
  nodes: Vec<Tree>
}

---

As a point of improvement, we might be able to add an attribute to the derive macro. I don't like the implication of encode_as, as it suggests it applies to encoding only. Maybe format?

struct Tree {
  #[format(Binary)]
  data: Vec<u8>,
  nodes: Vec<Tree>
}

Under the hood we could rely on From to convert from the type in the struct to the type we encode/other way around. We would require Binary: From<Vec<u8>> and Vec<u8>: From<Binary> in this case.

[filmor]

Another option is to use serde instead, see https://docs.rs/rustler/latest/rustler/serde/index.html, which by default converts Vec<u8> <=> binary(). That's especially useful, if your objects already implement Serialize and Deserialize.

[NinoScript]

Since Binary/OwnedBinary implements Encode/Decode, you can do this:

struct Tree<'a> {
  data: Binary<'a>,
  nodes: Vec<Tree>
}

We're in the same case as the original poster, that our project "is not Erlang-first oriented", the type is defined in another crate that does not use Rustler, so I think that is not the best solution for us.

I don't like the implication of encode_as, as it suggests it applies to encoding only. Maybe format?

Sure, any way to disambiguate between binary() and [non_neg_integer()] would be great, that was just the first thing that came to mind to get the idea across.

@hansihe Thank you for your time!

[NinoScript]

Another option is to use serde instead

@filmor Thank you, I'll look into it, that may help.