Hello

[luk-f-a]

hi @mdekstrand , I googled numba csr and this showed up, 18 hours after creation. what a nice coincidence! Thanks for sharing this library.

I see you based this on jitclasses. They will be eventually replaced by the new StructRef (https://numba.pydata.org/numba-doc/dev/extending/high-level.html#implementing-mutable-structures). If you are starting a new library maybe it makes sense to base it on those. Jitclasses won't go away any time soon, so it's also fine to keep it as it is.

Cheers,
Luk

[mdekstrand]

Thank you, I will take a look at that! I thought I had remembered seeing something like it, but wasn't finding it again in the docs when I reviewed. I'll at least create a branch to prototype using it.

I initially built this out with namedtuples to try for more stability, but the performance was rather bad.

One open question that will need a little thought is the compilation overhead - will the structref's methods be JITed at import time, or lazily when used? The predecessor to this package was a jitclass in LensKit itself, and the overhead of compiling the jitclass with methods at import time caused some obnoxious performance hits in multiprocessing code that didn't use it. I jumped through some weird hoops to fix that, and am addressing it here by having the jitclass not actually have any methods to speak of.

[luk-f-a]

I don't have a lot of experience with StructRef but I just ran the example in the documentation (the link above) and it seems that the compilation happens lazily like with njit functions.

[....code from example....]
alice = MyStruct("Alice", vector=np.random.random(3))
print(MyStruct_get_name.signatures) # returns empty list, which means that it hasn't been compiled.
print(alice.name) # 'Alice'
print(MyStruct_get_name.signatures) # [(numba.MyStructType(('name', unicode_type), ('vector', array(float64, 1d, C))),)]

The compilation only seems to take place after the first call to .name. I think this happens because it has been implemented as an njit function which gets called from functions, and therefore follows the normal lazy logic of njit functions.

[mdekstrand]

Fantastic. It also looks like structref might enable me to skip the explicit .R dereference to get a Numba representation, if CSR itself is a proxy for the structref. That would be very nice.

I will be looking more closely at this.

[mdekstrand]

I've got the code prototyped, and so far I'm quite happy with structref. I'm currently running the benchmark suite to make sure it doesn't hurt performance, and if we still get good performance I think I'll merge it in to main.