The buffer protocol has been implemented

[pyscripter]

The buffer protocol has been implemented in P4D. A new demo (Demo 35) has been added, that shows how you can achieve super fast read/write access to numpy arrays using this protocol.

[CorBer]

Hi,

I recently discovered your great work on python4Delphi. As I am a Linux/Lazarus/FPC user it would be handy if your announcements clarify if new features are also available for the FPC. Ive tried to run the Demo35 but it fails on FPC (even after changing the inline variants). Nothing unexpected but adding this in the announcement can help users. Keep up the good work !

cheers
Cor

---

waveSharp Free Image sharpen Software
https://github.com/CorBer/waveSharp

TeensyBat Bat Detector
https://github.com/CorBer/teensy_batdetector

RegiStax Free Image Processing Software
http://www.astronomie.be/registax

---

[pyscripter]

The buffer protocol stuff is FPC compatible. Although Demo35 may not work in FPC out of the box, it can be easily made FPC compatible, by rewriting the loop involving the VarPyIterate as a standard for loop.

There is a Wiki page Supported Platforms discussing the level of FPC support in general.

The master branch of FPC contains fixes to the variants issue, anonymous functions and importantly an Rtti unit. When it gets released, I will revisit the level of FPC compatibility, since it may well be possible to bring FPC support much closer to the Delphi support.

And by the way, I am looking for a volunteer to convert at least some of the tests (in the Tests subdirectory) to FPC, so that we know exactly what it does and what it doesn't work in FPC. Is it something you can help with?

[CorBer]

Hi,

I have tried to use that demo but cannot get this line working
arr := np.array(BuiltinModule.range(N));
the compiler does not accept the array part of np.array.
Error: Syntax error, "identifier" expected but "ARRAY" found.

And on your request for a volunteer, you contact me on registax at gmail dot com as I am interested to help.

Cor

[pyscripter]

Here is an FPC translation of Demo35 that works.
Buffer protocol.zip

I will email you regarding the tests,

[pyscripter]

Added Demo35 to FPC demos.

[CorBer]

Output (only needed to change the dll to a linux library)

Lazy but slow:
Sum from 0 to 100000 = 4999950000
Elapsed ms: 435

Using Py_Buffer:
Sum from 0 to 100000 = 1249975000
Elapsed ms: 0

Successfully modified the numpy array using Py_buffer

[pyscripter]

On Windows I get:

Lazy but slow:
Sum from 0 to 100000 = 4999950000
Elapsed ms: 20764120000

Using Py_Buffer:
Sum from 0 to 100000 = 4999950000
Elapsed ms: 17350000

Successfully modified the numpy array using Py_buffer

The sum is the same.

I think on Linux the buffer contains Int64 values. Could you please check whether casting to an Int64 array in line 100 resolves the issue?

[CorBer]

Lazy but slow:
Sum from 0 to 100000 = 4999950000
Elapsed ms: 458

Using Py_Buffer:
Sum from 0 to 100000 = 4999950000
Elapsed ms: 0

Successfully modified the numpy array using Py_buffer

TIntArray = array[0..N - 1] of int64;

[CorBer]

That speed is very promissing BTW. I hope this scales for larger matrices also. The project I am trying to use this in uses large matrices (images of 5000x4000 pixels) that get processed with openCV in python (using P4D) and I currently get the results back by saving a file. I read that file back into Lazarus.