Batch processing on Apple Silicon?

[TheJoffrey]

Hello,
When trying to use marker for batch processing on my Mac (M1), I got the following error:

Cannot use MPS with torch multiprocessing share_memory.  You have to use CUDA or CPU.  Set the TORCH_DEVICE environment variable to change the device.

I got the same error using "cpu" as TORCH_DEVICE. And it seems like CUDA is not available on macOS.

Did I miss something? Thanks!

And thanks very much for Marker!

[jeblister]

I have the same error, I use Apple Silicon M1

[bestcarly]

I'm facing a similar issue. I attempted to set TORCH_DEVICE to cpu, yet it's still utilizing mps. Could you guide me on how to correct this? Thank you for any responses.

[VikParuchuri]

I can patch this so you can use MPS for batch conversion, but it will be less efficient than with CUDA

[bestcarly]

I can patch this so you can use MPS for batch conversion, but it will be less efficient than with CUDA

This is an excellent project, thank you for your contributions. I'm supporting it by doing the following:
in models
...
def setup_order_model(device=None, dtype=None):
device = "cpu"
dtype = "auto"
...

Yes, it works but it's very slow.😭

[aifirstd3v]

It seems like supporting MPS but it is very slow IDK why.