Array split does not result in sub arrays with equal size: attempting 3 splits along axis 2 for shape

[jrp2014]

Looks like there is a bug in a recent version of mlx? This never happened before, but perhaps here was just a hidden bug:

mlx version: 0.22.0.dev20250201+80c863b9
mlx-vlm version: 0.1.12

vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
Running mlx-community/phi-4-4bit
Fetching 9 files: 100%|███████████████████████████████████████████████████████████████| 9/9 [00:00<00:00, 22550.02it/s]
ERROR:root:Model type phi3 not supported.
Failed to load model or config at mlx-community/phi-4-4bit: Model type phi3 not supported.
================================================================================

vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
Running mlx-community/pixtral-12b-8bit
Fetching 11 files: 100%|████████████████████████████████████████████████████████████| 11/11 [00:00<00:00, 11255.76it/s]
Fetching 11 files: 100%|█████████████████████████████████████████████████████████████| 11/11 [00:00<00:00, 6591.05it/s]
==========
Image: ['/Users/jrp/Pictures/Processed/20250201-163734_DSC02441-Edit.jpg'] 

Prompt: <s>[INST][IMG]Provide a factual caption, description and comma-separated keywords or tags for this image so that it can be searched for easily.[/INST]
Failed to generate output for model at mlx-community/pixtral-12b-8bit: Array split does not result in sub arrays with equal size: attempting 3 splits along axis 2 for shape (1,3,688,1024).
********************************************************************************

vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
Running mlx-community/pixtral-12b-bf16
Fetching 13 files: 100%|████████████████████████████████████████████████████████████| 13/13 [00:00<00:00, 14206.87it/s]
Fetching 13 files: 100%|████████████████████████████████████████████████████████████| 13/13 [00:00<00:00, 10260.81it/s]
==========
Image: ['/Users/jrp/Pictures/Processed/20250201-163734_DSC02441-Edit.jpg'] 

Prompt: <s>[INST][IMG]Provide a factual caption, description and comma-separated keywords or tags for this image so that it can be searched for easily.[/INST]
Failed to generate output for model at mlx-community/pixtral-12b-bf16: Array split does not result in sub arrays with equal size: attempting 3 splits along axis 2 for shape (1,3,688,1024).
********************************************************************************

[Blaizzy]

That's odd

I just ran the smoke test and Pixtral passed

[Blaizzy]

Here is the updated image with the latest official pip release and not the dev version

[neilmehta24]

I saw this issue too in #195. @Blaizzy This failure happens when using the dev build of transformers 4.49.0.dev0 . It might also be related to the image size or ratio. I noticed that mlx_vlm.generate failed this way for a specific image:

python -m mlx_vlm.generate --model ~/.cache/lm-studio/models/mlx-community/pixtral-12b-4bit --max-tokens 100 --temp 0.0 --image https://raw.githubusercontent.com/lmstudio-ai/mlx-engine/501e61df1ced33f7002b3b2a036210686c1be713/demo-data/toucan.jpeg

[Blaizzy]

Hey guys @neilmehta24 @jrp2014

I ran the example on dev build of transformers 4.49.0.dev0 from last week and the latest mlx-vlm and it works.

However, when I reinstalled it from source and I get the same issue now. It seems transformers finally fixed the data processing for pixtral.

I have updated mlx-vlm and a PR should be up soon.

[neilmehta24]

Hey @Blaizzy any update on this? I think that qwen2.5 VL is also broken with transformers 4.49.0

[Blaizzy]

Hey guys

The issue was fixed, just forgot to push it!

#216