Hacker News new | past | comments | ask | show | jobs | submit login

Running it on a MacBook with M1 Pro chip and 32 GB of RAM is quite slow. I expected to be as fast as phi4 but it's much slower.



With eval rate numbers:

- phi4: 12 tokens/s

- mistral-small: 9 tokens/s

On Nvidia RTX 4090 laptop:

- phi4: 36 tokens/s

- mistral-small: 16 tokens/s




Consider applying for YC's Summer 2025 batch! Applications are open till May 13

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: