github.com
Slotstream
An experimental inference tool for running mixture-of-experts language models on Macs by streaming expert weights from SSD storage.
https://github.com/carloslfu/slotstreamAbout This Resource
An experimental inference tool for running mixture-of-experts language models on Macs by streaming expert weights from SSD storage. It explores trading storage access for a lower memory requirement.