Hello everyone,

I wanted to start a project with a Raspberry Pi 5 device that incorporates Granite, to demonstrate that enterprise grade models can run locally on devices that are relatively affordable(RPi 5 Kit ~ $130). 

I created a GitHub repo which stores the guide to setting up Granite3.2:2b locally on a Raspberry Pi, for a fun a DIY project!

https://github.com/Jewelzufo/granitepi

Congrats, really cool! How fast is it?

Hello everyone,

I wanted to start a project with a Raspberry Pi 5 device that incorporates Granite, to demonstrate that enterprise grade models can run locally on devices that are relatively affordable(RPi 5 Kit ~ $130). 

I created a GitHub repo which stores the guide to setting up Granite3.2:2b locally on a Raspberry Pi, for a fun a DIY project!

https://github.com/Jewelzufo/granitepi

My apologies on the delay! Just now seeing this comment. 

Granite3.3:2b worked surprisingly well running solely on the RPi 5 CPU! Response time was reasonable enough to use for creating references tables when mixing audio in music production!

Yesterday, I was experimenting with granite3-moe:1b and was pleasantly surprised with its performance compared to the 2b parameter model. I created a simple OpenWeb UI for interacting with the moe:1b model! 

Essentially, these models can be run on single board computers for around $100USD for a basic kit. 4 years ago, I would have had a good laugh at the thought of running models this powerful on such tiny devices. I have found many uses for Raspberry Pi AI systems, especially in computer vision. 

I plan on implementing Granite models for computer vision technologies, as affordably as possible. Trying my best to ensure it is accessible to those without higher end computers! 

If you are ever interested in working on some RPi + Granite projects, feel free to reach out.

Ollama + RPi has proven to be a powerful combo for me!

Congrats, really cool! How fast is it?

Hello everyone,

I wanted to start a project with a Raspberry Pi 5 device that incorporates Granite, to demonstrate that enterprise grade models can run locally on devices that are relatively affordable(RPi 5 Kit ~ $130). 

I created a GitHub repo which stores the guide to setting up Granite3.2:2b locally on a Raspberry Pi, for a fun a DIY project!

https://github.com/Jewelzufo/granitepi