Raspberry Pi 5 NAS & Edge AI Server

A low-power system combining networked storage, secure remote access, and on-device AI inference.

System Architecture

This project involved designing and deploying a low-power NAS and application server using a Raspberry Pi 5.
I picked the RPi5 because of its energy efficiency. The system combines networked storage, remote access, and on-device machine learning inference while operating within the performance and thermal constraints of its hardware.
An M.2 NVMe SSD provides overkill-fast storage (its only gen 2.0, but my SSD is 3.0), while the AI accelerator module enables local inference without reliance on any cloud, all remaining local.

Hardware & System Setup

The system is built around a Raspberry Pi 5 paired with a Pironman 5 Max case, which provides active cooling, power management, and M.2 expansion.
The ingredients are below:

Raspberry Pi 5 (ARM64 8GB RAM) – main compute platform running Raspberry Pi OS.
Pironman 5 Max Case – active cooling, power control, NVMe expansion and an OLED screen to show your local IP, CPU temps, storage usage, RAM and more to option, just a very handy thing.
4TB NVMe SSD – high-throughput storage for datasets, media, and backups.
AI Accelerator Module – supports efficient edge AI inference.

Storage & Networking (NAS)

The Raspberry Pi runs on RPi OS a Linux-based operating system that's very lightweight, configured to act as Network Attached Storage (NAS).
The best thing you can do for your RPi is move the OS from an SD card to an SSD, even an external one, it makes a world of a difference speed wise, and also more reliable.
This gives me:

Centralised file storage hosted on the NVMe SSD.
Network file sharing over my local network.
User and permission management for controlled access.
Always-on, low-power design suitable for home and lab environments.

Secure Remote Access

Secure access to the system from outside the local network is provided using a WireGuard VPN. This enables encrypted remote connections without exposing services or ports directly to the internet.

Encrypted access to files and services.
Consistent IP routing regardless of location.
Production-style security model used for remote admin and access to my NAS.

On-Device Image Recognition

A lightweight image recognition pipeline runs directly on the Raspberry Pi using the AI accelerator module and miniature camera. The model is a compact convolutional neural network optimised for 'limited' memory and computational power.
It's all done locally! Preserving privacy, allowing me to add more to the dataset, and with no external data transfer.

How was it done?

Image recognition is handled by a small convolutional neural network trained in PyTorch on a custom dataset of everyday objects and personal data from my space.
I also ran and customised a lightweight pretrained language model directly on the Raspberry Pi, with a focus on local inference weights rather than large-scale conversational performance.

Images are captured from the camera, preprocessed (resized and normalised), and passed to the model for inference using the AI accelerator. The model architecture and input size were intentionally kept small to remain within the system’s memory, power, and thermal constraints.
I also experimented with adapting the language model on-device using a mixture of general knowledge and personal context, just to see what I could get out of an LLM on a Raspberry Pi.

For example, I asked: “What are the trade-offs of running an LLM on constrained hardware?”
The model responded:

Running a large language model on constrained hardware like a Raspberry Pi involves several trade-offs. Memory and compute limitations mean models must be smaller or quantised, which can reduce accuracy and reasoning ability. Latency is higher compared to powerful servers, so responses take longer. On the plus side, all inference happens locally, preserving privacy and eliminating the need for cloud connectivity.

A concise response like this was generated in around 5–10 seconds, demonstrating practical on-device inference without any cloud dependency.

While the camera-based recognition is largely experimental, the core objective was to get the training, inference, and storage all on one system.

Result

The final system operates successfully as a low-power NAS, a securely accessible remote server, and a local edge AI inference device.
This project brings together storage, networking, and an LLM on a Raspberry Pi?!