Introduction Over the past year, agentic AI frameworks have grown rapidly, but many of them are designed with large models in mind. This creates friction when developers want to run smaller models locally, either for cost efficiency, privacy, or speed. After experimenting with OpenClaw, NanoClaw, PicoClaw, and Nanobot, we decided to build our own agent: MrCrab.
Why Small Models Matter
- Local deployment without dependency on cloud quotas.
- Lower hardware requirements, making AI accessible to more teams.
- Faster iteration cycles for debugging and prototyping.
- Privacy and compliance advantages when data never leaves your infrastructure.
Design Principles of MrCrab
- Tool Registry: Instead of injecting a massive tool list into every prompt, MrCrab allows the agent to query tools dynamically by keyword.
- Hybrid Memory: Recent turns are kept in context, older ones are summarized, and full logs are stored in persistent memory for retrieval on demand.
- Backend Flexibility: MrCrab integrates with Ollama, AnythingLLM, and any provider compatible with the OpenAI API.
- Lightweight Prompts: Optimized for small models like Gemma4:e2b, Qwen3.5:2B, Granite4:1B.
Implementation Highlights
- Modular architecture written with simplicity in mind.
- Debugging and logging designed to be transparent.
- Easy integration with local or cloud‑based LLMs.
Lessons Learned
- Large prompts and tool lists overwhelm small models.
- Timeout handling must be explicit when working with local inference.
- Summarization should be progressive, not premature.
Future Work
- Extending MrCrab to real business use cases, such as community management systems.
- Adding support for multi‑agent collaboration.
- Exploring long‑context training for small models.
MrCrab is our attempt to make agentic AI practical for small models. By focusing on lightweight prompts, dynamic tool discovery, and hybrid memory, we believe it can bridge the gap between experimental frameworks and production‑ready agents.
MrCrab is written in PHP, without third‑party libraries. This design choice minimizes supply chain risks and ensures that the agent can be deployed in a secure and portable way. Developers can run MrCrab locally with minimal setup, while still benefiting from integrations with Ollama, AnythingLLM, and OpenAI‑compatible APIs.
