AI infrastructure / homelab
Local AI infrastructure as technical capability.
This is not a gaming build page. It is an operations-minded environment for learning inference tradeoffs, deployment patterns, security boundaries, and practical AI service design.
Local inference
Model-family testing across Qwen 3.5, Qwen 3.6, Gemma 4, GLM 4.7 Flash, and Bonsai ternary models for understanding capability, latency, memory pressure, and task fit outside hosted APIs.
Hardware constraints
RTX 4060 Ti 16GB class testing for VRAM, context, throughput, and quantisation tradeoffs.
Mini-server stack
Docker, Caddy, vLLM, llama.cpp, LM Studio, LangChain, PostgreSQL, pgvector, pgai, ChromaDB, FAISS, reverse proxying, and public demo hosting patterns.
Security-conscious design
Local inference, restricted endpoints, public frontends, protected backends, and careful exposure boundaries.
RAG deployment
Retrieval pipelines, reranking, answer grounding, prompt-injection protection, guardrails, source constraints, and operational monitoring considerations.
Deployment constraints
The same constraints appear repeatedly: cost, risk, throughput, data boundaries, and maintainability.