Skip to content

Blog

Practical write-ups on self-hosted AI, Linux systems and DevSecOps, from work on real products.

Self-hosted LLMs · · 13 min readServing LLMs in production with vLLM: a practical guideDeploy vLLM as an OpenAI-compatible server: Docker setup, the flags that control memory and concurrency, metrics to watch and mistakes to avoid.Linux and ISO builds · · 13 min readWriting systemd services: unit files, restarts and sandboxingWrite a systemd service unit for your own program: service types, restart policy, dependencies, credentials, sandboxing options and debugging.Web engineering · · 5 min readHow this site is built: a static Next.js blog with MarkdownThe design of this site: Next.js static export, Markdown articles rendered at build time, a flash-free theme switch and build checks for thin pages.Linux and ISO builds · · 13 min readSSH hardening: keys, sshd_config and fail2ban done properlySet up SSH keys, harden sshd_config with a drop-in file, restrict who can log in and add fail2ban, with checks that prevent locking yourself out.Self-hosted LLMs · · 13 min readSecuring a self-hosted LLM API: auth, TLS and rate limitsProtect a self-hosted LLM endpoint in layers: private binding, an nginx reverse proxy with TLS, per-client API keys, route allowlists and rate limits.DevSecOps · · 13 min readSecrets management for small teams: from .env files to a vaultA staged path for handling passwords and API keys: stop leaks, encrypt secrets in Git with SOPS and age, deliver them safely, and know when a vault is worth it.Self-hosted LLMs · · 11 min readHow to run an LLM locally with Ollama: a complete setup guideInstall Ollama on Linux, pull a model, call its API, tune context length and memory, run it as a hardened service and fix the usual problems.Web engineering · · 15 min readREST API design best practices: naming, errors and versioningDesign an HTTP API that clients can rely on: resource naming, correct methods and status codes, a standard error format, pagination and safe versioning.RAG and AI agents · · 13 min readRAG chunking strategies: how to split documents that retrieve wellCompare chunking strategies for RAG, from fixed-size and recursive splitting to structure-aware and contextual chunks, and learn how to choose by measuring.RAG and AI agents · · 14 min readPrompt injection: how it works and how to defend your appHow direct and indirect prompt injection work, why filters alone fail, and the layered controls that limit what an injected LLM app can do.Self-hosted LLMs · · 11 min readSelf-host a private ChatGPT with Open WebUI and OllamaDeploy Open WebUI with Ollama using Docker Compose, set up accounts, add documents, put it behind HTTPS and keep it backed up and updated.Self-hosted LLMs · · 12 min readOllama vs llama.cpp vs vLLM: which should you self-host with?A practical comparison of Ollama, llama.cpp and vLLM: how each loads models, uses memory and handles concurrency, with commands to try all three.Linux and ISO builds · · 13 min readnftables for beginners: a practical Linux firewall guideBuild a default-deny nftables firewall for a Linux server: tables, chains, sets, rate limits, safe testing over SSH and making rules persistent.RAG and AI agents · · 13 min readModel Context Protocol (MCP) explained with a working serverHow MCP connects AI applications to tools and data: the architecture, the messages on the wire, and a Python server you can run and test.Self-hosted LLMs · · 12 min readOne API for every model: setting up a LiteLLM gatewaySet up a LiteLLM proxy in front of vLLM, Ollama and hosted models: config file, virtual keys, per-key limits, load balancing and fallbacks.Linux and ISO builds · · 12 min readLinux server hardening: a practical checklist with commandsA step-by-step hardening checklist for Debian, Ubuntu, RHEL and Rocky servers: updates, accounts, SSH, firewall, services, sysctl and auditing.Linux and ISO builds · · 14 min readLinux performance troubleshooting: what to check firstA repeatable first-ten-minutes routine for a slow Linux server: load, CPU, memory, disk and network checks, and how to read what the tools print.RAG and AI agents · · 12 min readHybrid search and reranking: better retrieval for RAGCombine keyword and vector search with reciprocal rank fusion, then rerank with a cross-encoder, using PostgreSQL, Qdrant and Python examples.Self-hosted LLMs · · 12 min readHardware for local LLMs: GPU, VRAM and RAM explainedHow to size hardware for local LLMs: estimate memory from parameter count and quantization, and decide between GPU, CPU and Apple Silicon.Self-hosted LLMs · · 12 min readGGUF and quantization explained: choosing Q4, Q5 or Q8What GGUF files are, how to read names like Q4_K_M and IQ3_M, how much memory each level needs, and how to pick or build the right quantization.RAG and AI agents · · 14 min readHow to evaluate a RAG system: metrics, test sets and toolingBuild a test set, measure retrieval with recall and MRR, score answers for faithfulness with an LLM judge, and run it all as a regression check.DevSecOps · · 13 min readDockerfile best practices: smaller, faster, safer imagesWrite Dockerfiles that build quickly and ship small images: layer ordering, build cache, multi-stage builds, base image choice and build secrets.DevSecOps · · 13 min readDocker security best practices: hardening containers properlyHarden Docker at runtime: non-root users, dropped capabilities, read-only filesystems, resource limits, safer networking and daemon settings, with checks.DevSecOps · · 13 min readDocker Compose in production: what to change before you deployTurn a development Compose file into a production one: pinned images, restart policies, health checks, resource limits, log rotation, secrets and safe updates.DevSecOps · · 13 min readA DevSecOps pipeline in GitHub Actions: test, scan, deployBuild a GitHub Actions pipeline that tests code, scans for secrets and vulnerabilities, builds and signs a container image, and deploys with approval.Web engineering · · 13 min readDeploy a static Next.js site to Cloudflare Pages, step by stepConnect a Git repository to Cloudflare Pages, set the build for a Next.js static export, add a custom domain, headers and redirects, and fix failed builds.DevSecOps · · 13 min readContainer image scanning with Trivy: find and fix vulnerabilitiesScan container images with Trivy, read the report, fix what matters, record accepted risks with expiry dates, and fail CI builds on fixable serious issues.RAG and AI agents · · 13 min readpgvector vs Qdrant vs Chroma: choosing a vector databaseCompare pgvector, Qdrant and Chroma on indexing, filtering, operations and scale, with working setup commands and queries for each one.RAG and AI agents · · 13 min readBuild a RAG pipeline from scratch in Python, step by stepBuild retrieval-augmented generation in plain Python: load and chunk documents, embed them, search by similarity and generate answers that cite sources.Linux and ISO builds · · 12 min readHow to build a custom Debian ISO with live-buildBuild a bootable, installable Debian ISO with live-build: configuration, package lists, files, hooks, the installer, testing in QEMU and rebuilds.RAG and AI agents · · 14 min readBuild an AI agent with tool calling from first principlesWrite a working AI agent loop in plain Python: define tools, handle tool calls, feed results back and add the limits that keep it safe.