A space for technology notes on anything I find interesting, and a musing or two. From electrons through to delivering and developing code, as well as shiny emerging tech.
Crafting and optimising Dockerfiles can be a bit tedious with the edit-build-test cycle, especially when optimising layer caching and working with multi-stage builds. Running commands or clicking buttons on every change—where is our hot reloading?
An answer - Tilt. Tilt’s primary purpose is providing local dev environments for microservice development, like a whole Kubernetes stack or Docker Compose stack.
Tilt is an awesome, rewarding tool which I use regularly. Part of its offering is monitoring Dockerfiles for changes with automatic rebuilds and displaying this in a web UI.
It all started with a colleague asking “Could you send me a copy of a recent CV?”
In the world of consulting, it’s a fairly regular ask—albeit a bit rarer for me being non-client facing.
The cogs started turning and I thought, well this will take me an hour to collate and another few to format. Then, an idea started to form—what if I could invest a bit more time now, and reduce time spent fulfilling similar requests in the future.
Azure VPN Gateway’s SSTP Point-to-Site (P2S) connections to private VNETs can cause networking issues in WSL. Symptoms include hanging SSL connections, frozen database clients, and DB clients failing in Docker/Kubernetes networks. I encountered this with a SSL secured MySQL connection which would just hang with nothing informative. Worked fine from the Windows host.
The culprit: MTU mismatches.
The Problem MTU (Maximum Transmission Unit) is the largest packet size that can be transmitted over a network connection.
A short exploration of options for connecting to resources with private VNETs in Azure. Focused on boutique dev work. Think access to state maintaining services like SQL databases, CosmosDB, Redis etc.
Costs are indicative at time of writing. Costs vary between Azure regions. Figures are in AUD, multiply by .6 to get an approximate in USD.
I haven’t considered use of Site to Site (S2S) VPNs to connect to existing Point to Site (P2S) VPNs here, if that connectivity is a requirement then these options are likely not relevant.
This post was written almost entirely by an LLM.
A reflection on building LLM-powered writing assistance while maintaining authentic voice
The Challenge of Authentic LLM Assistance After well over a year of sporadic posting, I found myself facing a familiar challenge: maintaining consistency in voice and technical depth across content while leveraging the productivity benefits of LLM assistance. The solution emerged through an interesting meta-exercise—using Roo Code (the agentic coding plugin for VSCode/VSCodium) to create its own content creation persona.
After well over a year without a post…
Personal Update At the end of March 2024, my father died. May he rest in peace. With the process of grieving taking its natural course, this site went well on the back-burner, or even off the stove entirely for a time.
The Migration Journey Then we moved house, and my Proxmox cluster turned back into a single node. I stayed up late one night and transitioned the site’s K3s stack on to an Azure VM as an interim measure.
This post explores renting a cloud GPU from RunPod and using the vLLM inference engine to run a Large Language Model made available via an OpenAI compatible endpoint, and then load testing that endpoint with K6.
What is RunPod? RunPod is a paid cloud GPU provider. It offers:
Pods
We will utilise a pod in this example.
A pod is a container with one or more GPUs attached. We specify the docker image and the configuration.
A set of notes on converting a transformers model from Pytorch format to Safetensors format and then quantising to ExLlamaV2 (Exl2) using a code based calibration dataset.
This was inspired by posts which reported coding LLMs quantised to Exl2 format using the wikitext default calibration dataset resulted in relatively lower quality outcomes.
ExLlamaV2 is the excellent work of Turboderp. It is an inference library for running LLMs on consumer GPUs. It is fast and supports multi-GPU hosts.
A guide to using the Terraform bpg provider to create virtual machines on a Proxmox instance.
The bpg provider is a wrapper for the Proxmox API. It enables the provisioning of infrastructure on Proxmox using Terraform.
bpg is one of two terraform providers available for Proxmox at time of writing, the other option being telmate. Both are active based on their GitHub repos, at a quick glance bpg was a bit more active, and a few positive posts about bpg swayed the decision towards it.