Posts

MSI Evo Prestige A13M Linux Mint 21.3 Installation Notes

A brief set of notes on getting Debian-based Linux up and running on an MSI Evo Prestige A13M (028AU) Laptop. The two aspects documented in these notes are making the hard drive available to Linux and getting the WiFi working. These did not work out of the box on first setting up this machine.

Continue reading

Deploying an OpenAI Compatible Endpoint on Runpod with vLLM and K6 Load Testing

This post explores renting a cloud GPU from RunPod and using the vLLM inference engine to run a Large Language Model made available via an OpenAI compatible endpoint, and then load testing that endpoint with K6. What is RunPod? RunPod is a paid cloud GPU provider.

Continue reading

Converting a Pytorch Model to Safetensors Format and Quantising to Exl2

A set of notes on converting a transformers model from Pytorch format to Safetensors format and then quantising to ExLlamaV2 (Exl2) using a code based calibration dataset. This was inspired by posts which reported coding LLMs quantised to Exl2 format using the wikitext default calibration dataset resulted in relatively lower quality outcomes.

Continue reading

Automating Virtual Machine Creation on Proxmox with Terraform and bpg

A guide to using the Terraform bpg provider to create virtual machines on a Proxmox instance. The bpg provider is a wrapper for the Proxmox API. It enables the provisioning of infrastructure on Proxmox using Terraform. bpg is one of two terraform providers available for Proxmox at time of writing, the other option being telmate.

Continue reading

Log Alchemy: Meaningful Kubernetes Logs with Seq and Fluent Bit

Let’s progress from checking Kubernetes logs in a terminal to using structured log data for searching, visualising and setting alerts within a web based user interface. We will use our Nginx deployment to demonstrate. Structured logging involves defining shapes for log data, most often represented in JSON using key value pairs.

Continue reading

Navigating Unplanned Downtime: Turning Challenges into Opportunities

A reflection on how to react to unplanned downtime once services are restored. The opportunity for growth and improvement is often highest during and directly after the times when complex systems behave unexpectedly. The potential for damage to stakeholder relationships is present at these times, particularly within teams or management structures.

Continue reading

Scaling GitHub Actions with Kubernetes: A Guide to ARC Deployment

Let us walk through setting up an Actions Runner Controller (ARC) for GitHub in a Kubernetes cluster. This will enable running continuous integration and continuous deployment (CI/CD) pipelines using GitHub Actions on our infrastructure, or on cloud based Kubernetes. First, we’ll introduce a bit of the terminology:

Continue reading

Streamlining Secret Management with Vault in K3s Kubernetes

This post will explore deploying Hashicorp Vault to K3s (Kubernetes distribution) using Helm and then configuring it with Terraform. This will enable us to store our secret state data in Vault and make those secrets available to our K3s resources.

Continue reading

Migrating from Docker Compose to Kubernetes (K3s)

In this post, we will look at migrating Docker Compose run services to K3s, a lightweight version of Kubernetes. K3s provides an approachable way to experience Kubernetes. It is quick to spin up and takes care of a lot of boilerplate, which suits a test environment.

Continue reading

Self-Hosted Website with Hugo, Docker, and Cloudflare Tunnels

This post will step through the process of building a Hugo-based website image using Docker in Ubuntu Linux, setting up a Cloudflare tunnel, and using a Docker Compose stack to bring up the website and Cloudflared containers. This will make a website available on the internet using an existing top-level domain.

Continue reading