Dense 30 billion parameter models are a bit of a sweet spot. They're small enough to fit on cheap consumer GPUs and have enough knowledge to answer my beginner FreeCAD questions or write boring boilerplate code for me.

As of October 2026, I use:

• Qwen3.8 27B for tool calling tasks (e.g. agentic coding).
• Gemma4 31B for vision and conversational tasks.

At 4-bit quantisation, those models both come in under 20GB. For some inference headroom and somewhere to keep a KV cache, I want at least 24GB of VRAM.

My target is 30 inference (output) tokens/second, that's fast enough to feel about the same as the ChatGPT or Claude web experience.

How much does the hardware cost to hit that target? Surprisingly, about £500.

It's my personal mission to make my local coding model better for me than anything commercially available. The big models are always going to be more capable and my consumer grade hardware will never be a match for a data center of compute. Still, the big models have to work for everyone, my local model only has to suit my workflow.

The biggest improvement I've seen with smaller models comes from the initial steering. If I can tell it which files, modules, classes or concepts a change will impact it does a better job.

An obvious choice is to reach for an embedding model to navigate the codebase.

More of these models are claiming to work with source code, from Qwen3 Embedding's release page:

Multilingual Capability: The Qwen3 Embedding series support over 100 languages, including various programming languages, and provides robust multilingual, cross-lingual, and code retrieval capabilities.

But what does that actually mean? Does the model understand structure? Naming? What?

I ran a small experiment, and here's what I found.

Continuing from my last post, can I predict building energy ratings from features a homeowner would already know?

Linear regression with 17 features (year of construction, number of storeys, etc.) got me to 0.45 of variance explained (R²).

With just a few lines of code, I got that to 0.78.

If you've ever rented or bought a home in Ireland, you'll be familiar with Building Energy Ratings (BER). They tell you how cold you'll be in winter, graded from A1 (toasty) through to G (cold). "BER Exempt" means you're in for a rough time.

To get your BER, an assessor takes dozens of measurements, pokes at every vent, crevice and hole, then produces a number: the cost to heat the building in kWh/m²/year. That number gets bucketed into a category from A1 to G:

Rating	kWh/m²/year
A1	≤ 25
A2	25 - 50
A3	50 - 75
B1	75 - 100
...	...
E2	340 - 380
F	380 - 450
G	> 450

The Sustainable Energy Authority of Ireland (SEAI) is responsible for BER, and they handily publish an anonymised dataset.

Can I, a proven idiot, use machine learning to predict a home's BER using only information a homeowner is likely to know? No crevice exploration required?

This is a short series of posts to figure that out. What I tried, what worked, what didn't.

I want to take back control of the recommendation algorithms I'm exposed to.

Streaming platforms don't know what mood I'm in. I rarely know myself to be honest, it's usually just a vague sense of what I don't want.

Sometimes I want something challenging, the kind of film Cillian Murphy has gushed about in an interview, where I'm left thinking about the central question of the narrative for days. Sometimes I want helicopters and explosions and exploding helicopters.

I usually want more queer media. Don't get me wrong, some of my best friends are straight, but more and more I'm just not in the mood for a hetero B-plot.

Can I build something that answers: Films like Speed 2, but gay?

Direct Preference Optimisation (DPO) lets you shape an LLM's behaviour.

DPO is used for tasks like:

• Refusals and safety mechanisms: don't say harmful stuff, don't tell people how to do bad things
• Tone and style alignment: be concise, or talk in a certain way
• Decontamination: make a model "unlearn" specific content, like copyright material or personal information
• Format compliance: prefer valid JSON over word salad

The big AI corps use RLHF (Reinforcement Learning from Human Feedback) and a ton of compute to do this.

How far can I get on a €300 second-hand RTX 3060 with 12GB of VRAM and some questions about dogs?

This is part of a series on ML for generalists, you can find the start here.

On my machine, a dataset of 2,000 training samples taking about 10 minutes (for a single epoch) on my CPU. That's not terrible for one run, but I need to iterate and experiment. I might change a hyperparameter (like learning rate) or tweak the architecture to add more layers, or bigger layers, or... you get the idea. Waiting around makes that painful.

Ten minutes per epoch means I can't realistically try more than a handful of changes in a session. Context switching is rough. Kick off a training run, go do something else, come back, check the results, try remember what I was testing, hopefully I scribbled a note somewhere. Repeat. I can only change one variable at a time if I want to learn what's happening, so my progress crawls along.

Enter GPUs.

This is part of a series on ML for generalists, you can find the start here.

Backprop answers one question for every weight in your network: "how would the loss change if I tweaked this weight slightly?"

The answer is called the gradient. Once you have it, training is simple: nudge each weight in the direction that reduces the loss.

That's it! The rest is just working out the maths efficiently.

This is part of a series on ML for generalists, you can find the start here.

We've trained our model, it's ready to use. How do we do that?

This is part of a series on ML for generalists, you can find the start here.

We know the data is the important part. It can take a lot of work to compile an accurately labelled dataset. How do we make sure we get the most out of what we have?