Running image and speech generation at home

I generate images, short video clips and narration at home, on one graphics card. Most of what I have learned is not about model quality — the models are all astonishing and they all get better every few weeks. It is about the boring constraints around them: how long things take, how much memory they need, and whether you are allowed to use the output.

The GPU decides everything, and it decides it by VRAM

The single number that determines what you can run is video memory. Not the core count, not the generation. A card with 24GB will run things a faster card with 16GB simply cannot load.

That is why an RTX 3090 remains one of the best value cards for this work despite being several generations old. It has 24GB, it is widely supported, and the second-hand price reflects gamers wanting frame rates rather than capacity.

Quantisation moves the line but does not erase it. Running weights at fp8 rather than fp16 roughly halves the memory a model needs and, in my experience, costs very little visible quality for this kind of work. It is the difference between “this model runs” and “this model does not”, which is not a subtle trade.

Expect minutes, not seconds

The expectation set by hosted services is that you type something and an image appears. Locally, a still image is a handful of seconds and that expectation holds. Video does not.

Generating a short video segment on a 3090 takes me somewhere between four and eight minutes, depending on resolution and how many steps the sampler is doing. Chain eight of those together to make something a minute long and you are looking at the better part of an hour of continuous GPU time for one clip.

This has a design consequence that took me a while to accept: local video generation is a batch workload, not an interactive one. Once you stop trying to iterate on it live and start treating it like a build job — submit, go away, come back to a finished artefact, review it — the whole thing becomes pleasant. Fought against, it is agony.

The corollary is that scheduling matters. If one job wants four hours of GPU and another wants twenty minutes, they need to be told about each other. I ended up deliberately placing the long weekly job in the idle window between the short daily ones, which sounds obvious written down and was not obvious at the time.

Two-stage generation beats one-stage

The approach that finally worked for video was not “describe a clip and generate it”. It was:

1. Generate keyframes as still images, and iterate there. Stills are cheap — seconds, not minutes — so this is where you can actually be picky.
2. Feed the chosen frames to a first-frame/last-frame video model and let it interpolate the motion between them.

The reason this works is not technical, it is economic. Iteration belongs in the cheap stage. If you are refining a prompt by regenerating a five-minute video, you will get four attempts an hour and you will settle for the least bad one. If you refine the still and then generate once, you get what you actually wanted.

Two cheap post-processing passes are worth having as standard: frame interpolation to raise the effective frame rate, and an upscale to reach delivery resolution. Both are fast relative to generation, and generating at a lower resolution and upscaling afterwards is dramatically cheaper than generating at full size.

The licence is a real constraint, not paperwork

This is the part I would most want someone to take away.

Model quality varies less than you would think between the top open-weight options. Licences vary enormously. Some are Apache-2.0 and you can do what you like, commercially, without asking. Others are explicitly non-commercial, or research-only, or free until some revenue threshold.

I have twice chosen the second-best model on quality because the best one had a licence that was incompatible with what the output was for. Both times it was the right call and both times it was mildly annoying. If anything you generate might ever sit next to advertising, be sold, or be used to promote something, read the licence before you download forty gigabytes of weights and build a workflow around them.

The same applies, with more teeth, to voices. Text-to-speech has a second layer of rights on top of the model licence: whose voice is it. A synthetic voice cloned from a real person without consent is a legal and ethical problem regardless of what the model licence says. Stock or fully synthetic voices from a provider that grants commercial use are the boring, correct answer.

Where I did not self-host

I generate images and video locally. I do not generate speech locally, and that was a deliberate decision rather than a technical limit.

Local TTS has become genuinely good. But the gap between “good” and “does not sound like a robot on the fourth sentence” is still real, and narration is the one component where a listener notices immediately. A hosted voice API costs a trivial amount per video and removes an entire class of quality problem. The GPU is better spent on the thing that actually needs 24GB.

The general principle: self-host the thing that is expensive to buy and cheap to run badly. Buy the thing that is cheap to buy and expensive to run badly.

Sound effects are a solved problem you can solve yourself

A small thing that saved a disproportionate amount of hassle. Rather than sourcing sound effects from “free” packs — which routinely carry attribution requirements, or no-redistribution clauses that become a problem at any scale — I synthesise the simple ones. Clicks, tones, transitions, whooshes: a few dozen lines of code producing waveforms directly.

It is deterministic, which means a rebuilt machine produces byte-identical audio. It is owned outright, so the licensing question disappears. And clean synthetic tones suit an instrumentation aesthetic better than cinematic stock effects anyway. This will not work for anything organic, but for interface sounds it is strictly better than the alternative.

What it actually costs

Roughly: one graphics card, some electricity, and considerably more of your attention than you expect during setup. Running costs after that are close to zero, which is the whole point.

Whether that beats paying per generation depends entirely on volume. If you make a handful of things a month, hosted services win and it is not close. If you are producing something every day, the card pays for itself surprisingly quickly — and, more importantly, the marginal cost of trying something drops to nothing, which changes what you attempt.