Why the Old Articles Are Important

The vast majority of automated blogs all sound the same, especially if they use Claude. Additionally, while AI is pretty accurate these days, it can still be wrong, especially when diving deep into a topic.

The advantage I had with Garage Dreams was that I had spent a ridiculous amount of time going through forums, social media groups, blogs and YouTube videos to find out as much as I could about each car. The Eastern European DIY fix-it videos were always a pot of gold.

This meant that I already had a load of researched content I could use, which proved particularly important for the lightweight model I eventually ran on the Raspberry Pi. It still isn’t a guarantee that everything will be correct, but it gives the model something solid to work from rather than asking it to write an article from memory.

How does the model get the content?

The pipeline used for this automated version of Garage Dreams never asks the local model for facts or information about a particular car or topic. It simply takes a section of at least 400 words from one of my existing handwritten articles and feeds it to the local model.

This means I don’t have to worry as much about hallucinations and made-up facts, which can be a pretty serious issue when dealing with small local models and complex topics. The model can still misunderstand something or drop an important detail, but at least there is an original source that the generated article can be checked against.

Once an initial draft is made, I get Claude and Codex to automatically review the generated article. They each give it a score and a verdict, and if both pass, the article is published.

If either reviewer asks for changes, the local model gets one chance to adjust the article based on their feedback. The revised draft is then checked and reviewed again. If either reviewer rejects it, or they still don’t both pass it after the revision, the article is skipped for the night.

I noticed during testing that allowing more revisions usually made the writing worse. The model would fix one problem and then introduce another, and the article could wind up going around in circles. Limiting it to one review revision seemed to work better.

Can generated articles be used as a source?

No, they can’t with this setup. One thing I made sure to include was a tag in the frontmatter of each article that controls whether it can be used as source material (source: true or source: false).

The original published guides have source: true, while the shorter generated articles have source: false. I didn’t want generated articles being used to generate future articles, as I think the quality would drop enormously over time.

I do plan to add generated buyer’s guides using larger models such as Codex, ChatGPT and Claude, but that isn’t set up yet. Those guides will only be given source: true after I have reviewed and approved them myself.

You can see a section of the code below showing how this works. It essentially takes a post, checks whether the source field is true or false, makes sure it is a guide and then checks that it is not a draft.

/** Eligible to seed derivative articles: a published, human-or-reviewed guide. */
export function isSourcePost(d: SourceCandidate): boolean {
    return d.source && d.type === 'guide' && !d.draft;
}

Traceability setup

Every article records the guide and exact heading it came from. Every completed editorial run also commits its log, whether it publishes or not, in the same commit as the article when there is one. This helps me keep track of whether an article or image has been published, rejected or if the system didn’t run at all.