I am an advocate for gaming history and archiving – both retro and modern. I love the fact we have places like archive.org for preserving BIOS and ROM sets for many systems and consoles.

Artwork is also incredibly important. There are a number of good sources: ScreenScraper is one I have supported and used for a long time, and there are also data and artwork collections at places such as Lemon64 and LaunchBox.

Sometimes collecting the metadata is easy. The scraper software in RetroPie is good: it connects us to these services and helps match the games in a library. But what happens when filenames contain all the extra characters and classification tags commonly found in archives such as TOSEC? What happens when several releases have almost the same name, or when a technically successful match is simply the wrong game?

We are living through exciting, interesting, and exceptional times in computing, especially with the advent of artificial intelligence, the rise of large language models, and agents such as Codex.

I can hear your eyeroll – please stick with me and hear me out on this one.

My thoughts on AI generally

I have been heavily involved in the AI space for the past two years. I have participated in the development of LLM-based solutions, I have a good grasp of how to use an LLM as a tool, and I recognize the importance of keeping a human in the loop.

I have enough experience with prompt engineering to direct an agent toward what I want, recognize when it has misunderstood me, and steer the work back on course. Understanding the limitations is just as important as understanding the capabilities. I firmly believe that people who develop real experience with AI will be better placed to succeed in the future.

The difference between the amount of work I can complete in a day with and without an LLM is astounding.

I will write more about my thoughts on AI, and I have several related ideas percolating for my YouTube channel. I understand the fears—very much so. I also shake my head when I see these tools being used carelessly, or judged only by the results of using them carelessly.

I use LLMs and agents as a way to improve my own knowledge and capabilities:

• I have learned games development at a deeper level for the Playdate and Adafruit MacroPad platforms
• I use LLMs to help refine emails and make them sound more professional. I never simply use the generated text verbatim; I merge useful ideas into my own version.
• I have been learning to manage Linux more effectively through work on my homelab and this arcade cabinet.
• I have used agents to help debug difficult issues, especially by analyzing logs that might otherwise have taken me hours to correlate—for example, determining the exact cause of a Windows 11 blue screen.

This RetroPie metadata project is another example, but it goes beyond asking a chatbot a question or requesting a short script. All of the supporting code was developed through agentic work under my direction.

What “written using AI” means in this project

I described the problem, the behavior I wanted, and the risks I was unwilling to accept. The agent helped inspect the existing cabinet, wrote the tools, ran read-only analyses, interpreted the resulting reports, and refined the implementation as we encountered real-world edge cases.

I actually thought this was risky – giving the agent access to my fully curated 3.6Tb image to clean up scraping. I had full backups – two digital copies and a copy of my SDD on a hard drive, so I was prepared to fail. After all, you learn by failing and I’ve learned to fully embrace that.

My direction established the important rules:

• inspect the current library before changing it;
• preserve good work that already exists locally;
• never treat a fuzzy match as a fact;
• keep scraper access and credentials separate from the maintenance toolkit;
• make the default behavior read-only;
• require an explicit command before modifying anything;
• back up a gamelist before replacing it; and
• produce reports that a person can inspect at every stage.

The agent turned those requirements into a collection of working Python tools behind a single retropie-metadata command. It was developed iteratively: run an audit, examine what the real collection revealed, adjust the logic, and test again. The code was generated agentically, but the objective, constraints, acceptance decisions, and definition of “correct” remained mine.

That distinction matters. I did not ask an AI to decide what every game was and then blindly rewrite the cabinet. I used an agent to help build a system that makes ambiguity visible and gives me control over the decision.

In other words, the coding agent is handled much like the scraper itself: extremely capable, very useful, and always subject to verification.

Back to scraping

A well-organized RetroPie library looks effortless. Every title has the right name, description, artwork, and launch path. Browse the menu and everything appears to belong exactly where it is. That has been my dream with this project.

Maintaining that illusion is harder than creating it. LOL

Our arcade cabinet contains dozens of console, computer, handheld, arcade, and specialist systems. Over time, games move, filenames change, emulator collections are reorganized, and metadata comes from several generations of tools. A missing screenshot is easy to notice. A plausible-looking but incorrect match is much more dangerous.

The obvious answer is to run a scraper again. I deliberately chose a different approach.

Rather than allowing a scraper to rewrite the library, we treat it as an untrusted source of suggestions. The cabinet is audited first, existing local work is recovered wherever possible, external results are staged and scored, and only reviewed matches are allowed into a gamelist.

The result is slower than clicking “scrape everything”, but vastly easier to understand, verify, and repair.

Why a successful scrape can still be wrong

A scraper is good at finding likely matches. It cannot always know what a file means in a particular collection.

Several situations can produce a confident but incorrect result:

• regional releases with similar names;
• translated, hacked, prototype, or alternate versions;
• disc and cartridge editions of the same game;
• titles with punctuation, subtitles, or filename tags;
• computer software whose filename describes the media rather than the game;
• MAME short names that require a separate name map; and
• custom launch entries that look like games but actually invoke a script or mounted image.

A bad match is not necessarily obvious. The title may look reasonable and the artwork may even be from the same series. Once that result is written into gamelist.xml, it becomes difficult to remember which fields were carefully curated and which ones came from an automated guess.

There is also more to library health than missing artwork. A gamelist can contain a beautiful entry whose ROM path no longer exists. The filesystem can contain media that no game references. Two copies of a gamelist can disagree. A scraper cache can hold useful data that was never merged—or stale data that should never be merged.

We needed a process that could distinguish all of those conditions.

Our rule: audit before acquisition

The metadata workflow is coordinated by a cabinet tool called retropie-metadata. It does not contact ScreenScraper, store scraper credentials, or download media. Its job is to understand the local library and control what happens to external results.

The first step is verification:

cd /home/pi/mame-organizer/toolkit
./retropie-metadata verify
--output work/verification.json

This checks every known copy of each system’s gamelist.xml and verifies the files referenced by those lists. We want to know whether the current library is internally consistent before introducing any new data.

Next, we identify what is actually missing.

./retropie-metadata audit
--output-tsv work/missing.tsv
--output-summary work/missing-summary.json

The detailed TSV is intended for inspection and filtering. The JSON summary gives us a compact view of the scope. Separating those outputs is useful: the summary tells us whether we have a small repair or a collection-wide problem, while the tabular report tells us which individual entries require attention.

At this stage nothing has changed. That is an important feature, not an incomplete operation.

Recover local work before downloading replacements

Old gamelists, scraper caches, and media directories often contain perfectly usable information. A game that is missing a description in the active list may already have one in another trustworthy local source. Artwork may exist on disk under an older path. Reusing that work is safer than asking a remote service to identify the game again.

We search for recoverable local material.

This produces a recovery audit rather than silently inserting the results. We inspect the proposed sources, confirm that they refer to the same games, and only then repeat the operation with --apply.

That order gives local evidence priority:

1. Preserve correct metadata that has already been curated.
2. Reconnect media that already exists.
3. Resolve stale paths where the intended target is clear.
4. Send only the genuinely unresolved entries to the external scraping stage.

It also avoids unnecessary downloads and reduces the number of ambiguous matches that need human attention later.

Treat external scraping as an import operation

Once local recovery is complete, the unresolved report becomes the input to a preparation step:

./retropie-metadata prepare-scrape
--missing work/missing.tsv
--review work/local-recovery/local-recovery-audit.csv
--output-dir work/scraper-staging

Preparation creates normalized titles, mappings, and target lists. It still does not contact the network. The point is to make the request explicit before another tool sees it.

This is where known naming exceptions belong. The cabinet’s Skyscraper configuration preserves title aliases, platform corrections, MAME mappings, matching hints, targeted inclusion lists, and records of deliberately skipped items. Those files explain the collection’s intent in a repeatable form.

Aliases and hints are better than manually correcting the same title after every future scrape. They turn one-off knowledge into part of the maintenance system.

The external scraper operates outside our metadata tool. Its returned cache is imported or placed under the staging directory. We then analyze the results without merging them:

./retropie-metadata review-scrape
--staging work/scraper-staging
--output-dir work/scraper-review

The review stage scores candidate matches and produces material for human inspection. A score helps prioritize the work, but it is not permission to modify the library. We still review the normalized name, platform, proposed match, and any ambiguity before approval.

If the results look wrong, the response is not to accept them and clean up afterward. We stop, improve the aliases or hints, and repeat preparation and review.

Merge approved results, and nothing else.

Approved candidates are passed to the merge stage:

./retropie-metadata merge-scrape
--staging work/scraper-staging
--review-dir work/scraper-review
--output-dir work/scraper-merge

Even this is a dry run. The generated merge can be inspected before it reaches an active gamelist. Only after the approved output is correct do we add --apply.

All modifying operations require an explicit --apply or --execute. Gamelist updates are written atomically and receive timestamped backups. A partial write should never be able to leave the front end with half a valid XML file, and a newly discovered problem should have a clear rollback point.

We also exit EmulationStation before applying changes. The front end reads and may write its own metadata, so editing the same gamelists while it is running creates an avoidable ownership conflict.

The operating rule is simple:

Generation is automatic. Acceptance is deliberate.

Audit the media as well as the XML

A cleanly parsed gamelist.xml does not prove that the library is clean. The final audit includes the media directories:

./retropie-metadata audit-media
--output-dir work/media-audit

This lets us separate several different states that are often collapsed into “missing artwork”:

• missing: the gamelist expects media that is not present;
• reusable: suitable media already exists locally but is not connected to the entry;
• orphaned: a media file exists but no gamelist entry references it; and
• uncatalogued: content exists outside the expected catalogue relationship and needs review.

Those categories lead to different actions. A missing image may need to be sourced. A reusable image needs a corrected reference. An orphan may be left over from a renamed game, or it may be the only surviving copy of wanted artwork. Automatically deleting every orphan would throw away evidence before we understood it.

Stale game paths receive their own analysis:

./retropie-metadata analyze-stale
--verification work/verification.json
--output-tsv work/stale.tsv
--output-summary work/stale.json

This is especially valuable after reorganizing a ROM collection. Metadata should follow intentional moves, but an unresolved path should remain visible rather than being guessed into a similarly named file.

Verify again before declaring success

After approved changes are applied, we run the verification step again. The second pass should demonstrate that:

• the gamelists still parse correctly;
• referenced games and media exist;
• the intended missing fields were filled;
• no unrelated entries were replaced;
• repaired paths point to the correct files; and
• the remaining exceptions are understood and documented.

Only then do we restart EmulationStation and inspect the results in the interface.

This gives the process a useful shape:

verify
  ↓
audit what is missing
  ↓
recover trustworthy local data
  ↓
prepare unresolved targets
  ↓
scrape into staging
  ↓
score and review candidates
  ↓
preview the approved merge
  ↓
apply
  ↓
verify again

Every arrow leaves behind a report. If something unexpected appears at the end, we can trace where it entered the process instead of trying to reconstruct a single opaque scraper run.

Why we do not start over with a clean cache

When scraper results become confusing, deleting the cache and starting again is tempting. For a large library, that is an expensive and sometimes destructive reflex.

A cache may contain years of valid results, hand-resolved titles, and media that is no longer easy to retrieve. Rebuilding it also does not correct a bad alias, an incorrect platform, or an ambiguous filename. Without fixing the underlying interpretation, a clean scrape may simply reproduce the same mistake at a larger scale.

We preserve the existing cache and review reports when making targeted corrections. A clean-slate scrape is considered only when it is genuinely necessary, and its completion is verified before it is allowed to generate a replacement gamelist.

What changed for us

The biggest improvement was not better artwork. It was confidence.

We can now answer questions that a one-click workflow obscures:

• Which entries were incomplete before the run?
• Which fields came from existing local data?
• Which games were sent to the external scraper?
• What match did the scraper propose?
• Why was that match approved?
• Which files changed when the merge was applied?
• Does the final gamelist still point to real games and media?

The scraper remains useful. It is simply no longer trusted with the final decision.

The same principle applies to the AI agent that helped me build the toolkit. Agentic development made it possible to inspect a large, messy collection and create a purpose-built set of tools in a fraction of the time it would otherwise have taken me. That speed did not remove the need for judgment. It made good guardrails even more important.

Human-in-the-loop cannot mean clicking “yes” at the end of a process nobody understands. In this project it means setting the rules, examining intermediate evidence, challenging suspicious output, refining the implementation, and deciding what is allowed to become part of the live collection.

That distinction scales well beyond RetroPie. Any automated system—whether it is matching a game or writing the code that performs the match—can produce a convincing mistake. The safest workflow separates discovery from authority: let automation find possibilities, preserve the evidence, test what it produces, and require an explicit review before those possibilities become reality.

For our cabinet, that combination of agentic development and human control turned metadata maintenance from a risky rewrite into a controlled, repeatable repair process. It is also a practical example of why I am optimistic about AI: not because it replaces expertise, curiosity, or responsibility, but because it can amplify all three when it is directed thoughtfully.

I was very surprised at how effective using an agent was for this process. The fact it was able to identify and pull box artwork rather than, say, a marquee – which was a problem I had during my initial scraping attempts.

Despite what I said earlier about “The result is slower than clicking scrape everything”, I think this process actually ended up being faster. Why? I’m not repeatedly running a scraper constantly against Screenscraper.fr and hoping for the best; when even the scraper fails, I could then ask the agent to search other databases to find the correct art and details and it did perfectly because it had clear data to act on.

All under my control and guidance, with a human in the loop.

I plan on making the tools available on Github as an example that someone could augment and update.