Start

Home / Big ideas

Idea5-min read+5 pts

Use AI to Build the Tool, Not Guess Every Answer

With Josh Schultz ·

Answer. When you have a big pile of messy data, use AI to build a tool (a set of rules, a formula, or a master list you can read) instead of asking it to guess every row. You can check eight rules in a minute, but you cannot check two thousand guesses.

The picture

Compare diagram. Left, "Guess every row": 2,000 rows go to AI, which guesses each one, giving 2,000 answers, with the note "check all 2,000?" Right, "Build the tool": AI writes the rules, you check 8 rules (circled), Excel runs them on all 2,000 rows, with the note "same answer every time."

The diagram compares two paths. On the left, AI makes 2,000 small guesses, and you cannot see how it made any of them. On the right, AI writes the rules once, you read and fix them, and Excel then runs them on every row, which gives the same answer every time.

Think of it like this

Say you have 500 brackets to weld. You could ask your best welder to eyeball each one, and they would do good work, but every bracket would come out a little different and you would have to measure all 500.

Or you could build a jig, which is a fixture that holds each part in the same position. It takes an hour to build, and you check it once with care, after which every bracket comes out the same.

Asking AI to clean 2,000 rows is like eyeballing 500 brackets, while asking it for the rules is like building the jig. You still need the welder's skill, but you spend it once on the jig instead of on every part.

Why does it matter at work?

A guess you cannot see is a guess you cannot check, and with thousands of rows you will not check them anyway.

Here is how that plays out. Hector Lee, the production planner at Mesquite Ridge Metal Works, pulls a spend report from ShopLine ERP. One supplier shows up three ways: "Red Creek Steel Supply," "RED CREEK STEEL," and "Red Creek Stl Sup." His report splits one supplier into three.

He pastes all 2,000 rows into a chat and asks AI to clean them, but the answer stops partway, and the rows it did return have names he cannot trace back to the file.

So he asks for the rules instead and gets a short list of rules and a master list of supplier names. He spots one gap in a minute, gets it fixed, and runs the rules in Excel on all 2,000 rows, and the answer is the same every time. Next month's export can use the same rules.

I have seen this work at a much bigger size. On one data move, AI sorted the mess into buckets. In my words: "And it came up with over 10,000 problems just for the last 9 months." Nobody can fix 10,000 problems by hand, so you build a tool for each bucket.

How does it work?

You change what you ask for. Instead of asking for the finished list, you ask for the thing that makes the list.

Ask for the rules, not the rows. Say, "Don't clean this list. Write the rules for cleaning it." That puts the AI's thinking on the page, where you can read it.

Check the rules, not every answer. Eight rules take a minute to read, and you will spot a bad one quickly because you know your own suppliers.

Fix the rule, not the row. When a rule is wrong, say so and ask for a new version, and every row gets the fix at once.

Compare diagram. Left, "Fix by hand": rows 412, 977, and 1,503, with the note "gone next month." Right, "Fix the rule": rule 8, a master name list (circled), fixes all 2,000 rows, with the note "and next month too."

The diagram shows why this matters. A fix made by hand on a few rows is gone next month, but a fix made to the rule, such as a master name list, covers all 2,000 rows and next month's rows too.

Let a plain tool run it. Excel, find-and-replace, and a lookup table all follow rules the same way every time. AI does not, because it writes one likely word at a time.

AI is still very good at the first part, which is spotting the patterns and naming the buckets, because that takes judgment. Running the rules 2,000 times does not take judgment, so a plain tool can do it. As I told one team, "I was basically creating tools to help me do the job faster rather than just doing the job."

Where do people get it wrong?

Most mistakes come from wanting the finished list right now. These are four to watch for.

  1. Pasting the whole file and asking for the answer. A big file can fill the AI's desk, and it may stop partway without telling you. The lesson on what a context window is explains why.
  2. Taking the rules without reading them. A rule can hide a risk. In my test, one rule set dropped words like "Supply" to make names match, and the AI's own note warned that this could merge two real companies, which is easy to miss if you do not read the rules.
  3. Fixing rows by hand. If you fix row 412 yourself, the fix is gone next month, so fix the rule instead.
  4. Building a jig for ten parts. For a short list you will use once, just ask for the answer and check it. The tool pays off when the pile is big or comes back every month.

Try this in 2 minutes

You will ask for the answer in one chat and for the rules in another. You can use any chat tool.

  1. Copy this fake supplier list:
Red Creek Steel Supply
RED CREEK STEEL
Red Creek Stl Sup.
Panhandle Bar & Plate
Panhandle Bar and Plate Co
Bolt Barn Industrial
Bolt Barn Ind.
boltbarn
Blue Heron Coatings
Blue Heron Coating LLC
Lone Oak Harness
Lone Oak Harness Inc
Kestrel Electronics
Kestrel Elec (overseas)
Delta Freight Lines
  1. In a new chat, paste the list under this ask:
Clean up this supplier list so each supplier appears once, with one name.
  1. In a second new chat, paste the list under this ask:
Don't clean this list. Write short numbered rules for cleaning supplier names like these, so I can check the rules and use them on all 2,000 rows in our file. Then show 3 rows before and after.
  1. Read the rules. Look for two names that would still not match, or a rule that could merge two real suppliers. If you find a gap, send a fix like this:
Your rules leave "RED CREEK STEEL" and "Red Creek Steel Supply" as two different suppliers. Fix the rules so each real supplier ends up with one name.

Two chats side by side. On the left, asking for the answer gives a neat list of seven suppliers, with "Blue Heron Coatings LLC" underlined and the note: a blend, not a name in your list. On the right, asking for the rules shows rule 8, a master supplier list, underlined, with the note: it spots the Red Creek gap on its own.

When I tried this, the answer ask gave a neat list of seven suppliers and even explained its choices. On this short list that was fine, but look at item 4. The list had "Blue Heron Coatings" and "Blue Heron Coating LLC," and the AI wrote "Blue Heron Coatings LLC," a blend of the two that never appeared in the list. It also merged "Kestrel Elec (overseas)" into "Kestrel Electronics" and said so: "The Kestrel merge is the only judgment call that could be wrong." With 15 names I can check that, but with 2,000 I cannot.

The rules ask gave eight rules I could read, and this time it spotted the hard part on its own. Rule 8 said:

  1. Map each result to one approved name from a master supplier list. Rules 1-7 won't catch these:
  • Missing words: Red Creek Steel vs Red Creek Steel Supply.

It also said, "Rule 8 is where errors are most likely, so spot-check it on the 2,000 rows." That master list is the real jig, because you can read it and fix it.

Not every set of rules was that careful. Another one dropped words like "Supply" to make names match, and warned me that "Red Creek Steel" and "Red Creek Steel Supply" could be different vendors. When I sent the fix, the reply ended with "Rule 6 can wrongly merge different suppliers." It did not hide the risk, but you still have to read it.

Frequently asked questions

What counts as a "tool" here?

Anything that runs the same way every time and that you can read. A list of rules, an Excel formula, a lookup table, or a find-and-replace list all count.

Do I need to know Excel formulas?

No. Ask AI to write the formula and explain each part in plain words. Your job is to check that it does what the rule says, on a few rows you know.

When should I just ask for the answer?

Ask for the answer when the list is short enough to check by eye and you will only do it once. For two thousand rows, or a job you repeat every month, build the tool.

Isn't this slower?

The first time, a little. The second time it takes seconds, and you do not have to check everything again.

Use it today

From my training: this comes from my own data clean-up work, where I stopped asking AI for answers and started asking it for tools.

Ideas are worth 5 points. Guides are worth 10 and up.

Get your first win this week.

One minute. Three guides picked for your job.

Start my first win