You don't know what you don't know

I shared a prompt to help create “AI expert prompts” to help you learn a domain, and I’m realizing I didn’t mention the dangers. As every company that gives you a genie to talk to says “they can get things wrong.” The problem is that as a non-expert you don’t know what you don’t know, so you don’t know when the genie is so confidently wrong.

When I started a company I figured, why not do the books myself for the first couple of months and then pay someone to review it later? I created a genie armed with the standard Swedish accounting plan, verified some of the big things I did (inject some extra starting capital) and could find clear examples of how it’s done. I did some other things and just did as it said, but did get a bit complacent because the big one just worked, and didn’t research as thoroughly.

Buuut, I always felt uncomfortable with the whole thing. Then I got a paying gig, and one of the first sales calls was from a company that does bookkeeping, and I decided to sign them to do this for me because I didn’t want this unease hanging around. And better keep it clean from the start. Lo and behold, a month in, and they’ve done changes to most of the transactions I did myself.

I thought I did the right thing. That I had gotten complete answers from the genie. But I’m not good enough to evaluate it, so… no.

So: don’t trust the genie, verify what it says. Ask for references, for the terms that people in that field use, and to find a good writeup that’s available for newbies in that area. Then, actually read it and use your judgement: Does this make sense to me? Does what the genie said hold up? Ask it follow-up questions with the reference you got.

Try another session with the genie (preferably from another provider) and see what they think given what the first genie gave you (ask it to give you a summary of your conversation). Ask the other genie to provide evidence too, or even better, ask the genie to “red team why this would be a terrible idea” (or just “red team it”). Picked this up from Justin Searls who added red teaming to his base prompt for ChatGPT.

And get real experts. Luckily, now someone who actually knows bookkeeping is doing mine. 🙂