Create your own Instacart GPT shopper

I've briefly talked about my weekly meal planning habit before which tends to save us money on groceries. I use Instacart because time > money for a family of 5.

Even though Instacart already saves me time, it still takes up time to do the add-to-cart dance. 🕺🤸‍♂️🛒 It's mundane, annoying, and it's time that could be better spent.

I have been itching to find areas to experiment with how to put AI to work for me IRL so when I saw the demo at MS Build this past week of them ordering Instacart while viewing a recipe page using a Microsoft Edge copilot plugin, I was like – WAIT, WHAT.

I started twirling my mustache wondering what shenanigans this would enable.

Life Before ChatGPT

As part of Jonathan Stark's 10-Day Systems Challenge, I created an SOP for meal planning and it's quite involved. But roughly the steps are:

1. Add meals to the meal app (Paprika) calendar for the week
2. On weekends, go through each meal and add required groceries to the list
3. Order all items on the list using Instacart
4. Go to other stores as needed for specialty ingredients

Step 1 Cassie handles, no biggie, she can do it in the car, wherever.

Now, steps 2 and 3 are a big pain for me. I need a solid 30-minute block of time set aside (usually during a time when I'd rather be working or chilling). I sometimes need to be near the kitchen to double-check what we need.

Step 4 isn't a big deal because we drive past Target or Fresh Thyme nearly every day for daycare.

That's why if I can shorten or automate steps 2 and 3, I will do my damnedest.

Welcome to the brave new world of AI copilots

It seems Microsoft wants "copilots" to be absorbed into everyday vernacular. In my experience, it seems like GPT is now "AI" for the layperson (even if it wasn't intentional, the rhyming makes it memorable). Hence why I titled this post "Instacart GPT shopper" instead of "Instacart AI Shopper." Google Bing juice.

Well, Instacart's plugin is available wherever Copilots are sold and I'm already a ChatGPT Plus subscriber which means I have plugin access. DING DING DING. 💡

The question was: can I build automation to... automatically order my weekly groceries? Can I build an Instacart AI shopper? #firstworldproblems

Making the list of ingredients

If you have a list of ingredients, you only need to paste it in with the Instacart plugin enabled to prompt it to add them:

But the problem, of course, is how do you make that list?

I have a list, it's in Paprika but it looks like this:

Luckily, Paprika can either email or share the list and it has some specific formatting it uses:

So as you can see, it's got some section headers and tickboxes. It adds the quantities/notes to the line items too. It will also include everything on the list, and we tend to add non-grocery items too like for Target shopping because this is the easiest way to sync across our phones.

It's good this is structured because in order to simplify it to pass to Instacart, we need to transform it. And instead of hacking together some code with regex or... something, why not use ChatGPT?

Metaprompting the ingredient list

How do we get our Paprika output to ChatGPT? I could copy/paste it but then I'd also need to write the prompt the same way every time. We're gonna want a "metaprompt" to engineer this.

Zapier recently added ChatGPT / OpenAI integration which means we can now use the no-code platform to make an Instacart GPT Shopper zap.

I created a new Zap and added an Email by Zapier step, which gives me a unique email address I can use:

Then, to test the trigger, I just clicked the "Email list" button in Paprika to open Gmail on my phone to email all the text to this Zapier email address.

Now Zapier has the email as test data to use for the next step which is... ChatGPT!

For this step, you need to hook up your OpenAI developer account. This costs money.

Luckily, you can pay-as-you-go and set a maximum limit (mine is $60/mo). Each time your Zap runs, you'll use up tokens that will be charged.

Now here's where the magic happens ✨

We need to provide a metaprompt, the prompt that surrounds any user input we send. Technically, a metaprompt would live even higher up but what we're doing here serves the same purpose. We want to give ChatGPT the context it needs to do a better job.

This is the User Message I'm currently using to tell ChatGPT how to transform the data we're sending it via email:

I am providing a grocery list after "Grocery list:". ☒ means I already bought the item. ☐ means I don't have it yet. Remove all items I have bought. Remove all section headers. Discard all items under Miscellaneous or that are not food items. Remove quantity information. Remove measurements. Extract just the ingredient names. Send back only a single bulleted list of ingredients that I can bring to a store.

Grocery list:

{{Email: Body Plain}}

The {{Email: Body Plain}} is a Zapier field from the previous step.

The prompt does a few things:

• Tells ChatGPT what the tickboxes represent
• Removes sectional headers
• Removes measurements
• Specific output format

I adjusted the other fields like so (Zapier fields noted):

• Username: {{Email: From Name}}
• Assistant Name: Grocery Shopping Assistant
• Assistant Instructions: You are a helpful grocery shopper who understands meal planning.
• Model: gpt-3.5-turbo (I'm on the gpt-4 waitlist)
• Temperature: 0.5 (less "creative")
• Top P: 1.0 (unchanged)
• Memory Key: {{Email: Raw Message Id}}

Then, when you test the trigger, you can see the magic happen in the response:

response:
  role:
    - assistant
  content:
    - Pillsbury Grands biscuits
    - all purpose flour
    - baking powder
    - baking soda
    - sugar
    - chicken broth
    - tomato paste
    - buttermilk
    - eggs
    - sour cream
    - unsalted butter
    - unsweetened coconut milk
    - beef chuck
    - boneless, skinless chicken thighs
    - diced chorizo
    - ground beef
    - vegetable or canola oil
    - fruit
    - salt
    - tilapia (or other fish fillets)
name:
  - grocery-shopping-assistant

What what! That's pretty dang good!

As you can imagine, we could play with this prompt to tweak it further. Perhaps being specific about different ingredients ("Treat ground chuck as 90% lean ground beef"). Lot of possibilities here.

Plugging the prompt into ChatGPT

After this step, we've got our "generated prompt."

Ideally, what would happen next is another request to ChatGPT to add to our Instacart using the plugin. I don't know yet if that's supported by the gpt-4 model but I don't have access to it regardless.

That leaves somehow copying the prompt and pasting it into a chat conversation using the ChatGPT web interface, Windows Copilot, Microsoft Edge, etc.

To do that, I chose to email myself using the Email by Zapier step but you could also send a text, Slack, Discord, etc.

I then provide a simple message with instructions to copy/paste a prompt and I include a link to my personal chat conversation which has the GPT-4 + Instacart plugin enabled.

This requires having a ChatGPT Plus subscription (which is separate from your developer billing). Once you have it, you can enable plugins of which there are many (and growing by the day – it's like early App Store days in here):

Incredibly, there's no way to search (at least not on mobile) so I had to go to page 11 or something to find Instacart 🤦‍♂️.

But once you've got it, you're golden ✨

Paste in the prompt, and Instacart will generate a unique shopping list link. This is not part of your account but it does have smarts to figure out past ordered items and will try to intelligently exclude things you probably still have (and it was right).

Instacart AI shopper is surprisingly good

How well did it do? See for yourself!

• ❌ means I have to fix it
• ⭕ means the store doesn't carry it, so it's expected
• 🆗 means the item is OK but I'd prefer a different one

Here are the results:

• Pillsbury Grands biscuits ❌
• all-purpose flour ✅
• baking powder ✅
• baking soda ✅
• sugar ✅
• chicken broth ✅
• tomato paste ✅
• buttermilk ⭕
• eggs ✅
• sour cream ✅
• unsalted butter ✅
• unsweetened coconut milk ⭕
• beef chuck ❌
• boneless, skinless chicken thighs ✅
• diced chorizo ⭕
• ground beef 🆗
• vegetable or canola oil ✅
• fruit 🆗
• salt ✅
• tilapia (or other fish fillets) ✅

That's pretty dang good! The biscuits and the beef chuck are the only two that really were off the mark – one because those biscuits ARE in stock at Aldi and second, I want ground beef, not a chuck roast. But I can't blame ChatGPT much for the beef – I need to update the recipe to be more specific (which I did).

For both the OK items, all it would take is refining them upfront to be specific (I use 90% ground beef for most meals besides burgers). "fruit" is vague but Instacart still picked something my kids would eat. Usually, we're more specific than that.

This will actually save me time

I don't like to overhype AI. This won't 10X my week but it will save me time. Assuming I had better inputs and excluded what the store doesn't carry, it got an 88% success rate (15 of 17 items). That's "good enough" where it shouldn't end up adding time to correct it.

Normally, I have 20-35 items so we'll see if it can maintain that. I will need to test it more fully next week for the next order since this week we only needed 5 things as I got most ingredients at the store while shopping for Memorial Day.

With a few tweaks like triggering the plugin via API which I hope I can do, and refining my recipes in the app to use more specific ingredients I use, this could definitely give me back hours each year.

Updates

• 6/25/2023 – Updated the meta prompt to use stricter language. It wasn't handling some more involved line items like "2 slices 1/2" thick Pullman or other white bread" and now it successfully extracts "Pullman or other white bread."
• 6/3/2023 – It got about 12 items correct and Instacart didn't get a few even though GPT did them correctly. It took me 5 minutes to add all the items to the list, and 7 mins to check out on Instacart. Tweaked the prompt to say "measurements and quantities." Verdict: 👍

Steal my AI shopper

Of course, I made my Zap public so you can feel free to use the template. You'll have to customize the email but otherwise, it will "just work" as long as you meet the requirements.

Could this be made into an app? Not sure yet – the list metaprompt and generation could be. The Instacart integration either needs to be through their API (enterprise only?) or through OpenAI and I'm not sure yet if you can trigger plugins via API.

So what do you think, did this make you a bit more excited about AI?

Cheers,
Kamran