this post was submitted on 03 Jun 2024
1475 points (98.0% liked)

People Twitter

5225 readers
2459 users here now

People tweeting stuff. We allow tweets from anyone.

RULES:

  1. Mark NSFW content.
  2. No doxxing people.
  3. Must be a tweet or similar
  4. No bullying or international politcs
  5. Be excellent to each other.

founded 1 year ago
MODERATORS
 
you are viewing a single comment's thread
view the rest of the comments
[–] [email protected] 97 points 5 months ago (4 children)

The reality is, though, that there are no such APIs. LLMs on the other hand could be a valid tool for the use case.

[–] Semi_Hemi_Demigod 52 points 5 months ago (6 children)

It's not that there's no API. It's that there's probably a different API for every single grocery store. And they make random changes and don't have public documentation. That's why we need the AI.

[–] [email protected] 40 points 5 months ago

Yup, exactly, no standardized APIs.

[–] [email protected] 28 points 5 months ago (2 children)

The stores don't want you to have easy comparable access to their prices.

They'd quite like it if you just came in, saw that the item you wanted is out of stock, and then just buy some shit you didn't need.

[–] BeMoreCareful 7 points 5 months ago

Yeah, we're not going to make technology that drives prices down

[–] [email protected] 1 points 5 months ago

But they'll happily give you full access to everything they have if you're another corpo and you promise to marginally improve their sales anyhow. That's, sadly, how businesses work.

[–] [email protected] 10 points 5 months ago (2 children)

Indeed. LLMs read with the same sort of comprehension that humans have, so if a supermarket makes their website compatible with humans then it's also compatible with LLMs. We have the same "API", as it were.

[–] [email protected] 2 points 5 months ago (1 children)

Can LLMs interpret structured input like html?

[–] [email protected] 5 points 5 months ago

Yup. And those that can't can have a parser pull just the human-readable text out, like a blind person's screen-reader would do.

[–] grue 9 points 5 months ago

No, that's why we need regulations to enforce standards.

[–] [email protected] 6 points 5 months ago (2 children)

You just need someone to do it. Here in Austria someone did it: https://heisse-preise.io

It's only in German and most of the prices aren't from a public API but crawled from different sources.
It's open source. Nothing except greed is stopping them from providing something like this.

[–] Semi_Hemi_Demigod 3 points 5 months ago (1 children)

Imagine if instead of building their own bespoke systems, grocery stores (and other places) created an open source software foundation and worked together to produce the software they needed.

[–] [email protected] 3 points 5 months ago

I sometimes dream of such things. Less waste, better inventory, customers get to choose inventory based on their wishlist, better prices, then I wake up.

We actually have a small liquor store nearby that really puts stuff on the shelves if you casually mention something you like. But that's more the exception than the rule.

[–] [email protected] 1 points 5 months ago

That's impressive, and honestly looks like it was quite a bit of work. I wonder how the author finances himself? There doesn't even seem to be a donation button on the site. I found a lengthy article on Wired but it doesn't appear to mention how he can afford to do all of this for free.

It's open source. Nothing except greed is stopping them from providing something like this.

Nothing is stopping anyone from doing this except the amount of work it takes to write and maintain all those data import scripts. I think greed is the wrong word here. It's not unreasonable to expect some sort of monetary reward for providing a useful public service that actually helps people save money. Everyone's gotta eat, right?

[–] [email protected] 1 points 5 months ago

Actually, you'd be surprised. Instacart has up-to-date price and product data for TONS of grocery stores. And while their API likely isn't public, they MUST have one in order for their smartphone apps to work.

[–] [email protected] 13 points 5 months ago (3 children)

LLMs are not a good tool for processing data like this. They would be good for presenting that data though.

[–] [email protected] 3 points 5 months ago (1 children)

Make an LLM convert the data into a standardized format for your traditional algorithm.

[–] [email protected] 2 points 5 months ago

There's no way to ensure that data will stay in that standardized format though. A custom model could but they are expensive to train.

[–] [email protected] 1 points 5 months ago (1 children)

Llms are excellent at consuming web data.

[–] [email protected] 6 points 5 months ago (1 children)

Not if you want to ensure the validity of the compiled coupons/discounts. A custom algorithm would be best but data standardization would be the main issue, regardless of how you process it.

[–] [email protected] 0 points 5 months ago* (last edited 5 months ago) (1 children)

What does validity mean in this case? A functionary LLM can follow links and make actions. I'm not saying it's not "work" to develop your personal bot framework, but this is all doable from the home PC, with a self hosted llm

Edit and of course you'll need non LLM code to handle parts of the processing, not discounting that

[–] [email protected] 1 points 5 months ago* (last edited 5 months ago)

The LLM doesn't do that though, that the software built around it that does that which is what I'm saying. Its definitely possible to do, but the bulk of the work wouldn't be the task of the LLM.

Edit: forgot to address validity. By that I mean keeping a standard format and ensuring that the output is actually true given the input. Its not impossible, but its something that requires careful data duration and a really good system prompt.

[–] [email protected] 0 points 5 months ago (1 children)

Llms are great for scraping data

[–] [email protected] 9 points 5 months ago (1 children)

LLMs don't scrape data, scrapers scrape data. LLMs predict text.

[–] [email protected] 6 points 5 months ago (1 children)

there are no such APIs

Yes there are. You can obtain access to the Kroger API, the Meijer API, the Walmart API, and I'm sure others that I didn't bother to Google. Failing getting access to the actual APIs, there are tons of web scraper projects that just parse those stores' websites for product information, and web scrapers are still orders of magnitude more efficient than LLMs.

[–] [email protected] 1 points 5 months ago

Instacart has prices for all of these stores and more. Obviously they're not updating them by hand...

[–] Ibaudia 5 points 5 months ago

At the cost of huge amounts of wasted energy and the whole litany of concerns that are always co-morbid with AI, but technically yes they could work for this lol. Ideally we'd have standardized APIs and mandated pricing transparency, but unfortunately we live in a capitalist society where that will literally never happen ever.