this post was submitted on 11 Oct 2023
506 points (92.6% liked)

Technology

59708 readers
5537 users here now

This is a most excellent place for technology news and articles.

Our Rules

  1. Follow the lemmy.world rules.
  2. Only tech related content.
  3. Be excellent to each another!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, to ask if your bot can be added please contact us.
  9. Check for duplicates before posting, duplicates may be removed

Approved Bots

founded 1 year ago
MODERATORS
L3s
[email protected]
[email protected]
L4s
enu
506
So Far, AI Is a Money Pit That Isn't Paying Off (gizmodo.com)
submitted 1 year ago by FlyingSquid to c/technology
156 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[–] [email protected] 32 points 1 year ago (3 children)

This is why I, as a user, am far more interested in open-source projects that can be run locally on pro/consumer hardware. All of these cloud services are headed down the crapper.

My prediction is that in the next couple years we'll see a move away from monolithic LLMs like ChatGPT and toward programs that integrate smaller, more specialized models. Apple and even Google are pushing for more locally-run AI, and designing their own silicon to run it. It's faster, cheaper, and private. We will not be able to run something as big as ChatGPT on consumer hardware for decades (it takes hundreds of gigabytes of memory at minimum), but we can get a lot of the functionality with smaller, faster, cheaper models.

[–] [email protected] 9 points 1 year ago (1 children)

Hundreds of gigabytes of memory in consumer PCs is not decades away. There are already motherboards that accept 128 GB.

[–] [email protected] 6 points 1 year ago

You're right, I shouldn't say decades. It will be decades before that's standard or common in the consumer space, but it could be possible to run on desktops within the next generation (~5 years). It'd just be very expensive.

High-end consumer PCs can currently support 192GB, and that might increase to 256 within this generation when we get 64GB DDR5 modules. But we'd need 384 to run BLOOM, for instance. That requires a platform that supports more than 4 DIMMs, e.g. Intel Xeon or AMD Threadripper, or 96GB DIMMs (not yet available in the consumer space). Not sure when we'll get consumer mobos that support that much.