this post was submitted on 18 Dec 2024
11 points (92.3% liked)

Hardware

758 readers
182 users here now

All things related to technology hardware, with a focus on computing hardware.

Rules (Click to Expand):

  1. Follow the Lemmy.world Rules - https://mastodon.world/about

  2. Be kind. No bullying, harassment, racism, sexism etc. against other users.

  3. No Spam, illegal content, or NSFW content.

  4. Please stay on topic, adjacent topics (e.g. software) are fine if they are strongly relevant to technology hardware. Another example would be business news for hardware-focused companies.

  5. Please try and post original sources when possible (as opposed to summaries).

  6. If posting an archived version of the article, please include a URL link to the original article in the body of the post.

Some other hardware communities across Lemmy:

Icon by "icon lauk" under CC BY 3.0

founded 2 years ago
MODERATORS
Alphane_Moon
CSFFlame
11
Just how deep is Nvidia's CUDA moat really? (www.theregister.com)
submitted 6 hours ago by Alphane_Moon to c/hardware
12 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[–] Alphane_Moon 2 points 5 hours ago (2 children)

From my limited consumer-level perspective, Intel/AMD platforms aren't mature enough. Try looking into any open-source/commercial ML software aimed at consumers, Nvidia is guaranteed and first class.

The situation is arguably different in gaming.

[–] brucethemoose 4 points 5 hours ago* (last edited 5 hours ago)

Intel is not as bad in LLM land as you'd think. Llama.cpp support gets better every day.

Nvidia may be first class, but in this case, it doesn't matter if the model you want doesn't fit in VRAM. I'd trade my 3090 for a 48GB Arc card without even blinking, even if the setup is an absolute pain.

[–] anamethatisnt 2 points 5 hours ago (1 children)

Thanks for the insight. Kinda sad how selfhosted LLM or ML means Nvidia is a must have for the best experience.

[–] brucethemoose 2 points 5 hours ago

Only because AMD/Intel aren't pricing competitively. I define "best experience" as the largest LLM/context I can fit on my GPU, and right now that's essentially dictated by VRAM.

That being said, I get how most wouldn't want to go through the fuss of setting up Intel/AMD inference.