this post was submitted on 18 Nov 2024
19 points (100.0% liked)

Hardware

691 readers
124 users here now

All things related to technology hardware, with a focus on computing hardware.

Rules (Click to Expand):

  1. Follow the Lemmy.world Rules - https://mastodon.world/about

  2. Be kind. No bullying, harassment, racism, sexism etc. against other users.

  3. No Spam, illegal content, or NSFW content.

  4. Please stay on topic, adjacent topics (e.g. software) are fine if they are strongly relevant to technology hardware. Another example would be business news for hardware-focused companies.

  5. Please try and post original sources when possible (as opposed to summaries).

  6. If posting an archived version of the article, please include a URL link to the original article in the body of the post.

Some other hardware communities across Lemmy:

Icon by "icon lauk" under CC BY 3.0

founded 1 year ago
MODERATORS
Alphane_Moon
CSFFlame
19
Lenovo's enterprise hardware biz still struggles for profits (www.theregister.com)
submitted 3 days ago by Alphane_Moon to c/hardware
11 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[–] Alphane_Moon 1 points 1 day ago (3 children)

Cheers, will give it a go. I want to move away from cloud LLMs.

[–] brucethemoose 2 points 1 day ago* (last edited 1 day ago) (2 children)

Pick up a 3090 if you can!

Then you can combine it with your 3080 and squeeze Qwen 72B in, and straight up beat GPT-4 in some use cases.

Also, TabbyAPI can be tricky to set up, ping me if you need help.

[–] Alphane_Moon 1 points 1 day ago (1 children)

Not planning on getting a new/additional GPU at this point. My local LLM project is more of curiosity, I am more knowledgeable on the AI upscaling side. :)

Thanks for the offer, will consider it!

[–] brucethemoose 2 points 1 day ago* (last edited 1 day ago)

Last time bothering you! I used to be really into the GAN space myself, but the newer diffusion models really blow them away. Check this out: https://github.com/mit-han-lab/nunchaku

This can squeeze Flux 1D onto your 3080, and (with the right pipeline/settings) it should blow anything else away at "enhancing" a low res image with img2img. It should also work with batching and torch.compile so you can get quite a lot of throughput from your 3080. Of course, there's no temporal consistency yet (or it may be, it's hard to keep up with all the adapter releases), but I'm sure its coming... And you can kinda hack some in with 2D models anyway.