this post was submitted on 25 Jul 2024
641 points (99.1% liked)
196
16511 readers
3392 users here now
Be sure to follow the rule before you head out.
Rule: You must post before you leave.
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
VRAM would be 810Gb/403Gb/203Gb for FP16/FP8/INT4 for interferrence, according to their website.
Hot damn that's a lot! They ain't messing around with that requirement.
My current server has 32 MB of VRAM. Yes, MB not GB. Once I finish the hardware migration it's going to 8GB but that's not even a drop in the bucket compared to that requirement.