AI

4006 readers

2 users here now

Artificial intelligence (AI) is intelligence demonstrated by machines, unlike the natural intelligence displayed by humans and animals, which involves consciousness and emotionality. The distinction between the former and the latter categories is often revealed by the acronym chosen.

founded 3 years ago

LLM ASICs on USB sticks? (lemmy.ml)

submitted 3 months ago by [email protected] to c/[email protected]

9 comments fedilink hide all child comments

Source: nostr

https://snort.social/nevent1qqsg9c49el0uvn262eq8j3ukqx5jvxzrgcvajcxp23dgru3acfsjqdgzyprqcf0xst760qet2tglytfay2e3wmvh9asdehpjztkceyh0s5r9cqcyqqqqqqgt7uh3n

Paper: https://arxiv.org/abs/2406.02528

you are viewing a single comment's thread
view the rest of the comments

[–] [email protected] 2 points 3 months ago (7 children)

That would actually be insane. Right now, I still need my GPU and about 8-10 gigs of VRAM to run a 7B model tho, so idk how that's supposed to work on a phone. Still, being able to run a model that's as good as a 70B model but with the speed and memory usage of a 7B model would be huge.

[–] [email protected] 1 points 3 months ago (2 children)

I have never worked on machine learning, what does the B stand for? Billion? Bytes?

[–] [email protected] 2 points 3 months ago (1 children)

I think it's how many billion parameters the model has

[–] [email protected] 1 points 3 months ago

Thanks!

load more comments (4 replies)