$149, meh.
this post was submitted on 13 Jul 2023
10 points (100.0% liked)
RISC-V
879 readers
1 users here now
RISC-V (pronounced "risk-five") is a license-free, modular, extensible instruction set architecture (ISA).
riscv.org
Other RISC-V communities on Lemmy
founded 3 years ago
MODERATORS
What does 4 TOPS of AI performance mean? Can it do video streaming analytics or language model inference at reasonable speed or voice transcription?
It puts it at the same performance as the Coral edge TPU (also 4 TOPS). I've used the edge TPU at work a bit, and it's definitely enough to run object detection models like YOLOv8 on live video streams (although I could only get YOLOv8 to work for up to 192x192 resolution input; I've seen others like MobileNetv2 get up to 640x640 resolution). I don't think you could run an LLM on it for any reasonable performance, but maybe voice transcription? I'm more in computer vision, so I don't know how big those models get off the top of my head. Voice transcription definitely seems like a not insanely intensive thing that ought to be able to be done live on 4 TOPS.
I’ll have to check out YOLOv8. I’ve been looking at an edge compute device that is ruggedized and can do video streaming analytics. I would need higher resolution (4k) but could get away with very low frame rates (1/30s). If I remember right I can break the image into sections and do inference on each section. Might have to do some overlapping sections to not miss “half on one, half on another”. Thanks for posting
Yeah, if your objects within the 4k frame are pretty small, you can definitely tile over the whole image with 192x192 windows. With YOLOv8n (the smallest model size), I could run inference on 192x192 images in 15ms per inference on average, and about 25ms per inference for 192x192 images on YOLOv8s (the second-smallest model size). We've been getting pretty good results in terms of accuracy and so forth with both n and s.
Biggest benefit of YOLOv8, imo, is it has a really good software package that makes it very easy to finetune, export to the edge TPU (or other inference engines), and run inference. Feel free to message me if you run into any issues with it!
Yea would probably have to do a several passes at different resolutions. Looking for liquid pooling so doesn’t have to be fast, just has to be right