LocalLLaMA

2203 readers

23 users here now

Community to discuss about LLaMA, the large language model created by Meta AI.

This is intended to be a replacement for r/LocalLLaMA on Reddit.

founded 1 year ago

MODERATORS

[email protected]

WizardLM/WizardCoder-33B-V1.1 released! (huggingface.co)

submitted 8 months ago by [email protected] to c/[email protected]

11 comments fedilink hide all child comments

Based off of deepseek coder, the current SOTA 33B model, allegedly has gpt 3.5 levels of performance, will be excited to test once I've made exllamav2 quants and will try to update with my findings as a copilot model

you are viewing a single comment's thread
view the rest of the comments

[–] [email protected] 5 points 8 months ago (1 children)

First few quants are up: https://huggingface.co/bartowski/WizardCoder-33B-V1.1-exl2

4.25 should fit nicely into 24gb (3090, 4090)

Smaller sizes still being created, 3.5, 3.0, and 2.4

[–] [email protected] 2 points 7 months ago (1 children)

I am curious if you prefer WizardCoder over Phind https://huggingface.co/Phind/Phind-CodeLlama-34B-v2

I am running them on my 3090, although the machine also has 32GBs of RAM as well.

[–] [email protected] 2 points 7 months ago (1 children)

I don't have a lot of experience with either at this time, I've used them here and there for programming questions but usually I stick to 7b models because I use them for code completion and I only find that useful if it completes the code before I do lol

That said, I've had overall good answers from either whenever I've decided to pull them out, it feels like wizard coder should be better since it's so much newer but overall it hasn't been that different. Wish phind would release an update :(

[–] [email protected] 2 points 7 months ago

That makes sense, thank you for sharing.

I tend to use CoPilot for IDE code completion, and I use the 34B models for automated refactors and code transforms where accuracy is a requirement.