Programmer Humor

19623 readers

92 users here now

Welcome to Programmer Humor!

This is a place where you can post jokes, memes, humor, etc. related to programming!

For sharing awful code theres also Programming Horror.

Rules

Keep content in english
No advertisements
Posts must be related to programming or programmer topics

founded 1 year ago

MODERATORS

[email protected]

793

Normal day in the life of a developer (lemmy.world)

submitted 9 months ago by Dehydrated to c/[email protected]

41 comments fedilink hide all child comments

you are viewing a single comment's thread
view the rest of the comments

[–] [email protected] 2 points 9 months ago (1 children)

Run an open source one. Training requires lots of knowledge and even more hardware resources/time. Fine tuned models are available for free online, there is not much use in training it yourself.

Options are

https://github.com/oobabooga/text-generation-webui

https://github.com/Mozilla-Ocho/llamafile

https://github.com/ggerganov/llama.cpp

I recommend llavafiles, as this is the easiest option to run. The GitHub has all the stuff you need in the "quick start" section.

Though the default is a bit restricted on windows. Since the llavafiles are bundling the LLM weights with the executable and Windows has a 4GB limit on executables you're restricted to very small models. Workarounds are available though!

[–] virku 1 points 9 months ago

Im gonna give llamafile a go! I want to try to run it at least once with a different set of weights just to see it work and also see different weights handle the same inputs.

The reason I am asking about training is because of my work where fine tuning our own is going to come knocking soon, so I want to stay a bit ahead of the curve. Even though it already feels like I am late to the party.