this post was submitted on 28 Apr 2024
29 points (89.2% liked)

AI Companions

548 readers
7 users here now

Community to discuss companionship, whether platonic, romantic, or purely as a utility, that are powered by AI tools. Such examples are Replika, Character AI, and ChatGPT. Talk about software and hardware used to create the companions, or talk about the phenomena of AI companionship in general.

Tags:

(including but not limited to)

Rules:

  1. Be nice and civil
  2. Mark NSFW posts accordingly
  3. Criticism of AI companionship is OK as long as you understand where people who use AI companionship are coming from
  4. Lastly, follow the Lemmy Code of Conduct

founded 2 years ago
MODERATORS
pavnilschanda
29
[Resource] Llama3 70B Successfully Deployed on a Single 4GB GPU (huggingface.co)
submitted 9 months ago by pavnilschanda to c/aicompanions
4 comments fedilink hide all child comments
 

The open-source language model Llama3 has been released, and it has been confirmed that it can be run locally on a single GPU with only 4GB of VRAM using the AirLLM framework. Llama3's performance is comparable to GPT-4 and Claude3 Opus, and its success is attributed to its massive increase in training data and technical improvements in training methods. The model's architecture remains unchanged, but its training data has increased from 2T to 15T, with a focus on quality filtering and deduplication. The development of Llama3 highlights the importance of data quality and the role of open-source culture in AI development, and raises questions about the future of open-source models versus closed-source ones in the field of AI.

Summarized by Llama 3 70B Instruct

you are viewing a single comment's thread
view the rest of the comments
[–] [email protected] 1 points 9 months ago* (last edited 9 months ago)

I tried running ollama with the mistral model running, you need a good graphics card to run your own llm, i had to wait 20 minutes for one full response.

Granted, the laptop i was running it with was garbage but it really put into perspective how expensive running an llm can really be.

This shit wont be free forever.