Hi there, If I'm looking to use LLM AI in a similar way like Stable Diffusion, i.e. running it on my own PC using pre-trained models (checkpoints?) - where would I start?

If I would want to have access to it on my mobile devices - is this a possibility?

If I would then later want to create workflows using these AI tools - say use the LLM to generate prompts and automatically run them on Stable Diffusion - is this a possibility?

I'm consistently frustrated with ChatGPT seemingly not beeing able to remember a chat history past a certain point. Would a self-run model be better in that regard (i.e. will I be able to reference somethin in a chat thread that happened 2 weeks ago?)

Are there tools that would allow cross-thread referencing?

I have no expert knowledge whatsoever, but I don't shy away from spending hours learning new staff. Will I be able to take steps working towards my own personal AI assistant? Or would this be way out of scope for a hobbyist?

you are viewing a single comment's thread
view the rest of the comments

[–] Solvena 1 points 1 year ago (1 children)

Thanks for the input!

I recently built a new PC to handle Stable Diffusion, that gives me 12GB of VRAM to work with. I also started to self-host a few things on a VPS recently, so I have a bit of a basis there.

As for Stable Diffusion integration: I do storytelling/worldbuilding as a hobby and find LLM's to be an amazing tool to "brainstorm with myself". It would be amazing if I could tell the LLM to "make a picture of the new character" and it would connect to and prompt SD accordingly. I assume that this is out of scope of what's currently possible, but something like that would be my goal. I will certainly have a look at langchain as you proposed. Also that's the context of me asking about cross referencing and context length. I've been working with ChatGPT, and while it is an amazing tool it had me bummed out when it couldn't reference a character that was developed a couple hours earlier (even in the same thread). The cross referencing solution that you sketched above might work for me, but I guess it'll take a while to learn how to do it.

Given this as a bit of context: where should I start? Downloading Llama 2 as another reply suggests and go from there?

[–] dtjones 1 points 1 year ago

Seems reasonable. I'll add in that there are models specifically finetuned for storytelling. You might check out this thread for some other model suggestions. I think you will also likely want to find a framework for RLHF.