Hi, please excuse my noob status.
Im looking at the viability of trying to run the AI Text Plugin locally, or one of the servers i have access to. In a non public context as an aid to fantasy story development.
I have seen the pages for the AI Text Plugin, the true one, and the forked version. Although their details are quite comprehensive i have yet to find or understand a clarity to the potential use of the plugin in an independant context.
So, to quote:
This plugin allows you generate text with AI. It uses a Llama-based AI model. It doesn't run on your actual device like other Perchance plugins because it requires too much computational power (and would require a many-gigabyte download), so it runs on server GPUs, which means it costs me money to run. For that reason, this plugin is funded with ads, so an ad will appear on your generator for non-logged-in users if you import this plugin. The ad will appear at the bottom of the screen like this. The ad will go away if you remove the plugin, of course.
To use this plugin, you'll first need to import it by adding this line to your lists editor:
So, is the Llama-based AI model something i can obtain ? Is it a specific model (which one)?
I understand all the aspects of high gig download, GPUs (needs GPU on text output? Im not really after the image gen, its the text ai i am interested in setting up locally, the character gen and output as available here: https://perchance.org/ai-character-chat (minus the image aspect)), cost and money to run, therefore ads.. absolutely i fully get and sympathise with that, server fees can be very expensive and high etc.
So it then goes on to importing in via lists editor.. im not entirely sure of the context, im assuming that is an accessible element (here via my account?) to play about with plugin ai's to create different generators? Or something i need to install locally to work with? *As i say please excuse what might seem a stupid question, im used to a different background of working, but have coding and server skills etc.
It seems a bit of a headache to try to recreate something already created, if i can download the elements i need to replicate and hybridise locally.. reduced right down if the image ai element is unused i assume?
Thank you for any help and advice! 👍
Sadly the model isn't public. Though it might be one of the base LLaMa models.
Yes, the purpose of the plugin is essentially a bridge from the client to the server. You can send instructions, and options through the function, then it is sent to the server and the response is displayed. Here's a simple example from the plugin page. No, you don't have to install anything locally for it to work. The LLM is on a server and we post requests and get responses from it.
The Perchance 'ai-character-chat' was based on OpenCharacters which you can run locally and possibly use other LLM for generation. Instead of being able to change the LLMs, the AI Character Chat instead used the LLM that is Perchance is using and only that. Additionally, on Perchance it also integrated their Text to Image to the chat that the OpenCharacters doens't have.
Ahh o.k thank you for pointing me in the direction of OpenCharacters.. that give me a bit of hope is i can make something at least partly similar or at least useful. The Main Perchant version is excellent and i could compose stuff all day. The AI response creating a real drive and enjoyment when working with it. I guess at one level i am scared it would vanish one day and after trying for such a long to time to find such a cool thing, i want to help perserve it, if that makes any sense. Hopefully it's future is looking great though, adapting and upgrading with new things over time. I would be happy to pay a subscritption fee to aid support, if it comes to that sort of support being required. 👍