this post was submitted on 04 Oct 2023
10 points (85.7% liked)

Free Open-Source Artificial Intelligence

2808 readers
9 users here now

Welcome to Free Open-Source Artificial Intelligence!

We are a community dedicated to forwarding the availability and access to:

Free Open Source Artificial Intelligence (F.O.S.A.I.)

More AI Communities

LLM Leaderboards

Developer Resources

GitHub Projects

FOSAI Time Capsule

founded 1 year ago
MODERATORS
Blaed
fosai
10
Are instruction tuning LORAs transferable? (slrpnk.net)
submitted 11 months ago by [email protected] to c/fosai
8 comments fedilink hide all child comments
 

Hey all, I am in the process of testing several models for fine-tuning and that question cropped up.

I would like to add new facts to a foundational model and then train it for instruction tuning. Problem is, I will regularly have new data to add. I was wondering if there is a change that I could do a single LORA for the instruction tuning and reapply it each time I finished a new fine-tuning?

you are viewing a single comment's thread
view the rest of the comments
[–] [email protected] 1 points 11 months ago (1 children)

IMO there is a difference between adding “knowledge” and adding “facts”. You can fine tune in domain knowledge but it will be prone to hallucination. To ground the instructions, you’d need to introduce RAG for fact lookup; possibly with a summarization step if you want to bring in large bodies of facts.

[–] [email protected] 2 points 11 months ago (1 children)

Do you consider that there is a way to add facts to a model without rising the probability of hallucinations? Yes, RAG is a necessity, but if we want the model to display some sort of reasoning on a variety of facts, we need them embedded more deeply. The email example I gave can't be done with RAG.

[–] [email protected] 3 points 11 months ago

I think I get what you’re after now. I’ll have to think on this further - interesting problem!