I was demoing the system to a group of professors, and one of the questions that came up is if there was a way to basically annotate a chatbot response, say what they didn't like about the response, and then that automatically updates the prompt and inserts the Q&A into the knowledge base.
Presumably, the flow would go something like this: Prof annotates answer -> Annotates get fed into an LLM call that returns a json with the updated Q&A and an updated prompt -> insertIntoChatbotKnowledgeBase() and updateChatbotPrompt() (not real functions btw)
This is a kinda cool idea. Is it worth the work and maintenance? Idk. Probably not right now. It might even make a more confusing UX since you would both want to allow profs to just manually re-write the answer for the Q&A or provide annotations for the answer, and that might just add more confusion.
Maybe this idea could be expanded upon someplace else. Like some way of prompting an LLM to adjust the HelpMe course for you, like "disallow all materials marked with W7 to W10 from being synced with the chatbot".
I was demoing the system to a group of professors, and one of the questions that came up is if there was a way to basically annotate a chatbot response, say what they didn't like about the response, and then that automatically updates the prompt and inserts the Q&A into the knowledge base.
Presumably, the flow would go something like this: Prof annotates answer -> Annotates get fed into an LLM call that returns a json with the updated Q&A and an updated prompt -> insertIntoChatbotKnowledgeBase() and updateChatbotPrompt() (not real functions btw)
This is a kinda cool idea. Is it worth the work and maintenance? Idk. Probably not right now. It might even make a more confusing UX since you would both want to allow profs to just manually re-write the answer for the Q&A or provide annotations for the answer, and that might just add more confusion.
Maybe this idea could be expanded upon someplace else. Like some way of prompting an LLM to adjust the HelpMe course for you, like "disallow all materials marked with W7 to W10 from being synced with the chatbot".