Instead, our current models usually guess what the user intended. Ideally, the model would ask clarifying questions when the user provided an ambiguous query.These issues arise from biases in the training data (trainers prefer longer answers that look more comprehensive) and well-known over-optimization issues. The model is often excessively verbose and overuses certain phrases, such as restating that it’s a language model trained by OpenAI.For example, given one phrasing of a question, the model can claim to not know the answer, but given a slight rephrase, can answer correctly. ChatGPT is sensitive to tweaks to the input phrasing or attempting the same prompt multiple times. Fixing this issue is challenging, as: (1) during RL training, there’s currently no source of truth (2) training the model to be more cautious causes it to decline questions that it can answer correctly and (3) supervised training misleads the model because the ideal answer depends on what the model knows, rather than what the human demonstrator knows.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |