A recent study by researchers at the Oxford Internet Institute has raised concerns about the use of Large Language Models (LLMs) in chatbots. LLMs have been designed to generate helpful and convincing responses, but they come with no guarantees regarding their accuracy or alignment with fact.
The paper published in Nature Human Behaviour highlights that LLMs are treated as knowledge sources and used to generate information in response to questions or prompts. However, the data they are trained on may not be factually correct. This is because LLMs often use online sources which can contain false statements, opinions, and inaccurate information. Users trust LLMs as a human-like information source due to their design as helpful, human-sounding agents.
The researchers emphasize the importance of information accuracy in science and education and urge the scientific community to use LLMs as “zero-shot translators.” This means that users should provide the model with appropriate data and ask it to transform it into a conclusion or code rather than relying on the model itself as a source of knowledge. This approach makes it easier to verify that the output is factually correct and aligned with the provided input.
While LLMs will undoubtedly assist with scientific workflows, it is crucial for the scientific community to use them responsibly and maintain clear expectations of how they can contribute. The paper highlights that LLMs can hallucinate, generating false content and presenting it as accurate, posing a direct threat to science and scientific truth. Therefore, researchers must ensure that they are using these models ethically and responsibly.
In conclusion, while Large Language Models (LLMs) may be useful tools for generating helpful responses, researchers must be aware of their limitations and take steps to ensure their responsible use in chatbots and other applications. By doing so, we can harness their potential while avoiding potential pitfalls that could harm our scientific workflows and truths.