Stack Overflow, a programming-related Q&A site, has temporarily banned users from posting answers created by the OpenAI AI chatbot ChatGPT.
The site’s moderators said the ban is temporary and that a final ruling will be made soon in consultation with the community. As the moderators point out, ChatGPT easily returns answers to questions, but they often appear correct at first glance but are incorrect upon closer inspection.
The fundamental issue is that while the answers generated by ChatGPT are frequently incorrect, they appear to be correct and simple to make. So, for now, users cannot use ChatGPT to create posts on Stack Overflow.
After this temporary policy is posted, if a user appears to have used ChatGPT, sanctions will be imposed to prevent them from posting such content.
ChatGPT is an experimental chatbot created by OpenAI based on the company’s AI text generator GPT-3.5. Last week, a demo of this bot was released, and it was greeted with enthusiasm on the web because of its ease of use, as it simply answers questions by typing on the site.
From generating poems, songs, and news articles, to answering trivia questions, to writing and debugging code, the bot delivers impressive, fluid results for various queries. It also has a very intuitive interface.
But while many users are enthusiastic about ChatGPT’s capabilities, many also point to its tendency to generate plausible but false answers. For example, asking for an explanation of how to develop software for a certain purpose may result in reasonable but ultimately incorrect code. This is one of the well-known drawbacks of AI text generation models known as large-scale language models (LLMs).
These systems are trained by analysing patterns from vast amounts of text from the internet. It finds statistical regularities in this data and uses them to predict the next word in any given sentence. However, this system also tends to generate nonsense texts because it has no hard-coded rules for how any particular system in the world should behave.
Given the huge size of these systems, determining what proportion of their output is faulty is impossible. And in the case of Stack Overflow, even a small percentage of misleading users was too high.
So yet, there is little evidence that LLM does harm in the actual world. But Stack Overflow’s decision confirms the argument that the scale of such systems creates new challenges.
The worry is that this pattern will repeat itself on other platforms, with a flood of AI-generated content drowning out real users with plausible but false data. And the stability of such a platform will depend on the nature of the platform itself and its moderation capabilities.
