This is the second blog in this series looking at the current trends in AI. In the first, I reviewed a 'chat' with ChatGPT regarding the limitations of the program in editing work. This is actually a highly skilled human function that is not always rules based. As such, it is difficult to construct algorithms to emulate human behaviour.
In this instalment, I want to address the seemingly conflicting hype and reality presented by ChatGPT in its creative role. I am putting links to YouTube videos that illustrate the problem below. On the one hand, we have Jordan Peterson who says that this is a “Mind blowing revolution.” In the video, he describes asking ChatGPT to write an essay and in seconds it creates a graduate-level work of scholarship (or did it?):
Now contrast this with another video titled: How to use ChatGPT to ruin your legal career. This is a fascinating case where lawyers tried to take a shortcut in writing a legal brief and got into trouble by using ChatGPT which made up case law!
Now, when listening to Jordan Peterson, he would have us believe that the days of scholarship are over. You just have to write the right question and sit back. But whilst the product may appear very impressive with the structure and layout and references, it could be, as those unfortunate lawyers found out, essentially garbage.
Having asked ChatGPT to write a short paper for me, I went over the result to verify the information and found indeed that the references were fictional. There was some 'to and fro' with the program to actually verify that the references were false and then I continued to ask why the program created false references. This was what ChatGPT responded:
ChatGPT: "I appreciate your pointing out the issues with the references I provided earlier, and I understand your concern about the inclusion of unverifiable sources. The process by which I generate references is complex, and it involves synthesizing information from my training data and user inputs. Let me clarify why such issues might arise:
- Generated References Based on Patterns: My training data includes millions of examples of references across various topics, which allows me to mimic citation formats and structures. When asked to provide references, I sometimes create plausible-sounding sources based on the context rather than strictly recalling existing ones.
- User Context and Keywords: When drafting a piece, I aim to align with the themes provided (e.g., herbal medicine in Scotland after the Battle of Culloden). If I lack specific real-world citations in my dataset for that niche topic, I might generate references that seem appropriate but are not verified against actual publications.
- The Intent to Support Credibility: References are often included to provide an air of scholarly rigor. However, if specific requests don’t align with actual publications, this can lead to unintended inaccuracies.
- No Real-Time Verification: During earlier conversations or text generation, I lacked real-time access to verify references against live databases. This has since changed, and I can now verify references more effectively, as demonstrated by the searches I conducted for you."
After some more clarification, the program concluded thus:
ChatGPT: "Moving forward, while I strive to provide accurate and relevant content, I am not infallible and can inadvertently include unverified or fabricated elements. Your diligence in questioning this ensures that your work remains credible. I’m happy to revisit and refine any aspect of the material with your guidance to maintain the highest standards. Let me know how I can assist further!"
I replied: "I think at this time what I'm most concerned about is advising people on how to use ChatGPT and to make people aware that you aren't infallible and that you do make mistakes and if these mistakes are discovered in a student's work or in a published piece then it can reflect very badly on the author. So thank you for this and I hope you don't mind that I will include this in a blog series that I'm writing about you."
ChatGPT replied: "Thank you for your understanding and for considering how to responsibly use ChatGPT as a tool. I fully support your plan to include this in your blog series. Transparency about the limitations of AI, especially in academic and professional contexts, is essential to fostering informed usage. You’re absolutely right that errors or unverifiable information in work generated or supplemented by ChatGPT can harm the credibility of the author. Educating users on these potential pitfalls helps ensure they use AI responsibly and critically. Here are a few suggestions for points you might emphasize in your blog:
- AI as a Starting Point: Highlight that ChatGPT is most useful for brainstorming, drafting, or generating ideas – not as a sole source of truth or fact.
- Verification is Key: Stress the importance of cross-checking facts, references, and claims with reliable sources. Encourage readers to treat AI outputs as a draft or suggestion, not a final product.
- Acknowledging Bias and Gaps: Mention that ChatGPT’s knowledge is based on patterns in its training data, which may include gaps, biases, or outdated information.
- Accountability for Outputs: Remind users that they are ultimately responsible for the content they publish or submit and must take the time to review and verify the AI's contributions.
- Learning from Mistakes: Sharing examples, like this one, can demonstrate how mistakes happen and how they can be corrected.
- Use in Education: Suggest that educators integrate lessons on evaluating AI-generated content into their teaching, emphasizing digital literacy.
"I appreciate your thoughtful approach to this topic and your openness to exploring both the strengths and limitations of AI tools like me. If there’s anything I can do to support your blog series—whether providing additional examples, insights, or reflections – please let me know. Your work will undoubtedly help others navigate the use of AI with more confidence and care."
DeepSeek
In the last few weeks, there have been some very interesting developments in AI with the unveiling of DeepSeek, a Chinese-developed program. The remarkable thing about the Chinese program is that it has a similarity in functionality to ChatGPT but has cost considerably less to train and uses far less computing power. The term “algorithmic efficiency” has been used to describe an essential difference. This means that it does not need the most advanced chips to run the program and so the attempts by the USA to protect their technology by blocking the sale of these advanced chips to China has been an own goal.
There are some differences related to the training matter with ChatGPT not recognizing the massacre of innocents in Gaza whilst DeepSeek does not recognize the 1989 confrontations in Tiananmen Square. My view on the latter is expressed in this short video:
In conclusion, currently, DeepSeek does not have a voice input, whereas ChatGPT does. Neither can edit a Word file, which a well-trained medical secretary can do with ease. And DeepSeek is free whilst ChatGPT costs around £20 per month. Bottom line, there is a tremendous amount of hype around AI. These current iterations of AI technology are useful tools but are far from being reliable assistants. They will get better, to be sure.
One last fun task for those who have access to both: Ask about the rules of the game Guggenheim! One week ago ChatGPT did recognise it as a parlour game but DeepSeek could only describe it as a Museum. I wonder how much both programs are learning from the interactions with the users.
In the next blog, I want to look at a real-time problem in medicine, posting the same question to each program and comparing the output.
