The interface of OpenAI’s greatest creation is purposefully plain, in stark white and green. At first glance, it does not appear to be powered by the fifth most powerful supercomputer in the world. ChatGPT, an artificial intelligence chatbot developed by the company, is a large language model tool that may upend academia, computer science, and the business world.
Released on Nov. 30, 2022, the program accrued over 100 million users by January, making it the fastest-growing consumer application to date. It is free to the public, requiring only a valid email address and a linked phone number to sign up for an account. All a user must do is type in a statement, query, or question — the model then responds, even typing its words out with a hesitancy and flickering cursor that feels almost human.
Far from being a mere conversationalist, however, ChatGPT can also write essays, generate code, and take exams. It operates by searching vast repositories of data on the Internet and using input from its users to string words together. Therefore, it does not ‘think’ in the same way a human does, but rather does so by scanning available data and coming up with the most likely association between words.
As soon as the chatbot was released, instructors voiced concern about its potential for plagiarism. After all, every response that it generates is unique, making it difficult for teachers to tell whether a student’s work is based on ChatGPT or entirely created on their own. Additionally, the bot can generate code for introductory coding assignments, creating the potential for fraud and deceit.
The reason why ChatGPT can do first- or second-year assignments, as well as clear high school-level exams, is because a vast number of students are given such homework or exams. Therefore, a large repository of potential answers exists on the Internet — if you know where to find it. As one might imagine, these solutions, and therefore ChatGPT’s output, may not always be accurate, which has become a major complaint about the service.
After spending the first few months of the year ignoring the language model, I read about its ability to generate essays and was instantly piqued. Like most NJIT students today, I wrote out dozens of essays during high school; did the creation of programs such as this mean that English and humanities are obsolete? What about programming, or even math?
To understand the program’s capabilities further, I asked the chatbot questions that an NJIT student might face across multiple disciplines. To be clear, every exam question I asked is publicly available and published on NJIT’s past Common and Final Exams page, which provides past practice tests for students to use while studying.
As one might expect, ChatGPT is very good at giving responses to calculation-based or basic calculus and algebra-based questions. For example, one of the practice questions from a 2018 Calculus III exam involves finding the partial derivatives of an expression, which is relatively easy to type into the text entry box. For this, ChatGPT rapidly provides an accurate answer.
However, other questions further down the page are more application-based and involve changing the order of integration, sketching and characterizing the area to be integrated, and then doing the actual integration. I have no doubt that careful phrasing could allow a user to get a meaningful output; however, when larger expressions or problems are entered, such as this one, it may have problems understanding the question and, as a result, shows an error message.
Prior to this experiment, I had believed that ChatGPT could only ‘solve’ problems to which solutions already existed on the Internet. This is true, in a manner of speaking — a reliable answer is more likely with a larger data set. Thus, asking the bot for the answer to a unique question may not get a reliable answer.
I experienced this myself when I asked the chatbot to give me a MATLAB code to plot a series. For my first few tries, I used common expressions which would be used for learning how to use the programming language. After that, I tried one that I thought was obscure. The code that it generated was not functional, generating many iterative errors.
The key takeaway, however, is not that ChatGPT cannot do more advanced and unique problems — it can do that and give solutions that wouldn’t easily be found on the internet through a simple search. The question is, rather, whether those answers are fully accurate and whether the program has even understood your question fully. I found that giving the system an assumption first and then asking follow-up questions resulted in greater accuracy.
Although the program could deliver the answer to such problems, it will probably not render math exams useless. After all, many subjects do not allow students to have access to computers during exams. However, it may serve as a valuable resource when completing homework, especially as it fine-tunes itself.
Similarly, although ChatGPT may be able to generate code for some computer science questions, the lack of resources available online for some more complex problems means that the output is buggy or nonfunctional. Thus, the most interesting use of ChatGPT may actually be in the humanities department, as the model is able to generate essay responses based on the user’s specifications.
I asked it to “write an essay analyzing Hester Prynne’s character development in The Scarlet Letter, citing quotes and events,” which is similar to questions asked in high school English. It was able to generate a response that would probably get at least a passing grade. Yet it was bland and dull, taking the most common opinions from online and putting them together.
In some ways, it can be easy to tell when a response is generated by artificial intelligence, because there is nothing new or fresh within that essay — after all, it works by pulling together words and phrases that have been used together within its data repository. However, the application is not all bad; it can be extremely helpful to students who want to understand how to do a math problem, for example, as it can answer follow-up questions as well. Only time will tell how university departments adapt to this double-edged sword.