7 Chapter 7: Students Using and Outpacing AI

In this chapter the focus is on challenging students to identify and work on those very things that AI cannot do, and to use AI as a tool where appropriate in their efforts. Solving problems that currently have no solutions, or at least none that people agree on, is an area in which humans may struggle, but out of that struggle it is at least possible that something new will emerge. Mimicking the patterns of words in existing text will not come up with a new solution for climate change. And even the things that an AI can do, such as explore all possible configurations of atoms in certain types of molecules, will only do that when humans drive the process, and if it produces something of significance, that significance will only be recognized and verified in the eyes of the human beholder. In short, an AI may be able to produce a literature review, although what it produces will need to be checked for accuracy and completeness. If AI can help students understand the current state of our knowledge more quickly and accurately than before, the next step is an obvious one: get more of our students trying to break new ground already at the level of undergraduate research. Whether they succeed is not the point. The learning happens not because a student comes up with a genuinely new and workable solution to climate change or some social problem, but in the process of trying to do so.

Tackle Wicked Problems

Get students to work together to try to come up with innovative solutions to the world’s problems in ways that are informed by the humanities, but which may also engage with the natural and social sciences. The one thing that humans are at least sometimes capable of that LLMs are not is genuine innovation.^[1] I will never forget the time I asked one to provide me with science fiction plot ideas that have never been used before. What it offered were some of the most well-worn tropes in the history of the genre. An LLM’s inability to innovate is not a criticism. It does what it does rather impressively. On the other hand, the way it is constrained by what humans have produced before is incredibly heartening for those looking for the value of human beings in the AI age.

Find out what issues students care about and send them into the databases (the virtual stacks) to see what has been proposed but not implemented, and to try to come up with something that has yet to be proposed. Their innovations, especially at an early level, will not be vastly different from their prototypes in almost all instances. That is okay. Fans of a particular band who start their own band will emulate the style of their heroes. In a similar way, students’ first attempts at innovation will be highly derivative.^[2] The point is to require students to try to innovate. In order to do so, they may use AI, but they cannot rely on it to produce content that meets the requirements of this assignment.

Go Local

When it comes to things that are only available in a student’s experience and those of other individuals in a particular place, there is much less chance that an LLM will be able to bluff about them convincingly.^[3] That is why getting students to apply broadly applicable skills locally makes so much sense. Have them visit sites and report on the experience. Have them take photos of local instances of religious or political expression and analyze them. I asked ChatGPT about the meaning of the “I’m voting for the felon” bumper sticker and it was vague and general. It did better when I asked it about the bumper sticker that says, “I bought this before I knew Elon was crazy.” It had no clue about the meme with a cat pushing an elephant off of a table.

We tend to think of archival research as something that advanced graduate students and professional researchers do. It can be but doesn’t have to be. Undergraduate research in local archives is often possible and can be meaningful not only to students but the maintainers of the collection and the wider public. Many institutions have poorly-catalogued collections of papers and objects connected with their history. Even the ones that have them organized well will often tell you that they do not get the attention they deserve. There are individuals of the past whose stories have not been told widely or at all and yet who deserve to be. There are individuals who, because of their identities in marginalized groups, have not received the same attention as others. There has been a long-overdue shift away from focusing on important white men in classes on history and literature. There are still plenty of those who are neglected, to be sure. Add to them the many people whose skin color, gender, religion, nationality, or language in the past stood in the way of their voices being heard, and you will realize that there is a lot of work to be done. Students can do some of this work.

Undergraduate research in the humanities, perhaps resulting in publications that educators co-author with their students, is a real possibility. That possibility is pursued much too infrequently. Doing this kind of work that is of value to students and society, and perhaps also to your own career path as an educator, would be worthwhile even if LLMs had not appeared on the scene. Their arrival constitutes just one more reason to pursue this, since local archives with obscure and neglected materials are not accessible to AI. LLMs will bluff if asked about them, to be sure, but they are bound to mess up, in the same way that a bluffing student also will. Thus, once again, we offer you a type of assignment where you do not need to police inappropriate AI usage. Just evaluate the student’s submission for what it is.

There are also things that students can do that blur into the social sciences but still provide opportunities for meaningful engagement with humanities content. They can gather data and conduct interviews on campus. Think about how often we as educators are surprised by how our students think and what they assume. Having students conduct interviews and report on the results gives them useful skills and provides useful information. When it comes to anthropology, observing life on campus or in your city or town can be similarly instructive. Students can train to do ethnographic research elsewhere by observing people where they are. And of course, if they have the chance to study abroad, they can do likewise in other places. After all, “local” doesn’t necessarily mean where their educational institution is located, it means wherever they are. For online courses, the value of students undertaking the same type of local research in different places and then sharing it with one another may not only lead to their learning, but uncover differences between what is the case in different places that are worthy of further investigation, perhaps leading to publication.

Foreign Language

If you have used an app like DuoLingo recently, you’ll be aware of the potential of AI to facilitate language practice. All you need to practice a language is a conversation partner. In the most widely spoken modern languages, LLMs do very well. So far in this book, you have seen what LLMs do in English. The facts may not be right, but the vocabulary and grammar typically are. If you have never thought about the fact that much of the internet is in other languages, now is the time. Your students can work on their language skills at any time of day and benefit from conversations with LLMs. They can also use them and other AI tools to provide translations of academic sources in other languages than their own. Even with the familiar Google Translate, I have found myself able to get the gist of articles in Korean and Russian that I would otherwise have had to simply set aside and not cite in my research.

Reading sources in other languages also adds the cultural context of that language and its native speakers. They might view, interpret, or value the same events, texts, or images differently than your students, purely because of a different perspective. Or, their response might be to give the same perspective, but to frame it or describe it in a completely different way. Languages often have different turns-of-phrase that visualize the same ideas using wildly different metaphors. That can be fun to explore. For example, in Hindi (or Punjabi), we say “ullu ka pattha” which translates directly as “an owl’s son” but it means someone who is foolish or dumb. However, in English, an owl is typically associated with the ideas of wisdom and knowledge in Western traditions. Reading texts with this phrase will lead to drastically different interpretations if a student is not careful—and provide a wonderful opportunity for them to learn. In this example, students learn not only about differences between languages and cultures. They learn about the need to avoid assuming they have understood someone else’s meaning, and to avail themselves of the internet to find the meaning of words and phrases that puzzle them. As students navigate their way through this experience, it can be both instructive and eye opening. Once again, working in this “gap” between an AI’s (in this case, translation) skill and our humanities understanding is a key point in the book. Here is another opportunity to highlight that to students. We are never going to reach a point where AI is autonomously conducting our business and political interactions on our behalf. We need people able to communicate across cultural and linguistic differences. If you have ever had a conversation relying on Google Translate, you have probably found it both wonderful and frustrating. On the one hand, having your phone listen to a phrase or take a photo of a sign in another language and translate it for you was until recently the stuff of sci-fi. On the other hand, you wouldn’t want to have that technology mediate all your interactions for a long period of time. Helping students realize the benefits of language learning for their futures will help them succeed. Even if you do not teach a foreign language course, asking them to find and engage with materials in other languages will help them learn not only language but research skills and how to collaborate with AI effectively. This idea is even more interesting if you happen to know an additional language that students do not. You can engage with the source text directly and see how much students can think their way through a tricky or erroneous translation. The best part is that they can’t verify what they’ve done is right, so they have to infer that based on their own knowledge of the content.

My own field has me working not only with modern languages but also ancient ones. AI at present does less well with most of these. This is because it has less of a textual base to learn from. If you paste Syriac text into ChatGPT and ask it to translate, it will offer you a translation. Sometimes it is spot on and sometimes it has nothing whatsoever to do with the text you gave it. Unless you know the language, you won’t know which is which. That said, there is plenty of room to get students involved in exploring the potential and the limits of LLMs for the purpose of translation.

There are an incredible number of texts and inscriptions which have been published but which have never been translated. Translators have experimented with Greek and Latin, two languages whose online corpuses are large enough that an LLM may be able to produce a translation that is not bad.^[4] An important question is whether the LLM is able to do so because there are also English language translations online. The only way to tell is to experiment with LLMs and untranslated texts and inscriptions. There is an opportunity here for students at many levels to get involved in producing something that can be published. The LLM will do the translation, and the student can offer analysis of the text, perhaps with the help of the LLM as well. The LLM cannot do this work on its own, and a student pursuing this will need to have humanities research skills and will improve them through their work on this project. Exceptional students will do much more, and perhaps more students will excel as a result of the opportunity to work on projects like this. Imagine being an undergraduate student and being responsible for a surge of attention to an inscription or papyrus fragment whose significance had gone unnoticed until you provided an initial translation and commentary. There will be a need for better translations and commentaries thereafter, but that might not have happened any time soon were it not for this step. AI will also provide tools that can help identify similar handwriting in fragments published separately, things that a human eye might not have noticed.^[5] For an even more extreme version of this sort of work, see the work of Brent Seales where he explains the fancy 3D renderings necessary to read the Herculaneum papyri, among others.^[6]

Textual Analysis

Finding patterns in human speech—such as noticing significant echoes by one author of another, quantifying word usage, and tracking down relevant parallels—has historically been the work of the best scholars. Progress relied on human beings becoming familiar with large amounts of texts and noticing similarities. The first digital humanities project was an attempt to harness the power of computers to document the vocabulary of a large corpus and facilitate study thereof. That project, the index to the Corpus Thomisticum, involved Jesuit priest and scholar Roberto Busa collaborating with founder of IBM Thomas Watson. It has never been true in the history of computing that technology is inherently a threat to the humanities. Rather, it is a good thing to use technology to accomplish what we cannot and facilitate study. The technology that makes that possible is not the problem. How humans use it and misuse it often is. And it is important to remember why, and in which ways, humans can streamline their work and focus on doing that which AI cannot.

For example, I asked ChatGPT the significance of a particular word cloud, without indicating anything further. All I gave it was the image. It recognized that the word cloud was derived from the beatitudes in the Gospel of Matthew. Recognizing such a connection would normally require the attention of a learned scholar or a lot of time. In either case, it would have been an onerous or difficult task for a human being. AI was able to resolve this “query” easily. Using images of words rather than typed text, or scanned articles and book chapters without OCR, has long created problems of accessibility for students with visual impairments. Advances in AI will continue to make learning more accessible. I share this here, however, to emphasize why an LLM is a really powerful and useful tool when the focus is on what it was designed to do. As a source of information, it just happens to be right relatively often. As a means of engaging with patterns in human speech, you’re getting at the heart of what it was designed to do.

The humanities are full of questions that scholars have not had the time to investigate but have long thought would be worth pursuing. I have in the past thought that I might get one of my students to work on one of them. It was clear that software such as the Thesaurus Linguae Graecae would need to be involved. Even if a student doesn’t know the relevant languages, they can do a search on keywords and then consult an English translation of the vast majority of texts relevant to such an undertaking. One such project relates to the Coptic Gospel known as the Gospel of Philip. In line 36 it talks about three Marys in the life of Jesus: his mother, a sister, and Magdalene who was his partner.^[7] The last word is a Greek loanword that can cover a range of meanings similar to the English word. It can denote a romantic significant other, a business partner, a companion, and so on. The context in that Gospel does not clarify which. There might be something in the pattern of usage that could be helpful. Is the word ever used in a platonic sense in ancient Greek and Coptic literature when it refers to the connection between an individual man and woman? That is the kind of question that is worth asking and that technology can help us answer. Projects or questions like this one can get university students involved in publishable research using AI tools under the supervision of their professors.^[8]

Working Virtually with Artifacts and Archaeological Sites

Archaeology involves digging into the ground, but much of the process of evaluating the significance of what is found and interpreting it happens as historians read reports about excavations and seek to correlate the new discovery with a wider array of humanistic data. We do not always succeed – after all, there are many objects that lack descriptions in library and museum holdings. Students may begin to participate in cataloging and describing these objects even while they have a relatively introductory level of knowledge, and the process of doing such activities will help them learn a lot about history, and aspects of the making and interpreting of historical knowledge beyond what comes across in textbooks. By participating at whatever level they are able to, they come to understand the process and not just the end result.

There are ancient sites that have received a lot of attention, and yet about which useful questions still remain to be asked.^[9] Getting students to compare images and descriptions from diverse site reports may lead to discovery. Things that a keyword search on Google or even in an academic library database would struggle to find, an LLM may be able to connect you with. Even with the need to fact check the details, an LLM may provide something that is genuinely useful, something that either might not come to light using older methods or would, at the very least, have taken significantly longer. You may also task students with using their own artistic skills, or working with others and perhaps also with AI, in order to represent ancient realities in modern replicas.

Theses

The unifying theme of this chapter is having students do more advanced work of their own, harnessing what LLMs can do not so as to alleviate the need for students to do work, but so as to fast track students to doing more advanced work. Can an LLM provide a decent literature review? At least sometimes, it does an impressive job, and even when its output is only partially useful it may speed up the process.^[10] We will come back to that question, but first let’s ask a more important one. Even if it were to produce an acceptable literature review, will an LLM be able to build on that foundation to offer pioneering insights? Of course not—or rather, only a human can prompt it to do so, and only a human can recognize the significance of anything important in the AI’s output. In most cases, only a human researcher can take that further step, but at the very least a human in the loop is essential to the process. The fact that an LLM has done a literature review will not be enough. The human student will need to understand what that material covers and what gaps (and thus potential new avenues of investigation) it points to. In particular, the fact that LLMs will likely never have access to all copyrighted publications means that an LLM literature review will not be sufficient on its own, although it certainly may speed up the process.

Let’s dig in a little deeper. Assistance by an AI may allow a student to move more quickly to more advanced research. This will not always be the case, but it has the potential to be. It is important to emphasize that all of the relevant research to date, as well as extensive anecdotal evidence, indicates that an LLM will not produce an acceptable literature review that could serve as part of an academic dissertation such as an undergraduate or masters thesis, to say nothing of a doctorate, without significant additional work by a human being necessary. AIso an LLM is almost certain to leave things out, and in keeping with historic biases, what it omits is more likely to be the work of historically underrepresented and marginalized groups. (Recall that this behavior isn’t intentionally nefarious on the part of the LLM or its developers, but more a consequence of the representation human creators and curators have added or failed to add to the volume of text data in commonly accessible corpuses.) If they include less frequently cited authors, they are particularly prone to fabricate citations for articles and books that do not exist.^[11] They can nevertheless produce lengthy summaries of the contents and significance of these non-existent works. You will likely require students to submit a research proposal at the start of the writing process that includes an outline and bibliography. If students rely uncritically on AI, or simply make up nonexistent works themselves, you’ll catch them at this stage with little effort.

The further along one gets in one’s studies, the more the literature review is about identifying what is lacking from previous scholarship. That requires human insight and imagination. An LLM can nonetheless sometimes help the researcher find relevant material and summarize it. The human participant(s) in the process must always keep in mind that there was significant bias in past scholarship as far as the underrepresentation of certain voices. An LLM, building on past linguistic precedent, will tend to perpetuate those biases. It will take deliberate human effort to ensure broad inclusivity, especially if the scholarship in question is sensitive to those types of concerns.

If students succeed in moving on more quickly from the first stage of research with the help of AI, they can utilize the power of AI to help them consider larger textual, visual, or other data corpuses so as to break new ground.^[12] AI cannot do this on its own, but humans using AI tools can accomplish things that humans without them simply cannot accomplish. There is no point in listing the possibilities here. They are endless, at once too numerous to meaningfully enumerate and too discipline-specific for a single author to meaningfully provide examples of.

More importantly, it is easy to forget that the current phase of AI is still very much in its infancy. Because it can do so many computations so quickly, it can evolve quickly. But LLMs are relatively new and the question of how they will contribute to humanity’s future is not yet known, despite the attempts of some to implement them in various capacities, often with disastrous (if at times also comical) results. What is needed is for the humanistic study of and engagement with these new tools. If all you do is have your students experiment with the capacities and limitations of LLMs and other AI systems and report on their findings, that in itself would provide a valuable service. More advanced students may be able to publish their results, but no experimentation is at this point anything other than a helpful contribution to the process. Whether the results are new breakthroughs in our understanding of texts and history, or new breakthroughs in our understanding of what AI can and cannot do, there is plenty of room for us and for our students to contribute in ways that those who simply give up engaging with AI and retreat, and those who uncritically implement it as educationally valuable, will not. Far from this being a moment for despair in and about the humanities, this is a moment when wide open research vistas vast and innumerable have opened up. The question is whether you and your students will have the courage to charge in and explore.^[13] This pioneering mapping of the terrain, like that of early cartographers, will undoubtedly ultimately be superseded. That isn’t a reason not to do it. We won’t get to the better, deeper, and fuller understandings without the work that needs to be done now.