Jennifer Jill Fellows: Open AI released ChatGPT three to the general public at the end of 2022. And within five days of its release, it had over 1 million users. It’s fair to say that today, three years later, ChatGPT, and other large language models embedded in Google, Ben, Facebook, Snapchat, Adobe, and well, everywhere have become ubiquitous. They promise to increase our abilities and capacities in a whole host of ways. But what they promise most is convenience. And convenience always comes with a cost. So welcome to another episode of Cyborg Goddess. I’m your host, Jennifer Jill Fellows. Today, my guest is Dr. Nicole Ramsoomair, and she is going to help me answer the question of what the convenience of large language models are costing us.
JJF: Dr. Nicole Ramsamir is an assistant professor in the Department of Philosophy at Dalhousie University. She defines herself as an academic and mom who worked through 24-hour workdays to complete her PhD while raising two young children. Her research interests include applied ethics, social and political philosophy, feminist philosophy, and personal identity. And she has a special interest in freedom of expression. Today, Nicole is here to talk about her research work on generative AI technologies and epistemic injustice.
JJF: Hi, Nicole. Welcome to the show.
Nicole Ramsoomair: Ah, thanks for having me.
NR: I’m joining from Halifax, Nova Scotia and the Mi’kma’ki and the traditional unseeded territory of the Mi’kmaq People.
JJF: Wonderful. I would also like to invite my listeners to think about physical space required to sustain this digital podcast. The servers and cables that connect myself and Nicole today are built with material extracted from stolen land and occupy stolen land. They depend on water for cooling and energy to keep working. Our digital tools and lives are not separate from physical reality and increasingly exert an immense pressure on physical space in ways that are often hidden from the user. So to make that hidden space somewhat less hidden, I acknowledge that this podcast is recorded on the unceded territories of the Coast Salish people, including the territories of q̓íc̓əy̓ (Katzie), qʼʷa:n̓ƛʼən̓ (Kwantlen), kʷikʷəƛ̓əm (Kwikwetlem), xʷməθkʷəy̓əm (Musqueam), qiqéyt (Qayqayt), Skwxwú7mesh (Squamish), scəw̓aθən (Tsawwassen) and səlilwətaɬ (Tsleil-Waututh) Peoples.
JJF: So, Nicole, can you tell me how you became interested in pursuing a PhD in philosophy?
NR: Generally, my story is pretty typical as I took philosophy back in high school, and I honestly just never stopped. I always thought that I’ll just keep on going until something stops, and fortunately I haven’t. . . I haven’t had to stop yet, so I keep on it.
JJF: That’s so cool. Not all high schools offer philosophy courses, but I think that it’s really cool when a high school can do that. Whether people continue on or not, like, obviously, I’m biased, but a little background in philosophy, I think, is always a good thing. And obviously it hooked to you.
NR: Yeah, I think my first paper was on the existence of God, and I really enjoyed that one.
JJF: That’s awesome. And so you went ahead and did a bachelor’s degree. Was it in philosophy directly?
NR: I did a bachelor in English and philosophy. I think at first, I wanted to teach high school.
JJF: And then you were still hooked, so you kept going?
NR: I thought, Yeah, it just it’s just a year for a master’s might as well, and then it just kept on going after that, so . . .
JJF: Alright, cool. So you’ve been hooked since high school, which is amazing. I feel like when I was in high school, I didn’t even know what philosophy was, so I think that’s awesome. But more recently, you’ve become interested in exploring large language generative AI models. So probably the most familiar one for a lot of users would be either the Snapchat AI or ChatGPT or something like that. So how did you become interested in examining these generative AI models from an epistemological perspective?
NR: Well, I think it was actually a couple of years ago I started thinking about personalization and autonomy and how those two function together. And even before we had ChatGPT, we were seeing more and more of this personalization going into these little rabbit holes of echo chambers where you don’t get really exposed to much else that your advertisements are personalized and music is personalized. What you watch is personalized. And I had started thinking about that for a little while, but then when ChatGPT came around, I was starting to see it used as a sort of resource for knowledge, and that I found a little concerning because it’s one particular avenue in which persons can go and . . . yeah, it seems to be that kind of personalization to the extreme in these cases.
JJF: Right.
NR: Yeah.
JJF: So we already had and you’re right. Like, if we think about just kind of the algorithmic setup of a lot of our Internet tools and apps and things like that, we already had fairly strong personalization happening, even for other sites where people might go for knowledge. So, you know, before generative AI, a lot of my students would, like, Google the answer, even if they weren’t supposed to. Or you might ask Siri or Alexa to find something for you. So you’re using them not just for entertainment or for other reasons, but also to gather knowledge. And it does become personalized, right? So if you Google and answer from a different geographic location, for example, you might get a different result. And your search history is also supposed to play into that, although obviously this is all proprietary, so we’re trying to figure it out kind of from our end, from the user end, rather than necessarily knowing what goes into the back end. I think that’s really interesting, though, and you referenced echo chambers there and the relationship between personalization and echo chambers. And we’re going to talk about that more, I think, as we get into the conversation today. But, yeah, so we have now ChatGPT, which was increasingly being used as a knowledge tool, as you said, then large language model generative AI resources are also being put into other areas that we’re already accustomed to thinking of as knowledge areas, whether we should or not. So now when you Google something, for example, the very first hit is the AI generated answer, right?
NR: Yes.
JJF: And then there’s stuff that paid to be on the front page, and then there’s all the other stuff. So I think that’s really interesting is the idea of looking at this as not necessarily a tool for knowledge, but as something that’s being used as a tool for knowledge or being maybe framed as a tool for knowledge. Is that the right way of thinking about it?
NR: Yeah, I think it being framed as a tool for knowledge, or just even the convenience of it. It’s just a convenient, quick way to you don’t even have to click a website anywhere, as you said, but the generative answer you see right up at the top, that you don’t even have to look through an entire page. It just seems that it’s a means of obtaining knowledge in a way that just seems a little too convenient.
JJF: Then there is this personalization aspect that can divide us, because the answer I get might be different from the answer that someone else gets from a different geographic location or based on their search history or other tracking data on the way. Yeah, that’s before we talk about the personalized ads, which you did talk about briefly. Just even the content that you’re getting can be very specific.
JJF: Before we dive into how generative AI can perhaps create unique problems from an epistemological perspective, that is unique problems when it comes to knowledge, I want to do a little bit of a discussion of the background of some of the theoretical frameworks that you were bringing to this. So I want to do a bit of a discussion of what the social account of epistemology is in general. Like, what does it mean to say that knowledge is social or is a social activity?
NR: Knowledge as a social activity. Mainly I’m drawing from Miranda Fricker’s work. Where she discusses social knowledge or the knowledge that can exist within a community is something that is built and shared. She often uses the term pooling of knowledge that we all contribute to this larger pool of knowledge. And in this way that, honestly, I think this is actually what contributes to the success of humans in general, is that we have this pool of knowledge that we can access. If you threw me out into the woods right now, I would not know what to do. I would probably wouldn’t last any more than maybe a day or two.
JJF: Yeah, not unless I had, like, a YouTube video showing me how to build shelter and build a fire. But that’s exactly what you’re talking about, right? The pooling of knowledge. If I don’t have access to that pool, I’m gonna be in a lot of trouble. Even with access, I might not do that one.
NR: There was that one joke somebody had said that if I went back in time and tried to tell everybody, I was . . . I can’t remember who said it, but if I tried to tell everyone I was from the future, I wouldn’t be able to prove it anymore because if they asked me how do cell phones work? I would have no idea what to say to them.
JJF: It might as well be magic.
NR: It is.
JJF: Yeah, yeah.
NR: Nobody would believe me.
JJF: So one aspect of this is just that we rely on the expertise of others a whole lot just to kind of make it through our lives. But then there’s also, I think, just this idea of everybody getting to contribute and add to the pool of knowledge as kind of growing the pool and making there be even more stuff you can draw from. So I talked about YouTube videos. Like, I taught myself to crochet a few years ago by relying on YouTube videos. And this was really cool because my grandmother tried to teach me how to crochet when I was young, but she’s right-handed, and I’m left-handed. And she couldn’t teach me. Like, I wasn’t at the place where I could mirror it in my brain. And she didn’t know how to teach me using the other hand. But now I can find left-handed crocheters making tutorials online, and so it’s like if you don’t have those people in your immediate community to share that knowledge, the pool of knowledge with you, the pool is so much bigger now. And I feel like that was like a naive promise of the Internet in the early 2000s was like, and it has . . . like, I don’t want to say it’s totally naive. You do have a huge pool of knowledge with the Internet. There are a lot of problems, which we’re going to get into. But one thing the Internet has done is it’s really created this large pool of knowledge where you can go and read people’s posts and watch people’s videos and find tutorials on how to do things. And you can teach yourself a lot because you have access to that large pool. I think that helps me kind of understand what you mean by the social account of knowledge, right? We get to add to the pool, but we also get access to this huge pool, which means that I don’t have to teach myself things. I can go get a tutorial from somebody else, for example.
NR: Yeah, I think it’s also connected so much to feminist standpoint theory. I’m often influenced by Sandra Harding and even just her notion of strong objectivity, where here, she would say that all of her knowledge comes from a particular standpoint, and we don’t actually get a larger understanding of the world unless we do incorporate those perspectives from others. That strong objectivity for her is something. . . It’s not about having this view from nowhere. It’s a view from almost everywhere. We get a better sense of objectivity when we include those voices. So when we don’t pretend that we’re being neutral, but we’re actually including all of those, we actually get a better picture of the world. And in this way, when we say knowledge is social, it’s also dynamic that it has to sort of react and tract reality, and this is one way we can do it with a lot of voices. And the larger pooling of knowledge it can. . . it would be sensitive to the sort of lived experience, at least ideally, which as we’ll get into, is not necessarily what’s happening right now.
JJF: Right. So, ideally, like, a really simple example is that idea that, like, I needed somebody who was also left-handed to teach me how to do this, right? I needed that perspective. And, of course, we can see this also, so when we talk about feminist standpoint theory, the idea that your social location affects the kind of knowledge you have access to and affects your lived experience of reality and even affects what you can claim to know, which maybe will come to a little bit later. But the idea that, like, existing in the world in a specific social category, yeah, affects what kind of knowledge you can add to the pool, and what kind of knowledge would be helpful for you to be able to draw from the pool, for example. Yeah. And so having more voices in the pool is helpful because hopefully there’s somebody like you in the pool who’s added something that you can pull out, right? So, we’ve got this idea of knowledge as a social activity. The idea that what we really want to try and do is have a huge amount of voices from people who have different social locations, who experience and see the world differently. And this feminist push to understanding objectivity in terms of this kind of different perspectives, different voices, as opposed to, like, a view from nowhere kind of homogenized view, I think is really helpful. But there are a few problems, even before it comes to large language models that we can identify that make it harder for societies to generate and share knowledge, right? So now let’s talk about some of the problems. We’ve talked about the ideal. What are some of the problems that actually happened in reality?
NR: Yeah, so the ideal would be everyone gets to. . . I think Miranda Fricker says a fair crack at it, at contributing to this pool of knowledge. The problem is, it can be quite exclusive, and it does seemingly not include all the voices that could and this is due to things like Miranda Fricker’s understanding of testimonial injustice and hermeneutical injustice that according to Fricker, testimonial injustice happens when somebody’s credibility is unfairly discounted, mainly due to bias, and it is bias on the basis of one’s identity, not because there are many situations in which other persons would be more credible, like the one on YouTube, who’s showing you how to crochet. Or even somebody a survivalist who might know how to live in the woods in a way I wouldn’t be able to. I would trust their testimony in those cases. And it makes sense why they would have a surplus of credibility. However, when it comes to testimonial injustice, persons’ testimonies are discounted not because of a lack of knowledge, but because of who they are and the kinds of stereotypes that track them. So if they try to contribute to that pool of knowledge, what we get is they get excluded from it, that their testimony is not given as much credibility. And then we have the sort of complimentary injustice there with hermeneutical injustice, which would mean that pool itself is not going to be covering all experiences. That pool of knowledge that we have, if it is only certain dominant perspectives that are allowed into that pool, then there’s going to be a lot of experiences that persons aren’t going to be able to. . . They look to that pool, and they’re not going to find it.
JJF: Yeah.
NR: So if you were looking for a left handed crochet video. You would not find it in this case, because that’s more on the margins. And I think it’s really important because when you are being denied access or fair crack at this pool in the sense of hermeneutical injustice, where it’s a kind of disadvantage in the ability to know or having access to knowledge, is that it tracks you throughout your life. That maybe in some cases, I often use this example with my students that maybe there are stereotypes against philosophers, and I bet if I go to the University science department, there’ll probably be a couple of stereotypes right there, and be like, ugh, not another philosophy professor. But that doesn’t track me through my whole day in the same way as I’m somebody who identifies as a woman. So if I. . . that would track me through my education, through my personal life, my professional life, and so on.
JJF: Yeah.
NR: So if you’re having an inability to understand your experiences, it’s gonna track you throughout your life. And then if you don’t fully understand those experiences, you’re not gonna be able to talk on it or speak to it. So they’re somewhat mutually influential.
JJF: Or build a community about it, right?
NR: Yeah.
JJF: Because you can’t connect with other people who have similar experiences, ’cause you’re being prevented from sharing in that pool.
NR: Yeah.
JJF: Yeah. No, I think your example of the philosopher is really good because, yes, um, people have views about philosopher. One I’ve heard from a more science minded position is philosophers ask questions and never offer answers, and, like, that’s just a waste of time, right? But that doesn’t track my ability to be a knower everywhere, right? Like, sure, maybe some scientists are dismissing my knowledge claims, but certainly other people are not dismissing my knowledge claims, and I do have expertise with my PhD, and I do have the ability to add to a pool of knowledge. But I always think of, like, it’s a dated example, but I think it’s a very striking example. The example from Virginia Woolf from a Room of Ones Own, where she says, not only were women not allowed or not allowed to attend university classes when she was writing that. So women weren’t allowed to enroll. Women weren’t allowed to take classes. But she tells the story of being told to, like, get off the lawn at the college that she couldn’t even go to the library. She couldn’t be in that space. Obviously, libraries are a physical pool of knowledge. So not having access either to contribute to or to draw from that pool of knowledge is obviously going to affect her and did affect her throughout her life as she struggled to publish and write and add to the pool of knowledge. But it also affects all other women in that case or other people who might have other experiences that are similar to Woolf’s and she can’t share them, right? So that’s a really strong example. But there are more subtle ways we can be blocked from having access to knowledge, but that’s just a very strong example. And like you said, this often tracks biases, right? So people are not perceived as experts or not perceived as knowers, based on stereotypes, based on discrimination quite often. Sexism, racism, ableism, other issues.
JJF: So we know that people can be, um, experience injustice in their testimony, so their testimony isn’t recognized as being knowledge because of biases. And we know that people can be, experience injustice by not being able to add to or draw from the pool of knowledge. And that’s what you’ve called, following Fricker, you’ve called hermeneutic injustice. So now that we have kind of this idea of the social account of knowledge, testimonial injustice, hermeneutic injustice, let’s dive into our topic today. And let’s talk about large language models. One thing you’ve claimed is that large language models like ChatGPT can lead to what you call a homogenization of content. Can you talk about how they do this and how this might contribute to some of the injustices that you’ve identified for us?
NR: So when I talk about homogenization of content, I think when ChatGPT first came out, I was reading a little bit more about it, and they were saying a lot of the information that was being given was being drawn on what was mostly statistically relevant. And that I found just generally concerning because of these sorts of imbalances we see in the social pool of knowledge that if it is imbalanced, it seems that it would be likely that that imbalance would be reflected in the kind of data. And for the most part, this has panned out. To some extent. And in this case, we would see that AI systems wouldn’t just reflect bias, but they could actually sustain it if we use it as a source of knowledge, that it might allow for these sort of credibility deficits, credibility surplus if it keeps on feeding us these same sorts of knowledge. And if we are focusing on mainly what is statistically relevant, it makes me worried about those marginalized voices that are not necessarily being included into this larger conversation. That if we’re drawing from this pool, what sorts of voices are we going to take? And this is not to say that ChatGPT could never, or any of these LLMs could never include these kinds of voices, but it doesn’t seem to be the default. It seems that you have to ask it too, and if it is becoming convenient and more, and easier, as you mentioned earlier, that when you make a search, it’s right up there at the top. We don’t know how that search, uh, what the parameters of that search was or, I, even really know what the algorithm they’re using to generate that search. So we don’t really know what kind of statistics it’s drawing upon to give you that answer. So I think that is something that is very concerning that it homogenized content it perhaps might draw from the most dominant perspectives and provide those perspectives in a way that is readily convenient and easy to access.
JJF: Yeah, you’ve given me a lot to think about there. The first thing I want to draw on is this idea that it, like, picks up whatever is statistically most likely or statistically dominant. Because I’ve seen, for example, a criticism in the early days when it first launched. So I think this was GPT#, 4 has its own problems, but GBT3 when it first launched, one assumption that it often made, for example, was that administrative assistants would use she/her pronouns. Um, so if you asked it to tell a story about an admin assistant or something like that, it would default to she/her pronouns, whereas, like, doctors, lawyers, physicists, it would default to he/him pronouns. And so this was raised by a lot of people as, like, This is a red flag. Like, you’re telling us that expertise when it comes to doctors, lawyers, physicists is more likely to be found among men. Like, that’s sustaining a stereotype that could hurt the testimony of, like, female physicists, for example. And if I remember correctly, the response from Open AI at the time was like, but that’s the statistical truth. Like, it statistically is true. And this is the case in Canada and the United States that 90% of people who hold administrative assistant positions identify as women. And it is statistically still true that many other professions, like the ones I listed, tend to be more male dominated. So they didn’t see it as a problem. Like, they were like, This is correct. This is how this tool is supposed to work. But one of the things I heard you say is that in the tool working this way, it’s not just reflecting these potential stereotypes back at us but sustaining them, right? So it’s not just telling us, well, it’s true that 90% of admin assistants are women, but through the kind of homogenization, we’re erasing the 10% of admin assistants who are not women, for example, right? Like, they just become invisible. So I’m wondering, can we unpack that a little bit more. Like, this idea that it’s doing more than just reflecting biases back at us, that it’s sustaining them, can you speak a little bit more to that point?
NR: Yeah, I think the way these work and how they speak with a sort of authoritative voice that it doesn’t at least in the conversations I’ve had with ChatGPT and many of these, that it doesn’t come at you as this is a tentative stance, that it has, it speaks with this kind of authority, that it skims over some of the ambiguities and gives you a fairly definitive answer. And as I said, it’s very possible that we have and we will probably develop systems that are going to be able to sort of recognize nuance and perhaps if you filter it through enough that you can generate and ask it to look at some of the more marginalized perspectives. But the way I’ve been seeing, and AI has been integrated and everything, I think what I even see a fridge with AI integration?
JJF: Yeah.
NR: And it’s just getting integrated everywhere in the sense of convenience that I think is a problem because it doesn’t we don’t actually go back and ask about that, but we just get fed the same knowledge without having to sort of work for it to look at why and how that knowledge was developed.
JJF: Yeah, it’s all packaged up very neatly, isn’t it? And it doesn’t necessarily encourage nuance, which isn’t to say, as you said, that we couldn’t build models that encourage nuance or that are more tentative in their answers. But that might not be as easy.
NR: Yeah.
JJR: Right? So it’s like we’re trading off nuance for ease, which, you know, that tracks for humanity.
NR: Yeah.
JJF: Yeah. So, the other thing I wanted to talk about. . . So that’s kind of how it presents, that it presents very authoritatively, that it erases nuance, and we end up with the 90%, for example, of admin assistants, it just takes that as 100%, and we erase the 10% that don’t identify as women, for example. But you also were talking about, like, where the content is drawn for this. And this is something I’ve seen Timnit Gebru and Emily Bender talk about, as well that a lot of the content for a lot of the bigger, more commercial, more public large language models, that content came from the Open Web, right? And it, not just so happens, It just so happens that people with marginalized identities tend to not post as much on the Open Web because they get trolled. Like, the Open Web is not super friendly, right? I mean, I’ve been on there long enough to have people say, like, get in the kitchen and make me a sandwich. That’s one of the more polite things that people say to women on the Open Web. And then, of course, there’s a lot of racist stuff that happens and other stuff. And so there is a potential that the material that’s been used to create these tools is also not necessarily representative because it can’t go into things like closed Discord servers, closed Facebook groups, like subscription newsletters, stuff like that. Is that another concern you have when you’re thinking about testimonial and hermeneutic injustice?
NR: Yeah, absolutely. And I was even thinking about in terms of language and what is being represented as well. The majority of it is in English, and if it is going to scrub the Internet for a different perspective, it’s going to be primarily Western English speaking. And even the kinds of journals, any sort of biases that we have, it’s going to be likely to be reflected in that as well.
JJF: Right. So all the biases that already exist on the Internet, the Internet being dominantly English, the Internet being sustained largely in the United States and then spreading from there, so a very Western perspective being more dominant on the Internet, which is not actually true of the perspectives of 8 billion people on the planet.
NR: Yeah, ’cause you have to think of who has access to the Internet in those cases. So it’s going to be more wealthier countries as well. So there could be a class overrepresentation as well. Economic overrepresentation.
JJF: Yeah. So when we add in, like, language and the digital divide here. We end up with models that are definitely not sharing the entire pool of human knowledge. Right? Definitely. And because of their authoritative voice, not even acknowledging that there’s parts of the pool of knowledge that aren’t being shared.
NR: I guess as also a Canadian going into this election, one thing I. . . I’m not exactly sure how to theorize it, but with a level of disinformation and bots that we’re having now that is replicating that well, I shouldn’t say knowledge, but ‘knowledge’ and air quotes
JJF: Knowledge claims.
NR: Knowledge claims, that is being replicated and repeated over and over, so and especially at the times we are right now, certain leaders who might not always be verifying what they’re saying, but still have large platforms to say it. It makes me worried about where it’s getting its information, especially, and even now, we have an influx of AI generated content, which is now feeding on AI generated content.
JJF: Yeah. Yes. So now it’s not even representing us. It’s representing itself. Yeah. So, the more bots you have posting on Twitter and Facebook and other places. Sorry, X and Facebook and other places. And then, the more it scrapes data from the Internet to build the large language models, it’s bots feeding bots now, right? So we end up with even more and more homogenization of content and the loss of marginalized voices just because we’re all being drowned out by AI generated content, basically. Yeah, that’s deeply concerning. I’m laughing ’cause I can’t cry.
NR: I’ve seen conversations on what was a Instagram where there’s an AI generated image where bots are talking to bots about
JJF: About the bot generated imaged and Oh no!
NR: I don’t even know what I’m not even sure what to say about it. It’s so dystopian at this point.
JJF: We’re being drowned out by the expertise of the bots. That sounds bad. But one thing that we know from research and social epistemology is that one way we can try and address, identify and maybe in a fruitful direction, move past testimonial and hermeneutic injustice is to experience what José Medinacalls epistemic friction. So before we talk about epistemic friction in connection with large language models, can we talk about ideally what epistemic friction is and how it’s supposed to work?
NR: Yes, so Medina I really liked the term when I first came across it when he coined the term. It was this kind of friction that we have in conversation that causes us to slow down when we have miscommunication, it makes us go back trying to figure out what the other person is actually saying. So friction arises when our assumptions are questions, when we face unfamiliar ideas. And it is not just about generating discomfort, but that . . . let’s stop look back and try to figure out what the other person is saying. And this connects back to those sort of hermeneutical gaps and this kind of hermeneutical injustice because it is this friction that allows us to start seeing that there are limits to our knowledge base, that there may be things that we don’t know. You can think of friction, even in terms of Dubois or even Patricia Hill Collins with this notion of outside or within. They draw on a lot of well, I guess, they’re before Medina, but in this case, that they do discuss this kind of friction in having access to a dominant narrative and other kinds of narratives that you have within the community. And noticing the sorts of differences between them, the frictions between them allow them to start seeing the gaps in knowledge. So epistemic friction for Medina is this difficulty in conversation, which is actually productive and allows us to gain better understanding of that world because it might uncover parts of reality we might not necessarily be cognizant of. And we’re only cognizant of once we see these gaps, and once there is that failure of communication, that there is something there that isn’t being said.
JJF: Yeah. So the idea that, when you feel like you’re not totally understanding or you’re misunderstanding, or someone is having difficulty communicating to you, you’re having difficulty picking up on what they’re saying. That’s an indication that maybe there’s an aspect of the pool of knowledge that you haven’t accessed yet and that you’re not aware of, right? And so that friction and trying to figure out, wait, what’s happening? What is this person saying? So it’s not like a conversation breaking down, but it’s a conversation, getting messy, I guess. Does that make sense? Yeah. And so somebody says something from one point of view and you say something from another point of view, and you’re like, Oh, wait, wait. Like, this is. Yeah.
JJF: So that means that conversations across different perspectives within the pool of knowledge are really, really important for dealing with testimonial injustice and hermeneutical injustice. But you found in your research that large language models like ChatGPT generate what you call kind of a flattened voice. So how does that, maybe along with some of the other things we’ve said about the homogenization of content, reduce opportunities for this beneficial epistemic friction.
NR: So with homogenization, the problem there is that it homogenized or concerned what was being said, the content itself was coming from single source or what is statistically prevalent. Whereas a flattened voice is more just even about the language itself it takes on. That large language models. . . it can take on dialects of other regions. But generally speaking, if it is going to be trained on English, if it is going to be trained on the dominant perspective, how it speaks is going to reflect a dominant perspective, as well. And if there is ambiguity, when it encounters that ambiguity, it’s going to default to again, statistically prevalent. If this term is poorly understood, it’s going to be understood by the dominant understanding of that term. And yeah, so what we lose with the flattened voice that we have with ChatGPT is the idiosyncrasies of our own natural speech, that there may be ways of describing that. There may be even points where I might use a term that you might not know that I have a different understanding of that could even spark that kind of epistemic friction that we would have. So what we get with the flattened voice is even the way words are used, what kinds of words are used, how they’re used. Also contributes to the sort of flattening out of diversity that I think really concerns me with the use of AI for conversation and for knowledge building.
JJF: Mm hmm. So the idea that there, you know, like language can be quite malleable, quite nuanced, even if we’re just speaking about English. One example I’ve given on the podcast before is that here in Canada, this is kind of a silly example. Here in Canada, if I use the word fanny, it’s kind of like a silly word for your bum. But if I use that word in the United Kingdom, it’s a much more vulgar word. Right. And so, the same word can have a lot of different nuance and it really depends, like, which understanding of the term are you taking and running with, right? And that can cause friction. So if you travel even to other English-speaking countries or other English-speaking parts of Canada, even, people use words in different ways, and it can cause confusion and epistemic friction as you’re trying to figure out what the other person means by what they say, right? And so I think that the idea of a flattened voice, kind of, like, homogenizing or anchoring all of those terms would really lose a lot of the malleability of languages.
NR: And even the way we use a flat and voice I’ve been thinking a lot about more recently is, and especially in the school year, I’ve been noticing a lot of students almost email me with uh, ChatGPT. So they will email me ChatGPT, and then I started really thinking like, What are conversations that we’re having now? Are they mediate by ChatGPT? Am I sending an email out where I get a ChatGPT response? And then ChatGPT responds to that, that are we actually even talking anymore? So the level of convenience, it is convenient. This is emails is probably one of the worst parts of the day, just trying to answer them all and tackle that inbox.
JJF: Yeah.
NR: But at the same time, you’re not actually talking anymore. There’s not even not only is it we’re losing that moment where we have that misunderstanding, but we’re losing the conversation in a lot of these cases. I’ve even seen people say that they’ve used ChatGPT to talk to family members. Students are giving their essays in it that it’s become very prevalent in the way we talk and how we talk to persons. So if it has a flattened voice, it’s also flattening and sort of dulling conversation, I would say, as well.
JJF: Yeah, yeah. No, I think that’s really interesting. So we’re getting kind of a flattened voice, which is homogenizing and making the language less dynamic, making it more static. And that is really convenient. And part of the convenience, I think, is that it does eliminate or not eliminate, but reduce epistemic friction. And one of the ways it does that is by anchoring to the dominant, so that’s erasing marginalized views. Then it also, I feel like, is this right? Like, I’ve also seen this with some of my students and some other people that the more and more you see stuff written in this kind of homogenized flattened style, that even if you’re not using the AI tools, you feel compelled to, like, copy that style because it’s become the norm that this is what an email should look like, or this is what a paper should look like. And so even if people aren’t using the tools, their voices are still being controlled by the tools or influenced by the tools. Maybe influenced is a better word.
NR: Yeah, I would think so because yeah, just even in the kinds of emails I’ve been seeing, I’ve been noticing the exact same sort of words being used. I’m not 100% sure if it’s fully filtered through these systems, but it’s a lot of them sound the same, in a way that I haven’t encountered before.
JJF: Yeah.
NR: And it seems to becoming more prevalent. Even the essays itself.
JJF: Yeah,
NR: A lot of them are getting there’s less diversity, I would say, in what is being said in those maps.
JJF: Yeah. And how it’s being said.
NR: Yeah.
JJF: Yeah, so we’re losing the epistemic friction. Things are becoming very easy and convenient. And losing epistemic friction may feel good because epistemic friction being confused doesn’t always feel good. But that doesn’t mean it’s necessarily beneficial for us, right?
NR: Mm hmm.
JJF: And then we’re also just losing genuine communication with each other because we’re not really communicating anymore.
NR: Outsourcing it.
JJF: So we kind of already talked a little bit about echo chambers. I want to dive back in and bring that idea back in. This is, as you said, a concept that has plagued social accounts of knowledge online for a long time. It predates large language models. Really, the whole rise of the algorithm and the idea of, like, personalized content and personalized Internet really started introducing this concept of echo chambers into the Internet space. So can you talk about echo chambers? Perhaps we’ll just start, like, echo chambers and algorithms and how this is a problem for the social account of knowledge?
NR: Yeah. So echo tremors basically are informal environments where people only hear views that they already believe. And I would describe it more as a subversion of that sort of ideal marketplace of ideas as belief systems converge and they mutually reinforce each other and through acknowledged and peer conformation, so it could be, let’s say, on a social media space, you would have persons with similar beliefs, similar social locations, talking about the same things and giving the sort of false impression of general consensus on these topics. But the echo chamber doesn’t necessarily always have to be with other persons. I think this sort of personalization goes hand in hand, as well. That if we’re constantly exposed to only what we want to be exposed to or what the algorithm thinks that we are going to be exposed to, we’re not as I said, it’s in music, it’s in our searches. It’s in entertainment. Everything has that sort of personalized feature to it. So we’re constantly going to be exposed to the same sorts of ideas. And with this echo chamber, we’d lose, again, that kind of diversity that we’re supposed to have with this marketplace with the standpoint diversity that our beliefs would just go unchecked. It’s like swimming through water without any resistance.
JJF: Yeah, yeah. That’s kind of scary and really helpful. And like you said, it’s, it’s like the antithesis of what we were promised. So I’m old enough to remember, like, we were promised the Internet was gonna be this marketplace of ideas, and you could access anything.
NR: Mm hmm.
JJF: And, like, technically, that’s true. But that’s not how it actually pans out, because first of all, there’s just too much. People can’t access everything. There’s too much. So algorithms started filtering. And now we’ve ended up in these places where, yeah, you’re, you know, the videos that are recommended to me are similar videos to the videos I’ve already watched. And the music that’s recommended to me is similar to the music I’ve already listened to and articles that are recommended to me and people that I might want to follow and all this kind of stuff just kind of feeds into what the algorithm thinks I already like, which often actually is what I already like, and that does make it, as you’ve said, kind of deceptively easy. But that does mean that I’m missing out on the other voices in the knowledge pool. I’m missing out on potential epistemic friction because part of why I already like this is I understand it all and agree with it all.
NR: Yeah.
JJF: And so it feels great to have my beliefs reinforced back to me over and over again. But it’s not necessarily good for me in terms of, like, growing knowledge or allowing me to see diverse perspectives or things like that, right? Yeah.
NR: Yeah, it’s not. Like any Google or Facebook is there trying to their goal isn’t to assure a well-informed public or that your well-being is taken care of. The goal is to get you to engage.
JJF: Yeah, they’re not trying to help me grow.
NR: Yeah. You might get rage faded sometimes, which might be something that is opposite of what you think, but just get that engagement. And then, again, if it’s shown to you within a smaller circle, then you’re gonna get this idea of a false consensus.
JJF: Yeah, yeah. Yeah, so you think more people agree with you than maybe do, and we haven’t even talked about bots agreeing with you yet. We will in a minute.
NR: Yeah.
JJF: But, yeah, you get this false idea that your view is more dominant than perhaps it is or is shared very, very widely. And then, yeah, you talked about rage bait, which we might think of as a place for epistemic friction, but at least in my experience, this isn’t a place to have a nuanced conversation to try and figure out what you’ve misunderstood, right? Like, that’s not how that usually works. If you see something that is antithetical to what you already believe, it’s usually, again, fueling engagement because now you’re angry, and there isn’t a lot of space for, like, confusion, nuance or a deep discussion and a deep dive into, Well, why do you believe this thing that’s so different from what I believe? Like, those kind of good faith discussions don’t really happen in a lot of these spaces.
NR: No, ’cause it filters out the extreme.
JJF: Yeah.
NR: It’s very adversarial in the way.
JJF: It’s either extreme agreement or extreme disagreement is kind of what you’re drawn into.
NR: Yeah. And that’s what it’s prioritized. It’s what’s gonna get people most angry or most upset. Or most likely to engage with it, so. It’s even programmed not to have that. That sort of nuance that more invitation to dialogue rather than this is going to get people going.
JJF: Yeah. Yeah. I really liked your point that, like, these spaces aren’t really interested in developing us as well-rounded human beings with deep pools of knowledge. Yeah. That’s not what’s going on.
NR: And you have to it is I think it’s, yeah, worse since the pandemic because that became our whole space. That was our whole public sphere for a while. Yeah, now we live a lot online.
JJF: Yeah. So, okay, let’s bring let’s bring the bots in. So now we have not just algorithmically driven search engines, but also as we’ve already mentioned, Google has its AI overview. Bing has its Copilot answer. Lots of people are just straight up asking ChatGPT instead of using a search engine at all. And we have all the bots online in all of our online spaces. So how do you think this might affect people’s susceptibility to echo chambers or affect the strength of echo chambers, maybe?
NR: Well, I think it’s just again, I want to go back to that idea of convenience. That convenience is a problem that. . . well, it’s not a problem. It’s good. We can do things more efficiently. ChatGPT, a lot of these searches allow us to have these really quick access to information, that promise of the 90s Internet that we had that we’re going to be connected to everybody. But part of the problem is just it’s almost too easy that you don’t get fed any of the any sort of countervailing. . . There isn’t any opportunities really to have your beliefs disconfirmed. That even now with that AI answer right at the top of the Google search, you don’t even have to go through pages of information. You don’t even have to skim a paper to actually try to find what you’re looking at. You just look for the keyword and you can get right to it. You don’t have to skim. So it’s a little bit more that not only is there the content being homogenized, we have the flattened voice, but even added here, that the convenience reduces opportunities for epistemic friction. That there isn’t, that you just have this one stop shop, it’ll answer you. Done. And we’re losing that those opportunities.
JJF: Yeah.
NR: And, again, don’t really know how the algorithms are working. So we haven’t engaged in that process. We don’t know what has been prioritized. Or even if we can look at it, again, this convenience isn’t really conducive to that. I don’t think many people when they get that hit and they get the answer from the AI overview are going to really dig deeper into it.
JJF: To be like, where did this answer come from?
NR: Yeah. Yeah, so it hides how it’s constructed, as well.
JJF: Are we granting a credibility? There’s a possibility, I think, of granting a credibility excess to these tools that maybe because they only ‘know’ in air quotes, what we’ve already put online, right? So anything they tell you is coming from somewhere. And some of the more sophisticated ones will try and give you sources. Sometimes those are real sources. In some cases, they’re made-up sources. Some of the tools are getting better at that. But the more mass-produced ones like AI overview, for example, it doesn’t necessarily tell you where any of this is coming from. And so you end up having to grant credibility to the tool itself, which seems not great to me that human beings are getting a credibility deficit, and the tool is maybe getting a credibility excess in some of these cases.
NR: Absolutely. And then, again, with the bots, it’s really hard to even tell when you’re actually talking to a flesh and blood human being there.
JJF: Yeah. Because we’ve already have spaces prior to these bots really rolling out in force. As you said, we already had spaces that really did not encourage deep nuanced engagement. And so now that the bots are in those spaces, too, and all the engagement is superficial anyway, it’s really hard to tell.
NR: Yeah.
JJF: Yeah.
NR: And the online etiquette has really sort of changed how we even interact with people, too. But maybe this is a whole other can of worms.
JJF: That’s the next research.
NR: Yeah.
JJF: So we have one more concept that you talked about in your research that I want to ask you about. So we know we’re now in a situation where LLMs are exacerbating testimonial injustice, exacerbating hermeneutical injustice through this flattened voice, the homogenization of content, the credibility excess that maybe we’re giving these tools. And making echo chambers easier to fall into because you said there’s just such a convenience now you don’t even have to look for your people, right? Like, it’s just there. The LLM is just feeding it to you. And they’re making it really hard to produce and share knowledge, especially for marginalized folk, because as we said, even speaking in English, our communication is being policed. So it’s harder to have different Englishes or different expressions of the way you want to say something, how you want to say it, and what you want to say it. So this is all really wild when we think about how these things are often marketed as being tools for knowledge as being something that’s going to make knowledge production and distribution easier. But it looks like they aren’t necessarily tools for knowledge, but as you kind of point out in your work, they’re tools more for upholding and maintaining certain forms of ignorance. And so I wanted to talk about philosophical theories of ignorance and how that might change our perspective on large language models.
NR: Absolutely. So with philosophers like Charles Mills, Linda Martin Alcoff, they discuss ignorance in a way that shows that it’s not just this sort of accidental ignorance of things we don’t know that we might miss something that even when we talk about hermeneutical gaps or these gaps in knowledge, it could be just something that’s accidental, maybe nobody paid attention to, but these are structured. There are narratives. I think it’s Hilde Lindemann Nelson calls them master narratives that we have within the larger discourse, are the stories we tell, even the legal decisions that we have and so on, that we have these larger narratives that allow us to remain ignorant of what we don’t know. That it justifies. . . So not only are these larger narratives shutting out these more marginalized perspectives or more or less deferred to the larger narrative, it is actively upholding that ignorance and gives reason why you might distrust the other sources that. . . Examples are, it’s easier to lack sympathy for inmates or ignore systemic problems in the prison system if you believe all incarcerated people are violent criminals. It’s easier to believe colonization was beneficial if you already think it was for their own good. So these are all different types of epistemologies of ignorance, so ways of knowing or kinds of cognitive misunderstandings that uphold this kind of injustice, and it allows you what it’s saying allows you to remain ignorant of these other perspectives. I think Thi Nguyen in the paper itself has this term, the sort of epistemic inoculation. It can, by believing these narratives and seeing these sort of repeated to you, you become resistant to those counter narratives that you’re being told. And I think, especially without getting into it too deeply, but in the kinds of politics we’ve been seeing now, these seem incredibly prevalent, that there are ways of understanding that when you’re seeing many people deported, you’re just told they’re criminals. And that makes it okay to override their fundamental rights.
JJF: Yeah.
NR: But not getting too deep into that, but always that in the background.
JJF: No, but I think that was a really it is a really prevalent master narrative. Not only the master narrative that people who are fleeing their countries and migrating to somewhere else are somehow criminal or doing it illegally or. . . because there’s a difference between illegal and irregular, but I’m not going to get into all that. . . So, yeah, I really think that the idea of master narratives as contributing to ignorance is a really important one to highlight. And I liked the example you gave, particularly of the idea that, like, criminals are all violent or maybe can’t be rehabilitated and deserve what they get. Like, I feel like that master narrative is upheld in so many of our, like, police procedural dramas or movies about this, like, bringing criminals to justice. And it’s even upheld when it’s, like, from the criminal’s perspective. Like, there are a whole bunch of movies like glorifying violence and assassins and stuff like that. That are a certain type of very idealized criminal behavior, right? Video games. And then also it’s upheld in mainstream media by the stories that are told about incarcerated people and stuff like that. And so I think that’s just one example. But when we have these very strong, prevalent master narratives that are repeated both in fiction and in non-fiction over and over again, it makes it harder to see things like, you know, criminals are human beings. They should have human rights. They deserve respect and human justice and things like that. Like people don’t want to hear that because it goes against the master narrative. And so the idea of that kind of role that master narratives can play in allowing people to sustain ignorance and to avoid listening to other perspectives, avoid considering them, whether they get adopted or not, but even just considering them, I think is really important. And if I’m understanding, tying that back to large language models. If large language models tend to portray dominant views, they would tend to portray master narratives of one sort or another. Is that part of the idea?
NR: And even just feeding it to you in a way that. . . the way even ChatGPT works in a lot of these models, because this is a private company. They are trying to make money. Again, not for your well-being. So they’re going to feed you what you want to hear, for the most part. So they’re going to fit these narratives. And if we do have these, it doesn’t challenge you and yeah, so it might show up in the content, and it also might show up in this idea of not challenging you to consider those sorts of outside perspective. It’s going to give you that same narrative that allows you to ignore the other perspectives that might disconfirm that otherwise. So.
JJF: So it’s a master narrative that fits your prompt, right? So if I ask it, for example, what are some reasons to distrust vaccines? It is more than happy to give me all of those narratives and probably won’t say, By the way, most scientists in scientific research agrees that vaccines are safe or whatever. Yeah.
NR: They do have some of those, and when I played around with them.
JJF: they have changed it a bit yeah.
NR: It does push back, but it still has that model of choose one answer you prefer, and it’s going fully sort of, like, shunts you into a particular way of thinking and how it’s being said and so on. In a way you prefer.
JJF: Yeah, so that you can avoid any friction yourself.
NR: Yeah.
JJF: So now that we know how these large language models operate and the effects that they’re having and might come to have on the social production of knowledge. What can we do, Nicole?
NR: Well, I think, when I was looking into it, a lot of these sorts of technical fixes don’t seem to be enough in these cases that you can have reweighting of data, audits, synthetic data generation, actually, and these solutions can only go so far because what we really need is that diversity. We need to have that friction reintroduced back into these conversations. So even if it can be trained to detect these kinds of gaps, I find it difficult to see how they would detect these sort of hermeneutical gaps because it’s. . . these almost by definition, are gaps that we’re not aware of that we don’t have words for. So it’s not exactly clear to me how that kind of interaction would ever be replicated by an AI system because we don’t have the words for it. How would you necessarily program that?
JJF: Yeah, yeah. So if I’m thinking about, like, one of Miranda Fricker’s really early examples of this is that it wasn’t until women in the workplace got together and had conversations with each other about their experiences in the workplace that they were able to develop the concept of workplace sexual harassment. Like, that wasn’t a concept that legally or socially existed until individual women got together in communities and were like, Hey, so there’s uncomfortable stuff happening to me or around me at work, and this is what’s happening. Oh, that’s happening to me, too. I am also uncomfortable by it or made uncomfortable by it. And the concept developed through this kind of discussion. But if that concept just isn’t there, then it can’t be represented by a large language model, right? So that if that hermeneutical resource of the idea of workplace sexual harassment isn’t there, then these large language models can’t tell us about it, right? And so we might be able to fix some of the injustices that the large language models have through better and more diverse training data, through maybe less personalization, through things like that, but that won’t necessarily address the fact that certain marginalized voices are being silenced and being prevented from adding to the hermeneutical. . . or to the pool of knowledge, right? Like, that problem is still there, even if we do get better systems.
NR: Yeah, the convenience problem also comes in again because we’re sort of not. . . the way we use it is pushing us away from that. So even if we could fix some of the biases within the system itself, how we’re using the system is also encouraging the loss of epistemic friction in these cases. So it’s, in these multiple different ways that it can harm. And this is why I think that we do need to develop better literacy around it and more knowledge about how to interact with these systems. And to really have users understand its limitations of ChatGPT of these large language models that the AI overview isn’t going to necessarily give you even a correct answer. I’ve seen some really funny ones. So in a lot of these cases, we need to sort of reintroduce that standpoint diversity. And one way we can do that is training people in how to know. So. Again, that’s a really tall order to have everybody using it and trying to really think critically about the kind of answers that they have especially with this convenience, which people tend to go with what’s mostly convenient and the answer that they’re given. So I think that is necessary to understand and reintroduce that and understand the ways in which those systems do marginalize those voices, be aware of it. I think Miranda Fricker had at the end of her larger book was this idea of the virtuous hearer. That you need to develop these sort of sensitivities that somebody may be struggling trying to say something. And again, really difficult to see how these large systems will be able to develop that kind of sensitivity
JJF: Or encourage that in us.
NR: Yeah. But even if it did. Yeah, even if it did develop it. Using ChatGPT is outsourcing that sensitivity to others.
JJF: Yeah. So it feels like and this is something I feel like you’ve been talking about this whole through this whole conversation is like we are definitely losing a skill set if we make quote unquote knowledge acquisition too easy or too convenient. Not only are we losing actual knowledge through testimonial injustice, hermeneutical injustice, echo chambers, manufactured ignorance, all that stuff. But we’re also losing a skill. And before we outsource this skill, we should really think very carefully about what we’re losing and what we’re handing over to large tech companies, right?
NR: Yeah.
JJF: In making this so easy and convenient and in losing the skill of acquiring knowledge, of listening virtuously, of wrestling with epistemic friction. Like, what happens when we lose that skill? Yeah. I think that’s a really good question to ask and to really think about.
JJF: I want to thank you so much for sharing your research on social epistemology, epistemic injustice, and large language models with us today. Is there anything else, Nicole, that you’d like to leave the listeners with?
NR: Yeah, I guess, building off of that is basically that convenience always does have its drawbacks, and we should think about that epistemically, as well. That I guess, with our ancestors, finding sweet foods used to be really difficult. But now, with modern technology, sugar is everywhere.
JJF: But it’s not necessarily good for us.
NR: Exactly. Amazon made it very easy, and I know everything shipped to your door. You don’t really have to even think about it. You just click, and it’s there. You don’t have to go to the store.
JJF: Very convenient.
NR: Easy. Everyone can access it, but then think of the personal financial costs, environmental costs.
JJF: And the exploitation, right?
NR: Yeah. And we should think of the LLMs and the use of ChatGPT in the same sort of way that we’re trading convenience for something, but I think there is something being lost here and there are going to be these kinds of drawbacks. And that is not to say, I do actually think this access to ChatGPT and other kinds of AI tools are actually a really good thing. It’s democratizing. There’s a lot of journals that are behind paywalls. There’s a lot of access to information that most people cannot, cannot find access to. They might not be able to get the kinds of books you would get at a university library. There’s. . . it does allow for access to information, but we also need to sort of even contend and try to think about what we’re losing with this use. And I do think there are some of these epistemic losses. That it becomes too convenient. That we lose that sort of ability or lose that avenue for epic friction. So even if we don’t necessarily lose that ability, we don’t have many avenues to really even exercise it or to have opportunities to encounter that kind of epistemic friction. I guess friction or difficulty is not something we should always rush to eliminate in these cases, something you should really. . .
JJF: It’s okay to be uncomfortable.
NR: Think twice about it.
JJF: Yeah. No, I do really like that message that all convenience always comes at a cost, and that will be true here as well. And so we need to really acknowledge and wrestle with the cost to make sure that when we are accessing the convenience, it is mindfully and worth it, I guess, worth the cost. Because tech companies don’t want to tell us about the cost.
NR: No. Yeah, they, they don’t. . . and I guess that’s the final thing they don’t have our well-being in minds.
JJF: Yeah. They aren’t concerned with growing our pool of knowledge.
NR: No. Absolutely.
JJF: I want to thank Nikki again for sharing her research on epistemic injustice and large language models like ChatGPT with us today. And thank you, listener, for joining me for another episode of Cyborg Goddess. This podcast is created by me, Jennifer Jill Fellows, and it is part of the Harbinger Media Network. Music is provided by Epidemic Sound. You can follow us on X, formerly Twitter, BlueSky or follow me on Mastodon. Social media links are in the show notes. And if you enjoyed this episode, please consider leaving us a review because it really helps. Until next time, everyone. Bye.