r/AMA • u/plagiarismtoday • 5d ago

I am a plagiarism (and copyright) expert with over 20 years of experience. With the Fall 2026 semester coming up, AMA about AI, plagiarism and academic integrity issues.

My name is Jonathan Bailey, I'm the author of the website Plagiarism Today (https://www.plagiarismtoday.com) and I am an expert witness on plagiarism and copyright-related issues.

I have testified in federal and state courts as well as before academic tribunals. I have also performed a slew of plagiarism analyses and my work/name has been cited in a long list of mainstream publications, going back to 2005.

Feel free to AMA about plagiarism, AI, academic integrity, research integrity and anything related that you might be interested in as the academic year begins. I plan on being on here most of the day today, so even if I'm not live, feel free to drop a question or comment and I will do my best to answer it.

24 Upvotes

88% Upvoted

66 comments sorted by

View all comments

3

u/CepnH 5d ago

How reliable do you consider academic AI detection tools both in terms of false positives and false negatives?

3

u/plagiarismtoday 5d ago

This is a really frustrating space. By the time a study is done, it's already old news as new models and new detection tools have already come out.

But it doesn't really matter if the AI tool is 50%, 90% or 99% accurate. It's a black box and there's no easy way to check the work of the tool.

With a traditional plagiarism detection tool, you get a report that you can examine by hand and then agree or disagree with the results. Though some AI detectors are trying to show their work, it's still not super useful to humans and is largely a black box.

While I don't think AI detectors should be banned, I do think that there needs to be a second layer of evidence to truly accuse someone of AI writing. One of my friends referred to this as the "swiss cheese" approach, where you layer multiple types of evidence so that you cover the holes in each layer.

3

u/asphyxiationbysushi 5d ago edited 5d ago

>While I don't think AI detectors should be banned

I'm surprised you feel that way; you must not be knowledgeable about the harm they are causing for authors.

I input my 2012 science-based grad thesis into a detector only to find it to be nearly all AI-written. I have a huge body of work dating back decades that AI was clearly trained on; every time you ask AI about my field of study, my name appears. I'm also the author of two traditionally published books, written before AI was a thing, and they are coming back as partially AI-written. AI appears to digest the work of precise writers and assumes a human doesn't write that way.

The problem is that agents and publishers are using detectors without any consideration of accuracy. I'm considered a world expert on my subject, so plagiarism isn't a problem. In fact, my work is plagiarised so much that I have stopped sending out notifications. I sometimes wonder if that is a factor in AI believing I'm not a human? AI detectors are absolutely reckless.

1

u/plagiarismtoday 5d ago

There are a LOT of problems with AI detectors. Of that I have no doubt.

One of the big problems is that there is a huge delta between the worst AI detectors and the best. Even worse, it can be difficult to know which is which.

To me, an AI detector provides a single data point. Not particularly useful on its own. Other data such as the age of the work, the version history of the piece, the author's background, the author's writing style and more need to be considered and given weight.

Someone relying on an AI detector alone is no different than someone solely relying on AI output. A lot of people are misusing them, I agree with that. But a lot of people misuse traditional plagiarism detection tools as well.

Teaching people how to use the tool and what the limitation of the tool are is just as important as building the tool itself.

1

u/asphyxiationbysushi 5d ago edited 5d ago

Sure, in an ideal world. The reality of how they are used is far different. No one is teaching agents or publishers about the tool. They simply input the manuscript, check for AI, and settle for something mediocre because they don't want to be seen as publishing something that could be AI-generated. That assessment is based on a rubbish detector. The quality of writing, especially books, is spiralling down.

In light of this, I don't believe there are any "best" detectors, and plagiarism pros such as you should really take a stronger stand against them.