A note on the detection of LLM-generated political speeches

Abstract

We use a recent data set (ParliaBench) containing political speeches as well as LLM-generated versions of such speeches to investigate whether the latter can be detected as being LLM-generated. We find that, for this data set, a linear, interpretable classifier can easily do so, achieving an F1 score of around 0.96 - 0.98. The ability of a classifier to distinguish LLM-generated text from human-written text is significant since it has many potential uses, for example in detecting potential undisclosed use of LLM-generated text in political speeches.