ChatGPT and Gemini develop unique writing styles similar to humans

New research reveals that popular AI chatbots like ChatGPT and Gemini have developed distinct writing styles, or “idiolects,” that can be identified through linguistic analysis. This finding challenges assumptions about AI uniformity and has significant implications for detecting AI-generated content in educational settings and forensic applications.

What you should know: Linguist Karolina Rudnicka used computational methods to analyze hundreds of texts about diabetes generated by ChatGPT and Gemini, finding clear stylistic differences between the models.

• The Delta method, a standard authorship attribution technique, showed ChatGPT texts had a linguistic distance of 0.92 to other ChatGPT content and 1.49 to Gemini content.
• Gemini texts showed a distance of 0.84 to other Gemini content and 1.45 to ChatGPT, indicating distinct authorship patterns.
• These measurements demonstrate that each AI model has developed its own consistent writing style, similar to how humans have individual speech patterns.

The big picture: AI chatbots don’t just average their training data but develop distinctive linguistic habits that mirror human language development patterns.

• ChatGPT favors formal, clinical language with phrases like “individuals with diabetes,” “blood glucose levels,” and “characterized by elevated.”
• Gemini uses more conversational, accessible language with phrases like “high blood sugar,” “blood sugar control,” and “the way for.”
• ChatGPT uses “glucose” more than twice as often as “sugar,” while Gemini does the opposite.

Why this matters: Understanding AI idiolects could revolutionize how we detect AI-generated content and understand machine learning development.

• Teachers and educators could use these patterns to identify when students submit AI-generated assignments.
• Forensic linguists might need to adapt their methods to account for AI-generated text in legal contexts.
• The phenomenon suggests AI models develop “emergent abilities” not explicitly programmed during training.

Key details: The research focused on specific linguistic markers that distinguish the models’ writing styles.

• Analysis of trigrams (three-word combinations) revealed consistent patterns in how each model structures sentences.
• “Blood glucose levels” appeared only once in Gemini’s entire dataset, while “high blood sugar” appeared 158 times in Gemini compared to just 25 times in ChatGPT.
• Both models showed preference for certain sophisticated verbs and adjectives, with ChatGPT overusing words like “delve,” “align,” and “underscore.”

What researchers think: The development of AI idiolects may result from computational efficiency principles or self-priming mechanisms.

• The “principle of least effort” suggests models continue using familiar words and phrases once they become part of their repertoire.
• Self-priming may occur when models become more likely to use words they’ve recently generated, similar to human speech patterns.
• These idiolects might evolve with model updates, creating new challenges for detection methods.