Impact of Large Language Models on Scholarly Publication Titles and Abstracts: A Comparative Analysis
Teh, Phoey Lee and Uwasomba, Chukwudi Festus (2024) Impact of Large Language Models on Scholarly Publication Titles and Abstracts: A Comparative Analysis. Journal of Social Computing, 5 (2). pp. 105-121. ISSN 2688-5255
|
Text
WURO_Impact_of_Large_Language_Models_on_Scholarly.pdf - Published Version Available under License Creative Commons Attribution. Download (1MB) | Preview |
Abstract
Artificial Intelligence (AI) tools become essential across industries, distinguishing AI-generated from human-authored text is increasingly challenging. This study assesses the coherence of AI-generated titles and corresponding abstracts in anticipation of rising AI-assisted document production. Our main goal is to examine the correlation between original and AI-generated titles, emphasizing semantic depth and similarity measures, particularly in the context of Large Language Models (LLMs). We argue that LLMs have transformed research focus, dissemination, and citation patterns across five selected knowledge areas: Business Administration and Management (BAM), Computer Science and Information Technology (CS), Engineering and Material Science (EMS), Medicine and Healthcare (MH), and Psychology and Behavioral Sciences (PBS). We collected 15 000 titles and abstracts, narrowing the selection to 2000 through a rigorous multi-stage screening process adhering to our study's criteria. Result shows that there is insufficient evidence to suggest that LLM outperforms human authors in article title generation or articles from the LLM era demonstrates a marked difference in semantic richness and readability compared to those from the pre-LLM. Instead, it asserts that LLM is a valuable tool and can assist researchers in generating titles. With LLM's assistance, the researcher ensures that the content is reflective of the finalized abstract and core research themes, potentially increasing the impact and accessibility and readability of the academic work.
| Item Type: | Article |
|---|---|
| Keywords: | semantic alignment, cosine similarity, patterns, large language models, technology impact |
| Divisions: | Applied Science, Computing and Engineering |
| Depositing User: | Hayley Dennis |
| Date Deposited: | 09 Oct 2024 14:51 |
| Last Modified: | 09 Oct 2024 14:51 |
| URI: | https://wrexham.repository.guildhe.ac.uk/id/eprint/18219 |
Actions (login required)
 |
Edit Item |