Exploring the Impact of ChatGPT on Scientific Research: Assessing Strengths, Weaknesses, Opportunities, and Threats

Yousef F. Alfarraj; Yousef Wardat

doi:10.25159/1947-9417/16006

Authors

Yousef F. Alfarraj King Saud University https://orcid.org/0000-0001-7750-028X
Yousef Wardat Yarmouk University https://orcid.org/0000-0003-2370-9808

DOI:

https://doi.org/10.25159/1947-9417/16006

Keywords:

scientific research, ChatGPT, large language models, SWOT analysis

Abstract

One of the most influential technologies, ChatGPT, has extensive implications for several domains of our life, including scientific research. A SWOT (Strengths, Weaknesses, Opportunities, and Threats) analysis was conducted to evaluate the benefits and drawbacks of leveraging ChatGPT in scientific research. The results demonstrate potential advantages in terms of evaluation and assessment, individualised and continuous learning, linguistic competence, comprehensive knowledge, increasing accessibility, and information retrieval. At the same time, there are important shortcomings, such as the absence of contextual knowledge, outcome and information bias, and a limited advanced cognitive ability. The results point to several threats: plagiarism, academic dishonesty, ethical challenges, as well as cybersecurity and privacy concerns. Furthermore, the data point to various prospects, such as creating interactive settings, improving teaching and learning, contributing to literature, collaborative brainstorming, language translation, and knowledge sharing. These complicated considerations necessitate caution when employing artificial intelligence applications since the potential of ChatGPT to improve scientific research is dependent on how researchers utilise its strengths and opportunities while limiting its weaknesses and threats. Thus, great innovation is born out of smart technologies and smarter practices.

Author Biography

Yousef F. Alfarraj, King Saud University

Department of Curriculum and Instruction, College of Education, the Excellence Center of Science and Mathematics Education (ECSME), King Saud University, Saudi Arabia

References

Alfarraj, Y. F., and A. R. Althubyani. 2023. “Evaluation of Various Aspects of the 11th Grade Engineering Curriculum: A Mixed-Methods Study”. International Journal of Engineering Education 39 (6): 1400–1416.

Alneyadi, S., and Y. Wardat. 2023. “ChatGPT: Revolutionizing Student Achievement in the Electronic Magnetism Unit for Eleventh-Grade Students in Emirates Schools”. Contemporary Educational Technology 15 (4): ep448. https://doi.org/10.30935/cedtech/13417.

Alneyadi, S., Y. Wardat, Q. Alshannag, and A. Abu-Al-Aish. 2023. “The Effect of Using Smart E-Learning App on the Academic Achievement of Eighth-Grade Students”. Eurasia Journal of Mathematics, Science and Technology Education 19 (4): em2248. https://doi.org/10.29333/ejmste/13067.

Amal, S. F., I. A. Saiid, and H. Mansor. 2024. “An Empirical Study for the Dynamic and Personalised Learning Experience of the AI Course Generator”. International Journal on Perceptive and Cognitive Computing 10 (2): 23–30. https://doi.org/10.31436/ijpcc.v10i2.483.

Azaria, A., R. Azoulay, and S. Reches. 2024. “ChatGPT Is a Remarkable Tool—For Experts”. Data Intelligence 6 (1): 240–296. https://doi.org/10.1162/dint_a_00235.

Bai, L., X. Liu, and J. Su. 2023. “ChatGPT: The Cognitive Effects on Learning and Memory”. Brain‐X 1 (3): e30. https://doi.org/10.1002/brx2.30.

Balhorn, L. S., J. M. Weber, S. Buijsman, J. R. Hildebrandt, M. Ziefle, and A. M. Schweidtmann. 2023. “What Does ChatGPT Know about Natural Science and Engineering?” Preprint, arXiv, September 18, 2023. https://doi.org/10.48550/arXiv.2309.10048.

Benfatah, M., A. Marfak, E. Saad, A. Hilali, C. Nejjari, and I. Youlyouz-Marfak. 2024. “Assessing the Efficacy of ChatGPT as a Virtual Patient in Nursing Simulation Training: A Study on Nursing Students’ Experience”. Teaching and Learning in Nursing 19 (3): e486–e493. https://doi.org/10.1016/j.teln.2024.02.005.

Bin-Hady, W. R. A., A. Al-Kadi, A. Hazaea, and J. K. M. Ali. 2023. “Exploring the Dimensions of ChatGPT in English Language Learning: A Global Perspective”. Library Hi Tech, ahead of print. https://doi.org/10.1108/LHT-05-2023-0200.

Chinonso, O. E., A. Mfon-Ette Theresa, and T. C. Aduke. 2023. “ChatGPT for Teaching, Learning and Research: Prospects and Challenges”. Global Academic Journal of Humanities and Social Sciences 5 (2): 33–40. https://doi.org/10.36348/gajhss.2023.v05i02.001.

Çobanoğulları, F. 2024. “Learning and Teaching with ChatGPT: Potentials and Applications in Foreign Language Education”. The EuroCALL Review 31 (1): 4–15. https://doi.org/10.4995/eurocall.2024.19957.

Cotton, D. R., P. A. Cotton, and J. R. Shipway. 2023. “Chatting and Cheating: Ensuring Academic Integrity in the Era of ChatGPT”. Innovations in Education and Teaching International 61 (2): 228–239. https://doi.org/10.1080/14703297.2023.2190148.

Cox, C., and E. Tzoc. 2023. “ChatGPT: Implications for Academic Libraries”. College and Research Libraries News 84 (3): 99–102. https://doi.org/10.5860/crln.84.3.99.

Dale, R. 2021. “GPT-3: What’s It Good For?” Natural Language Engineering 27 (1): 113–118. https://doi.org/10.1017/S1351324920000601.

Derner, E., and K. Batistič. 2023. “Beyond the Safeguards: Exploring the Security Risks of ChatGPT”. Preprint, arXiv, May 13, 2023. https://doi.org/10.48550/arXiv.2305.08005.

Elmas, R., M. Adiguzel-Ulutas, and M. Yılmaz. 2024. “Examining ChatGPT’s Validity as a Source for Scientific Inquiry and Its Misconceptions Regarding Cell Energy Metabolism”. Education and Information Technologies 1–30. https://doi.org/10.1007/s10639-024-12749-1.

Else, H. 2023. “Abstracts Written by ChatGPT Fool Scientists”. Nature, January 12, 2023. https://doi.org/10.1038/d41586-023-00056-7.

Farrokhnia, M., S. K. Banihashem, O. Noroozi, and A. Wals. 2023. “A SWOT Analysis of ChatGPT: Implications for Educational Practice and Research”. Innovations in Education and Teaching International 61 (3): 460–474. https://doi.org/10.1080/14703297.2023.2195846.

Filippi, S. 2023. “Measuring the Impact of ChatGPT on Fostering Concept Generation in Innovative Product Design”. Electronics 12 (16): 3535. https://doi.org/10.3390/electronics12163535.

Ganguly, P., and S. Dutta. 2024. “Ethical Considerations on the Use of ChatGPT in Education”. Redshine Archives 6 (6). https://chapters.redshine.in/index.php/redshine/article/view/873.

Gao, J., X. Ding, B. Qin, and T. Liu. 2023. “Is ChatGPT a Good Causal Reasoner? A Comprehensive Evaluation”. Preprint, arXiv, October 12, 2023. https://doi.org/10.48550/arXiv.2305.07375.

Ghosh, A., and A. Bir. 2023. “Evaluating ChatGPT’s Ability to Solve Higher-Order Questions on the Competency-Based Medical Education Curriculum in Medical Biochemistry”. Cureus 15 (4): e37023. https://doi.org/10.7759/cureus.37023.

Giray, L., J. Jacob, and D. L. Gumalin. 2024. “Strengths, Weaknesses, Opportunities, and Threats of Using ChatGPT in Scientific Research”. IJTE: International Journal of Technology in Education 7 (1): 40–58. https://doi.org/10.46328/ijte.618.

Hidayat, R., and Y. Wardat. 2024. “A Systematic Review of Augmented Reality in Science, Technology, Engineering and Mathematics Education”. Education Information Technology 29: 9257–9282. https://doi.org/10.1007/s10639-023-12157-x.

Huang, K., and W. Ma. 2023. “Legal and Ethics Responsibility of ChatGPT”. In Beyond AI: ChatGPT, Web3, and the Business Landscape of Tomorrow, edited by K. Huang, Y. Wang, F. Zhu, X. Chen and C. Xing, 329–353. Cham: Springer.

Hwang, Y.-S., J.-S. Um, B. Pradhan, T. Choudhury, and S. Schlueter. 2024. “How Does ChatGPT Evaluate the Value of Spatial Information in the 4th Industrial Revolution?” Spatial Information Research 32 (2): 187–194. https://doi.org/10.1007/s41324-023-00567-5.

Jan, R. 2025. “Examining the Reliability of ChatGPT: Identifying Retracted Scientific Literature and Ensuring Accurate Citations and References”. In Impacts of Generative AI on the Future of Research and Education, edited by A. Mutawa, 367–392. Hershey: IGI Global. https://doi.org/10.4018/979-8-3693-0884-4.ch014.

Jarrah, A. M., Y. Wardat, and P. Fidalgo. 2023. “Using ChatGPT in Academic Writing Is (Not) a Form of Plagiarism: What Does the Literature Say?” Online Journal of Communication and Media Technologies 13 (4): e202346. https://doi.org/10.30935/ojcmt/13572.

Kalla, D., and N. Smith. 2023. “Study and Analysis of ChatGPT and Its Impact on Different Fields of Study”. International Journal of Innovative Science and Research Technology 8 (3): 827–833. https://doi.org/10.5281/zenodo.7767675.

Kasneci, E., K. Seßler, S. Küchemann, M. Bannert, D. Dementieva, F. Fischer, U. Gasser, G. Groh, S. Günnemann, E. Hüllermeier, S. Krusche, G. Kutyniok, T. Michaeli, C. Nerdel, J. Pfeffer, O. Poquet, M. Sailer, A. Schmidt, T. Seidel, M. Stadler, J. Weller, J. Kuhn, and G. Kasneci. 2023. “ChatGPT for Good? On Opportunities and Challenges of Large Language Models for Education”. Preprint, arXiv, January 29, 2013. https://doi.org/10.35542/osf.io/5er8f.

Kingdom of Saudi Arabia. 2021. “Media Document: Human Capability Development Program 2021–2025”. Accessed December 13, 2024. https://www.vision2030.gov.sa/media/pgid4z3t/2021-2025-human-capability-development-program-delivery-plan-en.pdf.

Kohnke, L., B. L. Moorhouse, and D. Zou. 2023. “ChatGPT for Language Teaching and Learning”. Relc Journal 54 (2): 537–550. https://doi.org/10.1177/00336882231162868.

Kooli, C. 2023. “Chatbots in Education and Research: A Critical Examination of Ethical Implications and Solutions”. Sustainability 15 (7): 5678. https://doi.org/10.3390/su15075614.

Kotsis, K. T. 2024. “ChatGPT as Teacher Assistant for Physics Teaching”. EIKI Journal of Effective Teaching Methods 2 (4): 18–27. https://doi.org/10.59652/jetm.v2i4.283.

Lee, T. 2024. “Artificial Intelligence and Posthumanist Translation: ChatGPT versus the Translator”. Applied Linguistics Review 15 (6): 2351–2372. https://doi.org/10.1515/applirev-2023-0122.

Lim, W. M., A. Gunasekara, J. L. Pallant, J. I. Pallant, and E. Pechenkina. 2023. “Generative AI and the Future of Education: Ragnarök or Reformation? A Paradoxical Perspective from Management Educators”. International Journal of Management Education 21 (2): 100790. https://doi.org/10.1016/j.ijme.2023.100790.

Liu, H., R. Ning, Z. Teng, J. Liu, Q. Zhou, and Y. Zhang. 2023. “Evaluating the Logical Reasoning Ability of ChatGPT and GPT-4”. Preprint, arXiv, May 5, 2024. https://doi.org/10.48550/arXiv.2304.03439.

Liu, S., G. Liu, B. Zhu, Y. Luo, L. Wu, and R. Wang. 2024. “Balancing Innovation and Privacy: Data Security Strategies in Natural Language Processing Applications”. In 5th International Conference on Machine Learning and Computer Application (ICMLCA), Hangzhou, China, 2024, 609–613. Piscataway: Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/ICMLCA63499.2024.10754062.

Lo, C. K., K. F. Hew, and M. S.-Y. Jong. 2024. “The Influence of ChatGPT on Student Engagement: A Systematic Review and Future Research Agenda”. Computers and Education 219: 105100. https://doi.org/10.1016/j.compedu.2024.105100.

Loos, E., and J. Radicke. 2024. “Using ChatGPT-3 as a Writing Tool: An Educational Assistant or a Moral Hazard? Current ChatGPT-3 Media Representations Compared to Plato’s Critical Stance on Writing in Phaedrus”. AI and Ethics. https://doi.org/10.1007/s43681-024-00470-1.

Lund, B. D., and T. Wang. 2023. “Chatting about ChatGPT: How May AI and GPT Impact Academia and Libraries?” Library Hi Tech News 40 (3): 26–29. https://doi.org/10.1108/LHTN-01-2023-0009.

Mathew, R. T. 2023. “ChatGPT: Proceed with Caution”. Cancer Research, Statistics and Treatment 6 (1): 122–124. https://doi.org/10.4103/crst.crst_95_23.

Mohamed, A. M. 2024. “Exploring the Potential of an AI-Based Chatbot (ChatGPT) in Enhancing English as a Foreign Language (EFL) Teaching: Perceptions of EFL Faculty Members”. Education and Information Technologies 29 (3): 3195–3217. https://doi.org/10.1007/s10639-023-11917-z.

Nandalwar, J., V. Shirashyad, P. Pandhare, and P. Kothawale. 2023. “An Intelligent Approach for Automating Data Inputs on Web Portals for Several Apex/Statutory Bodies in the Field of Education Using Robotics and Machine Learning”. International Journal for Multidisciplinary Research 5 (2): 1–9. https://doi.org/10.36948/ijfmr.2023.v05i02.2397.

Nath, A., S. Mwesigwa, Y. Dai, X Jiang, and Z. Zhao. 2024. “GENEVIC: GENetic Data Exploration and Visualization via Intelligent Interactive Console”. Bioinformatics 40 (10): btae500. https://doi.org/10.1093/bioinformatics/btae500.

Nikolic, S., S. Daniel, R. Haque, M. Belkina, G. M. Hassan, S. Grundy, S. Lyden, P. Neal, and C. Sandison. 2023. “ChatGPT versus Engineering Education Assessment: A Multidisciplinary and Multi-Institutional Benchmarking and Analysis of this Generative Artificial Intelligence Tool to Investigate Assessment Integrity”. European Journal of Engineering Education 48 (4): 559–614. https://doi.org/10.1080/03043797.2023.2213169.

Oertner, M. 2024. “ChatGPT als Recherchetool?: Fehlertypologie, technische Ursachenanalyse und hochschuldidaktische Implikatione”. Bibliotheksdienst 58 (5): 259–297. https://doi.org/10.1515/bd-2024-0042.

Ouh, E. L., B. K. S. Gan, K. J. Shim, and S. Wlodkowski. 2023. “ChatGPT, Can You Generate Solutions for My Coding Exercises? An Evaluation on Its Effectiveness in an Undergraduate Java Programming Course”. In ITiCSE 2023: Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education, Vol. 1, 54–60. https://doi.org/10.1145/3587102.3588794.

Pinzolits, R. 2024. “AI in Academia: An Overview of Selected Tools and Their Areas of Application”. MAP Education and Humanities 4: 37–50. https://doi.org/10.53880/2744-2373.2023.4.37.

Popovici, M.-D. 2023. “ChatGPT in the Classroom. Exploring Its Potential and Limitations in a Functional Programming Course”. International Journal of Human–Computer Interaction 40 (22): 7743–7754. https://doi.org/10.1080/10447318.2023.2269006.

Rane, N. L., M. Paramesha, and P. Desai. 2024. “Artificial Intelligence, ChatGPT, and the New Cheating Dilemma: Strategies for Academic Integrity”. In Artificial Intelligence and Industry in Society 5.0, edited by N. L. Rane, 1–23. Birkenhead: Deep Science Publishing. https://doi.org/10.70593/978-81-981271-1-2_1.

Ray, P. P. 2023. “ChatGPT: A Comprehensive Review on Background, Applications, Key Challenges, Bias, Ethics, Limitations and Future Scope”. Internet of Things and Cyber-Physical Systems 3: 121–154. https://doi.org/10.1016/j.iotcps.2023.04.003.

Rospigliosi, P. A. 2023. “Artificial Intelligence in Teaching and Learning: What Questions Should We Ask of ChatGPT?” Interactive Learning Environments 31 (1): 1–3. https://doi.org/10.1080/10494820.2023.2180191.

Roumeliotis, K. I., and N. D. Tselikas. 2023. “ChatGPT and Open-AI Models: A Preliminary Review”. Future Internet 15 (6): 192. https://doi.org/10.3390/fi15060192.

Sahlgren, O. 2024. “Action-Guidance and AI Ethics: The Case of Fair Machine Learning”. AI and Ethics 1–13.‏ https://doi.org/10.1007/s43681-024-00437-2.

Sebastian, G. 2023. “Do ChatGPT and Other AI Chatbots Pose a Cybersecurity Risk?: An Exploratory Study”. International Journal of Security and Privacy in Pervasive Computing (IJSPPC) 15 (1): 1–11.‏ https://doi.org/10.4018/IJSPPC.320225.

Shafik, W. 2024. “Data Privacy and Security Safeguarding Customer Information in ChatGPT Systems”. In Revolutionizing the Service Industry with OpenAI Models, edited by M. Demir and A. Dalgic, 52–86. Hershey: IGI Global.‏ https://doi.org/10.4018/979-8-3693-1239-1.ch003.

Shidaganti, G., R. Sanjana, K. Shubeeksh, V. M. Raman, and V. Thakshith. 2023. “ChatGPT: Information Retrieval from Image Using Robotic Process Automation and OCR”. In 2023 7th International Conference on Intelligent Computing and Control Systems (ICICCS), 1264–1270. Piscataway: Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/iciccs56967.2023.10142461.

Sreen, A. S., and M. H. M. Majid. 2024. “Leveraging ChatGPT for Personalized Learning: A Systematic Review in Educational Settings”. Amandemen: Journal of Learning, Teaching and Educational Studies 2 (1): 63–78.‏ https://doi.org/10.61166/amd.v2i1.72.

Stojanov, A. 2023. “Learning with ChatGPT 3.5 as a More Knowledgeable Other: An Autoethnographic Study”. International Journal of Educational Technology in Higher Education 20: 35. https://doi.org/10.1186/s41239-023-00404-7.

Sullivan, M., A. Kelly, and P. McLaughlan. 2023. “ChatGPT in Higher Education: Considerations for Academic Integrity and Student Learning”. Journal of Applied Learning and Teaching 6 (1): 31–40. https://doi.org/10.37074/jalt.2023.6.1.17.

Teoli, D., T. Sanvictores, and J. An. 2019. “SWOT Analysis”. In StatPearls. Treasure Island: StatPearls Publishing. https://europepmc.org/article/med/30725987.

Theelen, H., J. Vreuls, and J. Rutten. 2024. “Doing Research with Help from ChatGPT: Promising Examples for Coding and Inter-Rater Reliability”. International Journal of Technology in Education 7 (1): 1–18. https://doi.org/10.46328/ijte.537.

Thorat, V. A., P. Rao, N. Joshi, P. Talreja, and A. Shetty. 2024. “The Role of Chatbot GPT Technology in Undergraduate Dental Education”. Cureus 16 (2): e54193. https://doi.org/10.7759/cureus.54193.

Tlili, A., B. Shehata, M. A. Adarkwah, A. Bozkurt, D. T. Hickey, R. Huang, and B. Agyemang. 2023. “What If the Devil Is My Guardian Angel: ChatGPT as a Case Study of Using Chatbots in Education”. Smart Learning Environments 10: 15. https://doi.org/10.1186/s40561-023-00237-x.

Tony, Ma. 2023. “The Influence of ChatGPT on Autonomous Learning for College Students”. In Lecture Notes in Education Psychology and Public Media: Proceedings of the International Conference on Global Politics and Socio-Humanities, Vol. 25, edited by E. Mallen and J. Cifuentes-Faura, 172–179. Hennef: EWA Publishing. https://doi.org/10.54254/2753-7048/25/20230708.

Wang, J., Y. Liang, F. Meng, Z. Sun, H. Shi, Z. Li, J. Xu, J. Qu, and J. Zhou. 2023. “Is ChatGPT a Good NLG Evaluator? A Preliminary Study”. Preprint, arXiv, October 24, 2023. https://doi.org/10.18653/v1/2023.newsum-1.1.

Wang, Y., S.-R. Zhang, A. Momtaz, R. Moradi, F. Rastegarnia, N. Sahakyan, S. Shakeri, and L. Li. 2024. “Can AI Understand Our Universe? Test of Fine-Tuning GPT by Astrophysical Data”. Preprint, arXiv, April 14, 2024. https://doi.org/10.48550/arXiv.2404.10019.

Whalen, J., and C. Mouza. 2023. “ChatGPT: Challenges, Opportunities, and Implications for Teacher Education”. Contemporary Issues in Technology and Teacher Education 23 (1): 1–23. https://www.learntechlib.org/primary/p/222408/.

Wu, X., R. Duan, and J. Ni. 2024. “Unveiling Security, Privacy, and Ethical Concerns of ChatGPT”. Journal of Information and Intelligence 2 (2): 102–115. https://doi.org/10.1016/j.jiixd.2023.10.007.

Yang, C., and A. Ettinger. 2023. “Can You Follow Me? Testing Situational Understanding in ChatGPT”. Preprint, arXiv, October 24, 2023. https://doi.org/10.18653/v1/2023.emnlp-main.394.

Yeung, W. E., C. Qi, J. L. Xiao, and F. R. Wong. 2023. “Evaluating the Effectiveness of AI-Based Essay Grading Tools in the Summative Assessment of Higher Education”. In ICERI 2023 Proceedings: 16th International Conference of Education, Research and Innovation, edited by L. Gómez, C. G. Martínez and J. Lees, 8069–8073. Valencia: IATED. https://doi.org/10.21125/iceri.2023.2063.

Yilmaz, R., and F. G. K. Yilmaz. 2023. “Augmented Intelligence in Programming Learning: Examining Student Views on the Use of ChatGPT for Programming Learning”. Computers in Human Behavior: Artificial Humans 1 (2): 100005. https://doi.org/10.1016/j.chbah.2023.100005.

Yuan, H., Z. Che, S. Li, Y. Zhang, X. Hu, and S. Luo. 2024. “The High Dimensional Psychological Profile and Cultural Bias of ChatGPT”. Preprint, arXiv, May 6, 2024. https://doi.org/10.48550/arXiv.2405.03387.

Zhang, J., Y. Chen, C. Liu, N. Niu, and Y. Wang. 2023. “Empirical Evaluation of ChatGPT on Requirements Information Retrieval under Zero-Shot Setting”. In ICNGN 2023: International Conference on Intelligent Computing and Next Generation Networks, edited by G. Myoung, P. Loskot, Q. Yang and R. Su, 177–182. Piscataway: Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/ICNGN59831.2023.10396810.

Zhu, C., M. Sun, J. Luo, T. Li, and M. Wang. 2023. “How to Harness the Potential of ChatGPT in Education?” Knowledge Management and E-Learning: An International Journal 15 (2): 133–152. https://doi.org/10.34105/j.kmel.2023.15.008.

Żmudka, K., A. Spychał, B. Ochman, Ł. Popowicz, P. Piłat, and J. Jaroszewicz. 2024. “ChatGPT—A Tool for Assisted Studying or a Source of Misleading Medical Information? AI Performance on Polish Medical Final Examination”. Annales Academiae Medicae Silesiensis 78: 94–103. https://doi.org/10.18794/aams/176450.