Evaluation of critical success factors affecting the use of artificial intelligence in health services with analytic hierarchy process

Abstract

Objectives: The aim of this research is to identify the critical factors that affect the success of the use of artificial intelligence (AI) in healthcare and to rank these factors in a hierarchical structure.

Materials and methods: Analytic hierarchy process method, one of the multi-criteria decision-making methods, was used.

Results: As a result of the opinions of the 10 experts included in the study, the factor of hospital organizational structure and health IT infrastructure received the highest weight ratio with the code "BF2" (0.1581), while the factor of security, privacy and ethical issues was evaluated by the experts with the code BF5 (0.1491) as the factor with the second highest importance level. Within the scope of the study, the factor with the lowest level of importance was revealed as the factor of high cost of AI with the code BF6 (0.0614).

Conclusion: As a result, when the critical success factors affecting the use of AI in healthcare services are prioritized by experts, it is revealed that the importance of IT infrastructure in healthcare services, hesitations on sensitive issues such as ethics, security and privacy of personal data are critical success factors affecting the use of AI in healthcare services. However, it was also evaluated by the experts in the study that the cost of AI does not affect its use in healthcare services as much as other factors.

Introduction

Health services include the totality of services provided through the prevention, diagnosis and treatment of illness, disease, discomfort, injury and other physical and mental disorders for the purpose of maintaining or improving human health. These services are provided by health professionals and relevant health institutions and aim to achieve optimal health outcomes through timely use of personal health services by individuals. However, access to health care services can be limited by factors such as financial limitations, geographical barriers, and personal limitations. Health service limitations can negatively affect the utilization of medical services, the effectiveness of treatments, and overall health outcomes. In this context, it has become inevitable to use artificial intelligence (AI) applications to improve and make healthcare services more effective.[1] Artificial intelligence refers to the ability of a computer or a computer-controlled robot to perform processes that are usually unique to humans. This technology has caused significant changes in the field of healthcare and has provided various advantages in the delivery of healthcare services. By analyzing healthcare data, AI can identify clinically important information and assist healthcare professionals in clinical decision-making.[2] Furthermore, AI systems have the potential to reduce treatment and diagnostic errors.[3]

Rapid technological developments in healthcare services have gone beyond traditional practices and led to revolutionary changes in the healthcare sector. Artificial intelligence technologies play an important role in this change.[4] They provide significant advantages in diagnosis, treatment, patient follow-up, and management processes in healthcare.[5,6] However, a number of critical success factors need to be identified and evaluated in order to successfully implement these technologies in healthcare.

This study focuses on the identification and evaluation of critical success factors affecting the use of AI in healthcare. Understanding and solving the challenges that arise for the effective implementation of AI technologies in the healthcare sector will be an important step in providing more effective healthcare services. Given the complexity of the factors affecting the success of applications of AI technologies in healthcare, this study considers prioritizing these factors using the analytic hierarchy process (AHP).[7]

Use of artificial intelligence in healthcare

Artificial intelligence is being developed for many different applications in medicine. It is used to facilitate diagnosis, provide a better understanding of the progression of diseases, optimize drug/treatment dosages and reveal new treatments.[4] Artificial intelligence technologies, which have started to be actively used in almost every aspect of daily life, have had an important role in the coronavirus disease 2019 pandemic that has affected the whole world. For example, AI applications have been utilized in early diagnosis of infections, monitoring treatments, tracking contacted patients, predicting future mortality rates, developing drugs and vaccines, reducing the workload of healthcare workers and preventing diseases.[8] For example, a mobile application is being developed to detect heart diseases, and in the studies, the heart sounds of the patients are listened to using a digital stethoscope, and the detected disease is recorded on the screen of the mobile device every five seconds. It is also noted that the rate of detection of heart diseases by this mobile application is at the level of trained and experienced cardiologists.[9]

Artificial intelligence that helps patients manage their own symptoms or cope with chronic diseases also has advantages such as minimizing errors in diagnosis and diagnosis. It can also limit access to health care, give imprecise medical advice, and recommend health care that is not humane. While AI provides indisputable benefits, it also entails risks and dangers that need to be actively managed and minimized. One of the inevitable details in holistic medicine practices is the risk of AI making mistakes and choosing who will be held responsible for the mistakes made.[10] It is thought that the fact that AI, which is expressed with the idea that it will prevent or minimize errors in medical practices, actually carries the risk of making mistakes is contrary to the reason for its existence. When AI is utilized to facilitate treatment or as a decision-maker in the clinic, patient and caregiver safety issues draw attention. Errors can also occur in AI, but if the error is difficult to detect, it can cause serious consequences.[11] For example, an AI application was developed to predict patients who developed complications after pneumonia and required hospitalization; however, this application made erroneous decisions as it did not take into account the specific information of the patients. This led to errors in diagnosis and treatment decisions for both the physician and the patient.[12] When the physician sees inconsistencies in AI recommendations, they should consider their own clinical knowledge and experience.[13]

Patients and Methods

Given the complexity of the factors affecting the success of applications of AI technologies in healthcare, the study considers prioritizing these factors using the AHP.[7] Analytic hierarchy process is a problem-solving framework. Saaty[14] was the first to use the mathematical AHP formulation of various aspects, such as measurement of judgments. Analytic hierarchy process has been tried in the health sector and has been studied by many people, including scientists such as Saaty and Vargas.[15] The purpose of using this method is to identify the critical factors that affect the success of the use of AI in health services and to rank these factors in a hierarchical structure. In this respect, the study aims to provide a framework that can help decision-makers and practitioners in the healthcare sector to develop more informed and guiding strategies. This study involves decision-making based solely on expert opinion. Ethics committee approval was not obtained as it did not involve any intervention or experimental application on human subjects. The study was conducted in accordance with the principles of the Declaration of Helsinki.

Critical Success Factors Affecting the Application of Artificial Intelligence in Health Services are discussed under six titles in the literature:

• Uncertain strategic management and organizational culture in health,

• Hospital organization structure and information technology (IT) Infrastructure in Health,

• Responsibility, accountability, and integration complexity in health care,

• Poor data quality and data management in health,

• Security, privacy, and ethical issues

• The high cost of artificial intelligence.

The data of the research were obtained by 10 experts, five academicians and five computer engineers, who have studies or experience in the field of AI living in ‹stanbul.

It is necessary to calculate how many times the criteria are more important than each other. It is necessary to calculate the degree of importance between the criteria by adopting the 1-9 scale given in Table 1.[14]

Table 1. Pairwise comparison method basic 1-9 scale[14]

Results

Critical success factors affecting the use of AI in healthcare services were evaluated by each expert, and a common conclusion was reached. In this context, the answers given by all experts participating in the study were analyzed and tabulated. The table below shows the weight and consistency values of the answers given by each expert, the consistency ratio (CR) revealed by all the answers given by 10 experts, and the weight values for critical success factors.

Since the CR of the answers given in Table 2, which includes all expert opinions, is less than 10% (6.73), it is assumed that the answers given are consistent. According to the answers forming the common table of all expert matrices, the success factor with the highest weight value (0.1581) was revealed as “Hospital Organizational Structure and Health IT Infrastructure” with the code BF2. The success factor with the second highest weight value (0.1491) was Security, Privacy and Ethical Issues with the code BF5. The next success factor, which was weighted in third place with the closest value to this value, was preferred by the experts as BF3, Responsibility, Accountability, and Integration Complexity in Healthcare (0.1350). The success factor with the fourth weighted value (0.1283) was BF4, Poor Data Quality and Data Management in Healthcare, while the success factor with the fifth weighted value (0.1247) was BF1, Uncertain Strategic Management and Organizational Culture in Healthcare. As a result of the evaluations of all experts, BF6 coded High Cost of Artificial Intelligence (0.0614), which was at the end of the ranking with the lowest weight value, was evaluated as the least preferred factor.

Table 2. Expert opinions

Discussion

Artificial intelligence is systems that depend on data and technology. Data is recognized as “the main driver of current AI systems”.[16] Data accuracy and reliability should be the top priorities in the implementation of AI systems, as they negatively affect the accuracy of models and thus can lead to serious errors.[17] Unreliable, inaccurate, and poor data input affects the functionality of systems and leads to wrong decisions. Accordingly, integration between systems and the ability to collect, combine, store, and use accurate data is vital for the successful implementation of AI systems.[18,19]

The existing literature has not adequately assessed the relationships between the critical factors identified in this study and the application of AI in healthcare. Based on the findings and feedback from the experts who participated in the study, this study offers several recommendations for the application of AI in healthcare in terms of conceptual and theoretical development. Within the scope of the findings of the study, the most important critical success factor affecting the use of AI in healthcare services was the organizational structure of hospitals and the suitability of the infrastructure for IT in healthcare. The use of AI in healthcare services means savings in time, space and expenditures. For this reason, especially hospital organizational structures and IT infrastructure should be evaluated and strengthened within the framework of quality standards.

Uncertain Strategic Management and Organizational Culture in Health, which is one of the critical success factors, received an important weight value within the scope of the study since it is not clear in many countries who the regulatory authority for AI in healthcare services is, nor how the ever-changing black box of AI will be evaluated from a policy perspective.

Limitations of the study include the use of a 1-9 scale in the AHP method, which some participants had difficulty understanding correctly, and challenges in reaching a sufficient number of experts with relevant field experience, which limited the data collection process.

In conclusion, it is thought that cost is not the most important factor in the use of AI, especially countries with a good financial structure can easily invest in this field and provide the necessary support. In this context, it is recommended to focus on the critical success factors that rank high in weight values. Artificial intelligence is already being applied in different health sectors and will probably have a significant impact on the entire health sector in the future.

Cite this article as: Aydoğan B, Algan Çapraz N, Tapan B, Demirbaş MB, Kurt A. Evaluation of critical success factors affecting the use of artificial intelligence in health services with analytic hierarchy process. D J Med Sci 2025;11(1):10-14. doi: 10.5606/fng.btd.2025.156

Idea/concept, design, data collection and/or processing, literature review, writing the article, references and fundings,materials: B.A.; Control/supervision, critical review: N.A.Ç., B.T., A.K.; Idea/concept, analysis and/or interpretation, writing the article, references and fundings, materials: M.B.D.

The authors declared no conflicts of interest with respect to the authorship and/or publication of this article.

The authors received no financial support for the research and/or authorship of this article.

Data Sharing Statement:
The data that support the findings of this study are available from the corresponding author upon reasonable request.

References

1. Akalın B, Veranyurt Ü. Sağlık hizmetleri ve yönetiminde yapay zekâ. acta infologica Acta Infologica 2021;5:232-9. doi: 10.26650/ acin.850857.
2. Alp F, Akalın B. Artıfıcıal Intellıgence Methods in Oncology. Gevher Nesibe J IESDR 2022;7:81-92.
3. Güneç HG, Gökyay SS, Kaya E, Cesur Aydın K. Toplum yapay zeka ile dental tanı konmasına hazır mı? Selcuk Dent J 2022;9:200-7.
4. Turan B, Gülşen M, Yılmaz AE. Artificial intelligence applications in psychiatric disorders. J Ankara Univ Fac Med 2022;75:56-62. doi:10.4274/atfm. galenos.2022.36002.
5. Hüban Karakaya B, Akyol AS, Doğan Merih Y. Yapay zekâ teknolojisinin perinatal dönem bakımına entegrasyonu ve uygulama örnekleri. Türkiye Sağlık Enstitüleri Başkanlığı Derg 2022;5:1-11. doi: 10.54537/tusebdergisi.1154089.
6. Kundakçı ŞÇ. Yaşlı bakımında yapay zekâ kullanımı. Doğu Karadeniz Sağlık Bilim Derg 2023;2:77-87. doi: 10.59312/ebshealth.1318150.
7. Merhi MI. An evaluation of the critical success factors impacting artificial intelligence implementation. Int J Inf Manage 2023;69:102545. doi: 10.1016/j. ijinfomgt.2022.102545.
8. Pradhan D, Biswasroy P, Kumar Naik P, Ghosh G, Rath G. A Review of current Interventions for COVID19 Prevention. Arch Med Res 2020;51:363-74. doi: 10.1016/j.arcmed.2020.04.020
9. Hoşgör H, Güngördü H. Sağlıkta yapay zekanın kullanım alanları üzerine nitel bir araştırma. Eur J Sci Technol 2022;(35):395-407. doi: 10.31590/ ejosat.1052614.
10. Thomas S. Artificial intelligence, medical malpractice, and the end of defensive medicine - Bill of Health 2017. Available at: https://blog.petrieflom.law. harvard.edu/2017/01/26/artificial-intelligencemedical-malpractice-and-the-end-of-defensivemedicine/#more-20627
11. Robert M Wachter. The digital doctor: Hope, hype and harm at the dawn of medicine’s computer age. Malaysian Orthop J 2018;12:2018.
12. Caruana R, Lou Y, Gehrke J, Koch P, Sturm M, Elhadad N. Intelligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission. Proc ACM SIGKDD Int Conf Knowl Discov Data Min 2015;2015:1721-30. doi: 10.1145/2783258.2788613.
13. Güvercin CH. Artificial intelligence in medicine and research. In: Ekmekçi PE, editor. Yapay zeka ve tıp etiği. Ankara: Türkiye Klinikleri; 2020. s. 7-13.
14. Saaty TL. What is Analytical hierarchy process? NATO ASI Ser Math Model Decis Support 1988;F48:109-22.
15. Saaty TL, Vargas LG. Uncertainty and rank order in the analytic hierarchy process. Eur J Oper Res 1987;32:107-117.
16. Thierer A, O’Sullivan AC, Russell R. Artificial intelligence and public law. 2017. Available at: https:// www.mercatus.org/research/research-papers/ artificial-intelligence-and-public-policy
17. WilliamsAM,LiuY,RegnerKR, JotterandF,LiuP,Liang M. Artificial intelligence, physiological genomics, and precision medicine. Physiol Genomics 2018;50:237- 43. doi: 10.1152/physiolgenomics.00119.2017.
18. Asch FM, Abraham T, Jankowski M, Cleve J, Adams M, Romano N, et al. Accuracy and reproducibility of a novel artificial intelligence deep learning-based algorithm for automated calculation of ejection fraction in echocardiography. J Am Coll Cardiol 2019;73:1447. doi: 10.1016/S0735- 1097(19)32053-4.
19. Mehr H. Artificial Intelligence for Citizen Services and Government. [Internet], 2017. Available at: https:// ash.harvard.edu/files/ash/files/artificial_intelligence_ for_citizen_services.pdf