The RCSI New Technologies 2024 for Future of Surgery in Ireland Working Group has examined how these new technologies may be leveraged to support the development of surgical care (RCSI, 2024). Based on the findings of this group, together with our own strategic vision, it is likely that surgery in the future will place a greater emphasis on prevention, earlier diagnosis, improved decision-making, and digitally enhanced delivery of care. Post-operative pathways are also expected to change, with shorter hospital stays, increased use of remote monitoring, and a growing emphasis on models such as “hospital at home.”
This report focuses on digital technologies and artificial intelligence and their impact on surgical practice and the wider surgical ecosystem. This includes on only the application of these technologies to the delivery of surgical care but also their use in related activities such as education, research, and clinical trials.
We aim to define AI and digital technologies, discuss clinical opportunities as well as risks and discuss how we should prepare ourselves for trhese new technologies.
Definition of Artificial Intelligence and Digital Technologies in Surgery
Artificial Intelligence (AI):
Technology that enables computers and machines to stimulate human learning, comprehension, problem-solving, decision-making, creativity, and autonomy.
Machine Learning (ML):
Involves creating models by training an algorithm to make predictions or decisions based on data .
Deep Learning:
Subset of machine learning that uses multilayered neural networks, called deep neural networks, that more closely stimulate the complex decision-making power of the human brain.
Generative AI (Gen AI):
Deep learning models that can create complex original content such as long-form text, high-quality images, realistic video or audio and more in response to a user’s prompt or request.
AI Agent:
An autonomous AI program, it can perform tasks and accomplish goals on behalf of a user or another system without human intervention, by designing its own workflow and using available tools.
Agentic AI:
A system of multiple autonomous AI agents, the efforts of which are coordinated, or orchestrated, to accomplish a more complex task or a greater goal than any single agent in the system could accomplish.
Digital Medicine:
The use of digital technologies to improve the delivery, quality and outcomes of healthcare and to improve wellness. This includes information and communication technologies and data analytics.
Digital Surgery:
The application of digital medicine in the field of surgery.
Preparing surgeons for the advent of these novel AI & digital technologies requires a two-pronged approach to training. Firstly, it is essential that surgeons are literate in these technologies and understand the basic principles underpinning AI technology, digital medicine and data science. Surgeons should understand their capabilities and limitations and they should possesss the skills and attributes to allow them evaluate, implement and use specific technologies as they become available. This includes the ability to interpret AI-generated outputs, crtically appraise decision-support tools, and recognise issues such as bias, data limitations and uncertainity.
In addition, surgeons will require particular training for specific novel devices or clinical platforms. The more the AI mimics the role of the clinician, the more critical our full understanding of AI becomes.
THE BASIC CORE COMPETENCIES:
– Understanding basic concepts
– Different types of models and AI systems
– Ethical and legal awareness
– Critical Thinking
– Understanding interfaces and workflows
– Adaptability and life-long learning
– Communication skills
Challenges and Risks
Operation of AI Systems
Bias
Bias is defined as a consistent, systematic or undesirable distortion in the outputs produced by an AI system. In healthcare, an unbiased algorithm allocating resources should give the same score to those with the same basic needs (Obermeyer et al. 2021). Bias may be due to flawed data in the training dataset or due to the design of the algorithm.
Data features are individual elements of the dataset, e.g. age, gender, ASA grade, tumour stage etc. Datasets should contain all the relevant features required for the algorithm to make a decision or take an action. Many factors may lead to biased outcomes including training on outdated datasets or lack of diversity. Use of historical datasets may not consider modern concepts of disease or treatment or may propagate historical biases against ceretain demographic groups. Lack of demographic or geographical diversity in the datasets may lead to conclusions inappropriately applied to a non-congruent cohort such as rural patients or those from minority groups. Even when protected characteristics are accounted for in the model, use of proxy features which are linked these characteristics may lead to bias. AI has been found to differentiate between races on chest x-rays (Gichoya et al. 2022) or between genders on retinography (Korot et al. 2021) in a way that cannot be detected by human oversight.
Misaligned Goals of the Algorithm
AI algorithms are designed to maximise reward, that is to perform actions to achieve its policy goals. However, critical to this process is that the correct goals are identified and agreed. These are often value judgements which are independent of the algorithms but are critical for achieving the optimal or desired outcomes. For example, if the goal of the algorithm is to prioritise cost savings over radical treatment, or 30-day readmission rates over long term outcome, this is what the algorithm will strive to achieve, irrespective of patient, clinician or societal expectations. When goals do not reflect clinical utility, equity, and workflow realities, performance can achieve apparently excellent metrics yet be detrimental to patients in practice. Clinicians should have a central role in defining these goals.
Hallucinations
Hallucinations in AI are instances where a model produces fabricated but confident, fluent outputs that look plausible but are incorrect or unsupported. Fabricated citations can lead to dangerous decisions, misdiagnoses, financial loss, or compliance violations. Confident but incorrect answers reduce user confidence in the system and the organisation. Hallucinated statements can defame, infringe intellectual property, violate consumer protection laws, or breach sector regulations (e.g. GDPR) and by entering knowledge bases it can train self-reinforcing errors and degrade future model performance.
Lack of Explainability
The lack of explainability in AI and the difficulty of understanding how complex models reach their outputs may undermine trust, safety, and accountability, especially in high-stakes domains like healthcare. Blackbox systems can encode spurious correlations and biases that are hard to detect, making errors difficult to debug and contest. Post-hoc explanations can help but are often unstable, incomplete, or misleading and provide the appearance of transparency without guarantees of validity. This opacity complicates regulatory approval, informed consent, and liability. On the other hand, these are very complex systems with billions of interacting nodes in the most complicated models. We should not expect to forensically trace every decision or outcome at a granular level. Much work is ongoing to improve explainability and to systematically detect and correct bias within these systems.
Harm Caused by AI
The potential for AI to bring about actual harm is very considerable whether caused intentionally or unintentionally. Intentional harm may include harm by design, AI misuse or attacks on AI systems. Unintentional harm may involve AI failures, failures of human oversight or integration harm (Hoffmann, 2025). Use of flawed, unvalidated or biased systems may be intentional or unintentional.
Effect on Clinicians
Clinician Overreliance on AI Tools
Clinicians may become over-reliant on AI tools leading to erosion of judgement and the failure of clinical oversight. Radiologists who are aware of the label assigned by AI to chest radiographs may increase their false positive diagnosis rate and even more so their false negative rates (Bernstein et al., 2023). Overreliance may also lead to a reduction in hands-on training opportunities, and potential deskilling, especially in areas such as emergency procedures and conversions to open procedures.
As AI becomes more widely used, there is the risk that humans will defer judgement and decision making to computers, adopting AI outputs with minimal scrutiny and overriding intuition, a process recently described as “cognitive surrender” by Shaw and Nave (Shaw and Nave, 2026)
Loss of Skilled Humans
While automation with improvement of accuracy and efficiency will provide potential cost savings, it is inevitable that the business model will generate significant pressure to reduce human payroll costs as has already been seen in other industries. Entry level workers are most vulnerable to AI.
Currently, AI excels at data processing, pattern recognition, and automation, but it lacks essential human qualities like empathy, emotional support, ethical judgment, nuanced decision-making, holistic patient care and adaptability in novel situations as well as physical interaction that AI cannot replicate at the current time. Although there is unsubstantiated optimism that AI can be trained for many of these attributes, recent evidence shows that AI deployed in clinical practice may not behave at all as expected. (Bean et al., 2026)
It is accepted that a critical number of skilled humans is required in the healthcare system both to primarily deliver healthcare, and also to supervise AI. It is important that we continue to train and retain these staff.
Challenges with Staff Engagement
Difficulty arises with the engagement of clinical staff in new AI programmes when they already have high clinical workloads and limited knowledge of AI to implement and govern new AI Systems (Ramsay et al., 2025). These challenges require programme leadership support, dedicated project management and clinical leadership to champion the implementation of this technology.
Other Issues
AI Privacy and Cybersecurity
AI models, particularly those which are continuously trained and evolved using patient data in real time, are subject to a wide range of risks. Personal identifiable information may be accessible to a bad actor by training-data leakage (where models memorise and regurgitate sensitive data, enabling data extraction attacks), model inversion and re-identification (where inputs are reconstructed from model outputs) and where weakly pseudonymised data can be re-identified via linkage attacks. Adversarial or trojan attacks may lead to misclassifications and unsafe behaviour or may even purposely guide the algorithm to generate the output desired by the bad actors. A determined cyberattack can take an AI system completely offline or make it unusable. Whether the outage is brief or catastrophic depends on architecture, operational discipline, and recovery plans.
Regulation
There are a number of regulatory risks including data protection legislation (GDPR in the EU or CCPA and HIPAA in the USA), violations from unlawful processing, lack of consent, or cross-border data transfers. Data lineage, the ability to trace sources, purposes, and retention may be opaque leading to audit failure and fines. Data Protection Impact Assessments (DPIAs) may be inadequate and not include assessments for high-risk use cases. In addition, there is the “third-party risk” with Vendors or APIs mishandling data or sub-processors lacking appropriate safeguards.
With the rapid development of AI and current concerns about AI going out of control, regulation is very important but there are serious concerns that regulation applied injudiciously may hamper development.
Liability
The issue of liability in the event of an adverse outcome when AI is used for diagnosis and treatment is not clear. This has not been fully tested in court. It is possible that liability may fall on the clinician, the healthcare provider organisation or the product developer.
The current paradigm is that the responsibility for the final clinical decision is that of the clinician. This may be because of negligent use of AI, misinterpretation of AI output or misunderstanding of the limitations of the technology. However, a device which does not perform as advertised may be the responsibility of the developer, while lack of proper governance or provision of training may be the responsibility of the healthcare provider.
Sustainability
Developing and deploying AI systems requires substantial investment in hardware, software, training, and infrastructure across the healthcare system. High initial cost of technology can make them prohibitive for smaller hospitals or low-resource settings. This may lead to unequal adoption exacerbating healthcare disparities. In addition, AI has a significant carbon footprint with a high requirement for both electricity and water. Currently, data centres use 1.4% of global electricity supply and this may rise to up to 4.4% by 2035 (IEA, 2025).
What is the Role of the HSE/Hospital?
The HSE has a critical role in planning the health services including consideration of longer-term population initiatives. Proper planning and integration of future technology is imperative. Through the surgical programmes, RCSI is committed to working with the HSE to integrate these new technologies into practice for the benefit of the patient and for the health service as a whole.
What is the Responsibility of Individual Surgeons?
• Understand the new technologies including safety and ethical risks and how to evaluate these.
• Advocate for the introduction of new technologies that offer clinical benefit for our patients and that are cost effective.
• Ensure that they have completed appropriate training in the use of a new device or technology before deploying it.
• Work with health authorities and patient groups to ensure that new technologies are optimally integrated into healthcare infrastructure.
• Support proper governance structures within our institutions to ensure that new technologies are monitored in relation to safety, benefit and cost.
• Uphold ethical principles, professional responsibility, accountability and clinical oversight at all times.
Digital Twin – The concept of the “digital twin” represents a significant future development, where trainees may rehearse procedures on patient-specific models derived from imaging and clinical data.
RCSI Liaison with External Bodies
• RCSI should work with health service providers, legislators, regulators, patients and other stakeholders to align standards, develop policy and progress ethical frameworks for the introduction of these technologies into clinical practice.
• RCSI should work with other higher education institutions and industry to explore, develop and evaluate innovative applications of AI and digital surgery in the surgical setting.
Source and Credit – Short Life Working Group: Artificial Intelligence and Digital Surgery, RCSI (May 2026)
Scriveners Online© 