Recruiting the most suitable candidates is a task that has continuously challenged surgical educators worldwide. Indeed, identification of appropriate selection criteria is an onerous task, often supported by scant evidence [67]. However, this controversial topic has gained much interest in recent times, particularly given the increased economic pressures, growing cost of training, and accountability placed upon training bodies. This, coupled with the reduction in working hours available for training, means that selection of trainees that are most likely to succeed through training is vital [68–70]. Traditionally, selection of prospective surgeons into training programs is based largely on three aspects: clinical experience and academic achievements, referee reports, and performance at interview. In Australia and New Zealand, this highly competitive process adheres to the aforementioned principles, where a self-reported structured curriculum vitae (CV) is scored according to strict criteria with points given for clinical experience, publications and presentations, teaching, higher degrees, and postgraduate prizes. Further, referee reports are collated from nominated clinical supervisors that involve scoring applicants according to the Royal Australasian College of Surgeons (RACS) competencies of medical and technical expertise, clinical decision making and judgment, collaboration and communication, professionalism academic, teaching, and leadership aptitudes. Finally, applicants are scored during a semi-structured interview consisting of a number of clinical and nontechnical skill stations that aim to assess these competencies. A recent study evaluating the predictive validity of this process demonstrated that those who obtained high score in the CV component of the selection process did not score higher in any subsequent objective work-based assessments during training. In contrast, referee reports and interview scores, as well as the overall score, positively correlated with performance during subsequent objective work-based assessments during the training program [71].

This traditional selection process has been controversially criticized by some for not including assessment of abilities that are fundamental to surgical practice, such as psychomotor skills [68]. Recent advancements in surgical practice—in the form of endoluminal techniques , complex laparoscopic procedures, microsurgery, and robotic surgery—require surgeons to possess a number of critical abilities across the cognitive, psychomotor, and visuospatial domains beyond those required for traditional surgical modalities [68, 72–77]. Further, some of these fundamental abilities have been considered largely innate, and it is debated whether these abilities can be acquired and mastered through training at all [77]. This clearly has implications for the benefit, cost-effectiveness, and safety of individuals without these innate abilities undergoing the lengthy, rigorous, and expensive process of surgical training. Within other high-risk industries, like aviation and the military, assessments of attributes deemed important for performance are incorporated into the selection process [78]. Cuschieri and colleagues surveyed the opinion of senior surgeons and surgical leaders from Europe and the USA with regard to the attributes they considered to be important for selection of surgical trainees [79]. The authors concluded that innate dexterity including the abilities of spatial perception, hand-eye coordination, aiming, multi-limb coordination, and hand-arm steadiness and the ability to interpret and manipulate images is considered by this group of expert surgeons to be an important selection criteria. Indeed, when these fundamental abilities were present in a trainee, improved performance correlated with shorter time to proficiency during endoscopic performance [76].

These provocative studies raise questions about the reliability and validity of the trainee selection process in surgery, as well as identifying those who may require additional training to achieve competence. As a result, tests of technical skills and fundamental abilities are included in the selection process for Higher Surgical Training at the Royal College of Surgeons in Ireland [80]. Candidates are required to complete a full day of assessments including a ten-station surgical skills Objective Structured Clinical Examination (OSCE) , where they are tested on skills acquired during basic surgical training. These include suturing, knot-tying, basic anastomosis, and basic endoscopic and laparoscopic skills. Additionally, candidates undergo a variety of validated assessments of psychomotor skills, visuospatial ability, and perception.

To meet the demands of increasingly complex healthcare associated with delivering high-quality, efficient surgical care, the concept of the surgical team has changed significantly [81]. No longer can the surgeon operate as a patriarchal figure issuing orders with regard to all aspects of patient care. In order to provide the highest quality holistic and efficient care, surgeons must work collaboratively as equals with nursing, allied health, other medical, and administrative colleagues. Together this group of individuals constitutes the surgical team with the shared goal of delivering the best care possible for their patients. Working within such an intricate system containing so many moving parts poses another challenge to surgical trainees beyond the pursuit of technical excellence. Furthermore, traditionally, surgical training programs focus little on training and assessing skills required to be a proficient collaborator.

In 2008, the American College of Surgeons (ACS) and Association of Program Directors in Surgery (APDS) united to create Phase III of the ACS/APDS National Curriculum [82]. Contained within this was a course of team training modules that incorporated a number of validated simulation scenarios to be used with human patient simulators. These modules were specifically designed to teach a wide range of team-related competencies including communication skills, critical language, assertive and closed-loop communication, active listening, and leadership. The scenarios involve laparoscopic crisis, laparoscopic troubleshooting, latex allergy anaphylaxis, patient handover, pre-operating briefing, as well as trauma team training [83]. Performance during the modules is assessed by specific assessment tools, but other validated nonproprietary instruments can also be used, such as the NOTECHS (non-technical skills) scale [83] and other frameworks [84]. Despite this, it has been reported that 21 % of 117 surveyed program directors were unaware of this curriculum [85]. Further, the implementation rate of Phase III was only 16 % [85]; lack of faculty-protected time and personnel, significant costs, and resident work-hour restrictions were suggested as reasons for this low figure [85].

Crew resource management (CRM) within healthcare is a concept that describes the principles of individual and crew behavior during ordinary and crisis situations, and aims to optimize available resources and develop skills in dynamic decision making, interpersonal behavior, and teamwork that lead to safe outcomes [86–88]. Emerging from other high-risk industries, such as aviation, CRM has been successfully applied to healthcare since the mid-1980s with a number of variants and hybrids being developed [89]. The development of the Team Strategies and Tools to Enhance Performance and Patient Safety (TeamSTEPPS^TM) program, as a variation of CRM, by collaboration between the Agency for Healthcare Research and Quality and the United States Department of Defence has provided a standardized evidence-based curriculum for team training for healthcare providers [90, 91]. At its core, TeamSTEPPS^TM aims to teach four fundamental competencies that constitute teamwork (leadership, situation monitoring, mutual support, and communication) with the aid of patient scenarios, case studies, multimedia, and simulation [92]. Having been implemented in multiple regional training centers around the USA and Australia [93], the TeamSTEPPS^TM program has been shown to enhance teamwork within the operating room, improving operating room efficiency and reducing patient safety concerns in the process [94, 95]. Additionally, it has been demonstrated to increase perceptions and attitudes with regard to patient safety culture, teamwork, and communication [42, 94, 96]. A recent study investigated the use of CRM within the surgical ward environment, in which surgical trainees participated in simulated ward-based scenarios of a deteriorating postoperative patient before and after CRM training [97]. CRM training improved clinical assessment and decision making and resulted in improvements in teamwork, communication, and leadership [97].

Effective and efficient teamwork within the operating room (OR) is crucial to prevent process failures and adverse patient events during an operation [98]. The OR team is further subdivided into specialized collaborations that include the surgical team (surgeon, surgical assistant, and scrub nurse), anesthetic team (anesthesiologist and anesthetic nurse), and theatre nursing staff (scrub nurse and scout nurse) [81]. Teamwork can have a huge impact in the OR on patient safety and resulted in development of strategies to reduce complications such as medication errors, positioning errors, and more, and train individuals to work efficiently and collaboratively not only within their own sub-team, but also within the entire OR team. The development of simulated ORs that replicate the entire OR environment has provided a unique opportunity [44] to cultivate a number of nontechnical skills, including command, control, and conflict resolution teamwork [99]. Real equipment as well as virtual reality and mannequin simulators are incorporated into this simulated setting [100]. This allows trainee surgical, anesthetic, and nursing staff to interact and practice teamwork skills together, while simultaneously performing technical tasks, during a variety of routine and crisis scenarios, just as they would in “real life” [40, 99, 101]. Indeed, Gettman et al. demonstrated an improvement of the teamwork, communication, and laparoscopic skills of trainees undergoing training within a simulated OR [102]. Further, the simulated OR was validated as realistic and representative of actual practice [102]. Other studies have similarly shown the benefits of collaborative training within a simulator OR environment on trainees’ nontechnical skills including teamwork and situational awareness [103]. Widespread use of simulated ORs for training is still limited due to a lack of appreciation of the benefits of training, potential savings in operations, harm reduction, and building trust between team members. Recently, virtual reality models of the OR have been developed and used for team training [104], but further research is needed to appreciate the ethical dimension, effectiveness, transfer of training and demonstrate the effect on team skills on patient outcomes [105, 106].

Surgical expertise encompasses a wide range of competencies. Holistic analysis of a surgeon’s professional and technical performance ideally incorporates reliable assessments of these individual competencies. Assessment of surgical expertise must start with shared evidence driven definitions and has been compartmentalized into technical and nontechnical skills, with a variety of methodologies developed to do this [107, 108]. Some of these are discussed below, but ultimately, the most important and relevant measure of expertise, using an expert performance and assessment approach, [5] is a robust evaluation of patient process and outcomes measures, both at the level of the individual practitioner [109] and at the microsystem level [84]. Just as error detection and analysis reflect expert performance by an individual [110], the same strategy applied to teamwork will yield dividends in terms of the team’s collective expertise [40, 44].

There are a multitude of methods for measuring technical skills in surgery that use varying degrees of complexity [111, 112]. These range from measurement of simple metrics, such as time and dexterity, through to global and procedure-specific rating scales and error-based checklists, as well as more complex assessments of higher level cognitive function using gaze tracking and functional brain imaging.

Motion analysis systems, such as the Imperial College Surgical Assessment Device (ICSAD) , use an electromagnetic tracking system that monitors the motion through space of sensors placed on the dorsum of the surgeon’s hands to record a variety of dexterity parameters , such as time to task completion and economy of motion [99]. This system has been validated as an objective assessment tool, and can distinguish between surgeons of differing skill levels [113, 114]. Likewise, virtual reality surgical simulators provide an opportunity for users to practice tasks of varying complexity and produce similar objective measures of dexterity, as well as record errors made, in real time. Not only have such models been validated as accurate assessment tools, but they have also been used to evaluate expert skill level to generate performance goals for trainees to practice within structured curricula [115, 116].

In contrast to the aforementioned dexterity assessment tools, direct observational assessment tools utilize rating scales to quantitatively assess the quality of operative performance. Broadly classified into global and procedure-specific rating scales, these tools require an observer to evaluate performance. Global rating scales, such as the Objective Structured Assessment of Technical Skill (OSATS) scale , assess generic operative skills, such as respect for tissue and instrument handling. The OSATS scale has been demonstrated to be a reliable and valid method of assessing operative skill in both the simulated and actual operating room environment [114, 117]. Nevertheless, the lack of ability to provide feedback on specific aspects of a particular procedure has led to the development of procedure-specific rating scales. These allow objective assessment of performance during individual operations to define specific areas of weakness that then can be practiced deliberately. Such tools have been developed and validated for a number of operations including cholecystectomy, gastric bypass, and colorectal, ear, nose, and throat, and cardiac surgery [44, 114, 118–122]. In a landmark publication, Birkmeyer demonstrated that superior performance by expert surgeons during gastric bypass surgery—as assessed by a procedure-specific rating scale—was associated with fewer postoperative complications, reoperation rates, readmissions, and, crucially, mortality [109].

More recently, more sophisticated methods of assessing surgical skill have been developed, such as gaze tracking and functional brain imaging. By using stationary cameras or cameras integrated into standard eyeglasses to record corneal reflection of infrared light, pupil position can be tracked to generate a map of the surgeon’s focus of attention during surgery [84, 123]. Additionally, other eye metrics can be obtained, including fixation frequency and dwell time; these indicate the degree of importance ascribed by the surgeon to a particular stimulus. In addition, pupillary dilation is a surrogate marker of effort and concentration. Indeed, a recent systematic review concluded that gaze tracking is feasible and valid as an objective measure of ability, and can produce reliable quantitative data differentiating between varying levels of surgical skill [123].

Similarly, the use of functional brain imaging provides a novel approach to measuring surgical proficiency. Functional magnetic resonance imaging (fMRI) has been utilized in other highly skilled domains such as sport and music [124, 125]. In a recent feasibility study using fMRI, Morris measured the blood oxygen level-dependent signal changes (BOLD) in specific brain regions while subjects performed and imagined performing hand tying of surgical knots [126]. Decreased BOLD activity was observed during knot-tying by experts when compared to novices. Further, increased BOLD activity was observed in experts when imagining performing hand ties compared to novices. This study demonstrated that using fMRI to assess surgical skill was feasible and specific regions of interest were identified through brain mapping.

Increasingly, attention has been directed to the concept of the learning curve in surgery. As a strategy, preoperative warmup and pre-procedure rehearsal exercises performed by surgeons at all levels of expertise lead to improved performance during the operative procedure [24, 127], but also serve to document a surgeon’s learning curve by longitudinal analysis of repeated performance.

Nontechnical skills (NTS) encompass a range of competencies, including communication, teamwork, leadership, decision making, situational awareness, managing stress, and coping with fatigue. In contrast to methods of evaluating technical skills, the assessment of NTS almost exclusively relies on rating scales and checklists that include specific definitions and examples of behaviors representing superior or substandard performance at each measured NTS. These tools can be used in both the simulated and actual clinical environment, and rely on direct observation of subjects. Surgeons have been shown to be reasonably accurate at self-assessing their technical skill, but lack sufficient insight to accurately self-assess their own NTS [128]. Several instruments have been created to evaluate NTS with considerable overlap, demonstrating the importance of some of these competencies to a number of academic surgical teams. Some of these instruments are discussed below.

One of the pioneering tools for NTS assessment is the Observational Teamwork Assessment for Surgery (OTAS) tool, which was developed in 2006 [129] to comprehensively assess the interprofessional teamwork of an entire operating room team, including communication, coordination, cooperation/backup behavior, leadership, and team monitoring/situation awareness. While it is valid and reliable, OTAS requires real-time observation, and raters must be adequately trained to use the scale [130].

Non-Technical Skills for Surgeons (NOTSS) was also developed in 2006 through cognitive task analyses with expert surgeons to identify five categories of NTS, including situational awareness, decision making, task management, leadership, and communication/teamwork [131]. While NOTSS has been demonstrated as a reliable assessment of surgeons’ NTS [132], novice assessors tended to score lower than expert assessors, again indicating the need for formal training in using NOTSS [133]. Crossley evaluated NOTSS as a real-world assessment tool using a mix of minimally trained assessors and demonstrated evidence to suggest that the scale is reliable and feasible to be used in the actual operating room [134]. Developed using a similar methodology to NOTSS, the Anaesthetists’ Non-Technical Skills (ANTS) and Scrub Practitioners’ List of Intraoperative Non-Technical Skills (SPLINTS) rating scales have also been shown to be reliable and valid in assessing NTS of anesthetists and instrument nurses [135, 136].

The revised NOn-TECHnical Skills (NOTECHS) rating scale is a validated and reliable instrument adapted from the aviation industry by Sevdalis and colleagues [137] for use in the operating room, and designed to measure the NTS of both the individual surgeon and the team as a whole [138]. Categorizing NTS into five domains, including communication/interaction, situational awareness/vigilance, cooperation/team skills, leadership/managerial skills, and decision making, the NOTECHS rating scale can be used in real time and requires minimal prior training for assessors [138]. Mishra developed the Oxford NOTECHS, as a variant of the original scale, with the aim of assessing the NTS of the entire operating room team [139], and a modified, higher resolution version was subsequently developed, with an increased number of performance indicators, particularly in the normal spectrum of behavior [140]. Further modifications of NOTECHS include the trauma NOTECHS (T-NOTECHS), which allows assessment of NTS that are crucial for effective and efficient management of trauma [141, 142]. Henrickson Parker and colleagues conducted focus group discussions to identify leadership characteristics of a surgeon [143]. These included maintaining standards, managing resources, making decisions, directing, training, supporting others , and coping with pressure. From this, the Surgeons’ Leadership Inventory (SLI) was developed and subsequently demonstrated to be a reliable means of assessing leadership with the operating room [143].

“Failure to rescue” patients whose condition deteriorates during the postoperative course has been suggested to be responsible for a large proportion of variability seen in patient outcomes within surgery. As stated previously, experts are able to monitor and detect subtle deviations from the usual postoperative course, and act swiftly to prevent such failures. The ability to develop these skills and conduct an efficient, accurate, and safe ward round requires the same deliberate practice required to master technical skills in the operating room. Recent development and validation of the Surgical Ward care Assessment Tool (SWAT) has enabled evaluation of patient assessment and management by surgeons [144]. This instrument comprises a checklist of assessment tasks, ranging from reviewing the vital signs chart and laboratory test results to performing a physical examination of the abdomen. Additionally, the checklist includes a number of management tasks, such as reviewing requirements for analgesia, antibiotics, and fluids. Further, the authors modified the T-NOTECHS scale to produce and validate the W-NOTECHS rating scale. For each NTS domain—including leadership, cooperation, resource management, communication and integration, assessment and decision making, global awareness, and coping with stress—five-point Likert scales were used to rate performance. Both the SWAT and W-NOTECHS scales have been demonstrated to reliably assess performance during ward rounds and provide structure for the development of expertise in the art of conducting a ward round through a cycle of objective assessment and deliberate practice [145].