Man vs machine: plastic surgery edition
In an intriguing face off, a team of researchers have challenged an experienced rhinoplasty surgeon to contend against ChatGPT, by comparing and assessing the quality of responses to a series of patient questions related to plastic surgery.
Before you get too excited, let me just clarify that unfortunately no, a robot did not perform actual plastic surgery. Man was instead pitted against machine in a duel of questioning, rather than scalpels. This more refined competition, however, appears not to have detracted from the fascinating results.
The AI contender was none other than ChatGPT, while an expert rhinoplasty surgeon with over two decades of experience stood as the opposition. Both were asked a series of preoperative and postoperative patient questions pertaining to nasal surgery. Seven expert rhinoplasty surgeons, blinded to the origin of the answers, then independently evaluated the responses using a 5-point Likert scale in four performance areas: empathy, completeness, accuracy and overall quality.
Interestingly, the researchers discovered that ChatGPT’s responses scored significantly higher ratings in accuracy, completeness and overall quality, and were decidedly preferred over the surgeon’s responses. The seven evaluators favored ChatGPT’s responses 80.95% of the time.
These findings demonstrate that ChatGPT is capable of providing accurate high-quality responses to a wide range of patient questions. This implies that large language models like ChatGPT could greatly support patients by educating and counseling them at a level equal to or surpassing that of an expert clinician.
Although such AI tools are promising educational resources for patients, we must consider the concerns over safety and trust. “It remains our responsibility to counsel patients on the occasional inaccurate statements made in AI-generated content. We can both acknowledge that most of our patients will be using AI and focus our efforts on maintaining the integrity of our patient education and consultation practices,” stated Travis Tollefson, Professor and Director of Facial Plastic and Reconstructive Surgery at the University of California, Davis (CA, USA).
