Google Cloud Unlocks New AI Capabilities with Improved Vertex AI Search for Healthcare
Google Cloud has announced two major improvements to its Vertex AI Search for Healthcare, an AI-driven system to help medical professionals easily find and interpret complex health data. The changes will allow the AI to integrate different types of data while enhancing speed and accuracy, giving doctors a more comprehensive picture of patient health to guide clinical decisions.
Vertex AI Search for Healthcare: Tackling an Overstretched Workforce
Google Cloud initially launched their Vertex AI Search for Healthcare in October 2023 at the annual HLTH conference. With the buzz around AI in healthcare building, generative AI was added to the 2021 Vertex AI system to help developers build specialist assistance tools for healthcare organizations.
More specifically, these technologies help clinicians quickly retrieve and interpret health records scattered across electronic databases, creating a unified view of patient health.
For example, if a doctor wants to know a patient’s genetic predisposition to a certain disease, instead of sifting through laboratory reports or electronic records, they can simply ask the AI “what genetic markers might indicate my patient is more at risk of X?” and instantly access all the relevant results.
The release of Vertex AI Search for Healthcare came at a critical time, after a report from Google Cloud and The Harris Poll revealed that medical professionals are massively overburdened by administrative work. At the time of the survey, clinicians reportedly spent almost 28 hours per week on administrative tasks, taking up a shocking 70% of the average 40-hour working week.
Doctors are drowning in administrative work, according to a 2023 survey.
What’s more, a 2016 study from the Americal Medical Association revealed that physicians were spending double the time on administrative work than they were on patient care; for every hour dedicated to seeing a patient, an additional two hours were devoted to admin.
In a press release around the time of the launch, Lisa O’Malley, senior director, Cloud AI Applications, Google Cloud, said, “The administrative burden on healthcare workers is immense, taking precious time away from what matters most: patient care.”
“[With the general availability of Vertex AI System for Healthcare], we’re providing powerful tools for our customers to build a strong data foundation and harness the power of generative AI to create more efficient and effective healthcare systems.”
Expanding Horizons with Visual Q&A
On Monday, Google Cloud announced a new round of upgraded capabilities for Vertex AI Search for Healthcare, including the ability to reason with image data using ‘Visual Q&A’.
Visual Q&A means that tables, charts, and diagrams can be directly input to Vertex AI. This is a significant advancement given an estimated 90% of healthcare data are in image formats. The upgrade will give clinicians a more complete picture of a patient’s current status and will supposedly help to reduce their mental workload.
Gemini 2.0
Another key upgrade to Vertex AI Search for Healthcare, is the integration of Gemini 2.0, which significantly enhances the platform’s performance in processing and analyzing multimodal data.
Gemini 2.0 outperforms the previous 1.5 Pro version on key benchmarks at twice the speed while accelerating data retrieval and improving the accuracy of search results.In effect, this offers an opportunity to help doctors to interpret more complex data, such as when making treatment decisions for patients with multiple health complications and long medication histories.
Aashima Gupta, global director of Healthcare Strategy & Solutions, Google Cloud, tells press:
“We’re helping clinicians to work more efficiently with new multimodal capabilities embedded in Vertex AI Search for healthcare. [This] analysis processes diverse sources of patient data, like medical images and genetic information, for a more comprehensive understanding and improved decision-making.”
Real-world Applications
MEDITECH — a leading medical electronic health record company—has created an AI-driven search and summarization feature as part of its Expanse Electronic Health Record system, a cloud-based platform that helps healthcare workers manage patient data more effectively. The new capabilities of Vertex AI Search for Healthcare will enable MEDITECH to further refine their innovation.
Cathy Turner, Chief Marketing and Nursing Executive, MEDITECH, says: “Building on our successful work with Google Cloud in developing AI-powered search and summarization, these advancements will further enhance our ability to deliver cutting-edge tools that empower clinicians and improve workflows.”
Suki —another Healthtech company— has utilized Google Cloud technology to integrate patient summarization and clinical Q&A features into Suki Assistant, a voice-enabled clinical assistant. The AI-driven tool can transcribe physician’s notes into structured documentation and generate summaries of healthcare discussions, aiming to streamline administrative tasks.
Visual Q&A and Gemini 2.0 will help Suki to optimize their assistant for data retrieval across different modalities—for example, integrating transcribed notes with medical scans or patient histories to enhance clinical AI-generated summaries.
“By combining Suki’s expertise in ambient AI with Google Cloud’s powerful multimodal capabilities, we are equipping clinicians with instant access to the critical information they need, enabling them to deliver the best possible patient care,” says Punit Soni, founder and CEO of Suki.
More Time for Patients
Vertex AI Search for Healthcare offers a crucial opportunity to build innovative healthcare technologies that could save crucial time normally spent on administrative tasks, allowing doctors to focus on patient care delivery.
