Source URL: https://cloud.google.com/blog/products/ai-machine-learning/announcing-vertex-ai-prompt-optimizer/
Source: Cloud Blog
Title: Announcing Public Preview of Vertex AI Prompt Optimizer
Feedly Summary: Prompt design and engineering stands out as one of the most approachable methods to drive meaningful output from a Large Language Model (LLM). However, prompting large language models can feel like navigating a complex maze. You must experiment with various combinations of instructions and examples to achieve the desired output. Moreover, even if you find the ideal prompt template, there is no guarantee that it will continue to deliver optimal results for a different LLM.
Migrating or translating prompts from one LLM to another is challenging because different language models behave differently. Simply reusing prompts is ineffective, so users need an intelligent prompt optimizer to generate useful outputs.
To help mitigate the “prompt fatigue" experienced by users while they build LLM-based applications, we are announcing Vertex AI Prompt Optimizer in Public Preview.
What is Vertex AI Prompt Optimizer?
Vertex AI Prompt Optimizer helps you find the best prompt (instruction and demonstrations) for any preferred model on Vertex AI. It is based on Google Research’s publication (accepted by NeurIPS 2024) on automatic prompt optimization (APO) methods, and employs an iterative LLM-based optimization algorithm where the optimizer model [responsible for generating paraphrased instructions] and evaluator model [responsible for evaluating the selected instruction and demonstration] work together to generate and evaluate candidate prompts. Prompt Optimizer subsequently selects the best instructions and demonstrations based on the evaluation metrics the user wants to optimize against. Instructions include the system instruction, context, and task of your prompt template. Demonstrations are the few-shot examples you provide in your prompt to elicit a specific style or tone from the model response.
With just a few labeled examples and configured optimization settings, Vertex AI Prompt Optimizer finds the best prompt (instruction and demonstrations) for the target model and removes the need for manually optimizing existing prompts every time for a new LLM. You can now easily craft a new prompt for a particular task or translate a prompt from one model to another model on Vertex AI. Here are the key characteristics:
Easy optimization: Quickly optimize prompts for any target Google model, including migration and translation of prompts from any source model.
Versatile task handling: Accommodates any text-based task (such as question and answering, summarization, classification, and entity extraction) and expand support for multimodal tasks is coming soon.
Comprehensive evaluation: Supports a wide array of evaluation metrics, including model-based, computation-based, and custom metrics, to ensure optimal prompt performance against the metrics you care about.
Flexible and customizable: Tailor the optimization process and latency with advanced settings and utilize various notebook versions to suit your expertise level and needs.
Why use Vertex AI Prompt Optimizer?
Data-driven optimization: Many existing prompt optimization tools focus on tailoring your prompts to your preferred style and tone and oftentimes still require human verification. However, Vertex AI Prompt Optimizer goes beyond this by optimizing your prompts based on specific evaluation metrics, ensuring the best possible performance for your target model.
Built for Gemini: If you’re using Gemini, Vertex AI Prompt Optimizer is designed to keep Gemini’s underlying characteristics in mind. It is specifically designed to adapt to the unique attributes of the Gemini and other Google models. This tailored approach allows you to unlock the full potential of Gemini and achieve superior results.
Getting started with Vertex AI Prompt Optimizer
To start using Vertex AI Prompt Optimizer, you can use the Colab notebook available in the Google Cloud Generative AI repository on Github which contains sample code and notebooks for Generative AI on Google Cloud. Refer to the UI version for basic settings and the SDK version for more advanced settings. More versions of the notebook to support custom metrics and multimodal input will be added in the coming weeks. You can also access it via the Vertex AI Studio console. Look for entry points in the console that indicate “prompt optimizer” or “optimizer your prompt further” (refer to screencasts below).
To either optimize or translate prompts using Vertex AI Prompt Optimizer, follow these steps:
Configure your prompt template
Input your data (labeled examples)
Configure your optimization settings (target model, evaluation metrics, etc.)
Run the optimization job
Inspect the results
Vertex AI Prompt Optimizer supports any Google models and evaluation metrics supported by the Generative AI Evaluation Service.
Entry points from Vertex AI Studio to Vertex AI Prompt Optimizer Colab Enterprise Notebook
A. The Saved prompts page will include a new Prompt optimizer button.
B. The Prompt assist dialog pop-up will include a new Optimize your prompt further button.
How AdVon Commerce and Augmedix enhanced Gemini prompts using Vertex AI Prompt Optimizer
AdVon Commerce – a digital commerce platform – partnered with Google Cloud to create quality content at scale for retailers using tailored AI solutions. AdVon Commerce utilizes LLMs to generate accurate and engaging product page content at scale, which incorporates the right keywords and represents the products accurately. When optimizing retail pages, there’s a lot of missing or incorrect data to work through. Creating shopper-first content means accurately completing missing product attributes that are essential for product searchability and customer journey.
Vertex AI Prompt Optimizer streamlined the creation and refinement of AI prompts, resulting in higher accuracy and relevance. AdVon Commerce observed a 10% increase in attribute accuracy and are able to maintain their commitment to high-quality content while significantly reducing the time they spend on human verification, saving substantial costs. Coupled with Gemini Flash, they have received impressive results with a reduction in incorrect specs and better quality in product page content. For example, AdVon Commerce has recently helped one of the largest global retailers by using Vertex AI Prompt Optimizer and Gemini 1.5 Flash to automate the process of creating populating product attributes for hundreds of millions of items. This resulted in a 100x increase in productivity for the retailer, as it would have taken 100 times longer if they had tried to do that manually.
Vlad Barshai, Chief Technology Officer at AdVon Commerce, stated “Vertex AI Prompt Optimizer allows us to optimize our prompts for Gemini Flash with 10% incremental improvements for problematic Product Attributes and PDP (Product Detail Page) for retailer listings, significantly surpassing results from all other leading AI models on the market. With Vertex AI Prompt Optimizer, we save time on human verification, allowing us to enrich millions of products in a loop where we optimize prompts and generate AI attributes and PDP content at scale. Coupled with a solid human-in-the-loop process, Vertex AI Prompt Optimizer will help us produce high quality enrichment every time.”
Augmedix – a leader in ambient AI medical documentation and data solutions that has generated over 10 million medical notes to date – partnered with Google Cloud to enhance medical documentation for healthcare providers. Augmedix utilizes LLMs to improve efficiency and accuracy in capturing patient interactions, reduce clinician administrative burdens, and ultimately, improve patient care. Augmedix adopted a hybrid approach – models are fine-tuned and inputs are prompt-tuned. For many parts of note generation, fine-tuning is best, and basic prompts work well. In other parts of their system, where there may be hundreds of rules that instruct the LLM, prompt-tuning these rules is optimal.
Augmedix employed Vertex AI Prompt Optimizer to enhance medical note generation from doctor-patient conversations. The feature improved LLM output quality scores from 66% to 86%. In addition, with Vertex AI Prompt Optimizer, Augmedix can test prompt variations quickly, allowing for faster iteration and optimization. The optimized prompts run in 6 seconds, compared to 20 seconds for prompts without Vertex AI Prompt Optimizer.
Ian Shakil, Founder, Director, and Chief Strategy Officer at Augmedix, stated, “Our partnership with Google Cloud AI enabled us to pioneer the frontier of the LLM wave. With MedLM and Gemini, we have achieved revolutionary advancements, driving cutting-edge innovation in the digital health space. This collaboration empowers us to deliver higher quality outputs, reduced turnaround times, and a richer feature set.”
What’s next
If you want to know more about Vertex AI Prompt Optimizer, join the Vertex AI Google Cloud community to share your experiences, ask questions, and collaborate on new projects. Also check out the following resources:
Documentation
Optimize prompts with Vertex AI Prompt Optimizer
Github samples
Getting started with Vertex AI Prompt Optimizer UI
Getting started with Vertex AI Prompt Optimizer SDK
Google for Developers blog post
Enhance your prompts with Vertex AI Prompt Optimizer
Medium Google Cloud Community blog post
Vertex AI Prompt Optimizer Medium post
AI Summary and Description: Yes
Summary: The text describes Vertex AI Prompt Optimizer, a tool that uses automatic prompt optimization methods to enhance the effectiveness and adaptability of prompts for Large Language Models (LLMs). This innovation alleviates the challenge of prompt engineering by offering an intuitive, data-driven approach to optimize prompts for various LLMs, demonstrating significant improvements in application efficiency for users in sectors such as digital commerce and healthcare.
Detailed Description:
– **Overview of Vertex AI Prompt Optimizer**:
– Designed to optimize prompts for any model on the Google Vertex AI platform.
– Introduces an iterative approach to prompt refinement using LLMs, enhancing output quality.
– Offers a user-friendly interface via Google Cloud, simplifying the process of prompt creation and optimization.
– **Key Features**:
– **Easy Optimization**: Quickly optimize prompts for any Google model, facilitating prompt migration from one model to another.
– **Versatile Task Handling**: Supports various text-based tasks such as Q&A, summarization, classification, and plans for multimodal support.
– **Comprehensive Evaluation**: Incorporates various evaluation metrics—model-based, computation-based, and custom metrics—to ensure optimal performance.
– **Flexibility**: Allows users to tailor optimization settings to suit their level of expertise and specific needs.
– **Innovative Approach**:
– Focuses on data-driven optimization rather than merely styling prompts, which can often require human oversight.
– Specifically tailored for the Gemini model, allowing for enhanced performance and usability.
– **Use Cases**:
– **AdVon Commerce**: Utilized Vertex AI Prompt Optimizer to improve content creation for e-commerce. Achieved a 10% boost in attribute accuracy and improved efficiency leading to a 100x increase in productivity for a significant retailer.
– **Augmedix**: Enhanced medical documentation processing, raising output quality scores from 66% to 86% while decreasing processing times from 20 seconds to 6 seconds per prompt.
– **Practical Implications**:
– AI and cloud professionals can leverage Vertex AI Prompt Optimizer to enhance their LLM applications considerably, reducing labor-intensive prompt engineering efforts.
– The tool exemplifies the integration of AI innovations within enterprise frameworks, showcasing its ability to streamline operations and optimize performance.
– **Getting Started**:
– Available through Google Cloud with comprehensive resources for users to get involved, share experiences, and collaborate with others.
This analysis highlights the significance of Vertex AI Prompt Optimizer in advancing the capabilities of LLMs, making it a vital tool for professionals focused on AI, cloud computing, and innovative applications in various industries.