Spade: An Innovative Approach to Synthesize Assertions for Identifying Errors in Large Language Models

A team of researchers from UC Berkeley, HKUST, LangChain, and Columbia University have developed a new system called Spade that automatically generates tests to identify errors in large language models(LLMs) like ChatGPT, Gemini, Claude, and Others. 

Published on January 5th, 2024, 

The research demonstrates how Spade can generate customized tests to evaluate AI-powered data generation pipelines without the need for extensive training data.

LLMs such as ChatGPT have gained immense popularity in recent years due to their capability to generate human-like text and engage in natural conversations. However, these models are susceptible to unpredictable failures, such as generating inappropriate, incorrect, or nonsensical responses. As more companies incorporate LLMs into their data generation pipelines, it is crucial to have rigorous tests to identify these failures before integrating them into production systems.

"LLM errors are inevitable due to their statistical nature," explains lead author Shreya Shankar, PhD student at UC Berkeley. "Our key insight was that tests tailored to common failure modes observed during prompt development could catch bad outputs without needing thousands of labeled examples."

The spade system analyzes the evolution of prompt instructions to automatically generate a suite of customized tests, called assertions, that check desired criteria. For instance, if a prompt specifies "Use positive language," Spade will generate an assertion to verify no negative words are present.

"We found prompt refinements contain rich signals on how to evaluate LLMs," says co-author Aditya Parameswaran, Assistant Professor at UC Berkeley. "spade mimics what a developer intuitively does when phrasing prompts - any new instruction implies a test concept."

To filter out redundant and inaccurate tests, spade formulates the selection as an optimization problem using integer linear programming. The system chooses a small set of tests that maximize coverage of potential failures while minimizing false failures on valid responses.

"Reducing assertions is important to avoid overwhelming users, but balancing coverage and accuracy is tricky with minimal data," explains co-author Eugene Wu, Associate Professor at Columbia University. "We introduce the idea of subsumption to retain tests that broaden coverage beyond what's in the sample data."

In their empirical study, the researchers deployed spade across nine different real-world LLM pipelines spanning diverse domains such as finance, medicine, and IT. The objective was to evaluate the effectiveness of spade in automatically generating assertions and improving the robustness of these pipelines. Notably, the team collected labeled prompt-response pairs, with eight of them being open-sourced to contribute to the research community. This transparent approach allows other researchers and developers to scrutinize and build upon the findings.

The results showcased the prowess of spade, demonstrating its ability to generate assertions with substantial failure coverage and minimal false failures across all tested pipelines. The system's subsumption-based ILP (Integer Linear Programming) outperformed simpler baseline approaches by reducing the number of assertions by a commendable 14% and simultaneously lowering the false failure rate by an impressive 21%. This reduction in both the number of assertions and false failures indicates spade's efficiency in optimizing the trade-off between comprehensive testing and minimizing the risk of false negatives.

"We were excited by spade's strong performance with little labeled data," remarks Shankar. "Subsumption was key to ensuring comprehensive assessment of failures without exhaustive manual labeling."

An essential aspect of spade's functionality lies in its ability to leverage prompt version histories. The research introduces a comprehensive taxonomy of prompt deltas obtained from 19 LLM pipelines. This taxonomy provides a structured understanding of the different types of prompt refinements that may trigger specific failure modes in LLMs. Categories include structural changes, response format alterations, prompt clarification, and various qualitative criteria.

This taxonomy not only enriches the understanding of the spade methodology but also serves as a valuable resource for developers and researchers delving into the intricacies of large language model pipelines. The taxonomy acts as a guide, offering insights into the potential pitfalls and challenges that may arise during the development and refinement of prompt templates for LLM-based data generation tasks.

The researchers have taken a proactive approach by publicly releasing a tool associated with spade for generating candidate assertions. This tool has seen over 1300 deployments across diverse sectors, including finance, medicine, and IT. The widespread adoption of this tool signifies its practical utility and aligns with the broader trend of open-sourcing tools to foster collaboration and accelerate advancements in the AI community.

The tool not only provides an early indication of the real-world applicability of auto-generated assertions but also serves as a platform for gathering valuable feedback from users across different industries. This collaborative approach is crucial in refining and enhancing the tool's capabilities, ensuring that it continues to meet the evolving needs of developers and researchers engaged in LLM-based data generation.

"As LLMs continue maturing at an incredible pace, pipelines leveraging these models are becoming highly complex," cautions Parameswaran. "Thorough testing is crucial before deployment, but designing exhaustive, accurate tests manually simply doesn't scale."

While the results are promising, the researchers acknowledge the challenges and limitations in their current approach. Looking ahead, they envision spade and similar assertion generation techniques as crucial elements in increasing trust and preventing harmful failures induced by large language models. The emphasis on the importance of automated testing in the maturing landscape of LLMs is highlighted, laying the groundwork for future research and development in this critical domain.

"Assertion generation techniques like spade will be critical to increasing trust and preventing harmful LLM-induced failures," emphasizes Wu. "We're excited to see rapid adoption of spade and hope our work catalyzes more research in this important area."

In conclusion, the spade system emerges as a pioneering solution to the complex task of automatically generating assertions for data-generating LLM pipelines. Its empirical success and the associated taxonomy contribute not only to the specific field of LLM testing but also to the broader discourse on the responsible deployment of AI technologies in various industries. As LLMs continue to evolve at a rapid pace, the role of automated testing methodologies like spade becomes increasingly indispensable for ensuring the reliability and safety of AI-driven applications. With LLMs powering everything from medical diagnosis to financial planning, rigorous testing is paramount to safely realize their benefits. As spade's inventors aptly summarized, "LLM errors are inevitable, but identifying them doesn't have to be."

Checkout the Research Paper , for more details.

All credit for this research goes to the researchers of this project.

Hey, join our AI SubReddit, Facebook Community, Discord Channel, Email Newsletter, And also follow us on Facebook, Instagram, Twitter, there we share the latest AI research news, awesome AI projects, AI guides/tutorial, Best AI tools, and more.