In our previous blog (opens new window), we explored how combining vector search with reranking methods addresses inherent limitations of vector search, such as the possible loss of information during text-to-vector transformations. This integration makes it a robust solution for complex query intents, significantly improving overall search accuracy.

Amidst a variety of reranking solutions available today, Jina AI (opens new window) stands out for the exceptional performance of its Reranker API (opens new window). In response, MyScale promptly updated its reranking function to support the Jina Reranker, further simplifying the integration of advanced search functionalities for developers and businesses.

What is Jina AI Reranker

Jina AI's Reranker API (opens new window) enhances search and retrieval tasks by reordering retrieved documents in a manner that deeply understands both the context and the semantics of the search query. Benchmarks show that Jina Reranker significantly improves search accuracy, particularly for complex natural language queries where intent and context are critical.

Key Benefits

• Enhanced Precision and Contextual Relevance: The Jina AI Reranker refines search results to closely align with user intent, significantly boosting user satisfaction. It consistently outperforms competitors in essential reranking benchmarks, such as BGE (BAAI), BCE (Netease Youdao), and Cohere, establishing it as the leader in the field. This superior performance underscores its ability to deliver precise and contextually relevant results.
• Easy Integration: A simple API makes it easy to integrate Jina Reranker into existing search systems.
• Customizability: Users can easily specify different models and parameters to fit their specific needs.

What is MyScale Reranking Function

The MyScale Reranking Function integrates mainstream reranker APIs, such as HuggingFace, Amazon SageMaker, Cohere, and Jina AI, into a cohesive framework. This powerful feature allows users to easily call the APIs of these leading providers through a simple SQL statement, streamlining the implementation process. By incorporating such diverse and powerful reranking tools, MyScale enhances the adaptability and efficiency of search functionalities across different platforms.

For more detailed information on how to leverage these capabilities within MyScale, please visit our documentation (opens new window).

Using Jina AI Reranker in MyScale

A practical strategy to enhance efficiency in handling reranking with SQL is to declare a User-Defined Function (UDF) that encapsulates reranking parameters, including the provider name and API key. This approach prevents the need to repeatedly specify these details in every query and simplifies updates to credentials or provider details.

Below is an SQL statement that declares a function named JinaAIRerank. You can get a Jina Reranker API key from the Jina AI website. and get a million free tokens to try it out. Then replace 'JINAAI_API_KEY' with your actual API key:

CREATE FUNCTION JinaAIRerank ON CLUSTER '{cluster}' AS (x,y,z) -> Rerank(x, y, z, 'Jina', '', 'JINAAI_API_KEY', '')

Once the function is set up, you can rerank query results by simply calling JinaAIRerank. Here’s how to use this function to rerank a list of documents and return the most 'TOP_K' relevant documents with the 'query':

SELECT JinaAIRerank('query', ['doc', ...], TOP_K)

Conclusion

The integration of Jina AI's Reranker API with MyScale significantly enhances search capabilities, making it easier for developers to align search results with user intent. This collaboration marks a step forward in search technology, offering a simpler, more efficient way to handle complex queries. By adopting this advanced reranking solution, businesses can expect improved search accuracy and user satisfaction.