Databricks builds batch vector search into its runtime engine

Databricks says it has built vector search directly into its runtime engine, replacing a design that sent a separate network request for each query row to an external search endpoint. Its NEAREST BY join finds the best matching rows in one table for each row in another, supporting batch tasks such as deduplication, classification and record enrichment. A specialized Photon execution operator reuses vectors across batches and can retain only the best matches rather than emitting every scored pair. Queries marked EXACT exhaustively evaluate candidates to guarantee the true best matches; APPROX queries permit approximate strategies, including an optional index stored as a Delta table.