# Eclipse Aerospace replaces flight-data pipeline with direct Spark decoding

_Published Thursday, November 12, 2026 at 2:41 PM EST · Products · Latest · Tier 2 — Notable_

![Eclipse Aerospace replaces flight-data pipeline with direct Spark decoding — Primary](https://www.databricks.com/sites/default/files/2026-11/blog-og-eclipse-aerospace.png)

Eclipse Aerospace has replaced a multi-stage flight-data ingestion pipeline with a connector that decodes proprietary files directly inside Spark, Databricks said. The change removes an AWS Lambda processing step and intermediate Avro copies of flight records.

The connector wraps Eclipse's existing Rust decoder in Python and transfers decoded records through Apache Arrow without copying the data. Databricks says it can process records within a minute of upload, including startup time, and commonly handles large, multi-gigabyte batches in under two minutes.

Eclipse stores more than 3TB of flight data spanning nearly 20 years. The compiled connector requires care over software-version and processor-architecture compatibility.

## Sources

- [Databricks](https://www.databricks.com/blog/eclipse-aerospace-simplifies-flight-data-ingestion-databricks)

---
Canonical: https://techandbusiness.org/newswire/p8V8dcrlvqm8p7r_1qp5B-
Published: 2026-11-12T19:41:05.328Z
Story chronology: 2026-11-12T19:20:48.000Z
Retrieved: 2026-11-12T21:54:50.387Z
Publisher: Tech & Business (techandbusiness.org)