Open Standards and Lakehouse AI Development - Arrow Flight, Apache Iceberg and MCP
Open Data Lake, MCP, OSS AI
As AI development shifts toward more agentic and composable workloads, the modern data lakehouse must rise to meet new demands — performance, interoperability, and intelligent access to resources.
We’ll focus on three key technologies:
Arrow Flight – A high-performance data transfer protocol that enables low-latency, columnar data sharing between engines, perfect for real-time AI pipelines.
Apache Iceberg REST Catalog Spec – A critical standard that allows diverse tools and systems to communicate seamlessly with Iceberg catalogs, enabling broader compatibility and easier integration.
Model Context Protocol (MCP) – A new standard that empowers AI agents to discover and leverage available tools, datasets, and prompts in a structured, contextual way.
Together, these standards form the backbone of an open and flexible lakehouse ecosystem — enabling fast data movement, cross-system compatibility, and intelligent AI-agent orchestration.
Alex Merced is Head of DevRel for Dremio and co-author of "Apache Iceberg: The definitive guide" from O'reilly and has worked as a developer and instructor for companies like GenEd Systems, Crossfield Digital, CampusGuard and General Assembly.