Databricks Services Partner Ecosystem Update

We introduced a Databricks services partner ecosystem analysis mid-last year (https://alten.capital/blog/databricks-services-partner-ecosystem). With the company’s upcoming Data + AI Summit in San Francisco, CA, getting an updated view of Databrick’s partner efforts makes sense.

Databricks was started by the original creators of Apache Spark in 2013. The company's mission is to "accelerate the development of data science and machine learning." Databricks provides a cloud-based data platform that makes using Apache Spark for data engineering, data science, and machine learning easy. The company’s revenue was estimated at $1+ billion in 2022 and $1.5+ billion in 2023.

Apache Spark was created in 2009 at the University of California, Berkeley AMPLab. It was developed as a response to the limitations of MapReduce, a popular big data processing framework at the time. MapReduce is a batch processing framework, meaning it must read all input data into memory before it can begin processing. This can be a significant bottleneck for large datasets. On the other hand, Spark is a general-purpose cluster computing system that can be used for batch and streaming processing. It uses a more in-memory approach to processing, which can significantly improve performance for large datasets. Spark is very well suited for analytical workloads, and its user base includes data engineers, data scientists, and machine learning engineers.

Spark is now one of the world’s most popular big data processing frameworks, used by over 20,000 organizations (Databricks is estimated to have over 10,000 worldwide customers).

Like other enterprise software platforms, Databricks created a partner program to accelerate the adoption of its solutions. Below are some highlights of its service partner ecosystem, which has grown more than 60% in one year.

In the past year, Databricks has expanded its roster of consulting partners to cover a wide range of company sizes, particularly mid-tier companies with 500 to 999 employees. In particular, the number of partners in this category has doubled, highlighting the adaptability of Databricks' services to serve both large companies and smaller organizations. 

As of Q2 2024, Databricks has approximately 730 service partners (consulting, implementation, managed services, etc.). 300 partners (40%) are based in North America, and 230 (30%) are in Europe.

Looking at the data by partner headcount tiers, we find that 50% of Databricks services partners are small-scale (with fewer than 100 employees). 80% of all service partners have fewer than 1,000 employees, and these 580 partners account for only 1% of the total headcount tracked. There are estimated to be more than 100,000 certified Databricks and Apache Spark professionals worldwide. Boutique data engineering, analytics, and data science services companies see value in leveraging the Databricks platform for their customers.

Databricks is a key component of the modern data stack. Apache Spark’s open-source nature makes the overall approach more accessible for a larger set of users, and the platform’s web-based interface, notebook environment, and library of pre-built machine learning models make it appealing to data science and data analytics end-users.

At Alten Capital, we enjoy the technology services space and are fond of data / AI offerings. Please reach out to find ways to partner and scale your organization.