In a recent ETR Insights interview, we spoke to the Vice President and Principal Software Engineer for a large financials enterprise. The interview kicked off with a discussion of the macro environment. Although he is understandably cautious, our guest does not expect his organization to make any “drastic cuts” to the budget or any reductions in thier cloud workloads. Instead, his organization is looking at consolidating and controlling costs where possible.
The detailed conversation continues to assess competition among Public Cloud providers and differentiators between core players AWS and Microsoft Azure. After that, our guest discusses container adoption and serverless technologies, including Kubernetes and Lambda. Finally, our guest dives into analytics and data warehousing and explains why he doesn’t consider Redshift and Snowflake direct competitors but has his eyes on Databricks encroaching in the analytics space. In this article, we dive into that last topic to garner expert commentary and vendor feedback within data analytics. See below for direct excerpts from the interview:
Can AWS Usurp Snowflake?: So far, they have been unable to. This is my area of expertise, and so far, they're struggling. I think AWS has a “too many cooks in the kitchen” problem. Redshift is probably the closest Snowflake competitor, but they're definitely not building out everything else that Snowflake has. I don't see them taking over Snowflake at any point in this space. Snowflake is singularly focused on execution for the data warehousing niche they’re in, but I should point out Snowflake still runs on AWS, Azure, or GCP. AWS still gets a cut of whatever Snowflake is doing, which is essentially what I'm saying. Processing and storage are effectively passthrough bills, but Snowflake has a lot of value.
They're able to start attracting a lot of value-add things through their marketplace – direct sharing, listings, discoverability, and just the network effect of having all of your clients and all over your vendors on the same data platform. I don't think AWS is focused on that exact use case. Even though they have one too many products that are able to tackle it, I don't think they're focused on it because they are focused on so many other things and so many other use cases. So, I still see Snowflake staying at the top for this particular set of vendors within these use cases.
Competition with AWS Redshift: Snowflake is one main product, whereas at AWS there are several. Specific to Redshift, it is a fork of a very old Postgres database, but it's quickly evolving through multiple engine rewrites, etc., to be the type of product that, as you mentioned with Snowflake, has that separation of storage and compute. For Snowflake they have three things: 1) The separation of storage and compute, meaning multiple people can query stuff without bumping into each other, and all on-demand. 2) Direct data sharing, so I load a table on my side, I share it with you, you get it, you don't need to run ATL, and you can use it with the rest of your stuff. Redshift is catching up to that. 3) And then the marketplace. In the marketplace, they have capability through AWS Data Exchange. I don't know if that's integrating Redshift directly just yet, but that's also meant to be a Snowflake marketplace competitor, where you can look for content sets. So, the data warehousing capability is about the same. I think it also runs serverless at this point. They're quickly iterating on these features, but they're still playing catch up. Also, AWS has, like, eight other database products and various flavors, whereas, with Snowflake, there's no other database you can choose.
Future of Snowflake Marketplace: Right now, there are only so many financial service providers or even data providers by category in that marketplace. As soon as more vendors start opening up, I think the more intrinsic value Snowflake will have. I think there are plenty of legs to go there. I think they're only scratching the surface for it, and they keep innovating on the types of things they’re able to deliver. At this point, it's not just delivering your content, but you're able to package up your internal business logic – through AWS Lambda, mind you – and easily be able to sell that as well. So, for example, if I didn't want to charge multiple thousands of dollars for a content set, but if you just want to take a ticker and see its exposure to something, I can package that up and charge you a couple of hundred dollars per run. You only need to do so many of them, right? So that's the business. That's code that's running in my AWS region but is seamlessly integrated into your environment. You can run it as a Snowflake function, for example, and it spits you back the results. They're looking into that kind of pricing model, and I think they take a larger cut of that. There are native applications. They're looking forward to being able to share more and more stuff through this platform. I think they’re in the early stages of what this thing [Snowflake Marketplace] can really do.
Databricks Getting Bigger & Bigger: They [Databricks and Snowflake] seem to be attacking the same problem from different sides, if you will. Databricks has a lot of overlap with Snowflake. Databricks also has Delta sharing format, which is similar to Iceberg with in-place sharing. They're putting in SQL capacity. But yeah, for consumers of a certain level of sophistication, Databricks seems to be the way to go and the way to run your models for any of your analyses. I see them getting bigger and bigger in the space. Again, it's a joint effort with AWS, Microsoft, or whatever cloud platform they're running on. The processing model is the same as Snowflake. You’re running Snowflake software on some cloud vendor, the same with Databricks. Any of the popularity that these two vendors have here is passed through for AWS, Microsoft, or GCP.
If you want to explore more expert comments from ETR's ITDM community
start your free trial and access our searchable transcript library with hundreds of expert interviews.
Enterprise Technology Research (ETR) is a technology market research firm that leverages proprietary data from our targeted IT decision maker (ITDM) community to bring you actionable insights about spending intentions and industry trends. Since 2010, we have worked diligently at achieving one goal: eliminating the need for opinions in enterprise research, which are often formed from incomplete, biased, and statistically insignificant data. Our community of ITDMs represents $1+ trillion in annual IT spend and is positioned to provide best-in-class customer/evaluator perspectives. ETR’s proprietary data and insights from this community empower institutional investors, technology companies, and ITDMs to navigate the complex enterprise technology landscape amid an expanding marketplace. Discover what ETR can do for you at www.etr.ai