It will be interesting to see how the rise of GenAI influences the somewhat crowded database market. From what it seems so far, factors that were important in the past are even more critical now, e.g., the need for nearly unlimited storage (cheaply); the need to intelligently consolidate and centralize data; and the ability to efficinelty query in real time.
The simple truth is that if companies can't store all of their data for a reasonable cost (that breaks across silos throughout the organization), then any AI running on the system will be severely limited. The AI is only as intelligent as the data it's fed, and if it's missing context, there will inevitably be more hallucinations and less insights.
There are other pieces at play here, such as does the database of choice have a good developer experience; is it serverless to help meet demand without scaling cost, and is it agile enough to keep up with trends like the rise of Postgres.
Here are the top database technologies to consider as we gear up for the new year:
#1 – Neon
You may have seen that Neon recently received a $25 million injection of cash from M12, Microsoft's venture fund. You may have also seen the explosive growth of Postgres over the last few years, which has become the database-of-choice for engineers building cloud applications.
This ambitious company is striving to make their serverless Postgres platform the easiest and most efficient way to spin us Postgres databases; developers are already spinning up over four thousand databases every day on the Neon platform, and this will only accelerate with AI.
The company also recently launched Neon Authorize, which gives engineers granular access controls at the database layer. This new offering leverages Postgres RLS (Row-Level Security)—a Postgres primitive that protects data from malicious actors even when accessed through third-party tooling—and, according to the company, makes it much simpler to use via new deployment models for application developers.
The company seems to have struck the perfect balance for cutting-edge companies that are scaling fast, such as OpenAI, Vercel, and Retool, while simultaneously building out enterprise-ready features such as out-of-the-box integrations with Microsoft and AWS.
#2 – ClickHouse
ClickHouse has gained recognition as a live data repository for big businesses. Deutsche Bank, eBay, Fastly, GitLab, HubSpot, Microsoft, ServiceNow, and Spotify are among the customers on its list. The company recently purchased PeerDB to improve change data capture and replication for Postgres, providing speed enhancements up to 10x, according to the company.
#3 – Timescale
Timescale has extended PostgreSQL for time-series data and analytics so developers can build faster, scale further, and stay under budget. Their team is made up of top researchers and industry veterans with experience building scalable systems at a variety of large and small companies, and they are backed by top-tier investors with a track record of success in the industry.
The company claims to have 1,000x faster queries than MongoDB and virtually infinite scaling made possible by de-coupling storage and compute (Neon has also de-coupled storage and compute). By leveraging the Timescale platform, engineers get back time they'd otherwise spend taking care of database tuning, continuous backups and failovers, operational tasks, and various compliance concerns.
#4 – MongoDB
MongoDB is likely the most well-known name on this list. They've recently extended the scope of their integrations between their cloud database development platform and Microsoft Azure—a move the company says will make it easier for partners and customers to build real-time data analytics links and develop generative AI applications.
MongoDB also recently announced they are integrating the MongoDB Atlas cloud database with Microsoft's Azure OpenAI services and launching its MongoDB Enterprise Advanced database management tools on the Azure Marketplace.
MongoDB has said the new integrations will provide partners and customers with greater flexibility in data development on Azure—particularly to help meet the exploding demand for data for AI and generative AI applications.
Conclusion
Each of these database technologies is a fantastic option, with varying flavors and emphasis. While some are more modern, lean on open source, and embrace serverless technologies, others have stood the test of time and are likely the preferable option for established enterprises. All in all, the rise of GenAI is going to make the way you store and distribute your data more important than ever—choose wisely :)
