Cloud computing and artificial intelligence (AI) are producing unprecedented opportunities and challenges at a breakneck speed. Allen Cao (a.k.a. Kun Cao), a tech lead and senior AI scientist at Amazon Web Services (AWS), knows this by heart. He offers valuable insights into how theoretical concepts are developed into real-world solutions.
Protecting Advancements in the Tech Industry
The tech industry's handling of intellectual property requires a delicate balance. While companies publish research papers to contribute to the scientific community, they also strategically file patents to safeguard their most valuable developments.
Allen Cao explains, "Big tech firms file patents defensively. Obviously, it is about shielding ourselves and our customers from potential legal challenges."
This has proven beneficial for the tech leader throughout his career. During his tenure at AWS, Allen Cao has filed four patents. AWS has commercialized these technologies and now widely utilizes them across its platform, serving millions of customers globally.
Allen Cao's inclination toward advancement and commercialization began during his doctoral research at Imperial College London. There, he developed a technique that used machine learning to enhance the aerodynamic efficiency of turbomachinery. This work resulted in two patents filed through Imperial College, showcasing Cao's early ability to use his academic research for practical applications.
"My experience at Imperial taught me the importance of not just developing new technologies but also considering their real-world impact and potential for commercialization," Cao reflects.
The Life Cycle of Cloud AI Projects
In his current role with AWS, the tech star has gained profound insights into the intricate process of bringing cloud AI projects from conception to deployment.
Allen Cao outlines a life cycle for these projects, which begins with analyzing customer pain points and designing tailored solutions. This process continues through research, development, implementation, and evaluation in cloud beta spaces.
"Once the technology proves viable, AWS files patents to protect the intellectual property. We conclude the cycle by deploying the technology in cloud production environments, and continuously monitoring and retraining AI models in an automated manner to guarantee optimal performance for customers," he shares.
Cao emphasizes the significance of this structured method: "Each stage plays an important role in guaranteeing that we are creating advanced technologies while delivering solutions that stay true to what our customers need."
AWS re:Post: A Showcase of Cao's Leadership
One of Cao's most notable contributions at AWS has been his role as the founding member and lead scientist & inventor of re:Post, AWS's intelligent knowledge hub. Under his guidance, the team has successfully deployed multiple major AI services that have served countless unique customers monthly since 2021.
According to Allen Cao, "re:Post represents how we tackle knowledge sharing and problem-solving in cloud AI. It creates an intelligent ecosystem that can adapt and grow with our customers' needs."
Building on this success, Cao and his team are now concentrating on developing new generative AI cloud services and more advanced solutions for global customers in the cloud industry.
Trends of Cloud AI Services
The industry leader has also observed massive changes in the cloud AI arena, particularly in light of the recent generative AI boom. Drawing from his extensive experience, Allen Cao shares two key observations.
The first one relates to cloud infrastructure. He notes a shift in services from decentralization to centralization. In the previous phase of natural language understanding, exemplified by models like Bidirectional Encoder Representations from Transformers (BERT), customers typically trained and deployed their own AI models on the cloud. This decentralized strategy was the norm.
However, generative AI and natural language generation have ushered in a new way. Customers commonly access a single, centralized large language model via application programming interface (API) calls.
Allen Cao draws a parallel between this and cloud computing itself. The astronomical costs associated with training, deploying, and running inferences on generative AI models make decentralization impractical and unaffordable for individual customers—the same driving factor when the industry switched from on-premise infrastructure to the cloud.
His second observation concerns the changing capabilities of generative AI models. Allen Cao shares that large language models are transitioning into large reasoning models, explaining that language serves as a vehicle for various patterns, including grammar, knowledge, logic, and reasoning.
Current large language models primarily focus on learning grammar and knowledge from historical data to provide accurate answers to given questions. However, researchers have identified a limitation: while these models can generate content based on learned knowledge, they cannot create new knowledge.
Consequently, the research focus is shifting toward training AI models to perform inference and reasoning, allowing for the creation of new knowledge.
Plans for the Future
Addressing these developments in cloud AI, Allen Cao is leading the efforts at AWS to research and develop next-generation products that use modern AI infrastructure and techniques.
He illustrates this with an example: previously, his team developed AI solutions to help customers find existing answers through search and recommendation. Now, they are creating cloud AI solutions that generate answers to solve customer problems directly.
"We aim to move from search and recommendation to true problem-solving AI," Allen Cao states.
True to form, Allen Cao has recently submitted two new patents for AI solutions that promise to advance cloud services further. As these technologies are commercialized, they will join the growing suite of AWS AI services, continuing to benefit customers worldwide.
In an industry characterized by a dizzying pace of change, Allen Cao's capability to consistently translate advanced research into practical, impactful solutions is invaluable. His work at AWS points toward a future where cloud AI becomes an increasingly intelligent part of everyone's daily lives.
