Azure SDK Community Standup: Building intelligent web apps with the AI Chat Protocol library

Introduction

Welcome to another episode of the Azure SDK Community standup! I'm Hector Norzagaray, a product manager for the Azure SDK team, and I'll be your host for today. It's fantastic to have you here as we chat about some of the cool new releases we've been working on over the past few months. We've been super busy adding new features and making improvements, and we can't wait to share what we've been up to. We think you'll love these updates and find them really helpful for building better, faster, and more scalable apps.

Today, our focus is on one of those releases: the AI Chat Protocol Library. To help us dive into this exciting topic, we have a special guest, Rohith Ganguli, who is a product manager for AI Chat Protocol for the Azure SDK team. He will demonstrate how to use the library to create smart web apps that can significantly enhance user experiences.

Introducing the Azure SDK PM Team

Before diving into the main topic, we always like to introduce the Azure SDK PM team. We have dedicated product managers working on various services and languages for which we have SDKs, such as Python, JavaScript, Typescript, Java, and .NET. Please feel free to contact these individuals through social media if you have any issues or feedback about the products.

What’s New in the Azure SDK

Visit the Azure SDK blog every month to check out our latest releases. We feature detailed change logs, demos, and documentation that are immensely helpful. Recent highlights from May and June include updates for languages like JavaScript, Java, .NET, and Go, with services like Dev Center Application Insights, Front Door support, and Text Translation.

AI Chat Protocol Library Overview

AI Terminology

Understanding foundational AI terminology is crucial for leveraging the AI Chat Protocol effectively. Let's cover the basics:

Embeddings: Representations of semantic meaning in vector form.
Inference: Calling the trained model to predict or generate output.
Completions: The actual output generated by the model based on a given prompt or instruction.

Simple AI Scenario

A basic AI scenario involves a client device running a web application that uses an inference SDK to interact with an AI service like Azure OpenAI. This basic example is useful for understanding how to get started but is not recommended for production use due to security concerns, such as exposing API keys.

Making AI More Intelligent

We can make AI applications more intelligent by:

Model Improvements: Updating to newer, more advanced models like GPT-4.
Access to Data: Providing AI with access to specific datasets or documents through vector stores like Azure AI Search.
AI Orchestration Frameworks: Using frameworks like Semantic Kernel or LangChain to manage various aspects of AI behavior and responses.

Realistic AI Scenario

A more realistic AI setup separates the client and backend logic. The client makes requests to an API endpoint, which then handles AI processing, implementing business logic, and security features. Using the AI Chat Protocol, we can stream AI responses to the client effectively.

AI Chat Protocol: API Spec and Library

API Spec

The AI Chat Protocol defines a standard interface for AI endpoints, ensuring interoperability regardless of backend implementation. Responses can be either non-streaming or streaming, allowing flexibility based on application needs.

Library Usage

Installing and using the AI Chat Protocol Library involves basic steps:

Install via npm (npm install @microsoft/ai-chat).
Create a client object.
Use methods like getCompletion or getStreamedCompletion to handle AI responses, which come with full logging, tracing, and authentication features.

Demo: Serverless AI Chat with Rag using LangChain JS

Architecture

The demo architecture includes:

Static Web App: Makes requests to a serverless API.
Azure Functions: Handle requests and AI processing logic.
AI Services: Utilize Azure OpenAI Service, Azure AI Search, and Azure Blob Storage for data storage and retrieval.

Running the Demo

Using the Azure Developer CLI, you can deploy the setup with azd up, which provisions resources and deploys the application. The demo showcases how to stream AI responses to a frontend, enhancing the user experience significantly.

Future Plans

Future updates to the AI Chat Protocol Library include:

Support for file transfers.
Additional samples in Python, Java, and .NET.
Frontend components for easy integration, such as React chat components.

Conclusion

We encourage you to explore the AI Chat Protocol Library, contribute to the repository, and share how you are using this technology in your projects. Your feedback helps us improve and innovate!

Keywords

AI Chat Protocol
Azure SDK
Embeddings
Inference
Completions
Model Improvements
Vector Stores
AI Orchestration Frameworks
Streaming Responses
Semantic Kernel
LangChain
AI Applications

FAQ

What is the AI Chat Protocol Library?

The AI Chat Protocol Library is a JavaScript library designed to facilitate intelligent web apps by streaming AI-generated responses. It works seamlessly with multiple AI backends and frameworks.

What are embeddings in the context of AI?

Embeddings are vector representations of semantic meaning, allowing AI models to understand relationships between words in a form that computers can process.

How can I make my AI applications more intelligent?

You can make AI applications more intelligent by improving models, providing access to specialized data, and using AI orchestration frameworks like Semantic Kernel or LangChain.

What are the main benefits of using the AI Chat Protocol Library?

The AI Chat Protocol Library simplifies streaming AI responses to the frontend, offers a standard interface for AI backends, and comes integrated with Azure SDK features like logging, tracing, and authentication.

How do I get started with the AI Chat Protocol?

You can start by visiting the GitHub repository at aka.ms/ai-chat to access the library and related resources. Follow the provided samples and documentation to integrate it into your projects.