Building an end to end data strategy for analytics and generative AI | AWS Events

Introduction

The transformation and innovation offered by generative AI (Gen AI) has become a focal point for modern businesses. As organizations evolve, data processing, usage, and governance have become paramount. Here's a comprehensive guide on how Amazon Web Services (AWS) and WorkHuman have leveraged these principles to build efficient data strategies for analytics and generative AI applications.

Opening Remarks by Rick

Rick from AWS introduces the role of data in modern businesses. With advancements in technology and computational power, the significance of data has grown immensely. Rick emphasizes that data is at the center of business innovation, helping support and enhance customer experiences. He discusses how modern artificial intelligence and machine learning models can transform data into actionable insights and new experiences for customers.

Evolution of AI Models

Rick delves into the evolution of AI and ML models, discussing the shift from task-specific models to deep learning and foundational models. Foundational models, like those driving generative AI, benefit from massive datasets and can provide complex outputs. The current landscape showcases how data availability and computational advancements have enabled AI to be more integrated and transformative across various industries.

Data Availability and Connecting to Data Sources

Rick highlights the crucial theme of data availability. Businesses need to make as much data as possible accessible to feed into AI models. Running through numerous AWS tools, he explains how multiple data sources - like data lakes, warehouses, and other databases - can be connected, cataloged, and processed for AI applications. He also mentions the seamless integration features provided by AWS that reduce the complexities of ETL (Extract, Transform, Load) processes.

Data Quality and Responsible AI

Maintaining data quality is another critical realm Rick touches upon. Quality data is crucial for unbiased and accurate AI models. AWS offers solutions to ensure quality checks, such as AWS Glue and SageMaker. Rick also emphasizes the protection and governance of data, which spans from initial ingestion through to the model's evaluation and end-user interaction. This ensures security, compliance, and responsible use of AI.

WorkHuman's Journey with Data Strategy

Mark from WorkHuman recounts their journey of developing a data strategy. Starting with an on-premises setup, WorkHuman transitioned to utilizing AWS cloud infrastructure to manage and leverage vast amounts of data. Mark shares eight lessons learned in their transformation journey:

1. Tell a Story: Using storytelling to engage and inform stakeholders.
2. Take Action: Finding allies and starting initiatives without waiting for formal approval.
3. Demonstrate Value: Creating and showcasing useful data-driven projects.
4. Expand Person by Person: Engaging a community within the organization.
5. Get Executive Support: Securing funding and formal backing.
6. Bring in Experts: Leveraging external expertise.
7. Do Good Work and Publicize It: Sharing success stories to build momentum.
8. Continuous Improvement: Always anticipating new challenges and remaining adaptable.

Technical Overview by Kamal

Kamal outlines the technical architecture that supports WorkHuman's data strategy:

• Extraction Layer: Handling data ingestion from various sources using AWS tools.
• Curation Layer: Refining and cataloging data for analytics.
• Modeling Layer: Transforming data into business-centric models.
• Consumption Layer: Delivering data to end-users via API solutions and reporting tools like QuickSight.

Realizing Value: Key Solutions

Kamal discusses two significant customer-centric solutions:

1. Data API: Enables real-time dynamic data delivery via APIs.
2. Self-Service Reporting: Facilitates user-driven reporting and analytics using AWS QuickSight.

Closing Remarks

Rick concludes by reiterating the importance of a robust data strategy, embracing AWS services to build scalable, efficient, and governed systems to drive analytics and generative AI.

Keywords

• AWS
• Data Strategy
• Generative AI
• Machine Learning
• Data Governance
• Data Quality
• Data API
• Self-Service Reporting
• WorkHuman

FAQ

Q: What is the role of data in modern businesses according to AWS? A: Data is considered a central asset that drives innovation and provides personalized customer experiences.

Q: What are the key themes in building an AI strategy? A: The three key themes are data availability, data quality, and data protection.

Q: How does WorkHuman handle data ingestion and transformation? A: WorkHuman uses a layered architecture with AWS services like AWS Glue, AWS DMS, and Amazon Redshift to manage data ingestion, curation, modeling, and consumption.

Q: What are some lessons learned from WorkHuman's data strategy journey? A: Key lessons include the importance of storytelling, taking action, demonstrating value, securing executive support, bringing in experts, publicizing good work, and continuous improvement.

Q: What solutions has WorkHuman developed for their customers? A: WorkHuman has created solutions like a data API for real-time data access and self-service reporting capabilities via AWS QuickSight.