GenAI Workshop

Workshop Flow

Total Duration: 3 hours 30 minutes (9:00 - 12:30)

Topics Covered

We'll explore the essential concepts behind modern GenAI systems, including LLMs, embeddings, RAG, tokens, context windows, agents, and frameworks like MCP and LangChain.

LLM (Large Language Model)

Advanced AI systems trained on vast text datasets to understand and generate human-like text.

Embeddings

Numerical representations of text, images, or data in high-dimensional space for semantic search, similarity comparison, and efficient AI processing.

RAG (Retrieval Augmented Generation)

Technique combining information retrieval with text generation to produce accurate, factual responses by accessing external data sources.

Tokens

Fundamental text units in AI models (words, subwords, or characters) that determine model capacity and processing limitations.

Context Window

The maximum amount of text an AI model can process at once, affecting coherence and reference to previous information.

Agents

Intelligent systems that autonomously perform tasks or make decisions using tools, memory, and advanced reasoning to achieve specific goals.

MCP (Model Context Protocol)

A standardized protocol for AI models to access external tools and data sources, improving interoperability across different models and platforms.

LangChain

An open-source framework that simplifies building applications powered by large language models, providing standardized interfaces for chains, agents, memory, and tool integrations.

Comparing Traditional Data Engineering and GenAI

Understanding how traditional data engineering practices intersect with GenAI workflows is crucial for building robust, scalable AI systems.

Comparing traditional data engineering with GenAI workflows

Key Considerations

• Data pipelines for RAG systems
• Vector database management and optimization
• Embedding generation at scale
• Real-time vs batch processing for AI workloads
• Monitoring and observability for AI systems

Watch: Understanding Generative AI

Before we dive into the technical details, watch this introduction to understand the fundamentals of Generative AI and its transformative potential.

What is a Token?

The way text is tokenized directly impacts how the model interprets and generates language. Understanding tokens is crucial for optimizing AI interactions and managing costs.

Why Tokens Matter

• Model Capacity: Determine processing limits and context window size
• Cost Management: API calls are charged per token (input + output)
• Response Quality: Influence the quality and coherence of model responses

What are Embeddings?

Embeddings are a fundamental concept in deep learning. They transform words or other data into vectors, which are numerical representations that machines can understand and process. By converting all input data into vectors, deep learning models can analyze and learn from complex information efficiently.

Semantic Similarity

AI models represent words as vectors in a multi-dimensional space. Here, "tower" is mapped to a point, and the model finds the closest points—words with similar meanings like "towers," "gate," "building," and "skyscraper."

This allows AI to understand relationships and similarities between words, enabling more accurate language understanding.

Finding semantically similar words using vector distance

Vector Arithmetic

By calculating the distance between "man" and "woman," we can apply the same difference to "king" and discover "queen." This powerful property allows AI models to solve word analogies using vector arithmetic, enabling a deeper understanding of the relationships and structure within language.

Vector arithmetic: king - man + woman ≈ queen

What is a Context Window?

Input data passes through multiple layers of attention and processing (such as multilayer perceptrons) within the model. Each layer analyzes the relationships between different parts of the input, capturing context and meaning.

How context flows through multiple layers of the model

Why Context Window Matters

• Determines how much text the model can "remember" at once
• Affects the coherence of long conversations
• Influences the model's ability to reference previous information
• Larger windows enable more comprehensive understanding but increase computational cost

Token limits across different context window sizes

How RAG Works

1. Load various types of data sources
2. Split data into manageable chunks
3. Convert chunks into embeddings (vectors)
4. Store embeddings in a vector database
5. Retrieve relevant chunks based on user queries
6. Provide retrieved context to the LLM for enhanced responses

RAG pipeline: from data ingestion to response generation

Framework for Building LLM Applications

LangChain enables developers to create sophisticated applications by connecting LLMs with external data sources, APIs, and tools. It abstracts away the complexity of managing prompts, memory, and data retrieval, allowing you to focus on building innovative solutions.

Core Components

• Models: Integration with various LLM providers (OpenAI, Anthropic, HuggingFace, etc.)
• Prompts: Templates and prompt engineering utilities for consistent interactions
• Chains: Sequences of calls to LLMs or other utilities for complex workflows
• Agents: Systems that use LLMs to decide which actions to take and in what order
• Memory: Mechanisms to persist state between chain or agent calls
• Indexes: Ways to structure documents for optimal LLM interaction
• Retrievers: Interfaces for fetching relevant documents for a given query

LangChain Architecture - How It Works

Watch how data flows through LangChain components, from user input to final output.

Detailed LangChain Execution Flow

Deep Dive: Component Details

Click through each tab to learn more about individual LangChain components.

# Using different LLM providers with the same interface
from langchain.llms import OpenAI
from langchain.chat_models import ChatAnthropic

# OpenAI model
openai_llm = OpenAI(model="gpt-3.5-turbo", temperature=0.7)

# Anthropic model - same interface!
claude_llm = ChatAnthropic(model="claude-3-sonnet")

# Both work the same way
response = openai_llm.invoke("Explain quantum computing")

from langchain.prompts import PromptTemplate, ChatPromptTemplate

# Simple template with variables
template = PromptTemplate(
    input_variables=["product", "language"],
    template="""Write a compelling product description for {product}.
The description should be in {language} and highlight key benefits."""
)

# Fill in the variables
prompt = template.format(
    product="wireless headphones",
    language="English"
)

# Chat-style templates
chat_template = ChatPromptTemplate.from_messages([
    ("system", "You are a helpful assistant specialized in {domain}."),
    ("human", "{question}")
])

from langchain.chains import LLMChain
from langchain.prompts import PromptTemplate
from langchain.llms import OpenAI

# Create a simple chain
llm = OpenAI(temperature=0.7)
prompt = PromptTemplate(
    input_variables=["topic"],
    template="Write 3 creative ideas about {topic}"
)

# Combine into a chain
chain = LLMChain(llm=llm, prompt=prompt)

# Run the chain
result = chain.run(topic="sustainable energy")

from langchain.memory import ConversationBufferMemory
from langchain.chains import ConversationChain

# Create memory to store conversation history
memory = ConversationBufferMemory()

# Create a conversation chain with memory
conversation = ConversationChain(
    llm=llm,
    memory=memory,
    verbose=True
)

# The chain remembers previous messages!
conversation.run("Hi, my name is Alice")
conversation.run("What's my name?")  # It remembers: "Alice"

from langchain.agents import initialize_agent, Tool
from langchain.tools import DuckDuckGoSearchRun

# Define tools the agent can use
search = DuckDuckGoSearchRun()
tools = [
    Tool(name="Search", func=search.run,
         description="Search the web for current information")
]

# Create an agent that decides which tools to use
agent = initialize_agent(
    tools=tools,
    llm=llm,
    agent="zero-shot-react-description",
    verbose=True
)

# The agent autonomously decides to use Search
agent.run("What's the weather in New York today?")

Key Use Cases

• Question Answering: Build systems that answer questions over your documents using RAG
• Chatbots: Create conversational agents with memory and context awareness
• Data Analysis: Enable LLMs to interact with databases and perform data operations
• Summarization: Process and summarize large volumes of text efficiently
• Code Generation: Generate and execute code based on natural language instructions
• Multi-step Reasoning: Chain multiple LLM calls for complex problem-solving

What is LangChain?

LangChain is an open-source framework designed to simplify the development of applications powered by large language models (LLMs). It provides a standard interface for chains, lots of integrations with other tools, and end-to-end chains for common applications.

• Data Awareness: Connect a language model to other sources of data
• Agentic: Allow a language model to interact with its environment

Why Use LangChain?

LangChain accelerates development by providing pre-built components and patterns for common LLM application scenarios. It offers:

• 🔌 Modularity: Mix and match components to build custom solutions
• 🔄 Flexibility: Switch between different LLM providers without code changes
• 📚 Rich Ecosystem: Extensive library of integrations and tools
• 🛠️ Production-Ready: Built-in error handling, logging, and monitoring
• 🚀 Rapid Prototyping: Quickly test and iterate on AI application ideas

What is LangGraph?

LangGraph is a library for building stateful, multi-actor applications with LLMs. It extends LangChain to support cyclic flows, allowing for more complex agentic behaviors.

Conceptual View: How the pieces fit together

Switch View 🔄

Core Concepts

• Nodes: Functions or agents that process data and make decisions
• Edges: Connections between nodes (can be conditional)
• State: Shared memory that persists across the graph execution
• Checkpoints: Save and resume graph execution at any point

Cyclic Supervisor Pattern

The most powerful pattern in LangGraph is the Cyclic Supervisor, where a supervisor agent orchestrates multiple worker agents in a loop until the task is complete.

Why Use LangGraph?

• State Management: Built-in persistence for complex, multi-turn conversations
• Cycles & Loops: Support for iterative refinement and self-correction
• Conditional Logic: Route between different agents based on context
• Human-in-the-Loop: Checkpoint system for approval workflows
• Scalability: Coordinate multiple specialized agents efficiently

Key Use Cases

LangChain vs. LangGraph

While LangChain excels at building linear, directed acyclic graphs (DAGs) for straightforward applications, LangGraph introduces the ability to build stateful, cyclic graphs which are essential for complex agentic workflows.

Comparative architecture: Linear Chains vs. Cyclic Graphs

What are Guardrails?

Guardrails are safety mechanisms that control and validate AI model inputs and outputs, ensuring responses are safe, accurate, and aligned with business requirements.

Types of Guardrails

• Input Guardrails: Filter/validate user prompts before processing
• Output Guardrails: Validate and sanitize model responses
• Content Moderation: Block harmful, toxic, or inappropriate content
• PII Detection: Identify and mask personal information
• Topic Restriction: Keep responses within allowed domains

Implementation Best Practices

What are Hallucinations?

Hallucinations occur when LLMs generate plausible but factually incorrect information. This is one of the biggest challenges in production AI systems.

Why Hallucinations Happen

• Training Data Gaps: Model lacks information on specific topics
• Over-confidence: Model generates text even when uncertain
• Pattern Matching: Predicts likely tokens without factual verification
• Context Confusion: Misinterprets or conflates information
• Knowledge Cutoff: No access to recent information

Mitigation Strategies

Detection Techniques

• Citation Verification: Check if sources actually support claims
• Confidence Scoring: Monitor model uncertainty signals
• Cross-Reference: Validate against multiple knowledge sources
• Human Review: Critical outputs reviewed by domain experts

What is MCP?

MCP acts as a bridge between AI models and the external world, allowing them to access real-time information, execute actions, and integrate with existing enterprise systems without custom integrations for each tool or data source.

Key Features

• Standardized Interface: Single protocol for connecting to multiple data sources and tools
• Security First: Built-in authentication and permission controls for safe data access
• Vendor Agnostic: Works across different LLM providers and platforms
• Bidirectional Communication: Models can both retrieve data and execute actions
• Context Management: Efficiently manages and maintains context across interactions
• Extensible: Easy to add new tools and data sources without modifying core code

Why Use MCP?

MCP solves a critical challenge in AI development: the fragmentation of tool integrations. Instead of building custom connectors for each combination of AI model and data source, MCP provides a unified approach.

• 🔌 Plug-and-Play Integration: Connect new tools and data sources with minimal code
• 🔒 Enterprise-Grade Security: Controlled access to sensitive data and systems
• ⚡ Reduced Development Time: No need to build custom integrations for each tool
• 🔄 Interoperability: Switch between different AI models without rewriting integrations
• 📈 Scalability: Easily scale to hundreds of tools and data sources
• 🛡️ Reliability: Standardized error handling and connection management

MCP Architecture Overview

The Model Context Protocol follows a client-server architecture where an MCP host (an AI application like Claude Desktop) establishes connections to one or more MCP servers. The MCP host creates one MCP client for each MCP server, maintaining dedicated one-to-one connections.

MCP architecture showing the relationship between hosts, clients, and servers

Key Participants

• MCP Host: The AI application that coordinates and manages multiple MCP clients
• MCP Client: Maintains a connection to an MCP server and obtains context for the host
• MCP Server: A program that provides context to MCP clients

Before and After MCP

MCP eliminates the need for custom integrations between each AI model and data source. Instead of building separate connectors for every combination, MCP provides a standardized protocol that works universally across different systems.

Comparison showing how MCP simplifies integrations: Before (left) - multiple custom connections required; After (right) - single standardized protocol for all connections

MCP Scope: The protocol focuses solely on context exchange between AI applications and data sources. It includes the MCP specification, SDKs for different programming languages, development tools like the MCP Inspector, and reference server implementations.

Common Use Cases

• Database Access: Query SQL databases, NoSQL stores, and data warehouses
• API Integration: Connect to REST APIs, GraphQL endpoints, and web services
• File Systems: Read and write files from local or cloud storage
• Business Tools: Integrate with CRM, ERP, and productivity platforms
• Real-time Data: Access live data feeds, monitoring systems, and IoT devices
• Code Execution: Run scripts and interact with development environments

Evolution Beyond LLMs

Agents combine multiple components, such as access to external tools, short-term and long-term memory, document processing, and even multi-modal capabilities (combining text, images, and other data types).

Agent Capabilities

• Access to external tools (APIs, databases, company systems)
• Short-term and long-term memory management
• Document processing and knowledge retrieval
• Multi-modal capabilities (text, images, structured data)
• Complex reasoning and decision-making
• Autonomous task execution and planning

GenAI agents orchestrating tools, memory, and execution flow

📚 Courses - AI Agents and Agentic Architecture

Practical courses from DeepLearning.AI on building, optimizing, and coordinating intelligent agents.

📄 Articles & Reports - GenAI Strategy and Leadership

Industry insights from leading consulting firms on GenAI strategy, ROI, and organizational transformation.

📖 Transformers & Attention Mechanisms - Deep Dive

Essential resources to understand the technology behind modern LLMs.

Test Your Knowledge

Ready to test what you've learned? Take the full interactive quiz to assess your understanding of GenAI concepts.

🚀 Practical GenAI Implementation

Put your knowledge into practice with our comprehensive hands-on workshops. We have divided the practical sessions into three focused tracks to cater to different learning goals.

🤖 Workshop 1: ChatBot Basics (Files 1-5)

Start here to understand the fundamental building blocks of Generative AI. You will work with individual Python scripts to learn how chatbots, tools, and RAG systems work under the hood.

✨ Workshop 2: Production App (Full Agent + Streamlit)

Ready for the next level? Deploy a complete, production-ready AI assistant with a polished user interface. This workshop focuses on application architecture, dependency management, and user experience.