Get in Touch

Course Outline

Introduction to Multimodal LLMs in Vertex AI

  • Comprehensive overview of multimodal capabilities within Vertex AI.
  • Deep dive into Gemini models and their supported modalities.
  • Exploration of relevant use cases in enterprise environments and research.

Setting Up the Development Environment

  • Configuring Vertex AI to support multimodal workflows.
  • Managing and manipulating datasets across different modalities.
  • Hands-on lab: Environment configuration and dataset preparation.

Long Context Windows and Advanced Reasoning

  • Understanding the mechanics of long-context workflows.
  • Analyzing applications in strategic planning and decision-making processes.
  • Hands-on lab: Implementing long-context analysis techniques.

Cross-Modal Workflow Design

  • Integrating text, audio, and image analysis components.
  • Chaining multimodal steps to create cohesive pipelines.
  • Hands-on lab: Designing an end-to-end multimodal pipeline.

Working with Gemini API Parameters

  • Configuring inputs and outputs for multimodal interactions.
  • Strategies for optimizing inference speed and computational efficiency.
  • Hands-on lab: Fine-tuning Gemini API parameters.

Advanced Applications and Integrations

  • Developing interactive multimodal agents and virtual assistants.
  • Integrating external APIs and auxiliary tools.
  • Hands-on lab: Constructing a fully functional multimodal application.

Evaluation and Iteration

  • Methods for testing and validating multimodal performance.
  • Key metrics for accuracy, alignment, and detecting data drift.
  • Hands-on lab: Conducting comprehensive evaluations of multimodal workflows.

Summary and Next Steps

Requirements

  • Strong proficiency in Python programming.
  • Practical experience in developing machine learning models.
  • Working knowledge of multimodal data types, including text, audio, and images.

Target Audience

  • AI researchers
  • Senior developers
  • Machine learning scientists
 14 Hours

Number of participants


Price per participant

Upcoming Courses

Related Categories