AI Sports Recap: Long Video Summarizer & Highlights Generator

Awarded 2nd place at the Multimodal AI in Media & Entertainment Hackathon organized by Twelve Labs, FBRC.ai, and AWS, AISR is an AI-powered application that generates context-specific video highlights and textual summaries of sports press conferences based on user prompts. The project is featured in TwelveLabs' Official Documentation here.

How it works:

• User-Prompted Video Analysis:
  Users provide a video URL (e.g., YouTube) and ask questions.
• Smart Extraction:
  The system identifies and stitches relevant video clips into a highlights reel.
• AI-Powered Summarization:
  Generates concise text summaries alongside the video.
• Follow-Up Prompts:
  Users can refine results for deeper insights.

Key Features:

• Player/Coach Interviews: Extracts and summarizes crucial insights.
• Social Media Highlights: Creates shareable bite-sized content.

Mechanism:

• Transcript & Timestamp Extraction: Utilizes Twelve Labs' API to index videos.
• Context-Aware Processing: Merges short utterances for better comprehension.
• AI-Powered Summarization: GPT-4o filters relevant content and generates summaries.
• Automated Video Editing: Clips key segments and compiles highlights.
• User-Friendly Interface: Built with Streamlit for an interactive chat experience.

Challenges & Innovations:

• Latency Optimization: Pre-indexing videos to improve processing speed.
• Reducing AI Hallucinations: Fine-tuned prompts for accuracy.
• Scalability & Deployment: Dockerized for seamless use.