How Youtube generate video previews?

The code is available on my github repository. Feel free to fork/clone it and play around with it.
While working on a project where I had to improve the scrubbing experience of video previews, I wondered how youtube does it so well. And I went down that rabbit hole. This article we will discuss how Youtube generates the video previews and how does is it make them so performant.
What is a video preview?
In short, video preview is a series of frames taken from a video which then can be used to show preview of video at a certain timeframe of the video. It allows users to see beforehand what's there in the video without actually watching it. Its a genius feature that improved viewing experience for users on video streaming platforms like YouTube, Vimeo, Netflix etc.
Before we jump into how Youtube does it, we need to understand some basics of it first.
How to generate preview from a video
We will be using golang for all the server side code in this article.
Let's first understand the most basic way to generate a preview from a video.
We will be generating a thumbnail preview image for the following video
We need to install ffmpeg
and ffprobe
to generate the preview. ffmpeg
is a tool that can be used to extract frames from a video and ffprobe
is a tool that can be used to get the duration of a video.
brew install ffmpeg
brew install ffprobe
Now we can use the following code to generate the preview.
package main
import (
"bytes"
"fmt"
"image"
"image/draw"
"image/jpeg"
"os"
"os/exec"
"strconv"
)
// generatePreviewStripWithInterval creates a preview strip by taking frames every N seconds
func generatePreviewStripWithInterval(videoPath string, outputPath string, intervalSeconds float64) error {
if intervalSeconds <= 0 {
return fmt.Errorf("interval must be greater than 0 seconds")
}
// Check if input file exists
if _, err := os.Stat(videoPath); os.IsNotExist(err) {
return fmt.Errorf("input video file does not exist: %s", videoPath)
}
// First get video duration using ffprobe
cmd := exec.Command("ffprobe",
"-v", "error",
"-show_entries", "format=duration",
"-of", "default=noprint_wrappers=1:nokey=1",
videoPath)
durationBytes, err := cmd.Output()
if err != nil {
return fmt.Errorf("error getting video duration: %v", err)
}
duration, err := strconv.ParseFloat(string(durationBytes[:len(durationBytes)-1]), 64) // Remove trailing newline
if err != nil {
return fmt.Errorf("error parsing duration: %v", err)
}
if duration <= 0 {
return fmt.Errorf("invalid video duration: %f", duration)
}
// Calculate how many frames we'll extract
frameCount := int(duration/intervalSeconds) + 1
if frameCount <= 0 {
frameCount = 1
}
fmt.Printf("Video duration: %.2f seconds, taking frames every %.2f seconds (approximately %d frames)\n",
duration, intervalSeconds, frameCount)
// Extract frames at the specified interval
frames := []image.Image{}
for timestamp := 0.0; timestamp < duration; timestamp += intervalSeconds {
cmd = exec.Command("ffmpeg",
"-v", "error", // Suppress verbose output
"-ss", fmt.Sprintf("%.2f", timestamp),
"-i", videoPath,
"-vframes", "1",
"-f", "image2pipe",
"-vcodec", "mjpeg", // Use mjpeg instead of jpeg for better compatibility
"-q:v", "2", // High quality
"-")
output, err := cmd.Output()
if err != nil {
// Get stderr for better error info
if exitError, ok := err.(*exec.ExitError); ok {
return fmt.Errorf("error extracting frame at %.2f seconds: %v, stderr: %s", timestamp, err, string(exitError.Stderr))
}
return fmt.Errorf("error extracting frame at %.2f seconds: %v", timestamp, err)
}
frame, err := jpeg.Decode(bytes.NewReader(output))
if err != nil {
return fmt.Errorf("error decoding frame at %.2f seconds: %v", timestamp, err)
}
frames = append(frames, frame)
}
// Create a new image to hold all frames horizontally
if len(frames) == 0 {
return fmt.Errorf("no frames extracted")
}
frameWidth := frames[0].Bounds().Dx()
frameHeight := frames[0].Bounds().Dy()
stripWidth := frameWidth * len(frames)
strip := image.NewRGBA(image.Rect(0, 0, stripWidth, frameHeight))
// Place each frame side by side
for i, frame := range frames {
draw.Draw(strip,
image.Rect(i*frameWidth, 0, (i+1)*frameWidth, frameHeight),
frame,
image.Point{0, 0},
draw.Src)
}
// Save the final image
outFile, err := os.Create(outputPath)
if err != nil {
return fmt.Errorf("error creating output file: %v", err)
}
defer outFile.Close()
if err := jpeg.Encode(outFile, strip, &jpeg.Options{Quality: 90}); err != nil {
return fmt.Errorf("error encoding final image: %v", err)
}
fmt.Printf("Successfully extracted %d frames\n", len(frames))
return nil
}
func main() {
// Check if input file exists, provide helpful message if not
inputFile := "input.mp4"
if _, err := os.Stat(inputFile); os.IsNotExist(err) {
fmt.Printf("Input file '%s' does not exist.\n", inputFile)
fmt.Printf("Please place a video file named 'input.mp4' in the current directory, or modify the code to use a different path.\n")
fmt.Printf("Usage: Place your MP4 video file as 'input.mp4' and run this program.\n")
os.Exit(1)
}
// Generate preview strip taking a frame every 5 seconds
intervalSeconds := 5.0
outputFile := "preview-strip.jpg"
fmt.Printf("Generating preview strip from '%s', taking frames every %.1f seconds...\n", inputFile, intervalSeconds)
err := generatePreviewStripWithInterval(inputFile, outputFile, intervalSeconds)
if err != nil {
fmt.Printf("Error generating preview strip: %v\n", err)
os.Exit(1)
}
fmt.Printf("Successfully generated preview strip: %s\n", outputFile)
}
To run the code -
go run main.go
Result of the code
After running the code, you will see a file named preview-strip.jpg
in the current directory. For the video above, the preview strip looks like this -

And the final video progress bar with preview thumbnails would look something like this -
Understanding
ffmpeg
andffprobe
is not in the scope of this article. Although not needed for this article, but feel free to learn more about them online
Here is what the code above is doing -
- It uses
ffprobe
to get the duration of the video. - It uses
ffmpeg
to extract frames from the video at a given interval. The interval can be passed as an argument to thegeneratePreviewStripWithInterval
function. - It combines all the collected frames into a single image.
- It then finally saves the image to a file.
This code uses ffmpeg
and ffprobe
to extract frames from a video and combine them into a strip. It's a simple way to generate a preview strip, but it's not very efficient.
There are few problems with this approach -
- In case if video is very long, the preview thumbnail strip will also be very large. Creating such large images will be slow and will also consume a lot of memory.
- Only one preview thumbnail is generated with a particular frame interval. This may not give the best user experience when -
- User is hovering through the progress bar slowly and the frame interval is large. In this case user would see the same frame for longer span
- User is hovering through the progress bar fast and frame interval is small - with small frame interval, a large preview image with a large number of thumbnails would be generated. In this case we don't need a preview thumbnail strip with a lot of thumbnails, an image with a higher frame interval should be used.
How Youtube does it?
Before we jump into how Youtube does it - I should mention that Youtube also transcodes videos, which means it converts videos from one format, resolution, bitrate, or codec into other formats for optimized and adaptive streaming. Youtube automatically changes the resolution of the video when your internet bandwidth fluctuates to keep the streaming experience optimized. Youtube also generates thumbnail preview images for each segment of video for each resolution. That's a lot of work!! For the sake of this article, I'll be doing this for only one resolution of video.
Please feel free to learn more about transcoding on the internet. Its a vast topic in itself and way out of scope of this article.
Alright let's get to it!!
After a video is uploaded, following steps are taken -
- Video is transcoded into different resolutions
- Each resolution is then split into multiple segments.
- Each segment is then used to generate a thumbnail preview image.
We'll be skipping the transcoding part and will just deal with the original resolution of the video.
Segmenting the video is the process of dividing the video into smaller parts. This is done to make the video easier to process and to make the video preview generation faster. Segmentation is also helpful for streaming the video in smaller chunks.
In the below code, we are segmenting the video into 30 second segments using ffmpeg
and also generating the manifest file for the segments. The segments are saved in the
hls_segments
directory with name segment_[segment_index].ts
(.ts extension not to be confused with typescript). After the segments are generated, we are generating
the thumbnail preview images for each segment. The thumbnail preview images are saved in the thumbnails
directory with name segment-[segment_index]-thumbnails.jpg
.
package main
import (
"bufio"
"bytes"
"fmt"
"image"
"image/draw"
"image/jpeg"
"os"
"os/exec"
"path/filepath"
"regexp"
"strconv"
"strings"
)
// SegmentInfo holds information about an HLS video segment
type SegmentInfo struct {
Index int
Filename string
Duration float64
StartTime float64
EndTime float64
}
// HLSManifest represents the parsed m3u8 file
type HLSManifest struct {
TargetDuration float64
TotalDuration float64
Segments []SegmentInfo
}
// generateHLSSegments creates HLS segments using ffmpeg
func generateHLSSegments(videoPath string, segmentDuration int, outputDir string) (*HLSManifest, error) {
// Create output directory
if err := os.MkdirAll(outputDir, 0755); err != nil {
return nil, fmt.Errorf("error creating output directory: %v", err)
}
playlistPath := filepath.Join(outputDir, "playlist.m3u8")
segmentPattern := filepath.Join(outputDir, "segment_%03d.ts")
fmt.Printf("🎬 Creating HLS segments with ffmpeg...\n")
fmt.Printf(" Segment duration: %d seconds\n", segmentDuration)
fmt.Printf(" Output directory: %s\n", outputDir)
// Use ffmpeg to create HLS segments
cmd := exec.Command("ffmpeg",
"-i", videoPath,
"-c:v", "libx264", // Video codec
"-c:a", "aac", // Audio codec
"-f", "hls", // HLS format
"-hls_time", fmt.Sprintf("%d", segmentDuration), // Segment duration
"-hls_list_size", "0", // Keep all segments in playlist
"-hls_segment_filename", segmentPattern, // Segment filename pattern
"-y", // Overwrite output files
playlistPath)
fmt.Printf("Running: %s\n", strings.Join(cmd.Args, " "))
output, err := cmd.CombinedOutput()
if err != nil {
return nil, fmt.Errorf("error creating HLS segments: %v\nOutput: %s", err, string(output))
}
fmt.Printf("✅ HLS segmentation complete\n")
// Parse the generated m3u8 file
manifest, err := parseM3U8(playlistPath)
if err != nil {
return nil, fmt.Errorf("error parsing m3u8 file: %v", err)
}
// Update segment paths to be relative to output directory
for i := range manifest.Segments {
manifest.Segments[i].Filename = filepath.Join(outputDir, manifest.Segments[i].Filename)
}
fmt.Printf("📄 Parsed manifest: %d segments, total duration: %.2fs\n",
len(manifest.Segments), manifest.TotalDuration)
return manifest, nil
}
// parseM3U8 parses an m3u8 playlist file
func parseM3U8(playlistPath string) (*HLSManifest, error) {
file, err := os.Open(playlistPath)
if err != nil {
return nil, fmt.Errorf("error opening playlist file: %v", err)
}
defer file.Close()
manifest := &HLSManifest{
Segments: []SegmentInfo{},
}
scanner := bufio.NewScanner(file)
var currentDuration float64
var currentStartTime float64
segmentIndex := 0
// Regex to extract duration from #EXTINF line
extinfoRegex := regexp.MustCompile(`#EXTINF:([\d.]+),`)
for scanner.Scan() {
line := strings.TrimSpace(scanner.Text())
if strings.HasPrefix(line, "#EXT-X-TARGETDURATION:") {
durationStr := strings.TrimPrefix(line, "#EXT-X-TARGETDURATION:")
manifest.TargetDuration, _ = strconv.ParseFloat(durationStr, 64)
} else if strings.HasPrefix(line, "#EXTINF:") {
// Extract duration from EXTINF line
matches := extinfoRegex.FindStringSubmatch(line)
if len(matches) > 1 {
currentDuration, _ = strconv.ParseFloat(matches[1], 64)
}
} else if strings.HasSuffix(line, ".ts") && !strings.HasPrefix(line, "#") {
// This is a segment file
segment := SegmentInfo{
Index: segmentIndex,
Filename: line,
Duration: currentDuration,
StartTime: currentStartTime,
EndTime: currentStartTime + currentDuration,
}
manifest.Segments = append(manifest.Segments, segment)
manifest.TotalDuration += currentDuration
currentStartTime += currentDuration
segmentIndex++
}
}
if err := scanner.Err(); err != nil {
return nil, fmt.Errorf("error reading playlist file: %v", err)
}
return manifest, nil
}
// generateThumbnailStripFromSegment creates a thumbnail strip from an HLS segment
func generateThumbnailStripFromSegment(segmentPath string, segment SegmentInfo, outputPath string, frameInterval float64) error {
if frameInterval <= 0 {
return fmt.Errorf("frame interval must be greater than 0 seconds")
}
// Check if segment file exists
if _, err := os.Stat(segmentPath); os.IsNotExist(err) {
return fmt.Errorf("segment file does not exist: %s", segmentPath)
}
// Calculate how many frames we'll extract from this segment
frameCount := int(segment.Duration/frameInterval) + 1
if frameCount <= 0 {
frameCount = 1
}
fmt.Printf("Processing segment %d (%s): %.2fs duration, ~%d frames every %.2fs\n",
segment.Index, filepath.Base(segmentPath), segment.Duration, frameCount, frameInterval)
// Extract frames at the specified interval from the segment
frames := []image.Image{}
maxTimestamp := segment.Duration - 1.0 // Leave 1 second buffer at the end
for timestamp := 0.0; timestamp < maxTimestamp; timestamp += frameInterval {
cmd := exec.Command("ffmpeg",
"-v", "error", // Suppress verbose output
"-ss", fmt.Sprintf("%.2f", timestamp),
"-i", segmentPath,
"-vframes", "1",
"-f", "image2pipe",
"-pix_fmt", "yuvj420p", // Force compatible pixel format
"-vcodec", "mjpeg", // Use mjpeg for better compatibility
"-q:v", "3", // Good quality but more forgiving
"-an", // No audio
"-")
output, err := cmd.Output()
if err != nil {
// Log the error but continue processing - some frames might fail near segment boundaries
fmt.Printf(" Warning: Could not extract frame at %.2fs from segment %d, continuing...\n",
timestamp, segment.Index)
continue
}
frame, err := jpeg.Decode(bytes.NewReader(output))
if err != nil {
fmt.Printf(" Warning: Could not decode frame at %.2fs from segment %d: %v\n",
timestamp, segment.Index, err)
continue
}
frames = append(frames, frame)
}
if len(frames) == 0 {
return fmt.Errorf("no frames extracted from segment %d", segment.Index)
}
fmt.Printf(" Successfully extracted %d frames from segment %d\n", len(frames), segment.Index)
// Create a new image to hold all frames horizontally
frameWidth := frames[0].Bounds().Dx()
frameHeight := frames[0].Bounds().Dy()
stripWidth := frameWidth * len(frames)
strip := image.NewRGBA(image.Rect(0, 0, stripWidth, frameHeight))
// Place each frame side by side
for i, frame := range frames {
draw.Draw(strip,
image.Rect(i*frameWidth, 0, (i+1)*frameWidth, frameHeight),
frame,
image.Point{0, 0},
draw.Src)
}
// Save the segment thumbnail strip
outFile, err := os.Create(outputPath)
if err != nil {
return fmt.Errorf("error creating output file: %v", err)
}
defer outFile.Close()
if err := jpeg.Encode(outFile, strip, &jpeg.Options{Quality: 90}); err != nil {
return fmt.Errorf("error encoding final image: %v", err)
}
fmt.Printf("✓ Generated thumbnail strip for segment %d: %s (%d frames)\n",
segment.Index, outputPath, len(frames))
return nil
}
// generateHLSBasedPreviewStrips creates thumbnail strips from HLS segments
func generateHLSBasedPreviewStrips(videoPath string, segmentDuration int, frameInterval float64) error {
// Check if input file exists
if _, err := os.Stat(videoPath); os.IsNotExist(err) {
return fmt.Errorf("input video file does not exist: %s", videoPath)
}
// Create HLS segments
hlsOutputDir := "hls_segments"
manifest, err := generateHLSSegments(videoPath, segmentDuration, hlsOutputDir)
if err != nil {
return fmt.Errorf("error generating HLS segments: %v", err)
}
// Create output directory for thumbnails
thumbnailsDir := "thumbnails"
if err := os.MkdirAll(thumbnailsDir, 0755); err != nil {
return fmt.Errorf("error creating thumbnails directory: %v", err)
}
// Generate thumbnail strips for each segment
for _, segment := range manifest.Segments {
thumbnailPath := fmt.Sprintf("%s/segment-%d-thumbnails.jpg", thumbnailsDir, segment.Index)
err := generateThumbnailStripFromSegment(segment.Filename, segment, thumbnailPath, frameInterval)
if err != nil {
return fmt.Errorf("error generating thumbnails for segment %d: %v", segment.Index, err)
}
}
// Create a metadata file with segment information
metadataPath := fmt.Sprintf("%s/segments.txt", thumbnailsDir)
metadataFile, err := os.Create(metadataPath)
if err != nil {
return fmt.Errorf("error creating metadata file: %v", err)
}
defer metadataFile.Close()
fmt.Fprintf(metadataFile, "total_duration=%.2f\n", manifest.TotalDuration)
fmt.Fprintf(metadataFile, "segment_duration=%d\n", segmentDuration)
fmt.Fprintf(metadataFile, "frame_interval=%.2f\n", frameInterval)
fmt.Fprintf(metadataFile, "segment_count=%d\n", len(manifest.Segments))
fmt.Fprintf(metadataFile, "hls_playlist=%s/playlist.m3u8\n", hlsOutputDir)
for _, segment := range manifest.Segments {
fmt.Fprintf(metadataFile, "segment_%d=%.2f-%.2f\n", segment.Index, segment.StartTime, segment.EndTime)
}
// Copy the HLS playlist to thumbnails directory for reference
playlistSrc := filepath.Join(hlsOutputDir, "playlist.m3u8")
playlistDst := filepath.Join(thumbnailsDir, "playlist.m3u8")
if err := copyFile(playlistSrc, playlistDst); err != nil {
fmt.Printf("Warning: Could not copy playlist file: %v\n", err)
}
fmt.Printf("\n✅ Successfully generated %d thumbnail strips from HLS segments\n", len(manifest.Segments))
fmt.Printf("📁 HLS segments: %s/\n", hlsOutputDir)
fmt.Printf("📁 Thumbnails: %s/\n", thumbnailsDir)
fmt.Printf("📄 Metadata: %s\n", metadataPath)
fmt.Printf("📺 HLS Playlist: %s\n", playlistSrc)
return nil
}
// copyFile copies a file from src to dst
func copyFile(src, dst string) error {
input, err := os.ReadFile(src)
if err != nil {
return err
}
return os.WriteFile(dst, input, 0644)
}
func main() {
// Check if input file exists
inputFile := "input.mp4"
if _, err := os.Stat(inputFile); os.IsNotExist(err) {
fmt.Printf("Input file '%s' does not exist.\n", inputFile)
fmt.Printf("Please place a video file named 'input.mp4' in the current directory.\n")
os.Exit(1)
}
// Configuration
segmentDuration := 30 // Each HLS segment will be 30 seconds
frameInterval := 2.0 // Take a frame every 2 seconds within each segment
fmt.Printf("🎬 Generating HLS-based preview thumbnails from '%s'\n", inputFile)
fmt.Printf("📊 Configuration:\n")
fmt.Printf(" - HLS segment duration: %d seconds\n", segmentDuration)
fmt.Printf(" - Frame interval: %.1f seconds\n", frameInterval)
fmt.Printf("\n")
err := generateHLSBasedPreviewStrips(inputFile, segmentDuration, frameInterval)
if err != nil {
fmt.Printf("❌ Error generating HLS-based preview strips: %v\n", err)
os.Exit(1)
}
fmt.Printf("\n🎉 Done! You now have:\n")
fmt.Printf(" 📺 HLS video segments (.ts files) for streaming\n")
fmt.Printf(" 🖼️ Thumbnail strips for video previews\n")
fmt.Printf(" 📄 m3u8 playlist for video playback\n")
}
To run the code -
go run segment.go
Now we have solved the problem of having to generate thumbnail preview of the whole video all at once in one big image. Now we have smaller chunks of the thumbnail preview which can be served to client lazily as needed.
But we have one other problem - while generating the thumbnail preview images, we haven't accounted for the speed at which user is scrubbing through the progress bar. For an optimal user experience, we should be generating thumbnail preview images of bigger segments(with higher frame interval) for higher scrubbing speeds and vice versa for lower scrubbing speeds. This optimization may be an overkill for short videos, but it is a must have for longer videos.
So how many segment durations should we consider? Well, this would be subjective and depends on the usage pattern of the users of how fast or how slow they are scrubbing through the video. For the sake of this article, let's just consider 3 different segment durations - 30 seconds(with 2 seconds frame interval), 60 seconds(with 4 seconds frame interval) and 90 seconds(with 6 seconds frame interval).
Following is the updated code -
package main
import (
"bufio"
"bytes"
"fmt"
"image"
"image/draw"
"image/jpeg"
"os"
"os/exec"
"path/filepath"
"regexp"
"strconv"
"strings"
)
// SegmentInfo holds information about an HLS video segment
type SegmentInfo struct {
Index int
Filename string
Duration float64
StartTime float64
EndTime float64
}
// HLSManifest represents the parsed m3u8 file
type HLSManifest struct {
TargetDuration float64
TotalDuration float64
Segments []SegmentInfo
}
// ThumbnailConfig represents a configuration for thumbnail generation from existing segments
type ThumbnailConfig struct {
FrameInterval float64 // Interval between frames in seconds
LogicalGrouping int // How many segments to group together (1=no grouping, 3=group 3 segments)
OutputSuffix string // Suffix for output directories and files
Description string // Description of the configuration
}
// generateHLSSegments creates HLS segments using ffmpeg
func generateHLSSegments(videoPath string, segmentDuration int, outputDir string) (*HLSManifest, error) {
// Create output directory
if err := os.MkdirAll(outputDir, 0755); err != nil {
return nil, fmt.Errorf("error creating output directory: %v", err)
}
playlistPath := filepath.Join(outputDir, "playlist.m3u8")
segmentPattern := filepath.Join(outputDir, "segment_%03d.ts")
fmt.Printf("🎬 Creating HLS segments with ffmpeg...\n")
fmt.Printf(" Segment duration: %d seconds\n", segmentDuration)
fmt.Printf(" Output directory: %s\n", outputDir)
// Use ffmpeg to create HLS segments
cmd := exec.Command("ffmpeg",
"-i", videoPath,
"-c:v", "libx264", // Video codec
"-c:a", "aac", // Audio codec
"-f", "hls", // HLS format
"-hls_time", fmt.Sprintf("%d", segmentDuration), // Segment duration
"-hls_list_size", "0", // Keep all segments in playlist
"-hls_segment_filename", segmentPattern, // Segment filename pattern
"-y", // Overwrite output files
playlistPath)
fmt.Printf("Running: %s\n", strings.Join(cmd.Args, " "))
output, err := cmd.CombinedOutput()
if err != nil {
return nil, fmt.Errorf("error creating HLS segments: %v\nOutput: %s", err, string(output))
}
fmt.Printf("✅ HLS segmentation complete\n")
// Parse the generated m3u8 file
manifest, err := parseM3U8(playlistPath)
if err != nil {
return nil, fmt.Errorf("error parsing m3u8 file: %v", err)
}
// Note: segment paths are kept as just filenames (e.g., "segment_000.ts")
// The hlsDir will be prepended when accessing them later
fmt.Printf("📄 Parsed manifest: %d segments, total duration: %.2fs\n",
len(manifest.Segments), manifest.TotalDuration)
return manifest, nil
}
// parseM3U8 parses an m3u8 playlist file
func parseM3U8(playlistPath string) (*HLSManifest, error) {
file, err := os.Open(playlistPath)
if err != nil {
return nil, fmt.Errorf("error opening playlist file: %v", err)
}
defer file.Close()
manifest := &HLSManifest{
Segments: []SegmentInfo{},
}
scanner := bufio.NewScanner(file)
var currentDuration float64
var currentStartTime float64
segmentIndex := 0
// Regex to extract duration from #EXTINF line
extinfoRegex := regexp.MustCompile(`#EXTINF:([\d.]+),`)
for scanner.Scan() {
line := strings.TrimSpace(scanner.Text())
if strings.HasPrefix(line, "#EXT-X-TARGETDURATION:") {
durationStr := strings.TrimPrefix(line, "#EXT-X-TARGETDURATION:")
manifest.TargetDuration, _ = strconv.ParseFloat(durationStr, 64)
} else if strings.HasPrefix(line, "#EXTINF:") {
// Extract duration from EXTINF line
matches := extinfoRegex.FindStringSubmatch(line)
if len(matches) > 1 {
currentDuration, _ = strconv.ParseFloat(matches[1], 64)
}
} else if strings.HasSuffix(line, ".ts") && !strings.HasPrefix(line, "#") {
// This is a segment file
segment := SegmentInfo{
Index: segmentIndex,
Filename: line,
Duration: currentDuration,
StartTime: currentStartTime,
EndTime: currentStartTime + currentDuration,
}
manifest.Segments = append(manifest.Segments, segment)
manifest.TotalDuration += currentDuration
currentStartTime += currentDuration
segmentIndex++
}
}
if err := scanner.Err(); err != nil {
return nil, fmt.Errorf("error reading playlist file: %v", err)
}
return manifest, nil
}
// generateOptimizedPreviewConfigurations creates one set of HLS segments and multiple thumbnail configurations
func generateOptimizedPreviewConfigurations(videoPath string) error {
// Check if input file exists
if _, err := os.Stat(videoPath); os.IsNotExist(err) {
return fmt.Errorf("input video file does not exist: %s", videoPath)
}
fmt.Printf("🎬 Generating optimized HLS-based preview system from '%s'\n", videoPath)
fmt.Printf("🚀 Efficiency: One segmentation pass, multiple thumbnail configurations\n")
fmt.Printf("💡 Benefits:\n")
fmt.Printf(" ⚡ Faster processing (single HLS segmentation)\n")
fmt.Printf(" 💾 Less storage (reuse same .ts files)\n")
fmt.Printf(" 🎯 Multiple preview granularities\n")
fmt.Printf(" 🎬 Better accuracy (10s segments vs 30s)\n")
fmt.Printf(" 💫 More precise hover previews\n")
fmt.Printf("\n")
// Step 1: Generate one set of fine-grained HLS segments (10s for better accuracy)
segmentDuration := 10
hlsOutputDir := "hls_segments"
fmt.Printf("📹 Step 1: Creating HLS segments (%ds each)...\n", segmentDuration)
manifest, err := generateHLSSegments(videoPath, segmentDuration, hlsOutputDir)
if err != nil {
return fmt.Errorf("error generating HLS segments: %v", err)
}
fmt.Printf("✅ Generated %d HLS segments, total duration: %.2fs\n\n", len(manifest.Segments), manifest.TotalDuration)
// Step 2: Define multiple thumbnail configurations using the same segments
configs := []ThumbnailConfig{
{
FrameInterval: 2.0,
LogicalGrouping: 1,
OutputSuffix: "detailed",
Description: "Detailed (2s intervals, individual 10s segments)",
},
{
FrameInterval: 4.0,
LogicalGrouping: 3,
OutputSuffix: "balanced",
Description: "Balanced (4s intervals, 30s logical groups)",
},
{
FrameInterval: 6.0,
LogicalGrouping: 6,
OutputSuffix: "overview",
Description: "Overview (6s intervals, 60s logical groups)",
},
}
fmt.Printf("🖼️ Step 2: Generating thumbnail configurations...\n")
for i, config := range configs {
fmt.Printf(" %d. %s\n", i+1, config.Description)
}
fmt.Printf("\n")
// Step 3: Generate thumbnail strips for each configuration
for _, config := range configs {
fmt.Printf("🔄 Processing: %s\n", config.Description)
err := generateThumbnailsFromExistingSegments(manifest, hlsOutputDir, config)
if err != nil {
return fmt.Errorf("error generating thumbnails for config %s: %v", config.OutputSuffix, err)
}
fmt.Printf("✅ Completed: %s\n\n", config.Description)
}
fmt.Printf("🎉 Optimized preview system completed!\n")
fmt.Printf("📁 Generated:\n")
fmt.Printf(" - %s/ (single set of HLS segments)\n", hlsOutputDir)
for _, config := range configs {
fmt.Printf(" - thumbnails_%s/ (%s)\n", config.OutputSuffix, config.Description)
}
return nil
}
// generateThumbnailsFromExistingSegments creates thumbnail strips from existing HLS segments
func generateThumbnailsFromExistingSegments(manifest *HLSManifest, hlsDir string, config ThumbnailConfig) error {
// Create output directory for thumbnails
thumbnailsDir := fmt.Sprintf("thumbnails_%s", config.OutputSuffix)
if err := os.MkdirAll(thumbnailsDir, 0755); err != nil {
return fmt.Errorf("error creating thumbnails directory: %v", err)
}
// Group segments according to configuration
segmentGroups := groupSegments(manifest.Segments, config.LogicalGrouping)
fmt.Printf(" Grouped %d segments into %d logical units\n", len(manifest.Segments), len(segmentGroups))
// Generate thumbnail strips for each group
for groupIndex, group := range segmentGroups {
var thumbnailPath string
var err error
if config.LogicalGrouping == 1 {
// Single segment - use existing function
segment := group[0]
segmentPath := filepath.Join(hlsDir, segment.Filename)
thumbnailPath = fmt.Sprintf("%s/segment-%d-thumbnails.jpg", thumbnailsDir, segment.Index)
err = generateThumbnailStripFromSegment(segmentPath, segment, thumbnailPath, config.FrameInterval)
} else {
// Multiple segments - create combined thumbnail strip
thumbnailPath = fmt.Sprintf("%s/segment-group-%d-thumbnails.jpg", thumbnailsDir, groupIndex)
err = generateCombinedThumbnailStrip(hlsDir, group, thumbnailPath, config.FrameInterval)
}
if err != nil {
return fmt.Errorf("error generating thumbnails for group %d: %v", groupIndex, err)
}
}
// Create metadata file
return createThumbnailMetadata(thumbnailsDir, manifest, config, segmentGroups)
}
// groupSegments groups segments according to the logical grouping configuration
func groupSegments(segments []SegmentInfo, groupSize int) [][]SegmentInfo {
if groupSize <= 1 {
// No grouping - each segment is its own group
groups := make([][]SegmentInfo, len(segments))
for i, segment := range segments {
groups[i] = []SegmentInfo{segment}
}
return groups
}
// Group segments
var groups [][]SegmentInfo
for i := 0; i < len(segments); i += groupSize {
end := i + groupSize
if end > len(segments) {
end = len(segments)
}
groups = append(groups, segments[i:end])
}
return groups
}
// generateThumbnailStripFromSegment creates a thumbnail strip from an HLS segment
func generateThumbnailStripFromSegment(segmentPath string, segment SegmentInfo, outputPath string, frameInterval float64) error {
if frameInterval <= 0 {
return fmt.Errorf("frame interval must be greater than 0 seconds")
}
// Check if segment file exists
if _, err := os.Stat(segmentPath); os.IsNotExist(err) {
return fmt.Errorf("segment file does not exist: %s", segmentPath)
}
fmt.Printf(" Processing segment %d (%s): %.2fs duration, %.2fs intervals\n",
segment.Index, filepath.Base(segmentPath), segment.Duration, frameInterval)
// Extract frames at the specified interval from the segment
frames := []image.Image{}
maxTimestamp := segment.Duration - 1.0 // Leave 1 second buffer at the end
for timestamp := 0.0; timestamp < maxTimestamp; timestamp += frameInterval {
cmd := exec.Command("ffmpeg",
"-v", "error",
"-ss", fmt.Sprintf("%.2f", timestamp),
"-i", segmentPath,
"-vframes", "1",
"-f", "image2pipe",
"-pix_fmt", "yuvj420p",
"-vcodec", "mjpeg",
"-q:v", "3",
"-an",
"-")
output, err := cmd.Output()
if err != nil {
continue // Skip failed frames
}
frame, err := jpeg.Decode(bytes.NewReader(output))
if err != nil {
continue // Skip failed frames
}
frames = append(frames, frame)
}
if len(frames) == 0 {
return fmt.Errorf("no frames extracted from segment %d", segment.Index)
}
// Create a new image to hold all frames horizontally
frameWidth := frames[0].Bounds().Dx()
frameHeight := frames[0].Bounds().Dy()
stripWidth := frameWidth * len(frames)
strip := image.NewRGBA(image.Rect(0, 0, stripWidth, frameHeight))
// Place each frame side by side
for i, frame := range frames {
draw.Draw(strip,
image.Rect(i*frameWidth, 0, (i+1)*frameWidth, frameHeight),
frame,
image.Point{0, 0},
draw.Src)
}
// Save the segment thumbnail strip
outFile, err := os.Create(outputPath)
if err != nil {
return fmt.Errorf("error creating output file: %v", err)
}
defer outFile.Close()
if err := jpeg.Encode(outFile, strip, &jpeg.Options{Quality: 90}); err != nil {
return fmt.Errorf("error encoding final image: %v", err)
}
fmt.Printf(" ✓ Generated strip: %s (%d frames)\n", outputPath, len(frames))
return nil
}
// generateCombinedThumbnailStrip creates a thumbnail strip from multiple segments
func generateCombinedThumbnailStrip(hlsDir string, segments []SegmentInfo, outputPath string, frameInterval float64) error {
fmt.Printf(" Creating combined strip from %d segments: ", len(segments))
for i, seg := range segments {
if i > 0 {
fmt.Printf(", ")
}
fmt.Printf("%d", seg.Index)
}
fmt.Printf("\n")
var allFrames []image.Image
// Extract frames from each segment in the group
for _, segment := range segments {
segmentPath := filepath.Join(hlsDir, segment.Filename)
// Extract frames from this segment
maxTimestamp := segment.Duration - 1.0
for timestamp := 0.0; timestamp < maxTimestamp; timestamp += frameInterval {
cmd := exec.Command("ffmpeg",
"-v", "error",
"-ss", fmt.Sprintf("%.2f", timestamp),
"-i", segmentPath,
"-vframes", "1",
"-f", "image2pipe",
"-pix_fmt", "yuvj420p",
"-vcodec", "mjpeg",
"-q:v", "3",
"-an",
"-")
output, err := cmd.Output()
if err != nil {
continue // Skip failed frames
}
frame, err := jpeg.Decode(bytes.NewReader(output))
if err != nil {
continue // Skip failed frames
}
allFrames = append(allFrames, frame)
}
}
if len(allFrames) == 0 {
return fmt.Errorf("no frames extracted from segment group")
}
// Create combined strip
frameWidth := allFrames[0].Bounds().Dx()
frameHeight := allFrames[0].Bounds().Dy()
stripWidth := frameWidth * len(allFrames)
strip := image.NewRGBA(image.Rect(0, 0, stripWidth, frameHeight))
// Place all frames side by side
for i, frame := range allFrames {
draw.Draw(strip,
image.Rect(i*frameWidth, 0, (i+1)*frameWidth, frameHeight),
frame,
image.Point{0, 0},
draw.Src)
}
// Save the combined thumbnail strip
outFile, err := os.Create(outputPath)
if err != nil {
return fmt.Errorf("error creating output file: %v", err)
}
defer outFile.Close()
if err := jpeg.Encode(outFile, strip, &jpeg.Options{Quality: 90}); err != nil {
return fmt.Errorf("error encoding final image: %v", err)
}
fmt.Printf(" ✓ Generated combined strip: %s (%d frames)\n", outputPath, len(allFrames))
return nil
}
// createThumbnailMetadata creates metadata file for the thumbnail configuration
func createThumbnailMetadata(thumbnailsDir string, manifest *HLSManifest, config ThumbnailConfig, segmentGroups [][]SegmentInfo) error {
metadataPath := filepath.Join(thumbnailsDir, "segments.txt")
metadataFile, err := os.Create(metadataPath)
if err != nil {
return fmt.Errorf("error creating metadata file: %v", err)
}
defer metadataFile.Close()
fmt.Fprintf(metadataFile, "total_duration=%.2f\n", manifest.TotalDuration)
fmt.Fprintf(metadataFile, "frame_interval=%.2f\n", config.FrameInterval)
fmt.Fprintf(metadataFile, "logical_grouping=%d\n", config.LogicalGrouping)
fmt.Fprintf(metadataFile, "segment_count=%d\n", len(segmentGroups))
fmt.Fprintf(metadataFile, "output_suffix=%s\n", config.OutputSuffix)
fmt.Fprintf(metadataFile, "description=%s\n", config.Description)
// Calculate timing for logical groups
currentTime := 0.0
for i, group := range segmentGroups {
groupDuration := 0.0
for _, segment := range group {
groupDuration += segment.Duration
}
fmt.Fprintf(metadataFile, "segment_%d=%.2f-%.2f\n", i, currentTime, currentTime+groupDuration)
currentTime += groupDuration
}
return nil
}
// copyFile copies a file from src to dst
func copyFile(src, dst string) error {
input, err := os.ReadFile(src)
if err != nil {
return err
}
return os.WriteFile(dst, input, 0644)
}
func main() {
// Check if input file exists
inputFile := "input.mp4"
if _, err := os.Stat(inputFile); os.IsNotExist(err) {
fmt.Printf("Input file '%s' does not exist.\n", inputFile)
fmt.Printf("Please place a video file named 'input.mp4' in the current directory.\n")
os.Exit(1)
}
fmt.Printf("🎬 Generating optimized HLS-based preview system from '%s'\n", inputFile)
fmt.Printf("🚀 Efficiency: One segmentation pass, multiple thumbnail configurations\n")
fmt.Printf("💡 Benefits:\n")
fmt.Printf(" ⚡ Faster processing (single HLS segmentation)\n")
fmt.Printf(" 💾 Less storage (reuse same .ts files)\n")
fmt.Printf(" 🎯 Multiple preview granularities\n")
fmt.Printf(" 🎬 Better accuracy (10s segments vs 30s)\n")
fmt.Printf(" 💫 More precise hover previews\n")
fmt.Printf("\n")
err := generateOptimizedPreviewConfigurations(inputFile)
if err != nil {
fmt.Printf("❌ Error generating optimized preview system: %v\n", err)
os.Exit(1)
}
fmt.Printf("\n🎉 Success! Optimized preview system ready:\n")
fmt.Printf(" 📺 One set of HLS segments (reusable for streaming)\n")
fmt.Printf(" 🖼️ Three thumbnail configurations (detailed, balanced, overview)\n")
fmt.Printf(" ⚡ ~3x faster than generating separate segment sets\n")
}
go run main.go
If you run this code you would see files getting generated in following structure -
- hls_segments/ ==> HLS segments
- playlist.m3u8
- segment_000.ts
- segment_001.ts
- segment_002.ts
- ...
- thumbnails_detailed/ ==> Thumbnail preview images for detailed or slow scrubbing
- segments.txt
- segment-0-thumbnails.jpg
- segment-1-thumbnails.jpg
- segment-2-thumbnails.jpg
- ...
- thumbnails_balanced/ ==> Thumbnail preview images for balanced or medium scrubbing
- segments.txt
- segment-0-thumbnails.jpg
- segment-1-thumbnails.jpg
- segment-2-thumbnails.jpg
- ...
- thumbnails_overview/ ==> Thumbnail preview images for overview or fast scrubbing
- segments.txt
- segment-0-thumbnails.jpg
- segment-1-thumbnails.jpg
- segment-2-thumbnails.jpg
- ...
Here we made few interesting changes. Let me explain -
- We have reduced the segment duration from 30 seconds to 15 seconds. This is to reduce the variance in the resulting segment durations - even though we have declared
in code that segments should be 10 seconds, but due to the nature of the video, the actual segment durations may vary from one segment to the other. One segment could be of 8 seconds
and another could be of 12 seconds. Hence shorter the segment duration, lesser the variance in the segment durations.
In practice, the segment durations are kept even smaller ~2-4 seconds long. This results in even less variance and smaller segments can be streamed easily even on lower bandwidths.
- We have created 3 different types of thumbnails preview images - detailed, balanced and overview. Detailed thumbnails are generated with 2 seconds frame interval, balanced thumbnails
are generated with 4 seconds frame interval and overview thumbnails are generated with 6 seconds frame interval.
- If the user is scrubbing through the video at a slower speed, we can use the detailed thumbnails to get a more accurate preview of the video.
- If the user is scrubbing through the video at a medium speed, we can use the balanced thumbnails to get a more accurate preview of the video.
- If the user is scrubbing through the video at a faster speed, we can use the overview thumbnails to get a more accurate preview of the video.
- We have created a metadata files for each of the thumbnail preview type. These metadata files can be sent to the client so that it can decide which thumbnail preview type to use based
on the user's scrubbing speed. The metadata files contains the following information -
- Total duration of the video
- Frame interval for each thumbnail preview image
- Logical grouping of the thumbnail preview images
- Output suffix for the thumbnail preview images
- Description of the thumbnail preview images
So now we have a system that can generate thumbnail preview images for different segment durations and also for different scrubbing speeds.
We are leaving out the frontend part for now because the article is already getting too long. But the frontend part is not that complex. I might write a separate article on that.
Performance considerations
I have to admit that the code is not optimized for performance. It is just a proof of concept. Although one quick performance improvement we can do is using goroutines to generate the thumbnail preview images concurrently. This would parallelize the thumbnail preview generation and would speed up the process.
Conclusion
In this article, we have seen how we can generate thumbnail preview images for a video in a more efficient way. We have seen how we can use HLS segments to generate thumbnail preview images for different segment durations and also for different scrubbing speeds.
Well ofcourse this is definitely not how Youtube would have been doing it. Youtube would have been using a more sophisticated system to generate the thumbnail preview images. But this is a good starting point to understand how we can generate thumbnail preview images for a video in a more efficient way.
The code is available on my github repository. Feel free to fork/clone it and play around with it.