Meta Kotlin Buck2 Build Optimization: Technical Intelligence Summary

Performance Breakthrough Overview

• Achievement: 3x reduction in Android build times for massive Kotlin codebases
• Impact: Critical modules build 3x faster, average developer sees 30% improvement
• Scope: Meta's internal Android development with plans for open source release

Critical Problem Context

The Build Time Crisis

• Root Cause: Legacy code growth transformed 5-file modules into 50-file monsters
• Developer Impact: 20+ minute build times destroying productivity
• Business Impact: Developers spending half their day waiting for builds
• Scale: Massive mobile codebases where traditional small-module philosophy failed

Technical Implementation Details

Core Solution Components

1. Kotlin Build Tools API Integration with Buck2
2. Incremental compilation preservation across distributed builds
3. Custom compiler plugin adaptation for partial recompilation

Critical Implementation Challenges and Solutions

1. The Shading Nightmare

• Problem: Kotlin Build Tools API expects shaded compiler (kotlin-compiler-embeddable)
• Meta's toolchain: Used unshaded version causing java.lang.NoClassDefFoundError crashes
• Workaround: Strip org.jetbrains.kotlin prefix with jarjar and rebuild library
• Warning: This is a hack - proper solution requires migrating entire toolchain to shaded compiler
• Operational Impact: 3 days wrestling with compiler version conflicts

2. Cache Relocation Hell

• Problem: Buck2 distributed builds require portable compilation caches
• Failure Mode: Incremental compiler loses state across different environments
• Solution: Explicitly configure root project and build directories
• Risk: Conflicting overloads and ambiguity errors when builds run on different machines

3. Plugin Compatibility Crisis

• Problem: Custom compiler plugins designed for full codebase compilation
• Failure Mode: Plugins expecting complete codebase context break with incremental compilation
• Solution: Make plugins accumulate results across multiple compilation rounds instead of replacing
• Requirement: Plugin architecture redesign for incremental compatibility

Resource Requirements and Constraints

Prerequisites for Success

• Build System: Buck2 (solution is Buck2-specific)
• Annotation Processing: KSP2 required (KAPT incompatible)
• Infrastructure: Distributed build system with cache relocation support
• Team Expertise: Deep build system integration knowledge

Time Investment Reality

• Integration Complexity: 8-step technical integration process
• Development Time: Months of engineering effort for Meta's team
• Debugging Overhead: Significant time spent on compiler version conflicts and cache issues

Performance Impact Analysis

Quantified Results

• Best Case: Modules without annotation processing nearly doubled speed
• Average Case: 30% improvement across Meta's codebase
• Critical Modules: Up to 3x faster build times
• CI/CD Impact: Pull requests reduced from 20+ minutes to under 10 minutes

Performance Limiting Factors

• KAPT Usage: Teams using KAPT remain "stuck in the slow lane"
• Annotation Processing: Still requires full non-incremental step before compilation
• Plugin Overhead: Custom plugins without incremental support create bottlenecks

Operational Warnings and Failure Modes

Critical Failure Scenarios

1. Cache Corruption: Distributed builds with misconfigured paths cause compilation failures
2. Plugin Incompatibility: Non-incremental plugins break partial recompilation
3. Compiler Version Conflicts: Shaded/unshaded compiler mismatches crash builds
4. Legacy Code Dependencies: Large modules still create compilation bottlenecks

Common Implementation Pitfalls

• Assumption: All compiler plugins work with incremental compilation (false)
• Misconception: Solution applies to Gradle-based projects (Buck2-specific)
• Hidden Cost: Requires complete toolchain migration to shaded compiler

Decision Support Matrix

When This Solution Applies

Scenario	Applicability	Alternative
Buck2 + Large Kotlin modules	High	Direct implementation
Gradle + Large Kotlin modules	Low	Wait for Gradle adoption
KAPT-based annotation processing	None	Migrate to KSP2 first
Small module architecture	Low	Existing approach sufficient

Trade-off Analysis

Worth It Despite:

• Complex 8-step integration process
• Months of engineering investment
• Risk of cache corruption in distributed builds
• Need for plugin architecture changes

Not Worth It If:

• Using Gradle (no current path)
• Dependent on KAPT
• Small modules already providing fast builds
• Limited build system expertise

Migration Path and Prerequisites

Essential Requirements

1. Build System: Must use Buck2
2. Annotation Processing: Migrate from KAPT to KSP2
3. Compiler Plugins: Redesign for incremental compatibility
4. Infrastructure: Distributed build cache with proper relocation

Breaking Changes to Expect

• Toolchain Migration: Complete switch to shaded Kotlin compiler
• Plugin API Changes: Custom plugins require incremental compilation support
• Cache Configuration: Explicit path management for distributed builds

Community and Support Reality

• Open Source Timeline: Meta plans Buck2 open source integration
• Gradle Adoption: No current timeline for Gradle ecosystem
• Community Support: Limited to Buck2 user base
• Documentation: Meta's engineering blog provides technical implementation details

Critical Success Factors

1. Technical Expertise: Deep understanding of Kotlin compilation internals
2. Infrastructure: Robust distributed build system
3. Team Commitment: Months of integration and debugging effort
4. Architecture: Proper module boundaries still essential for optimal performance

Bottom Line Assessment

This breakthrough proves incremental compilation scales for massive Kotlin codebases but requires Buck2 adoption and significant engineering investment. For teams stuck with slow Kotlin builds on Buck2, this is transformational. For Gradle teams, this demonstrates what's possible but offers no immediate solution.