rust-lldb: Rust Debugging Technical Reference

Overview

rust-lldb is a specialized debugger that understands Rust's type system, preventing manual memory layout decoding during production incidents. Essential for debugging complex Rust types where plain LLDB shows internal memory structures instead of actual data values.

Critical Value Proposition

• Plain LLDB shows: ptr, cap, len fields requiring manual pointer dereferencing
• rust-lldb shows: Some("hello") - actual readable values
• Time savings: Reduces debugging from hours to minutes for complex types

Configuration Requirements

Build Settings That Work in Production

# Debug build (standard)
cargo clean && cargo build

# Release build with debugging capability
cargo build --release --config "profile.release.debug=true" --config "profile.release.opt-level=1"

Critical Setting: opt-level=1 is the maximum usable optimization level

• opt-level=2: Optimizes out local variables
• opt-level=3: Inlines everything, deletes stack frames
• Result: Variables show as <optimized out>

Installation Verification

# Check rust-lldb exists
which rust-lldb

# If missing, reinstall Rust toolchain
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
rustup component add rust-src    # Required for stepping through stdlib

Platform-Specific Failure Modes

macOS Critical Issues

• Monterey: Broke rust-lldb for months, requires Xcode version coordination
• Xcode 13.1: Won't load Rust symbols
• Xcode 13.2: Crashes on launch
• Xcode 13.2.1: Works only with codesigned binaries
• Codesigning requirement: codesign -s - target/debug/your_binary
• Status: Check rust-lang GitHub issues for current Apple-induced breakage

Linux DWARF Version Compatibility

• Rust 1.70+: Changed DWARF format
• LLDB <15: Silently fails to load symbols (no error message)
• Symptom: Debugging appears to work but shows no useful information
• Solution: Update LLDB to version 15+ or downgrade Rust
• ptrace security: echo 0 | sudo tee /proc/sys/kernel/yama/ptrace_scope

Windows WSL2 Failure Patterns

• Intermittent hangs: LLDB hangs when inspecting Vec with 100+ strings
• Memory corruption: SIGSEGV in Python pretty printer after restart cycles
• Recommendation: Use native Windows Rust installation instead of WSL2

Performance Characteristics

Memory Usage Impact

Data Structure	Plain LLDB	rust-lldb Memory Cost	Failure Threshold
Simple types	Minimal	2x overhead	Non-issue
HashMap <1000 entries	Minimal	5x overhead	Acceptable
HashMap >1000 entries	Minimal	10x+ overhead	LLDB crashes
Vec >100 items	Minimal	Exponential growth	System memory exhaustion

Critical Threshold: Python pretty printer fails catastrophically on large collections

• Symptom: rust-lldb consumes 4-8GB RAM
• Cause: 10,000 Python function calls for HashMap pretty printing
• Solution: Disable pretty printing or use print statements

Performance Degradation

• Python overhead: 2-10x slower than plain LLDB
• Conditional breakpoints: 10x performance penalty (evaluates Python on every hit)
• Large collection inspection: Can consume more memory than target program

Emergency Debugging Procedures

Production Core Dump Analysis

rust-lldb -c core.dump target/release/your_service
(lldb) bt              # Backtrace
(lldb) fr v            # Frame variables
# Time constraint: ~30 seconds before management escalation

Live Process Attachment (High Risk)

rust-lldb --attach-pid $(pgrep your_service)
# WARNING: May crash the service
# Ensure backup instances are running

Essential Commands (Copy-Paste Ready)

# Breakpoints that work reliably
(lldb) b main
(lldb) b src/lib.rs:42
(lldb) b "my_crate::my_function"

# Navigation
(lldb) run             # Start execution
(lldb) bt              # Where am I?
(lldb) fr v            # What variables exist?
(lldb) p my_variable   # What's in this thing?
(lldb) n               # Next line (don't step into functions)
(lldb) c               # Continue execution

Failure Recovery Procedures

When Pretty Printer Crashes

(lldb) settings set target.enable-synthetic-value false

Result: Shows raw pointers and discriminant values, but prevents crashes

When Symbols Are Missing

(lldb) target symbols add target/debug/deps/*.so

Nuclear Option (Complete Reset)

cargo clean
rm -rf target/
cargo build

Alternative Tool When rust-lldb Fails

rust-gdb target/debug/your_binary
# Different failure modes, sometimes works when rust-lldb doesn't

Tool Comparison Matrix

Tool	Performance Cost	Platform Reliability	Memory Usage	Failure Modes
rust-lldb	2-10x slower	macOS: Broken by Xcode updates	High on large collections	Pretty printer crashes
rust-gdb	Minimal overhead	Linux: Excellent, macOS: Broken since 2019	Minimal	Limited on macOS
Plain LLDB	No overhead	Consistent cross-platform	Minimal	Requires manual memory interpretation
VS Code + CodeLLDB	UI lag + Python overhead	Extension randomly fails	High	UI complications + rust-lldb issues
JetBrains RustRover	Heavy IDE overhead	Windows/Linux only	Very high	$200/year cost
println! debugging	Hot loop performance impact	Universal	None	Requires recompile+deploy

Critical Debugging Scenarios

Async Code Failure Patterns

• State machine names: __async_gen_0_state_machine with fields like __field_37_variant_0_async_suspend
• Backtrace pollution: 90% executor internals, 10% actual code
• Recommendation: Use tracing framework instead of debugger
• Breakpoint strategy: Set on line numbers, not async function names

Complex Type Debugging

• HashMap<String, Vec<Option<Result<User, Error>>>>: 47 levels of pointer indirection in plain LLDB
• Solution time: Can take hours to manually decode vs. seconds with rust-lldb
• Memory corruption detection: Use AddressSanitizer for unsafe code instead of manual debugging

Production Incident Time Estimates

Issue Type	Identification Time	Understanding Time	Fix+Test Time
Simple panic	5 minutes	2 hours	Variable
Memory corruption	30 minutes	Half day minimum	Possibly career-ending
Async deadlock	30 minutes	Rest of afternoon	Entire evening
Proc macro issues	N/A - debugger won't help	N/A	Use print statements only

When to Abandon rust-lldb

Unwinnable Scenarios

• Release builds with full optimization: Everything shows as "optimized out"
• Embedded targets: Use RTT instead
• Proc macro generated types: Pretty printer crashes, use cargo expand
• Meta-debugging situations: When debugging the debugger itself

Performance Threshold Violations

• Conditional breakpoints making progress impossible: Use regular breakpoints + manual continue
• Memory usage exceeding system resources: Switch to print statements
• Pretty printer infinite loops: Disable synthetic values

Alternative Tools for Specific Use Cases

Memory Issues

• AddressSanitizer: RUSTFLAGS="-Z sanitizer=address" cargo run
• Better than manual debugging: Automated memory error detection

Time-Travel Debugging (Linux only)

• rr (Record and Replay): Deterministic execution replay
• Use case: Race conditions and non-deterministic bugs
• Command: rr record ./target/debug/binary && rr replay

Async-Specific Debugging

• Tokio tracing framework: Structured logging more reliable than debugger attachment
• Performance impact: Lower than interactive debugging
• Production safety: No process attachment risk

Resource Requirements

Expertise Prerequisites

• Understanding of Rust memory layout
• LLDB command proficiency
• Python pretty-printer debugging (advanced scenarios)
• Platform-specific toolchain management

Time Investment Realistic Expectations

• Learning curve: 2-4 hours for basic proficiency
• Platform setup: 30 minutes to 4 hours depending on OS
• Production debugging session: 30 minutes to multiple days
• Fighting debugger toolchain issues: Can exceed actual debugging time

Community Resources (Ranked by Utility)

High-Value Resources

• Rust Users Forum: Real problems, real solutions, search before posting
• GitHub rust-lang/rust issues: Current breakage reports and workarounds
• cargo-expand: Essential for proc macro debugging (cargo install cargo-expand)

Platform-Specific Resources

• macOS debugging guide: Covers codesigning requirements
• Rust embedded debugonomicon: Hardware debugging setup (embedded only)

Low-Value Resources

• Official LLDB documentation: 800 pages without Rust mention
• Rust Book debugging section: Three paragraphs of "use a debugger"
• LLDB tutorial for GDB users: C++ focused, skip entirely

Security Considerations

Production Environment Risks

• Process attachment: Can crash target service
• Core dump analysis: Safer than live attachment
• ptrace scope changes: Security implications on Linux systems
• Remote debugging: Network exposure of debugging interface

Symbol Information Exposure

• Debug builds: Contain more information than necessary for production
• Core dumps: May contain sensitive data in memory
• Remote debugging ports: Should not be exposed to internet

Breaking Changes and Compatibility

Version Compatibility Matrix

• Rust 1.70+: DWARF format change requires LLDB 15+
• macOS updates: Regularly break rust-lldb compatibility
• VS Code extension updates: Introduce new failure modes
• Xcode updates: Require rust-lldb version coordination

Monitoring for Breakage

• Symptom: Previously working debugging setup suddenly fails
• Check sequence: GitHub issues → Rust version → LLDB version → OS updates
• Mitigation: Version pinning in CI/CD environments