Lightpanda Browser

A Headless Browser Built from Scratch in Zig for AI Agents

Deep Technical Dive — Architecture, Internals & Benchmarks

The Problem: Browsers Were Built for Humans

Modern browsers are engineering marvels — Chrome has ~35 million lines of code. But most of that code serves visual rendering: CSS layout, compositing, rasterization, GPU acceleration, font shaping.

For headless automation, this is pure waste

  • AI agents don't have eyes — they read the DOM
  • Web scrapers extract data, not pixels
  • Test runners check assertions, not aesthetics

Even "headless Chrome" starts the full rendering pipeline. You're paying for a cinema ticket to read the subtitles.

Lightpanda's Bet: Start from Zero

"What if we built a browser that never had a screen?"

What It Keeps

  • HTML parsing (html5ever from Servo)
  • JavaScript execution (V8)
  • DOM tree + Web APIs
  • HTTP networking (libcurl)
  • Cookie / session management
  • CDP server for tool compatibility

What It Throws Away

  • CSS layout engine
  • Visual rendering / compositing
  • Image decoding
  • Font shaping and rasterization
  • GPU acceleration layer
  • Media playback / WebRTC
  • Extension system

Why Zig? Language Choice Is Architecture

Zig's Advantages for a Browser

  • No hidden allocations — every byte of memory is explicit
  • No garbage collector — deterministic, predictable performance
  • Comptime — compile-time code execution for zero-cost abstractions
  • C ABI compatible — direct FFI to libcurl, V8, html5ever
  • Cross-compilation — build for Linux/macOS/ARM from any host

vs. C++ (Chrome/Firefox)

  • No undefined behavior footguns
  • No header file soup
  • Explicit error handling (no exceptions)
  • Built-in test runner
  • Compiles 10-100x faster than C++

Zig 0.15.2 is used — pre-1.0 but stable enough for production. The team explicitly chose it for "low-level system programming" with optimization headroom.

Architecture Overview

Four layers, three foreign dependencies

CDP Server — Chrome DevTools Protocol (WebSocket)
↓
Browser Core (Zig) — DOM, Events, Web APIs, Cookie Jar, Routing
↓
html5ever
HTML Parser (Rust)
V8
JS Engine (C++)
libcurl
HTTP Client (C)
↓
OS / Network Stack — Linux, macOS, Docker

HTML Parsing: html5ever via Zig FFI

Lightpanda uses Servo's html5ever — the same spec-compliant HTML parser that powers Firefox's experimental engine.

  • Written in Rust, called from Zig via C ABI
  • Full HTML5 spec compliance
  • Handles malformed HTML gracefully
  • Builds a DOM tree that Zig code owns and manages

Key insight: By using html5ever, Lightpanda avoids reimplementing the gnarliest part of browser development — spec-compliant HTML parsing with all its edge cases.

The Pipeline

// Simplified flow
Raw HTML bytes
    ↓
html5ever tokenizer
    ↓
html5ever tree builder
    ↓
Zig DOM tree
    ↓
Web API bindings
    ↓
V8 JavaScript context

JavaScript Execution: V8 Integration

Lightpanda embeds Google V8 — the same JS engine powering Chrome and Node.js — for full ECMAScript compliance.

What Works

  • Full ES2024+ JavaScript execution
  • DOM manipulation (createElement, querySelector, etc.)
  • fetch() and XMLHttpRequest
  • Promises, async/await
  • setTimeout / setInterval
  • Event listeners and dispatch

The Binding Challenge

V8's C++ API must be bridged to Zig. Lightpanda uses Zig's C ABI interop to create bindings that expose DOM objects as V8 JavaScript objects.

// Zig exposes DOM nodes to V8
// so JS like:
document.getElementById('app')
// resolves to a Zig-managed
// DOM node, not a C++ object

Networking: libcurl as the HTTP Layer

Rather than implementing HTTP/1.1, HTTP/2, TLS, and connection pooling from scratch, Lightpanda delegates to libcurl — the battle-tested C library powering billions of HTTP requests daily.

  • HTTP/1.1 and HTTP/2 support
  • TLS via OpenSSL/LibreSSL
  • Cookie jar management
  • Proxy support (HTTP, SOCKS5)
  • Custom header injection
  • Request interception — block, modify, or mock

Request Interception

One of Lightpanda's most useful features for automation:

// Via CDP, you can intercept
// any network request:

// Block images (save bandwidth)
page.setRequestInterception(true)
page.on('request', req => {
  if (req.resourceType() === 'image')
    req.abort()
  else
    req.continue()
})

CDP: Speaking Chrome's Language

The Chrome DevTools Protocol is the standard wire protocol for controlling browsers programmatically. Lightpanda implements a CDP server so existing tools work unchanged.

How It Works

// Connect Puppeteer to Lightpanda
const browser = await puppeteer.connect({
  browserWSEndpoint:
    'ws://127.0.0.1:9222'
});

const page = await browser.newPage();
await page.goto('https://example.com');
const title = await page.title();

CDP Domains Supported

  • Page — navigation, lifecycle events
  • Runtime — JS evaluation
  • DOM — tree inspection, manipulation
  • Network — request interception
  • Target — page/context management
  • Input — keyboard/mouse events

Multiple concurrent CDP connections are supported within a single Lightpanda process.

Playwright & Puppeteer Compatibility

Puppeteer (Full Support)

const puppeteer = require(
  'puppeteer-core'
);

const browser = await
  puppeteer.connect({
    browserWSEndpoint:
      `ws://127.0.0.1:9222`
  });

// Use exactly like Chrome
const page = await browser.newPage();
await page.goto('https://...');
await page.click('#submit');

Playwright (Growing Support)

const { chromium } = require(
  'playwright'
);

const browser = await
  chromium.connectOverCDP(
    'http://127.0.0.1:9222'
  );

// Works, but Playwright may
// use different code paths as
// new APIs become available

Playwright's internal feature detection may route to different implementations as Lightpanda exposes more Web APIs. Test your specific workflows.

Performance Benchmarks

Puppeteer requesting 100 pages — AWS EC2 m5.large

2.3s
Lightpanda time
25.2s
Chrome time
11x
Speed advantage
24 MB
Lightpanda peak RAM
207 MB
Chrome peak RAM
9x
Memory savings

Why it matters: At cloud scale, 9x memory reduction means 9x more browser sessions per server. For a fleet running 10,000 concurrent sessions, that's the difference between 100 servers and 12.

Why Is It So Fast?

It's not one trick — it's removing entire subsystems

Subsystem Chrome Headless Lightpanda
CSS Layout (Blink) Full layout tree computation Eliminated
Rendering / Paint Skia rasterization (skipped but loaded) Never compiled in
GPU Process Separate process, IPC overhead Does not exist
Image Decode PNG/JPEG/WebP decoders loaded Eliminated
Font Shaping HarfBuzz, FreeType loaded Eliminated
Process Model Multi-process (browser + renderer + GPU) Single process
Memory Allocator PartitionAlloc (general purpose) Zig's explicit allocators

Zig's Memory Model in Practice

Zig gives Lightpanda explicit control over every allocation. There is no hidden memory overhead from a garbage collector or runtime.

Allocator Strategy

  • Arena allocators for per-page DOM trees — allocate fast, free all at once
  • General purpose allocator for long-lived objects
  • No GC pauses — deterministic latency
  • Memory is freed the moment a page context closes
// Zig allocator pattern
const allocator = std.heap
    .page_allocator;

// Arena for a page session
var arena = std.heap
    .ArenaAllocator
    .init(allocator);
defer arena.deinit();

// All DOM nodes allocated here
// When page closes: one free()
const node = try arena
    .allocator()
    .create(DomNode);

Result: 24 MB peak for 100 pages vs Chrome's 207 MB. Each page context is a clean arena — no fragmentation, no leaks.

DOM Implementation

Lightpanda implements a custom DOM tree in Zig that supports the Web APIs needed for headless automation.

Supported Web APIs

  • document.querySelector() / querySelectorAll()
  • document.getElementById()
  • document.createElement()
  • element.innerHTML / textContent
  • element.getAttribute() / setAttribute()
  • element.click()
  • addEventListener() / dispatchEvent()
  • window.location, window.history

The Coverage Challenge

The full Web API surface has hundreds of interfaces with thousands of methods. Lightpanda's approach:

  • Implement the most-used APIs first
  • Validate against Web Platform Tests (WPT)
  • Prioritize APIs that AI agents and scrapers actually call
  • Accept that full coverage is a multi-year journey

Built-in robots.txt Compliance

Unlike Chrome, which has no concept of robots.txt, Lightpanda has native support for respecting crawl directives.

# Start with robots.txt compliance
lightpanda --obey_robots

# Lightpanda will:
# 1. Fetch /robots.txt first
# 2. Parse Allow/Disallow rules
# 3. Block navigation to
#    disallowed paths
# 4. Respect Crawl-delay

Why This Matters

  • AI agents scraping at scale must respect site policies
  • Built-in is better than bolted-on — no chance of forgetting
  • Avoids IP bans and legal issues
  • Sets a precedent for responsible AI browsing

This is a deliberate design choice that signals Lightpanda's positioning: not a scraping bypass tool, but infrastructure for responsible automation.

AI Agent Integration Patterns

How AI agents use Lightpanda in practice

LLM
Claude, GPT, etc.
→
Agent Framework
browser-use, LaVague
→
Puppeteer/Playwright
CDP client
→
Lightpanda
CDP server

The Agent Loop

  • LLM decides what action to take (click, type, navigate)
  • Framework translates to Puppeteer commands
  • Puppeteer sends CDP messages to Lightpanda
  • Lightpanda executes and returns DOM state
  • Framework extracts page content for the LLM
  • LLM decides next action — repeat

Why Lightpanda Wins Here

  • Instant startup — no Chrome boot overhead
  • Low memory — run hundreds of agents on one box
  • Embeddable — single binary, no Chrome install
  • Deterministic — no rendering jitter or timing issues

Deployment Options

Self-Hosted

# Docker (amd64 + arm64)
docker run -p 9222:9222 \
  ghcr.io/lightpanda-io/browser

# Binary (Linux x86_64, macOS arm64)
curl -L https://github.com/
  lightpanda-io/browser/
  releases/latest \
  -o lightpanda
chmod +x lightpanda
./lightpanda --host 0.0.0.0 \
  --port 9222

Lightpanda Cloud

Managed infrastructure — no servers to run:

const browser = await
  puppeteer.connect({
    browserWSEndpoint:
      'wss://cloud.lightpanda.io'
      + '?token=YOUR_TOKEN'
  });

// Supports both Lightpanda
// and Chrome backends
// for maximum compatibility

The cloud option gives you both engines — use Lightpanda for speed, fall back to Chrome for full rendering when needed.

Concurrency: Single Process, Multiple Sessions

Chrome spawns separate OS processes for each tab (renderer, GPU, utility). Lightpanda runs everything in a single process with multiple CDP connections.

Chrome's Model

Chrome (1 page = ~4 processes)
browser   PID 1001  120 MB
renderer  PID 1002   45 MB
gpu       PID 1003   30 MB
utility   PID 1004   12 MB
Total: ~207 MB for one page

Lightpanda's Model

Lightpanda (N pages = 1 process)
lightpanda PID 2001 24 MB
  - CDP session 1 (page A)
  - CDP session 2 (page B)
  - CDP session 3 (page C)
  ...
Total: ~24 MB for 100 pages

Trade-off: Single process means one crash affects all sessions. But for automation workloads, this is acceptable — sessions are short-lived and can be retried.

Testing: Web Platform Tests (WPT)

Lightpanda validates its implementation against the Web Platform Tests — the same test suite used by Chrome, Firefox, and Safari to ensure interoperability.

  • Forked WPT repo with custom test harness
  • Tests run against DOM, Fetch, Events, HTML parsing
  • Incremental coverage — passing percentage grows with each release
  • Ensures spec compliance, not just "it works for one site"

Why WPT Matters

A browser that passes WPT for a given API means you can trust it behaves exactly like Chrome for that API. No surprises, no edge-case differences.

The Honest Reality

Full WPT coverage is a massive undertaking. Chrome has thousands of engineers; Lightpanda is a small team. Coverage will grow over time, and some niche APIs may never be implemented.

Limitations & Trade-offs

What you give up for 11x speed

Limitation Impact Workaround
No screenshots Can't capture visual state Use Chrome for visual testing
No CSS layout getBoundingClientRect() unavailable Use DOM selectors instead of coordinates
Incomplete Web APIs Some sites may break Report issues; coverage improves monthly
No media Audio/video/WebRTC unsupported Not needed for scraping/automation
Pre-1.0 Zig Language may introduce breaking changes Pinned to Zig 0.15.2
Single process One crash affects all sessions Restart + retry (sessions are short-lived)

Building from Source

Dependencies and compilation

Requirements

  • Zig 0.15.2 — the compiler
  • Rust toolchain — for html5ever
  • libcurl-dev — HTTP networking
  • V8 build deps — JavaScript engine
# Clone and build
git clone https://github.com/
  lightpanda-io/browser.git
cd browser
zig build -Doptimize=ReleaseSafe

Multi-Language Build

The build system is notable for orchestrating three languages:

build.zig         // Zig build script
  → compiles Zig source
  → calls cargo for html5ever
  → links V8 static lib
  → links libcurl
  → single binary output

Zig's build system handles C/C++ and Rust dependencies natively — no CMake, no Makefiles, no autoconf. One build.zig file orchestrates everything.

Competitive Landscape

Tool Engine Memory (100pg) Time (100pg) JS Support
Lightpanda Custom (Zig + V8) 24 MB 2.3s Full (V8)
Chrome Headless Blink + V8 207 MB 25.2s Full (V8)
Playwright (Chromium) Blink + V8 ~220 MB ~27s Full (V8)
jsdom (Node.js) JS DOM simulation ~80 MB ~5s Partial
Splash (Python) QtWebKit ~150 MB ~15s Full (WebKit)

Lightpanda occupies a unique niche: full V8 JavaScript with the memory footprint of a DOM simulator. No other tool offers both.

Open Source: AGPL-3.0

The License Choice

AGPL-3.0 is a strong copyleft license. If you modify Lightpanda and offer it as a service, you must open-source your changes.

  • Free for self-hosted use
  • Modifications must be shared if offered as a service
  • Protects the project from closed-source forks
  • Drives users toward Lightpanda Cloud for hassle-free use

Project Health

23K
GitHub stars
Zig
Primary language

Business model: Open-source core + managed cloud. The AGPL ensures competitors can't take the code and offer a competing cloud service without contributing back.

Key Takeaways

"The fastest code is the code that never runs."

github.com/lightpanda-io/browser • lightpanda.io • AGPL-3.0