Burst Load · Performance

Move the full dataset at the speed of the available system.

Omni Loader uses a massively parallel, in-memory migration engine. It slices large tables across workers, keeps work flowing, and lets you cap bandwidth when the migration shares infrastructure with production traffic.

Why this path matters

Use every useful core without hand-partitioning the job.

Omni Loader profiles the run, divides large tables into concurrent slices, and keeps multiple tables moving at once. More workers increase concurrency until CPU, source reads, network, or target ingestion becomes the limiting resource.

  • ✓Automatic slicing for very large tables
  • ✓One worker per core as a practical starting point
  • ✓Configurable local worker concurrency
  • ✓Progress visible per table and slice

What changes for you

Fast does not have to mean uncontrolled.

A full-speed migration can consume the same network and target capacity that applications need. Set a bandwidth ceiling or change worker allocation so the move advances quickly inside an explicit operating budget.

The payoff

See what the capability changes.

10.5M rows/sec

Measured BigQuery run

A published run demonstrates the upper end of a favorable target path; your source and target set the real boundary.

N-way

Parallel table slices

Workers process independent ranges instead of waiting on one serial reader.

In memory

No framework spill path

The engine was built for direct movement rather than layered onto a general ETL runtime.

Where the work happens

The details that make the result usable.

At a glance

A visual summary of the work.

Illustration of a high-performance server cluster around Omni Loader

Measure the whole path

Benchmark boundaries stay visible.

Parser, source-to-spool, and target-ingestion figures answer different questions. Size the deployment with a representative source, target, table shape, and network path instead of treating one component number as an end-to-end promise.

Take the next step

Test Burst Load on the path you actually need.

Bring a representative source, target, and table shape. We will identify the active bottleneck and show the measured migration boundary on your data.

Plan my migration