What changes for you
Use timestamped output, clear-before-load, folder or table cleanup, overwrite behavior, and clear-after-load according to the migration workflow.
Cloud and warehouse · Staging
Prepare data in parallel. Let the destination ingest natively.
Omni Loader can transform rows into intermediate files, stream them to object storage, and invoke target-side ingestion or stop after file creation when the files are the deliverable.
Cloud staging pipeline Encode · compress · upload
Ingestion queued
Inputs
Source tablesREADY
Selected columnsREADY
Mapped valuesREADY
File pipeline Encode · compress · upload
Outputs
CSVREADY
ParquetREADY
AvroREADY
The job to be done
Move the full dataset without turning failure into a restart.
A large migration is not finished when the first rows arrive. Omni Loader keeps parallel work productive, adapts the loading path to the target, and records enough progress to recover the part that failed.
Write to Amazon S3, Azure Blob Storage, Azure Data Lake Storage Gen2, Fabric Lakehouse, Google Cloud Storage, or local storage.
Why this path matters
Stage where the target can ingest efficiently.
Write to Amazon S3, Azure Blob Storage, Azure Data Lake Storage Gen2, Fabric Lakehouse, Google Cloud Storage, or local storage.
The outcome
Tell us the destination, staging location, format, compression, and cleanup policy.
The payoff
See what the capability changes.
3 formats
CSV · Parquet · Avro
Choose the intermediate shape the target expects.
6 locations
Cloud or local staging
Use S3, Azure, Fabric, GCS, or local storage.
Bounded
Streaming backpressure
Keep producers from outrunning uploads indefinitely.
Where the work happens
The details that make the result usable.
Read each capability as part of the operating path. The important question is not whether a feature exists, but what work it removes and what evidence it leaves behind.
At a glance
A visual summary of the work.
Warehouse targets can ingest an intermediate representation and complete conversion inside the target, avoiding slow row-by-row loading.
Storage choices
Stage where the target can ingest efficiently.
Write to Amazon S3, Azure Blob Storage, Azure Data Lake Storage Gen2, Fabric Lakehouse, Google Cloud Storage, or local storage.
- CSV, Parquet, and Avro
- Gzip and Snappy options
- Row-count file splitting
- Configurable path layouts
Lifecycle control
Decide what happens before and after the load.
Use timestamped output, clear-before-load, folder or table cleanup, overwrite behavior, and clear-after-load according to the migration workflow.
Intermediate remapping
Separate file schema from final warehouse schema.
Warehouse targets can ingest an intermediate representation and complete conversion inside the target, avoiding slow row-by-row loading.
Before you choose
Check the path against your environment.
Use the page-specific details below as a short discovery checklist. They are the conditions and work areas that shape the capability, not generic product promises.
- CSV, Parquet, and Avro
- Gzip and Snappy options
- Row-count file splitting
- Configurable path layouts
The path to confidence
See the work. Make the call. Prove the result.
The page keeps the product detail close to the decision it supports. Your team can see what the software handles automatically, what it changes for the target, and what the result leaves ready to operate.
Stage where the target can ingest efficiently.
Write to Amazon S3, Azure Blob Storage, Azure Data Lake Storage Gen2, Fabric Lakehouse, Google Cloud Storage, or local storage.
Decide what happens before and after the load.
Use timestamped output, clear-before-load, folder or table cleanup, overwrite behavior, and clear-after-load according to the migration workflow.
Separate file schema from final warehouse schema.
Warehouse targets can ingest an intermediate representation and complete conversion inside the target, avoiding slow row-by-row loading.
Keep the plan connected
Take the next useful step.
A capability becomes easier to operate when the next decision follows from the constraint you just uncovered. Explore the related work that completes this part of the path.
Your working set
What this page leaves you with
- CSV · Parquet · Avro
- Cloud or local staging
- Stage where the target can ingest efficiently.
- Decide what happens before and after the load.
- Feed the warehouse the way it wants to ingest.
Take the next step
Feed the warehouse the way it wants to ingest.
Tell us the destination, staging location, format, compression, and cleanup policy.
Plan my migration