FastTransfer Configuration

MigratorXpress delegates all data movement to FastTransfer (>= 0.15.0 required), which uses a streaming architecture. Rows flow directly from source to target without being buffered entirely in memory, so tables with billions of rows can be transferred on modest hardware.

FastTransfer Location

Point --fasttransfer_dir_path to the directory containing the FastTransfer binary. Since v0.6.25, if this flag is omitted, MigratorXpress auto-detects a bundled FastTransfer in the engine/ subdirectory next to the binary. Trial distributions ship with FastTransfer pre-bundled.

Key Parameters

Parameter	Type	Default	Description
--fasttransfer_dir_path	Path	Auto-detected	Directory containing the FastTransfer binary
--fasttransfer_p	Integer	1	Number of parallel streams within FastTransfer for large table partitioning
--ft_large_table_th	Integer	250,000,000	Row count threshold above which a table is eligible for parallel transfer
--n_jobs	Integer	1	Number of tables to transfer simultaneously

Dual-Level Parallelism

MigratorXpress supports two levels of parallelism:

• --n_jobs controls how many tables are transferred concurrently.
• --fasttransfer_p controls how many parallel streams each large table uses internally.

For example, --n_jobs 4 --fasttransfer_p 2 transfers 4 tables simultaneously, and any table above the large table threshold is split into 2 parallel streams.

Parallel Transfer Methods

Each source database uses a different method to partition tables for parallel transfer:

Source Database	Method	Key Column Required	Description
PostgreSQL	Ctid	No	Splits by physical tuple ID
Oracle	Rowid	No	Splits by physical ROWID; requires SELECT_CATALOG_ROLE
Netezza	NZDataSlice	No	Splits by internal data slice distribution
SQL Server	Standard	N/A	Standard transfer without physical partitioning

Method Selection Logic

MigratorXpress selects the parallel transfer method automatically based on the source database type. No manual configuration is needed. If --fasttransfer_p is set to 1 or the table row count is below --ft_large_table_th, the table is transferred as a single stream regardless of the source type.

Performance Tuning

PostgreSQL Sources

Ctid-based parallelism works well with minimal configuration. Increase --fasttransfer_p for tables with hundreds of millions of rows. Start with 2-4 parallel streams and scale up based on available CPU and I/O bandwidth.

Oracle Sources

Rowid-based parallelism requires SELECT_CATALOG_ROLE (see Oracle configuration). Oracle parallel transfers benefit from higher stream counts on large tables. Consider 4-8 streams for tables exceeding 100M rows.

Netezza Sources

NZDataSlice parallelism aligns with Netezza's internal data distribution. Set --fasttransfer_p to match or be a factor of the number of data slices on the appliance.

SQL Server Sources

SQL Server uses standard (non-partitioned) transfers. Parallelism is achieved at the table level via --n_jobs. For SQL Server sources, increase --n_jobs rather than --fasttransfer_p.

SQL Server Target Options

When SQL Server is the target database, MigratorXpress supports additional options for columnstore indexes and foreign key handling.

Columnstore Indexes

Parameter	Type	Default	Description
--cci_threshold	Integer	1,000,000	Create Clustered Columnstore Index for tables above this row count
--aci_threshold	Integer	100,000,000	Use archive compression (COLUMNSTORE_ARCHIVE) above this row count

Tables >= cci_threshold but < aci_threshold get a standard CCI. Tables >= aci_threshold get a CCI with DATA_COMPRESSION = COLUMNSTORE_ARCHIVE. Tables below cci_threshold remain as standard rowstore tables.

Foreign Key Mode

Parameter	Type	Default	Description
--fk_mode	String	disabled	FK enforcement: disabled (created with NOCHECK and disabled), untrusted (NOCHECK but enabled), trusted (full enforcement)

Row Count Options

Parameter	Type	Default	Description
--compute_nbrows	Boolean	false	Use COUNT(*) for exact row counts instead of database estimates
--without_xid	Flag	false	Disable transaction ID usage in Netezza and Oracle

Configuration Examples

Small Migration (< 10 tables, < 1M rows total)

./MigratorXpress --auth ./credentials.json \
  --source_db_auth_id oracle_prod \
  --source_schema_name HR \
  --target_db_auth_id postgres_dev \
  --target_schema_name hr \
  --fasttransfer_dir_path ./FastTransfer_linux-x64_v0.12.4/ \
  --migration_db_auth_id ms_mig_log \
  --task_list translate create transfer diff

Default parallelism is sufficient. Single-stream transfer per table, one table at a time.

Medium Migration (10-100 tables, up to 100M rows)

./MigratorXpress --auth ./credentials.json \
  --source_db_auth_id oracle_prod \
  --source_schema_name SALES \
  --target_db_auth_id postgres_dev \
  --target_schema_name sales \
  --fasttransfer_dir_path ./FastTransfer_linux-x64_v0.12.4/ \
  --migration_db_auth_id ms_mig_log \
  --n_jobs 4 \
  --fasttransfer_p 2 \
  --task_list translate create transfer diff

Four tables in parallel, large tables split into 2 streams each.

Large Migration (100+ tables, billions of rows)

./MigratorXpress --auth ./credentials.json \
  --source_db_auth_id oracle_prod \
  --source_schema_name WAREHOUSE \
  --target_db_auth_id postgres_dev \
  --target_schema_name warehouse \
  --fasttransfer_dir_path ./FastTransfer_linux-x64_v0.12.4/ \
  --migration_db_auth_id ms_mig_log \
  --n_jobs 8 \
  --fasttransfer_p 4 \
  --ft_large_table_th 500000 \
  --task_list translate create transfer diff

Eight tables in parallel, large tables (> 500K rows) split into 4 streams. Adjust thresholds based on hardware capacity.

Monitoring Performance

During and after a migration, you can query the migration tracking database to check progress and identify slow tables.

Check Overall Progress

SELECT
    task_name,
    COUNT(*) AS total_tables,
    SUM(CASE WHEN status = 'completed' THEN 1 ELSE 0 END) AS completed,
    SUM(CASE WHEN status = 'running' THEN 1 ELSE 0 END) AS running,
    SUM(CASE WHEN status = 'failed' THEN 1 ELSE 0 END) AS failed
FROM migration_tasks
WHERE run_id = 'YOUR_RUN_ID'
GROUP BY task_name;

Identify Slow Tables

SELECT
    table_name,
    task_name,
    DATEDIFF(SECOND, start_time, ISNULL(end_time, GETDATE())) AS duration_seconds,
    row_count
FROM migration_tasks
WHERE run_id = 'YOUR_RUN_ID'
    AND task_name = 'transfer'
ORDER BY duration_seconds DESC;

Troubleshooting

FastTransfer binary not found: Verify the --fasttransfer_dir_path points to the correct directory containing the FastTransfer executable. Check file permissions and ensure the binary is executable (chmod +x FastTransfer on Linux).

Transfer stalls on large tables: Reduce --fasttransfer_p if the source or target database is under heavy load. Check network bandwidth between the MigratorXpress host and the databases.

Out of memory errors: FastTransfer's streaming architecture normally prevents memory issues. If they occur, reduce --n_jobs to lower the number of concurrent table transfers.

Slow transfer throughput:

• Increase --fasttransfer_p for large tables.
• Increase --n_jobs if the bottleneck is small-table overhead rather than individual table speed.
• Check that the network path between source and target is not saturated.
• Verify that the source database is not under heavy concurrent load.

See Also

• Migration Tasks -- Task execution order and combinations
• Database Connections -- Auth file configuration
• Quick Start Guide -- Getting started with MigratorXpress
• CLI Reference -- Full command-line reference