Goods-to-Material Integration: A Practical Guide to Incremental Synchronization

What Problem This Strategy Solves

Moving material master data from a business system to a financial and supply-chain platform may look like simple field transport, but poorly designed identifiers, time windows, and save behavior can eventually cause omissions, duplicate materials, or failed updates. In one project, we used the Qeasy Data Integration Platform to support a goods-to-material flow with incremental pulls based on creation time, idempotent target identification by business key, and staggered scheduling, keeping foundational data consistent across systems.

Data Flow and Field Mapping

The data flow is: source goods list → Qeasy pagination, transformation, and record identification → target material batch save. The source uses the goods number as its business number and the goods identifier as an internal identifier. The target uses the material number as its business key. Duplicate checking allows the target to determine whether to create or update a material.

The current strategy covers only core master-table fields. Units, barcodes, and weights can later be added to the basic information entity. A common customer-site approach is to stabilize name, number, and specification first, then add sub-entity fields rather than expanding the entire model at launch.

Coding mappings should be centrally managed in Qeasy. Even when a field is currently mapped directly, its source, transformation rule, and target should remain explicit. When unit-name or material-group mappings are added, a centralized code-mapping table can avoid scattering conversion logic across synchronization records.

How to Configure It in Qeasy

First, configure the source query task. Set the operation to a query, use the goods number as the business number, and use the goods identifier as the record identifier. Start pagination at 0 and configure the page size according to the interface; this strategy uses 50. Filter by the creation-time range from the last successful synchronization time to the current execution time, with consistent formatting and time zone.

Second, configure intermediate processing. The main goal is not to rewrite names but to validate that the number, name, and specification are present before applying direct mappings. Source pagination must be processed as a loop. Never treat the first page as the full result. If a task is interrupted, continue from a reliable watermark so records are not skipped.

Third, configure target saving. Use the material number as the business number and enable idCheck=true, allowing the target to determine whether the record is new or an update. Specify the material form and save operation in the request, and decide whether automatic submission and approval are appropriate for the actual process. Creating and using organization values should come from configuration rather than business data.

Fourth, establish operational monitoring. At minimum, monitor the number of source records read, the number successfully saved, failure reasons, and the last successful synchronization time. Store the latest successful time in Qeasy synchronization state or another reliable store. Do not rerun an arbitrary page using only the current time, because a network interruption could otherwise produce duplicate requests.

Implementation Steps

Before go-live, perform a baseline review of source fields, organizational scope, and target numbering rules. In the test environment, prepare one new and one updated record. Verify that the target creates or updates material records by number and inspect the actual status after automated saving.

Do not initialize the incremental watermark to the system’s current time without a baseline. Trigger a full synchronization first, then use its successful completion time as the incremental watermark. If modification timestamps are available, assess switching to a modification-time incremental strategy. A creation-time strategy can omit historical updates, so this decision must be accepted by the business before activation.

This design uses staggered execution: the source runs every five minutes during the defined business period, while the target executes in a later minute window. The intent is not to claim absolute real-time processing, but to leave time for extraction, pagination, transformation, and saving. If an organization adopts parallel incremental and full loads, full synchronization should be constrained to a business window and avoid peak incremental periods.

After go-live, use phased rollout: limited validation, stable observation, and then broader business coverage. Whenever organization constants, sub-entity fields, or automatic approval rules change, regression-test create, update, and empty-value scenarios before restoring the schedule.

Lessons Learned

1. Using creation time for incrementals while ignoring historical modifications. The typical mistake is assuming that a new integration only creates materials. This is a common source of failure. Decide explicitly whether the flow covers only new records or both new and modified records. If modifications must be included, use modification timestamps and handle boundary times correctly.

2. Processing only the first page. The interface starts at page 0 and limits the page size. Without looping through every page, later goods are permanently omitted. Check the total page count or next-page indicator and retain page-level read status in platform logs.

3. No target-side idempotency by material number. Relying only on a source identifier for direct creation can create duplicate materials over time. Use the material number as the business key, enable duplicate checking, and verify before go-live that updating the same number does not create a second material.

4. Assuming the material is complete because sub-entities are empty. If base unit, barcode, and weight are not mapped, the target may use default values. In phased delivery, separate required fields from optional fields: block execution when required values are missing and issue a warning for optional omissions instead of silently accepting everything.

5. Embedding constants that require manual recall. Hard-coding creating and using organizations makes cross-environment migration error-prone. When Qeasy handles implementation, store such values as auditable configuration and verify them with every release note.

Suitable and Unsuitable Scenarios

This strategy suits supply-chain environments where a business system creates goods while a target platform maintains material records and supports scheduled batch saving. It is especially appropriate for an initial phase that synchronizes only core master-table fields. It is unsuitable for strong real-time bidirectional maintenance, scenarios where historical modifications must appear immediately, or projects that require complete initial synchronization of units, barcodes, weights, images, and other complex sub-entities. Those cases need modification-time incrementals, explicit conflict rules, and phased expansion.