Customer Master Data Sync from a Source ERP Downstream: A Hands-on Breakdown of One QUERY Strategy

What This Strategy Solves

Pushing customer master data from a source system down to a downstream ledger looks like the least technical kind of sync — pull a customer profile table, extract an increment by code or time, write it to the target system, done. But anyone who has actually run this in the field knows: the place that breaks is almost always this kind of job.

The meaning of "customer code" diverges between the two systems, and three months later the numbers no longer match;
After a one-off full pull, the increment's starting point is recorded wrong, causing data loss or duplication;
The customer changes a name or category in the source system, and the target side does not refresh in time.

The strategy we are dissecting here aims to do "query customer profile" properly: on the Qeasy data integration platform, use QUERY mode to paginate from the source system, use modification time as the increment, and land the data in the downstream master data store. The strategy itself is not complex, but it is the dependency for every subsequent transactional sync (orders, inbound, outbound) — if its quality is poor, the whole chain gets dragged down.

Data Flow and Field Mapping

The data flow is unidirectional: source system → Qeasy middleware layer → target ledger. The source side is fundamentally a QUERY action (reading data from the source system), and the target side is a lightweight "write-empty operation" or a write to the target table.

The table below is the key field mapping. In multiple customer engagements we have used this contract with the lowest error rate:

Business meaning	Source field	Middleware field	Target ledger field	Handling notes
Customer code	customerCode	customer_code	Customer code	Pass through as-is, use as the reconciliation key
Customer primary key	customerId	customer_id	Internal customer code	When landing in middleware, build a source-PK → target-PK mapping table
Customer name	customerName	customer_name	Customer full name	Source may have leading/trailing spaces — remember to trim
Customer type	customerType	customer_type	Customer category	Centralize the encoding mapping; do not scatter it across strategies
Start modification time	gmtModifiedBegin	last_sync_time	(N/A)	Use last sync time as the left-closed, right-open window start
End modification time	gmtModifiedEnd	current_time	(N/A)	Use current time at dispatch as the window end

One thing worth emphasizing: encoding mappings must be centrally managed. We have seen teams write a copy of "customer type 1=VIP, 2=normal" inside every strategy. Three months later, the source system adds a new type and every strategy breaks. Qeasy provides a unified mapping dictionary — change in one place, the whole chain updates.

How to Configure on Qeasy

This strategy is configured on the Qeasy data integration platform, typically in four blocks:

1. Source-side QUERY configuration

The source side is a standard QUERY API using POST. Pagination parameters pageIndex / pageSize must be passed; the page size can be set to the source system's upper limit (commonly 50/100/200). The incremental window uses two variables:

{{LAST_SYNC_TIME|datetime}} as gmtModifiedBegin — Qeasy automatically picks up the last successful dispatch time;
{{CURRENT_TIME|datetime}} as gmtModifiedEnd — picks up the current dispatch start time.

idCheck should stay false in the QUERY phase, because this is "query a list by time window," not "query a document by primary key" — enabling the primary-key check here causes false negatives.

2. Middleware field mapping

customerCode from the source passes through as the business key. customerId, the system key, lands in the middleware's id_mapping table. All downstream strategies (orders, inventory) look up correspondences through this table.

3. Target-side "write-empty operation"

This is the most interesting bit of the configuration: the target side is configured with api=写入空操作, type=WebAPI, effect=EXECUTE. Its job is not to actually call an external API — it is to let Qeasy land the middleware data and produce a master data snapshot that downstream strategies can consume. In multiple customer engagements we use this pattern: in the master data sync phase, first "sink" the data into Qeasy's intermediate store, and in the downstream transactional sync phase read from there rather than hitting the source system every time.

4. Scheduling

Source-side pull schedule 0 0,12 * * * (every day at 00:00 and 12:00). Target-side landing schedule 23 2 * * * (02:23 daily, intentionally offset by 30 minutes to give the pull and buffer room). Offsetting the two schedules is the safe pattern.

Implementation Steps

We recommend a three-phase rollout — do not jump straight into full + high-frequency on day one.

Phase 1: Full initialization

Manually trigger once with LAST_SYNC_TIME left empty; the source side paginates the entire dataset in ascending customerCode order.
After the middleware landing, manually sample and reconcile customer counts and key fields on both sides.
In this phase, do not enable target-side writes — only verify "are we pulling correctly."

Phase 2: Increment starting point

Once full reconciliation passes, set LAST_SYNC_TIME to the end time of this full run (note: use the source system time, not our execution time).
Switch to the production schedule, run at low frequency first (e.g. once a day), observe for 3–5 days, confirm no data is lost or duplicated in the increment.

Phase 3: Schedule frequency

Once stable, raise the schedule to 0 0,12 * * * (twice daily) or higher based on how often customers edit records in the source.
This step is incremental and full in parallel: high-frequency increment for daily coverage, weekly full validation, alert immediately on drift.

Pitfall Retrospective

This kind of strategy looks simple but has many failure modes. We pick the four most common:

Using local time for the increment's starting point Classic mistake: setting LAST_SYNC_TIME to "yesterday 00:00 local." If the source system's clock drifts from the dispatcher's clock, you either miss data (source modification time earlier than window start) or re-pull data (within the window but actually unchanged). The safe pattern is to use the gmtModified field returned by the source as the next window's start — in Qeasy, use a variable like {{LAST_SYNC_VALUE}}.
Missing the last page in pagination A common source-side pagination gotcha: pageIndex starts at 0, page size 50, "done" is determined only when the total is reached. If the code mishandles "last page has fewer than 50," data is dropped. We let Qeasy's paginator stop automatically once response.total is fully consumed.
Encoding mappings written inside every strategy As emphasized above: centralize. Each customer has different customer-type enums in the source, some with dozens of values — never scatter them across strategies.
idCheck not toggled correctly on the target side Here the target is a "write-empty operation" and idCheck=true is fine; but many teams leave idCheck on in the QUERY phase, so when pulling by time window the primary key fails to match and Qeasy drops the data. Rule of thumb: idCheck=false in QUERY, idCheck=true in EXECUTE.

When This Applies and When It Does Not

Applies: the source system has a standard paginated customer-profile query API, the target system requires persistence of customer master data, and there are subsequent transactional syncs (orders, inventory) that depend on the master data landed by this strategy.

Does not apply: the source system does not support modification-time-based increments (only full pulls), the customer data volume is extremely small (a few thousand or fewer — manual import is cheaper), or the target system requires real-time delivery but the source has no change-notification mechanism.