Shop-to-Customer Sync: A Practical Incremental Strategy with idCheck
What This Strategy Solves
In e-commerce retail companies, shop master records are usually created in an ERP-adjacent mid-platform first, then settled into the ERP customer archive with "the shop as the customer entity." In one real project we discovered that pushing all shops every day resubmitted thousands of historical records, producing duplicate audit entries on the target side and breaking reconciliation between codes on both sides. Collapsing the strategy to "incremental sync by shop_no, skip anything already synced" immediately tamed the issue. That is the core pain point this Shop-to-Customer strategy addresses: keep master data consistent with the smallest possible data flow.
Data Flow and Field Mapping
The flow is straightforward and unidirectional: source (shop archive) → Qeasy Data Integration Platform middleware (deduplicate, map, fill constants) → target (customer archive).
| Target Field | Source Field / Rule | Mapping Type | Description |
|---|---|---|---|
| FName | shop_name | DIRECT | Customer name comes from shop name |
| FNumber | shop_no | DIRECT | Customer code uses shop number directly |
| FCreateOrgId | Constant "100" | CONSTANT | Fixed create org |
| FUseOrgId | Constant "100" | CONSTANT | Fixed use org |
| FDescription | Constant "" | CONSTANT | Description left empty |
The source API is a shop query (QUERY); the target API is a customer batch save (EXECUTE). This strategy does not require cross-strategy lookups, so _findCollection or _mongoQuery are not needed.
How to Configure on Qeasy
In the Qeasy platform, this strategy is essentially a "source QUERY → target EXECUTE" chain. Key configuration points:
- Source metadata: Select the shop query API (QUERY). Set both number and id to shop_no, and set idCheck to true—this is the switch that prevents already-synced data from being resent.
- Target metadata: Select the batch save API (EXECUTE). FormId is fixed to BD_Customer, Operation is fixed to BatchSave, and IsAutoSubmitAndAudit is set to true.
- Field mapping: Configure DIRECT and CONSTANT mappings as in the table above; organization IDs use constants. For multi-org scenarios, we recommend converting FCreateOrgId into a COLLECTION mapping that resolves from platform_id or sub_platform_id to the corresponding ERP org code.
- Centralized code mapping: A common Qeasy customer pattern is to maintain a separate "shop → customer" mapping table. When the org splits or new shop types appear later, you only add a row to the mapping table—no need to touch the strategy itself.
Implementation Steps
We recommend a three-step rollout for a steady rhythm.
Step 1: Initialize the incremental baseline. Before the first run, manually create or import every "shop-as-customer that should exist in ERP" on the target side so idCheck has a reference baseline. The platform's first run only fills gaps and does not push historical shops again.
Step 2: Trigger the full-volume go-live. In Qeasy, set the source schedule to */56 7-20 * * * (every 56 minutes from 7 AM to 8 PM), and offset the target by one minute (*/57) to avoid same-second concurrency. Observe two full schedule cycles and confirm that the "skipped" and "created" counts in the logs match expectations.
Step 3: Steady-state scheduling and monitoring. Keep the schedule at */56, focusing on two alert types: empty shop_no in the source response causing empty FNumber, and IsAutoSubmitAndAudit rollback failures on the target. Run incremental and full-volume in dual tracks: incremental is driven by idCheck, and full-volume is a manual monthly trigger as a safety net.
Pitfall Retrospective
- idCheck field is set incorrectly. A typical mistake is setting number to shop_id and id to shop_no—the platform compares two different fields, effectively disabling dedup. The safe approach is to fill number and id with the same business key: shop_no.
- Hard-coded org IDs break down under multi-org. It works with a single org at go-live, but six months later the business splits orgs and every customer lands in org 100, unnoticed. Improve by using COLLECTION mapping for the org from day one.
- IsAutoSubmitAndAudit mistakenly left on. If the customer requires draft status and manual review before audit, you cannot default to true. With this strategy, however, leaving auto-audit on together with idCheck is highly efficient—the business boundary should decide.
- No baseline initialization at go-live. Source has 8,000 shops, target is empty; idCheck finds none of them "existing," and the full batch floods in, triggering a batch save performance alert. Initializing the baseline first avoids this trap.
- Header-body layered template misused. The shop archive is a flat structure on the source side, but some teams reflexively wrap it into a header-body template, which breaks dedup logic. The customer archive is a header-only document—keep this strategy as a single-layer mapping and do not force a multi-layer pattern.
Applicable and Non-applicable Scenarios
Applicable: e-commerce mid-platform shops pushed to ERP master data as customers; single-org or multi-org with enumerable orgs; shop count stable with primarily incremental changes. Non-applicable: scenarios that require syncing shop contact, address, or disabled status to ERP customer extension fields (requires extended DIRECT mapping); and advanced scenarios that need cross-strategy lookup (e.g., reverse lookup of related suppliers by shop), which require _findCollection.