Querying Wangdiantong Goods Archive: An Incremental Sync Strategy with Qeasy
What this strategy solves
The goods archive is the foundation of any supply-chain integration: every downstream document, stock movement and costing record depends on it. In one real engagement, the customer wanted to pull Wangdiantong's goods master into Kingdee Cosmos incrementally by last-modified time, so that purchasing, sales and inventory could share a single source of truth. A 'just pull the goods' job sounds trivial, but in production you must answer three hard questions — which time window, how to paginate, and where to stop without losing or duplicating rows.
Data flow and field mapping
The flow is unidirectional: Wangdiantong · QiMen → Qeasy Integration Platform → Kingdee Cosmos. The Qeasy platform acts as both the staging buffer and the execution point for scheduling and mapping.
The source wdt.goods.query is a QUERY-type WebAPI. In the request, start_time is bound to {{LAST_SYNC_TIME|datetime}}, end_time to {{CURRENT_TIME|datetime}}, and pagination is controlled by page_size. The target metadata is a 'write no-op' — this is intentional: the actual write into Kingdee is performed by a downstream 'write goods' strategy. This strategy's job is only to fetch from the source, land the data in the staging layer, and stamp the time window.
Key field mapping:
| Meaning | Source (Wangdiantong) | Staging | Target (Kingdee) |
|---|---|---|---|
| Goods number | goods_no | goods_no | Mapped by downstream strategy |
| Window start | start_time | LAST_SYNC_TIME | — |
| Window end | end_time | CURRENT_TIME | — |
| Include deleted | deleted | deleted_flag | — |
| Page size | page_size | page_size | — |
We recommend keeping all code mappings in Qeasy's central mapping center instead of duplicating them per strategy.
How to configure it on Qeasy
- Connection: Create a source connection in the Qeasy Data Integration Platform, choose the 'Wangdiantong · QiMen' adapter and fill in the credentials. Qeasy ships adapters for this kind of ERP/e-commerce backend, so you do not need to hand-roll an HTTP client.
- Metadata modeling: Register every field in the
wdt.goods.queryrequest as JSON metadata. Turn onidCheck, leavebuildModeloff (query APIs normally do not need a document model), and turn onautoFillResponseso the response shape is auto-detected. - Variable binding: Bind
start_timeto{{LAST_SYNC_TIME|datetime}}andend_timeto{{CURRENT_TIME|datetime}}. This single line is the soul of the incremental logic — get the boundaries wrong and you lose or duplicate rows. - Stub the target: The target being a 'write no-op' is a deliberate design choice. It keeps the source query pipeline independent and lets a downstream strategy consume the staged data.
- Logging and alerts: Enable run-log retention, retry on failure and exception alerts inside Qeasy — for a high-volume goods archive they are lifesavers.
Implementation steps
We usually roll out this kind of strategy in three phases:
- Initialize the incremental anchor: Before go-live, set
LAST_SYNC_TIMEto a historical point (e.g. three months back) and run a one-off full reload so the baseline archive is flat. - Trigger the full load: Manually trigger a full run from Qeasy and verify the target side is complete. A common Qeasy pattern is to keep 'incremental + full reload' on parallel tracks: full reload as a safety net, incremental as the daily driver.
- Switch on the schedule: Set the crontab to
0 */2 * * *, i.e. roll a new window every two hours. Do not run too often (it competes with the source's business peak) nor too rarely (downstream write latency grows).
For the first two weeks after go-live, check three numbers daily: rows in this window, cumulative rows, and rows in error.
Lessons learned
- Typical mistake: using
NOW()instead of the platform variable forend_time.NOW()drifts across time zones and nodes, causing the same batch to be pulled by two consecutive windows. The safe choice is Qeasy's{{CURRENT_TIME|datetime}}. - Where it tends to break: coupling between pagination and the time window. If pagination errors mid-way and the window has already moved forward, the next run skips that segment. Enable retry + window rollback in Qeasy rather than naively advancing
LAST_SYNC_TIMEtoCURRENT_TIME. - Typical mistake: dropping the 'deleted' flag as noise.
deleted=1in Wangdiantong means 'include deleted', so keep adeleted_flagin staging and let Kingdee decide whether to disable or hard-delete. Otherwise the two sides diverge within three months. - Pitfall: misreading the 'write no-op' target. Many engineers first see this metadata and think the configuration is broken. In fact it is a typical pattern of 'header/body staged separation' — query and write are decoupled for reuse.
- Pitfall: crontab semantics vs. window semantics.
0 */2 * * *fires on the hour, but data is consumed by window. If a batch lands near a window boundary, leave at least one schedule cycle of slack.
When to use and when not to use
Use it when the upstream exposes a last-modified-time query API (Wangdiantong, most ERP master-data APIs) and the downstream needs a stable, time-ordered staging feed. Do not use it when the upstream only supports full export without pagination, or when the downstream requires hard real-time (sub-second) consistency — in those cases switch to change-notification or message-subscription patterns.