Material Master Query Strategy in Practice: Encoding Mapping and Incremental Scheduling from Source System to Middleware
What This Strategy Solves
In supply chain integration between Jushuitan and Kingdee YXC, material master data is the root of all downstream documents. Inventory quantities, unit prices, unit conversions, purchase receiving, and sales outbound all hang on the material code. If the source system adds or modifies an item but the target system does not sync in time, the warehouse may ship the wrong goods the next day. The core objective of this strategy is to pull material codes from the source system into the middleware under defined conditions, complete encoding mapping for consumption by subsequent sync strategies, and ensure consistency of foundational data on both sides from the source.
Data Flow and Field Mapping
The overall flow is "Source System (Jushuitan) → Middleware (Qeasy Data Integration Platform) → Target System (Kingdee YXC)." The material query strategy only covers the first two steps, i.e., pulling from the source and landing into the middleware; the target-side write is triggered by subsequent sync strategies.
The key field mapping table is as follows:
| Business Meaning | Source Field (Jushuitan) | Middleware Field (Qeasy) | Remarks |
|---|---|---|---|
| Material code | number | material_code | Primary key, cannot be empty |
| Material name | name | material_name | Used for fuzzy search |
| Specification | spec | spec_model | Optional |
| Barcode | barcode | barcode | Optional |
| Item category | category_id | category_id | Linked to category mapping table |
| Enable status | enable | enable | 1 enabled / 0 disabled |
| Create time | create_time | create_time | Incremental start field |
How to Configure on Qeasy
First, register the source platform (Jushuitan) and target platform (Kingdee YXC) connection information in the Qeasy Data Integration Platform, selecting the corresponding adapter for each platform type.
Key configuration of the strategy body:
- API and Method: Call the source system material query interface, method is GET, effect is set to QUERY.
- Pagination and Cursor: The source side usually returns by page; you need to configure the two request fields page_no and page_size, and pass the total and current page count in the response to the paginator until completion.
- Filter Parameters:
enableis fixed to "1", only enabled materials;create_start_timeandcreate_end_timeuse incremental timestamps. - Target Action: The target of this strategy is "write empty operation" (WebAPI, POST), used to trigger middleware landing, not directly writing to the target system.
- Centralized Encoding Mapping: In Qeasy's mapping configuration, map Jushuitan's category_id to Kingdee YXC's item category ID. All downstream strategies share the same mapping table, minimizing subsequent maintenance.
Implementation Steps
We typically proceed in three phases at the customer site:
Phase 1: Full trigger, establish the baseline. On the first run, no timestamp is passed; pull all enabled materials from the source into the middleware in one go, completing the first round of encoding mapping. Closely monitor whether pagination is complete, and whether there are duplicates or missing records during execution.
Phase 2: Switch to incremental start point. Use the first full run as the baseline and record the maximum value of create_time as the incremental start point. After that, only pull materials created later than that value each time. A typical scheduling frequency is set during business off-peak hours, such as early morning.
Phase 3: Scheduled execution and monitoring. Configure the scheduling cycle. The sample given in the source material is */10 7-21 * * *, i.e., runs every 10 minutes from 7:00 to 21:00 every day, covering the real-time requirements during business hours. After middleware landing, the actual downstream write strategies trigger on their own rhythm, achieving "decoupling of query and write."
Another common response pattern among Qeasy customers: make the full trigger and incremental scheduling two independent strategies, run incremental normally, and run a full verification once at the beginning of each month as a safety net.
Pitfall Review
- Inconsistent timestamp units. The source side sometimes returns seconds and sometimes milliseconds; direct comparison can easily miss data. The safe approach is to use a script in Qeasy to uniformly convert to seconds, then perform difference comparison.
- Misuse of fuzzy search parameter. The
searchfield does fuzzy matching on name, code, specification, and barcode. If historical keywords are also passed in incremental scheduling, historical data will be pulled again. It is recommended to leave search empty in incremental scenarios and rely only on timestamp filtering. - Insufficient enable status filtering. If source-side disabled materials (enable=0) are also synced to the target side, downstream document creation will fail. A typical pitfall is forgetting enable=1 in the first version.
- Pagination and total count mismatch. Some interfaces have missing or lagging total fields, and pagination will terminate early. It is recommended to add a maximum page protection in Qeasy's pagination configuration to avoid infinite loops or premature termination.
- Encoding mapping scattered across multiple strategies. If each downstream strategy maintains its own category mapping, changing one category requires modifying a dozen places. Centralized mapping is the standard for material-type integration.
Applicable and Non-Applicable Scenarios
Applicable: Multiple business systems coexist, source side is the single source of truth for item master data, target side requires stable material codes as the root of downstream documents. Not applicable: Both sides are allowed to freely create materials (no single source of truth), or the material volume is extremely small (within a few dozen), making it not worth building an independent query strategy.