Qeasy Cloud
Get Started

Practical Tutorial for the Query Xiaoman Products Strategy: Pulling Master Data from CRM into the Middleware Layer

· 卢剑航· Integration Solutions· 12 views· 4 min read
小满OKKICRM鼎捷t100策略教程物料主数据QUERY_ONLYIncremental Sync轻易云CRM与ERP集成

What This Strategy Solves

In a real engagement with a retail-and-manufacturing customer, the CRM held the authoritative product master data while the downstream ERP was the source for all subsequent business documents. Once product codes diverged between the two, orders, inventory and pricing all silently went wrong. The first thing we did on the Qeasy integration platform was to pull the product data from the CRM into a middleware layer in a stable, repeatable way. The strategy named Query Xiaoman Products plays exactly this role — it writes nothing to a target system, only fetches data through an incremental window.

Data Flow and Field Mapping

The data flow is clean: Xiaoman OKKI CRM → Qeasy Integration Platform (middleware) → waiting for the next write strategy to land in the downstream ERP. This strategy only covers the first half.

DimensionSource (Xiaoman OKKI CRM)Target (Qeasy Middleware)
API path/v1/product/list (GET)Write no-op (POST)
EffectQUERYEXECUTE
Business keyproduct_no0 (placeholder, mapped by next strategy)
Idempotency checkidCheck disabledidCheck enabled
Auto-fill responseautoFillResponse=true—

The request is a typical page-plus-time-window payload: start_index, count, start_time, end_time, removed. start_time and end_time are the scheduling core and are bound to {{LAST_SYNC_TIME|datetime}} and {{CURRENT_TIME|datetime}} so that each run only takes changes since the last successful sync.

How to Configure on Qeasy

On the Qeasy data integration platform, configure this strategy with the following points:

  1. Create the strategy using the Query template. Set the source platform to Xiaoman OKKI CRM and the target platform to the Qeasy platform itself.
  2. API metadata follows the source JSON: /v1/product/list, GET, product_no as the business key, idCheck kept off.
  3. Request parameters are maintained in the request array: hard-code start_index=1, count=20, removed=0; bind start_time and end_time to the platform's built-in time variables.
  4. Response handling turns on autoFillResponse so the response is materialized as an observable data view, which is invaluable for troubleshooting.
  5. Target side selects the Write no-op API, which lands the data in the middleware layer instead of writing directly to the ERP. This is a common pattern on customer sites: land first, distribute later. Centralized code-mapping management is kept in the downstream strategy rather than mixed into the first one.
  6. Scheduling is set per crontab: source at 03:02, target at 02:23 — staggered to avoid overlap.

Implementation Steps

We recommend a three-phase rollout: incremental start point, full backfill, schedule frequency.

  • Incremental start point: the first run should not rely on LAST_SYNC_TIME. Manually set an explicit starting date (for example, "2024-01-01 00:00:00") to avoid scanning the entire table on day one.
  • Full backfill: add a one-shot manual trigger on Qeasy for initial historical data. After the full backfill is complete, switch back to incremental-window mode — this is the typical dual-track approach of incremental plus full.
  • Schedule frequency: once a day in the early morning is enough. CRM product master data rarely changes more than a few times per day, and over-scheduling only burns API quota.
  • Dependencies and sequence: in the orchestration, this strategy is sequence A; the downstream write-to-ERP strategy is sequence B with depends_on A, forming a clear chain.

Pitfalls Recap

  1. Using CURRENT_TIME for start_time: a classic mistake is to set both start_time and end_time to the current time, which returns an empty window every time. The safe approach is start_time = last successful time, end_time = current time.
  2. Pagination not fully traversed: count=20 is the default and will silently lose records once the product catalog grows. Configure a paging loop in Qeasy until the returned record count is smaller than count.
  3. Ignoring the removed field: on customer sites, we often see reconciliation disputes where a product is deleted in the CRM but still in use in the ERP. Always run removed=0 and removed=1 in separate windows and periodically sync deletion events.
  4. Writing directly to the downstream ERP in this first strategy: this strategy is intentionally limited to landing in the middleware. Forcing an ERP write into the very first query strategy mixes code-mapping, field trimming and retry logic together and becomes unmaintainable.
  5. Scheduling collisions: source at 03:02 and target at 02:23 are deliberately staggered to give the landing enough time. When both are set to the same minute, the downstream strategy will pick up data that has not yet finished landing, producing mysterious mismatches.

When to Use and When Not to Use

Use when: master data lives in the CRM and needs to be stably pulled into a middleware layer by a time window, with a separate write strategy distributing to the ERP afterward. Do not use when: real-time sync is required (use event triggers instead of polling); the source itself is already a middleware and no second landing is needed; or the business is highly sensitive to deletion events and must use a subscription API rather than polling.

Original content. Please credit the source when reposting: https://www.qeasy.cloud/insights/solutions/strat-okkicrm-t100-0138-nd8a66a8e-b16332b9

Comments