YiDa Customer Source Lookup Platform: A Practical Guide to Customer Master Data Sync with Qeasy

What This Strategy Solves

Where did a customer come from, who registered them, and which category do they belong to — these questions come up almost daily during customer operations reviews. In one real project, a retail enterprise recorded customer source information in a DingTalk YiDa form, while downstream business teams needed to directly look up 'customer ID ↔ source category' inside the integration platform. The two systems were not connected, so operations had to manually take screenshots every time — inefficient and error-prone. The 'YiDa Customer Source Lookup Platform' strategy is designed to sync customer source master data from the YiDa form into the integration platform on a scheduled basis, forming a traceable, queryable customer profile view.

Data Flow and Field Mapping

The flow is one-way: DingTalk YiDa (source) → Qeasy Data Integration Platform (target). The source side calls the YiDa open API to pull form instances, and the target is an internal execution endpoint (WebService) on the platform that writes the records into the lookup view.

Key field mapping:

The code and name come from the same field to preserve the 'code as name' display habit. If your organization has a dedicated name field, simply replace b_label with the corresponding field.

How to Configure on Qeasy

In the Qeasy Data Integration Platform, this strategy takes the shape 'Source WebAPI (pull) → Platform internal execution endpoint (write)'. Key configuration points:

• Source connector: Select the Dingtalk platform, interface type 'WebAPI', request method POST, and fill in the required parameters: appType, formUuid, pageNumber, pageSize, systemToken. Credentials like systemToken should be stored in Qeasy's credential management, not hard-coded in the strategy.
• Pagination and rate limiting: The YiDa API defaults to a page size of 50. If the form has a large volume of data, consider increasing pageSize to the platform's maximum to reduce pull rounds. Pay attention to the platform's QPS limits — Qeasy's scheduler can be configured with a maximum calls-per-minute value to avoid throttling.
• Target execution endpoint: The target is a WebService on the platform side with effect set to EXECUTE, which writes the records into the lookup view.
• Field mapping: Use Qeasy's 'visual field mapping' to map source fields to the target fields a_value, a_label, b_value, b_label, category. Since a_value is a date type, make sure to convert YiDa's string to a date in Qeasy's type conversion.
• Centralized code mapping management: Enum values like customer source categories are best managed through Qeasy's 'code mapping' feature rather than scattered across each strategy. Adding a new source category otherwise means editing many strategies — a typical pitfall.

Implementation Steps

We recommend breaking the implementation into three phases:

Phase 1: Incremental starting point and baseline collection. Pull the source side's full data once as a baseline and write it into the lookup view. This uses a one-time full trigger, not regular scheduling. After the baseline run, use 'updated time > last sync time' as the incremental starting point.

Phase 2: Scheduled jobs go live. The crontab in the strategy is 0 0 * * *, but in actual customer projects we generally recommend a dual-track approach: 'incremental at high frequency, full sync at low frequency'. Run incremental syncs every 10–30 minutes, and the full sync once daily as a safety net outside the incremental window. This ensures timeliness while avoiding missed records.

Phase 3: Exceptions and reconciliation. Qeasy's runtime logs record the success/failure status of each record. For failed records, first check whether it's an authentication issue on the source side or a write failure on the target side, then decide whether to re-run or fix the data manually. Perform a weekly 'row count reconciliation' between the two sides to prevent long-term drift.

Lessons Learned

1. systemToken expiration is silent. YiDa's app keys do not notify before expiration, but they may suddenly fail after permission changes. Add a 'heartbeat probe' strategy in Qeasy that pulls a fixed record daily as a health check and raises an alert on failure.
2. Pagination data loss. If new data is inserted during incremental pulls, the YiDa paginated API may produce duplicates or misses across pages. The safe approach is to use formInstanceId + updated time as the unique key, and write with 'upsert' semantics in Qeasy rather than pure insert.
3. a_value date type mismatch. YiDa returns time fields as strings. Writing them directly into a date target field will be rejected. Always go through a date converter during mapping and specify the timezone.
4. Hard-coded category ID. If customer source categories change in the future, hard-coding them as string constants becomes hard to maintain. The best practice is 'source field + code mapping' — category changes only update the mapping table, not the strategy.
5. Lookup view query performance. After a large write volume, the lookup view gets slower without proper indexes. Confirm that appropriate composite indexes (recommended on code + date) are built in the downstream view of Qeasy's target endpoint.

Suitable and Unsuitable Scenarios

Suitable: master data like customer source or customer tags with 'single record, few fields, infrequent changes' that need cross-system lookup and tracing. Unsuitable: high-volume, high-frequency transactional data such as orders and inventory — those scenarios should use streaming or dedicated sync channels rather than scheduled polling.