Web API Streams
Ce contenu n’est pas encore disponible dans votre langue.
Web API Streams
Section titled “Web API Streams”Overview
Section titled “Overview”A Web API stream is a specific dataset retrieved from a connection. Use streams for resources such as orders, tickets, events, measurements, or inventory snapshots.
How it works
Section titled “How it works”Each stream has extraction settings, a transform plan, a target table, and a sync mode. A full refresh replaces the source’s table contents. An incremental stream uses a cursor or timestamp strategy to fetch only new or changed records when the provider supports it.
Step-by-step
Section titled “Step-by-step”- Open the parent Web API connection.
- Create a stream for one endpoint or resource.
- Test a sample response.
- Review the generated transform plan.
- Choose full refresh or incremental cursor mode.
- Configure the recurring schedule.
- Observe the first Dispatcher-submitted sync in the Job Queue and validate its output before depending on the stream.
Current execution path
Section titled “Current execution path”Web API syncs currently enter the Worker only through the scheduled Dispatcher path. The manual Sync and Full Sync actions in the UI and Sidekick are unavailable while the Worker contract is hardened to resolve the canonical connection and stream server-side.
Use the first scheduled run as the end-to-end validation of the endpoint, transform plan, and target table. A successful sample-response test validates discovery and mapping, but it does not prove that a complete background sync can load the target.
Common mistakes
Section titled “Common mistakes”Do not use incremental mode unless the source provides a reliable cursor, timestamp, or comparable change marker.
Do not treat a failed scheduled sync as only a dashboard problem. Open the stream and the Job Queue to see whether extraction, transform, or load failed.