Discussions
API throughput limits & extraction patterns for Location entity
6 hours ago by Jayanth Koye
Hi,
We're working on onboarding the Corrigo Location entity to support asset tagging. During our investigation we've hit some constraints and want to understand what's possible on the API side.
Context:
- We currently extract WorkOrder data via api/v1/query/WorkOrder using time-based partitioning (24-hour windows), which works well for incremental daily loads.
- For Location(api/v1/query/Location), there's no date/timestamp field to partition on, and the total record count is ~1.73M
Questions:
- Change detection / delta support - Is there any mechanism (last-modified timestamp, change feed) that could tell us which Location records changed since a given point in time? This would
allow incremental loads instead of full extractions. - TPS / rate limits - What are the current transactions-per-second limits on the querying Corrigo endpoints? Are there different limits for read-heavy bulk extraction vs. normal operational queries?
- Alternative endpoints — Beyond api/v1/query/Location, are there other endpoints or export mechanisms (bulk export) that are better suited for extracting location data (Our current scope is to extract the TypeId: 'Unit', 'Equipment', but might be extended based on future requirements)?
- Concurrent connections — How many parallel API sessions/connections can we sustain without being throttled? We're evaluating a hash-bucketing strategy with range-based filtering in none of the above asks are not possible:
- We've confirmed that Location.Id supports > and < operators. Can we reliably use ID-range bucketing (e.g., Id > 0 AND Id < 100000, Id > 100000 AND Id < 200000, etc.) for parallel
extraction? Are there any gotchas with ID gaps or non-sequential IDs?
- We've confirmed that Location.Id supports > and < operators. Can we reliably use ID-range bucketing (e.g., Id > 0 AND Id < 100000, Id > 100000 AND Id < 200000, etc.) for parallel
What we're trying to solve:
Daily full-load of 1.7 million Location records isn't viable with sequential pagination at current response limit and times. We need either (a) a way to parallelize extraction safely, or (b) an incremental/delta mechanism to only pull changes.
Thanks,
Jayanth

