Bulk: chunked work and partial failure

Bulk divides an item set into bounded chunks and sends each chunk to your action handler. The operation records delivery attempts, aggregate progress, cancellation, and permanent item failures.

Create a bounded operation

const operation = await incld.bulkOperations.create(
 {
  action: "sync_contacts",
  items,
  chunkSize: 500,
  metadata: { importId: "imp_42" },
 },
 { idempotencyKey: "contacts:imp_42" },
)

This example assumes incld is constructed with the authenticated organization scope. Items are metered when the operation is accepted. Keep individual items JSON-serializable and small enough that a chunk remains within your application’s webhook body limit.

Chunk handler payload

interface BulkActionPayload<Item> {
 operationId: string
 chunkId: string
 chunkIndex: number
 items: Item[]
 metadata: Record<string, unknown>
}

Use the delivery event ID—or the stable chunk ID—as your downstream deduplication key. Do not rely on the chunk index alone across unrelated operations.

Progress and inspection

Field	Meaning
totalItems / totalChunks	Accepted work and derived chunk count.
completedChunks	Chunks in a terminal state.
succeededChunks / failedChunks	Terminal outcome split.
percentage	Completed chunks divided by total chunks.
chunks(id)	Payload, attempts, response status/body, error, and timestamps for each chunk.
events(id)	Lifecycle and operator actions for the operation.

Retries and partial failure

A transient delivery error moves a chunk through retrying and increments its attempt count. When retry policy is exhausted, the chunk becomes failed. The operation completes with completed_with_errors if any chunk failed permanently; successful chunks are not rolled back.

Compensation belongs to your application

Bulk provides execution durability, not distributed transactions. If partial completion requires reversal, implement a compensating action using the chunk and item identifiers.

Cancellation

await incld.bulkOperations.cancel(operationId, 'Requested by operator')

Cancellation prevents queued chunks from starting. A chunk already being handled may finish, so the handler still needs idempotency and application-level cancellation checks for long-running work.