> ## Documentation Index
> Fetch the complete documentation index at: https://developers.teampascal.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Uploads

> Open an upload, send records in batches, close it, and read the result.

## Records

Send JSON records with your own field names:

```json theme={null}
{"id": "1001", "doc_type": "purchase_order", "data": {"PONumber": "PO-88213", "Customer": "Example Foods", "OrderDate": "2026-09-01"}}
```

| Field      | Contract                                                         |
| ---------- | ---------------------------------------------------------------- |
| `id`       | String, 1–200 characters, no whitespace                          |
| `doc_type` | `purchase_order` or `invoice`; must be declared on the connector |
| `data`     | JSON object in your own field layout                             |

A purchase order must carry the order number, the customer's name, and the order date. An invoice must carry the invoice number, the invoice date, and the total amount. Dates in common formats are normalized. Unparseable required dates are reported as missing.

Ids are unique across the datasource. An invoice and a purchase order sharing a source number need different ids. An upload may contain at most 20 distinct combinations of document type and record layout.

## Open an upload

Send `POST /uploads`:

```json theme={null}
{"datasource": "sales_orders", "mode": "incremental"}
```

Choose `incremental` or `full` using the [mode table](/). The 201 response includes `upload_id`, `status: "in_progress"`, `mode`, `dry_run`, `datasource`, and `created_at`. Save the upload id.

An `in_progress` upload is writable while `closed_at` is null. Opening another upload expires the previous writable upload and discards its staged records. If the current upload has `closed_at` set, opening returns 409 `upload_processing` with `Retry-After: 30`.

## Send records

Send batches to `PUT /uploads/{upload_id}/records`:

```json theme={null}
{"records": [{"id": "1001", "doc_type": "purchase_order", "data": {"PONumber": "PO-88213", "Customer": "Example Foods", "OrderDate": "2026-09-01"}}]}
```

Each call allows at most 1,000 records and 10 MB (10,485,760 bytes). The entire batch is validated before any write. A rejected batch stages nothing.

Invalid ids or undeclared document types return 400 `invalid_record`. Its `details.records` entries contain `index`, `id`, and `reason`: `invalid_id` or `undeclared_doc_type`. A non-object `data` value returns 400 `invalid_request`.

Sending the same id again within an upload replaces its staged value: last write wins. Retrying the same records is safe. The response contains `accepted` (records accepted in this batch), `upload_id`, and `record_count` (distinct staged ids).

## Close

Call `POST /uploads/{upload_id}/close`. Close sets `closed_at` and returns 202 with `upload_id` and `status`. Processing is asynchronous. Further writes are rejected.

Repeated closes return the current status, except expired uploads return 410 `upload_expired`. If scheduling fails, close returns 503 `publish_failed` with `Retry-After: 5`. Wait five seconds and close again to retry scheduling.

## Poll

Call `GET /uploads/{upload_id}`. While the upload is `in_progress` with `closed_at` set, honor `Retry-After: 15`.

| Status        | Meaning                                                   |
| ------------- | --------------------------------------------------------- |
| `in_progress` | Writable if `closed_at` is null; otherwise processing     |
| `done`        | Processing succeeded                                      |
| `rejected`    | Validation or the deletion brake rejected the upload      |
| `failed`      | Processing exceeded its time limit                        |
| `expired`     | Idle too long before close, or replaced by a newer upload |

The response includes record count and timestamps. `merged_counts` holds `inserted`, `updated`, and `deleted` after a successful merge. `report` is null until validation finishes; then it contains errors and per-document-type findings. Rejected or failed uploads carry an `error` envelope. See [Errors and recovery](/errors).

```mermaid theme={null}
stateDiagram-v2
    [*] --> in_progress
    in_progress --> in_progress: close sets closed_at
    in_progress --> done: merge succeeds
    in_progress --> rejected: validation or deletion brake
    in_progress --> failed: processing timeout
    in_progress --> expired: idle or replaced before close
    done --> [*]
    rejected --> [*]
    failed --> [*]
    expired --> [*]
```

The hourly sweep expires writable uploads after at least 24 hours without a write. Worker failures leave closed uploads `in_progress` during retries. After at least one hour of processing, the stuck-upload sweep marks them `failed` with `processing_timeout`. Rejected, failed, and expired uploads have their staging discarded.

## Full snapshots and the deletion brake

`full` upserts submitted records and removes current records absent from the upload. With a non-empty current set, removing more than 20% or submitting zero records rejects the upload with `too_many_deletions`. Nothing changes.

Error details include `current_count`, `would_delete`, and `max_delete_fraction`. The 20% threshold is fixed across the deployment. Resend the intended full set, and contact Pascal if a larger removal is intentional.

## Dry runs

Set `dry_run: true` when opening. Close validates the records and returns `done` or `rejected` with a report. It discards staging and leaves `merged_counts` null. Documents and the last accepted upload remain unchanged.

## Inspect the datasource

Call `GET /datasources/{name}` with your bearer token. The name must match the token's datasource, or the response is 403 `datasource_mismatch`.

| Response field         | Meaning                                                                                                |
| ---------------------- | ------------------------------------------------------------------------------------------------------ |
| `datasource`           | Datasource name                                                                                        |
| `doc_types`            | Declared document types                                                                                |
| `record_count`         | Records in the current set                                                                             |
| `last_accepted_upload` | Latest non-dry-run upload with status `done`: `upload_id`, `finished_at`, and `merged_counts`, or null |
| `connector_stats`      | Document counts and crawl timing; may be null before the first processing run                          |