Skip to content

Automatic translation — the Portuguese version is the reference. View in Portuguese

Updated on 2026-10-04

Tutorial — connect a generic REST API

Step-by-step to describe a REST API (URL, list path, pagination, token), test the connection and load the records as a Bronze table.

On this page (9)

The generic REST API connector can be registered and tested today. What is proven by tests is our side — request assembly, pagination, resumption, retry with backoff and token protection. The format of your API is what the connection test checks. Synthetic examples.

Read first: Como os dados fluem · Catálogo de conectores.


What it is#

A connector for systems without a ready connector: you describe the API (base URL, the path that lists records, where the list is in the JSON and how it pages) and Ingestia reads the records on schedule and loads them as one table in the Bronze layer. Columns are discovered on first read (names normalized to snake_case; objects and lists become JSON text).

What it's for#

Bring data from an internal system, a niche vendor or a partner API into the datalake without waiting for a dedicated connector — and without exporting spreadsheets by hand.

When to use / when not to use#

  • Use when: the API responds JSON over HTTPS, lists records at an endpoint (GET or POST) and pages by page number, by offset or by next-page URL in the body (or does not page).
  • Do not use when: the system already has a connector in the catalog (prefer it — it knows the API envelope and limits), the API requires OAuth with token refresh, per-request signing or a custom header key (these modes are planned for a future connector evolution), or the response is XML/CSV.

Plan and permissions#

  • Product core · access check (workspace active).
  • The load goes through the shutdown key of the capacity execução de ingestão.
  • Recommended: a dedicated read-only token for Ingestia.

Prerequisites#

  1. The API base URL (starts with https://) and the resource path that lists the records (e.g.: /v1/pedidos).
  2. Know where the list is in the response: the response is already an array ([...]), or the array is under a key (data, results, data.items).
  3. Know how the API pages and the maximum page size it accepts.
  4. A Bearer token, if the API requires it (sent as Authorization: Bearer …; encrypted, never re-displayed).

Step by step#

  1. In the console, go to Fontes › Conectores and choose API REST genérica (category APIs genéricas).
  2. Fill the API description:
    • URL base — e.g.: https://api.exemplo.com
    • Caminho do recurso — e.g.: /v1/pedidos
    • Método HTTP — GET (parameters in the query) or POST (in the JSON body)
    • Caminho da lista na resposta — e.g.: data (empty = the response is already the array)
  3. In Paginação, choose the API style and fill the requested fields:
    • Nenhuma — a single response.
    • Por número de página — page parameter (e.g.: page), page size parameter (e.g.: per_page) and the maximum size the API accepts.
    • Por deslocamento (offset) — offset parameter (e.g.: offset, skip), page size parameter (e.g.: limit) and the maximum size.
    • URL da próxima página no corpo — where the response provides the next URL (e.g.: next, links.next, paging.next; accepts relative URLs).
  4. If the API requires a token, paste it into Token Bearer. In Query extra, pin parameters that every call must include (e.g.: status=active&sort=desc).
  5. Click Testar conexão. Ingestia requests a small page, checks that the list exists at the provided path and shows the discovered table.
  6. With a successful test, save the source and add it to a pipeline. The Bronze table is named registros.

How reading works (and where it stops)#

  • For page-number and offset pagination, reading stops at the first page shorter than the requested size — that's why the field asks for the maximum the API accepts. If the API ignores the size and always returns fewer, provide the size it actually returns.
  • For next-page URL in body, reading follows the next URL until it is empty or absent. A URL that repeats is treated as the end (it won't loop).
  • With an empty token, no Authorization header is sent (open APIs).
  • Responses 429 and 5xx are retried with backoff (and honoring Retry-After when present). 401/403 stop the process with the message "credencial recusada" — the token is never shown in the error.
  • Extraction is full on each run (no incremental window). Very large sources may need a narrower path or Query extra filter.

Common problems#

SymptomLikely causeWhat to do
"A resposta não tem uma lista em <caminho>"wrong list pathcheck the API JSON; use dot notation (data.items)
Only the first page arriveswrong page/size parameter names, or size bigger than acceptedfix the parameter names and use the API's real maximum
"Credencial recusada"token invalid, expired or lacking permissiongenerate a new read token and update the source
Columns with odd namesthe API uses keys with spaces/accentsnormal — we normalize to snake_case; rename in the Silver layer
Objects appear as textnested fieldexpected behavior (JSON as text); extract in Silver

Related#

Related links