Skip to content

Automatic translation — the Portuguese version is the reference. View in Portuguese

Updated on 2026-10-04

Generic REST API — connector

Reads the records from a JSON REST API you describe (base URL, list path, pagination by page, offset or next-page URL, optional Bearer token) and loads them into the Bronze layer.

On this page (15)

Read before: ../conectores.md (state map) and fichas.md (index of cards).


What it is#

Reads the records from a JSON REST API you describe (base URL, resource path that lists records, where the array is in the JSON, and how pagination works: page, offset or next-page URL in the body). Optional Bearer token supported. Loads data into the Bronze layer.

Supported ingestion modes:

  • Nós buscamos — We fetch the data from your source.

What it's for#

Bring data from a API REST genérica into your datalake and make it available to the Bronze/Silver/Gold layers, BI models and the natural language question wizard.

When to use / when not to use#

Use when you need to consolidate this source into the datalake on a scheduled basis, with isolation per workspace.

Remember that the first run with a real credential is followed by our team before the connector is fully released.

Plan and permissions#

  • Plan: core product (connectors are part of all plans; the limit is the number of sources allowed by the plan).
  • Access: requires an active workspace — with the subscription overdue, suspended or canceled, runs are blocked.
  • Shutdown: ingestion runs can be temporarily disabled (maintenance or incident). When that happens, the UI indicates the feature is currently unavailable.

Prerequisites#

  • A Generic REST API account with access to the data you want to ingest.
  • Describe the API: the base URL, the endpoint that lists records, where the array is in the JSON (e.g.: data) and how it paginates (page, offset or next-page URL in the body). If the API requires a token, paste it below — it is sent as Authorization: Bearer and is encrypted (AES-256-GCM). Columns are discovered on the first read.

Credentials#

Authentication method: Access token pasted by you. It is encrypted (AES-256-GCM) and never shown again, nor appears in logs or error messages.

Sensitive fields. They are encrypted (AES-256-GCM) and are never kept in the connection config, pipeline definition, export or DAG. By default the encrypted value is stored in the database; moving the credential of this connection to an external credential vault is an explicit choice made per connection after saving — see ../../administracao/credenciais-e-rotacao.md.

  • Token Bearer (token) — Sent as Authorization: Bearer <token>. Leave empty for open APIs.

Secrets are never re-displayed and go through redaction in any error/log (raw token, URL-encoded and in JSON).

Form fields#

CampoRótuloTipoObrigatórioObservação
baseUrlBase URLtextyesStarts with https://. The path below is appended to it.
caminhoResource pathtextyesEndpoint that LISTS the records (one Bronze table per source).
metodoHTTP methodselectionnodefault: GET; options: GET / POST
caminhoListaPath to list in responsetextnoWhere the array of records is in the JSON (ex.: data, results, data.items). Empty = response is already the array.
paginacaoPaginationselectionnodefault: none; options: none / pagina / offset / urlNoCorpo
paramPaginaPage / offset parametertextnodefault: page; appears when paginacao = pagina / offset; For page: the page number parameter (ex.: page). For offset: the offset parameter (ex.: offset, skip).
paramTamanhoPage size parametertextnodefault: limit; appears when paginacao = pagina / offset / urlNoCorpo; Ex.: limit, per_page, page_size. Empty = do not send.
tamanhoPaginaPage sizenumbernodefault: 100; appears when paginacao = pagina / offset / urlNoCorpo; Use the MAX the API accepts: reading stops at the first page shorter than this.
caminhoProximaUrlPath to next URLtextnodefault: next; appears when paginacao = urlNoCorpo; Where the response provides the URL for the next page (ex.: next, links.next, paging.next). Accepts relative URL.
token 🔒Token Bearerpassword/secretnosecret (encrypted; external vault optional); Sent as Authorization: Bearer <token>. Leave empty for open APIs.
queryExtraExtra querytextnoFixed parameters appended to every call (format key=value&key=value).

Step by step#

  1. In the menu Dados → Conexões, use Nova conexão.
  2. Choose API REST genérica in the catalog.
  3. Fill the form fields (see the table above).
  4. Use Testar conexão. The result is shown broken down into Conectividade, Autenticação and Listagem/leitura — each with OK, Failed, Not applicable or Not tested. A capability with no test of its own for this connector shows as Not applicable, and Not applicable never counts as success.
  5. Use Salvar conexão. Saving DOES NOT start any ingestion.
  6. On the connection card, use Montar extração (or go to Orquestração → Pipelines and use Conectar fonte). The pipeline points to the saved connection — the credential is not copied.
  7. In the review, Publicar (agenda desligada) creates version 1 without running anything. Then, Ativar agenda and Executar agora are separate actions.

Example — connection test#

Example configuration (all values are fictitious; secrets are masked):

text
# Exemplo sintético (valores fictícios — não são credenciais reais)
baseUrl = <baseUrl>
caminho = <caminho>
metodo = GET
caminhoLista = <caminhoLista>
paginacao = nenhuma
token = •••••• (cifrado — nunca exibido)
queryExtra = <queryExtra>

Expected result#

The Testar conexão returns the list of discovered tables/resources. After the first load, rows appear in your workspace's Bronze layer, ready for the pipeline.

Limits and costs#

  • Pagination: hard cap per run (default 10,000 pages / 5,000,000 rows); anti-loop guard.
  • SaaS load: default batchSize 5,000, maxRows 200,000 per resource.
  • Estimated volume: ~0.1 GB/month (catalog estimate, not measurement).
  • Cost: the extraction itself is not charged by the platform; the Bronze load consumes datalake processing (billable).

Security#

  • Workspace scope: each customer reads and writes only to their own datalake (<prefix>_bronze / _silver / _gold); there is never leakage between workspaces.
  • Encrypted secrets: sensitive material is encrypted with AES-256-GCM before being written. External credential vault is optional and enabled per connection; without it, the encrypted value is stored in the database. Changing the value here DOES NOT revoke the credential at the origin.
  • Redaction: every error/log goes through secret redaction (raw, URL-encoded and in JSON).
  • Production does not mock: a path that would hit a mock triggers production guard (explicit failure instead of simulating success).

Common errors#

  • Credential rejected (401/403) — the provider did not accept the credential: expired, revoked or lacking required permission. Reconnect the source (the source and history are preserved; do not recreate the source).
  • Rate limit (429) — the provider asked to wait. The extraction respects Retry-After and retries; if it repeats, wait and reduce the schedule frequency.
  • Resource not found — check the resource identifier (account, object, report) and the granted permissions.
  • Pagination cursor stuck — the extraction stops when the API cursor does not advance (anti-loop guard). If this happens in consecutive runs, open a support ticket with the connector and resource information.
  • Empty records path — the response arrived without records where expected. Check the selected resource and filters (period, account, status) before assuming no data.

Troubleshooting#

  • Use Testar conexão to isolate credential/network issues before scheduling.
  • Monitor runs in the observability per connector (error rate, p50/p95, freshness).
  • A schema change at the source is signaled by schema drift (new column = info; removed column/type changed = attention).

Related#


Last review: 2026-10-04. Card automatically generated from the product connector catalog.

Related links