Read before:
../conectores.md(state map) andfichas.md(index of cards).
What it is#
Reads the records from a JSON REST API you describe (base URL, resource path that lists records, where the array is in the JSON, and how pagination works: page, offset or next-page URL in the body). Optional Bearer token supported. Loads data into the Bronze layer.
Supported ingestion modes:
- Nós buscamos — We fetch the data from your source.
What it's for#
Bring data from a API REST genérica into your datalake and make it available to the Bronze/Silver/Gold layers, BI models and the natural language question wizard.
When to use / when not to use#
Use when you need to consolidate this source into the datalake on a scheduled basis, with isolation per workspace.
Remember that the first run with a real credential is followed by our team before the connector is fully released.
Plan and permissions#
- Plan: core product (connectors are part of all plans; the limit is the number of sources allowed by the plan).
- Access: requires an active workspace — with the subscription overdue, suspended or canceled, runs are blocked.
- Shutdown: ingestion runs can be temporarily disabled (maintenance or incident). When that happens, the UI indicates the feature is currently unavailable.
Prerequisites#
- A Generic REST API account with access to the data you want to ingest.
- Describe the API: the base URL, the endpoint that lists records, where the array is in the JSON (e.g.: data) and how it paginates (page, offset or next-page URL in the body). If the API requires a token, paste it below — it is sent as Authorization: Bearer and is encrypted (AES-256-GCM). Columns are discovered on the first read.
Credentials#
Authentication method: Access token pasted by you. It is encrypted (AES-256-GCM) and never shown again, nor appears in logs or error messages.
Sensitive fields. They are encrypted (AES-256-GCM) and are never kept in the connection config, pipeline definition, export or DAG. By default the encrypted value is stored in the database; moving the credential of this connection to an external credential vault is an explicit choice made per connection after saving — see ../../administracao/credenciais-e-rotacao.md.
- Token Bearer (
token) — Sent as Authorization: Bearer <token>. Leave empty for open APIs.
Secrets are never re-displayed and go through redaction in any error/log (raw token, URL-encoded and in JSON).
Form fields#
| Campo | Rótulo | Tipo | Obrigatório | Observação |
|---|---|---|---|---|
baseUrl | Base URL | text | yes | Starts with https://. The path below is appended to it. |
caminho | Resource path | text | yes | Endpoint that LISTS the records (one Bronze table per source). |
metodo | HTTP method | selection | no | default: GET; options: GET / POST |
caminhoLista | Path to list in response | text | no | Where the array of records is in the JSON (ex.: data, results, data.items). Empty = response is already the array. |
paginacao | Pagination | selection | no | default: none; options: none / pagina / offset / urlNoCorpo |
paramPagina | Page / offset parameter | text | no | default: page; appears when paginacao = pagina / offset; For page: the page number parameter (ex.: page). For offset: the offset parameter (ex.: offset, skip). |
paramTamanho | Page size parameter | text | no | default: limit; appears when paginacao = pagina / offset / urlNoCorpo; Ex.: limit, per_page, page_size. Empty = do not send. |
tamanhoPagina | Page size | number | no | default: 100; appears when paginacao = pagina / offset / urlNoCorpo; Use the MAX the API accepts: reading stops at the first page shorter than this. |
caminhoProximaUrl | Path to next URL | text | no | default: next; appears when paginacao = urlNoCorpo; Where the response provides the URL for the next page (ex.: next, links.next, paging.next). Accepts relative URL. |
token 🔒 | Token Bearer | password/secret | no | secret (encrypted; external vault optional); Sent as Authorization: Bearer <token>. Leave empty for open APIs. |
queryExtra | Extra query | text | no | Fixed parameters appended to every call (format key=value&key=value). |
Step by step#
- In the menu Dados → Conexões, use Nova conexão.
- Choose API REST genérica in the catalog.
- Fill the form fields (see the table above).
- Use Testar conexão. The result is shown broken down into Conectividade, Autenticação and Listagem/leitura — each with OK, Failed, Not applicable or Not tested. A capability with no test of its own for this connector shows as Not applicable, and Not applicable never counts as success.
- Use Salvar conexão. Saving DOES NOT start any ingestion.
- On the connection card, use Montar extração (or go to Orquestração → Pipelines and use Conectar fonte). The pipeline points to the saved connection — the credential is not copied.
- In the review, Publicar (agenda desligada) creates version 1 without running anything. Then, Ativar agenda and Executar agora are separate actions.
Example — connection test#
Example configuration (all values are fictitious; secrets are masked):
# Exemplo sintético (valores fictícios — não são credenciais reais)
baseUrl = <baseUrl>
caminho = <caminho>
metodo = GET
caminhoLista = <caminhoLista>
paginacao = nenhuma
token = •••••• (cifrado — nunca exibido)
queryExtra = <queryExtra>Expected result#
The Testar conexão returns the list of discovered tables/resources. After the first load, rows appear in your workspace's Bronze layer, ready for the pipeline.
Limits and costs#
- Pagination: hard cap per run (default 10,000 pages / 5,000,000 rows); anti-loop guard.
- SaaS load: default
batchSize5,000,maxRows200,000 per resource. - Estimated volume: ~0.1 GB/month (catalog estimate, not measurement).
- Cost: the extraction itself is not charged by the platform; the Bronze load consumes datalake processing (billable).
Security#
- Workspace scope: each customer reads and writes only to their own datalake (
<prefix>_bronze/_silver/_gold); there is never leakage between workspaces. - Encrypted secrets: sensitive material is encrypted with AES-256-GCM before being written. External credential vault is optional and enabled per connection; without it, the encrypted value is stored in the database. Changing the value here DOES NOT revoke the credential at the origin.
- Redaction: every error/log goes through secret redaction (raw, URL-encoded and in JSON).
- Production does not mock: a path that would hit a mock triggers production guard (explicit failure instead of simulating success).
Common errors#
- Credential rejected (401/403) — the provider did not accept the credential: expired, revoked or lacking required permission. Reconnect the source (the source and history are preserved; do not recreate the source).
- Rate limit (429) — the provider asked to wait. The extraction respects
Retry-Afterand retries; if it repeats, wait and reduce the schedule frequency. - Resource not found — check the resource identifier (account, object, report) and the granted permissions.
- Pagination cursor stuck — the extraction stops when the API cursor does not advance (anti-loop guard). If this happens in consecutive runs, open a support ticket with the connector and resource information.
- Empty records path — the response arrived without records where expected. Check the selected resource and filters (period, account, status) before assuming no data.
Troubleshooting#
- Use Testar conexão to isolate credential/network issues before scheduling.
- Monitor runs in the observability per connector (error rate, p50/p95, freshness).
- A schema change at the source is signaled by schema drift (new column = info; removed column/type changed = attention).
Related#
../conectores.md— connector catalog by state../ingestao-incremental-backfill.md— full, incremental by watermark, backfill and checkpoint../pipeline-medalhao.md— Bronze/Silver/Gold layers and quality../agendamento-retry-dlq.md— scheduling, retry/backoff and DLQ../custo-limites-seguranca.md— cost, caps and workspace security
Last review: 2026-10-04. Card automatically generated from the product connector catalog.