Skip to content

Automatic translation — the Portuguese version is the reference. View in Portuguese

Query API (REST) — `/api/v1/query` and `/api/v1/tables`

On this page (12)

The REST query API is in early access and not yet open for external use. It runs billable processing in the datalake; it opens together with the final request-limit and cost protections.

Don't build critical integrations on these routes yet. For external data consumption today, use the OData feed (Power BI/Excel), which is available.

This page documents the REST API contract for transparency and planning. The behavior described is that of the current code; the contract may change with notice while the API is in early access.

Availability#

  • Status: early access. Reason: these routes trigger paid queries in the datalake, and the rate limit guard needs to operate in a proven fail-closed mode (deny when the limiter is unavailable) before release.
  • Available alternative: OData v4 / Power BI / Excel (available) covers reading tables from published marts.
  • Region: Brazil (São Paulo).

Permissions#

  • API key authentication (see Authentication).
  • /api/v1/tables requires the data export action (member+ role / owner's data key).
  • /api/v1/query (free-form SQL) requires the data query action (admin+); the key also needs the read/read_query scope.

Endpoints (contract)#

GET /api/v1/tables

Lists the tables in the workspace's published marts (gold + silver).

json
[
  { "dataset": "prd_aurora_gold", "table": "vendas", "layer": "gold",
    "columns": [{ "name": "data", "type": "DATE" }, { "name": "total", "type": "NUMERIC" }] }
]

GET /api/v1/tables/{dataset}/{table}?limit=&offset=

Paginated rows from a published table. dataset must be a workspace mart (<prefixo>_gold or <prefixo>_silver) — anything else returns 403.

  • limit: default 100, maximum 1000.
  • offset: starting offset.
json
{ "table": "vendas", "rows": [ /*... */ ], "rowCount": 100, "nextOffset": 100 }

nextOffset is null when there are no more pages.

POST /api/v1/query

Read-only SQL (SELECT/WITH … SELECT) scoped to the workspace.

json
// body
{ "sql": "SELECT data, SUM(total) AS total FROM prd_aurora_gold.vendas GROUP BY data", "limit": 500 }
json
// response
{ "rows": [ /*... */ ], "rowCount": 42 }

If the result is cut off for safety, the response includes "truncated": true and "totalRows" — so the sum on the other side doesn't come out wrong.

Defenses applied before touching the datalake (in layers):

  1. SELECT only — any INSERT/UPDATE/DELETE/DDL/… → 403.
  2. No crossing workspaces — a reference to another customer's dataset or another project → 403.
  3. No dangerous constructs — EXTERNAL_QUERY, project metadata (INFORMATION_SCHEMA/JOBS_*), etc. → 403.
  4. Published marts only — only *_gold/*_silver; a reference to bronze or to any other dataset → 403.
  5. Cost guard — a billed-bytes ceiling applied to the job.

Validation (once released)#

  • GET /api/v1/tables returns 200 with the list of tables.
  • A query against a dataset outside the workspace returns 403 with an explicit message.
  • While in early access, integrations should treat unavailability responses (503) as an expected state.

Example#

bash
# Listar tabelas
curl -H "Authorization: Bearer ing_live_..." https://ingestia.io/api/v1/tables

# Puxar 100 linhas
curl -H "Authorization: Bearer ing_live_..." \
  "https://ingestia.io/api/v1/tables/prd_aurora_gold/vendas?limit=100&offset=0"

# SQL livre (read-only)
curl -X POST -H "Authorization: Bearer ing_live_..." -H "Content-Type: application/json" \
  -d '{"sql":"SELECT * FROM prd_aurora_gold.vendas LIMIT 10"}' \
  https://ingestia.io/api/v1/query

Cost#

  • Charges for the bytes actually processed in the datalake — LIMIT/OFFSET do not reduce the bytes read. Paginating a large table is the most expensive way to extract data; prefer a SELECT with filtering/aggregation.
  • Identical queries (same workspace version) are served from cache at R$ 0 cost.
  • The amount in x-ingestia-cost-brl is a projection, not a guaranteed billed price today.

Security#

  • Read-only + workspace scope (published marts only).
  • PII columns are not exposed through this API.
  • Queries and charges are recorded in the workspace audit log.

Limits#

  • Maximum limit per page: 1000.
  • Workspace-level key (no per-member filtering).
  • Rate limit per key and per workspace (see Errors and limits).
  • No writes — there is no INSERT/UPDATE/DELETE endpoint.

Troubleshooting#

SymptomLikely causeWhat to do
Recurring 503capability in early access / datalake not provisioneduse the OData feed; follow up with support
403 Apenas consultas SELECT (SELECT only)write/DDL SQLuse only SELECT/WITH … SELECT
403 fora do workspace (outside the workspace)reference to another customer's/project's datasetquery only *_gold/*_silver from your workspace
403 só marts publicados (published marts only)reference to bronze/another datasetpoint to gold/silver
High cost billpaginating a large tablefilter/aggregate in the SELECT instead of paginating everything

Next steps: OData v4 / Power BI / Excel (available alternative) · Errors and limits.

Related links