Skip to content

Automatic translation — the Portuguese version is the reference. View in Portuguese

Updated on 2026-10-04

Parquet — connector

Ingests Parquet files (self-describing columnar format), which you upload to the receiving area of your environment (mode "You deposit"). To read from a bucket of yours, the connector is the one for the storage service itself.

On this page (16)

Read before: ../conectores.md (state map) and fichas.md (index of datasheets).


What it is#

Ingests Parquet files (self-describing columnar format), which you upload to the receiving area of your environment (mode "You deposit"). To read from a bucket of yours, the connector is the one for the storage service itself.

Supported ingestion modes:

  • You deposit — You deposit the data in our cloud.

What it's for#

Bring Parquet data into your datalake and make it available to the Bronze/Silver/Gold layers, BI models, and the natural language question wizard.

When to use / when not to use#

Use when you need to consolidate this source into the datalake on a scheduled basis, with isolation by workspace.

Avoid sending files with unstable schema without first checking the profiling and schema drift.

Plan and permissions#

  • Plan: core product (connectors are part of all plans; the limit is the number of sources allowed by the plan).
  • Access: requires an active workspace — if the subscription is overdue, suspended, or canceled, the run is blocked.
  • Shutdown: ingestion runs can be temporarily turned off (maintenance or incident). When that happens, the screen indicates the feature is currently unavailable.
  • Upload (mode "You deposit"): the file goes to the landing area of your own workspace.

Prerequisites#

  • The files you will upload (mode "You deposit").

Credentials#

This connector requires no secrets of its own (upload mode / no credential on the platform).

Secrets are never re-displayed and are redacted in all errors/logs (raw token, URL-encoded and in JSON).

Form fields#

Mode "You deposit" (push)

FieldLabelTypeRequiredNote

Step by step#

  1. In the menu Dados → Conexões, use Nova conexão.
  2. Choose Parquet in the catalog and select the ingestion mode.
  3. Fill in the form fields (see the table above).
  4. Upload the files. A file format only accepts deposit — there is no connection to test here.
  5. Use Salvar conexão. Saving DOES NOT start any ingestion.
  6. On the connection card, use Montar extração (or go to Orquestração → Pipelines and use Conectar fonte). The pipeline points to the saved connection — the credential is not copied.
  7. In the review, Publicar (agenda desligada) creates version 1 without running anything. After that, Ativar agenda and Executar agora are separate actions.

Example — connection test#

Example configuration (all values are fictitious; secrets are masked):

text
# Exemplo sintético (valores fictícios — não são credenciais reais)

Expected result#

The Testar conexão returns the list of discovered tables/resources. After the first load, the rows appear in the Bronze layer of your workspace, ready for the pipeline.

Limits and costs#

  • Pagination: hard cap per run (default 10,000 pages / 5,000,000 rows); anti-loop guard.
  • Estimated volume: ~0.5 GB/month (catalog estimate, not measurement).
  • Cost: the extraction itself is not charged by the platform; the Bronze load consumes datalake processing (billable).

Security#

  • Workspace-scoped: each customer reads and writes only to their own datalake (<prefix>_bronze / _silver / _gold); there is never leakage between workspaces.
  • Encrypted secrets: sensitive material is encrypted with AES-256-GCM before being written. The external credential vault is optional and toggled per connection; without it, the encrypted value stays in the database. Changing the value here DOES NOT revoke the credential at the source.
  • Redaction: all errors/logs pass through secret redaction (raw, URL-encoded and in JSON).
  • Production does not fake: a path that would fall into a mock triggers the production guard (explicit failure instead of simulating success).

Common errors#

  • Delimiter/encoding (CSV) — check the delimiter and prefer UTF-8; glued columns often indicate the wrong delimiter.
  • JSON format — choose between lines (ndjson) and a single array according to the file.

Troubleshooting#

  • Use Testar conexão to isolate credential/network issues before scheduling.
  • Monitor runs in the connector observability (error rate, p50/p95, freshness).
  • A schema change at the source is flagged by the schema drift (new column = info; removed column/type changed = attention).

Last revision: 2026-10-04. Datasheet auto-generated from the product connector catalog.

Related links