Read before:
../conectores.md(state map) andfichas.md(index of sheets).
What it is#
Extracts data from a project in your Google data warehouse (via SQL, with optional billing project) and loads it into our cloud.
Supported ingestion modes:
- Nós buscamos — Nós buscamos os dados na sua fonte.
What it's for#
Bring data from your Google data warehouse into your datalake and make it available to the Bronze/Silver/Gold layers, BI models and the natural language question wizard.
When to use / when not to use#
Use when you need to consolidate this source into the datalake on a scheduled basis, with isolation by workspace.
Avoid sending files with unstable schemas without first checking profiling and schema drift.
Plan and permissions#
- Plan: core of the product (connectors are part of all plans; the limit is the number of sources allowed by the plan).
- Access: requires an active workspace — with the subscription overdue, suspended or canceled, runs are blocked.
- Shutdown: the ingestion run can be temporarily turned off (maintenance or incident). When that happens, the screen indicates the resource is currently unavailable.
Prerequisites#
- A read-only user on the database, with access to the desired schema/tables.
- Network: choose how we reach the database — public IP + allowlist, SSH tunnel (bastion), Cloud SQL or VPN.
Credentials#
Sensitive fields. They are encrypted (AES-256-GCM) and never stored in the connection config, in the pipeline definition, in the export or in the DAG. By default the encrypted value is kept in the database; moving this connection's credential to an external credential store is an explicit choice, made per connection after saving — see ../../administracao/credenciais-e-rotacao.md.
- Access key (JSON file) (
serviceAccountJson) — Open the service account .json file and paste the entire content here.
Secrets are never re-displayed and go through redaction on any error/log (raw token, URL-encoded and in JSON).
Form fields#
| Field | Label | Type | Required | Note |
|---|---|---|---|---|
projectId | Projeto dos dados (Project ID) | text | yes | The project code where the tables live (appears at the top of the console). |
billingMode | Cobrança das consultas (billing) | selection | no | default: same; options: same / separate |
billingProjectId | Projeto de billing (Project ID) | text | no | appears when billingMode = separate |
dataset | Dataset | text | no | — |
location | Location | text | no | — |
serviceAccountJson 🔒 | Chave de acesso (arquivo JSON) | password/secret | yes | secret (encrypted; external vault optional); Open the service account .json file and paste the entire content here. |
Step-by-step#
- In the menu Dados → Conexões, use Nova conexão.
- Choose Data warehouse in the catalog.
- Fill in the form fields (see the table above).
- Use Testar conexão. The result is shown broken down into Conectividade, Autenticação and Listagem/leitura — each with OK, Failed, Not applicable or Not tested. Here Conectividade shows as Not applicable: BigQuery is HTTPS against the Google API, not a host:port — the OK comes from Listagem/leitura.
- Use Salvar conexão. Saving does NOT start any ingestion.
- On the connection card, use Montar extração (or go to Orquestração → Pipelines and use Conectar fonte). The pipeline points to the saved connection — the credential is not copied.
- In the review, Publicar (agenda desligada) creates version 1 without executing anything. After that, Ativar agenda and Executar agora are separate actions.
Example — connection test#
Example configuration (all values are fictitious; secrets appear masked):
# Exemplo sintético (valores fictícios — não são credenciais reais)
projectId = meu-projeto-exemplo
billingMode = same
dataset = vendas
location = southamerica-east1
serviceAccountJson = •••••• (cifrado — nunca exibido)Expected result#
Testar conexão returns the list of discovered tables/resources. After the first load, the rows appear in the Bronze layer of your workspace, ready for the pipeline.
Limits and costs#
- Pagination: hard cap per run (default 10,000 pages / 5,000,000 rows); anti-loop guard.
- Estimated volume: ~5 GB/month (catalog estimate, not measurement).
- Cost: the extraction itself is not charged by the platform; the Bronze load consumes datalake processing (billable).
Security#
- Scope by workspace: each customer reads and writes only to their own datalake (
<prefix>_bronze/_silver/_gold); there is never leakage between workspaces. - Encrypted secrets: sensitive material is encrypted with AES-256-GCM before being written. The external credential vault is optional and enabled per connection; without it, the encrypted value is kept in the database. Changing the value here DOES NOT revoke the credential at the source.
- Redaction: all error/log output goes through secret redaction (raw, URL-encoded and in JSON).
- Production does not mock: a path that would hit a mock triggers the production guard (explicit failure instead of simulating success).
Common errors#
- Connection refused / timeout — incorrect host, port or network rule (allowlist/SSH/VPN). Check the chosen network mode in Como conectamos.
- Authentication failed — invalid user/password or user without read permission on the schema.
- SSL/handshake — adjust
sslmode(PostgreSQL) orencrypt(SQL Server) according to the server's requirement. - Large table/query — the extraction respects the page/row cap; use incremental load by watermark to reduce volume.
Troubleshooting#
- Use Testar conexão to isolate credential/network issues before scheduling.
- Monitor runs in the connector observability (error rate, p50/p95, freshness).
- A schema change at the source is signaled by schema drift (new column = info; removed column/type changed = attention).
Related#
../conectores.md— connector catalog by state../ingestao-incremental-backfill.md— full, incremental by watermark, backfill and checkpoint../pipeline-medalhao.md— Bronze/Silver/Gold layers and quality../agendamento-retry-dlq.md— scheduling, retry/backoff and DLQ../custo-limites-seguranca.md— cost, caps and security by workspace
Last revision: 2026-10-04. Sheet generated automatically from the product's connector catalog.