Skip to content

Automatic translation — the Portuguese version is the reference. View in Portuguese

Updated on 2026-10-04

Google data warehouse — connector

Extracts data from your Google data warehouse project (via SQL, with optional billing project) and loads it into our cloud.

On this page (15)

Read before: ../conectores.md (state map) and fichas.md (index of sheets).


What it is#

Extracts data from a project in your Google data warehouse (via SQL, with optional billing project) and loads it into our cloud.

Supported ingestion modes:

  • Nós buscamos — Nós buscamos os dados na sua fonte.

What it's for#

Bring data from your Google data warehouse into your datalake and make it available to the Bronze/Silver/Gold layers, BI models and the natural language question wizard.

When to use / when not to use#

Use when you need to consolidate this source into the datalake on a scheduled basis, with isolation by workspace.

Avoid sending files with unstable schemas without first checking profiling and schema drift.

Plan and permissions#

  • Plan: core of the product (connectors are part of all plans; the limit is the number of sources allowed by the plan).
  • Access: requires an active workspace — with the subscription overdue, suspended or canceled, runs are blocked.
  • Shutdown: the ingestion run can be temporarily turned off (maintenance or incident). When that happens, the screen indicates the resource is currently unavailable.

Prerequisites#

  • A read-only user on the database, with access to the desired schema/tables.
  • Network: choose how we reach the database — public IP + allowlist, SSH tunnel (bastion), Cloud SQL or VPN.

Credentials#

Sensitive fields. They are encrypted (AES-256-GCM) and never stored in the connection config, in the pipeline definition, in the export or in the DAG. By default the encrypted value is kept in the database; moving this connection's credential to an external credential store is an explicit choice, made per connection after saving — see ../../administracao/credenciais-e-rotacao.md.

  • Access key (JSON file) (serviceAccountJson) — Open the service account .json file and paste the entire content here.

Secrets are never re-displayed and go through redaction on any error/log (raw token, URL-encoded and in JSON).

Form fields#

FieldLabelTypeRequiredNote
projectIdProjeto dos dados (Project ID)textyesThe project code where the tables live (appears at the top of the console).
billingModeCobrança das consultas (billing)selectionnodefault: same; options: same / separate
billingProjectIdProjeto de billing (Project ID)textnoappears when billingMode = separate
datasetDatasettextno—
locationLocationtextno—
serviceAccountJson 🔒Chave de acesso (arquivo JSON)password/secretyessecret (encrypted; external vault optional); Open the service account .json file and paste the entire content here.

Step-by-step#

  1. In the menu Dados → Conexões, use Nova conexão.
  2. Choose Data warehouse in the catalog.
  3. Fill in the form fields (see the table above).
  4. Use Testar conexão. The result is shown broken down into Conectividade, Autenticação and Listagem/leitura — each with OK, Failed, Not applicable or Not tested. Here Conectividade shows as Not applicable: BigQuery is HTTPS against the Google API, not a host:port — the OK comes from Listagem/leitura.
  5. Use Salvar conexão. Saving does NOT start any ingestion.
  6. On the connection card, use Montar extração (or go to Orquestração → Pipelines and use Conectar fonte). The pipeline points to the saved connection — the credential is not copied.
  7. In the review, Publicar (agenda desligada) creates version 1 without executing anything. After that, Ativar agenda and Executar agora are separate actions.

Example — connection test#

Example configuration (all values are fictitious; secrets appear masked):

text
# Exemplo sintético (valores fictícios — não são credenciais reais)
projectId = meu-projeto-exemplo
billingMode = same
dataset = vendas
location = southamerica-east1
serviceAccountJson = •••••• (cifrado — nunca exibido)

Expected result#

Testar conexão returns the list of discovered tables/resources. After the first load, the rows appear in the Bronze layer of your workspace, ready for the pipeline.

Limits and costs#

  • Pagination: hard cap per run (default 10,000 pages / 5,000,000 rows); anti-loop guard.
  • Estimated volume: ~5 GB/month (catalog estimate, not measurement).
  • Cost: the extraction itself is not charged by the platform; the Bronze load consumes datalake processing (billable).

Security#

  • Scope by workspace: each customer reads and writes only to their own datalake (<prefix>_bronze / _silver / _gold); there is never leakage between workspaces.
  • Encrypted secrets: sensitive material is encrypted with AES-256-GCM before being written. The external credential vault is optional and enabled per connection; without it, the encrypted value is kept in the database. Changing the value here DOES NOT revoke the credential at the source.
  • Redaction: all error/log output goes through secret redaction (raw, URL-encoded and in JSON).
  • Production does not mock: a path that would hit a mock triggers the production guard (explicit failure instead of simulating success).

Common errors#

  • Connection refused / timeout — incorrect host, port or network rule (allowlist/SSH/VPN). Check the chosen network mode in Como conectamos.
  • Authentication failed — invalid user/password or user without read permission on the schema.
  • SSL/handshake — adjust sslmode (PostgreSQL) or encrypt (SQL Server) according to the server's requirement.
  • Large table/query — the extraction respects the page/row cap; use incremental load by watermark to reduce volume.

Troubleshooting#

  • Use Testar conexão to isolate credential/network issues before scheduling.
  • Monitor runs in the connector observability (error rate, p50/p95, freshness).
  • A schema change at the source is signaled by schema drift (new column = info; removed column/type changed = attention).

Related#


Last revision: 2026-10-04. Sheet generated automatically from the product's connector catalog.

Related links