Skip to content

Automatic translation — the Portuguese version is the reference. View in Portuguese

Updated on 2026-10-03

First ingestion (create, publish, and run the pipeline)

The six stages of building the pipeline, publishing version 1 with the schedule off, the explicit run, and checking the result — with a screenshot of each screen.

On this page (15)

What you'll do#

Define a load, review its stages, publish a version, and run the first load explicitly.

Publishing the definition and running the data are different actions

The first publication creates version 1 and leaves the schedule off. Nothing is read or written at that moment. Turn on schedule and Run now are two other decisions, each with its own button.

The transformation follows the medallion architecture:

  • Bronze — raw data, as it arrived from the source.
  • Silver — clean, standardized data (types, dates, columns, conditionals).
  • Gold — datamart ready for BI (aggregations, joins, business model).

Before you start#

  • An authorized connection (see Connect your first source) or a file in the supported import flow.
  • Be the workspace owner: Pipelines and Runs are owner areas.
  • Know what can be read from the source and where it can be written.
  • For the tutorial, use a small, synthetic dataset, with keys and values you know in advance.
  • Subscription up to date. Running is an expensive action: when payment is past due, suspended, or canceled, the run is blocked with an explicit message and a link to Plans.
  • The first load consumes cloud resources, charged in credits.

Where pipelines live#

In ETL/ELT → Pipelines. The list shows, per pipeline, the source connector, the schedule, and the last run — and the New pipeline button opens the builder.

Pipeline list, with each one's type, connector, status, schedule, and last run

Create the pipeline — guided mode (the simplest path)#

New pipeline opens four questions:

#QuestionWhat you choose
1Where does the data come from?A saved connection, Upload a file or spreadsheet, or Connect a database
2What to bring in?The tables (database) or the files — and, for files, whether the new one replaces or is combined with the previous ones
3When to update?Only when I say so, Every day, at a set time (Brasília time), or Whenever I upload a new file
4Review and createCreate and run now or Just create
New pipeline in guided mode, question 1 of 4: saved connections, upload a file, or connect a database

Guided mode brings in the whole table on every run. For loading only what changed, transformations, partitioning, alerts, or tools (Google Sheets, Ads, CRM), use Use full mode, described below.

Build the pipeline in full mode — the six stages#

Full mode (Use full mode, or New report in the list) has a single six-stage ruler, the same for extraction and for a Gold report:

#StageWhat you decide in it
1ConnectionsWhich saved connection the pipeline will reference. The credential isn't copied.
2DataWhich tables, files, or endpoints come in, with fields, types, and a sample when authorized.
3LoadThe load mode and the effect on the target: replace, merge, and append are not the same thing.
4TransformationsThe Silver/Gold stages that fit the task, and the dependencies in the graph.
5OperationSchedule, limits, retries, and alerts.
6ReviewTarget, write mode, run stages, schedule, limits, and alerts before publishing.
Building the pipeline in full mode, stage 1 of 6 (Connections): the single ruler Connections · Data · Load · Transformations · Operation · Review, above the source catalog

Starting from a saved connection, the equivalent shortcut is Use in a new pipeline, in the connection's menu under ETL/ELT → Connections — and, if you open the connection wizard from inside the builder, when you save you return to the same draft with the new connection already selected.

About stage 1

The "— enter one-off credentials —" option exists for specific cases, but loses the advantage of changing the password in one place. Prefer referencing a connection.

Publish version 1#

Publishing happens in the Review stage, with the Publish pipeline button. Next to it, before you click, the screen tells you what will happen: "Publishing creates the pipeline version. The schedule will stay off and no load will start."

Once published, the wizard responds "Pipeline published — version 1", with the Schedule off badge and the sentence "Version 1 of the definition is frozen. No run was created, no load ran and no earlier interval was recovered." The three next actions — Run now, Turn on schedule, and Open operation — appear separately, each with its own button.

On the pipeline screen, the Publishing and schedule panel shows the same state: the Version 1 published badge, the schedule state, and three actions — Review and publish, Turn on schedule, and Run now. The middle button is Turn on schedule precisely because the schedule is off; it only becomes Pause schedule after you turn it on.

Pipeline right after the first publication: the Version 1 published badge, the schedule off, the Review and publish, Turn on schedule, and Run now actions, and the product's response saying no load was started

Schedule vocabulary on this screen

The status badge uses "Schedule paused" when a time is configured and the switch is off; the scheduling card, just below, uses "Schedule OFF". They're the same thing: nothing is dispatched until you turn it on.

Run the first load#

Use Run now. Review context, version, and target before confirming. Turn on schedule is a separate action, for later — and turning on the schedule does not trigger historical recovery.

Track the run#

In ETL/ELT → Runs. The list separates failed on last run (what needs action now) from failures in period (history), shows the schedule per pipeline, and shows times in Brasília time.

Runs list, with the Running, Failed on last run, Failures in period, and Paused schedules counters, and the pipeline table

Open the run that was created and drill down from Run to Task, from Task to Attempt, and only then to the Logs tab. The log shown is the one for that attempt — the attempt selector is at the top, next to the run selector.

Detail of a failed run: the grid on the left, the run and attempt selectors, the selected task, and the log for that attempt

The Graph tab shows the dependencies between the tasks of the version that ran.

Graph of a run, with the tasks and their dependencies

Check the result in the data#

A "success" status doesn't replace checking. Open Data → Data console and validate: the tables exist, the count matches the test dataset, and the known keys and values are there.

Legacy pipelines (created before versioned publishing)#

Not every pipeline in your workspace went through this flow. Those created before versioned publishing have no recorded version, and the screen says so instead of faking a retroactive version 1:

  • the badge is "Version not recorded", not "Version 1 published";
  • the schedule appears as Manual when no frequency/time is defined — even if the schedule switch is on, because the switch alone doesn't create a schedule;
  • if the old registration stored a cron expression, the screen states that it is not dispatched: only the pipeline's scheduling (frequency and time) triggers automatic runs.

The screenshot below is of a legacy pipeline in the demo environment. It's here so you can recognize that state — it is not the screen for a first publication (that's in the Publish version 1 section).

Legacy pipeline: the publishing and schedule panel stating “Version not recorded”, the Manual schedule, and the warning that the old registration's cron expression is not dispatched

To bring a legacy pipeline into versioning, use Review and publish: publishing creates the version from the current definition. On this path the schedule is kept as it is — updating a pipeline never turns off the schedule as a side effect.

Expected result#

  • A published definition, identifiable by version.
  • A traceable run, with tasks, attempts, and logs.
  • Data checked at the target, not just a green badge.
  • The schedule stays as you left it: off, if you didn't turn it on.

The end of a run doesn't refresh every dashboard automatically

Refreshing the dashboard is a separate step. See View a dashboard.

If something goes wrong#

SymptomWhat to checkNext action
Publishing blockedThe validation pointed out in the reviewFix the item; don't make up a dependency or credential to move forward
I published and nothing ranThe schedule stateExpected behavior — use Run now or Turn on schedule
It didn't startSubscription status, concurrency limit, permissionCheck the message on screen; settle it in Plans if it's billing
Authentication failureThe referenced connectionFix it in the connection; don't re-enter the password in the pipeline
Schema or write failureInput contract, mapping, effect on the targetSee pipeline and schema
Retry pendingLimit and time of the attemptDon't trigger parallel loads unnecessarily
Success, but empty dashboardTarget, materialization, BI binding, filters, and dashboard refreshSee View a dashboard
I reran and it duplicatedLoad mode and recovery scopeSee Recover from a failure
The historical run doesn't show a versionRuns from before versioningThe screen warns you; don't treat a later date as proof of content

Limits and costs#

  • Creating and publishing: R$ 0. Running consumes cloud resources (credits).
  • We don't publish run times: they depend on the volume of your data, the source, and the queue. There's no published measurement.

Back to the app#

Next task#


Last reviewed: 2026-09-17. The screenshots on this page come from the delivered version, in a demo environment with fictional data (the project seed's synthetic workspace), 1366×768 viewport; the failed run shown is a failure induced in the fixture, not a customer incident. The provenance of each image is in the screenshot manifest e2e/baseline/galeria-v3.json. A screenshot from a local environment documents the interface; it doesn't prove execution in the cloud.

Related links