Build your first pipeline
Where: sidebar → Jobs, then open a pipeline in its Data Catalog database.
This walkthrough uses illustrative names: source training-orders-source, target database training, and target table orders_copy. These resources are not installed by the documentation. Use an approved small source table and an isolated target in your own workspace.
Before you start
- You have workspace-write permission and appropriate catalog permissions.
- A saved connection is verified.
- A crawler snapshot describes the source table, where supported.
- An approved target database exists in the Data Catalog.
- The platform's execution service is available.
Create the job
Open Jobs and select New pipeline. Choose the Data Catalog database, enter a Pipeline name, and submit the dialog. Open the created pipeline.
Configure a source node
In the visual editor's Graph nodes catalogue, choose a source node supported by your connector. Select the verified connection and source relation in its property panel. Use the fields offered by that node; do not paste a database password into a SQL transformation.
Configure the target
Add a supported target node and connect the source output to it. Set the target database and table. Review the target write mode and any schema-related settings before saving.
For the first exercise, use a new training table rather than an existing production table. Transformations can be added after the simple source-to-target path works.
Save and review
Save the graph and resolve validation errors. Inspect node properties and the generated script where available. The generated script represents a saved version, not necessarily unsaved canvas edits.
A node preview requires a saved pipeline and is a limited preview, not proof of a completed target write.
Run the saved pipeline
Select Run when available. Note the execution identifier and open Job running. Wait for Succeeded; Queued, Submitted, and Running are intermediate states.
Verify the managed table
Open the target database under Data Catalog, locate the table, and inspect its columns and preview. Run a small query if authorized. Compare a few known source records and confirm that the write mode produced the intended result.
Safe iteration
Re-running a pipeline can append duplicates or replace data, depending on the target configuration. Confirm the write mode and expected output before every retry. Cancellation does not guarantee that already-written data is rolled back.
Use Clone to explore changes in a separate pipeline. Archive changes pipeline availability; it is not a command to drop its target tables.
If another editor saved a newer version, review or reload that version before continuing. Do not repeatedly overwrite a conflict without understanding the other changes.
Continue with Monitor runs or Query your data.