Prerequisite — Ingesting source data with os-nexus
Connect source accounts in Nexus and use the existing BigQuery warehouse as their destination, with MCP tools for pipeline setup and sync verification.
Module 2 used seed data. Module 3 uses source data landed by Nexus before dbt transforms it into entities, events, and relationships.
Learning Objectives
- Reuse the warehouse created in lesson 1.2
- Connect source accounts through Nexus integrations
- Configure and verify ingestion with the Nexus MCP
- Map actual destination tables into the course's dbt examples
Where ingestion fits
Source APIs → Nexus ingestion → BigQuery raw tables → dbt + dbt-nexus
Nexus manages source authentication through Nango and runs ingestion. dbt reads the landed tables and builds your modeled outputs. Connecting Google for warehouse setup does not also authorize Gmail or Calendar; source accounts need their own connections.
Step 1: Reopen your connected warehouse
In your Nexus workspace, open Integrations → BigQuery and reopen the completed setup from 1.2 Setting up BigQuery. Record its warehouse ID, actual project ID, dataset, and location.
For this course, the destination dataset is os_nexus. The setup grants Nexus
write access to that selected dataset. It does not grant permission to create
arbitrary additional datasets.
A new warehouse does not replace your workspace's default warehouse or move existing pipelines. Select it explicitly when wiring each source.
Step 2: Connect a source account
Open Integrations, choose the source integration available in your workspace, and complete its connection flow. Start with one account so you can verify a complete sync before adding more.
Authorize the intended Google account for Gmail or Calendar, or select the pages you want Nexus to access when connecting Notion. The available connector determines the source-specific settings and models you can sync.
Step 3: Wire the source to this warehouse with MCP
Use the Nexus MCP to set the destination explicitly. Some integration UI flows
use a source-named namespace such as notion; for the warehouse created in this
course, the connection namespace must match the approved dataset, os_nexus.
Ask your assistant:
Wire my connected source account to the warehouse from my completed BigQuery setup. Use its warehouse ID and os_nexus as the connection namespace. Check whether this account already has a pipeline before creating one, and show me the destination and models before syncing.
The assistant uses list_ingestion_pipelines to inspect existing wiring and
create_ingestion_pipeline for a new pipeline:
| Argument | Value to use |
|---|---|
organizationId |
Your workspace's ID |
providerConfigKey |
The connected source integration's provider key |
nangoProviderConnectionId |
The source's Nango provider connection ID |
dataWarehouseId |
The warehouse ID returned by your completed BigQuery setup |
connectionNamespace |
os_nexus, or the actual dataset approved in your setup |
sourceConfig |
Connector-specific settings, when needed, encoded as a JSON string |
Use the source connection, not the Google Cloud setup connection. The create tool returns an existing pipeline if that account is already wired; it does not move it to a new destination. If the existing pipeline points elsewhere, review that configuration with your Nexus administrator before syncing.
Step 4: Run and verify the initial sync
After reviewing the source and destination, ask the assistant to run an initial
sync. It calls trigger_sync with your workspace ID and the pipeline's
connectionId, then follows the returned run using get_sync_status. The
pipeline connection ID is different from the Nango provider connection ID.
Check that the run completed and wrote records. If it fails, inspect the reported error before retrying the same pipeline. A dataset permission error can mean the namespace points outside the approved destination; check that mapping first.
Record the actual destination table names and schemas. Repeat for the other source accounts once the first source works. A successful warehouse setup verifies access; this source sync verifies that real source data has arrived.
Step 5: Map raw data into dbt
Nexus source ingestion commonly stores JSON in _raw_record, with metadata such
as _ingested_at, _connection_id, _stream_id, and _sync_id. Other write
paths use _raw_data; inspect the actual table schema before writing extraction
SQL.
The course's later examples use simplified tables such as
messages.sender_email and tasks.assignee_id. Build staging models to extract
those typed fields from the raw payload first. Connector output does not
automatically include all of these teaching tables.
The names gmail, google_calendar, and notion in dbt source() calls are
logical source names. Map database, schema, and each table's identifier to
the actual relations you prepared. They do not require three additional BigQuery
datasets. For staging models inside the same dbt project, use ref() instead of
source().
Your dbt identity needs read access to the ingestion dataset and write access to its development dataset. Use the dbt connection from Module 1; there is no Nexus service-account JSON key to export into your project.
Hands-On Exercise
- Connect one source account in Nexus.
- Wire it to your completed warehouse with the approved dataset namespace.
- Trigger a sync and confirm that records landed in the intended tables.
- Record the table mappings and stage the fields the course models need.
- Repeat for the other sources when ready.
If you are following the course with sample data, keep using dbt seeds and
reference them with ref(). You do not need to manually provision another
BigQuery project or service account to practice the models.
Next Lesson
Continue to 3.2 Introduction to dbt-nexus. The source model walkthrough is in 3.4 Setting up sources.