Analytics Hub lets you share BigQuery datasets with other Google Cloud organizations, or even publicly, without copying data.
Here’s a quick peek at it in action. Imagine you have a dataset called my_project.my_dataset containing sales data. You want to share this with a partner in partner_org.partner_project.
First, you’d create a listing in Analytics Hub. This listing acts as a catalog entry pointing to your data.
# On the publisher side (your project)
bq mk --transfer_config \
--project_id=my_project \
--target_dataset=my_project.analytics_hub_listings \
--display_name="My Sales Data" \
--data_source="analytics_hub" \
--params='{"dataset_id": "my_project.my_dataset", "source_type": "DATASET"}'
This creates a listing. The transfer_config here is a bit of a misnomer; it’s how we manage Analytics Hub listings via the bq CLI. The params specify what you’re sharing (my_project.my_dataset) and that it’s a BigQuery DATASET.
Once the listing is created, you get a listing_id. You then grant access to this listing.
# On the publisher side
# Replace <listing_id> with the actual ID from the previous step
bq query \
--project_id=my_project \
'GRANT analytics_hub.viewer TO "user:analyst@example.com", "group:data-team@example.com", "serviceAccount:my-sa@my-project.iam.gserviceaccount.com"' \
--use_legacy_sql=false
This GRANT command is key. It doesn’t grant access to the underlying BigQuery dataset directly. Instead, it grants the analytics_hub.viewer role on the listing resource. This role allows principals to discover and subscribe to the listing.
Now, on the subscriber side (partner_org.partner_project), they can find and subscribe to your shared data.
# On the subscriber side (partner's project)
# First, list available listings in your organization
bq ls --transfer_config --project_id=partner_org.partner_project \
--data_source="analytics_hub"
# Then, subscribe to the specific listing
# Replace <publisher_listing_id> with the actual ID from the publisher's side
bq mk --transfer_config \
--project_id=partner_org.partner_project \
--target_dataset=partner_org.analytics_hub_subscriptions \
--display_name="Subscribed to My Sales Data" \
--data_source="analytics_hub" \
--params='{"listing_id": "<publisher_listing_id>"}'
When the subscriber runs the bq mk --transfer_config command with the listing_id, Analytics Hub creates a linked dataset in their project. This linked dataset appears as a regular BigQuery dataset (e.g., partner_org.analytics_hub_subscriptions.Subscribed_to_My_Sales_Data), but it doesn’t contain any data itself. Instead, it’s a read-only view pointing back to the publisher’s original dataset.
The magic happens when the subscriber queries this linked dataset.
# On the subscriber side
SELECT * FROM partner_org.analytics_hub_subscriptions.Subscribed_to_My_Sales_Data.my_table LIMIT 10;
When this query runs, BigQuery on the subscriber’s side intercepts it. It recognizes that partner_org.analytics_hub_subscriptions.Subscribed_to_My_Sales_Data is a linked dataset originating from Analytics Hub. It then rewrites the query to be executed directly against the publisher’s my_project.my_dataset.my_table in the publisher’s project. The results are then streamed back to the subscriber’s project. This means data isn’t copied, and the publisher retains control over their data, including its access policies and costs.
The surprising truth about Analytics Hub is that it fundamentally shifts the paradigm from data movement to data access. Instead of ETL pipelines, you manage access to shared data resources. The underlying BigQuery engine handles the query routing and execution transparently.
The listing_id is not just a pointer; it’s a resource in its own right, managed via IAM. You grant permissions on the listing resource, not the underlying BigQuery dataset. This allows for fine-grained control over who can discover and subscribe to your data products. The subscription process on the subscriber side creates a reference, not a copy, meaning updates to the publisher’s data are immediately available to subscribers.
What most people don’t realize is that the params for the bq mk --transfer_config command when creating a listing can also include description and restricted_exchange fields, allowing for richer metadata and control over whether the listing is discoverable across all of Google Cloud or only within a specific "exchange" (a curated collection of listings).
The next hurdle you’ll encounter is managing the lifecycle of these shared data products, including versioning and de-provisioning.