Skip to content

docs: add data connection concept to core-concepts page - #46

Open
tylere wants to merge 4 commits into
mainfrom
tylere/data-connection-concept-docs
Open

tylere wants to merge 4 commits into
mainfrom
tylere/data-connection-concept-docs

Conversation

@tylere

@tylere tylere commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Adds a Data Connections section to the Core Concepts page (docs/about-source/core-concepts.md), between "Objects" and "Data Viewers", and links to the Bring Your Own Bucket page for setup.

A data connection describes a storage location and how Source accesses it. It can be an Amazon S3 bucket (or an S3-compatible store such as Cloudflare R2), an Azure Blob container, or a Google Cloud Storage bucket (DataProvider in source.coop's src/types/data-connection.ts). Source's own storage is configured as data connections too, ones with no owner, and BYOB lets an account add connections to storage it controls.

The section lists the fields in the page's bullet style:

  • Location: a bucket for S3 and GCS; a storage account and container for Azure.
  • Authentication: none (anonymous reads), or federated access through an assumed IAM role. These are the two cases the data proxy serves today (backend_auth.rs in data.source.coop). The schema also defines access-key and SAS-token types, but the proxy refuses them, and the GCP and Azure workload-identity types are scaffolded but not implemented, so the docs leave them out.
  • Base Prefix: with each product's path set beneath it by the prefix template.
  • Allowed Visibilities.
  • Read Only: required for connections without authentication (DataConnectionSchema's refinement).

It also notes that creating connections must be enabled for an account (the CREATE_DATA_CONNECTIONS account flag), which matches the BYOB page's beta note.

Closes #41

Test plan

  • Verify the new section renders correctly on the Core Concepts page
  • Confirm the link to /bring-your-own-bucket resolves (docs/using-source/bring-your-own-bucket.md has that slug)

🤖 Generated with Claude Code

Adds a Data Connections section to the Core Concepts page, defining what
a data connection is, its key configuration fields, and linking to the
Bring Your Own Bucket guide for setup instructions.

Closes #41

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@vercel

vercel Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
docs-source-coop Ready Ready Preview Oct 24, 2026 12:44am UTC

Request Review

A data connection can point at S3 (or an S3-compatible store such as R2), Azure Blob or Google Cloud Storage, not only S3. Source's own storage is also configured as data connections, so the section no longer presents them as BYOB-only.

The authentication bullet now describes what the data proxy serves today: no authentication, or federated access via an assumed IAM role. The read-only rule is tied to having no authentication, not to public visibility, and the section says creating connections must be enabled for an account.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The data proxy prepends a connection's base prefix to every object key whatever the provider, so for Azure it is a folder inside the container. The Base Prefix bullet now says bucket or container, matching the Location bullet.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The section's closing line already points to the BYOB page for setup, matching the single see-also link that ends the Data Products section, so the first mention is now plain text.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@tylere

tylere commented Oct 24, 2026

Copy link
Copy Markdown
Contributor Author

@alukach for accuracy review

@tylere
tylere requested a review from alukach October 24, 2026 00:47

This branch was successfully deployed

1 active deployment
Preview — 5c3d493d Deployed Oct 24, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add data connection concept to docs

1 participant