Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 49 additions & 2 deletions registry/workflows.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,58 @@
"type": "workflow",
"description": "Gather competitor pages, extract structured data, and keep it fresh.",
"status": "available",
"requires_backend_capabilities": ["protected_fetch", "extract"],
"requires_backend_capabilities": [
"protected_fetch",
"extract"
],
"requires_auth": true,
"version": "0.1.0",
"path": "workflows/competitor-intelligence",
"tags": ["competitive", "extract", "fetch"]
"tags": [
"competitive",
"extract",
"fetch"
]
},
{
"name": "price-monitoring",
"type": "workflow",
"description": "Watch prices across many product pages, validated once and then run as a managed batch.",
"status": "available",
"requires_backend_capabilities": [
"protected_fetch",
"extract",
"batch"
],
"requires_auth": true,
"version": "0.1.0",
"path": "workflows/price-monitoring",
"tags": [
"pricing",
"fetch",
"extract",
"batch"
]
},
{
"name": "gated-page-extraction",
"type": "workflow",
"description": "Get structured data out of pages that need interaction first, escalating to a browser only with evidence.",
"status": "available",
"requires_backend_capabilities": [
"protected_fetch",
"browser",
"extract"
],
"requires_auth": true,
"version": "0.1.0",
"path": "workflows/gated-page-extraction",
"tags": [
"browser",
"escalation",
"extract",
"fetch"
]
}
]
}
51 changes: 51 additions & 0 deletions workflows/gated-page-extraction/WORKFLOW.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
---
name: gated-page-extraction
description: Get structured data out of pages that need interaction first, escalating to a browser only with evidence.
requires_backend_capabilities: [protected_fetch, browser, extract]
status: available
---

# Workflow: gated-page extraction

Some pages only show their data after an interaction: a login, a "load more",
a form. The rule from [[interact-browser]] applies end to end: **escalation
only** — a browser session costs more than fetch/extract (billed by bandwidth
plus session time, 15-minute cap), so it enters the flow only when the cheap
step has failed with evidence.

## Steps

1. **Try the cheap primitive first** ([[protected-fetch]]):
```
zenrows fetch https://portal.example/reports
```
If the response already carries the data, stop here — the rest of this
workflow is unnecessary cost. If it fails, read the trace before escalating
([[trace-debug]]).
2. **Escalate to a scripted session.** Put the interaction (navigate, type,
click, read) in a script and let the session close itself
([[interact-browser]]):
```
zenrows browser run script.json
```
Prefer the scripted form over an interactive session: interactive sessions
you must `close` yourself, and every open minute bills.
3. **Turn the captured page into data** ([[extract]]):
```
zenrows extract https://portal.example/reports --autoparse
```
Or a selector map when you know the shape:
```
zenrows extract https://portal.example/reports --css '{"report":"h2","total":".sum"}' --validate
```

## Explain

```
zenrows workflow explain gated-page-extraction
```

Each step declares its capability; the toolkit refuses honestly when a required
primitive is not available yet (`ASSET_REQUIRES_CAPABILITY`). Browser Sessions
is GA and on by default; a workspace that opted out re-enables it with
`zenrows policy set allow_browser true` ([[compliance-policy]]).
52 changes: 52 additions & 0 deletions workflows/price-monitoring/WORKFLOW.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
---
name: price-monitoring
description: Watch prices across many product pages, validated once and then run as a managed batch.
requires_backend_capabilities: [protected_fetch, extract, batch]
status: available
---

# Workflow: price monitoring

Three primitives in a row: prove the page with **Fetch**, prove the structure
with **Extract**, then let **Batch** operate the loop. Each step is the cheapest
tool that answers the next question, per [[cost-control]].

## Steps

1. **Prove the page loads with its price.** One protected fetch, waiting for
the selector you care about ([[protected-fetch]]):
```
zenrows fetch https://shop.example/p/123 --wait-for ".price"
```
2. **Prove the structure once.** A selector map that must come back as valid
JSON, or fail loudly ([[extract]]):
```
zenrows extract https://shop.example/p/123 --css '{"title":"h1","price":".price"}' --validate
```
3. **Estimate before spending.** Put the full URL list in `jobs.jsonl` and ask
what the run would cost — this validates the spec locally and needs no key
([[batch-jobs]]):
```
zenrows batch estimate jobs.jsonl
```
4. **Run it managed.** Batch submits every URL, retries transient failures and
stores results, so you do not operate the loop yourself:
```
zenrows batch create jobs.jsonl --wait
zenrows batch results <id> --status all --out results.jsonl
```
5. **Keep it fresh.** Rerun on a schedule (cron or CI). Failures do not restart
the world:
```
zenrows batch retry-failed <id>
```

## Explain

```
zenrows workflow explain price-monitoring
```

Each step declares its capability; the toolkit refuses honestly when a required
primitive is not available yet (`ASSET_REQUIRES_CAPABILITY`). Batch is beta —
`estimate` runs locally today, the cloud steps need beta access.
Loading