Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ci-templates/azure-devops/azure-pipelines.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ stages:
version: 22.x
# An exact version: change it when you upgrade, as
# https://docs.ingestron.io/docs/how-to/upgrade#choose-the-version-you-run says.
- script: npm install -g ingestron@2.3.0
- script: npm install -g ingestron@2.3.1
displayName: Install Ingestron
- script: ingestron check
displayName: ingestron check
Expand Down
2 changes: 1 addition & 1 deletion ci-templates/azure-devops/ingestron-check.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ steps:
version: 22.x
# An exact version: change it when you upgrade, as
# https://docs.ingestron.io/docs/how-to/upgrade#choose-the-version-you-run says.
- script: npm install -g ingestron@2.3.0
- script: npm install -g ingestron@2.3.1
displayName: Install Ingestron
- script: ingestron check
displayName: ingestron check
2 changes: 1 addition & 1 deletion ci-templates/github-actions/deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ jobs:
- uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
with:
node-version: 22
- run: npm install -g ingestron@2.3.0
- run: npm install -g ingestron@2.3.1
- run: ingestron check

build:
Expand Down
2 changes: 1 addition & 1 deletion ci-templates/github-actions/ingestron-check.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,5 +18,5 @@ jobs:
- uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
with:
node-version: 22
- run: npm install -g ingestron@2.3.0
- run: npm install -g ingestron@2.3.1
- run: ingestron check
2 changes: 1 addition & 1 deletion content/docs/(reference)/reference/versions.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -29,5 +29,5 @@ Connector maturity, such as Public Preview or Beta, is shown on each
The [deployment templates](/docs/how-to/deploy-through-environments) pin the
tools their stages run: `@microsoft/azure-data-factory-utilities` 1.0.3, the
Databricks CLI v1.23.0, and sqlcmd (Go) v1.10.0. Their check stage installs
`ingestron@2.3.0`, and each GitHub action is pinned to a commit, with its
`ingestron@2.3.1`, and each GitHub action is pinned to a commit, with its
version beside it.
4 changes: 2 additions & 2 deletions content/docs/(reference)/tools/vscode.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -279,8 +279,8 @@ Explorer.
<Screenshot
light="/images/vscode/reference-light.png"
dark="/images/vscode/reference-dark.png"
alt="The Ingestron tab beside standard.yaml, with the cursor on mode: incremental. The Reference section shows mode, a setting of adf-copy: what each window copies, its values full and incremental with incremental marked, the default full, that it is usually set in a flow or transform, and a Reference link."
caption="The Reference section follows the cursor: here mode, a setting of adf-copy, with its values, the one written marked, and its default."
alt="The Ingestron tab beside standard.yaml, with the cursor on incremental in load: default: incremental. The Reference section shows default, a field of Standard: how data is loaded when no rule matches and the flow says nothing, its values files, incremental, changes and full with incremental marked, and a Reference link."
caption="The Reference section follows the cursor: here load's default, with its values and the one written marked."
/>

## See what will be generated
Expand Down
2 changes: 1 addition & 1 deletion content/docs/how-to/(build)/check-in-ci.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ from the definitions project folder on every pull request: it fails when
someone changed the definitions without generating, so the files you deploy
are always what the definitions say. It needs only Node.js 22.13 or later:
each template installs an exact version of the command from npm,
`npm install -g ingestron@2.3.0`, then runs it. Change the version when you
`npm install -g ingestron@2.3.1`, then runs it. Change the version when you
upgrade, as
[Upgrade Ingestron and its packs](/docs/how-to/upgrade#choose-the-version-you-run)
shows, so a new release never fails a pull request that changed nothing.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ The tools are pinned: Microsoft's `@microsoft/azure-data-factory-utilities`
1.0.3, the Databricks CLI v1.23.0 and sqlcmd (Go) v1.10.0, each download
checked by its SHA-256 where the template downloads it. Each GitHub action is
pinned to a full commit, with its version beside it, and the check installs an
exact Ingestron, `ingestron@2.3.0`: change them when you upgrade. The ADF
exact Ingestron, `ingestron@2.3.1`: change them when you upgrade. The ADF
build fetches its build tool from `adf.azure.com`, so the build stage needs
network access there.

Expand Down
2 changes: 1 addition & 1 deletion content/docs/how-to/(build)/start-from-a-template.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -176,7 +176,7 @@ written for `dev` and for `prod`, counts once for `init` and twice for `plan`.
For the project above, plan ends:

```text
9 values are still REPLACE_ME; ingestron check fails until each is written.
9 places are still REPLACE_ME; ingestron check fails until each is written.
```

Then make the rest yours:
Expand Down
24 changes: 17 additions & 7 deletions content/docs/how-to/(standards)/ask-the-platform-team.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -192,17 +192,27 @@ how to supply them in CI.

## When the definitions have problems

Any other error stops the request. The request is then only its title and
one line, and the command exits with `1`:
Any other error stops the request. The request is then its title, a note and
each error, written as `ingestron plan` writes a problem, and the command
exits with `1`. For an environment the project does not declare:

```text
# Platform request (dev)
````text
# Platform request (nope)

The definitions have problems; fix them first:

The definitions have problems; fix them first.
```text
project.yaml:7:3 error UNKNOWN_ENVIRONMENT
Environment "nope" is not declared in project.yaml.
Declared environments: dev.
```
````

Run `ingestron plan` to see the problems. Unset variables alone never do
this: with nothing else wrong, the command exits with `0`.
The request reads the value of each variable that is set, so it can find an
error that `ingestron plan` does not: a value the deploying pipeline supplies,
written in your environment, that its setting does not accept. Unset variables
alone never stop the request: with nothing else wrong, the command exits with
`0`.

## Next

Expand Down
20 changes: 17 additions & 3 deletions content/docs/how-to/(standards)/upgrade.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ choose to upgrade. The [CI templates](/docs/how-to/check-in-ci) and the
[deployment templates](/docs/how-to/deploy-through-environments) do:

```sh
npm install -g ingestron@2.3.0
npm install -g ingestron@2.3.1
```

`npm ls -g ingestron` shows the version a computer has installed.
Expand Down Expand Up @@ -95,7 +95,7 @@ next; see [Deploy through environments](/docs/how-to/deploy-through-environments

### The standard and the folders

1. Install `ingestron@2.3.0` and plan. Every pack is version 2, so a standard that
1. Install `ingestron@2.3.1` and plan. Every pack is version 2, so a standard that
names `databricks@1`, `adf@1`, `synapse@1` or `duckdb@1` stops with
`PACK_VERSION`, once for all the environments that have it:

Expand Down Expand Up @@ -135,7 +135,7 @@ next; see [Deploy through environments](/docs/how-to/deploy-through-environments
[`DEPLOY_VALUE`](/docs/reference/errors#DEPLOY_VALUE): write it in
`project.yaml` for each environment. See
[Supplied when deploying](/docs/concepts/yaml-references#supplied-when-deploying).
5. Change CI: the check templates install `ingestron@2.3.0`, and the deploy
5. Change CI: the check templates install `ingestron@2.3.1`, and the deploy
pipelines deploy `build/<pack>/` by target, parameter file or script, as
[Deploy through environments](/docs/how-to/deploy-through-environments)
shows.
Expand Down Expand Up @@ -517,6 +517,20 @@ For code that calls the packages:
- `@ingestron/authoring` 2.3 adds `ProjectSnapshot.replaceMeHelp`, what a
`REPLACE_ME` stands for, and lists a table's intent in its hover.

Release 2.3.1 changes only what the command and the editor say; the generated
files are the same:

- When an error stops `ingestron request`, the request lists each error, as
`ingestron plan` writes it, after its note. See
[When the definitions have problems](/docs/how-to/ask-the-platform-team#when-the-definitions-have-problems).
- `plan` and `apply` count the places still `REPLACE_ME`:
`8 places are still REPLACE_ME; …`. `init` counts the README's rows, one
per value, so a value written for each environment is one row there and
one place per environment here.
- In VS Code 2.2.1, completion offers a method's settings that are now intent,
such as `mode` and `sequence_by`, after the others, each with the intent
that replaces it.

## What changed in a release

Recent releases are commits whose message starts with the version, such as
Expand Down
2 changes: 1 addition & 1 deletion content/docs/platforms/adf.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -185,7 +185,7 @@ appears on every plan, apply and check while the factory uses a runtime, as

For the project created without `--set`, plan also warns about each value
still `REPLACE_ME`, and ends
`9 values are still REPLACE_ME; ingestron check fails until each is written.`:
`9 places are still REPLACE_ME; ingestron check fails until each is written.`:
it counts each place a value is written, where `init` counts names. The files
are still written, and `ingestron check` fails until each is written. Run
`ingestron apply` to write the files.
Expand Down
2 changes: 1 addition & 1 deletion content/docs/platforms/databricks.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -168,7 +168,7 @@ create src/retail/silver/daily_sales.sql

For the project created without `--set`, plan first warns about each value
still `REPLACE_ME`, then lists the same files, and ends
`8 values are still REPLACE_ME; ingestron check fails until each is written.`:
`8 places are still REPLACE_ME; ingestron check fails until each is written.`:
it counts each place a value is written, where `init` counts names. Plan and
apply still write the files, and `ingestron check` fails until each is
written. Run `ingestron apply` to write them. If plan reports a problem
Expand Down
6 changes: 3 additions & 3 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,9 @@
"sync:vscode": "node scripts/sync-vscode-commands.mjs"
},
"dependencies": {
"@ingestron/authoring": "^2.3.0",
"@ingestron/cli": "^2.3.0",
"@ingestron/core": "^1.1.0",
"@ingestron/authoring": "^2.3.1",
"@ingestron/cli": "^2.3.1",
"@ingestron/core": "^1.1.1",
"@ingestron/definitions": "^1.13.0",
"@ingestron/pack-adf": "^2.3.0",
"@ingestron/pack-databricks": "^2.3.0",
Expand Down
34 changes: 17 additions & 17 deletions pnpm-lock.yaml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Binary file modified public/images/vscode/platform-request-dark.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified public/images/vscode/platform-request-light.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified public/images/vscode/reference-dark.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified public/images/vscode/reference-light.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion scripts/ci-templates.test.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ test("every YAML template parses, and the folder holds no other kind of file", (
* The ingestron bundle the templates install: an exact version, the one the
* docs build with (test/standards-and-teams.test.ts checks the cli's version).
*/
const bundle = "2.3.0";
const bundle = "2.3.1";

test("each check installs an exact ingestron from npm, then runs ingestron check", () => {
for (const file of [...checks, "github-actions/deploy.yml", "azure-devops/azure-pipelines.yml"]) {
Expand Down
2 changes: 1 addition & 1 deletion test/extracts.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -243,7 +243,7 @@ describe("the platform request", () => {

it("fails with the request's output when the request fails", () => {
expect(() => exampleRequest("landing-only", "nope")).toThrow(
'Example "landing-only": the request for nope fails:\n# Platform request (nope)\n\nThe definitions have problems; fix them first.\n',
'Example "landing-only": the request for nope fails:\n# Platform request (nope)\n\nThe definitions have problems; fix them first:\n\n```text\nproject.yaml:5:3 error UNKNOWN_ENVIRONMENT\n Environment "nope" is not declared in project.yaml.\n Declared environments: dev, prod.\n```\n',
);
});

Expand Down
2 changes: 1 addition & 1 deletion test/platform-pages.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -291,6 +291,6 @@ describe("reference/versions", () => {
expect(tools).toContain(`\`@microsoft/azure-data-factory-utilities\` ${adf}`);
expect(tools).toContain(`Databricks CLI ${databricks}`);
expect(tools).toContain(`sqlcmd (Go) ${sqlcmd}`);
expect(tools).toContain("`ingestron@2.3.0`");
expect(tools).toContain("`ingestron@2.3.1`");
});
});
26 changes: 14 additions & 12 deletions test/standards-and-teams.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -39,8 +39,11 @@ const joined = (text: string) =>
);

/** The page's request command, run in a folder: its exit code and output. */
const request = (page: string, cwd: string) =>
cli(cwd, command(commands(section(page, "## What each route asks for")), "request"));
const request = (page: string, cwd: string, environment?: string) => {
const args = command(commands(section(page, "## What each route asks for")), "request");
if (environment) args[args.indexOf("--env") + 1] = environment;
return cli(cwd, args);
};

/** The `shop` project the request guide creates from the minimal starter, with its own commands. */
function shop(page: string): string {
Expand Down Expand Up @@ -124,17 +127,16 @@ describe("how-to/ask-the-platform-team", () => {
expect(values).toContain("`.env.dev`");
});

it("is only the title and one line when the definitions have problems, and exits with 1", () => {
it("lists the errors when the definitions have problems, as the page shows, and exits with 1", () => {
const problems = section(page, "## When the definitions have problems");
const cwd = shop(page);
const connection = join(cwd, "connections/shop.yaml");
writeFileSync(
connection,
readFileSync(connection, "utf8").replace("connector: files", "connector: nothing"),
);
const broken = request(page, cwd);
// The page's example: an environment the project does not declare. Its block holds a
// fenced block, so it is fenced with four backticks.
const shown = /^````text\n([\s\S]*?)^````$/m.exec(problems)?.[1];
expect(shown, "the four-backtick text block").toBeDefined();
expect(problems).toContain("For an environment the project does not declare:");
const broken = request(page, shop(page), "nope");
expect(broken.code).toBe(1);
expect(broken.output.trim()).toBe(fences(problems, "text")[0]!.trim());
expect(broken.output.trim()).toBe(shown!.trim());
expect(problems).toContain("exits with `1`");
});
});
Expand Down Expand Up @@ -324,7 +326,7 @@ describe("how-to/upgrade", () => {

it("pins the exact bundle these pages describe, as the templates install it", () => {
// The ingestron bundle these pages describe: the installed cli's version.
const bundle = "2.3.0";
const bundle = "2.3.1";
const choose = section(page, "## Choose the version you run");
const version = JSON.parse(
readFileSync(join(process.cwd(), "node_modules/@ingestron/cli/package.json"), "utf8"),
Expand Down
Loading