Analyse the repository with CodeQL

[DZPM]

Turns on code scanning with a workflow rather than with the default setup, because this repository needs path exclusions that the default cannot express.

This one is independent of the open stack: it comes off edition and touches one new file, so it can be reviewed and merged on its own.

Three languages, and the reason for each:

actions is the most useful one here. The queries find a token with wider permissions than the job needs, an action pinned to a tag instead of a commit, and untrusted input inside a workflow expression. Note that the query for an unpinned action has medium precision, so the default query suite does not run it: this workflow asks for security-extended so that it does. Expect it to report one thing we already know about, contents: write in gh-pages.yml, which the deploy to master needs.

python covers bin/check-content and bin/check-html-safety. They have no .py extension, only a shebang, so whether CodeQL finds them at all was checked against the extractor source rather than assumed: it accepts a file with no extension when its first bytes match a python shebang, and the five shell scripts in bin/ correctly do not match. In the first run, the python job's log should report two files.

javascript covers our own scripts and the inline scripts in the templates. The vendored libraries under themes/pybcn_theme/assets/vendor/ and the frozen snapshots under static/archives/ are excluded. Those are jQuery, Bootstrap and two retired sites, committed as they are and never edited, and findings in code nobody will touch are how a security tool gets ignored. CodeQL skips *.min.js on its own.

Every action is pinned to a full commit SHA with its version in a comment, which is the style pr.yml and gh-pages.yml already use, and each SHA was checked against the GitHub API rather than written from memory. The job gets contents: read and security-events: write and nothing else.

It runs on push and on pull request against edition, and weekly on Monday at 05:37 UTC. The odd minute is on purpose: GitHub queues every top-of-the-hour schedule together and delays or drops them under load.

After the merge

The first successful run turns code scanning on by itself, so there is nothing to enable first. Results appear under the Security tab, with one entry per language.

One thing not to do: in Settings, under Code security, do not click "Set up" on the default CodeQL configuration. Default setup rejects the results of an advanced workflow, so it would turn this off.

[github-advanced-security]

You are seeing this message because GitHub Code Scanning has recently been set up for this repository, or this pull request contains the workflow file for the Code Scanning tool.

What Enabling Code Scanning Means:

• The 'Security' tab will display more code scanning analysis results (e.g., for the default branch).
• Depending on your configuration and choice of analysis tool, future pull requests will be annotated with code scanning analysis results.
• You will be able to see the analysis results for the pull request's branch on this overview once the scans have completed and the checks have passed.

For more information about GitHub Code Scanning, check out the documentation.

[DZPM]

The first run checked the two things that were assumptions when I wrote this.

CodeQL does find the Python scripts. They have no .py extension, so whether the extractor would pick them up at all was read off its source rather than observed. The job log of this run says it extracted exactly two files:

bin/check-content
bin/check-html-safety

and none of the five shell scripts next to them. When the open stack merges there will be five Python scripts in bin/, and they will be picked up the same way with no change here.

The workflow analyses itself and passes. Analyze actions ran over all three workflows, this one included, in 45 seconds with no finding. That matters because security-extended turns on the query for an action pinned to a tag instead of a commit, which is exactly the kind of thing a CodeQL workflow tends to be guilty of itself.

All seven checks are green.

What to expect once this merges

The first successful run on edition turns code scanning on by itself. There is nothing to enable first: the repository currently reports default-setup: not-configured, so there is no conflict to clear.

I expect one alert, on gh-pages.yml: contents: write, which the deploy to master needs. It is correct and it should be dismissed as "Won't fix" with a note, rather than left open for ever as an alert nobody reads.

One thing not to do afterwards: in Settings > Advanced Security > Code scanning, do not click "Set up" on the default CodeQL configuration. Default setup rejects the results of an advanced workflow, so it would quietly turn this off. That page should read "Advanced".

There is also a Protection rules section there with a "Check Failure" threshold, which can make a pull request go red on a finding. I would leave it alone for a couple of weeks: set before we know how much noise this makes, the first false positive blocks somebody's pull request and the whole thing gets switched off.

Unrelated, while we were in that screen

Secret scanning and push protection are now on for this repository. They were off. Push protection is the one that earns its keep: it rejects a git push that carries a key, before it is ever published.

The first scan of the full history since 2013 found zero alerts, which is worth knowing: nothing has ever been leaked here.