Repository navigation
Stop CI from committing generated files and check they are up to date - #38
Merged
Merged
Conversation
CI previously regenerated the TSV files and lexicon_statistics.md and pushed them back to the repository using a token with write access. That exposed the write token to every dependency and script, interpolated github.ref into a shell command, raced between overlapping runs, and cannot work once master is branch protected. Contributors now generate these files themselves and commit them in their pull request. CI regenerates them and fails if they differ from what is committed. CI workflow: - Use a read-only token and set persist-credentials: false on checkout - Pin actions/checkout and actions/setup-python to full commit SHAs - Only run on pushes to master and on pull requests - Run txt_to_tsv.py and lexicon_statistics.py before test_all_collections.py so the generated TSV files are the ones that get validated - Add a check that fails if any generated file is modified or missing Documentation: - CONTRIBUTING.md: add steps to set up the development environment and run the generation commands before opening a pull request, note that a maintainer can run them if the contributor cannot, and fix broken links (../ paths and lexicon_resources.json -> language_resources.json) - README.md: add a top-level Development Environment section covering local (venv) and dev container setup, and update the lexicon statistics and Text to TSV sections to say contributors generate the files and CI checks them - .gitignore: ignore .venv/ Fixes UCREL#37
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
CI previously regenerated the TSV files and lexicon_statistics.md and pushed them back to the repository using a token with write access. That exposed the write token to every dependency and script, interpolated github.ref into a shell command, raced between overlapping runs, and cannot work once master is branch protected.
Contributors now generate these files themselves and commit them in their pull request. CI regenerates them and fails if they differ from what is committed.
CI workflow:
Documentation:
Fixes #37