Revise setup instructions for macOS

[moylstmed360]

Three fixes found while working through MACOS_SETUP.md on a fresh machine.
pip install -e ".[dev]" no longer works, the dev, test and docs dependencies are now Poetry groups rather than extras, and pip does not understand groups. The "pytest<9" cap is needed today: pytest 9.1.1 with pytest-cases 3.10.1 raisesIdMaker.__init__() takes 7 positional arguments but 8 were given. Happy to drop it if you would rather not pin in the docs.

[kebarr]

Running through these commands fixed the problems I was experiencing with the previous setup for MacOS instructions on a clean machine,.

In the long run I think it would be nicer to use poetry to keep it consistent between OS's.

Tested with a few of the notebooks.

When trying to build the docs, two dependencies were missing, fixed by running:

conda install sphinx-design pydata-sphinx-theme

[moylstmed360]

Thanks Katie, really appreciate you giving it a go on a clean machine, great to see it hold up there too! Good tip on the docs package, and agreed, Poetry would be nicer in the long run.

[ahernank]

Thanks @moylstmed360, @kebarr. Would you mind taking a look at the previous PR (#1205) that addresses some of these issues, and incorporating anything from it into this updated version (inc. @kebarr's notes on the extra dependecies)? We can then close the older one, and merge this one after review.

[kebarr]

I tested the previous PR (#1205) in a clean environment and couldn't get the tests or docs to run, unsure if this is due to other parts of the code base that have changed since then. Have discussed with @moylstmed360, and he will work based on his PR to ensure:

sphinx-design
pydata-sphinx-theme

are included.

[moylstmed360]

Thanks @kebarr for testing #1205 and confirming what was missing. Folded it into this PR, so #1205 can be closed.

pyproject.toml, added a [project] table with dependencies and the dev, test and docs groups. #1205 had the groups but no dependencies, so pip installed the tooling and not the runtime stack, and the docs build failed on ModuleNotFoundError: plotly. sphinx-design and pydata-sphinx-theme are in the docs group.

MACOS_SETUP.md, the install step is now pip install -e ".[dev,test]".

Verified in a clean venv: pip install -e ".[dev,test,docs]" then make html, build succeeds.

Worth flagging: runtime dependencies are now declared twice in pyproject.toml, under [project].dependencies and [tool.poetry.dependencies], and only the Poetry ones carry full constraints. Consolidating into [project] would remove the risk of drift, and could be a separate PR.

[kebarr]

Testing in a clean env got same issues with running tests as the other PR. Can you please run:

conda env export | grep -v "^prefix: " > environment.yml

And attach the results so I can see where our setups differ.

I double checked that poetry setup isn't impacted by any of the changes and it isn't.

Mine are attached, but.I also get the attached stdout. I think this could be (or at least is being made worse by) the combination of using conda and pip for dependency management. It is possible that soomething like this could be useful as it comes with integrated pip support and supports pyproject.toml-managed packages.

I get the following error, which is currently being discussed in the python-pytest-cases repo, when running the tests:

Traceback (most recent call last):
  File "/opt/homebrew/Caskroom/miniconda/base/envs/malariagen_pr2/bin/pytest", line 6, in <module>
    sys.exit(_console_main())
             ^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniconda/base/envs/malariagen_pr2/lib/python3.11/site-packages/_pytest/config/__init__.py", line 253, in _console_main
    code = _main(prog=_get_prog_name(sys.argv))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniconda/base/envs/malariagen_pr2/lib/python3.11/site-packages/_pytest/config/__init__.py", line 223, in _main
    config = _prepareconfig(new_args, plugins, prog=prog)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniconda/base/envs/malariagen_pr2/lib/python3.11/site-packages/_pytest/config/__init__.py", line 410, in _prepareconfig
    config: Config = pluginmanager.hook.pytest_cmdline_parse(
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniconda/base/envs/malariagen_pr2/lib/python3.11/site-packages/pluggy/_hooks.py", line 512, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniconda/base/envs/malariagen_pr2/lib/python3.11/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniconda/base/envs/malariagen_pr2/lib/python3.11/site-packages/pluggy/_callers.py", line 167, in _multicall
    raise exception
  File "/opt/homebrew/Caskroom/miniconda/base/envs/malariagen_pr2/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
  File "/opt/homebrew/Caskroom/miniconda/base/envs/malariagen_pr2/lib/python3.11/site-packages/_pytest/helpconfig.py", line 124, in pytest_cmdline_parse
    config = yield
             ^^^^^
  File "/opt/homebrew/Caskroom/miniconda/base/envs/malariagen_pr2/lib/python3.11/site-packages/pluggy/_callers.py", line 121, in _multicall
    res = hook_impl.function(*args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniconda/base/envs/malariagen_pr2/lib/python3.11/site-packages/_pytest/config/__init__.py", line 1232, in pytest_cmdline_parse
    self.parse(args)
  File "/opt/homebrew/Caskroom/miniconda/base/envs/malariagen_pr2/lib/python3.11/site-packages/_pytest/config/__init__.py", line 1583, in parse
    self.pluginmanager.load_setuptools_entrypoints("pytest11")
  File "/opt/homebrew/Caskroom/miniconda/base/envs/malariagen_pr2/lib/python3.11/site-packages/pluggy/_manager.py", line 416, in load_setuptools_entrypoints
    plugin = ep.load()
             ^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniconda/base/envs/malariagen_pr2/lib/python3.11/importlib/metadata/__init__.py", line 202, in load
    module = import_module(match.group('module'))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniconda/base/envs/malariagen_pr2/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1126, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "/opt/homebrew/Caskroom/miniconda/base/envs/malariagen_pr2/lib/python3.11/site-packages/_pytest/assertion/rewrite.py", line 188, in exec_module
    exec(co, module.__dict__)
  File "/opt/homebrew/Caskroom/miniconda/base/envs/malariagen_pr2/lib/python3.11/site-packages/pytest_cases/__init__.py", line 8, in <module>
    from .fixture_core1_unions import fixture_union, NOT_USED, unpack_fixture, ignore_unused
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "/opt/homebrew/Caskroom/miniconda/base/envs/malariagen_pr2/lib/python3.11/site-packages/_pytest/assertion/rewrite.py", line 188, in exec_module
    exec(co, module.__dict__)
  File "/opt/homebrew/Caskroom/miniconda/base/envs/malariagen_pr2/lib/python3.11/site-packages/pytest_cases/fixture_core1_unions.py", line 40, in <module>
    from .common_pytest import get_fixture_name, is_marked_parameter_value, get_marked_parameter_values, pytest_fixture, \
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "/opt/homebrew/Caskroom/miniconda/base/envs/malariagen_pr2/lib/python3.11/site-packages/_pytest/assertion/rewrite.py", line 188, in exec_module
    exec(co, module.__dict__)
  File "/opt/homebrew/Caskroom/miniconda/base/envs/malariagen_pr2/lib/python3.11/site-packages/pytest_cases/common_pytest.py", line 644, in <module>
    _idval = IdMaker([], [], None, None, None, None, None)._idval
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: IdMaker.__init__() takes 7 positional arguments but 8 were given

In my normal dev environment, I have pytest version pytest 8.4.2, whereas when building in a new conda environment with pip I get pytest version 9.1.1. Fixing the pytest version to pytest==8.4.2 as per line 55 here works as a workaround for the time being.

environment.yml
conda_env_list_output.txt

[moylstmed360]

Thanks @kebarr, I get the same failure on my Mac. A new conda environment with pip install -e ".[dev,test]" installs pytest 9.1.1. pytest-cases doesn't support pytest 9.1.1, so pytest crashes on startup.

I've capped pytest below 9 in pyproject.toml, for both the pip extras and the Poetry test group, so both routes now resolve pytest 8.4.2.

I created a conda environment with Python 3.11 and installed with pip, as you did, to reproduce the failure. After the pin, pip installs pytest 8.4.2 and pytest -v tests/anoph/test_base.py passes, 44 tests.

I've attached the environment, Python 3.11 with pytest 8.4.2 and pytest-cases 3.10.1, so you can compare it with yours.

Worth flagging: this also downgrades CI's pytest from 9.0.1 to 8.4.2. The conda-lock suggestion could be picked up in a separate issue.

pr1348_environment.yml

[kebarr]

Tested new version- test, docs, and basic commands from an interpreter run. I think this is ready to be merged.

[ahernank]

Thanks for the updates and the thorough testing @kebarr, @moylstmed360 - this is great!