Karmaflu fails to execute in directory with only genome and metadata files

[fanninpm]

Describe the bug
The karmaflu shell script does not know that it is being executed from a different place that where it is being stored, so Snakemake gets passed incorrect parameters.

Impact
Not applicable, since this is just a test.

To Reproduce
Steps to reproduce the behavior:

1. mkdir my_karmaflu_run
2. cp DAIS_ribosome.gen metadata.csv my_karmaflu_run
3. cd my_karmaflu_run
4. apptainer run --bind "${PWD}":/opt/karmaflu docker://cdcgov/karmaflu:v0.0.0 classify --genome DAIS_ribosome.gen --metadata metadata.csv (I expect that IT will never allow us to use Docker under any circumstances!)
5. See error

Expected behavior
The workflow should run to completion without errors.

Screenshots
If applicable, add screenshots to help explain your problem.

Logs

Traceback (most recent call last):

  File "/opt/conda/envs/Snakemake/lib/python3.12/site-packages/snakemake/cli.py", line 2173, in args_to_api
    config_settings=ConfigSettings(
                    ^^^^^^^^^^^^^^^

  File "<string>", line 7, in __init__

  File "/opt/conda/envs/Snakemake/lib/python3.12/site-packages/snakemake/settings/types.py", line 408, in __post_init__
    self.overwrite_config = self._get_overwrite_config()
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/opt/conda/envs/Snakemake/lib/python3.12/site-packages/snakemake/settings/types.py", line 416, in _get_overwrite_config
    update_config(overwrite_config, load_configfile(f))
                                    ^^^^^^^^^^^^^^^^^^

  File "/opt/conda/envs/Snakemake/lib/python3.12/site-packages/snakemake/common/configfile.py", line 38, in load_configfile
    config = _load_configfile(configpath)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/opt/conda/envs/Snakemake/lib/python3.12/site-packages/snakemake/common/configfile.py", line 12, in _load_configfile
    obj = open(configpath_or_obj, encoding="utf-8")
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

FileNotFoundError: [Errno 2] No such file or directory: 'workflows/classify_config.yaml'

Desktop (please complete the following information):

• OS: [e.g. iOS] SLES
• Browser [e.g. chrome, safari] N/A
• Version [e.g. 22] 15.7

Smartphone (please complete the following information):

• Device: [e.g. iPhone6] N/A
• OS: [e.g. iOS8.1] N/A
• Browser [e.g. stock browser, safari] N/A
• Version [e.g. 22] N/A

Additional context
cc @nbx0 @friesac

[fanninpm]

A potential fix: use --overlay instead of --bind.

The real fix: put EXEC_DIR=$(dirname $(realpath $0)) into the karmaflu shell script, then use that where there are relative paths.

[fanninpm]

Food for thought:

cd $(which karmaflu)  # or change directory to where the 'karmaflu' shellscript is
mkdir data
# copy DAIS_ribosome.gen and metadata.csv into data/
cd data
../karmaflu classify --genome DAIS_ribosome.gen --metadata metadata.csv

If karmaflu were sane, this should work!

[fanninpm]

I came up with this crazy workaround:

apptainer exec --bind "${PWD}:/opt/karmaflu/data --writable-tmpfs docker://cdcgov/karmaflu:v0.0.0 /bin/bash <<EOF
cd /opt/karmaflu
mkdir data/snakelog
ln -s data/snakelog snakelog
START_KARMAFLU_TIME=\$(date +"%Y-%m-%dT%H:%M:%S%z")
./karmaflu classify --genome data/DAIS_ribosome.gen --metadata data/metadata.csv
find . -path ./data -prune -o -type f -newermt "\${START_KARMAFLU_TIME}" -exec cp --parents -t ./data {} +
EOF

[uee9]

Hi @fanninpm Thanks for testing.

Does your system have singularity or just apptainer?

Thanks,
Matt

[fanninpm]

Does your system have singularity or just apptainer?

On systems with Apptainer only, the singularity command is symlinked to the apptainer executable.

[uee9]

So the bash script is just a wrapper around snakemake. Take a look at the new branch. That should simplify the snakemake linkages. It calls sigularity from snakemake using rule directives.

Most folks here also do not have access to docker on enterprise-level compute so I think the singularity/apptainer process is probably best.

./install.sh ## this will install snakemake and micromamba locally within the KARMAflu git repo. 

## make the micromamba cli active
export MAMBA_ROOT_PREFIX=$(pwd)/micromamba
eval "$(payloads/micromamba shell hook --shell=bash)"

$ micromamba run -n Snakemake ./karmaflu annotate --fasta testdata/test2.fasta 
##The wrapper uses singularity by default no need for docker.

## a directory with $(date -I) with the results.