Major refactor

.flake8

@@ -0,0 +1,14 @@
+[flake8]
+extend-ignore = E500, E501
+exclude =
+    # No need to traverse our git directory
+    .git,
+    # There's no value in checking cache directories
+    __pycache__,
+    # The conf file is mostly autogenerated, ignore it
+    docs/source/conf.py,
+    # This contains our built documentation
+    build,
+    # This contains builds of flake8 that we don't want to check
+    dist
+max-complexity = 10

.github/workflows/test.yml

@@ -0,0 +1,34 @@
+name: Tests
+
+on:
+  push:
+    branches: [ main, master, cc-development ]
+  pull_request:
+    branches: [ main, master, cc-development ]
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: ["3.12"]
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+
+      - name: Set up Python ${{ matrix.python-version }}
+        uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python-version }}
+          cache: "pip"
+
+      - name: Install package and test dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install -e ".[dev]"
+
+      - name: Run tests with pytest
+        run: |
+          pytest -v --tb=short

.gitignore

@@ -0,0 +1,10 @@
+_*
+.*
+_*/
+.*/
+*.pyc
+__pycache__/
+build/
+dist/
+*.egg-info/
+.pytest_cache/

CHANGELOG.md

@@ -0,0 +1,11 @@
+# Changelog
+
+v0.0.2
+------
+- major refactor
+- added tests
+- package scaffolding
+
+v0.0.1
+------
+- initial version

Makefile

@@ -0,0 +1,30 @@
+# Makefile for building and publishing the package
+#
+# Usage:
+#   make build    - Build source and wheel distributions
+#   make upload   - Upload distributions to PyPI (requires twine credentials)
+#   make publish  - Build and upload in one step
+#   make clean    - Remove build artifacts
+
+.PHONY: build upload publish clean help
+
+# Default target
+help:
+	@echo "Available targets:"
+	@echo "  make build    - Build sdist and wheel into dist/"
+	@echo "  make upload   - Upload dist/* to PyPI via twine"
+	@echo "  make publish  - Build then upload"
+	@echo "  make clean    - Remove dist/ and build artifacts"
+
+build:
+	python -m build
+
+upload:
+	python -m twine upload dist/* --verbose
+
+publish: build upload
+
+clean:
+	rm -rf dist/ build/ *.egg-info src/*.egg-info
+	find . -type d -name __pycache__ -exec rm -rf {} + 2>/dev/null || true
+	find . -type f -name '*.pyc' -delete 2>/dev/null || true

README.md

@@ -1,50 +1,65 @@
 # IPython plugin for DataLinter
 
-> Warning: This work is still very experimental.
+[![License](https://img.shields.io/pypi/l/datalintermagic)](https://pypi.org/project/datalintermagic/)
+[![Tests](https://github.com/zgornel/Ipython-datalinter/actions/workflows/tests.yml/badge.svg?branch=main)](https://github.com/zgornel/Ipython-datalinter/actions/workflows/tests.yml?query=branch%3Amain)
+[![PyPI version](https://img.shields.io/pypi/v/datalintermagic)](https://pypi.org/project/datalintermagic/)
+[![Python versions](https://img.shields.io/pypi/pyversions/datalintermagic)](https://pypi.org/project/datalintermagic/)
+![til](./gifs/jupyter.gif)
 
 This is a Ipython magic that allows one to run [DataLinter](https://github.com/zgornel/DataLinter) in Jupyter notebooks.
 
-[![License](http://img.shields.io/badge/license-GPL-brightgreen.svg?style=flat)](LICENSE)
+## Installation
 
-![til](./gifs/jupyter.gif)
+Make sure you have up-to-date stable versions of [IPython](https://ipython.org/) and [Jupyter](https://jupyter.org/). In order for the linter to work, both the Jupyter plugin and DataLinter (server in a Docker image) need to be installed in the system.
 
-## Installation
+### Jupyter plugin package
 
-Make sure you have up-to-date stable versions of [IPython](https://ipython.org/) and [Jupyter](https://jupyter.org/).
+The Jupyter plugin can be installed with `pip`:
+```
+pip install datalintermagic
+```
 
 ### DataLinter
 
-Install [DataLinter](https://github.com/zgornel/DataLinter) by pulling the Docker image:
-```
+[DataLinter](https://github.com/zgornel/DataLinter) installation is done with:
+```bash
 docker pull ghcr.io/zgornel/datalinter-compiled:latest
 ```
-and start it with:
-```
-docker run -it --rm -p10000:10000 \
+The command to start the linting server is:
+```bash
+docker run -d --rm -p10000:10000 \
     ghcr.io/zgornel/datalinter-compiled:latest \
         /datalinterserver/bin/datalinterserver \
             -i 0.0.0.0 \
-            --config-path /datalinter/config/default.toml \
-            --log-level debug
+            --config-path /datalinter/config/python_modelling_config.toml \
+            --log-level info
 ```
 If the server starts correctly, it should display something like:
 ```
-Warning: KB file not correctly specified, defaults will be used.
-└ @ datalinterserver /DataLinter/apps/datalinterserver/src/datalinterserver.jl:83
 [ Info: • Data linting server online @0.0.0.0:10000...
-[ Info: Listening on: 0.0.0.0:10000, thread id: 1
 ```
 
-### Ipython-datalinter magic
-Clone the repository:
-```
-git clone https://github.com/zgornel/Ipython-datalinter
-```
+### Quick start
+```python
+%reload_ext datalintermagic
+import numpy as np
+X = np.random.random((10,100))
 
-To run the demo, enter the `Ipython-datalinter` folder, run
-```
-jupyter notebook
+%add --tracked-variable X  --data-header --data-delim ','
+%lintline X --show-stats
 ```
-and open `ipython/demo.ipynb`.
 
-> Note: In order for the `datalintermagics` magic to be available, the `datalintermagics` folder/module needs to be available to the jupyter notebook. The easy way to do this is to have the `datalintermagics` directory and notebooks in the same location.
+Further examples of linter usage in Jupyter are found in this [notebook](https://github.com/zgornel/Ipython-datalinter/blob/main/notebooks/workflow_example.ipynb).
+## License
+
+This code has an GNU GPLv3 license.
+
+## Contributing
+
+To report a bug or request a feature, please [file an issue](https://github.com/zgornel/Ipython-datalinter/issues/new).
+
+Recent changes can be found in [CHANGELOG.md](CHANGELOG.md).
+
+## References
+
+[1] https://en.wikipedia.org/wiki/Lint_(software)