MicroHS support

[ulysses4ever]

What's achieved

lib:Cabal and cabal-install can build, run, test and install packages with MicroHs, mhs, the way its own package manager, mcabal, does.

AI disclosure: this was written largely by Claude Code (Fable 5.1 or Opus 5.5 depending on the weather outside). I did review it myself and it's a result of many iterations...

Size

The patch looks big but fear not: most of it is contained in the new MHS.hs module and tests.

	Files	Lines added	Lines removed
Total	73	1362	21
cabal-install	4	47	11
Cabal, except MHS.hs	9	38	2
Distribution.Simple.MHS (MHS.hs)	1	915	0
cabal-testsuite	31	277	0
Docs, with the changelog	28	85	8

(Counted with git diff --numstat against master at aeac769.)

Notes:

• The most annoying part and the one to focus your review on is cabal-install. In an ideal world, we'd not need to touch it at all but as the Commit history (below) explains, concessions had to be made.

• Changes to Cabal outside MHS.hs are trivial, which is a strong point.

• Of the lines added to cabal-testsuite, 57 are in the 4 test scripts and 220 in the packages they build.

• 22 of the 28 files in docs are the generated help texts of the commands, which gain --mhs and the mhs program.

Commit history

Each commit ends with how many of the 91 packages of MicroHs's own list build, and run a program, with cabal at that commit, replayed for every commit in the same way, with nothing installed for mhs besides its base (see Testing below and all results in a separate repo).

• 2e55647 Adds the mhs program and the backend Distribution.Simple.MHS, to which configure, build, install and register dispatch: it builds libraries into mhs package files and executables against them, compressed with -z as mcabal does, and installs and registers both, in place and in the store; the packages of mhs itself are described with mhs -L. Every extension is accepted, as mhs ignores the pragmas of those it lacks, and so are reexported-modules, as mhs sees every module of the packages it loads. 3/91 build and run: with base at 0.16.9.0, the solver rejects most packages from Hackage, or falls back to ancient versions of them, until the next commit.
• 1b449d1 Presents the base of mhs at version 4.19.1.0, and the packages it bundles, such as bytestring and text, as installed packages at the versions MicroCabal uses, except hashable, at 1.4.7.0, the API that mhs implements, rather than MicroCabal's 1.0.0.0. 39/91 build and run.
• 6528204 Passes the macros of cabal_macros.h, such as MIN_VERSION_base, to the preprocessor of mhs as -D options. 58/91 build and run.
• b289e8f Rewrites, in cabal-install, the packages that the solver sees, as MicroCabal does: ghc-compat, which has the modules that are part of base with GHC, becomes a dependency of every component, and array and random become their forks array-mhs and random-mhs, so that cabal plans and builds all three from Hackage, and mhs needs nothing installed besides base. Rewriting what the solver sees is enough because cabal-install now builds packages in-library (#11703): it configures each package with the description in its plan, which is rewritten already, rather than running a Setup script that would read the original .cabal file again. 88/91 build and run.
• cbfde3d Puts the C sources of a library beside its package file, where mhs compiles them when it links a program, and gives mhs the include directories of the dependencies with -I. 91/91 build and run.
• 5bcc71f cabal install --lib installs libraries and their dependencies with mhs -Q into the directory where mcabal installs packages, which is the package directory of mhs itself after make minstall, or into the one given with --package-env, so that plain mhs finds them, rather than writing a GHC environment file. 91/91 build and run.

Testing: 3 parts

Part 1: cabal-testsuite

cabal-testsuite/PackageTests/MHS/Build builds and runs a library with a hidden module and an executable using it, with the bounds of a package from Hackage, and is skipped when mhs is not on the PATH. MHS/Macros checks the version macros and is skipped unless cpphs, which mhs runs for CPP, is on the PATH too. MHS/CSources checks the C sources of a library built in place and of one installed in the store from a local repository, and MHS/InstallLib installs a library with cabal install --lib and compiles a program against it with mhs alone:

cabal run cabal-testsuite:cabal-tests -- --with-cabal="$(cabal list-bin cabal)" cabal-testsuite/PackageTests/MHS/Build/cabal.test.hs cabal-testsuite/PackageTests/MHS/Macros/cabal.test.hs cabal-testsuite/PackageTests/MHS/CSources/cabal.test.hs cabal-testsuite/PackageTests/MHS/InstallLib/cabal.test.hs

Part 2: MicroHs's own package list

MicroHs's Makefile.packages is the list of 91 Hackage packages its hackage-ci workflow installs with mcabal on every push. https://github.com/ulysses4ever/cabal-mhs-testsuite builds them with cabal and mhs instead, one at a time and in the same order, each into a store shared by all the steps, with this PR's commits applied; its CI also runs cabal-testsuite/PackageTests/MHS. Every step also runs a small executable depending on the package. As mhs compiles C sources when it links a program, even the executable that does nothing compiles the C sources of the package and its dependencies. For time, splitmix, random, QuickCheck, hspec, terminfo and binary, the executable calls into the package: QuickCheck passes 100 tests, hspec reports 1 example, 0 failures, and terminfo links with the curses library its configure script found. Where mcabal differs from cabal, the replay does what mcabal does, and its README says how: the --git overrides become source-repository-package stanzas, and since mcabal ignores version bounds, a few are lifted in project.extra, each with its reason. mhs has nothing installed besides base; cabal builds ghc-compat and the forks itself.

All the counts in the list of commits are for compiling each package as it is published, with no settings of its own: the project file sets no flags and no compiler options for any package of the list. It only chooses versions and sources: the index-state, the git repositories of MicroHs's Makefile, the three bounds that project.extra lifts and its one constraint, and a copy of hspec-core with its missing dependency added. The tiny executable of each step, which is not one of the packages, is linked with --no-as-needed on Linux.

The version macros take the count from 39 to 58: without them, transformers, which defines its Semigroup instances under #if MIN_VERSION_base(4,9,0), blocked 26 packages, and prettyprinter stopped at its own #error "MIN_VERSION_base macro not defined!". The rewrite takes it from 58 to 88: array from Hackage, which mhs cannot compile, blocked 17 of those 30 packages until array-mhs replaced it, and without ghc-compat, exceptions and call-stack failed on GHC.Stack, and time on Language.Haskell.TH.Syntax. The last three, time, binary and these, through binary, needed the headers of C sources, such as HsTime.h, which the commit for C sources puts on the include path.

One package needs a fix to build with cabal: hspec-core imports async without depending on it, as it compiles its vendored copy of async only for GHC. That works with mcabal because mhs sees every package that mcabal has installed. Cabal gives mhs only the declared dependencies, so the replay adds if impl(mhs) / build-depends: async to a copy of hspec-core, which is the fix that belongs upstream; without it, hspec-core and hspec above it fail.

Running the programs found problems outside cabal, in mhs and in some packages, which mcabal has too; INVENTORY.md lists them with reproductions.

Part 3: Building MicroHs itself (after patching it up)

MicroHs builds its base with mcabal build in lib. We allow cabal build it with mhs too, and the compiler from MicroHs.cabal, once four problems of those .cabal files that mcabal does not mind are fixed:

1. library base has to be library,
2. Data.Float is listed twice,
3. six listed modules of base have no source,
4. MicroHs.cabal lists for every compiler seven modules that only GHC needs.

microhs-cabal-fixes.patch fixes them, and MicroHs still builds with GHC with it. base also needs allow-boot-library-installs: True, as with any boot library.

The base that cabal builds has the same modules, exports and definitions as the one mcabal builds. cabal is no slower than the Makefile and mcabal, and, as it compresses programs as mcabal does, it builds an mhs of exactly the size of the Makefile's: the same machine code, and the same combinator code but for how 52 source locations of base are spelled. The testsuite's README has the times and sizes. An installation of mhs whose base and compiler cabal built passes the tests of cabal-testsuite/PackageTests/MHS and builds and runs the whole package list, and the testsuite's CI builds both on every run (CI run). cabal install exe:mhs does not give a working mhs, though: mhs looks for its packages and runtime only beside its binary, in the layout of its own installer.

INVENTORY.md lists every issue or wrinkle this work found outside cabal: in mhs, the MicroHs repository, MicroCabal, and some packages on Hackage, with reproductions.

Not covered yet

• mhs-options in .cabal files, cabal repl, Backpack, foreign libraries, profiling and shared ways.
• The replay runs a small program per package, but not the test suites of the packages. Running those with mhs is the next step there.
• Upstream: hspec-core should depend on async for mhs, mhs could publish the versions at which its bundled packages should be presented, mhs could pass the libraries of packages after the C files, declare the C functions a program imports and expand $VERSION in mhs.conf to its version; MicroCabal could move to time 1.16.0.1, which fixed the time zone that time 1.15 gets wrong under mhs.

Checklist

• Patches conform to the coding conventions.
• Any changes that could be relevant to users have been recorded in the changelog.
  • Is the change significant? If so, remember to add significance: significant in the changelog file.
• The documentation has been updated, if necessary.
• Manual QA notes have been included.
• Tests have been added. (Ask for help if you don’t know how to write them! Ask for an exemption if tests are too complex for too little coverage!)

QA Notes

A smoke test, and how to extend it in the direction you want to check.

Set up. Install MicroHs v0.16.9.0 from source, which also installs cpphs, the preprocessor that mhs runs for CPP and that the nixpkgs package lacks, and build cabal from this branch:

git clone --branch v0.16.9.0 https://github.com/augustss/MicroHs
make -C MicroHs -j all && make -C MicroHs minstall   # installs into ~/.mcabal; add ~/.mcabal/bin to PATH
cabal build cabal-install:exe:cabal
alias cabal-mhs="$(cabal list-bin cabal)"

Smoke test. A library, and an executable using it:

mkdir -p mhs-demo/src mhs-demo/app && cd mhs-demo
printf 'packages: .\ncompiler: mhs\nwith-compiler: mhs\n' > cabal.project
cat > mhs-demo.cabal <<'EOF'
cabal-version: 3.0
name:          mhs-demo
version:       0.1.0.0

library
  exposed-modules:  MyLib
  build-depends:    base ^>=4.19, bytestring
  hs-source-dirs:   src
  default-language: Haskell2010

executable mhs-demo
  main-is:          Main.hs
  build-depends:    base, mhs-demo
  hs-source-dirs:   app
  default-language: Haskell2010
EOF
cat > src/MyLib.hs <<'EOF'
module MyLib (greeting) where

import qualified Data.ByteString.Char8 as B

greeting :: String
greeting = B.unpack (B.pack "hello from mhs")
EOF
cat > app/Main.hs <<'EOF'
module Main (main) where

import MyLib (greeting)

main :: IO ()
main = putStrLn greeting
EOF
cabal-mhs run mhs-demo   # prints: hello from mhs

Expected: Build profile: -w mhs-0.16.9.0 -O1, and ghc-compat built from Hackage first, as every package depends on it with mhs; base ^>=4.19 and bytestring resolve to what the base of mhs provides. With -v2, the mhs commands show the library compiled into a package file, and the executable compiled with that file and base preloaded (-p) and the search path of mhs cleared (-a).

Extend it. The tests in cabal-testsuite/PackageTests/MHS do most of these:

• A dependency from Hackage: add random to the library's build-depends, and cabal-mhs build builds the fork random-mhs instead.
• CPP: add {-# LANGUAGE CPP #-} to MyLib, and choose code with #if MIN_VERSION_base(4,19,0) (MHS/Macros).
• C sources: give the library c-sources: cbits/twice.c, include-dirs: include and install-includes: twice.h, and MyLib a foreign import ccall "twice.h twice" twice :: CInt -> CInt; mhs compiles the C file when it links the executable (MHS/CSources).
• The store: cabal-mhs install exe:mhs-demo builds the package in the store and installs the executable.
• Plain mhs: cabal-mhs install --lib installs the library where mcabal installs packages, into ~/.mcabal/mhs-0.16.9.0, which make minstall made the package directory of mhs itself, so that mhs Prog.hs compiles a program that imports MyLib (MHS/InstallLib).

🤖 Generated with Claude Code

https://claude.ai/code/session_0176vAefrw8jPKXdh9h1wKuj

[ulysses4ever]

I got as far as solving dependencies: on a freshly cabal inited package with cabal.project like this:

packages: .
compiler: mhs
with-compiler: mhs

I'm getting:

❯ cabal build
Warning: don't know how to find change monitoring files for the installed
package databases for mhs
Resolving dependencies...
Error: [Cabal-7107]
Could not resolve dependencies:
[__0] trying: mhs-test-0.1.0.0 (user goal)
[__1] next goal: base (dependency of mhs-test)
[__1] rejecting: base; 4.21.0.0, 4.20.2.0, 4.20.1.0, 4.20.0.1, 4.20.0.0, 4.19.2.0, 4.19.1.0, 4.19.0.0, 4.18.3.0, 4.18.2.1, 4.18.2.0, 4.18.1.0, 4.18.0.0, 4.17.2.1, 4.17.2.0, 4.17.1.0, 4.17.0.0, 4.16.4.0, 4.16.3.0, 4.16.2.0, 4.16.1.0, 4.16.0.0, 4.15.1.0, 4.15.0.0, 4.14.3.0, 4.14.2.0, 4.14.1.0, 4.14.0.0, 4.13.0.0, 4.12.0.0, 4.11.1.0, 4.11.0.0, 4.10.1.0, 4.10.0.0, 4.9.1.0, 4.9.0.0, 4.8.2.0, 4.8.1.0, 4.8.0.0, 4.7.0.2, 4.7.0.1, 4.7.0.0, 4.6.0.1, 4.6.0.0, 4.5.1.0, 4.5.0.0, 4.4.1.0, 4.4.0.0, 4.3.1.0, 4.3.0.0, 4.2.0.2, 4.2.0.1, 4.2.0.0, 4.1.0.0, 4.0.0.0, 3.0.3.2, 3.0.3.1 (constraint from non-reinstallable package requires installed instance)
[__1] fail (backjumping, conflict set: base, mhs-test)
After searching the rest of the dependency tree exhaustively, these were the goals I've had most trouble fulfilling: base, mhs-test

I guess I need to better understand how MHS treats base.

[ulysses4ever]

@augustss I hope to get your input here. This work (allowing cabal to drive mhs) uncovered many things some of which need fixing on mhs' side if you are willing to review. The full list is here: https://github.com/ulysses4ever/cabal-mhs-testsuite/blob/main/INVENTORY.md I can start submitting them to the mhs repo if this seems useful to you. Also, I'm happy to discuss them elsewhere if that's preferable.

[augustss]

Let me look at the list.

[Copilot]

🟡 Changes recommended

Unresolved backend correctness issues can break supported package builds and leave installed-package information stale.

6 open issues

• Reject unsupported renamed module reexports · New
• Inject ghc-compat independently into each component · New
• Monitor package files and subdirectories for index changes · New
• Preserve private headers and relative include layout · New
• Search preprocessing output for generated main-is files · New
• Include preprocessing output in module search paths · New

What changed in this PR

Adds MicroHs support to Cabal’s compiler backend and cabal-install’s planning and installation workflows.

Changes:

• Implements MHS configuration, package discovery, building, registration, and installation.
• Adapts dependency solving and library installation for MicroHs.
• Adds integration fixtures, documentation, and generated help updates.

File	Description
doc/​setup-help/​configure.txt	Documents MHS compiler selection.
doc/​setup-help/​build.txt	Lists the MHS program.
doc/​setup-commands.rst	Documents MHS options.
doc/​config.rst	Lists MicroHs among known programs.
doc/​cmd-v2-help/​update.txt	Adds MHS selection help.
doc/​cmd-v2-help/​test.txt	Adds MHS selection help.
doc/​cmd-v2-help/​target.txt	Adds MHS selection help.
doc/​cmd-v2-help/​run.txt	Adds MHS selection help.
doc/​cmd-v2-help/​repl.txt	Adds MHS selection help.
doc/​cmd-v2-help/​path.txt	Adds MHS selection help.
doc/​cmd-v2-help/​outdated.txt	Adds MHS selection help.
doc/​cmd-v2-help/​list-bin.txt	Adds MHS selection help.
doc/​cmd-v2-help/​install.txt	Adds MHS selection help.
doc/​cmd-v2-help/​haddock.txt	Adds MHS selection help.
doc/​cmd-v2-help/​gen-bounds.txt	Adds MHS selection help.
doc/​cmd-v2-help/​freeze.txt	Adds MHS selection help.
doc/​cmd-v2-help/​exec.txt	Adds MHS selection help.
doc/​cmd-v2-help/​configure.txt	Adds MHS selection help.
doc/​cmd-v2-help/​build.txt	Adds MHS selection help.
doc/​cmd-v2-help/​bench.txt	Adds MHS selection help.
doc/​cmd-v1-help/​reconfigure.txt	Adds MHS selection help.
doc/​cmd-v1-help/​install.txt	Documents MHS selection and program.
doc/​cmd-v1-help/​configure.txt	Documents MHS selection and program.
doc/​cmd-v1-help/​build.txt	Lists the MHS program.
doc/​cabal-project-description-file.rst	Lists MHS compiler configuration.
doc/​cabal-package-description-file.rst	Updates documented compiler flavors.
doc/​cabal-commands.rst	Explains MHS library installation.
changelog.d/​pr-11392.md	Records support and limitations.
Cabal/​src/​Distribution/​Simple/​Setup/​Config.hs	Adds the --mhs option.
Cabal/​src/​Distribution/​Simple/​Register.hs	Dispatches MHS registration operations.
Cabal/​src/​Distribution/​Simple/​Program/​Builtin.hs	Defines the MHS program.
Cabal/​src/​Distribution/​Simple/​Program.hs	Exports the MHS program.
Cabal/​src/​Distribution/​Simple/​MHS.hs	Implements the MicroHs backend.
Cabal/​src/​Distribution/​Simple/​Install.hs	Dispatches MHS installation.
Cabal/​src/​Distribution/​Simple/​Configure.hs	Integrates MHS configuration and discovery.
Cabal/​src/​Distribution/​Simple/​Compiler.hs	Generalizes reexport capability detection.
Cabal/​src/​Distribution/​Simple/​Build.hs	Dispatches MHS builds.
Cabal/​Cabal.cabal	Exposes the backend module.
cabal-testsuite/​PackageTests/​MHS/​Macros/​src/​Versions.hs	Exercises version macros.
cabal-testsuite/​PackageTests/​MHS/​Macros/​mhs-macros.cabal	Defines the macro fixture.
cabal-testsuite/​PackageTests/​MHS/​Macros/​cabal.test.hs	Checks macro output.
cabal-testsuite/​PackageTests/​MHS/​Macros/​cabal.project	Selects MHS for macro tests.
cabal-testsuite/​PackageTests/​MHS/​Macros/​app/​Main.hs	Exercises header and library macros.
cabal-testsuite/​PackageTests/​MHS/​InstallLib/​src/​Triple.hs	Provides an FFI library fixture.
cabal-testsuite/​PackageTests/​MHS/​InstallLib/​prog/​Prog.hs	Consumes the installed library.
cabal-testsuite/​PackageTests/​MHS/​InstallLib/​mhs-installlib.cabal	Defines the installation fixture.
cabal-testsuite/​PackageTests/​MHS/​InstallLib/​include/​triple.h	Supplies the installed C header.
cabal-testsuite/​PackageTests/​MHS/​InstallLib/​cbits/​triple.c	Implements the C fixture.
cabal-testsuite/​PackageTests/​MHS/​InstallLib/​cabal.test.hs	Tests installation and standalone consumption.
cabal-testsuite/​PackageTests/​MHS/​InstallLib/​cabal.project	Selects MHS for installation tests.
cabal-testsuite/​PackageTests/​MHS/​CSources/​src/​Twice.hs	Exercises in-place library FFI.
cabal-testsuite/​PackageTests/​MHS/​CSources/​repo/​mhs-cdep-1.0/​src/​Add.hs	Exercises dependency C API imports.
cabal-testsuite/​PackageTests/​MHS/​CSources/​repo/​mhs-cdep-1.0/​mhs-cdep.cabal	Defines the repository dependency.
cabal-testsuite/​PackageTests/​MHS/​CSources/​repo/​mhs-cdep-1.0/​include/​add.h	Supplies dependency declarations and macro.
cabal-testsuite/​PackageTests/​MHS/​CSources/​repo/​mhs-cdep-1.0/​cbits/​add.c	Implements dependency C functionality.
cabal-testsuite/​PackageTests/​MHS/​CSources/​mhs-csources.cabal	Defines C compilation fixtures.
cabal-testsuite/​PackageTests/​MHS/​CSources/​include/​twice.h	Supplies the library header.
cabal-testsuite/​PackageTests/​MHS/​CSources/​cbits/​twice.c	Implements library C functionality.
cabal-testsuite/​PackageTests/​MHS/​CSources/​cabal.test.hs	Checks C linking and compiler options.
cabal-testsuite/​PackageTests/​MHS/​CSources/​cabal.project	Selects MHS for C tests.
cabal-testsuite/​PackageTests/​MHS/​CSources/​app/​offset.h	Declares executable C functionality.
cabal-testsuite/​PackageTests/​MHS/​CSources/​app/​offset.c	Exercises a C compiler definition.
cabal-testsuite/​PackageTests/​MHS/​CSources/​app/​Main.hs	Runs the C integration checks.
cabal-testsuite/​PackageTests/​MHS/​Build/​src/​Greeting/​Internal.hs	Exercises hidden modules and bytestring.
cabal-testsuite/​PackageTests/​MHS/​Build/​src/​Greeting.hs	Exposes the library greeting.
cabal-testsuite/​PackageTests/​MHS/​Build/​mhs-test.cabal	Defines the build fixture.
cabal-testsuite/​PackageTests/​MHS/​Build/​cabal.test.hs	Checks building and running.
cabal-testsuite/​PackageTests/​MHS/​Build/​cabal.project	Selects MHS for build tests.
cabal-testsuite/​PackageTests/​MHS/​Build/​app/​Main.hs	Consumes the fixture library.
cabal-install/​src/​Distribution/​Client/​ProjectPlanning.hs	Applies compiler-specific dependency rewriting.
cabal-install/​src/​Distribution/​Client/​Dependency.hs	Rewrites MHS source-package descriptions.
cabal-install/​src/​Distribution/​Client/​CmdPath.hs	Resolves the MHS compiler program.
cabal-install/​src/​Distribution/​Client/​CmdInstall.hs	Routes MHS library installation.

_{Review effort: Balanced}