Don't re-run the solver unnecessarily (#12119)

[tdammers]

Previously, the solver would be re-run whenever anything about the package description changed. This changes that so that only the relevant bits (roughly: the information found in PInfo) is taken into account, ignoring other changes to the package description.

• Patches conform to the coding conventions.
• Any changes that could be relevant to users have been recorded in the changelog.
  • Is the change significant? If so, remember to add significance: significant in the changelog file.
• The documentation has been updated, if necessary.
• Manual QA notes have been included.
• Tests have been added. (Ask for help if you don’t know how to write them! Ask for an exemption if tests are too complex for too little coverage!)

[sheaf]

I think the problem is that Distribution.Solver.Modular.modularResolver.postprocess re-injects a bunch of outside information into what the solver has computed beyond the input we use as the recompilation cache key. Specifically, Distribution.Solver.Modular.ConfiguredConversion.convCP re-injects things like SourcePackage which leaks things like local directory names etc.

I think we need to be slightly more ambitious in how we update Distribution.Client.ProjectPlanning.rebuildInstallPlan.phaseRunSolver (man aren't these a mouthful).

Here is a possible roadmap (still very sketchy):

• remove precisely depResolverInstalledPkgIndex and depResolverSourcePkgIndex from DepResolverParams, and add on the local packages but as [SourcePackage ()] so that it contains no filesystem-local information
• update resolveDependencies so that it takes the two fields you removed from DepResolverParams as extra arguments
• update phaseRunSolver so that it uses rerunIfChanged on (platform, compilerInfo, depResolverParams) and move it to a top-level function so that it can't possibly capture other local data that it isn't supposed to.
• The convCP logic that re-injects data moves to the outside of the solver's rerunIfChanged.

So the code in rebuildInstallPlan around the call to phaseRunSolver would become something like:

(plan0, ...) <- phaseRunSolver solverEnv depResolverParams -- has the rerunIfChanged described above
let solverPlan = addSourceLocations ... plan0 
(elaboratedPlan, elaboratedShared) <-
  phaseElaboratePlan projectConfig compilerEtc pkgConfigDB solverPlan localPackages

where addSourceLocations is a new function that does the "add back source locations" parts of convCP.

[sheaf]

I think the easiest way forward would be to introduce a new datatype for the solver inputs without changing anything else, as a first step. Then we can think about which datatypes can be removed later.

------------------------------
-- cabal-install-solver 

-- Copy of PackageDescription based on what `convGPD` reads off it,
-- replacing 'BuildInfo' with a separate new 'SolverComponent'.
data SolverPackageDescription = SolverPackageDescription
  { spdPackageId      :: PackageId
  , spdSpecVersion    :: CabalSpecVersion
  , spdScannedVersion :: Maybe Version
  , spdBuildType      :: BuildType
  , spdSetupBuildInfo :: Maybe SetupBuildInfo
  , spdFlags          :: [PackageFlag]
  , spdLibrary        :: Maybe (CondTree ConfVar [Dependency] SolverComponent)
  , spdComponents :: [(ComponentName, CondTree ConfVar [Dependency] SolverComponent)]
  }

data SolverComponent = SolverComponent
  { scBuildable :: Bool, scVisibility :: LibraryVisibility
  , scExtensions :: [Extension], scLanguages :: [Language]
  , scPkgconfigDeps :: [PkgconfigDependency]
  , scToolDeps :: [ExeDependency] }

projectOutSolverPackageDescription :: SourcePackage loc -> SolverPackageDescription

---------------------------------
-- cabal-install

-- The recompilation cache key (input to `solveProject`)
data SolverInputs = SolverInputs
  { siCompiler        :: Compiler
  , siPlatform        :: Platform
  , siPackageDbs      :: PackageDBStackCWD
  , siRepoConfig      :: SolverRepoConfig 
  , siSettings        :: SolverSettings
  , siProjectPackages :: [PackageSpecifier SolverPackageDescription]
  , siEnabledStanzas  :: Map PackageName (Map OptionalStanza Bool)
  , siExtraInstalled  :: InstalledPackageIndex
  }

and then update convSP, convGPD, condCondTree, condBranch to use these types instead.

For the return type of the solver, delete ResolverPackage, SolverPackage, InstSolverPackage and DependencyResolver, replacing them by introducing

---------------------------
-- cabal-install-solver

data SolverPlanPackage ipkg srcpkg
  = PreExisting PackageId UnitId ipkg SolverDeps
  | Configured  PackageId srcpkg FlagAssignment OptionalStanzaSet SolverDeps

Then generalise SolverInstallPlan to GenericSolverPlan ipkg srcpkg, defining type SolverInstallPlan = GenericSolverPlan InstalledPackageInfo UnresolvedSourcePackage.

----------------------
-- cabal-install
data SolverOutput = SolverOutput
  { soPlan         :: GenericSolverPlan () ()
  , soInstalled    :: InstalledPackageIndex       -- only units the plan references
  , soRepoPackages :: [UnresolvedSourcePackage]   -- only non-project packages the plan references
  , soPkgConfigDb  :: Maybe PkgConfigDb
  , soIndexState   :: TotalIndexState, soActiveRepos :: ActiveRepos }

Then:

• The solver returns GenericSolverPlan () (), and that's what gets cached.
• convCP becomes a pure function which adds back on the InstalledPackageInfo and UnresolvedSourcePackage from the outside.

I retract any prior suggestions to use DepResolverParams directly as the cache key because making that actually work brings in unneeded changes (e.g. figuring out what to do with goalOrder, basicInstallPolicy etc).