Use compilation jobs for caching and shared reflection

.buildkite/pipeline.yml

@@ -53,7 +53,6 @@ steps:
               - "13.4"
               - "13.3"
               - "13.2"
-              - "13.1"
         commands: |
           echo -e "[CUDA_Compiler_jll]\nversion = \"{{matrix.cuda}}\"" >LocalPreferences.toml
 

Project.toml

@@ -15,8 +15,10 @@ CUDA_Tile_jll = "2068806d-a867-5dbd-af0e-42c2eb5d895d"
 CompilerCaching = "9db33cc3-5358-4881-8759-fa4194144afd"
 EnumX = "4e289a0a-7415-4d19-859d-a7e5c4648b56"
 GPUArrays = "0c68f7d7-f131-5f86-a1c3-88cf8149b2d7"
+GPUCompiler = "61eb1bfa-7361-4325-ad38-22787b887f55"
 GPUToolbox = "096a3bc2-3ced-46d0-87f4-dd12716f4bfc"
 IRStructurizer = "93e32bba-5bb8-402b-805d-ffb066edee93"
+ObjectFile = "d8793406-e978-5875-9003-1fc021f44a92"
 PrecompileTools = "aea7be01-6a6a-4083-8856-8a6e6704d82a"
 Preferences = "21216c6a-2e73-6563-6e65-726566657250"
 REPL = "3fa0cd96-eef1-5676-8a61-b3b8758bbffb"
@@ -40,9 +42,11 @@ CompilerCaching = "0.5"
 DLFP8Types = "0.1.0"
 EnumX = "1.0"
 GPUArrays = "11"
+GPUCompiler = "2.6"
 GPUToolbox = "3"
 IRStructurizer = "0.6.3"
 Microfloats = "0.2"
+ObjectFile = "0.5"
 PrecompileTools = "1"
 Preferences = "1.5"
 REPL = "1"

docs/src/lib/reflection.md

@@ -11,12 +11,28 @@ code_tiled
 code_typed
 code_ircode
 code_structured
+code_ptx
+code_sass
+```
+
+## Jobs
+
+```@docs
+TileJob
+tile_job
 ```
 
 ## Intercepting a launch
 
 ```@docs
 @device_code_tiled
-@device_code_typed
 @device_code_structured
+@device_code_ptx
 ```
+
+`@device_code_typed` and `@device_code_warntype` are shared with GPUCompiler;
+they work for cuTile kernels through the shared compile hook.
+
+`CUDA.@device_code_sass` works for cuTile kernels: it intercepts module loads
+at the driver level (via CUPTI), so it captures any backend's kernels without
+backend-specific support.

docs/src/man/debugging.md

@@ -36,6 +36,23 @@ Spelling out those types is only worth it when you have no GPU, since
 `code_tiled` does not need CUDA.jl. Otherwise let the launch site derive them
 for you with `ct.@device_code_tiled`, described next.
 
+By default `code_tiled` compiles for the active CUDA device with the
+toolchain's bytecode version, so its output matches what a launch emits,
+including architecture-dependent `ct.@compiler_options` hints. Pass `sm_arch`
+to target another architecture, or to inspect code without a CUDA device, and
+`bytecode_version` to emit an older version. Combinations the bytecode does
+not support are rejected, so an older `bytecode_version` may need an explicit
+`sm_arch` on older devices.
+
+To inspect several stages with the same compilation options, create a job:
+
+```julia
+job = ct.tile_job(vadd, argtypes; sm_arch=v"10.0", num_ctas=2)
+ct.code_typed(job)
+ct.code_structured(job)
+ct.code_tiled(job)
+```
+
 With a CUDA 13.4 or newer `tileiras`, pass `remarks=true` to compile the Tile IR
 and print optimization diagnostics after it:
 
@@ -44,9 +61,35 @@ ct.code_tiled(vadd, argtypes; remarks=true, sm_arch=v"10.0")
 ```
 
 The remarks report successful and failed optimizations, including tensor-core
-selection and memory alignment issues. `sm_arch` may be omitted when a CUDA
-device is available. Remarks are generated afresh for reflection and are not
-read from or written to the compilation cache.
+selection and memory alignment issues. Remarks are generated afresh for
+reflection and are not read from or written to the compilation cache.
+
+
+## Inspecting PTX and SASS
+
+The output of `tileiras` can be inspected with `ct.code_ptx` and
+`ct.code_sass`, taking the same signature as `ct.code_tiled`:
+
+```julia
+ct.code_ptx(vadd, argtypes; sm_arch=v"10.0")
+ct.code_sass(vadd, argtypes; sm_arch=v"10.0")
+```
+
+`code_ptx` shows the thread-level SIMT program the tile-level kernel is
+lowered to, with every compiler decision (thread mapping, CTA size,
+pipelining, synchronization) already made; `code_sass` shows the final machine
+code, disassembled with `nvdisasm`. Both compile the Tile IR with `tileiras`
+but do not need a GPU.
+
+!!! warning "PTX reflection is unstable"
+    PTX is an implementation detail of `tileiras`, not an interface, and may
+    stop being produced or recorded at any time. `code_ptx` reads it from an
+    undocumented debug section of the CUBIN and will be removed with it.
+
+Note that `tileiras` always generates architecture-specific code: targeting
+`sm_arch=v"10.0"` produces `sm_100a` PTX and SASS, which only runs on that
+exact architecture. Launches therefore always compile for the active device,
+and reject an explicit `sm_arch` that differs from it.
 
 
 ## Intercepting a launch
@@ -73,17 +116,37 @@ intercepted kernel:
 ct.@device_code_tiled remarks=true @cuda backend=cuTile blocks=grid kernel(args...)
 ```
 
-Three are available, corresponding to successive stages of the pipeline:
+The following macros are available:
 
 | Macro | Output |
 |-------|--------|
-| `ct.@device_code_tiled` | Final Tile IR (MLIR textual format) |
-| `ct.@device_code_typed` | Typed Julia IR after overlay resolution |
+| `ct.@device_code_warntype` | Typed Julia IR with type-instability highlighting |
+| `ct.@device_code_typed` | Typed Julia IR after overlay resolution, returned per job |
 | `ct.@device_code_structured` | Structured IR (after control-flow structurization) |
-
-Read top to bottom, they run backwards through the pipeline: Julia IR with Tile
-IR intrinsics substituted by the overlay method table, then structured control
-flow, then emitted Tile IR.
+| `ct.@device_code_tiled` | Final Tile IR (MLIR textual format) |
+| `ct.@device_code_ptx` | PTX generated by `tileiras` (unstable, as above) |
+
+The first two are shared with GPUCompiler: cuTile reports its
+compilations through the same hook as the LLVM backends, so
+`CUDA.@device_code_typed` and `CUDA.@device_code_warntype` cover cuTile
+launches as well, including expressions that launch kernels of both kinds.
+`@device_code_typed` returns a dictionary
+from job to typed code rather than printing.
+Both Julia-level views use cuTile's optimized inferred code, including constant
+argument specializations. `CUDA.@device_code_ptx` also includes cuTile launches.
+The cuTile-specific stage macros only inspect cuTile jobs and ignore other backends;
+they report an error when the expression contains no cuTile compilations.
+
+The macros use the launch's actual target, constant arguments, and compilation
+hints, including when the compiled kernel is already cached. Each distinct
+job is inspected once per macro invocation. Hooks are scoped to the current task
+and inherited by child tasks; nested macros restore the enclosing hook.
+
+For the machine code, `CUDA.@device_code_sass` works for cuTile kernels: it
+intercepts module loads at the driver level (via CUPTI), showing the binary
+that was actually loaded — complete with Julia source locations, since
+`tileiras` compiles with line info. Under Nsight or another active profiler,
+which conflicts with CUPTI, use `ct.code_sass` on the signature instead.
 
 
 ## Dumping bytecode

src/compiler/codegen/kernel.jl

@@ -387,15 +387,7 @@ function emit_subprogram!(ctx::CGCtx, func, arg_types::Vector,
     if !haskey(ctx.cache, mi)
         error("Expected $func($(join(arg_types, ", "))) to be cached already by inference.")
     end
-    # Suppress compile_hook to avoid @device_code_tiled treating
-    # region bodies (e.g. reduce combiners) as standalone entries.
-    old_hook = compile_hook[]
-    compile_hook[] = nothing
-    sci, _, _ = try
-        emit_structured!(ctx.cache, mi)
-    finally
-        compile_hook[] = old_hook
-    end
+    sci, _, _ = emit_structured(emit_julia(ctx.cache, mi)...)
 
     # 2b. Run the pass pipeline on subprogram IR
     sub_divby, sub_bounds = run_passes!(sci)