Add NFS and SMB volume drivers via direct guest mounts

[tonycoco]

Add NFS and SMB volume drivers via direct guest mounts

Adds --driver smb and --driver nfs to container volume create. Both drivers are experimental: they mount network shares directly inside the Linux guest through the kernel's CIFS and NFS clients, so there is no host-side mount hop and no block image.

How it works

VolumesService stores the share and driver options as volume metadata. At container run, the client resolves the volume into a Filesystem.smb or Filesystem.nfs mount, which RuntimeService translates into a .any guest mount of type cifs or nfs.

A new ContainerResource.MountOptionValidator checks that option keys match ^[A-Za-z][A-Za-z0-9_-]{0,63}$ and values contain no commas, whitespace, or control characters, so a value can't inject extra mount options. It runs in the CLI, VolumesService, ContainersService.create (for filesystems attached directly through the client API), and RuntimeService before the guest mount is built. The share value is validated separately as a mount source — it is passed to mount(2) as its own argument and never joined into the options string — so share names containing spaces work.

SMB passwords

The SMB password is stored in the macOS keychain under com.apple.container.volume, keyed by volume name. It is never written to VolumeConfiguration, the container bundle, or shown by volume inspect / volume list. ContainersService reads it from the keychain at bootstrap and passes it to the runtime in the bootstrap message; the runtime adds it only to the in-memory guest mount. The keychain item is removed when the volume is deleted. A password requires username; without either, the share is mounted as a guest.

Secret keys (password, pass, credentials, cred) are rejected for filesystems attached directly through the client API and for NFS volumes, and the runtime refuses to start a container whose stored configuration carries one.

Design discussion: #1428 (comment)

Usage

SMB:

container volume create --driver smb \
  --opt share=//fileserver/share \
  --opt username=user \
  --opt password=secret \
  --opt vers=3.0 \
  myshare

NFS:

container volume create --driver nfs \
  --opt share=nas.local:/exports/data \
  --opt addr=nas.local \
  --opt vers=3 \
  --opt proto=tcp \
  myexport

container run -v myshare:/media alpine ls /media

Boolean mount flags such as nolock or mfsymlinks are passed as --opt nolock= with an empty value and serialized as bare flags.

Kernel requirement

The guest kernel needs CONFIG_CIFS (SMB) and CONFIG_NFS_FS (NFS). The default kernel, Kata 3.32.0, includes both (CIFS via kata-containers/kata-containers#12374), and the containerization example config has CIFS enabled as of apple/containerization#681. If you use a custom kernel, verify these options are set; without them the mount fails with ENODEV.

Known limitations / follow-ups

• Passwords may not contain commas or whitespace. cifs supports comma escaping (,,), which can be added later.
• containerization's Mount failure message includes the mount data string, so a wrong SMB password is echoed in the mount error. Follow-up in apple/containerization to redact password=.
• No container volume update; to change a password, delete and recreate the volume.

Type of Change

• Bug fix
• New feature
• Breaking change
• Documentation update

Motivation and Context

Closes #1911
Closes #1413

Testing

• Tested locally
• Added/updated tests
• Added/updated docs

[jglogan]

@tonycoco Could you fix the conflicts? Thanks for the contribution, I need to take a closer look but this seems fine.

One thing to note is that changing the kernel/config-arm64 file down in containerization won't actually do anything to the kernel in our release distribution, as it uses the Kata kernel defined here: https://github.com/apple/container/blob/main/Sources/ContainerPersistence/DefaultsStore.swift#L222.

Users would need to build and specify their own kernel that includes SMB and/or NFS client capability.

One possible alternative/workaround would be if the public homebrew recipe (which isn't managed by the Containerization project maintainers, incidentally) were capable of building and installing a default kernel from using a more expansive config that it embeds.

[tonycoco]

@jglogan Merge conflicts are now fixed. I have commits for the kernel and upstream kata as well in the loop but at least this unlocks the container side of things. I also have some changes to the container-compose project that will allow for this feature to get turned on for that too.

Reference:

apple/containerization#681
https://github.com/tonycoco/Container-Compose/tree/feature/smb-volume-driver (not yet a PR but this feature branch works well)

Tracking:
kata-containers/kata-containers#12374

[jglogan]

@tonycoco Thanks! I'll let @dcantah handle apple/containerization#681 as he's already working with you on it.

[tonycoco]

Looks good and also kata-containers/kata-containers#12374 has been merged!

[tonycoco]

@jglogan this change is ready to go

[tonycoco]

cleaned up the commit and rebased

[tonycoco]

@jglogan @dcantah ping for activity here

[tonycoco]

Ping for activity again. Let me know if anything else is needed for the merge.

[tonycoco]

Resolved merge conflicts in VolumesService.swift. Mostly due to the rename of the Volume model to VolumeConfiguration. I updated the switch driver logic to use the new VolumeConfiguration structure. The rebase introduced compilation failures in the newly added test suites NFSVolumeTests.swift and SMBVolumeTests.swift due to reference of the old Volume name so I updated both test files to use the new VolumeConfiguration model.

[tonycoco]

Any update on this?

[tonycoco]

Gentle ping for activity here. @jglogan @dcantah the containerization and kata changes are merged in. This should be ready to roll.

[jglogan]

@tonycoco Thanks for the pings, and for your patience. I'll put this into the plan for this sprint.

IIRC I said I wanted to test it locally (which I haven't done yet).

I also need to do a little thinking about the architecture for the volume model. I don't think it means any changes to what your PR does, but what we might want to do is mark these options as experimental so that folks don't get wedded to a particular CLI shape.

[jglogan]

@tonycoco Could you create an issue describing the feature and use case(s), and add something to the description above indicating that this PR closes the issue?

[tonycoco]

@jglogan #1911 issue created. Added a bunch of details.

[tonycoco]

@jglogan made a few changes and rebased to latest on this. Ready for merge.

[jglogan]

@tonycoco See the review comments for this. I think most of this is workable, but the one thing that gives me concern is that this implementation can store a secret value (the SMB share password) as plain text in mount options on the volume and container (possible if someone creates a container using the client API and attaches an .smb Filesystem).

Before making code changes for this, could you propose a design here in the issue or PR comments for how we can retain the secret value securely (this would likely involve the keychain)? Requirements include:

• Keep the password out of both VolumeConfiguration.options and the persisted ContainerConfiguration
• Retrieve the secret server-side at bootstrap and not in Utility.swift on the client
• Reject password keys in Filesystem for the direct-API case
• Ensure the keychain item gets deleted with the volume

[jglogan]

Should we validate "local" here? Would there be any backward compatibility issues if we did?

We have the server side validation, but fail-fast would be cheap here.

[tonycoco]

Good call. Switched the if/else if/else to a switch with explicit "smb", "nfs", and "local" cases and a throwing default.

No backward-compatibility concern that I can find: driver is a required (non-optional) key in VolumeConfiguration's decoder, and before this PR the server only ever persisted "local" since _create threw driverNotSupported for anything else. So every stored volume already has driver == "local".

[jglogan]

Could you create a ContainerResource.MountOptionValidator as a package enum that ensures that keys and values are syntactically valid?

Perhaps a key validation regex like ^[A-Za-z][A-Za-z0-9_-]{0,63}$ and a value regex like ^[^,\s\p{Cc}]{0,1024}$? The principal thing we want to avoid is injecting commas into values, so the regex rejects them for now (if we need escaped commas in values, we can add that as a separate feature later).

For defense in depth, the validator should also be invoked for smb and nfs mounts in ContainerService.create and in RuntimeService.

An alternative we could look into is to represent the driver and mount options using a type that performs validation on init and deserialization so that we know the options are correct by construction. The one thing we'd need to do with this approach is ensure that we gracefully handle deserialization errors at startup - one invalid option on disk shouldn't break the whole app.

[tonycoco]

Added ContainerResource.MountOptionValidator as a package enum with the key/value regexes you suggested (^[A-Za-z][A-Za-z0-9_-]{0,63}$ and ^[^,\s\p{Cc}]{0,1024}$). Options are checked in key order so errors are deterministic, and the violation message names the key but never echoes the value, since it may be a credential.

One refinement: share is validated as a mount source rather than a mount option. It's handed to mount(2) as its own argument and never joined into the comma-separated data string (resolvedMountOptions already filters it out), so the comma/whitespace rule doesn't buy anything there and would reject common Windows share names like //server/My Share. validateShare instead requires non-empty and no control characters (which would truncate or corrupt the C string), with a 4096-character cap. Everything else gets the strict rule.

It's invoked at all four points: VolumesService._create for smb/nfs, ContainersService.create for every mount (covers the direct-API case), RuntimeService.configureContainer before building the guest mount (rejects a tampered bundle with .invalidState, mirroring AttachmentNameValidator.validateStored), and the CLI. The server and runtime call sites go through a new Filesystem.validateMountOptions(), which checks the .smb/.nfs associated share as a source and mountOptions as options, so the validated set is exactly what resolvedMountOptions emits. It's a no-op for non-network types.

I went with validation-at-the-boundary rather than the validate-on-init type for now, since it keeps the Filesystem Codable path unchanged and avoids the "one bad option on disk breaks startup" problem you mentioned. Tests in MountOptionValidatorTests (including that share leniency doesn't leak into username=), plus service-level coverage in VolumesServiceTests and a Filesystem test in FilesystemSMBTests.

[tonycoco]

@jglogan Here's a proposed design for keeping the SMB password out of persisted state. It follows the existing registry-credential pattern (container registry login → KeychainQuery) and keeps all keychain access inside container-apiserver.

Summary

Requirement	How it's met
Keep password out of VolumeConfiguration.options	VolumesService._create strips password from driverOpts before constructing/persisting VolumeConfiguration, and writes it to the keychain keyed by volume name.
Keep password out of persisted ContainerConfiguration	Filesystem.smb(mountOptions:) is built from volume.options, which no longer contains the password, so the container bundle JSON never sees it. Server-side validation rejects it if a caller injects it anyway.
Retrieve server-side at bootstrap, not in Utility.swift	ContainersService.bootstrap resolves credentials from the keychain and passes them to the runtime in the bootstrap XPC message (same mechanism as dynamicEnv / networkBootstrapInfos). RuntimeService.bootstrap merges them into the in-memory mount only.
Reject password keys in Filesystem for the direct-API case	ContainersService.create validates every .smb mount and throws invalidArgument if password / pass / credentials / cred is present in mountOptions.
Delete keychain item with the volume	VolumesService._delete removes the keychain item after store.delete(name).

Storage

• Use ContainerizationOS.KeychainQuery directly (not the registry-shaped KeychainHelper). ContainerAPIService already depends on ContainerizationOS.
• New security domain: Constants.volumeKeychainID = "com.apple.container.volume" alongside the existing com.apple.container.registry, so volume items never collide with registry items and container registry list doesn't show them.
• Item mapping (kSecClassInternetPassword, same as registry):
  • kSecAttrSecurityDomain → com.apple.container.volume
  • kSecAttrServer → volume name (unique, already validated against VolumeStorage.volumeNamePattern)
  • kSecAttrAccount → SMB username
  • kSecValueData → SMB password
• Keyed by volume name rather than share path so two volumes can point at the same share with different credentials, and so delete maps 1:1.
• Wrap it in a small VolumeCredentialStore protocol inside ContainerAPIService/Server/Volumes/ with a KeychainVolumeCredentialStore (prod) and an in-memory implementation for VolumesServiceTests.

Write path — container volume create

CLI --opt password=...  ──XPC──▶  VolumesService._create
                                    ├─ validate: password requires username
                                    ├─ MountOptionValidator (from your other comment)
                                    ├─ credentialStore.save(volume: name, username:, password:)
                                    ├─ driverOpts.removeValue(forKey: "password")
                                    └─ store.create(VolumeConfiguration(options: driverOpts))   // no secret

• The password transits XPC once (local mach IPC, never written to disk) and is then owned solely by the keychain.
• Writing server-side (rather than from the CLI, as registry login does) means create/read/delete all happen from the same signed binary, which avoids cross-process keychain ACL prompts, and means client-API callers of ClientVolume.create get identical behavior to the CLI.
• If the keychain write fails, the volume is not created (fail closed).
• username and domain stay in VolumeConfiguration.options — they're not secret and are useful for inspect/ls. The existing password=*** redaction in VolumeResource+ListDisplayable becomes defense-in-depth only.

Read path — container run / container start

ContainersService.bootstrap(id:)
  ├─ config = getContainerConfiguration(at: path)            // from bundle, no secret
  ├─ for mount in config.mounts where mount.isSMB, let name = mount.volumeName:
  │     credentialStore.lookup(volume: name) → [name: password]     // keyNotFound ⇒ no entry
  └─ runtimeClient.bootstrap(stdio:, networkBootstrapInfos:, dynamicEnv:, mountCredentials:)

RuntimeService.bootstrap(message)
  ├─ var config = bundle.configuration                         // in-memory copy
  ├─ for each .smb mount with a credential: mountOptions["password"] = secret
  └─ configureContainer(czConfig:config:) → mount.asMount → cifs options

• New RuntimeKeys.mountCredentials carrying [String: String] (volume name → password), JSON-encoded like dynamicEnv. The bundle's config.json on disk is never rewritten.
• Resolution happens in ContainersService so container-runtime-linux never touches the keychain; the runtime only sees the secret in memory for the lifetime of the mount setup.
• No keychain item for an SMB volume ⇒ mount proceeds without password (guest / anonymous share). An error from the keychain other than not-found fails bootstrap with a clear message.
• ContainersService.bootstrap currently logs dynamicEnv at debug; the new field must be excluded from that log line.

Reject path — direct API

In ContainersService.create, before writing the runtime configuration:

for mount in configuration.mounts {
    guard case .smb(_, _, let opts) = mount.type else { continue }
    try MountOptionValidator.validate(opts)                          // syntax (comma/whitespace)
    if let key = opts.keys.first(where: Self.smbSecretKeys.contains) {
        throw ContainerizationError(.invalidArgument,
            message: "smb mount option '\(key)' is not allowed; create a volume with `container volume create --driver smb` instead")
    }
}

smbSecretKeys = ["password", "pass", "credentials", "cred"] — these are the cifs.ko aliases that would carry or point at a secret. The same check runs in RuntimeService for defense in depth, as you suggested for the validator.

Delete path

try await self.store.delete(name)
try self.removeVolumeDirectory(for: name)
if volume.driver == "smb" {
    try? self.credentialStore.delete(volume: name)   // log a warning on failure, don't fail the delete
}

KeychainQuery.delete already tolerates errSecItemNotFound, so this is safe for SMB volumes created without a password. container volume prune goes through the same _delete, so it's covered.

Out of scope / follow-ups

• Rotation: no container volume update; delete and recreate. Can revisit if there's demand.
• --password-stdin on volume create to keep the secret out of shell history, mirroring registry login. Small and additive; happy to include if you'd like it in this PR.
• Migration: none needed — the feature hasn't shipped, so no on-disk volumes carry a plaintext password.

Open questions

1. Are you OK with the server (VolumesService) performing the keychain write, rather than the CLI? I prefer it for the single-process ACL reason above, but it does mean the password crosses XPC once at create time.
2. Any objection to com.apple.container.volume as the security domain name?
3. Should an SMB volume with a missing keychain item fail bootstrap instead of mounting anonymously? Anonymous mounting is friendlier for guest shares, but failing is stricter.

[tonycoco]

@jglogan Fixed up all the feedback sans the Keychain stuff. Let me know what you want to do there and I can write up another commit to get that in the same PR, or push it in a different one.

[tonycoco]

I implemented the Keychain stuff in 56e6fdd so you can see the shape, happy to rework any of it based on the questions below. Should be contained in that one commit.

[tonycoco]

Also, the containerization Mount.swift failure message includes data=…, so a wrong SMB password would be echoed in the mount error. a follow-up for apple/containerization, not this PR.