Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
246 changes: 139 additions & 107 deletions api/v1alpha1/vm_types.go

Large diffs are not rendered by default.

19 changes: 9 additions & 10 deletions api/v1alpha1/zz_generated.deepcopy.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

106 changes: 42 additions & 64 deletions dist/files/crds/cobaltcore.cloud_virtualmachines.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -26,9 +26,6 @@ spec:
- jsonPath: .spec.resources.memory
name: Memory
type: string
- jsonPath: .spec.targetHost.name
name: TargetHost
type: string
- jsonPath: .status.host.name
name: ActualHost
type: string
Expand Down Expand Up @@ -89,7 +86,7 @@ spec:
Region, AZ, and Resources are populated at creation time either by the Cortex
placement API (at scheduling time) or by the vm-operator's Nova sync
controller (when it adopts a pre-existing Nova server that has no CRD yet).
Both write the same create-time values; the fields are immutable thereafter.
Region and AZ are immutable thereafter; Resources may change on resize.
properties:
az:
description: |-
Expand Down Expand Up @@ -158,15 +155,16 @@ spec:
- uuid
type: object
type: array
explanation:
description: |-
Explanation is a human-readable explanation of the scheduling decision
made by Cortex.
type: string
hostCandidates:
description: HostCandidates is the set of host references Cortex
has considered for the most recent scheduling operation of the
VM.
scheduledGeneration:
description: ScheduledGeneration is the last generation observed
by cortex for this VM.
format: int64
minimum: 0
type: integer
schedulingCandidates:
description: SchedulingCandidates is the set of host references
Cortex has considered for the most recent scheduling operation
of the VM.
items:
description: ObjectReference contains enough information to
let you inspect or modify the referred object.
Expand Down Expand Up @@ -212,14 +210,30 @@ spec:
type: object
x-kubernetes-map-type: atomic
type: array
intent:
schedulingError:
description: |-
SchedulingError is a human-readable explanation of the scheduling error,
if any occurred. Typically, this is non-empty when the set of scheduling
candidates is empty.
type: string
schedulingExplanation:
description: |-
SchedulingExplanation is a human-readable explanation of the scheduling decision
made by Cortex.
type: string
schedulingIntent:
description: |-
Intent is the last operation performed on this virtual machine,
SchedulingIntent is the last operation performed on this virtual machine,
e.g. initial placement, evacuation, live-migration, rebuild, etc.
This is an information provided by Nova to Cortex, but it is
persisted nowhere inside the Nova database, so Cortex propagates
this information into the CortexSpec.
type: string
schedulingTime:
description: SchedulingTime is the timestamp of the last transition
of the VM's intent.
format: date-time
type: string
type: object
hypervisorType:
description: HypervisorType is the hypervisor type of the host where
Expand Down Expand Up @@ -287,9 +301,13 @@ spec:
name, used for logging and debugging only.
type: string
ownership:
description: Ownership identifies the OpenStack project and user
owning this VM.
description: Ownership identifies the OpenStack project, user,
and domain owning this VM.
properties:
domain:
description: Domain is the OpenStack domain UUID that owns
the project, when known.
type: string
project:
description: Project is the OpenStack project (tenant) UUID.
type: string
Expand Down Expand Up @@ -330,50 +348,6 @@ spec:
x-kubernetes-validations:
- message: both cpu and memory must be specified
rule: '"cpu" in self && "memory" in self'
targetHost:
description: TargetHost is a reference to the host where this VM should
be placed.
properties:
apiVersion:
description: API version of the referent.
type: string
fieldPath:
description: |-
If referring to a piece of an object instead of an entire object, this string
should contain a valid JSON/Go field access statement, such as desiredState.manifest.containers[2].
For example, if the object reference is to a container within a pod, this would take on a value like:
"spec.containers{name}" (where "name" refers to the name of the container that triggered
the event) or if no container name is specified "spec.containers[2]" (container with
index 2 in this pod). This syntax is chosen only to have some well-defined way of
referencing a part of an object.
type: string
kind:
description: |-
Kind of the referent.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
type: string
name:
description: |-
Name of the referent.
More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
type: string
namespace:
description: |-
Namespace of the referent.
More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/
type: string
resourceVersion:
description: |-
Specific resourceVersion to which this reference is made, if any.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency
type: string
uid:
description: |-
UID of the referent.
More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#uids
type: string
type: object
x-kubernetes-map-type: atomic
required:
- az
- nova
Expand All @@ -384,8 +358,13 @@ spec:
description: VMStatus defines the observed state of a VirtualMachine.
properties:
conditions:
description: Conditions holds standard Kubernetes conditions for this
VM.
description: |-
Conditions are exclusively reconciled and owned by the vm operator.
They reflect the current state of the VM as observed by the operator.

If no conditions are set, the operator has not yet recorded observations.
This includes resources just scheduled for the first time by Cortex.
See the CortexSpec for scheduling details.
items:
description: Condition contains details for one aspect of the current
state of this API Resource.
Expand Down Expand Up @@ -533,7 +512,6 @@ spec:
selectableFields:
- jsonPath: .spec.region
- jsonPath: .spec.az
- jsonPath: .spec.targetHost.name
- jsonPath: .status.host.name
served: true
storage: true
Expand Down
6 changes: 3 additions & 3 deletions internal/nova/controller.go
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,7 @@ func (c *Controller) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Resu
meta.SetStatusCondition(&vm.Status.Conditions, metav1.Condition{
Type: v1alpha1.ConditionTypeNovaSynced,
Status: metav1.ConditionFalse,
Reason: v1alpha1.ConditionReasonNovaSyncedError,
Reason: v1alpha1.ConditionReasonNovaSyncedFailed,
Message: err.Error(),
})
if perr := c.Status().Patch(ctx, &vm, client.MergeFrom(base)); perr != nil {
Expand Down Expand Up @@ -227,7 +227,7 @@ func (c *Controller) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Resu
meta.SetStatusCondition(&created.Status.Conditions, metav1.Condition{
Type: v1alpha1.ConditionTypeNovaSynced,
Status: metav1.ConditionTrue,
Reason: v1alpha1.ConditionReasonNovaSyncedUpToDate,
Reason: v1alpha1.ConditionReasonNovaSyncedOK,
Message: "resource created from nova state",
})
if err := c.Status().Update(ctx, created); err != nil {
Expand All @@ -246,7 +246,7 @@ func (c *Controller) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Resu
// NovaSynced condition, so operators can still see what the VM was). LastSyncTime,
// conditions and Status.Host are preserved either way.
base := vm.DeepCopy()
reason, message := v1alpha1.ConditionReasonNovaSyncedUpToDate, "resource reflects the current nova state"
reason, message := v1alpha1.ConditionReasonNovaSyncedOK, "resource reflects the current nova state"
outcome := outcomeSynced
var requeueAfter time.Duration
var changed bool
Expand Down
17 changes: 7 additions & 10 deletions internal/nova/controller_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -279,7 +279,7 @@ func TestController_Reconcile_usesServerGet(t *testing.T) {
if got.Status.Nova == nil || got.Status.Nova.Status != "ACTIVE" || got.Status.Nova.Host != "compute-1" {
t.Errorf("status.nova = %+v, want ACTIVE on compute-1", got.Status.Nova)
}
assertSyncedCondition(t, &got, metav1.ConditionTrue, v1alpha1.ConditionReasonNovaSyncedUpToDate)
assertSyncedCondition(t, &got, metav1.ConditionTrue, v1alpha1.ConditionReasonNovaSyncedOK)
}

func TestController_Reconcile_syncsExistingVM(t *testing.T) {
Expand Down Expand Up @@ -317,7 +317,7 @@ func TestController_Reconcile_syncsExistingVM(t *testing.T) {
if got.Status.Nova.LastSyncTime == nil {
t.Error("expected LastSyncTime to be stamped on sync")
}
assertSyncedCondition(t, &got, metav1.ConditionTrue, v1alpha1.ConditionReasonNovaSyncedUpToDate)
assertSyncedCondition(t, &got, metav1.ConditionTrue, v1alpha1.ConditionReasonNovaSyncedOK)
}

// TestController_Reconcile_preservesCortexSpec verifies the full update path (the
Expand All @@ -331,7 +331,7 @@ func TestController_Reconcile_preservesCortexSpec(t *testing.T) {
ObjectMeta: metav1.ObjectMeta{Name: name},
Spec: v1alpha1.VMSpec{
Cortex: &v1alpha1.CortexSpec{
HostCandidates: []corev1.ObjectReference{{Kind: "Hypervisor", Name: "compute-9"}},
SchedulingCandidates: []corev1.ObjectReference{{Kind: "Hypervisor", Name: "compute-9"}},
},
},
}
Expand Down Expand Up @@ -360,8 +360,8 @@ func TestController_Reconcile_preservesCortexSpec(t *testing.T) {
if got.Spec.Cortex == nil {
t.Fatal("spec.cortex was cleared by the nova sync")
}
if len(got.Spec.Cortex.HostCandidates) != 1 || got.Spec.Cortex.HostCandidates[0].Name != "compute-9" {
t.Errorf("spec.cortex.hostCandidates = %+v, want the seeded compute-9", got.Spec.Cortex.HostCandidates)
if len(got.Spec.Cortex.SchedulingCandidates) != 1 || got.Spec.Cortex.SchedulingCandidates[0].Name != "compute-9" {
t.Errorf("spec.cortex.schedulingCandidates = %+v, want the seeded compute-9", got.Spec.Cortex.SchedulingCandidates)
}
}

Expand Down Expand Up @@ -429,9 +429,6 @@ func TestController_Reconcile_createsMissingVM(t *testing.T) {
if got.Spec.HypervisorType == nil || *got.Spec.HypervisorType != v1alpha1.HypervisorTypeQEMU {
t.Errorf("spec.hypervisorType = %v, want QEMU", got.Spec.HypervisorType)
}
if got.Spec.TargetHost == nil || got.Spec.TargetHost.Name != "compute-1" {
t.Errorf("spec.targetHost = %+v, want name compute-1", got.Spec.TargetHost)
}
cpu := got.Spec.Resources[v1alpha1.ResourceCPU]
if cpu.Value() != 4 {
t.Errorf("spec.resources[cpu] = %s, want 4", cpu.String())
Expand All @@ -447,7 +444,7 @@ func TestController_Reconcile_createsMissingVM(t *testing.T) {
if got.Spec.Nova.Flavor.ExtraSpecs[hypervisorTypeExtraSpec] != "qemu" {
t.Errorf("spec.nova.flavor.extraSpecs = %+v, want hypervisor type qemu", got.Spec.Nova.Flavor.ExtraSpecs)
}
assertSyncedCondition(t, &got, metav1.ConditionTrue, v1alpha1.ConditionReasonNovaSyncedUpToDate)
assertSyncedCondition(t, &got, metav1.ConditionTrue, v1alpha1.ConditionReasonNovaSyncedOK)
}

func TestController_Reconcile_skipsCreateWithoutAZ(t *testing.T) {
Expand Down Expand Up @@ -655,7 +652,7 @@ func TestController_Reconcile_recordsFetchFailure(t *testing.T) {
}

got := getVM(t, c)
assertSyncedCondition(t, &got, metav1.ConditionFalse, v1alpha1.ConditionReasonNovaSyncedError)
assertSyncedCondition(t, &got, metav1.ConditionFalse, v1alpha1.ConditionReasonNovaSyncedFailed)
}

func TestController_Reconcile_bothMissingIsNoop(t *testing.T) {
Expand Down
43 changes: 26 additions & 17 deletions internal/nova/conversion.go
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,6 @@ import (
"k8s.io/apimachinery/pkg/api/equality"
"k8s.io/apimachinery/pkg/api/resource"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/runtime/schema"

v1alpha1 "github.com/cobaltcore-dev/vm-operator/api/v1alpha1"
)
Expand Down Expand Up @@ -72,32 +71,36 @@ func ConvertToVirtualMachine(server *servers.Server, flavor *flavors.Flavor, reg
ObjectMeta: metav1.ObjectMeta{
Name: server.ID,
Labels: map[string]string{
v1alpha1.LabelRegion: region,
v1alpha1.LabelAZ: server.AvailabilityZone,
v1alpha1.LabelTargetHost: server.Host,
v1alpha1.LabelRegion: region,
v1alpha1.LabelAZ: server.AvailabilityZone,
},
},
Spec: v1alpha1.VMSpec{
AZ: server.AvailabilityZone,
Region: region,
HypervisorType: &hypervisorType,
TargetHost: &v1alpha1.HostRef{
// So we avoid the import of the Hypervisor operator:
APIVersion: schema.GroupVersion{Group: "kvm.cloud.sap", Version: "v1"}.String(),
Kind: "Hypervisor",
Name: server.Host,
},
Resources: map[v1alpha1.ResourceName]resource.Quantity{
v1alpha1.ResourceCPU: *resource.NewQuantity(int64(flavor.VCPUs), resource.DecimalSI),
v1alpha1.ResourceCPU: *resource.
NewQuantity(int64(flavor.VCPUs), resource.DecimalSI),
// RAM is reported in MB; ResourceMemory is bytes.
v1alpha1.ResourceMemory: *resource.NewQuantity(int64(flavor.RAM)*1024*1024, resource.BinarySI),
v1alpha1.ResourceMemory: *resource.
NewQuantity(int64(flavor.RAM)*1024*1024, resource.BinarySI),
},
Nova: v1alpha1.NovaSpec{
InstanceName: server.InstanceName,
CreatedAt: createdAt,
Ownership: v1alpha1.Ownership{Project: server.TenantID, User: server.UserID},
Flavor: v1alpha1.Flavor{Name: flavor.Name, FlavorID: flavor.ID, ExtraSpecs: flavor.ExtraSpecs},
Image: v1alpha1.Image{UUID: imageUUID},
Ownership: v1alpha1.Ownership{
Project: server.TenantID,
User: server.UserID,
// Note: the domain information is not available on nova servers,
// it needs to be populated through keystone.
},
Flavor: v1alpha1.Flavor{
Name: flavor.Name,
FlavorID: flavor.ID,
ExtraSpecs: flavor.ExtraSpecs,
},
Image: v1alpha1.Image{UUID: imageUUID},
},
},
Status: v1alpha1.VMStatus{
Expand All @@ -110,12 +113,15 @@ func ConvertToVirtualMachine(server *servers.Server, flavor *flavors.Flavor, reg

// HasChanged reports whether applying this conversion to vm would change any
// sync-owned field — i.e. vm is out of sync. It compares Spec.Nova (excluding
// Flavor, which never drives a sync on its own) and Status.Nova.Status, ignoring
// Flavor, which never drives a sync on its own, and Ownership.Domain, which Nova
// does not expose) and Status.Nova.Status, ignoring
// LastSyncTime, conditions and Status.Host.
func (c ConvertedVirtualMachine) HasChanged(vm *v1alpha1.VirtualMachine) bool {
want := c.Spec.Nova
got := vm.Spec.Nova
want.Flavor, got.Flavor = v1alpha1.Flavor{}, v1alpha1.Flavor{}
// Nova does not expose the project's domain; it must not drive a sync.
want.Ownership.Domain, got.Ownership.Domain = "", ""
// A nil Status.Nova compares as empty so a never-synced resource is clean.
var gotStatus, gotHost string
if vm.Status.Nova != nil {
Expand All @@ -126,7 +132,8 @@ func (c ConvertedVirtualMachine) HasChanged(vm *v1alpha1.VirtualMachine) bool {
}

// PatchInto applies the sync-owned fields (Spec.Nova and Status.Nova) onto vm,
// leaving the immutable spec, conditions, Status.Host and LastSyncTime untouched.
// leaving Ownership.Domain, the immutable spec, conditions, Status.Host and
// LastSyncTime untouched.
//
// It writes only Spec.Nova and Status.Nova, so the Cortex subspec (Spec.Cortex,
// owned by the Cortex placement API) is deliberately left as-is: the Nova sync
Expand All @@ -136,7 +143,9 @@ func (c ConvertedVirtualMachine) PatchInto(vm *v1alpha1.VirtualMachine) {
if vm.Status.Nova != nil {
lastSync = vm.Status.Nova.LastSyncTime // carried forward across syncs.
}
domain := vm.Spec.Nova.Ownership.Domain
vm.Spec.Nova = c.Spec.Nova
vm.Spec.Nova.Ownership.Domain = domain
vm.Status.Nova = &v1alpha1.NovaStatus{
Status: c.Status.Nova.Status,
Host: c.Status.Nova.Host,
Expand Down
Loading
Loading