Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update Flavor APIs #25

Merged
merged 1 commit into from
Jun 26, 2024
Merged

Update Flavor APIs #25

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions charts/region/Chart.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@ description: A Helm chart for deploying Unikorn's Region Controller

type: application

version: v0.1.16
appVersion: v0.1.16
version: v0.1.17
appVersion: v0.1.17

icon: https://raw.githubusercontent.com/unikorn-cloud/assets/main/images/logos/dark-on-light/icon.png

Expand Down
137 changes: 110 additions & 27 deletions charts/region/crds/region.unikorn-cloud.org_regions.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,34 +64,108 @@ spec:
compute:
description: Compute is configuration for the compute service.
properties:
flavorExtraSpecsExclude:
flavors:
description: |-
FlavorExtraSpecsExclude discards any flavors with the listed
extra specs keys.
items:
type: string
type: array
gpuDescriptors:
description: |-
GPUDescriptors defines a set of keys that can be probed to
list GPU topology information.
items:
properties:
expression:
description: |-
Expression describes how to extract the number of GPUs from the property
if it exists. This must contain exactly one submatch that is a number
e.g. "^(\d+)$".
type: string
property:
description: Property is the property name to examine
e.g. "resources.VGPU".
type: string
required:
- expression
- property
type: object
type: array
Flavors defines how flavors are filtered and reported to
clients. If not defined, then all flavors are exported.
properties:
exclude:
description: Exclude inhibits the export of flavors from
the region service.
items:
properties:
id:
description: ID flavor ID is the immutable Openstack
identifier for the flavor.
type: string
required:
- id
type: object
type: array
include:
description: |-
Include allows or augments flavors that can be exported by the region
service as defined by the "selectionPolicy" property. This explcitly
allows a flavor to be used, and or allows metadata to be mapped to the
flavor e.g. CPU/GPU information that isn't supported by OpenStack.
items:
properties:
baremetal:
description: |-
Baremetal indicates that this is a baremetal flavor, as opposed to a
virtualized one in case this affects image selection or even how instances
are provisioned.
type: boolean
cpu:
description: CPU defines additional CPU metadata.
properties:
family:
description: |-
Family is a free-form string that can communicate the CPU family to clients
e.g. "Xeon Platinum 8160T (Skylake)", and allows users to make scheduling
decisions based on CPU architecture and performance etc.
type: string
type: object
gpu:
description: |-
GPU defines additional GPU metadata. When provided it will enable selection
of images based on GPU vendor and model.
properties:
count:
description: Count is the number of logical
GPUs in the flavor.
type: integer
memory:
anyOf:
- type: integer
- type: string
description: Memory is the amount of memory
each logical GPU has access to.
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
model:
description: |-
Model is a free-form model name that corresponds to the supported models
property included on images, and must be an exact match e.g. H100.
type: string
vendor:
description: |-
Vendor is the GPU vendor, used for coarse grained flavor and image
selection.
enum:
- NVIDIA
- AMD
type: string
required:
- count
- memory
- model
- vendor
type: object
id:
description: |-
ID is the immutable Openstack identifier for the flavor.
While most flavor metadata (CPUs/Memory) should be immutable, the name is
not, and may change due to sales and marketing people.
type: string
required:
- id
type: object
type: array
selectionPolicy:
description: |-
SelectionPolicy defines the default set of flavors to export. "All" exports
all flavors, the "include" property defines additional metadata to
merge with matching flavors and the "exclude" inhibits export. "None" is a
more secure policy that only exports those flavors defined in the "include"
property, the "exclude" property is ignored as it's redundant.
enum:
- All
- None
type: string
required:
- selectionPolicy
type: object
serverGroupPolicy:
description: |-
ServerGroupPolicy defines the anti-affinity policy to use for
Expand Down Expand Up @@ -132,6 +206,15 @@ spec:
format: byte
type: string
type: object
network:
description: Network is configuration for the network service.
properties:
physicalNetwork:
description: |-
PhysicalNetwork is the neutron provider specific network name used
to provision provider networks e.g. VLANs for bare metal clusters.
type: string
type: object
serviceAccountSecret:
description: |-
ServiceAccountSecretName points to the secret containing credentials
Expand Down
50 changes: 33 additions & 17 deletions charts/region/templates/region.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,34 +27,50 @@ spec:
{{- end }}
{{- end }}
{{- end }}
{{- with $compute := $openstack.compute }}
{{ printf "compute:" | nindent 4 }}
{{- with $policy := $compute.regionGroupPolicy }}
{{- with $compute := $openstack.compute -}}
{{- printf "compute:" | nindent 4 }}
{{- with $policy := $compute.regionGroupPolicy -}}
{{ printf "regionGroupPolicy: %s" $policy | nindent 6 }}
{{- end }}
{{- with $specs := $compute.flavorExtraSpecsExclude }}
{{ printf "flavorExtraSpecsExclude:" | nindent 6 }}
{{- range $spec := $specs }}
{{ printf "- %s" $spec | nindent 6 }}
{{- with $flavors := $compute.flavors -}}
{{- printf "flavors:" | nindent 6 }}
{{- printf "selectionPolicy: %s" $flavors.selectionPolicy | nindent 8 }}
{{- with $includes := $flavors.include }}
{{- printf "include:" | nindent 8 }}
{{- range $include := $includes }}
{{- printf "- id: %s" $include.id | nindent 8 }}
{{- with $cpu := $include.cpu -}}
{{- printf "cpu:" | nindent 10 }}
{{- with $family := $cpu.family -}}
{{ printf "family: %s" $family | nindent 12 }}
{{- end }}
{{- end }}
{{- with $gpu := $include.gpu -}}
{{- printf "gpu:" | nindent 10 }}
{{- printf "vendor: %s" $gpu.vendor | nindent 12 }}
{{- printf "model: %s" $gpu.model | nindent 12 }}
{{- printf "memory: %s" $gpu.memory | nindent 12 }}
{{- printf "count: %v" $gpu.count | nindent 12 }}
{{- end }}
{{- end }}
{{- end }}
{{- end }}
{{- with $descriptors := $compute.gpuDescriptors }}
{{ printf "gpuDescriptors:" | nindent 6 }}
{{- range $descriptor := $descriptors }}
{{ printf "- property: %s" $descriptor.property | nindent 6 }}
{{ printf " expression: %s" $descriptor.expression | nindent 6 }}
{{- with $excludes := $flavors.exclude -}}
{{- printf "exclude:" | nindent 8 }}
{{- range $exclude := $excludes }}
{{- printf "- id: %s" $exclude.id | nindent 8 }}
{{- end }}
{{- end }}
{{- end }}
{{- end }}
{{- with $image := $openstack.image}}
{{ printf "image:" | nindent 4 }}
{{- with $properties := $image.propertiesInclude }}
{{- with $image := $openstack.image -}}
{{- printf "image:" | nindent 4 }}
{{- with $properties := $image.propertiesInclude -}}
{{ printf "propertiesInclude:" | nindent 6 }}
{{- range $property := $properties }}
{{ printf "- %s" $property | nindent 6 }}
{{- end }}
{{- end }}
{{- with $signingKey := $image.signingKey }}
{{- with $signingKey := $image.signingKey -}}
{{ printf "signingKey: %s" $signingKey | nindent 6 }}
{{- end }}
{{- end }}
Expand Down
19 changes: 19 additions & 0 deletions charts/region/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,25 @@ imagePullSecret:
# compute:
# # Kubernetes control plane scheduling policy.
# serverGroupPolicy: soft-anti-affinity
# # Flavor selection and configuration.
# flavors:
# # The selection policy can be "All" or "None"
# selectionPolicy: All
# # Include specific flavors when the policy is "None". In all cases
# # allows additional metadata to be exposed by the API that Openstack
# # cannot act as a source of truth for.
# include:
# - id: 60ab8c22-ac61-467d-8680-03d0ecca23c9
# cpu:
# family: Intel Xeon Platinum 8160T (Skylake)
# gpu:
# vendor: NVIDIA
# model: H100
# memory: 192Gi
# count: 2
# # Eclude specific flavors when the policy is "All".
# exclude:
# - id: d04d82d9-faa6-4b3b-9247-28b8a158b7ed
# # Flavors containing any of the specified extra specs will be discarded.
# flavorExtraSpecsExclude:
# - resources:CUSTOM_BAREMETAL
Expand Down
96 changes: 83 additions & 13 deletions pkg/apis/unikorn/v1alpha1/types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ package v1alpha1
import (
unikornv1core "github.com/unikorn-cloud/core/pkg/apis/unikorn/v1alpha1"

"k8s.io/apimachinery/pkg/api/resource"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)

Expand Down Expand Up @@ -77,6 +78,8 @@ type RegionOpenstackSpec struct {
Compute *RegionOpenstackComputeSpec `json:"compute,omitempty"`
// Image is configuration for the image service.
Image *RegionOpenstackImageSpec `json:"image,omitempty"`
// Network is configuration for the network service.
Network *RegionOpenstackNetworkSpec `json:"network,omitempty"`
}

type NamespacedObject struct {
Expand All @@ -97,21 +100,82 @@ type RegionOpenstackComputeSpec struct {
// ServerGroupPolicy defines the anti-affinity policy to use for
// scheduling cluster nodes. Defaults to "soft-anti-affinity".
ServerGroupPolicy *string `json:"serverGroupPolicy,omitempty"`
// FlavorExtraSpecsExclude discards any flavors with the listed
// extra specs keys.
FlavorExtraSpecsExclude []string `json:"flavorExtraSpecsExclude,omitempty"`
// GPUDescriptors defines a set of keys that can be probed to
// list GPU topology information.
GPUDescriptors []OpenstackGPUDescriptor `json:"gpuDescriptors,omitempty"`
// Flavors defines how flavors are filtered and reported to
// clients. If not defined, then all flavors are exported.
Flavors *OpenstackFlavorsSpec `json:"flavors,omitempty"`
}

type OpenstackGPUDescriptor struct {
// Property is the property name to examine e.g. "resources.VGPU".
Property string `json:"property"`
// Expression describes how to extract the number of GPUs from the property
// if it exists. This must contain exactly one submatch that is a number
// e.g. "^(\d+)$".
Expression string `json:"expression"`
// +kubebuilder:validation:Enum=All;None
type OpenstackFlavorSelectionPolicy string

const (
OpenstackFlavorSelectionPolicySelectAll OpenstackFlavorSelectionPolicy = "All"
OpenstackFlavorSelectionPolicySelectNone OpenstackFlavorSelectionPolicy = "None"
)

type OpenstackFlavorsSpec struct {
// SelectionPolicy defines the default set of flavors to export. "All" exports
// all flavors, the "include" property defines additional metadata to
// merge with matching flavors and the "exclude" inhibits export. "None" is a
// more secure policy that only exports those flavors defined in the "include"
// property, the "exclude" property is ignored as it's redundant.
SelectionPolicy OpenstackFlavorSelectionPolicy `json:"selectionPolicy"`
// Include allows or augments flavors that can be exported by the region
// service as defined by the "selectionPolicy" property. This explcitly
// allows a flavor to be used, and or allows metadata to be mapped to the
// flavor e.g. CPU/GPU information that isn't supported by OpenStack.
Include []OpenstackFlavorInclude `json:"include,omitempty"`
// Exclude inhibits the export of flavors from the region service.
Exclude []OpenstackFlavorExclude `json:"exclude,omitempty"`
}

type OpenstackFlavorInclude struct {
// ID is the immutable Openstack identifier for the flavor.
// While most flavor metadata (CPUs/Memory) should be immutable, the name is
// not, and may change due to sales and marketing people.
ID string `json:"id"`
// Baremetal indicates that this is a baremetal flavor, as opposed to a
// virtualized one in case this affects image selection or even how instances
// are provisioned.
Baremetal bool `json:"baremetal,omitempty"`
// CPU defines additional CPU metadata.
CPU *CPUSpec `json:"cpu,omitempty"`
// GPU defines additional GPU metadata. When provided it will enable selection
// of images based on GPU vendor and model.
GPU *GPUSpec `json:"gpu,omitempty"`
}

type OpenstackFlavorExclude struct {
// ID flavor ID is the immutable Openstack identifier for the flavor.
ID string `json:"id"`
}

type CPUSpec struct {
// Family is a free-form string that can communicate the CPU family to clients
// e.g. "Xeon Platinum 8160T (Skylake)", and allows users to make scheduling
// decisions based on CPU architecture and performance etc.
Family *string `json:"family,omitempty"`
}

// +kubebuilder:validation:Enum=NVIDIA;AMD
type GPUVendor string

const (
NVIDIA GPUVendor = "NVIDIA"
AMD GPUVendor = "AMD"
)

type GPUSpec struct {
// Vendor is the GPU vendor, used for coarse grained flavor and image
// selection.
Vendor GPUVendor `json:"vendor"`
// Model is a free-form model name that corresponds to the supported models
// property included on images, and must be an exact match e.g. H100.
Model string `json:"model"`
// Memory is the amount of memory each logical GPU has access to.
Memory *resource.Quantity `json:"memory"`
// Count is the number of logical GPUs in the flavor.
Count int `json:"count"`
}

type RegionOpenstackImageSpec struct {
Expand All @@ -125,6 +189,12 @@ type RegionOpenstackImageSpec struct {
SigningKey []byte `json:"signingKey,omitempty"`
}

type RegionOpenstackNetworkSpec struct {
// PhysicalNetwork is the neutron provider specific network name used
// to provision provider networks e.g. VLANs for bare metal clusters.
PhysicalNetwork *string `json:"physicalNetwork,omitempty"`
}

// RegionStatus defines the status of the region.
type RegionStatus struct {
// Current service state of a region.
Expand Down
Loading
Loading