The Deckhouse Platform installs CRDs but does not remove them when a module is disabled. If you no longer need the created CRDs, delete them.

The module lifecycle stage: General Availability

FencingFailedNodeState

Short names: ffns

Scope: Cluster
Version: v1alpha1

Fencing incident of one node.

The name of the object is the name of the failed node. The object has exactly one owner reference to this node. When the Node object is deleted, the Kubernetes garbage collector deletes this object too. The controller does nothing for an object whose owner reference does not have the UID of the node with this name, for example after the node is created again with the same name. A healthy node has no FencingFailedNodeState: the object exists only while the node has an active incident.

A fencing agent creates the object. The agent of any node can create it, also the agent on a node of a group in the Disabled mode. The creator writes spec. Nobody can change spec after that. Each part of status has one owner. The fencing controller reads the object and decides if it must evacuate the node.

The object is deleted in these cases:

  • The failed node comes back with the same boot ID before the phase Proven. Then the fencing agents delete the object.
  • The Node is Ready and has a new boot ID. Then the fencing controller removes the node.kubernetes.io/out-of-service taint from the Node and deletes the object.
  • The Node object is deleted. Then the Kubernetes garbage collector deletes the object.

Do not create the object manually. Do not edit spec or status.

  • apiVersion
    string
  • kind
    string
  • metadata
    object
  • spec
    object

    Required value

    Immutable description of the incident. The creator of the object writes it at creation. You cannot change it after that.

    • spec.armed
      boolean

      Required value

      true if the watchdog of the failed node was armed at the time of the failure. A fencing agent takes the value from the memberlist metadata of the node.

    • spec.bootID
      string

      Required value

      Boot ID of the failed node at the time of the failure. A fencing agent takes it from the memberlist metadata of the node. For the NodeLeaseStale source, the controller takes it from status.nodeInfo.bootID of the Node.

    • spec.detectedAt
      string

      Required value

      Time when the creator detected the failure, with microsecond precision.

      The value is for reference only. The fencing controller counts its deadlines from its own observations.

      Pattern: ^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}\.[0-9]{6}(Z|[+-][0-9]{2}:[0-9]{2})$

    • spec.detectedBy
      string

      Required value

      Name of the node whose agent created the object. If the fencing controller created the object, the value is fencing-controller.

    • spec.profileRef
      object

      SLA profile of the node group of the failed node at the time when the object was created. The group is the value of the node.deckhouse.io/group label of the Node. The mode of the group selects the profile. If the group is in the Disabled mode, the value is the profile of the memberlist network of the cluster.

      The field is set only if armed is true. The fencing controller takes the timings of the failed node only from this profile.

      • spec.profileRef.name
        string

        Required value

        Name of the built-in SLA profile. The value can be only the name of a built-in profile.

        The name of the related FencingSLAProfile object is this value in lower case, because the name of a Kubernetes object cannot have capital letters. To find the profile, the controller changes this value to lower case.

        Allowed values: Critical, Normal, Slow

    • spec.source
      string

      Required value

      Source of the incident. It tells who created the object:

      • MemberlistDead: a fencing agent that saw the node as dead in the memberlist network of the cluster.
      • DeclareSuicide: a fencing agent that received the DeclareSuicide message of the node. This agent is the witness.
      • NodeLeaseStale: the fencing controller, because the Lease of the node is stale. The current version of the controller does not create objects with this source.

      Allowed values: MemberlistDead, DeclareSuicide, NodeLeaseStale

  • status
    object

    Observed state of the incident. Each part of the status has one owner:

    • attestation: the fencing agent of the witness node.
    • contact: the fencing agent that received a report of a contact with the failed node. The current version of the agent does not write this section.
    • fallback: the fencing agent of the failed node.
    • phase, correlation and conditions: the fencing controller. The current version of the controller does not write correlation.

    The status is a subresource. The creator of the object cannot set it in the create request.

    • status.attestation
      object

      Attestation of the failure. The witness is a node whose agent received the DeclareSuicide message of the failed node. The agent of the witness writes the name of its own node.

      • status.attestation.at
        string

        Required value

        Time of the attestation, with microsecond precision.

        Pattern: ^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}\.[0-9]{6}(Z|[+-][0-9]{2}:[0-9]{2})$

      • status.attestation.witness
        string

        Required value

        Name of the witness node.

    • status.conditions
      array of objects

      Conditions of the incident. A condition does not replace the phase, so the controller can report the same problem in any phase.

      The controller sets ConfigurationError with the status True and the reason ProfileUnavailable while it cannot read the timings that it needs from the SLA profile in spec.profileRef.name. The message of the condition tells the cause. In this state, the controller does not evacuate the node. The status False with the reason ProfileResolved tells that the controller found the profile. An incident with armed: false needs no profile and also has this status.

      The Queued condition with the status True tells why the evacuation waits in the Queued phase. The reasons are ConcurrencyLimit and MassFailure. The current version of the controller does not use this condition.

      The controller sets Progressing with the status False when the evacuation in the Evicting phase is not complete after 60 seconds. The controller continues to examine the node. The reason tells what stops the evacuation now, and the message gives the details:

      • VolumesAttached: the node still has VolumeAttachment objects, so the volumes of its pods are not free for other nodes. The message gives their names.
      • PodsRemaining: the controller could not delete pods of the node, or another error of the API stops the evacuation. The message gives the pods and the errors.
      • TooManyUnhealthyMachines: the controller does not delete the Node of a cloud CAPI machine, because the MachineHealthCheck does not replace the machine now: more than 50 % of the machines are unhealthy, or the cluster has no MachineHealthCheck that reports its machines. The message gives the numbers.

      When the evacuation is complete, the phase is Evicted, and the controller removes the Progressing condition.

      • status.conditions.lastTransitionTime
        string

        Required value

        Time the condition last changed its status.

      • status.conditions.message
        string

        Human-readable description of the last transition.

      • status.conditions.observedGeneration
        integer

        Generation of the resource the condition was set from.

        Allowed values: 0 <= X

      • status.conditions.reason
        string

        Required value

        Machine-readable reason for the last transition.

      • status.conditions.status
        string

        Required value

        Condition status.

        Allowed values: True, False, Unknown

      • status.conditions.type
        string

        Required value

        Condition type.

    • status.contact
      object

      Last contact with the failed node. A fencing agent that received a report of a contact with the node writes this section.

      • status.contact.at
        string

        Required value

        Time of the last contact with the failed node, with microsecond precision.

        Pattern: ^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}\.[0-9]{6}(Z|[+-][0-9]{2}:[0-9]{2})$

      • status.contact.reportedBy
        string

        Required value

        Name of the node that reported the contact.

    • status.correlation
      object

      Correlation of the incident with the incidents of other nodes. The current version of the controller does not write this section.

      • status.correlation.episode
        string

        Identifier of the episode of correlated failures that includes this incident.

    • status.fallback
      object

      Liveness evidence from the failed node. Only the fencing agent of the failed node writes this section, and only through the Kubernetes API. Thus the section exists only if the node reached the API.

      The fencing controller keeps the FallbackAlive phase while the last heartbeat is not older than 2 * heartbeat.interval + heartbeat.apiTimeout of the profile.

      • status.fallback.lastHeartbeatAt
        string

        Required value

        Time of the last heartbeat of the node, with microsecond precision.

        When fresh heartbeats stop, this is the signal that the node cannot reach the API.

        Pattern: ^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}\.[0-9]{6}(Z|[+-][0-9]{2}:[0-9]{2})$

    • status.phase
      string

      Phase of the incident:

      • Suspected: a failure of the node is suspected.
      • FallbackAlive: the node writes heartbeats through the Kubernetes API. For example, an ACL or a firewall blocks the network between the nodes, but the node can reach the API.
      • Proven: the failure is proven. The controller proves a failure in one of two ways:
        • attestation: a witness confirmed the stop of the node, and the time watchdog.timeout + watchdog.margin of the profile passed after the controller saw the attestation. The node did not send a heartbeat after the attestation;
        • silence of the node: the time controlPlaneLostAfter + 0.2s + watchdog.timeout + watchdog.margin + 0.9s of the profile passed after the last sign of life of the node. The last sign of life is the later of two moments: when the controller saw the object first, and when it saw the last heartbeat plus heartbeat.interval.

        While the Fencing condition of the Node is not True, the controller does not write Proven, and the object stays in the phase Suspected.

      • Unproven: the failure is a part of a correlated failure and has no attestation. The current version of the controller does not set this phase.
      • Queued: the evacuation waits. The Queued condition tells the reason. The current version of the controller does not use this phase.
      • Evicting: the node has the node.kubernetes.io/out-of-service taint, and the evacuation is in progress. If the evacuation is not complete after 60 seconds, the Progressing condition tells the reason.
      • Evicted: the evacuation is complete: the controller deleted the pods of the node, and the node has no VolumeAttachment objects.

      The controller writes Evicted before it deletes the Node or the Machine of a node that the cloud replaces. The deletion of the Node also deletes this object. If the write of Evicted fails, the controller does not delete the Node or the Machine. If the deletion fails, the controller sends it again in the Evicted phase.

      An object with armed: false never gets the phase Proven. It stays in the phase Suspected or FallbackAlive, and the node never gets the taint.

      An empty phase means that the controller did not process the object yet. Read it as Suspected.

      An error is not a phase. The controller reports an error in conditions and keeps the phase.

      Allowed values: Suspected, FallbackAlive, Proven, Unproven, Queued, Evicting, Evicted

FencingSLAProfile

Short names: fsp

Scope: Cluster
Version: v1alpha1

SLA profile of fencing. A profile holds only timings. Correlation, the brake and the eviction limit are settings of the module, not of a profile.

The module ships three profiles: critical, normal and slow. Community Edition (CE) has no critical profile, because the Critical mode is available only in the commercial editions. Only Deckhouse can change the profiles. NodeGroup spec.fencing.mode selects the profile of a group. The mode name in lower case is the name of the object, because the name of a Kubernetes object cannot have capital letters.

All agents of the cluster run one memberlist network. They use the memberlist and rejoin sections of the profile of the fastest group that is not Disabled. An agent of a group that is not Disabled takes the other sections from the profile of its own group.

A profile is a cluster-scoped object. It has no state, so the resource has no status.

  • apiVersion
    string
  • kind
    string
  • metadata
    object
  • spec
    object

    Required value

    Timings of the profile. Every value is a duration or a count. The CEL rules of this object check the inequalities of one profile; the agent checks the others.

    The profile has no probe timings of kubernetes-api-proxy. The proxy uses the same fixed values on all nodes:

    • probe interval: 1.2s at most (1s with a jitter of 20 %);
    • probe timeout: 0.5s;
    • no retry of a failed probe.

    The largest age of a probe is 1.7s (1.2s + 0.5s).

    The profile has no timeouts of the poll of the control plane state and no read period of kubernetes-api-proxy. These timings are the same for all profiles: they are internal values of the module.

    • spec.controlPlaneLostAfter
      string

      Required value

      Maximum age of the last successful control plane probe. The node sees the control plane while the last Ready answer of any apiserver upstream is not older than this value.

      Pattern: ^([0-9]+(\.[0-9]+)?(ms|s|m|h))+$

      Maximum length: 32

    • spec.eviction
      object

      Required value

      Parameters of the creation of FencingFailedNodeState objects.

      The profile has no delay before the taint of a silent node, a node without signs of life. The fencing controller calculates it from the other values of the profile: controlPlaneLostAfter + 0.2s + watchdog.timeout + watchdog.margin + 0.9s. The controller counts the delay from the last sign of life of the node.

      • spec.eviction.writerWaveInterval
        string

        Required value

        Interval between the waves of the agents that create FencingFailedNodeState objects.

        Pattern: ^([0-9]+(\.[0-9]+)?(ms|s|m|h))+$

        Maximum length: 32

    • spec.heartbeat
      object

      Required value

      Parameters of the heartbeat that a failed node writes through the Kubernetes API.

      • spec.heartbeat.apiTimeout
        string

        Required value

        Timeout of one request of the agent to the Kubernetes API. The memberlist TCP timeout uses it too.

        Pattern: ^([0-9]+(\.[0-9]+)?(ms|s|m|h))+$

        Maximum length: 32

      • spec.heartbeat.interval
        string

        Required value

        Interval between the heartbeat writes of a failed node into its own FencingFailedNodeState.

        Pattern: ^([0-9]+(\.[0-9]+)?(ms|s|m|h))+$

        Maximum length: 32

    • spec.memberlist
      object

      Required value

      Parameters of memberlist and Lifeguard. All agents of the cluster use this section of the fastest profile among the node groups that are not Disabled.

      • spec.memberlist.awarenessMaxMultiplier
        integer

        Required value

        Maximum slowdown of the probe interval of a locally degraded agent (Lifeguard awareness).

        Allowed values: 1 <= X

      • spec.memberlist.deadNodeReclaimTime
        string

        Required value

        Time after which a new node can take the name or the address of a dead node.

        Pattern: ^([0-9]+(\.[0-9]+)?(ms|s|m|h))+$

        Maximum length: 32

      • spec.memberlist.gossipInterval
        string

        Required value

        Interval between gossip messages.

        Pattern: ^([0-9]+(\.[0-9]+)?(ms|s|m|h))+$

        Maximum length: 32

      • spec.memberlist.gossipNodes
        integer

        Required value

        Number of peers that receive gossip in each gossip interval.

        Allowed values: 1 <= X

      • spec.memberlist.gossipToTheDeadTime
        string

        Required value

        Time during which a dead node still receives gossip. The node can refute the dead state after it recovers.

        Pattern: ^([0-9]+(\.[0-9]+)?(ms|s|m|h))+$

        Maximum length: 32

      • spec.memberlist.indirectChecks
        integer

        Required value

        Number of indirect checks when the direct probe of a peer fails.

        Allowed values: 1 <= X

      • spec.memberlist.probeInterval
        string

        Required value

        Interval between probes of one peer. A shorter interval gives faster detection and more probe traffic. The value must be at least twice probeTimeout.

        Pattern: ^([0-9]+(\.[0-9]+)?(ms|s|m|h))+$

        Maximum length: 32

      • spec.memberlist.probeTimeout
        string

        Required value

        Time to wait for the answer to a probe. The value must be at most half of probeInterval.

        Pattern: ^([0-9]+(\.[0-9]+)?(ms|s|m|h))+$

        Maximum length: 32

      • spec.memberlist.retransmitMult
        integer

        Required value

        Retransmit multiplier of gossip broadcasts: retransmitMult * ceil(log10(nodeCount + 1)), where nodeCount is the number of nodes.

        Allowed values: 1 <= X

      • spec.memberlist.suspicionMaxTimeoutMult
        integer

        Required value

        Upper bound of the suspicion timer when no confirmations arrive.

        Allowed values: 1 <= X

      • spec.memberlist.suspicionMult
        integer

        Required value

        Multiplier of the suspicion window before memberlist marks a peer as dead. The value must be at least 3.

        Allowed values: 1 <= X

    • spec.poll
      object

      Required value

      Parameters of the poll of the control plane state between agents.

      • spec.poll.witnessFresherBy
        string

        Required value

        Minimum gap between the own last successful probe and the probe of the answering node. An answer with a smaller gap is not a witness.

        Pattern: ^([0-9]+(\.[0-9]+)?(ms|s|m|h))+$

        Maximum length: 32

    • spec.rejoin
      object

      Required value

      Parameters of the join attempts of the agent. All agents of the cluster use this section of the fastest profile among the node groups that are not Disabled.

      • spec.rejoin.interval
        string

        Required value

        Interval between the join attempts of the agent. The agent repeats the attempt at this fixed interval while some nodes are missing from the memberlist network.

        Pattern: ^([0-9]+(\.[0-9]+)?(ms|s|m|h))+$

        Maximum length: 32

    • spec.watchdog
      object

      Required value

      Parameters of the watchdog of the node.

      • spec.watchdog.feedInterval
        string

        Required value

        Interval between two feeds of the watchdog. The value must be at most a quarter of timeout.

        Pattern: ^([0-9]+(\.[0-9]+)?(ms|s|m|h))+$

        Maximum length: 32

      • spec.watchdog.margin
        string

        Required value

        Reserve that is added to timeout. The value must be at least 300ms and at least 5 % of timeout.

        Pattern: ^([0-9]+(\.[0-9]+)?(ms|s|m|h))+$

        Maximum length: 32

      • spec.watchdog.timeout
        string

        Required value

        Timeout of the watchdog. The node panics when the agent does not feed the watchdog for this time. The value is a whole number of seconds, because the watchdog device takes whole seconds.

        Pattern: ^([0-9]+(\.[0-9]+)?(ms|s|m|h))+$

        Maximum length: 32

Instance

Scope: Cluster

  • v1alpha2
  • v1alpha1

Instance describes the full lifecycle of a server in Deckhouse Platform, aggregating status from the Machine object (infrastructure) and Bashible service on the node (configuration).

  • spec
    object

    Specification of the Instance, including references to related resources.

    • spec.classReference
      object

      The reference to the InstanceClass object. It is unique for each cloud-provider-* or infra-provider-* module.

      • spec.classReference.kind
        string

        The object type (e.g., OpenStackInstanceClass). The object type is specified in the documentation of the corresponding cloud or infrastructure provider module.

      • spec.classReference.name
        string

        The name of the required InstanceClass object (e.g., finland-medium).

    • spec.machineRef
      object

      Reference to instance in the cloud.

      • spec.machineRef.apiVersion
        string

        APIVersion defines the versioned schema of this representation of an reference object.

      • spec.machineRef.kind
        string

        Kind is a string value representing the REST resource referenced object represents.

      • spec.machineRef.name
        string

        Name of referenced object.

      • spec.machineRef.namespace
        string

        Namespace of referenced object.

    • spec.nodeGroupRef
      object

      Reference to the NodeGroup that ordered this instance. It is known before the node joins the cluster.

      • spec.nodeGroupRef.name
        string

        NodeGroup name.

    • spec.nodeRef
      object

      Reference to the kubernetes node object.

      • spec.nodeRef.name
        string

        Node name.

  • status
    object

    Most recently observed status of the Instance.

    • status.bashibleStatus
      string

      Aggregated status of the bashible configuration process.

      Derived from BashibleReady condition.

      Allowed values: Error, Unknown, Ready, WaitingApproval

    • status.bootstrapStatus
      object

      Information about instance bootstrapping process.

      • status.bootstrapStatus.description
        string

        Description about getting logs.

      • status.bootstrapStatus.logsEndpoint
        string

        http endpoint for getting bootstrap logs.

    • status.conditions
      array of objects

      Detailed conditions of the instance (MachineReady, BashibleReady, etc.).

      On a node that has a NodeConfig, NodeAgentReady is True while the node agent reports and Unknown with reason NoReport after 10 minutes without a report.

      • status.conditions.lastHeartbeatTime
        string

        Last time the condition heartbeat was updated by its source.

      • status.conditions.lastTransitionTime
        string

        Last time the condition transitioned from one status to another.

      • status.conditions.message
        string

        A human-readable message indicating details about the transition.

      • status.conditions.observedGeneration
        integer

        The generation observed by the controller.

      • status.conditions.reason
        string

        Required value

        The reason for the condition’s last transition.

      • status.conditions.severity
        string

        Severity of the condition (Error, Warning, Info).

      • status.conditions.status
        string

        Required value

        Status of the condition (True, False, Unknown).

      • status.conditions.type
        string

        Required value

        Type of the condition (e.g., MachineReady, BashibleReady).

    • status.machineStatus
      string

      Aggregated status of the infrastructure machine.

      Derived from MachineReady condition.

      Allowed values: Progressing, Error, Ready, Blocked, Rebooting

    • status.message
      string

      A human-readable message indicating details about the current status or error.

      Displayed in CLI only when using -o wide for d8 k.

    • status.phase
      string

      The high-level lifecycle phase of the Instance.

      Mapping: Pending -> Provisioning -> Provisioned -> Running -> Terminating.

      Allowed values: Pending, Provisioning, Provisioned, Running, Terminating, Unknown

Describes an implementation-independent ephemeral machine resource.

  • status
    object
    • status.bootstrapStatus
      object

      Information about instance bootstrapping process.

      • status.bootstrapStatus.description
        string

        Description about getting logs.

      • status.bootstrapStatus.logsEndpoint
        string

        http endpoint for getting bootstrap logs.

    • status.classReference
      object

      The reference to the InstanceClass object. It is unique for each cloud-provider-* module.

      • status.classReference.kind
        string

        The object type (e.g., OpenStackInstanceClass). The object type is specified in the documentation of the corresponding cloud-provider- module.

        Allowed values: OpenStackInstanceClass, GCPInstanceClass, VsphereInstanceClass, AWSInstanceClass, YandexInstanceClass, AzureInstanceClass, VCDInstanceClass, ZvirtInstanceClass, DynamixInstanceClass, HuaweiCloudInstanceClass, DVPInstanceClass, BareMetalInstanceClass

      • status.classReference.name
        string

        The name of the required InstanceClass object (e.g., finland-medium).

    • status.currentStatus
      object

      Current status of the instance object.

      • status.currentStatus.lastUpdateTime
        string

        Last update time of current status.

      • status.currentStatus.phase
        string

        InstancePhase is a label for the condition of a instance at the current time.

    • status.lastOperation
      object

      Last operation refers to the status of the last operation performed.

      • status.lastOperation.description
        string

        Description of the current operation.

      • status.lastOperation.lastUpdateTime
        string

        Last update time of current operation.

      • status.lastOperation.state
        string

        State of operation.

      • status.lastOperation.type
        string

        Type of operation.

    • status.machineRef
      object

      Reference to instance in the cloud.

      • status.machineRef.apiVersion
        string

        APIVersion defines the versioned schema of this representation of an reference object.

      • status.machineRef.kind
        string

        Kind is a string value representing the REST resource referenced object represents.

      • status.machineRef.name
        string

        Name of referenced object.

      • status.machineRef.namespace
        string

        Namespace of referenced object.

    • status.nodeRef
      object

      Reference to the kubernetes node object.

      • status.nodeRef.name
        string

        Node name.

InstanceTypesCatalog

Scope: Cluster
Version: v1alpha1

Contains information about discovered data from cloud.

  • instanceTypes
    array of objects

    Required value

    List of available instance types.

    • instanceTypes.cpu
      integer or string

      Required value

      The number of CPUs for the instance type.

      Pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$

    • instanceTypes.memory
      integer or string

      Required value

      The amount of RAM a instance type has.

      Pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$

    • instanceTypes.name
      string

      Instance type name.

    • instanceTypes.rootDisk
      integer or string

      Root disk size.

      Pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$

NodeGroup

Short names: ng

Scope: Cluster

  • v1
  • v1alpha2
  • v1alpha1

Describes the runtime parameters of the node group.

Example:

# NodeGroup for cloud nodes in AWS.
apiVersion: deckhouse.io/v1
kind: NodeGroup
metadata:
  name: test
spec:
  nodeType: CloudEphemeral
  cloudInstances:
    zones:
      - eu-west-1a
      - eu-west-1b
    minPerZone: 1
    maxPerZone: 2
    classReference:
      kind: AWSInstanceClass
      name: test
  nodeTemplate:
    labels:
      tier: test
---
# NodeGroup for static nodes on bare metal servers (or VMs).
apiVersion: deckhouse.io/v1
kind: NodeGroup
metadata:
  name: worker
spec:
  nodeType: Static
  • metadata
    object
    • metadata.name
      string

      Pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$

      Maximum length: 42

  • spec
    object

    Required value

    • spec.chaos
      object

      Chaos monkey settings.

      Example:

      chaos:
        mode: DrainAndDelete
        period: 24h
      
      • spec.chaos.mode
        string

        The chaos monkey mode:

        • DrainAndDelete — drains and deletes a node when triggered;
        • Disabled — leaves this NodeGroup intact.

        Default: Disabled

        Allowed values: Disabled, DrainAndDelete

      • spec.chaos.period
        string

        The time interval to use for the chaos monkey.

        It is specified as a string containing the time unit in hours and minutes: 30m, 1h, 2h30m, 24h.

        Default: 6h

        Pattern: ^([0-9]+h([0-9]+m)?|[0-9]+m)$

    • spec.cloudInstances
      object

      Parameters of the node group cloud virtual machines.

      The cloudInstances parameter is used for CloudEphemeral nodes. When the cloud provider is configured through ModuleConfig, it is also used for CloudPermanent nodes, including master nodes. In this case, classReference specifies the instance class with the virtual machine parameters.

      • spec.cloudInstances.classReference
        object

        Required value

        The reference to the InstanceClass object. It is unique for each cloud-provider-* module.

        • spec.cloudInstances.classReference.kind
          string

          The kind of the InstanceClass object (e.g., OpenStackInstanceClass). Supported kinds are defined by the installed cloud provider modules and specified in their documentation.

        • spec.cloudInstances.classReference.name
          string

          The name of the required InstanceClass object (e.g., finland-medium).

      • spec.cloudInstances.maxPerZone
        integer

        Required value

        For CloudEphemeral nodes, sets the maximum number of nodes in each zone and the upper bound for autoscaling.

        For CloudPermanent nodes, the field is required when cloudInstances is set. Specify a value greater than or equal to minPerZone.

        Allowed values: 0 <= X

      • spec.cloudInstances.maxSurgePerZone
        integer

        The maximum number of instances the platform creates in each zone above the desired number while it replaces the instances of the group.

        Replacement happens when parameters that require recreating a virtual machine change, for example in the InstanceClass. The parameter does not limit the number of instances that the cluster autoscaler orders when it scales the group up. For details on what NodeGroup parameters affect, see NodeGroup parameters and their result.

        The value 0 is treated as if the parameter is not set, so the default value applies.

        Default: 1

        Allowed values: 0 <= X

      • spec.cloudInstances.maxUnavailablePerZone
        integer

        The maximum number of unavailable instances (during rollout) in the group in each zone.

        Default: 0

        Allowed values: 0 <= X

      • spec.cloudInstances.minPerZone
        integer

        Required value

        For CloudEphemeral nodes, sets the minimum number of nodes in each zone and the lower bound for autoscaling.

        For CloudPermanent nodes with a ModuleConfig-based provider configuration, sets the desired number of nodes in the group.

        Allowed values: 0 <= X

      • spec.cloudInstances.priority
        integer

        Priority of the node group.

        When scaling a cluster, the autoscaler will first select node groups with a higher priority set. If several node groups have the same priority, the autoscaler randomly selects a group of them. You can control the rate at which nodes are switched to lower-priority groups on demand (for more details, see the FAQ).

        Using priorities can be convenient to prefer ordering cheaper nodes (for example, spot instances) over more expensive ones.

      • spec.cloudInstances.quickShutdown
        Deprecated
        boolean

        Lowers CloudEphemeral machine drain timeout to 5 minutes.

      • spec.cloudInstances.standby
        integer or string

        The summary number of overprovisioned nodes for this NodeGroup in all zones.

        An overprovisioned node is a cluster node on which resources are reserved that are available at any time for scaling. The presence of such a node allows the cluster autoscaler not to wait for node initialization (which may take several minutes), but to immediately place a load on it.

        The value can be an absolute number (for example, 2) or a percentage of desired nodes (for example, 10%). If a percentage is specified, the absolute number is calculated based on the percentage of the maximum number of nodes (the maxPerZone parameter) rounded down, but not less than one.

        Pattern: ^[0-9]+%?$

      • spec.cloudInstances.standbyHolder
        object

        Amount of reserved resources.

        Used to determine whether to order overprovisioned nodes.

        • spec.cloudInstances.standbyHolder.notHeldResources
          Deprecated
          object

          Deprecated: the parameter is no longer used. Use the overprovisioningRate parameter.

          Describes the resources that will not be held (consumed) by the standby holder.

          • spec.cloudInstances.standbyHolder.notHeldResources.cpu
            integer or string

            Describes the amount of CPU that will not be held by standby holder on Nodes from this NodeGroup.

            The value can be an absolute number of cpus (for example, 2) as well as a milli representation (for example, 1500m).

            Pattern: ^[0-9]+m?$

          • spec.cloudInstances.standbyHolder.notHeldResources.memory
            integer or string

            Describes the amount of memory that will not be held by standby holder on Nodes from this NodeGroup.

            The value can be an absolute number of bytes (for example, 128974848) as well as a fixed-point number using one of memory suffixes: G, Gi, M, Mi.

            Pattern: ^[0-9]+(\.[0-9]+)?(E|P|T|G|M|K|Ei|Pi|Ti|Gi|Mi|Ki)?$

        • spec.cloudInstances.standbyHolder.overprovisioningRate
          integer

          Percentage of reserved resources calculated from the capacity of a node of a NodeGroup.

          Default: 50

          Allowed values: 1 <= X <= 80

      • spec.cloudInstances.zones
        array of strings

        List of availability zones to create instances in.

        The default value depends on the cloud provider selected and usually corresponds to all zones of the region being used.

        Example:

        zones:
        - Helsinki
        - Espoo
        - Tampere
        
    • spec.cri
      object

      Container runtime parameters.

      • spec.cri.containerd
        object

        Containerd runtime parameters.

        If used, cri.type must be set to Containerd.

        • spec.cri.containerd.maxConcurrentDownloads
          integer

          Set the max concurrent downloads for each pull.

          Default: 3

      • spec.cri.containerdV2
        object

        ContainerdV2 runtime parameters.

        If used, cri.type must be set to ContainerdV2.

        When using ContainerdV2, CgroupsV2 will be used (providing improved security and resource management).

        To use ContainerdV2 as a container runtime, the nodes in the group must meet the following requirements:

        • Support for CgroupsV2.
        • Linux kernel version 5.8 or newer.
        • Systemd version 244 or newer.
        • Support for erofs kernel module.
        • spec.cri.containerdV2.maxConcurrentDownloads
          integer

          Set the max concurrent downloads for each pull.

          Default: 3

      • spec.cri.docker
        Deprecated
        object

        Docker settings for nodes.

        • spec.cri.docker.manage
          boolean

          Enable Docker maintenance from bashible.

          Default: true

        • spec.cri.docker.maxConcurrentDownloads
          integer

          Set the max concurrent downloads for each pull.

          Default: 3

      • spec.cri.notManaged
        object

        Settings for not managed CRI for nodes.

        • spec.cri.notManaged.criSocketPath
          string

          Path to CRI socket.

      • spec.cri.type
        string

        Container runtime type.

        If not specified, the defaultCRI parameter value of ClusterConfiguration is used.

        Note. Docker support is deprecated.

        Note. The CSE edition supports containerd v2 only.

        Allowed values: Docker, Containerd, ContainerdV2, NotManaged

    • spec.disruptions
      object

      Disruptions settings for nodes.

      Example:

      disruptions:
        approvalMode: Automatic
        automatic:
          drainBeforeApproval: false
          windows:
          - from: '06:00'
            to: '08:00'
            days:
            - Tue
            - Sun
      
      • spec.disruptions.approvalMode
        string

        The approval mode for disruptive updates:

        • Manual — disable automatic disruption approval; the alert will be displayed if disruption is needed. Nodes are not drained automatically in Manual mode. If necessary, manually drain nodes before setting the update.node.deckhouse.io/disruption-approved= annotation. Caution. The master node group update mode must be Manual to avoid issues with draining.
        • Automatic — automatically approve disruption-involving updates. Nodes could be drained automatically before reboot, as configured by the drainBeforeApproval option.
        • RollingUpdate — in this mode, a new node with new settings will be created; then, the old node will be deleted. Available only for cloud nodes.

        If the RollingUpdate mode is not used, when updating, the node is first drained and then updated (rebooted) and put back into operation (uncordoned). Note that in this case, the cluster must have sufficient resources to accommodate the load while the node being updated is unavailable. In the RollingUpdate mode, the node is replaced by the updated node, i.e., an extra node appears in the cluster for the duration of the update. In cloud infrastructures, the RollingUpdate mode is convenient, for example, if there are no resources in the cluster to temporarily host the load from the node being updated.

        Default: Automatic

        Allowed values: Manual, Automatic, RollingUpdate

      • spec.disruptions.automatic
        object

        Additional parameters for the Automatic mode.

        • spec.disruptions.automatic.drainBeforeApproval
          boolean

          Drain Pods from the nodes before approving disruption.

          Caution. This setting ignores (nodes will be approved without draining Pods):

          • for the nodeGroup master with a single node;
          • for a single ready node in a nodeGroup picked out for Deckhouse placement.

          Default: true

        • spec.disruptions.automatic.windows
          array of objects

          Time windows for node disruptive updates.

          • spec.disruptions.automatic.windows.days
            array of strings

            Days of the week when node could be updated.

            Examples:

            days: Mon
            
            days: Wed
            
            • Element of the array
              string

              Day of the week.

              Allowed values: Mon, Tue, Wed, Thu, Fri, Sat, Sun

          • spec.disruptions.automatic.windows.from
            string

            Required value

            Start time of disruptive update window (UTC timezone).

            Pattern: ^(?:\d|[01]\d|2[0-3]):[0-5]\d$

            Example:

            from: '13:00'
            
          • spec.disruptions.automatic.windows.to
            string

            Required value

            End time of disruptive update window (UTC timezone).

            Pattern: ^(?:\d|[01]\d|2[0-3]):[0-5]\d$

            Example:

            to: '18:30'
            
      • spec.disruptions.rollingUpdate
        object

        Additional parameters for the RollingUpdate mode.

        • spec.disruptions.rollingUpdate.windows
          array of objects

          Time windows for node disruptive updates.

          • spec.disruptions.rollingUpdate.windows.days
            array of strings

            Days of the week when node could be updated.

            Examples:

            days: Mon
            
            days: Wed
            
            • Element of the array
              string

              Day of the week.

              Allowed values: Mon, Tue, Wed, Thu, Fri, Sat, Sun

          • spec.disruptions.rollingUpdate.windows.from
            string

            Required value

            Start time of disruptive update window (UTC timezone).

            Pattern: ^(?:\d|[01]\d|2[0-3]):[0-5]\d$

            Example:

            from: '13:00'
            
          • spec.disruptions.rollingUpdate.windows.to
            string

            Required value

            End time of disruptive update window (UTC timezone).

            Pattern: ^(?:\d|[01]\d|2[0-3]):[0-5]\d$

            Example:

            to: '18:30'
            
    • spec.fencing
      object

      Fencing settings for the node group.

      • spec.fencing.mode
        string

        Fencing mode of the node group. Every mode except Disabled selects the built-in FencingSLAProfile with the reaction timings of the group:

        • Disabled: fencing is off for the node group. A node group without fencing is in this mode too.
        • Critical: the critical profile, reaction in about 12 seconds. Available only in the commercial editions. In Community Edition (CE), Deckhouse does not let you set this mode. A group that already has this mode works as Normal.
        • Normal: the normal profile, reaction in about 30 seconds.
        • Slow: the slow profile, reaction in about 60 seconds.
        • Watchdog: deprecated. This mode works as Slow. Use Normal, Slow, or Critical (Critical is available only in the commercial editions).

        When at least one node group has a mode other than Disabled, the fencing-agent runs on every node of the cluster. If all node groups are Disabled, Deckhouse does not deploy the fencing-agent.

        In a cluster with one master, Deckhouse does not let you turn on fencing of an existing node group and shows a warning when you create a node group with fencing, because a failure or a restart of this master stops all nodes with fencing.

        How the fencing-agent works:

        • The agents of all cluster nodes form a single gossip network (memberlist). Every agent uses the timings of the fastest mode of the node groups in this network. The memberlist protocol detects failures correctly only when all members have the same timings.
        • On the nodes of a group with a mode other than Disabled, the agent arms the /dev/watchdog device with the watchdog timeout of the profile of the group. While the agent runs, it resets the timer. If the node or the agent hangs, the timer expires and the kernel panics.
        • When the node does not see the control plane for longer than controlPlaneLostAfter of the profile, the agent stops the reset of the timer at once. When the watchdog timeout expires, the node stops. If the control plane is available again before the timeout expires, the agent resets the timer again, and the node continues to work. Thus, a short pause of the control plane does not stop the nodes.
        • After the stop of the reset, the agent asks the agents of the other nodes for a witness, that is, a node that sees the control plane. The witness writes an attestation of the stop. After that, the stop is final.
        • On the nodes of a Disabled group, the agent only takes part in failure detection. The agent does not arm the watchdog.
        • The agent respects maintenance annotations node-manager.deckhouse.io/fencing-disable, update.node.deckhouse.io/approved, and update.node.deckhouse.io/disruption-approved, temporarily disabling the watchdog during planned operations.

        For a mode other than Disabled, the NodeGroupConfiguration script loads the softdog kernel module with soft_panic=1. The fencing-agent applies the watchdog timeout when it arms the device.

        On nodes of a group with systemType: Immutable (Deckhouse Engine), the NodeGroupConfiguration script does not run. On these nodes, softdog is built into the kernel and starts with soft_panic=1. The operating system image sets the permissions on /dev/watchdog. The fencing-agent applies the timeout of the profile in the same way as on other nodes. If you installed a node before the Deckhouse Engine release with softdog in the kernel, the node does not have softdog. The node gets it only after a reinstall or an update of the boot partition.

        For a mode other than Disabled, Deckhouse disables the automatic reboot of the node after a kernel panic at the OS level. This prevents a node from returning in an undefined state: the stopped node stays down until you restore it manually.

        When the stop of the node is certain, the fencing-controller sets the node.kubernetes.io/out-of-service taint on the node. After the taint, the controller sets the Failed phase and the DisruptionTarget condition on the pods of the node and deletes the pods immediately. This includes the pods that tolerate the taint. The controller does not delete the pods of DaemonSets and the static pods. The controller does not remove the finalizers of the pods: a pod with a finalizer stays in the API until the owner of the finalizer removes it. For such a pod of a StatefulSet, the controller sends a warning event with the names of the finalizers. The evacuation is complete when the node has no VolumeAttachment objects. If the evacuation is not complete after 60 seconds, the incident stays in the Evicting phase and gets the Progressing condition with the status False and the reason. The controller continues to examine the node. After the evacuation, Deckhouse acts on the node as follows:

        • For a node of a CloudPermanent, CloudStatic, or Static group (this includes a Static group with staticInstances, that is, CAPS), for a master node, and for an etcd arbiter node, Deckhouse only keeps the taint. It never deletes the Node object.
        • For a node of a CloudEphemeral group with status.engine: CAPI, Deckhouse deletes the Node object only when the MachineHealthCheck of the Cluster API cluster reports its machines and 50 percent or fewer of them are unhealthy. Deckhouse counts the stopped machine as unhealthy. MachineHealthCheck then sets the HealthCheckSucceeded condition of the Machine to False with the NodeDeleted reason and replaces the machine. If more than 50 percent of the machines are unhealthy, or the MachineHealthCheck does not report its machines yet, Deckhouse does not delete the Node object yet. The controller continues to examine the machines. When 50 percent or fewer of them are unhealthy, the controller deletes the Node object and completes the evacuation.
        • For a node of a CloudEphemeral group with status.engine: MCM, Deckhouse deletes the Machine (machine.sapcloud.io). Machine Controller Manager creates a new machine.

        When the node returns (the Node is Ready and it has a new boot ID), the fencing-controller removes the taint automatically. The controller works only while at least one node group has a mode other than Disabled. If you turn fencing off in all node groups, the taint stays on the node, and you must remove it manually.

        The stop is certain in one of two cases: a witness confirmed the stop and watchdog.timeout + watchdog.margin of the profile passed, or the node is silent for controlPlaneLostAfter + 0.2s + watchdog.timeout + watchdog.margin + 0.9s of the profile after its last sign of life. The controller does not set the taint while the Fencing condition of the Node is not True.

        If the fencing-controller does not set the taint, Kubernetes evicts the pods of the node after the time in the failedNodePodEvictionTimeoutSeconds parameter of the control-plane-manager module (300 seconds by default).

        For details on how fencing handles a failed node, see FAQ.

        Default: Disabled

        Allowed values: Disabled — Fencing is off for the node group. A node group without fencing is in this mode too.
        Normal — The normal profile, reaction in about 30 seconds.
        Slow — The slow profile, reaction in about 60 seconds.
        Critical — The critical profile, reaction in about 12 seconds. Available only in the commercial editions.
        Watchdog — Deprecated. Works as Slow. Use Normal, Slow, or Critical (Critical is available only in the commercial editions).

      • spec.fencing.watchdog
        object

        Deprecated. Deckhouse keeps the settings of fencing 1.0 only for compatibility and ignores them.

        • spec.fencing.watchdog.timeout
          string

          Deprecated. Deckhouse ignores this field. The watchdog timeout comes from the profile of the fencing mode.

          Pattern: ^[0-9]+[smh]{1}$

    • spec.gpu
      object

      Available in editions: Ultimate/EE

      GPU parameters.

      • spec.gpu.exclusive
        object

        Without GPU sharing.

      • spec.gpu.mig
        object

        MIG sharing strategy parameters.

        • spec.gpu.mig.customConfigs
          array of objects

          Custom MIG partitioning by GPU index.

          Used only when partedConfig: custom is set.

          Represents an array of devices (a map keyed by index), where each entry refers to a GPU by its index and describes the slices to be created.

          Examples:

          customConfigs:
            profile: 3g.40gb
            count: 2
          
          customConfigs:
            profile: 1g.10gb.me
            count: 1
          
          • spec.gpu.mig.customConfigs.index
            integer

            Required value

            GPU index on the node.

            Allowed values: 0 <= X

          • spec.gpu.mig.customConfigs.slices
            array of objects

            Required value

            List of MIG slices to create.

            • spec.gpu.mig.customConfigs.slices.count
              integer

              Required value

              Number of slices in the selected profile.

              Default: 1

              Allowed values: 1 <= X <= 7

            • spec.gpu.mig.customConfigs.slices.profile
              string

              Required value

              MIG profile name.

              Pattern: ^[0-9]+g\.[0-9]+gb(\.me)?$

              Examples:

              profile: 3g.40gb
              
              profile: 1g.10gb
              
              profile: 1g.10gb.me
              
        • spec.gpu.mig.partedConfig
          string

          MIG configuration name.

          Standard all-* profiles are supported (the same partitioning applied to all GPUs), as well as the custom value, which allows defining partitioning individually for each GPU.

          Default: all-1g.5gb

      • spec.gpu.sharing
        string

        GPU sharing strategy:

        • Exclusive — no sharing (each GPU is fully allocated to a single Pod).
        • TimeSlicing — time-sliced sharing (multiple Pods can share one GPU concurrently by splitting GPU time).
        • MIG — use NVIDIA Multi-Instance GPU (MIG) to split each GPU into multiple instances (if supported by hardware).

        Allowed values: TimeSlicing, MIG, Exclusive

      • spec.gpu.timeSlicing
        object

        TimeSlicing sharing strategy parameters.

        • spec.gpu.timeSlicing.partitionCount
          number

          Count of partition per GPU.

          Default: 4

          Allowed values: 1 <= X <= 100

    • spec.kubelet
      object

      Kubelet settings for nodes.

      • spec.kubelet.containerLogMaxFiles
        integer

        How many rotated log files to store before deleting them.

        Default: 4

        Allowed values: 2 <= X <= 20

      • spec.kubelet.containerLogMaxSize
        string

        Maximum log file size before it is rotated.

        Default: 50Mi

        Pattern: \d+[Ei|Pi|Ti|Gi|Mi|Ki|E|P|T|G|M|k|m]

      • spec.kubelet.maxPods
        integer

        Set the max count of pods per node for given NodeGroup. When left unspecified, the system automatically determines this limit according to the value of the podSubnetNodeCIDRPrefix parameter of the control-plane-manager module:

        • 120 pods for podSubnetNodeCIDRPrefix ≥ 24;
        • 250 pods for podSubnetNodeCIDRPrefix = 23;
        • 500 pods for podSubnetNodeCIDRPrefix = 22;
        • 1000 pods for podSubnetNodeCIDRPrefix ≤ 21.
      • spec.kubelet.memorySwap
        object

        Configuration of swap memory usage for kubelet.

        • spec.kubelet.memorySwap.limitedSwap
          object

          Configuration for the limited swap mode. Required when swapBehavior is set to LimitedSwap.

          • spec.kubelet.memorySwap.limitedSwap.size
            string

            Required value

            Size of the swap file to create, specified in GB (for example, “2G”, “4G”).

            Pattern: ^\d+G$

            Example:

            size: 1G
            
        • spec.kubelet.memorySwap.swapBehavior
          string

          Defines how swap memory is handled on the node:

          • NoSwap: Swap is disabled. This is the traditional Kubernetes behavior.
          • LimitedSwap: Kubelet can use swap memory up to the limit specified in limitedSwap.size.

          For details on swap memory usage on nodes, refer to Kubernetes documentation.

          Allowed values: NoSwap, LimitedSwap

        • spec.kubelet.memorySwap.swappiness
          integer

          Defines the kernel’s tendency to use the swap.

          • 0: Minimal swap usage (only to avoid OOM).
          • 60: Default balanced behavior.
          • 100: Aggressive swap usage.

          This value is set via vm.swappiness sysctl parameter.

          Default: 60

          Allowed values: 0 <= X <= 100

      • spec.kubelet.resourceReservation
        object

        Management of resource reservation for system daemons on a node.

        More info in the Kubernetes documentation.

        • spec.kubelet.resourceReservation.mode
          string

          Defines the system resource reservation mode:

          • Off — disables resource reservation.
          • Auto — reserves resources based on the Node capacity.
          • Static — reserves resources based on the values provided by the user via the static parameter.

          Note that currently we do not use a dedicated group cgroup for resource reservation (-system-reserved-cgroup is not used).

          Default: Auto

        • spec.kubelet.resourceReservation.static
          object

          Resource reservation parameters for the ‘Static’ mode.

          • spec.kubelet.resourceReservation.static.cpu
            integer or string

            Pattern: \d+[m]

          • spec.kubelet.resourceReservation.static.ephemeralStorage
            integer or string

            Pattern: \d+[Ei|Pi|Ti|Gi|Mi|Ki|E|P|T|G|M|k|m]

          • spec.kubelet.resourceReservation.static.memory
            integer or string

            Pattern: \d+[Ei|Pi|Ti|Gi|Mi|Ki|E|P|T|G|M|k|m]

      • spec.kubelet.rootDir
        string

        Directory path for managing kubelet files (volume mounts,etc).

        Default: /var/lib/kubelet

      • spec.kubelet.seccompDefault
        boolean

        Use RuntimeDefault seccomp profile for workloads that don’t specify seccomp profile explicitly.

      • spec.kubelet.topologyManager
        object

        Controls the kubelet resource managers on NUMA nodes: the CPU manager, the memory manager and the topology manager.

        With a policy other than None, containers of Guaranteed pods with an integer CPU request get exclusive CPUs, and the memory of Guaranteed pods is bound to NUMA nodes. With Restricted and SingleNumaNode, they get whole physical cores. Deckhouse calculates NUMA-dependent kubelet settings on each node of the group: it splits reservedMemory between NUMA nodes and sets max-allowable-numa-nodes on nodes with more than 8 NUMA nodes.

        A policy other than None requires resourceReservation.mode Auto, or Static with a positive static.cpu. Changing the settings restarts the kubelet and can require a disruption on nodes with Guaranteed pods. See How do I give pods dedicated CPUs and NUMA alignment?

        For more information, see the Kubernetes documentation.

        • spec.kubelet.topologyManager.enabled
          Deprecated
          boolean

          Deprecated, has no effect. The resource managers are controlled by policy alone.

        • spec.kubelet.topologyManager.policy
          string

          Resource alignment policy and the related kubelet settings:

          • None: no alignment. The CPU and memory managers are not used.
          • BestEffort: Guaranteed pods get exclusive CPUs and NUMA-bound memory. Resources are aligned to NUMA nodes when possible; a pod is admitted even if they cannot be aligned, but it is rejected if its memory cannot be placed.
          • Restricted: Guaranteed pods get exclusive whole physical cores and NUMA-bound memory. A pod is rejected if its resources cannot be aligned; when a pod needs several NUMA nodes, the closest ones are preferred.
          • SingleNumaNode: as Restricted, but all resources of a pod (or of a container, see scope) must come from one NUMA node.

          More information can be found in the Kubernetes documentation.

          Default: None

          Allowed values: None, BestEffort, Restricted, SingleNumaNode

        • spec.kubelet.topologyManager.scope
          string

          Defines granularity with which resources are aligned:

          • Container: resources are aligned for each container separately.
          • Pod: all containers of a pod are aligned together.

          Default: Pod

          Allowed values: Container, Pod

    • spec.nodeDrainTimeoutSecond
      integer

      Maximum duration (in seconds) for which the controller attempts to reschedule pods to a new machine during node draining.

      When a machine is deleted, the Drain operation evicts the pods for this long. Then the machine is deleted anyway. In a Static or CloudStatic NodeGroup without this field, the Drain of a deleted machine has no time limit.

      The range is the one the module’s own settings have always enforced. A value outside it used to be accepted here and then rejected when the module read it, which stopped node-manager reconciling anything at all.

      Allowed values: 30 <= X <= 7200

    • spec.nodeTemplate
      object

      Specification of some of the fields that will be maintained in all nodes of the group.

      Example:

      nodeTemplate:
        labels:
          environment: production
          app: warp-drive-ai
        annotations:
          ai.fleet.com/discombobulate: 'true'
        taints:
        - effect: NoExecute
          key: ship-class
          value: frigate
      
      • spec.nodeTemplate.annotations
        object

        Similar to the standard metadata.annotations field.

        Example:

        annotations:
          ai.fleet.com/discombobulate: 'true'
        
      • spec.nodeTemplate.labels
        object

        Similar to the standard metadata.labels field.

        Example:

        labels:
          environment: production
          app: warp-drive-ai
        
      • spec.nodeTemplate.taints
        array of objects

        Similar to the .spec.taints field of the Node object.

        Available fields: effect, key, and values.

        Example:

        taints:
        - effect: NoExecute
          key: ship-class
          value: frigate
        
        • spec.nodeTemplate.taints.effect
          string

          Allowed values: NoSchedule, PreferNoSchedule, NoExecute

        • spec.nodeTemplate.taints.key
          string
        • spec.nodeTemplate.taints.value
          string
    • spec.nodeType
      string

      Required value

      The type of nodes this group provides:

      • CloudEphemeral — nodes for this group will be automatically created (and deleted) in the cloud of the specified cloud provider;
      • CloudPermanent. Nodes are created and deleted using dhctl. Virtual machine parameters are set in the cloud provider configuration. When the provider is configured through ModuleConfig, the parameters are set in the instance class referenced by cloudInstances.classReference;
      • CloudStatic — a static node (created manually or using any external tools) hosted in the cloud integrated with one of the cloud providers. This node has the CSI running, and it is managed by the cloud-controller-manager: the Node object automatically gets the information about the zone and region based on the cloud data; if a node gets deleted from the cloud, its corresponding Node object will be deleted in Kubernetes;
      • Static — a static node hosted on a bare metal or virtual machine. The cloud-controller-manager does not manage the node even if one of the cloud providers is enabled.

      Allowed values: CloudEphemeral, CloudPermanent, CloudStatic, Static

    • spec.operatingSystem
      object

      Operating System settings for nodes.

      • spec.operatingSystem.manageKernel
        Deprecated
        boolean

        This parameter has no effect. Earlier, it enabled kernel maintenance on behalf of bashible.

        Default: true

    • spec.providerType
      string

      Cloud provider type of the node group. None or an empty value — for a static node group.

    • spec.staticInstances
      object

      Parameter for provisioning static machines to the cluster.

      • spec.staticInstances.count
        integer

        Target number of StaticInstance resources (matching the labelSelector) that DP bootstraps and adds into the cluster.

        In a NodeGroup with Static or CloudStatic nodes, this parameter sets the expected number of nodes. DP uses this value for state control and automation purposes.

        Warning. Set this parameter explicitly. The default value is 0, which means DP won’t bootstrap any StaticInstance resource or add any node to the group unless this value is modified. To add static nodes, specify the required number here (greater than 0).

        Default: 0

        Allowed values: 0 <= X

      • spec.staticInstances.labelSelector
        object

        Label selector for choosing the StaticInstance resources the node group uses.

        If both matchLabels and matchExpressions are set, a StaticInstance resource must satisfy all the conditions at once.

        An empty selector ({}) chooses every StaticInstance resource. If the selector is not set, the node group chooses no StaticInstance resources.

        Warning. A selector with at least one condition cannot be modified or removed. To change it, create a new node group and move the static nodes to it.

        Adding a selector to a node group that already uses StaticInstance resources is possible only when the labels of all those resources satisfy the new selector. Otherwise the webhook rejects the change and lists the resources that do not match the selector in the error message.

        After the selector is added, existing static nodes are not recreated and keep their previous configuration, but the platform applies the selector to every new node. A StaticInstance resource that does not match the selector cannot be reused, so the node group ends up with fewer nodes than the count parameter specifies.

        To add a selector:

        • Label all the StaticInstance resources of the node group with the d8 k label staticinstance <STATIC_INSTANCE> <KEY>=<VALUE> command.
        • Set the selector in the spec.staticInstances parameter of the NodeGroup resource.
        • spec.staticInstances.labelSelector.matchExpressions
          array of objects

          A list of label selector requirements. The requirements are ANDed.

          A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

          • spec.staticInstances.labelSelector.matchExpressions.key
            string

            key is the label key that the selector applies to.

          • spec.staticInstances.labelSelector.matchExpressions.operator
            string

            operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.

          • spec.staticInstances.labelSelector.matchExpressions.values
            array of strings

            values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.

            • Element of the array
              string

              Pattern: [a-z0-9]([-a-z0-9]*[a-z0-9])?

              Length: 1..63

        • spec.staticInstances.labelSelector.matchLabels
          object

          matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.

    • spec.systemType
      string

      How the nodes of the group are managed.

      • GenericOS: Generic OS nodes, configured by bashible on the fly. An unset field means the same.
      • DeckhouseEngine: Deckhouse Engine nodes. The node agent brings the node to the state described in a NodeConfig object, without bashible. The machine must boot from a Deckhouse Engine image.

      DeckhouseEngine is supported for:

      • Static: Machines must be added with d8 node add. staticInstances cannot be used.
      • CloudEphemeral: Machines must be created by the cloud provider through Cluster API (CAPI).
      • CloudPermanent: Machines are created by dhctl.

      DeckhouseEngine is not supported for CloudStatic groups.

      A group can become DeckhouseEngine only while bashible has configured none of its nodes. DeckhouseEngine cannot be removed or changed to GenericOS.

      Allowed values: GenericOS, DeckhouseEngine

    • spec.update
      object
      • spec.update.maxConcurrent
        integer or string

        Maximum number of concurrently updating nodes.

        Can be set as absolute count or as a percent of total nodes.

        Default: 1

        Pattern: ^[1-9][0-9]*%?$

  • status
    object
    • status.conditionSummary
      object
      • status.conditionSummary.ready
        string

        Status of the condition summary.

        Allowed values: True, False

      • status.conditionSummary.statusMessage
        string

        Status message about group handling.

    • status.conditions
      array of objects
      • status.conditions.lastTransitionTime
        string

        Last time the condition transit from one status to another.

      • status.conditions.message
        string

        Human readable message indicating details about last transition.

      • status.conditions.status
        string

        Required value

        Status of the condition, one of True, False.

      • status.conditions.type
        string

        Required value

        Type of node group condition.

    • status.deckhouse
      object
      • status.deckhouse.observed
        object

        Contains last timestamp when the resource change was noted by the operator and its checksum.

        • status.deckhouse.observed.checkSum
          string

          The checksum of the last observed resource.

        • status.deckhouse.observed.lastTimestamp
          string

          Last timestamp when a change of the resource was noted by the operator.

      • status.deckhouse.processed
        object

        Contains last timestamp when the resource was applied in the cluster by the operator and its checksum.

        • status.deckhouse.processed.checkSum
          string

          The checksum of the last applied resource.

        • status.deckhouse.processed.lastTimestamp
          string

          Last timestamp when the resource was applied in the cluster.

      • status.deckhouse.synced
        string

        True if last observed version of the resource was successfully applied in the cluster.

    • status.desired
      integer

      Number of desired machines in the group.

    • status.engine
      string

      Machinery used to manage nodes in this group.

      The field value indicates which component manages the nodes:

      • None — nodes in the group are not managed by Machine Controller Manager or CAPI.
      • MCM — nodes in the group are managed by Machine Controller Manager.
      • CAPI — nodes in the group are managed by Cluster API.

      Allowed values: None, MCM, CAPI

    • status.error
      string

      Error message about possible problems with the group handling.

    • status.instances
      integer

      Number of instances (in any state) in the group.

    • status.kubernetesVersion
      string

      The current version of kubernetes on the nodes, or the version to which the nodes will be upgraded.

    • status.lastMachineFailures
      array of objects
      • status.lastMachineFailures.lastOperation
        object
        • status.lastMachineFailures.lastOperation.description
          string

          Last operation’s description.

        • status.lastMachineFailures.lastOperation.lastUpdateTime
          string

          Timestamp of last status update for operation.

        • status.lastMachineFailures.lastOperation.state
          string

          Machine’s operation state.

        • status.lastMachineFailures.lastOperation.type
          string

          Type of operation.

      • status.lastMachineFailures.name
        string

        Machine’s name.

      • status.lastMachineFailures.ownerRef
        string

        Machine owner’s name.

      • status.lastMachineFailures.providerID
        string

        Machine’s ProviderID.

    • status.max
      integer

      Maximum amount of instances in the group.

    • status.min
      integer

      Minimal amount of instances in the group.

    • status.nodes
      integer

      Number of Kubernetes nodes (in any state) in the group.

    • status.ready
      integer

      Number of ready Kubernetes nodes in the group.

    • status.standby
      integer

      Number of overprovisioned instances in the group.

    • status.upToDate
      integer

      Number of up-to-date nodes in the group.

Describes the runtime parameters of the node group.

  • metadata
    object
    • metadata.name
      string

      Pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$

      Maximum length: 42

  • spec
    object

    Required value

    • spec.chaos
      object

      Chaos monkey settings.

      Example:

      chaos:
        mode: DrainAndDelete
        period: 24h
      
      • spec.chaos.mode
        string

        The chaos monkey mode:

        • DrainAndDelete — drains and deletes a node when triggered;
        • Disabled — leaves this NodeGroup intact.

        Default: Disabled

        Allowed values: Disabled, DrainAndDelete

      • spec.chaos.period
        string

        The time interval to use for the chaos monkey (can be specified in the Go format).

        Default: 6h

        Pattern: ^[0-9]+[mh]{1}$

    • spec.cloudInstances
      object

      Parameter for provisioning the cloud-based VMs.

      Caution. Can only be used together with nodeType: CloudEphemeral.

      • spec.cloudInstances.classReference
        object

        Required value

        The reference to the InstanceClass object. It is unique for each cloud-provider-* module.

        • spec.cloudInstances.classReference.kind
          string

          The kind of the InstanceClass object (e.g., OpenStackInstanceClass). Supported kinds are defined by the installed cloud provider modules and specified in their documentation.

        • spec.cloudInstances.classReference.name
          string

          The name of the required InstanceClass object (e.g., finland-medium).

      • spec.cloudInstances.maxPerZone
        integer

        Required value

        The maximum number of instances for the group in each zone.

        This value is used as the upper bound in cluster-autoscaler.

        Allowed values: 0 <= X

      • spec.cloudInstances.maxSurgePerZone
        integer

        The maximum number of instances the platform creates in each zone above the desired number while it replaces the instances of the group.

        Replacement happens when parameters that require recreating a virtual machine change, for example in the InstanceClass. The parameter does not limit the number of instances that the cluster autoscaler orders when it scales the group up. For details on what NodeGroup parameters affect, see NodeGroup parameters and their result.

        The value 0 is treated as if the parameter is not set, so the default value applies.

        Default: 1

        Allowed values: 0 <= X

      • spec.cloudInstances.maxUnavailablePerZone
        integer

        The maximum number of unavailable instances (during rollout) in the group in each zone.

        Default: 0

        Allowed values: 0 <= X

      • spec.cloudInstances.minPerZone
        integer

        Required value

        The minimum number of instances for the group in each zone.

        This value is used in the MachineDeployment object and as a lower bound in cluster-autoscaler.

        Allowed values: 0 <= X

      • spec.cloudInstances.standby
        integer or string

        The summary number of overprovisioned nodes for this NodeGroup in all zones.

        An overprovisioned node is a cluster node on which resources are reserved that are available at any time for scaling. The presence of such a node allows the cluster autoscaler not to wait for node initialization (which may take several minutes), but to immediately place a load on it.

        The value can be an absolute number (for example, 2) or a percentage of desired nodes (for example, 10%). If a percentage is specified, the absolute number is calculated based on the percentage of the maximum number of nodes (the maxPerZone parameter) rounded down, but not less than one.

        Pattern: ^[0-9]+%?$

      • spec.cloudInstances.standbyHolder
        object

        Amount of reserved resources.

        Used to determine whether to order overprovisioned nodes.

        • spec.cloudInstances.standbyHolder.notHeldResources
          object

          Describes the resources that will not be held (consumed) by the standby holder.

          • spec.cloudInstances.standbyHolder.notHeldResources.cpu
            integer or string

            Describes the amount of CPU that will not be held by standby holder on Nodes from this NodeGroup.

            The value can be an absolute number of cpus (for example, 2) as well as a milli representation (for example, 1500m).

            Pattern: ^[0-9]+m?$

          • spec.cloudInstances.standbyHolder.notHeldResources.memory
            integer or string

            Describes the amount of memory that will not be held by standby holder on Nodes from this NodeGroup.

            The value can be an absolute number of bytes (for example, 128974848) as well as a fixed-point number using one of memory suffixes: G, Gi, M, Mi.

            Pattern: ^[0-9]+(\.[0-9]+)?(E|P|T|G|M|K|Ei|Pi|Ti|Gi|Mi|Ki)?$

      • spec.cloudInstances.zones
        array of strings

        List of availability zones to create instances in.

        The default value depends on the cloud provider selected and usually corresponds to all zones of the region being used.

        Example:

        zones:
        - Helsinki
        - Espoo
        - Tampere
        
    • spec.cri
      object

      Container runtime parameters.

      • spec.cri.containerd
        object

        Containerd runtime parameters.

        If used, cri.type must be set to Containerd.

        • spec.cri.containerd.maxConcurrentDownloads
          integer

          Set the max concurrent downloads for each pull.

          Default: 3

      • spec.cri.docker
        object

        Docker settings for nodes.

        Note. the Docker is deprecated.

        • spec.cri.docker.manage
          boolean

          Enable Docker maintenance from bashible.

          Default: true

        • spec.cri.docker.maxConcurrentDownloads
          integer

          Set the max concurrent downloads for each pull.

          Default: 3

      • spec.cri.notManaged
        object

        Settings for not managed CRI for nodes.

        • spec.cri.notManaged.criSocketPath
          string

          Path to CRI socket.

      • spec.cri.type
        string

        Container runtime type.

        Value defaultCRI from the initial cluster configration (cluster-configuration.yaml parameter from the d8-cluster-configuration secret in the kube-system namespace) is used if not specified.

        Note. Docker support is deprecated.

        Allowed values: Docker, Containerd, NotManaged

    • spec.disruptions
      object

      Disruptions settings for nodes.

      Example:

      disruptions:
        approvalMode: Automatic
        automatic:
          drainBeforeApproval: false
          windows:
          - from: '06:00'
            to: '08:00'
            days:
            - Tue
            - Sun
      
      • spec.disruptions.approvalMode
        string

        The approval mode for disruptive updates:

        • Manual — disable automatic disruption approval; the alert will be displayed if disruption is needed. Caution. The master node group update mode must be Manual to avoid issues with draining.
        • Automatic — automatically approve disruption-involving updates.
        • RollingUpdate — in this mode, a new node with new settings will be created; then, the old node will be deleted. Available only for cloud nodes.

        If the RollingUpdate mode is not used, when updating, the node is first drained and then updated (rebooted) and put back into operation (uncordoned). Note that in this case, the cluster must have sufficient resources to accommodate the load while the node being updated is unavailable. In the RollingUpdate mode, the node is replaced by the updated node, i.e., an extra node appears in the cluster for the duration of the update. In cloud infrastructures, the RollingUpdate mode is convenient, for example, if there are no resources in the cluster to temporarily host the load from the node being updated.

        Default: Automatic

        Allowed values: Manual, Automatic, RollingUpdate

      • spec.disruptions.automatic
        object

        Additional parameters for the Automatic mode.

        • spec.disruptions.automatic.drainBeforeApproval
          boolean

          Drain Pods from the nodes before approving disruption.

          Caution. This setting ignores (nodes will be approved without draining Pods):

          • for the nodeGroup master with a single node;
          • for a single ready node in a nodeGroup picked out for Deckhouse placement.

          Default: true

        • spec.disruptions.automatic.windows
          array of objects

          Time windows for node disruptive updates.

          • spec.disruptions.automatic.windows.days
            array of strings

            Days of the week when node could be updated.

            Examples:

            days: Mon
            
            days: Wed
            
            • Element of the array
              string

              Day of the week.

              Allowed values: Mon, Tue, Wed, Thu, Fri, Sat, Sun

          • spec.disruptions.automatic.windows.from
            string

            Required value

            Start time of disruptive update window (UTC timezone).

            Pattern: ^(?:\d|[01]\d|2[0-3]):[0-5]\d$

            Example:

            from: '13:00'
            
          • spec.disruptions.automatic.windows.to
            string

            Required value

            End time of disruptive update window (UTC timezone).

            Pattern: ^(?:\d|[01]\d|2[0-3]):[0-5]\d$

            Example:

            to: '18:30'
            
      • spec.disruptions.rollingUpdate
        object

        Additional parameters for the RollingUpdate mode.

        • spec.disruptions.rollingUpdate.windows
          array of objects

          Time windows for node disruptive updates.

          • spec.disruptions.rollingUpdate.windows.days
            array of strings

            Days of the week when node could be updated.

            Examples:

            days: Mon
            
            days: Wed
            
            • Element of the array
              string

              Day of the week.

              Allowed values: Mon, Tue, Wed, Thu, Fri, Sat, Sun

          • spec.disruptions.rollingUpdate.windows.from
            string

            Required value

            Start time of disruptive update window (UTC timezone).

            Pattern: ^(?:\d|[01]\d|2[0-3]):[0-5]\d$

            Example:

            from: '13:00'
            
          • spec.disruptions.rollingUpdate.windows.to
            string

            Required value

            End time of disruptive update window (UTC timezone).

            Pattern: ^(?:\d|[01]\d|2[0-3]):[0-5]\d$

            Example:

            to: '18:30'
            
    • spec.kubelet
      object

      Kubelet settings for nodes.

      • spec.kubelet.containerLogMaxFiles
        integer

        How many rotated log files to store before deleting them.

        Warning. This parameter does nothing if CRI type is Docker.

        Default: 4

        Allowed values: 2 <= X <= 20

      • spec.kubelet.containerLogMaxSize
        string

        Maximum log file size before it is rotated.

        Warning. This parameter does nothing if CRI type is Docker.

        Default: 50Mi

        Pattern: \d+[Ei|Pi|Ti|Gi|Mi|Ki|E|P|T|G|M|k|m]

      • spec.kubelet.maxPods
        integer

        Set the max count of pods per node for given NodeGroup. When left unspecified, the system automatically determines this limit according to the value of the podSubnetNodeCIDRPrefix parameter of the control-plane-manager module:

        • 120 pods for podSubnetNodeCIDRPrefix ≥ 24;
        • 250 pods for podSubnetNodeCIDRPrefix = 23;
        • 500 pods for podSubnetNodeCIDRPrefix = 22;
        • 1000 pods for podSubnetNodeCIDRPrefix ≤ 21.
      • spec.kubelet.rootDir
        string

        Directory path for managing kubelet files (volume mounts,etc).

        Default: /var/lib/kubelet

      • spec.kubelet.seccompDefault
        boolean

        Use RuntimeDefault seccomp profile for workloads that don’t specify seccomp profile explicitly.

    • spec.nodeTemplate
      object

      Specification of some of the fields that will be maintained in all nodes of the group.

      Example:

      nodeTemplate:
        labels:
          environment: production
          app: warp-drive-ai
        annotations:
          ai.fleet.com/discombobulate: 'true'
        taints:
        - effect: NoExecute
          key: ship-class
          value: frigate
      
      • spec.nodeTemplate.annotations
        object

        Similar to the standard metadata.annotations field.

        Example:

        annotations:
          ai.fleet.com/discombobulate: 'true'
        
      • spec.nodeTemplate.labels
        object

        Similar to the standard metadata.labels field.

        Example:

        labels:
          environment: production
          app: warp-drive-ai
        
      • spec.nodeTemplate.taints
        array of objects

        Similar to the .spec.taints field of the Node object.

        Available fields: effect, key, and values.

        Example:

        taints:
        - effect: NoExecute
          key: ship-class
          value: frigate
        
        • spec.nodeTemplate.taints.effect
          string

          Allowed values: NoSchedule, PreferNoSchedule, NoExecute

        • spec.nodeTemplate.taints.key
          string
        • spec.nodeTemplate.taints.value
          string
    • spec.nodeType
      string

      Required value

      The type of nodes this group provides.

      • Cloud — nodes for this group will be automatically created (and deleted) in the cloud of the specified cloud provider;
      • Static — a static node hosted on a bare metal or virtual machine. The cloud-controller-manager does not manage the node even of one of the cloud providers is enabled;
      • Hybrid — a static node (created manually or using any external tools) hosted in the cloud integrated with one of the cloud provider. This node has the CSI running, and it is managed by the cloud-controller-manager: the Node object automatically gets the information about the zone and region based on the cloud data; if a node gets deleted from the cloud, its corresponding Node object will be deleted in Kubernetes.

      Allowed values: Cloud, Static, Hybrid

    • spec.operatingSystem
      object

      Operating System settings for nodes.

      • spec.operatingSystem.manageKernel
        boolean

        Enable kernel maintenance from bashible.

        Default: true

    • spec.providerType
      string

      Cloud provider type of the node group. None or an empty value — for a static node group.

    • spec.systemType
      string

      How the nodes of the group are managed.

      • GenericOS: Generic OS nodes, configured by bashible on the fly. An unset field means the same.
      • DeckhouseEngine: Deckhouse Engine nodes. The node agent brings the node to the state described in a NodeConfig object, without bashible. The machine must boot from a Deckhouse Engine image.

      DeckhouseEngine is supported for:

      • Static: Machines must be added with d8 node add.
      • Cloud: Machines must be created by the cloud provider through Cluster API (CAPI).
      • Hybrid: Machines must be created by dhctl (CloudPermanent in deckhouse.io/v1).

      DeckhouseEngine is not supported for other Hybrid groups (CloudStatic in deckhouse.io/v1).

      A group can become DeckhouseEngine only while bashible has configured none of its nodes. DeckhouseEngine cannot be removed or changed to GenericOS.

      Allowed values: GenericOS, DeckhouseEngine

  • status
    object
    • status.conditionSummary
      object
      • status.conditionSummary.ready
        string

        Status of the condition summary.

        Allowed values: True, False

      • status.conditionSummary.statusMessage
        string

        Status message about group handling.

    • status.desired
      integer

      Number of desired machines in the group.

    • status.engine
      string

      Machinery used to manage nodes in this group.

      The field value indicates which component manages the nodes:

      • None — nodes in the group are not managed by MCM or CAPI.
      • MCM — nodes in the group are managed by machine-controller-manager.
      • CAPI — nodes in the group are managed by Cluster API.

      Allowed values: None, MCM, CAPI

    • status.error
      string

      Error message about possible problems with the group handling.

    • status.instances
      integer

      Number of instances (in any state) in the group.

    • status.kubernetesVersion
      string

      The current version of kubernetes on the nodes, or the version to which the nodes will be upgraded.

    • status.lastMachineFailures
      array of objects
      • status.lastMachineFailures.lastOperation
        object
        • status.lastMachineFailures.lastOperation.description
          string

          Last operation’s description.

        • status.lastMachineFailures.lastOperation.lastUpdateTime
          string

          Timestamp of last status update for operation.

        • status.lastMachineFailures.lastOperation.state
          string

          Machine’s operation state.

        • status.lastMachineFailures.lastOperation.type
          string

          Type of operation.

      • status.lastMachineFailures.name
        string

        Machine’s name.

      • status.lastMachineFailures.ownerRef
        string

        Machine owner’s name.

      • status.lastMachineFailures.providerID
        string

        Machine’s ProviderID.

    • status.max
      integer

      Maximum amount of instances in the group.

    • status.min
      integer

      Minimal amount of instances in the group.

    • status.nodes
      integer

      Number of Kubernetes nodes (in any state) in the group.

    • status.ready
      integer

      Number of ready Kubernetes nodes in the group.

    • status.standby
      integer

      Number of overprovisioned instances in the group.

    • status.upToDate
      integer

      Number of up-to-date nodes in the group.

Defines the runtime parameters of a node group.

  • metadata
    object
    • metadata.name
      string

      Pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$

      Maximum length: 42

  • spec
    object

    Required value

    • spec.chaos
      object

      Chaos monkey settings.

      Example:

      chaos:
        mode: DrainAndDelete
        period: 24h
      
      • spec.chaos.mode
        string

        The chaos monkey mode:

        • DrainAndDelete — drains and deletes a node when triggered;
        • Disabled — leaves this NodeGroup intact.

        Default: Disabled

        Allowed values: Disabled, DrainAndDelete

      • spec.chaos.period
        string

        The time interval to use for the chaos monkey (can be specified in the Go format).

        Default: 6h

        Pattern: ^[0-9]+[mh]{1}$

    • spec.cloudInstances
      object

      Parameter for provisioning the cloud-based VMs.

      Caution. Can only be used together with nodeType: CloudEphemeral.

      • spec.cloudInstances.classReference
        object

        Required value

        The reference to the InstanceClass object. It is unique for each cloud-provider-* module.

        • spec.cloudInstances.classReference.kind
          string

          The kind of the InstanceClass object (e.g., OpenStackInstanceClass). Supported kinds are defined by the installed cloud provider modules and specified in their documentation.

        • spec.cloudInstances.classReference.name
          string

          The name of the required InstanceClass object (e.g., finland-medium).

      • spec.cloudInstances.maxPerZone
        integer

        Required value

        The maximum number of instances for the group in each zone.

        This value is used as the upper bound in cluster-autoscaler.

        With a value of 0, you need to set capacity for some InstanceClass. Get more details in the description of the necessary InstanceClass.

        Allowed values: 0 <= X

      • spec.cloudInstances.maxSurgePerZone
        integer

        The maximum number of instances the platform creates in each zone above the desired number while it replaces the instances of the group.

        Replacement happens when parameters that require recreating a virtual machine change, for example in the InstanceClass. The parameter does not limit the number of instances that the cluster autoscaler orders when it scales the group up. For details on what NodeGroup parameters affect, see NodeGroup parameters and their result.

        The value 0 is treated as if the parameter is not set, so the default value applies.

        Default: 1

        Allowed values: 0 <= X

      • spec.cloudInstances.maxUnavailablePerZone
        integer

        The maximum number of unavailable instances (during rollout) in the group in each zone.

        Default: 0

        Allowed values: 0 <= X

      • spec.cloudInstances.minPerZone
        integer

        Required value

        The minimum number of instances for the group in each zone.

        This value is used in the MachineDeployment object and as a lower bound in cluster-autoscaler.

        Allowed values: 0 <= X

      • spec.cloudInstances.standby
        integer or string

        The summary number of overprovisioned nodes for this NodeGroup all zones.

        An overprovisioned node is a cluster node on which resources are reserved that are available at any time for scaling. The presence of such a node allows the cluster autoscaler not to wait for node initialization (which may take several minutes), but to immediately place a load on it.

        The value can be an absolute number (for example, 2) or a percentage of desired nodes (for example, 10%). If a percentage is specified, the absolute number is calculated based on the percentage of the maximum number of nodes (the maxPerZone parameter) rounded down, but not less than one.

        Pattern: ^[0-9]+%?$

      • spec.cloudInstances.standbyHolder
        object

        Amount of reserved resources.

        Used to determine whether to order overprovisioned nodes.

        • spec.cloudInstances.standbyHolder.notHeldResources
          object

          Describes the resources that will not be held (consumed) by the standby holder.

          • spec.cloudInstances.standbyHolder.notHeldResources.cpu
            integer or string

            Describes the amount of CPU that will not be held by standby holder on Nodes from this NodeGroup.

            The value can be an absolute number of cpus (for example, 2) as well as a milli representation (for example, 1500m).

            Pattern: ^[0-9]+m?$

          • spec.cloudInstances.standbyHolder.notHeldResources.memory
            integer or string

            Describes the amount of memory that will not be held by standby holder on Nodes from this NodeGroup.

            The value can be an absolute number of bytes (for example, 128974848) as well as a fixed-point number using one of memory suffixes: G, Gi, M, Mi.

            Pattern: ^[0-9]+(\.[0-9]+)?(E|P|T|G|M|K|Ei|Pi|Ti|Gi|Mi|Ki)?$

      • spec.cloudInstances.zones
        array of strings

        List of availability zones to create instances in.

        The default value depends on the cloud provider selected and usually corresponds to all zones of the region being used.

        Example:

        zones:
        - Helsinki
        - Espoo
        - Tampere
        
    • spec.cri
      object

      Container runtime parameters.

      • spec.cri.containerd
        object

        Containerd runtime parameters.

        If used, cri.type must be set to Containerd.

        • spec.cri.containerd.maxConcurrentDownloads
          integer

          Set the max concurrent downloads for each pull.

          Default: 3

      • spec.cri.type
        string

        Container runtime type.

        If not specified, the defaultCRI parameter value of ClusterConfiguration is used.

        Note. Docker support is deprecated.

        Allowed values: Docker, Containerd, NotManaged

    • spec.disruptions
      object

      Disruptions settings for nodes.

      Example:

      disruptions:
        approvalMode: Automatic
        automatic:
          drainBeforeApproval: false
          windows:
          - from: '06:00'
            to: '08:00'
            days:
            - Tue
            - Sun
      
      • spec.disruptions.approvalMode
        string

        The approval mode for disruptive updates:

        • Manual — disable automatic disruption approval; the alert will be displayed if disruption is needed. Caution. The master node group update mode must be Manual to avoid issues with draining.
        • Automatic — automatically approve disruption-involving updates.
        • RollingUpdate — in this mode, a new node with new settings will be created; then, the old node will be deleted. Available only for cloud nodes.

        If the RollingUpdate mode is not used, when updating, the node is first drained and then updated (rebooted) and put back into operation (uncordoned). Note that in this case, the cluster must have sufficient resources to accommodate the load while the node being updated is unavailable. In the RollingUpdate mode, the node is replaced by the updated node, i.e., an extra node appears in the cluster for the duration of the update. In cloud infrastructures, the RollingUpdate mode is convenient, for example, if there are no resources in the cluster to temporarily host the load from the node being updated.

        Default: Automatic

        Allowed values: Manual, Automatic, RollingUpdate

      • spec.disruptions.automatic
        object

        Additional parameters for the Automatic mode.

        • spec.disruptions.automatic.drainBeforeApproval
          boolean

          Drain Pods from the nodes before approving disruption.

          Caution. This setting ignores (nodes will be approved without draining Pods):

          • for the nodeGroup master with a single node;
          • for a single ready node in a nodeGroup picked out for Deckhouse placement.

          Default: true

        • spec.disruptions.automatic.windows
          array of objects

          Time windows for node disruptive updates.

          • spec.disruptions.automatic.windows.days
            array of strings

            Days of the week when node could be updated.

            Examples:

            days: Mon
            
            days: Wed
            
            • Element of the array
              string

              Day of the week.

              Allowed values: Mon, Tue, Wed, Thu, Fri, Sat, Sun

          • spec.disruptions.automatic.windows.from
            string

            Required value

            Start time of disruptive update window (UTC timezone).

            Pattern: ^(?:\d|[01]\d|2[0-3]):[0-5]\d$

            Example:

            from: '13:00'
            
          • spec.disruptions.automatic.windows.to
            string

            Required value

            End time of disruptive update window (UTC timezone).

            Pattern: ^(?:\d|[01]\d|2[0-3]):[0-5]\d$

            Example:

            to: '18:30'
            
      • spec.disruptions.rollingUpdate
        object

        Additional parameters for the RollingUpdate mode.

        • spec.disruptions.rollingUpdate.windows
          array of objects

          Time windows for node disruptive updates.

          • spec.disruptions.rollingUpdate.windows.days
            array of strings

            Days of the week when node could be updated.

            Examples:

            days: Mon
            
            days: Wed
            
            • Element of the array
              string

              Day of the week.

              Allowed values: Mon, Tue, Wed, Thu, Fri, Sat, Sun

          • spec.disruptions.rollingUpdate.windows.from
            string

            Required value

            Start time of disruptive update window (UTC timezone).

            Pattern: ^(?:\d|[01]\d|2[0-3]):[0-5]\d$

            Example:

            from: '13:00'
            
          • spec.disruptions.rollingUpdate.windows.to
            string

            Required value

            End time of disruptive update window (UTC timezone).

            Pattern: ^(?:\d|[01]\d|2[0-3]):[0-5]\d$

            Example:

            to: '18:30'
            
    • spec.docker
      object

      Docker settings for nodes.

      If used, cri.type must be set to Docker.

      Note. the Docker is deprecated.

      • spec.docker.manage
        boolean

        Enable Docker maintenance from bashible.

        Default: true

      • spec.docker.maxConcurrentDownloads
        integer

        Set the max concurrent downloads for each pull.

        Default: 3

    • spec.kubelet
      object

      Kubelet settings for nodes.

      • spec.kubelet.containerLogMaxFiles
        integer

        How many rotated log files to store before deleting them.

        Warning. This parameter does nothing if CRI type is Docker.

        Default: 4

        Allowed values: 2 <= X <= 20

      • spec.kubelet.containerLogMaxSize
        string

        Maximum log file size before it is rotated.

        Warning. This parameter does nothing if CRI type is Docker.

        Default: 50Mi

        Pattern: \d+[Ei|Pi|Ti|Gi|Mi|Ki|E|P|T|G|M|k|m]

      • spec.kubelet.maxPods
        integer

        Set the max count of pods per node for given NodeGroup. When left unspecified, the system automatically determines this limit according to the value of the podSubnetNodeCIDRPrefix parameter of the control-plane-manager module:

        • 120 pods for podSubnetNodeCIDRPrefix ≥ 24;
        • 250 pods for podSubnetNodeCIDRPrefix = 23;
        • 500 pods for podSubnetNodeCIDRPrefix = 22;
        • 1000 pods for podSubnetNodeCIDRPrefix ≤ 21.
      • spec.kubelet.rootDir
        string

        Directory path for managing kubelet files (volume mounts,etc).

        Default: /var/lib/kubelet

      • spec.kubelet.seccompDefault
        boolean

        Use RuntimeDefault seccomp profile for workloads that don’t specify seccomp profile explicitly.

    • spec.kubernetesVersion
      string

      The desired minor version of Kubernetes.

      By default, it corresponds to the version selected for the cluster globally (see installation documentation) or to the current version of the control plane (if the global version is not defined).

      Allowed values: 1.33, 1.34, 1.35, 1.36, 1.37

      Example:

      kubernetesVersion: '1.33'
      
    • spec.nodeTemplate
      object

      Specification of some of the fields that will be maintained in all nodes of the group.

      Example:

      nodeTemplate:
        labels:
          environment: production
          app: warp-drive-ai
        annotations:
          ai.fleet.com/discombobulate: 'true'
        taints:
        - effect: NoExecute
          key: ship-class
          value: frigate
      
      • spec.nodeTemplate.annotations
        object

        Similar to the standard metadata.annotations field.

        Example:

        annotations:
          ai.fleet.com/discombobulate: 'true'
        
      • spec.nodeTemplate.labels
        object

        Similar to the standard metadata.labels field.

        Example:

        labels:
          environment: production
          app: warp-drive-ai
        
      • spec.nodeTemplate.taints
        array of objects

        Similar to the .spec.taints field of the Node object.

        Available fields: effect, key, and values.

        Example:

        taints:
        - effect: NoExecute
          key: ship-class
          value: frigate
        
        • spec.nodeTemplate.taints.effect
          string

          Allowed values: NoSchedule, PreferNoSchedule, NoExecute

        • spec.nodeTemplate.taints.key
          string
        • spec.nodeTemplate.taints.value
          string
    • spec.nodeType
      string

      Required value

      The type of nodes this group provides.

      • Cloud — nodes for this group will be automatically created (and deleted) in the cloud of the specified cloud provider;
      • Static — a static node hosted on a bare metal or virtual machine. The cloud-controller-manager does not manage the node even of one of the cloud providers is enabled;
      • Hybrid — a static node (created manually or using any external tools) hosted in the cloud integrated with one of the cloud provider. This node has the CSI running, and it is managed by the cloud-controller-manager: the Node object automatically gets the information about the zone and region based on the cloud data; if a node gets deleted from the cloud, its corresponding Node object will be deleted in Kubernetes.

      Allowed values: Cloud, Static, Hybrid

    • spec.operatingSystem
      Deprecated
      object

      Operating System settings for nodes.

      • spec.operatingSystem.manageKernel
        Deprecated
        boolean

        Enable kernel maintenance from bashible.

        Default: true

    • spec.providerType
      string

      Cloud provider type of the node group. None or an empty value — for a static node group.

    • spec.static
      object

      Static node parameters

      • spec.static.internalNetworkCIDRs
        array of strings

        Subnet CIDR

    • spec.systemType
      string

      How the nodes of the group are managed.

      • GenericOS: Generic OS nodes, configured by bashible on the fly. An unset field means the same.
      • DeckhouseEngine: Deckhouse Engine nodes. The node agent brings the node to the state described in a NodeConfig object, without bashible. The machine must boot from a Deckhouse Engine image.

      DeckhouseEngine is supported for:

      • Static: Machines must be added with d8 node add.
      • Cloud: Machines must be created by the cloud provider through Cluster API (CAPI).
      • Hybrid: Machines must be created by dhctl (CloudPermanent in deckhouse.io/v1).

      DeckhouseEngine is not supported for other Hybrid groups (CloudStatic in deckhouse.io/v1).

      A group can become DeckhouseEngine only while bashible has configured none of its nodes. DeckhouseEngine cannot be removed or changed to GenericOS.

      Allowed values: GenericOS, DeckhouseEngine

  • status
    object
    • status.conditionSummary
      object
      • status.conditionSummary.ready
        string

        Status of the condition summary.

        Allowed values: True, False

      • status.conditionSummary.statusMessage
        string

        Status message about group handling.

    • status.desired
      integer

      Number of desired machines in the group.

    • status.engine
      string

      Machinery used to manage nodes in this group.

      The field value indicates which component manages the nodes:

      • None — nodes in the group are not managed by Machine Controller Manager or CAPI.
      • MCM — nodes in the group are managed by Machine Controller Manager.
      • CAPI — nodes in the group are managed by Cluster API.

      Allowed values: None, MCM, CAPI

    • status.error
      string

      Error message about possible problems with the group handling.

    • status.instances
      integer

      Number of instances (in any state) in the group.

    • status.kubernetesVersion
      string

      The current version of kubernetes on the nodes, or the version to which the nodes will be upgraded.

    • status.lastMachineFailures
      array of objects
      • status.lastMachineFailures.lastOperation
        object
        • status.lastMachineFailures.lastOperation.description
          string

          Last operation’s description.

        • status.lastMachineFailures.lastOperation.lastUpdateTime
          string

          Timestamp of last status update for operation.

        • status.lastMachineFailures.lastOperation.state
          string

          Machine’s operation state.

        • status.lastMachineFailures.lastOperation.type
          string

          Type of operation.

      • status.lastMachineFailures.name
        string

        Machine’s name.

      • status.lastMachineFailures.ownerRef
        string

        Machine owner’s name.

      • status.lastMachineFailures.providerID
        string

        Machine’s ProviderID.

    • status.max
      integer

      Maximum amount of instances in the group.

    • status.min
      integer

      Minimal amount of instances in the group.

    • status.nodes
      integer

      Number of Kubernetes nodes (in any state) in the group.

    • status.ready
      integer

      Number of ready Kubernetes nodes in the group.

    • status.standby
      integer

      Number of overprovisioned instances in the group.

    • status.upToDate
      integer

      Number of up-to-date nodes in the group.

NodeConfig

Short names: nc

Scope: Cluster
Version: v1alpha1

NodeConfig is the desired state of one node and the node agent’s report on it.

The node agent (nodelet) applies spec.

  • apiVersion
    string

    APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources

  • kind
    string

    Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds

  • metadata
    object
  • spec
    object

    Required value

    Desired state of the node.

    • spec.apiServerEndpoints
      array of strings

      Kubernetes API server URLs with scheme and port, for example https://10.0.0.1:6443.

      • Element of the array
        string

        Pattern: ^https://(\[[0-9A-Fa-f:]+\]|[A-Za-z0-9]([-A-Za-z0-9]*[A-Za-z0-9])?([.][A-Za-z0-9]([-A-Za-z0-9]*[A-Za-z0-9])?)*):(6553[0-5]|655[0-2][0-9]|65[0-4][0-9]{2}|6[0-4][0-9]{3}|[1-5][0-9]{4}|[1-9][0-9]{0,3})/?$

        Maximum length: 253

    • spec.containerRuntime
      object

      ContainerRuntime holds containerd configuration.

      • spec.containerRuntime.maxConcurrentDownloads
        integer

        Maximum number of image layers downloaded in parallel.

        0 means no limit. Defaults to 8 when unset.

        Allowed values: 0 <= X

      • spec.containerRuntime.registryConfigOwner
        string

        Component that writes the containerd registry.d directory.

        • Nodelet: The node agent writes the configuration from spec.registry to the directory and removes everything else in it, except the route directories of spec.registry.upstreams.
        • RegistryAgent: The registry agent static pod owns the directory. The node agent writes nothing there and only removes the directories it wrote before.

        Defaults to Nodelet when unset. Set by node-manager.

        Allowed values: Nodelet, RegistryAgent

      • spec.containerRuntime.sandboxImage
        string

        The pause image used for pod sandboxes.

        If not specified, the pause image imported by the containerd system extension is used.

        Pattern: ^[^[:space:]]+$

        Maximum length: 1024

    • spec.extensions
      array of objects

      Extensions the node installs, each pinned by digest.

      On a Deckhouse Engine node an entry is a signed dm-verity system extension image. On a Generic OS node an entry is a registry package: the node agent runs the package’s own install script as root.

      An entry named nodelet updates the node agent itself.

      Extension is a signed verity sysext built from a release channel, fetched from the registry-packages-proxy by digest. Optional repository selects the proxy’s per-registry config; optional path is its “path” parameter.

      • spec.extensions.digest
        string

        Required value

        Digest is the sha256 digest of the sysext image.

        Pattern: ^sha256:[a-f0-9]{64}$

      • spec.extensions.name
        string

        Required value

        Name is the extension name (also the sysext image basename).

        Pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$

        Length: 1..63

      • spec.extensions.path
        string

        Path to the artifact inside the registry.

        Maximum length: 512

      • spec.extensions.repository
        string

        Repository optionally selects the proxy’s per-registry client config.

        Pattern: ^[^[:space:]]+$

        Maximum length: 1024

      • spec.extensions.requestedBy
        object

        Object that requested the extension.

        Unset for an extension of the platform itself.

        • spec.extensions.requestedBy.kind
          string

          Required value

          Kind of the object.

          Allowed values: NodeExtensionRequest, NodeStaticPodRequest, RegistryUpstream, ModuleSource

        • spec.extensions.requestedBy.name
          string

          Required value

          Name of the object.

          Pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$

          Length: 1..253

    • spec.featureGates
      object

      Node agent features to switch on or off, keyed by feature name.

      A feature missing from this map is on for a Deckhouse Engine node if the node type supports it, and off for a Generic OS node. A Generic OS node runs only the features switched on here.

      The node rejects a document that names an unknown feature or switches on a feature its node type does not support.

      Set by node-controller.

    • spec.internalNetworkCIDRs
      array of strings

      InternalNetworkCIDRs are the subnets the cluster’s own traffic runs on. The node picks the interface holding an address inside one of them and registers with that address; a node with one NIC needs none of this. IPv4 only.

      • Element of the array
        string

        Pattern: ^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(\/(3[0-2]|[1-2][0-9]|[0-9]))$

        Maximum length: 43

    • spec.kernel
      object

      Kernel holds sysctl settings and kernel modules to load.

      • spec.kernel.modules
        array of objects

        Kernel modules to load, in this order.

        Dependencies are not loaded automatically: list each module after the modules it depends on. Do not list a driver built into the kernel: KernelModulesLoaded becomes False.

        A module added to the list is loaded only after the disruption is approved.

        KernelModule is a module to load with optional parameters.

        • spec.kernel.modules.name
          string

          Required value

          Module name, as lsmod lists it. A - and a _ in the name are the same.

          Length: 1..64

        • spec.kernel.modules.params
          array of strings

          Module parameters, passed to insmod in this order, for example debug=1.

          A changed list applies the next time the module loads: a loaded module is not reloaded.

          • Element of the array
            string

            Maximum length: 256

      • spec.kernel.sysctl
        object

        Kernel parameters the node sets at runtime, keyed by their dotted name, for example net.ipv4.ip_forward.

        A key the kernel does not have is skipped. The keys the node agent requires keep its values, and vm.overcommit_memory can only be 1. The size of the connection tracking table, which the node computes from its cores, yields to a value set here.

    • spec.kubelet
      object

      Kubelet holds kubelet configuration parameters.

      • spec.kubelet.bootstrapToken
        string

        BootstrapToken is the bootstrap token used by kubelet to obtain its client certificate on first boot.

        Maximum length: 4096

      • spec.kubelet.clusterCABundle
        string

        PEM bundle of the cluster CA.

        Maximum length: 65536

      • spec.kubelet.clusterDNS
        array of strings

        ClusterDNS is the list of DNS server IP addresses for the cluster.

        • Element of the array
          string

          Maximum length: 45

      • spec.kubelet.clusterDomain
        string

        ClusterDomain is the DNS domain for this cluster (e.g. “cluster.local”).

        Pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$

        Maximum length: 253

      • spec.kubelet.containerLogMaxFiles
        integer

        Number of rotated log files kept per container.

        Defaults to 4 when unset.

        Allowed values: 2 <= X <= 1000

      • spec.kubelet.containerLogMaxSize
        string

        Maximum size of a container log file before it is rotated.

        Defaults to 50Mi when unset.

        Maximum length: 64

      • spec.kubelet.externalCloudProvider
        boolean

        ExternalCloudProvider enables –cloud-provider=external so the cloud-controller-manager manages the node (zone, region, providerID).

      • spec.kubelet.kubernetesVersion
        string

        KubernetesVersion is the cluster’s minor version, e.g. “1.34”. It decides which feature gates kubelet is started with (bashible turns DRA gates on by version); without it DRA workloads cannot run on Deckhouse Engine nodes.

        Pattern: ^[0-9]+\.[0-9]+$

        Maximum length: 16

      • spec.kubelet.maxPods
        integer

        Maximum number of pods on the node.

        Defaults to 120 when unset.

        Allowed values: 1 <= X <= 1000

      • spec.kubelet.nodeLabels
        object

        NodeLabels are labels added to the node object when kubelet registers. A NodeGroup caps neither their number nor their shape, so neither does this.

      • spec.kubelet.registerWithTaints
        array of objects

        RegisterWithTaints is a list of taints to add to the node object when kubelet registers itself. Only takes effect on initial registration.

        Taint represents a Kubernetes taint applied to a node during registration. Key, value and effect take what a NodeGroup takes: bare strings and an optional effect, so a taint copied from one still fits.

        • spec.kubelet.registerWithTaints.effect
          string

          Taint effect. A taint without an effect is skipped: kubelet registers a taint only with one of these.

          Allowed values: NoSchedule, PreferNoSchedule, NoExecute

        • spec.kubelet.registerWithTaints.key
          string

          Required value

          Taint key. A taint kubelet cannot register, such as one whose key is not a qualified name, is skipped, and the other taints are registered.

        • spec.kubelet.registerWithTaints.value
          string

          Taint value.

      • spec.kubelet.resourceReservation
        object

        ResourceReservation controls how much CPU, memory and disk are held back from pods for the system itself (systemReserved).

        • spec.kubelet.resourceReservation.mode
          string

          Reservation policy.

          • Auto: Computed from the node’s capacity.
          • Off: Nothing is reserved.

          Defaults to Auto when unset.

          Allowed values: Auto, Off

      • spec.kubelet.serverTLSBootstrap
        boolean

        ServerTLSBootstrap makes kubelet request a serving certificate from the cluster instead of signing its own.

        Defaults to true when unset.

    • spec.network
      object

      Network holds hostname, DNS, NTP, interfaces and routes.

      • spec.network.dns
        object

        DNS resolver configuration.

        • array of strings

          Search domains for host names that are not fully qualified, tried in this order.

          • Element of the array
            string

            Maximum length: 253

        • spec.network.dns.servers
          array of strings

          Name server IP addresses.

          • Element of the array
            string

            Maximum length: 45

      • spec.network.hostname
        string

        Host name of the node.

        Maximum length: 253

      • spec.network.interfaces
        array of objects

        Network interfaces, keyed by name.

        NetworkInterface describes a single NIC.

        • spec.network.interfaces.addresses
          array of strings

          Static addresses in CIDR notation, used when dhcp is false.

          • Element of the array
            string

            Maximum length: 43

        • spec.network.interfaces.cluster
          boolean

          Cluster names this NIC the one the cluster’s traffic runs on, for a machine whose addressing spec.internalNetworkCIDRs cannot tell apart.

        • spec.network.interfaces.dhcp
          boolean

          Required value

          DHCP enables DHCPv4 on the interface.

        • spec.network.interfaces.gateway
          string

          Default gateway IP address.

          Maximum length: 45

        • spec.network.interfaces.name
          string

          Required value

          Interface name, for example eth0.

          Length: 1..253

      • spec.network.ntp
        object

        NTP time-sync configuration.

        • spec.network.ntp.servers
          array of strings

          NTP servers, each a host name or an IPv4 address. On Immutable nodes the time-sync feature checks them once per boot, before kubelet starts, and waits at most 30 seconds for a usable answer. The deckhouse chrony module keeps the clock in time after that.

          • Element of the array
            string

            Maximum length: 253

      • spec.network.routes
        array of objects

        Static routes, added on every interface listed in interfaces.

        Route is a static route.

        • spec.network.routes.gateway
          string

          Gateway IP address.

          Maximum length: 45

        • spec.network.routes.name
          string

          Label for the route. It does not change how the route is applied.

          Maximum length: 253

        • spec.network.routes.networks
          array of strings

          Destination networks in CIDR notation.

          • Element of the array
            string

            Maximum length: 43

    • spec.nodeName
      string

      Required value

      Name of the Kubernetes node this configuration applies to.

      Equals metadata.name and cannot be changed.

      Pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$

      Length: 1..253

    • spec.osImage
      object

      Operating system image to use on the node.

      To determine whether an update is required, the node compares the image digest with the value saved to /config/os-digest during installation. The digest identifies a specific image publication and may change when the same content is republished.

      Required when systemType is DeckhouseEngine or its former name Immutable.

      • spec.osImage.digest
        string

        Required value

        Operating system image digest.

        Identifies a specific image and is used by the node to determine whether an update is required. During installation, the node saves the digest and compares it with the specified value during subsequent checks.

        Pattern: ^sha256:[a-f0-9]{64}$

      • spec.osImage.path
        string

        Path to the image inside the registry.

        Maximum length: 512

      • spec.osImage.repository
        string

        Address of the container registry from which the operating system image is downloaded.

        The value is passed to registry-packages-proxy as the repository parameter.

        If the parameter is not specified, repository is not passed and registry-packages-proxy uses its default container registry.

        Pattern: ^[^[:space:]]+$

        Maximum length: 1024

      • spec.osImage.rootHash
        string

        Dm-verity root hash of the root filesystem inside the image, in ASCII hexadecimal format.

        Used to determine whether the actual image content has changed. Unlike digest, which may change when an image is republished, rootHash remains the same if the root filesystem content has not changed.

        Deckhouse Platform obtains the value from an image label. If the value cannot be obtained from the label, the node determines the root hash after downloading the image.

        Pattern: ^[a-f0-9]{64}$

    • spec.registry
      object

      Registry is the container registry the node talks to directly, without the registry-packages-proxy. A node bootstrapping a control plane has no proxy yet but must pull images and sysexts. upstreams lists the additional registries containerd routes pulls to.

      On a Generic OS node bashible configures containerd for the registry itself, and the node agent writes only the upstreams routes, with the registry-routes feature on. It reads only address there, which no route may take over, and never downloads from the registry directly.

      • spec.registry.address
        string

        Required value

        Address is the registry host, optionally with a port, e.g. “registry.deckhouse.io” or “registry.example.com:5000”.

        Pattern: ^[^[:space:]]+$

        Length: 1..1024

      • spec.registry.auth
        string

        Auth is the base64-encoded “user:password” pair, as it appears in the “auth” field of a docker config.

        Maximum length: 4096

      • spec.registry.caBundle
        string

        PEM bundle of the CA certificates the registry is verified with.

        Maximum length: 65536

      • spec.registry.path
        string

        Path is the repository path within the registry, e.g. “/deckhouse/ce”.

        Maximum length: 1024

      • spec.registry.scheme
        string

        Protocol the node uses to connect to the registry.

        Defaults to HTTPS when unset.

        Allowed values: HTTPS, HTTP

      • spec.registry.upstreams
        array of objects

        Additional registries the node routes pulls to: each entry sends the pulls of one registry host to another registry and its mirrors.

        The node writes one containerd host directory per entry, named after match in lower case, and removes the directories of entries that are gone. A route change does not restart containerd. An entry the node cannot apply is skipped and reported in the RegistryRoutesReady condition; the other entries are still applied.

        The registry-routes feature applies this list. It is on by default on a Deckhouse Engine node; a Generic OS node applies the list only when featureGates switch the feature on. With the feature off, the node leaves the directories it wrote before as they are.

        Set by node-controller from the accepted RegistryUpstream objects and the CAs of ModuleSource objects. On a Deckhouse Engine node the list is empty while the registry module’s agent configures containerd. On a Generic OS node the list is always set but has no ModuleSource entry, since bashible writes those CAs itself, and the node writes no route while the agent’s _default directory is listed in bashible’s deckhouse_hosts_state.json.

        RegistryUpstream is one route of spec.registry.upstreams.

        • spec.registry.upstreams.match
          string

          Required value

          Registry host, optionally with a port, whose pulls the route takes over. The name of the containerd host directory.

          Pattern: ^[a-zA-Z0-9]([-a-zA-Z0-9]*[a-zA-Z0-9])?(\.[a-zA-Z0-9]([-a-zA-Z0-9]*[a-zA-Z0-9])?)*(:[0-9]{1,5})?$

          Length: 1..255

        • spec.registry.upstreams.requestedBy
          object

          Required value

          Object the route comes from: a RegistryUpstream, or a ModuleSource whose registry needs its own CA.

          • spec.registry.upstreams.requestedBy.kind
            string

            Required value

            Kind of the object.

            Allowed values: NodeExtensionRequest, NodeStaticPodRequest, RegistryUpstream, ModuleSource

          • spec.registry.upstreams.requestedBy.name
            string

            Required value

            Name of the object.

            Pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$

            Length: 1..253

        • spec.registry.upstreams.upstream
          object

          Required value

          Registry the pulls are sent to, and its mirrors.

          • spec.registry.upstreams.upstream.auth
            object

            Credentials for the registry.

            • spec.registry.upstreams.upstream.auth.auth
              string

              Base64-encoded username:password pair. Takes precedence over username and password when set.

              Maximum length: 4096

            • spec.registry.upstreams.upstream.auth.password
              string

              Password for basic authentication.

              Maximum length: 4096

            • spec.registry.upstreams.upstream.auth.username
              string

              Username for basic authentication.

              Maximum length: 1024

          • spec.registry.upstreams.upstream.ca
            string

            PEM bundle of the CA certificates the registry is verified with over HTTPS. The node skips the route unless the bundle is only CERTIFICATE blocks that all parse.

            Maximum length: 65536

          • spec.registry.upstreams.upstream.host
            string

            Required value

            Registry host, optionally with a port.

            Length: 1..255

          • spec.registry.upstreams.upstream.mirrors
            array of objects

            Additional addresses serving the same content, tried after the registry in the listed order.

            RegistryEndpoint is one address of a route.

            • spec.registry.upstreams.upstream.mirrors.auth
              object

              Credentials for the registry.

              • spec.registry.upstreams.upstream.mirrors.auth.auth
                string

                Base64-encoded username:password pair. Takes precedence over username and password when set.

                Maximum length: 4096

              • spec.registry.upstreams.upstream.mirrors.auth.password
                string

                Password for basic authentication.

                Maximum length: 4096

              • spec.registry.upstreams.upstream.mirrors.auth.username
                string

                Username for basic authentication.

                Maximum length: 1024

            • spec.registry.upstreams.upstream.mirrors.ca
              string

              PEM bundle of the CA certificates the registry is verified with over HTTPS. The node skips the route unless the bundle is only CERTIFICATE blocks that all parse.

              Maximum length: 65536

            • spec.registry.upstreams.upstream.mirrors.host
              string

              Required value

              Registry host, optionally with a port.

              Length: 1..255

            • spec.registry.upstreams.upstream.mirrors.path
              string

              Repository prefix inside the registry, for example /virtualization. The node prepends it to every repository it pulls through this address.

              Maximum length: 1024

            • spec.registry.upstreams.upstream.mirrors.scheme
              string

              Protocol the node uses to connect to the registry.

              Defaults to HTTPS when unset.

              Allowed values: HTTP, HTTPS

          • spec.registry.upstreams.upstream.path
            string

            Repository prefix inside the registry, for example /virtualization. The node prepends it to every repository it pulls through this address.

            Maximum length: 1024

          • spec.registry.upstreams.upstream.scheme
            string

            Protocol the node uses to connect to the registry.

            Defaults to HTTPS when unset.

            Allowed values: HTTP, HTTPS

    • spec.registryPackagesProxy
      object

      How the node reaches registry-packages-proxy, which serves it extensions and operating system images.

      • spec.registryPackagesProxy.accessToken
        string

        Bearer token the node presents to the proxy.

        Base64-encoded in the object.

        Maximum length: 4096

      • spec.registryPackagesProxy.endpoints
        array of strings

        registry-packages-proxy URLs with scheme and port.

        When unset, the node uses port 4219 of the apiServerEndpoints hosts. Set only in the initial configuration of the first master node.

        • Element of the array
          string

          Pattern: ^https?://(\[[0-9A-Fa-f:]+\]|[A-Za-z0-9]([-A-Za-z0-9]*[A-Za-z0-9])?([.][A-Za-z0-9]([-A-Za-z0-9]*[A-Za-z0-9])?)*):(6553[0-5]|655[0-2][0-9]|65[0-4][0-9]{2}|6[0-4][0-9]{3}|[1-5][0-9]{4}|[1-9][0-9]{0,3})/?$

          Maximum length: 253

    • spec.staticPods
      array of objects

      Manifests of static pods that kubelet runs without using the Kubernetes scheduler.

      The node writes the manifests to /etc/kubernetes/manifests. Static pods must obtain any other required resources on their own. For example, a pod can issue its own certificates, use the node’s kubeconfig, and mount directories for persistent state using hostPath with the DirectoryOrCreate type.

      Control plane component manifests created by the node agent are not included in spec.staticPods and are not modified when this parameter is processed.

      Static pod.

      • spec.staticPods.manifest
        string

        Required value

        Complete pod manifest.

        When the manifest is written to disk, the $MY_IP substring is replaced with the node’s IP address, similarly to control plane component manifests.

        Length: 1..32768

      • spec.staticPods.name
        string

        Required value

        Manifest file name without an extension.

        Used as the list item key and is independent of metadata.name of the pod in the manifest. Kubelet determines the mirror pod name from the manifest rather than the file name.

        Pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$

        Length: 1..63

      • spec.staticPods.requestedBy
        object

        Object that requested the static pod.

        Unset for a static pod of the platform itself.

        • spec.staticPods.requestedBy.kind
          string

          Required value

          Kind of the object.

          Allowed values: NodeExtensionRequest, NodeStaticPodRequest, RegistryUpstream, ModuleSource

        • spec.staticPods.requestedBy.name
          string

          Required value

          Name of the object.

          Pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$

          Length: 1..253

    • spec.statusToken
      string

      StatusToken is the bearer presented to the node’s :50000 status port. Whoever writes the document mints it (dhctl, or the NodeConfigTemplate render for a hand-installed machine); the cluster carries it over untouched, having no way to tell the node a new one.

      Maximum length: 4096

    • spec.storage
      object

      Disk the operating system is installed on, and additional filesystems.

      The installation disk cannot be changed once set.

      • spec.storage.device
        string

        Device is the whole-disk block device to install onto, e.g. “/dev/sda”, “/dev/nvme0n1”, or a stable “/dev/disk/by-id/…” path. Ignored when diskSelector is set.

        Pattern: ^/dev/[A-Za-z0-9._/-]+$

        Maximum length: 1024

      • spec.storage.diskSelector
        object

        Selects the installation disk by attributes instead of a fixed path.

        All specified fields must match.

        • spec.storage.diskSelector.busPath
          string

          BusPath matches the hardware bus path (glob).

          Maximum length: 253

        • spec.storage.diskSelector.model
          string

          Model matches the device model (glob), e.g. “Samsung*”.

          Maximum length: 253

        • spec.storage.diskSelector.name
          string

          Name matches the kernel device name (glob), e.g. “nvme0n1”.

          Maximum length: 253

        • spec.storage.diskSelector.rotational
          boolean

          Rotational matches spinning (true) vs solid-state (false) disks.

        • spec.storage.diskSelector.serial
          string

          Serial matches the disk serial number (glob).

          Maximum length: 253

        • spec.storage.diskSelector.size
          string

          Size matches the disk capacity, optionally with a comparison operator, e.g. “>=100Gi”, “>1Ti”, “512Gi”.

          Maximum length: 253

        • spec.storage.diskSelector.type
          string

          Type matches the disk kind.

          Allowed values: SSD, HDD, NVMe, SD

        • spec.storage.diskSelector.wwid
          string

          WWID matches the World Wide Identifier (glob).

          Maximum length: 253

      • spec.storage.mounts
        array of objects

        Mounts are additional filesystems the node makes available, at /mnt/ or wherever bindTo names. Nothing here is partitioned: only existing partitions and blank whole disks are formatted (when empty) and mounted.

        Mount is one additional filesystem, at /mnt/ unless bindTo says otherwise. Exactly one of device or partitionSelector names the partition; an empty one is formatted (label = name), an existing filesystem mounted as is.

        • spec.storage.mounts.bindTo
          string

          BindTo mounts the filesystem at this path instead of /mnt/, for a directory something else knows by name (/var/lib/etcd). The directory must be empty: mounting over files hides them, and hidden etcd data is lost data.

          Pattern: ^/[A-Za-z0-9._/-]+$

          Maximum length: 1024

        • spec.storage.mounts.device
          string

          Device is the partition to use, e.g. “/dev/sdb1” or a stable “/dev/disk/by-id/…-part1” path.

          Pattern: ^/dev/[A-Za-z0-9._/-]+$

          Maximum length: 1024

        • spec.storage.mounts.filesystem
          string

          Filesystem to create on an empty partition.

          Defaults to Ext4 when unset.

          Allowed values: Ext4

        • spec.storage.mounts.mode
          string

          Mode is the mode of the filesystem root after mounting, as an octal string, e.g. “0700”. Left alone when unset. A freshly made ext4 has its root at 0755, which is a mode etcd refuses to start on.

          Pattern: ^0[0-7]{3}$

        • spec.storage.mounts.name
          string

          Required value

          Name identifies the mount, and is both the mount point (/mnt/) and the filesystem label written when this node formats the partition. Capped at 16 characters because that is the size of the ext4 volume label field.

          Pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$

          Length: 1..16

        • spec.storage.mounts.partitionSelector
          object

          PartitionSelector picks the partition by attributes instead of a fixed path.

          • spec.storage.mounts.partitionSelector.blank
            boolean

            Blank makes whole disks selectable, and only ones that carry nothing: no partition table, no filesystem — a cloud disk never touched. Without it a selector sees partitions only; a whole disk is where somebody’s layout lives.

          • spec.storage.mounts.partitionSelector.fsType
            string

            FSType matches the type of the filesystem already present (glob).

            Maximum length: 253

          • spec.storage.mounts.partitionSelector.label
            string

            Label matches the filesystem label (glob).

            Maximum length: 253

          • spec.storage.mounts.partitionSelector.name
            string

            Name matches the kernel device name (glob), e.g. “sdb1” or “nvme0n1p*”.

            Maximum length: 253

          • spec.storage.mounts.partitionSelector.partLabel
            string

            PartLabel matches the GPT partition name (glob).

            Maximum length: 253

          • spec.storage.mounts.partitionSelector.partUUID
            string

            PartUUID matches the GPT partition UUID exactly, ignoring case.

            Maximum length: 253

          • spec.storage.mounts.partitionSelector.size
            string

            Size matches the size, optionally with a comparison operator, e.g. “>=100Gi”, “>1Ti”, “512Gi”. Without an operator the comparison is “>=”; “=” allows 1%, since a disk rarely reports an exact round size.

            Maximum length: 253

          • spec.storage.mounts.partitionSelector.uuid
            string

            UUID matches the filesystem UUID exactly, ignoring case.

            Maximum length: 253

      • spec.storage.wipe
        boolean

        Wipe controls whether an already-provisioned disk is wiped and re-partitioned. Default false: only an unprovisioned (or non-matching) disk is set up, so an existing layout stays and a reboot never destroys data.

    • spec.systemType
      string

      Required value

      How the node is managed.

      • DeckhouseEngine: a Deckhouse Engine node. The node agent configures the whole node.
      • GenericOS: a node configured by bashible. The node agent manages only what featureGates turn on.

      The former names Immutable (for DeckhouseEngine) and Mutable (for GenericOS) are deprecated and accepted during the transition period.

      The node type cannot be changed. Replacing a former name with the new name of the same type, or back, is allowed.

      Allowed values: Immutable, Mutable, DeckhouseEngine, GenericOS

    • spec.updatePolicy
      object

      UpdatePolicy controls how and when the node is updated.

      • spec.updatePolicy.mode
        string

        Update mode.

        The node checks the value and does not act on it: in either mode it updates within windows.

        Allowed values: Manual, Automatic

      • spec.updatePolicy.windows
        array of objects

        Time windows in UTC when the node may download and apply updates.

        An update is allowed if any window matches. Without windows, updates are allowed at any time.

        One update window, in UTC.

        • spec.updatePolicy.windows.days
          array of strings

          Days of the week the window applies to, as calendar days in UTC.

          A time matches when its own day is listed, also in the part of a window that runs past midnight. When empty, the window applies every day.

          • Element of the array
            string

            Allowed values: Mon, Tue, Wed, Thu, Fri, Sat, Sun

        • spec.updatePolicy.windows.from
          string

          Required value

          Start of the window, H:MM or HH:MM (24-hour, UTC).

          Pattern: ^(?:\d|[01]\d|2[0-3]):[0-5]\d$

        • spec.updatePolicy.windows.to
          string

          Required value

          End of the window, same format as from.

          A window whose end is before its start runs over midnight. The part after midnight belongs to its own calendar day: with days: [Sat], a window from 22:00 to 04:00 allows Saturday 00:00 to 04:00 and Saturday 22:00 to midnight, and not Sunday 01:00.

          Pattern: ^(?:\d|[01]\d|2[0-3]):[0-5]\d$

  • status
    object

    Status is reported by the on-node agent after each reconcile pass.

    • status.agent
      object

      Node agent build that wrote this status.

      • status.agent.version
        string

        Build version of the node agent.

    • status.appliedGeneration
      integer

      AppliedGeneration is the spec generation the node has applied.

      The node has applied the current spec when AppliedGeneration equals metadata.generation and ConfigurationApplied is True at that generation. The value stays behind while a disruptive change waits for approval, and while a new agent build installed for this generation has not applied it yet.

    • status.conditions
      array of objects

      Conditions of the node.

      A False condition whose reason ends with Pending is a wait. Any other False reason and Unknown mean a failure. AddressResolved does not affect phase.

      In Ready, ConfigurationApplied and DisruptionApproved, observedGeneration is the generation the node has processed. In the other conditions, it is the generation the node runs.

      Condition types:

      • Ready: True with reason ReconcileSucceeded when phase is Ready. Otherwise False, with the phase as the reason. The message names the condition that determined the phase.
      • ConfigurationApplied: True when the node runs the published configuration.

        With True, the reason is ReconcileSucceeded, or UnknownFields when the object has fields unknown to this agent build. The message lists them.

        Reasons with False that mean a wait:

        • DisruptionApprovalPending
        • Pending
        • RestartPending

        Reasons with False that mean a failure:

        • Rejected
        • RolledBackToLastKnownGood
        • RollbackFailed
        • PartiallyApplied
        • FieldsNotApplicable
      • DisruptionApproved: Whether the interruption that the configuration of the node needs is approved.

        Reasons:

        • ApprovalPending (False): A configuration that interrupts the node waits for approval.
        • AttemptsExhausted (False): The interruptions node-manager attempted failed, and it makes no new attempt for now.
        • Approved (True): The interruption is approved.
        • NoDisruptionPending (True): Nothing waits for approval.
      • AddressResolved: Whether the node has determined the address it registers with.

        Reasons with False:

        • NoneInCIDR, Ambiguous: The node cannot choose an address.
        • AddressDrift: The node keeps the address it registered with, but that address is no longer on the machine or the configuration now selects another one.
      • Subsystem conditions, one per subsystem the pass ran or skipped:

        • BootTimeSyncConfigured
        • APIEndpointsReachable
        • SysctlApplied
        • KernelModulesLoaded
        • MountsReady
        • ExtensionsReady
        • ContainerdReady
        • KubeletReady
        • StaticPodsReady
        • RegistryRoutesReady
        • UnitsRunning
        • OSImageUpToDate
        • ControlPlaneBootstrapped

        Values of a subsystem condition:

        • True: The reason is ReconcileSucceeded, or a specific reason of the subsystem listed below.
        • False: The message gives the cause. The reason is Pending while the subsystem waits for something outside the node, otherwise ReconcileFailed or a more specific reason of the subsystem listed below.
        • Unknown: The subsystem did not run in this pass. The reason is ReconcileSkipped.

        Specific reasons with True:

        • OwnedByRegistryAgent (RegistryRoutesReady): The registry agent configures containerd’s registry hosts on this Generic OS node. The node writes no route and has removed the ones it wrote.

        Specific reasons with False that mean a failure:

        • StaticPodFailed (StaticPodsReady): A static pod manifest is not written or not removed. The staticPods entry gives the cause.
        • OSImageRolledBack (OSImageUpToDate): The node rolled back from the image or refused it before. The image is not tried again.
        • OSImageRejected (OSImageUpToDate): The kernel refused the root hash signature of the image. The image is not tried again.
        • OSImageRootHashMismatch (OSImageUpToDate): The image does not carry the root hash the configuration declares. The image is not tried again.
        • OSImageDownloadFailed (OSImageUpToDate): The node could not download the image. The node retries.
        • OSImageStaged (OSImageUpToDate): The image is staged for the next boot, but the node could not request the reboot.
        • OSImageOnTrial (OSImageUpToDate): The node did not confirm the new image in time and could not request the reboot that tries it again.

        Waits of OSImageUpToDate are published with the reason Pending, and the message tells them apart:

        • OSImageBaselineUnknown: The node has no record of the image it runs and cannot update it until it is reinstalled.
        • OSImageUpdateDeferred: The update waits for the update window.
        • OSImageStaged, OSImageOnTrial: The node reboots into the new image or waits for its subsystems before it confirms the image.

      Condition contains details for one aspect of the current state of this API Resource.

      • status.conditions.lastTransitionTime
        string

        Required value

        lastTransitionTime is the last time the condition transitioned from one status to another. This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable.

      • status.conditions.message
        string

        Required value

        message is a human readable message indicating details about the transition. This may be an empty string.

        Maximum length: 32768

      • status.conditions.observedGeneration
        integer

        observedGeneration represents the .metadata.generation that the condition was set based upon. For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date with respect to the current state of the instance.

        Allowed values: 0 <= X

      • status.conditions.reason
        string

        Required value

        reason contains a programmatic identifier indicating the reason for the condition’s last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. The value should be a CamelCase string. This field may not be empty.

        Pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$

        Length: 1..1024

      • status.conditions.status
        string

        Required value

        status of the condition, one of True, False, Unknown.

        Allowed values: True, False, Unknown

      • status.conditions.type
        string

        Required value

        type of condition in CamelCase or in foo.example.com/CamelCase.

        Pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$

        Maximum length: 316

    • status.extensions
      array of objects

      Extensions is the outcome of each configured system extension, one entry per extension. Republished on every pass, empty when the pass checked nothing — never “the previous outcome still holds”.

      ExtensionStatus is the reconcile outcome of one system extension.

      • status.extensions.digest
        string

        Digest depends on State.

        • Ready: the digest installed on the node.
        • Pending: the digest being installed. For the agent’s own entry, the digest installed and waiting for the new build to start.
        • Failed: the digest the node refused or could not install.
      • status.extensions.message
        string

        Message carries the cause when State is Failed.

      • status.extensions.name
        string

        Required value

        Name is the extension name: a spec.extensions[].name, or the name of an extension removed from spec whose uninstall failed on a Generic OS node.

      • status.extensions.state
        string

        Required value

        State of the extension.

        • Ready: the extension is installed (on a Deckhouse Engine node, also merged).
        • Pending: the extension is being fetched or waits for the update window, or the agent restarts with a new build of itself (reason RestartPending).
        • Failed: the cause is in Message.

        Allowed values: Ready, Pending, Failed

    • status.features
      array of strings

      Features that ran on the node during its last reconciliation, sorted by name: the gates switched on in spec.featureGates plus the defaults of the node type.

      On a Deckhouse Engine node spec.featureGates is usually empty because unset gates are on there. This list shows what actually runs on the node.

    • status.lastReconcileTime
      string

      LastReconcileTime is when the node last finished a reconcile pass — the only thing in the status that ages, so it is what tells a dead agent from a healthy one. Republished coarsely: judge staleness in tens of minutes.

    • status.maintenanceToken
      string

      MaintenanceToken is the bearer token an operator presents to the node’s :50000 config-push port once the node has lost the API. Root-equivalent, so the apiserver keeps it out of every answer without the sensitive subresource, which nodes are not granted.

    • status.network
      object

      Network is what the node resolved for itself: the interface it took its cluster address from, and that address. Written by the node alone.

      • status.network.address
        string

        Address is the address kubelet registered the node with.

      • status.network.clusterInterface
        string

        ClusterInterface is the NIC the address was taken from.

    • status.observedGeneration
      integer

      ObservedGeneration is the latest spec generation the node has processed: it reaches the newest generation as soon as the node has looked at it, even while it is still held for approval (see AppliedGeneration).

    • status.osImage
      object

      Status of the operating system image on the node. Filled in by the node.

      • status.osImage.attemptsLeft
        integer

        Number of remaining boot attempts with the image being verified before initramfs automatically rolls back to the previous image.

      • status.osImage.digest
        string

        Digest of the operating system image currently used by the node. The value is read from the node configuration partition.

        May be absent on nodes installed before the digest persistence mechanism was introduced.

      • status.osImage.failedDigest
        string

        Digest of an image that caused the node to roll back after booting. No further attempts are made to boot this image.

        Because a digest uniquely identifies immutable image content, a fixed version of the image has a different digest.

      • status.osImage.rootHash
        string

        Dm-verity root hash of the image currently used by the node. The value is read from the active A/B slot.

        Used to identify the actual root filesystem content independently of the image digest. For example, the digests of the desired and currently used images may differ after republishing, while identical rootHash values indicate that the root filesystem content has not changed.

      • status.osImage.slot
        string

        A/B slot (a or b) containing the image currently used by the node.

      • status.osImage.trialDigest
        string

        Digest of the image prepared for the next boot or being verified during the current boot.

        Set only during an update, while digest still contains the digest of the previous image.

    • status.phase
      string

      Summary of the node.

      • Ready: The node runs the published configuration and every subsystem is healthy.
      • Pending: The node is healthy but does not run the published configuration yet.
      • Degraded: At least one condition other than AddressResolved reports a failure.

      Allowed values: Ready, Pending, Degraded

    • status.staticPods
      array of objects

      Results of processing static pods. A separate entry is created for each pod.

      Value is regenerated on each reconciliation and remains empty if no static pods were checked during the reconciliation.

      Processing result for a single static pod.

      • status.staticPods.manifestDigest
        string

        sha256 of the manifest as published in spec, in the form sha256:<hex>.

        Set when state is Written.

      • status.staticPods.message
        string

        Detailed information about the error when state is Failed.

      • status.staticPods.name
        string

        Required value

        Static pod name corresponding to spec.staticPods[].name.

      • status.staticPods.reason
        string

        Reason for the Failed state.

        Possible values:

        • ManifestRejected: Node rejected the manifest. The manifest in the resource must be corrected.
        • WriteFailed: Manifest file could not be written to the node. Check the node state.
        • RemoveFailed: Static pod was removed from spec, but the corresponding manifest file could not be removed from the node. The pod continues to run.

        Allowed values: ManifestRejected, WriteFailed, RemoveFailed

      • status.staticPods.state
        string

        Required value

        Manifest processing state.

        Possible values:

        • Written: File on disk matches the manifest from spec.
        • Failed: File does not match the manifest from spec, or the required operation could not be completed. The reason is specified in reason.

        Allowed values: Written, Failed

    • status.units
      array of objects

      Units is the outcome of each managed systemd unit (containerd, kubelet and every unit an extension ships), one entry per unit. Republished on every pass like Extensions, and empty when the pass checked nothing.

      UnitStatus is the reconcile outcome of one managed systemd unit.

      • status.units.message
        string

        Message carries the cause when State is Failed.

      • status.units.name
        string

        Required value

        Name is the systemd unit name (e.g. containerd.service).

      • status.units.state
        string

        Required value

        State is Active when the unit is running, Pending when it is queued to be started later this pass, or Failed with the cause in Message.

        Allowed values: Active, Pending, Failed

NodeExtensionRequest

Short names: ner

Scope: Cluster
Version: v1alpha1

Request to install an extension on the selected nodes: a system extension (a sysext image, optionally with kernel modules) or a registry package.

The requester sets the kind in spec.extension.kind. A registry package (Package) is installed on the selected nodes that bashible configures. A system extension (SystemExtension) is installed on the selected Deckhouse Engine nodes. The kind is never inferred from the image. If none of the selected groups installs extensions of this kind, the Accepted condition reports the NotDeliverable reason.

The image is addressed the same way as any package in the registry-packages-proxy: a name, a digest, and an optional repository and path. node-controller copies these fields unchanged into spec.extensions of the NodeConfig, and the node agent installs the extension. node-controller does not look up the image by module or ModuleSource.

Example:

# A system extension with kernel modules, fetched from the registry of a ModuleSource.
apiVersion: deckhouse.io/v1alpha1
kind: NodeExtensionRequest
metadata:
  name: drbd
spec:
  extension:
    name: drbd
    digest: sha256:4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945
    kind: SystemExtension
    repository: registry.example.com/deckhouse/modules
    path: drbd/extensions
    kernelModules:
      - name: drbd
        params:
          - usermode_helper=disabled
      - name: drbd_transport_tcp
  nodeGroupSelector:
    matchNames:
      - worker
---
# A registry package from the cluster's main registry, on the nodes of every
# group bashible configures that carry the label.
apiVersion: deckhouse.io/v1alpha1
kind: NodeExtensionRequest
metadata:
  name: nvme-cli
spec:
  extension:
    name: nvme-cli
    digest: sha256:9b2a6f2bb7e3c5f6d1a0e4c8b7d6e5f4a3b2c1d0e9f8a7b6c5d4e3f2a1b0c9d8
    kind: Package
  nodeSelector:
    matchLabels:
      storage.example.com/nvme: "true"
  • apiVersion
    string
  • kind
    string
  • metadata
    object
  • spec
    object

    Required value

    • spec.extension
      object

      Required value

      Image to install and its kind. The image is addressed the same way as any package in the registry-packages-proxy.

      • spec.extension.digest
        string

        Required value

        Image manifest digest.

        Pattern: ^sha256:[a-f0-9]{64}$

      • spec.extension.kernelModules
        array of objects

        Kernel modules the node agent loads after merging the system extension, in the listed order. Allowed only with the SystemExtension kind. Each module is listed once.

        Dependencies are not loaded automatically: list every module the extension needs, each one after the modules it depends on.

        Adding a module interrupts each selected node: the node waits for an approved Disruption NodeOperation.

        • spec.extension.kernelModules.name
          string

          Required value

          Kernel module name.

          Pattern: ^[a-zA-Z0-9][a-zA-Z0-9_-]*$

          Length: 1..64

        • spec.extension.kernelModules.params
          array of strings

          Module parameters, passed as key=value strings.

          • Element of the array
            string

            Maximum length: 256

      • spec.extension.kind
        string

        Required value

        Image kind.

        • Package: a registry package. The node agent installs it on the selected nodes that bashible configures.
        • SystemExtension: a system extension. The node agent merges it on the selected Deckhouse Engine nodes.

        Nodes of groups of the other type do not receive the image. If none of the selected groups installs extensions of this kind, the Accepted condition reports the NotDeliverable reason.

        Cannot be changed after creation. To change the kind, delete the request and create it again.

        Allowed values: Package, SystemExtension

      • spec.extension.name
        string

        Required value

        Extension name.

        The image of a system extension must contain the extension-release file for this name.

        The names containerd, kubelet, kubernetes-cni, nodelet and qemu-guest-agent belong to platform extensions and are not accepted.

        Pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$

        Length: 1..63

      • spec.extension.path
        string

        The image’s path within the repository (the proxy’s --rpp-path). Left empty, the repository root is used.

        Maximum length: 512

      • spec.extension.repository
        string

        The key the registry-packages-proxy resolves credentials by (its --rpp-repository), for example a ModuleSource’s spec.registry.repo. Left empty, the image is pulled from the cluster’s main registry.

        Pattern: ^[^[:space:]]+$

        Maximum length: 1024

    • spec.nodeGroupSelector
      object

      Narrows the extension to nodes of the named NodeGroups.

      • spec.nodeGroupSelector.matchNames
        array of strings

        The set of NodeGroup names the extension applies to.

        • Element of the array
          string

          Pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$

          Maximum length: 63

    • spec.nodeSelector
      object

      Narrows the extension to nodes carrying the given labels.

      • spec.nodeSelector.matchLabels
        object

        The set of node labels a node must carry to be selected.

  • status
    object
    • status.appliedNodes
      integer

      Number of selected nodes that report the extension installed at the digest of this request.

      Allowed values: 0 <= X

    • status.conditions
      array of objects

      Conditions of the request: Accepted and Ready.

      Accepted shows whether node-manager accepted the request. It is True with the Accepted reason or False with one of these reasons:

      • InvalidExtension: The extension has no name or no digest.
      • ReservedName: The name belongs to a platform extension.
      • Conflict: An older request of the same kind already uses this name or digest, or the platform installs a registry package with this name or digest on the same nodes.
      • KernelModuleConflict: An older request already loads one of the kernel modules with different parameters.
      • NotDeliverable: None of the selected NodeGroups installs extensions of this kind.

      Ready shows whether the selected nodes run the request’s digest:

      • True with the AllNodesApplied reason: Every selected node runs it.
      • True with the NoNodesSelected reason: The request selects no node. The selected NodeGroups do not exist, have no nodes, or have no nodes that match nodeSelector.
      • False with the RolloutInProgress reason: Some selected nodes do not run it yet.
      • False with the RefusedByNodes reason: At least one node refused the extension.
      • False with the NotAccepted reason: The Accepted condition is False.
      • status.conditions.lastTransitionTime
        string

        Required value

      • status.conditions.message
        string

        Required value

        Maximum length: 32768

      • status.conditions.observedGeneration
        integer

        Allowed values: 0 <= X

      • status.conditions.reason
        string

        Required value

        Pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$

        Length: 1..1024

      • status.conditions.status
        string

        Required value

        Allowed values: True, False, Unknown

      • status.conditions.type
        string

        Required value

        Pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$

        Maximum length: 316

    • status.failedNodes
      integer

      Number of selected nodes that refused the extension at the request’s digest.

      Allowed values: 0 <= X

    • status.matchedNodeGroups
      array of strings

      The selected NodeGroups whose nodes install this kind of image: Deckhouse Engine groups for a system extension, groups bashible manages for a registry package.

    • status.matchedNodes
      integer

      Number of nodes in matchedNodeGroups that the selectors pick.

      Allowed values: 0 <= X

    • status.nodes
      array of objects

      Selected nodes that have not applied the request, at most 32.

      • status.nodes.message
        string

        Why the node refused the request or what it waits for.

        Maximum length: 4096

      • status.nodes.name
        string

        Required value

        Node name.

      • status.nodes.state
        string

        Required value

        Failed if the node refused the request or Pending if it has not answered yet.

        Allowed values: Pending, Failed

    • status.observedGeneration
      integer

      The generation of the spec this status reflects.

      Allowed values: 0 <= X

    • status.pendingNodes
      integer

      Number of selected nodes that have reported neither the extension installed at the request’s digest nor a refusal yet.

      Allowed values: 0 <= X

    • status.phase
      string

      Summary of the conditions.

      • Pending: The selected nodes are still installing the extension (the Ready condition has the RolloutInProgress reason).
      • Ready: The Ready condition is True.
      • Degraded: node-manager did not accept the request (the Accepted condition is False), or nodes refused the extension (the Ready condition has the RefusedByNodes reason).

      Allowed values: Pending, Ready, Degraded

NodeGroupConfiguration

Short names: ngc

Scope: Cluster
Version: v1alpha1

Executes bash scripts on nodes.

Read more in the module documentation.

  • spec
    object

    Required value

    • spec.bundles
      array of strings

      Required value

      Bundles for step execution. You can set '*' for selecting all bundles.

      Examples:

      bundles:
      - ubuntu-lts
      - centos-7
      
      bundles:
      - ubuntu-lts
      
      bundles:
      - "*"
      
      • Element of the array
        string

        Allowed values: ubuntu-lts, centos, debian, redos, rosa, astra, altlinux, opensuse, *

    • spec.content
      string

      Required value

      A bash script that does the same things you would do in a configuration step.

      You can use Go Template to generate a script.

      The list of parameters available for use in templates can be retrieved from the bashible-apiserver-context Secret as follows:

      d8 k -n d8-cloud-instance-manager get secrets bashible-apiserver-context -o jsonpath='{.data.input\.yaml}'| base64 -d
      

      For example:

      {{- range .nodeUsers }}
      echo 'Tuning environment for user {{ .name }}'
      # Some code for tuning user environment
      {{- end }}
      

      You can also use the pre-defined bashbooster commands in the script. For example:

      bb-event-on 'bb-package-installed' 'post-install'
      post-install() {
        bb-log-info "Setting reboot flag due to kernel was updated"
        bb-flag-set reboot
      }
      

      Example of using the proxy parameters of ClusterConfiguration resource:

      bb-set-proxy
      curl -I https://google.com
      bb-unset-proxy
      
      test-function-with-proxy() {
        bb-set-proxy
        trap bb-unset-proxy RETURN
        curl -I https://google.com
      }
      test-function-with-proxy
      
    • spec.nodeGroups
      array of strings

      Required value

      List of NodeGroups to apply the step for. You can set '*' for selecting all NodeGroups.

      Examples:

      nodeGroups:
      - master
      - worker
      
      nodeGroups:
      - worker
      
      nodeGroups:
      - "*"
      
    • spec.weight
      integer

      Order of the step execution.

      Default: 100

NodeOperation

Short names: nop

Scope: Cluster
Version: v1alpha1

Describes an operation on a node that interrupts its work.

The spec of an existing operation cannot be changed, except .spec.approved. To approve a Disruption operation, run:

d8 k patch nodeoperation <OPERATION_NAME> --type=merge -p '{"spec":{"approved":true}}'

In the command, replace <OPERATION_NAME> with the name of the operation.

Operations for a deleted Cluster API Machine carry the label node-manager.deckhouse.io/machine-uid. Only node-manager can set it.

Example:

# Reboot a node.
apiVersion: deckhouse.io/v1alpha1
kind: NodeOperation
metadata:
  name: reboot-worker-1
spec:
  type: Reboot
  nodeName: worker-1
  • apiVersion
    string
  • kind
    string
  • metadata
    object
  • spec
    object

    Required value

    • spec.approved
      boolean

      Whether the interruption may go ahead. If the field is not set, the operation is not approved.

      Only meaningful for Disruption. Reboot and Drain ignore it.

      In the Manual approval mode of the NodeGroup (spec.disruptions.approvalMode), node-manager sets it to false, and an operator sets it to true. In the other modes, node-manager sets it to true on creation. An approval cannot be withdrawn.

    • spec.cause
      string

      Why the node is interrupted.

      For a Disruption, node-manager sets it on creation. For Reboot and Drain, an operator may set it.

      Maximum length: 1024

    • spec.configGeneration
      integer

      Revision of the node’s NodeConfig this disruption is for (.metadata.generation of the NodeConfig object).

      Allowed only for Disruption. The API server rejects the field for other types. If the NodeConfig moves to a newer generation before the operation is approved, node-manager deletes the operation.

      Allowed values: 1 <= X

    • spec.drain
      object

      How to evict the workload before the node is interrupted.

      Allowed for Reboot and Disruption. The API server rejects the field for Drain.

      • spec.drain.skip
        boolean

        Interrupt the node without evicting its workload first.

        When node-manager creates a Disruption, it sets the field to true for these nodes:

        • The single node of the master NodeGroup
        • The node of the Deckhouse pod, while fewer than two nodes of its group are Ready
        • Every node of a NodeGroup with spec.disruptions.automatic.drainBeforeApproval: false

        Default: false

    • spec.nodeConfigUID
      string

      NodeConfig object this disruption is for (.metadata.uid of the node’s NodeConfig).

      Only meaningful for Disruption. Other types ignore it.

      Maximum length: 36

    • spec.nodeName
      string

      Required value

      The name of the node the operation applies to.

      Pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$

      Length: 1..253

    • spec.type
      string

      Required value

      Operation to perform on the node.

      • Reboot: Reboot the node. Only the nodes of a NodeGroup with systemType: DeckhouseEngine perform this operation. A node that bashible manages refuses it, and the operation fails with the reason NodeRefused.
      • Drain: Evict the workload from the node. The node stays unschedulable until an operator makes it schedulable again. node-manager also creates a Drain for the node of a deleted Cluster API Machine. .spec.cause names the machine.

        To cancel a running Drain, delete it. node-manager withdraws its eviction request from the node before the object is gone. The cordon the eviction placed stays: make the node schedulable again yourself.

      • Disruption: Interrupt the node’s work so that the node can apply a new configuration.

      node-manager creates Disruption operations. Creating one by hand is refused. Reboot and Drain are meant for an operator.

      Allowed values: Reboot, Drain, Disruption

  • status
    object
    • status.bootID
      string

      Boot ID of the node (status.nodeInfo.bootID of the Node object) when the reboot started.

      Maximum length: 64

    • status.conditions
      array of objects

      Latest details of the operation’s progress:

      • Drained: Written by node-manager. While the workload is being evicted, the condition is False with the reason Draining. Once the workload has left, it is True with the reason Drained. When the operation skips the eviction, it is True with the reason DrainSkipped.
      • EvictionBlocked: Written by node-manager while evictions of the workload are refused, for example by a PodDisruptionBudget. The condition is True with the reason EvictionRefused, and its message lists the refusals as the eviction reports them, one per pod. The same text is in the operation’s message. The condition is removed once the refusals stop or the eviction ends. A finished operation keeps the refusals it ended on.
      • NodeReported: Written by the node. The condition is True with the reason Succeeded, Released, Failed, Refused or RebootStarted.
      • Retry: Written by node-manager on a failed Disruption. The condition says whether node-manager makes another attempt for the same NodeConfig revision.

      Reasons of the Retry condition:

      • Created (True): The next attempt exists, and message names it.
      • Scheduled (True): node-manager creates the next attempt at the time given in message.
      • AttemptsExhausted (False): 3 attempts for the same revision have failed. node-manager makes no new attempt until the NodeConfig changes or the oldest failed attempt is deleted.
      • Superseded (False): The NodeConfig has moved to another generation or has been recreated.
      • NotRequested (False): The node no longer asks to be interrupted for this revision.
      • status.conditions.lastTransitionTime
        string

        Required value

      • status.conditions.message
        string

        Required value

        Maximum length: 32768

      • status.conditions.observedGeneration
        integer

        Allowed values: 0 <= X

      • status.conditions.reason
        string

        Required value

        Pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$

        Length: 1..1024

      • status.conditions.status
        string

        Required value

        Allowed values: True, False, Unknown

      • status.conditions.type
        string

        Required value

        Pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$

        Maximum length: 316

    • status.deadline
      string

      Time when the current stage of the operation times out and the operation fails.

      The node gets 30 minutes from startedAt to report the result or to come back after a reboot. The eviction gets the drain timeout of the NodeGroup (nodeDrainTimeoutSecond) plus 30 minutes.

    • status.drainDeadline
      string

      When the eviction this operation asked for runs out of the time it was given.

      On the Drain of a deleted machine, the operation fails with PreparationTimedOut at this moment, and the machine is deleted with the pods that did not leave. The field is not set for a Static or CloudStatic NodeGroup without nodeDrainTimeoutSecond, and such a Drain has no time limit.

      It is pinned at the moment the eviction is requested, from the NodeGroup’s nodeDrainTimeoutSecond as it stood then, because that is the bound the draining controller itself started running to. Deriving it again later would let a change to the NodeGroup cut short a drain that is still running.

    • status.eviction
      object

      Progress of the eviction this operation asked for, written by node-manager while the eviction runs.

      The field appears once the pods to evict are listed and is updated every few seconds. It is removed when the eviction ends. A failed operation keeps the progress it ended on.

      • status.eviction.remainingPods
        integer

        Required value

        How many of the pods the eviction listed are still on the node, including the terminating ones. The eviction leaves DaemonSet pods, static pods and finished pods on the node, and they are not counted.

        Allowed values: 0 <= X

      • status.eviction.terminatingPods
        array of strings

        Pods that have been evicted and have not stopped yet, as <namespace>/<name>.

    • status.finishedAt
      string

      When the operation reached a terminal phase. A finished operation is kept as the record of what was done to the node and is collected once it is a day old.

    • status.message
      string

      Human-readable explanation of reason.

      Maximum length: 32768

    • status.nodeWasUnschedulable
      boolean

      Whether the node was already out of the scheduler when the operation reached it.

      Releasing the node at the end of the operation restores this value instead of making the node schedulable, so an operation does not quietly undo a cordon an operator put there by hand.

    • status.phase
      string

      Current phase of the operation, set by node-manager.

      • Pending: The operation is queued or waits for approval.
      • Approved: The operation may go ahead. The workload is being evicted from the node before the node receives the operation.
      • InProgress: For Drain, the workload is being evicted. For the other types, the node has been prepared and can perform the operation.
      • Completed: The operation finished successfully.
      • Failed: The operation failed.

      Allowed values: Pending, Approved, InProgress, Completed, Failed

    • status.reason
      string

      Reason for the current phase of the operation.

      • Pending: Queued (nothing has been done yet), WaitingForApproval.
      • Approved: Draining (the workload is being evicted from the node), WaitingForOperation (another eviction of the node is running, and message names who requested it).
      • InProgress: Draining and WaitingForOperation (for Drain), WaitingForNode (the node has received the operation and has not reported yet), Rebooting (the node is rebooting and has not come back yet).
      • Completed: Succeeded, Released (the node gave up on the NodeConfig revision the Disruption is for), Drained (for Drain).
      • Failed: NodeFailed and NodeRefused (reported by the node), PreparationTimedOut (the workload did not leave the node in time), NodeTimedOut (the node did not report back or did not come back after a reboot in time), NodeNotFound, NodeGroupMissing, MachineDeleted (the machine of the node is being deleted).

      Pattern: ^[A-Za-z][A-Za-z0-9]*$

      Maximum length: 128

    • status.startedAt
      string

      When the node was handed the operation. The wait for the node is measured from here.

NodeStaticPodRequest

Short names: nspr

Scope: Cluster
Version: v1alpha1

Request to run a static pod on the selected nodes without using the Kubernetes scheduler.

node-manager adds the manifest to spec.staticPods of the NodeConfig of each selected node. The node writes the manifest to disk, and kubelet runs the pod from it. The request reports whether the nodes wrote the manifest. It does not report whether the pod runs.

A user with permission to create NodeStaticPodRequests can run arbitrary manifests with root privileges on all selected nodes.

Example:

# A node-local agent on the nodes of the worker NodeGroup.
apiVersion: deckhouse.io/v1alpha1
kind: NodeStaticPodRequest
metadata:
  name: node-local-agent
spec:
  nodeGroupSelector:
    matchNames:
      - worker
  manifest: |
    apiVersion: v1
    kind: Pod
    metadata:
      name: node-local-agent
      namespace: kube-system
    spec:
      hostNetwork: true
      priorityClassName: system-node-critical
      containers:
        - name: agent
          image: deckhouse.local/images:node-local-agent
          args:
            - --listen-address=$MY_IP:5002
          env:
            - name: NODE_NAME
              valueFrom:
                fieldRef:
                  fieldPath: spec.nodeName
          volumeMounts:
            - name: state
              mountPath: /var/lib/node-local-agent
      volumes:
        - name: state
          hostPath:
            path: /var/lib/node-local-agent
            type: DirectoryOrCreate
  • apiVersion
    string
  • kind
    string
  • metadata
    object
  • spec
    object

    Required value

    • spec.manifest
      string

      Required value

      Complete pod manifest.

      Requirements for the manifest:

      • metadata.name is a DNS subdomain (RFC 1123), metadata.namespace is a DNS label (RFC 1123).
      • The namespace is kube-system or starts with d8-.
      • spec.nodeName is not set.
      • The pod does not reference API objects. Not allowed: spec.serviceAccountName, volumes of the secret, configMap, persistentVolumeClaim and projected types, envFrom and env[].valueFrom with secretKeyRef or configMapKeyRef in containers and init containers.

      A manifest that does not meet these requirements is rejected on creation and update.

      When the manifest is written to disk, the $MY_IP substring is replaced with the node’s IP address.

      Maximum manifest size is 32 KiB. A node can run no more than 16 static pods defined by NodeStaticPodRequests.

      Length: 1..32768

    • spec.nodeGroupSelector
      object

      Restricts the pod to nodes in the specified NodeGroups. If no groups are specified, all NodeGroups are selected.

      The node agent writes the manifest from the NodeConfig of each selected node. On nodes managed by bashible, a resource named registry-agent or describing the kube-system/registry-agent pod is not applied, because bashible writes that manifest itself. If the resource reaches no other node, the Ready condition reports the ServedByBashible reason.

      • spec.nodeGroupSelector.matchNames
        array of strings

        Names of NodeGroups on whose nodes the pod must run.

        • Element of the array
          string

          Pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$

          Maximum length: 63

  • status
    object
    • status.appliedNodes
      integer

      Number of selected nodes that reported the current manifest written.

      Allowed values: 0 <= X

    • status.conditions
      array of objects

      Conditions of the request: Accepted and Ready.

      Accepted shows whether node-manager accepted the request. It is True with the Accepted reason or False with one of these reasons:

      • InvalidName: The request name is not a DNS label (RFC 1123) of at most 63 characters.
      • ReservedName: The request name, or the metadata.namespace and metadata.name of the pod, is already used by a static pod of the platform.
      • InvalidManifest: The manifest does not meet the requirements of spec.manifest.
      • Conflict: An older request already defines a pod with the same metadata.namespace and metadata.name.
      • LimitExceeded: At least one selected NodeGroup already has 16 static pods from older requests. Only the nodes of the other selected NodeGroups get the manifest.

      Ready shows whether the selected nodes wrote the current manifest:

      • True with the AllNodesApplied reason: Every selected node wrote it.
      • True with the NoNodesSelected reason: The selector matches no existing NodeGroup, or the selected NodeGroups have no nodes.
      • True with the ServedByBashible reason: The selected NodeGroups are managed by bashible, which writes the registry-agent manifest on their nodes itself.
      • False with the RolloutInProgress reason: Some selected nodes have not written the current manifest yet.
      • False with the RefusedByNodes reason: At least one node refused the manifest.
      • False with the NotAccepted reason: The Accepted condition is False.
      • status.conditions.lastTransitionTime
        string

        Required value

      • status.conditions.message
        string

        Required value

        Maximum length: 32768

      • status.conditions.observedGeneration
        integer

        Allowed values: 0 <= X

      • status.conditions.reason
        string

        Required value

        Pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$

        Length: 1..1024

      • status.conditions.status
        string

        Required value

        Allowed values: True, False, Unknown

      • status.conditions.type
        string

        Required value

        Pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$

        Maximum length: 316

    • status.failedNodes
      integer

      Number of selected nodes that refused the manifest.

      Allowed values: 0 <= X

    • status.matchedNodeGroups
      array of strings

      NodeGroups that match the current selector.

    • status.matchedNodes
      integer

      Number of nodes in the selected NodeGroups to which the node agent writes the manifest. Nodes of NodeGroups on which bashible writes the manifest itself are not counted.

      Allowed values: 0 <= X

    • status.nodes
      array of objects

      Selected nodes that have not written the current manifest, at most 32.

      • status.nodes.message
        string

        Why the node refused the manifest or what it waits for.

        Maximum length: 4096

      • status.nodes.name
        string

        Required value

        Node name.

      • status.nodes.state
        string

        Required value

        Failed if the node refused the manifest or Pending if it has not answered yet.

        Allowed values: Pending, Failed

    • status.observedGeneration
      integer

      Generation of spec that corresponds to the current status.

      Allowed values: 0 <= X

    • status.pendingNodes
      integer

      Number of selected nodes that have not yet reported either the current manifest written or a failure.

      Allowed values: 0 <= X

    • status.phase
      string

      Summary of the conditions.

      • Pending: The selected nodes are still writing the manifest (the Ready condition has the RolloutInProgress reason).
      • Ready: The Ready condition is True.
      • Degraded: node-manager did not accept the request (the Accepted condition is False), or nodes refused the manifest (the Ready condition has the RefusedByNodes reason).

      Allowed values: Pending, Ready, Degraded

NodeUser

Scope: Cluster

  • v1
  • v1alpha1

Defines the linux users to create on all nodes.

The user’s home directory is created in the /home/deckhouse/ directory.

  • spec
    object

    Required value

    • spec.extraGroups
      array of strings

      Node user additional system groups.

      Examples:

      extraGroups:
      - docker
      
      extraGroups:
      - docker
      - ftp
      
    • spec.isSudoer
      boolean

      Persistence of node user in sudo group.

      Default: false

      Example:

      isSudoer: true
      
    • spec.nodeGroups
      array of strings

      List of NodeGroups to apply the user for.

      Default: ["*"]

      Examples:

      nodeGroups:
      - master
      - worker
      
      nodeGroups:
      - worker
      
      nodeGroups:
      - "*"
      
    • spec.passwordHash
      string

      Hashed user password.

      The format corresponds to the password hashes in /etc/shadow. You can get it using the following command: openssl passwd -6.

      If the caller is not allowed to read the nodeusers/sensitive subresource, the API returns <omitted> instead of the value.

      Example:

      passwordHash: "$2a$10$F9ey7zW.sVliT224RFxpWeMsgzO.D9YRG54a8T36/K2MCiT41nzmC"
      
    • spec.sshPublicKey
      Deprecated
      string

      Node user SSH public key.

      Either sshPublicKey or sshPublicKeys must be specified.

      Example:

      sshPublicKey: ssh-rsa AAABBB
      
    • spec.sshPublicKeys
      array of strings

      Node user SSH public keys.

      Either sshPublicKey or sshPublicKeys must be specified.

      Example:

      sshPublicKeys:
      - ssh-rsa AAABBB
      - cert-authority,principals="name" ssh-rsa BBBCCC
      
    • spec.uid
      number

      Required value

      Node user ID.

      We recommend using the values >= 1100 to avoid conflicts with manually created users.

      This parameter does not change during the entire resource life.

      Allowed values: 1001 <= X

      Example:

      uid: 1100
      
  • status
    object

    User creation status information

    • status.errors
      object

      List of user creation errors on each node

Deprecated resource. Support for the resource might be removed in a later release.

Defines the linux users to create on all nodes.

The user’s home directory is created in the /home/deckhouse/ directory.

  • spec
    object

    Required value

    • spec.extraGroups
      array of strings

      Node user additional system groups.

      Examples:

      extraGroups:
      - docker
      
      extraGroups:
      - docker
      - ftp
      
    • spec.isSudoer
      boolean

      Persistence of node user in sudo group.

      Default: false

      Example:

      isSudoer: true
      
    • spec.nodeGroups
      array of strings

      List of NodeGroups to apply the user for.

      Default: ["*"]

      Examples:

      nodeGroups:
      - master
      - worker
      
      nodeGroups:
      - worker
      
      nodeGroups:
      - "*"
      
    • spec.passwordHash
      string

      Hashed user password.

      The format corresponds to the password hashes in /etc/shadow. You can get it using the following command: openssl passwd -6.

      If the caller is not allowed to read the nodeusers/sensitive subresource, the API returns <omitted> instead of the value.

      Example:

      passwordHash: "$2a$10$F9ey7zW.sVliT224RFxpWeMsgzO.D9YRG54a8T36/K2MCiT41nzmC"
      
    • spec.sshPublicKey
      Deprecated
      string

      Node user SSH public key.

      Either sshPublicKey or sshPublicKeys must be specified.

      Example:

      sshPublicKey: ssh-rsa AAABBB
      
    • spec.sshPublicKeys
      array of strings

      Node user SSH public keys.

      Either sshPublicKey or sshPublicKeys must be specified.

      Example:

      sshPublicKeys:
      - ssh-rsa AAABBB
      - cert-authority,principals="name" ssh-rsa BBBCCC
      
    • spec.uid
      number

      Required value

      Node user ID.

      We recommend using the values >= 1100 to avoid conflicts with manually created users.

      This parameter does not change during the entire resource life.

      Allowed values: 1001 <= X

      Example:

      uid: 1100
      

SSHCredentials

Scope: Cluster

  • v1alpha2
  • v1alpha1

Contains credentials required by Cluster API Provider Static (CAPS) to connect over SSH. CAPS connects to the server (virtual machine) defined in the StaticInstance custom resource to manage its state.

A reference to this resource is specified in the credentialsRef parameter of the StaticInstance resource.

  • apiVersion
    string

    APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources

  • kind
    string

    Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds

  • metadata
    object
  • spec
    object

    SSHCredentialsSpec defines the desired state of SSHCredentials.

    • spec.privateSSHKey
      string

      Private SSH key in PEM format encoded as base64 string.

      If the user does not have permission to view sensitive SSHCredentials data, the API returns instead of the actual value.

    • spec.sshExtraArgs
      string

      A list of additional arguments to pass to the openssh command.

      Examples:

      sshExtraArgs: "-vvv"
      
      sshExtraArgs: "-c chacha20-poly1305@openssh.com"
      
      sshExtraArgs: "-c aes256-gcm@openssh.com"
      
      sshExtraArgs: "-m umac-64-etm@openssh.com"
      
      sshExtraArgs: "-m hmac-sha2-512-etm@openssh.com"
      
    • spec.sshPort
      integer

      A port to connect to the host via SSH.

      Default: 22

      Allowed values: 1 <= X <= 65535

    • spec.sudoPasswordEncoded
      string

      Base64 encoded sudo password for the user.

      If the user does not have permission to view sensitive SSHCredentials data, the API returns instead of the actual value.

    • spec.user
      string

      Required value

      A username to connect to the host via SSH.

Contains credentials required by Cluster API Provider Static (CAPS) to connect over SSH. CAPS connects to the server (virtual machine) defined in the StaticInstance custom resource to manage its state.

A reference to this resource is specified in the credentialsRef parameter of the StaticInstance resource.

  • apiVersion
    string

    APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources

  • kind
    string

    Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds

  • metadata
    object
  • spec
    object

    SSHCredentialsSpec defines the desired state of SSHCredentials.

    • spec.privateSSHKey
      string

      Private SSH key in PEM format encoded as base64 string.

      If the user does not have permission to view sensitive SSHCredentials data, the API returns instead of the actual value.

    • spec.sshExtraArgs
      string

      A list of additional arguments to pass to the openssh command.

      Examples:

      sshExtraArgs: "-vvv"
      
      sshExtraArgs: "-c chacha20-poly1305@openssh.com"
      
      sshExtraArgs: "-c aes256-gcm@openssh.com"
      
      sshExtraArgs: "-m umac-64-etm@openssh.com"
      
      sshExtraArgs: "-m hmac-sha2-512-etm@openssh.com"
      
    • spec.sshPort
      integer

      A port to connect to the host via SSH.

      Default: 22

      Allowed values: 1 <= X <= 65535

    • spec.sudoPassword
      string

      A sudo password for the user.

      If the user does not have permission to view sensitive SSHCredentials data, the API returns instead of the actual value.

    • spec.user
      string

      Required value

      A username to connect to the host via SSH.

StaticInstance

Scope: Cluster

  • v1alpha2
  • v1alpha1

StaticInstance describes a machine for the Cluster API Provider Static.

  • apiVersion
    string

    APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources

  • kind
    string

    Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds

  • metadata
    object
  • spec
    object

    StaticInstanceSpec defines the desired state of StaticInstance.

    • spec.address
      string

      Required value

      The IP address of the host.

      Pattern: ^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}$

    • spec.credentialsRef
      object

      Required value

      The reference to the SSHCredentials object.

      • spec.credentialsRef.apiVersion
        string

        API version of the referent.

      • spec.credentialsRef.fieldPath
        string

        If referring to a piece of an object instead of an entire object, this string should contain a valid JSON/Go field access statement, such as desiredState.manifest.containers[2]. For example, if the object reference is to a container within a pod, this would take on a value like: “spec.containers{name}” (where “name” refers to the name of the container that triggered the event) or if no container name is specified “spec.containers[2]” (container with index 2 in this pod). This syntax is chosen only to have some well-defined way of referencing a part of an object.

      • spec.credentialsRef.kind
        string

        Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds

      • spec.credentialsRef.name
        string

        Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names

      • spec.credentialsRef.namespace
        string

        Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/

      • spec.credentialsRef.resourceVersion
        string

        Specific resourceVersion to which this reference is made, if any. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency

      • spec.credentialsRef.uid
        string

        UID of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#uids

  • status
    object

    StaticInstanceStatus defines the observed state of StaticInstance.

    • status.conditions
      array of objects

      Conditions defines current service state of the StaticInstance.

      Condition contains details for one aspect of the current state of this API Resource.

      • status.conditions.lastTransitionTime
        string

        Required value

        lastTransitionTime is the last time the condition transitioned from one status to another. This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable.

      • status.conditions.message
        string

        Required value

        message is a human readable message indicating details about the transition. This may be an empty string.

        Maximum length: 32768

      • status.conditions.observedGeneration
        integer

        observedGeneration represents the .metadata.generation that the condition was set based upon. For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date with respect to the current state of the instance.

        Allowed values: 0 <= X

      • status.conditions.reason
        string

        Required value

        reason contains a programmatic identifier indicating the reason for the condition’s last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. The value should be a CamelCase string. This field may not be empty.

        Pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$

        Length: 1..1024

      • status.conditions.status
        string

        Required value

        status of the condition, one of True, False, Unknown.

        Allowed values: True, False, Unknown

      • status.conditions.type
        string

        Required value

        type of condition in CamelCase or in foo.example.com/CamelCase.

        Pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$

        Maximum length: 316

    • status.currentStatus
      object
      • status.currentStatus.lastUpdateTime
        string
      • status.currentStatus.phase
        string

        Allowed values: Error, Pending, Bootstrapping, Running, Cleaning

    • status.machineRef
      object

      The reference to the StaticMachine object.

      • status.machineRef.apiVersion
        string

        API version of the referent.

      • status.machineRef.fieldPath
        string

        If referring to a piece of an object instead of an entire object, this string should contain a valid JSON/Go field access statement, such as desiredState.manifest.containers[2]. For example, if the object reference is to a container within a pod, this would take on a value like: “spec.containers{name}” (where “name” refers to the name of the container that triggered the event) or if no container name is specified “spec.containers[2]” (container with index 2 in this pod). This syntax is chosen only to have some well-defined way of referencing a part of an object.

      • status.machineRef.kind
        string

        Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds

      • status.machineRef.name
        string

        Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names

      • status.machineRef.namespace
        string

        Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/

      • status.machineRef.resourceVersion
        string

        Specific resourceVersion to which this reference is made, if any. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency

      • status.machineRef.uid
        string

        UID of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#uids

    • status.nodeRef
      object

      The reference to the Node object.

      • status.nodeRef.apiVersion
        string

        API version of the referent.

      • status.nodeRef.fieldPath
        string

        If referring to a piece of an object instead of an entire object, this string should contain a valid JSON/Go field access statement, such as desiredState.manifest.containers[2]. For example, if the object reference is to a container within a pod, this would take on a value like: “spec.containers{name}” (where “name” refers to the name of the container that triggered the event) or if no container name is specified “spec.containers[2]” (container with index 2 in this pod). This syntax is chosen only to have some well-defined way of referencing a part of an object.

      • status.nodeRef.kind
        string

        Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds

      • status.nodeRef.name
        string

        Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names

      • status.nodeRef.namespace
        string

        Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/

      • status.nodeRef.resourceVersion
        string

        Specific resourceVersion to which this reference is made, if any. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency

      • status.nodeRef.uid
        string

        UID of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#uids

StaticInstance describes a machine for the Cluster API Provider Static.

  • apiVersion
    string

    APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources

  • kind
    string

    Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds

  • metadata
    object
  • spec
    object

    StaticInstanceSpec defines the desired state of StaticInstance.

    • spec.address
      string

      Required value

      The IP address of the host.

      Pattern: ^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}$

    • spec.credentialsRef
      object

      Required value

      The reference to the SSHCredentials object.

      • spec.credentialsRef.apiVersion
        string

        API version of the referent.

      • spec.credentialsRef.fieldPath
        string

        If referring to a piece of an object instead of an entire object, this string should contain a valid JSON/Go field access statement, such as desiredState.manifest.containers[2]. For example, if the object reference is to a container within a pod, this would take on a value like: “spec.containers{name}” (where “name” refers to the name of the container that triggered the event) or if no container name is specified “spec.containers[2]” (container with index 2 in this pod). This syntax is chosen only to have some well-defined way of referencing a part of an object.

      • spec.credentialsRef.kind
        string

        Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds

      • spec.credentialsRef.name
        string

        Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names

      • spec.credentialsRef.namespace
        string

        Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/

      • spec.credentialsRef.resourceVersion
        string

        Specific resourceVersion to which this reference is made, if any. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency

      • spec.credentialsRef.uid
        string

        UID of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#uids

  • status
    object

    StaticInstanceStatus defines the observed state of StaticInstance.

    • status.conditions
      array of objects

      Conditions defines current service state of the StaticInstance.

      Condition contains details for one aspect of the current state of this API Resource.

      • status.conditions.lastTransitionTime
        string

        Required value

        lastTransitionTime is the last time the condition transitioned from one status to another. This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable.

      • status.conditions.message
        string

        Required value

        message is a human readable message indicating details about the transition. This may be an empty string.

        Maximum length: 32768

      • status.conditions.observedGeneration
        integer

        observedGeneration represents the .metadata.generation that the condition was set based upon. For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date with respect to the current state of the instance.

        Allowed values: 0 <= X

      • status.conditions.reason
        string

        Required value

        reason contains a programmatic identifier indicating the reason for the condition’s last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. The value should be a CamelCase string. This field may not be empty.

        Pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$

        Length: 1..1024

      • status.conditions.status
        string

        Required value

        status of the condition, one of True, False, Unknown.

        Allowed values: True, False, Unknown

      • status.conditions.type
        string

        Required value

        type of condition in CamelCase or in foo.example.com/CamelCase.

        Pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$

        Maximum length: 316

    • status.currentStatus
      object
      • status.currentStatus.lastUpdateTime
        string
      • status.currentStatus.phase
        string

        Allowed values: Error, Pending, Bootstrapping, Running, Cleaning

    • status.machineRef
      object

      The reference to the StaticMachine object.

      • status.machineRef.apiVersion
        string

        API version of the referent.

      • status.machineRef.fieldPath
        string

        If referring to a piece of an object instead of an entire object, this string should contain a valid JSON/Go field access statement, such as desiredState.manifest.containers[2]. For example, if the object reference is to a container within a pod, this would take on a value like: “spec.containers{name}” (where “name” refers to the name of the container that triggered the event) or if no container name is specified “spec.containers[2]” (container with index 2 in this pod). This syntax is chosen only to have some well-defined way of referencing a part of an object.

      • status.machineRef.kind
        string

        Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds

      • status.machineRef.name
        string

        Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names

      • status.machineRef.namespace
        string

        Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/

      • status.machineRef.resourceVersion
        string

        Specific resourceVersion to which this reference is made, if any. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency

      • status.machineRef.uid
        string

        UID of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#uids

    • status.nodeRef
      object

      The reference to the Node object.

      • status.nodeRef.apiVersion
        string

        API version of the referent.

      • status.nodeRef.fieldPath
        string

        If referring to a piece of an object instead of an entire object, this string should contain a valid JSON/Go field access statement, such as desiredState.manifest.containers[2]. For example, if the object reference is to a container within a pod, this would take on a value like: “spec.containers{name}” (where “name” refers to the name of the container that triggered the event) or if no container name is specified “spec.containers[2]” (container with index 2 in this pod). This syntax is chosen only to have some well-defined way of referencing a part of an object.

      • status.nodeRef.kind
        string

        Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds

      • status.nodeRef.name
        string

        Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names

      • status.nodeRef.namespace
        string

        Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/

      • status.nodeRef.resourceVersion
        string

        Specific resourceVersion to which this reference is made, if any. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency

      • status.nodeRef.uid
        string

        UID of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#uids