The managed-hive-metastore module manages instances of Hive Metastore (HMS) in Deckhouse Platform (DP). HMS is the centralized metadata store in the big data ecosystem. The module provides:

  • Automatic Deployment: Creates an HMS instance using a simple YAML configuration.
  • Standalone: Supports deployment of a standalone instance.
  • Configuration Management: Separate HiveMetastoreClass resource for templating service creation and flexible validation of user parameters.
  • Status: Displays the current status of the deployed HMS instance.

For more details about module settings and usage examples, refer to the module documentation.

Module architecture

The following assumptions are used to simplify the diagram:

  • The diagram shows direct communication between containers in different Pods. In practice, components communicate through Kubernetes Services (internal load balancers). Service names are omitted when obvious from context. In other cases, a service name is shown above the arrow.
  • Pods can run with multiple replicas, but only one replica of each Pod is shown in the diagram.

The level-2 C4 architecture of the managed-hive-metastore module and its interactions with other components of DP are shown in the following diagram:

Managed-hive-metastore module architecture

Module components

The module consists of the following components:

  1. Managed-hive-metastore-operator (Deployment): Kubernetes operator consisting of a single container manager that performs the following operations:

    • Reconciles HiveMetastore custom resources in all user namespaces. The HiveMetastore and HiveMetastoreClass custom resources define the settings of the HMS instance.

    • Creates and manages the Certificate custom resource and Deployment, Service, and Secret resources related to the HMS instance.

    • Validates and mutates HiveMetastore custom resources using the Validating/Mutating Admission Controllers mechanism.

  2. D8ms-hms-<INSTANCE_NAME> (Deployment): A component that starts and runs the HMS instance.

    It consists of the following containers:

    • truststore-prepare: Init container that prepares the truststore.
    • agent: Sidecar container that monitors and updates the TLS certificate status.
    • hivemetastore: Main container.

Module interactions

The module interacts with the following components:

  1. PostgreSQL instance: Processes metadata on the database server.

  2. Object storage: Processes data in the object storage.

  3. Kube-apiserver:

    • Manages HiveMetastore and Certificate custom resources.
    • Watches HiveMetastoreClass custom resources.
    • Manages Deployment, Service, and Secret resources.

The following external components interact with the module:

  1. Kube-apiserver: Validates and mutates HiveMetastore custom resources.

  2. User applications: Send requests to the HMS instance.

Additional resources