crate.operator.restore_backup module

class crate.operator.restore_backup.AfterRestoreBackupSubHandler(namespace, name, ref, context, depends_on=None, run_on_dep_failures=False)

Bases: StateBasedSubHandler

Parameters

  • namespace (str) –

  • name (str) –

  • ref (str) –

  • context (dict) –

async handle(namespace, name, status, repository, logger, **kwargs)
Parameters
class crate.operator.restore_backup.BeforeRestoreBackupSubHandler(namespace, name, ref, context, depends_on=None, run_on_dep_failures=False)

Bases: StateBasedSubHandler

Parameters

  • namespace (str) –

  • name (str) –

  • ref (str) –

  • context (dict) –

async handle(namespace, name, patch, logger, **kwargs)
Parameters
class crate.operator.restore_backup.ResetSnapshotSubHandler(namespace, name, ref, context, depends_on=None, run_on_dep_failures=False)

Bases: StateBasedSubHandler

Parameters

  • namespace (str) –

  • name (str) –

  • ref (str) –

  • context (dict) –

async handle(namespace, name, logger, **kwargs)

Reset the snapshot name in the CrateDB spec to ensure the same snapshot can be restored again if it failed for any reason. This has to be done last because kopf recognizes it as a new change in the restoreSnapshot field.

Parameters

  • namespace (str) – The Kubernetes namespace of the CrateDB cluster.

  • name (str) – The CrateDB custom resource name defining the CrateDB cluster.

  • logger (Logger) – the logger on which we’re logging

  • kwargs (Any) –

class crate.operator.restore_backup.RestoreAll(*, tables=None, sections=None, partitions=None, **_kwargs)

Bases: RestoreType

Parameters
get_restore_keyword()

Each subclass needs to return the keyword to be used in the RESTORE SNAPSHOT command based on the type of restore operation.

async validate_restore_complete(*, conn_factory, snapshot, logger, **_kwargs)

Each subclass needs to define a method which indicates that the restore operation has been completed successfully. Or raise a kopf.TemporaryError in case the operation is still in progress.

Parameters
class crate.operator.restore_backup.RestoreBackupSubHandler(namespace, name, ref, context, depends_on=None, run_on_dep_failures=False)

Bases: StateBasedSubHandler

Parameters

  • namespace (str) –

  • name (str) –

  • ref (str) –

  • context (dict) –

async handle(namespace, name, repository, snapshot, restore_type, tables, partitions, sections, logger, **kwargs)
Parameters
class crate.operator.restore_backup.RestoreDataSections(*, tables=None, sections=None, partitions=None, **_kwargs)

Bases: RestoreType

Parameters
DATA_SECTION_TABLES: str = 'tables'
get_restore_keyword()

Each subclass needs to return the keyword to be used in the RESTORE SNAPSHOT command based on the type of restore operation.

async validate_restore_complete(*, conn_factory, snapshot, logger, **_kwargs)

Each subclass needs to define a method which indicates that the restore operation has been completed successfully. Or raise a kopf.TemporaryError in case the operation is still in progress.

Parameters
class crate.operator.restore_backup.RestoreMetadata(*, tables=None, sections=None, partitions=None, **_kwargs)

Bases: RestoreType

Parameters
get_restore_keyword()

Each subclass needs to return the keyword to be used in the RESTORE SNAPSHOT command based on the type of restore operation.

async validate_restore_complete(*, conn_factory, snapshot, logger, **_kwargs)

Each subclass needs to define a method which indicates that the restore operation has been completed successfully. Or raise a kopf.TemporaryError in case the operation is still in progress.

Parameters
class crate.operator.restore_backup.RestorePartitions(*, tables=None, sections=None, partitions=None, **_kwargs)

Bases: RestoreType

Parameters
get_restore_keyword()

Each subclass needs to return the keyword to be used in the RESTORE SNAPSHOT command based on the type of restore operation.

async validate_restore_complete(*, conn_factory, snapshot, logger, **_kwargs)

Each subclass needs to define a method which indicates that the restore operation has been completed successfully. Or raise a kopf.TemporaryError in case the operation is still in progress.

Parameters
class crate.operator.restore_backup.RestoreSystemUserPasswordSubHandler(namespace, name, ref, context, depends_on=None, run_on_dep_failures=False)

Bases: StateBasedSubHandler

Parameters

  • namespace (str) –

  • name (str) –

  • ref (str) –

  • context (dict) –

async handle(namespace, name, logger, **kwargs)

Restore the system user password from the secret in the namespace. Use crash here because during a restore the system user password was probably set to a different value.

Parameters

  • namespace (str) – The Kubernetes namespace of the CrateDB cluster.

  • name (str) – The CrateDB custom resource name defining the CrateDB cluster.

  • logger (Logger) – the logger on which we’re logging

  • kwargs (Any) –

class crate.operator.restore_backup.RestoreTables(*, tables=None, sections=None, partitions=None, **_kwargs)

Bases: RestoreType

Parameters
get_restore_keyword()

Each subclass needs to return the keyword to be used in the RESTORE SNAPSHOT command based on the type of restore operation.

async validate_restore_complete(*, conn_factory, snapshot, logger, **_kwargs)

Each subclass needs to define a method which indicates that the restore operation has been completed successfully. Or raise a kopf.TemporaryError in case the operation is still in progress.

Parameters
class crate.operator.restore_backup.RestoreType(*, tables=None, sections=None, partitions=None, **_kwargs)

Bases: ABC

Base class for the different types of restore operations. New subclasses are identified by their restore_type and can be added dynamically by using the register_subclass decorator.

Parameters
classmethod create(restore_type, *args, **kwargs)
Parameters

restore_type (str) –

abstract get_restore_keyword()

Each subclass needs to return the keyword to be used in the RESTORE SNAPSHOT command based on the type of restore operation.

classmethod register_subclass(restore_type)
Parameters

restore_type (str) –

subclasses: Dict[str, Any] = {'all': <class 'crate.operator.restore_backup.RestoreAll'>, 'metadata': <class 'crate.operator.restore_backup.RestoreMetadata'>, 'partitions': <class 'crate.operator.restore_backup.RestorePartitions'>, 'sections': <class 'crate.operator.restore_backup.RestoreDataSections'>, 'tables': <class 'crate.operator.restore_backup.RestoreTables'>}
abstract async validate_restore_complete(*, conn_factory, snapshot, logger, **_kwargs)

Each subclass needs to define a method which indicates that the restore operation has been completed successfully. Or raise a kopf.TemporaryError in case the operation is still in progress.

Parameters
class crate.operator.restore_backup.SendSuccessNotificationSubHandler(namespace, name, ref, context, depends_on=None, run_on_dep_failures=False)

Bases: StateBasedSubHandler

A handler which depends on all other subhandlers having finished successfully and schedules a success notification of the restore process.

Parameters

  • namespace (str) –

  • name (str) –

  • ref (str) –

  • context (dict) –

async handle(namespace, name, logger, **kwargs)

Schedule success notification and send it after the cluster has been restored successfully.

Parameters

  • namespace (str) – The Kubernetes namespace of the CrateDB cluster.

  • name (str) – The CrateDB custom resource name defining the CrateDB cluster.

  • logger (Logger) – the logger on which we’re logging

  • kwargs (Any) –

class crate.operator.restore_backup.ValidateRestoreCompleteSubHandler(namespace, name, ref, context, depends_on=None, run_on_dep_failures=False)

Bases: StateBasedSubHandler

Parameters

  • namespace (str) –

  • name (str) –

  • ref (str) –

  • context (dict) –

async handle(namespace, name, snapshot, restore_type, tables, partitions, sections, logger, **kwargs)
Parameters
async crate.operator.restore_backup.drop_repository(cursor, repository, logger)

Drops a backup repository if it exists.

Parameters

  • cursor (Cursor) – A database cursor to a current and open database connection.

  • repository (str) – The name of the repository to drop.

  • logger (Logger) – the logger on which we’re logging

async crate.operator.restore_backup.ensure_no_restore_in_progress(namespace, name, snapshot, pod_name, scheme, logger)

This checks if there is a restore operation of the given snapshot currently in progress by querying the sys.snapshot_restore table. If there is a restore in progress, it queries the sys.shards table to get the progress of the operation. It sends this information to the API and raises a kopf.TemporaryError. Use crash here because during a restore the system user password might be restored to a different value already.

Parameters

  • namespace (str) – The Kubernetes namespace of the CrateDB cluster.

  • name (str) – The CrateDB custom resource name defining the CrateDB cluster.

  • snapshot (str) – The name of the snapshot.

  • pod_name (str) – The pod name where the crash command should be run.

  • scheme (str) – The host scheme for running the crash command.

  • logger (Logger) – the logger on which we’re logging

crate.operator.restore_backup.get_crash_pod_name(spec, name)

Returns the pod name where crash commands should be run.

Parameters

  • spec (dict) – The CrateDB custom resource definition.

  • name (str) – The CrateDB custom resource name defining the CrateDB cluster.

Return type

str

crate.operator.restore_backup.get_crash_scheme(spec)

Return the host scheme for running crash commands.

Parameters

spec (dict) – The CrateDB custom resource definition.

Return type

str

async crate.operator.restore_backup.get_snapshot_tables(conn_factory, snapshot, logger)

Returns a list of tables included in a snapshot.

Parameters

  • conn_factory – A function that establishes a database connection to the CrateDB cluster used for SQL queries.

  • snapshot (str) – The name of the snapshot where to lookup the tables.

  • logger (Logger) – the logger on which we’re logging

Return type

List[Any]

async crate.operator.restore_backup.get_source_backup_repository_data(core, namespace, name, logger)

Read the secret values to access the backup repository of the source cluster defined by secretKeyRef in restoreSnapshot.

Parameters

  • core (CoreV1Api) – An instance of the Kubernetes Core V1 API.

  • namespace (str) – The namespace where to lookup the secret and its value.

  • name (str) – The CrateDB custom resource name defining the CrateDB cluster.

  • logger (Logger) – the logger on which we’re logging

Return type

dict

crate.operator.restore_backup.is_valid_snapshot(new, **kwargs)

This checks if the new snapshot name is valid or not (empty). This check is necessary to avoid another restore operation is triggered after we reset the snapshot field at the end of a completed restore operation.

Parameters

new (Body) – The new CrateDB resource

Return type

bool

async crate.operator.restore_backup.run_crash_command(namespace, pod_name, scheme, command, logger, delay=30)

This connects to a CrateDB pod and executes a crash command in the crate container. It returns the result of the execution.

Parameters

  • namespace (str) – The Kubernetes namespace of the CrateDB cluster.

  • pod_name (str) – The pod name where the command should be run.

  • scheme (str) – The host scheme for running the command.

  • command (str) – The SQL query that should be run.

  • logger – the logger on which we’re logging

  • delay (int) – Time in seconds between the retries when executing the query.

async crate.operator.restore_backup.shards_recovery_in_progress(conn_factory, snapshot, tables, logger)

Checks if there is at least one shard which has not fully recovered after an operation of type SNAPSHOT.

Parameters

  • conn_factory – A function that establishes a database connection to the CrateDB cluster used for SQL queries.

  • snapshot (str) – The name of the snapshot to restore.

  • tables (List[str]) – A list of tables which should be checked for shards that have not been restored completely.

  • logger (Logger) – the logger on which we’re logging

async crate.operator.restore_backup.update_cratedb_admin_username_in_cratedb(namespace, cluster_name, new_admin_username)