crate.operator.restore_backup module

class crate.operator.restore_backup.AfterRestoreBackupSubHandler(namespace, name, ref, context, depends_on=None, run_on_dep_failures=False, operation=None)

Bases: StateBasedSubHandler

Parameters:
async handle(namespace, name, status, repository, logger, **kwargs)
Parameters:
class crate.operator.restore_backup.BeforeRestoreBackupSubHandler(namespace, name, ref, context, depends_on=None, run_on_dep_failures=False, operation=None)

Bases: StateBasedSubHandler

Parameters:
async handle(namespace, name, patch, logger, **kwargs)
Parameters:
class crate.operator.restore_backup.ResetSnapshotSubHandler(namespace, name, ref, context, depends_on=None, run_on_dep_failures=False, operation=None)

Bases: StateBasedSubHandler

Parameters:
async handle(namespace, name, logger, **kwargs)

Reset the snapshot name in the CrateDB spec to ensure the same snapshot can be restored again if it failed for any reason. This has to be done last because kopf recognizes it as a new change in the restoreSnapshot field.

Parameters:

  • namespace (str) – The Kubernetes namespace of the CrateDB cluster.

  • name (str) – The CrateDB custom resource name defining the CrateDB cluster.

  • logger (Logger) – the logger on which we’re logging

  • kwargs (Any)

class crate.operator.restore_backup.RestoreAll(*, tables=None, sections=None, partitions=None, **_kwargs)

Bases: RestoreType

Parameters:
get_restore_keyword(*, cursor)

Each subclass needs to return the keyword to be used in the RESTORE SNAPSHOT command based on the type of restore operation.

Parameters:

cursor (Cursor)

async validate_restore_complete(*, conn_factory, snapshot, logger, **_kwargs)

Each subclass needs to define a method which indicates that the restore operation has been completed successfully. Or raise a kopf.TemporaryError in case the operation is still in progress.

Parameters:
class crate.operator.restore_backup.RestoreBackupSubHandler(namespace, name, ref, context, depends_on=None, run_on_dep_failures=False, operation=None)

Bases: StateBasedSubHandler

Parameters:
async handle(namespace, name, repository, snapshot, restore_type, backup_provider, tables, partitions, sections, logger, **kwargs)
Parameters:
class crate.operator.restore_backup.RestoreDataSections(*, tables=None, sections=None, partitions=None, **_kwargs)

Bases: RestoreType

Parameters:
DATA_SECTION_TABLES: str = 'tables'
get_restore_keyword(*, cursor)

Each subclass needs to return the keyword to be used in the RESTORE SNAPSHOT command based on the type of restore operation.

Parameters:

cursor (Cursor)

async validate_restore_complete(*, conn_factory, snapshot, logger, **_kwargs)

Each subclass needs to define a method which indicates that the restore operation has been completed successfully. Or raise a kopf.TemporaryError in case the operation is still in progress.

Parameters:
class crate.operator.restore_backup.RestoreInternalTables(conn_factory, repository, snapshot, logger)

Bases: object

Parameters:
async cleanup_tables()

After a successful restore, the temporary renamed gc tables can be dropped.

has_tables_to_process()
Return type:

bool

async rename_duplicated_tables()

If the snapshot contains grand central tables, rename them if they exist in the cluster in order to recreate the new ones from the snapshot.

async restore_tables()

If the restore operation failed, rename back the gc tables to their original names.

async set_gc_tables(restore_type, tables=None)

Retrieve the grand central tables from the snapshot to be restored.

Parameters:
class crate.operator.restore_backup.RestoreInternalTablesSubHandler(namespace, name, ref, context, depends_on=None, run_on_dep_failures=False, operation=None)

Bases: StateBasedSubHandler

Parameters:
async handle(namespace, name, snapshot, repository, restore_type, tables, logger, **kwargs)

Handles restoring or cleaning up internal tables after a snapshot restore.

This subhandler runs after dependent handlers (e.g. RestoreSubHandler and ValidateRestoreCompleteSubHandler) have completed. It verifies their results and then either restores temporary internal tables (on failure) or drops them (on success).

Parameters:
class crate.operator.restore_backup.RestoreInternalUsersPasswordSubHandler(namespace, name, ref, context, depends_on=None, run_on_dep_failures=False, operation=None)

Bases: StateBasedSubHandler

Parameters:
async handle(namespace, name, logger, **kwargs)

Restore the system user and grand-central user passwords from the secret in the namespace. Use crash here because during a restore the system user password was probably set to a different value.

Parameters:

  • namespace (str) – The Kubernetes namespace of the CrateDB cluster.

  • name (str) – The CrateDB custom resource name defining the CrateDB cluster.

  • logger (Logger) – the logger on which we’re logging

  • kwargs (Any)

class crate.operator.restore_backup.RestoreMetadata(*, tables=None, sections=None, partitions=None, **_kwargs)

Bases: RestoreType

Parameters:
get_restore_keyword(*, cursor)

Each subclass needs to return the keyword to be used in the RESTORE SNAPSHOT command based on the type of restore operation.

Parameters:

cursor (Cursor)

async validate_restore_complete(*, conn_factory, snapshot, logger, **_kwargs)

Each subclass needs to define a method which indicates that the restore operation has been completed successfully. Or raise a kopf.TemporaryError in case the operation is still in progress.

Parameters:
class crate.operator.restore_backup.RestorePartitions(*, tables=None, sections=None, partitions=None, **_kwargs)

Bases: RestoreType

Parameters:
get_restore_keyword(*, cursor)

Each subclass needs to return the keyword to be used in the RESTORE SNAPSHOT command based on the type of restore operation.

Parameters:

cursor (Cursor)

async validate_restore_complete(*, conn_factory, snapshot, logger, **_kwargs)

Each subclass needs to define a method which indicates that the restore operation has been completed successfully. Or raise a kopf.TemporaryError in case the operation is still in progress.

Parameters:
class crate.operator.restore_backup.RestoreTables(*, tables=None, sections=None, partitions=None, **_kwargs)

Bases: RestoreType

Parameters:
get_restore_keyword(*, cursor)

Each subclass needs to return the keyword to be used in the RESTORE SNAPSHOT command based on the type of restore operation.

Parameters:

cursor (Cursor)

async validate_restore_complete(*, conn_factory, snapshot, logger, **_kwargs)

Each subclass needs to define a method which indicates that the restore operation has been completed successfully. Or raise a kopf.TemporaryError in case the operation is still in progress.

Parameters:
class crate.operator.restore_backup.RestoreType(*, tables=None, sections=None, partitions=None, **_kwargs)

Bases: ABC

Base class for the different types of restore operations. New subclasses are identified by their restore_type and can be added dynamically by using the register_subclass decorator.

Parameters:
classmethod create(restore_type, *args, **kwargs)
Parameters:

restore_type (str)

abstractmethod get_restore_keyword(*, cursor)

Each subclass needs to return the keyword to be used in the RESTORE SNAPSHOT command based on the type of restore operation.

Parameters:

cursor (Cursor)

classmethod register_subclass(restore_type)
Parameters:

restore_type (str)

subclasses: Dict[str, Any] = {'all': <class 'crate.operator.restore_backup.RestoreAll'>, 'metadata': <class 'crate.operator.restore_backup.RestoreMetadata'>, 'partitions': <class 'crate.operator.restore_backup.RestorePartitions'>, 'sections': <class 'crate.operator.restore_backup.RestoreDataSections'>, 'tables': <class 'crate.operator.restore_backup.RestoreTables'>}
abstractmethod async validate_restore_complete(*, conn_factory, snapshot, logger, **_kwargs)

Each subclass needs to define a method which indicates that the restore operation has been completed successfully. Or raise a kopf.TemporaryError in case the operation is still in progress.

Parameters:
class crate.operator.restore_backup.SendSuccessNotificationSubHandler(namespace, name, ref, context, depends_on=None, run_on_dep_failures=False, operation=None)

Bases: StateBasedSubHandler

A handler which depends on all other subhandlers having finished successfully and schedules a success notification of the restore process.

Parameters:
async handle(namespace, name, logger, **kwargs)

Schedule success notification and send it after the cluster has been restored successfully.

Parameters:

  • namespace (str) – The Kubernetes namespace of the CrateDB cluster.

  • name (str) – The CrateDB custom resource name defining the CrateDB cluster.

  • logger (Logger) – the logger on which we’re logging

  • kwargs (Any)

class crate.operator.restore_backup.ValidateRestoreCompleteSubHandler(namespace, name, ref, context, depends_on=None, run_on_dep_failures=False, operation=None)

Bases: StateBasedSubHandler

Parameters:
async handle(namespace, name, snapshot, restore_type, tables, partitions, sections, logger, **kwargs)
Parameters:
async crate.operator.restore_backup.drop_repository(cursor, repository, logger)

Drops a backup repository if it exists.

Parameters:

  • cursor (Cursor) – A database cursor to a current and open database connection.

  • repository (str) – The name of the repository to drop.

  • logger (Logger) – the logger on which we’re logging

async crate.operator.restore_backup.ensure_no_restore_in_progress(namespace, name, snapshot, pod_name, scheme, logger)

This checks if there is a restore operation of the given snapshot currently in progress by querying the sys.snapshot_restore table. If there is a restore in progress, it queries the sys.shards table to get the progress of the operation. It sends this information to the API and raises a kopf.TemporaryError. Use crash here because during a restore the system user password might be restored to a different value already.

Parameters:

  • namespace (str) – The Kubernetes namespace of the CrateDB cluster.

  • name (str) – The CrateDB custom resource name defining the CrateDB cluster.

  • snapshot (str) – The name of the snapshot.

  • pod_name (str) – The pod name where the crash command should be run.

  • scheme (str) – The host scheme for running the crash command.

  • logger (Logger) – the logger on which we’re logging

async crate.operator.restore_backup.get_snapshot_tables(conn_factory, snapshot, logger)

Returns a list of tables included in a snapshot.

Parameters:

  • conn_factory – A function that establishes a database connection to the CrateDB cluster used for SQL queries.

  • snapshot (str) – The name of the snapshot where to lookup the tables.

  • logger (Logger) – the logger on which we’re logging

Return type:

List[Any]

async crate.operator.restore_backup.get_source_backup_repository_data(core, namespace, name, backup_provider, logger)

Read the secret values to access the backup repository of the source cluster defined by secretKeyRef in restoreSnapshot.

Parameters:

  • core (CoreV1Api) – An instance of the Kubernetes Core V1 API.

  • namespace (str) – The namespace where to lookup the secret and its value.

  • name (str) – The CrateDB custom resource name defining the CrateDB cluster.

  • logger (Logger) – the logger on which we’re logging

  • backup_provider (BackupStorageProvider)

Return type:

BackupRepositoryData

crate.operator.restore_backup.is_valid_snapshot(new, **kwargs)

This checks if the new snapshot name is valid or not (empty). This check is necessary to avoid another restore operation is triggered after we reset the snapshot field at the end of a completed restore operation.

Parameters:

new (Body) – The new CrateDB resource

Return type:

bool

crate.operator.restore_backup.quote_table(table, cursor)

Ensure table names are correctly quoted. If it contains a schema (e.g., ‘doc.nyc_taxi’), quote both the schema and the table using psycopg2.extensions.quote_ident.

Return type:

str

crate.operator.restore_backup.restore_internal_tables_context(apps, namespace, name, conn_factory, repository, snapshot, logger, restore_type, tables)
async crate.operator.restore_backup.shards_recovery_in_progress(conn_factory, snapshot, tables, logger)

Checks if there is at least one shard which has not fully recovered after an operation of type SNAPSHOT.

Parameters:

  • conn_factory – A function that establishes a database connection to the CrateDB cluster used for SQL queries.

  • snapshot (str) – The name of the snapshot to restore.

  • tables (List[str]) – A list of tables which should be checked for shards that have not been restored completely.

  • logger (Logger) – the logger on which we’re logging

crate.operator.restore_backup.table_without_schema(table, cursor)

Returns the table name without schema, ensuring it’s correctly quoted.

Parameters:

  • table – The full table name, possibly including schema.

  • cursor – The database cursor used for quoting.

Return type:

str

Returns:

The quoted table name without schema.

async crate.operator.restore_backup.update_cratedb_admin_username_in_cratedb(namespace, cluster_name, new_admin_username)