Backup and Restore for DigitalOcean Kubernetes Using Kasten K10
In this blog post, we will guide you step by step on how to use Kasten K10 to backup and restore MongoDB databases operating in a DigitalOcean managed Kubernetes environment.
MongoDB is an open-source and NoSQL database. MongoDB uses a JSON-like document and optional schema. MongoDB database has the ability to easily store a large amount of data and scales very easily.
Kasten K10 is a Kubernetes-native data management platform that provides operations teams an easy-to-use, scalable, and secure system for backup/restore, disaster recovery, and mobility of Kubernetes applications.
We will assume you already have set up your DigitalOcean account and completed the steps to spin up the cluster. You can spin up the clusters through guided UI, which will take about four minutes to start the cluster. Connect to the cluster through this Link.
There are three steps involved to use Kasten K10 backup and restore feature in DigitalOcean managed Kubernetes service:
- Installing Kasten K10 on your DigitalOcean Kubernetes
- Installing MongoDB
- Backup and restore workflow using Kasten K10
Tools that are used in this blog post are:
- kubectl - Kubernetes client
- doctl - DigitalOcean client
- Helm v3
Step 1: Installing Kasten K10 on Your DigitalOcean Cluster
You can find an overview and detailed documentation for K10 here. Please make sure you have set up your tools that are mentioned above, before starting this installation. We will use Kasten K10 Helm chart to install K10 on a Kubernetes cluster and to add the repo, we will use the following command:
$ helm repo add kasten https://charts.kasten.io/
Next we will create a Kasten namespace to deploy the K10 application:
$ kubectl create namespace kasten-io
Now we will install K10 using the command below:
$ helm install k10 kasten/k10 -n kasten-io
Helm install will create multiple deployments and services, and you can watch the status and validate the install by the following command:
$ kubectl get pods -n kasten-io --watch
Once the pods are in running condition, you can port-forward the service to access K10 dashboard from the browser.
You can access the K10 dashboard at 127.0.0.1:8080/k10/#/ after running the following command:
$ kubectl --namespace kasten-io port-forward service/gateway 8080:8000
Step 2: Installing MongoDB on Cluster
We start with creating MongoDB namespace by using the following command:
$ kubectl create namespace mongodb
Then we will add MongoDB repo by following this command:
$ helm repo add bitnami https://charts.bitnami.com/bitnami
Now we will install MongoDB in the MongoDB namespace:
$ helm install mongodb bitnami/mongodb -n mongodb
This will provision Persistent Volume, Persistent Volume Claim, ReplicaSet, Deployment, and pod.
To validate the installation:
$ kubectl get all -n mongodb
|PORT(S) 27017/TCP||AGE 35s|
We also have a Persistent Volume claim of 8gb attached to MongoDB pod to persist data over container restarts. MongoDB will automatically use DO block storage as a Persistent Volume.
We can verify that by:
$ kubectl get pvc -n mongodb
K10 automatically discovers MongoDB instances, and you will see the data and associated artifacts for this instance.
Step 3: Backup and Restore using Kasten K10
Here comes the main point of this blog post; you will see how easy it is to backup and restore using Kasten K10 in a Kubernetes cluster. K10’s default backup mechanism uses storage volume snapshots. K10 supports several backup consistency levels including logical database level backups as well as exporting to external target storage systems such as object stores. In this post, we will cover the local volume snapshot option. A backup may be created manually or on a schedule through a backup policy.. DigitalOcean Block Storage is exposed in Kubernetes through the community-defined Container Storage Interface (CSI). To configure K10 to use CSI, we need to annotate the existing VolumeSnapshotClass.
Following command will show us the storage class name that is attached to MongoDB instance:
$ kubectl get pvc -n mongodb
Now we can annotate the do-block-storage by the following command:
$ kubectl annotate volumesnapshotclass do-block-storage
For more information on CSI, please see the documentation.
Create a restore point (Manual Backup)
On the MongoDB instance, click “snapshot” to manually backup.
On the dashboard, you can see the progress of the backup.
Wait for the operation to complete, then you will see one restore point available.
Click on the restore point you want to restore to. If there is more than the one, you will see all the restore points here. Clicking on Today, restore point will start operation to restore.
You will see the process in the dashboard as it starts the restoring process.
As you can see here, restoring data is completed.
Using Backup Policies
We have seen the workflow of backup using the manual method, another method is to use the backup policies to make daily, weekly, or hourly backups. You can set the backup and snapshot retention schedule independently for detailed control over how often backups are performed and the amount of total storage they consume.
Click on create policies. You will see the policy creation sidebar. Policies are extremely configurable, and multiple options are available.
When a policy is applied to the application, this executes a backup for the first time. The application’s compliance with the policy is reported in the application card. You will see the application is then compliant with all the policies.
This blog post shows how easy it is to use Kasten K10 in your Kubernetes environment. How you can back up your application data with a single click and manually create a restore point. You can also create policies in the production environment; policies create an hourly, daily, or monthly backup based on a defined policy, without manual intervention. You can even create one policy for multiple applications. In this example, we have used DigitalOcean managed Kubernetes service to deploy a MongoDB instance and used this instance to create a backup, but you may use any SQL (PostgreSQL or MySQL) or NoSQL database (Elastic).
Tom made the jump from hardware into software at his first role in 2013. He worked on the server team at Maginatics, cloud-based file system company which was acquired by EMC late in 2014. After the acquisition, he joined Dropbox where he focussed on improving the efficiency and reliability of Dropbox's databases. In 2017, Tom joined the founding team at Kasten, a startup solving storage problems in cloud native environments. He now manages teams whose work includes Kanister, an open source execution framework for Kubernetes and K10, an enterprise data management solution. He's an active member of the Kubernetes Data Protection Working Group.