Using the Crunchy Data PostgreSQL Operator - A Walkthrough

What is an Operator?

The question should really be "What is a Kubernetes Operator?" since the word "operator" appears in many contexts.

In the Kubernetes context, an Operator abstracts away the Kubernetes resource configurations for a specific application run on Kubernetes.  It deploys, manages and monitors those resources. There are operators for all sorts of things - logging, databases, even CI/CD integration. 

More specifically, the Operator creates a Kubernetes Custom Resource Definition (CRD) for specific technologies.

What does the Postgres Operator do?

Crunchy Data provides enterprise Postgres support to clients. The Postgres Operator is their community edition for Kubernetes Postgres applications.

The Postgres Operator deploys Postgres databases on your Kubernetes cluster. Databases have very specific requirements compared to other applications, persistence being chief among them. The Operator leverages Kubernetes Persistent Volumes for this purpose. It exposes a Service for communication inside the cluster, and it uses Secrets for database user authorization purposes. Oh, and it creates Pods that run your actual database.

Thanks to the Operator, we don't have to worry about making all of the cluster level resources work together - the operator does that for us.

To talk to the operator, the user gets a command line tool called pgo, and several scripts to deploy the Operator onto a working Kubernetes cluster.

The PostgreSQL Operator can:

• Deploy Postgres clusters of the latest version with all the bells and whistles
• tag metadata to the database clusters
• control user access
• backup databases, both from snapshot and by timestamp
• Collect and store metrics
• be customized further since it is an open source tool.

A note of disambiguation: The term "Postgres cluster" is a Postgres-specific term that refers to the particular way of how Postgres databases are set up for access and maintenance. It has absolutely nothing to do with "Kubernetes cluster". In my head I think of "Postgres cluster" as "Postgres database".

Installing and using the Operator

I wanted to learn about how to install and interact with Operators, so I explored this one's user documentation and ran through the demo installation. I created a walkthrough video with the help of a coworker. In the video, I deploy a Postgres Operator onto a Minikube Kubernetes cluster using the following steps:

1. Set up a cluster
2. Choose and set a namespace
   
3. Set env vars for make
4. Run make pull and make deployoperator
5. Get pgo binary from source but realize binary names differ by OS

Note: The pgo binary download is a single download that comes with separate binaries for Mac, Linux, and Windows, as well as a number of configurations. In my demo, I alias "pgo-mac" to "pgo" to be consistent with the documentation.

To simulate how an app would interact with a database, I create a separate Pod, with an init-container that creates the database, and a postgres container to simulate the app that would access the database. I also run a Cron Job to create periodic backups of my database. The source code for that lives here. Here are the steps:

1. The app accessing the database will run in a pod that has an init container that sets up the database with pgo
2. centos for the init container, postgres for the database app
3. Kubernetes Cron Job will backup every 5 minutes (for demo purposes)
4. Exec into database app container (mypod) to interact with database (in database pod)
5. Secrets have been set up as env vars so you can immediately access postgres without login

For restoring a database from a backup, I run pgo locally from my computer instead, and port-forward the Service IP address to my localhost. Here are the steps:

1. Configure pgo client
2. kubectl port-forward operator service and database service
3. Create backup following these instructions
4. Exec into mypod to access restored database and see restored data

Here is the video of the walkthrough, with comments from a coworker, who preferred to remain anonymous. Keep in mind that it's a live walkthrough, with little glitches and minor bumps; please feel free to click around. I hope to put some breaks into the video in the near future to make it more user friendly.

Wrapup and notes

• Remember pgo has different names depending on OS
• Postgres clusters are not Kubernetes clusters
• Postgres cluster names need to be unique - if you try to deploy another cluster with the same name, nothing will happen 
• Must have Mercurial installed on machine (I ran into this for one of the dependencies)
• Although there is a separate script to create persistent volumes on the host file system, I did not use it, and relied instead on the default hostpath (for which no extra steps are needed). For actual production databases, the script is a handy reference for a more customized setup.

Would I use this Operator?

The short answer is yes.

The documentation is fairly extensive. There is even a helm chart to deploy the operator. The database backup and restoring worked perfectly every time. There are a lot of supported configurations that look really useful, metrics and logging being among them. I started looking into this tool right as it was undergoing a major version upgrade, and I ran into some documentation hiccups and a few bugs (that may or may not have been version related). When I mentioned these concerns, the code maintainers were immediately on top of it, and assistance and fixes were underway. Of course, when it was possible for me to do so, I submitted some doc changes myself.

The folks at Crunchy Data are friendly, responsive and easy to work with. I really appreciated their receptiveness to my doc changes and their willingness to explain and follow up on bug reports. Additionally, there are quite a few Crunchy Data Postgres Operator resources on youtube so I encourage folks to run a quick search and check those out.