Checking the deployed tools
Before we get our hands into a fully orchestrated data pipeline, we need to make sure that all the necessary operators are correctly deployed on Kubernetes. We will check for the Spark operator, the Strimzi operator, Airflow, and Trino. First, we’ll check for the Spark operator using the following command:
kubectl get pods -n spark-operator
This output shows that the Spark operator is successfully running:
NAME READY STATUS spark-operator-74db6fcf98-f86vt 1/1 Running spark-operator-webhook-init-5594s 0/1 Completed
Now, we will check Trino. For that, type the following:
kubectl get pods -n trino
Check if all pods are correctly...
