kubectl version
To get the versions of kubectl and your Kubernetes cluster, which requires querying its APIs, run this:
$ kubectl version --short
Client Version: v1.24.3
Kustomize Version: v4.5.4
Server Version: v1.23.3
You should ensure that the versions of kubectl and the Kubernetes cluster are close to each other; otherwise, strange behavior might occur.
Managing Resources
Now let’s move on to commands that actually perform some actions on your cluster, such as creating, updating, and deleting resources.
Creating Objects
To create Kubernetes objects, you can use imperative commands, but as we saw earlier, this is not recommended except to quickly create a resources while doing some development. As an example, here is how to create a pod using imperative mode:
$ kubectl run www --image=nginx
This will create a pod named “www” running the “nginx” image. As you can see, some object types use specific commands, and complex object specifications will be nearly impossible to specify using the command line only. If you want to learn more about imperative commands, Kubernetes has a dedicated page for them.
Here is another example of an imperative creation, this time for a deployment:
$ kubectl create deployment www --image=nginx --replicas=3
deployment.apps/www created
$ $ kubectl get pod
NAME READY STATUS RESTARTS AGE
www-6d49b97f5-ct7cn 1/1 Running 0 50s
www-6d49b97f5-jb689 1/1 Running 0 50s
www-6d49b97f5-nggsm 1/1 Running 0 50s
You probably noticed that this time, we used the standard kubectl create
command. Most objects can be created imperatively using the kubectl create
command, but a few (like pods) have their own commands.
The recommended method is to use the declarative mode like this:
$ kubectl apply -f MANIFEST.YAML
The manifest file is a YAML-formatted file. Here are its specifications. Using a manifest file allows you to better track the changes you make to your cluster, and such files can be kept in revision control. As an example, here is the manifest file for a pod:
apiVersion: v1
kind: Pod
metadata:
name: debug
namespace: myns
spec:
containers:
name: debug
image: nicolaka/netshoot
command: [sh, -c, while true; do sleep 60; done]
This example is actually very useful because the “nicolaka/netshoot
” image contains a lot of networking-related tools that are very helpful for debugging connectivity issues inside a cluster.
Pro-tip: After creating some CRDs, you won’t be able to immediately create objects of the types defined by these CRDs. This is because Kubernetes has a race condition internally, and you will need to wait for a few seconds before attempting to create objects of the new types.
Updating Objects
Again, it is possible to update objects using imperative mode. There are actually four different ways to effect changes to objects.
First, you can make the changes directly using the edit
command:
$ kubectl edit deploy/www
This will fire up your favorite text editor with the manifest and status of the object you selected. You will then need to make the desired changes directly, and once you exit and save, Kubernetes will apply the changes.
The second way is to use the replace
command, which works like this:
$ kubectl get deploy/www -o yaml > tmp.yaml
$ myeditor tmp.yaml # make some changes
$ kubectl replace -f tmp.yaml
Please note that the replace command is not equivalent to the apply command, even though they look very similar.
Pro-tip: Not everything can be modified on an existing Kubernetes object. Sometimes you will need to delete and recreate the object.
The next imperative way to modify an object is to use the patch
command, which works like this:
$ kubectl patch deployment/www -p '{"spec": {"replicas": 2}}'
deployment.apps/www patched
$ k get pod
NAME READY STATUS RESTARTS AGE
www-6d49b97f5-ct7cn 1/1 Running 0 82s
www-6d49b97f5-nggsm 1/1 Running 0 82s
The -p option accepts both YAML and JSON, although you will struggle to type any YAML on the command line.
Finally, you can use the set
command to make direct changes to objects. This command is quite idiosyncratic, so very few people actually use it and we won’t cover it here. If you are interested in learning more, just run kubectl set --help
.
Once again, we recommend that you use the declarative method via the apply -f
command. It is actually simpler because you just need to edit the manifest file you used to create the objects in the first place and run the same command again:
$ kubectl apply -f MANIFEST.YAML
As you can see, this is mucher easier, less error-prone, and allows you to revision-control your code. You can apply all the manifest files in a given directory at once as well:
$ kubectl apply -f DIR
You can see that the same command, kubectl apply -f
, is used both to create and update objects. Kubernetes will compute the differences between the new manifest files and the current state of the objects and will apply the differences intelligently. It should be noted that, as discussed previously, this method won’t delete objects that you removed from the manifest files.
If you used kustomization files, you should use the -k option rather than -f (and the argument must be a directory):
$ kubectl apply -k DIR
Pro-tip: Unfortunately, when making changes to role-based access control (RBAC) permissions, Kubernetes requires a bit more gymnastics. The reasons for this are complex and outside the scope of this article. Just remember to run kubectl auth reconcile -f MANIFEST.YAML
before running kubectl apply -f MANIFEST.YAML
whenever you make some changes to RBAC objects.
Deleting Objects
At the time of writing, the only way to delete objects is with the imperative delete
command. You can still use the -f flag and pass the manifest files you used to create/update the objects in the first place, so that makes your life a bit easier:
$ kubectl delete -f MANIFEST.YAML
Pro-tip: Use the --now
flag to speed up the delete command. This is especially useful when deleting pods; otherwise, it takes Kubernetes quite a while to go through the motions to actually delete the pod.
Commands for Deployments
Deployments in Kubernetes receive special treatment with some added bonuses. When you update a deployment — and the same applies to daemon sets and stateful sets — you can view the status of the update using:
$ kubectl rollout status deploy/www
If the update is still in progress, you can cancel it like so:
$ kubectl rollout undo statefulset/mydb
You can view the update history using this:
$ kubectl rollout history sts/mydb
Next, you can manually modify the number of replicas using the following imperative command:
$ kubectl scale --replicas=N deployment/www
Now, in practice, it is unlikely that you will use these commands. You will be much more likely to use a higher-level tool such as Helm to manage such deployments, and you will probably set up some sort of autoscaling for your deployments rather than manually scaling them out and in.
Debugging
In Kubernetes, things sometimes go wrong, especially during the development phase. However, problems can occur even in production, so it’s very useful to know how to debug issues in your cluster.
In many cases, it makes sense to first try to get a high-level picture of the state of various objects. You can do this using the describe command that we saw earlier. To check a pod, for example, you would run:
$ kubectl describe pod/MYPOD
You might also want to manually check what a given pod or service is serving; the port-forward command is very helpful here. For example, to access the service MYSVC running on port 80 via port 8000 of your computer, you would run the following:
$ kubectl port-forward service/MYSVC 8000:80
You could then manually run some requests to port 8000 on your local computer, for example using curl or Postman.
Next, you probably want to know what a misbehaving pod is doing, and the first port of call is to check its logs, which would be done like so:
$ kubectl logs MYPOD
You can also add the -f flag for the kubectl logs
command to continuously poll the logs in real time and display them as they come:
$ kubectl logs -f MYPOD
One of the most useful debugging commands is kubectl exec
, which allows you to run commands inside a suspicious pod. The drawback of this command is that the program you want to execute must exist inside the pod. Here is a typical example to start a shell to allow you to explore what’s inside the pod:
$ kubectl exec -it www-6d49b97f5-j4dlb -- /bin/sh
root@www-6d49b97f5-j4dlb:/#
If you have more than one container running inside the pod, you might need to specify which container you want to execute the command in using the -c flag:
$ kubectl exec -it www-6d49b97f5-j4dlb -c istio-proxy -- /bin/sh
istio-proxy@www-6d49b97f5-j4dlb:/$
Another command that is sometimes useful is kubectl cp
, which allows you to copy files in and out of the pod.
Other Helpful kubectl Commands
Now let’s have a look at some helpful kubectl commands.
First of all, it’s often useful to see what containers are running in a pod. Here’s how to get a list of the names of a pod’s containers:
$ kubectl get pod www -o jsonpath='{.spec.containers[*].name}'
nginx istio-proxy
You can build some pretty complex expressions using jsonpath. Here’s an example to get the name of the image for a given container:
$ kubectl get pod www \
-o jsonpath='{.spec.containers[?(@.name=="istio-proxy")].image}'
docker.io/istio/proxyv2:1.13.6
If your Docker images are pulled from a private registry, Kubernetes will need some credentials to connect to the registry and pull your images. Here is a quick solution suitable for a development environment (not a production one, though).
First, manually login to the Docker registry using the docker login command. Then run the following:
$ kubectl create secret generic SECRET_NAME \
--type=kubernetes.io/dockerconfigjson \
--from-file=.dockerconfigjson=$HOME/.docker/config.json
You will obviously need to replace the path to your Docker config.json file for your own operating system.
Possibly the most useful command of all is, of course, the “help” command:
$ kubectl --help
To get help on a specific subcommand, add -–help
after the command name. For example:
$ kubectl logs --help
Kubectl’s help is quite well-written and accessible. Make it your first port of call when you have questions about kubectl.
Finally, here is a very useful command to test your RBAC rules:
$ kubectl --as=system:serviceaccount:NAMESPACE:SERVICE_ACCOUNT \
auth can-i get secret/SECRET_NAME
yes
The example above will test whether the specified service account can get the given secret. Of course, you will need to replace the capitalized names with what is relevant for your setup.