Adopting New Technologies - A Lesson

At dinner the other night with a former colleague of mine, he shared some war stories from the trenches with Kubernetes. Don’t get me wrong: I think Kubernetes is incredibly powerful and useful. But, it brings a lot of complexity and with complexity comes trouble.

So, what did he share? He’s working at a very cool startup that has not yet launched their product/platform. But, they are up and running with a few folks using the platform. And, they use kubernetes and manage the kubernetes cluster themselves. That is, they are not leveraging a kubernetes system managed by their cloud provider, where they just have to define what to run in their cluster. Apparently, doing it this way is more trouble than many may realize when they choose to go this path.

So, in this situation where they are not yet even fully launched, they’ve already hit issues with network connection delays, dropped network black holes (packets never reach their destination), and intermittent connection resets. These are bugs that have existed for quite a while. The time it took him to diagnose the symptoms he saw and identify that they were in fact bugs with k8s was significant, when what he really wanted to be doing was finishing up the platform so they could launch. His plan now is to vastly simplify how they use kubernetes networking to avoid these and other potential issues.

It’s not surprising that bugs like this exist. It’s actually quite natural to expect that. The problem is that too often we adopt new technologies without regard to the likelihood that we will encounter such issues. That is, we systematically underestimate the cost of such decisions. Keep this in mind when considering adopting new technologies, whether kubernetes, the latest distributed database, or whatever.

Thank You!

Your comment has been submitted. It will appear on this page shortly! OK

Yikes, Sorry!

Error occured. Couldn't submit your comment. Please try again. Thank You! OK

Only 1 comment on Adopting New Technologies - A Lesson

  1. Victor Mendiluce's avatar Victor Mendiluce says:

    Yeah, networking is where a lot of Kubernetes’ complexity lies, which lends to some of the nastier bugs. The CNI architecture, multiple network plugins (Flannel, Weave, Calico…) with different approaches and trade-offs which make the decision complex.

    Also, choosing a “Kubernetes as a service” provider may be fine for some companies, and I’m sure that at least gCloud solves a lot for you. But some companies can’t just do that for business reasons, and they have to fight all the installing, running and upgrading complexities by themselves.

    Leave a comment