CatOps
5.73K subscribers
94 photos
4 videos
19 files
2.21K links
DevOps and other issues by Yurii Rochniak (@grem1in) - SRE @ Preply && Maksym Vlasov (@MaxymVlasov) - Engineer @ Star. Opinions on our own.

We do not post ads including event announcements. Please, do not bother us with such requests!
Download Telegram
Some good practices for Argo Workflows. I'm not super familiar with this tool, so I'd just list the points here:

- Use common templates
- Limit parallelism to control cluster resources
- Avoid nested loops
- Collect metrics from your workflows
- Use lifecycle hooks
- Control long-running workflows or break down long workflows into separate smaller ones
- Integrate with Argo Events

#argocd #cicd #kubernetes
Fixing a memory leak in Kubernetes with a single line of code.

I love such debug stories because you can read them almost as a thriller or a detective literature. Also, this case is very interesting indeed and it took that team a couple of months to figure it out.

A takeaway for myself:
- It‘s useful to have a dashboard (and potentially a monitor) that shows the difference of the node memory usage and the memory usage by pods.

#kubernetes #postmortem
A comparison between EKS and AKS.

Tors article provides some insights into what to expect from each managed service. It’s a pity, GKE is not included into this comparison, I’ve heard a lot of good things about GKE. I believe, this is because this article originated from a specific use-case.

#kubernetes #aws #azure
AWS Karpenter is in beta now.

Karpenter is AWS’ tool to manage and autoscale node pools in Kubernetes, which has more features compared to the Cluster Autoscaler and has deeper integration with AWS features. Yet, support for other major clouds is somewhere in the roadmap, IIRC.

This article in particular, describes what are the changes in Karpenter Beta compared to the previous versions and also guides you through depreciations and upgrade notes.

#kubernetes #aws
An article that could be a tweet (xeet?) but with more context.

tl;dr: avoid Helm hooks when possible.

I totally agree with the statement, yet an example in this article is rather weak. Anyways, if you need to have some imperative actions for your app, it’s always better to use higher level abstractions like ordering on the CD tool level or even an operator.

Having an init container that has access to the API is rather a security concern than a workaround.

#kubernetes
Here's a neat article with some good practices regarding security when configuraing an EKS cluster.

If you work with AWS and Kubernetes a lot, it won't give you any dramatic insights, but you could still use it as a checklist / cheat sheet when configuring a cluster, since it's easy to forget something when there are many moving parts.

#aws #kubernetes
This article with a clickbait title got my attention recently. It was even translated in Ukrainian by one of the largest Ukrainian developers-oriented media.

The most interesting part of this article, in my opinion, isn’t its premise and even not the points that the author is making (some of those points are 5 years late, TBH). The thing that caught my attention was what the author chooses to compare Kubernetes to. And those are managed platforms like Heroku, etc.

This is interesting for a couple of reasons: first of all it seems like other orchestration solutions are out of the table already. Second, it reaffirms the statement of Viktor Farcic (you can find those in the Den Vasyliev’s blog). Basically, the idea is that in the future Kubernetes will “disappear”. Not in the sense that it will fade away, but in the same way hypervisors have “disappeared”. The majority of us use them today, but we rarely think about what virtualization powers our cloud instances, etc. In the same way at some point there will an extendable API that allows one to run workloads and whatever cluster technology is underneath would be a concern of a cloud provider.

#kubernetes
Aqua Security warn people about the danger of the supply chain attacks using Kubernetes Secrets.

We all know this story: base64 is not an encryption, Kubernetes Secrets may have a ton of sensitive information, etc. The article just provides some data from Aqua’s recent research.

However! There’s another viewpoint on Kubernetes Secrets - Plain Kubernetes Secrets are fine. This is the thing I wanted to share with you today.

The main gist of this article is that you cannot really tell if something is secure or not without a threat modeling. Also, that the auto-unsealing feature in Vault kinda negates some if its security features.

Apparently, there’s a discussion about this article in a form or a podcast but I haven’t checked it out yet.

Also, if you need some guides for threat modeling, OWASP website is a good place to start.

So, do your due diligence, do threat modeling, and have a nice day!

#security #kubernetes
An interesting read by Monzo about how they implemented Kubernetes Network Policies for 1.5k microservices.

There are some questionable parts in there, in my opinion. For example, why building your own tool to "guess" where an app connects to if you could use a network monitoring tool. However, those are not directly related to the main topic.

An interesting part is how folks in Monzo "reverted" the idea behind Network Policies using templating. So, instead of a target services allowing internal connections, a caller can specify the groups of services it wants to connect to.

Although, I think it partially negates the idea of Network Policies, I can completely understand, why Monzo did that from the UX perspective.

Also, here's a Reddit discussion on the topic. I love the top comment there:

 How would you even know that another team plans to connect your apps?
- By communicating...


#kubernetes #networking
Best Bluetooth Speaker Sound Quality