Skip to content

Category: Containers and Kubernetes

Google Next 2017, Amsterdam Edition

On June, 21 there was the “Google NEXT” conference, 2017 edition, in the Kromhouthal in Amsterdam. Google had a dedicated ferry running to ship people over to the IJ north side, delivering directly at the Kromhouthal.
 
The event was well booked, about 1400 people showing up (3500 invites sent). That is somewhat over the capacity of Kromhouthal, actually, and it showed in the execution in several places (Toilet, Catering, and room capacity during keynotes).
 
The keynotes were the expected self-celebration, but if you substract that, they were mostly useful content about the future of K8s, about Googles Big Data offerings and about ML applications and how they work together with Big Data.
 
For the two talk slots before the lunch, I attended K8s talks. After lunch, I switched to the Big Data track. I did not attend any ML stuff, and I missed the last talk about Spanner because I got sucked into a longer private conversation.
Leave a Comment

Scaleway now with 2FA

Cloud Provider Scaleway now has ARM64 based bare metal in Amsterdam. They are also now offering 2FA auth based on Google Authenticator (or other, compatible 2FA apps).

No U2F token support, yet, though (but still a better solution than steam).

This blog is hosted on a Scaleway instance.

Leave a Comment

“Usage Patterns and the Economics of the Public Cloud”

The paper (PDF) is, to say it in the words of Sascha Konietzko, eine ausgesprochene Verbindung von Schlau und Dumm (“a very special combination of smart and stupid”)

The site mcafee.cc is not related to the corporation of the same name, but the site of one of the authors, R. Preston McAfee.

The paper looks at the utilization data from a number of public clouds, and tries to apply some dynamic price finding logic to it. The authors are surprised by the level of stability in the cloud purchase and actual usage, and try to hypothesize why is is the case. They claim that a more dynamic price finding model might help to improve yield and utilization at the same time (but in the conclusion discover why in reality that has not happened).

Leave a Comment

A case for IP v6

So when companies talk about IP V6, it is very often at the scope of “terminating V6 at the border firewall/load balancer and then lead it as V4 into the internal network. Problems that arise there are most often tracking problems (»Our internal statistics can’t handle V6 addresses in Via: headers from the proxy«).

But when you do containers, the need for V6 is much more urgent and internal. Turns out that Docker Port Twiddling is exactly the nuisance that it looks like and networkers strongly urge you to surgically remove all traces of native Docker networking bullshit and go all in on IP-per-Container. Mostly, because that’s what IPs are for: Routing packets, determining their destination and stuff. Networkers have ASICs and protocols that are purpose-built for this stuff.

Now, let us assume you have a modern 40- or 56-core machine that you are running stuff on in your Kubernetes cluster. It means that you will easily at least 30 and up to 100 pods per machine. In a moderately sized cluster with some 100 nodes you get to use 100×100, 10.000 IPs to handle that. And because IP space is not handed out in sets of one, but in the form of subnets per node, you will have need for more than 10k addresses. Expect to consume a /17 or /16 to handle this.

Even if you are digging into 10/8 for internal addressing here, this is going to be a problem – it’s unlikely that you will be able to use all of 10/8, because non-cluster things exist, too, in your environment, and you will likely have more than one cluster.

With V6, things are becoming a complete non-issue, with the minor issue of getting V6 running on the inside of your organisation.

Leave a Comment

Cloud Costs

Cloud cost models are sometimes weird, and billing sometimes is not quite transparent. The cost model can also change at will.

The Medium story reported by Home Automation is an extreme example, and contains a non-trivial amount of naiveté on their side, but underlines the importance of being spread through more than one cloud provider and having an exit strategy. Which is kind of a dud, if you are using more than simple IaaS – if you tie yourself to a database-as-a-service offer, you can’t really have an exit strategy at all.

 

TL;DR: Firebase accidentally wasn’t billing some traffic, and fixed that (the billing). They did not communicate the change, they did not update their status panels to report the increased traffic, and they did not measure the billing impact of their change to find extreme cases before the change and contact them.

The customer, Home Automation, has close to zero clue to using TLS correctly, was using connection inefficiently and kind of maximised overhead, ran into the worst case scenario for the change, got fucked. They would want out, but also had zero strategy for that, because DBaaS fuckup.

In the cloud you don’t need operations. Until you do.

4 Comments

Toybox: Writing a new command line from scratch

Rob Landley, of Busybox/Toybox fame, spoke four years ago about the Toybox project in the context of Android and whatever else was recent back then. The talk contains a brilliant deconstruction of the problems that GPL v3 has, and why it is in decline.

It also shows a lot of vision re containers, and what is needed in this context. If you are deploying Alpine today, with musl and toybox in it, here is why and how it came to be.

Leave a Comment

jq

When dealing with Kubernetes, you will inevitably have to deal with config and data that is in JSON format.

jq is a cool tool to handle this, but while the man page is complete, it is also very dry. A nice tutorial can be found at The Programming Historian, which uses some real world use cases. My personal use case is Converting JSON to CSV, and the inverse of that. There also is a mildly interesting FAQ.

Learning jq takes about one quiet afternoon of time.

4 Comments

Understanding sysdig

The open source sysdig is a piece of software that does not quite, but almost, what strace or oprofile do: It instrument the kernel, and traces system calls as well as a few other kernel activities.

It does not utilize the ptrace(2) kernel facility, though, but its own interface. This interface picks up data in the kernel and writes it into a ring buffer.

A userspace component extracts this data, interprets, filters and formats it, and then shows it. If the data source outpaces the userspace, the ring buffer overflows and events are lost, but the actual production workload is never slowed down.

Leave a Comment