Die wunderbare Welt von Isotopp

Fertig gelesen: The Vanishing Middle Class

Kristian Köhntopp - April 7, 2021
The Vanishing Middle Class: Prejudice and Power in a Dual Economy by Peter Temin analyzes the economical and political structure of the USA. Temin does this through the lens of the “Dual Economy ” Model. Dual Economies are normally prevalent in colonial societies of “less developed” countries. These countries usually have local production and production for a global export market, and besides economical and income structures around this separation, there are also social segration and a certain degree of isolation of the social groups from each other - in politics as well as attempts to prevent social migration into the higher ranked group.

MySQL and UUIDs

Kristian Köhntopp - April 6, 2021
In ALTER TABLE for UUID we discuss currently proper way to store and handle UUID in MySQL. Currently it works, even in a performant way, but it still hurts. It should not. Definition of UUID The RFC 4122 defines various types of UUID, and how they are being formatted for presentation and as a bit field on the wire. As this document was written bei Leach and Salz, among others, RFC 4122 UUIDs are also called “Leach-Salz UUIDs” (for example in the Java Documentation ).

Making an unexpected leap with interval syntax

Kristian Köhntopp - April 2, 2021
(based on a find by Ruud van Tol, and several Twitter contributions) Ruud commented on our DST discussion with mysql> SELECT '2019-02-28 12:34:56'+ INTERVAL 1 YEAR + INTERVAL 1 DAY as a, '2019-02-28 12:34:56'+ INTERVAL 1 DAY + INTERVAL 1 YEAR as b\G a: 2020-02-29 12:34:56 b: 2020-03-01 12:34:56 2019 is a year before a leap year. Adding (left to right) a year brings us to 2020-02-28, and then adding a day makes this 2020-02-29, because it’s a leap year.

Things you didn't know about MySQL and Date and Time and DST

Kristian Köhntopp - March 29, 2021
(based on a conversation with a colleague, and a bit of Twitter ) A Conundrum A developer colleague paged me with this: mysql> select UNIX_TIMESTAMP("2021-03-26 03:07:00" + INTERVAL 2 YEAR) - UNIX_TIMESTAMP("2021-03-26 02:07:00" + INTERVAL 2 YEAR) as delta\G delta: 420 It is obviously wrong, and weirdly so. It only works for “2 year”, not with other values: mysql> select UNIX_TIMESTAMP("2021-03-26 03:07:00" + INTERVAL 1-11 year_month) - UNIX_TIMESTAMP("2021-03-26 02:07:00" + INTERVAL 1-11 year_month) as delta\G delta: 3600 mysql> select UNIX_TIMESTAMP("2021-03-26 03:07:00" + INTERVAL 1-12 year_month) - UNIX_TIMESTAMP("2021-03-26 02:07:00" + INTERVAL 1-12 year_month) as delta\G delta: 3600 mysql> select UNIX_TIMESTAMP("2021-03-26 03:07:00" + INTERVAL 1-13 year_month) - UNIX_TIMESTAMP("2021-03-26 02:07:00" + INTERVAL 1-13 year_month) as delta\G delta: 3600 It has to be exactly 730 days (2 * 365 days, 2 years):

That's a lot of databases

Kristian Köhntopp - March 24, 2021
Where I work, we are using MySQL a lot. The databases are being organized in replication hierarchies, and each hierarchy is a tree topology with a single primary and a number of intermediate replicas. Replication is a tree managed by Orchestrator We are using MySQL orchestrator to manage the replication topology. MySQL Orchestrator shows a typical replication hierarchy. Each color indicates a different data center/availability zone. Replication is a tree, from the primary to per-AZ intermediate replicas for fan-out to leaf replicas.

Memory saturated MySQL

Kristian Köhntopp - March 12, 2021
»If at all possible, we build databases so that the working set of the database fits into memory.« What does that even mean? Working Set In computer science, the “Working Set” of a program is the set of things it will be accessing in the near future. Because computer science has not yet solved looking into the future, we are looking at the set of things we accessed most recently and hope for The Best™.

Rechenaufgaben lösen

Kristian Köhntopp - February 28, 2021
Das Kind möchte ein Programm zum Üben von Rechenaufgaben sehen. Nun gut. Hier ist eine Version in PyQt5. Unsere Oberfläche soll so aussehen. Wir wollen ein kleines Fenster, in dem eine billig generierte Rechenaufgabe angezeigt wird. Der Schüler soll die Antwort eingeben und den Knopf “Antworten” drücken. Danach wird angesagt, ob die Antwort korrekt war, oder ob sie falsch war. Wenn sie falsch war, wird auch die korrekte Antwort angezeigt.

MySQL from Below

Kristian Köhntopp - February 25, 2021
When you insert data into a database and run COMMIT you expect things to be there: Atomically, Consistent, Isolated and Durable , like Codd commanded us 40 years ago, but also quickly. There is a surprising amount of sophistication being poured into this, but since I do not want to shame MongoDB and Redis developers in this post, I am not going to talk about that much in this place.

Validating storage

Kristian Köhntopp - February 24, 2021
Where I work, we try to run databases in a memory saturated way. That is, we try to provide so much memory that the working set of the database is memory resident, or in other words, the number of disk reads after an initial warmup is no longer dependent on the database load. Workload Intelligence Analytics showing “IOPS over time” for a mixed read/write benchmark on Datera iSCSI. We can validate and prove that with automated load testing: For each replication chain we single out a production host, and increase the hosts weight in the load balancer until the system load1 becomes critical.

Database as a Queue

Kristian Köhntopp - January 28, 2021
The DBA experience at work suggests that every single schema at some point in its lifecycle holds a queue table. These are tables in which some processes (the “producers”) put rows, which a swarm of other processes (the “consumers”) lock and consume. A variation on that theme is the state machine, in which jobs are placed by producers. Consumers do not immediately delete them, but update them a few times to indicate processing progress, before the rows are ultimately being deleted.

Unlimited Mail Addresses

Kristian Köhntopp - January 20, 2021
When people ask for my mail address, they usually get a personalized address from me. That is particularly true for all commercial email. So you don’t get to send mail to my main account, but to kris-yourbusiness@koehntopp.de , and that will end up going into INBOX.special.yourbusiness. At least until it leaks, receives spam or is otherwise burned. In which case I will short it out and route all incoming mail on that address to /dev/null.

940.000 User in Baden-Württemberg

Kristian Köhntopp - January 12, 2021
Deutschland ist im Lockdown, die Schulen sind endlich geschlossen und es wird remote unterrichtet. Weil es Deutschland ist, passiert das in jedem Bundesland anders und uneinheitlich. In Baden-Württemberg verwendet man Moodle . Wer sich da drunter nichts vorstellen kann, kann es sich hier ansehen. In Bawü wird eine getrennte Moodle-Instanz pro Schule installiert, aber halt viele Instanzen pro Server, weil Server recht groß sind. In Summe muß man bummelig 940.000 Schüler abfrühstücken.

Labyrinths (in Python)

Kristian Köhntopp - January 10, 2021
My son wants labyrinths. Ok, let’s make them like it’s the first semester. We need a class Labyrinth that holds the dimensions of our maze, and the actual maze, in grid - a List of Lists of integers. That’s not exactly a two-dimensional array (it can have a ragged right edge), but it will do for us. A basic container for labyrinths class Labyrinth: """Store a labyrinth as a List of Lists of Integers.

Using Python to bash

Kristian Köhntopp - January 5, 2021
Heise writes an introduction to bash programming (in german) : Bash ist eine vollwertige Programmiersprache, mit der Sie alltägliche Aufgaben leicht automatisieren. Bash is a fully featured programming language that you can use to automate everyday tasks. Bash is not a fully featured programming language at all, and nothing in bash is ever easy. You are advised to use a proper programming language early on in development, and if possible never put bash commands into a file.

fork, exec, wait and exit

Kristian Köhntopp - December 28, 2020
This is the english version of a 2007 article . In de.comp.os.unix.linux.misc somebody asked: Are commands in a script executed strictly sequentially, that is, will the next command only be executed when the previous command has completed, or will the shell automatically start the next command if the system has spare capacity? Can I change the default behavior - whatever it may be - in any way? If you are looking into the fine manual, it may explain at some point that the shell starts each command in a separate process.

SQL Clause is coming to town

Kristian Köhntopp - December 26, 2020
Olya Kudriavtseva has an ugly Christmas sweater : He’s making a table. He’s sorting it twice. SELECT * FROM contacts WHERE behavior = “nice”; SQL Clause is coming town! (buy here ) Katie Bauer observes : I mean, except for the fact that sorting something twice is TERRIBLY optimized So how bad is this? Let’s find out. Some test data We are defining a table santa, where we store peoples names (GDPR, EU Regulation 2016/679 applies!

Der Testing-in-Production-Blues

Kristian Köhntopp - December 17, 2020
Kalenderwoche 51/2020: Lockdown mit Schulschließungen. Nachdem es im März schon einmal Schulschließungen wegen Corona gab, und dort die Defizite technischer und organisatorischer Natur offenbar wurden, hat man in Deutschland die Zeit genutzt und sich auf die vorhergesagte 2. Welle vorbereitet, die jetzt genau eingetroffen ist. Deutschland ist schließlich nicht nur das Land der Dichter und Denker, sondern auch ein Land der Ingenieure und Tüftler, und bekannt für seine funktionierende und effiziente Bürokratie.

Embracing the Stream

Kristian Köhntopp - December 9, 2020
So this happened: CentOS Project shifts focus to CentOS Stream The future of the CentOS Project is CentOS Stream, and over the next year we’ll be shifting focus from CentOS Linux, the rebuild of Red Hat Enterprise Linux (RHEL), to CentOS Stream, which tracks just ahead of a current RHEL release. CentOS Linux 8, as a rebuild of RHEL 8, will end at the end of 2021. CentOS Stream continues after that date, serving as the upstream (development) branch of Red Hat Enterprise Linux.

Not JOINing on PERFORMANCE_SCHEMA

Kristian Köhntopp - December 1, 2020
The tables in PERFORMANCE_SCHEMA (P_S) are not actually tables. You should not think of them as tables, even if your SQL works on them. You should not JOIN them, and you should not GROUP or ORDER BY them. Unlocked memory buffers without indexes The stuff in P_S has been created with “keep the impact on production small” in mind. That is, from a users point of view, you can think of them as unlocked memory buffers - the values in there change as you look at them, and there are precisely zero stability guarantees.

MySQL: Backups and Replication

Kristian Köhntopp - November 27, 2020
There was a question at work about MySQL backups and restore. I needed to explain more. We use databases to make state persistent. That is: As a developer you can think of your database as a single giant, structured global variable with a weird access method, and to make things worse, concurrent access. A database is just a global variable to your code We can log statements that change the state of our database in a log.