Die wunderbare Welt von Isotopp

Memory saturated MySQL

- March 12, 2021
»If at all possible, we build databases so that the working set of the database fits into memory.« What does that even mean? Working Set In computer science, the “Working Set” of a program is the set of things it will be accessing in the near future. Because computer science has not yet solved looking into the future, we are looking at the set of things we accessed most recently and hope for The Best™.

Rechenaufgaben lösen

- February 28, 2021
Das Kind möchte ein Programm zum Üben von Rechenaufgaben sehen. Nun gut. Hier ist eine Version in PyQt5. Unsere Oberfläche soll so aussehen. Wir wollen ein kleines Fenster, in dem eine billig generierte Rechenaufgabe angezeigt wird. Der Schüler soll die Antwort eingeben und den Knopf “Antworten” drücken. Danach wird angesagt, ob die Antwort korrekt war, oder ob sie falsch war. Wenn sie falsch war, wird auch die korrekte Antwort angezeigt.

MySQL from Below

- February 25, 2021
When you insert data into a database and run COMMIT you expect things to be there: Atomically, Consistent, Isolated and Durable , like Codd commanded us 40 years ago, but also quickly. There is a surprising amount of sophistication being poured into this, but since I do not want to shame MongoDB and Redis developers in this post, I am not going to talk about that much in this place.

Validating storage

- February 24, 2021
Where I work, we try to run databases in a memory saturated way. That is, we try to provide so much memory that the working set of the database is memory resident, or in other words, the number of disk reads after an initial warmup is no longer dependent on the database load. Workload Intelligence Analytics showing “IOPS over time” for a mixed read/write benchmark on Datera iSCSI. We can validate and prove that with automated load testing: For each replication chain we single out a production host, and increase the hosts weight in the load balancer until the system load1 becomes critical.

Database as a Queue

- January 28, 2021
The DBA experience at work suggests that every single schema at some point in its lifecycle holds a queue table. These are tables in which some processes (the “producers”) put rows, which a swarm of other processes (the “consumers”) lock and consume. A variation on that theme is the state machine, in which jobs are placed by producers. Consumers do not immediately delete them, but update them a few times to indicate processing progress, before the rows are ultimately being deleted.

Unlimited Mail Addresses

- January 20, 2021
When people ask for my mail address, they usually get a personalized address from me. That is particularly true for all commercial email. So you don’t get to send mail to my main account, but to kris-yourbusiness@koehntopp.de , and that will end up going into INBOX.special.yourbusiness. At least until it leaks, receives spam or is otherwise burned. In which case I will short it out and route all incoming mail on that address to /dev/null.

940.000 User in Baden-Württemberg

- January 12, 2021
Deutschland ist im Lockdown, die Schulen sind endlich geschlossen und es wird remote unterrichtet. Weil es Deutschland ist, passiert das in jedem Bundesland anders und uneinheitlich. In Baden-Württemberg verwendet man Moodle . Wer sich da drunter nichts vorstellen kann, kann es sich hier ansehen. In Bawü wird eine getrennte Moodle-Instanz pro Schule installiert, aber halt viele Instanzen pro Server, weil Server recht groß sind. In Summe muß man bummelig 940.000 Schüler abfrühstücken.

Labyrinths (in Python)

- January 10, 2021
My son wants labyrinths. Ok, let’s make them like it’s the first semester. We need a class Labyrinth that holds the dimensions of our maze, and the actual maze, in grid - a List of Lists of integers. That’s not exactly a two-dimensional array (it can have a ragged right edge), but it will do for us. A basic container for labyrinths class Labyrinth: """Store a labyrinth as a List of Lists of Integers.

Using Python to bash

- January 5, 2021
Heise writes an introduction to bash programming (in german) : Bash ist eine vollwertige Programmiersprache, mit der Sie alltägliche Aufgaben leicht automatisieren. Bash is a fully featured programming language that you can use to automate everyday tasks. Bash is not a fully featured programming language at all, and nothing in bash is ever easy. You are advised to use a proper programming language early on in development, and if possible never put bash commands into a file.

fork, exec, wait and exit

- December 28, 2020
This is the english version of a 2007 article . In de.comp.os.unix.linux.misc somebody asked: Are commands in a script executed strictly sequentially, that is, will the next command only be executed when the previous command has completed, or will the shell automatically start the next command if the system has spare capacity? Can I change the default behavior - whatever it may be - in any way? If you are looking into the fine manual, it may explain at some point that the shell starts each command in a separate process.

SQL Clause is coming to town

- December 26, 2020
Olya Kudriavtseva has an ugly Christmas sweater : He’s making a table. He’s sorting it twice. SELECT * FROM contacts WHERE behavior = “nice”; SQL Clause is coming town! (buy here ) Katie Bauer observes : I mean, except for the fact that sorting something twice is TERRIBLY optimized So how bad is this? Let’s find out. Some test data We are defining a table santa, where we store peoples names (GDPR, EU Regulation 2016/679 applies!

Der Testing-in-Production-Blues

- December 17, 2020
Kalenderwoche 51/2020: Lockdown mit Schulschließungen. Nachdem es im März schon einmal Schulschließungen wegen Corona gab, und dort die Defizite technischer und organisatorischer Natur offenbar wurden, hat man in Deutschland die Zeit genutzt und sich auf die vorhergesagte 2. Welle vorbereitet, die jetzt genau eingetroffen ist. Deutschland ist schließlich nicht nur das Land der Dichter und Denker, sondern auch ein Land der Ingenieure und Tüftler, und bekannt für seine funktionierende und effiziente Bürokratie.

Embracing the Stream

- December 9, 2020
So this happened: CentOS Project shifts focus to CentOS Stream The future of the CentOS Project is CentOS Stream, and over the next year we’ll be shifting focus from CentOS Linux, the rebuild of Red Hat Enterprise Linux (RHEL), to CentOS Stream, which tracks just ahead of a current RHEL release. CentOS Linux 8, as a rebuild of RHEL 8, will end at the end of 2021. CentOS Stream continues after that date, serving as the upstream (development) branch of Red Hat Enterprise Linux.

Not JOINing on PERFORMANCE_SCHEMA

- December 1, 2020
The tables in PERFORMANCE_SCHEMA (P_S) are not actually tables. You should not think of them as tables, even if your SQL works on them. You should not JOIN them, and you should not GROUP or ORDER BY them. Unlocked memory buffers without indexes The stuff in P_S has been created with “keep the impact on production small” in mind. That is, from a users point of view, you can think of them as unlocked memory buffers - the values in there change as you look at them, and there are precisely zero stability guarantees.

MySQL: Backups and Replication

- November 27, 2020
There was a question at work about MySQL backups and restore. I needed to explain more. We use databases to make state persistent. That is: As a developer you can think of your database as a single giant, structured global variable with a weird access method, and to make things worse, concurrent access. A database is just a global variable to your code We can log statements that change the state of our database in a log.

Gitlab in Docker

- November 22, 2020
These installation notes are mostly a note to myself, documenting the installation process of a Gitlab Omnibus Container in Docker, plus Gitlab Runners. OS Setup We are installing into /export/gitlab, a 10G xfs slice from the local flash pool: # lvcreate -n gitlab -L 10G data # mkfs -t xfs /dev/data/gitlab # mkdir /export/gitlab # mount /dev/data/gitlab /export/gitlab # echo "/dev/data/gitlab\t/export/gitlab\txfs\tbsdgroups,usrquota,grpquota,attr2,nofail,noatime 1 2" >> /etc/fstab # mkdir /export/gitlab/{gitlab,gitlab-runner} # mkdir /export/gitlab/gitlab/{config,data,logs} Docker We are using docker-compose to run this, with a .

On the Observability of Outliers

- November 19, 2020
At work, I am in an ongoing discussion with a number of people on the Observability of Outliers. It started with the age-old question “How do I find slow queries in my application?” aka “What would I want from tooling to get that data and where should that tooling sit?” As a developer, I just want to automatically identify and isolate slow queries! Where I work, we do have SolarWinds Database Performance Monitor aka Vividcortex to find slow queries, so that helps.

My home sensor network

- November 15, 2020
I have been asked to document my home sensor network. Being married to a person with a background in web security sets boundary conditions: No cloud. We are running all services locally. No control, only metrics. I am collecting data from a number of plugs with power meters over Wi-Fi, using the MQTT protocol. I am also collecting data from a number of temperature sensors over Zigbee, and convert to MQTT.

Rechenzentren und ihren Stromverbrauch regulieren

- November 1, 2020
Es gibt ein Interview mit Stefan Ramesohl vom Umweltministerium (des Bundes) in Netzpolitik.org: “Warum niemand weiß, wie viele Rechenzentren es in Europa gibt ”. Im Wesentlichen hat das Umweltministerium angesagt, daß es auf europäischer Ebene Rechenzentren erfassen und katalogisieren will, um in einem zweiten Schritt den Energieverbrauch von Rechenzentren zu regulieren. Das ist sehr spannend, denn derzeit gibt es keine Übersicht über Rechenzentren in Europa, und tatsächlich sind einige Rechenzentrumsbetreiber sehr paranoid, was den genauen Standort ihrer Hardware angeht und wieviel und welche Hardware darin ist oder was diese tut.

MySQL: Ecosystem fragmentation

- October 28, 2020
Sometimes things change in a way that is hard to put a finger on, but I am doing this MySQL thing since 3.23, and commercially since 2005, and the environment is changing. These days, when you talk to people in need of MySQL, the first thing you have to ask them is “Which MySQL”. And by that I do not mean a version number in the first place. The answer may be: