Die wunderbare Welt von Isotopp

Cloning and splitting logical volumes

Avatar of @isotopp@infosec.exchange Kristian Köhntopp - December 2, 2019

Where I work, we routinely run our databases on XFS on LVM2.

The setup

Each database has their database on /mysql/schemaname, with the subdirectories /mysql/schemaname/{data,log,tmp}. The entire /mysql/schemname tree is a LVM2 Logical Volume mysqlVol on the Volume Group vg00, which is then formatted as an XFS filesystem.

# pvcreate /dev/nvme*n1
# vgcreate vg00 /dev/nvme*n1
# lvcreate -n mysqlVol -L...G vg00
# mkfs -t xfs /dev/vg00/mysqlVol
# mount -t xfs /dev/vg00/mysqlVol /mysql/schemaname

Basic Ops

You can grow an existing LVM Logical Volume with lvextend -L+50G /dev/vg00/mysqlVol or similar, and then xfs_grow /dev/vg00/myqlVol.

Fertig gelesen: Exit Strategy

Avatar of @isotopp@infosec.exchange Kristian Köhntopp - November 22, 2019

Exit Strategy by Martha Wells is the fourth book in the the Murderbot Diaries (All Systems Red , Artificial Condition, Rogue Protocol, Exit Strategy, Network Effect).

This novella is the culmination of the Murderbot Novellas, in which our nameless murderbot cyborg tries to nail the evil GrayCris corporation for the series of crimes against settlers and researchers, has to rescue it’s “owner”, sponsor and friend Dr. Mensah from the claws of GrayCris, and make friends on the way. And while Murderbot has no problems with violent action and explosions, it is still scared of emotions and … people.

Fertig gelesen: Fashion and Its Social Agendas: Class, Gender, and Identity in Clothing

Avatar of @isotopp@infosec.exchange Kristian Köhntopp - November 21, 2019

Fashion and Its Social Agendas: Class, Gender, and Identity in Clothing by Diane Crane is an analysis of the changes in society during the various phases of the industrial revolution through the lens of clothing in various societal strata.

The book studies fashion and clothing choices in three countries - France, the United States, and England - over a period of 150 years. France and England are examples of class societies in the nineteenth century. It has available four sets of data: Interviews and inventories of clothing in a large number of households the above countries, spaced out over an extended period of time. The long timescale and the temporal distance between the various studies allow comparison across major events (such as the Franco-Prussian war) and as industrialisation plays out.

Fertig gelesen: The Wiz Biz

Avatar of @isotopp@infosec.exchange Kristian Köhntopp - November 20, 2019

The Wiz Biz by Rick Cook is a book in the genre of ‘magical realism’. Some real-world person is being transported into a universe with predictable magic in a classical fantasy setting and is applying scientic method and industrial production methods to the setting.

In this book, the real-world person is Walter Zumwalt, a stereotypical hacker, which as part of a hail-mary summoning rital is being transported into a magical war between the good guys and the bad guys. Walter does not know a thing about magic at all, but apparently that can be learned and then turns out to be a lot like computer programming, if you make it so.

When a file changes, do a thing

Avatar of @isotopp@infosec.exchange Kristian Köhntopp - November 19, 2019

When developing there is often an edit-compile-test cycle, or an edit-distribute-changes cycle or a similar repetetive task. You could poll changes, for example with cron every minute or similarly, but that is wasteful and slow.

All modern operating systems have mechanisms for processes to subscribe to file or directory changes. In MacOS, we do have the File System Events API since 10.5, in Linux we got three different implementations (as described in LWN ): The original dnotify, its replacement inotify and the even more recent fanotify (which got its own LWN article ). BSD has kqueue.

A blast from the past

Avatar of @isotopp@infosec.exchange Kristian Köhntopp - November 18, 2019

TL:DR: If you have long running transactions, MySQL does not deal well with this, and it will slow down the box. That’s okay as long as you are basically alone on your box, but if you aren’t, the others will hate you.

The database machine ‘somehierarchy-02’ in a general purpose load balancer pool for somehierarchy had replication delay.

It’s a MySQL replica and is receiving the same write workload than all the other boxen in that pool. Yet, somehierarchy-03 is fine, while somehierarchy-02 is not. Both machines have comparable hardware: -02 and -03 are both Dell M630 with 128 GB of memory and two SSD. They should behave identically, yet one runs from memory, but the other is reading 40 MB/s from disk.

Fertig gelesen: Change Agent

Avatar of @isotopp@infosec.exchange Kristian Köhntopp - November 18, 2019

Change Agent is Daniel Suarez fifth book.

As usual, he builds his stories around one technology or technological change and constructs an action-rich storyline around it. For “Change Agent”, this is CRISPR genetic modification of living beings. He basically imagines a programmable genetic agent that rewrites living beings including humans on the go - the eponymous change agent.

It’s 2045, and an Interpol Agent Kenneth Durand is trying to pursue the Huli Jing cartel, a criminal black market corporation that produces enhanced designer babies for rich prospectice parents. Huli Jing so far has proven to be able to completely evade any police action. As Durand tries to close in on Marcus Demag Wyckes, the cartel leader, he’s being attacked, injected with a drug and the change agent turns him into Wyckes, Face-Off-style. Through some unlikely circumstances Durand survives the procedure that was expected to kill him, and pursues the cartel.

Fertig gelesen: Infinity Engine

Avatar of @isotopp@infosec.exchange Kristian Köhntopp - November 18, 2019

Infinity Engine is the third and final book in Neal Asher’s Transformation Sequence , a trilogy set in the Polity Universe .

This is the conclusion of the three book arc about the redemption of the mis-manufactured and temporarily mad AI Penny Royal. Asher takes us through a set of encounters he prepared two other books for, contrasting the reconstruction and the redemption of Penny Royal with the descend into madness of the forensic AI Brockle, and weaving together all the other story-strands that he set up earlier into a literally star-shattering finale.

Everything was a file, but we got better

Avatar of @isotopp@infosec.exchange Kristian Köhntopp - November 14, 2019

I fell into the Twitters again. @CarrickDB joked about Unix, Files and Directories:

And that is a case of “Haha, only serious”. Because directories used to be files, and that was a bad time. Check out the V7 Unix mkdir command. At this point in history we do not have a mkdir(2) syscall, yet, so we need to construct the entire directory in multiple steps.

This fragile and broken: mkdir could be interrupted while doing that or another program could try to race mkdir while it is doing that. In both cases we get directories that are invalid and dangerous to traverse, because they break crucial assumptions users make about directories.

Filling disk space fast

Avatar of @isotopp@infosec.exchange Kristian Köhntopp - November 11, 2019

Some of the databases at work are a tad on the large side, in the high 2-digit terabytes of size. Copying these to new machines at the moment takes a rather long time, multiple days, up to a week. Speeding it up pays twice, because with shorter copy times there is also less binlog to catch up.

I have been looking into disk copy speeds in order to better understand the limits. When creating a partition from NVME devices, the most simple layout is a concatenation: