Skip to content

The Isoblog. Posts

Using MySQL Partitions (a Python example)

Today somebody had a problem with expiring a large table (a Serendipity Blog table).

In MySQL InnoDB, tables are physically ordered by primary key (InnoDB data is a B+ tree, a balanced tree where the data pages are the leaves of the tree). If you are expiring old data from such a log table, you are deleting from the left hand side of the tree, and since it is a balanced tree, that triggers a lot of rebalancing – hence it is very slow.

If you rename the old table and INSERT … SELECT the data you want to keep back into the original table, that can be faster. But if the data you want to keep is larger than memory, the indexing of the data will still be slow.

A nice way to handle log tables are partitions. Here is an example. It’s not very cleaned up, but it works on my system.

Leave a Comment

New Technology vs Planned Obsolescence

based on an old Google plus article from 2015:

What you observe as Planned Obsolescence is often the natural outcome of fast product cycles that are necessary for any new technology.

When a new thing arrives in the market, it is often barely viable, a minimum viable product. We are remembering the iPhone 1 as revolutionary, but we chose to forget about is slowness, its clunkyness and the very limited feature set it had. And those of us having purchased a car with built-in satnav now have to deal with a car radio where you have to choose between listening to a CD or putting in the outdated CD-ROM with navigation data – and then wait for a minute until you get the route.

3 Comments

The Data Center in the Age of Abundance

We are currently experiencing a fundamental transition in the data center. In recent discussions, it occured to me how little this is understood by people in the upper layers of the stack, and how the implications are not clear to them.

In the past, three fundamentally scarce resources limited the size of the systems we could build: IOPS, bandwidth and latency. All three of them are gone to a large extent, and the systems we are discussing now are fundamentally different from what we had in “The Past™”, with “The Past” being a thing five to ten years ago.

4 Comments

LetsEncrypt Wildcard Certs

In their continuing quest to ground the certificate market, LetsEncrypt now offers wildcard Certificates.

TLS Certificates are digital passports that provide proof of identity in encrypted connections. In the past, a duopoly of two companies has been selling these through many differently branded outlets.

LetsEncrypt managed to break into this, providing TLS Certificates for free and fixing other problems on the way. For example, previously these certificates had a very long lifetime, making revocation of compromised certificates a complicated affair and discouraging users of these certificates from automating rollover and renewal, driving up costs for running encrypted connections.

By doing what they do, LetsEncrypt also forced the existing TLS brands of the CA duopoly to adjust their prices and rework procedures and APIs in order to make automation simpler.

Wildcard certificates are TLS identities that work on an entire domain (*.koehntopp.info, “any name in the koehntopp.info domain”), where regular certificates only work on one specific name.

Next step are EV certificates.

Leave a Comment

On collecting the right kind of data

So Microsoft just blogged this:

Transform your organization with Microsoft Workplace Analytics

Workplace Analytics taps into Office 365 email and calendar metadata, including to/from data, subject lines and timestamps, to shine a light on how the organization collaborates and spends time. It turns this digital exhaust—the data that comes naturally from our everyday work—into a set of behavioral metrics that can be used to understand what’s going on in an organization.

1 Comment

Pi-HR-anhas

So Soundcloud is unwell, financially, and has laid off 173 people, about 40% of their workforce. Such things are never welcome, and usually they are sad affairs.

Except when somebody throws a bunch of Data Scientists, ML people or Backend people into the water. Check out the thread below this tweet:

Tweet

Data Scientists in the water, the Pi-HR-anhas have a feeding frenzy.

Leave a Comment