Rem.co: Remco Overdijk

Git: Automatically moving a tag using a custom command

Mon, 18 Feb 2019 16:56:55 +0100

Ever find yourself moving a git tag to a new commit? You’ll probably know this procedure consists out of three steps;

Removing the existing tag from your origin
Manually moving the tag (using -f to allow moving)
Pushing the tag back your origin

Since this procedure is more cumbersome that it could be, behold, a quick and easy life hack to automate this process into a single custom command.

Puppet: Calculating average catalog compilation times

Mon, 09 Mar 2015 18:25:18 +0100

Just a quick post with the oneliner of the day.

When you are debugging catalog compilation issues or other puppet performance issues in general, it is good to know exactly which catalogs are slow to compile. Knowing which catalogs are substantially slower than others allows you to focus on those catalogs and the modules they contain.

Git: The difference between lightweight and annotated tags

Thu, 12 Feb 2015 14:05:50 +0100

I was reviewing some pull requests at work today. One of the PR’s had an updated composer.lock file. We usually check if the reference matches the version for this update, to see if that commit is actually released on the module’s master branch:

"name": "company/module_name",
- "version": "0.11.0",
+ "version": "0.12.0",
"source": {
	"type": "git",
	"url": "ssh://git@stash.company.net/packages/module_name.git",
-	"reference": "19ecfcb286052457697caad3359d7817e2dfa2f5"
+	"reference": "2c539864d72baede7f169f15eec8c3317e26c1bc"
 },
- "time": "2014-10-08 11:12:23"
+ "time": "2014-11-18 16:47:02"

Usually, this reference matches the hash of the commit we’ve tagged as this version. In this particular case however, the hash mentioned in reference was nowhere to be found in the commit log. So what’s going on here?

Bacula: Cancelling all jobs that are currently writing

Mon, 09 Feb 2015 14:08:07 +0100

Just a quick post with the oneliner of the day.

Scenario: after a bacula director restart a couple of jobs were stuck on the FD with message:

Running Jobs:
Writing: Incremental Backup job node.cluster.company.com JobId=8702 Volume=""
    pool="bacula.director.company.com:pool:default.incremental" device="DefaultFileStorage" (/mnt/bacula/default)
    spooling=0 despooling=0 despool_wait=0
    Files=0 Bytes=0 AveBytes/sec=0 LastBytes/sec=0
FDSocket closed

There were a couple of these jobs that were stuck, preventing all other jobs from running, because those were waiting for a free slot on the FD.

Bacula: Purging and deleting old volumes

Thu, 15 Jan 2015 14:08:02 +0100

I’ve been using bacula for a couple of months now in conjunction with puppet to make automated backups of all servers that are managed by puppet. My bacula setup labels a volume for every job it runs with a unique name:

`1`	`Label Format = "${Job}.${Year}${Month:p/2/0/r}${Day:p/2/0/r}.${Hour:p/2/0/r}${Minute:p/2/0/r}"`

These volumes are automatically purged once the retention of all files contained on the volume expires (which is configured per-pool). Due to the unique names however, the volumes cannot be recycled. The result of this is that the volumes that have been marked as purged in the catalog remain as-is on the disk. After some time this ultimately resulted in a full disk, thus halting all backups performed on that pool. Not good. Not good at all.

Music: End Of The Year Mix 2014

Wed, 14 Jan 2015 16:55:06 +0100

Need some fresh tunes? Enjoy my End Of The Year Mix for 2014.

Gentoo: Running a Minecraft 1.8 server

Fri, 28 Nov 2014 13:36:57 +0100

Running a dedicated Minecraft server can be a challenging job. You have to find a balance between performance and usability using “server software” that doesn’t seem to be designed to provide for long running, resilient services.

Being a first-time Minecraft server operator I had to tackle various challenges in order to come up with a way to provide a stable and reliable service to my players. The following article is a recollection of the things I implemented and scripts I wrote in order to run a Minecraft 1.8 server. The scripts mentioned are specific to Gentoo Linux, but could also be used on most other Linux flavours, albeit with some modifications to match that platform’s init.d scripts.

Diceware Password Generator

Sat, 14 Jun 2014 20:59:51 +0200

Select a language: English Dutch Japanese Polish Swedish Rolls: Add number Clear Words: What is this? This is a diceware password generator. Read about diceware here here and here Is this safe? Probably not, but I’ve taken some precautions to make sure this is as safe as a browser-based generator can be: This generator is implemented in javascript. It runs in your browser, and doesn’t send your rolls and/or words to me or anyone else.

Git: Moving a repository to a new server

Tue, 10 Jun 2014 16:59:56 +0200

The quick, easy and complete way:

Fixing 'Error opening terminal: screen-256color.'

Thu, 06 Jun 2013 00:00:00 +0000

If you’re using tmux or screen as a wrapper for your terminal sessions (such as when using the Text Triumvirate), chances are that you are presented with the following error when you invoke a command that uses 256 color mode (such as multitail or htop):

Error opening terminal: screen-256color.

OnApp: Using PHP to launch a VNC connection

Fri, 01 Mar 2013 00:00:00 +0000

If you’re an OnApp user you probably know you can make a VNC connection to your VM’s using the Control Panel. This uses a java applet in your browser as the VNC client.

Wouldn’t it be nice to use your own VNC client (like Remotix) instead?

In fact, you can, but OnApp spawns a VNC server on a (somewhat) random port and a new random VNC password for each sessions which you’ll have to figure out before setting up your connection.

SELinux: Allowing SSH public key authentication

Tue, 19 Feb 2013 00:00:00 +0000

The issue

I experienced a seemingly weird issue with a freshly installed CentOS server today.

SSH Public key authentication was correctly set up; The sshd_config was properly configured and a ~/.ssh/authorized_keys was present with the correct rights and verified correct contents (as the file was yanked from another, working, server with scp).

All attempts to connect to the machine using key authentication silently failed however.

Ubuntu: Automatically selecting a fast mirror

Tue, 22 Jan 2013 00:00:00 +0000

Letting Ubuntu pick the apt-get mirror closest to you is pretty easy.

Munin: failing with Storable error

Mon, 21 Jan 2013 00:00:00 +0000

I suffered from a Munin version 2.0.10 installation that refused to update the majority of the graphs. Only the first two of a long list were being updated, the rest all ‘hung’ at the same moment.

After a little investigating, the problem surfaced:

1
2
3

$ su - munin --shell=/bin/bash munin-cron
File is not a perl storable at blib/lib/Storable.pm (autosplit into blib/lib/auto/Storable/fd_retrieve.al) line 398, at /usr/lib64/perl5/vendor_perl/5.12.4/Munin/Master/Utils.pm line 362
File is not a perl storable at blib/lib/Storable.pm (autosplit into blib/lib/auto/Storable/fd_retrieve.al) line 398, at /usr/lib64/perl5/vendor_perl/5.12.4/Munin/Master/Utils.pm line 362

I started out by fixing all items the munin-check script suggested, which is always a good starting point.

Accelerating TYPO3 with Nginx & Varnish

Thu, 17 Jan 2013 18:16:04 +0200

So, you have a TYPO3 website and despite all your best efforts, it’s still too slow in your opinion.

Maybe it’s time to start using Varnish as a caching reverse proxy to help speed things along. It’s fairly easy to set up, but there are some caveats when it comes to TYPO3. I’ll try to outline a fairly basic scenario below that should fit a number of TYPO3 installations.

Gentoo: Updating and Cleaning

Wed, 16 Jan 2013 23:40:44 +0200

Keeping your Gentoo Linux server up to date isn’t as straightforward as let’s say an Ubuntu box, where you would just run $ apt-get update && apt-get upgrade && apt-get clean for example.

Gentoo is far too flexible for a one size fits all approach. The commands outlined below come pretty close for daily use though:

2013: Blogging with Octopress

Wed, 16 Jan 2013 19:06:33 +0200

Well. Here we are!

New year, new approach to blogging.

In the past year I haven’t managed to push out a lot of blog articles though I’ve been working with loads of new, interesting technologies and approaches.

Being a programmer, I blame the software I use to blog. In my case that’s the TYPO3 installation at rem.co. It’s recently been updated to TYPO3 6.0, and I haven’t gotten around to fixing all the bugs that appeared after such a major upgrade.

Moreover, that site uses the ancient tt_news plugin as a blog substitute, which makes blogging a bit… troublesome to put it mildly. Especially “advanced” stuff like including code snippets and multiple layouts in one article.

Fail2Ban PhpMyAdmin script

Wed, 09 Dec 2009 16:44:27 +0200

While examining my webserver statistics, I noticed that quite a lot 404’s are being served on most of my domains to scan bots that are trying to find exploits in possible running PHPMyAdmin configurations. Though harmless if you keep a clean ship with a decently configured PHPMyAdmin and the latest updates like I do, I still decided I couldn’t let this behaviour unanswered. So I took action, and wrote a small fail2ban filter that permanently drops all traffic from the IP addresses these scans originate from, like I do with every address that misbehaves in any way.

The regex used won’t capture all attempts, but with my configuration only 1 hit is enough to get you banned (the scripts these scans call are main.php and config.inc.php, which aren’t to be called directly, especially not when they fail with a 404 like these), and all scanning attempts I’ve seen so far cycle through at least 20 different combinations.

IPMI graphs in Munin

Sun, 20 Sep 2009 17:22:59 +0200

It is possible to monitor fan speeds and temperatures on Dell Poweredge servers under Linux. You can achieve this by reading out the IPMI data that is available on the system.

I used the steps on this website to buffer the data gathered by IPMI to use in Cacti.

However, in addition to Cacti I also use Munin to monitor various system parameters. Wouldn’t it be nice to incorporate graphs for fan speeds and temperatures in Munin? I thought so, so I developed a way to do this.

Cisco VoIP oplossing voor Ziggo telefonie

Fri, 04 Sep 2009 11:39:48 +0200

This article is only available in Dutch.

Sinds een recente verhuizing beschik ik thuis over een Ziggo Alles-in-1 Plus pakket, met internet, tv, én telefonie.

Daarvoor maakte ik gebruik van een Cisco VoIP netwerk op basis van een externe SIP provider. Natuurlijk wilde ik mijn Cisco netwerk blijven gebruiken, maar dan wel op basis van de Ziggo telefonie aansluiting.

Helaas maakt Ziggo gebruik van het PacketCable protocol over EuroDocsis, in plaats van SIP. Daarnaast heeft het Motorola SurfBoard modem dat bij het Ziggo abonnement geleverd wordt geen SIP interface voor het LAN, maar beschikt over 2 POTS poorten op RJ11 connectoren.

Iptables: Creating persistent bans from Fail2Ban

Tue, 18 Aug 2009 15:53:10 +0200

On my servers I use the nifty program Fail2Ban to perform logbased automatic firewalling of ‘bad’ ip’s.

The idea behind this is easy: Some IP performs an action I don’t approve of. This can be any number of things, e.g. requesting pages in Apache that are commonly accessed by bots and/or scanners, or trying to log in to SSH with accounts that do not exist on the system. This bad behavior gets logged, and Fail2Ban keeps tabs on those logs, and using a number of rules it determines if a host is ‘bad’ enough to temporarily or permanently ban all access to the server. It does so by adding a few chains to Iptables (one for each thing it checks for), and dynamically adding/removing IP’s to/from these chains.

This all works perfectly. However, there’s one issue; When Iptables gets reloaded, it restores its default rules, removing the Fail2Ban chains and all the rules they contain, even if the ip’s in the chain were marked as permanent.

CSS: Overlapping Flash content with CSS

Fri, 29 May 2009 19:06:33 +0200

By default, flash movies are always shown on the top-level of a display tree.

However, it can be very useful to be able to move the flash content to the background, and having it overlapped by other content; e.g. You have a flash movie in the header of your website, but there’s a sidebar menu which should be displayed over the header.

PHP: Testing the Pseudo Random Number Generator

Tue, 26 May 2009 18:44:44 +0200

Every programmer uses them.. PRNG’s, better known as Pseudo-Random Number Generators; in PHP represented by the rand(min,max) function.

Unlike True Random Number Generators (TRNG’s) that use true random data like atmospheric noise to create their numbers, PRNG’s rely on software algorithms to come up with seemingly random numbers.. but are they?

And is there a difference between Linux and Windows PRNG results?

MySQL: The complete FULLTEXT checklist

Thu, 14 May 2009 18:16:04 +0200

In an effort to help a colleague with FULLTEXT search troubles today, I tried to find out everything that could go wrong with setting up this search method on a table. My short research resulted in this checklist. Failure to comply with these checks will result in catastrophic failure :P

MySQL: Boolean substitution

Mon, 27 Apr 2009 00:00:00 +0000

Today I faced a quite interesting problem, that originated from pure laziness. I’m developing a backend system for a quite complex database structure. Within this backend, an almost limitless amount of table views have to be created for the end user. Because I’m extremely lazy, and didn’t want to develop the html view code for each table view, I created a PHP html-table-generator-class, which takes a mysql_result_set as parameter, and outputs the html table in string format.

This method works great, unless for some cases, where a value in the query has to be substituted by a user readable value. A boolean is a good example of such a value.