rem

.co

Puppet: Calculating Average Catalog Compilation Times

| Comments

Just a quick post with the oneliner of the day.

When you are debugging catalog compilation issues or other puppet performance issues in general, it is good to know exactly which catalogs are slow to compile. Knowing which catalogs are substantially slower than others allows you to focus on those catalogs and the modules they contain.

I devised this bash oneliner, which is almost a direct copy of this example by Jadu Saikia, so full credits go to him!

Calculating Average Catalog Compilation Times in Puppet
1
cat /var/log/puppet/master.log | grep -i "compiled catalog for" | awk '{printf "%s\t%s\r\n",$9,$14}' | sort | awk 'BEGIN{FS="\t"; printf("%-35s\t\t%s\t\t%s\t\t%s\n", "Node","Count","Total","Average")} NR!=1 {a[$1]++;b[$1]=b[$1]+$2}END{for (i in a) printf("%-35s\t%10.0f\t%10.0f\t%10.2f\n", i, a[i], b[i], b[i]/a[i])}' | sort -n -k4

Git: The Difference Between Lightweight and Annotated Tags

| Comments

I was reviewing some pull requests at work today. One of the PR’s had an updated composer.lock file. We usually check if the reference matches the version for this update, to see if that commit is actually released on the module’s master branch:

Example of an updated composer.lock
1
2
3
4
5
6
7
8
9
10
11
"name": "company/module_name",
- "version": "0.11.0",
+ "version": "0.12.0",
"source": {
  "type": "git",
  "url": "ssh://git@stash.company.net/packages/module_name.git",
-    "reference": "19ecfcb286052457697caad3359d7817e2dfa2f5"
+    "reference": "2c539864d72baede7f169f15eec8c3317e26c1bc"
 },
- "time": "2014-10-08 11:12:23"
+ "time": "2014-11-18 16:47:02"

Usually, this reference matches the hash of the commit we’ve tagged as this version. In this particular case however, the hash mentioned in reference was nowhere to be found in the commit log. So what’s going on here?

Bacula: Cancelling All Jobs That Are Currently Writing

| Comments

Just a quick post with the oneliner of the day.

Scenario: after a bacula director restart a couple of jobs were stuck on the FD with message:

Bacula File Director Error
1
2
3
4
5
6
Running Jobs:
Writing: Incremental Backup job node.cluster.company.com JobId=8702 Volume=""
    pool="bacula.director.company.com:pool:default.incremental" device="DefaultFileStorage" (/mnt/bacula/default)
    spooling=0 despooling=0 despool_wait=0
    Files=0 Bytes=0 AveBytes/sec=0 LastBytes/sec=0
FDSocket closed

Bacula: Purging and Deleting Old Volumes

| Comments

I’ve been using bacula for a couple of months now in conjunction with puppet to make automated backups of all servers that are managed by puppet. My bacula setup labels a volume for every job it runs with a unique name:

Bacula Label Format
1
Label Format = "${Job}.${Year}${Month:p/2/0/r}${Day:p/2/0/r}.${Hour:p/2/0/r}${Minute:p/2/0/r}"

These volumes are automatically purged once the retention of all files contained on the volume expires (which is configured per-pool). Due to the unique names however, the volumes cannot be recycled. The result of this is that the volumes that have been marked as purged in the catalog remain as-is on the disk. After some time this ultimately resulted in a full disk, thus halting all backups performed on that pool. Not good. Not good at all.

Gentoo: Running a Minecraft 1.8 Server

| Comments

Running a dedicated Minecraft server can be a challenging job. You have to find a balance between performance and usability using “server software” that doesn’t seem to be designed to provide for long running, resilient services.

Being a first-time Minecraft server operator I had to tackle various challenges in order to come up with a way to provide a stable and reliable service to my players. The following article is a recollection of the things I implemented and scripts I wrote in order to run a Minecraft 1.8 server. The scripts mentioned are specific to Gentoo Linux, but could also be used on most other Linux flavours, albeit with some modifications to match that platform’s init.d scripts.