This morning, I did something unquestionably naughty, and totally got away with it. A little background. We just had some brand new workstations delivered.. They turned up yesterday afternoon. They’re high performance 3d workstations with an Intel DX79TO mainboard. This mainboard has the intel 87529 Gigabit Ethernet controller. I wouldn’t normally pay so much attention to the controllers and so on that are actually on a board, but in future, I will. ...

Well, that’s certainly another strange problem. We have a tendency to build our own DMG images for certain bits of software we roll out here. Sometimes we’ll incorporate our own patches, other times it’s just to make the application structure more FHS compliant, and stop it “shitting all over the filesystem”, as we so charmingly term it. In the past, we’ve used InstallEase to build a DMG and PKG installer for OSX. It’s been pretty good up until last week, when one of our engineers built a PKG that had a nasty side effect of destroying the System Preferences once installed. We initially pegged this as “another weird thing about Lion”, and rebuilt the image, and so on. ...

Puppet really is great. Don’t ever get me wrong there. It’s saved me masses of time and money over the last few years, and allowed me to do my job quickly and efficiently. That said, it really does have issues with scalability. After about 20-30 clients using WEBBrick, everything kinda falls over a bit. We had this problem at Baseblack. We’ve now got ~60 workstations and rendernodes all using Puppet for configuration management and software deployment. It’s great. It far simplifies the process of rolling out updates and upgrades to new build machines. ...

Originally a comment here In the above article, Bhumi gives a method for storing files in the database, using MySQL and PHP. My personal distaste for PHP aside, I don’t think I could ever find a reason to store files in the database, rather than on the filesystem. I’m also primarily talking about RDBMS type databases, not NoSQL, which tend to have a mechanism for storing files a little bit more sanely than “old-fashioned” databases. ...

Firstly, welcome back. It’s now 2012, and there’s lots more to write about. Recently, Oracle withdrew the ability for Linux distributions to repackage Java and distribute their own packages. This has been widely regarded as a bad idea. I tend to agree. So, let’s re-roll an old sun-java6 deb file, with a new content to contain the latest 6u30 java release. You will need: A set of build packages (I’ve got a set for lucid, so if this goes away, I’ll find some way to host them.) from http://archive.canonical.com/ubuntu/pool/partner/s/sun-java6/ The latest Java packages: http://download.oracle.com/otn-pub/java/jdk/6u30-b12/jdk-6u30-linux-i586.bin and http://download.oracle.com/otn-pub/java/jdk/6u30-b12/jdk-6u30-linux-x64.bin dch. just install devscripts package to get this. Some idea of how packaging on debian/ubuntu works. Let’s get started. ...

Hah. So it’s New Year’s Eve. Twitter is down, and has been for about 3-4 hours. That’s because the NYE celebrations have already started. Somewhere on the other side of the world. I include Twitter on my personal website. The one you’re reading. There’s a template tag that displays my five latest tweets. About 2-3 hours ago, I got some error reports about XML parse errors on that template tag. I use api.twitter.com and pull in the XML feed for parsing. No problems there, it’s always worked. ...

Right. Let’s have a look at the massive technical implications of the Fix Puppet idea. As I mentioned in my earlier blogpost, in order to fix puppet in a sensible way, we’ll have to review all, and overhaul some of the underlying infrastructure that allows it all to run. The interlinks and dependencies between all the parts are a little tricky to visualise. So, here’s a picture. ...

I’ve decided. We need to start doing points poker here at Baseblack if we’re going to carry on this Agile DevOps thing. I’ve got to admit, the first time I came across the Agile methodology was quite late in my career. In the past, prioritisation of “operations” projects was reasonably first come first serve, or by order of priority (frequently, business need, and seldom operational requirement). For software development teams, Agile is a pretty good, native fit. The concepts embodied by stories and sprints fit a development team very cleanly. When it comes to systems administration and engineering, or what I’ve come to refer to as DevOps, Agile can be a bit more awkward initially. ...

Our new PowerDNS cluster (of 2 nodes, so far).. Is backed by Postgresql. In the past, I’ve found that Postgres performs far better for a PowerDNS backend, than MySQL, and certainly better than the BIND, LDAP or SQLite backends. Until version 9.x, Postgres replication was a pretty sorry state of affairs. There were a few options for replication. Slony was commonly used, if not very good.. You’d tend to get a horrific SPoF around the single master. In total, there were 9 or 10 different third party solutions for Postgres replication and clustering. They all had their pros and cons, and some were great, and some were downright awful. ...

This article relies on having a soon-to-expire SSL certificate on an Apple OSX Server. Ours are running Snow Leopard, and I’m yet to try the whole thing on Lion. I’ve got to admit, I went through a bit of a rigmarole to do this. To generate a new certificate, you need a key, and a CSR. To get the key, you need to export a PKCS12 file from KeychainAccess as ROOT. Yes, Root. Yes, OSX = Toy operating system. No, another admin user won’t cut it. Yes it’s a pain in the arse. ...