I thought I’d have a play with AWS’s latest offering “OpsWorks”, and see if they’ve done us all out of a job. Well, kinda. OpsWorks is interesting. It’s basically hosted chef, with EC2 integration. The immediate “drawbacks” to some are that there’s only two supported distributions, Ubuntu 12.04 LTS and Amazon Linux. It’s also incredibly rough around the edges. It’s easy to sign up to, and it adds a service to the AWS management console. ...

Do not use any of the defaults shown below. I would no longer recommend a security group rule for port 22 to 0.0.0.0/0 This has been roughly adapted from this ServerFault question for the case when it gets removed/deleted/closed. The question was about how to configure a Flash game server on Linux, but on EC2. I had a good look around, but didn’t find any true step-by-step EC2 tutorials for proper beginners. So I made one. This one is fairly specific to SmartFox Server towards the end, but the first few bits about creating an instance, and adding stuff to the security group should be generic enough to be useful. ...

This is the brief story of how PDFTribute.net came to be, and survived being linked to from some of the top world news sites. My good friend, Patrick Socha, so moved by the outpouring of data associated with the twitter hashtag #pdftribute set up a quick and dirty twitter archive, extracting tweets containing links. I had a look at this, realised it looked awful on mobile (I was on a bus at the time!), and requested the source on Git. ...

2012: A year in review. So.. 2012.. What can I say.. Quite a lot, actually. When I last wrote an annual retrospective, it was 2011 going into 2012, and I’d just started at Baseblack, one of many Soho-based VFX studios. I had a good 12-odd months there, before the credit crunch hit the entire London VFX industry, and I was made redundant in October. Over my time there I wrote a lot of puppet manifests, built a render farm based on Dell Blade servers, undertook a Hitachi HNAS administrator course, learned how to use Maya, Realflow, Nuke, Shake, PFTrack, Silhouette, and a whole bunch of other VFX packages too numerous to mention. “Oh, your 2D render won’t complete? Did you use Paint nodes? Oh.. Well .. Good Luck With That.” It was a good year. I got my head around 2D and 3D LUTs, Figured out monitor and projector calibrations. Before it all went pearshaped, we’d built a really awesome pipeline, with the help of Paul Nendick, Andrew Bunday, and Michael Nguyen. We had a truly fantastic team. And then it all went wrong. ...

“John, I can’t get Jurassic Park back on line without Dennis Nedry.” Words you never want to hear uttered. Unless you work for InGen, it’s highly unlikely. Although there is the remaining problem of the Human Single Point of Failure (HSPOF). After you’ve spent the last year or two eliminating the single points of failure from your computational infrastructure, you realise that you’re the only one who knows which cronjobs run when, and on which servers. You’re the only one who knows how to kickstart the postgresql streaming replication, and pg_basebackup isn’t documented in the wiki. ...

So.. it transpires that I have a friend who studied Physics at Imperial College, and as a part of that, was taught how to use C++ “As a tool to help with computational physics”. - His words, not mine. As a result, he has no explicit knowledge of some of the finer points of C++ programming, no idea how a binary search algorithm works, why you’d use a Deque and when you’d use a Vector. ...

I get easily tired of doing the same thing over and over again, and will, wherever possible, script or automate it to make life easier for myself. This could be in the form of a lightweight webapp/REST api for stuff, or in this case, I used Jenkins. So on one server, we sometimes need to reload apache. As we don’t like developers randomly executing shells on live servers, it’s better to just allow access to a few specific commands, in this case, a wrapper script on the target server’s /usr/local/bin that just wraps “/etc/init.d/httpd restart” or “/etc/init.d/httpd reload”. ...

Interesting thing of the day: Milo, Ron, et al. “Network motifs: simple building blocks of complex networks.” Science Signalling 298.5594 (2002): 824. Fulltext available from Google Scholar: - http://bit.ly/YAstgD It occurs to me that in scalable systems engineering (the sort of thing I do for a living), you only tend to see Bi-fan networks and Bi-parallel ones. Bi-fan is rougly equivalent to a cross-connected core switch whereas Bi-parallel is a good representation of a Virtual IP with Load balancer. ...

I think this seems like an appropriate time to say a few words in favour of the great United States of America. There’s some things they just do excellently. Customer service springs to mind as one of the best I’ve ever encountered. I’m not exactly sure why this is, but people do seem to be far more willing to be kind, courteous and helpful. There’s other things too, the weather is, generally better (and if not better, then certainly more predictable). There’s also the stunningly beautiful scenery. I challenge anyone to gaze deeply at the Grand Canyon, or the Yellowstone Paint Pots and not be stricken with a sense of awe and grandeur. ...

I’ve heard quite a bit about the “G-WAN Application Server” over the past few weeks. Initially it was a Serverfault question that left me thinking “WTF” (http://serverfault.com/questions/445835/virtual-host-on-g-wan) I took a look at their website and thought: “Those are pretty insane claims”. They’re also the kind of crap you tend to see where the intended audience is somebody who has absolutely no clue about scalability, or production-readiness. Y’know, Managers. Quite well summarised by this comment: GWAN isn’t designed to be a robust webserver, it’s designed to perform exceptionally well in contrived and outlandish benchmarks, so PHBs will demand the IT team use it and buy support… – Chris S♦ 19 hours ago Interestingly enough, the only person who answered that question was a fellow called Gil who apparently works for G-WAN.. I don’t normally take much offense to product owners on Serverfault et al, but the vast majority of his answers do seem to be a bit spammy. ...