I was quoted in an IOL article: Landmark judgment liberates VANS

It’s going to be interesting how many new national telecoms operators pop up… 3, 6, 10? You need a good chunk of change to build network infrastructure.

Reminds me of a theory from iWeek a year or two ago.. somebody commented on why the ISP/ICT industry was not “representational” (too many white faces).. he figured that smart BEE investors would not be interested in telecoms: too risky, unstable, low margins, fast moving, hard to predict.. and very capital intensive.

It is ridiculous to call this an industry. This is not. This is rat eat rat, dog eat dog. I’ll kill ‘em, and I’m going to kill ‘em before they kill me. You’re talking about the American way of survival of the fittest. — Ray Kroc, founder of McDonald’s

Comes down to a simple question.. how long do you need to have that fibre in the ground before you recover your costs and what else could you have done with your money?

Don’t get me wrong, I’m very happy with the news that VANS get IECNS licenses, but this is the very beginning of a long journey..

Sometimes a little deadlining does miracles.

So here we go. I will be officially launching a private beta of Half Price Tuesdays on the 29th of September at the September Geekdinner, final details of which will be available shortly.

Unfortunately, if you’re not already on the list for the event you won’t be because it’s full, but I will open the beta to others shortly thereafter.

If you are coming to the dinner, please don’t expect fireworks or rocket-science; hpt was meant to be coded over a weekend, but I got carried away in the design phase.

A few months ago while working on a Linux system using XFS, I came across an interesting “feature”. When moving a directory such that its owner changed, the moved directory’s mtime was changed to the current date and time.

For example:

[0] mj@majestic:~/tmp$ mount |grep home
/dev/mapper/data-home on /home type xfs (rw)
[0] mj@majestic:~/tmp$ mkdir test
[0] mj@majestic:~/tmp$ ls -ld test
drwxr-sr-x 2 mj mj 6 Jun 18 15:28 test
[0] mj@majestic:~/tmp$ touch -t 200801011530 test
[0] mj@majestic:~/tmp$ ls -ld test
drwxr-sr-x 2 mj mj 6 Jan 1 15:30 test
[0] mj@majestic:~/tmp$ stat test
File: `test'
Size: 6 Blocks: 0 IO Block: 4096 directory
Device: fd00h/64768d Inode: 951267331 Links: 2
Access: (2755/drwxr-sr-x) Uid: ( 1000/ mj) Gid: ( 1000/ mj)
Access: 2008-01-01 15:30:00.000000000 +0000
Modify: 2008-01-01 15:30:00.000000000 +0000
Change: 2008-06-18 15:29:08.173750666 +0100
[0] mj@majestic:~/tmp$ mv test test1
[0] mj@majestic:~/tmp$ ls -ld test1
drwxr-sr-x 2 mj mj 6 Jan 1 15:30 test1
[0] mj@majestic:~/tmp$ mv test1 ..
[0] mj@majestic:~/tmp$ ls -ld ../test1
drwxr-sr-x 2 mj mj 6 Jun 18 15:30 ../test1
File: `../test1'
Size: 6 Blocks: 0 IO Block: 4096 directory
Device: fd00h/64768d Inode: 951267331 Links: 2
Access: (2755/drwxr-sr-x) Uid: ( 1000/ mj) Gid: ( 1000/ mj)
Access: 2008-01-01 15:30:00.000000000 +0000
Modify: 2008-06-18 15:30:02.814078187 +0100
Change: 2008-06-18 15:30:02.814078187 +0100


I’d never seen this happen before, so I tried to reproduce the behaviour on systems using ext3, UFS and HFS+ filesystems. None of them updated the moved directory’s mtime when the parent directory changed, which is what one would expect. XFS does have an active mailing list, so I reported my findings and was quickly provided with a patch that fixed the problem. Kudos to the XFS developers for providing a fix so quickly.

Rather odd that no-one had spotted this behaviour in the past though…

This week I flew domestically within the UK and encountered British Airways’ strange special meals policy for the first time. According to their website:

Special meal requests cannot be made for travel on any of the following services: all UK Domestic flights, Euro Traveller flights from the UK to Paris, Luxembourg, Dusseldorf, Jersey, Cologne, Amsterdam & Brussels, and passengers travelling on any flight on a stand-by or waitlist basis.

That’s rather poor (and rather strange, if you ask me). Heck, in South Africa BA even have extra vegetarian meals on their domestic flights for those who haven’t requested them in advance. I’m not at all impressed, but unfortunately BA seem to be the only airline that fly from Gatwick to Manchester.

Update: If you ask, they do have vegetarian meals onboard, even if you can’t pre-book one.

It has come to my attention that it has been far too long since I last wrote in here. Sorry guys, been far too busy living life to write about it.

I hate those all-in-one-post blog entries that just contain point-form activities, so I'll do mine in paragraphs in random order. (Which is only slightly better.)

The jazz band has started up again (first real rehearsal tonight) for a gig on the 12th at Kirstenbosch. I don't have any further details, but it's going to be a good one. I only realised during last week's trombone section rehearsal how much I missed Wednesday night jazz. The Dukes band is fun, but the music isn't as challenging (some of it is downright boring, actually, but we need to do it) and we usually have big gaps where we're missing instruments.

I have been domesticated. I now own a pot and a pan and am actually cooking myself breakfast most mornings. Usually eggs on toast and such, but today I experimented with fried onion as well. I need to tune the herbs I add (basil and thyme were handy, but not that great) and reduce the quantity (a whole onion is too much) but it's an overall win. One of these days I might be able to actually cook some proper food. (Mom, if you're reading this, this paragraph is a lie. I eat muesli and yoghurt for breakfast and cook marvelous and healthy suppers every night.)

In the last month or so, I have watched most of Doctor Who (the new version) and Torchwood. Definitely near the top of my list of things I really want there to be more of.

I have received two emails about my Erlang stuff. One was a wonderful ego-boosting thank-you and the other seemed to be a question in a language my browser refuses to display except as hex-runes.

On browsers: I finally updated to Firefox 3 on my primary machine. Two things had been delaying this. Two extensions I really struggle without (Chromatabs and Tab Mix Plus) didn't have fx3 versions. They both now have beta versions (although in the case of the former it's actually a new extension). Fx3 really struggles with malformed certificates and won't let you add an exception to visit the site anyway. I think this is still the case, but the one site I relied on with a broken cert has finally fixed it. I quite like the new UI. The biggest win has to be that auth windows and plugin loads no longer lock the whole browser, something that had really irritated me in fx2.

Tomorrow morning shall see me giving my flat a much-overdue clean in preparation for the arrival of a houseguest. I should probably finish putting up my last blind, too. (pkeike, if you're reading this, ignore this paragraph. My flat is always spotless and I did nothing special to prepare for your arrival.)

And now, a rant:
Dear Google, please give me a version of Chrome I can actually use. I realise that most of your userbase is quite happy with a Windows-only version, but some of us prefer an OS that isn't actively hostile. At the very least, can we have one that works in wine By alienating a large class of free-software developers you're not buying any favours. Yours, $linux_lad.

I have been playing with Flickr's oEmbed implementation just now and wrote the following script.

<?php
require_once 'db.php';
$rows = mysql_query('SELECT `post_content` FROM `post` WHERE `post_content` REGEXP "http://flickr.com/photos/[a-zA-Z0-9]+/[0-9]+"');
while ($row = mysql_fetch_assoc($rows)) {
  preg_match_all('#http://flickr.com/photos/[a-zA-Z0-9]+/[0-9]+#', $row['post_content'], $images);
  foreach ($images[0] as $image) {
    $data = @file_get_contents('http://flickr.com/services/oembed?url=' . urlencode($image));
    if ($data) {
      $xml = @new SimpleXMLElement($data);
      if ($xml && $xml->url) {
        $jpg = str_replace('.jpg', '_m.jpg', $xml->url);
        $alt = "$xml->title by $xml->author_name";
        echo "<a href='$image'><img src='$jpg' alt='$alt' title='$alt'></a>";
      }
    }
  }
}

This should work together with starstar. The API calls need to be cached though so will make a plan there before I commit it to subversion. Oh yes and the regular expression for the flickr usernames also probably needs checking. This is essentially WIP.

After a recent comment on my blog by Paul from ITWeb and an IOD presentation I attended, I was interested in what you should and should not be blogging about as the director of a company.

The basics:

As a basic principle a director has a fiduciary duty to the company and the shareholders. This boils down to: you must act in the best interests of the company and shareholders when acting as a director. This would certainly extend to the release of confidential information without proper authorisation. It all becomes much more serious when a company becomes public / widely-held.

So you have to ask yourself.. is the information public knowledge and could it harm the business? Would the other directors of the business he happy with the information you publish?

I found this interesting:

..the information in a blog can be considered to be of an uneven distribution and therefore could be considered prejudicial to others who would not normally find it.

I guess it makes sense, but.. truth is free, information costs.. you have to work at finding relevant information. I don’t see how people can expect to be guaranteed to be made aware of information which is freely available.

When I blog about the ventures I’m involved with I most definitely do not want to be the official voice with the clean clinical PR message.. nobody wants to read that. I think people like a more fun, personal and subjective view and a story.

Often I can’t share all the facts, but I’m betting it’s the story you remember more than the facts.

A while ago I created this for graphjam:

It should be obvious to most of the readers of my blog that the use of “Tigers” and the placement of “Sudan” was purposeful.

Unfortunately half the commenters seem to be dumb people called “Brittany” (guess where she’s from) who are just clever enough to check the wikipedia article on tigers and yet not quite clever enough to grasp the point of the map.

Um, shouldn’t this be on Failblog? There are no tigers in Africa.

and:

Perhaps this should be on failblog due to the completely inaccurate placement of the Sudan.

With first prize going to this gentleman:

Technically most countries in africa carry no international significance. that’s the whole point of this map. and my lol’s.

I only wish I could convince myself that he’s trying to be incredibly sarcastic… but I can’t.

:(

Yesterday Lynnae and I went to the Homemakers Expo and the best thing about it was that I got a free back-issue Popular Mechanics which will live aside the toilet for the next few weeks. The second best thing was that we had media passes and didn’t waste R45 getting in.

I guess the problem with shows like this is that all the big vendors are totally over them. Why else would companies like Defy, Bosch, Smeg etc not be there? I can only imagine (as someone who has a tiny bit of trade show experience) that these companies learnt a long time ago that trade shows cost a bucketload of cash to put together and generally just frustrate your staff who end up having to work on weekends.

Nobody buys anything at trade shows, except, Wonder Mops (and other idiotic things like an egg cracking device).

Wait, picture this: We walk past a stand for something along the lines of “The Amazing Egg Cracker 6000″. Lynnae sighs and wants to walk on, but I, in a trainwreckian desire to hurt myself decide I *have* to know how they sell this thing. I walk over and ask for a demonstration. So the poor girl, who’s run out of eggs, begins “So what’s the problem with cracking eggs? Simple, you crack it on the side of the pan and egg ends up on your fingers, eggshells in the pan, and you invariably break the yolk!” she says, like she’s probably said over a thousand times in the last 5 hours. She looks up at me for that reciprocal head-nod.

I shake my head and say “Not really”.

She’s momentarily stunned by then carries on undeterred, “Well, with the Miracle Egg 6000 that’ll never happen again, you just put the egg in the device and squeeze the handle and instantly you have a perfectly cracked egg.”

“Thanks” I said as I walked away. I imagine that this is a wonderful invention for people with physical disabilities but for everyone else I think you’d be better off just buying 6 eggs and practicing on a Sunday afternoon. I’ve perfected (with a little help from my private chef) the single egg separation technique where you separate the egg white and yolk with the egg shell… Gotta learn how to do that with one hand though.

But, slightly more mind blowing than the egg cracker 6000 is the perpetual rollout of Wonder Mops and Miracle Orange Fibre stuff. The perveyors of this modern day snake oil have their routine so slick that you find yourself entranced by the bright colours and the disturbingly charming man with the Britney Spears microphone and smooth talking demeanor who occationally winks at the woman in the front row.

I was reminded of the old Westerns where the travelling salesmen stands beside his wagon and annouces his miracle cures for athritis and the black lung. Often telling the townsfolk that it is the most popular thing in the big cities and was invented in New York!

If there is one thing I remember from my youth (and we’re talking 20 years ago now) it’s the miracle mops, incredible dirt trapping floor mats and designed by NASA multicoloured dusters being sold at tradeshows (like Design For Living etc)… WHAT THE HELL DOES NASA NEED A MULTICOLOURED DUSTER FOR?

People! Nothing has changed in the past 20 years… Just like your multi-action, triple-flex, anti-bacterial, pro-enzyme, plaque-fighting toothbrush with the matrix-eque 3D graphics advertisements, everything at tradeshows is a big lie to make you pay too much for crap you don’t actually need.

Which is why the people who sell stuff you do actually need aren’t at the trade shows.

over.

More about me here.

Well, it is two months since I sold my Gomoto for R1600.00. At the time that I sold it the frame had begun to rust (because I made it sleep outside too many nights). The battery was flat and the back tyre had a puncture. The state of disrepair was partly the result of me not looking after the bike as I should. I had allowed the stresses and strains of my life to take over and this was even visible on this blog.

I have decided to try something new. I moved to Northern Ireland. I live in a small town, Banbridge in County Down. I work as a pastor of two congregations of the Seventh-day Adventist Church. Twice a week I drive to Larne, another small town on the County Antrim coast. On the way there I drive past this garden feature on one of the roundabouts (We call them "Traffic Circles" in South Africa). It always makes me think of my days on the Gomoto and I really do miss them.

I will probably begin a new blog soon and post a link here. Exactly what theme it will take is still uncertain, but I am playing with the idea of toilets and toilet paper. I might call it "2-Ply". I doubt that many people would read it, but I need a place to document some of my experiences.

For now, I can only thank Gomoto for the good experience I had on one of their bikes. One day I'll return and buy myself a new one. Thanks to all my readers. I hope you enjoy your Gomoto's and please try to be adventurous as possible on them.

Godspeed

What is exciting about Cosmology today is how much we do not know. The observable universe - baryonic matter we are fairly sure comprises only a few percent of the total mass of the universe. The rest is a mystery, but mainstream theories split it between Dark Matter and Dark Energy.

I work at the African Institute of Mathematical Sciences (AIMS), and we are blessed with some great visiting lecturers, and I have followed the courses on Cosmology and Quantum Mechanics with interest. I have written about AIMS and their new research centre before.

Cosmological Scale

There have been recent discoveries, newer than the Bullet Cluster, which seem to confirm the presence of Dark Matter.

Since we can't 'see' Dark Matter, where do these observations come from? The most promising method is through Gravitational Microlensing, similar to passing a weak magnifying glass over a desk. A large massive object, whether visible (baryonic) or not (Dark Matter) will bend light waves coming from directly behind the object. It is not really the same as an optical lens, because light passing closest to the centre bends further than that further out.

But - it does increase the apparent brightness of objects directly behind. If we observe the objects over time, our magnifying glass will show varying brightness as the lens and target shift from our perspective. From these calculations can be made of the mass of the gravitational lens. So, observing using gravitational microlensing requires a time dimension to viewing - a historical survey perhaps.

Separating out the observable mass (stars and galaxies) we can see the distribution of non-observable mass. The MCSJ0025 cluster referred to above appears to show two galaxy clusters colliding. When galaxies collide, they mostly pass right through each other - odds are against one star going Blam! right into another. However, the interstellar gas does interact on a local scale, 'tangling' the visible portions of the galaxy clusters.

The Hubble Space Telescope and Chandra X-ray Observatory appear to show that the Dark Matter portions of the galaxy clusters - significant invisible lumpiness that is gravitationally attracted to the cluster - passes straight through each other - flying out the other side through the windscreen, as it were. That is the portion coloured blue in the pictures above.

Quantum scale

At the Quantum scale other intriguing discoveries are being made - watch that space (not this one).

There is another aspect of mass that we completely fail to understand. The baryons that make up the vast majority of our tangible world are protons and neutrons - both made up of three quarks - UUD and UDD. However, if their masses are compared to their components :-

• Proton - 938 MeV
• Neutron - 939 MeV
• Up Quark - 1.5 – 4.0 MeV
• Down Quark - 4 – 8 MeV

You can see the massive disparity. The strong force appears to have multiplied the mass of the component quarks a hundred-fold. Quark masses cannot be measured in isolation, as they seem always bound into Hadrons. Their relative masses can be calculated though.

We understand our ignorance in this area. There are huge discoveries to be made that might be as revolutionary as General Relativity. Maybe Weakly Interacting Massive Particles (WIMPs) that are theorised to be behind Dark Matter glue in at the Quantum level to add mass to Hadrons. Or maybe I display my ignorance by even suggesting that.

Nobody really knows.

…are trains cancelled due to rough seas.

(Click to view; taken from the Cape Metrorail Traffic Report.)

For those of you not familiar with the Simon’s Town line, this is actually entirely reasonable. In places, the railway line runs right next to the beach; with a spring tide and a heavy storm surge, the sea can wash right over the tracks.

My week in bullet form..

• Monday, storage space shopping, sunset run in Mouille Point, sushi with friends.
• Tuesday, packing, sorting and getting rid of old and unused stuff, Lions Head walk, drinks at Sinn’s, pizza at Knead, Jazz at Asoka, drinks with ma.tt.
• Wednesday, gym, house shopping in Bo-Kaap, heavy chef meeting at 24.com, Jason gave me a WordCamp t-shirt, turns out Rob reads my blog.. Hi Rob!, 27 Dinner, nice Japanese salad at Balducci’s with Cath.
• Thursday, lunch with Chris, chatted about WordPress things mostly, cable car run, 5km at ~5min/km pace.
• Friday, packing and moving, Newlands forest walk with Meebs, sushi at the Vineyard Hotel with the ~ex-wife.
• Saturday, packing and moving, Betty’s Bay with Mia, Georg, Cath and Parri.
• Sunday, breakfast in Betty’s, afternoon in Jonkershoek for Debby’s birthday, drinks in Stellenbosch.

Moving is not fun.

These days, you can hardly get any service without having to sign some kind of service contract. I make a point of reading every single one of them. I hate signing them, because they often have clauses in them that I consider unreasonable. Yet, I put up with it and sign it, since it would just be too big a deal to argue against those clauses, especially when I need to have the service right now.

These corporations also bend and twist and mangle the laws to such an extent that the laws are essentially defeated, and as consumers our rights are compromised.

I was happy to read on Slashdot today that the Washington State Supreme Court ruled that AT&T’s service agreement may not waiver a client’s right to file a lawsuit against the company.

“AT&T’s Consumer Services Agreement is substantively unconscionable and therefore unenforceable to the extent that it purports to waive the right to class actions, require confidentiality, shorten the Washington Consumer Protection Act statute of limitations, and limit availability of attorney fees. … Courts will not be easily deceived by attempts to unilaterally strip away consumer protections and remedies by efforts to cloak the waiver of important rights under an arbitration clause.”

The full ruling is available in PDF format.

I think that the big corporates who attempt to downplay our rights are scumbags, and the people who take the time to deal with them and bring them to justice are my heroes.

(rant)

I’m so frustrated with SARS (South African Revenue Service). They sent me an e-mail to notify me that I’ve received communication from them that require my attention. I click on the link which takes me to their website, enter my username and password, and it takes me to a page where I have to download the contents of the documentation in a PDF file. So, I do so.

When I opened the file with my PDF viewer, and it displayed that I require the latest version of Adobe Reader. This was already a bit frustrating, having to do so much work and still not being able to read my message. At least Adobe reader is packaged in Medibuntu, so I add the repositories and install it. 20 minutes worth of downloads later, I open the PDF. Yet again, it displayed “You need the latest version of Adobe reader to open this file”. Grrr.

I head over to Adobe.com, saw a link that says “Adobe Reader download”, where it also said “Latest Version”, and downloaded it. Turns out it is also Version 8.1.2, the same version available from Medibuntu, and it also doesn’t want to open the document.

After trying all different kinds of things, I’m still unable to open the PDF. Thanks Adobe, for implementing such horrible technologies in our government, and thank you SARS, for absolutely not caring about free software users, even when our government has a mandate to move over to free software itself.

Then again, it’s not like any government ever had a good reputation for keeping its promises. I don’t know how I expected ours to be any different.

A friend and former colleague of mine uploaded a photo of a whiteboard to Flickr today he took a couple of years back in the office we used to work from. We were both protesting all the public holidays we experience in South Africa, each costing the economy lots of money. I mean, as if having two sevenths of your days off (the weekends) is not enough. Yes, we were geeks without broadband at home.

The whiteboard reads something like this. I don't know, maybe this is abuse of the blockquote element.

Petition against silly public holidays!

• Charl
• Aryan

We shall refuse to stay home!

We shall come to work by force!

Yes, we are geeks and we are militant. :P

Akismet

By the time I write this, Akismet will have caught a quarter of a million of spam entries on my blog. Thanks again, Akismet!

Michelle Obama

This morning while driving to work, I heard excerpts from Michelle Obama’s speech that she delivered in Denver yesterday. I haven’t cried in nearly 13 years, but every now and again something happens that brings me close, that speech got brought me one of those moments. I think Michelle Obama will make an awesome first lady.

Monopoly World Edition

Hasbro (the company behind the popular game, Monopoly), will be releasing a new version of the game that contains popular cities around the world. The cities were chosen by votes on their website, and they have made the results public. My home town, Cape Town, got in at number 3. Awesome.

Perhaps entirely randomly, I decided that TechGeneral would need Sitemaps before I put it live.

A Sitemap (sometimes called a Google Sitemap, although you won't see Google calling it that, and it is a standard that Yahoo!, Ask, and Live all support) is an XML file (or bunch of XML files) that describe the various resources on your web site which allows search engines and other programs to discover them more easily.

There are a few advantages to putting together a Sitemap.  Generally, search engines give up after they travel a few links into a web site to avoid infinite automatically generated links (not because of malicious intent necessarily, but because of weird programming).  With a Sitemap, each listed resource can potentially be treated as a first visit.  Also, if a site has navigation that search engines can't traverse to get to certain pages, Sitemaps can assist search engines to find those resources.

They also optionally assign a priority to each resource as a way to influence the importance assigned to the resource relative to other resources on your web site.  Similarly, an optional update frequency per resource can influence how often a search engine or other program should check back for new versions of that resource.  Last modified dates also optionally help to determine whether to try revisit a resource earlier or later than would normally happen.

Example Sitemap File

<urlset
    xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9
        http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd">
 
    <url>
        <loc>http://techgeneral.org/diary</loc>
        <lastmod>2008-08-16T22:52:41+00:00</lastmod>
        <changefreq>daily</changefreq>
        <priority>0.9</priority>
    </url>
 
    <url>
        <loc>http://techgeneral.org/speaking</loc>
        <lastmod>2008-08-16T22:52:13+00:00</lastmod>
        <changefreq>daily</changefreq>
        <priority>0.9</priority>
    </url>
 
    <url>
        <loc>http://techgeneral.org/contact</loc>
        <lastmod>2008-08-10T16:59:32+00:00</lastmod>
        <changefreq>weekly</changefreq>
        <priority>0.9</priority>
    </url>
 
    <url>
        <loc>http://techgeneral.org/about</loc>
        <lastmod>2008-08-10T12:42:06+00:00</lastmod>
        <changefreq>weekly</changefreq>
        <priority>0.9</priority>
    </url>
</urlset>

There are two types of Sitemaps - individual Sitemap files and Sitemap Index files.  Why would you want a Sitemap Index?  One, less relevant to many, reason is that individual Sitemap files can only contain 50 000 URLs (which, admittedly, the average blog isn't going to hit) and be less than 10MB uncompressed.  Another reason is that you might be using multiple systems that each generate Sitemap files (or you've hacked them to do so) but you don't want to merge them yourself.

Example Sitemap Index

<sitemapindex
    xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9
        http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd">
 
    <sitemap>
        <loc>http://techgeneral.org/sitemap_posts.xml</loc>
    </sitemap>
 
    <sitemap>
        <loc>http://techgeneral.org/sitemap_archives.xml</loc>
    </sitemap>
 
    <sitemap>
        <loc>http://techgeneral.org/sitemap_pages.xml</loc>
    </sitemap>
</sitemapindex>

One useful side-effect of using a Sitemap with Google's webmaster tools is that you can see errors that occur on resources listed in the Sitemap.  So, if a request for a resource starts returning 404 or 500 errors, you can separate that more specific set of errors from those caused by broken links on your site or on other sites.

However, Google's webmaster tools doesn't seem to like having a whole bunch of separate Sitemap files with a central Sitemap Index.  I mean, it seems to work, but it complains (warnings, not errors) that many of the Sitemaps (all on this site, most on my personal web site) have only entries with the same priority.  I'm setting the priority of all the archives low (they have noindex, follow set anyway, so won't show up in search results), the frontpage high, and the posts are priorities based on age.

I get the feeling that the priorities only apply within the same file, and not within the same site.  This somewhat makes sense, since one can delegate a sitemap for a particular folder on your web site, and you wouldn't want an overeager person assigning "1.0" to all content within the folder, overriding your beautifully crafted values for the base site.  However, in this case, they're all at the same level, and I really want the archives lower than the posts, and the frontpage higher than most of the posts.

Oh well, I'll push on and see whether it's just a matter of warnings that aren't affecting things (my favourite kind) or an indication of things being as I suspect.

At WordCamp South Africa 2008, held in Cape Town yesterday, we were given a brief overview of how Wordpress.com is set up to scale.

Matt Mullenweg set the scene with some idea of just how huge Wordpress.com is.  I may mess up a few numbers mentioned, but there've been something like 6.5 billion page views on Wordpress.com since the beginning of the year, there are 3.8 million Wordpress.com hosted blogs (only Blogger is bigger), and there are 1.4 billion words in posts created on Wordpress.com.

Warwick Poole then gave us some more in-depth numbers, although pointing out that Wordpress.com was bigger than AdultFriendFinder was a pretty good and well-understood indication from the audience's reaction.  In May 2008, Wordpress.com was served 693 million page views, but this rose to 812 million page views in July.  Over 1TB of media was uploaded in May, 1.3TB in July.  In May, 417TB of traffic left the Wordpress.com data centres.  These numbers are available in the "July wrap-up" post on the Wordpress.com web log.

Apparently, across the approximately 710 servers, 10 000 web requests and 10 000 databases requests are handled per second (I wasn't intelligent to write down whether this was the average).  110 requests per second are done to Amazon's S3 storage service, while 3TB of media is cached on their own media caches.  They output 1.5TB/s (I wrote TB, so it probably is TB and not Tb.  I'm guessing this is peak). They experience approximately 5 server failures a week.

How is it put together?  They use Round Robin DNS which determines the data centre (from testing, it seems there round robin six IPs - two IPs for each of three data centres).  There it hits a load balancer using some combination of nginx, wackamole, and spread.  They use Varnish for serving at least media, and currently use Litespeed web servers.  They also use MySQL and memcached.

They use (and developed) the batcache Wordpress plugin to serve content from memcached - according to the documentation, batcache only potentially servers stale content to first-time visitors - visitors who have interacted with the web log receive up to date content.

When new media is uploaded, its existence and initial location is stored in a table.  As necessary, the other data centres will create their own local copies of that media, and update that table.  The backup media stores in the data centres are write-only - apparently nothing is ever deleted from them.

That's about all I wrote down, but there's quite a bit of information about how Wordpress.com is set up and the sort of load/traffic it has on the Wordpress.com blog and on the blogs of various employees (such as this post on nginx replacing Pound, this one on Pound, and another on varnish) giving some useful information which will probably inform some technology choices we might make at SynthaSite.

People that know me well will so appreciate this. :)

Good version control system usage prevents many disasters, but that doesn't necessarily mean you won't make your own mistakes.  Today, I mistakenly included a file in a commit that I didn't want to commit yet.  I learned two new tricks while spending a few minutes puzzling the best way to get back to where I was before with that file.

First, make a mistake:

$ svn commit -m "..."
Sending        dev.cfg
Sending        gibe/plugin.py
Transmitting file data ..
Committed revision 114.

svn merge is the tool to use for this:

merge: Apply the differences between two sources to a working copy path.
usage: 1. merge sourceURL1[@N] sourceURL2[@M] [WCPATH]
       2. merge sourceWCPATH1@N sourceWCPATH2@M [WCPATH]
       3. merge [-c M | -r N:M] SOURCE[@REV] [WCPATH]

Trick #1: use svn merge's 3rd usage pattern with the -c option with the negative of the revision you've committed, and (here comes the trick) use . (the current directory) as the source of the merge:

$ svn merge -c -114 .
U    gibe/plugin.py
U    dev.cfg

With that your working copy is now where the repository was before your commit.  Commit that to the repository, and the repository is back where it was before your commit.

Now your working copy is where it was before you made any changes - but you probably want those changes back.  Easy enough:

$ svn merge -c 114 .
U    gibe/plugin.py
U    dev.cfg

Now your working copy is back where it was before you did the mistaken commit.

Trick #2: Of course, if your mistake is like mine and you only messed up one file and everything else is as it should be, you can just do this on one file, by using svn merge's 2nd usage pattern:

$ svn merge dev.cfg@114 dev.cfg@113
U    dev.cfg

Commit that, and your repository is back to normal.  Then run:

$ svn merge dev.cfg@113 dev.cfg@114
U  dev.cfg

Now the file is back where it was before your botch.

Just received my Instructor's Manual (only a week late!) for Sun's SL-425 "Architecting and Designing J2EE Applications"¹ and I'm very happy to see that it is basically the same course as the old "Architecture and Design" course I taught several times lo' those many years ago when I was so frequently on my feet as "Herr Instructor".

This is easily the best² course I ever "taught"! It is aimed a senior, experienced designers and developers³, and confronts head-on the sticky few-good-answers stuff, the ill-defined and the fuzzy grey areas. I have always run the course in a round-table "workshop" format. When you get 8 or 10 senior developers into a room, each with a decade or two of experience, you're not going to be their Teacher. And you sure as hell better not have a tender ego. "I don't know" is a frequent answer. The job is much more one of facilitation: Keeping discussion on-track, drawing quieter participants into the discussion, acknowledging expertise and encouraging people to share their (often vast!) experience. I found it hugely enjoyable to engage with seriously expert people, and to facilitate drawing out their expertise. And I learned a lot!

It has been a very long time that I have wanted to run this course again, so I'm looking forward to it hugely!

So: I have a couple of weeks to catch-up on the changes since last it taught facilitated this course. Mainly the technology has caught up with the tech: Where there was only CORBA or RMI a decade ago, there are now a host of J2EE technologies, and the course now includes them. Where a decade ago the whole idea of enterprise-architecture patterns was pretty new and unheard-of, now there's the J2EE Blueprints Catalogue (even if those yanks can't spell "Catalogue"!)

Of course I'll probably be unable to resist the temptation fo slipping in a few sly teasers about Jini and Javaspaces -- but Hey! Sun seem to encourage instructors to go beyond the boundaries of the course material, and encourage us to bring our own experience into the classroom. Or workshop in this case.

I am still thinking about floating my own version of this course, perhaps a little less attached to Java tech, so that more people might engage. The absolutely best runs of the course were when we had many people from different organisations and backgrounds. Cross-pollination really works. Ask plants!
----
[1] Is there such a word as "Architecting"? My spelling-chequer doesn't seam to think sew.

[2] "Best" from my point of view, anyway. Though, I can honestly report that every participant I've ever had on this course reported exactly the same sentiment!

[3] I don't believe the term/job-title "Architect" had any currency back then. Whilst it had certainly been invented4 it was certainly not popular. In fact there still was no such Job Title as "Software Designer" back then. My boss had to invent it for me!

[4] I believe that I was one of the (probably many) inventors of "Software Architect"a s a job title. Certainly not unique in that, though! Back in 1989 a technical career-path looked much like "Junior Programmer - Programmer - SeniorProgramer - Junior Systems Analyst - Systems Analyst - Senior Systems Analyst - Business Analyst..." I rejected the whole deal⁵ stating "Analysis is The Art of Taking Things Apart. I don't want to do that. I want to put things together, and that's called Design (and, at the high-end, Architecture)."

[5] ... in a bi-annual merit-assessment that became (in)famous as the one where I told my Manager, "If you want loyalty, get a dog! This is a business relationship; didn't you understand that?" Needless to say I got no increase that year. ;-)