I have recently been contracting at the Centre for High Performance Computing. This is just an update on how to map the particular problem I have onto the computing cluster.

What this is not

This is not a complicated expose of a whole computer cluster co-operating on a single task.

First steps

It is an explanation of how to get to the much easier job of splitting the main task up into bite-sized pieces that can be fed independently via a job submission system called MOAB.

Task description

Each day satellites MODIS (NASA) and MERIS (ESA) do a few passes of a polar orbit over our region of interest, African coastal and inland waters. The (large amounts) of daily data arrive via a dedicated satellite link - about 1 Gig every day. They are in "swaths", about 20 or so files from 10Meg to 800Meg.

We need to keep all this input data. It is a few terabytes and growing of several years of historical data.

This needs to be processed - mostly to cut it up and combine swaths to get smaller files of country-specific data, and a PNG picture of some attributes in the country netcdf file.

So I have a python script that reads the input files, brackets them by latitude and longitude, re-grids (lossily) the data onto a north-south grid as opposed to 'along-track' delivered data, writes the netcdf and png.

Collecting the data

All the input data was on a number of other machines in the oceanography department. The supercomputer cluster has four 79 terabyte cluster drives, accessible from all nodes. The network drives are themselves a distributed network of directory nodes and storage nodes, for speed and parallel access. See LUSTRE filesystems.

Each node runs linux, and can directly access these drives. For organisational purposes, I create a heirarchy of SATELLITE/YEAR/MONTH/* and copy all the terabytes down.

Python

I will skip the off-topic but lengthy installation of python 2.6.5 on the NFS read-only share. Consider it done :)

I can now split the problem up by day, and satellite. I have a shell script that copies and decompresses a day's input files to a 'scratch' area, and feeds that days-worth of files to a python program, which processes it repeatedly for each of our Regions of Interest, dropping the output files in the same directory. It can take up to about 6 hours to process a days files - mostly spent on the regridding operation.

You can see in this png of the Western coast of South Africa two separate swaths, and artifacts caused by the regridding operation at the lower resolution edges of the swath. MODIS orbits south to north, MERIS (above) the other way. Most of the gaps are clouds over the sea at the time of the satellite overpass.

MERIS have their own file format, but provide a java program to convert that to hdf5, a more widely recognised format that has python modules for input and output. Luckily the cluster has java :)

The shell script then copies the results back to a target file heirarchy, and deletes the input files, and finishes.

Firewall

The firewall setup around the computing cluster prevents any machine in the cluster from making network connections out. When logging into the cluster, you 'land' on the login node, and from there you can ssh to any node in the cluster on its private 10.* Infiniband network. So copying the output data to the webserver must be orchestrated from outside.

MOAB

The product page talks about graphical installations and point and click, but I prefer the command line. Usually the GUI just pokes command line tools anyway, so you are closer to the metal as well as being convenient for remote use.

The submission tool, msub, takes as parameters a number of accounting switches, where to send script output (stdout and stderr), target blade classes, and a host of other things including my script.

A problem comes because I cannot pass parameters to the script on the command line. So, if I want to tell it which day to process, I have to use an environment variable.

There are ways of naming environment variables to be passed through, but now it is getting complicated. I chose to use the 'jobname' - a simple string available in the accounting switches that I can set to anything I like. I set it to the date - for instance "2010:02:29" is a leap-day.

You can embed all the msub switches (like expected job run time, feature set of target blades needed, stdin and stout) in the script itself, prefixed by "#MSUB", which is mighty convenient. so now submitting a day's job becomes

msub -N 2010:08:11 ~/bin/meris.sh

and a trivial loop will do a month.

moab is probably sniggering at me for submitting so many single-threaded jobs and nothing more complex. I may even be penalised in the queue for jobs, but it works fine for me.

Loading and unloading

It is convenient to keep all the historical input data on the cluster filesystems, and that means setting up a daily job to rsync the satellite data as it arrives down to the cluster.

Output is currently served by the MRSU webserver at http://www.afro-sea.org.za/ - and thus needs the data copied off the cluster to be served. Because of the difficulty of synchronising, I actually do the daily processing on a beefy desktop outside that is otherwise idle, just because all the network copying can be coordinated there, and I can use cron.

There is a plan to make the site more dynamic, where you could request a custom region, at which point I would consider poking the supercomputer cluster with long spoons from the webserver. First - we need users.

Users

This is a bit double-edged - probably the folks most interested in the daily information would be foreign fishing vessels :) Regardless, potentially interested parties must be made aware of this resource funded by the taxpayer.

Some weeks I wake up and I’m somebody’s dad. Some weeks I wake up and I’m a kid on an adventure…

• Monday, woke up in Athens.. babies don’t sleep this well, only got out of the hotel at 2pm, no point if the light is bad for photography right?, blogged, found a new hotel, the first one was in an area that looked like Mogadishu at night, train to Piraeus and back.. harbour area, good time to read the travel guide, Greek salad and octopus legs with a Mythos beer, Roman Agora, bought a few tshirts, followed my nose for a while, Hadrian’s Arch, Temple of Olympian Zeus, Zappeion Hall, Panathinaiko Stadium, National Gardens, Parliament, a 10 Euro Gin Martini on a rooftop in Gazi, a beer at Gazi Collage again, picked up a fresh orange juice on the way to the hotel.
• I think I need a Cape Town sticker for my camera bag.. but the official Cape Town tourism branding may look a bit gay.
• Pet hate: tourists who hang around monuments (inconsiderately) when I want to take photos.
• Tuesday, pretty useless hotel breakfast, moved to a new hotel, National Archaeological Museum, a late lunch, nap, the 16mm and I went to explore the Ancient Agora, The Acropolis, watched the sunset from the Nike Temple, g&t on the balcony overlooking the Agora and the Pantheon, Jacques arrived late.. he fell off his bike on the way to the airport.. on the highway, we went out for a few drinks.
• Maybe we should go to NZ for the rugby world cup in Sept 2011.
• Nice music on the subway in Athens, Dave Brubeck, etc.
• Wednesday, got up late, Jacques got himself a Fear and Loathing hat and shades, I got myself 3 pairs of sunglasses, lunch, nap, we solved the world’s problems on the roof of our hotel.

• “Truth seeking is a dangerous business.” — Jacques

• Lawyer: What about that tattoo on your chest, doesn’t it say “DIE BART DIE”?
  Bob: “No! That’s German…for “Die Bart Die” (Courtroom laughs.)”
  “No one that speaks German can be a bad man.” — Simpsons

• When I’m not really making phone calls my phone can go about 8 days on a single charge.. Jacques does not even get one day with his iPhone.

• “Somewhere in the world it’s 5pm” — about opening a beer at 11am.

• “Ek’s op ‘n pakkie ‘n dag.. Nurofen.” — Jacques about toothache.

• “Buy then there is blood on the streets.” — Jacques about property investment.

• Thursday, Jacques’s bag got stolen in a cafe with his passport in it, took a ferry to Mykonos.. Jacques was not happy, so we depleted the ferry’s mini-bar sized Johnnie Red and there was probably a bit more Afrikaans swearing than necessary, two friendly girls took us to their family run hotel, had a swim.

• “Soos on nou geleer het uit Californication.. floozies is nie die antwoord nie.” — Jacques, about relationships

• I met a girl who within a day pretty much knew everything about me. To be mysterious or not?.. I think I have more personal information online than anybody I know, but I’d still like to think I’m not giving away the whole story.
• Did I mention how hot the woman in Athens are, even the Police woman are hot.
• Street names are a bitch in Greece, often a street has three names.
• Why did I come to Greece?.. because I enjoyed the Egyptian history lesson or because Mia watched Mamma Mia over and over?
• Focus. Relax. The two most frequent (opposing) instructions Mia gets from her dad.
• I really need a travel companion who likes to read travel guides and plan things. Maybe Mia will be into that.
• Seems Santorini is one of the islands where Atlantis is believed to have been.
• I think I need to meet a girl for whom I’d like to make compilation CDs again.
• Mykonos is pure hedonism.
• Jacques thinks it’s strange that I tell people I’m from Cape Town and not South Africa.
• Friday, woke up on an island, a nice breakfast overlooking the sea, walked to the closest scooter rental place, hired at 50cc quad bike.. which was not designed to carry 210kg up a hill so Jacques had to walk a few times, cruised around or the right-hand side of the road without a helmet, this country really needs to update their laws to allow 125cc bikes without a license, had lunch next to the beach, walked around the old harbour town, nap, sundowner by the pool, walked to a restaurant close to the hotel and had Moussaka, Jacques kept me awake with a crazy romantic plan to cure me of Naulene.
• Saturday, had a Red Bull for breakfast.. needed the B vitamins, took the bike to Ornos beach, Mykonos old harbour, had some Swordfish fillet at a small shoreline cafe, took the orange lawnmower over the island to Super Paradise Beach.. which turned out to be a (half) nudist beach, got some rays, crazy sundowner beach party, Cob and garlic mash for supper, figured it was a bad idea to go to Space Dance, g&t’s and some photo processing.
• Sunday, greek yoghurt with honey, Paradise Beach, a Scuba dive at Paradise Point with an Afrikaans dive instructor from Somerset West, got some rays, Paradise beach party at Tropicana, Jacques convinced some people he was from Texas and I was from California, saw the hottest woman I’ve ever seen (true story), headed back to our local restaurant for supper, g&t’s at the hotel.
• Seems Jacques is pretty screwed without a passport. He’s taking a longer than planned holiday.
• My current travelling setup is pretty efficient. I guess I could go without the laptop and the heavy camera, but at least this way I can process my photos while travelling and not end up with a backlog when I get home.
• The SA dive instructor figures it’s a good idea to hit Thailand (Phi Phi islands) in January.

• “Trust me, it’s paradise. This is where the hungry come to feed. For mine is a generation that circles the globe and searches for something we haven’t tried before. So never refuse an invitation, never resist the unfamiliar, never fail to be polite and never outstay the welcome. Just keep your mind open and suck in the experience. And if it hurts, you know what? It’s probably worth it.” — Richard, The Beach.. goodbye Mykonos

• Next: I’m in London for a week, entertaining Mia.

Have a fun week, crazy kids.

There have been a number of countries, like UAE and India that have demanded access to encrypted communications of Research-In-Motion's (RIM) BlackBerry smartphones. These efforts are misguided, and unfairly target RIM's business.

Public key encryption

We need a quick primer on today's encryption. We pick the standard scenario where Alice wants to talk to Bob, and Charlie is trying to listen in.

In the old days Alice had to get a 'secret' to Bob before they can chatter. This was vulnerable because there was always the possibility that Charlie could intercept that initial exchange.

Nowadays, public key encryption makes that unnecessary. Bob can publish a (large) number, in his newspaper, or on his business card. This Bob's public key. If Alice uses some well-known, open-source software like GPG that even Charlie has, Alice can send a message to Bob that only Bob (with a second, private large number key) can decrypt.

Let us be clear. The code that does the encryption and decryption is open, available for inspection by Charlie, who may spend years with scientists and huge computers and will still never break the code. The secret to decoding the message is Bob's private key, which he has never given to anyone.

Duality of the key pair

The two keys, Bob's Private and Public key, are a dual - what is done by one can only be undone by the other. There are infinitely many of these key pairs, and when you need them they can be generated. The bigger these numbers are, the harder it is to decrypt. The numbers can be made up to any size.

They can be used for signatures too - Bob can create an email, and sign it - attach a (very big) number to that email created with his private key. Anyone, like Alice or Charlie, can verify that only someone with access to Bob's private key could have 'guessed' the number Bob put at the bottom of the Email - proving that Bob wrote it.

Secure communication protocols

Public key encryption is at the cornerstone of SSL - the secure wrapper to a number of protocols - https:// that is used to access your banking website, ssmtp for email submission, and imaps for email retrieval to name a few. It is a very well studied mathematical science - we have a pretty good idea what America's National Security Agency is able to do concerning cracking these protocols.

Blackberry phones

Research-In-Motion have two types of customers - corporate, and casual smartphone buyers that like the BlackBerry phone. For corporate customers, they install the BlackBerry Enterprise Server (BES) - a computer at their premises that interfaces to their corporate mail server and securely delivers the mail to the smartphone.

This traffic is encrypted at the BES (Alice), send over RIM's network and the public phone network (Charlie) and decrypted on Bob's smartphone - using the private key that has never left the innards of his phone. RIM corporate clients know this, and that is why they buy RIMs devices.

Snooping traffic

The only way the message could be read by Charlie is if he installed snooping software on either the BES or the phone itself. UAE tried to install at the phone, but they were outed. RIM go into exhaustive detail on the steps used to generate and exchange the keys. If this process is followed nobody except that corporation can read those messages. Especially not Charlie, RIM themselves, or the UAE or Indian government.

With proper attention to detail, SSL protocols like SSMTP and IMAPS can be made just as secure, with no possibilities of eavesdropping. RIM has just come under the spotlight because its business model is based on this security. My Nokia E71 can access my mailserver, and the traffic between the two is secured as tightly as RIM's service. What is different is that key management on my phone is too sloppy. A Corporate Nokia customer could request more strictness.

BlackBerry Internet Service

The BlackBerry Internet Service is the service non-corporate individuals get if they just buy a BlackBerry at the corner store. Depending where that server is located, and who has access to it, will entirely determine the security of the service. If Charlie controls this server, then naturally Alice and Bob can expect no privacy.

If the UAE or India lean too hard on RIM, hard enough to either control the key generation process so private keys are revealed, or the numbers representing the keys are small enough to be broken by a large computer, or if either the BES or smartphone have spyware installed, there will no longer be any reason for corporates to buy their service, and RIM will go out of business in those markets.

Security-conscious Blackberry buyers will instead buy another smartphone, most of which (with attention to detail) can be made just as secure to eavesdroppers. The government will have gained very little in snooping on someone determined to keep their privacy.

Android

Furthermore, the new Google smartphone OS, Android, is open-source. Soon individuals or companies will be able to install their own version of the same encryption software RIM uses and neither their network provider, cellphone provider or government will be able to do anything about it.

Conclusions

Perhaps India and the UAE are just going after the conveniently-packaged security-in-a-box that the casual smartphone user has been able to buy in the BlackBerry brand - and they want to at least deny that to their perceived enemies. But it is a limited solution, will not stand the test of time, and unfairly targets one company - RIM.

As a user, don't stand for this bullying by the governments. Take your privacy and security seriously. Hold companies like Yahoo and Google and Facebook and Twitter accountable for your privacy using their services, or use them understanding that it might be, and probably is, routinely snooped.

Until I can use the same software I use on my two computers, on a mobile phone, I don’t consider the “smart phone” to be a general purpose computing device. Paul Graham has a similar metric, he wants the device to be capable of hosting it’s own development environment.

Graham, among many others, has done a good job of describing how Apple has crippled the iPhone in this regard by locking down the platform and imposing a broken app store model to “publish” software that runs on it. Instead of re-hashing the ecosystem issues surrounding mobile phone platforms, I have a technical wish list:

1. Hardware and software feature parity with best smart phones out there:
   • Quality touch interface, high resolution display, good battery life, all the right radio hardware (GPS, GSM, Wifi, UMTS), sufficient storage and CPU power
   • Software to support standard phone use-cases out of the box: turn-by-turn navigation, tethering, web browsing, full multimedia support, camera functions, etc.
2. The same Open Source UNIX-like operating system I use on other platforms: this happens to be Linux
3. A familiar, quality package system like Debian’s.
4. The ability to cross-compile and run any application I could use on my PC (this implies X11 support).
5. Support for standard peripherals (USB host/otg).

Neither the iPhone, nor any Android-based phone meet these criteria.

The Nokia N900 comes very close, but it’s still missing software support for some standard smart phone use cases. Although I’m a bit disappointed about the Maemo -> Meego move throwing dpkg out in favor of RPM, I still have hope for the Nokia, and I’m interested to see them finish the software (Hi Edwin).

Finally, I want hardware support for an external display. That might sound weird, but if a phone is really a general purpose portable computing device, then you should be able to “dock” it at your desk, like one does a laptop. Maybe I don’t “get” smart phones, but I prefer the view that the world is wrong.

Anyone with me on this? I would appreciate some comments.

Update: The Nokia N9 prototype running Meego seems promising. I guess we’ll see when it’s launched.

One of the weird ironies about this blog has been that although it is focused on networking in South Africa, the web server hosting it for the last ~two years was a Linode.com virtual machine in New Jersey, USA. The server’s IPv6 connectivity was via tunnel to HE.net in New York.

Now, thanks to the generosity of my friends at IPINX, this blog is hosted in Midrand, Gauteng and it has native IPv6. I would guess this is probably one of the few web servers in South Africa with native v6 outside of AS2018.

I took the opportunity to upgrade to the latest WordPress. If you spot anything thats broken, please let me know.

I’m writing this one from Athens..

• ~30 hours in the air this week.
• Monday, backups, camera battery charging, gym, swim, lunch at Balduci’s with Jonathan and hardware crypto vendors, bought a UPS and installed it for my workstation, last minute packing, packed Mia’s bee suitcase, Georg gave Mia and I a lift to the airport, flew to London with BA, Club World seats are pretty awesome, had a Kir Royale while listening to Blonde on Blonde, read a book about Greece, watched Sherlock Holmes.. not bad, Mia watched How to train your dragon again and slept for a good 7 hours.
• 11 hour flight, 1 hour backwards. Must say the Boeing 747-400 is like a flying fortress, smoothest take-off ever.
• Alexander the Great died when he was 33. Impressive.
• Tuesday, we woke up to land at Heathrow, tea, took a bus to Stansted airport to drop Mia off with Naulene, train to London CBD, met Jacques for a sushi lunch, work, London curry and some Johnnie Black.
• Wednesday, woke up at Jacques house, flew from Heathrow to Boston Logan with American Airlines (not something I’ll easily do again), sat next to a Alvaro, who is from Guatemala.. doing a PhD related to the flexible electronics market at MIT, bus and train ride to Cambridge, nice place, found a guest house, took a walk down Mass. Ave.
• 7 hour flight, 6 hours backwards.
• Mia went to Cambridge for a week, then Germany for a week. So both Joe and Mia were in Cambridge, just not the same Cambridge. I phoned her, she seemed happy.
• Thursday,VRM+CRM 2010 day1 at Harvard Law School, dinner at a seafood restaurant with the conference people, when we finished dinner it was 3:30 SA time.. tired.
• Cambridge is a bit like Stellenbosch, only there is free wifi pretty much anywhere and everybody has an iPhone4.
• Friday, VRM+CRM 2010 day2, a sunset walk.
• US food: Naked Smoothies and the salads at Harvard were great. I tried a Dunkin’ Doughnut (half), had a Big Mac, had (most of) a Hershey bar, wanted to try Wendy’s but did not get around to it.
• I figure people like order. Deterministic outcome. After a week of creating order it’s time to go experience some unknowns. A ten day mini-break in Greece. Time to go visit the birth place of democracy.
• A thought about visa applications from countries who are very strict about their visas: I wonder what percentage of their own citizens would make it through their visa process if they were not born there.
• Alvaro made an interesting comment about not really knowing where home is.. South America, UK, US? I think you don’t feel at home anymore if you leave a place for more than 5 years. Alvaro also commented that the most popular technologies are usually not the best technologies.
• Saturday, blogged about VRM things, a tourist/photography day, took the train to MIT, found the MIT Media Lab, had McDonalds for lunch.. seemed like an experience you need to have at least once in the US.. they did not ask me if I wanted to supersize my meal, train/bus to Logan airport, flew to Heathrow with BA, watched She’s out of my league.. pretty good.
• 7 hour flight, 6 hours forward.
• Sunday, flew from Heathrow to Athens, train ride to the centre of Athens, found a hotel, nap, took my camera and explored on foot, had a salad and a beer at a cafe in the Gazi area.. nice vibe, meandered along some side streets and walked past the Acropolis.
• 5 hour flight, 2 hours forward.
• I have 4 different currencies in my wallet. Finding the right coin is silly.
• Travel tip: don’t wear a belt or shoes, just easier for airport security.
• I think my body clock must be a bit confused, but at least I’m not feeling jet-lagged.
• Tune of the week: Why Don’t You Get A Job? – Offspring, reminds me of somebody I know (-;

Have a fun week, crazy kids.

Time for a bit of adventure again, kids. I need some new travel photos.

Mia and I flew to London yesterday. She’s spending two weeks with her mother.. who still hates my guts.

I’m off to Boston tomorrow for VRM+CRM 2010.

Then Jacques and I are off to France/Nice for a few days and then Greece. Keen to find some good free diving spots on the islands. I’m not very good at planning these things, but it’s fun jumping on a plane and reading the travel guide on the way.

Then back to London for a few days.

“Walk tall, kick ass, love music and never forget that you come from a long line of truth seekers, lovers and warriors.” – Dr. HST.. a quote to inspire some adventure

Rock ‘n Roll.

Last month wikileaks published a large tranche of Afghan war documents, that the Pentagon doesn't like. Actually, they want it all back, if such a thing is possible in the download age.

Crowd-sourcing

This is the new 'enemy' - the indiscriminate information age. Even during the case, wikileaks gave its followers - 120 thousand and growing - a sharply-focused feed into this continually-evolving situation. Wikipedia is there to helpfully keep the permanent record, written by the crowds, not by chosen editors. We choose our own news these days - I don't read what just one editor says.

Kenya Election

I believe in free information - I host on my ftp server "statements of domestic election observers the night before the announcement of the results" in case that information were lost. I get about 100 downloads a month.

Iran

Authoritarian governments everywhere - most recently demonstrated in Iran, fear this uncontrollable melange of information - created by twitter and short urls. It is funny to see government trying to swat the fly.

South Africa

There is a proposed Media Tribunal in South Africa - with the government's motive questioned. All governments have to acknowledge that it is impossible now to control the information - perhaps they think that if they have the TV, radio and newspapers covered they have dealt with the majority of the ANC electorate. I will miss the straight-talking Desmond Tutu in the South African media.

Europe

Julian Assange is the founder of wikileaks, reportedly permanently on the move like the Cathar Perfecti. He was bizarrely accused of rape and molestation in Sweden, and equally hastily withdrawn. Even the legal system cannot keep up with this instant feedback loop.

America

America is the land of the free. It is the homesteaders, the hackers, the start of the open source movement and the tools that built Linux and the software I use every day.

Democracy

America is also proud of its democratic heritage and sees nothing wrong with exporting informed choice for the electorate onto countries whose government would prefer the old days.

It has been hard to see the hacker culture that made a lot of what is good today in a struggle with its own government. Omnipresent cameras mean that The Man already has powerful tools. But the strength of the American system is they will resolve it, if necessary by counting heads.

More Afghan War Documents

Wikileaks has published in association with the War Diarys above, a huge, unknown, encrypted file. Possibly more war documents ? A few hundred pages of /dev/random ? All it needs is a password to unlock it. Yes, I already tried "Assange" and all the dog names I could think of.

Lets get one thing straight. Thomas Alva Edison did not invent the lightbulb.

This story requires context:

Thomas Alva Edison - Douchebag Extraordinaire

So all that Edison ever “invented” was that a vacuum increased the lifespan of the filament.

So, the next obvious question is, why does everyone think that Edison invented the light bulb? The answer to that question is an interesting one because it has more to do with propaganda than it does to do with invention.

A lightbulb is useless without electricity, Edison knew that… and like many clever businessman Edison knew that real inventors are generally very bad at making money out of their inventions because they’re always too busy working on version 2.

Edison realised that the only way to make real money was to get electricity into people’s houses so that you could sell them lightbulbs, and electricity… This was brilliant because it’s the “Give away the razor, sell the razorblades” plan except both lightbulbs and electricity are consumables… so its more like “Give away the connection and sell the lightbulbs and the electricity”. Edison also wasn’t the first person to figure this out, he was just really really good at marketing… Or really really bad at marketing, depending on how you feel about Elephants.

Topsy the Elephant

Another person who realised that Electricity was going to make a lot of money was a rich businessman called Westinghouse. Westinghouse had become friends with an eccentric Yugoslav scientist called Nikola Tesla who had been experimenting with electricity his whole life. When Tesla wasn’t busy building Tesla Coils or trying to harness the power of lightning (or trying to harness energy from outer space, I shit you not) he worked with Westinghouse to build electricity generators and plan electricity distribution systems. Tesla knew more about electricity than pretty much anyone else alive at that point and early on had realised that Alternating Current (AC) was far better at distributing electricity than Direct Current (DC). So Westinghouse and Tesla started generating and distributing power to the rich and famous.

Edison didn’t like this. Not only were Tesla and Westinghouse competitors, but they were also proposing a different, better, system (AC) that Edison knew would eventually win the battle. Edison had managed to market himself as the father of electricity — a magician and folk hero — and he was getting incredibly rich.

Nikola Tesla - True Genius

So Edison did what any self respecting douche-bag marketer would do… he started publicly torturing animals… and filming it. Thomas Alva Edison, the “inventor of the lightbulb” went out into the streets and publicly electrocuted, to death, animals in a bid to show the public that Alternating Current was far too dangerous to be in their homes. And yes, in case you’re wondering, that reference to elephants earlier is because Edison even electrocuted, to death, an adult elephant.

Of course AC isn’t really any more dangerous than DC and Edison knew that… While there are some issues with AC’s 50-60 Hertz frequency being closer to that of your heart, both AC and DC are equally able to kill elephants and small children.

Sadly the public was gullible and Tesla was devastated… He went from being an eccentric socialite magician to being that guy who wants to murder small children and animals. Westinghouse had thicker skin and managed to keep his chin up, but Tesla became a recluse and started working in isolation on increasingly crazier ideas like harnessing power from the stratosphere. Tesla believed that it would be possible to get this power so cheaply that it would become free. Whether Tesla was onto something or whether his eccentric genius mind had finally snapped will never be known… In 1943 he died, alone, drowning in debt, in a hotel room.

Thomas Edison was no doubt a very clever man, but he was also ethically a disgusting person who thought nothing of destroying others to elevate his own fame.

I’m visiting Santa Barbara, California for ten days, attending CRYPTO 2010 (the 30th International Cryptology Conference) and the Second SHA-3 Candidate Conference. This post describes my journey to Santa Barbara, and the conference reception on Sunday 15th August.

My journey from Cape Town to Santa Barbara was as comfortable as could be expected. Of course, a trip that lasts thirty hours and involves three economy-class flights, transiting through Jo’burg and Frankfurt, can never be expected to be particularly comfortable. That said, all the planes were on time, and all of them were uneventful.

Cape Town-Joburg was the standard 2-hour SAA domestic flight; all that needs to be said about it was that the turkey wrap served for lunch was surprisingly quite edible. OR Tambo International Airport, despite all the reconstruction, was rather irritating. Firstly, if you’re coming off a domestic flight at Terminal B there’s virtually no signage to direct you to the Terminal A international departures area. Secondly, there’s hardly anywhere to sit at the boarding gates. A Boeing 747 carries 400-500 people, and an Airbus A340 slightly less than that; why, then, at our country’s flagship international airport, do the boarding gates for long-haul flights only have seats for about 50 people to wait?

Joburg-Frankfurt was an overnight flight, also SAA, on the Airbus A340-600. This plane had personal video screens in the back of the seats, which made for a nice distraction; however, the software that runs them is obviously rather buggy. Warning: geeky stuff begins. At one point it segfaulted, dumping to a Linux console. Evidently the software is some kind of SDL-based program launched by a rather hackish batch script; that was all I could tell before it rebooted automatically. Unfortunately I couldn’t get a photo in the low-light conditions. Geeky stuff over.

We hit quite a lot of turbulence over the Congo, and again over Switzerland, but nothing seriously scary. My attempts to sleep on the flight failed, so I resorted to watching movies to pass the time. Guy Ritchie’s recent Sherlock Holmes was fun but not, of course, particularly true to the original books. I also saw Invictus, which I will admit had me in tears a few times.

Frankfurt Airport, though much larger than OR Tambo, was considerably easier to navigate. Apart from an incredibly long queue for the security check, there were no hassles, and I reached the gate with three hours to wait before boarding – at least, I thought I had reached the gate. I was waiting to the one specified on my boarding pass, and as more and more people arrived, the vast majority of them appeared to be Indian. Now, I know there are a lot of Indians who work in IT in California, but this was getting to be little suspicious. So I checked, and yes indeed the Los Angeles flight had been moved to another gate. This is my one complaint about Frankfurt Airport: there are lots of boards listing the gate assignments before you clear security, but afterwards there is only one board in the whole wing.

Ironically, the passenger next to me on the flight to Los Angeles was an Indian working in IT in San Diego. This flight, although only an hour longer than the previous one, seemed absolutely interminable. I was, of course, more tired that I had been on the first flight; I also had a piercing headache for the first two hours. (Thank you, mother, for suggesting I take a bottle of Panado tablets along.)

Also, the flight only had overhead video screens, which showed first some incomprehensible German dance music videos, then one of the sequels to Shrek. After that, as is apparently inevitable with Germans, they showed episodes of Mr. Bean. So I spent most of the time trying to sleep, with little success: all I managed was a series of ten-minute naps.

Anyway, in the end the flight arrived at LAX thirty minutes early. Unlike some unpleasant stories I’d heard, clearing Customs & Border Protection was quick and easy – in fact, the only question I was asked was, “what is the purpose of your visit?” My baggage arrived fairly quickly and undamaged, and ultimately I was out of the terminal by the time the plane was originally scheduled to land.

It was quite a relief to breathe fresh air after such a long time in an air-conditioned environment. Of course, this was Los Angeles, so the air was not particularly fresh. The bus to Santa Barbara, along “the 101″, took a little more than two hours; the overwhelming impression for me was the sprawl – every hillside seemed to have a house on it, every valley had malls and office buildings. The later part of the drive, along the coast from Ventura to Santa Barbara, was more pleasant and reminded me of the R44 from Gordon’s Bay to Betty’s Bay, though the road was wider and the slope not as steep.

We finally arrived at the conference accommodation at UCSB sometime after five in the evening; at this point I had been awake, excepting short naps on the plane, for something like 40 hours. But there was still the conference reception dinner to attend – and I’m glad I did, not only because I was rather hungry after all that traveling (and the food was very good), but also because I had the opportunity to meet many people and have some very interesting conversations. As an aside, this is something I am particularly coming to like about the crypto community: the people in it, and even its most eminent members, are friendly and willing to chat, even with mere graduate students like me. No matter how interesting the conversation, though, by eight o’clock I couldn’t keep my eyes open and I had to retire to bed.

Today, Debian turns 17. It’s great to see how one of the first GNU/Linux distributions out there still manages to be important and relevant. Happy birthday, Debian!

This may also be a good time to say thanks to a team or package maintainer who makes it possible.

Jonathan just ran into this on his new HTC Desire (updated to 2.2 “Froyo”); I haven’t confirmed it on any other handset, but I imagine the problem is common to many HTC Sense + Android handsets. Anyhow, I figured I’d put this information out there to help anyone that runs into this issue. If anyone has a contact at HTC, please pass this along, so that they can look into a fix/workaround.

When “Facebook for HTC Sense” tries to connect to Facebook to retrieve contacts or profiles, the client sends Expect: 100-Continue in the HTTP header. Facebook’s HTTP servers do support 100 Continue, but some proxy servers (such as squid) do not. If you have configured your phone to use a proxy server, or are behind a transparent proxy (which many ISPs do inflict on their users), and the proxy server does not support this, it will (correctly) respond with 417 Expectation failed response. At this point, HTC’s client does not attempt to retry the request without requesting 100-Continue, but instead fails with an error message. This results in an empty Facebook directory / contacts list, and an error when you attempt to refresh it.

If the proxy server is on your own network, simply bypassing it should solve the issue for you. If this is your ISP’s proxy server, then you’ll either need to work around it somehow, or contact HTC and/or your ISP for assistance.

An open access network (OAN) is a horizontally layered business model for telecoms that has some interesting advantages. It is typically implemented by separating physical infrastructure from services. This creates a virtual market place, where end users have a choice of multiple service providers, and the service providers in turn are freed from having to invest in expensive and difficult to maintain access networks.

The wikipedia article describes OAN in more detail, and refers to a number of examples where this model has been implemented in broadband access networks.

But if the service provider doesn’t pay for the infrastructure, who does? Governments have already done this once, discovered they’re no good at it and consequently privatized the state owned telcos. These telcos are moving in the opposite direction of open access, instead trying to figure out ways out of the dumb-pipe trap through vertical integration, value added services, and so on.

An idea called homes with tails proposes to include the last mile as part of your property when you build or buy a house. This is smart, because it opens up a brand new (and also bandwidth/throughput/services independent) way to amortize the cost of the broadband access network.

Yesterday, Joe Botha blogged about what he imagines a network architecture/billing model for broadband OAN could be. What he doesn’t mention is that this idea is not new to South Africa. This prompted me to dump my thoughts on the matter (many of which I’ve been ruminating on for some time) into this post:

A few years ago, Neology (who I’ve previously blogged about) did an OAN pilot project in Hatfield for the Tswhane (Pretoria) municipality. If I have the story straight, this was a Wifi based mesh network that had an http captive portal that let you choose between 2 or 3 different ISPs, and it worked. This initiative subsequently stalled (apparently due to council politics) but my point is that OAN broadband been demonstrated in the local context.

If the access network infrastructure was not owned by government, but by the end users instead, perhaps the outcome would have been different. So let’s push this idea further. The next best thing would be to avoid relying on local government for wayleaves/right of way/servitudes. This might be possible using wireless local loop, but I’m just going to ignore that option because a bunch of home owners have little chance competing against multi-billion Rand tel/cellcos for exclusive access to radio spectrum (and the ISM-band WISP approach has some drawbacks).

Now it only takes a short drive around your city in South Africa to notice the transformation from traditional suburban townships to gated communities/housing complexes/residential estates. Herein lies the opportunity to do exactly what I’ve described in the paragraph above. I’ve head some entrepreneurs and local ISPs (who?) have begun to wire up these estates internally and provide centralized Internet break-out. I don’t know if this is done on an open access basis, but again, my point is that this piece of the puzzle exists.

Finally, Dark Fiber Africa have recently begun doing something similar in office parks, and as with their metro dark fiber networks, their model is open access.

So where to from here?

Vodacom, MTN and Cell C have all invested in fiber-based back-haul to their cellular base stations (ever wondered why the roads are being dug up and trenches filled with brightly coloured plastic ducts, deep in residential areas?). These base stations now represent an extensive “curb” (in the network sense), and since in many cases towers are shared by two, or all three operators, I think base stations make an ideal location for OAN broadband “meet me” points.

I haven’t asked, but I suspect the cellcos aren’t thinking this way (yet). If they venture into fixed line residential broadband, they will probably first attempt PON-based overbuild in affluent suburbs.

A final thought to leave you with is to consider the explosive growth in recent years of Wireless User Groups. These non-profit hobbyist metro Wifi networks are mostly used for fire-sharing and gaming (Internet access is specifically not provided). It shows that at least some South African consumers are willing to invest in their own telecoms infrastructure. Perhaps the way to bridge the gap between WUG and WISP is to form a fully licensed and professionally operated telecoms cooperative, like this one.

#!/usr/bin/php
<?php
error_reporting(-1);
$doc = new DOMDocument();
$doc->load($_SERVER['argv'][1]);
$entries = $doc->getElementsByTagName('entry');
$posts = array();
$comments = array();
for ($i = 0; $i < $entries->length; $i++) {
 $entry = $entries->item($i);
 $links = $entry->getElementsByTagName('link');
 for ($j = 0; $j < $links->length; $j++) {
  $link = $links->item($j);
  if ($link->getAttribute('rel') == 'alternate') {
   $href = $link->getAttribute('href');
   if (substr($href, 0, 24) == 'http://blog.jokevn.com/2') {
    if (strstr($href, '#') === false) {
     $post = new stdClass();
     $post->id = substr($href, 23);
     $post->title = $entry->getElementsByTagName('title')->item(0)->textContent;
     $post->content = $entry->getElementsByTagName('content')->item(0)->textContent;
     $post->date = $entry->getElementsByTagName('published')->item(0)->textContent;
     $posts[] = $post;
    } else {
     $comment = new stdClass();
     $comment->post = substr($href, 23, strpos($href, '.html') - 18);
     $comment->author = $entry->getElementsByTagName('name')->item(0)->textContent;
     $comment->content = $entry->getElementsByTagName('content')->item(0)->textContent;
     $comment->date = $entry->getElementsByTagName('published')->item(0)->textContent;
     $comments[] = $comment;
    }
   }
  }
 }
}
file_put_contents('posts.json', json_encode($posts));
file_put_contents('comments.json', json_encode($comments));

I’m a little bored, and the whole internet (well, at least the portion of it composed of Desire owners) seems to be going crazy waiting around for the recently-announced rolling out of the Android 2.2 update for the HTC Desire, so I thought I’d poke around a little in the OTA update process.

UPDATE (2010/08/03): The update seems to be rolling out to most people, but some people still aren’t getting it, even though they have unlocked/unbranded phones. From what I’ve been able to discover, HTC appears to be employing an IMEI “blacklist” of some kind; checkins with specific IMEI values fail to return the 2.2 update, but anything else (including completely invalid IMEI values) returns the update. I’m still not sure whether this is just their way of staggering the update, or if there is some other reason for withholding the update from the affected devices.

UPDATE (2010/08/05): HTC now seems to either have a bad content server, or be in the process of pulling the update from their servers, including the update file itself on liveimage.htc.com. If it is deliberate, I have no idea why they’re doing it, and I can’t currently find any information that sheds any light.

The process seems to be relatively straightforward: when you hit “Check now”, the phone sends an HTTP POST to http://andchin.htc.com/android/checkin (not https, thus all in the clear, making it easy for me to sniff :P), with a Content-Type of “org/x-json” (wat), and a JSON-encoded payload. The payload looks as follows (I’ve pretty-printed it and stripped personal identifiers, but it’s still good enough to get a response):

{
  "locale": "en_GB",
  "checkin": {
    "build": {
      "product": "bravo",
      "serialno": "HTXXXXXXXXXX",
      "changelist": "174215",
      "build_type": "user",
      "carrier": "htc_wwe",
      "radio": "32.36.00.28U_4.06.00.02_2",
      "firmware_version": "1.21.405.2 CL174215 release-keys",
      "bootloader": "0.80.0000",
      "id": "htc_wwe/htc_bravo/bravo/bravo:2.1-update1/ERE27/174215:user/release-keys",
      "revision": "129"
    }
  },
  "model_number": "HTC Desire",
  "logging_id": 0,
  "imei": "",
  "id": 0,
  "digest": "8fccfd93dcbe837d072e5f1494da2"
}

Note the “carrier” value there; this is “htc_wwe” for phones that are not carrier-locked / otherwise carrier-customized. If you had a Sprint phone, for example, this would be “sprint”. I believe this value is ultimately determined by the the “androidboot.carrier” kernel parameter passed to the kernel by the boot loader.

I’m not sure what “digest” is of; it is 29 hex digits long, which doesn’t make any sense for a cryptographic digest. It doesn’t seem to be a personal identifier of any kind, though; if you modify it, you get some “upload crash” response back, which contains (among other things) the correct digest, so I can only guess that it corresponds to the product information in some way. I’ll be interested to see if it changes once I have the new update installed.

The response that comes back looks like this:

{"time_msec":"1280538364433","stats_ok":true,"intent":[{"action":"android.server.checkin.FOTA_CANCEL"}]}

The “intent” part will probably be obvious to Android developers (it’s specifying what the UI should do next), and time_msec is a UNIX timestamp in milliseconds, corresponding to the time the response was sent.

If an update *is* available, the response looks like this instead:

{
  "stats_ok": true,
  "intent": [
    {
      "action": "android.server.checkin.FOTA_UPDATE",
      "data_uri": "http://liveimage.htc.com/OTA_Bravo_HTC_WWE_1.21.405.2-1.15.405.4_release_P4ohim1mtp6zggcyon.zip",
      "extra": [
        {
          "name": "promptMessage",
          "value": "System Upgrade"
        },
        {
          "name": "promptFeature",
          "value": "Update:HTC application improvement"
        },
        {
          "name": "promptVersion",
          "value": "System upgrade1.21.405.2 (27.07 MB)We would recommend using a free Wi-Fi hotspot or an unlimited data plan to apply this update. If not, standard data connection charges may apply. Any questions? Contact us via http://www.htc.com/www/CA_Hotline.aspx"
        },
        {
          "name": "promptSize",
          "value": "27.07 MB"
        },
        {
          "name": "promptMinutes",
          "value": "30,..."
        },
        {
          "name": "timeoutSeconds",
          "value": "120"
        },
        {
          "name": "force_update",
          "value": "false"
        }
      ]
    }
  ],
  "time_msec": "1280513282008"
}

Now, it seems as if they are planning to do a rolling release of the 2.2 update, somehow. I assume this means that they will only be responding with the updated version to some requests, but I don’t know exactly what mechanism they’ll be using to do that. Oh well…

So the M&G did an article on ICON, the recent gaming convention up in Jo’burg. It’s about as awful as you would expect, only worse. It makes it sound as if all the women were walking around in revealing outfits with their tits falling out, and that the main attraction of the con to women is the possibility of attracting a horde of desperate nerds no matter how shy or unconventionally attractive they are. Not, you know, participation in any of the games, or the purchase of cool toys; those are for the smelly man-children running around and whacking each other with rubber swords.

Here’s where you can read this wonder of modern journalism in all its glory. Use of latex and bleach is recommended if you have a gentle and sensitive nature or are allergic to fail.

Since I appear to be kind of half maybe possibly not actually logged in to the website, and I have no idea whether it’s eaten my comment, and if it hasn’t eaten my comment it has almost certainly eaten the paragraph breaks in my comment, thus rendering it unreadable, here it is again:

This is yet another “quirky human interest” article written by a clueless mainstream journalist who briefly encounters a subculture and then regurgitates a hopelessly confused mess of whatever they found most memorable. But it’s a lot worse than average.

I came here expecting to get a bit annoyed about the conflation of LARPing with the SCA, or the portrayal of all LARPing as MEAD-style boffer LARPing. But the horrible and sensationalist gender stereotypes on display here kind of eclipse the geeky details.

I disagree that the article helps any of the hobbies it features. It tells men “these subcultures welcome sexist jerks” and tells women “stay far away from here unless you would enjoy wading through a cesspit of sexist jerks”.

I was there, I wore normal clothes (like a lot of other women there, who I guess were not exciting or exotic enough to be mentioned), I had a great geeky time and nobody was rude to me. But if all I knew about ICON came from this article, I would avoid it like the plague.

I know women who dress up at cons, sometimes in revealing outfits, satirically and non-satirically. I know a lot of the women from the T&A RPG team. I may have to agree to disagree with them about the feminist implications of dressing up in revealing outfits at cons, but that’s a whole other post — and on the whole I think they should wear whatever the hell they want and not be assumed to be wearing it for the benefit of dudely eyeballs. Our differences of opinion notwithstanding, I think it’s safe to say that the satirical element went so far over the reporter’s head it may have ended up in that giant football dirigible floating over the city.

(In non-ranty post-ICON news, I have giant blisters on my hands from screwing my giant alien’s legs in, but it was so worth it. It looks totally badass. )

P.S. The really annoying thing about this is that in South Africa roleplaying isn’t that male-dominated anymore. OK, maybe the nationwide roleplaying diaspora — all those tiny cells of friends who have played AD&D all by themselves since high school and never talk to other roleplayers — is male-dominated, but the larger communities centered on e.g. ICON or CLAWs at UCT are full of women and have been for over a decade. And they’re not just “girlfriends and sisters”; they write LARPs and modules and organise things and come to conventions every year. Anyone who doesn’t recognise that is seriously misrepresenting the community.

A few weeks ago I moved to Québec, Canada. Today I bought this car from a colleague at Révolution Linux, he doesn’t need it anymore since he’s moving to Montréal. He bough it just a few months ago from another colleague who moved to France.

It doesn’t have a CD player, instead, it has a tape deck, which plays analogue magnetic tapes. They’re horribly obsolete now but this was how I listened to music way back when I was a kid. Another college that lives down the street from me has a hi-fi with a CD player AND a tape deck. I was planning to go there to write the Offspring Smash album to tape, it’s one of my favourite driving albums and I never tire from it.

All the tapes from 2 owners back was still in the car. I decided to remove them when I got home this afternoon. The *first* tape I took out happened to be this one:

I was amazed and delighted! Then I removed the next tape:

It had “High voltage” written on it, which is very close to my IRC/Identica/Twitter nickname, “highvoltage”. If I were at all superstitious, I’d take it as a very good sign that this car will serve me very well!

Encumbered forever by desire and ambition,
there’s a hunger still unsatisfied;
our weary eyes still stray to the horizon,
though down this road we’ve been so many times.

— Pink Floyd, High Hopes

Introduction

As some of you already know, I recently became the proud new owner of an HTC Desire. You can find the detailed specs elsewhere, but the phone is pretty much a Nexus One with an optical trackpad, real buttons, a little extra flash memory, and the HTC Sense software. This post isn’t exactly going to be an in-depth review, but I thought I’d give a brief rundown of my impressions, and also the apps I’m using on the phone at the moment.

Overall, I’m pretty happy with the phone. I can get 1-2 days of use between charges (depending on usage), which is good enough for my purposes, and that does include push email (quite a lot of email, in the case of my personal account) and syncing all sorts of stuff on a pretty regular schedule. The phone UI is generally way more responsive and snappy than my previous phones (Nokia E71, E65), and the touch screen works reasonably well. Also, I can actually get apps for a lot of the services I use now, whereas I couldn’t before (not many people are making Series 60 apps).

Input

The lack of a real keyboard turned out to be not as bad as I expected. Tap-typing with a single finger is not as fast as using both thumbs was on the E71, and holding the phone so that I can use both thumbs is somehow awkward enough that I never did it. Selecting predicted words is a lot easier with the touch screen, though (no need to scroll through the list). However, I am now using the Swype Beta, an alternative input method where you “swype” your finger through the letters of the word you want, without lifting it, along with a lot of other nice tweaks to the on-screen keyboard. It works pretty nicely; it doesn’t always select the right word, but you can finish your sentence (or paragraph) and then go back and select a different word where it made mistakes. If a word isn’t in the dictionary, or isn’t really a word at all, you can just tap it out; regular words will be added to the dictionary after you press space, but you can also add “non-words” like “http://” or “.co.za” for convenience. I’m fast enough on the Swype keyboard now that I can use it for taking notes during a conversation in real-time, which is great. Unfortunately, the beta is closed at the moment; if you didn’t have a chance to grab the app while it was open, you’ll have to wait around until they decide to let people have a crack at it again, or they open it up to everyone.

The multi-touch screen is great. A lot of apps (like Maps, or the browser) support the “pinch” and “spread” (what’s the real name for the opposite of pinch?) to zoom in and out, which easily lends itself to an “arbitrary precision” navigation style: zoom out to cover a lot of ground, then zoom back in to precisely select a button or other page element. The “momentum-based” scrolling mechanism (again, what the heck is this actually called?) is also used by most apps, and is similarly awesome. It sort of behaves like a globe: flick the screen to start scrolling; over time, it slowly slows down, but you can touch the screen to “catch” it once you get to where you want to be, or flick it again to keep going.

Multi-tasking

The Android approach to multi-tasking is awesome, and I kinda wish my desktop operating systems worked the same way. Under the hood, it’s basically just standard Linux processes being scheduled by the kernel, but with a bit of a twist. In terms of the UI, there’s no explicit process management. Some apps have an “exit” option, and you can burrow into the Applications list to “Force Stop” any app, but there isn’t even any built-in way to get a list of currently-running apps (although third-party apps for this purpose are available), simply because you don’t really need it. When the OS runs out of memory (think “OOM killer” if you’re familiar with Linux), it kills off the oldest apps to free up memory. In addition, provision is made for apps to save and restore their state. As a result, if you switch back and forth between apps, they’ll probably still be running, so switching will be immediate. If you open up an app that you haven’t used for a while, it may not be running anymore, so it’ll take a little longer (but usually not that much longer) for the app to start up again and restore the state. Most apps don’t do anything if they’re not in the foreground, so there’s no need to worry about wasting your battery running apps you’re not using. There are also “services”, which are for long-running background tasks that shouldn’t be killed off to reclaim memory; synchronization, for example.

Unfortunately, some people seem to be confused by all of this; as a result, there are a plethora of “advanced task killer” apps and suchlike on the Android market. These are all just about pointless; all you are likely to accomplish by running around killing random processes is breaking certain functionality like background synchronization that you turned on. If you want to stop your contacts syncing, for example, just go turn that off in the settings, instead of using an app that kills the process every time it starts up. Also, if an app really is draining your battery in the background, it’ll show up in the battery info screen, and you can then uninstall it or otherwise take steps to solve the problem. The only exception to the latter is built-in functionality that gets aggregated into the “Android Platform” entry; it may not be obvious what you need to disable or reconsider if “Android Platform” shows up as using too much juice.

Intents

Think of intents like file type associations on steroids. Instead of just “open this file which is of some type”, you also have “open this URL” and “dial this phone number” and “view these map coordinates” and “search for this text” and “display the home screen” and “the battery is running low” and just about anything else you can think of, including brand new intents created by third-party developers.

Since any application can register itself to handle an intent, this means that third-party applications can do anything from hooking in as a phone dialler (eg. a SIP client), to replacing the home screen; just about any aspect of the “default” OS software can be replaced. This can also be done selectively; for example, if you follow a link to a Google Maps URL (even from another web page in the browser), you’ll be given the option to open it in the “Maps” app as well as opening it in the browser.

The net result is that third-party software can replace and enhance the OS in a practically unlimited fashion; and different third-party apps can interoperate and integrate without a great deal of special effort being exerted on the part of the application developers.

GPS / Location

The Desire has A-GPS functionality. My E71 also supposedly had this functionality, but it never seemed to work; getting a GPS lock could take up to 5 minutes if I started the Ovi Maps app up in a different location to where I last closed it. On my Desire, under optimal conditions (outdoors, clear view), I can generally get a GPS lock in seconds, no matter how far I’ve moved since the last time I got a lock. I can even get a GPS lock in under 30 seconds standing indoors in the middle of my house! I guess this is probably a combination of a better A-GPS implementation and a better GPS receiver, but I’m definitely much happier with the results.

Battery lifetime

A few quick notes I wanted to make on battery lifetime. Firstly, the battery lifetime seems to go up after the first several charge-discharge cycles; I guess this probably applies to all batteries in this class, but I thought it was worth mentioning. Secondly, active wifi use generally uses less battery than an active cell data (GPRS/EDGE/3G/HSDPA) connection; but an idle wifi connection will use more battery than an idle cell data connection. You can adjust the wifi “sleep” policy to take advantage of this by disabling the wifi connection after a certain timeout, although you may wish to avoid the extra cell data charges. Also, if you are in an area with poor cell reception, the cell radio will use more battery as the transmit strength is increased in order to successfully transmit; this applies even if you are not making use of cell data.

My applications

You can see a list of all of the applications I have installed (thanks to AppBrain), but not all of them are worth mentioning, so I’ll run through the interesting ones here, with some commentary. Unfortunately paid apps are not currently available from the Android Market in South Africa, which some apps are either not available at all, while others are only available in free+ads form.

• Aldiko Book Reader
  A pretty decent eBook reader for ePub format books. It includes support for purchasing books from some sites (they have a mechanism whereby any online store can integrate their catalogue); otherwise you can download and import them from anywhere you can find them, or copy them onto the phone from your PC.
• Android Agenda Widget
  This is a ridiculously customizable home screen widget for viewing your calendar and tasks. It has support for “styles”, but you can also individually customize just about every aspect of the widget. It also provides widgets in almost every every possible size from 1×1 up to 4×4. It supports the various calendars (Google, HTC, etc.) as well as Astrid and some other tasks app that I can’t remember offhand.
• AppBrain App Market
  The name is a little misleading: this is not a separate market, but just a way to sync and share your installed apps list, as well as scheduling installs / uninstalls via the web interface (which will be performed the next time you sync using the phone app).
• Astrid Task/Todo List
  A nifty tasks list manager. You can schedule multiple reminders for tasks, tag them, track expected time / time spent, and sync with Remember The Milk.
• ASTRO File Manager
  Android does not come with any kind of file manager, although you won’t need one very often since apps tend to avoid explicit file management. Still, there is the occasional need to rename or move a file around; it can also be very useful to transfer files over the network, especially with the SMB module.
• Barcode Scanner
  Scan a variety of barcodes, including QR codes and UPC / EAN barcodes. Many sites provide download / market links for apps in the form of QR codes; scanning the code from another display is obviously easier than retyping the URL.
• Dropbox
  Nothing much to this app; it just lets you view / download files you have stored on Dropbox.
• Evernote
  Create and access your Evernote notes, including geotagging support.
• Foursquare
  Check in on Foursquare, get alerts when your friends check in nearby, view nearby venues, and search for venues.
• FoxyRing: Smart Ringtone
  I’m not actually using this app currently, but you may find it useful regardless. The app allows you to adjust your ringtone settings based on ambient noise levels, time of day, and location, as well as overriding them for selected contacts.
• Glympse
  Unlike Google Latitude, which shares your location to all of your contacts the whole time you have it active, Glympse provides more limited and granular sharing. When you create a “Glympse”, you specify a lifetime (eg. 30 minutes) after which the Glympse expires, and optionally start and destination locations; a unique code is then generated. You can then share the URL by text message or email, or you can post it on Facebook. The recipients can then view your location in realtime, until the Glympse expires, either when the specified time has elapsed, or by you manually expiring it at any time of your choosing.
• Google Goggles
  This app is described as a “visual search” application. You take a picture of anything, and the application attempts to recognise it. This might mean scanning a barcode / QR code, scanning and OCRing text (which you can then translate or search for), detecting a well-known landmark, a work of art, and more.
• GPS Status
  A simple app that allows you to visualize the GPS satellites in range, how many satellites you are locked onto, and see other information from the GPS receiver.
• Kindle for Android
  Amazon’s Kindle application. Great for reading Kindle books, but if you can get them in ePub, rather use Aldiko (see above), as the Kindle app is not nearly as featureful.
• Nimbuzz
  This is an IM / voice chat app. It has support for Skype, which is useful, as Fring no longer has Skype support, and the official Skype Android app is currently only available to Verizon subscribers through a special deal. It also supports a variety of IM networks, including Facebook, but unfortunately generic Jabber support is no longer available.
• Real Signal
  A simple app that gives you a configurable cell signal strength meter, as well as allowing you to see the actual numerical signal strength.
• Remember The Milk
  Only usable by RTM Pro subscribers (although you get a 14-day trial of Pro if you aren’t a subscriber). This lets you sync your RTM tasks to your phone, create tasks on your phone, and includes a widget for displaying tasks. Not strictly necessariy in conjuction with Astrid (see above), but I find it useful for managing additional task views (eg. my shopping list).
• ScrobbleDroid
  Just what it sounds like: scrobble your music activity to last.fm. Compatible with various music players.
• Shazam
  Want to know what song is currently playing? Capture an audio sample via the phone’s microphone with this app, and then search their online database to identify the song.
• Shopper
  This app overlaps with Google Goggles somewhat (see above). You can scan media (DVDs and CDs) or books by barcode or cover image, and then search for them online with Google Shopper; useful for doing some comparative shopping when you’re in a physical store.
• SpeedView
  A simple speedometer app that uses the GPS to track your speed.
• TripIt
  Sync your TripIt travel plans to your phone; allows for offline access, so you can get your important travel details even if you can’t get an internet connection from where you are.
• Wifi Analyzer
  Visualize the wifi networks in range, including the channel and signal strength. Useful for selecting an unused channel for your wifi network, or picking an internet access point.

Finished all the levels in the default Frozen Bubble levelset? Run out of custom levels from the interwebs? Random levels too boring? Making levels too hard? Sounds like you need Forever Bubble — what I made on Tuesday night instead of working.

Randomly generate pretty levels — for ever! Run it with --help to see all the options. You need the argparse and networkx python modules, and obviously python. It should work on any *NIX. The Frozen Bubble levels directory needs to exist if you want to save there. Patches, ports and comments welcome. If you generate a particularly awesome level, paste it into a comment; maybe I’ll compile a “greatest hits” levelset.