come on down to clug park and meet some geeks online

20 June 2018

Jonathan Carter (highvoltage)

Plans for DebCamp18

Dates

I’m going to DebCamp18! I should arrive at NCTU around noon on Saturday, 2018-07-21.

My Agenda

  • DebConf Video: Research if/how MediaDrop can be used with existing Debian video archive backends (basically, just a bunch of files on http).
  • DebConf Video: Take a better look at PeerTube and prepare a summary/report for the video team so that we better know if/how we can use it for publishing videos.
  • Debian Live: I have a bunch of loose ideas that I’d like to formalize before then. At the very least I’d like to file a bunch of paper cut bugs for the live images that I just haven’t been getting to. Live team may also need some revitalization, and better co-ordination with packagers of the various desktop environments in terms of testing and release sign-offs. There’s a lot to figure out and this is great to do in person (might lead to a DebConf BoF as well).
  • Debian Live: Current live weekly images have Calamares installed, although it’s just a test and there’s no indication yet on whether it will be available on the beta or final release images, we’ll have to do a good assessment on all the consequences and weigh up what will work out the best. I want to put together an initial report with live team members who are around.
  • AIMS Desktop: Get core AIMS meta-packages in to Debian… no blockers on this but just haven’t had enough quite time to do it (And thanks to AIMS for covering my travel to Hsinchu!)
  • Get some help on ITPs that have been a little bit more tricky than expected:
    • gamemode – Adjust power saving and cpu governor settings when launching games
    • notepadqq – A linux clone of notepad++, a popular text editor on Windows
    • Possibly finish up zram-tools which I just don’t get the time for. It aims to be a set of utilities to manage compressed RAM disks that can be used for temporary space, compressed in-memory swap, etc.
  • Debian Package of the Day series: If there’s time and interest, make some in-person videos with maintainers about their packages.
  • Get to know more Debian people, relax and socialize!

by jonathan at 20 June 2018 08:32 AM

17 June 2018

Johann Botha (joe)

Quick Update

Windy, wet and cold…

Sea Point
  • Week of 11-17 June.
  • Kinda weird week, over before it started.
  • Monday, audio book listening school run, gym, a walk with Alan around Stellenbosch – got a cap, office, sunset audio book listening walk, general tinkering and tea, some reading about small towns, in bed early.
  • Tuesday, published a blog post, walked to the office, catch up with Kenny and Nick, Sababa lunch, catch up with Nigel, catch up with Roelf at Strolla, ribs special at Van Hunks with Georg and Chris.
  • Wednesday, office, Sababa lunch, after work catch up with Nick and Michael at Nu, dinner at 91 with Heleen.
  • Time in REM sleep heals all emotional wounds.
  • Thursday, rainy and windy day, worked from home, Sabada lunch with Heleen, cooked some mexican food.
  • School should start later, 08:30 or 09:00.
  • Friday, office, dev meeting, after work drink on the roof, pizza at Da Vinci’s with Heleen, watched some Soccer World Cup at Harvey’s.
  • Saturday, tea, backups, gym, Green Point Park walk, nap, watched the rugby at Rockpool with Georg, Harvey’s.
  • Sunday, rainy day, breakfast at Clark’s with Heleen, random reading and tinkering, cold and windy sunset walk – mostly because I needed two photos for this week’s blog post.
  • No time well spent this week, sadly.
  • This is gold! Read it twice. How to Get Rich

    Seek wealth, not money or status. Wealth is having assets that earn while you sleep. Money is how we transfer time and wealth. Status is your place in the social hierarchy.

    Understand that ethical wealth creation is possible. If you secretly despise wealth, it will elude you. Ignore people playing status games. You’re not going to get rich renting out your time. You must own equity – a piece of a business – to gain your financial freedom. You will get rich by giving society what it wants but does not yet know how to get. At scale. Pick an industry where you can play long term games with long term people.

    The Internet has massively broadened the possible space of careers. Most people haven’t figured this out yet. Play iterated games. All the returns in life, whether in wealth, relationships, or knowledge, come from compound interest.

    Pick business partners with high intelligence, energy, and, above all, integrity. Don’t partner with cynics and pessimists. Their beliefs are self-fulfilling.

    Learn to sell. Learn to build. If you can do both, you will be unstoppable.

    Arm yourself with specific knowledge, accountability, and leverage. Specific knowledge is knowledge that you cannot be trained for. If society can train you, it can train someone else, and replace you. Specific knowledge is found by pursuing your genuine curiosity and passion rather than whatever is hot right now. Building specific knowledge will feel like play to you but will look like work to others. When specific knowledge is taught, it’s through apprenticeships, not schools. Specific knowledge is often highly technical or creative. It cannot be outsourced or automated.

    Fortunes require leverage. Business leverage comes from capital, people, and products with no marginal cost of replication (code and media).

    Capital means money. To raise money, apply your specific knowledge, with accountability, and show resulting good judgment. Labour means people working for you. It’s the oldest and most fought-over form of leverage. Labour leverage will impress your parents, but don’t waste your life chasing it. Capital and labour are permissioned leverage. Everyone is chasing capital, but someone has to give it to you. Everyone is trying to lead, but someone has to follow you. Code and media are permissionless leverage. They’re the leverage behind the newly rich. You can create software and media that works for you while you sleep. An army of robots is freely available – it’s just packed in data centres for heat and space efficiency. Use it. If you can’t code, write books and blogs, record videos and podcasts. Leverage is a force multiplier for your judgement.

    Judgement requires experience, but can be built faster by learning foundational skills. There is no skill called “business”. Avoid business magazines and business classes. Study microeconomics, game theory, psychology, persuasion, ethics, mathematics, and computers. Reading is faster than listening. Doing is faster than watching. You should be too busy to “do coffee,” while still keeping an uncluttered calendar.

    Become the best in the world at what you do. Keep redefining what you do until this is true. Apply specific knowledge, with leverage, and eventually you will get what you deserve. When you’re finally wealthy, you’ll realize that it wasn’t what you were seeking in the first place. But that’s for another day.

Have a fun week, crazy kids.

by joe at 17 June 2018 04:20 PM

12 June 2018

Johann Botha (joe)

Quick Update

The attempt to be here and now, and sleep…

Cape Town Lights
  • Week of 4-10 June.
  • Monday, gym, work, catch up with Pieter at Foodlovers Market, a bloody mary at Van Hunks with Anton, Wellness Warehouse, paleo burger at Da Vinci’s, non-vegan ice cream at unframed, a Bains at Harvey’s with Georg and Anton – chats about life, the universe and the 9th planet, La Vie.
  • Tuesday, walked to the office, upgraded to an iPhone X, lunchtime walk, after work drink at La Vie with Heleen, date night dinner at HQ with 7 guys, romantic cable car road drive to watch the city lights (not with the 7 guys).
  • Remember Strega and Pu Na Na bar?
  • iPhone X mini review: Works well. No home button or headphone jack, but you get used to that quickly. It’s a tiny bit big for one hand use, I guess I need a bigger hand. Face ID works well, in low light also. Dual camera portrait mode photos are cool and good fun to play with (more than one portrait photo this week, see below) . The screen is great, quality and size. I’ll need Airpods soon, to get to Apple fanboy level 10 status. The iPhone 8 is fine but it’s not the future. The new needs friends.
  • Must say, may latest iPhone upgrade process was fairly painless, more apps seem to keep their data and state in backups now, still took a while to move to a new device.
  • “A good traveller has no fixed plans, and is not intent on arriving.” — Lao Tzu

  • Wednesday, gym, office, dev meeting, Sababa lunch, Piazza St John, Nu with Michael and Nick, take-out from Food Inn India, in bed early.
  • Been listening to Why We Sleep, good book. I’m very happy that I’m a good sleeper, and always have been. Take afternoon naps. You will live longer.
  • Maybe sleep is our default mode. You only wake up to eat, so you can go back to sleep. I’ve been thinking, humans are maybe just some elaborate distributed analogue computer, we wake up – gather ideas, stories and images, then process these in REM sleep – in a range of combinations and permutations. The human colossus. Opportunity finding machines. Remix, defrag and reorganise – Dark City. I expect to see more research on psychedelics and sleep coming out soon. The line between our conscious and unconscious is fascinating.
  • Thursday, catch up with Jonathan and Ian, office, stayed in the office till well after dark, La Vie winter pizza special with Heleen.
  • Friday, office, fetched Mia, watched Jurassic World: Fallen Kingdom at the Waterfront with Mia – not bad, entrecote steak special at Den Anker, gelato.
  • Saturday, rainy day, milky tea, backups, configured my new phone – did some phone housekeeping to make all my apps fit in two screens again, omelette at gym, grid area workout, Granger Bay market, Moro ice cream – best ice cream in Cape Town, took a few photos from the roof of our building, a glass of Bouchard Finlayson Kaaimansgat Chardonnay at 12 Apostles Leopard Bar, podcast listening drive through Constantia via Hout bay, a Botanist G&T at the Secret Gin Bar, Mia and I shared the 1kg ribs at Da Vincis and watched the 2nd half of the rugby, in bed early, photo processing, time well spent.
  • Sunday, smoothies at Wellness Warehouse, watched Please Stand By at the Labia – nice movie – LLAP, Company Gardens walk, watched The Search for Life at the Planetarium – Harrison Ford narrated NASA film – pretty good, Company Gardens Cafe, burgers and a sunset grapefruit mimosa at Mantra Cafe in Camps Bay, updated my geeky movies list, photo processing, in bed early.
  • The new Planetarium is pretty cool, go check it out.
  • I’m in the mood for health sprint soon.
  • I think we’re ready to call the recent Jolp app release Version 1.0.
  • Fat people need more sleep.

Have a fun week, crazy kids.

by joe at 12 June 2018 06:08 AM

11 June 2018

Graham Poulter (verdant)

How to spring-clean your blog

How to spring-clean your blog
Here are ways I found to clean up a Blogger blog. I've found and fixed many things with content, images, navigation, theme, descriptions, and analytics. [4 minutes]

Here's a 2-minute commentary podcast to listen to while you read, let me know what you think:

Spring-Cleaning Steps

Clean up the content

  • Remove old posts: delete old posts with no traffic and no usefulness to readers (some were Facebook-update type posts, from before Facebook was a thing). Also delete the comments.
  • Use headings: create proper headings and subheadings, since Blogger added support.
  • Remove cruft: remove broken links, outdated information and unhelpful asides.
  • Enhance clarity: clean up rambling bits, convert list-like paragraphs to bullets, add bold titles to bullets to aid readers in skimming the post.
  • Clean HTML: remove extra HTML elements that crept in over time, to be tidy.
  • Remove blank paragraphs: removing random white space improves post appearance.
  • Add read times: add [5 minutes] etc time estimates to help readers decide.
Some old posts I edited so extensively that I could republish them as new and redirect from the old URL, but I haven't bothered with that.

Clean up the images

  • Replace hotlinks: replace hotlinks with Blogger uploads of the images. Also helps prevent http resource on https page.
  • Design Title Images: square 800x800 title images work best on Blogger, though tall works better on Pinterest. The mobile preview uses middle 50% of the image. Avoid having te 
  • Increase Resolutions: if old images are looking low-res on new monitors, pick a higher-resolution image.
  • Title text: text that shows on mouse-over, and it overrides page title and alt text to be the image description on Pinterest.
  • Alt text: a text description of the image, overrides post title as Pinterest description. If you're not using Search Description, default description comes from first 140 characters of the post, which includes any alt text.

Clean up the navigation

  • Revise titles: rework post titles be more direct (8-12 words, under 60 characters) using one or more key phrases from top organic searches.
  • Make intro concise: write intro paragraph under 200 characters to avoid truncation in mobile post list or Pinterest description. Each sentence must be under 140 characters to avoid truncation when used in search result snippets or social shares.
  • Add Search Descriptions: enable under Settings:Search Preferences:Meta Description. Then add a Search Description to each post based off the 200-character intro.
  • Use broad labels: re-label using only a few categories, a category for each major audience, each post in 1 or 2 categories only.
  • Add internal links: add internal links to other posts a reader of this post might be interested in.

Clean up the blog

  • Pick theme: change from Contempo to Notable since images are not important in my posts.
  • Tweak timestamps: I removed the year from dates shown on the blog. My posts are so old 👴
  • Revise blog description: rewritten to sound better when sharing blog on social media.
  • Enable comments: switch from Google+ comments to Blogger comments, since commenting dropped off radically under Google+.
  • Tune AdSense fill rate: reduce fill rate from 60% to 5% to show fewer ads. I'm just playing with AdSense, only made €15 in 5 years anyway.

Clean up your discoverability

  • AdWords: run a campaign for niche posts that aren't ranking yet, with custom redirects for a short URL for the ad.
  • Set up HTTPS: enable HTTPS by default and HTTPS redirection.
  • Webmaster Tools: added https:// domain and sitemap.xml to Google Webmaster Tools, also register with Bing Webmaster Tools.
  • Google Analytics: add filter to exclude referrals from spam domains ("Campaign Source" filter), and exclude "m" in "Exclude URL Query Parameters" on the view, so you don't have two rows for each page.
  • Sharing + Follow Buttons: add proper sharing and follow buttons as Blogger built-in ones aren't great. Right now using Shareaholic with all the tracking and advertising crud turned off.

Things I should do

  • Social sharing: I should actually share my posts on Twitter, Facebook, Pinterest and Google+.
  • Pinterest Images: for Pinterest, best images have post title and nice photo, often a portrait.
  • Respond to comments: Many old comments I never replied to. I should spare time to reply to people soon after they comment.
  • Podcasts / Videos: I want to try creating companion podcasts and videos, now that it's much easier to create rich media with decent video and sound quality than a few years ago.

Q & A

Why the cleanup effort? After letting the blog languish for 5 years I checked Analytics and found it was getting a few hundred hits a month to a few popular posts. I'd recently started playing with AdWords out of interest, and began to see the blog from a whole new perspective of internet advertising, though I don't expect to make any money from it. So, I decided to clean things up.

What's the blog topic? There's no topic 😔 I post informational content - summaries, how-to's, reviews - on a wide range of subjects with mostly non-overlapping audiences. Wide-ranging helpful blogs do not gain a loyal followers, unlike niche-topic blogs or entertaining blogs. Were I more serious about wasting my time on blogging I'd have separate blogs on sub-domains for "books", "tech" and "lifestyle", instead of just labels.

Where do you get traffic? Mostly organic search for low-competition key phrases, and some referrals from attribution links for some of the how-to's. I usually write "niche posts" (not like this one) when I can't find what I want online, then I figure it out and post about, then other people find my post.

What'll you be doing next? I'm hoping to do more book summaries. Literature reviews were my strong point in academia. Probably mixing in random how-to's, but unlikely to be highly technical like some of the old ones (use only in-house tools now). I do want to try embedding companion podcasts into some posts.

P.S. After you've decluttered your blog, try decluttering your home with the outbox method.

Any more advice for cleaning up the blog? Drop me a comment.

by Graham Poulter (noreply@blogger.com) at 11 June 2018 08:27 PM

06 June 2018

Johann Botha (joe)

WordPress Backup Script

My WordPress backup script, which keeps daily, weekly and monthly backups of website files and database dumps.

Installation:

This should work on your garden variety Debian or Ubuntu server running WordPress and Mysql.

Copy this script to /usr/local/sbin/backup-websites

cat /etc/cron.d/backup-websites
# Backup websites and databases at around 3am
15 03 * * * root /usr/local/sbin/backup-websites

btw. sorry the indentation seems to go missing when I use the /code tag in WordPress.

Obligatory Muppets Photo

The script…

backup-websites:


#!/bin/bash
# Backup websites and databases

####
# Set some variables and paths

BACKUP_DIR=home/backups/websites
# Must contain backup.monthly backup.weekly backup.daily folders

FILE_DIRS_BASE=var/www
DATABASES=`echo "show databases" | /usr/bin/mysql --defaults-file=/etc/mysql/debian.cnf | grep -v Database | grep -v schema`

email=root@localhost

######
# Make appropriate destination directory

# Destination file names
date_daily=`date +"%Y-%m-%d"`

# Get current month and week day number
month_day=`date +"%d"`
week_day=`date +"%u"`

# On first month day do monthly
if [ "$month_day" -eq 1 ] ; then
destination=backup.monthly/$date_daily
else
# On Saturdays do weekly
if [ "$week_day" -eq 6 ] ; then
destination=backup.weekly/$date_daily
else
# On any regular day do daily
destination=backup.daily/$date_daily
fi
fi

cd /$BACKUP_DIR
mkdir $destination

# latest backup symlink
ln -snf $destination /$BACKUP_DIR/latest

####
# Create backup files in today's directory

stamp=`date +%Y-%m-%d_%H-%M-%S`

# Dump MySQL tables in incoming dir
for db in $DATABASES; do
/usr/bin/mysqldump --defaults-file=/etc/mysql/debian.cnf $db | bzip2 > \
/$BACKUP_DIR/$destination/$db.$stamp.sql.bz2
done

FILE_DIRS=`ls /$FILE_DIRS_BASE`

# Compress files, one tar file per sub directory
for filedir in $FILE_DIRS; do
tar -C / -cjf /$BACKUP_DIR/$destination/$filedir.$stamp.tar.bz2 $FILE_DIRS_BASE/$filedir
done

#####
# Email backup info

ls -l /$BACKUP_DIR/$destination/ | mail -s "[`uname -n`] Websites Backup $date_daily" $email

#####
# Clean up old backups

# Daily - keep for 7 days
find /$BACKUP_DIR/backup.daily/ -maxdepth 1 -mtime +7 -type d -exec rm -rf {} \;

# Weekly - keep for 30 days
find /$BACKUP_DIR/backup.weekly/ -maxdepth 1 -mtime +30 -type d -exec rm -rf {} \;

# Monthly - keep for 365 days
find /$BACKUP_DIR/backup.monthly/ -maxdepth 1 -mtime +365 -type d -exec rm -rf {} \;

#END

Keep your backups for less time by updating mtime above.

To recover your backups:

tar jxvf your-backup-file.tar.bz2
bunzip2 your-backup-file.sql.bz2

LLAP.

by joe at 06 June 2018 11:24 AM

28 May 2018

Graham Poulter (verdant)

How to extend the life of your clothes

How to extend the life of your clothes
Here are some tips on preserving your clothes! Ways to make the wash easier on the clothes, and increase the number of wears one can get between washes. [3 minutes]

Reduce the wear of washing

1. Wash clothes in laundry nets

Mesh laundry nets stop socks getting lost - they're all in a bag - and shield clothes from most of the friction-wear of laundering.

2. Choose the delicate wash cycle

The delicate cycle causes the least heat and friction damage. If animal fibers such as wool thermals are in the wash, use a non-bio detergent since the enzymes of bio detergents eat the proteins. Wool jumpers of course wash by hand with wool detergent.

3. Dry clothes on an airing rack

Air-drying causes less wear than tumble drying. The tumble-dried sheets and towels have visible wear over time.

Get more wears per wash

4. Air clothes on a rail after use

Airing worn clothes on a rail allows odors to disperse. One can cycle between shirts, with worn shirts airing on the rail instead of retaining odors in the wardrobe.

5. Air clothes outdoors

If otherwise clean pants or jackets fail the sniff test say due to spending the evening in a bar or restaurant, airing them outside for a day or two likely restores them without washing. Winter coats go years without dry-cleaning if aired properly.

6. Keep shoes on a shoe rack

Using a shoe rack keeps the smells of the city near the door instead of in the wardrobe, and helps keep the floor clean.

7. Shower in the evening

By showering in the evening the bed sheets can go 3 or even 4 weeks before feeling like they need a change. Showering in the morning, expect to wash the sheets every week.

Bonus tip: Down Jackets

Down jackets can be washed once a year after the cold season, with down detergent on a woolen cycle, then dried flat on an airing rack. To remove clumps from the down, follow with a 20-minute delicate-cycle tumble dry with tennis balls.

Conclusion

What do you think of these ways of making clothes last longer? Less wear from washing, more wears between washes.  If you disagree or have more tips, comment below.

P.S. check out my my advice for removing smoke smells from waterproof jackets and decluttering your home with the outbox method.

Try this short companion podcast:

by Graham Poulter (noreply@blogger.com) at 28 May 2018 10:13 PM

19 May 2018

Christel Breedt (Pirogoeth)

Some feminists are transphobic - deal with it.


Lately there's been a lot in the news about high profile transgender folk. Seems people have been talking over tea a bit more than before, and recently a close friend of mine asked me to comment on a thread where a troll was bandying about some choice words regarding transsexual women wanting to be accepted as, you know, women.This made me feel a bit ranty. So without further ado - to the rant blog!

Transwomen are women. Transmen are men. And yes, I mean like "real" men, and "real" women (why do those concepts give me the heebie-jeebies?)
Why do I say this? Because some really clever scientists who did some really complicated sciency stuff (you know, like brainscans and twin studies and post mortem brain structure studies) tells me that this is probably a good thing to do.

Some very pushy feminist types, especially some very butch lesbians and a couple of very camp gay guys, have given me some really pretty sexy philosophical arguments as to why they think gender is all a evil societal construct. They seemed legit, being homosexual and all, and some of them even had philosophy majors. You could be forgiven for thinking their opinions are authoritative. Still, maybe its just me, but where medical conditions are concerned, I tend to prefer theories that contain data that came from a medical professional, not a minority activist or a philosophy major.

While from a purely genotypical and phenotypical perspective it is technically accurate to describe transsexual individuals as being male/female  according to  birth sex, it turns out that what defines ones gender as either male or female is rather more complex than either genetics OR genitalia.


I can refer you to a very informative article which explains in fairly thorough detail where the diagnosis, treatment and definition of trans comes from historically, and what current medical science has come to know in recent years about this condition. My favourite bit is where they come to the conclusion that brain development  in-utero largely determines gender identity, and that definite brain structure differences are observable between genders - also in transsexual folk who identify as those genders.

The  long and the short of it seems to be that our brains are actually structurally linked to our gender identities, and, we were born that way.


If you want to know more, feel free to hit up some Google scholar articles or a couple of good libraries. I was required to do so myself not so long ago. Oh and look at recent publications, since the really juicy mind altering discoveries are less than 10 years old.

As for the philosophers:

I have several issues with the inherent sexism and patriarchal structures that tend to crop up in transgendered circles. I wondered why it was so important for transwomen/men to be "women" or "men", and not just "transwomen"/ "transmen". Surely we should be fighting for these terms to be accepted, instead of trying to obscure the problem behind genetic and phenotypical word games?

I'm pansexual. I'm in a polyamorous triad. I'm into BDSM. I do not subscribe to any religious or philosophical dogma for much longer than it takes me to read up about it. To paraphrase Henry Rollins I "burned my closet for kindling". Lets face it, I have a desperate incompatibility with things coercively normative.

However, when I fell in love with a thoroughly no-nonsense young trans individual who had a great sense of humour and a low bullshit quotient, I discovered to my shame that I had been propagating coercively normative ideologies in the name of equality and activism for a number of years. I for a long time believed I was Cisgendered (my wetware matches my software, I was assigned my chosen gender at birth) and therefore was rather blind to cissexism.

What 20 years of feminism could not teach me, I learned when I loved someone who lived with this on a daily basis. In time I discovered that what I thought I knew about gender equality was littered with prejudice and propaganda propagated by all factions of the gender wars.

Interestingly, transpeople have often been invisible observers in the gender wars. For those who do not appear obviously transgendered (they "Pass") a world is opened up where they become privy to the secret intimate world that is guarded by each individual gender grouping. 

The cisgendered majority accept every day in their basic interactions a rather remarkable amount of silent sexism from each other. Take it from a transperson  to know just how many small prejudices we are all blind to. They have been taking this stuff in their stride for so long, nobody even knows it's bullshit - because none of them will ever see the way the other side lives. I sure as hell learned a lot about invisible sexism from my trans partner, and I'm a better feminist for broadening my mind to include that worldview.

Without shame or rancour, I was set straight about what transsexuality is and isn't. It is, simply put, a medical condition. That is all. 


 

The hysterical rantings often held up in an attempt to protect the sanctified and holy spaces of the cisgendered (like bathrooms and womens only concerts and feminism) are, in my opinion, roughly on par with the arguments proposed for "protecting" heterosexual marriage. Same bullshit, different topic. Transgender people are as much of a threat to cisgendered interests and spaces as homosexuals are to heterosexual marriage.

There are no really tasty benefits to being transgendered. The suicide rate is over 50%, the overall death rate 75%, due to such factors as starvation, exposure to the elements, assualt, and murder. They face the most frequent and most serious discrimination of any minority group in the LGBT cluster. They are frequently profiled as being mentally ill, pathological liars and sexually deviant (which, research shows, is no more likely to be true of them than of their cisgendered counterparts, or other minority groupings). Hell, when their condition is managed using internationally approved treatment protocols, they actually look pretty...normal. You know, like that kid with bad eyes who got glasses and could then live the rest of their lives reading the board from the back row like anyone else.

Still, it turns out that being accepted as your target gender at all is a pretty shitty second prize when you have to live as "Freaky Friday" for your entire childhood, youth and reproductive years (and if you are treated in your youth, the treatment leaves you sterile, and you live the rest of your life afraid to have people see your childhood photos).

Lets not even BEGIN to talk about the cost and scarcity of medical treatment. How would you feel if you needed that pair of glasses, but you had to wait 25 years* to get one because your minority is so small that no facilities are available to receive treatment sooner? Unless, of course, you're willing to fork out half a million Rand for good treatment in Thailand.

For transsexuals, being accepted as a woman/man is not primarily an attempt at subdueing an unaccepting society, but a best effort attempt at treating a truly craptastic medical condition.There is, at best, only poor palliative and prosthetic treatment, and you'll just have to live with the fact that there is no real remedy for the chimerism of having brain structures mapped onto physiology that is opposite to your gender.

You know, like having to wear scratchy contact lenses or bottlebottom brainy-specs. Even if they are really cool specs, or really nice contact lenses, you are still going to get called four eyes, or end up with pink eye twice a year. But then...glasses won't get you  beaten to death by bullies the way peeing standing up/ not peeing standing up potentially can.

Its not a lame defense against the patriarchy or an attempt to gain male privilage. Its not generally motivated purely by self hatred or a desire to conform to gender normativity. Its about having what all of the rest of the Cisgendered schmucks take for granted - having your phenotypical carpet match your neurological drapes.

For me, accepting transsexual women as women is about widening the parameters of what it means to be a man or a woman. To permit anyone who wants to claim that gender construct to do so if they wish - much like allowing modern feminists to be Volvo-driving soccer moms (if that's what floats their neo-feminist boats). Its about letting go of policing gender. Its about conceding that despite our desperate attempt to claim that gender is purely a societal construct, science has shown us this is NOT the case but gender IS more multifaceted than previously believed.

It really sucks for activists when their arguments dissolve in a puff of scientific smoke, but I am a scientist first, and an activist second, so my brain didn't hurt so much the day after I woke up from this particular hangover.

We tend not to deal with that kind of defeat well, especially when our entire argument rests on (faulty) assumptions about gender that we have been fighting to entrench for generations - the idea that we have roughly interchangeable brains gender-wise but VERY definite physical differences.

As usual, in nature, the argument is rather more complex. Boys and girls can be whatever they want to be, because for every type of girl or boy or gender variant individual, there is a place in the wide continuum of human diversity. If we learn anything at all from the Kinsey report (other than that humans are a really creatively horny species), it is that human beings are extremely diverse.

Everyone deserves to be whatever they were born to be, or want to be - without judgement from either the moral majority OR minority.


I've reached a place in my life where I don't fight for us to wipe clean the slate of gender constructs anymore. I fight for the right for anyone to claim whatever gender construct they choose for themselves - because to me that is true equality. That is honouring the bell-curve. That is accepting that we are both a product of nature AND nurture.
Accepting that some men have vaginas and some women have penises is really of no consequence to anyone who genuinely wants to fuck the patriarchy and free women from eons of oppression. Becoming comfortable with the reality that we can't know the content of a persons mind or underwear by looking at them is likewise a good way to advance the cause of equality. We really need to get over the compulsive need to know what whether they have the right equipment down south BEFORE we buy a person a drink. The answer is that that's just one more thing about you'll need to learn the polite way: By getting to know them.

Trans people generally fight really hard for gender equality and to destroy sexism and patriarchal power - because transmen really hate the patriarchy for
telling them that being a man isn't an option, and transwomen really hate sexism because society treats them like even bigger crap than cis women.

They hate this stuff JUST LIKE WE DO.

So in short, friends, please can we stop hating on them for wanting to be men and women just like you?

:)

*Recent information from the Transgender therapy team at Grootte Schuur Hospital

by Whizper (noreply@blogger.com) at 19 May 2018 10:02 PM

16 May 2018

Christel Breedt (Pirogoeth)

My child just got diagnosed today. What do I need to know?



Dear carer.

Everything and nothing has changed.

When we arrive we get handed a set of default character traits from a lotto basket, and sent off to the world without a user guide or a walkthrough.

All of us.

But two things are known.

There are those who understand that kindness and cooperation is the only path, not just between equal adults but between all living things on the earth. ALL living things.

And there are those that believe they control this world and things or other people in it.

Be the former. There's no control over your life.

There's just learning to surf the waves that would drown you with style, reveling in that with others, learning the skill or teaching it. Control is the dream, the illusion we all want to share.



I'm going to be counting on you to teach me when you learn, because my waves are the kind that put people under the ground early more often than normal.

But dude, if I lick it, you're gonna watch me surf some epic waves.

People will try to stop you seeing me as a person like you. They will tell you what to do or think or feel or believe, as they try to determine how to get me to behave.

Aim to misbehave. It's said no well behaved woman ( or for that matter man) ever made history. Think on that.


Trust the ones who try to understand what YOU think. Who help you better understand what other people are thinking by helping you hear and see them better when they try to tell you.

Those are the good ones with the knowings of the way of life.

And remember I'm not little you, or less you, or kinda you, or you plus.

I'm just you. We all are. We are all one thing, and the same thing: Sentient.

Forget literally everything else you think you know. I'm about to matrix your life.

You can let that break you.  You can try to fight the sudden break in your world and sell me out for a steak and some potatoes and the illusion of that comfortable reality back again...

Or you can realise that for the most part there really is no spoon.

I don't need to speak, or write normally to communicate.

I don't need to hit milestones to develop.

I don't need to recover from being different. We're literally all different.

I am also sentient like you. Nothing changes.

But things just got real because someone noticed I'm off the proscribed child behavioural script and are trying to put me back on track.

I'm likely not liking that track.



Get me out of here if you need to. I might not be able to run. But I can show you if you watch me closely. Watch my eyes.

We're going to need to stick together, you and I, if we want to get through.

But you got this.

Up to you. Blue Pill, Red Pill.

by Whizper (noreply@blogger.com) at 16 May 2018 07:13 PM

Jonathan Carter (highvoltage)

Video Channel Updates

Last month, I started doing something that I’ve been meaning to do for years, and that’s to start a video channel and make some free software related videos.

I started out uploading to my YouTube channel which has been dormant for a really long time, and then last week, I also uploaded my videos to my own site, highvoltage.tv. It’s a MediaDrop instance, a video hosting platform written in Python.

I’ll still keep uploading to YouTube, but ultimately I’d like to make my self-hosted site the primary source for my content. Not sure if I’ll stay with MediaDrop, but it does tick a lot of boxes, and if its easy enough to extend, I’ll probably stick with it. MediaDrop might also be a good platform for viewing the Debian meetings videos like the DebConf videos. 

My current topics are very much Debian related, but that doesn’t exclude any other types of content from being included in the future. Here’s what I have so far:

  • Video Logs: Almost like a blog, in video format.
  • Howto: Howto videos.
  • Debian Package of the Day: Exploring packages in the Debian archive.
  • Debian Package Management: Howto series on Debian package management, a precursor to a series that I’ll do on Debian packaging.
  • What’s the Difference: Just comparing 2 or more things.
  • Let’s Internet: Read stuff from Reddit, Slashdot, Quora, blogs and other media.

It’s still early days and there’s a bunch of ideas that I still want to implement, so the content will hopefully get a lot better as time goes on.

I have also quit Facebook last month, so I dusted off my old Mastodon account and started posting there again: https://mastodon.xyz/@highvoltage

You can also subscribe to my videos via RSS: https://highvoltage.tv/latest.xml

Other than that I’m open to ideas, thanks for reading :)

by jonathan at 16 May 2018 06:19 PM

14 April 2018

Christel Breedt (Pirogoeth)

We are all different and that's ok, or Why you should disclose your child's diagnosis to them young




CW: Bullying and victimisation stuff.

On the topic of whether to reveal a diagnosis of autism or ADHD to a child young:

I am personally against hiding any confirmed diagnosis with the potential to affect a child's quality of life, especially if it causes cognitive impairments....and the reason will probably strike you as obvious once I'm done explaining.
When you have cognitive impairments, such as with executive function, you know. You know the way that people who are sober now will know they are drunk later. They won't know WHAT they know, but you know because you live in your brain and it is always part of your awareness.
If no explanation is given you think what you are experiencing IS EXPERIENCED BY EVERYONE. You may misunderstand fundamental concepts explained to you about how consciousness, concentration, awareness, attention, focus, discipline, effort and intentionality works if you aren't told that your experience is not the most commonly described experience.

You also begin to think that you are doing it wrong. That your character is flawed. That you are stupider than you actually are, or that you're missing something even when you aren't. It leads to a certain special sort of pervasive gaslighting, invalidation and minimisation that is extremely destructive to our sense of self.


This causes you to fail to have safe boundaries in place to protect your energy and your health - when you think your best effort is normal and it's actually you burning yourself up like a sun just to do what Joe Soap can do half asleep you are setting yourself up for a lot of stress related health disorders and mental health issues later in life. This is DANGEROUS.

Let's illustrate using an example: Curly hair.

If I wasn't told as a child that my curly hair was natural for me, that my granny had curly hair, that my curly hair is normal for me and isn't at all needed to be straightened of flattened or controlled but is instead beautiful and rich and lovely, I might have been very self conscious about how ...curly ... it is.
I mean I'd watch others combing their hair and begin to feel like I must be doing my hair wrong or something because I can't make it stay that flat and I'm doing EXACTLY the same thing that Susie Q does with her hair at the sleepover but I still look like a golliwog and she's all sleek. I might have thought I was BAD at doing my hair, or that my hair was UNTIDY. I might have started to feel guilty and insecure instead of angry and offended when someone told me my hair was messy.

Or another example: Freckles

A doctor (a resident from central africa who was practicing in our hospitals here and had only READ about freckles in textbooks) once asked me whether my sister's freckles were a hyperpigmentation disorder. If we'd never been raised to understand that ginger people just have freckles (because we do, duh!) we might have had a doctor go on a wild goose chase trying to pin down what the strange illness was that was causing our pigmentation errors....and we'd have walked around telling everyone we had a hyperpigmentation disorder which depending on the context could have caused people to treat us either as diseased pariahs, or possibly mentally ill persons who are delusional and hypochondriac for supposing that freckles are a disease..... And this would develop based on what we were told to expect (which shapes the way we perceive ourselves and our traits).

BUT Hiding a diagnosis from a child isn't empowering the way some people think. It doesn't help us NOT to be told we are different. That is like conspiciously not mentioning the fact that my hair is EXTREMELY red. I mean I can see myself in a mirror....why aren't we talking about this? Is it something bad? .......



You know I'm right guys. This is how we all work.


At best, It is trying to straighten their curly hair, or letting them silently wonder why their hair doesn't comb out right. It's letting them walk around thinking they have a hyperpigmentation disorder or that the fact they are the only red haired child in the family must mean they were adopted.....

At worst it is teaching them to associate feeling sick, dysfunctional, exhausted, overwhelmed, scared, sad, misunderstood and confused - with BEING NORMAL.

You REALLY don't want that. Being weird? I EMBRACE that shit. In the words of Brene Brown, what makes us vulnerable makes us beautiful.
My weird IS MY AWESOME. And that is a lesson my parents DID teach me, alongside the truth of why my mind doesn't work the way other people's minds do.

I was made very aware of the sad fact that because the world doesn't give a rat fuck about us on the institutional level, we may need medication to survive our childhoods and that's NOT OUR FAULT.


That's like our curly hair. Or our freckles. We're RARE. But we aren't a MISTAKE.

Understanding the sad but real fact that statictically neurodiverse people are a minority, that we require different things to be happy and those things are not commonly available and we may get looked at funny for needing them (BUT THAT IS NOT OUR FAULT) helped me.
I still developed a self esteem problem, but that was mainly because my school mates and fucked up teachers used to part the way in the halls to let me through like I was Moses at the red sea, and they hectored me calling me "Stinky" and told me "You must have held up a sieve when your mom threw you with shit" and "You look like stuck your head in period blood" and "You're the laziest child I've ever known". It wasn't because I was told I was different. It was because I was treated differently, and when I didn't know why I thought it was MY FAULT.

Let's face it having things like that said to you still hurts even when you know they are wrong. But at least you cant TALK to people about it in some way that makes sense because there is a vocabulary for you to describe your experience.
Give them a reason. Make sense of the cruelty of others. Tell them the truth. But remind them that DIFFERENT is not UGLY or BAD or WRONG.

It's just the reason there are more than one kind of flower, or all sorts of different birds or trees. It's why we have shoes that aren't one size fits all, and haircolours that arent all one colour, and why we have tall people and short people and why we have different kinds of music or why not everyone likes the same things you do or eats drinks their tea the same way.... because difference is EVERYWHERE when you look for it.

We just tend to forget that the rarerst and most special things in the world sometimes only bloom in the tops of the highest trees in the amazon, or grow in the depths of an ocean vent so hot that no other thing lives there. We are extremophiles. We need special environments, we need our hothouses....but it's because this culture is broken. Western civilisation doesn''t CARE if you need a specific diet like the orchid food for amazonian orchids. It doesn't CARE if you can only grow in places that have a temperature gradient that never varies under your hottest oven setting, or that you turn to jelly when you are brought up from the ocean depths because you cannot live at normal pressure.


It's NOT OUR FAULT.


AND THE WORLD NEEDS US.

Autistic people have a purposeful life too. They have a reason to exist in the ecosystem of earth. We are NOT an evolutionary fuckup. The ecosystem doesn't keep trash on the payroll guys. There's far too many of us for our geneset to be all bad things and no good.

Understanding that we might be like those extermophiles. that we need special environments and special diets and special accommodations to live helps me.
Understanding that extremophiles fulfill niche roles in the ecosystem helps me. They go where no other creature dares to go and grows in untenable conditions so that we may have from the crook of a tree canopy one of the most beautiful yet delicate flowers in the world, the Orchid.
So what if you are a hot house flower? The world should be privileged they didn't have to climb a tree in the amazon to see you!

We really just need to stop feeling so bad about being different, needing support, being sensitive....
Because you are beautiful. Like. All of us are friggin wandering miracles of nature that are fully formed breathing art. We have universes inside of us, and we carry libraries behind our eyes and recording studios behind our ears and we have a band in our mouth and we are extraordinary biomechanical machines that are puppets which dance for themselves by themselves.
That's just NOT OUR FAULT.
It is our ARTISTRY.

by Whizper (noreply@blogger.com) at 14 April 2018 10:01 AM

24 December 2017

Jonathan Carter (highvoltage)

Hello, world! – Welcome to my Linux related videos

I’ve been meaning to start a video channel for years. This is more of a test video than anything else, but if you have any ideas or suggestions, then don’t hesitate to comment.

by jonathan at 24 December 2017 06:13 PM

22 November 2017

Graham Poulter (verdant)

The keys to doing long-form Narrative Improv

"Do it Now" book cover
Here are some key ingredients for full-length improvised plays known as Narrative Improv. Providing tips on story structure, normalcy, the protagonist, consequences and clarity. [4 minutes]

In Narrative Improv the group creates a full-length play with a more-or-less coherent story that runs all the way through from "once upon a time" through to the "happily ever after" (or not so happily). It's a flexible form that stands in contrast to both highly structured forms like the Armando or Harold, and to the unstructured "montage" improv of possibly-connected scenes with once-off characters.

The following tips are summarised from the excellent little book, Do It Now: Essays on Narrative Improv by Parallelogramophonograph ("Pgraph" for short), especially the "What comes next?" conclusion on page 73:

1. Build the story spine

The story spine is a generic story structure: "Once Upon A Time ... And Every Day ... Until One Day ... And Because Of That ... And Because Of That (repeat to taste) ... Until Finally ... And Ever Since Then". Virtually any plot can be arranged or rearranged into a story spine form, and you can practice making story spines in a group exercise where each person, in turn, adds an element to the story.

2. Invest in normalcy

It's almost impossible to spend too much time on the "once upon a time ... and every day" part of the show where everything that happens is stuff that always happens. Even if two characters start in a sword fight, that's just something they do every day. Normalcy builds the world, the characters, the relationships. Take up multiple scenes, a large fraction of your show, just showing different parts of the everyday world of these people. If someone brings in the plot too soon just act like their proposed plot is something that happens all the time. One day, when the "One Day" finally happens, you will have loads of material to work with!

3. Identify the protagonist

Around the "Until One Day" point, different characters will be making offers of goals and desires. As a group, you gradually home in with a "spotlight" on different characters to settle on one with a strong offer, who will become the protagonist for the rest of the story. Give the protagonist a strong and simple desire - even if it's just to get their rug back (it really tied the room together!). Then the rest of the story becomes scenes that either help or hurt the protagonist in pursuit of their goal.

4. Make scenes have clear consequences

More than "making bold choices", by giving actions and scenes clear, strong consequences, the "And Because Of That" will come naturally, and you get a feel for where the story is going. This frees you from "oh no, where are we going?" scrambling for the next step, giving you space to have playful moments in your scenes.

5. Be clunky but clear

Be overt and clear about the details of scenes and relationships, particularly early in your group's development. It's the opposite of being vague and hoping that the others figure out what you're getting at or fill in the gaps. "My dear brother Kevin, what a dank cave this is!" comes out clunky but makes the relationship and location clear! Subtlety will develop over time.

6. Use a range of acting styles and tones

Deliberately try out new genres, and new acting styles from everyday realism, to stage realism, to over-the-top stylized acting. Try out new character tropes from movies and theatre. These deepen your stories and make your shows distinct, taking your improv to new places.

Conclusion: PGraph advise us to "play, practice, enjoy getting it horribly wrong - that's how you learn! -and revel in getting it right." I hope this taste inspires you to get the book or take the class on Narrative Improv!

P.S. I bought and read "Do It Now" as preparation for a 6-week class on Narrative Improv facilitated by Neil Curran of Lower The Tone, and have no affiliation with Parallelogramophonograph.

Do you have some advice to share on doing Narrative Improv? Comment below!

by Graham Poulter (noreply@blogger.com) at 22 November 2017 09:49 PM

08 August 2017

Adrian Frith (htonl)

Linguistic diversity map of South Africa

The linguistic diversity index measures the probability that two people selected at random from a population speak different home languages. The map below, which I produced, depicts the linguistic diversity index calculated on a 10-kilometre-wide hexagonal grid across South Africa.

A map of South Africa showing the linguistic diversity index calculated on a 10-kilometre-wide hexagonal grid Linguistic diversity; click to enlarge

08 August 2017 10:00 PM

25 September 2016

Michael Gorven (cocooncrash)

XBMC for Raspberry Pi

This page describes how to install XBMC on a Raspberry Pi running Raspbian. You can either install packages on an existing Raspbian installation, or you can download a prebuilt image and flash it to an SD card.

Installing packages on an existing installation

I've published a Debian archive containing packages for Kodi/XBMC and some dependencies which it requires. This can be setup on an existing Raspbian installation (including the foundation image).

Installing

The easiest way to install the package is to add my archive to your system. To do this, store the following in /etc/apt/sources.list.d/mene.list:

deb http://archive.mene.za.net/raspbian wheezy contrib

and import the archive signing key:

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-key 5243CDED

Then update the package lists:

sudo apt-get update

You can then install it as you would with any other package, for example, with apt-get:

sudo apt-get install kodi

The user which you're going to run Kodi as needs to be a member of the following groups:

audio video input dialout plugdev tty

If the input group doesn't exist, you need to create it:

addgroup --system input

and setup some udev rules to grant it ownership of input devices (otherwise the keyboard won't work in Kodi), by placing the following in /etc/udev/rules.d/99-input.rules:

SUBSYSTEM=="input", GROUP="input", MODE="0660"
KERNEL=="tty[0-9]*", GROUP="tty", MODE="0660"

The GPU needs at least 96M of RAM in order for XBMC to run. To configure this add or change this line in /boot/config.txt:

gpu_mem=128

You will need to reboot if you changed this value.

Running

To run XBMC, run kodi-standalone from a VT (i.e. not under X). XBMC accesses the display directly and not via Xorg.

If you want Kodi to automatically start when the system boots, edit /etc/default/kodi and change ENABLED to 1:

ENABLED=1

Run sudo service kodi start to test this.

Release history

  • 15.2-2: Isengard 15.2 release, and most PVR addons.
  • 14.2-1: Helix 14.2 release.
  • 14.1-1: Helix 14.1 release.
  • 14.0-1: Helix 14.0 release.
  • 13.1-2: Link to libshairplay for better AirPlay support.
  • 13.1-1: Gotham 13.1 release.
  • 12.3-1: Frodo 12.3 release.
  • 12.2-1: Frodo 12.2 release.
  • 12.1-1: Frodo 12.1 release. Requires newer libcec (also in my archive).
  • 12.0-1: Frodo 12.0 release. This build requires newer firmware than the raspberrypi.org archive or image contains. Either install the packages from the raspberrypi.org untested archive, the twolife archive or use rpi-update. (Not necessary as of 2013/02/11.)

Flashing an SD card with a prebuilt image

I've built an image containing a Raspbian system with the XBMC packages which you can download and flash to an SD card. You'll need a 1G SD card (which will be completely wiped).

Flashing

Decompress the image using unx:

% unxz raspbian-xbmc-20121029.img.xz

And then copy the image to the SD card device (make sure that you pick the correct device name!)

% sudo cp xbmc-20121029-1.img /dev/sdb

Customising

The image uses the same credentials as the foundation image, username "pi" and password "raspberry". You can use the raspi-config tool to expand the root filesystem, enable overclocking, and various other configuration tasks.

Updating

Both Raspbian and Kodi can be updated using normal Debian mechanisms such as apt-get:

# sudo apt-get update
# sudo apt-get dist-upgrade

Release history

Unstable versions

I've started building packages for the upcoming Jarvis release. These are in the new unstable section of the archive. To install these packages update your source list to look like this:

deb http://archive.mene.za.net/raspbian wheezy contrib unstable

Release history

  • 16.1-1: Jarvis 16.1
  • 16.0-1: Jarvis 16.0
  • 16.0~git20151213.a724f29-1: Jarvis 16.0 Beta 4
  • 15.2-2: Isengard 15.2 with packaging changes to support PVR addons, and most PVR addons.
  • 15.2-1: Isengard 15.2
  • 15.1-1: Isengard 15.1
  • 15.0-1: Isengard 15.0
  • 15.0~git20150702.9ff25f8-1: Isengard 15.0 RC 1.
  • 15.0~git20150501.d1a2c33-1: Isengard 15.0 Beta 1.
  • 14.2-1: Helix 14.2 release.
  • 14.1-1: Helix 14.1 release.
  • 14.0-1: Helix 14.0 release.
  • 14.0~git20141203.35b4f38-1: Helix 14.0 RC 2
  • 14.0~git20141130.ea20b83-1: Helix 14.0 RC 1
  • 14.0~git20141125.4465fbf-1: Helix 14.0 Beta 5
  • 14.0~git20141124.ec361ca-1: Helix 14.0 Beta 4
  • 14.0~git20141116.88a9a44-1: Helix 14.0 Beta 3
  • 14.0~git20141103.d6947be-1: Helix 14.0 Beta 1. This requires firmware as of 2014/10/06 and libcec 2.2.0 (both included in the archive). There are also builds for Jessie but I haven't tested them. PVR addons are also updated.
  • 14.0~git20141002.d2a4ee9-1: Helix 14.0 Alpha 4

by mgorven at 25 September 2016 04:54 AM

06 January 2016

Michael Gorven (cocooncrash)

Memory optimised decompressor for Pebble Classic

TLDR: DEFLATE decompressor in 3K of RAM

For a Pebble app I've been writing, I need to send images from the phone to the watch and cache them in persistent storage on the watch. Since the persistent storage is very limited (and the Bluetooth connection is relatively slow) I need these to be as small as possible, and so my original plan was to use the PNG format and gbitmap_create_from_png_data(). However, I discovered that this function is not supported on the earlier firmware used by the Pebble Classic. Since PNGs are essentially DEFLATE compressed bitmaps, my next approach was to manually compress the bitmap data. This meant that I needed a decompressor implementation ("inflater") on the watch.

The constraint

The major constraint for Pebble watchapps is memory. On Pebble Classic apps have 24K of RAM available for the compiled code (.text), global and static variables (.data and .bss) and heap (malloc()). There is an additional 2K for the stack (local variables). The decompressor implementation needed to have both small code size and variable usage. I discovered tinf which seemed to fit the bill, and tried to get it working.

Initially, trying to decompress something simply crashed the app. It took some debug prints to determine that code in tinf_uncompress() wasn't even being executed, and I realised that it was exceeding the 2K stack limit. I changed the TINF_DATA struct to be allocated on the heap to get past this. At this stage it was using 1.2K of .text, 1.4K of .bss, 1K of stack, and 1.2K of heap (total 4.8K). I set about optimising the implementation for memory usage.

Huffman trees

Huffman coding is a method to represent frequently used symbols with fewer bits. It uses a tree (otherwise referred to as a dictionary) to convert symbols to bits and vice versa. DEFLATE can use Huffman coding in two modes: dynamic and fixed. In dynamic mode, the compressor constructs an optimal tree based on the data being compressed. This results in the smallest representation of the actual input data; however, it has to include the computed tree in the output in order for a decompressor to know how to decode the data. In some cases the space used to serialise the tree negates the improvement in the input representation. In this case the compressor can used fixed mode, where it uses a static tree defined by the DEFLATE spec. Since the decompressor knows what this static tree is, it doesn't need to be serialised in the output.

The original tinf implementation builds this fixed tree in tinf_init() and caches it in global variables. Whenever it encounters a block using the fixed tree it has the tree immediately available. This makes sense when you have memory to spare, but in this case we can make another tradeoff. Instead we can store the fixed tree in the same space used for the dynamic tree, and rebuild it every time it is needed. This saves 1.2K of .bss at the expense of some additional CPU usage.

The dynamic trees are themselves serialised using Huffman encoding (yo dawg). tinf_decode_trees() needs to first build the code tree used to deserialise the dynamic tree, which the original implementation loads into a local variable on the stack. There is an intermediate step between the code tree and dynamic tree however (the bit length array), and so we can borrow the space for the dynamic instead of using a new local variable. This saves 0.6K of stack.

The result

With the stack saving I was able to move the heap allocation back to the stack. (Since the stack memory can't be used for anything else it's kind of free because it allows the non-stack memory to be used for something else.) The end result is 1.2K of .text, 0.2K of .bss and 1.6K of stack (total 3.0K), with only 1.4K counting against the 24K limit. That stack usage is pretty tight though (trying to use app_log() inside tinf causes a crash) and is going to depend on the caller using limited stack. My modified implementation will allocate 1.2K on the heap by default, unless you define TINF_NO_MALLOC. Using zlib or gzip adds 0.4K of .text. You can find the code on bitbucket.

by mgorven at 06 January 2016 06:28 AM

31 October 2015

Adrian Frith (htonl)

Historical topographic maps of Cape Town

I’ve made an interactive website with six sets of topographic maps of Cape Town and surrounds covering the period from 1940 to 2010. You can zoom in and move around the maps, switching from one era to another.

31 October 2015 11:00 PM

03 October 2015

Simon Cross (Hodgestar)

Where to from here?

Closing speech at the end of PyConZA 2015.

We’ve reached the end of another PyConZA and I’ve found myself wondering: Where to from here? Conferences generate good idea, but it’s so easy for daily life to intrude and for ideas to fade and eventually be lost.

We’ve heard about many good things and many bad things during the conference. I’m going to focus on the bad for a moment.

We’ve heard about imposter syndrome, about a need for more diversity, about Django’s flaws as a web framework, about Python’s lack of good concurrency solutions when data needs to be shared, about how much civic information is locked up in scanned PDFs, about how many scientists need to be taught coding, about the difficulty of importing CSV files, about cars being stolen in Johannesburg.

The world is full of things that need fixing.

Do we care enough to fix them?

Ten years ago I’d never coded Python professionally. I’d never been to a Python software conference, or even a user group meeting.

But, I got a bit lucky and took a job at which there were a few pretty good Python developers and some time to spend learning things.

I worked through the Python tutorial. All of it. Then a few years later I worked through all of it again. I read the Python Quick Reference. All of it. It wasn’t that quick.

I started work on a personal Python project. With a friend. I’m still working on it. At first it just read text files
into a database. Slowly, it grew a UI. And then DSLs and programmatically generated SQL queries with tens of joins. Then a tiny HTML rendering engine. It’s not finished. We haven’t even released version 1.0. I’m quietly proud of it.

I wrote some games. With friends. The first one was terrible. We knew nothing. But it was about chickens. The second was better. For the third we bit off more than we could chew. The fourth was pretty awesome. The fifth wasn’t too bad.

I changed jobs. I re-learned Java. I changed again and learned Javascript. I thought I was smart enough to handle
threading and tons of mutable state. I was wrong. I learned Twisted. I couldn’t figure out what deferreds did. I wrote my own deferred class. Then I threw it away.

I asked the PSF for money to port a library to Python 3. They said yes. The money was enough to pay for pizza. But it was exciting anyway.

We ported another library to Python 3. This one was harder. We fixed bugs in Python. That was hard too. Our patches were accepted. Slowly. Very slowly. In one case, it took three years.

Someone suggested I run PyConZA. I had no idea how little I knew about running conferences, so I said yes. I asked the PSF for permission. They didn’t know how little I knew either, so they said yes too. Luckily, I got guidance and support from people who did. None of them were developers. Somehow, it worked. I suspect mostly because everyone was so excited.

We got amazing international speakers, but the best talk was by a local developer who wasn’t convinced his talk would interest anyone.

I ran PyConZA three more times, because I wasn’t sure how to hand it over to others and I didn’t want it to not happen.

This is my Python journey so far.

All of you are at different places in your own journeys, and if I were to share some advice from mine, it might go as follows:

  • Find people you can learn from
    • … and make time to learn yourself
  • Take the time to master the basics
    • … so few people do
  • Start a project
    • … with a friend(s)
  • Keep learning new things
    • … even if they’re not Python
  • Failure is not going to go away
    • … keep building things anyway
  • Don’t be scared to ask for money
    • … or for support
    • … even from people who aren’t developers
  • Sometimes amazing things come from one clueless person saying, “How hard can it be?”
  • Often success relies mostly on how excited other people are
  • Stuff doesn’t stop being hard
    • … so you’re going to have to care
    • … although what you care about might surprise you.

Who can say where in this complicated journey I changed from novice, to apprentice, to developer, to senior
developer? Up close, it’s just a blur of coding and relationships. Of building and learning, and of success and
failure.

We are all unique imposters on our separate journeys — our paths not directly comparable — and often wisdom seems largely about shutting up when one doesn’t know the answer.

If all of the broken things are going to get fixed — from diversity to removing the GIL — it’s us that will have to fix them, and it seems unlikely that anyone is going to give us permission or declare us worthy.

Go build something.

by admin at 03 October 2015 03:05 PM

16 August 2015

Simon Cross (Hodgestar)

So what is this roleplaying thing anyway?

I ran a roleplaying module [1] for some friends from work and after initially neglecting to explaining what roleplay is, I wrote this:

Roleplaying is a form of collaborative storytelling — a group of people gathering to tell a story together. This broad definition covers quite a range of things — one can tell very different kinds of stories and collaborate in very different ways.

What I’m planning to run is called “tabletop roleplaying” [2]. The stories told centre around a group of characters (the protagonists in a movie). Each person playing is in charge of one of these main characters, except for one person who has no character and instead handles everything that isn’t one of the main characters (they are a bit like the director of a movie).

Tabletop roleplaying is a little like a radio drama — almost everything is done by narrating or speaking in character. You’ll be saying things you want your character to say and describing the actions you want your character to take. Light acting, such as putting on accents or changing tone of voice or changing posture, can be quite fun, but is by no means a requirement.

The “director”, also called the “storyteller” or “DM” [3], describes the situations the main characters find themselves in, decides on the consequences of their actions and takes on the role of minor supporting characters and antagonists (often villains, because they’re exciting). The storyteller is also an arbitrator and a facilitator and attempts to maintain consensus and suitable pacing of the story.

Often you’ll want to have your character attempt an action that might not succeed [4]. For example, they might want to shoot a villain, charm their way past a bouncer, pick a lock or run across a burning bridge. In some cases success will be more likely than others. A character who is good looking or persuasive might find charming the bouncer easy. A character who has never picked up a gun might find shooting the villain hard.

The set of rules used to determine success or failure is called “the system”. The rules might be as simple as “flip a coin” or they might take up a whole book [5]. Dice are a commonly used way of randomly determining results with high numbers typically indicating more successful outcomes and lower numbers less successful ones.

Since the real world is very very complicated, the rules usually model different aspects of it in varying degrees of detail and this often sets the tone of the story to some extent. For example, a system for telling stories about bank robbers in the 1920s might have very detailed rules on vault locks, while a system for telling fantasy stories will likely have special rules for elves and dwarves.

All systems have shortcomings, and when these are encountered it’s usually the storyteller’s job to apply common sense and tweak the outcome accordingly.

The system I’m planning to use is an extremely simplified version of Dungeons & Dragons, 3rd Edition. The full rules run to many books. I’m hoping to explain the few rules we’ll be using in 10-15 minutes.

The story I’m planning to run focuses on a down-on-their-luck rock band about to enter a battle of the bands contest. The twist is that it’s a fantasy setting so there are elves and dwarves and, of course, in the hands of suitably skilled musicians, music is literally magical.

Some practical considerations

Someone has to supply the table to sit around. This is the person hosting the game. Ke can be a player or the storyteller or even uninvolved [6]. Traditionally the host also supplies a kettle and tea or coffee.

Everyone needs to show up and sit at the table, preferably roughly at the same time. This is surprisingly hard.

In order to remain at the table for prolonged periods, one needs things to nibble on. Traditionally people who are not the host bring snacks and drinks of various kinds. Roleplayers seem to take a perverse delight in bringing the unhealthiest snacks they can find, but this is perhaps a tradition best improved on.

Remembering the details of the story can be tricky, so it’s often useful to scribble short notes for oneself. A pen and paper come in handy.

I’ll give each player a couple of pages of information about their character. These are called the “character sheet”. The primary role of the character sheet is to supply margins to scribble notes and doodles in (see above).

It’s likely that time will fly remarkably quickly. If there are six of us, each person will get on average less than ten minutes of “screen time” per hour and probably a lot less given that the storyteller usually uses more than their fair share and there are always distractions and side tracks like discussing the rules or office gossip [7]. If we run out of time, we can always continue the story another day if we’re excited enough.

Lastly, the point is to have fun and tell an interesting story [8].

Glossary

Host: Person who supplies the table, and usually warm beverages like tea and coffee.

Table: Thing one plays at.

Storyteller: The person managing the world the story takes place in, the consequences of players actions and playing the minor characters and antagonists.

Players: The people who are not the storyteller.

Player character: One of the protagonists of the story. Each player has their own player character to narrate.

NPC: Non-player character. All the characters in the story who are not player characters.

RPG: Roleplaying Game. Also rocket-propelled grenade.

System: The rules used to determine the outcomes of risky actions.

Dice: Things one rolls. Usually because one is required to do so by the rules, but often just for fun.

Fun: The point. :)

Footnotes

[1] The module was This is Vörpal Mace. If you’re keen to play it, you can download it from Locustforge.

[2] So called because it usually takes place around a table.

[3] “DM” stands for “Dungeon Master” and is a silly legacy term from the earliest tabletop roleplaying games which mostly focused on a group of heroes running around vast dungeons full of traps and monsters. The storyteller’s role was mostly to invent traps and monsters, hence the title.

[4] Because otherwise the story would be very boring. :)

[5] A whole book is far more common. :P

[6] Although letting six people invade your house for an evening for an activity you’re not involved in requires a special kind of friendship.

[7] One can avoid this time-divided-by-number-of-people limit by having multiple scenes running concurrently. This is a lot of fun, but hell on the storyteller. :)

[8] And it’s easy to lose track of this amongst all the details of playing your character, keeping track of what’s happening and figuring out the rules.

[9] This footnote is not related to anything.

by admin at 16 August 2015 11:06 PM

13 August 2015

Adrian Frith (htonl)

South African provinces as they might have been

The post-apartheid political map of South Africa might well have looked quite different. The Eastern Cape might have been divided into two provinces, with the Kat River and Great Fish River on the boundary. The Northern Cape might not have existed, with the Western Cape meeting North West at the Orange River. Gauteng might have been much bigger – or much smaller. The Western Cape might have stopped south of Citrusdal – or it might have incorporated all of Namaqualand.

13 August 2015 10:00 PM

07 June 2015

Tristan Seligmann (mithrandi)

Adventures in deployment with Propellor, Docker, and Fabric

Preface

After playing around with Docker a bit, I decided that it would make an ideal deployment platform for my work services (previously we were using some ad-hoc isolation using unix users and not much else). While Docker’s security is…suspect…compared to a complete virtualization solution (see Xen), I’m not so much worried about complete isolation between my services, as things like easy resource limits and imaging. You can build this yourself out of cgroups, chroot, etc. but in the end you’re just reinventing the Docker wheel, so I went with Docker instead.

However, Docker by itself is not a complete solution. You still need some way to configure the Docker host, and you also need to build Docker images, so I added Propellor (which I recently discovered) and Fabric to the mix.

Propellor

Propellor is a configuration management system (in the sense of Puppet, Chef, Salt, et al.) written in Haskell, where your configuration itself is Haskell code. For someone coming from a systems administration background, the flexibility and breadth offered by a real programming language like Haskell may be quite daunting, but as a programmer, I find it far more straightforward to just write code that does what I want, extracting common pieces into functions and so on. Our previous incarnation of things used Puppet for configuration management, but it always felt very awkward to work with; another problem is that Puppet was introduced after a bunch of the infrastructure was in place, meaning a lot of things were not actually managed by Puppet because somebody forgot. Propellor was used to configure a new server from scratch, ensuring that nothing was done ad-hoc, and while I won’t go into too much detail about Propellor, I am liking it a lot so far.

The role of Propellor in the new order is to configure things to provide the expected platform. This includes installing Docker, installing admin user accounts, SSH keys, groups, and so on.

Docker

The Docker workflow I adopted is based on the one described by Glyph. I would strongly recommend you go read his excellent post for the long explanation, but the short version is that instead of building a single container image, you instead build three: A “build” container used to produce the built artifacts from your sources (eg. Python wheels, Java/Clojure JARs), a “run” container which is built by installing the artifacts produced by running the “build” container, and thus does not need to contain your toolchain and -dev packages (keeping the size down), and a “base” container which contains the things shared by the “build” and “run” containers, allowing for even more efficiency of disk usage.

While I can’t show the Docker bits for our proprietary codebases, you can see the bits for one of our free software codebases, including instructions for building and running the images. The relative simplicity of the .docker files is no accident; rather than trying to shoehorn any complex build processes into the Docker image build, all of the heavy lifting is done by standard build and install tools (in the case of Documint: apt/dpkg, pip, and setuptools). Following this principal will save you a lot of pain and tears.

Fabric

The steps outlined for building the Docker images are relatively straightforward, but copy/pasting shell command lines from a README into a terminal is still not a great experience. In addition, our developers are typically working from internet connections where downloading multi-megabyte Docker images / packages / etc. is a somewhat tedious experience, and uploading the resulting images is ten times worse (literally ten times worse; my connection at home is 10M down / 1M up ADSL, for example). Rather than doing this locally, this should instead run on one of our servers which has much better connectivity and a stable / well-defined platform configuration (thanks to Propellor). So now the process would be “copy/paste shell command lines from a README into an ssh session” — no thanks. (For comparison, our current deployment processes use some ad-hoc shell scripts lying around on the relevant servers; a bit better than copy/pasting into an ssh session, but not by much.)

At this point, froztbyte reminded me of Fabric (which I knew about previously, but hadn’t thoughto f in this context). So instead I wrote some fairly simple Fabric tasks to automate the process of building new containers, and also deploying them. For final production use, I will probably be setting up a scheduled task that automatically deploys from our “prod” branch (much like our current workflow does), but for testing purposes, we want a deploy to happen whenever somebody merges something into our testing release branch (eventually I’d like to deploy test environments on demand for separate branches, but this poses some challenges which are outside of the scope of this blog post). I could build some automated deployment system triggered by webhooks from BitBucket (where our private source code is hosted), but since everyone with access to merge things into that branch also has direct SSH access to our servers, Fabric was the easiest solution; no need to add another pile of moving parts to the system.

My Fabric tasks look like this (censored slightly to remove hostnames):

@hosts('my.uat.host')
def build_uat_documint():
    with settings(warn_only=True):
        if run('test -d /srv/build/documint').failed:
            run('git clone --quiet -- https://github.com/fusionapp/documint.git /srv/build/documint')
    with cd('/srv/build/documint'):
        run('git pull --quiet')
        run('docker build --tag=fusionapp/documint-base --file=docker/base.docker .')
        run('docker build --tag=fusionapp/documint-build --file=docker/build.docker .')
        run('docker run --rm --tty --interactive --volume="/srv/build/documint:/application" --volume="/srv/build/documint/wheelhouse:/wheelhouse" fusionapp/documint-build')
        run('cp /srv/build/clj-neon/src/target/uberjar/clj-neon-*-standalone.jar bin/clj-neon.jar')
        run('docker build --tag=fusionapp/documint --file=docker/run.docker .')


@hosts('my.uat.host')
def deploy_uat_documint():
    with settings(warn_only=True):
        run('docker stop --time=30 documint')
        run('docker rm --volumes --force documint')
    run('docker run --detach --restart=always --name=documint --publish=8750:8750 fusionapp/documint')

Developers can now deploy a new version of Documint (for example) by simply running fab build_uat_documint deploy_uat_documint. Incidentally, the unit tests are run during the container build (from the .docker file), so deploying a busted code version by accident shouldn’t happen.

by mithrandi at 07 June 2015 10:57 PM

07 March 2015

Tristan Seligmann (mithrandi)

Axiom benchmark results on PyPy

EDIT: Updated version now available.

EDIT: Fixed the issue with the store-opening benchmark

Axiom conveniently includes a few microbenchmarks; I thought I’d use them to give an idea of the speed increase made possible by running Axiom on PyPy. In order to do this, however, I’m going to have to modify the benchmarks a little. To understand why this is necessary, one has to understand how PyPy achieves the speed it does: namely, through the use of JIT (Just-In-Time) compilation techniques. In short, these techniques mean that PyPy is compiling code during the execution of a program; it does this “just in time” to run the code (or actually, if I understand correctly, in some cases only after the code has been run). This means that when a PyPy program has just started up, there is a lot of performance overhead in the form of the time taken up by JIT compilation running, as well as time taken up by code being interpreted slowly because it has not yet been compiled. While this performance hit is quite significant for command-line tools and other short-lived programs, many applications making use of Axiom are long-lived server processes; for these, any startup overhead is mostly unimportant, the performance that interests us is the performance achieved once the startup cost has already been paid. The Axiom microbenchmarks mostly take the form of performing a certain operation N times, recording the time taken, then dividing that time by N to get an average time per single operation. I have made two modifications to the microbenchmarks in order to demonstrate the performance on PyPy; first, I have increased the value of “N”; second, I have modified the benchmarks to run the entire benchmark twice, throwing away the results from the first run and only reporting the second run. This serves to exclude startup/”warmup” costs from the benchmark.

All of the results below are from my desktop machine running Debian unstable on amd64, CPython 2.7.5, and PyPy 2.1.0 on a Core i7-2600K running at 3.40GHz. I tried to keep the system mostly quiet during benchmarking, but I did have a web browser and other typical desktop applications running at the same time. Here’s a graph of the results; see the rest of the post for the details, especially regarding the store-opening benchmark (which is actually slower on PyPy).

[graph removed, see the new post instead]

To get an example of how much of a difference this makes, let’s take a look at the first benchmark I’m going to run, item-creation 15. This benchmark constructs an Item type with 15 integer attributes, then runs 10 transactions where each transaction creates 1000 items of that type. In its initial form, the results look like this:

mithrandi@lorien> python item-creation 15
0.000164939785004
mithrandi@lorien> pypy item-creation 15
0.000301389718056

That’s about 165µs per item creation on CPython, and 301µs on PyPy, nearly 83% slower; not exactly what we were hoping for. If I increase the length of the outer loop (number of transactions) from 10 to 1000, and introduce the double benchmark run, the results look a lot more encouraging:

mithrandi@lorien> python item-creation 15
0.000159110188484
mithrandi@lorien> pypy item-creation 15
8.7410929203e-05

That’s about 159µs per item creation on CPython, and only 87µs on PyPy; that’s a 45% speed increase. The PyPy speed-up is welcome, but it’s also interesting to note that CPython benefits slightly from the changes to the benchmark. I don’t have any immediate explanation for why this might be, but the difference is only about 3%, so it doesn’t matter too much.

The second benchmark is inmemory-setting. This benchmark constructs 10,000 items with 5 inmemory attributes (actually, the number of attributes is hardcoded, due to a limitation in the benchmark code), and then times how long it takes to set all 5 attributes to new values on each of the 10,000 items. I decreased the number of items to 1000, wrapped a loop around the attribute setting to repeat it 1000 times, and introduced the double benchmark run:

mithrandi@lorien> python inmemory-setting
4.86490821838e-07
mithrandi@lorien> pypy inmemory-setting
1.28742599487e-07

That’s 486ns to set an attribute on CPython, and 129ns on PyPy, for a 74% speed increase. Note that this benchmark is extremely sensitive to small fluctuations since the operation being measured is such a fast one, so the results can vary a fair amount between benchmarks run. For interest’s sake, I repeated the benchmark except with a normal Python class substituted for Item, in order to compare the overhead of setting an inmemory attribute as compared with normal Python attribute access. The result was 61ns to set an attribute on CPython (making an inmemory attribute about 700% slower), and 2ns on PyPy (inmemory is 5700% slower). The speed difference on PyPy is more to do with how fast setting a normal attribute is on PyPy, than to do with Axiom being slow.

The third benchmark is integer-setting. This benchmark is similar to inmemory-setting except that it uses integer attributes instead of inmemory attributes. I performed the same modifications, except with an outer loop of 100 iterations:

mithrandi@lorien> python integer-setting
1.23480038643e-05
mithrandi@lorien> pypy integer-setting
3.80326986313e-06

That’s 12.3µs to set an attribute on CPython, and 3.8µs on PyPy, a 69% speed increase.

The fourth benchmark is item-loading 15. This benchmark creates 10,000 items with 15 integer attributes each, then times how long it takes to load an item from the database. On CPython, the items are deallocated and removed from the item cache immediately thanks to refcounting, but on PyPy a gc.collect() after creating the items is necessary to force them to be garbage collected. In addition, I increased the number of items to 100,000 and introduced the double benchmark run:

mithrandi@lorien> python item-loading 15
9.09668397903e-05
mithrandi@lorien> pypy item-loading 15
5.70205903053e-05

That’s 90µs to load an item on CPython, and 57µs on PyPy, for a modest 37% speed increase.

The fifth benchmark is multiquery-creation 5 15. This benchmark constructs (but does not run) an Axiom query involving 5 different types, each with 15 attributes (such a query requires Axiom to construct SQL that mentions each item table, and each column in those tables) 10,000 times. I increased the number of queries constructed to 100,000 and introduced the double benchmark run:

mithrandi@lorien> python multiquery-creation 5 15
5.5426299572e-05
mithrandi@lorien> pypy multiquery-creation 5 15
7.98981904984e-06

55µs to construct a query on CPython; 8µs on PyPy; 86% speed increase.

The sixth benchmark is query-creation 15. This benchmark is the same as multiquery-creation, except for queries involving only a single item type. I increased the number of queries constructed to 1,000,000 and introduced the double benchmark run:

mithrandi@lorien> python query-creation 15
1.548528409e-05
mithrandi@lorien> pypy query-creation 15
1.56546807289e-06

15.5µs to construct a query on CPython; 1.6µs on PyPy; 90% speed increase.

The final benchmark is store-opening 20 15. This benchmark simply times how long it takes to open a store containing 20 different item types, each with 15 attributes (opening a store requires Axiom to load the schema from the database, among other things). I increased the number of iterations from 100 to 10,000; due to a bug in Axiom, the benchmark will run out of file descriptors partway, so I had to work around this. I also introduced the double benchmark run:

mithrandi@lorien> python store-opening 20 15
0.00140788140297
mithrandi@lorien> pypy store-opening 20 15
0.00202187280655

1.41ms to open a store on CPython; 2.02ms on PyPy; 44% slowdown. I’m not sure what the cause of the slowdown is.

A bzr branch containing all of my modifications is available at lp:~mithrandi/divmod.org/pypy-benchmarking.

by mithrandi at 07 March 2015 11:24 AM

Axiom benchmark results on PyPy 2.5.0

This is a followup to a post I made about 1.5 years ago, benchmarking Axiom on PyPy 2.1.0. Not too much has changed in Axiom since then (we fixed two nasty bugs that mainly affected PyPy, but I don’t expect those changes to have had much impact on performance), but PyPy (now at 2.5.0) has had plenty of work done on it since then, so let’s see what that means for Axiom performance!

Unlike my previous post, I’m basically just going to show the results here without much commentary:

Graph of Axiom performance

A few notes:

  • I didn’t redo the old benchmark results, but the hardware/software I ran the benchmarks on is not significantly different, so I think the results are still valid as far as broad comparisons go (as you can see, the CPython results match fairly closely).
  • The benchmark harness I’m using now is improved over the last time, using some statistical techniques to determine how long to run the benchmark, rather than relying on some hardcoded values to achieve JIT warmup and performance stability. Still could use some work (eg. outputting a kernel density estimate / error bars, rather than just a single mean time value).
  • There is one new benchmark relative to the last time, powerup-loading; PyPy really shines here, cutting out a ton of overhead. There’s still room for a few more benchmarks of critical functions such as actually running and loading query results (as opposed to just constructing query objects).
  • The branch I used to run these benchmarks is available on Github.
  • The horizontal axis is cut off at 1.0 so you can’t actually see how store-opening lines up, but the raw data shows that PyPy 2.1.0 took about 53% longer on this benchmark, whil PyPy 2.5.0 only takes about 2% longer.

by mithrandi at 07 March 2015 11:22 AM

28 February 2015

Andre Truter (Cacofonix)

Old software in business

One thing that I cannot understand is why a business that is not a one-man-show would use outdated software, especially something like Internet Explorer 6 or 7 or anything under 10.
I can understand if a very small business of private person who do not have an IT department of IT support can still be stuck with such old software, but a big business with branches all over the country should know that especially IE < 10 has security risks and do not support HTML5.

So if you then ask a supplier to write you a web application and the web application makes use of HTML5, then you should not wonder why it does not work with your IE 8 browser.

I can understand that it might not always be possible to upgrade all workstations to the latest operating system if you use expensive proprietary operating systems, but then you can at least standardise on an open source browser like Firefox or a free browser like Chrome. Both of them have very good track records for security and they support HTML5 and keeping them up to date does not cost anything.

So why are your staff stuck with an old, insecure browser that does not support HTML5? We are not living the the 90's anymore!

The same goes for office suites. LibreOffice is kept up to date and there are other alternatives like OpenOffice (Backed by Apache, Oracle and IBM). With a little training, you can move all your users over to LibreOffice or OpenOffice and never have to pay for an upgrade again and always have the latest stable and secure version of the software available, no matter what OS you run.

To me it just makes sense to invest some time and money once to get ensure a future of not being locked in. In the long run it saves money and if enough businesses does that, it might even force Microsoft to come to the Open Document Standards table and bring the price of it's Office suite down or even make it also free, which will benefit everybody as we will have a real open document format that everybody can use and nobody can be locked in.

Just my humble opinion.

Some links:
Firefox Web Browser
Google Chrome Web Browser
Best Microsoft Office Alternatives

by andre at 28 February 2015 03:35 PM

03 December 2014

Neil Blakey-Milner (nbm)

Starting is one of the hardest things to do

Just over six years ago I stopped posting regularly to my fairly long-lived blog (apparently started in April 2003), and just under four years ago I noticed it was not loading because the server it was on had broken somehow, and I didn't care enough to put it back up. (Astute readers will note how those two dates are roughly a few months in to my last two job start dates...)

I've written a number of blog posts over the last few years, but I haven't posted any of them. I don't even have most of them anymore, although a few are in the branches of git repos somewhere. I'll try dig some of them up - one of the things I enjoyed about my old blog was rereading what I was thinking about at various times in the past. (I guess I'll try get that back online.)

I've also written two different bits of blog software since then - since that's what one does whenever one contemplates starting a blog up again (although, to be fair, I also started setting up Wordpress and Tumblr blogs too).

The first used Hyde, a static blog generation tool (whose development seems to have halted a year ago). The second is what I'm using now - a collection of Flask modules and some glue code that constitutes gibe2, which I wrote just under a year ago. It uses Frozen-Flask to freeze a Flask app into an HTML tree that can be served by any static web server.

Putting something out there - let alone a first pass, a beginning with some implied commitment to continue - is quite scary. In my career, I've tended to be the joiner and sometimes the completer to a project, not the initiator. I find it easy to take something from where it is to where I think it should go, and hard to imagine what should exist.

I should mention that I have succeeded in posting a bit to play.TechGeneral.org about the occasional spurts of time I get to work on a little D language OpenGL someday-might-be-a-game. The minimum expended effort there is a lot higher - building a new feature or doing a major refactor, and then explaining it all in a post - in fact the posts are sometimes a lot harder to put together than the code, and the code is quite a bit further ahead than the posts now. It will probably be very sporadic with posts mostly coinciding with my long weekends and vacations. (And, unsurprisingly, I've just finished up a vacation where I finally got around to putting the final touches on this here blog.)

Connecting together starting up being scary and game programming: I've been watching Handmade Hero religiously since it started (the first few on Youtube the day after, the last few live). Talk about being a tough thing to contemplate starting - one hour a day of writing a game from scratch streaming to the public every weekday - every single little mistake in your code and in your explanations there for people to pick apart without any opportunity for editing. And having ~1000 streamers in your first few shows - no pressure!

That definitely put just putting together a few words a week with no implied time-bound commitment into perspective, so here we are.

I stopped reading blogs roughly when I stopped caring about mine. I hope to find a few interesting people to follow. A few of my friends used to post, albeit infrequently. I hope I can convince them to do so again as well.

by Neil Blakey-Milner at 03 December 2014 07:45 AM

22 August 2014

Adrianna Pińska (Confluence)

Yet another way to play videos in an external player from Firefox

I spent some time today trying to figure out how to get Firefox to play embedded videos using anything other than Flash, which is an awful, stuttering, resource-devouring mess. The internet is littered with old browser extensions and user scripts which allegedly make this possible (e.g. by forcing sites like YouTube to use the vlc media plugin instead), but I was unable to get any of them to work correctly.

Here’s a quick hack for playing videos from YouTube and any other site that youtube-dl can handle in an external mplayer window. It’s based on several existing HOWTOs, and has the benefit of utilising a browser extension which isn’t dead yet, Open With, which was designed for opening websites in another browser from inside Firefox.

I wrote a little script which uses youtube-dl to download the video and write it to stdout, and pipes this output to mplayer, which plays it. Open With is the glue which sends URLs to the script. You can configure the extension to add this option to various context menus in the browser — for example, I can see it if I right-click on an URL or on an open page. You may find this less convenient than overriding the behaviour of the embedded video on the page, but I prefer to play my videos full-screen anyway.

This is the script:

#!/bin/sh
youtube-dl -o - $1 | mplayer -

Make it executable. Now open Open With’s preferences, add this executable, and give it a nicer name if you like. Enjoy your stutter-free videos. :)

(Obviously you need to have youtube-dl and mplayer installed in order for this to work. You can adapt this to other media players — just check what you need to do to get them to read from stdin.)

by confluence at 22 August 2014 02:41 PM

21 August 2014

Simon Cross (Hodgestar)

Character Creation 3000W

by Simon Cross, Mike Dewar and Adrianna Pińska

Your character creation skills have progressed far beyond writing
numbers on paper. Your characters have deftly crafted manerisms and
epic length backgrounds. They breathe emotion and seem more life-like
than many of your friends.

Yet, somehow, when you sit down at a table to play your beautiful
creations, things don’t quite work out.

Perhaps the story heads in an unexpected direction, leaving your
creation out of place and struggling to fit in? Or maybe they’re fun
to play initially but their actions begin to feel repetitive and
uninteresting?

If any of this sounds familiar, read on.

Reacting to failure

It’s easy to spend all your time imagining a character’s successes —

their victories and their crowning moments — but what happens when
they fail? How do they respond to minor setbacks? And big ones?

Maybe they’re stoic about it? Perhaps it’s likely to cause a crisis of
faith? Maybe they react by doubling down and uping the stakes? Maybe
they see failure as an opportunity to learn and grow? Perhaps they’re
accustomed to failure? Perhaps they see failure as a sign that they’re
challenging themselves and pushing their abilities?

The dice and the DM are going to screw you. Make sure you have a plan
for how to roleplay your character when they do.

Philosophy

A character’s goals are things strongly tied to specific events. A
philosophy colours every situation. The two are often aligned, but a
philosophy is more broadly useful. It gives you a handle on how your
character might behave in circumstances where it is not otherwise
obvious what they would do.

To take a hackneyed example: your backstory might involve punishing an
old partner who screwed you. This goal could feed a number of
rather different philosophies:

  • “I always keep my word, and I promised Jimmy I’d get him back.”
  • “Any situation can be solved with enough violence.”
  • “Karma controls the universe. What goes around comes around.”

The goal is the same, but each philosophy implies very different
day-to-day behaviour.

There are going to be times when other characters’ plots and goals are
centre-stage, and it behooves us as roleplayers to have a plan for
these awkward (and hopefully brief) moments. A philosophy allows your
character to participate in others’ plots as a unique and distinct
individual, rather than as a bored bystander.

Your character’s philosophy becomes vitally important when paradigm
shifts occur in-game. Setting changes erode the importance of lesser
goals and past history and create a strong need for a philosophy that
guides your character’s immediate responses and further development.

It may be interesting to construct characters with goals that
contradict their philosophy. For example, a pacifist might wish to
exact revenge on the person who killed their brother. This creates an
interesting conflict that will need to be resolved.

Randomly fucking with people is not a philosophy.

Interacting with colleagues

Your character is going to spend a lot of time interacting with their

colleagues — the other player characters — so it’s worthwhile
thinking about how they do that.

It’s tempting (and a bit lazy) to think of characters as relating to
everyone else the same way. This leads to loners and overly friendly
Energizer bunnies, both of which get old very quickly.

Avoid homogenous party dynamics.

If your character’s interactions with the other player characters are
all the same, you have failed.

Varied interactions also help make party disagreements more
interesting. Without varied interactions, you have to resolve all
disagreements by beating each other over the head with the logic stick
until consensus (or boredom) is reached. Unique relationships and
loose factions make disagreements more interesting to roleplay and
help the party find plausible lines along which to unite for a given
scenario.

If your character is part of a command structure, spend some time
thinking about how they respond to orders they disagree with. Remember
that the orders are likely issued by someone your character knows and
has an existing relationship with. What is that relationship?

Also keep in mind that your character has likely been given such
orders before, and since they appear to still be part of the command
structure, they’ve probably come to terms with this in some way that
both they and their immediate superiors can live with.

Obviously everyone has their limits, though — where are your
character’s? How much does it take for other player characters or
NPCs to cross the line?

Development

Sometimes even if you do everything right you find yourself in a
situation where your character is no longer fun to play. Maybe the
campaign took an unexpected turn or you’ve just run out of ideas for
them as they are. It’s time for your character to change — to embark
on a new personal story arc.

Great characters aren’t static. They grow and react to events around
them. Perhaps a crushing defeat has made them re-consider their
philosophy — or made them more committed to it? Or maybe frustration
with their current situation has made them reconsider their options?

It helps to think broadly about how your character might develop while
you’re creating them. Make sure you’d still find the character
interesting to play even if their stance on some important issues
shifted. Don’t become too invested in your character remaining as they
are. Be flexible — don’t have only one plan for character
development.

Your character’s philosophy and general outlook can be one of the most
interesting things to tweak. Small changes can often have big
ramifications for how they interact with others.

Don’t feel you have to leave character development for later in the
campaign! The start of a campaign is often when character changes are
most needed to make a character work well and it sets the stage for
further character development later on.

Sharing

Think about how you convey who your character is to the other
players. They’re probably not going to get to read your epic
backstory, so they’re going to have to learn about who your character
is in other ways.

Likely the first thing people will hear about your character is his or
her name — so make it a good one. It’s going to be repeated a lot so
make sure it conveys something about who your character is. If they’re
an Italian mobster, make sure their name sounds like they’re an
Italian mobster. That way whenever the DM or another player says your
character’s name, it reminds everyone who your character is.

The second thing people hear will probably be a description of your
character. Take some time to write one. Don’t rely on dry statistics
and descriptions. Stick to what people would see and remember about
your character if they met him or her for a few minutes. Don’t mention
hair colour unless hair is an important feature.

After introductions are done, you probably won’t get another
invitation to monologue about your character. So do it in character
instead. Tell the NPC about that time in ‘Nam. Regale the party with
tales from your epic backstory. As in real life, try not to ramble on,
but equally, don’t shy away from putting your character in the
spotlight for a few moments. Continually remind the others at the
table who your character is.

Last but not least, remember that the most epic backstory is pointless
if no one finds out about it. The point of dark secrets is for them to
be uncovered and for your character to confront them.

Epilogue

Don’t fear failure. Have a philosophy. Have varied interactions with
others. Embrace change. Share who you are.

Thanks

  • Kululaa dot COMMMM!
  • Mefridus von Utrecht (for a philosophy that involves others)
  • Attelat Vool (for starting life after failure)

This article was also published in the CLAWmarks 2014 Dragonfire edition.

by admin at 21 August 2014 09:36 AM

17 March 2014

Adrianna Pińska (Confluence)

Why phishing works

Let me tell you about my bank.

I would expect an institution which I trust to look after my money to be among the most competent bureaucratic organisations I deal with. Sadly, interactions which I and my partner H have had with our bank in recent years suggest the opposite.

Some of these incidents were comparatively minor: I once had to have a new debit card issued three times, because I made the fatal error of asking if I could pick it up from a different branch (the first two times I got the card, my home branch cancelled it as soon as I used it). More recently, when I got a replacement credit card, every person I dealt with had a completely different idea of what documents I needed to provide in order to collect it. When H bought a used car, it turned out that it was literally impossible for him to pay for it with a transfer and have the payment clear immediately — after he was assured by several employees that it would not be a problem.

Some incidents were less minor. H was once notified that he was being charged for a replacement credit card when his current card was years away from expiring. Suspicious, he called the bank — the employee he spoke to agreed that it was weird, had no idea what had happened, and said they would cancel the new card. H specifically asked if there was anything wrong with his old card, and was assured that everything was fine and he could keep using it. Of course everything was not fine — it suddenly stopped working in the middle of the holiday season, and he had to scramble to replace it at short notice. All he got was a vague explanation that the card had to be replaced “for security reasons”. From hints dropped by various banking employees he got the impression that a whole batch of cards had been compromised and had quietly been replaced — at customers’ expense, of course.

What happened last weekend really takes the cake. The evening before we were due to leave on a five-day trip, well after bank working hours, H received an SMS telling him that his accounts had been frozen because he had failed to provide the bank with some document required for FICA. This was both alarming and inexplicable, because we had both submitted documents for FICA well before the deadline years ago. The accounts seemed fine when he checked them online. When he contacted the bank, he was assured that he had been sent the SMS in error, everything was fine, and he didn’t need to provide any FICA documents.

So we left on our trip. I’m sure you can see where this is going.

On Thursday evening H’s accounts were, in fact, frozen. He tried unsuccessfully during the trip to get the bank to unfreeze them, but since it was closed for the weekend there was pretty much nothing he could do until we got back home.

Hilariously, although employees from the bank made a house call this morning to re-FICA him, he had to go to the branch anyway because they needed a photocopy of his ID (scanning and printing is magically not the same as photocopying).

Again, what actually happened is a mystery — the bank claims that they have no FICA documents on record for H (as an aside, why are customers not given receipts when they submit these documents to the bank? If there is no record of the transaction, the bank can shift blame to the customer with impunity if it loses vital documentation).

We’re very fortunate that none of these incidents had devastating consequences for us, since they only impacted one of us at a time. If we relied on a single bank account, we could easily have ended up without food, without electricity, or stranded in the middle of the Karoo with no fuel. This is pretty clearly not an OK situation.

The common thread running through all these incidents is a lack of communication: both between the bank and its customers, and within the bank. We rapidly discovered through our dealings that while the bank maintains the facade of a single, unified entity, it in fact comprises several more-or-less autonomous divisions, and communication between these divisions often resembles diplomacy in the Balkans.

I would not be surprised to discover that various aspects of customers’ information are distributed over several disconnected databases. There appears to be no way for certain types of information to be linked to accounts, which necessitates the use of bizarre hacks. When H’s credit card was disabled, this was in no way reflected in the online interface — he just had an enormous amount of money reserved on the card. When his accounts were frozen, this was again not apparent in the interface (which was recently updated) — his available credit balance was just zeroed, and he got a non-specific error whenever he tried to transfer funds. I believe that the back-end infrastructure for managing this information effectively and making it available to customers and employees simply does not exist.

As a result of this, I have learned not to believe a word that a bank employee says if there is any chance that the issue crosses some internal jurisdictional boundary. In the best case scenario they are aware of their own lack of information and are able to direct you to the appropriate department. In the worst case scenario, they dispense outright misinformation — something which can have devastating consequences (for you).

We live in an age with an abundance of instant communications methods. In spite of this, it appears to be beyond the bank’s abilities to inform people timeously about what is going on with their accounts. It has backed off from electronic communications presumably because of fears of phishing, and has fallen back to two obsolete and inefficient channels: voice phonecalls, which are prone to misunderstandings and human error and leave the customer with no written record, and SMSes, which have a character limit. Both these channels are vulnerable to blips in the cellphone network infrastructure, customers having no airtime, or customers just not being available to answer their phones at certain times (since voice phonecalls are synchronous). The bank also uses these channels as if it believes that they are intrinsically secure and trustworthy, which is ludicrous, as anyone who keeps getting calls from “the Microsoft Support Centre” can attest.

In order for this dysfunctional system to work, the bank appears to expect its customers to accept unexpected additional charges unquestioningly, and to follow instructions issued by unknown persons calling from unverified internal numbers, even if they are nonsensical or suspicious and cannot be corroborated by other bank employees contacted through more reputable public-facing channels. It is standard procedure for such callers to demand personal information from customers in order to verify their identities, but they offer no evidence of their own legitimacy.

In short, the bank expects us to be the kind of credulous chumps who fall prey to phishing scams. It’s easy to see why phishing works when genuine communications from the bank are so unprofessional and follow such laughable security practices that they are virtually indistinguishable from phishing.

I haven’t named my bank, although it’s pretty easy to find out what it is if you know where to look, because I’m sceptical that there are significant differences between the way South African banks operate; particularly the Big Four. I’ve certainly heard the same kinds of horror stories from customers of other banks.

This is the final straw which has led us to investigate other banking options. Whether the pastures we’re moving to are greener or just differently green remains to be seen.

If you feel strongly that there is a bank which is notably better or worse than the others, or you just want to share your own tale of banking woe, let me know in the comments. I’ll be over here, stuffing my money into a sock to hide under my mattress.

by confluence at 17 March 2014 12:10 PM

25 November 2013

Michael Gorven (cocooncrash)

Statically linked e2fsprogs binaries for Android

I needed to resize the ext4 filesystem of the /data partition on my Android phone and spent ages looking for a prebuilt binary of resize2fs (since it isn't included in ROMs or recovery). I didn't find one and so spent ages trying to build it myself. To save others the trouble, here are the binaries.

Just the main ones (mke2fs, resize2fs, tune2fs, fsck): tar.xz 630K or zip 1.5M
Everything tar.xz 1.6M or zip 5.8M

by mgorven at 25 November 2013 04:58 AM

17 October 2013

Simeon Miteff (simeon)

DBE is right about Delphi and Office

Some thoughts on the current furore about the Department of Basic Education’s policy to standardise on MS Office and Delphi in teaching:

MS Office

Word processors and spreadsheets are simple tools for solving simple problems. For hard typesetting and number crunching problems we have better tools.

My point is that we’re already making a pragmatic choice with computer literacy training in schools: most scholars will use computers to solve simple problems, so schools train them to be able to do that (not to build mathematical models in Octave or to typeset engineering manuals in LaTeX). Should we now insist that they be trained to use a relatively obscure FOSS office suite instead? LibreOffice is a fork of one piece of crap, emulating another piece of crap known as MS Office. DBE is standardising, on the de-facto (albeit, piece of crap) standard, and that is the sensible choice, get over it.

Delphi

I did computer programming in Pascal (with Borland Turbo Pascal) at high school. The curriculum covered the basic constructs of the imperative programming style. At the time, Borland had already enhanced their Pascal implementation to support much of the language features currently available in Delphi. When I started university, the Computer Science program I took required basic programming ability as a prerequisite, and having done Pascal at school was sufficient to fulfil this requirement. Students who didn’t have programming at school took an additional semester course (Introduction to programming in C).

I never touched Pascal again.

Not only was the Pascal I was taught at school not an immediately useful skill for the workplace, but in retrospect, the abstract concepts I was taught served only as a grounding for getting into real computer science which followed at university: data structures and algorithms, computational complexity, computer architecture, AI, compilers and the study of programming languages, distributed systems, and so on. I wouldn’t have become a capable programmer without studying mathematics and science (as apposed to being trained in programming).

In my opinion, Pascal (and consequently Delphi) is good enough for education. Stated another way: Java is no better for the purpose of teaching a teenager how a for loop works. No programming language taught at school can turn a scholar into serious career programmer.

Real problems

There are so many difficult problems to solve in South Africa’s education system: consider that the majority of scholars are being taught all subjects in their second, or third (natural) language. Why aren’t we having a debate about whether Afrikaans should be taught as a second language to kids who speak Xhosa at home, when they’re expected to understand what’s going on in a history lesson delivered in English?

Conclusion

On this particular issue, y’all need to calm down and give DBE a break.

by Simeon Miteff at 17 October 2013 10:27 PM

08 September 2013

Andy Rabagliati (wizzy)

Richard Stallman visits Cape Town

Myself with Richard Stallman Richard Stallman visited Cape Town, and gave a talk at the University of Cape Town titled "A Free Digital Society"

On the 4th September I picked Richard Stallman up from the airport and we went to dinner with a number of people that had helped organise the event at Brass Bell, Kalk Bay.

Around Cape Town

He stayed with me and my friend Lerato in Muizenberg, and he is a gracious and accommodating guest. I took the Thursday off work, and we had lunch at a little local restaurant in Marina da Gama, and I then took him for a drive around the peninsula on a beautiful Cape day.

A Free Digital Society

The talk was at the Leslie Social Sciences building at University of Cape Town, but as we arrived Richard realised he had left behind a Gnu fluffy toy he wanted to auction, so I had to dash back to Muizenberg to pick it up.

Gnu Public Licence

The other organisers delayed the start of the talk by ten minutes so I could do the introductions, thank you. He talked for about two hours on a range of topics, dwelling on the GNU Public Licence, his 30 year old prescription for how developers should set about writing code that will become a living part of a commonly-owned set of tools and procedures that other people can build upon and improve.

Tracking

He outlined his suspicions regarding the trackability of cellphones, and the vast amounts of information that is stored by cellphone companies - mostly metadata (number called, duration, cellphone location) and the implications for privacy. He does not own one.

Education

He touched on software used in schools, exhorting educators to think about the larger consequences of software usage, and putting students on the right track - to empower them to realise they are in control of the software, and not just treat it as a purchased item that remains the property of the software company.

Voting Software

He highlighted the many dangers and pitfalls of computerising the counting of votes, from the closed nature of the software, the possibility of bugs or deliberate malware in the code, the difficulty of knowing if the software that was verified is indeed the software that is running on the machine for the duration of the vote, and the difficulty of a recount, or even what that means.

E-Tolling

Bringing a South African flavour to his talk, he discussed eTolling - which he objected to because of the potential for abuse for surveillance purposes. He also pointed out that if proper attention was paid to anonymising the data, he would no longer object.

Lerato and Richard

Farewell

He let me keep a South African music CD in his collection, which was very nice of him, and I like it very much. In short, we had a capacity audience from diverse backgrounds, no glitches, and he very much enjoyed his trip, saying he would look forward to coming again. We are certainly blessed to live in such a beautiful part of the world, that people like Richard want to visit.

Other commentary

A couple of other posts :-

by Andy at 08 September 2013 02:32 PM