RAW: A Data Visualisation Application for Non Programmers

air-bender

I created the above SVG using RAW which shows the steady decline in the popularity of M. Night Shyamalan’s filmography. It really was just a cut and paste of the data from the web, to excel and into RAW. RAW creates vector graphics (i.e. SVG) from spreadsheets (i.e. TSV). The app should have been called TSVG. I’ll make some calls.

It helps if you have used some of the graph types before to know how they work best with the source data. Maybe it will spur some people on to get to grips with D3 more.

Visualising a DDoS attack

The makers of the very awesome VLC media player were recently the subject of a DDoS attack. They made a video that visualised the attack using a program called Logstalgia. Its very pretty to watch.

It would be interesting to correlate this data with a location map to see just how widely distributed the attack was, similar to this one I built for the Security Presentation.

Census Data

Building the Census Data Explorer required that I have the South African Census data in a format that worked easily with D3, i.e. JSON. I therefore had to get the data from the Statistics South Africa website and convert it into JSON format. I’m making it available here for you to use. Its not complete, just the data I needed for this project.

South African Census Data in JSON format

South African Number Formats

A friend of mine recently asked for help with Excel number formatting. His version of Excel was using a comma for the decimal point and it was irritating him as it seemed wrong. I then saw the use of spaces for the thousands separator and figured it was using the wrong region settings. It turned out that the region settings were correct and we had both been subjected to the Americanisation of a standard. I found that other people had experienced a similar problem. Technically speaking, the official format for numbers in South Africa is spaces for the thousands separator and a comma for the decimal mark.

1 234 567,89

The American variation that we had become used to is commas for the thousands separator and a full stop for the decimal point.

1,234,567.89

In programming, I have only ever seen the use of the full stop for the decimal mark, and no thousands separator. However when printing numbers for humans to read, its important to use the thousands separator and to use a number format that the largest group of readers will easily understand.

If your readers are South African, this handy JavaScript function will format your numbers correctly with 2 decimal places, which is mostly useful for displaying currency values. This code is a modified version of this script.

function za_format(number) {
  number = parseInt(number).toFixed(2);
  number += '';
  var x = number.split('.');
  var x1 = x[0];
  var x2 = x.length > 1 ? ',' + x[1] : '';
  var rgx = /(\d+)(\d{3})/;
  while (rgx.test(x1)) {
    x1 = x1.replace(rgx, '$1' + ' ' + '$2');
  }
  return x1 + x2;
}

Rating our Presidents

As any person who reads the news in South Africa, I’m prone to the problem of having a two week memory of news that made headlines. But our brains fool us into thinking that we remember more of the past and are better able to predict the future. At the time of writing this (17 Feb 2013), the following news items about Jacob Zuma were published:

http://www.iol.co.za/news/politics/nothing-inspiring-in-zuma-s-speech-glaser-1.1470658#.USDan6USZ8E

http://www.thenewage.co.za/83097-1007-53-DA_wants_clarity_from_President_Jacob_Zuma_on_State_land_audit

http://www.bdlive.co.za/economy/2013/02/15/news-analysis-national-plans-backers-still-await-details-from-zuma

As with all quick news reports, it tends to focus on a particular issue that will be newsworthy and helps confirm people’s biases, whichever way they may lean. You seldom get a clear picture of the track record of a president, only a snapshot of current sentiment. It seems to me, the job of a president that wants to stay is power is to first manage public opinion, and then to be in charge of running the country. How do you tell if they are doing a good job on the second part? At this point, I read a lot of negative sentiment around Jacob Zuma. The usual “the country is falling apart” rhetoric mixed with cultural biases around his behaviour that is contrary to what is deemed acceptable.. Also there are our biases towards viewing short term change in economic indicators as representative of a longer term trend. When the Rand weakens against the dollar or inflation goes up, without looking at a much longer timeline, these data point blips shouldn’t matter. Any short term negative change is usually highlighted in the news and tends to cloud the view of actual job performance.

Continue reading “Rating our Presidents”

Monitoring System Dashboard

three6five probe dashboard

Another three6five related project was the requirement to create a dashboard for our support team to have an overview of all the probes that we have deployed on site at our customers. Specifically it needed to firstly highlight any probes that are not functioning, and then any services on an active probe that are not functioning.

This will live on a big TV screen in our support centre and can be seen from anywhere in the office. While we send out notifications, its good to have a easy to view list of all the issues that are being dealt with. Continue reading “Monitoring System Dashboard”