tomcarnell.com

Sunday, August 31, 2014

Google driverless car - Who gets to die?

I was just thinking about the driverless Google Car (actually I was thinking how Apple's lack of interest in this area is probably a terrible mistake). Then I started to think that it is actually very interesting that the computer 'driver' could be tested within a software 'sandbox/test harness' environment over thousands of scenarios - all that is required is an extremely accurate computer model of a road (different surfaces, weather etc).

BUT then I started to think about something far more thought provoking - lets give an example: You are in the Google Car enjoying a coffee on your way to work but suddenly up ahead a lorry tyre blows-out and begins to loose control, it swerves, looses balances and tips over, sliding along the tarmac. As Google knows everything, it knows that the occupants of car driving next to you are a single-mother with 2 children in the back. Now, in sight of this knowledge, does your Google Car act to save your life by swerving to avoid the lorry (at the calculated possible risk of the loss of live of the mother and children), or does your own car act for 'the greater good' by putting your single life on the line to improve the chances of survival of the mother and children?

There are many ways a driver can react in an emergency situation, but a human driver is generally unlikely to react in the most logical or calculated way. I remember a long distance coach driver telling me that his 'mandate' was absolutely to the safety of his passengers - if he were to round a bend and find a car stranded in the middle of the road, however awful it might sound, his training would tell him to drive straight through the car rather than attempt to swerve round it which could potentially result in a far higher loss of life if the coach were to loose control.

So what 'mandate' does the Google Car have? And who gets to choose how your car will react in an emergency situation? Might there even be a dial on the dashboard so YOU can choose where the car's priorities lie?

Imagine the first time it happens - that a Google Car is involved with in an accident and CCTV reply footage shows the Google Car swerving to save its single occupant as it careers into a car containing young children... Hmm, perhaps the Google PR team would have preferred that the single male adult was compromised and that the Google Car acted to save young children? So perhaps you wont get the choice after all...

...and a cynical person may even suppose that the Google Car acting in the greater good by saving the most number of human lives would also conveniently fit with their business model.

But really, it would be fascinating to know exactly how the Google Car would react in these situations and exactly who has decided 'who gets to die?'.

Saturday, June 22, 2013

Horizon: Fracking - the New Energy Rush, BBC Two - Propaganda

I have just watched the Horizon documentation "Fracking - the New Energy Rush". I was looking forward to a neutral and informative documentary about 'fracking' - the process of extractive gas trapped in deep layers of highly compressed rock.

However, the documentary was at best a farse and at worse, suspiciously close to a propaganda video. If I was a Governement or energy company wanting to convince the British public that fracking was a magic new source of clean energy, I would have probably made exactly the same video, specifically:


  1. The use of a geologist as a narrator to make everything he said more credible, honest and less politically motivated.
  2. Scare the British audience by stating that Britain is dependant on energy from terrible rogue nations like the dreaded Iraq, with the implicit implication that fracking could secure our energy needs.
  3. Enforce the general message that fracking has no negative effects when mines are built properly.
  4. State that any other negative effects like earthquakes or gas polluted water are irrelevant because that might occur naturally anyway.
  5. Do not interview anybody from the Royal Society.
  6. Do not interview anybody from a British energy company.
  7. Do not interview any British politicians.
  8. Do not interview any British scientists.
  9. Undertake an unofficial interview with an american doctor to show that US policy is the problem and not the secret concoction of obviously very toxic chemicals - and later explain that European energy companies would have to disclose their cocktails of toxic chemicals... which is okay then.

And I too would have definitely avoided these issues:
  1. Can we trust energy companies to indeed build these wells safely?
  2. What are the consequences if after thousands of miles of wells are built under major British cities, it is discovered that infact, the wells had not been built safely? Or some other disastrous and previously unknown side-effect is discovered.
  3. Can mines be 'unbuilt'?
  4. What are these secret fracking chemicals that were used in the 2011 British trials?

It seemed to me that Professor Iain Stewart did not want to give his real opinion about fracking, even though he just spent an hour telling us how wonderful is it when it is done properly and not how those cowboy Americans are doing it...

I knew little about fracking before watching the documentary, but this "pre-emptive reassurance" by Professor Iain Stewart makes me worry why exactly Horizon think I need to be reassured...

Wednesday, May 8, 2013

Innovation, Creativity, Software and Success

It is amazing just how many software companies are "creative" and "innovative". In addition, by reading CV's and online profiles, it is amazing just how many people are also "creative" and "innovative".

At this point, let's loosely define our three terms "Innovate", "Creativity" and "Success":

  • Innovate - Provide a new solution to problem that is distinctly different from existing approaches
  • Creativity - The desire, ability and resourcefulness to create 'something'.
  • Success - The measure by which an individual or organization can be said to have achieved its objectives.
So it might seem reasonable to expect that the world is full of innovative, creative companies that are full to the brim of innovative and creative people. And that all of these innovative and creative companies are creating innovative new stuff....

Thus it stands to reason that the majority of these companies are NOT successful, because most new 'stuff' is created by the same relatively small number of companies.

In most cases, not being a creative and innovative company is perfectly fine. For example, a restaurant does not have to be particularly creative or innovative to make a profit (although I am not saying it may not help).

However, if you are a software company and you are not genuinely creative or innovative I would bet that you are probably not successful either. Even the big mega-corporations like Apple, IBM, Oracle, Microsoft, Facebook and SAP were at some point in their history creative and innovative and to varying degrees continue to be so.

So if these mega-corporations still continue to be innovative and creative, then your small software company DEFINITELY has to be.

But most software companies are neither innovative nor creative. And I wonder if this is because most people are not actually innovative or creative, despite what they say.

Companies do need different types of people, but software companies do need a genuine element of creativity and innovation which, if not always, mostly takes precedence over the best practices, rules of thumbs, processes and procedures that 'most' people would govern by.

So find genuinely innovative and creative people and let them do their thing...

Tuesday, March 27, 2012

IMified - Please dont shutdown!

This morning I received the following email from IMified informing me that the service intends to shut down due to lack of profit and what seems like continual problems with the IM companies. The real shame is that IMified is an awesome service with absolutely no alternative - please do correct me if I am wrong, I'd love to know.

For those of you that are not aware of IMified (http://imified.com), it is a service which allows application developers to create 'bots' which are essentially IM users on networks like AOL, Google Chat, MSN, XMPP etc and provides a single consitent API to allow asynchronous communication with a client. This service is absolutely brilliant for notification services like Femtoo.com (http://femtoo.com - online web page tracker and notification service). Maybe IMified could have a future as an open source, cloud-based service. Eitherway, I really hope that we can save IMified.

Here is the email for your information:

===================

Dear IMified Customer,

The purpose of this email is to alert you to a change in the IMified service.

Over the last several months, it has become apparent that running an IM bot hosting service is something that cannot be done profitably. When we created IMified several years ago, the popular IM networks had programs for running bots - bypassing limits in contact lists and message volume, for example. This made it feasible to run a large scale IM bot. However, over the last year, these programs have started to disappear. Our contacts at IM providers have left the companies or been reassigned. It's become clear that this is a business that public IM providers no longer want to be in.
We've tried to keep IM support operational despite the fact that not only have we had no support from IM networks, but we have also had to fight the networks actively stopping and shutting down IM bots. Continuing to do this requires a significant amount of resources, and does not bring in enough revenue to justify the battle. So we're shutting down the service.
The logical question that many of you will have next is "what can I do instead?" Unfortunately, we're not aware of any viable IM bot hosting services. Those that we knew of have all gone away. One alternative is to host one yourself. Building a basic XMPP (Jabber) bot isn't hard. You just need to sign up with an XMPP host, someone like Gmail or Jabber.org will work for low-traffic bots. There are numerous tutorials and frameworks for building XMPP bots in your language of choice. A discussion thread on Quora has some other ideas. Once you have a XMPP bot up, you can connect it to other IM networks using XMPP Gateways (also known as Transports). These sign on to an IM network for you, and relay messages to and from an XMPP account. There's not much in the way of public gateways available, so to go this route, you may need to host your XMPP server and your gateways yourself. Spectrum and Kraken are two popular gateways, and Kraken is available as an easy-to-install plugin for Openfire, an open source XMPP server.

If you have concerns or questions about the shutdown, please email shutdown@imified.com


- The IMified Team
===================

Wednesday, March 21, 2012

Software development and the dangers of the word "Done"

People like using the word "done" and with good reason - if something is "done" it means we no longer have to worry about it and we can get on with the next task - its a decisive and positive word. But this word is usually misleading and in the real world, software development tasks are very rarely "done".

To say that a software development task is "done" tends to send a message to the project manager and client that the specific user story or feature will never need any revision and is completely free from errors. This is almost certainly not the case. It's a good assumption that new code will contain errors, even if you haven't found them yet and you can be pretty sure that when the new feature is demonstrated to the client or as soon as real people start to use the system, changes will be required.

And at the engineering level, well organized code will generally adhere to the DRY (dont repeat yourself) principal. This means that engineers will try to 'abstract' code, functions and structures to be as general as possible so that they are used as often as possible. This implies that although a feature may appear unchanged from one version to the next, the engineers really know that actually a lot of the underlying code may well have changed and although unit testing can 'positively' test and confirm consistent behaviour, a good dose of manual testing will do no harm.

But this article is not about code organization or testing techniques, it is about why we should try not to use the word "done". When we say this word, we probably really mean: "done to the point where another task is of higher priority". If this definition of "done" is generally accepted then project managers and clients can continue as usual, but should not be alarmed when something marked as "done" contains bugs, needs changing or the code requires rework.

The Internet - and what it could be

The Internet is still basically a mass of insular websites that do not interact with each other. Websites are still primerily designed to be read/browsed by a human, regardless of whether it is done from a PC, mobile or tablet device. Most websites are still just like books in a library which do not communicate and share information with one another.

Websites have got prettier and include more media, like music, videos and animations but essentially, websites have not changed and the Internet as a whole as not significantly changed for many years.

But the Internet has an amazing potential, behind all of these insular websites are computers capable of complex data processing and hi-speed communications, but for the most part, this potential power is just used to serve up pre-defined static content.

I think it is now time to build the REAL "web 2.0". For me, this implies websites that communicate and share information automatically with other websites and services in (pseudo) real-time. The end goal of which is to turn the Internet into something really powerful - a single massively parallel computer. The hard part has been done for us with technologies like HTTP, DNS, TCP/IP, JSON etc already defined and well supported by tried and tested technologies - all the building blocks of a new Internet are in place.

But to create this new Internet, we must create a mechanism such that all data currently formatted and published for humans (the majority) is also available to other web services. Consider my own personal website:

http://tomcarnell.com

If you visit this website from a standard web browser, you would be presented with a mixture of graphics, text and hyperlinks organised to be human readable and formatted to be attractive (like any other website). Unfortunately, this type of content is completely unsuitable for a third party web service to make use of, let alone use in a meaningful way. So, lets consider the following two proposed URL's:

http://tomcarnell.com/services.json
http://tomcarnell.com/data.json

Now, a third party web service can read the contents of 'services.json' and can learn that the 'tomcarnell.com' site has a service called 'contact_me', a programatic version of the 'contact form' (http://tomcarnell.com/contact_me) which allows the service to submit a contact request. The services.json file will also detail how the 'contact_me' service should be called, what type of data it returns and maybe some human readable text to explain what the service does and how it should be used. We could also implemenent a 'subscribeToService_xxx' service which means that before allowing just anything to use the 'contact_me' service, one must first subscribe. The rules and details of how these services work are completely up to the website owner.

Our third party web service could also decide to read the contents of 'data.json' which contains semantically structured data. This data might contain the details of my portfolio, my curriculum, my contact details, links to social profile pages and any of the text found on the human readable website. It could also detail daily contractor rates, availability, location and phone number, which a third party recruitment service might find useful when searching for an available web developer in the Madrid area with experience with jQuery for example. Again, the contents of 'data.json' are completely up to the website owner.

Online shops could employ the same standard to publish special offers and could provide services to allow 3rd application to check an order status or product stock levels.

A news website could publish their latest headlines in 'data.json' and maybe provide a search service in 'services.json' which could return a list of news articles that fulfill the search criteria.

But the fun really starts when we start to combine these services. We could have language translation services, hotel reservation services, transport reservation systems, services that interact with social networks, services to access and query government and official data and all this opens us up to the possibility of creating sophisticated 'problem specific' services, for example a service that knows how to plan an entire wedding and knows which other services it needs to locate available venues, check guests calendars, send and recieve invitations, book a chauffeur driven car, obtain estimates for food, gift lists etc...

It would be really great to move in this direction, but it will take the support of lots of people. However, as a stepping stone, website owners can start by using http://livedirectory.org/ - a service which provides the functionality to publish and manage an arbitrary set of data and allow 3rd party applications to subscribe to this data, which means that LiveDirectory will send change notifications to all subscribers when this data is updated. For example, I have implemented the equivilent of the 'data.json' file as a LiveDirectory profile:

http://livedirectory.org/profile/tom-carnell

The above LiveDirectory profile details information about me and allows anybody to 'subscribe' to any part of that data structure. Another example is the jQuery profile:

http://livedirectory.org/profile/jquery

Where services can subscribe to be notified whenever a new version of jQuery is released, which is specifically located here:

http://livedirectory.org/profile/jquery/latest_version

The LiveDirectory.org aims to provide a simply but very powerful API based data manangement, retrieval and subscription system to help the Internet take one step toward becoming a more useful tool for everyone.

Wednesday, September 14, 2011

Why database column names should be globally unique

After years of working with relational databases (RDBMS's), I have just started to realise the importance of globally unique names for database columns. Generally I use the term 'globally unique' to mean unique within the database schema, but I it could potentially extend to being unique across all schemas.

The problem that arises from using repeating column names is when two or more tables are used in a single query. For example, if we have two tables: 'customers' (which has a column 'id') and 'customer_orders' (which also has the column 'id') when we perform the following query:

"SELECT * FROM customers c, customer_orders o WHERE c.name = 'tom';"

The "*" clause means we will return every column of both tables. So considering a single row returned from the query, what would be the value of 'id'? Would it be the 'customers.id' field or the 'customer_orders.id' field? What if our application needed to use both values? Of course, we could change our query and give aliases to the columns, which would also mean we would need to explicitly list the columns we wish to be returned:

"SELECT c.id customer_id, o.id order_id, c.name, o.value FROM customers c, customer_orders o WHERE c.name = 'tom';"

So we can now reference either the customers.id value and the order.id value. But this solution makes our query fragile and susceptible to schema changes - what if we later rename the column 'c.name' to 'c.first_name'? The query would then break. In addition, and for the purposes of consistancy of any 'data interface' we might define, we would be forced to use these explict column aliases in all our queries, which would reduce the readability of the SQL and introduce the same problems associated with any form of code duplication.

In reality, the first SQL query would be perfectly good had we used unique column names. For example, 'customers.cst_id' instead of 'customer.id' and 'customer_orders.cso_id' instead of 'customer_orders.id'. Here I have chosen a three character 'prefix' for column names that gives some clue to the name of the containing table. Therefore 'customers.name' would also become 'customers.cst_name'.

Implementing a simple column naming convention across the entire database schema will greatly improve the effectiveness of SQL queries and will reduce the likelihood of application errors that may occur by accidentally referencing the wrong data within an application.