The Internet - and what it could be

The Internet is still basically a mass of insular websites that do not interact with each other. Websites are still primerily designed to be read/browsed by a human, regardless of whether it is done from a PC, mobile or tablet device. Most websites are still just like books in a library which do not communicate and share information with one another.

Websites have got prettier and include more media, like music, videos and animations but essentially, websites have not changed and the Internet as a whole as not significantly changed for many years.

But the Internet has an amazing potential, behind all of these insular websites are computers capable of complex data processing and hi-speed communications, but for the most part, this potential power is just used to serve up pre-defined static content.

I think it is now time to build the REAL "web 2.0". For me, this implies websites that communicate and share information automatically with other websites and services in (pseudo) real-time. The end goal of which is to turn the Internet into something really powerful - a single massively parallel computer. The hard part has been done for us with technologies like HTTP, DNS, TCP/IP, JSON etc already defined and well supported by tried and tested technologies - all the building blocks of a new Internet are in place.

But to create this new Internet, we must create a mechanism such that all data currently formatted and published for humans (the majority) is also available to other web services. Consider my own personal website:

http://tomcarnell.com

If you visit this website from a standard web browser, you would be presented with a mixture of graphics, text and hyperlinks organised to be human readable and formatted to be attractive (like any other website). Unfortunately, this type of content is completely unsuitable for a third party web service to make use of, let alone use in a meaningful way. So, lets consider the following two proposed URL's:

http://tomcarnell.com/services.json
http://tomcarnell.com/data.json

Now, a third party web service can read the contents of 'services.json' and can learn that the 'tomcarnell.com' site has a service called 'contact_me', a programatic version of the 'contact form' (http://tomcarnell.com/contact_me) which allows the service to submit a contact request. The services.json file will also detail how the 'contact_me' service should be called, what type of data it returns and maybe some human readable text to explain what the service does and how it should be used. We could also implemenent a 'subscribeToService_xxx' service which means that before allowing just anything to use the 'contact_me' service, one must first subscribe. The rules and details of how these services work are completely up to the website owner.

Our third party web service could also decide to read the contents of 'data.json' which contains semantically structured data. This data might contain the details of my portfolio, my curriculum, my contact details, links to social profile pages and any of the text found on the human readable website. It could also detail daily contractor rates, availability, location and phone number, which a third party recruitment service might find useful when searching for an available web developer in the Madrid area with experience with jQuery for example. Again, the contents of 'data.json' are completely up to the website owner.

Online shops could employ the same standard to publish special offers and could provide services to allow 3rd application to check an order status or product stock levels.

A news website could publish their latest headlines in 'data.json' and maybe provide a search service in 'services.json' which could return a list of news articles that fulfill the search criteria.

But the fun really starts when we start to combine these services. We could have language translation services, hotel reservation services, transport reservation systems, services that interact with social networks, services to access and query government and official data and all this opens us up to the possibility of creating sophisticated 'problem specific' services, for example a service that knows how to plan an entire wedding and knows which other services it needs to locate available venues, check guests calendars, send and recieve invitations, book a chauffeur driven car, obtain estimates for food, gift lists etc...

It would be really great to move in this direction, but it will take the support of lots of people. However, as a stepping stone, website owners can start by using http://livedirectory.org/ - a service which provides the functionality to publish and manage an arbitrary set of data and allow 3rd party applications to subscribe to this data, which means that LiveDirectory will send change notifications to all subscribers when this data is updated. For example, I have implemented the equivilent of the 'data.json' file as a LiveDirectory profile:

http://livedirectory.org/profile/tom-carnell

The above LiveDirectory profile details information about me and allows anybody to 'subscribe' to any part of that data structure. Another example is the jQuery profile:

http://livedirectory.org/profile/jquery

Where services can subscribe to be notified whenever a new version of jQuery is released, which is specifically located here:

http://livedirectory.org/profile/jquery/latest_version

The LiveDirectory.org aims to provide a simply but very powerful API based data manangement, retrieval and subscription system to help the Internet take one step toward becoming a more useful tool for everyone.