Javascript

Click Tracking with jQuery

Standard

Generally I use only Google Analytics for the stats of my web apps, but for one of my current projects (Bröllopia – sökmotorn för ditt bröllop – a swedish wedding site) I needed a bit more. I wanted to have detailed statistics for links leading to other sites to use these stats in the app itself. One way of doing this is of course to make all these links go to a page on my site and then when this page loads I record all the stats I want and then redirects to the external site, for example the link www.digitalistic.com/tracker.php?id=123 loads tracker.php and logs a click on the link with the id “123” (this id needs to be mapped to a link in the database) and then forwards to www.webhostninja.com. The link on my site would then look like this:

<a href='http://www.digitalistic.com/tracker.php?id=123'>WebHostNinja</a>

The problems with this is that it is slow (one more page load) and that the link the user sees does not lead to the page, ie the link text “WebHostNinja” and then the user expects the link to be something like “http://www.webhostninja.com” and nothing else. I don’t like this from a usability point of view. Not that I have any evidence for it, but I suspect that relevant outgoing links help the sites Google juice. Definitly it helps the Page Rank of the site I link to, and why should I not help them?

My solution is to use some jQuery magic. Download and install jQuery (btw, if you like jQuery, then take a look at the fantastic jQuery tools). Then add the class “track_this_link” to each link tag that you want to track as well as give each link tag an unique id (so you can track which link is which). The href attribute should point to the external site directly and not to some internal page. The link above would now look like this:

<a href='http://www.webhostninja.com' id='123' class='track_this_link'>WebHostNinja</a>

Next step is to add a some javascript that adds an onClick event to all the links with the class “external_link”:

$(document).ready(function(){
$('a.track_this_link').click(function() {
$.post(http://www.digitalistic.com/tracker.php, {id:this.id});
return true;
});
});

This means that each time an external link is clicked a post request is sent to digitalistic.com/tracker.php with a unique id for the link in question. What is left is to implement the tracker.php script so that it can handle the post and save the data you are interested in to the database in a secure and correct way. I am happy to just summarize the number of clicks per link on a daily basis, but if you want to save detailed data for each click.

Do you have a better solution to this problem? Let me know!

Writing an iPhone page in minutes using IUI

Standard

The 3G iPhone was released a few weeks ago, so it’s time to write a page or application for the iPhone and try to ride the hype wave for all it is worth. So I started working on an iPhone specific page for enlatele.com.mx – after all why shouldn’t Mexicans be able to find their TV-schedule on their iPhone? You can see the finished result on iphone.enlatele.com.mx. Since all I want to do is show web based content in the iPhone there is no need to start using the iPhone API and build an actual application, so instead I built a web page aimed specifically at the iPhone, and it was suprisingly easy!

Some general lessons I learned before we jump into the actual code:

  • iPhone pages are all about lists, the more lists the better apparently. On the smaller screen you can not really do much more than lists, and if you want to do more then just let the iPhone user browse your normal web page.
  • Write a seperate page for the iPhone, do not detect if a site visitor is using the iPhone and automatically server up the iPhone page. One of the cool things about the iPhone is that it is possible to interact with normal web pages quite well on a small screen, dont take this away from the iPhone user. Instead create a subdomain specifically for this, for example iphone.enlatele.com.mx, or m.enlatele.com.mx.
  • The iPhone connection is not always the best, so do not serve lots of images etc if there is not a real need for it. This might not be true with the new 3G iPhone in the middle of New York or Stockholm, but writing an iPhone page for Mexico I’d rather be safe than sorry and keep the number of images to a minimum.

I used just HTML, CSS and JavaScript, packaged up very nicely in the IUI framework. Basically all you do is download a CSS file and a JavaScript file, import them into your HTML page and write lists with certain CSS classes and IUI takes care of all the rest. IUI even has minimized versions of the CSS and JavaScript file you can use to make the page even smaller (and thus a quicker download to the iPhone). This is what I have in the header of my page:

<meta name="viewport" content="width=320; initial-scale=1.0; maximum-scale=1.0; user-scalable=0;"/>
<style type="text/css" media="screen">@import "iui/iuix.css";</style>
<style type="text/css" media="screen">@import "styles.css";</style>

The meta tag “viewport” is needed to let the Safari browser on the iPhone know how to deal with the content, without that tag you risk that the page acctually looks worse in the iPhone than in a regular browser.

In the body of my HTML document I specify the menus of my iPhone, ie lists lists and more lists. The code for the start menu is:

<ul id="home" title="Enlatele.com.mx" selected="true">
<li class="group">Canales</li>
<li><a href="channelgroup.html?=canales_abc">A-B-C</a></li>
<li><a href="channelgroup.html?=canales_def">D-E-F</a></li>
<li><a href="channelgroup.html?=canales_ghi">G-H-I</a></li>
<li><a href="channelgroup.html?=canales_jkl">J-K-L</a></li>
<li><a href="channelgroup.html?=canales_pqr">P-Q-R</a></li>
<li><a href="channelgroup.html?=canales_stu">S-T-U</a></li>
<li><a href="channelgroup.html?=canales_vwxyz">V-W-Y-X-Y-Z</a></li>
<li class="group">Proveedores</li>
<li><a href="#sky">Sky</a></li>
<li><a href="#cablevision">Cablevision</a></li>
<li><a href="#mastv">MASTV</a></li>
<li><a href="#tvabierta">TV Abierta</a></li>
</ul>

And this results in this page (as can be seen on iphone.enlatele.com.mx):

startpage.png

IUI styles it all nicely. The key parts of this list is:

  • the id “home” on the ul tag. Ids like this are used throught IUI to identify submenus etc. For example the link to Sky is <a href="#sky">Sky</a> this links to another ul list on the same page with the id “sky” – <ul id="sky" title="Sky">. This means that I can have most of my menu structure, submenus and all, on one page. So once this page is downloaded to the iPhone the phone do not need to access the internet just because you click on a link to a submenu.
  • title="Enlatele.com.mx" is the header of the page
  • selected="true" on the ul means that this is the first menu to load.
  • <li class="group">Canales</li> creates a grouping in the menu, a nice way to make things look fancy without acctually doing any work
  • now to a really nice thing about IUI, it takes care of loading content dynamically into your current page seamlessly. For the enlatele iPhone page there are listings of TV channels that are not loaded to the iPhone unless the user clicks on a link in the menu to do this. So if a user wants a list of all channels that starts with A, B or C he would click on the following link in the menu – <li><a href="channelgroup.html?=canales_def">D-E-F</a></li>. IUI would then insert the content of channelgroup.html into the page basically using AJAX (don’t worry about the ?=canales_def – that is just so that the enlatele backend knows what to load). The channelgroup.html page should _only_ contain a list of channels to show once this link is clicked, nothing else:

<ul id="canales_abc" title="Canales A-B-C">
<li><a href="channel.html?canal=a_e_mx_west">A&E MX West</a></li>
<li><a href="channel.html?canal=american_network">American Network</a></li>
<li><a href="channel.html?canal=animal_planet_latinam">Animal Planet LatinAm</a></li>
<li><a href="channel.html?canal=animax">Animax</a></li>
</ul>
So no HTML start tags or BODY tags or anything else, just the list you see above, that is the whole of channelgroup.html.

IUI also deals with other things you really want on an iPhone page, such as automatically inserting a back button (that only looks good in Safari since it is using Safari specific functions, so if it looks crappy when you test in Firefox do not worry), showing a nice loading graphic etc. There are also a lot of more fancy stuff that can be done with IUI, but just the basic can get you a long way. This is pretty cool stuff I think, and it makes is really quick to implement something that works great and looks great on the iPhone. Of course there is a big part missing here, and that is the backend acctually filling these pages with content (in the enlatele case dynamically creating the channel list based on what is in the enlatele database), but that is standard programming stuff so I wont cover it here.

If you are interested in doing more mobile content, especially design, check out Yahoo’s Design Stencil Kit for mobile. It contains all the building blocks you need to do mockups and designs for the iPhone and other mobile platforms.

HTML is the worlds most common API

Standard

Most folks that are working with Mashups just assume that services and APIs will magically appear but unfourtunatly there are not that many public APIs around today. Just check out programmableweb and you will see. More and more are added every day, but it will bever reach the level that a majority of systems have an API, especially not if you think about systems within the coporate firewalls. Simply put there is a painful lack of APIs, and if that is not addressed it will stop the mashup wave in it’s tracks. Fortunatly there are already smart people working at this, and one of the solutions is to start using HTML as it is an API. That’s right, start using all the data and functionality that today is available in HTML to build new innovative mashups and solutions.

The potential of HTML

All new interesting applications (Skype being the exception that proves the rule) has an HTML interface. And this is true not just for the consumer facing applications, but for Enterprise level applications as well. So with the millions and millions of HTML pages in existance today it is not unlikely that HTML is one of the worlds most common data formats (I wonder how it compares to printed text and audio for example). The great thing with HTML is that it does not just contain data, it also is the interface to a whole lot of functionality (when you search Google you do that via HTML don’t you?). What if we could use HTML as one big API? That would make HTML the worlds most widespread API and that would give mashup developers and programmers access to more data and more functionality than ever before.

The problem with HTML

Almost not sites on the web today are following the HTML 4 standards. So todays browsers are very good at interpreting the tag soup that most pages consists of (ie broken HTML, missing end tags etc). Furthermore HTML is used to both mark up data in a document, for example with the <title> tag, and to mark up how the data should be presented, for example the <b> tag. All this together makes HTML documents unstructured documents (by implementation, not by nature) with data in very application specific formats (microformats will help here, but there will be some time before that is widespread enough to be really usefull).

Another problem is of course that there is fewer and fewer pages on the web that uses pure (albeit broken) HTML, there are more and more Javascript around. Especially in the Web 2.0 applications most of the really interesting functionality is available via AJAX. So it is not only HTML, but also Javascript that has to be taken into consideration when one wants to get to a web applications functionality.

Parsing

So we have huge amounts of data and functionality in HTML and we want to use it to make our latest funky Mashup. The good old approach is to try to parse the page in question using Perl, now it can be done pretty well using almost any modern programming language. There are several problems with parsing though:

  • It is damn complicated to get to work on serious web pages and once it is done it breaks easily
  • Good luck handling a real tag soup, already that breaks most parsers (since using XML parsers for this means that the parser simply stops at the first error it encounters)
  • It is boring to program those parsers (if you havent tried then lucky you)
  • Can not access functionality that uses javascript and AJAX
  • It is hard to handle things like login into a web application (ie session handling) and to navigate over several pages

Still this is a very usual approach to get to data and functionality in HTML. But there is a much easier way…

Web Scraping

I bet that a fair portion of the people reading the word “Web Scraping” think of old mainframe terminals and “Screen Scraping” and frown. Don’t worry, technology has moved forward lightyears since the days of mainframes. Web Scraping is to interact with HTML (including Javascript if it is a good scraper) and to either extract data from the HTML or repackage the functionality in the HTML. The data can be saved into a database or a file for example, and the functionality can be made available as a REST service, a programming language API or whatever else makes sense. Suddenly HTML is wide open. Just imagine that you wanted to get data from Digg (before the Digg API that came out a few weeks ago) for some reason, without an API that would be hard. But using a web scraper you could for example build a REST service out of the search on Digg only by accessing the HTML. Web Scrapers are used more and more for doing things like collecting large amounts of job ads or flight information and then repackage that data into sites that then allow users to search for a job or a cheap flight.

Openkapow

The web scraper of my choice is the one supplied on openkapow.com (disclaimer: I am working for Kapow Technologies, the company behind openkapow.com, but trust me in that I am not plugging openkapow to make my boss happy – it is really a great product). Using openkapow one can access data and functionality on any web page and access it as a REST service or and RSS/Atom feed. Of course JavaScript is handled automatically, it is possible to navigate multiple pages, login to restricted pages and have full control over the process flow with conditions and error handling. I recommend that anybody that is interested in how to use HTML as an API takes a look at openkapow.

An Eye Opener…

Thinking of HTML as an API does significantly expand your horizons as a developer. I have literaly seen a light go on in fellow geeks eyes when they realize the potential. Suddenly the web is really yours to use in your programs and mashups. When suddenly APIs and services are abundant then you can start using the other cool mashup tools around (Teqlo, jMaki etc).