To further help focus the attention on local Swedish APIs we are now starting a competition to find out which is the best Swedish API. After about 2 weeks of voting we hope to present the best API the country has to offer!

We have noticed a real need for this during the last few years when we have been working with our customers and their APIs. There have been a real lack of a one-stop-shop to let developers find Swedish APIs and it has been hard for Swedish API providers to reach developers. Now we hope to have solved that problem…

• You have to trust the API providers to keep that API up and running. Anyone that has done Twitter mashups know how much keeping an API up and running is worth.
• You have to trust the API providers to keep the API (somewhat) stable and to not remove features that you are depending on, or add features you really dont want. An example of this is all the mashups built on Google Maps, and we all know that there are way too many of those. Have the developers of those mashups really thought about what happens when Google puts ads on the Maps (and they will, they have already started in parts of the US)? I dont think so, the developers just trust Google to keep on going. If you have a commercial site using Google Maps you probably do not want ads for your competitors on those maps.
• If your mashup becomes a success you have to trust the API providers you are depending on not to change their terms of service so that your mashup suddenly becomes an illegal use of the API. Also, you have to trust the API provider not to use the fact that you are depending on their API for your business against you in a business negotiation.
• If you use web scraping you have to trust the site you scrape not to change to often (or at least trust your own ability to roll with the punches and update your scraping so it works with the new version of the site as well) etc. That is a lot of trust, and when mashups move from toys into real applications this becomes an issue.

All these things makes me a bit nervous, as I am a bit paranoid (note: this has not been clinically proven, I still think that THEY are out to get me). So how to mix a healthy bit of paranoia into my mashup building and get something good out of it all? What I do is that I always try to be aware of that I might have to switch API provider. Are you building a Twitter mashup? Why not also take a look at Jaiku’s API, or Pownce’s API (or Plurk or FriendFeed etc etc). You dont have to build your mashup so you can switch API provider in a matter of minutes, just be aware what else is out there so that you see the commonalities and don’t use to many features unique to one provider. This is the approach I am currently using when I am building mashups, at least I know that if shit hits the fan I can always go with somebody else. It will hurt a bit and take some effot, but I am not dead in the water. For the Google Maps example this would mean looking at Yahoo Maps and see what features the Yahoo Maps API have in common with the Google Maps API, and just use those common features. This can also come in handy if you hit the maximum number of requests on Google Maps, then it would be nice switch to Yahoo Maps automatically.

The risk with all this is of course to spend to much time preparing for something that won’t happen. It is the same situation as developers spending so much time making their code perfectly scalable and optimized that they acctually never ship anything in time. So dont go too far, but be aware of the situation. Trust is nice, but trusting several API providers to always do a good job and to not be evil in order for you to survive is quite risky.

Now when I am writing some of my own web apps I would like to avoid spam to overwhelm me, both via contact forms and comments. Lo and behold, Akismet is available as a web service, and via some ready made libraries it is very easy to use from outside WordPress as well. I am using Elliot Haughin’s CodeIgniter Library, but there are also libraries for PHP, .NET, Ruby on Rails, Drupal etc etc. All you need to do is to get a Akismet API Key from WordPress.com (register, then go to My Account and Edit Profile and you will find the key), download a library (or start writing your own from Akismets API) and start sending all comments to Akismet. The API Key is free for personal use, cost $5/month if you make more than $500/month from your site and cost a bit more for Enterprises. Well worth it I say, it will save you time and frustration all around, and make your website so much cleaner. There is no longer any excuse to suffer comment spam.

Thanks Akismet for one of the best and most usefull web services around!

It is quite easy to post to several microbloggins services at once. Jaiku does a great job of importing RSS feeds, so posts to Twitter or Pownce can easily be imported to Jaiku. Via Twitterfeed it is easy to get an RSS feed into your Twitter as well. I haven’t found any easy way of getting an RSS feed into Pownce. Also there are apps like Twhirl that let’s you post to all 3 platforms at once. In my case I also use the Twitter Facebook app to get my tweets into my Facebook status and I am looking for a way to do the same with LinkedIn (no success yet). So posting cross-platform is not a problem, even if it means that you need to do some configuration and that all your posts gets trippled or quadrupled.

Reading friends posts from several platforms could also easily be done. I could of course go to all the different services and read each posts on each one, but since I want to do other things with my day than that I would rather use services like FriendFeed or SocialThing that aggregates it all into one place. FriendFeed imports from most sources and have some nice comment features, but the UI really desperatly needs a designers touch. SocialThing imports from just a few places so far (please please add RSS now!) but I still prefer it to FriendFeed. These services and others make it easy to see your friends posts from several platforms in one place. What is missing is a way to naturally post back to the microblogging platforms from these services.

What is needed, and what will come very soon I am sure, is a mashup of all these microbloggin platforms to allow users to be active on several platforms at once all from one place. I would like to see an app that allows me to interact with Twitter, Pownce and Jaiku completely. This means reading other peoples posts, replying to posts to have a conversation going cross-platform and posting to all platforms at once. Since all of the platforms have APIs this should be possible to implement (and for all I know it already exists somewhere, if you know of such an app please let me know through a comment on this post!). This would be a great mashup that would breach the community silos that exists today. Short of everyone moving to one platform a mashup is the best answer to this problem.

Btw, I have some extra SocialThing invites so if anybody wants one please let me know via a comment on this post.

The API Provider
You have this great service that lifts humanity to a new level, makes the sun shine brighter, makes TV sucks less and give the gift of limitless bandwidth to the people (or at least it is really cool). You have even added this great API, now what? How do you get developers to start using the API and spread the word of your great service to everyone and their grandmother?

Well, let’s back up a bit. First of all, do you really have a great service? If you do, then do you really have a great API? Without a product people want to use there is no need to go through the hassle of promoting it. Make sure that the API actually is usefull for developers, that it will enable them to do cool and usefull stuff easier than if they would just hack it all together from scratch. Also make sure that there are plenty of documentation, examples, code snippets etc for the developers to get their hands on to minimize the barrier to entry. Hack together some mashups yourself with your API included in the mix, to give people and idea of what can be done. The key to get an API used by developers is to get the developers excited about the possibilities and get them talking. So give them something to be excited about and something to talk about.

Once all that hard work is done then you can promote your API via directories such as programmableweb and webmashup so that developers can find you. If you have made your own example mashups, then go through the steps below to market that, that is a good way of getting some recognition.

Last, but not at all least, show some love for the developers that has taken their time and built something using your API. Have an example gallery where they can list their creations. Blog about them. Talk about them at conferences. “Link love shall be bestowed upon those who link love showeth”.

The Mashup Developer
For the developer of the mashup there is Google AdSense money on the line, or maybe just recognition from peers. Most mashups result in web pages anyway, so make sure to do all the SEO stuff – have good page titles, have a good copy, have validating HTML, have a sitemap available etc. If there is money down the line for you then also throw some money at advertising (Google & Facebook makes this a walk in the park). All this is standard, but as there are differences between mashups and a regular web page you should also use that to your advantage.

What APIs do you use? What tools have you used to piece things together? Explain how you made your mashup, what the moving parts are. If you used Yahoo! Pipes, then link to the pipes used and explain how they were done. If you used Google Maps (and if you are a mashup newbie then I guarantee that you have, just admit it… “my name is Andreas, and I am a Google Maps addict”) then explain how. If you used openkapow robots, then explain how you developed them. Since API providers are suckers for traffic, just as everyone else, it is not unlikely that they would be interested in adding your mashup (assuming it kicks-ass, which of course it does) to their example gallery. All this creates more link love, more Google baits and really increases the chances of your mashup being found and appreciated by fellow developers. Another plus is that all this also increases the chances to be blogged about, do not forget that bloggers are suckers both for traffic and content.

There’s both money and recognition in entering your mashup in a contest, see programmableweb for a good list of what you can enter right now. You might not have to redo the mashup from the ground up, just add another API to the already great mashup you have made and you could already be a winner. If you go to Mashup Camp you could enter the traditional Speed Geeking (like speed dating for mashups basically) and go home with a shiny new Macbook.

Of course also list your mashups in directories such as programmableweb and webmashup , but by now you should know that already

Thanks for everyone that discussed this with me at Mashup Camp! For the notes from this session check out the Mashup Camp wiki.

Facebook vs. OpenSocial
The Facebook API was compared to the OpenSocial API. There were quite a few differences, but what caught my attention was that it takes 60 lines of code in OpenSocial what it takes 2 lines in Facebook using the Facebook Markup language (FBML). In general it seems like the OpenSocial API’s are quite imature so far and that Facebook will keep ahead with it’s aggressive constant innovation for quite a while. Still, go for both Facebook and OpenSocial if you are going to do an app. If the social platform war will stay limited to just two standards we are quite well of …

You are defined by your friends
In addition to the information you explicitly have defined, like your age and if you are married or not, there is a lot of information about you that can be deducted from your list of fiends. Are most of your friends from London? If so there is a good chance that so are you. If most of your friends are working in the banking industry then there is a good chance that you are as well. This information is of course worth a lot in the hands of an advertiser that can target the ads to the right people. Now with Facebooks new Social Advertisting initiative that makes it possible for them to target ads to Facebook users even if that user is on another site than Facebook this kind of information is worth even more.

Techlightenment have developed a cool Facebook application called Socialistics that on-the-fly makes best guesses about a person based on their friends. It is a really interesting proof of concept of what can be done. Install it yourself and see what can be deducted about you based on your friends. The more friends the better guesses of course, but it is amazing what can be figured out about you. The power of these kind of statistical analyzis will explode in the near future, which is going to make both Mark Zuckerberg and the Google guys more money.

Who am I according to my friends?
Using the Socialistic app I get the following data about myself. I most likely live either in Stockholm (I did for years) or in Mexico City (not a bad guess since I do spend a lot of time thre), there is a 10% chance I have worked for Microsoft in Copenhagen (wrong) and I most likely took my university degree at Universidad Iberoamericana in Mexico City (I have studied spanish there for all of 1 hour once, but that’s it). The good stuff is that there is a 65% chance that I am a man (just that I work with IT should bring that number up to at least 90%) and that I am from Sweden (guilty as charged)

According to my wife I am a social outcast since I only have 28 friends on Facebook. If the name calling I have to take out a Facebook divorce (ie “cancel relationship”), after all it is 28% chance that I am single already.

Enterprise Mashup Infrastructure: How Web 2.0 technologies are used inside Enterprises today
This is the Web 2.0 Expo presentation and it will be about how Mashups enable companies to solve problems quickly and efficiently, and also deal with problems that are impossible to figure out using traditional technologies. I will cover the basics of mashups and the technology behind them, real-life examples of how mashups are used within enterprises today and what the mashup infrastructure looks like.

In an enterprise environment, mashups can be seen as an extension and a compliment to SOA, and not just as simple Google Maps applications that are viewed on the web. This approach enables self-service IT that lets business users build the situational applications they need to solve the problems at hand. A key problem in enterprise mashup building is how to get access to abundance of unstructured data both inside and outside the company firewall. And, this is a problem that can now be solved in minutes rather than hours or days. How this is done will be shown in a short demonstration.

Where are all the APIs and Web Services? Build APIs to any web site in minutes using Kapow Mashup Server.
In this Mashup Camp presentation I will zoom in on one of the most forgotten parts of Mashups. There are many great tools available to build the user interface on a Mashup, but were are all the APIs and Web Services needed to feed the Mashup? These APIs will not appear magically and even if more APIs are created every day it is going quite slow. Using the Kapow Mashup Server and openkapow.com you can build an API to almost any web site in a matter of minutes. Suddenly you can use the whole internet as a structured data source to feed your mashups.

See you there!
If this is anything you like to hear about and happen to come to Berlin or Dublin then please drop by and hear me speak.

The official title of the presentation is “Integration Gets All Mashed Up – Bridging Web 1.0 and Web 2.0 Applications”, and it is an overview of what mashups are, what technologies are involved and how the different components fit together inside a mashup. It was a lot of work getting the presentation together, but it was well worth it since I got a lot of good feedback from people (thanks to eveybody that came and heard me speak!).

The presentation starts out with the standard (= Wikipedia) definition of mashups and then move on to describing the enabling technologies such as REST, JSON, Atom and RSS. Of course all this is from a Java perspecive since it is after all a JavaOne presentation. Next are the components of a mashup such as Mashup Builders, Widgets, Mashup Enablers and how they all fit together.

Next I describe the the mashup programming model, which is a lightweight programming model that means that the focus is on assembling and not coding. In the end I highlight what it means to be a developer in a mashup world.

If you are in too much a hurry to go through the presentation I can give you the summary right here:

• Mashups are all about
  • Building a situational application ad hoc
  • Reuse and remix both data and funcitonality
• Using Mashups means that
  • You can do more in less time
  • You don’t have to reinvent the wheel
  • You are not in total control of how your applications and data will be used

If you are interested in Mashups I hope that this presentation will give you a good overview. Please take a look and a listen and let me know what you think!

The Web is full of Ecosystems

There are already plenty of Ecosystems out there. Amazon was one of the first companies that understood the power of this, and created an open API that developers all over the world now are using (and thereby increasing the sales of books for Amazon, plus the brand recognition of course). YouTube’s success is largly due to the ease of including and showing videos on your own sites and blogs (the real success was of course that Google pay a ridiculous amount of dollars for them, but that is a completely different story). Flickr is doing the same for photos online. WordPress is an OK blog engine, but it would be nothing without all the plugins, so that is another successful Ecosystem. In the Enterprise world there is of course Salesforce that now has more requests coming in via their API than via their web page.

There are also successful such Ecosystems that are not based on web applications. Firefox (OK, that one is almost web based, just humor me please) is a great browser, but without all the add-ons it is not that usefull (me love Firebug!). Apple has been very successful in creating an Ecosystem around the iPod, just think of how many companies make great money by just producing skins and other extensions to the iPod.

The Platform Wins

What is in it for the Platform is pretty evident: more loyal users that have more reason to keep coming back. This is hard to create, and if System Providers build apps on your Platform then basically you are expanding your development deparment and you have more brilliant minds trying to figure out the killer app for your users. Also this is a great way to keep the competition at bay. Now when Facebook is becoming a Platform and not just a site it will either force the main competition MySpace to do the same or be left behind.

The System Providers Win

The System Providers that write applications that run on the Platform (most likely called something fancy like widget, gadget, addon, plugin or extension) gets access to a whole lot of ready made functionality and a huge existing user base. There is no longer any need to implement photo album funcitonality for your web app, just use Flickr. No need to build video capabilities, just use YouTube. If you build an app for Salesforce you can sell it on their AppExchange and make some money. So in short you get both functionality and a userbase, that is a pretty good deal.

The Users Win

The users can stay on one platform and take full advantage of that the platform continously gets more and more functionality added. Since each of the System Providers do not have to implement the most basic functionality (user handling, file uploading, photo album etc) they can concentrate on new great features, which of course benefits the end users. So more cooler useful features, that makes uses happy.

Win Win Win and Lose

The winners are the Platform providers that are in the center of the Ecosystem, the System Providers that gets features and a user base for free and the users that get more features quicker. That is a pretty impressive Win-Win-Win scenario.

The losers are the companies that do not understand that Ecosystems are the new type of light weight business partnerships. Either you try to be a Platform or you take advantage of whatever the Platforms around you offers. To build a new big web application and not provide an API is simply failing to take use of a huge opportunity. To build any web site and not take advantage of one or more existing Ecosystems is to do more work than you need to. It seems like Facebook really has realized that (that is no good reason to say no to the $1 billion that Yahoo is rumoured to have offered, saying no to $1 billion is always a bad move in my humble opinion).

The potential of HTML

All new interesting applications (Skype being the exception that proves the rule) has an HTML interface. And this is true not just for the consumer facing applications, but for Enterprise level applications as well. So with the millions and millions of HTML pages in existance today it is not unlikely that HTML is one of the worlds most common data formats (I wonder how it compares to printed text and audio for example). The great thing with HTML is that it does not just contain data, it also is the interface to a whole lot of functionality (when you search Google you do that via HTML don’t you?). What if we could use HTML as one big API? That would make HTML the worlds most widespread API and that would give mashup developers and programmers access to more data and more functionality than ever before.

The problem with HTML

Almost not sites on the web today are following the HTML 4 standards. So todays browsers are very good at interpreting the tag soup that most pages consists of (ie broken HTML, missing end tags etc). Furthermore HTML is used to both mark up data in a document, for example with the <title> tag, and to mark up how the data should be presented, for example the <b> tag. All this together makes HTML documents unstructured documents (by implementation, not by nature) with data in very application specific formats (microformats will help here, but there will be some time before that is widespread enough to be really usefull).

Another problem is of course that there is fewer and fewer pages on the web that uses pure (albeit broken) HTML, there are more and more Javascript around. Especially in the Web 2.0 applications most of the really interesting functionality is available via AJAX. So it is not only HTML, but also Javascript that has to be taken into consideration when one wants to get to a web applications functionality.

Parsing

So we have huge amounts of data and functionality in HTML and we want to use it to make our latest funky Mashup. The good old approach is to try to parse the page in question using Perl, now it can be done pretty well using almost any modern programming language. There are several problems with parsing though:

• It is damn complicated to get to work on serious web pages and once it is done it breaks easily
• Good luck handling a real tag soup, already that breaks most parsers (since using XML parsers for this means that the parser simply stops at the first error it encounters)
• It is boring to program those parsers (if you havent tried then lucky you)
• Can not access functionality that uses javascript and AJAX
• It is hard to handle things like login into a web application (ie session handling) and to navigate over several pages

Still this is a very usual approach to get to data and functionality in HTML. But there is a much easier way…

Web Scraping

I bet that a fair portion of the people reading the word “Web Scraping” think of old mainframe terminals and “Screen Scraping” and frown. Don’t worry, technology has moved forward lightyears since the days of mainframes. Web Scraping is to interact with HTML (including Javascript if it is a good scraper) and to either extract data from the HTML or repackage the functionality in the HTML. The data can be saved into a database or a file for example, and the functionality can be made available as a REST service, a programming language API or whatever else makes sense. Suddenly HTML is wide open. Just imagine that you wanted to get data from Digg (before the Digg API that came out a few weeks ago) for some reason, without an API that would be hard. But using a web scraper you could for example build a REST service out of the search on Digg only by accessing the HTML. Web Scrapers are used more and more for doing things like collecting large amounts of job ads or flight information and then repackage that data into sites that then allow users to search for a job or a cheap flight.

Openkapow

The web scraper of my choice is the one supplied on openkapow.com (disclaimer: I am working for Kapow Technologies, the company behind openkapow.com, but trust me in that I am not plugging openkapow to make my boss happy – it is really a great product). Using openkapow one can access data and functionality on any web page and access it as a REST service or and RSS/Atom feed. Of course JavaScript is handled automatically, it is possible to navigate multiple pages, login to restricted pages and have full control over the process flow with conditions and error handling. I recommend that anybody that is interested in how to use HTML as an API takes a look at openkapow.

An Eye Opener…

Thinking of HTML as an API does significantly expand your horizons as a developer. I have literaly seen a light go on in fellow geeks eyes when they realize the potential. Suddenly the web is really yours to use in your programs and mashups. When suddenly APIs and services are abundant then you can start using the other cool mashup tools around (Teqlo, jMaki etc).