Metadata has a bad name. People find it a necessary evil that they need to deal with when they're dealing with content. These days you can't just stick a file in folder. When it comes to content management systems, authors need provide a variety document and version history details before a document can be placed in the correct location and the right document lifecycle workflows can kick-in.

If its so bad, why do we need metadata ? Search engines do an excellent job of providing accurate, secure, and fast access to massive volumes of content held in many disparate stores. Isn't that enough ? What more would you want ? Unfortunately the world of content and unstructured information grows ever bigger, is more regulated, and becomes more important to organisations day-by-day, which means that single scenario of the knowledge worker using the search engine to access content doesn't describe accurately the world we live in anymore.

The world of keeping documents on file systems and emailing them to colleagues is fast disappearing. Governments are introducing new regulations on how we conduct business with each other, and this affects how emails, documents, and other unstructured content is created, accessed, maintained, and ultimately disposed of.

The large volumes of content we deal with means vendors such as IBM, Stellent etc. develop content and records management systems to allow organisations to implement these new regulations. The policies that run on top of these systems are driven by metadata. Metadata is required to decide where a document is to be stored or when documents should be disposed of. Its unavoidable, we need metadata, unless we build artificial intelligence into our records management systems and teach the AI how to read.

Metadata has a bad name, because organisations treat it as transactional data, like a unit price on an order line item.

Metadata is different from content and its different from transactional data.

Metadata describes content in a given context or series of contexts, some of these are user-generated, some machine-generated and potentially interpreted differently across consuming systems.

Metadata needs to changes over time, to reflect the changing contexts in which it is used.

Therefore we need to start to think about metadata in a new light and consider what some people are calling a metadata lifecycle model.

Search is a magical thing. You type in a few keywords into the little box at the top right-hand corner of your screen or into your web browser and a world of relevant information unfolds itself right in front of your eyes.

Search is a half-baked truth. You type in a few keywords into that same little box and that lists comes back. You have no idea whether or not that list of results is "the truth, the whole truth and nothing but the truth". Take look at the screenshot taken from my MacBook Pro's Spotlight desktop search for "trip to London". God knows what email message about swimsuits has to do with my trip to London this week. I dare not open that email for fear of embarrassment.

I have no way of knowing whether or not the results in my Spotlight search is an exhaustive list or merely a subset. It probably isn't and without a shadow of a doubt there are better search engines out there, but for me its good enough. The "top hit" was the right document I was looking for.

Search engine vendors have a hard time. They find themselves in a continuous war of words with one and another. Rocket-scientist generals explaining to their foot soldiers in the marketing departments why their rocket-science algorithms are better, more accurate, faster, and more truthful than the next guy.

You the user/buyer get bombarded by all sorts of messages such as "multi-axial guided navigation" (no wonder the military funds this stuff, its got the word 'guided'), "smart previews" (for the dumb user), "information transformation layers" and (my personal favourite) "meaning based computing". Selling complex solutions to complex problems. Yet in software design terms all these products pretty much only deal with one single use case: the information overloaded knowledge worker sitting in front of their search box wondering what to type in next. Sure some vendors take matters a step further by screen-scraping your screen for text and using that to "guide" your search by "understanding your current context", but its just more of the same, saving you a few keyboard strokes.

The problem with the typical search engine vendor is that they need to convince you that more complexity is better not worse, which goes completely against anything we've ever learned from Mother Nature.

Take Autonomy, pretty much top-of-the-heap king-of-the-hill in search vendor terms. I like Autonomy. Yes its expensive, yes its complex, it hits all of my geeky-techy buttons. I wish I had designed & built their software. I wish I was that smart.

Autonomy's message is typical: complexity complexity complexity. They're the "meaning based computing" guys. They even provide a page explaining why other current popular approaches are unworkable and what they say is pretty much correct, but in doing so only reinforce the perception that their own sector consists of a bunch of snake oil vendors.

How do you, the corporate buyer, measure "meaning based computing"? How do you know that when you type in "trip to london" into that little box you are getting the whole truth? Unfortunately you can't.

It's magic, it's a secret and they won't tell you.

I have a confession to make: I am a Power User. (GGGGGWWAAARRRRR!) That means I spend all day geeking out over technology and software. In fact this blog entry is a perfect example of "me".

The final day of cmf2006 and I think I've got a fairly good impression on where the CMS industry is at in 2006. Yesterday I was an audience member at a Web Idol competition (think Pop Idol but with content management systems). Each of the 5 vendors (ez Systems, Sitecore, TERMINAL FOUR, FatWire, and tridion) choose to demonstrate their goods around the topic of content authoring and ease-of-use with mixed results. I left feeling a bit cynical and dissappointed. Its 2006 and vendors still competing on how good their content authoring features are compared to the next guy.

Earlier in the day, I sat into Ted Nelson's keynote, which was as interesing as it was entertaining. We spend all year trying to get web technology to work in 1000s of different situations and environments, and he's decided: nope, we need a fresh start, we need a new model. His project Xanadu has clearly defined but limited scope and it will be interesting to see how far he gets with it. The web is far from perfect and the issues discussed over the last couple of days show that, but on reflection I came away thinking Ted Nelson's project, if his aspiration is to 'reinvent the web', is comparable to reinventing the industrial revolution after it reached critical mass.

The afternoon leading up to the Web Idol competition saw me sitting on Erik Hartman's session on 'real world taxonomies', JoAnn Hackos' presentation on navigation, search, and DITA, before being royally entertained but also seriously warned about text mining in Stephen Arnold.

The thread that ran through all of yesterday and I think this whole conference is that fundementally we're all trying to solve a individuals facets of very complex and difficult problem: making content manageable - be it through search, information architecture, or ease-of-use. 

I think I'll come away today with more questions then answers, which basically guarantees my attendance next year.

No sign of the promised land just yet. Nope, we're still too busy selling 2000 CMSs to unsuspecting buyers. As they say... there's one born every minute.... a CMS buyer, that is....

Just sat in a session presented by Thomas Vogel and Travis Wissink. Travis' half-hour was of more interest to me than Thomas', due to subject matter. Travis' highlighted two near-future (I suspect next 2-3 years) directions for ECM implementations, namely Content Bridges and Enterprise Service Busses. All great stuff, and much of it confirming what we've already suspected: integration is available today, but much of it is pointless unless you can easily (read cost-effectively) integrate content & metadata stores uniformly, specifically tackling the 80% of unstructured & semi-structured content. And yes, using XML examples is cheating... 80% of content is not XML!

Right, I think I need a new CMS.... but first lunch.

I need a timemachine. I need a timemachine so I can go live 5-10 years in the future, so I can live in a world without 2000 different CMSs. I mean really ? Are they all different ? I don't think there were ever 2000 database vendors, 2000 CRM vendors, or 2000 ERP vendors selling packaged software. Where is the competitive edge ? Where are differentiating factors ? 

• They all provide a template-based authoring experience.
• They all provide a workflow process (some better than others, but big deal)
• They all provide some kind of content lifecycle management, metadata, user management, yada yada yada yada.....

So I propose we stop building content management systems and put our effort in to building a time machine and go live in an era where users/buyers aren't dumb-founded by the massive choice and complexity involved in implementing theses systems.

Seems like this week, the Glasgow-side of the company is spread to the winds. George is Boston, Owen's in Cannes (read his post here). I'm currently in Aarhus, Denmark at the cmf2006 convention, a convention without sales, without product demos, and without endless hours of hovering around a stand. Instead, I've found a 300 likeminded people who are willing to discuss the issues of web content management in all its forms. 

Being based in Glasgow its often difficult to find the right people to discuss the good & bad practices surrounding content management. Although you can have the most amazing lifestyle in Glasgow, its not the epicentre of the world regarding CM. We always talked internally within Vamosa, that we need to go out and meet people and hear new ideas, debunk old ones, and listen to the doubts that people have with regards to our product set. With the lack of commerciality, this has turned out to be an excellent forum.

Yesterday (at breakfast), I found out that my friend Tony Wood from VisionWT was staying at the same hotel. Good start to the day, we both sat in on Tony Byrne's session on Portal & CMS architectures - a good session that we're not barking up the wrong tree at Vamosa with regards to Portals, i.e. they are difficult & costly beasts and out of the box offer little more than a web development environment.

The afternoon session was with Nick Carr on the general topic of IT and IT innovation in particular. I struggled with this session, not because it was bad, quite the opposite. I struggled with it because for Vamosa it had so many aspects to it. Although the audience was mostly made up of people with in-house CMS implementations, I think it suited Vamosa and the other 2 vendors more. As small vendors (Vamosa has 40 staff) in a big pond, we are under a lot of pressure to continuously innovate. The points Nick Carr made were good, i.e. you can either innovate process or innovate product, doing both is risky and expensive, something Vamosa cannot afford to do. So I came away from that session "struggling", but that's good. It will take me a few days to digest.

Last week I met with eZ Systems, a Norwegian company developing an open source web content management system (WCMS) called eZ Publish. Now there are plenty of open source WCMSs available on the market, some better than others. In fact you're currenly using one in the shape of VamosaGroups, which runs on Drupal. But back to eZ Systems...

eZ Systems are a large, mostly European-focused, WCMS vendor who do two things really well:

1. make the open source mindset and philosophy the heart of their offering
2. develop a business-model around their own GPL-ed open source CMS

They've done this so well that they've grown to an 80-strong workforce in 7 years across 5 countries. My point is this, gone are the days that meant that open source was either produced by hobby hackers or faceless projects such as the Apache Foundation. These days, a SME software vendor can go to market with an open source business model and compete effectively with the big boys (read closed source ECM vendors) not just on price or functionality, but on customer experience. Open source vendors such eZ Systems and Alfresco, make it easy to buy from them. There's no hiding behind secret price lists, hidden implementation costs, or expensive support. You get what you pay for, but more importantly you pay for what you want to buy.

Earlier this week I spent two and a half days in Southampton with one of Vamosa's partners, ActiveNavigation. I'd travelled down on the Sunday afternoon so I could spend sometime in this historical city. Not sure why I bothered. This once beautiful town has had its naval history ruined by modern apartment buildings, cinemas, and shopping centres along the sea front. I don't know what was going on Sunday, but it was more or less nothing. The city centre was more or less deserted as the pictures below attest to.

Thanks for the patience and allowing the downtime. VamosaGroups should be

• running a lot faster now
• use searchengine-friendly URLs
• and support trackbacks

Thanks once again.