The Need for Metadata

Metadata has a bad name. People find it a necessary evil that they need to deal with when they're dealing with content. These days you can't just stick a file in folder. When it comes to content management systems, authors need provide a variety document and version history details before a document can be placed in the correct location and the right document lifecycle workflows can kick-in.

If its so bad, why do we need metadata ? Search engines do an excellent job of providing accurate, secure, and fast access to massive volumes of content held in many disparate stores. Isn't that enough ? What more would you want ? Unfortunately the world of content and unstructured information grows ever bigger, is more regulated, and becomes more important to organisations day-by-day, which means that single scenario of the knowledge worker using the search engine to access content doesn't describe accurately the world we live in anymore.

The world of keeping documents on file systems and emailing them to colleagues is fast disappearing. Governments are introducing new regulations on how we conduct business with each other, and this affects how emails, documents, and other unstructured content is created, accessed, maintained, and ultimately disposed of.

The large volumes of content we deal with means vendors such as IBM, Stellent etc. develop content and records management systems to allow organisations to implement these new regulations. The policies that run on top of these systems are driven by metadata. Metadata is required to decide where a document is to be stored or when documents should be disposed of. Its unavoidable, we need metadata, unless we build artificial intelligence into our records management systems and teach the AI how to read.

Metadata has a bad name, because organisations treat it as transactional data, like a unit price on an order line item.

Metadata is different from content and its different from transactional data.

Metadata describes content in a given context or series of contexts, some of these are user-generated, some machine-generated and potentially interpreted differently across consuming systems.

Metadata needs to changes over time, to reflect the changing contexts in which it is used.

Therefore we need to start to think about metadata in a new light and consider what some people are calling a metadata lifecycle model.