Saturday, 28 November 2009

The unknown approach to Database Publishing

What do you associate with the words Database Publishing? Heavy scripting, clumsy, old fashioned, hard to adopt, impossible to sustain over time, only for really big productions?

These would probably have been my answers had you asked me years ago. Now, on the other hand I’ve come to realise that there’s another approach to this. An approach that I fear not many are aware of. Without the knowledge of this approach you might unfortunately miss out on a lot of opportunities.

Before explaining the approach I’m talking about, I’d like to start by explaining the traditional approach to Database Publishing.

Traditional Database Publishing can basically be described as this:
  1. Export the data from the data source into a text file.
  2. Read the exported text with a custom written script or a commercial tool.
  3. Let the script or tool create the publication, page per page, formatting the text and creating the layout as the pages are built - one after the other.
This sounds fairly easy but there are a few hurdles to pass. First of all you are likely to encounter problems with the exported data. It probably doesn’t contain all the data you need like units, headings etc. Then there could be problems with structure of the data. The data might for instance come in another order then you’d expect. All these data “hurdles” need to be addressed and that’s usually handled by the script or the tool used.

Then there’re all the “hurdles” associated with changes, adjustments and reuse.

What do you do when the publication is generated and you find an error in the original data? Repeat the process from step 1 should be the obvious answer, but what if you had done several manual adjustments to the layout? Repeating from step 1 would mean re-doing all the manual adjustments all over again. If you repeat this cycle a couple of times... well, you know where I’m going with this, don’t you?

Another interesting question is what you do in the highly customised layout case? It’s not that uncommon that a layout have to look a certain way. A way that doesn’t accomodate data “flowing” very good.

In this case I’m sure most consultants and power users come to the rescue and offer to customise the script or tool to do exactly what you need. This might just solve your immediate problems but be aware. This is usually a good way of making sure your solution will become costly to maintain and very hard to reuse for other database publishing needs.

All these “hurdles” are the reasons why database publishing is associated with words like heavy scripting, clumsy, old fashioned, hard to adopt and impossible to sustain over time. The consequence is that database publishing is really only considered for really big productions where the manual approach isn’t a feasible choice.

Traditional database publishing like this can be described by one word - “push”. This of course comes from the approach where the data is being pushed onto to layout.

So, this approach obviously have limitations. What’s the other approach then? Well, if there’s a “push”, then there’s of course a “pull”.

Imagine a solution where the data is being pulled instead of pushed. What would it look like? What’s being pulled and who’s doing the pulling?

Of course it’s the data that’s being pulled and it’s the layout that’s doing the pulling. And since you are in control of the layout, you’re the one doing the pulling. In fact the idea is not that different from how you place an image in a layout.

You create the page, including the boxes and all data that isn’t to be found in the database. Then you pull the desired data from the database into the correct place in the layout and format it. Just like when placing images in the layout, the pulled data stays linked to the database. When the data in the database gets updated it will show in the layout and you can choose to update the data with a single click. Just like an image.

This works with pricing information, product text, headings, units and tables. It even works with images.

This “pull” approach gives you total control over the entire layout and it is very easy to adopt to different productions. It allows you to create entire catalogues as well as a single advertisement without having to alter the data source or the tool being used.

It’s also very easy to get started. It’s just a matter of opening a previous production and replace the static information with data pulled from the database. Then you have your linked layout and you can always be confident that your prices, product information etc will stay up to date. This should also affect the approval process in a very positive way, right? There’s no longer the same need to check information correctness.

Does this sound like magic? It really isn’t.
There are in fact several plugins to Adobe InDesign that makes exactly this possible. There’s LiveMerge from Cacidi, Smart Catalog from Woodwing, CCR from Ctrl Publishing and my personal favorite EasyCatalog from 65bit Software. I’m sure there are even more tools than these out there. If you know of any please mention them in the comments.

EasyCatalog presents the data in an Excel like panel. This allow you to restructure the data and format it in different ways, for instance add units. Inserting and linking data to the layout is easy. Just select the data in the panel and then click the little pen icon in the panels bottom left and the data will be inserted (or pulled) into the layout.

EasyCatalog also works with InDesign Librarys. This is very handy when creating for instance DM leaflets that are commonly built on modules. Create a Library item with linked data for each module type, then select the data row for a product in the Excel like panel, then drag & drop the module (Library item) onto the page. All corresponding data including images will be inserted and formated automatically.

I’ve used EasyCatalog to help reduce the production time of 12-16 page DM leaflets from over a week to 1-2 days.

Before you get too excited and start downloading the trial of one of these tools you should reflect over the following lessons I learned from helping customers implement this.

1. Does the database really contain all the data you need?
You may find that not all data you need exist in the database. If so, you need to find a way to add that data and that can unfortunately prove to be a rather cumbersome process in some cases. You might even need to consider implementing a Product Information Management System.

2. Is the data in the database correct?
A common problem you might discover is that some data, for instance product names, are incorrect. They might be the correct internal names but not the names used when marketing the product.

3. Is the data accessible?
If the database you want to access is a proprietary ERP-system you might have trouble accessing the data. If you can’t access the data directly via ODBC or similar, make sure the data can be exported into a text file or something similar.

4. Be prepared to change the way you work
Usually proof reading and approval are performed by other people than the people creating the publications. If the people doing the proof reading and approval are used to marking text adjustments on the physical proof, this have to change. They now have to make the changes themselves directly in the database instead.

5. Image database or a Digital Asset Management system?
If you want to include images into the database publishing process you need to have your images structured. They need to be tagged with, or named according to a unique identifier. Something that can be connected to a product in the exported data. If you have many images per product you also need a way of identifying which image should be used.

If this still seem tough, I will offer a last piece of advise. If you have a database solution serving the web, I’m willing to bet you can check many of above points right off away. Looking into the systems powering the web site is usually a good place to start.

To summarise. Database Publishing doesn’t have to involve heavy scripting and isn’t just something for the really big productions. There is another approach that not that many are aware of. The “pull” approach. With standard commercial plugins data can be linked to layouts just like images, making data population and data updates a very easy. This makes database publishing feasible for pretty much any publication containing product information.

The great benefit of all this is that you will be able to shorten production times, which in return will make it possible to either same money or find the time to create other publications. Perhaps all those targeted or personalised publications you are unable to create today?

Feel free to drop a comment if you have questions or need advise.
Good luck!

Tuesday, 10 November 2009

How to buy a system

So, you've decided to investe in a new system. Be it a Brand Asset Management System, Product Information Management System, Proofing System or any other system, how do you go about it?

The usual approach is to start writing a Software Requirements Specification. This is not an easy task. Often IT gets involved in the process which is good – in most cases anyway. The goal of the specification is to get a complete description of the behaviour of the desired system. Complete being the operative word here. That's what's making it so hard and why consultants like me often gets involved.

A completed Software Requirements Specification usually contains segments like this:

Functional Requirements:
F-01: The system must support single sign on
F-02: Users must be able to upload a single image
F-03: Users should be able to upload images in batches
F-04: Users must be able to search for images using categories

Once the specification is completed it's usually sent to different suppliers as a Request for Information or Proposal. Basically this is asking: ”Can your system do these things?

I've read and answered my fair share of these specifications, and I also confess to having written quite a few of them. As I said this is the usual approach. By learning the hard way I'm now convinced this is a bad way.

Specifications like this are:
- hard to write
- hard to read
- hard to answer
- incredibly boring! (which probably results in very few people actually reading them)

Besides, the only answers you are likely to get are ”yes”, ”no” or ”requires customising”. The answers won't tell you if you have to click two buttons, switch user, log back in and then click again to do what you want.

In this situation it's tempting to blame the supplier for not answering good enough, but frankly you haven't exactly made it easy for the supplier. The supplier have absolutely no idea of why you want to do these things and in what situation or context you plan to be doing it. The best they can do is assume.

So, is there an alternative?
First of all. I'm a fan of Use Case Modeling. If your IT-department is envolved they might be able to help by creating nice looking Use Case diagrams in UML that clearly illustrates ”who” can do ”what” with the system. UML diagrams are much better than the above listed requirements. But if you don't know UML there's also a third alternative.

Write scenarios or stories. Storytelling is more or less what every marketer does so this should be familiar turf. A scenario for a Brand Asset Management system could look something like this:

Meet Lisa. Lisa is a Marketing Assistant. Lisa has just received an email from a subsidiary that needs the latest photo of the CEO. This is the third email today asking for images and it's not even noon yet.
Lisa grunts a little but then heads to the ”System”, enters the search, finds the image and forwards it to the contact at the subsidiary. Lisa is really pleased by the fact that this entire process just took seconds to complete. She is also glad she didn't have to download the entire file to her local computer before sending it.

If you write scenarios like this, send them to suppliers, and – instead of asking ”Can your system do this?” – ask ”How does your system assist in this scenario” you'll get a much better answers that will help you evaluate different answers and choose the ”best fit” for your needs.

Thursday, 5 November 2009

3 things that will make you smarter

Three things that will make you smarter, wiser and more insightful:

  1. Allow yourself 20 minutes of quality time and watch Kevin Kelly talking about the next 5000 days of the web. This will definitely give you an idea of how you need to develop your products and marketing strategy for the future. 
  2. Start subscribing to the Radiolab Podcast. Listen to them on your way to work, when you're in the car or whenever really. Allow yourself to be amazed, intrigued and surprised!
  3. Add Seth Godin's blog to your daily reading list. His posts are short but always spot-on and relevant. His words of wisdom will sometimes give you new ideas and sometimes make you question the things you take for granted.

Tuesday, 3 November 2009

A DAM 2.0 user experience, please

Pretty much every Digital Asset Management system out there today have the features needed to ingest, annotate and store digital assets. Most systems also provide tons of different ways to add metadata and permissions. No doubt, all available systems provide asset owners the functionality needed to manage their digital assets in one way or the other. But today that's not enough. It's not just about the asset owner any longer.

This is what a typical DAM system presents to a regular user:

The user can either enter one or many search words into a search field or browse using some sort of taxonomy tree. 

This is however not Web 2.0 experience.

Two of the characteristics of Web 2.0 are rich user experience and user participation. Take a look at Amazon, iTunes Music Store och Wikipedia. Users today expect to be guided to other relevant material based on their searches. They also expect to be able to contribute by rating and commenting everything and everywhere because they've learned that this is a great way of adding value to other users. For instance, when I search for a song in iTunes I always look for user reviews and other recommended songs. These reviews and recommendations lets me discover other songs I otherwise might have missed. Let us call these reviews and recommendations ”filters” for future discussions.

Did you know that every single song in iTunes Music Store has in fact been sold at least once? That's really interesting because it wouldn't have happend without these filters. Have you ever looked at the thousands of assets managed by your DAM system? How many of them have been reused at least once? This is actually a very relevant question since the arguments for implementing a DAM system almost always include increased asset utilization and cost avoidance – for instance not having to recreate assets you can't find. Well, maybe you should take another look into your DAM system? If your assets aren't being reused, how can you tell that your DAM system is fulfilling it's purpose?

So, filters are important as they encourage users to venture down the long tail of assets in the system. This leads to increased asset utilization, but it's also important for another reason. It's simply great fun! One of the big challenges with implementing a DAM system is getting users to actually use the system. A key factor in this is the user interface. If it's perceived intuitive and easy to use, less training is required. If it also allows them to contribute and communicate with other users through comments, it could almost go viral.

Unfortunately most DAM systems today don't allow user contribution in this way. Some even have licensing models that prohibit it all together. In this aspect DAM systems are still very web 1.0, i.e. only the owner can modify the content. It's time to invite the end user to the table.

Okay, what do we need?

Step 1: Allow user authoring
I suggest:
  • Allow users to rate assets and have the system display the number of votes and average rating
  • Allow users to comment assets as well as comment other's comments
  • Allow users to tag assets by adding keywords. (Now I expect some of you might object? Allowing users to tag without guidelines and predefined lists of approved keywords? I say, why not? This could actually help reducing one of the most daunting tasks in DAM. Besides it's not new. In fact it even has a name - folksonomy)
Step 2: Add filter features
Now, what would these filters need to look like?
Imagine yourself searching for ”flower” in your DAM system. The system performs the search and displays the resulting 53 assets. Now, as any normal consumer you automatically ask yourself what the difference is between all these 53 assets? How can I group them? In what way can organise them to help me narrow my search? This is where filters should help you.

One type of filter are filters based on metadata. They can help you organise the asset by type, date and size. In fact the beta of Elvis seem to do this i a very nice way. In fact the best I've seen so far.

Elvis interface. Notice how you can filter by limiting the search result to kind. Also notice how the tags available to the images in the result are presented. The tag mexico is bigger because there are more images associated with that tag. Clicking on mexico narrows the search result to only those tagged with mexico

The other type of filter are filters based on user contributions. These are the ones you find in Amazon, iTunes etc. and the ones you typically don't find in most DAM systems. These filters should help you identify the assets recommended by the management, the asset not yet tagged and therefore possibly unique, the assets other users who entered this query liked etc.

Step 3: Make it all search- and sortable. 
This should be the easy part. Make it possible to search all user interactions. Search for all images rated by Johan, commented by Lisa etc. I would also like to search on user actions. It's perhaps a bit trickier to implement but it would be great to be able to search for images viewed by Johan, downloaded this week, most popular this week etc.

Step 4: Facilitate sharing
Almost every blog out there has functionality to make it easy for users to share a post. With sites like and it's very easy to implement. Of course it should be just as easy to share an asset in the DAM system via Twitter, Facebook, e-mail etc.

Step 5: Create a stunning and super simple user interface
This is the really tricky part. I'd suggest hiring a highly skilled interface designer. If you don't agree take a look at this - Why Apple & Google win - and your company doesn't.
How many of the DAM systems out there today look like an Apple or Google product?

Last but not least. Why is this important?
The single most important reason for doing all this is that it will make DAM implementations much easier. It will reduce the need for user training and thus speed up the entire implementation process. It will also make it fun and it will ultimately help you fulfilling your goals faster.

What's the next step?
If you disagree or think I left something out, please add value to this by submitting your comments to this post.
If you agree, please forward this to the person responsible for your DAM system or even to your DAM vendor. Help spreading the word.