Museum transparency and the IMA Dashboard – an interview with Rob Stein

Last year the Indianapolis Museum of Art launched their Dashboard – a visual display of various data about the museum and its activities. Updated regularly the Dashboard gives open public access to much data that would usually be buried deep in an annual report.

The ‘transparency’ that the Dashboard offers is remarkable – it not only makes that information available on an ‘almost live’ basis, most importantly it makes it ‘accessible’. Everything from the number of new artworks on view and website visitors to new plantings in the gardens and budget performance, many statistics are available, and many more can be drilled down and explored in greater depth.

I’ve been very interested in the project and how it might have impacted organisational change at the IMA. I conducted an interview with Rob Stein, Chief Information Officer at the IMA, about the project.

Rob Stein explains the genesis of the project;

“Our CEO, Maxwell Anderson, has been interested in the role of transparency in museums for quite a long time. He has also spent a lot of time thinking about what kinds of metrics museums can use to measure their performance against mission based statistics (see his paper ‘Metrics for success in Art Museums‘). So, the decision to investigate what it might look like to create an institutional dashboard that could inform both the staff and the public was certainly initiated by Max, and reflects the museum’s commitment to operating as transparently and openly as possible.

Our goal was to create a site that could be accessed and understood easily and that would be of general interest at one level, but that could also eventually support a depth of investigation into how this museum measures up to our mission and strategic priorities. We don’t see that there’s any reason to keep these things secret, and in fact believe that making it easy for the public to see how we’re performing will offer a great incentive to the staff to understand why this is important.

The visual design of the Dashboard has been instrumental in making ‘sense’ of the data and opening up access.

Since statistics can sometimes been seen as boring and not interesting to the general public, we tried to keep the site as visually engaging as possible. We took quite a bit of inspiration from the dashboard screen of the Google Analytics tool as well as from Apple’s general design aesthetic. Google’s Analytics tool is great in that it offers bits of relevant information in one screen (the dashboard) that are easily digested, but also provides the ability for users to dig deeper into statistics that interest them. I think we were able to accomplish this effectively by incorporating “teaser” modes for each statistics on the Dashboard.

We also make pretty liberal use of an underlying set of taxonomies to help in organizing the information. Each statistic can be tagged, or categorized against a pretty simple taxonomic structure. This allows us to group statistics by department, or topic, or to say which statistics need to be grouped together for some reason (i.e. 2007 end of year reporting) The navigational tabs at the top of the screen are generated on the fly from these underlying taxonomies, so that whenever we add a new statistic, or topic to the dashboard they are immediately available.

Each node of the dashboard currently [also] provides its own RSS feed that users can subscribe to, In addition, users can also subscribe to any topic or departmental page as well. So, for instance if our finanace committee wants to always pull in new statistics related to IMA’s finances they could subscribe to the RSS for the Finance Topic.

Likewise there have been some sensible choices in the data made available to the public. Rob explains the focus on ‘interestingness’ in the choice of what to present;

We polled the staff and asked them to solicit information or statistics that they felt would be interesting, and particularly focused on those statistics that mirrored an activity or priority that supported the museum’s mission. We intentionally chose a small set of these statistics to start with, and probably placed a bit more emphasis on those statistics than changed relatively frequently. Now we’re in the process of going back and expanded on those sets of statistics and opening up the Dashboard to items that maybe don’t change very often, but are of general interest (i.e. the square footage of gallery space, the acreage of the museum’s grounds, etc…)

Dashboards rely on timeliness and there are many challenges in integrating back-end systems to deliver cross-organisational data. I asked Rob how important ‘live data’ was to the objectives of the project, and some of the implications that this has had internally to the museum.

This question illustrates a pretty important philosophical nuance that we’ve been talking about internally at the IMA. Currently, most of the statistics shown on the Dashboard are updated by staff members from many different departments at the museum. Our thinking in this is that the staff member responsible for tracking any particular statistic should, as part of their normal workflow be required to report that statistic to the dashboard. The reasoning here, is that we feel that when reports are automated, it is easy for staff members to become disconnected to the tracking of the information. We want the dashboard to be useful and interesting to the public, but also a tool for the museum to use in tracking its own progress across time. By asking staff members to take on the responsibility for reporting this information, we’re sure that they are aware of the trends in performance.

The flip side of this coin is that this does open the door to having statistics that could be fudged to make them look better than they really are. Also, if the responsible staff member is away from the office on vacation or a trip, the statistics they are responsible for may lag in their updates.

Our implementation of the Dashboard is definitely a work in progress. Since we created the software and web design in house, we’ve planned all along that we would feel our way through some of these difficult issues, and make changes to the sytem as we got feedback from our audience and the community on how dashboards should be operated. For example, in the next few weeks, we will be launching a set of new Dashboard nodes that will completely automate the reporting of attendance. We have systems in place in the museum that make this possible, and now understand and feel comfortable that these systems are reporting their numbers accurately. We’ll be moving from updating these numbers by hand on a weekly basis to live data being reported to the web every 5 min.

We’ve also been toying around with the idea of producing some kind of integration between the Dashboard software that we’ve built and Crystal Reports and the Flash integration that they support. We feel like this might give us a way to better integrate the Dashboard framework that we’ve built with a typical way that business systems support automated reporting. In doing this however, we’d still want to address the staff’s connection with the data and it’s reporting, and we haven’t quite figured that one out yet.

Might Dashboards ever ‘replace’ annual reports in terms of general public access?

I’m not sure that Dashboards should ever replace annual reports. I think annual reports are sometimes criticized in that they are hard to digest and tedious to produce, and of ultimately limited value in some settings. This may actually have more to do with poor execution than a critique of the annual report as a medium for communicating about the state of an organization. Dashboards, by their nature are designed to be good at communicating small bits of information in a somewhat random order. Navigation can facilitate themes or grouping of statistics, but Dashboards will have a hard time supporting much of a narrative regarding the performance of an institution. Annual reports or other long format texts can do a much better job of this, but will probably always struggle to interrelate large sets of seemingly disparate statistics. Given those two thoughts, it seems that dashboards and annual reports are probably pretty good compliments to each other, and we should probably leverage that relationship better than we do today!

With all this data now being publicly available there would be some exciting possibilities for cross-institutional data sharing and analysis. Rob explains some tentative future plans for the project;

We’d love to open source this product at some point in the future and hope that it would encourage other institutions to take the transparency of their operations to the next level. In order to make that really successful we’re in need of some good partner institutions that could look at the existing system with a critical eye and provide insight and resource in determining how the Dashboard could be more broadly applicable for many different kinds of institutions.

We take software quality pretty seriously here, and feel like there are some projects that treat open sourcing their software as almost a dumping ground for stuff they’re done with. We think that casts open source software in a bad light, and is not helpful in communicating that there are a lot of open source projects that are of extremely high quality and value. We’re not underselling what it would take to package the Dashboard in a way that’s easy to use and install, and to providing at least some amount of support to a future community of users.

Thanks to Rob for taking the time to answer my questions and being so open in his responses.

Try out the Dashboard.