Category: User behaviour

Categories
Conferences and event reports User behaviour User experience

Unexpected lessons with technology in museums #1

#advice

A photo posted by Bim Ricketson (@bim_cd) on

Early in June I was back in Sydney presenting one of the keynotes [slides] at Remix, a cross-sector/cross-industry event I also spoke at last year when it passed through New York. The keynote was based on a very long paper that Aaron Cope and I co-authored for Museums and the Web in April (a much shorter ‘clean 7″ radio edit’ is forthcoming in Curator too!).

There’s a couple of key bits that seem to have resonated particularly well and bear elaboration. So this is one of a series of posts that will do that elaboration.

#1 Have an opinion about the visitor behaviour that you want, then design explicitly for it

It sounds so benign and obvious – of course your museum has an opinion about how visitors should behave when they visit. Usually this is couched in “no this, no that” – or subtly in the social cues emanating from the architecture, the dress and attitudes of staff, and the behaviour of other visitors. There’s a whole slew of problems with ‘museum-going culture’ – and it is important to acknowledge the bountiful existing literature on who is already excluded or included in the ‘traditional museum’.

Writing about the ‘omg, those new museum visitors are doing what? photography! selfies!’ moral panic of 2013, Ed Rodley’s summary and discussion is worth re-reading;

“The solution seems to me to be to establish a new contract, and actually state it, instead of hoping that visitors will infer intent. I’d love to see museums generate explicit policies that state what the museum encourage, allows, forbids, and why.”

When we were thinking about Cooper Hewitt in the early days, the MONA experience was very much top of mind. The lack of object labels, the O – it all expressed a firm opinion about how owner, David Walsh, wanted you to experience his museum. As it turns out, even if you found this annoying, you admired the bravado – and it has and continues to be a huge, popular success.

Before the galleries were tackled, Cooper Hewitt’s online collection began to develop a very particular style – an opinion that carried through to the architecture of the website, and the linguistic choices on the front-end. That ended up influencing the entire ‘voice’ of the museum online – some of which you can see in the 2014 website redesign.

In the galleries and exhibitions we knew what we didn’t want. We didn’t want people staring at their own devices – they made the choice to come to the museum, so we wanted them to be ‘present’ – after all, everything they saw they could easily get access to later on online, and museum going should be a full body experience, right?

Amy Liprot writes about a visit to legendary Berghain club in Berlin;

On the way in, the door staff put stickers over the camera on my phone. There is an open minded attitude here to nudity, drugs and sex, yet taking a photo will get you thrown out. It’s highly refreshing that everyone’s not filming stuff. It’s hard for internet kids, by which I mean it’s hard for me, to have an unphotographed experience but I am really here, more than ever. This is not a place for observers but for active participants.

Whilst we did want active participants, we wouldn’t go that far – but we did think, and this is important, about the impact of everybody engaging in whatever it was we came up with.

Everyone’s usage (or non-usage) would impact the overall atmosphere of the gallery. If it was a mobile App, then how would it feel to have everyone in the museum using it at once? If it wasn’t an App but something else, then what would that feel like for visitors as a collective mass.

We knew – from the experience of MONA and of audio/media guides at other museums – that it was likely a choice between 90% take-up or <10% take-up with a chasm of un-met user frustrations in-between. So thinking about maximal usage was an important design consideration once we aimed for ubiquity. As it turned out, The Pen has had some interesting impacts. Usage has been pretty much ubiquitous with over 90% of visitors using it, and using it a lot [details over at Cooper Hewitt Labs]. There’s several years’ worth of research topics for enterprising museum studies and audience researchers in the data too!

Because it is very visible to others – a large-ish un-pocketable size, but has no screen – visitors seem willing to help each other when they see people having difficulties or using it ‘wrongly’. People don’t tend to do this sort of ‘social helping’ with mobile Apps because there’s nothing to indicate that the other person is actually using the ‘official App’ or just texting their friends.

As for photography, yes, that’s very much welcomed at Cooper Hewitt but you don’t see cameras out anywhere near as much as in nearby museums.

And once a behaviour becomes normalised, it starts to change expectations elsewhere.

—

In the next instalment I’ll talk about some lessons around ‘internal literacy’.

Don’t forget, these are ‘riffs’ based on the very long paper that Aaron Cope and I co-authored for Museums and the Web in April. If you’ve got a spare half hour then there is a lot of detail in that paper.

Categories
User behaviour User experience

On ‘farewellers’ and exit marketing

So ridiculously busy right now that there is little time to blog. But stay tuned for some cool stuff over at the Labs shortly.

But here’s a the first of a few quick thoughts on some topics bouncing around the blogosphere.

This week Nina Simon wrote about her ideas of having a staff of ‘goodbyers’ instead of ‘greeters’ in order to better build continuing engagement with visitors. She writes –

We realized from this discussion that we have a huge missed opportunity when people are leaving the museum. On their way in, they are excited, curious, ready to engage. They are not ready to hear about membership or take a newsletter about what’s coming up next time. They bolt right past those tables to the “good stuff.” But at the end, they’ve had a great time, and they want a takeaway from the experience. They WANT to join the email list. If we’re smart, we should be developing a takeaway that both memorializes the visit and leads them to another. In other words, we should be giving them a string for their new pearl.

This reminded me a lot of the efforts we’d go to back in the early 90s putting on all night parties. Before this was a task given to ‘street teams’ (no one had commercialised enough to hire people to do the least exciting tasks), you’d take a stack of flyers to parties at the very end of the night just as the dawn anthems were blasting through the bassbins and start giving them out as people exited. Others would go and plaster the windscreens of parked cars to similar effect. No one would ever give out flyers early on in the party – they’d get forgotten, sweaty, destroyed, or just ‘repurposed’. It was all about ‘exit marketing’ – and it was an important part of building bonds within the subculture. Flyers for the next month’s worth of warehouse parties made for a strong encouragement to ‘stay involved’ – especially as most people would be returning to their ‘ordinary lives’ during the week, saving their living for the weekends. It gave newcomers a sense that this wasn’t just a fleeting ‘temporary autonomous zone‘ but something they could regularly return to, and for the hardcore flyers and their effective distribution became core ‘subcultural media’. I’d argue that they were more effective than the more scattergun street press advertising, and definitely more successful than ‘record shop drops’.

Now museums rarely ignite the sort of passion that subcultures do. Perhaps they should, but that’s unlikely to happen given the age demographics. But there’s plenty to be had in Nina’s idea – the farewelling experience is likely to be the only opportunity to remind visitors that museum visits need not be a ‘one-off occurrence’ or a ‘once a year’ activity, but an essential part of their cultural calendars.

And of course, ‘farewelling’ behaviours are exactly the sort of things that you’d be hoping the staff in your ‘well placed gift shop‘ are doing as just good business.

Categories
Mobile User behaviour

Chickens, eggs & QR codes

Adam Greenfield at Urbanscale just posted some interesting research his team has been doing in NYC on the citizen familiarity of QR codes.

This is especially timely as QR codes are getting a lot of interest (finally) from the cultural sector. The Powerhouse Museum in Sydney has been doing QR codes for a few years – first failing – but now perhaps getting good traction with them now that the code scanner is built into the exhibition catalogue App. Shelley Bernstein’s team at the Brooklyn Museum have also been rolling them out. And Wikipedia’s been promoting the nifty language ‘auto-detect’ QR codes that Derby Museum & Art Gallery have developed (QRpedia).

But there are still very valid concerns about the appropriateness of them – especially now that visual recognition is coming along rapidly (see Google Goggles at the Getty) and maybe even NFC might gain traction (see Museum of London’s Nokia trial). QR codes feel very much like a short term intermediate solution that isn’t quite right.

Here’s Greenfield:

While general awareness of the codes was frankly rather higher than we’d expected, and a majority of our respondents knew more or less what they were for, very few … were successfully able to use QR codes to resolve a URL, even when coached by a knowledgeable researcher.

…

A strong theme that emerged — which we certainly found entirely unsurprising, but which ought to give genuine pause to the cleverer sort of marketers — is that, even where respondents displayed sufficient awareness and understanding of QR codes to make use of them, virtually no one expressed any interest in actually doing so. As one of our respondents put it, “I’ve already seen the ad, and now I’m going to spend my data plan on watching your commercial? No thanks.”

These findings mirror the anecdotal experience most of us have had with QRs ourselves. The value proposition just isn’t obvious – and the amount of scaffolding required to encourage scanning can, in museums, sometimes take up as much visual space as the content that ends up being displayed (especially for object labels).

Is this just a chicken and egg situation? I’m not sure.

Greenfield’s initial findings do show that even when there is awareness there isn’t interest. And, I’d add, even when there is interest, museums need to be especially careful to consider what visitors actually want/expect to see when they scan vs what museums are able to show/tell. This is a crucial distinction that is often missed in discussions of in-gallery content delivery.

Categories
Mobile User behaviour

Early MoveMe wi-fi heat maps from Love Lace exhibition

Several months ago I announced that the Powerhouse Museum was a partner in the MoveMe pilot project funded under NSW Government’s Collaborative Solutions Program.

We’ve been working with Ramp, MOB Labs, ShopperTrak and Smarttrack RFID to deploy the pilot in our recent Love Lace exhibition.

This exhibition is ideal for trialling location aware content delivery because it is already kitted out with public wi-fi and we have the cross platform iOS and Android free exhibition App. Even better, the exhibition uses QR codes and the QR code reader in the exhibition App which gives the pilot project a great baseline to compare usage against.

While we don’t yet have the location aware content delivery working – that will come in a future version of the exhibition App – we have started to get access to wi-fi tracking data using the ShopperTrak system. As explained by Christopher Ainsley & Julian Bickersteth in their paper for Museums & the Web earlier this year, the ShopperTrak system is already used to create heatmaps and visitor journeys through shopping centres (or ‘malls’ as some readers might describe them).

The first data has started to emerge from the system and it is already very interesting.

Here’s a dwell time heat map that shows the areas of the exhibition where the wi-fi enabled devices (presumably carried by visitors) spend the longest time. This shows data from Sunday Oct 30 and 226 tracked devices.


(click for larger version)

A couple of important caveats.

Whilst the sample sizes are unexpectedly quite high (largely because the wi-fi tracking doesn’t require an actual connection to our wi-fi network, just that it is switched on on the device/phone), the sample rate at which devices are ‘pinged’ is quite low. iOS devices, for example, are only pinged every 2 minutes and so the resolution is very low – unless they are actively connected to our wi-fi network for the exhibition. This means that if an iOS device has wi-fi switched on but they aren’t using our Love Lace App and not connected to the exhibition wi-fi and they spend 10 minutes walking around the gallery their device will be counted in a maximum of 5 locations. Of course this can be offset by the volume of tracked devices (which almost certainly exceeds that of other manual people counting methods employed by traditional audience research).

What is interesting about the data is that it pretty much mirrors the distribution of the QR code usage I blogged about earlier. Unsurprisingly the longer dwell times are where the sit-down video experience is.

Categories
Collection databases Interviews User behaviour

“Do curators dream of electric collection records?” Exploring how the Powerhouse online collection is used

As one of the first of a ‘new style’ of museum online collections, launching several internet generations ago in 2006, the Powerhouse Museum’s collection database has been undergoing a rethink in recent times. Five years is a very long time on the web and not only has the landscape of online museum collections radically changed, but so to has the way researchers, including curators, use these online collections as part of their own research practices.

Digging through five years of data has revealed a number of key patterns in usage, which when combined with user research paints a very different picture of the value and usefulness of online collections. Susan Cairns, a doctoral candidate at the University of Newcastle, has been working with us to trawl through oodles of data, and interviewing users to help us think about how the next iteration of an online museum collection might need to look like.

I asked Susan a number of questions about what she’s been discovering.

F&N – You’ve been looking over the last few years of data for the Powerhouse’s collection database. Can you tell me about the different types of users you’ve identified?

Based on the Google Analytics, there seem to be four main types of OPAC users. I’ve given each of them a nickname, in order to better identify them.

The first group is the FAMILIARS, composed of people who access the OPAC intentionally. FAMILIARS know of the collection through either experience (having used the online collection previously, or from visiting the museum), or via reputation (ie GLAM professionals, researchers or amateur collectors). FAMILIARS come to OPAC with the highest level of expectations and have the most invested in the experience. Trust and authority are hugely important for the people in this segment.

The second group, I’ve called the SEEKERS. Like FAMILIARS, SEEKERS are driven by a desire for information they can trust. However, unlike FAMILIARS, SEEKERS do not yet know about the museum and/or its collection. This group includes people who are new to collecting communities, or student researchers etc. If they find what they are looking for on the OPAC, SEEKERS have the potential to become FAMILIARS.

The final group for whom authority and trust in information is important are the UTILISERS. These visitors, primarily education users (like school students), have specific and particular research needs, which are externally defined (ie they might be looking for answers to set questions). This group is task-oriented.

The last group that comes to the OPAC is the WANDERERS. These are casual browsers who seek fast and convenient information, but don’t necessarily need depth in their answers. Seb once nicknamed them “pub trivia” users, and that seems pretty apt.


F&N – What sort of proportions do each of these make up?


By far the greatest number of OPAC visitors are WANDERERS. More than 80% of all OPAC users – whether in a two-year period, or a six-month timeframe – visited the collection online once. Obviously not all of these will be WANDERERS, but a significant proportion of OPAC users are clearly coming to meet short-term information needs.

At the opposite end of the scale, around 5% of OPAC users visited the collection five times or more during the last six months. These visitors have the most invested in the current OPAC, having spent time learning to negotiate it.


F&N – Have these users changed over time? (As other collections have come online etc)


The actual make up over time doesn’t seem to have changed that much, although the numbers of visitors dropped a little after a peak in early 2010.

Having said that, there are seasonal trends in the users. The search terms that UTILISERS often use to find the collections (such as “gold license”) are more popular during the school year than at other times. Similarly search terms go through peaks, depending on media interest, such as a high number of searchers who come to the OPAC looking for Australian media personality Claudia Chan Shaw, whose dress is in the collection.

Some search terms are just weird. One of the most popular search terms ever was “blue fur felt” which skyrocketed to popularity in January – July 2010, but has not been used to bring visitors to the OPAC since.

F&N – Are overseas users different from Australian ones?


During the last six months, the OPAC actually had more international users than domestic ones, with the top ten international countries visitors coming from the USA, UK, Canada, New Zealand, India, Germany, France, Netherlands and Philippines. The search terms that lead international users to the OPAC are very different from those within Australia. After all, many of the most searched for items are that link up with the school curriculum, and that is very Australia-specific. These items also make up a significant proportion of the most-looked-at references.

The search terms overseas users to access the collection are often far more specific – such as particular clock brands etc, which would indicate a higher proportion of amateur collectors (SEEKERS and FAMILIARS) than WANDERERS.

Australian users spend longer on the site, and have a far lower bounce rate, so once on site they engage more.

F&N- You’ve been speaking to our curators about how they use ours and others collection databases. What are some of the things you’ve learned from this?

Talking to the curators has been absolutely fascinating. Every single curator that I have spoken to has his or her own ways of researching and gathering collection information. Some curators rely heavily on books, while others spend a significant amount of time conducting face-to-face interviews. Others use websites like Trove, or conduct community consultation online, using wikis and blogs. However, every researcher utilises Google and the Web in some way in their search for information.

No matter how a curator conducts collection research however, all are looking for two main types of information. The first is the broad contextual information for an object that places it into an historical and social framing. This includes the broader history or biography of the creator or manufacturer, and information on the social period in which it is or was used.

The second type of information is specific to the object itself, and includes information about maker’s marks, the object’s history (including provenance, such as how, when and why it came into the collection, why it was owned and used), and any stories that relate specifically to the object.

In order to find this information however, very few of our curators use museum collection databases – even those curators who conduct a significant amount of their research online. The reasons for this varied, but emerging themes included a difficulty navigating online collections (once it could be located on the institution website in the first place), a sense of frustration at being unable to find relevant information/objects, and most important, a lack of trust in online collection databases.

Not one curator that I spoke to trusted either our own OPAC or other online collections as a resource that could provide complete and authoritative information. Where a number of curators did find online collections useful however, was in providing immediate access to images of objects and to get a sense of whether another institution held objects that might be important to their own search. Knowledge about what was in a collection was useful, but not necessarily the collection knowledge that was included in the online record.

A number of curators did use our own OPAC to see what information was being communicated to the public, and to answer public enquiries. However, it was very clear that there are ongoing issues with trust and authority.

Two things that did increase trust for curators however were good quality images (through which they could get a visual sense of the object), and PDFs of original documents. Curators trust that which they can see themselves. For most curators, their expertise is such that they will have an intuitive sense when information they come across is likely to be correct.

—

Following Susan’s initial work we started looking at the SEEKERS in more detail. Why were they coming to the site? And, more importantly, were they satisfied with what they found?

We’ve had a pop up survey running for the last two months – again using Kiss Insights – and the numbers have started coming in.

In order to survey only the SEEKERS we have set the survey to only show to visitors who’ve arrived via organic search, have visited at least three pages, and, obviously, are in the museum’s online collection. The survey, thus, has quite a limited reach and has been triggered by only 3900 visitors in the time – and has been completed by 229 respondents.

It is somewhat heartening to find that the largest subgroup of Seekers – those doing ‘amateur research, hobbyist and collectors’ – feel the content they find is ‘good’, and that the lowest positive ratings are for the ‘other’ group. This is especially interesting if we look by object and see which object records are being rated as ‘poor’. Here we find a mix of well documented (at least according to us) and very scantily documented (no image, metadata last copied from a paper stock book entry in the 1980s).

Once we get to a critical mass of respondents – 1000 or more – in this group we should have some more actionable findings. Then we move on to looking at the the other groupings.

Categories
User behaviour Web metrics

Let’s Get Real report from Culture24 now available

Over in the UK right now Culture 24 are launching a report I worked on with them and many of the major cultural institutions in the UK. Coming from a need amongst web/digital people to find better ways of measuring the effectiveness of their work in the sector, the report – Let’s Get Real – pulls together analytics data from 3 years of activities online and in social media and makes a number of recommendations that are aimed at kickstarting, in the words of Culture24 Director, Jane Finnis, “a dramatic shift in the way we plan, invest and collaborate on the development of both the current and next generation digital cultural activities”.

The inability to effectively communicate the connection between delivering the institutional mission and digital projects is an ongoing concern to everyone working in museums. And at a time when there are increasing calls for museums to take roles that are more akin to broadcasters and publishers in the digital space, yet the majority of internal and external stakeholder value is still perceived as coming from visits to exhibitions and buildings, there is a pressing need to keep thinking about the ways digital projects report success (or otherwise!).

From my perspective, working with this diverse group of institutions was a lot of fun and very illuminating. It helped consolidate much of my thinking about the state of digital projects in the cultural sector and the long road ahead to really transform the way, particularly museums (less so the performing arts), use and adequately resource digital in their institutions. At the same time there were many unexpected surprises – the very different geographies of online visitors between institutions, and the comparatively low impact of social media in terms of website traffic, even for particularly well-promoted campaigns were revealing. The social media work by Rachel Clements also demonstrated that the easy option – reporting the numbers – greatly undersells the value of social media. The alternative, qualitative analysis, is much harder and requires more time and an understanding of why you are active in social media in the first place.

Have a read of the report (PDF) and see what you think.

For those involved in the project there was a lot more than number crunching – there were some amazingly productive working sessions and meetups – and the launch conference that is taking place right now in Bristol (check the #C24LGR hastag conversations!). In many ways the report captures only a fragment of the ‘value’ of the project as a whole.

Categories
Mobile User behaviour User experience

More on mobile tech impacts in museums (extended Mashable remix)

There’s a nice introductory piece today that features some of the recent Powerhouse Museum work in Mashable. It is a broad overview piece of how the Smithsonian, the NY Museum of Jewish Heritage and the Powerhouse have been utilising mobile technologies in galleries and exhibitions.

Reading some of the comments and picking up on some of the chatter on Twitter I thought it might be valuable to include two of the Q&A from the journalist that didn’t make the cut in the final story. They add a little more context and introduce more complexity into the issue – probably less interesting for non-museum people but useful to those deeply engaged in the field.

Q – How are you measuring the effectiveness of the technology you’ve deployed? Downloads? Data capture? Usage stats? I noticed you are going to put in moveME wifi triangulation system. What will the data from this tell you – you had mentioned in a post dwell time and loves but how will you put those findings to use? (Why are you doing this?)

We’re really interested in changing the physical design of our galleries so that they are able to deliver better experiences and tell more effective stories to and with our visitors. Once a visitor carries a fully searchable encyclopedia in their pocket (not too mention access to all our collection including the objects not on display), the whole idea of a ‘museum’ and how it could and should be designed, changes.

The ‘effectiveness’ of technologies has a number of different facets –

1. We look at raw usage data – downloads, views, interactions in order to redesign and iterate new versions of the technology itself.

2. Then we look at how visitors are using it both individually as as groups through observation and also data collection. This helps us to think about the social impact of our technologies in the galleries. For example, are our mobile apps meaning that families visiting together are talking to each other less than before? (a possibly negative outcome!)

3. We also look at the aggregate usage data to help us think about what content is being accessed (and what is being ignored) and then follow up with qualitative research to understand why. This, over time, helps us better understand which objects, for example, visitors are interested in finding out more about, and which, perhaps need a little more prompting.

4. Finally, and holistically, we aim to bring all this data together to better inform the spatial layout of galleries, and also the ancillary services such as education kits for teachers or curator-guided tours, that might further enhance a visit.

As we move from 1 to 4 the impact and time taken gets longer and longer obviously – and impacts much more broadly on the museum and its various operations.

Q – Where do you think things are going in terms of digital tech in your museum and in museums in general?

At the Powerhouse we are certainly getting far more strategic in our deployments rather than being seduced by novelty. This has been largely possibly because of the way digital has evolved at the museum with significant internal capacity and on-staff developers, digital producers, and strategy.

Broadly in the museum world we are seeing much higher volumes of technologies deployed – Google Goggles at the Getty, NFC at the Museum of London, AR at the Stedelijk, touch-tables everywhere – and I expect that over the next decade we will see the very idea of a ‘digital team’ or ‘digital unit’ or even ‘CTO’ at a museum as quaint. Simply because the very definition of a museum will be, itself, ‘digital’ and cross-platform.

Categories
Mobile QR codes User behaviour

Early App and QR code scanning data from Love Lace exhibition

I promised updates on the data coming from the QR code implementation in the Love Lace exhibition so here are the results of the last 4 weeks since opening.

Already we’ve released updates to both the iOS and Android versions of the Love Lace App. Perhaps surprisingly it has been the Android App that has given us the most trouble. Carlos has been troubleshooting various Android devices and OS versions to make the QR code scanning work properly – something that has been made much easier on iOS because of the consistency of hardware and lockdown of other apps. Now, though both are humming along nicely.

In terms of downloads we’ve had 572 iOS and 165 Androids. And using Flurry we’ve tracked 3,126 sessions on iOS and 502 on Android.

But let’s jump to the meaty data.

When we designed this App the QR code scanning tool was built in to try to maximise the use of QR code scans in the exhibition. Of course users could still just browse the scrolling list of objects and artists if they wished, but we hoped to get the QR scanning up to a reasonably good level by reducing user friction.

Looking only at the iOS figures we can see that browsing is by far the preferred behaviour although we haven’t segmented this by location. Obviously the QR code scanning only works when the visitor is in the gallery and outside of the gallery any App use would involve the scrolling browser only.

233 items (objects and artist records) have been viewed a total of 6933 times using the scrolling interface.

The QR code scanner has had 844 scans including 45 failed scans and 17 non-exhibition codes. Many objects have not been scanned at all.

Where this becomes interesting at this early stage is when we overlay the scans on the exhibition floor plan.


(click to open this at full size in a new window/tab – 457kb)

Visitors enter this gallery space from the bottom left and then complete a circuit counter clockwise. The triangular grey area in the very bottom left is the exhibition title wall that has signed promoting the App and the free in-gallery wifi.

Not unexpectedly the first hemisphere of Room 1 followed by Room 2 attract the most scans. However after that things become interesting.

What is striking about the overlay is that the most popular object (Meghan Price’s Habitat Wave) is near the end of the circuit of this part of the gallery in Room 8 and this is a rare outlier, being surrounded by almost entirely unscanned objects. Similarly Room 6, full of smaller objects, has a cluster of scanned objects but these are comparatively low numbers.

The cluster at the top of Room 10 are a set of five QR codes linking to the Inter Lace microdoumentaries that are projected in a remixed form in this space. Visitors dwell for significant time in this area but from the low figures would not seem to be aware of the full versions of these documentaries that lie in wait on YouTube.

Next?

In the next few weeks we will be rolling out a newer version of the App which will incorporate both these documentary videos as well as the ability to ‘love’ objects and share them more easily. We will be able to compare this data with the scan and view data and see if there are any correlations. Then, in about six weeks time the moveME wifi triangulation system will also be integrated allowing us to overlay and correlate dwell times in the space against ‘actions’ such as ‘love’ or scanning.

Stay tuned for the results of that.

Categories
User behaviour User experience

Prototyping moveME – a location-aware indoor mobile App with tracking

Last week at CeBIT the announcement came through that a project that has been under wraps for a little while now received NSW Government funding to move ahead. It is a collaborative project bringing together commercial partners with the Museum being used, as our Director puts it in the media release, “to directly support the NSW technology industry by being a ‘living laboratory’ for the development of this product”.

A consortium formed by Smarttrack RFID, RAMP RFID, MOB and the Powerhouse Museum is delighted to be selected as one of the first recipients of funding from the NSW Government under the Collaborative Solutions program. Announced on Tuesday 31 May, the NSW Government’s Collaborative Solutions programme aims to help build the digital economy in NSW.

And although there’s two companies involved with RFID in their names, this project is not about RFID at all – but instead is developing and trialling a mobile platform which, combined with indoor location awareness, is able to deliver customised content to visitors and also deliver valuable spatial analytics data back to the museum to assist with future exhibition design and spatial configuration.

Julian Bickersteth and Christopher Ainsley delivered a paper, Mobile Phones & Visitor Tracking, at Museums and the Web 2011 that outlined the broad premise of the project in April.

Discussing the use of tracking applications in the retail sector they wrote,

The museum sector is small and consequently does not have the resources to make use of this opportunity. However, significant components of museum operations have synergies with the retail sector, a part of the economy with deeper pockets for exploring new technologies. The museum sector has a history of piggy backing on the technological developments of its retail cousins, whether in the overt area of streamlining their own retail operations (both in the museum shop and on-line) or more subtly in using retail counting systems to accurately count museum visitors.

Shopping malls in particular share many physical characteristics with museums. They are both likely to be large masonry structures with a limited number of entrances, to contain a series of retail or exhibition spaces along with catering areas, joined by large open spaces. Both shopping mall and museum operators want to know where the more and less popular areas are located, what the dominant paths followed by visitors are, how long they spend in catering outlets and retail stores, and how long they spend in the mall/museum as a whole.

For the retail mall operators (known in the business as RAMs or Retail Asset Managers), critical to their thinking is adjacencies and synergies of the retail and catering mix so as to maximise rental income through more intelligent leasing. This in turn allows them to charge top dollar for shops in prime positions, and also control who gets a lease in the first place.

I sent Julian a number of questions about the project to expand on.

F&N – How did moveMe come about?

MoveMe started as visitor counting project which then expanded into visitor tracking. Visitor tracking works best if the visitors are using an App (giving them a reason to turn their devices on etc), hence the addition of the App. But then the packaging of this offering to include location specific content delivery and way finding clearly became a much more interesting space to be in, hence the evolution into moveMe.

F&N – How effective has this technology been in the shopping mall environment? What sorts of business decision making has this enable or improved?

This is still being trialled, but where it has in US department stores and super-hardware stores, it clearly has the ability to help staffing and also response to promotions. Since both these areas are high cost items, it is data they are very keen to get their hands on. [There’s some expansion on this in the MW paper]

F&N – You’ve seen a lot of mobile projects around the world. How does this differ in potential from what you’ve seen at AMNH and, locally, MONA?

Both AMNH and MONA cost absolutely heaps to design and install. Our solution is going to be much cheaper to deliver because it uses a different technology for tracking. Also it will be designed as a generic rather than a bespoke solution, so can be provided over the Internet for museums to locally self install and use.

F&N – What do you hope will be the key benefits of this system that is being prototyped?

For the visitor, focused delivery of far wider information than they could ever get from a label, plus the removal of having to squint at and possibly queue to see a label before they know what you are looking at. Also way finding. This is a critical issue in any big museum, which actually puts people off exploring as they might get lost. For the museum the ability to provide access to much more information plus also understand visitor patterns and behaviour in a way which up to now has just not been possible in a systematic and regular way

—

The prototype will be bundled into a cross-platform exhibition App that we are building in-house for the upcoming Love Lace – Powerhouse International Lace Award exhibition that launches with Sydney Design 2011. This will provide the first access to the technologies involved and also demonstrate how it can be potentially plugged into existing mobile tour platforms if required (as well as provide a full service for those without an existing App).

We will keep you posted on the project as it develops.

Categories
Interviews Social media User behaviour

Interview with Mia Ridge on museum metadata games

museum games logo

Mia Ridge is the lead developer at the Science Museum in London. She approached us in 2010 to use our collection database in her Masters research project which looks at the impact of different interfaces in museum collection-related ‘games’. Her research project is up and running at Museumgam.es where you can partake in a variety of different collection description activities.

We’ve had tagging on our collection database since 2006 and the results have, after an initial phase of interest, been quite mixed. During 2011 we’re rebuilding the entire collection database from the ground up and we’ve been rethinking the whole idea of tagging and its value in both metadata enhancement and community building.

I am particularly excited by Mia’s research because it looks explicitly ways of enhancing the opportunities for metadata enhancement of the ‘least interesting’ objects in online museum collections – the ones that have minimal documentation, never get put out on public display, have unknown provenance. These objects make up the vast bulk of the collections of museums like the Science Museum and the Powerhouse, and whilst sometimes they connect online with family historians or specialist communities, they do require a certain amount of basic documentation in order to do so. Similarly, being at the far end of the long tail they don’t generate enough views and engagement to be able to effectively ‘validate’ crowdsourced contributions.

I’m hoping we can use Mia’s findings to help us design better minigames in our new collection database, and I’m also hoping others, especially those outside of the museum community, will use her findings to build better games with our collection API as well as those of other museums.

Mia answered some questions about her project whilst snowed in in London.

Q – What was the inspiration/s behind Museum Metadata Games (MMG)?

The inspiration for the museum metadata games I’ve made was my curiosity about whether it was was possible to design games to help improve the quality of museum catalogue records by getting people to create or improve content while having fun with collections.

I’m also exploring ways to encourage public engagement with the less glamorous bulk of museum collections – I wondered if games could tap into everyone’s inner nerd to create casual yet compelling experiences that would have a positive impact on a practical level, helping improve the mass of poorly catalogued or scantily digitised records that make up the majority of most museum collections.

People ask for access to the full records held by museums, but they rarely realise how little information there is to release once you’ve shared those for objects that have been on display or fully documented at some point. Museum metadata games are a way of improving the information as well as providing an insight into the challenges museum documentation and curatorial teams face.

The motivation to actually build them was my dissertation project for my MSc in Human-Centred Systems. I’ll keep working on the games on MMG after my project is finished, partly because I want to release the software as a WordPress plugin, and partly because now that the infrastructure is there it’s quite easy to tweak and build new games from the existing code.

Q – What do you think are the main challenges for crowdsourcing metadata in the cultural sector?

Quite a few projects have now demonstrated that the public is willing to tag content if given the chance, but the next step is properly integrating user-created content into existing documentation and dissemination work so that public work is actually used, and seen to be used. The people I’ve interviewed for this project are so much more motivated when they know the museum will actually use their content. Museums need to start showing how that content is enriching our websites and catalogue systems. In some interviews I’ve shown people the tags from Flickr on objects on the Powerhouse collection site, and that’s immediately reduced their scepticism.

My research suggests that results are improved when there’s some prep work put into selecting the objects; and while museums can build games to validate data created by the public, I think a small time investment in manually reviewing the content and highlighting good examples or significant levels of achievement helps motivate players as well as encouraging by example. However it’s often difficult for museums to commit time to on-going projects, especially when there’s no real way of knowing in advance how much time will be required.

Museums also need an integrated approach to marketing crowdsourcing projects to general and specialist audiences.

And it might seem like a small thing, but most museum crowdsourcing sites require registration before you can play, or even check out how
the crowdsourced task works, and that’s an immediate barrier to play, especially casual play.

Identifying gaps in existing collections that can realistically be filled by members of the public or targeted specialist groups and then tailoring gameplay and interactions around that takes time, and the ideal levels of prototyping and play testing might require a flexible agency or in-house developers. This became apparent when I found that the types of game play that were possible changed as more data was added – for example, I could use previously added content to validate new content, but if I wasn’t writing the code myself I might not have been able to work with those emergent possibilities.

Q – Can you give some examples of what you see as ‘best practice’ in metadata crowdsourcing both from the cultural sector and also from elsewhere?

The work of Luis von Ahn and others for the ‘games with a purpose’ project at Carnegie Mellon University has inspired many of the projects in the cultural heritage sector.

Also I think Brooklyn Museum have done a great job with their tagging game – it’s full of neat touches and it feels like they’ve really paid attention to the detail of the playing experience.

I also like the experience the National Library of Australia have designed around digitising newspapers. The Dutch project Waisda? was designed to encourage people to tag multimedia, and seemed to produce some really useful analysis.

Q – What is MMG specifically trying to determine/ascertain with Dora, Donald and the Tag challenges?

My original research question was “which elements of game mechanics are effective when applied to interfaces to crowdsource museum collections enhancement?”.

Over the life of the project, my question changed to ‘can you design data crowdsourcing games that work on ‘difficult’ types of museum content? e.g. technical, randomly chosen or poor-quality records?’ and ‘can you design to encourage enhancements beyond tags (but without requiring more advance data cleaning, selection or manual game content validation)?’.

The designs were based around user personas I’d created after research into casual games, and the tagging game, Dora seems to work particularly well for people close to the design persona, which is encouraging.

I think I’d revisit the personas and create a new one for the fact-finding game (Donald) if I was continuing the research project, and I’d re-examine the underlying game mechanics to deal with the different motivations that would emerge during that process. I’d also like to tweak the ‘success’ state for Donald – how does a player know when they’ve done really well? How does the game know which content is great and which is just ok, if it can’t rely on manual review by the game producers?

The ‘tagging activity’ was created as a control, to test the difference game mechanics made over the simple satisfaction of tagging objects.

Q – What happens to the data after your dissertation?

I’ll pass it onto the museums involved (PHM and SciM) and hopefully they’ll use it. I’ve noticed that people have tagged objects in games
that aren’t tagged on PHM site, so I think the content already supplements existing tags.

Q – What do you think of the debates around ‘gamification’, motivation and rewards?

I think Margaret Robertson’s post, ‘Can’t play, won’t play’ summed it up really well and Use Game Mechanics to Power Your Business also covers some of the dangers of cheap badgeification.

Gamification isn’t a magic elixir. There’s a risk that it all sounds really easy, and that museums will be tempted to skip the hard work of thinking about what a successful experience looks and feels like for their project, audiences and content, choosing their core goals and designing a game around them. If you don’t understand what engagement, fun and learning mean for your content, you can’t build a game around it.

Q – What mistakes do you see museums making with gamification?

I think I covered most of the burning issues in ‘challenges’ above… Requiring the visitor to sign-up to start playing is a huge barrier to participation, and in most cases it’s trying to prevent something that wouldn’t happen anyway – like spam. I haven’t been running my games for long but they’ve been posted widely on Facebook and twitter and I’ve not had any malicious content added yet, and there’s only been two spam attempts in over 500 turns on the two games.

In the evaluation I’ve done, people have said they’re more motivated when they think a museum will actually use their data. If you can show how it’s used, people are much more likely to believe you than if you just tell them.

Q – How much granularity are you tracking with MMG? (By this I mean are you segmenting behaviour by gender, age, location etc?)

I’m using two evaluation methods – in-depth interviews alongside play tests, and releasing the games to the public and seeing what kinds of
data is generated.

For the second, I haven’t tried to collect demographic data as I was more concerned with analysing the types of content generated and looking for factors such as:

Image quality e.g. black and white vs colour images
Technical vs social history objects
Photos vs objects
Extent of existing content – title, dates, places, description
‘Nice’ vs reference images

I’m also looking at factors like number of tags or facts per session, bounce rate, number of repeat sessions, sign-up rates vs play rates, time on site; and analysing the data to see if the types of content created can be usefully categorised.

—

Now go and have a play with Mia’s games!