Given that assigned tasks are completed in a different way to self-selected/naturalistic tasks, it crossed my mind some time ago that an interesting way to gather data on naturalistic tasks might be to ask people to install an ‘event sampling’ app which could be tied into research and some nugget of information users might find interesting. A nice example of this sort of approach, using apps for event (or time) sampling, is the mappiness app () which pings users once (or more) a day to gather GPS data and ask the user about their current mood. The idea is to collect data on times/locations of moods, the user pull is 1) help research 2) see your own personal graph. If someone interested in search created an app of this type, information could include things (of interest to users and researchers) like: * Normally you take x time searching for answers —- this time you spent y time searching for your answer * Normally you select m number of links from a search page —- This time you selected n number of links from the search page * You said page p had the correct answer – you could have take shorter route q (or clicked x fewer times or something) perhaps either involving a path analysis that includes either an algorithm based ‘shortest path’ or a collaborative filtering based one (‘users who search for x found this result much faster than your search for y’). * Something about types of queries made/whatever, this could include (if installed in more than one device, e.g. phone and PC) something about the types of queries made on different devices (and other differences in measures across the two) If you could also include questionnaires based on some sort of self-efficacy, or search-satisfcation measure I imagine the data would be quite revealing. Of course, this might lose, for example some of the (uninterupted) nuance in multi-part search tasks, for example those on agoogleaday often include two parts e.g. “how many floors does the building on xy street in New York have?” in which one first must find out what the building is, and then the number of floors. It also might not be very good for exploratory tasks – although perhaps asking people to ‘bundle’ the searches they’ve made into a particular “need” they were trying to meet would be interesting. …writing this I can’t help but suspect such a tool already exists? Or perhaps there are arguments for just using custom systems to track all searches (rather than event sampling) which I’ve neglected? I’d welcome thoughts/feedback! [EDIT 29/04/2013] Just been reminded of HCI Browser “The HCI Browser is a tool designed to help administer and collect data for studies of web information seeking behavior. Studies involving web search and browsing often involve presenting participants with a set of tasks to do on the Web and then recording information about the web sites visited, searches conducted, windows opened, and other interactions with the browser such as clicks and scrolling events. Sometimes, questionnaires are administered before or after each task to ask participants about their experiences. The HCI browser supports these types of studies and is designed to automatically present tasks, administer questionnaires, and collect interaction data. A major goal of the design is that it will guide participants through the questionnaires and tasks with minimal experimenter intervention.” It’s not quite what I’m describing because it’s less random…but if it could be triggered periodically then the only thing that would need adding is that trigger mechanism, the rest of the system is built there I think!