How many users for social design problems?

I’ve been thinking a lot about research methods for social design problems. For example, how do you research what goes on on Facebook? How does Facebook do this currently? I believe they use metrics from their data team — which I do not believe really uncover social interaction design issues.

How do you research how the unemployed find jobs? Surely they use their social networks. How do you research how to financial investors get advice from their community? How do you research consumer coffee drinking culture? How do you research how people shop for clothes? Shopping is one of the most social activities, even if it’s done alone. In fact the most social shoppers are elderly women: they shop with their deceased husbands in mind (“What would he have liked?”) and with their children in mind (“If I buy the cheaper shirt, I’ll have more to pass on to my daughter.”) But shopping for a dress might take several days to weeks before the purchase is actually made. What goes on in that time?

Okay, those are what I was referring to as “social design problems”. But what I care about is how we design for those things? I know very little about financial planners make decisions. I know my coffee drinking habits — but do I know yours? What about your shopping habits? Research is necessary to uncover all the nuances in our social behaviors, and is especially important to do right when working on such social design problems.

But how? Our user research methods have been fine tuned over the past decade so that we know exactly how to recruit the right sample, how to ask the right questions, and how many users to talk to to unveil most usability issues in an interface. I just don’t believe that we have the right methods yet for social interaction design issues.

How do we study a community that spans the online and offline world? How do we capture emotions, feelings, expectations, and privacy issues without biased research prompts. How do we understand social context?

I am not convinced that 5 users is enough for social interaction design issues — or for studying social interfaces. There are too many variations in people’s social behavior to capture everything in small samples. There are too many social contexts in which even the same user would respond differently. How many users is necessary? What is the best way to complement data metrics (quants) with qualitative user behavior?

I have been talking to several people about this recently and would love to hear from you. What social research methods do you use? How do you use them and what benefits do they provide? If you’re willing to share your methods, I am interested in collaborating with you to create a set of best practices here!

On why people ask questions on social networks

Social search behavior is now generally agreed to be common enough that companies like Google are buying up companies like Aardvark, and academic researchers are asking good questions about the value of networks for question-answering (Q-A). I have done a bit of research in this area myself, and so I was quite pleased to read about this new study by Microsoft researchers Merrie Morris, Jamie Teevan, and Katrina Panovich.

They were building upon (among other things) my work on asking and answering behavior on social networks (“Do your friends make you smarter“) and wanted to know more details about things like: why do people post questions, answer questions, and what motivates people to go the social search route over traditional search.

I want to talk about their conclusion here first: where they discuss the tradeoffs between using social networks and search engines. This was the most interesting part of the paper and is the part that has the most practical take-aways for the UX community.

Basically, for questions requiring trustful and personalized responses like for recommendations or opinions, social networks were strongly preferred over search engines. People occasionally perceived a delay in response time (but sometimes did not!) — and either way, this delay was not a downer since it saved people the effort of making multiple attempts on a search engine before getting the right answer back (if they ever did). Also, subjective questions got faster responses on social networks than objective questions!

Additionally, they talk about how being on social networks is fun for people. It’s too bad that they under-emphasize this point because it’s really important from a social interaction design perspective! People engage on social networks because it’s fun! There’s an emotional tie to the community that keeps bringing them back. When it comes time to ask a question, people turn to their friends for help because it keeps them connected with others, helps them to share information about themselves, and keeps them engaged in their community. Fun is an important element of design — and until Google, Facebook, Yahoo, Microsoft, or other search providers make the information seeking part fun, they will lose. It’s not enough to simply pipe in real-time Twitter results and expect that to be fun or more personalized!

That (above) is the juiciest part of the paper, but I do summarize the whole paper below for other nuggets of inspiration.

To start, the authors note that Q-A on social networks is unique (like unique from Q&A sites like Yahoo Answers) because:

  • questions are posted with true identities (rarely anonymously)
  • people’s audience is smaller (being only people’s direct contacts, or possibly friends-of-friends)
  • status messages are short: so questions are succinct

Then they collected surveys from 624 Microsoft employees (1/4 were female) about their Q-A behavior on Facebook and Twitter mostly. The survey asked about:

Asking questions:

  • whether they ever used their status messages to ask questions
  • if yes, to share a recent example of a question they asked
  • what kind of responses they received
  • how often they logged onto social networking sites

And answering questions:

  • whether they responded to someone else’s question (from a status update)
  • if yes, to share a recent example of a question they answered

50% of people who replied to the survey did indeed use their status messages to ask questions! Sometimes they were silly questions (“Why are men so stupid?”) and sometimes they were serious (“Point and shoot camera just died — need to replace it asap. What should I buy? Think under $200″).

On asking questions

Questions took many shapes and forms. Some were targeted to “Anyone” in the social network; others were general statements put out to the network to interpret. Most questions were asking about recommendations (29%) or opinions (22%). Some were about facts (17%) and some were rhetorical (14%).

Questions also related to everything from technology (29%) to entertainment (17%), places (8%), current events (5%), and ethics and philosophy (2%). People said they’d generally avoid asking their network about topics like health, pornography, religion, and financial issues because those are too personal.

Another finding was that Twitter users were more likely to ask questions about technology, while Facebook users more likely to ask about home and family issues. This isn’t too surprising, but does show that the makeup of people’s networks affects what kinds of questions people will ask.

One looming question in this research is: why do people choose to ask their friends over searching on Google? In this study, people reported having more trust in their friends’ responses or that they thought traditional search wouldn’t work for them with this particular question. Sometimes it was obvious that their friends would know their history, family situation, and other preferences better than a search engine.

People also generally knew the makeup of their networks (meaning they could get targeted responses) and people wanted to advertise their current interests by way of asking questions of their friends.

What about answer speed or answer quality? There were mixed results, but often questions asked on social networks were not urgent so any delay was considered acceptable. It was also easier than sorting through tons of results on search engines and quite often more fun to engage with friends.

On answering questions

Almost everyone has answered a question on one occasion or another. And of the questions offered up in this survey — nearly every one received an answer! That’s kind of amazing considering Q-A sites in general have a much lower response rate, but it’s also likely due to the intimate make up of people’s social networks. Not to be an under-appreciated finding in this paper, however!

About one-quarter of questions got an answer in 30 minutes or less, and 90% were answered within one day. (I’d LOVE to see how Aardvark’s data compares here. I have several questions out to Aardvark that have never been answered; but of the ones that do get answered, it’s typically faster than 30 minutes.)

Yet people don’t seemed to be bothered by the 1-day response lag. Perhaps this is because replies from trustworthy friends are more valuable than junk search engine or Yahoo Answers responses?

Why did people respond to questions by others? Most often they were just trying to be helpful (37%) or had expertise in the area (32%). Those seem like good reasons to me, and ones that are often overlooked when thinking about how valuable personal social networks can be to people. At other times, people replied because they had a good relationship with the asker, were connected socially, or had some notion of social currency (that answering now may result in a favor later).

When people didn’t respond to questions by others, it was mostly because they didn’t know the answer! That’s pretty neat. But some said they’d prefer a personal request and ignored questions that were asked to the network as a whole. In fact, my research on Q-A in social networks revealed the very same thing — that people gave great responses when asked privately or one-on-one, even when they weren’t very knowledgeable about the topic.

Another interesting, but not very surprising finding, is that people who were more frequent social networks users received quicker responses! Why duh! People who are engaged with their communities have developed rapports and back-and-forths and social capital with folks — what goes around comes around. But it is cool to see this intuition confirmed in real data!

Unfortunately, this paper is weak on design recommendations. My colleagues and I have already suggested the very same ideas (e.g., surface your friends status messages on the results page — what I call “friend-filtered search” — or have a search engine post a question as a status message on your behalf.) Even as I say this, there are serious social interaction design issues with this approach. Expectations about search engine behavior, expectations about social network intrusion, privacy (obviously), interactions with online agents, etc. all come into play. I very much hope that these issues are being considered by the big search engine companies, but somehow I’m afraid they’re not.

Enterprise social search: a design workshop in San Francisco


This just in! Will Evans and I are putting on a design workshop in San Francisco around the theme of enterprise social search. The workshop will be an all-day affair on Friday May 7 at the Bolt | Peters offices, near the Civic Center.

Detailed information and registration can be found here: http://socialsear.ch

Why Enterprise Social Search?

Knowledge management and information retrieval in large organizations is a huge problem. A number of orgs are making efforts to address these issues; and leveraging social data — or information that people within the company hold — is one promising route.

Why a design workshop?

Although the premise of our workshop is social search in the enterprise, we won’t be satisfied by just writing or thinking about it. We want to bring together the sharpest minds in the enterprise world and mix them with designers, researchers, and IT professionals to come up with some practical solutions that actually could be implemented.

What next?

After our workshop in San Francisco, we’ll take our findings and carry them onto future workshops in Australia and Washington DC — to build upon our ideas and find ways to develop them even further.

Where can I learn more?

You can read more about who we are and what the workshop will entail over at http://socialsear.ch.

Also don’t miss…

…this presentation that we gave at the Enterprise Search Summit last Fall. This presentation gives a flavor of that topics that will be covered in our Enterprise Social Search Design Workshop in San Francisco.

We hope to see you at the workshop!

Leaving flatland: designing for holistic user experiences

I sat in on a great workshop at IA Summit yesterday about Leaving Flatland: Designing Services and Systems Across Channels, by Jess McMullin (Center for Citizen Experience) and Samantha Starmer (Manager of User Experience at REI). These are my notes!

If I could summarize the workshop in one sentence, it would be: Don’t design for users’ needs on one website, design for the experience customers expect to have.

Now, that’s actually what designers should always strive for. And it gets complicated in large organizations where marketing, product, executives, and designers don’t necessarily all work together.

So it’s important — albeit challenging —  to design across channels. People are leading multi-track lives these days and our experiences span across spaces and mediums. College kids prefer to receive a text message than an email…we visit a new city and have to navigate signage, paper maps, word-of-mouth information, iPhone directions, etc…so many touch points (pain points!) while traveling to the airport and taking a flight.

At REI, for example, customers research online but buy in the store (for bikes), but often research and purchase tents both online. Strategies (or user needs) are different for different products! But to customers, they just care about their experience purchasing gear with REI — whether it’s online or in the store.

Why do we need to sell multi-channel design? Practitioners aren’t thinking enough about designing for holistic experiences, but people’s experiences are always holistic. Also, marketers are way ahead of us! (Only 4 out of ~80 people at Adaptive Path’s Managing Experience conference raised their hand saying that system design was something they thought about or practiced.)

Multi-channel design requires full company buy-in. Executives need to be on board. Designers need to understand the organizational structure, executives’ perspectives, the business goals — and importantly, how things actually get done in the company. UX here needs to work both top-down and bottom-up. REI’s got a new mandate: “Transform the Customer Experience” which was delivered from the higher-ups. This changes how everyone approaches their work.

Mid-morning we went on a field trip to understand how the Hyatt Regency hotel deals with customer service. My group looked at front desk interactions, but other groups went to the restaurant services and meeting services place. We asked a lot of questions. Took a lot of notes. Drew a bunch of pictures.

Then we came back and did a customer journey mapping exercise, where we thought about all the touch points for different services. For example, for front desk check-in, touch points include: face-to-face, web, TV, email, kiosk, phone, and physical space (lobby). An experienced business traveler might flow through this system like this:

The next exercise involved Business Origami! The point was to map out spheres of interaction for the check-in process.

This was really fun because it got us thinking about all the players in the system (not just the agent & the guest). Of course, we started by thinking about the front-desk interaction, but increasingly drew out more and more spheres that all played into this one interaction. All the training and support the agent has…the immediate Hyatt chain…Hyatt corp…but also the whole sphere of the business man’s life.

We learned that this is a method of doing “service blueprinting” where you have to think about all the physical actions, guest actions, customer-facing actions, back-stage actions (like training), etc.

Finally we talked about some tips for building good experiences across multiple touch points. On websites, you have to consider how every single page looks because lots of users will come into your site via Google (e.g., through a “back door”). So on every page, every touchpoint, you have to:

  • be consistent
  • use the same information
  • and reinforce your brand everywhere.

Designing holistic experiences also means listening holistically:

  • the usual UX research, along with:
  • call center
  • social media
  • sentiment analysis
  • market research
  • analytics
  • store follows/shop alongs
  • email queries and feedback

And finally, make new friends (marketing, finance, distribution center, packaging, different industries); wander the halls; get out of your comfort zone and have new experiences. This will give you perspectives on customer experiences that you wouldn’t otherwise glean.

Please tell Congress to renew the flood insurance program

This is a quick plea for your help and attention. My sister and her husband — first time homebuyers — are on the brink of losing the house they have a winning bid on and are totally on track to close on April 14. EXCEPT that they cannot get the flood insurance they need for the loan because Congress went on Easter Recess!

This is their (hopefully) new house — or watch the slideshow. And this matters so much, they just wrote an op-ed in the San Francisco Chronicle explaining the situation.
There will be so much good karma flowing your way if you can reach out for your local congressperson to urge them to renew this non-controversial federal program!

For the really easy route, use this website and simply copy/paste our pre-written text below. For the full story of why Erinn and Chris Andrews might lose their house, scroll down.

Draft for your Congressperson

Dear Congressperson,

I’m writing to you today about a very urgent matter. My good friends are in the process of buying their very first home. They have been saving for years and are ecstatic about owning a home where they can raise a family.

However, because of the Easter/Spring recess that Congress took, the bill # HR 1264 (the Multiple Peril Insurance Act of 2009) was not voted to be reinstated, and the National Flood Insurance Program (NFIP) expired on March 28th. Because our friends’ new home is in a flood zone, they cannot remove the final flood insurance contingency because no insurance company will insure homes in a flood zone until the NFIP is reinstated. This means they may LOSE the home they are trying to purchase. This also means that ALL home purchasing in flood zones around the United States has come to a complete halt because of the recess.

At a time when the government is trying to encourage home buying, it is painfully ironic that such an obstacle would come in the way. We know our friends are not the only people affected by this delay which means both buyers and sellers are at the mercy of Congress’ inaction. We ask that you follow-up with your colleagues in Congress to reinstate this legislation the moment you return to session.

Thank you very much for the work you do on behalf of your constituents.
Sincerely,

Erinn & Chris Andrews’ full story

“As some of you know, our new house is in a flood zone in Menlo Park, and all mortgage lenders require loan insurance in order to issue a mortgage on these sorts of properties. Unfortunately, as of March 28th, the National Flood Insurance Program (NFIP) expired because Congress did not renew the program before it’s Easter recess lasting through April 12th. This means that no one in the United States can get flood insurance on a property in a flood zone, and in turn, no new mortgages can be approved for the purchase of those properties!

“In order for us to close escrow by April 14th per our offer agreement, we need Congress to take action as soon as they return to session. Please help us by going to your local congressperson’s webpage, emailing them, and asking them to take action. Feel free to use/modify the attached draft, and we sincerely appreciate your help in getting our first home purchase approved!

“Thank you so much — we look forward to returning the favor if/when you ever need a place to stay in Menlo Park!”

–Chris and Erinn