I recently ran two user studies to determine the effectiveness of Android permissions in practice. Our results are now available as a technical report: Android Permissions: Attention, Comprehension, Behavior. Android permissions are supposed to inform users of the risks of using applications. However, researchers have speculated that users ignore them. We decided to establish how well they actually work.

Here are our primary findings:

• Attention. In both a self-reported Internet survey and observational laboratory study, 17% of participants paid attention during a given installation. 42% of laboratory study participants were completely unaware of permissions.
• Comprehension. Only 3% of Internet survey respondents could correctly answer three multiple-choice comprehension questions. No lab study participants could correctly describe all of the permissions of familiar apps.
• Behavior. A majority of Internet survey respondents and 20% of lab study participants say they have decided not to install an app in the past because of its permissions.

We categorized 20% of the lab study participants as “power users”: they sometimes look at permissions and scored reasonably well on our comprehension test. It’s possible that a small fraction of “power users” could write negative reviews when they encounter troubling permission requests, thereby protecting other users.

Our studies identified several factors that contribute to the low attention and comprehension rates. Here are two of them: Continue reading →

If you had to guess:

1. What percentage of Android users know about permissions?
2. What percentage of Android users look at permissions during installation?

This is a serious, non-rhetorical question. Please leave your guesses as comments.

Update: Interesting! I was curious about whether anyone actually had faith that the permission system works well. (It looks like the answer is no.)

Researchers have been working on tools (like AppFence) to help users control how their personal information is used by applications. Recently, some colleagues and I asked 25 Android users to tell us about their bad experiences with applications, to provide insight into whether there is real user demand for more control over how applications access and use their personal information.

We asked, “Have you ever uninstalled (or stopped using) an application because you didn’t like what it was doing with your personal information?” 6 of 25 people said yes:

• Spam. Three people said that they’d uninstalled apps that sent them spam and/or used their accounts to send spam to others. For example, one person reported that an application had misused her Twitter account: “I logged on and it was posting crazy stuff like, ‘Oh! I just won $1000 while doing this or something’ … And I’m like, that definitely wasn’t me, it was the app.”
• Social privacy. One person said that she didn’t like how an application shared information about her on Facebook. “I went on [app name] to buy tickets and if you went on Facebook you could see who bought tickets where and what their names are. So I chose not to buy tickets on [app name] because I didn’t want anyone to see where I sat if they went on to Facebook.”
• Programming errors. Two people said that they had used applications with logic errors that affected their personal information. For example, “But what it did was, if I sent a text message to one person, it would send it to everyone in my list.”
• General privacy. One person read a Wall Street Journal article that listed applications that share user data with other parties like advertising networks. “…I went through my phone and went through that list, and got rid of all of them except for one. … Shazam was the only one I kept, because the functionality. I know people are taking my stuff but functionality-wise I need that.”

Two other users said that they would uninstall applications if the applications misused their personal information, but they had no way of knowing how their data was being used.

Recently, I had the opportunity to sit down and chat with a few Android users who aren’t computer scientists. One of the things that we talked about was advertising.  If a person mentioned using an app with ads, I asked, “Do you think the advertiser can use the app’s permissions?”  Twelve people answered this question:

• Yes: 5
• No: 2
• I don’t know: 5

People were right to be confused by the question.  The correct answer is complex (check out the last paragraph of this blog post).  However, I wasn’t trying to test people.  Instead, I wanted to get a general sense of what people think the relationship is between ads, applications, and permissions.

Some people didn’t think the advertiser should get permissions:

• “I gave these permissions to [app name]. No permissions to the ad.”
• “They can have advertisements like TV has advertisements, a million people’s phones can have advertisements, but they shouldn’t have access to your phone just cause they have advertisements on them.  They shouldn’t get a permission at all.”
• “I guess it’s possible, depending on their relationship with whoever made the app. But I couldn’t imagine why they would have permissions that would go through my phone state and identity. … I hope not.”

On the other hand, some people said that they think it’s reasonable for advertisers to get access to information about them via permissions:

• “I’ve read that advertisers are sending people things on mobile devices based on, like– you might be near some store, so you might get something that pops up on your phone that says, ‘We’re giving a two for one deal today in the next two hours,’ or so.”
• “They probably get at least location-based so they know where people are accessing it and where they’re downloading it from, how they’re getting to their ads most likely.”
• “The advertisers could use what I look up on the Internet to better, I guess, suit the ads that they put in for my specific [ads]. When I’m searching for things if it has access to what I look for on the Internet, it could find and put ads that are similar to interests I’ve looked up on the Internet to my phone.”

These are just simple anecdotes, but I find them interesting.

It’s hard to gauge how people feel about advertising and permissions because they don’t understand Android permissions to begin with. For example, the person who mentioned Internet search history made that statement because he misunderstood an unrelated permission that does not give access to Internet search history.

Can an advertiser use an app’s permissions?
When you see an advertisement in an application, there are three parties.  First, there’s the application itself, which asks the user for permissions.  Second, there’s the advertising library, which is shoved into the application and therefore gains access to all of the app’s permissions.  Third, the advertising library displays the advertisement itself.  The advertisement can’t directly use any of the permissions, but the advertising library might share information with the company that is running the ad.  So if you see an REI ad while playing a game, you should know that the invisible ad library gets all of the game’s permissions, and it might share information like your location with REI.

P.S. Thanks to Elizabeth Ha, Serge Egelman, Ariel Haney, and Erika Chin, who interviewed users with me.