A while ago I was having drinks with a Google employee, and we started discussing privacy problems. He asked me why Buzz had received so much bad press for its email analysis when Facebook and other social networks had been doing the same thing for years? He also pressed me on why the iPhone tracking story had become such a big issue.
People have a mental model of what devices and services are for, and get freaked out when someone changes the rules. Nobody understands constantly-changing space-shuttle-control-panel privacy settings within services, but everyone knows that LinkedIn is for business relationships, and Facebook is for friends. Users try to protect their privacy by limiting information to sites that serve the audiences they want it to reach.
When Google changed from an email and search provider to a service that could broadcast semi-public updates to her friends, it became unclear where information she'd previously shared would end up. When Apple switched from a phone and computer builder to something that followed your movements, that crossing of boundaries was the real problem. Nobody would have blinked an eye at the idea of a Garmin device keeping a file showing where you'd been.
If you're worried about how users will react to something innovative you're trying, think about how they understand your purpose. Why did they sign up for you in the first place? Ignore the grand vision in your head, what do they think you do? If what you're doing makes sense for that goal, you'll be surprised at how generous and supportive they can be, even for potentially scary applications. If you're working towards something they don't expect, if you're moving outside of the silo they think you're in, you may be in trouble!