What I find strange (not sure how true) is what I hear from random MS employees that customer feedback is more valuable, or weighs more, than the feedback from MS employees. To me, this shows a low level of trust from management in their own employees' ability to gauge the market.
Another thing I can't wrap my mind around is how can it be possible to have one thousand of your customers yelling at you they don't like something -- that can be easily changed -- and have the management respond with "Oh well, we're still going to do what we want. What ya gonna do about it?!"
>What I find strange (not sure how true) is what I hear from random MS employees that customer feedback is more valuable, or weighs more, than the feedback from MS employees.
In my experience it is the classic (and often correct) "You aren't the target audience". For instance, I could write code on the Excel team (I don't). I am not a heavy Excel user, so my feelings on how Excel should be set up or operate likely do not represent the actual users of Excel and thus my feedback should be weighted lower than real (paying) customer feedback. Talk is cheap, people exchanging money for a product should be listened to above whiny devs (the 'whiny dev' label is intended to apply only to me and my feedback. External customers giving feedback are not whiny devs, at least not in my eyes :)).
Visual Studio is somewhat of an exception to this general rule because us oddball developer types (who don't represent the mainstream customer of most any product) actually do represent (mostly) the mainstream customer of our product.
>Another thing I can't wrap my mind around is how can it be possible to have one thousand of your customers yelling at you they don't like something -- that can be easily changed -- and have the management respond with "Oh well, we're still going to do what we want. What ya gonna do about it?!"
thousands << millions. The user base of Visual Studio is large, I don't know exact install numbers (I am sure I could find out, I just have never been curious enough to pester someone in marketing or the PM org) but I know it is definitely in the millions. It isn't clearly incorrect to argue that 1000 people playing with the beta may be a different group and have different beliefs than the millions of eventual users. I am not saying it is true or not, I haven't done the market research, but it isn't implausible.
I agree that thousands < millions, but was there a study to see how those millions feel about the UI? The change was from beta to RC. Was there really a study done in such a short time to ask those millions? If such studies exist then why even bother asking for feedback in a public forum? Anything you'd get would be statistical noise.
Since the changes were quite drastic between beta and RC, my guess would be no, there was no study. If there was a study then I would question the competency of the people of carried out the study given how off the mark the results were (VS beta UI).
My guess is that the UX/UI people didn't really know what they were doing. They saw the massive backlash and went "Oh OK, I guess we were a little bit off. Let us remove most of these upper case captions, but we can't get rid of them entirely, so let's change the menu to upper case." When the second wave of backlash came (over the menu) the UX/UI people already had their pride bruised so they couldn't back down again.
>When the second wave of backlash came (over the menu) the UX/UI people already had their pride bruised so they couldn't back down again.
Possibly, but I suspect it is, as they have already stated, driven by the desire to align some UX look and feel consistency across 'major products'. I don't know if I quite 'buy' the idea that all caps menus in VS and all caps menus in Office give you any 'alignment', but you know, I am but a dev not a UX person :)
You do not need that study of millions. Let's say you talk to 100 customers who you think to be a representatieve sample. One or two make a remark on the issue. Two weeks later, you ask them about it, and they say it grows on you.
Now, you send out the beta. You expect 1-3% or so of your customers to make a remark on it. Maybe 1‰ will contact you or write a blog post. Should that worry you? No.
And of course, you should be even less worried if you had measured performance related to the issue, and saw no differences.
In short: those thousands are self-selected and not representative of the audience. On top of that, all reviewers have to have some critique. Critisizing (nicely symmetrical, with all vowels being i's, and the symmetry of consonants; but I am digressing) such an obviously visible feature is easy.
Listening to customers is not the end all or be all of UX or product design and neither should it be. It should be taken into account, but UX experts surprisingly know better much of the time. The one constant is that people don't like change, whether good or bad, and that decision cannot be made by listening to a vocal group of commenters.
This has always bothered me: I'm not the target user, but then who is? Some of the more unprofessional product managers can use this as a mantra when they really have no clue.
On the other hand, when you include a designer in your product, you really have to let them do their job! Yes, feedback (external and internal) will be killer, but if they are good and know what they are doing, they won't let this sway them: they want to deliver a product that you will love, that you didn't know you wanted, and not necessarily what you asked for. Professional UX design with IDEs seems rather new (e.g., Cloud9), and developers are surprisingly a rather conservative lot. But if you want innovation from your IDE vendor, you should give them some room to experiment and possibly make mistakes.
Another thing I can't wrap my mind around is how can it be possible to have one thousand of your customers yelling at you they don't like something -- that can be easily changed -- and have the management respond with "Oh well, we're still going to do what we want. What ya gonna do about it?!"