Comparing JS libraries with web server frameworks is pretty apples-to-oranges, first of all. But MVC frameworks pretty much have to be "everything". Can you use a desktop UI library without going with it completely?
Further, who has ever advocated for building desktop apps _without_ a framework? The comparison of client-side frameworks to server-side frameworks is the wrong comparison. If I'm going to build a sufficiently complex Mac app I'm going to use Cocoa, no question. If I just need to automate something and don't necessarily need a UI, I may look at AppleScript or even a simple shell script.
This is more of an apt comparison for web front ends. The choice of Backbone (or, in my case something that provides a little more value like Flight) vs. Ember/Angular is like choosing to build a CLI app vs. a full-on desktop app.
There are perfectly valid reasons for choosing either and in the right context, the right tool is worth its weight in gold.
