I say this with confidence because there is a large amount of research showing that people make decisions with a good deal of input- if not entirely- from the non-conscious parts of their brain. In fact, there are some people that argue that our conscious brains don't make decisions at all, they merely rationalize it post facto. (This goes too far, for me)

The classic example of this are the split brain studies, where they would show one half of a person's brain a sign that said "stand up." The patient would stand up. Then they would ask the other half of the brain (the one that was capable of speaking; in most humans, only one half of the brain can) why they stood up, and they would be completely convinced it was because they wanted to get a drink or water, or go to the bathroom.

That said, I'm pretty much the most gung-ho person on reason I know. The reason for this is that, like someone mentioned on a blog post on here, intuition is not transferable. If someone wants to convince me of anything at all, they're going to have to make reasoned arguments, not emotional ones.

Hmm, I'm not as sure on that one. Aren't there studies that show that convincing people via logical argument is just about the least effective possible means of persuasion? Politicians might be seen as something like professional intuition-transferrers: they want you to come around to their way of thinking, and they don't stick to only using rational arguments as the primary way of making that happen.

Even in philosophy, arguments by analogy often seem aimed at least partly aimed at transferring intuition, by mapping a situation to one where that you hope your reader has the intuition you want.

For instance, it's possible to mathematically prove in some cases that one algorithm is better than another; a set of ten nested for loops simply has greater time complexity than one. You don't need to actually test which algorithm is better.

And secondly, your intuition can be bad, but it's not guaranteed to be wrong, either. If you're a very experienced developer you might have a feeling about what's a more efficient way to code something up that could be in general, correct. Not every decision you make needs to be tested.

However, there is absolutely no point in arguing whose algorithm is better if there IS an argument about it since it can be tested easily. Also, if you personally are not sure which approach is better, test it.

And finally, you're actually in luck if you have two algorithms that you can test and find the better one. That's an easy situation to deal with. It's actually much harder if you only have one algorithm and it's a slow memory hog. Testing is only as good as the options you give it.

(On a related note, this is especially important to remember in science, too- testing hypotheses is only as good as the hypotheses you come up with to test. There could always be a correct hypothesis out there and you never even conceived of)

At the risk of being pedantic, I want to point out that it's not always so clear-cut. Take sorting, for example. Everybody knows that quicksort is quick; it says so right there in the name. It runs in O(n lg n) time with small constant factors. But for small arrays, or arrays which are mostly sorted already, you can do better by using insertion sort, which runs in O(n^2) time. Somewhere in the bowels of glibc is a highly-optimized quicksort which does exactly this: it drops down to insertion sort for subarrays of size 6 or less.

And don't even get me started on the effects of cache locality. The point is, sometimes intuition and analysis of algorithms can only go so far; at some point you've got to either test it or just call it "good enough" and think about something else.

The constants would make it less efficient for most cases, though.

there is no pure or complete ordering of reason, intuition or empiricism. Wisdom is knowing when each approach is the best fit, and understanding that even the best approach selection heuristic is not 100% reliable.

If so, that's an interesting hypothesis. Is it true?

There's no substitute for doing your best to rationally analyze something, gather data, analyze arguments, etc., and not doing so in a particular situation is simply sloppy decision making. But that's merely one input into a decision making process that can also include things that fall under the category of intuition but which can't easily be articulated: ideas that give you a bad feeling or make you nervous, gut feelings about things, consideration of taste and aesthetics.

In the end there's no substitute for having good judgment, and good judgment and wisdom come from being able to use both rational arguments/data and intuition/feelings to arrive at the best choice.

-- But let me clarify! I want to directly attack those who you refer to thinking that "if you can't explain or prove something rationally then it's not of any value". I don't think anyone is a constant calculator like that. It's important to learn because we're building computers that operate that way, but it doesn't appear that strict rationality (whatever that is) is our true operating mode.

So if you want to get the gains of rationality, the best you can do is use rational structure to train and shape your intuition. If you're skilled at the math required then you can use it to empower your intuition -- but never replace it.

I say this with strong fear of thought paralysis. Those who spend too long rationalizing every thing seem likely to trap themselves in local minima, to argue endlessly over two similar choices while missing out benefit of either.

I like to think about MCMC algorithms, actually. They're guaranteed to converge to the most liable posterior beliefs, but do so by jumping randomly. Each step forward is technically blind and hopeful, but by keeping a goal in mind and learning from every jump you improve, even considering the immense ignorance that Markov methods maintain.

Intuition can do better.

It feels empowering to hear from a fellow intuition enthusiast, too! :)

Is money as valuable as a marketable skill? Is getting somewhere quickly as valuable as a vehicle?

The intuition of a person who doesn't value rational thinking is going to be worthless in an empirical field. The gut feeling of someone who can't show they are right is not likely to be correct.

Intuition is just the fast-read cache of pre-computed notions. If a problem looks similar but not exactly like a dozen or so others that you have solved in the past, you take a sort of snapshot of all that past work and distill it into an intuitive grasp of the problem.

If you see a problem unlike any you have ever encountered, you are unlikely to have an intuitive idea of the solution. Sometimes people will pull past-solved problems out of their fast-read cache and apply them to unrelated problems in an irrational way; sometimes people will feel their intuition guiding them in a certain way and rationally realize they are projecting subconsciously an unrelated situation onto the problem.

So intuition is only as valuable as its rational basis. Intuition is much more efficient, much like using a rainbow table to crack a password; the table is only as effective as its precomputed hashes; it will never crack a password that hasn't been precomputed. However, compared to a traditional brute-force approach, it's orders of magnitudes more efficient. A brute-force approach however, with enough time will crack any password.

I, for one, often struggle to explain to others why my gut feeling is right, but it often is.

I agree that it can connect past experiences and solutions in new ways; that's obviously what it does, but my point is those past experiences and solutions have to exist already; it's literally what they are drawing from. If your brain categorized failures as 'solutions' due to irrational thinking, whatever you intuit from those 'solutions' is going to be a failure as well.

Struggling to explain why your gut feeling 'is right' when it pops into your head is normal and completely separate than being absolutely unable to make a good argument for it when people offer contradictory solutions.

Personally, when my intuition is ahead of my reasoning, yet I have a strong personal belief my intuition 'is right' I will often later, in a spontaneous way rationally link it all together in a sort of "ohh thats why it's like that."

However, if I've made an important choice intuitively and it turns out right, I absolutely dig back into it and try to discover why it was the right choice in a rational fashion. Understanding that choice rationally lets me learn how to apply what I learned later, and sometimes helps me see second- and third-order effects which I can't see intuitively.

After you've practised and gotten good at something, you've got your brain to start using the fast side to handle things. For instance, bike riding. At first, you're consciously thinking and trying to control everything. After a while, you get heuristics that take over and it becomes natural.

This happens in a lot of subjects, as far as I can tell. So, you shouldn't trust your intuition blindly. But understand the biases you're likely to have and how your brain is immediately popping up these ideas can be useful. Perhaps intuition can be more powerful because it's our brain using some fast ways of coming to a conclusion?

http://www.amazon.com/Heuristics-Biases-Psychology-Intuitive...

I don't think it's in our nature as human beings to act purely in a rational realm, but some degree of it has to exist or we'd never get anywhere. When I focus on matters of personal finance and productivity I need to think and act rationally or I lose objectivity (and ultimately much more). However in my work and much of the rest of my life, intuition goes hand in hand with experience and I find myself relying on what I "feel" to be the best course of action given whatever situation.

Again, this intuitive process that makes use of my experience (and that of those I've entrusted my thoughts to) is something I've given myself over to on a rational basis. If my experience-based intuition has served me well in the past, isn't using that to my advantage the rational thing to do?

We have created a society that honors the servant and has forgotten the gift."

Albert Einstein

I also like "Logic will get you from A to B. Imagination will take you everywhere."

His quotes are basically the proof of the pudding that rationality needs intuition, since he's thought of as one of the greatest scientific minds ever.

Off topic but interesting: the wonderful definition of insanity that's always attributed to Einstein (doing the same thing over again while expecting different results) never came from Einstein at all. It has been traced to a 1983 novel by mystery writer (and one-time partner of Martina Navratilova, how's that for trivia) Rita Mae Brown.

Secondly, he did speak of the sacred very frequently in fact, especially later in life: http://www.time.com/time/magazine/article/0,9171,1607298,00....

Regardless, it's a good quote either way, don't you think?

Most such quotes on the internet are fake. I'm fascinated by how catchy lines get attributed to famous people, so this is a bit of a hobby.

Secondly, he did speak of the sacred very frequently in fact

The article you link to doesn't include the word "sacred". I think the word is significant. It doesn't quite seem like a word Einstein would use. Especially in the context of a "sacred gift", which implies a gift-giver. Einstein's deism was resolutely impersonal.

As you doubtless are aware, Einstein's religious views or lack thereof are the subject of perennial quarreling among camps eager to claim his name for their own. It's all rather tedious. But it's at least a credit to Einstein's open-mindedness and tolerance that he wasn't a finger-wagging scientistic scold of the likes of Dawkins.

Regardless, it's a good quote either way, don't you think?

Not quite, no. Certainly my sympathies are all with this line of thinking, so I ought to like it, but it feels like it hits the nail askew. I mean, if the intuitive mind is a sacred gift, why isn't the rational mind also? Presumably there can be more than one gift. Also, the implied symmetry between "intuitive mind" and "rational mind" seems slightly facile. I'm a big relier on intuition but I'm not sure I'd call it a mind. But I'm just rambling out loud here. Thanks for an interesting discussion.

Edit: do you know Eugene Gendlin's work on intuition? One of these days I want to read him in greater depth. But I'm familiar with his book Focusing which was an original contribution to psychotherapy. He speaks of intuition (actually, he uses the phrase felt sense) as a kind of bodily knowing distinct from cognitive knowing.

I'm the type that will routinely head off to a restaurant with only vague information about where it is or what its name is.

"It's probably down here, this seems like a street that would have mexican restaurants on it..."

"I've been there once before, and I remember it was on a block that was on the edge of a hill that sloped off toward the morning sun..."

It seems crazy, but the reason I end up doing this is because it works a surprising amount of the time. I end up getting there, somehow.

With work, I find that I end up relying on intuition the same way. The downside is that until I'm familiar with the problem space, I tend to be more unsure of myself than my rational peers. The upside is that once I learn the space, I can move more quickly with less information than they do.

Edit: Like others said below, the other downside is that this can sometimes lead to surprisingly bad results, especially in new and unfamiliar areas. I've been trying to exercise my logic muscles more recently to help with this.

To answer your question. I have personally been more right than wrong when i have followed a gut feeling. I also act when i go with it more often than when i dont. When i am 'rational' i often DO NOT act because rationality tends to lead to pessimism. Atleast with me.

Intuition is a means of taking in information. Psychologists call it irrational because it isn't really a decision making process.

Instinct is very rational, we just don't usually recognize it as such because it isn't concerned with verbalizing its logic. For instance, the game of basketball actually requires a lot of logic if you think about it. It's just that it requires a very fast-acting form of logic. Imagine how ridiculous it would be for a basketball player to say something like "I know I can make this goal because of Jordan's theorem". By the time they finished that thought, the other team might have already scored.

Rationality can definitely lead to pessimism if you're not careful, but I think most pessimism is actually quite irrational. Rationality can also be used to support optimism (for example, it's rationally possible that I become really successful). So I try to live by what I call "optimistic rationality" (if that hasn't been coined already. :) )

I think that pessimism and rationality are close cousins. In that modern terms like 'be rational' tend to really mean that we should 'be skeptical'. Which tend to mean that we should 'be pessimistic'

The leading thought of the book is that we should put more attention to the R-mode of thinking, traditionally associated with the right brain, which is responsible for creativity, pattern matching and intuition.

Moreover there is a fragment which states that depending on intuition works only when you reach a certain level of competence and gather enough experience to "feed" the unconsciousness processing.

http://pragprog.com/titles/ahptl/pragmatic-thinking-and-lear...

It's been a while since I read it, but the long and short of it is that intuition is the only game in town, especially when making decisions under pressure, and the people with good intuition are the one's with experience.

In other words, our mind is constantly watching and learning, and then matching that data to patterns we've observed in the past; not all of this information reaches the front of our consciousness.

Of course, if it's subconscious, it is easy to get misled, or match patterns inaccurately since there is no rigorous examination involved about the underlying assumptions.

* Of course, rationalizing a gut feeling is far from objective. But at least it makes you think things through.

Rational thinking is mostly understood to be a deductive form of "thinking" that a electrical/mechanical device could simulate with the technology we have no. (Evidenced by, say, Mathematica.)

Intuition is a more organic form of rational though; generally unchecked for errors (before your rational thought process gets ahold of it and examines it thoroughly), but exploring possibilities for solutions that may not immediately follow from the data you have in front of you. It skips the intermediate steps and delivers an abstract pathway to your solution, which you follow, and prove to work, with your rational thought.

Intuition is more like +N un-processed rational steps at once, with a higher probability for error. As long as we assume there is some difference, they're are co-dependent and just as valuable as the other; you need equal "amounts" of both to solve the difficult problems.

But however much we rely on emotions and intuition in our own thinking, they're not easy to communicate, especially to skeptics. Those of us with a scientific and skeptical mindset try not to accept irrational arguments from other people. [2]

So, in some sense, an idea without a rational justification isn't worth very much (yet), because you can't convince most people that it's correct. But it still might have a lot of potential! And if we're talking about hacking then there are other forms of persuasion (code and demos).

[1] http://changingminds.org/explanations/emotions/emotion_decis...

[2] This is especially true of mathematics where rational arguments (that is, proofs) are valued far above any other kind of argument.

In this case (emotion), my guess is that parsimonious applies. When there’s all that useful circuitry in the cerebellum (emotion), why reinvent it in the cortex? The cortex obviously has new and very useful circuitry but there will still be many applications where the old circuitry - and thus the cerebellum and emotion - are what’s needed.

Emotion has been around a much longer time than reason (and is found in a wider range of animals). It’s true that it was probably easier for evolution to invent emotion but history is also telling you that the evolutionary advantages of emotion are longer-standing and better tested than reason. Current events tell me that the evolutionary jury is still out as regards human-type reason - or at least the way these various quantities (reason, emotion, intuition, etc) are wired together in the case of human beings. Evolution may need a redo.

"For the first time in human history, we can look inside our brain and see how we think. It turns out that we weren't engineered to be rational or logical or even particularly deliberate. Instead, our mind holds a messy network of different areas, many of which are involved with the production of emotion. Whenever we make a decision, the brain is awash in feeling, driven by its inexplicable passions. Even when we try to be reasonable and restrained, these emotional impulses secretly influence our judgment."

Put another way, there is no real bright-line divide between rational, intuitive and emotional. Rather they form a decision-making fabric and even when we believe we are deciding in an entirely rational manner, we're not.

Rational thinking is good at explaining why does it work, then. This is connecting the new dot back to what you already know.

Further, my observations suggest that those two are not mutually exclusive. Intuition and feelings can indeed be based on what you already rationally know. They just operate on the unknown area that your consciousness can't reach yet. Intuition isn't too helpful unless you know something about the problem domain. Similarly, rational thinking can't really warp to anything truly new without intuition: the rational mind per se can only take what is already known and expand it a bit. To do otherwise would be, to the rational mind, well, irrational.

tldr: Intuition has a larger short-term memory space. It can process more complex systems than verbalized thinking. However, because it is not verbalized, it is difficult to componentize, communicate or reapply. It also does not work well on-demand or under pressure.

Say, if a person doesn't know programming, his 'intuition' in programming would just be wild guesses and unlikely to be useful. However, he can still gather facts on programming and think rationally on it, no matter how slow and painful.

So in your original context of your post, I would suggest that we should only use intuition when we have extensive experience in that market or working with the target users. Otherwise, stay close with rationality.

What about general life decisions? It's hard to say if you have enough "experience" in life to have good intuition about it.

I don't believe the two approaches can be meaningfully separated, but some people try to do so. Which can only cause problems as each belongs to a different type of problem. One doesn't rationalized where a lobbed ball will land when going to catch it, but a non-expert doesn't intuit where a mortar will land upon firing it.

We might not base those intuitive decisions on clear cut reasons (that guarantee results, 100%), but we base them on past experiences and observed phenomena (that increase the likelihood of the result, sometime even to 100%.)

If you are asking about an intuitive decision that is not based on any of the above, then your question is really, "Should I make random decisions?" Sure, just because something has a minute chance doesn't mean it won't happen. It might just work.

At first glance, yes.

I suggest you read 'Blink' by Malcolm Gladwell. The author gives many examples of situations where spontaneous decisions yield better results compared to rational thinking methods.

    And [that] data eventually becomes a crutch for every
    decision, paralyzing the company and preventing it from
    making any daring design decisions.

So yes, let's use it, with care, and be honest to yourself.

But it is quite fallible in a way that "pure" rational thought is not.

Revelation means ask an expert, read the documentation. It is most appropriate when you don't know what you're doing at all -- when you have no sound first principles to feed into the engine of reason, and no experience on which to build intuition. Revelation is fast but limited; you instantly gain a conclusion as sound as your expert, but you cannot improve upon or critique it.

Reason is most appropriate when you have moderate experience in a field. Through revelation and limited experience, you have developed some sound, inviolable principles, and can reason your ways to new ones. You know what must go here because you know what must go there, and you can figure out how this works because you know how that works. Reason converges to truth slowly but inexorably. It will eventually get there. But it takes a long time to process a lot of input or to navigate a complex landscape.

Intuition is an appropriate tool when you have high experience in a field. Even presented with a complex problem, you know what to do. You just know that OO is the wrong paradigm for this, and that code must go on the server, and this bug is almost certainly caused by a mismatched type somewhere in the parser. Given time, you could probably justify it. Doing so would be equivalent work to writing a paper -- there's just a lot of stuff to consider and weigh. But the power of intuition is that you can decide nearly instantly, and often decide right. Intuition as a tool doesn't converge to truth; it quickly leaps, and then it either gets there or it doesn't. And even when does get there, you don't know immediately whether it actually worked or not. It always uncertain, being only as good as your necessarily incomplete mental models. But there are times when a fast guess is way better than a slow conclusion. And there are times certainty isn't possible anyway. One problem with intuition is that you can't improve its results for a particular problem. If it's wrong, it's just plain wrong. You can't doggedly grind on like you could with reason, or ask another source like you can with revelation.

The highest level of competence comes from using all three tools in concert. With a lot of experience, revelation is sharp: you know who the experts are and what they are likely to know. Reason is sharp: you know a lot of useful rules, and the fastest ways to check truth for certainty. Intuition is sharp: you have a feel for the rhythm of the hills and valleys in the problem landscape. You can guess the existence of a distant mountain the hill-climbing rationalist would take forever to find, quickly and soundly check the local landscape for its unexpected little pits and spikes, and borrow a good map to cross a desert quickly.

This engine of canny guesses and rapid checks and good borrowed maps is precisely what makes communities of hackers work so impossibly, frighteningly fast.

Could the two concepts merely be component parts of a more complex feedback loop?

but i guess this strongly depends on your intuition :)

Intuitive thinking is highly valuable when there is no time to collect the necessary data for rational thinking, which is most of the time. When data is available there is little point to intuitive thinking.

They are both very valuable tools, but like any tool it's value is realized by applying it to the right task.

Intuition will sometimes lead to ludicrous solutions...

Guess you have to use rationality to filter out the ridiculous ideas you can come up with intuitively...