Here's my observation: People buy products with features, not products that let you build features. Programmers are no different. Most of them will choose a language that does x over a language that can easily be extended to do x, even if the library that provides x has already been written. The fact that their favorite language doesn't do y and z doesn't matter because they don't know about y and z, and the fact that their favorite language prevents them from inventing alpha and beta doesn't matter because they think in terms of using language features, not extending the language. They want something they can use without thinking very much about it. They don't want to change it.
Of course it is hard to do programming without doing any metaprogramming at all, so new languages, like C#, have some basic metaprogramming features. But you have to market them as features of an otherwise fixed programming language or you'll scare people off. It's all about minimizing the number of things your customer has to think about. It's even worse if management is driving the choice/design of the language.
I'm not arguing things like LINQ shouldn't come with the language. I'm arguing that I should be able to add them if I want to. In Smalltalk (or Lisp, or Scheme), I can. In C#, I have to wait for Microsoft to get around to it. Look at the situation with lamdbas in Java, if you want another example.
That doesn't change the fact that I'm very glad most Smalltalks ship with ROE or GLORP, or that C# ships with LINQ. It just says that I want to be able to add features as easily as the language designer can.
I've expressed exactly this point to many people involved in C#. Partly, they do not appreciate that most PL features can be implemented as libraries in more powerful languages. Another part is they understand that blub programmers are interested in features, not libraries. Being popular means appealing to the masses.
If a language does not have closures nor more complex control flow features, you really can't implement PL features as equivalently easy-to-use libraries. IMHO, that's why most mainstream languages continue to add superficial features rather than fix the core problems in their PL.
I'm working with matlab currently. It is simultaneously the best engineering tool ever and the worstest, lousiest development environment ever. It has tremendous shortcuts for doing things quickly and powerfully but everything has been added in haphazard, half-cocked, half-assed fashion.
I can see how the two face of matlab have evolved together in the way that Perl has amazing power and horrible maintainability. I don't see that they would have to have evolved together. Ruby, for example, does what Perl does but with more sanity.
Matlab is a horrible language, although at least it has lambdas now. Have you looked at SciPy? It can't replace some of the more specialized toolboxes of course.
Matlab is a requirement for what I'm currently doing.
I'll research lambdas.
But I meant it also in saying that Matlab is the greatest engineering tool ever. You can do amazing things in 200 lines. Then you do 400 lines and get caught in some BS for days. But if you remember to do only 200 lines (even just 2 lines), then you can be a very, very good and feel very productive.
In my experience, Perl has inherent problems with readability on a line-by-line basis. I don't think that there's anything that can be added to change this - testing would prove a line does certain things but given that a test isn't complete specification, testing couldn't by itself change the readability problem.
Again, Perl would have more justification if Ruby wasn't visible
If someone references a book with author/title, PLEASE at least check a review on the web before commenting.
I was hoping that any answer might be insightful.
(Readability is mostly a function of familiarity. Everything has advantages/disadvantages, if I'd list the problems with Perl, readability wouldn't be there.)
This is mostly true. Why do people use Rails instead of Catalyst? Because they can see everything at once, and they know they won't realize halfway through their project that they are doing something wrong. People hate learning, and making sure there's nothing to learn will make a language (framework/environment/whatever) popular. Sad but true.
(I wrote a whole book on how to best use Catalyst. A year later, it's nearly all wrong. This seems to upset people, as now they "are wrong". But it doesn't really matter; they can incrementally move in the direction of what we consider "not wrong" today, and, wrong or not, it's not like their app doesn't work. Personally, I am happy that everything becomes outdated so quickly, as it means that I can be more productive, and my apps can be easier to write and maintain. But people are weird...)
I think programmers will also trade actual flexibility for an IDE that appears to make their language flexible. Are there any really popular languages today where bloated IDEs aren't "required"? (I consider Java and C# to be the "really popular" languages today. I don't use them, but millions of other people do.)
The main problem with all of this is that programmers aren't really responsible for deciding which programming technology will be used. A programmer wants macros. A hiring manager wants the ability to throw a dart at a list of names and find a programmer that knows that language. Guess whose input wins? (Combine this with the fact that people think programming is a "cool" profession, and you end up with 90% of the practitioners not actually knowing how to program or what programming is. Does it surprise anyone that cool GUIs beat complex computer science topics?)
The good news is, you don't have to be part of the problem. There is hardly any reason to use the "popular" languages. They don't have better platform support, they don't have better libraries, and they don't have better tools. So don't worry about them, they are for other people :)
On the contrary, there are lots of reasons to use the popular languages. The reasons mostly have to do with straightforwardness, lots of well-tested libraries (even if they do things in a way that's not the most clever), lots of people who've already had any problem you're likely to encounter, as long as you're doing things in the obvious way, etc.
These things together mean that using a popular language in the way that most people use it is easy -- it's the path of least resistance in a lot of instances. Using languages that are cool, have neat features (or the ability to build them), and/or have higher abstractions is fun for play, but it turns out to feel like you're swimming upstream when you have to do the boring requirements-laden task that you've been handed.
There's more than one reason for that, but collectively they add up to why I use PHP for things I just want to get working quickly, or for job-like things, and lisps or other small-community languages when I'm doing something just for fun.
> The reasons mostly have to do with ... lots of well-tested libraries ... lots of people who've already had any problem you're likely to encounter, as long as you're doing things in the obvious way, etc.
Economics has given us the wonderfully pretentious sounding term "positive network externalities" to describe products that become more valuable, the more people who use them. Simple concept, big, impressive term.
Many languages have this kind of community IMO. After a certain size, it doesn't matter really how big the community is because there is enough to sustain help. After a certain point, it only helps in finding specialized sub-communities which need to be that minimum size too. I think almost any language that a person here would consider for some project or another has that community, and you'll really notice the difference w/ languages that don't. For example, SML is a language that is so tiny that you can really notice the lack of community (and free documentation...) compared to, lets say, python, or objective-c.
>The good news is, you don't have to be part of the problem. There is hardly any reason to use the "popular" languages. They don't have better platform support, they don't have better libraries, and they don't have better tools. So don't worry about them, they are for other people :)
This is why I'm not interested in Clojure, regardless of how nice a lisp it may be. Java libraries are poorly-designed, the prospect of wrestling with them again brings back memories of frustration. And you can bet that no one will ever write better ones, because the community will discourage people from "duplicating functionality."
"Java libraries are poorly-designed, the prospect of wrestling with them again brings back memories of frustration."
One of the best things about Clojure is how it makes Java libraries suck much less. For example, line-seq turns a Java BufferedReader into a lazy sequence of String's, one for each line in the file.
You still have the suckiness of the multiple lines it takes to instantiate and clean up the BufferedReader, but macros like with-open makes that suck a little bit less, too.
Stir in some macro-fu of your own, and now you have the benefits of Java libraries minus the boilerplate suckiness. Basically, it turns Java into a "your language features are my libraries" language, no mean feat.
So your criticism of Clojure is based upon unfounded assumptions about what the community will do in future? Maybe you should look at what libraries are being created before you leap to conclusions?
There are many Clojure libraries that attempt to reimplement existing Java libraries. For instance, Ring is a HTTP specification for Clojure that reimplements much of the functionality of Java servlets.
"Most of them will choose a language that does x over a language that can easily be extended to do x, even if the library that provides x has already been written."
I nominate Ruby as the first "your language features are my libraries" language to become legitimately popular. This is likely due to the fact that Rails provides the libraries that most web programmers want, in addition to the cool language features. But it also seems that the "Rails community" is enamored with how easy Ruby's flexible syntax makes it to extend the language in interesting ways. Thus, all of the fuss over adding methods to base classes, and whether or not, or when, it is a good idea to do so.
So I would say that Ruby has brought the idea of language features as libraries into the mainstream, as much as anything as geeky as programming languages can be considered mainstream.
If a language includes more than just processing directives and code organization then that becomes more of a language/platform/framework than just a language. Often these are geared to a specific domain such as desktop or web software and try to give you tools that you'll most often need.
If you don't need additional libraries then you are probably doing things that the authors of the language/platform/framework thought would be necessary.
The other language language/platform/framework, which I am familiar with, is ColdFusion. If you are doing web development and fit with-in only the functionality available to you by default then you are fine. If you want to abstract away certain things or do more than average web development then outside libraries are necessary, especially in the user-interface side of things.
Well, some languages have so much apparent power that it's easy to be seduced into thinking you hardly need libraries, since you can just build it yourself right quick. Common Lisp, for example.
Of course it is hard to do programming without doing any metaprogramming at all, so new languages, like C#, have some basic metaprogramming features. But you have to market them as features of an otherwise fixed programming language or you'll scare people off. It's all about minimizing the number of things your customer has to think about. It's even worse if management is driving the choice/design of the language.