I don't know Lisp, so correct me if I am wrong. The whole idea of embedding your own language sound pretty much the same as a growing the language, except that you are doing it yourself, in a non standard way.
If you build one new feature that subsumes N>1 old features then you're effectively shrinking the language because you never need to use those N old features any more. You can then support legacy code with macros that compile the old features into the new feature. At that point you've actually shrunk the language. You can even have those legacy support macros give compile-time warnings that the old, now-deprecated feature is being used and that the code ought to be changed. In fact, you can even easily write tools that will do this translation automatically. So yes, you really can shrink the language.
But even then, you have only shrunk the language for you - unless you can get everyone else to use your new feature. But if several such things are proposed, typically none of them gains traction, and so they all stay as just individual solutions.
Which is still fine - for the individuals who write them. But the language, as generally used, does not shrink.
Obviously anyone who chooses not to use a particular macro will not reap the benefits that macro provides. That doesn't change the fact that it's possible to write language-shrinking macros in Common Lisp, and so it's ironic to choose Common Lisp as the poster child for how languages inevitably grow without bound once they reach a certain level of complexity.
Indeed. Lisp is a poster child for how you can have one standard put out in 1994, and everything is reasonably fine 21 years later.
I really respect that about Common Lisp: the committee had the good sense (lacking in C and C++ and their ilk) of going their separate ways when they got the job done.
Good sense in a way that is related to good taste, I should add.
The way Common Lisp is generally used is by writing macros that shrink the language at higher levels. It's used that way because it can be. Whether one considers the lack of such capability in other languages a handicap or feature doesn't change that it is common Common Lisp practice. There aren't Common Lisp shops that don't use macros since Common Lisp includes standard macros.
>unless you can get everyone else to use your new feature.
When you are writing functions, classes you are extending language in similar way, and everyone else have to use this extension. You just not used to macros and your opinion is based only on your habits.
There are many articles on lisp that explain that in lisp you can try out many things in a few evenings and find suitable solution. In this way you shouldn't be smart enough to make an ideal, general design in one turn, but you will find it through many iterations. And this is what makes lisp different - these iterations are very cheap.
The 'standard' thing is both a gift and an issue. You're free to fulfill your needs, but everyone will do so. The community needs to be mature and sensitive, sharing good ideas, not building silos.
Both rigidly defined and flexibly defined languages appear to have their issues. I'm not much of a historian of Lisp but my basic understanding is that it's great for lone developers, but a pain for larger teams, and the flexibility of the language is the reason behind both of these.
In the Go language the tool gofmt formats the source code to certain standards. I haven't heard a single developer complain about gofmt, developers seem to appreciate that it takes away the bikeshedding over style, there's one established way to lay out your Go code and that's to use gofmt.
Now if bikeshedding can happen over something as trivial as code layout, imagine what's going to happen if you make it trivial to change the language. Someone didn't design a regex library the way you like it? Code your own. So will everyone else. You end up with thousands of developers reinventing the wheel, all building their own versions of the same thing, that aren't necessarily compatible.
CL now has its own package manager (Quicklisp), I'd be curious to know if this has had an impact on fragmentation, that's CL's best bet for reestablishing itself as a language that's growing in use.
I'm calling out this nonsense; "larger teams" of Common Lisp programmers are a myth. If such a thing happens, they will make it a priority to get along so that their good luck persists as long as possible. :) :)
> there's one established way to lay out your Go code and that's to use gofmt.
There is basically one established way to lay out Lisp, and even Vim's Lisp mode gets it about 95% right. (The main issue is not quite having the correct vocabulary, out of the box, of what symbols denote forms that have a body requiring sub-form indentation rather than list element alignment.)
(operator (with maybe some args)
(and a)
;; here is a header comment
(body of forms) ; margin comment
(indented by one level such as two spaces))
(function call with many arguments
split across several
lines lined up with the first argument)
'(quoted list
split across lines
lining up with first element
(oh here comes a nested
list))
;;;
;;; three semi's for block comment
;;;
#(vector literal
follows quoted list)
The above represents the majority of what you need for writing nice looking Lisp.
You've missed my point. I used gofmt as an example of a tool that exists because programmers like to do things multiple ways, even when it can be beneficial to do things in a standardised way.
The formatting of Lisp is not what I was referring to, it was the flexibility of the language. Lisp encourages writing DSLs. DSLs by their nature are opinionated. The only way to stop the language going in lots of directions at once is to have a canonical implementation of supporting features to work against. My point was that without a package manager, this focus was not as strong as it should be, but there's a hope that with standardisation on packages the language would grow with a sense of direction.
To use another example, look at how many implementations of CL and Scheme there are, why does all this fragmentation exist? What barriers are in place for coders working on extending a smaller subset of implementations?
For what it's worth, I'm not saying this is a Lisp-only trait (for another example in the computing world, just look at how many Linux distros exist), but the simplicity and flexibility of Lisp does lend itself to fragmentation. Whether this is a good thing or not is a matter of debate.
Out of interest, what impact have you noted since Quicklisp began being used?
> To use another example, look at how many implementations of CL and Scheme there are, why does all this fragmentation exist?
For Common Lisp the reasons are: competition, different licenses, create proprietary property to sell, different runtime implementation strategies, different compilation strategies.
Basically the same reason why there are several different Java implementations.
Common Lisp was created, because it was clear that there is a fragmentation of Lisp, but there shouldn't one for the basic language. Originally it was also thought to standardize on some libraries, but the whole thing ran out of steam. But you can read the Common Lisp standardization groups on various topics.
Quicklisp discourages people from re-inventing the wheel because it makes the pre-existing wheels trivial to access. No one in their right mind would try to reinvent regexps in Common Lisp (except perhaps as an exercise) because CL-PPCRE is just so fucking awesome. There's a CL library for just about everything nowadays, and most of them are very high quality.
That's what I meant about being mature. Lisp literature spends a lot of time explaining when to or not to use macros. Don't be silly and write something that won't be a game changer just because you want regexes literals like expressed #r#.... instead of (rx ...). I'm pretty positive any seasoned lisper can sense when abstractions must be added.
Btw, large teams may exist because tools don't solve the problem easily.
ps: It would be interesting to ask the CL community about Quicklisp social effects.
I don't think it's much different than the library situation with most other programming languages.
The one big difference is that Lisp libraries can add new controls structures to the language in ways that most language's libraries can't.
For example, in Python, I can fetch a website using the stdlib's http.client, or I can use the more convenient functions from the Requests library.
In Lisp, I can iterate and loop using the built-in "loop" construct, or I can use the more convenient control structures defined in the Iterate library.
The standard way to embed your own language is to use a Lisp to begin with.
Any Lisp language is effectively designed to provide standard tools and mechanisms for adding new language constructs so that you can write your own language without having to write either a full implementation from the tokenizer to the interpreter or use non-standard glue to hook an existing scripting language into a compatible C runtime.
The key idea is to grow layers of your own DSLs embedded in a Lisp. It is a bottom-up process of growing of a layers of abstractions, similar to building if a house (but, paradoxically, without blueorints) which some people called 'exploratory programming'.
PG in his "On Lisp" book explained this process in details. In some sense, those who never read it cannot claim to be a Lisp programmer.
So, you are not extending the language, you are growing a several new ones (layers) according to the domain at hands.
This is good one!) The insight were there for s long time - DSLs were popularized in SICP and AoMOP, and CL's looping expressions and SETF/GETF are the most famous, but PG is good at concise writing and he walked his talk with the Arc language.)
Don't get me wrong, I liked On Lisp a lot. I just think it's important to recognize that much of it is a distillation of pre-existing understandings and traditions.
