I knew before I even clicked on this article, that something inside was going to cause me nerd rage; it's almost always the case when I read these Baby's first UNIX type articles. I thought this one might be different when I saw Andy's formidable neck beard, and for the most part it was decent but then I saw it.
..plus Ack is 4 or 5 times faster than grep.
Ack is implemented in Perl, which has a full featured regex engine, but is not generally known for being super fast. It's going up against GNU grep, which is a highly tuned regex engine, which is very fast. The ack guys make the argument that ack is faster because it doesn't search stuff you, the developer, don't care about, which in my estimation is horseshit. Try it yourself, I threw a regex at my email spool from the 90s which is about 128M. I made sure it was in disk cache first and then did:
chunky:~$ time egrep -c -i '(d|m|j)ingle' RMAIL
23
real 0m3.129s
user 0m3.075s
sys 0m0.051s
chunky:~$ time ack -c -i '(d|m|j)ingle' RMAIL
RMAIL:23
real 0m26.208s
user 0m26.060s
sys 0m0.121s
chunky:~$ otool -L /usr/bin/grep | grep pcre; grep --help | grep perl
/usr/lib/libpcre.0.dylib (compatibility version 1.0.0, current version 1.1.0)
-P, --perl-regexp PATTERN is a Perl regular expression
Many users choose ack over grep for the same reason they choose htop over top, tmux over screen, zsh over bash, and postgres over mysql -- the perception that one is significantly better than the other. This advocacy often comes from folks whose use and understanding of either tool is superficial at best.
htop over top, tmux over screen, zsh over bash, and postgres over mysql
That doesn't seem like a good comparison to me. I use htop over top because it has features that top doesn't have (the funky coloured ASCII graphs). I use tmux over screen because unicode seems to work without any effort on my part. I use bash because I have never looked at zsh in detail and don't care enough. I use postgres because I like its license better than mysql and its not controlled by any one company.
So its not at all about the perception that one is significantly better than the other, at least for me, its about specific features (license is a feature). Or in the case of zsh vs bash, where there are no specific features, I use bash because it seems to be the default on most distros.
I often see ack recommended by word of mouth but unfortunately I feel it's between programmers that don't understand the Unix command-line enough to instruct grep, find, xargs, etc., sufficiently accurately to do what they intend and ack has the allure of doing the right thing, though often slowly, as you say. GNU grep's -I option to ignore binary files for example, or -boa to search binary files.
Once the standard commands are learnt then by all means use ack if you understand the trade-offs and can fall back to the normal way when on another system or when ack's insufficient.
Normally my directories aren't going to have .bzr, .git, .hg, and .svn in the same directory tree, but if they did it wouldn't be that much harder.
alias nocrap="egrep -v \\.\(git\|bzr\|hg\|svn\)"
find . -type f -name \*.[ch]| nocrap | xargs grep whatever
This is how I do it because I'm lazy. I suppose I could tell find to exclude directories so it wouldn't have to recurse into them, but the list of directories and files in my source tree is almost always in buffer cache anyway, so I never really worry about it.
You might want to consider using single quotes to protect your globs and regexps more rather than backslashes. \✳.[ch] will match the ./✳.c that may have been accidentally created by an earlier error, then find will only look for a file called precisely ✳.c and nothing else. '✳.[ch]' or \✳.\[ch] is what's meant.
(Edited to use ✳ to workaround unescapable mark-up.)
> In Grep grep -R 'some term' * , in Ack ack 'some term'. The -R tells grep to search through all directories and subdirectories while the * tells it what types of files.
No, that's not how it works with grep. The shell is expanding the glob * to all the things in the current directory that don't start with a dot and that's what grep sees. It then reads each of them in turn or if they're a directory it descends into them recursively; the * plays no further part in the descent.
I was under the impression that * expanded to all things not starting with a dot in all directories-- or better: in each directory that -R dictates. I understand that "* tells it what types of files," isn't correct, but I don't understand what you mean by "plays no further part in the descent."
The reason there are files that start with a dot is that ls(1) and globs starting with an asterisk ignore them by default. ls needs -a (all) to show them. Some shells, but not bash, provide a globbing syntax for the recursive expansion you think you require here.
But if, as normal, you want to grep recursively down from the current directory then that's what you should specify; just pass grep one path to search, dot, the current directory.
grep -r foo .
This is very basic knowledge of the Unix command line for a programmer and I highly recommend the classic text _The Unix Programming Environment_ by Kernighan and Pike to learn the philosophy behind Unix. It has nothing on window systems, ssh, etc., nor should it have as that's modern-day noise that hides the essence. It and Kernighan and Ritchie's _The C Programming Language_ were the key books every starting-out Unix programmer read. http://www.amazon.com/exec/obidos/ASIN/013937681X/mqq-20
just run echo * in your directory. That's the same argument grep will get(instead of the * ) when you pass it * .
grep will run through all those files, if any, and recurse through all the directories, if any. grep recursing down a directory will process all the files/subdirs in that directory. (which includes . files).
If the shell can't expand the * , grep will receive a literal * and try to open a file/directory named * .
Personally, I `shopt -s failglob' in my .bashrc so any glob that fails to expand is treated as an error by the shell and the command never gets started. I then quote all special globbing characters that I intend to pass to the command as it avoids the sometimes hard to spot errors where a glob expanded unintentionally and the command didn't see the wildcards. Relying on non-matching globbing to reach the command will bite one day.
I've wanted to try zsh for a long time now, but I worry about the mental context switching I'll have to do when developing locally and then ssh'ing into another server to perform other tasks (servers, I should add, where I can't install zsh).
Is anyone in a similar situation? If you switched, was it worth it?
I switched cold turkey from bash to zsh a few months ago, and going back to bash on servers hasn't been a problem. In my experience, if you've used bash for a long time you don't forget it.
Minor tip - you can use the same .aliases file for both shells, I just source it in .bashrc and .zshrc on my dev machine. So if I ever need to switch to bash for some reason, my most used aliases are still available.
I'm actually not very taken with oh-my-zsh. It seems to add a layer of complexity while not really adding anything you couldn't just paste into your .zshrc. If it eliminated tinkering overhead that would be one thing, but in my experience it just gave me a new layer of software that needed tinkering.
It's along similar lines to copying someone else's rc file willy hilly, without understanding what you're getting. Extra layer of quirks and features that you need to figure out, on top of the already existing layer.
zsh can mostly be thought of as a collection of enhancements to bash, so I find that rather than changing my thought process, I just try to use a zsh feature, realize it's not working, and do it the crummy bash way instead. If a feature exists in bash, the zsh syntax is usually identical.
Also just for the record zsh is installed by default on many OSes, including OSX, so you might be surprised to learn that some of your servers already have it.
I've been on zsh for ages, and the thing that always kills me dead when stuck in bash is the terrible default command line editor. Whereas in zsh, if I go back to edit a multi-line loop or something, it preserves the white-space, bash by default presents the commands as a semicolon delimited list on one line. That's ass. Is there a handy bash shortcut to mimic zsh's behavior here?
I've tried living in iPython and eshell, and while I prefer python or lisp as a programming language to almost anything, neither is nearly as quick and dirty as sh for doing stuff with stdin/out from a pipeline of tools. I don't quite know what a better language than sh would look like, but a man can dream.
Also, universe, if you're going to get on fixing the sh problem, can we restructure how the shell operates entirely? Don't do argument expansion! This is one area where the Unix Haters got it really right.
In many (all?) unix shells, globs are expanded by the shell before being passed to the program.
'ls *.jpg'
is passed expanded to 'ls 1.jpg 2.jpg 3.jpg' BEFORE ls is run. This choice means that if a program wants to use another style of regular expression (ie: pcre), it must be enclosed in quotes on the shell, ie:
Yes, that's globbing, but I was wondering what the OP meant by argument expansion. It could be just globbing, or include globbing but extend to variable expansion, braces, command substitution... It's hard to put a case that having the shell do these things is the right way to do it without understanding the OP's point clearly.
See `failglob' elsewhere on this page. What you suggest is sometimes useful but also sometimes wrong and hard to spot because the glob might match locally and one not realise.
As someone who's converted from bash -> fish (arguably a much bigger context switch as fish isn't bash compatible) I haven't found this to be an issue.
Contrast rebellion refers to W3C's Web Content Accessibility Guidelines, which specify a contrast ratio of at least 4.5. Just for fun, I plugged in the numbers, the site has a contrast ratio of about 4.9.
I'll take a look when I've got a few more moments to see where various of the W3C's recommendation levels fall on my own assessment of readability.
Speaking as a 40-something with decent vision, the site isn't impossible to read, but it's most definitely annoying. For someone with compromised or old eyes, it would be much worse.
This was a good list for the intermediate command line user looking to customize. For the absolute newbie I'd reduce it down to git/hg, ssh, grep/ack and curl.
tmux is a CLI terminal multiplexer. As an interactive CLI program, you need a shell to use it. To run such a shell, you need a terminal emulator. iTerm2 is a Mac OS X only GUI terminal emulator.
I don't use any advanced features of iTerm, though, I just use it over Terminal.app because it supports 256 colors. We have not been updated to Lion or Mountain Lion at work so Terminal.app is still limited to 16 colors for me.
tmux - Without a doubt, the biggest upgrade to my workflow in years. If you aren't using tmux (or at least screen), you are doing yourself a big disservice.
Also, I keep seeing iTerm2 get mention. You mentioned split panes and what not, which are not critical when I have that with tmux (as well as the tabs). And while before Lion, Terminal's color support was weak, that's changed. So, I really wonder what the big reason for using iTerm2 is over Terminal?
Because I can shut down my computer, move to another one, and pick up right where I left off. This means even if I leave a process running, it doesn't stop.
tmux is like screen, only it has more features.
Which is really why I asked. What does iTerm2 offer that I can't get from tmux and current terminal? People always refer to iTerms ability to do multiple windows/tabs. For me, this is useless if it's just available client side.
There is an amazing amount of freedom being able to just shut down, move to another computer, and be able to pick up exactly where I left off.
So, first (and most obvious) point of interest. Your function "server" will not work. It contains "python -m SImpleHTTPServer .." .. Notice the typo? It should be SimpleHTTPServer. In addition, that other fluff around it is unneeded in general. Just "python -m SimpleHTTPServer $port" is almost always "good enough", though the SimpleHTTPServer module contains severe bugs. Secondly, your link right above that snipped to Paul Irish contains an href="" which, obviously, links it to your own page. It makes it seem like you're trying to take credit for his work. This is all sloppy and bad form.
Honestly, anyone who puts code into a blog post without even bothering to RUN IT THEMSELVES first should get out of the business of writing even slightly technical posts. Feel free to review food without tasting it first instead; at least then it's subjective so you can't be wrong. Here, the "/usr/bin/python: No module named SImpleHTTPServer" you'd get if you had arsed yourself to actually check your code to be valid easily clues the reader into your carelessness.
In addition to all that, there's the audacity of posting a post for the "novice" to hacker news when the post is clearly neither relevant for any hacker who has performed any real work or has any real intelligence (they would already know all the mentioned tools) and neither is news for all the tools mentioned are fairly old and well known. You might as well just call us all stupid and dilute the front page further with crap.
I appreciate you taking the time not only to read the post, but to offer your opinion and corrections to my article. I've updated those two mistakes you pointed out.
As I understand it, SimpleHTTPServer offers basic functionality to the user, which is all I use it for...a quick way to view pages using AJAX requests. I attempted to use http.server instead, but as I'm not running Python 3.0 I'm not able to use that option. Since Lion ships with Python 2.7, and many people are using that OS, I felt safe leaving the reference in place.
I do run the server function myself, many times a day in fact, but I made the mistake of typing it in manually rather than copying and pasting from my .zshrc file.
This post isn't intended for hackers, thus the title "...for the novice". The comments I've received about the article thus far, most likely from people not as advanced as yourself, have been positive and appreciative for the information. I'd love to hear some of the tools that improve your workflow daily...would you care to share them?
..plus Ack is 4 or 5 times faster than grep.
Ack is implemented in Perl, which has a full featured regex engine, but is not generally known for being super fast. It's going up against GNU grep, which is a highly tuned regex engine, which is very fast. The ack guys make the argument that ack is faster because it doesn't search stuff you, the developer, don't care about, which in my estimation is horseshit. Try it yourself, I threw a regex at my email spool from the 90s which is about 128M. I made sure it was in disk cache first and then did: