I find that the `pbpaste | something | pbcopy` idiom is common enough that it's worth having a shell function for it:
pbfilter() {
if [ $# -gt 0 ]; then
pbpaste | "$@" | pbcopy
else
pbpaste | pbcopy
fi
}
Then you can use something like `pbfilter json_pp` or `pbfilter base64 -d` or `pbfilter sed 's/this/that/'` or whatever.
This version also can also act as a plain-text-only filter. If you just use `pbfilter` with no argument, it'll remove any formatting from the text in the pasteboard, leaving just straight plain text.
It does have a some limitations, though: you can't use it with an alias, or pipeline, or anything complex like that. The filter command must be a single regular command (or function) and its arguments.
I love this flow! Such a powerful and clean way to solve text issues.
# This will remove Windows double-spaced empty lines from your copy/paste buffer
alias winlines="sed '/^$/{$!{N;s/\n//;};}'"
# pbw = [P]aste [B]uffer to fix [W]indows line endings
alias pbw="pbpaste | winlines | pbcopy"
Also - if you want `pbpaste` and `pbcopy` on Linux...
# imitate MacOS's paste buffer copy/paste:
alias pbcopy='xsel --clipboard --input'
alias pbpaste='xsel --clipboard --output'
I find it so annoying that these only work with plain text and RTF. On X11 there is `xclip`[0] and on Wayland there is `wl-clipboard`[1] both of which support binary file formats either through parsing the header or explicitly setting the MIME type.
This means you can do things like copy an image from the terminal and paste it into a graphical program like a browser or chat client and vice-versa. Also can be very useful in shell scripts for desktop automation.
The workaround on MacOS is to use AppleScript via `osascript` to `set the clipboard to...`.
It creeps me out when the clipboard is unexpectedly shared between my phone and computer. And since the feature seems to turn on randomly but not reliably when I want it to, I’d rather it just didn’t exist.
I have a clipboard manager application called "Paste" (creative i know). Its an awesome app for a million reasons. But one thing I like is that it allows me to see and hear when my iphone copy worked.
So I have it enabled so there is a sound when something goes into the clipboard. Even on my mac, I have come to rely on that audio feedback. But it has the added benefit that when I am using my phone in front of my computer and I copy something on my phone, I immediate (and it is impressively fast... maybe a 200ms delay), I hear the chime that something was added to my clipboard on my mac. So it gives you that good feedback that a copy "worked".
You can also shift+cmd+V to see the clipboard history, which is another complimentary tool with universal clipboard because if a paste isn't working as expected you can see if the universal copy never "took" (as you mentioned it is semi-unreliable), or if it just got overridden. You can then use the navigator to paste the older item.
I've noticed that more and more apps on both macOS and iOS sniff the clipboard contents and randomly clobber it. I usually notice it in apps like Sourcetree, where I'll click something or do a certain action and suddenly I can't paste anymore. I even get a feel for it, like my mind detects the pattern that empties the clipboard so I sense when I can no longer paste, but I can't figure out concrete repeatable steps to make it happen. On iOS it's more random, and I feel like it's probably Facebook doing it, or maybe websites in Safari. I just assume that everything is spying on my clipboard contents now, hoping to log secrets/passwords and PII to sell to scammers.
I have to say, this is one of the more disappointing developments from Apple, that they certainly must know by now about these clipboard shenanigans, but have done nothing to stop them. They need to implement permissions that deny all apps the ability to get/set the clipboard by default, and have an option to ask the user whether so-and-so app can access the clipboard (outside of normal copy/paste), every time with the option to allow always. And all clipboard access attempts should probably get logged somewhere.
I would think requiring opt-in for clipboard functionality would be the more radical option that would leave most users (myself included, I would imagine) scratching their heads when they can’t copy/paste as a matter of course. Maybe you meant something more specifically related to 3rd-party sniffing/modifying clipboard contents, but I haven’t really encountered that outside of apps such as CopyQ and Paste, and they are pretty explicit and intentional about their functions.
I have found a lot of utility with cross-device copy/paste. I know it requires the somewhat mysterious phantom Bluetooth/Wi-Fi connectivity that AirPlay/Airdrop use, so if I have disabled Bluetooth on my device, for example, it will no longer work. I could see where it might not be fully reliable enough to count on, I have experienced inexplicable failures, not often but enough to understand that it might not be some folks’ default preference. As part of the “handoff” function, it can be disabled in Settings at least.
Some of the “not working” cases may be due to the application you’re copying from setting the items as `localOnly`, ex: from a password manager. I don’t have an explanation for other failures.
I’m on the same version of 1password, probably for similar reasons.
In their iOS app, there’s an entry in Settings -> Security -> Allow Universal Clipboard which lets you opt-in to passwords through the clipboard. I suspect there’s something similar on macOS.
Yes, I use this app multiple times a day, each time with the added satisfaction of knowing I own the app and not a subscription. Even if my credit card expires I will still have access to my passwords.
There’s a fix for it where you remove all the unlock entries in Keychain, and then re-enable it. Only thing that worked for me long term (but it did fix it 100%)
I tried this yesterday. It... helped. But what I've now found is that if I escape out of the password prompt (which will turn the screen off again) and then try to unlock it a second time, the watch unlock will be triggered. I wonder if it's something about waking from sleep. (I also wonder if it would have worked before to do the same thing, but it never occurred to me to try.)
Requiring WiFi makes (so the phone/computer is on a network and can communicate with the other devices), but what's the benefit of Bluetooth? Does it only work when the phone and computer are near each other?
Bluetooth is used to discover peers and to initiate communication. Thus handoff works even on a wifi network that blocks broadcasts. It even works when no wifi network is present, by setting up an ad hoc network for the connection. (Disclaimer: This is all I know. The details seem rather murky, as handoff is a proprietary Apple protocol.)
This is great utility I didn't know about! Thanks!
But do you know why it doesn't seem to work with the `pbfilter` function?
If I do directly `pbpaste | vipe | pbcopy`, then it opens vim and the clipboard text is pasted there.
But if I run `pbfilter | vipe`, then vim opens with a blank buffer.
function pbfilter() {
if [ $# -gt 0 ]; then
pbpaste | "$@" | pbcopy
else
pbpaste | pbcopy
fi
}
It seems that the number of args is 0 for some reason
So much of my Linux use is over ssh from a MacOS client that I've made a `pbcopy` executable that just pipes stdin over ssh to my MacBook to its pbcopy (with a dedicated ssh key that runs this as a forced command). Makes it super nice to be on an SSH session and `pbcopy` some content to my MacOS clipboard!
I'm also super interested in this! I've had many amusing moments of instinctively typing either `pbcopy` or `pbpaste` on remote boxes followed by a brief moment of confusion when my local clipboard isn't updated :)
I have! Unfortunately not supported in MacOS Terminal.app, which I'm otherwise very satisfied with (have tried iTerm2, use Alacritty on Linux, just like Terminal.app).
I don't think wrapping my entire shell session in a moderately complex third party tool (that maybe just uses pbcopy under the hood[1]) counts as "simply" when compared to my existing solution which just pipes over ssh and a couple bash scripts.
This tool isn’t doing anything particularly complex.
It sets up a new pty, attaches the child process to it, listens for OSC52 control codes, and calls pbcopy when appropriate.
You can wrap your ssh session with it and you’re done.
It’s very elegant and multiple orders of magnitude less complex than something like tmux.
I should use `git format-patch` instead of creating a mock draft PR (which I end up deleting) and modifying the URL to add `.patch` and then downloading the patch file haha. `git format-patch` would probably be faster :)
Speaking for myself, the first form is more natural- even if it’s a useless cat, because I’m always cat-ing files to see their structure. Then progressively tacking on different transforms. And then finally putting it in whatever I want as output.
It’s so ingrained, I’m more likely than not to just write it out that way even when I know exactly what I’m doing from the onset.
generally, speaking, if you don't have an idea of how big the file is, or it would take up too much real-estate on your terminal window, sure. 100%. It was just an example.
lot's of times we sort of know what we are working with, but don't remember the particulars especially
I really recommend folks use "less" over cat, especially keyboard oriented folks. Different terminal emulators don't always have the scroll behavior I want, not do they always allow me to search the file I'm looking at. "less" does all those things, in nearly every environment no matter the terminal emulator, and has other wonderful options to boot (chop long lines so they don't wrap can be nice for logs, line numbers can be VITAL, etc).
I still uselessly use cat though, it's such a nice way to build a pipeline.
As a scientist who cares about reproducibility, the big difference between the "useless cat" and providing the input file name on the command line is that, in the latter case, the program can capture that file name and reproduce it. That is harder when using stdin.
Many of my programs and scripts start output with the line:
# cmd arg1 arg2 arg3 ...
and simply echo back lines that start with '#'. That way, I have an internal record of the program that was run and the data file that was read (as well as previous parts of the analysis chain).
And, 'R' ignores lines starting with '#', so the record is there, but does not affect later analyses.
The “useless cat” meme needs to die. Everyone is aware that most commands accept a file argument, but looking up the arguments and their ordering is annoying and using cat for things like this is just fine.
granted, it is a little snarky and maybe the snark isn't appropriate in today's tech environment. but no, things like "useless use of cat" do not need to go away, because they make me better at what I do in little ways. those little ways add up over time.
> but looking up the arguments and their ordering is annoying
you seem to be arguing for complacency. taking your idea to an extreme, why learn to do _anything_ well?
It usually doesn't matter much, but there are some situations where it can matter a lot. For one thing, you can't use seek() on a pipe, so e.g. `cat bigfile | tail` has to read through the entire file to find the end, but `tail bigfile` will read the file backward from the end, completely skipping the irrelevant beginning and middle. With `pv bigfile | whatever`, pv (which is basically a pipeline progress indicator) can tell how big file is and tell you how for through you are as a percentage; with `cat bigfile | pv | whatever`, it has no idea (unless you add a flag to tell it). Also, `cat bigfile | head` will end up killing cat with a SIGPIPE signal after head exits; if you're using something like "Unofficial bash strict mode" [1], this will cause your script to exit prematurely.
Another sometimes-important difference is that if there are multiple input files, `somecommand file1 file2 file3` can tell what data is coming from which file; with `cat file1 file2 file3 | somecommand` they're all mashed together, and the program has no idea what's coming from where.
In general, though, I think it's mostly a matter of people's expertise level in using the shell. If you're a beginner, it makes sense to learn one very general way to do things (`cat |`), and use it everywhere. But as you gain expertise, you learn other ways of doing it, and will choose the best method for each specific situation. While `cat |` is usually an ok method to read from a file, it's almost never the best method, so expert shell users will almost never use it.
If the command is meant to stream through something really fast by using a large buffer size, then prepending a cat(1) will limit the incoming buffer size to ~4k.
I teach shell scripting. Cat invocations are cheap and help learners understand and keep clear where input is coming from, and where it is going. There are no awards or benefits to reducing the number of lines, commands invoked, or finding the shortest possible way to perform a task in a script. There are plenty of detriments to reading and understanding though when we try to obfuscate this to save 1ms of execution time on a script that is going to execute near instantaneously anyways.
I 100% agree with you. My only defense of OP is that `<` is something tends to be forgotten. Like everyone else in this thread I go to `cat` first for things like this. But sometimes I forget that even `<` exists, and the callout is a nice reminder.
Not just that, but also all the bytes have to go through an extra pipe. Presumably they're copied an extra time because of this.
When you run "cmd < file", the command reads from stdin, which pulls directly from the file. When you do "cat file | cmd", "cat" opens the file, reads from there, and writes to a pipe. Then "cmd" reads from its stdin, which is a pipe.
GNU cat will use the copy_file_range syscall when possible!
copy_file_range allows a user land program to copy data between two files without doing any user space work. Instead of reading data into a buffer and writing it back out to the destination, the kernel will somehow manage to move the data for you.
I think this will prevent any extra copies from occurring in situations where it can be used.
To add, searching for “useless use of cat” will yield several results for those interested in learning more. Other examples include “useless use of echo” and “useless use of ls *”.
All of them do. Including bash. It’s just not the same syntax (ie ‘< filename’).
But I honestly think people who try to optimise away ‘cat’ are optimising the wrong thing. If one extra fork() is that detrimental then don’t use a shell scripting language.
For a lot of people, “useless” ‘cat’ enables them to write a pipeline in the order that their brain farts out the requirements for the pipeline. So they’ve optimised for human productivity. And given the human brain is slower than a few extra fork()s, I think optimising for one’s brain makes more sense here.
Literally the next sentence after the one you quoted explains my point:
> It’s just not the same syntax (ie ‘< filename’).
Reading from a file isn’t a hard problem. Having a good UX for doing that is where most shells fall apart. And that’s basically what ‘cat’ offers here: an improved UX.
Having ‘cat’ as a shell builtin wouldn’t really solve the complaints raised by “useless use of” anyway because you’d still be piping (and in some cases, fork()ing too). You couldnt really use ‘cat’ as syntactic sugar for ‘<‘ because things start to get really weird if you want to pass flags to ‘cat’ or even redirect the output to something other than a pipe. And given ‘cat’ is POSIX (https://en.m.wikipedia.org/wiki/List_of_Unix_commands#/media...) the current behaviour of shells is, in my opinion, correct. This is why my own shell has a differently named builtin that approximately serves the purpose of ‘cat’ but for instances when you need the command built into the shell and it can’t just be passing a file handle to the next command (in my case, because i wanted to pass metadata out-of-band as well as the file contents)
And sent it straight away to my coworker on Slack.
I then spent the rest of the day making a new private key and adding my new pubkey to all of the 1000+ servers I had root access to. I mean we had tools to help but it still wasn’t fun.
With great power/convenience comes the potential to do dumb things at lightning speeds!
also userify allows you to set up sudo access on some of the servers and not others, so that'd take care of the other root-access issue you have. (sudo also provides auditing/logging controls that are useful in a multi-user environment)
That's not a bad idea. I've never actually made the same mistake, but I have caught it at the last moment and having tab complete not pick the private one first would help.
It would have avoided it! I was using tab and forgot to select .pub as you correctly surmised. I was a junior dev at the time and all the seniors got a good laugh out of it, and I use it as a cautionary tale about trying to be TOO overeager and efficient.
On Linux I have these wrap xsel or xclip, and likewise open to xdg-open.
Now, for your Mac example — if that's a specific pipeline you often use, you can write a Service menu entry to do it in place, without switching to a terminal.
Since you mention both pbcopy and iTerm - I love https://github.com/skaji/remote-pbcopy-iterm2. I do most of the work on a remove Linux server, treating my MacBook as mostly a dumb terminal, and being able to transparently copy from the remove to my local clipboard is so nice.
I have tried it, but for whatever reason I just don't like it. I prefer just running tmux in iTerm with no integration.
On the topic, you can also integrate tmux with the native clipboard - I have set copy-pipe to the remote pbcopy, so any selection done in tmux get copied to my local clipboard. I also just found out that tmux also support it natively (https://github.com/tmux/tmux/wiki/Clipboard#the-set-clipboar...).
This is interesting, thank you. I've been automating various things with Hammerspoon but (I think) it's limited to what you can reach with either a11y or osascript / the NS dictionary for the app you want to manipulate, but Shortcuts seems to have some actions that aren't in the NS dictionary.
For instance, in Shortcuts, I see that there's a "Pin Notes" action for Notes.app, but I don't see anything for pinning notes when I open Notes.app with "File -> Open Dictionary..." in Script Editor.
(In this case it's likely that Notes.app has the a11y bits necessary to run that action from Hammerspoon, but it would probably be easier to go through Shortcuts.)
Yes. I use it a great deal, but I haven't gotten used to using the linux equivalents. I guess that would be either xsel or xclip. Maybe I should create a "pbcopy" that runs one of those. I like to minimize the cognitive load when I switch between mac and linux command line environments.
Oh man. I recently threw together a "j2p" script to make converting between json and python dicts simpler, and combining it with pbcopy/pbpaste will make it so much better:
$ python -m json.tool --help
usage: python -m json.tool [-h] [--sort-keys] [--no-ensure-ascii] [--json-lines] [--indent INDENT | --tab | --no-indent | --compact] [infile] [outfile]
A simple command line interface for json module to validate and pretty-print JSON objects.
positional arguments:
infile a JSON file to be validated or pretty-printed
outfile write the output of infile to outfile
options:
-h, --help show this help message and exit
--sort-keys sort the output of dictionaries alphabetically by key
--no-ensure-ascii disable escaping of non-ASCII characters
--json-lines parse input using the JSON Lines format. Use with --no-indent or --compact to produce valid JSON Lines output.
--indent INDENT separate items with newlines and use this number of spaces for indentation
--tab separate items with newlines and use tabs for indentation
--no-indent separate items with spaces rather than newlines
--compact suppress all whitespace separation (most compact)
And I have one that unifies _both_ to `clip` so you can put the same command in both sides of the pipe, e. g. to turn a line-delimited blob on your clipboard to a space-separated one:
I've aliased that (and its equivalents on Linux and Android/Termux) to 'xc' and 'xp' (for X11 Copy and X11 Paste, as I'd originated this on Linux).
Being able to populate or read from the system clipboard (or secondary clipboard!), or to feed it, including by reading from or writing to pipes is wonderful.
alias pbg='pbpaste | fgrep --color -i "`pbpaste -pboard find`"'
select all in a terminal window with pages of log data and cmd-c copy; find the one phrase you want to find in that data and cmd-e to put it in the find pasteboard; cmd-n new window, type pbg to isolate the log lines.
I recognize that your pbg alias works for pretty much any text you could copy, but I wanted to mention, in case you're looking at log files with plain old less, there's the & limiter, which limits the current view to only lines matching a regular expression (or, if you type ^R during a & prompt, for a text match).
If you type ^N or ! during a & prompt it will limit the view to those lines that do not match the expression.
These view limits stack, so you can "&WARN<enter>" to see all lines that have WARN in them, and then maybe you want to see just a certain PID so "&12345<enter>" and you'll only see lines with both WARN and 12345, but then that one module is printing out a bunch of messages you think are safe to ignore so you do "&!modulename<enter>" and it filters out log lines that match modulename. Very handy and less is everywhere.
I don't remember when I learned about these, but they've been game changers, and everyone I've shared them with feels the same way. I use your use case often as well, though through `jq` because I'm more familiar with it, and sometimes wish to do transforms.
You can use `python -m json.tool` for just JSON formatting, which is convenient now that Python is available by default in most Linux distros. Jq is really excellent though.
I have an alias that while trivally simple is quicker to type and remember. It copies whatever file you give it to the clipboard which is super handy. I use it with the "Compare with Clipboard" to diff a file in Rubymine for example.
Could be an alias, I have a limited set of aliases for each type of system. But I keep a repository of hundreds of personal shell scripts and it fit better there.
Dealing with some minified json, switching to iTerm, doing `pbpaste | json_pp | pbcopy` and having a clean output is _so_ nice.