How do I combine -I and -n with xargs?

xargs -i
"xargs -i {}" example
xargs '-i;
xargs replace string
xargs capital i
"xargs -n 1"
xargs placeholder
"xargs -i%"

I want to move a large set of files using find and xargs. Normally I'd do this:

find /foo -name 'bar*' | tr '\n' ' ' | xargs -I % echo mv % /dest

However, when there are too many files to move, I hit the limit of the number of arguments to pass to mv. xargs has a -n which seems like it would be perfect for this:

$ echo {0..9} | xargs -n 3 echo
0 1 2
3 4 5
6 7 8
9

However, -I implies -L 1, so I can't use -I with -n:

$ echo {0..9} | xargs -n 3 -I % echo % /dest
0 1 2 3 4 5 6 7 8 9 /dest

I was hoping for behaviour like this:

$ echo {0..9} | xargs -n 3 -I % echo % /dest
0 1 2 /dest
3 4 5 /dest
6 7 8 /dest
9 /dest

Is this possible with xargs? I don't have GNU Parallel on my machines.

The mv command (at least from Linux coreutils) has the convenient -t flag that perfectly matches this use case:

find /foo -name 'bar*' | tr '\n' ' ' | xargs mv -t /dest

Above also supports keeping any weirdo filename without filename massaging:

find /foo -name 'bar*' -print0 | xargs -0 mv -t /dest

If for whatever reason you want to use mv as usual, below could also work (i.e. use a sh scriptlet to "consume" all arguments ($@)):

find /foo -name 'bar*' | tr '\n' ' ' | xargs sh -c 'mv "$@" /dest' --

8 Practical Examples of Linux Xargs Command for Beginners, , and then pass the name of the file (or type of files) you want to find to search as input through stdin. Combine multiple pdfs, potentially with spaces in their name, using pdftk and xargs - combine-pdf.sh

With GNU Parallel you would do:

find /foo -name 'bar*' |
  parallel -X echo mv {} /dest

This works for file names containing ', " and space. If the file names contain '\n' then you have to resort to:

find /foo -name 'bar*' --print0 |
  parallel -0 -X echo mv {} /dest

If you do not have GNU Parallel on your machine and you do not have rights to install software as root, you can make a personal installation:

wget https://ftpmirror.gnu.org/parallel/parallel-20180422.tar.bz2
bzip2 -dc parallel-20180422.tar.bz2 | tar xvf -
cd parallel-20180422
./configure --prefix=$HOME && make && make install

Or you can use --embed on a machine that has GNU Parallel and then copy the resulting script to you other machine:

parallel --embed > new_script
# Change the end of new_script
# Copy new_script to the other machine

find, xargs, and parallel: fun toys - Yakking, that it should run gzip on any arguments following the -- , or in other words, the -- separates the command to be run from the filenames to give the command as arguments. If you forget to do so, your credit will be deducted as long as the instance is running and you may get extra charges. TIP To easily stop all your running instances, you can combine this command with --quiet or -q option and xargs .

If you're looking for regular files with find, you should use -type f, or you may find directories. There is no need to use xargs here as you already have that ability built into find.

With GNU mv:

find /foo -type f -name 'bar*' -exec mv -t /dest {} +

This would not detect that a destination file with the same name already existed. To do that, use

find /foo -type f -name 'bar*' -exec sh -c '
    for name do
        destname="/dest/${name##*/}"
        if [ ! -f "$destname" ]; then
            mv "$name" "$destname"
        else
            printf '%s exists already, will not copy %s\n' "$destname" "$name" >&2
        fi
    done' sh {} +

This would refuse to move files that would overwrite files in /dest (and report these).

These commands will work with all valid Unix filenames.

find . -type f -print0, uses). This allows file names that contain newlines or other types of white space to be correctly interpreted by programs that process the find output. You can do this without xargs using find‘s -exec command, but the xargs method will be faster, especially with a large number of files, because it will run as a single invocation of cp. Find out more. This post only scratches the surface of what find can do. find supports testing based on permissions, ownership, access time, and much more. It

Paste command in Linux with examples, operating system. It is used to join files horizontally (parallel merging) by outputting lines consisting of lines from each file specified, separated by tab as delimiter, to the standard output. Merging multiple files as columns There are different cases when we need to concatenate files by their columns. We may need each file's content to appear in separate columns. Usually, … - Selection from Linux Shell Scripting Cookbook - Second Edition [Book]

Xargs and pipes, combining multiple commands, Xargs converts a list of arguments into small segments and passes them to other commands. A simple example: echo '--help' | cat. What does the xargs command do? Can you combine it with find and grep to find yet another way of searching all files in the /home subdirectory tree for the word hello? What does the cut command do? Can you use it together with w to produce a list of login names and CPU times corresponding to each active process?

Finding Files - Overview, The name xargs , pronounced EX-args, means "combine arguments." xargs builds and executes command lines by gathering together arguments it reads on the  To obtain the absolute directory of a file, we combine the readlink and dirname commands. We can do this in one of two ways. 3.1. xargs. First,

Comments
  • From you last sentence it seems you are aware that this can easily be solved with GNU Parallel. Can you elaborate on why you do not have GNU Parallel on your machines? Are any of the reasons covered by oletange.wordpress.com/2018/03/28/…
  • @OleTange I'm working on a script that runs on a large cluster of Jenkins worker machines. These are spun up on demand (so extra tools installed don't persist), and I don't have root on them.
  • @WilfredHuges But what stops you from installing without root access using one of the methods described on oletange.wordpress.com/2018/03/28/…
  • Bingo! That works as expected, thanks. Note that -t isn't available on BSDs (e.g. OS X) but for Linux this is perfect.
  • This would break if any directory matched the pattern, also the first and last would break if any name contained newlines or spaces.