How do I join pairs of consecutive lines in a large file (1 million lines) using vim, sed, or another similar tool?

sed join every 2 lines
bash join every 3 lines
awk join lines with comma
join every other line
python join every 2 lines
how to merge two lines in a file in unix
powershell join every 2 lines
linux join lines in file

I need to move the contents of every second line up to the line above such that line2's data is alongside line1's, either comma or space separated works.




line1 line2
line3 line4

I've been doing it in vim with a simple recording but vim seems to crash when I tell it to do it 100 000 times... I'm thinking maybe sed would be a good alternative but not sure how to do what I want or maybe there's a better option?

Each line only contains 1 numerical value, I just have a million lines...

If I understand correctly, you have:


and you want:


then you can do it easily with (g)awk like this:

awk 'NR % 2 == 1 { o=$0 ; next } { print o "<sep>" $0 }' INPUTFILE

See it in action here.

Update: if the number of lines is odd, the above will omit the last line (as Martin Stettner pointed out) so this will not:

awk 'NR % 2 == 1 { o=$0 ; next } { print o "<sep>" $0 } END { if ( NR % 2 == 1 ) { print o } }' INPUTFILE


15 Useful 'sed' Command Tips and Tricks for Daily Linux System , Every system administrator has to deal with plain text files on a daily basis. Tools such as head and tail allow us to view the bottom or the top of a file. It's possible that you're interested in set of non-consecutive lines, or in Another use of sed consists in printing the lines from a file that match a given� Hi, Can anyone help me for merge the following multi-line log which beginning with a number and time: into one line. For each line need to delete the return and add a space.

The paste command can do this. Its "-s" option will join consecutive lines; and the "-d" option specifies a list of characters to use as delimiters, repeating them cyclically. Join first with a space, then with a newline, and repeat:

seq 10 | paste -sd" \n" -

Vim join lines, Try other Vim commands to apply them to the highlighted text. 2) delete this line with dd and put it after 1) 1) first line (position cursor anywhere in this Vim is an editor to create or edit a text file. i - start insert mode at cursor; I - insert at the Using the :substitute Command to Join Lines. u/gfiorav. md Vim will open in that� The paste command can do this. Its "-s" option will join consecutive lines; and the "-d" option specifies a list of characters to use as delimiters, repeating them cyclically. Join first with a space, then with a newline, and repeat: seq 10 | paste -sd" " -

try this:

sed -rn 'N;s/\n/ /;p' yourFile

test with seq:

kent$  seq 10

kent$  seq 10|sed -rn 'N;s/\n/ /;p'
1 2
3 4
5 6
7 8
9 10

awk works too:

awk 'NR%2{printf $0" ";next;}1' yourFile


kent$  seq 10|awk 'NR%2{printf $0" ";next;}1'
1 2
3 4
5 6
7 8
9 10

Bash join lines, BASH – awk & paste – joining lines – every other – join 3 lines repeat – every 4 lines (8 Replies) For each pair of input lines with identical join fields, write a line to The file lines: line 1 line 2 line three line 4 Script, that replaces the line with the Using a common command line tool like sed or awk, is it possible to join all� At the end of the article, we also show the performance of the different ways of counting the total number of lines in a large file that contains 5 million lines and 1053 characters per line. 1. Files.lines (Java 8) This Files.lines is the most straightforward implementation.

Well your example is this in Vim.


But then what about the last line?

Or did you mean this?


You might also be interested in this Vim script, which makes dealing with large files easier.

Advanced Bash-Scripting Guide, Examples of sed operators; E-1. List of Examples; 2-1. cleanup: A script to clean up log files in /var/log Another command-line arg test using an and list; 26-3. Virtually the entire repertoire of UNIX commands, utilities, and tools is However, the gethostname() function sets the Bash internal variable� Write each FILE to standard output, with line numbers added. With no FILE, or when FILE is -, read standard input. 1. A basic example. Here is a basic example that explains how nl command can be used to number lines of a file.

This might work for you:

sed 'N;s/\n/ /' file


cat file | paste -d' ' - - 

Or another couple of ways for the above:

paste -d\  - - <file

paste -sd' \n' file

Add consecutive numbers succintly and elegantly to index lines , Just for the completeness of this list, sed can also do it: In Vim you could add line numbers to the actual text of a buffer like this: You could make it a bit fancier by automatically calculating and using a of the “hard coded” 5, as above) for the current number of lines in the file: 1:Content 2:. Some other solutions:. result file should be: >test100-1 AACTGCTCGTTACTGTG GCCTCACCCTAG >test100-2 ACCGTTATAAGACA GTTCACCCAT >test100-3 AAGGC. PROBLEM IN MY CODE: In the file where i have 2 lines I am not able to join them so that i get the results correct. here is my code:

Vim Regular Expressions 101, Estimation is that there are about half a million people using Vim as their main editor. I started this tutorial for one simple reason - I like regular expressions. For each line in the range replace a match of the pattern with the string where: Tip 1: frequently you need to do S&R in a text which contains UNIX file paths - text� Counting the lines of a text file may not seem very exciting but recently I had to deal with a number of large files (for an upcoming article which you will soon read about on this blog) and one of the operations that I had to do was to count the number of lines but at the same time ensuring that I am nice to the .NET Garbage Collector and don't cause any blocking collection.

Replace Comma With Line Break Online, Remove or replace line breaks With this online tool, you can remove all line breaks from a text. replaced and deleted by using regular expression with `sed command. a large text file, but I want to replace the found text with a carriage return (as if I would like to replace the newline (line break) characters with commas,� But you asked for finding any non consecutive duplicate lines in large file and not just a well known small set of lines. I posted several macro and script solutions for a task like searching in file A for lines or strings listed in file B, the last one just six days ago, see Need help comparing two files and outputting to a new file the

Bash comment multiple lines, bash comment multiple lines org # top-most EditorConfig file root = true line arguments so I can use this bash script on any other files I want. Simply write some commands, each on a new line in the large text field below. I wrote a simple Linux shell script to run the sed command on every “ node*.

  • Please format your question, and show us proper input and expected output.
  • Wouldn't this script omit the last line in case you have an odd number of lines?
  • best answer The '-' at the end is not necessary because paste reads from input as default.
  • The trailing '-' is required by some versions of paste (such as MacOS)
  • Hi,wow, i knew there had to be a better way than I was trying!! :) Thanks for the loads of options, :g/^/j was the most simple and does the job great!
  • Edited the OP following the above clarification above. The first suggestion in my answer can now be ignored.
  • Great Vim tip! In my case, wasn't the whole file, so can use a Visual Selection :'<,'>g/^/j or between the a mark and the current line :'a,.g/^/j to limit the joining to my range of interest. Beats doing it 773 times manually!
  • How about perl? seq 10 | perl -pe '$a++%2||s/\n/ /'