sed: print all lines THROUGH the LAST match of pattern A, then print ONLY lines that match pattern B

sed print lines between two patterns exclusive
sed print lines between two line numbers
man sed
awk print between two patterns same line
sed extract lines matching pattern
sed tutorial
sed regex
sed command

Esteemed colleagues...

I generate log files of several thousand lines of the form:

a
b
X
d
X
e
b
g
Y
a
Y
d

For example, I would like the output of my script to print all lines THROUGH the last match of "^X" and then print only matches of "^Y". Desired output for above:

a
b
X
d
X
Y
Y

"X" will always appear, but "Y" may not. In cases where "Y" does not appear, I believe that the last "X" will be on the last line of the file.

It was quite difficult to do this with "sed", although that's what I've been trying. I use "tac" to flip the line order so I can delete anything not matching "^Y" through the FIRST match of "^X". Because I don't use "-n", after that first match of "^X", all lines are echoed. I just use "tac" again to flip it around and put it in a file.

This seems to work...

tac /path/to/logfile | \
sed -e '1,/^X/ { /^Y/!d }' | \
tac > /output/path/logfile.processed

No...?

PS: Is "tac" commonly available on all Linux?

without tac, a double-pass approach with awk

$ awk 'NR==FNR{if(/^X$/) lx=NR; next} FNR<=lx || /^Y$/' file{,}

a
b
X
d
X
Y
Y

mark the last index of X and print all before that index and other matching pattern.

Print lines of a file between two matching patterns, You can make sed quit at a pattern with sed '/pattern/q' , so you just need your matches and then quit at the second pattern match: sed -n '/^pattern1/,/^pattern2/​p  B. sed - Print Lines with Regular Expression/Pattern 1. Print lines containing a Pattern This will print the line that contains the pattern provided. Syntax: sed -n '/PATTERN/p' FILE.txt Example: $ sed -n '/4/p' sedtest.txt This is line #4 2. Print lines excluding a Pattern This will print all those lines which do not contain the pattern

To avoid reading twice, you could use a perl:

$ perl -0777 -lnE 'say $1 while (/(\A[\s\S]*^X$|^Y$)/gm)' file
a
b
X
d
X
Y
Y

Or, with sed and common utilities:

$ sed_cmd=$(printf "1,%sp; /^Y/p" $(cat -n file | sed -nE 's/^[[:space:]]*([[:digit:]][[:digit:]]*)[[:space:]]*X/\1/p' | tail -n 1))
$ sed -nE "$sed_cmd" file
# same output

sed, $ sed -n '/--START--/{:a;N;/--END--/!ba; /Device=A/p}' file --START-- Device=A Data=asdfasdf Lorem=Ipsum --END-- --START-- Device=A  grep -v will exclude matching lines, but I want something that will print all lines but exclude a matching field. The pattern that I want excluded is '/mnt/svn' If there is a better solution than awk I am happy to hear about it, but I would like to see this done in awk as well.

Here's a somewhat more logically explicit version using Perl.

perl -MList::Util=max -lnE '
    $lines{$.} = $_; 
    eof || next; 
    $last_match = max grep {$lines{$_} =~ /^X/} keys %lines;
    say for @lines{1 .. $last_match};
    say for grep {$_ =~ /^Y/} @lines{$last_match .. $.};
' /path/to/logfile

How to print lines between two patterns, inclusive or exclusive (in , occurring from the time PAT1 occurs and up to the next PAT2 is seen. Awk print all lines on match? Ok so I can use awk to match a pattern and print the whole line with print $0. Is there any way to just tell awk to print every line of output when the pattern matches?

grep command in Unix/Linux, Which of the below command select lines in files that match patterns? Full Discussion: sed print all lines after pattern match Top Forums Shell Programming and Scripting sed print all lines after pattern match Post 302383557 by steadyonabix on Wednesday 30th of December 2009 07:05:31 AM

! Aware to Perl: How can I pull out lines between two patterns that , is referred to as the regular expression (grep stands for globally search for regular expression and print out). Full Discussion: sed print all lines after pattern match Top Forums Shell Programming and Scripting sed print all lines after pattern match Post 302383548 by steadyonabix on Wednesday 30th of December 2009 06:39:55 AM

Different ways to print the next few lines after , How do I print a line between two patterns in Perl? grep -v will exclude matching lines, but I want something that will print all lines but exclude a matching field. The pattern that I want excluded is '/mnt/svn' If there is a better solution than awk I am happy to hear about it, but I would like to see this done in awk as well.

Comments
  • Is it possible that there is a Y between two X lines?
  • What should be output if there is no X in the input?
  • @kvantour: No. There should never be an "X" there, but I believe that rule is secondary to the need to find the LAST "X".
  • @EdMorton: There will always be a "X" in the input. Thanks!
  • If a platform has "sed", but not "awk", I'd be surprised. Both "awk" and "tac" are in "/usr/bin" on some platforms, but "sed" is in "/bin". I'm not sure if it's riskier to depend on "tac" or "awk"... Thanks for your help. I like your short answer...
  • If a platform is POSIX, it will have awk
  • @LanceE.T.Compte it's riskier to depend on tac than awk. Over the past 35 years or so Ive used many machines that didn't have tac but none that didn't have some flavor of awk.
  • How does your "sed" improve on mine, other than to get rid of "tac"? It seems you're finding the line number of the last match...
  • Also can use '.' instead of \s\S by adding the s modifier: perl -0777 -nE 'say $1 while (/(^.*^X$|^Y$)/gms)' file
  • How does your "sed" improve on mine, other than to get rid of "tac"? That is the improvement. tac is not that common on non Linux platforms, but the sed I have there is POSIX compliant. Indeed, the pipeline with cat is to determine the line number of the last X
  • I do find this kind of brute force approach is quite readable/maintainable. I'm already worried about portability since I need to run on many different flavors of Linux. Isn't list::util in coremodules now?
  • Yes, List::Util is a core module.