mjhjmtu

I have a text file like so:

<!--START OF FILE -->
random text
<meta> more random text </meta>
x x x x x x x 
more random text
that I dont need 
x x x x x x x

I need everything
from this point
onwards
...

I need to remove everything between <!--START OF FILE --> and the second
x x x x x x x like so:

I need everything
from this point
onwards
...

I tried using sed '/<!--START OF FILE -->/,/x x x x x x x/d' test.txt but this removes the block between the first occurence of x x x x x x x which is not what I want.

This is quite the opposite of

How to print lines between pattern1 and 2nd match of pattern2?

With sed you'd do something like:

sed -n '/PATTERN1/,$! # if not in this range
p;d # print and delete

/PATTERN2/!d # delete if it doesn't match PATTERN2
x;//!d # exchange and then, again, delete if no match
: do # label "do" (executed only after the 2nd match)
n;p # get the next line and print
b do' infile # go to label "do"

or, in one line (on gnu setups):

sed -n '/PATTERN1/,$!p;d;;/PATTERN2/!d;x;//!d;: do;n;p;b do' infile

Sure, it's easier with awk and counters. I'll leave that as an exercise for you...

Straightforward awk:

$ awk '/<!--START OF FILE -->/ a=2; !a; /x x x x x x x/ && a a--' < data

I need everything
from this point
...

It just prints whenever a is zero and decrements it when it sees the x x x ....

Or starting from the actual start of the file instead of a pattern, change the first block to BEGIN a=2.

Note that your sample input has an empty line after the second x x x..., and it remains in the output if we stop removing lines at the x x x... line.

grep -Pz '(?s)<!--START OF FILE(.*?x x x x x x x)2K.*' input.txt

Explanation

1. 
   

   grep -Pz

   
   
   
   • 
     -P - Interpret the pattern as a Perl-compatible regular expression (PCRE).
   
   
   • 
     -z - process the input.txt as one big line.
   
   


2. 
   

   (?s)<!--START OF FILE(.*?x x x x x x x)2K.*

   
   
   
   • 
     (?s) - Turn on "dot matches newline" for the remainder of the regular expression.
   
   
   • 
     .*? - non-greedy matching.
   
   
   • 
     2 - amount of repetitions of the pattern.
   
   
   • 
     K - any previously-matched characters to be omitted from the final matched string.
   
   

This snippet:

# Utility functions: print-as-echo, print-line-with-visual-space.
pe() for _i;do printf "%s" "$_i";done; printf "n"; 
pl() pe;pe "-----" ;pe "$*"; 
pl " Input data file $FILE:"
head -v -n 20 $FILE

pl " Expected output on file $E:"
head -v $E

pl " Results:"
cgrep -V -D -w '<!--START OF FILE -->' +2 +w 'x x x x x x x' 'meta' $FILE

produces:

-----
 Input data file data1:
==> data1 <==
<!--START OF FILE -->
random text
<meta> more random text </meta>
x x x x x x x 
more random text
that I dont need 
x x x x x x x

I need everything
from this point

-----
 Expected output on file expected-output1:

I need everything
from this point
onwards
...

-----
 Results:

I need everything
from this point
onwards
...

This omits (-V) a window beginning (-w) with '...START...', and ending (+w) with the second occurrence (+2) of a string '...x x...' that has the string 'meta' inside the window.

On a system like:

OS, ker|rel, machine: Linux, 3.16.0-4-amd64, x86_64
Distribution : Debian 8.9 (jessie) 
bash GNU bash 4.3.30

Some details for cgrep:

cgrep shows context of matching patterns found in files (man)
Path : ~/executable/cgrep
Version : 8.15
Type : ELF 64-bit LSB executable, x86-64, version 1 (SYS ...)
Home : http://sourceforge.net/projects/cgrep/ (doc)

Although one would need to get and compile cgrep, I have had no trouble doing that on 32-bit or 64-bit systems, and it is available on macOS (High Sierra) with brew. The execution time is on a par with GNU grep.

Best wishes ... cheers, drl