How to match a block of lines?

[genivia-inc]

Question

I've been wondering whether it could be feasible to implement a block-level or paragraph match mode for ugrep.
There are a few implementations in the wild doing just that, but they usually are dedicated tools.

By block-level I mean a tool which would match on a block of contiguous lines defined by regex patterns matching one a start line and another an end line of the block.
It would be interesting in order to allow, for instance, to match and return only matching blocks or only non-matching blocks against a given pattern.

See #369 by @trantor

Anwer

This works out-of-the-box with ugrep with lazy quantifiers and \n newline matching. The block of text between two matching lines is matched by the lazy pattern (.*\n)*?, which is followed by the ending line pattern to end the lazily-matching block.

Given a BEGIN and an END pattern to match the begin and end line of the block to match:

$ ug 'BEGIN.*\n(.*\n)*?.*END' FILE

Place the lazy pattern .*\n(.*\n)*?.* between BEGIN and END.

To match a BEGIN and an END pattern also when on the same line as well as a block:

$ ug 'BEGIN(.*\n)*?.*END' FILE

For example, to block-match C++ /*-*/ block comments look for regex /\* then match text with lazy regex (.*\n)*? and end with the regex .*\*+\/:

$ ug '/\*(.*\n)*?.*\*+\/' file.cpp

To capture the text between two matching lines use option -P and --format to output the group capture. For example:

$ ug -P '/\*((.*\n)*?.*)\*+\/' --format="%1%~" file.cpp

where %1 is the captured text of the group ((.*\n)*?.*) and %~ outputs a newline. See ug --help format for more info.

To invert the match and output only the blocks that don't match a specific pattern but do match the BEGIN and END criteria?

Note that option -v inverts matches. But that won't include the begin and end line, which is what you want.

To include begin and end lines, it will be necessary to use three patterns:

1. from the start of the file anchor \A to a line that matches BEGIN: \A(.*\n)*?.*BEGIN
2. from a line that matches END to a line that matches BEGIN, i.e. inverted order: END.*\n(.*\n)*?.*BEGIN
3. from a line that matches END to the end of the file anchor \Z: END(.*\n)*\Z

$ ug -e '\A(.*\n)*?.*BEGIN' -e 'END.*\n(.*\n)*?.*BEGIN' -e 'END(.*\n)*\Z' FILE

where the third pattern doesn't need to use a lazy quantifier since it matches greedily to the end of the file. Because of this lazy/non-lazy "conflict" at a zero-width anchor \Z, it's best to combine the second and third patterns like so:

$ ug --e '\A(.*\n)*?.*BEGIN' -e 'END.*\n(.*\n)*?(.*BEGIN|\Z)' FILE

All of this assumes that BEGIN ... END isn't nested. Nested blocks cannot be handled with regex, because it requires a context-free grammar, not a linear grammar. To do so requires a parser.

[genivia-inc]

Note: see answer in the question post.