Feature/matcher alignment

[broaddeep]

Description

Support for match alignments.

Many users wanted rule-base matcher to support subgroup labeling(#3275) or Group capture(#4642), Look-around operator like regex(#6420).
However, it wasn't an easy task as seen in the issue(#3275).

To address this issue, I propose the concept of match alignments.
It represents the part of a token pattern that contributed to the match.

For example, suppose we have pattern
[{"ORTH": "a", "OP": "+"}, {"ORTH": "b"}]
and the text is given as
a a a b

The matched span will have four tokens(in the longest greedy setup).
We can easily verify that the first three matched tokens(a a a) was matched by the first token pattern ({"ORTH": "a", "OP": "+"}),
and the last token(b) was matched by the second token pattern ({"ORTH": "b"})

We can rewrite this in List[int], [0, 0, 0, 1].

Using this information, it can be applied to have the same effect as group capture or look around operator, subgroup labeling.

Implementation details

• Each time the state changes, it keeps track of the index of the token pattern at that time and the length of the span.
• See be3a664

API

import spacy
from spacy.matcher import Matcher

nlp = spacy.load('en_core_web_sm')

pattern = [
    {'ENT_TYPE': 'PERSON', 'OP': '+'}, 
    {'LEMMA': 'love'}, 
    {'ENT_TYPE': 'PERSON', 'OP': '+'}
]

matcher = Matcher(nlp.vocab)
matcher.add("test", [pattern], greedy='LONGEST')

doc = nlp("John Doe loves Jane Doe. John loves Jane.")

matches = matcher(doc, match_alignments=True)

for m in matches:
    print(m)
# (1618900948208871284, 0, 5, [0, 0, 1, 2, 2])
# (1618900948208871284, 6, 9, [0, 1, 2])

• It does not require breaking changes. (All test passed)
• Test case added.

Types of change

New feature,

Checklist

• I have submitted the spaCy Contributor Agreement.
• I ran the tests, and all new and existing tests passed.
• My changes don't require a change to the documentation, or if they do, I've added all required information.