python-从difflib获取更细粒度的diff(或对diff进行后处理以实现相同效果的方法)

下载this页并对其进行较小的编辑,将本段中的前65更改为68：

然后,我使用BeauifulSoup解析这两个源,并使用difflib对其进行比较.

url = 'https://secure.ssa.gov/apps10/reference.nsf/links/02092016062645AM'
response = urllib2.urlopen(url)
content = response.read()  # get response as list of lines

url2 = 'file:///Users/Pyderman/projects/temp/02092016062645AM-modified.html'
response2 = urllib2.urlopen(url2)
content2 = response2.read()  # get response as list of lines
import difflib
d = difflib.Differ()

diffed = d.compare(content, content)

soup = bs4.BeautifulSoup(content, "lxml")
soup2= bs4.BeautifulSoup(content2, "lxml")
diff = d.compare(list(soup.stripped_strings), list(soup2.stripped_strings))
changes = [change for change in diff if change.startswith('-') or  change.startswith('+')]
for change in changes:
    print change

打印更改将给出：

- The Achieving a Better Life Experience (ABLE) Act, H.R. 5771, legislation passed on December 19, 2014. It contains a Title II provision that changes the age at which workers compensation/public disability offset ends for disability beneficiaries from age 65 to full retirement age (FRA).  This provision will apply to any individual who attains age 65 on or after December 19, 2015 (the one year anniversary of enactment of this bill).  Two new Universal Text Identifiers (UTIs), UTI WCP060 and WCP061 were created to comply with this change.
+ The Achieving a Better Life Experience (ABLE) Act, H.R. 5771, legislation passed on December 19, 2014. It contains a Title II provision that changes the age at which workers compensation/public disability offset ends for disability beneficiaries from age 68 to full retirement age (FRA).  This provision will apply to any individual who attains age 65 on or after December 19, 2015 (the one year anniversary of enactment of this bill).  Two new Universal Text Identifiers (UTIs), UTI WCP060 and WCP061 were created to comply with this change.

因此,尽管有很小的变化,但它还是打印了整个段落.我想这是一件好事,它显示的是整个段落的差异而不是句子的差异,但是我们可以以某种方式使输出更细粒度吗？就目前而言,似乎我只想突出显示已更改的文本,就必须对这两个几乎完全相同的字符串进行一些额外的增量比较.

解决方法:

您可以使用nltk.sent_tokenize()将汤串分割成句子：

from nltk import sent_tokenize

sentences = [sentence for string in soup.stripped_strings for sentence in sent_tokenize(string)]
sentences2 = [sentence for string in soup2.stripped_strings for sentence in sent_tokenize(string)]

diff = d.compare(sentences, sentences2)
changes = [change for change in diff if change.startswith('-') or  change.startswith('+')]
for change in changes:
    print(change)

仅在检测到更改的地方打印适当的句子：

- It contains a Title II provision that changes the age at which workers compensation/public disability offset ends for disability beneficiaries from age 65 to full retirement age (FRA).
+ It contains a Title II provision that changes the age at which workers compensation/public disability offset ends for disability beneficiaries from age 68 to full retirement age (FRA).

标签：difflib,python,beautifulsoup,lxml,html-parsing
来源： https://codeday.me/bug/20191009/1878290.html