pogmommy/rssguard
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

scraper for searching of feeds in html sites

Browse source
This commit is contained in:
Martin Rotter 2021-03-16 09:02:21 +01:00
parent 24df33dd89
commit a96e83fb10
1 changed files with 6 additions and 3 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

9
resources/scripts/scrapers/search-xml-feeds.py
View file

@ -16,9 +16,12 @@ regexp_href = re.compile("href=\"([^\"]+)\"")
for url in urls_lines: for url in urls_lines:
# Download HTML data. # Download HTML data.
url_response = urllib.request.urlopen(url) try:
html = url_response.read().decode("utf-8") url_response = urllib.request.urlopen(url)
html = url_response.read().decode("utf-8")
except:
continue
# Search for XML feeds with regexps. # Search for XML feeds with regexps.
for link_tag in re.findall(regexp_link, html): for link_tag in re.findall(regexp_link, html):
for link_xml_feed in re.findall(regexp_href, link_tag): for link_xml_feed in re.findall(regexp_href, link_tag):