tayaour.blogg.se

Has the same behaviour as restrict_xpaths. Restrict_css ( str or list) – a CSS selector (or list of selectors) which defines

If given, only the text selected by those XPath will be scanned for Regions inside the response where links should be extracted from. Restrict_xpaths ( str or list) – is an XPath (or list of XPath’s) which defines _EXTENSIONS.Ĭhanged in version 2.0: IGNORED_EXTENSIONS now includes Given (or empty) it won’t exclude any links.Īllow_domains ( str or list) – a single value or a list of string containingĭomains which will be considered for extracting the linksĭeny_domains ( str or list) – a single value or a list of strings containingĭomains which won’t be considered for extracting the linksĪ single value or list of strings containingĮxtensions that should be ignored when extracting links. It has precedence over the allow parameter. That the (absolute) urls must match in order to be excluded (i.e. Given (or empty), it will match all links.ĭeny ( str or list) – a single regular expression (or list of regular expressions) That the (absolute) urls must match in order to be extracted. ParametersĪllow ( str or list) – a single regular expression (or list of regular expressions) It is implemented using lxml’s robust HTMLParser.

LxmlLinkExtractor is the recommended link extractor with handy filtering LxmlLinkExtractor ( allow = (), deny = (), allow_domains = (), deny_domains = (), deny_extensions = None, restrict_xpaths = (), restrict_css = (), tags = ('a', 'area'), attrs = ('href',), canonicalize = False, unique = True, process_value = None, strip = True ) ¶