EVOLUTION-NINJA
Edit File: element.cpython-36.pyc
3 &6]$ � @ s d Z yddlmZ W n. ek rB Z zddlmZ W Y ddZ[X nX ddlZddlZddlZddl Z ddl m Z dZ ej d dkZejd�Zdd � ZG d d � d e�ZG d d � d e�ZG dd� de�ZG dd� de�ZG dd� de �ZG dd� de�ZG dd� de�ZG dd� de�ZG dd� de�Z G d d � d e�Z G d d � d e�Z G d d!� d!e�Z G d"d#� d#ee �Z G d$d%� d%e �Z!G d&d'� d'e!�Z"G d(d)� d)e!�Z#G d*d+� d+e#�Z$G d,d-� d-e!�Z%G d.d/� d/e!�Z&G d0d1� d1e!�Z'G d2d3� d3e �Z(G d4d5� d5e�Z)G d6d7� d7e*�Z+dS )8ZMIT� )�CallableN)�EntitySubstitutionzutf-8� z\s+c s& t � fdd��}|j� fdd��}|S )z>Alias one attribute name to another for backward compatibilityc s t | � �S )N)�getattr)�self)�attr� � /usr/lib/python3.6/element.py�alias s z_alias.<locals>.aliasc s t | � �S )N)�setattr)r )r r r r s )�property�setter)r r r )r r �_alias s r c @ s e Zd Zddd�ZdS )�NamespacedAttributeNc C sV |d krt j| |�}n*|d kr,t j| |�}nt j| |d | �}||_||_||_|S )N�:)�str�__new__�prefix�name� namespace)�clsr r r �objr r r r " s zNamespacedAttribute.__new__)N)�__name__� __module__� __qualname__r r r r r r s r c @ s e Zd ZdZdS )�%AttributeValueWithCharsetSubstitutionz=A stand-in object for a character encoding specified in HTML.N)r r r �__doc__r r r r r / s r c @ s e Zd ZdZdd� Zdd� ZdS )�CharsetMetaAttributeValuez�A generic stand-in for the value of a meta tag's 'charset' attribute. When Beautiful Soup parses the markup '<meta charset="utf8">', the value of the 'charset' attribute will be one of these objects. c C s t j| |�}||_|S )N)r r �original_value)r r r r r r r 9 s z!CharsetMetaAttributeValue.__new__c C s |S )Nr )r �encodingr r r �encode> s z CharsetMetaAttributeValue.encodeN)r r r r r r r r r r r 2 s r c @ s. e Zd ZdZejdej�Zdd� Zdd� Z dS )�ContentMetaAttributeValuea A generic stand-in for the value of a meta tag's 'content' attribute. When Beautiful Soup parses the markup: <meta http-equiv="content-type" content="text/html; charset=utf8"> The value of the 'content' attribute will be one of these objects. z((^|;)\s*charset=)([^;]*)c C s6 | j j|�}|d kr tjt|�S tj| |�}||_|S )N)� CHARSET_RE�searchr r r )r r �matchr r r r r M s z!ContentMetaAttributeValue.__new__c s � fdd�}| j j|| j�S )Nc s | j d�� S )N� )�group)r$ )r r r �rewriteX s z1ContentMetaAttributeValue.encode.<locals>.rewrite)r" �subr )r r r' r )r r r W s z ContentMetaAttributeValue.encodeN) r r r r �re�compile�Mr" r r r r r r r! B s r! c @ sV e Zd ZdZeddg�Zedg�Zeddg�Zedd� �Z edd � �Z ed d � �Z d S ) �HTMLAwareEntitySubstitutiona% Entity substitution rules that are aware of some HTML quirks. Specifically, the contents of <script> and <style> tags should not undergo entity substitution. Incoming NavigableString objects are checked to see if they're the direct children of a <script> or <style> tag. ZscriptZstyle�preZtextareac C s. t |t�r&|jd k r&|jj| jkr&|S ||�S )N)� isinstance�NavigableString�parentr �cdata_containing_tags)r �ns�fr r r �_substitute_if_appropriatem s z6HTMLAwareEntitySubstitution._substitute_if_appropriatec C s | j |tj�S )N)r4 r �substitute_html)r r2 r r r r5 w s z+HTMLAwareEntitySubstitution.substitute_htmlc C s | j |tj�S )N)r4 r �substitute_xml)r r2 r r r r6 | s z*HTMLAwareEntitySubstitution.substitute_xmlN) r r r r �setr1 Zpreformatted_tags�preserve_whitespace_tags� classmethodr4 r5 r6 r r r r r, \ s r, c @ s e Zd ZdZdZdd� ZdS )� Formatterz6Contains information about how to format a parse tree.�/c O s t � �dS )z1Transform certain characters into named entities.N)�NotImplementedError)r �args�kwargsr r r �substitute_entities� s z Formatter.substitute_entitiesN)r r r r �void_element_close_prefixr? r r r r r: � s r: c @ s e Zd ZdZdd� ZdS )� HTMLFormatterzThe default HTML formatter.c O s t j||�S )N)r, r5 )r r= r> r r r � substitute� s zHTMLFormatter.substituteN)r r r r rB r r r r rA � s rA c @ s e Zd ZdZdd� ZdS )�MinimalHTMLFormatterzA minimal HTML formatter.c O s t j||�S )N)r, r6 )r r= r> r r r rB � s z MinimalHTMLFormatter.substituteN)r r r r rB r r r r rC � s rC c @ s e Zd ZdZdZdS )�HTML5Formatterz5An HTML formatter that omits the slash in a void tag.N)r r r r r@ r r r r rD � s rD c @ s e Zd ZdZdd� ZdS )� XMLFormatterz+Substitute only the essential XML entities.c O s t j||�S )N)r r6 )r r= r> r r r rB � s zXMLFormatter.substituteN)r r r r rB r r r r rE � s rE c @ s e Zd ZdZdd� ZdS )�HTMLXMLFormatterz Format XML using HTML rules.c O s t j||�S )N)r, r5 )r r= r> r r r rB � s zHTMLXMLFormatter.substituteN)r r r r rB r r r r rF � s rF c @ s` e Zd ZdZe� e� e� dd�Ze� e � dd�Z d]dd�Z e dd � �Z d d � Zd^d d �Zed�Zed�Zdd� ZeZdd� ZeZeZdd� Zdd� Zd_dd�ZeZdd � Z d d � Z d d � Z d!d"� Z di dfd#d$�Z e Z!di ddfd%d&�Z"e"Z#di dfd'd(�Z$e$Z%di ddfd)d*�Z&e&Z'e&Z(di dfd+d,�Z)e)Z*di ddfd-d.�Z+e+Z,e+Z-di dfd/d0�Z.e.Z/di ddfd1d2�Z0e0Z1e0Z2di fd3d4�Z3e3Z4di dfd5d6�Z5e5Z6e5Z7e d7d8� �Z8e d9d:� �Z9d;d<� Z:d=d>� Z;e d?d@� �Z<e dAdB� �Z=e dCdD� �Z>e dEdF� �Z?e dGdH� �Z@eAjBdI�ZCeAjBd`�ZDdadLdM�ZEdNdO� ZFdbdQdR�ZGdSdT� ZHdUdV� ZIdWdX� ZJdYdZ� ZKd[d\� ZLdS )c� PageElementzeContains the navigational information for some part of the page (either a tag or a piece of text)N)�htmlZhtml5�minimalN)rH rI NrI c C sB t |t�r| j|�}|dkr"|}n t|�r4||�}n |j|�}|S )z2Format the given string using the given formatter.N)r. r �_formatter_for_name�callablerB )r �s� formatter�outputr r r � format_string� s zPageElement.format_stringc C s. | j dk r| j S | jdkr&t| dd�S | jjS )aJ Is this element part of an XML tree or an HTML tree? This is used when mapping a formatter name ("minimal") to an appropriate function (one that performs entity-substitution on the contents of <script> and <style> tags, or not). It can be inefficient, but it should be called very rarely. N�is_xmlF)� known_xmlr0 r �_is_xml)r r r r rR � s zPageElement._is_xmlc C s* | j r| jj|t� �S | jj|t� �S dS )z<Look up a formatter function based on its name and the tree.N)rR �XML_FORMATTERS�getrE �HTML_FORMATTERSrA )r r r r r rJ � s z PageElement._formatter_for_namec C s~ || _ || _|dk r | | j_|| _| jr0| | j_|| _| jrD| | j_| rh| j dk rh| j jrh| j jd }|| _|rz| | j_dS )zNSets up the initial relations between this element and other elements.Nr% ���)r0 �previous_element� next_element� next_sibling�previous_sibling�contents)r r0 rW rX rZ rY r r r �setup� s zPageElement.setuprY rZ c C sV | j std��|| krd S || j kr,td��| j }| j j| �}| j� |j||� | S )Nz]Cannot replace one element with another when theelement to be replaced is not part of a tree.z%Cannot replace a Tag with its parent.)r0 � ValueError�index�extract�insert)r � replace_withZ old_parent�my_indexr r r ra s zPageElement.replace_withc C sT | j }| j std��| j j| �}| j� x&t| jd d � �D ]}|j||� q<W | S )NzSCannot replace an element with its contents when thatelement is not part of a tree.)r0 r] r^ r_ �reversedr[ r` )r Z my_parentrb �childr r r �unwrap% s zPageElement.unwrapc C s | j |�}|j|� |S )N)ra �append)r Z wrap_inside�mer r r �wrap3 s zPageElement.wrapc C s� | j dk r | j j| j j| �= | j� }|j}| jdk rF| j|k rF|| j_|dk r`|| jk r`| j|_d| _d|_d| _ | jdk r�| j| jk r�| j| j_| jdk r�| j| jk r�| j| j_d | _| _| S )z0Destructively rips this element out of the tree.N)r0 r[ r^ �_last_descendantrX rW rZ rY )r � last_childrX r r r r_ 8 s( zPageElement.extractTc C sN |r| j r| j j}n$| }x t|t�r6|jr6|jd }qW | rJ|| krJd}|S )z8Finds the last element beneath this object to be parsed.r% NrV )rY rW r. �Tagr[ )r Zis_initializedZ accept_selfrj r r r ri U s z PageElement._last_descendantc C s� |d krt d��|| kr t d��t|t�r>t|t� r>t|�}ddlm} t||�r�x&t|j�D ]}| j||� |d7 }q`W d S t |t | j��}t |d�r�|j d k r�|j | kr�| j |�}||k r�|d8 }|j� | |_ d }|dkr�d |_| |_n(| j|d }||_||j_|jd�|_|jd k �r,||j_|jd�}|t | j�k�r�d |_| }d } x2| d k�r�|d k �r�|j} |j }| d k �rVP �qVW | d k �r�| |_nd |_n*| j| } | |_|jd k �r�||j_| |_|jd k �r�||j_| jj||� d S )Nz Cannot insert None into a tag.z Cannot insert a tag into itself.r )� BeautifulSoupr% r0 F)r] r. r r/ Zbs4rl �listr[ r` �min�len�hasattrr0 r^ r_ rZ rW rY ri rX ) r Zposition� new_childrl ZsubchildZ current_indexZprevious_childZnew_childs_last_elementr0 Zparents_next_siblingZ next_childr r r r` c sh zPageElement.insertc C s | j t| j�|� dS )z2Appends the given tag to the contents of this tag.N)r` ro r[ )r �tagr r r rf � s zPageElement.appendc C sR | |krt d��| j}|dkr&t d��t|t�r8|j� |j| �}|j||� dS )z�Makes the given element the immediate predecessor of this one. The two elements will have the same parent, and the given element will be immediately before this one. z&Can't insert an element before itself.Nz2Element has no parent, so 'before' has no meaning.)r] r0 r. rG r_ r^ r` )r Z predecessorr0 r^ r r r � insert_before� s zPageElement.insert_beforec C sV | |krt d��| j}|dkr&t d��t|t�r8|j� |j| �}|j|d |� dS )z�Makes the given element the immediate successor of this one. The two elements will have the same parent, and the given element will be immediately after this one. z%Can't insert an element after itself.Nz1Element has no parent, so 'after' has no meaning.r% )r] r0 r. rG r_ r^ r` )r Z successorr0 r^ r r r � insert_after� s zPageElement.insert_afterc K s | j | j|||f|�S )zjReturns the first item that matches the given criteria and appears after this Tag in the document.)� _find_one� find_all_next)r r �attrs�textr> r r r � find_next� s zPageElement.find_nextc K s | j ||||| jf|�S )zbReturns all items that match the given criteria and appear after this Tag in the document.)� _find_all� next_elements)r r rw rx �limitr> r r r rv � s zPageElement.find_all_nextc K s | j | j|||f|�S )z{Returns the closest sibling to this Tag that matches the given criteria and appears after this Tag in the document.)ru �find_next_siblings)r r rw rx r> r r r �find_next_sibling� s z PageElement.find_next_siblingc K s | j ||||| jf|�S )zqReturns the siblings of this Tag that match the given criteria and appear after this Tag in the document.)rz � next_siblings)r r rw rx r| r> r r r r} � s z PageElement.find_next_siblingsc K s | j | j|||f|�S )zkReturns the first item that matches the given criteria and appears before this Tag in the document.)ru �find_all_previous)r r rw rx r> r r r � find_previous� s zPageElement.find_previousc K s | j ||||| jf|�S )zcReturns all items that match the given criteria and appear before this Tag in the document.)rz �previous_elements)r r rw rx r| r> r r r r� � s z PageElement.find_all_previousc K s | j | j|||f|�S )z|Returns the closest sibling to this Tag that matches the given criteria and appears before this Tag in the document.)ru �find_previous_siblings)r r rw rx r> r r r �find_previous_sibling s z!PageElement.find_previous_siblingc K s | j ||||| jf|�S )zrReturns the siblings of this Tag that match the given criteria and appear before this Tag in the document.)rz �previous_siblings)r r rw rx r| r> r r r r� s z"PageElement.find_previous_siblingsc K s& d}| j ||df|�}|r"|d }|S )zOReturns the closest parent of this Tag that matches the given criteria.Nr% r )� find_parents)r r rw r> �r�lr r r � find_parent s zPageElement.find_parentc K s | j ||d|| jf|�S )zFReturns the parents of this Tag that match the given criteria.N)rz �parents)r r rw r| r> r r r r� ! s zPageElement.find_parentsc C s | j S )N)rX )r r r r �next* s zPageElement.nextc C s | j S )N)rW )r r r r �previous. s zPageElement.previousc K s& d }||||df|�}|r"|d }|S )Nr% r r )r �methodr rw rx r> r� r� r r r ru 4 s zPageElement._find_onec s8 |dkr d|kr |d }|d= t �t�r.�}nt�||f|�}|dkr�| r�| r�| r҈dksh�dkr�dd� |D �}t||�S t �t�r҈jd�dkr��jdd�\�� nd��� � ��fdd�|D �}t||�S t|�} xXy t|�} W n tk �r P Y nX | r�|j| �} | r�| j | � |r�t | �|kr�P q�W | S ) z8Iterates over a generator looking for things that match.N�stringTc s s | ]}t |t�r|V qd S )N)r. rk )�.0�elementr r r � <genexpr>J s z(PageElement._find_all.<locals>.<genexpr>r r% c 3 sB | ]:}t |t�r|j�ks6|j� kr�d ks6|j�kr|V qd S )N)r. rk r r )r� r� )� local_namer r r r r� W s ) r. � SoupStrainer� ResultSetr �count�splitr� � StopIterationr# rf ro ) r r rw rx r| � generatorr> Zstrainer�result�results�i�foundr )r� r r r rz ; s< zPageElement._find_allc c s$ | j }x|d k r |V |j }qW d S )N)rX )r r� r r r r{ q s zPageElement.next_elementsc c s$ | j }x|d k r |V |j }qW d S )N)rY )r r� r r r r x s zPageElement.next_siblingsc c s$ | j }x|d k r |V |j }qW d S )N)rW )r r� r r r r� s z PageElement.previous_elementsc c s$ | j }x|d k r |V |j }qW d S )N)rZ )r r� r r r r� � s z PageElement.previous_siblingsc c s$ | j }x|d k r |V |j }qW d S )N)r0 )r r� r r r r� � s zPageElement.parentsz ^[a-zA-Z0-9][-.a-zA-Z0-9:_]*$zX^(?P<tag>[a-zA-Z0-9][-.a-zA-Z0-9:_]*)?\[(?P<attribute>[\w-]+)(?P<operator>[=~\|\^\$\*]?)z=?"?(?P<value>[^\]"]*)"?\]$c C s. | j ||�}t|t�s t|t�r*dj|�}|S )z�Force an attribute value into a string representation. A multi-valued attribute will be converted into a space-separated stirng. � )rT r. rm �tuple�join)r �value�defaultr r r �_attr_value_as_string� s z!PageElement._attr_value_as_stringc s �s� S � �fdd�}|S d S )Nc s | j �ko� | �S )N)r )rr )�function�tag_namer r �_match� s z1PageElement._tag_name_matches_and.<locals>._matchr )r r� r� r� r )r� r� r �_tag_name_matches_and� s z!PageElement._tag_name_matches_and� c s� |dkr� �fdd�S |dkr0� �fdd�}|S |dkrF� �fdd�S |d kr\� �fd d�S |d krr� �fd d�S |d kr�� �fdd�}|S � fdd�S dS )z�Create a function that performs a CSS selector operation. Takes an operator, attribute and optional value. Returns a function that will return True for elements that match that combination. �=c s | j � ��kS )N)r� )�el)� attributer� r r �<lambda>� s z0PageElement._attribute_checker.<locals>.<lambda>�~c s& | j � g �}t|t�s |j� }�|kS )N)rT r. rm r� )r� �attribute_value)r� r� r r �_includes_value� s z7PageElement._attribute_checker.<locals>._includes_value�^c s | j � d�j��S )Nr� )r� � startswith)r� )r� r� r r r� � s �$c s | j � d�j��S )Nr� )r� �endswith)r� )r� r� r r r� � s �*c s �| j � d�kS )Nr� )r� )r� )r� r� r r r� � s �|c s" | j � d�}|�kp |j�d �S )Nr� �-)r� r� )r� r� )r� r� r r �_is_or_starts_with_dash� s z?PageElement._attribute_checker.<locals>._is_or_starts_with_dashc s | j � �S )N)�has_attr)r� )r� r r r� � s Nr )r �operatorr� r� r� r� r )r� r� r �_attribute_checker� s z PageElement._attribute_checkerc C s | j S )N)r{ )r r r r � nextGenerator� s zPageElement.nextGeneratorc C s | j S )N)r )r r r r �nextSiblingGenerator� s z PageElement.nextSiblingGeneratorc C s | j S )N)r� )r r r r �previousGenerator� s z PageElement.previousGeneratorc C s | j S )N)r� )r r r r �previousSiblingGenerator� s z$PageElement.previousSiblingGeneratorc C s | j S )N)r� )r r r r �parentGenerator� s zPageElement.parentGenerator)rI )NNNNN)TTzs^(?P<tag>[a-zA-Z0-9][-.a-zA-Z0-9:_]*)?\[(?P<attribute>[\w-]+)(?P<operator>[=~\|\^\$\*]?)=?"?(?P<value>[^\]"]*)"?\]$)N)r� )Mr r r r rA rD rC rU rF rE rS rO r rR rJ r\ r Z nextSiblingZpreviousSiblingra Z replaceWithre Zreplace_with_childrenZreplaceWithChildrenrh r_ ri Z_lastRecursiveChildr` rf rs rt ry ZfindNextrv Z findAllNextr~ ZfindNextSiblingr} ZfindNextSiblingsZfetchNextSiblingsr� Z findPreviousr� ZfindAllPreviousZ fetchPreviousr� ZfindPreviousSiblingr� ZfindPreviousSiblingsZfetchPreviousSiblingsr� Z findParentr� Z findParentsZ fetchParentsr� r� ru rz r{ r r� r� r� r) r* � tag_name_re�attribselect_rer� r� r� r� r� r� r� r� r r r r rG � s� J 6 +rG c @ s\ e Zd ZdZdZdZdd� Zdd� Zdd� Zd d � Z dd d �Z e dd� �Z e j dd� �Z dS )r/ r� Nc C s2 t |t�rtj| |�}ntj| |t�}|j� |S )a- Create a new NavigableString. When unpickling a NavigableString, this method is called with the string in DEFAULT_OUTPUT_ENCODING. That encoding needs to be passed in to the superclass's __new__ or the superclass won't know how to handle non-ASCII characters. )r. r r �DEFAULT_OUTPUT_ENCODINGr\ )r r� �ur r r r � s zNavigableString.__new__c C s t | �| �S )z�A copy of a NavigableString has the same contents and class as the original, but it is not connected to the parse tree. )�type)r r r r �__copy__ s zNavigableString.__copy__c C s t | �fS )N)r )r r r r �__getnewargs__ s z NavigableString.__getnewargs__c C s$ |dkr | S t d| jj|f ��dS )z�text.string gives you text. This is for backwards compatibility for Navigable*String, but for CData* it lets you get the string without the CData wrapper.r� z!'%s' object has no attribute '%s'N)�AttributeError� __class__r )r r r r r � __getattr__ s zNavigableString.__getattr__rI c C s | j | |�}| j| | j S )N)rO �PREFIX�SUFFIX)r rM rN r r r � output_ready s z NavigableString.output_readyc C s d S )Nr )r r r r r # s zNavigableString.namec C s t d��d S )Nz)A NavigableString cannot be given a name.)r� )r r r r r r ' s )rI )r r r r� r� rQ r r� r� r� r� r r r r r r r r/ � s r/ c @ s e Zd ZdZddd�ZdS )�PreformattedStringz�A NavigableString not subject to the normal formatting rules. The string will be passed into the formatter (to trigger side effects), but the return value will be ignored. rI c C s | j | |� | j| | j S )zUCData strings are passed into the formatter. But the return value is ignored.)rO r� r� )r rM r r r r� 2 s z PreformattedString.output_readyN)rI )r r r r r� r r r r r� + s r� c @ s e Zd ZdZdZdS )�CDataz <![CDATA[z]]>N)r r r r� r� r r r r r� 8 s r� c @ s e Zd ZdZdZdZdS )�ProcessingInstructionz A SGML processing instruction.z<?�>N)r r r r r� r� r r r r r� = s r� c @ s e Zd ZdZdZdZdS )�XMLProcessingInstructionz An XML processing instruction.z<?z?>N)r r r r r� r� r r r r r� C s r� c @ s e Zd ZdZdZdS )�Commentz<!--z-->N)r r r r� r� r r r r r� H s r� c @ s e Zd ZdZdZdS )� Declarationz<?z?>N)r r r r� r� r r r r r� N s r� c @ s e Zd Zedd� �ZdZdZdS )�Doctypec C sN |pd}|d k r2|d| 7 }|d k rF|d| 7 }n|d k rF|d| 7 }t |�S )Nr� z PUBLIC "%s"z "%s"z SYSTEM "%s")r� )r r Zpub_idZ system_idr� r r r �for_name_and_idsU s zDoctype.for_name_and_idsz <!DOCTYPE z> N)r r r r9 r� r� r� r r r r r� S s r� c @ sH e Zd ZdZdhdd�Zed�Zdd� Zedd � �Z e Z ed d � �Z e j d d � �Z d e effdd�Zee�Zedd� �Zdd e effdd�ZeZee�Zdd� Zdidd�Zdd� Zdjdd �Zdkd d �Zd d � Zd!d"� Zd#d$� Z d%d&� Z d'd(� Z d)d*� Z d+d,� Z d-d.� Z!d/d0� Z"d1d2� Z#d3d4� Z$d5d6� Z%d7d8� Z&dld:d;�Z'd<d=� Z(d>d?� Z)e*�rTe( Z)Z'e+dd@dAfdBdC�Z,dDdE� Z-de+d@fdFdG�Z.dmdHdI�Z/de+d@fdJdK�Z0de+d@fdLdM�Z1e+d dNfdOdP�Z2di dQdfdRdS�Z3e3Z4di dQddfdTdU�Z5e5Z6e5Z7edVdW� �Z8edXdY� �Z9dZd[d\gZ:d Z;e<j=d]�Z>d^d_� Z?dnd`da�Z@dbdc� ZAddde� ZBdfdg� ZCdS )ork z=Represents a found HTML tag with its attributes and contents.Nc C s� |dkrd| _ n|j| _ |dkr(td��|| _|| _|| _|dk rJ|j} n| rTg } ntj} | | _|dkrni }n4|r�|dk r�|jr�|j | j|�}q�t |�}nt |�}|r�|j | _ n| | _ || _ g | _| j||� d| _|dk r�|j| � |j|�| _nd| _dS )zBasic constructor.Nz%No value provided for new tag's name.F)� parser_classr� r] r r r r8 r, Zcdata_list_attributesZ$_replace_cdata_list_attribute_values�dictrP rQ rw r[ r\ �hiddenZset_up_substitutions�can_be_empty_element) r �parser�builderr r r rw r0 r� rP r8 r r r �__init__i sB z Tag.__init__r� c C sh t | �d| j| j| j| j| j| jd�}x dD ]}t||t| |�� q,W x | j D ]}|j |j � � qNW |S )z�A copy of a Tag is a new Tag, unconnected to the parse tree. Its contents are a copy of the old Tag's contents. N)rP r� r� )r� r� ) r� r� r r r rw rR r r r[ rf r� )r Zcloner rd r r r r� � s z Tag.__copy__c C s t | j�dko| jS )a7 Is this tag an empty-element tag? (aka a self-closing tag) A tag that has contents is never an empty-element tag. A tag that has no contents may or may not be an empty-element tag. It depends on the builder used to create the tag. If the builder has a designated list of empty-element tags, then only a tag whose name shows up in that list is considered an empty-element tag. If the builder has no designated list of empty-element tags, then any tag with no contents is an empty-element tag. r )ro r[ r� )r r r r �is_empty_element� s zTag.is_empty_elementc C s0 t | j�dkrdS | jd }t|t�r*|S |jS )aq Convenience property to get the single string within this tag. :Return: If this tag has a single string child, return value is that string. If this tag has no children, or more than one child, return value is None. If this tag has one child tag, return value is the 'string' attribute of the child tag, recursively. r% Nr )ro r[ r. r/ r� )r rd r r r r� � s z Tag.stringc C s | j � | j|j|�� d S )N)�clearrf r� )r r� r r r r� � s Fc c s^ xX| j D ]N}|dkr t|t� s|dk r6t|�|kr6q|rP|j� }t|�dkrPq|V qW dS )z�Yield all strings of certain classes, possibly stripping them. By default, yields only NavigableString and CData objects. So no comments, processing instructions, etc. Nr )� descendantsr. r/ r� �stripro )r r� �typesZ descendantr r r � _all_strings� s zTag._all_stringsc c s x| j d�D ] }|V q W d S )NT)r� )r r� r r r �stripped_strings� s zTag.stripped_stringsr� c C s |j dd� | j||d�D ��S )zP Get all child strings, concatenated using the given separator. c S s g | ]}|�qS r r )r� rL r r r � <listcomp>� s z Tag.get_text.<locals>.<listcomp>)r� )r� r� )r Z separatorr� r� r r r �get_text� s z Tag.get_textc C s8 | j � | }x&|dk r2|j}|jj� g |_|}qW dS )z/Recursively destroys the contents of this tree.N)r_ rX �__dict__r� r[ )r r� r� r r r � decompose� s z Tag.decomposec C s^ |r:xT| j dd� D ] }t|t�r,|j� q|j� qW n x | j dd� D ] }|j� qJW dS )zP Extract all children. If decompose is True, decompose instead. N)r[ r. rk r� r_ )r r� r� r r r r� s z Tag.clearc C s0 x"t | j�D ]\}}||kr |S q W td��dS )z� Find the index of a child by identity, not value. Avoids issues with tag.contents.index(element) getting the index of equal elements. z Tag.index: element not in tagN)� enumerater[ r] )r r� r� rd r r r r^ s z Tag.indexc C s | j j||�S )z�Returns the value of the 'key' attribute for the tag, or the value given for 'default' if it doesn't have that attribute.)rw rT )r �keyr� r r r rT s zTag.getc C s | j ||�}t|t�s |g}|S )z-The same as get(), but always returns a list.)rT r. rm )r r� r� r� r r r �get_attribute_list s zTag.get_attribute_listc C s || j kS )N)rw )r r� r r r r� & s z Tag.has_attrc C s t | �j� S )N)r �__hash__)r r r r r� ) s z Tag.__hash__c C s | j | S )zqtag[key] returns the value of the 'key' attribute for the tag, and throws an exception if it's not there.)rw )r r� r r r � __getitem__, s zTag.__getitem__c C s t | j�S )z0Iterating over a tag iterates over its contents.)�iterr[ )r r r r �__iter__1 s z Tag.__iter__c C s t | j�S )z:The length of a tag is the length of its list of contents.)ro r[ )r r r r �__len__5 s z Tag.__len__c C s || j kS )N)r[ )r �xr r r � __contains__9 s zTag.__contains__c C s dS )z-A tag is non-None even if it has no contents.Tr )r r r r �__bool__< s z Tag.__bool__c C s || j |< dS )zKSetting tag[key] sets the value of the 'key' attribute for the tag.N)rw )r r� r� r r r � __setitem__@ s zTag.__setitem__c C s | j j|d� dS )z;Deleting tag[key] deletes all 'key' attributes for the tag.N)rw �pop)r r� r r r � __delitem__E s zTag.__delitem__c O s | j ||�S )z�Calling a tag like a function is the same as calling its find_all() method. Eg. tag('a') returns a list of all the A tags found within this tag.)�find_all)r r= r> r r r �__call__I s z Tag.__call__c C sv t |�dkr@|jd�r@|d d� }tjdt|d� � | j|�S |jd� r`|dk r`| j|�S td| j|f ��d S ) N� rk z�.%(name)sTag is deprecated, use .find("%(name)s") instead. If you really were looking for a tag called %(name)sTag, use .find("%(name)sTag"))r �__r[ z!'%s' object has no attribute '%s'���) ro r� �warnings�warnr� �findr� r� r� )r rr r� r r r r� O s zTag.__getattr__c C s� | |kr dS t |d� sXt |d� sXt |d� sX| j|jksX| j|jksXt| �t|�kr\dS x(t| j�D ]\}}||j| krhdS qhW dS )zReturns true iff this tag has the same name, the same attributes, and the same contents (recursively) as the given tag.Tr rw r[ F)rp r rw ro r� r[ )r �otherr� Zmy_childr r r �__eq__` s z Tag.__eq__c C s | |k S )zZReturns true iff this tag is not identical to the other tag, as defined in __eq__.r )r r� r r r �__ne__q s z Tag.__ne__�unicode-escapec C s t r | j� S | j|�S dS )z Renders this tag as a string.N)�PY3K�decoder )r r r r r �__repr__v s z Tag.__repr__c C s | j � S )N)r )r r r r � __unicode__� s zTag.__unicode__c C s t r | j� S | j� S d S )N)r r r )r r r r �__str__� s z Tag.__str__rI �xmlcharrefreplacec C s | j |||�}|j||�S )N)r r )r r � indent_levelrM �errorsr� r r r r � s z Tag.encodec C s |dk o| j | jkS )z"Should this tag be pretty-printed?N)r r8 )r r r r r �_should_pretty_print� s zTag._should_pretty_printc C sR t |t� r t|� r | j|�}g }| jr�x�t| jj� �D ]�\}}|dkrP|}nrt |t�sdt |t�rpdj |�}n0t |t �s�t |�}n t |t �r�|dk r�|j |�}| j ||�}t |�d tj|� }|j|� q:W d} d} d} | jr�| jd } | j�rd} t |t��r |j�p| } nd| | jf } | j|�} d} d}|dk �rHd|d }| �r\|} |d }nd}| j|||�}| j�r||}n�g }d}|�r�ddj |� }|dk �r�|j|� |jd| | j|| f � | �r�|jd � |j|� | �r|�r|d d k�r|jd � | �r| �r|j| � |j| � |dk �rD| �rD| j�rD|jd � dj |�}|S ) a� Returns a Unicode representation of this tag and its contents. :param eventual_encoding: The tag is destined to be encoded into this encoding. This method is _not_ responsible for performing that encoding. This information is passed in so that it can be substituted in if the document contains a <META> tag that mentions the document's encoding. Nr� r� r� r z</%s%s>r% z <%s%s%s%s>� rV )r. r: rK rJ rw �sorted�itemsrm r� r� r r r rO r Zquoted_attribute_valuerf r r� r@ r r �decode_contentsr� rY )r r �eventual_encodingrM rw r� �valZdecodedrx �closeZcloseTagr � pretty_printZspaceZ indent_spaceZindent_contentsr[ rL Zattribute_stringr r r r � sx z Tag.decodec C s* |d kr| j d|d�S | j|d|d�S d S )NT)rM )r r )r r rM r r r �prettify� s z Tag.prettifyc C s� t |t� r t|� r | j|�}|dk }g }x�| D ]�}d}t |t�rP|j|�}n t |t�rn|j|j|||�� |r�|r�| j dk r�|j � }|r2|r�| j dk r�|jd|d � |j|� |r2| j dk r2|jd� q2W dj |�S )av Renders the contents of this tag as a Unicode string. :param indent_level: Each line of the rendering will be indented this many spaces. :param eventual_encoding: The tag is destined to be encoded into this encoding. This method is _not_ responsible for performing that encoding. This information is passed in so that it can be substituted in if the document contains a <META> tag that mentions the document's encoding. :param formatter: The output formatter responsible for converting entities to Unicode characters. Nr- r� r% r r� ) r. r: rK rJ r/ r� rk rf r r r� r� )r r r rM r rL �crx r r r r s( zTag.decode_contentsc C s | j |||�}|j|�S )ac Renders the contents of this tag as a bytestring. :param indent_level: Each line of the rendering will be indented this many spaces. :param eventual_encoding: The bytestring will be in this encoding. :param formatter: The output formatter responsible for converting entities to Unicode characters. )r r )r r r rM r[ r r r �encode_contents+ s zTag.encode_contentsr c C s |sd }| j ||d�S )N)r r )r )r r Z prettyPrintZ indentLevelr r r �renderContents= s zTag.renderContentsTc K s* d}| j ||||df|�}|r&|d }|S )zLReturn only the first child of this Tag matching the given criteria.Nr% r )r� )r r rw � recursiverx r> r� r� r r r r� F s zTag.findc K s& | j }|s| j}| j|||||f|�S )a� Extracts a list of Tag objects that match the given criteria. You can specify the name of the Tag and any attributes you want the Tag to have. The value of a key-value pair in the 'attrs' map can be a string, a list of strings, a regular expression object, or a callable that takes a string and returns whether or not the string matches for some custom definition of 'matches'. The same is true of the tag name.)r� �childrenrz )r r rw r rx r| r> r� r r r r� Q s z Tag.find_allc C s t | j�S )N)r� r[ )r r r r r e s z Tag.childrenc c s@ t | j�sd S | j� j}| jd }x||k r:|V |j}q$W d S )Nr )ro r[ ri rX )r ZstopNodeZcurrentr r r r� j s zTag.descendantsr� �+r� z "[^"]*:[^"]*"c C s | j |dd�}|r|d S dS )z9Perform a CSS selection operation on the current element.r% )r| r N)�select)r �selectorr� r r r � select_oney s zTag.select_onec ) s: d|kr�g }dd� |j d�D �}t� }t� }x�|D ]~}|dkrJtd| ��||krTq2|j|� �j||d�} x0| D ](} t| �} | |krr|j| � |j| � qrW |r2t|�|kr2P q2W |S tj |�} �g} | d3 �j kr�td| d4 ���j r�t d | � �x�t | �D �]�\}}g }tg �}| |d �j k�rB�j �rt d � �q�j �rVt d | � d �d �d }�j j|�}|d k �r�|j� \�}}}�j|||�}�n�d |k�r�|j d d�\���fdd�}|}�nxd|k�r|j dd�\�}t|j d��� � fdd�}|}�n<d|k�rԈjj|� �r�|j dd�\�}�dk�r@td��tjd|�}g } |d k�rd|} d } n |j� \} } | dk�r�y t| �} W n td��Y nX | dk �r�td��G dd� dt�} | | �j}ntd��nl|d k�r�n`|d k�r�d d � �nL|d k�rd!d � �n8|d"k�r d#d$� } | �n �jj|��r4|�n td%| ����rh| |d ����fd&d'�}!|!}"nV|d k�r��j �r���r�d(}#n�}#t d)|# � �j �r��fd*d+�}$|$}"nd,d � }"n|}"d-}%�x| D �]}&�j �r�t d.|&jt|&j�f � x�|"|&�D ]�} t | t ��s�q���r$| j�k�r$�q�|d k �rTy || �}'W n t k �rR P Y nX |d k�sd|'�r��j �r�t d/| jt| j�f � t| �|k�r�|j| � |jt| �� n �j �r�t d0| jt| j�f � �q�W �q�W |} �qW |�rt| �|k�r| d |� } �j �r6t d1� x"| D ]}(t d2|(j|(jf � �qW | S )5z9Perform a CSS selection operation on the current element.�,c S s g | ] }|j � �qS r )r� )r� r� r r r r� � s z Tag.select.<locals>.<listcomp>r� z"Invalid group selection syntax: %s)r| r% z-Final combinator "%s" is missing an argument.zRunning CSS selector "%s"z0 Token was consumed by the previous combinator.z Considering token "%s"N�#c s | j dd �� kS )N�id)rT )rr )�tag_idr r � id_matches� s z Tag.select.<locals>.id_matches�.c s � j | jdg ��S )N�class)�issubsetrT )� candidate)�classesr r � classes_match� s z!Tag.select.<locals>.classes_matchr z0A pseudo-class must be prefixed with a tag name.z ([a-zA-Z\d-]+)\(([a-zA-Z\d]+)\)z nth-of-typezMOnly numeric values are currently supported for the nth-of-type pseudo-class.z2nth-of-type pseudo-class value must be at least 1.c @ s e Zd Zdd� Zdd� ZdS )zTag.select.<locals>.Counterc S s d| _ || _d S )Nr )r� � destination)r r' r r r r� � s z$Tag.select.<locals>.Counter.__init__c S s& | j d7 _ | j | jkr dS dS d S )Nr% TF)r� r' )r rr r r r �nth_child_of_type� s z-Tag.select.<locals>.Counter.nth_child_of_typeN)r r r r� r( r r r r �Counter� s r) z?Only the following pseudo-classes are implemented: nth-of-type.r� r� c S s | j S )N)r )rr r r r r� s z Tag.select.<locals>.<lambda>r� c S s | j S )N)r )rr r r r r� s r c s s | j d�V d S )NT)r~ )rr r r r �next_tag_sibling s z$Tag.select.<locals>.next_tag_siblingz)Unsupported or invalid CSS selector: "%s"c 3 st �j r(td� | j| jf � tdd � x4| j� ��D ]$}�j rTtd|j|jf � |V q6W �j rptdd � d S )Nz- Calling select("%s") recursively on %s %sr� �( z,(Recursive select picked up candidate %s %s))� _select_debug�printr rw r )rr r� )� next_token� recursive_candidate_generatorr r r �recursive_select s z$Tag.select.<locals>.recursive_selectz[any]z- Default candidate generator, tag name="%s"c 3 s8 x2| j D ](}t|t�sq� r*|j� k r*q|V qW d S )N)r� r. rk r )rr rd )r� r r �default_candidate_generator7 s z/Tag.select.<locals>.default_candidate_generatorc S s | j S )N)r� )rr r r r r� @ s r z( Running candidate generator on %s %sz SUCCESS %s %sz FAILURE %s %szFinal verdict:z %s %srV rV ) r� r7 r] �addr r rf ro �shlex�_selector_combinatorsr, r- r� r� r$ �groupsr� � quoted_colonr# r) �intr< �objectr( r� r �reprrw r. rk r� ))r r Z_candidate_generatorr| �contextZ selectorsZused_selectorsZselected_object_idsZpartial_selectorZ candidatesr$ Z object_id�tokensZcurrent_contextr^ �tokenZ new_contextZnew_context_idsZchecker�mr� r� r� r �klassr& ZpseudoZpseudo_attributesr� Z pseudo_typeZ pseudo_valuer) r* r0 Z_use_candidate_generatorZcheckr1 r� rr r� r� r )r% r. r/ r r r� r r � s $ z Tag.selectc C s | j S )N)r )r r r r �childGeneratork s zTag.childGeneratorc C s | j S )N)r� )r r r r �recursiveChildGeneratorn s zTag.recursiveChildGeneratorc C s t jd| � | j|�S )z�This was kind of misleading because has_key() (attributes) was different from __in__ (contents). has_key() is gone in Python 3, anyway.z2has_key is deprecated. Use has_attr("%s") instead.)r� r� r� )r r� r r r �has_keyq s z Tag.has_key) NNNNNNNNN)F)N)N)r )NrI )NN)Dr r r r r� r Z parserClassr� r r� Z isSelfClosingr� r r/ r� r� Zstringsr� r� ZgetTextrx r� r� r^ rT r� r� r� r� r� r� r� r� r� r� r� r� r� r� r r r r r� r r r r r r r r� Z findChildr� ZfindAllZ findChildrenr r� r4 r, r) r* r6 r r r? r@ rA r r r r rk e s� 3 \ ) lrk c @ sT e Zd ZdZdi dfdd�Zdd� Zdd� Zdi fd d �ZeZd d � Z dd d�Z dS )r� zMEncapsulates a number of ways of matching a markup element (tag or text).Nc K s� | j |�| _t|t�s"||d< d }d|kr<|d |d< |d= |r\|rX|j� }|j|� n|}i }x&t|j� �D ]\}}| j |�||< qnW || _| j |�| _ d S )Nr" Zclass_) �_normalize_search_valuer r. r� �copy�updaterm r rw rx )r r rw rx r> Znormalized_attrsr� r� r r r r� ~ s" zSoupStrainer.__init__c C s� t |t�s.t|�s.t|d�s.t |t�s.|d kr2|S t |t�rF|jd�S t|d�r�g }xJ|D ]B}t|d�r�t |t� r�t |t� r�|j|� qZ|j| j|�� qZW |S tt|��S )Nr$ �utf8r� ) r. r rK rp �bool�bytesr rf rB )r r� Z new_value�vr r r rB � s z$SoupStrainer._normalize_search_valuec C s | j r | j S d| j| jf S d S )Nz%s|%s)rx r rw )r r r r r � s zSoupStrainer.__str__c C s$ d }d }t |t�r|}|}t | jt�o0t |t� }| j sd|sd|rP| j|| j�sd| r�| j|| j�r�|rv| j||�}nrd}d }xht| jj� �D ]V\}} |s�t|d�r�|}n i }x|D ]\} } | || < q�W |j |�} | j| | �s�d}P q�W |r�|r�|}n|}|�r | j �r | j|j | j � �r d }|S )NTrT F) r. rk r r �_matchesrm rw r rp rT rx r� ) r Z markup_nameZ markup_attrsr� �markupZcall_function_with_tag_datar$ Zmarkup_attr_mapr � match_against�krH Z attr_valuer r r � search_tag� sB "zSoupStrainer.search_tagc C s� d }t |d�rHt|ttf� rHx�|D ] }t|t�r$| j|�r$|}P q$W npt|t�rr| j sf| jsf| jr�| j |�}nFt|t�s�t|t�r�| j r�| j r�| j || j�r�|}nt d|j ��|S )Nr� z&I don't know how to match against a %s) rp r. rk r r/ r# rx r rw rM rI � Exceptionr� )r rJ r� r� r r r r# � s" zSoupStrainer.searchc C s� d}t |t�st |t�rPx |D ]}| j||�r dS q W | jdj|�|�rLdS dS |dkr`|d k S t |t�rr||�S |}t |t�r�|j}| j|�}|d kr�| S t |d�o�t |t � �r|s�t � }xL|D ]@}|j r�|}nt |�}||kr�q�q�|j|� | j|||�r�dS q�W dS d}| �r.t |t ��r.||k}| �rLt |d��rL|j|�S | �r�t |t��r�|j�r�| j|jd |j |�S |S )NFTr� r� r# r )r. rm r� rI r� r rk r rB rp r r7 r� r r2 r# r ) r rJ rK Z already_triedr� �itemZoriginal_markupr� r$ r r r rI sV zSoupStrainer._matches)N) r r r r r� rB r rM Z searchTagr# rI r r r r r� z s 'r� c s, e Zd ZdZf f� fdd� Zdd� Z� ZS )r� zTA ResultSet is just a list that keeps track of the SoupStrainer that created it.c s t t| �j|� || _d S )N)�superr� r� �source)r rQ r� )r� r r r� V s zResultSet.__init__c C s t d| ��d S )Nz�ResultSet object has no attribute '%s'. You're probably treating a list of items like a single item. Did you call find_all() when you meant to call find()?)r� )r r� r r r r� Z s zResultSet.__getattr__)r r r r r� r� � __classcell__r r )r� r r� S s r� ),Z __license__�collections.abcr � ImportError�e� collectionsr) r3 �sysr� Z bs4.dammitr r� � version_infor r* Z whitespace_rer r r r r r! r, r8 r: rA rC rD rE rF rG r/ r� r� r� r� r� r� r� rk r� rm r� r r r r �<module> s^ % R9 Z