EVOLUTION-NINJA
Edit File: diagnose.cpython-36.pyc
3 #6]� � @ s d Z dZddlZddlmZ ddlmZ ddlZddlmZm Z ddl m Z ddl Z ddl Z ddlZddlZddlZddlZddlZddlZdd � Zd#d d �ZG d d� de�Zdd� ZdZdZd$dd�Zd%dd�Zd&dd�Zd'd d �Z d(d d!�Z e d"k�reej j � � dS ))z=Diagnostic functions, mainly for use when doing tech support.ZMIT� N)�StringIO)� HTMLParser)� BeautifulSoup� __version__)�builder_registryc ; C sX t dt � t dtj � dddg}x>|D ]6}x0tjD ]}||jkr6P q6W |j|� t d| � q*W d|kr�|jd� y*dd l m } t d d j t t |j�� � W n* tk r� } zt d � W Y d d }~X nX d|k�r ydd l}t d|j � W n, tk �r } zt d� W Y d d }~X nX t| d��r4| j� } n�| jd��sL| jd��rdt d| � t d� d S y:tjj| ��r�t d| � t| ��}|j� } W d Q R X W n tk �r� Y nX t � x�|D ]�}t d| � d} yt| |d�} d} W n8 tk �r" } zt d| � tj � W Y d d }~X nX | �rBt d| � t | j � � t d d � �q�W d S ) z/Diagnostic suite for isolating common problems.z'Diagnostic running on Beautiful Soup %szPython version %sz html.parser�html5lib�lxmlz;I noticed that %s is not installed. Installing it may help.zlxml-xmlr )�etreezFound lxml version %s�.z.lxml is not installed or couldn't be imported.NzFound html5lib version %sz2html5lib is not installed or couldn't be imported.�readzhttp:zhttps:z<"%s" looks like a URL. Beautiful Soup is not an HTTP client.zpYou need to use some other library to get the document behind the URL, and feed that document to Beautiful Soup.z7"%s" looks like a filename. Reading data from the file.z#Trying to parse your markup with %sF)�featuresTz %s could not parse the markup.z#Here's what %s did with the markup:�-�P ) �printr �sys�versionr Zbuildersr �remove�appendr r �join�map�strZ LXML_VERSION� ImportErrorr �hasattrr � startswith�os�path�exists�open� ValueErrorr � Exception� traceback� print_excZprettify) �dataZ basic_parsers�nameZbuilderr �er �fp�parser�success�soup� r) � /usr/lib/python3.6/diagnose.py�diagnose sj r+ Tc K sN ddl m} x<|jt| �fd|i|��D ] \}}td||j|jf � q(W dS )z�Print out the lxml events that occur during parsing. This lets you see how lxml parses a document when no Beautiful Soup code is running. r )r �htmlz %s, %4s, %sN)r r Z iterparser r �tag�text)r" r, �kwargsr Zevent�elementr) r) r* � lxml_traceZ s $r1 c @ s` e Zd ZdZdd� Zdd� Zdd� Zdd � Zd d � Zd d � Z dd� Z dd� Z dd� Z dd� Z dS )�AnnouncingParserz?Announces HTMLParser parse events, without doing anything else.c C s t |� d S )N)r )�self�sr) r) r* �_pg s zAnnouncingParser._pc C s | j d| � d S )Nz%s START)r5 )r3 r# Zattrsr) r) r* �handle_starttagj s z AnnouncingParser.handle_starttagc C s | j d| � d S )Nz%s END)r5 )r3 r# r) r) r* � handle_endtagm s z AnnouncingParser.handle_endtagc C s | j d| � d S )Nz%s DATA)r5 )r3 r" r) r) r* � handle_datap s z AnnouncingParser.handle_datac C s | j d| � d S )Nz %s CHARREF)r5 )r3 r# r) r) r* �handle_charrefs s z AnnouncingParser.handle_charrefc C s | j d| � d S )Nz %s ENTITYREF)r5 )r3 r# r) r) r* �handle_entityrefv s z!AnnouncingParser.handle_entityrefc C s | j d| � d S )Nz %s COMMENT)r5 )r3 r" r) r) r* �handle_commenty s z AnnouncingParser.handle_commentc C s | j d| � d S )Nz%s DECL)r5 )r3 r" r) r) r* � handle_decl| s z AnnouncingParser.handle_declc C s | j d| � d S )Nz%s UNKNOWN-DECL)r5 )r3 r" r) r) r* � unknown_decl s z AnnouncingParser.unknown_declc C s | j d| � d S )Nz%s PI)r5 )r3 r" r) r) r* � handle_pi� s zAnnouncingParser.handle_piN)�__name__� __module__� __qualname__�__doc__r5 r6 r7 r8 r9 r: r; r< r= r> r) r) r) r* r2 d s r2 c C s t � }|j| � dS )z�Print out the HTMLParser events that occur during parsing. This lets you see how HTMLParser parses a document when no Beautiful Soup code is running. N)r2 Zfeed)r" r& r) r) r* �htmlparser_trace� s rC ZaeiouZbcdfghjklmnpqrstvwxyz� c C s> d}x4t | �D ](}|d dkr$t}nt}|tj|�7 }qW |S )z#Generate a random word-like string.� � r )�range� _consonants�_vowels�random�choice)�lengthr4 �i�tr) r) r* �rword� s rO � c C s dj dd� t| �D ��S )z'Generate a random sentence-like string.� c s s | ]}t tjd d��V qdS )rP � N)rO rJ �randint)�.0rM r) r) r* � <genexpr>� s z rsentence.<locals>.<genexpr>)r rG )rL r) r) r* � rsentence� s rV �� c C s� dddddddg}g }x~t | �D ]r}tjdd �}|dkrRtj|�}|jd | � q |d krr|jttjd d ��� q |d kr tj|�}|jd| � q W ddj|� d S )z+Randomly generate an invalid HTML document.�pZdiv�spanrM �bZscript�tabler � z<%s>� rP rF z</%s>z<html>� z</html>)rG rJ rS rK r rV r )� num_elementsZ tag_names�elementsrM rK Ztag_namer) r) r* �rdoc� s ra 順 c C s( t dt � t| �}t dt|� � x�dddgddgD ]z}d}y"tj� }t||�}tj� }d}W n6 tk r� } zt d | � tj� W Y d d }~X nX |r6t d ||| f � q6W d d l m } tj� }|j |� tj� }t d|| � d d l } | j � }tj� }|j|� tj� }t d|| � d S )z.Very basic head-to-head performance benchmark.z1Comparative parser benchmark on Beautiful Soup %sz3Generated a large invalid HTML document (%d bytes).r r, r z html.parserFTz %s could not parse the markup.Nz"BS4+%s parsed the markup in %.2fs.r )r z$Raw lxml parsed the markup in %.2fs.z(Raw html5lib parsed the markup in %.2fs.)r r ra �len�timer r r r! r r ZHTMLr r �parse) r_ r" r& r' �ar( rZ r$ r r r) r) r* �benchmark_parsers� s4 rg r c C sX t j� }|j}t| �}tt||d�}tjd|||� tj |�}|j d� |j dd� d S )N)�bs4r" r& z bs4.BeautifulSoup(data, parser)Z cumulativez _html5lib|bs4�2 ) �tempfileZNamedTemporaryFiler# ra �dictrh �cProfileZrunctx�pstatsZStatsZ sort_statsZ print_stats)r_ r& Z filehandle�filenamer" �varsZstatsr) r) r* �profile� s rp �__main__)T)rD )rP )rW )rb )rb r )!rB Z __license__rl �ior Z html.parserr rh r r Z bs4.builderr r rm rJ rj rd r r r+ r1 r2 rC rI rH rO rV ra rg rp r? �stdinr r) r) r) r* �<module> s8 C !