EVOLUTION-NINJA
Edit File: scanner.cpython-36.pyc
3 <z�W�� � @ sL d dgZ ddlmZ ddlT G dd� de�ZG dd� d�ZG dd � d �Zd S ) �Scanner� ScannerError� )�MarkedYAMLError)�*c @ s e Zd ZdS )r N)�__name__� __module__� __qualname__� r r � /usr/lib64/python3.6/scanner.pyr s c @ s e Zd Zdd� ZdS )� SimpleKeyc C s( || _ || _|| _|| _|| _|| _d S )N)� token_number�required�index�line�column�mark)�selfr r r r r r r r r �__init__& s zSimpleKey.__init__N)r r r r r r r r r # s r c @ sx e Zd Zdd� Zdd� Zdd� Zdd� Zd d � Zd d � Zd d� Z dd� Z dd� Z dd� Z dd� Z dd� Zdd� Zdd � Zd d � Zd d � Zd!d"� Zd#d$� Zd%d&� Zd'd(� Zd)d*� Zd+d,� Zd-d.� Zd/d0� Zd1d2� Zd3d4� Z d5d6� Z d7d8� Z d9d:� Z d;d<� Z d=d>� Z!d?d@� Z"dAdB� Z#dCdD� Z$dEdF� Z%dGdH� Z&dIdJ� Z'dKdL� Z(dMdN� Z)dOdP� Z*dQdR� Z+dSdT� Z,dUdV� Z-dWdX� Z.dYdZ� Z/d[d\� Z0d]d^� Z1d_d`� Z2dadb� Z3dcdd� Z4dedf� Z5dgdh� Z6didj� Z7dkdl� Z8dmdn� Z9dodp� Z:dqdr� Z;dsdt� Z<dudv� Z=dwdx� Z>dydz� Z?d{d|� Z@d}d~dd�d�d�d�d�d�d�d�d�d�d�d�d�d�d��ZAd�d�d�d��ZBd�d�� ZCd�d�� ZDd�d�� ZEd�d�� ZFd�d�� ZGd�d�� ZHd�d�� ZId�d�� ZJd�d�� ZKd�S )�r c C s< d| _ d| _g | _| j� d| _d| _g | _d| _i | _dS )zInitialize the scanner.F� r TN���) �done� flow_level�tokens�fetch_stream_start� tokens_taken�indent�indents�allow_simple_key�possible_simple_keys)r r r r r 0 s zScanner.__init__c G sJ x| j � r| j� qW | jrF|s$dS x |D ]}t| jd |�r*dS q*W dS )NTr F)�need_more_tokens�fetch_more_tokensr � isinstance)r �choicesZchoicer r r � check_tokenq s zScanner.check_tokenc C s* x| j � r| j� qW | jr&| jd S d S )Nr )r r r )r r r r � peek_token} s zScanner.peek_tokenc C s: x| j � r| j� qW | jr6| jd7 _| jjd�S d S )Nr r )r r r r �pop)r r r r � get_token� s zScanner.get_tokenc C s2 | j r dS | jsdS | j� | j� | jkr.dS d S )NFT)r r �stale_possible_simple_keys�next_possible_simple_keyr )r r r r r � s zScanner.need_more_tokensc C s� | j � | j� | j| j� | j� }|dkr4| j� S |dkrL| j� rL| j� S |dkrd| j� rd| j � S |dkr|| j � r|| j � S |dkr�| j � S |dkr�| j � S |dkr�| j� S |dkr�| j� S |d kr�| j� S |dkr�| j� r�| j� S |d kr�| j� r�| j� S |d k�r| j� �r| j� S |d k�r*| j� S |d k�r<| j� S |dk�rN| j� S |dk�rj| j �rj| j� S |dk�r�| j �r�| j � S |dk�r�| j � S |dk�r�| j � S | j � �r�| j � S t!dd d| | j"� ��d S )N� �%�-�.�[�{�]�}�,�?�:r �&�!�|�>�'�"z!while scanning for the next tokenz.found character %r that cannot start any token)#�scan_to_next_tokenr' � unwind_indentr �peek�fetch_stream_end�check_directive�fetch_directive�check_document_start�fetch_document_start�check_document_end�fetch_document_end�fetch_flow_sequence_start�fetch_flow_mapping_start�fetch_flow_sequence_end�fetch_flow_mapping_end�fetch_flow_entry�check_block_entry�fetch_block_entry� check_key� fetch_key� check_value� fetch_value� fetch_alias� fetch_anchor� fetch_tagr � fetch_literal� fetch_folded� fetch_single� fetch_double� check_plain� fetch_plainr �get_mark)r �chr r r r � s^ zScanner.fetch_more_tokensc C s: d }x0| j D ]&}| j | }|d ks,|j|k r |j}q W |S )N)r r )r Zmin_token_number�level�keyr r r r( s z Scanner.next_possible_simple_keyc C s` xZt | j�D ]L}| j| }|j| jks6| j|j dkr |jrPtd|jd| j� ��| j|= q W d S )Ni zwhile scanning a simple keyzcould not find expected ':')�listr r r r r r rX )r rZ r[ r r r r' s z"Scanner.stale_possible_simple_keysc C s^ | j o| j| jk}| jrZ| j� | jt| j� }t||| j | j | j| j � �}|| j | j < d S )N) r r r r �remove_possible_simple_keyr �lenr r r r rX r )r r r r[ r r r �save_possible_simple_key$ s z Scanner.save_possible_simple_keyc C s@ | j | jkr<| j| j }|jr2td|jd| j� ��| j| j = d S )Nzwhile scanning a simple keyzcould not find expected ':')r r r r r rX )r r[ r r r r] 5 s z"Scanner.remove_possible_simple_keyc C sD | j r d S x4| j|kr>| j� }| jj� | _| jjt||�� q W d S )N)r r rX r r% r �appendZ BlockEndToken)r r r r r r r; B s zScanner.unwind_indentc C s& | j |k r"| jj| j � || _ dS dS )NTF)r r r` )r r r r r � add_indentZ s zScanner.add_indentc C s$ | j � }| jjt||| jd�� d S )N)�encoding)rX r r` ZStreamStartTokenrb )r r r r r r d s zScanner.fetch_stream_startc C sB | j d� | j� d| _i | _| j� }| jjt||�� d| _d S )Nr FTr ) r; r] r r rX r r` ZStreamEndTokenr )r r r r r r= p s zScanner.fetch_stream_endc C s, | j d� | j� d| _| jj| j� � d S )Nr Fr )r; r] r r r` �scan_directive)r r r r r? � s zScanner.fetch_directivec C s | j t� d S )N)�fetch_document_indicatorZDocumentStartToken)r r r r rA � s z Scanner.fetch_document_startc C s | j t� d S )N)rd ZDocumentEndToken)r r r r rC � s zScanner.fetch_document_endc C sH | j d� | j� d| _| j� }| jd� | j� }| jj|||�� d S )Nr F� r )r; r] r rX �forwardr r` )r � TokenClass� start_mark�end_markr r r rd � s z Scanner.fetch_document_indicatorc C s | j t� d S )N)�fetch_flow_collection_startZFlowSequenceStartToken)r r r r rD � s z!Scanner.fetch_flow_sequence_startc C s | j t� d S )N)rj ZFlowMappingStartToken)r r r r rE � s z Scanner.fetch_flow_mapping_startc C sJ | j � | jd7 _d| _| j� }| j� | j� }| jj|||�� d S )Nr T)r_ r r rX rf r r` )r rg rh ri r r r rj � s z#Scanner.fetch_flow_collection_startc C s | j t� d S )N)�fetch_flow_collection_endZFlowSequenceEndToken)r r r r rF � s z Scanner.fetch_flow_sequence_endc C s | j t� d S )N)rk ZFlowMappingEndToken)r r r r rG � s z Scanner.fetch_flow_mapping_endc C sJ | j � | jd8 _d| _| j� }| j� | j� }| jj|||�� d S )Nr F)r] r r rX rf r r` )r rg rh ri r r r rk � s z!Scanner.fetch_flow_collection_endc C s< d| _ | j� | j� }| j� | j� }| jjt||�� d S )NT)r r] rX rf r r` ZFlowEntryToken)r rh ri r r r rH � s zScanner.fetch_flow_entryc C s� | j sF| js td d d| j� ��| j| j�rF| j� }| jjt||�� n d| _| j � | j� }| j � | j� }| jjt ||�� d S )Nz%sequence entries are not allowed hereT) r r r rX ra r r r` ZBlockSequenceStartTokenr] rf ZBlockEntryToken)r r rh ri r r r rJ � s zScanner.fetch_block_entryc C s� | j sD| js td d d| j� ��| j| j�rD| j� }| jjt||�� | j | _| j � | j� }| j � | j� }| jjt ||�� d S )Nz!mapping keys are not allowed here) r r r rX ra r r r` �BlockMappingStartTokenr] rf �KeyToken)r r rh ri r r r rL s zScanner.fetch_keyc C s | j | jkr|| j| j }| j| j = | jj|j| j t|j|j�� | j st| j|j �rt| jj|j| j t |j|j�� d| _ n\| j s�| j s�t d d d| j � ��| j s�| j| j �r�| j � }| jjt ||�� | j | _ | j� | j � }| j� | j � }| jjt||�� d S )NFz#mapping values are not allowed here)r r r �insertr r rm r ra r rl r r rX r` r] rf Z ValueToken)r r[ r rh ri r r r rN s2 zScanner.fetch_valuec C s$ | j � d| _| jj| jt�� d S )NF)r_ r r r` � scan_anchorZ AliasToken)r r r r rO V s zScanner.fetch_aliasc C s$ | j � d| _| jj| jt�� d S )NF)r_ r r r` ro Z AnchorToken)r r r r rP a s zScanner.fetch_anchorc C s" | j � d| _| jj| j� � d S )NF)r_ r r r` �scan_tag)r r r r rQ l s zScanner.fetch_tagc C s | j dd� d S )Nr6 )�style)�fetch_block_scalar)r r r r rR w s zScanner.fetch_literalc C s | j dd� d S )Nr7 )rq )rr )r r r r rS z s zScanner.fetch_foldedc C s$ d| _ | j� | jj| j|�� d S )NT)r r] r r` �scan_block_scalar)r rq r r r rr } s zScanner.fetch_block_scalarc C s | j dd� d S )Nr8 )rq )�fetch_flow_scalar)r r r r rT � s zScanner.fetch_singlec C s | j dd� d S )Nr9 )rq )rt )r r r r rU � s zScanner.fetch_doublec C s$ | j � d| _| jj| j|�� d S )NF)r_ r r r` �scan_flow_scalar)r rq r r r rt � s zScanner.fetch_flow_scalarc C s" | j � d| _| jj| j� � d S )NF)r_ r r r` � scan_plain)r r r r rW � s zScanner.fetch_plainc C s | j dkrdS d S )Nr T)r )r r r r r> � s zScanner.check_directivec C s. | j dkr*| jd�dkr*| jd�dkr*dS d S )Nr re z---u T)r �prefixr< )r r r r r@ � s z Scanner.check_document_startc C s. | j dkr*| jd�dkr*| jd�dkr*dS d S )Nr re z...u T)r rw r< )r r r r rB � s zScanner.check_document_endc C s | j d�dkS )Nr u )r< )r r r r rI � s zScanner.check_block_entryc C s | j r dS | jd�dkS d S )NTr u )r r< )r r r r rK � s zScanner.check_keyc C s | j r dS | jd�dkS d S )NTr u )r r< )r r r r rM � s zScanner.check_valuec C s6 | j � }|dkp4| j d�dko4|dkp4| j o4|dkS )Nu -?:,[]{}#&*!|>'"%@`r u r+ z?:)r< r )r rY r r r rV � s zScanner.check_plainc C s� | j dkr | j� dkr | j� d}xb|s�x| j� dkr@| j� q*W | j� dkrhx| j� dkrf| j� qPW | j� r~| js�d| _q$d}q$W d S )Nr u F� �#u T)r r< rf �scan_line_breakr r )r �foundr r r r: � s zScanner.scan_to_next_tokenc C s� | j � }| j� | j|�}d }|dkr:| j|�}| j � }n>|dkrV| j|�}| j � }n"| j � }x| j� dkrv| j� q`W | j|� t||||�S )NZYAMLZTAGu )rX rf �scan_directive_name�scan_yaml_directive_value�scan_tag_directive_valuer< �scan_directive_ignored_lineZDirectiveToken)r rh �name�valueri r r r rc s zScanner.scan_directivec C s� d}| j |�}xfd| ko"dkn s`d| ko:dkn s`d| koRdkn s`|dkrt|d 7 }| j |�}qW |s�td |d | | j� ��| j|�}| j|� | j � }|d kr�td |d | | j� ��|S ) Nr �0�9�A�Z�a�zz-_r zwhile scanning a directivez6expected alphabetic or numeric character, but found %ru )r< r rX rw rf )r rh �lengthrY r� r r r r| # s$ J zScanner.scan_directive_namec C s� x| j � dkr| j� qW | j|�}| j � dkrJtd|d| j � | j� ��| j� | j|�}| j � dkr�td|d| j � | j� ��||fS )Nrx r, zwhile scanning a directivez%expected a digit or '.', but found %ru z%expected a digit or ' ', but found %r)r< rf �scan_yaml_directive_numberr rX )r rh �major�minorr r r r} 8 s z!Scanner.scan_yaml_directive_valuec C s� | j � }d| kodkn s6td|d| | j� ��d}x*d| j |� koTdkn rd|d7 }q<W t| j|��}| j|� |S )Nr� r� zwhile scanning a directivez expected a digit, but found %rr r )r< r rX �intrw rf )r rh rY r� r� r r r r� I s z"Scanner.scan_yaml_directive_numberc C sP x| j � dkr| j� qW | j|�}x| j � dkr<| j� q&W | j|�}||fS )Nrx )r< rf �scan_tag_directive_handle�scan_tag_directive_prefix)r rh �handlerw r r r r~ V s z Scanner.scan_tag_directive_valuec C s6 | j d|�}| j� }|dkr2td|d| | j� ��|S )N� directiverx zwhile scanning a directivezexpected ' ', but found %r)�scan_tag_handler< r rX )r rh r� rY r r r r� ` s z!Scanner.scan_tag_directive_handlec C s6 | j d|�}| j� }|dkr2td|d| | j� ��|S )Nr� u zwhile scanning a directivezexpected ' ', but found %r)� scan_tag_urir< r rX )r rh r� rY r r r r� i s z!Scanner.scan_tag_directive_prefixc C sr x| j � dkr| j� qW | j � dkr@x| j � dkr>| j� q(W | j � }|dkrftd|d| | j� ��| j� d S )Nrx ry u zwhile scanning a directivez0expected a comment or a line break, but found %r)r< rf r rX rz )r rh rY r r r r r s z#Scanner.scan_directive_ignored_linec C s | j � }| j� }|dkr d}nd}| j� d}| j|�}xfd| koLdkn s�d| koddkn s�d | ko|d kn s�|d kr�|d 7 }| j|�}q:W |s�td | |d| | j � ��| j|�}| j|� | j� }|dkr�td | |d| | j � ��| j � }||||�S )Nr �aliasZanchorr r� r� r� r� r� r� z-_r zwhile scanning an %sz6expected alphabetic or numeric character, but found %ru ?:,]}%@`)rX r< rf r rw ) r rg rh Z indicatorr� r� rY r� ri r r r ro � s2 J zScanner.scan_anchorc C s& | j � }| jd�}|dkrdd }| jd� | jd|�}| j� dkrZtd|d| j� | j � ��| j� n~|dkr~d }d }| j� ndd}d }x,|d kr�|d kr�d }P |d7 }| j|�}q�W d }|r�| jd|�}n d }| j� | jd|�}| j� }|d k�r td |d| | j � ��||f}| j � }t|||�S )Nr �<� �tagr7 zwhile parsing a tagzexpected '>', but found %ru r5 Fu Tzwhile scanning a tagzexpected ' ', but found %r)rX r< rf r� r r� ZTagToken) r rh rY r� �suffixr� Z use_handler� ri r r r rp � sH zScanner.scan_tagc C s� |dkrd}nd}g }| j � }| j� | j|�\}}| j|� | jd }|dk rTd}|d krv| j� \}} } t|| �} n|| d } | j| �\}} d} x�| j| ko�| j � dk�rt|j |� | j � dk} d}x| j |�d kr�|d7 }q�W |j | j |�� | j|� | j � } | j| �\}} | j| k�rp| j � dk�rp|�rd| d k�rd| �rd| j � dk�rd|�sn|j d � n |j | � q�P q�W |dk �r�|j | � |dk�r�|j |� tdj|�d|| |�S ) Nr7 TFr � r) z r u � rx )rX rf � scan_block_scalar_indicators� scan_block_scalar_ignored_liner � scan_block_scalar_indentation�max�scan_block_scalar_breaksr r< �extendr` rw rz � ScalarToken�join)r rq Zfolded�chunksrh �chomping� incrementZ min_indent�breaks� max_indentri r � line_breakZleading_non_spacer� r r r rs � sP zScanner.scan_block_scalarc C s� d }d }| j � }|dkrn|dkr&d}nd}| j� | j � }|dkr�t|�}|dkrdtd|d| j� ��| j� n\|dkr�t|�}|dkr�td|d| j� ��| j� | j � }|dkr�|dkr�d}nd}| j� | j � }|d kr�td|d | | j� ��||fS ) Nz+-�+TF� 0123456789r z while scanning a block scalarz<expected indentation indicator in the range 1-9, but found 0u z9expected chomping or indentation indicators, but found %r)r< rf r� r rX )r rh r� r� rY r r r r� sF z$Scanner.scan_block_scalar_indicatorsc C sr x| j � dkr| j� qW | j � dkr@x| j � dkr>| j� q(W | j � }|dkrftd|d| | j� ��| j� d S )Nrx ry u z while scanning a block scalarz0expected a comment or a line break, but found %r)r< rf r rX rz )r rh rY r r r r� A s z&Scanner.scan_block_scalar_ignored_linec C sh g }d}| j � }xL| j� dkr\| j� dkrB|j| j� � | j � }q| j� | j|kr| j}qW |||fS )Nr u rx )rX r< r` rz rf r )r r� r� ri r r r r� O s z%Scanner.scan_block_scalar_indentationc C s� g }| j � }x"| j|k r.| j� dkr.| j� qW xJ| j� dkrz|j| j� � | j � }x"| j|k rv| j� dkrv| j� qVW q2W ||fS )Nrx u )rX r r< rf r` rz )r r r� ri r r r r� ^ s z Scanner.scan_block_scalar_breaksc C s� |dkrd}nd}g }| j � }| j� }| j� |j| j||�� x4| j� |krt|j| j||�� |j| j||�� qBW | j� | j � }tdj|�d|||�S )Nr9 TFr� )rX r< rf r� �scan_flow_scalar_non_spaces�scan_flow_scalar_spacesr� r� )r rq �doubler� rh Zquoteri r r r ru k s zScanner.scan_flow_scalarr) ��� r� � � � �rx r9 �\� � u u )r� r� �b�tr� �n�v�f�r�erx r9 r� �N�_�L�Pr� � � )�x�u�Uc C s� g }�x�d}x| j |�dkr&|d7 }qW |rF|j| j|�� | j|� | j � }| r�|dkr�| j d�dkr�|jd� | jd� q|r�|dks�| r�|dkr�|j|� | j� q|o�|dk�r�| j� | j � }|| jkr�|j| j| � | j� n�|| jk�r|| j| }| j� xBt|�D ]6}| j |�dk�rtd |d || j |�f | j� ���qW t | j|�d �}|jt |�� | j|� n<|d k�r�| j � |j | j ||�� ntd |d | | j� ��q|S qW d S )Nr u '"\ r r8 r� z"\r� �0123456789ABCDEFabcdefz%while scanning a double-quoted scalarz?expected escape sequence of %d hexdecimal numbers, but found %r� u z!found unknown escape character %r)r< r` rw rf �ESCAPE_REPLACEMENTS� ESCAPE_CODES�ranger rX r� �chrrz r� �scan_flow_scalar_breaks)r r� rh r� r� rY �k�coder r r r� � sL z#Scanner.scan_flow_scalar_non_spacesc C s� g }d}x| j |�dkr"|d7 }q W | j|�}| j|� | j � }|dkr\td|d| j� ��nT|dkr�| j� }| j||�}|dkr�|j|� n|s�|jd � |j|� n |j|� |S ) Nr z r r) z while scanning a quoted scalarz found unexpected end of streamu r� rx ) r< rw rf r rX rz r� r` r� ) r r� rh r� r� � whitespacesrY r� r� r r r r� � s( z Scanner.scan_flow_scalar_spacesc C s� g }xx| j d�}|dks |dkr@| jd�dkr@td|d| j� ��x| j� dkrX| j� qBW | j� dkrv|j| j� � q|S qW d S ) Nre z---z...u z while scanning a quoted scalarz#found unexpected document separatorz u )rw r< r rX rf r` rz )r r� rh r� rw r r r r� � s z Scanner.scan_flow_scalar_breaksc C sN g }| j � }|}| jd }g }�xd}| j� dkr4P xP| j|�}|dksx| j rj|dkrj| j|d �dksx| jrz|dkrzP |d7 }q6W | jr�|dkr�| j|d �dkr�| j|� td|d | j � d ��|dkr�P d | _|j|� |j| j |�� | j|� | j � }| j ||�}| �s4| j� dk�s4| j r"| j |k r"P q"W t d j |�d ||�S )Nr r ry u r3 z,:?[]{}u ,[]{}z while scanning a plain scalarzfound unexpected ':'zGPlease check http://pyyaml.org/wiki/YAMLColonInFlowContext for details.Fr� T)rX r r< r rf r r r� r` rw �scan_plain_spacesr r� r� )r r� rh ri r Zspacesr� rY r r r rv � sF zScanner.scan_plainc C s. g }d}x| j |�dkr"|d7 }q W | j|�}| j|� | j � }|dk�r| j� }d| _| jd�}|dksr|dkr�| j d�d kr�d S g } x`| j � d kr�| j � dkr�| j� q�| j| j� � | jd�}|dks�|dkr�| j d�d kr�d S q�W |d kr�|j|� n| �s|jd� |j| � n|�r*|j|� |S ) Nr rx r u Tre z---z...u u r� )r< rw rf rz r r` r� ) r r rh r� r� r� rY r� rw r� r r r r� ! s> zScanner.scan_plain_spacesc C s� | j � }|dkr*td| |d| | j� ��d}| j |�}|dkr�xfd| koTdkn s�d| kold kn s�d | ko�d kn s�|d kr�|d7 }| j |�}qBW |dkr�| j|� td| |d| | j� ��|d7 }| j|�}| j|� |S ) Nr5 zwhile scanning a %szexpected '!', but found %rr rx r� r� r� r� r� r� z-_)r< r rX rf rw )r r� rh rY r� r� r r r r� F s& J zScanner.scan_tag_handlec C s� g }d}| j |�}x�d| ko&dkn sdd| ko>dkn sdd| koVdkn sd|dkr�|d kr�|j| j|�� | j|� d}|j| j||�� n|d 7 }| j |�}qW |r�|j| j|�� | j|� d}|s�td | |d | | j� ��d j|�S )Nr r� r� r� r� r� r� z-;/?:@&=+$,_.!~*'()[]%r* r zwhile parsing a %szexpected URI, but found %rr� )r< r` rw rf �scan_uri_escapesr rX r� )r r� rh r� r� rY r r r r� ^ s( J zScanner.scan_tag_uric C s� g }| j � }xx| j� dkr�| j� x>td�D ]2}| j|�dkr,td| |d| j|� | j � ��q,W |jt| jd�d�� | jd� qW yt|�j d�}W n8 t k r� } z td| |t |�|��W Y d d }~X nX |S )Nr* r� r� zwhile scanning a %szBexpected URI escape sequence of 2 hexdecimal numbers, but found %rr� zutf-8) rX r< rf r� r r` r� rw �bytes�decode�UnicodeDecodeError�str)r r� rh Zcodesr r� r� �excr r r r� w s (zScanner.scan_uri_escapesc C sN | j � }|dkr6| jd�dkr*| jd� n| j� dS |dkrJ| j� |S dS )Nu r� z r� u r� )r< rw rf )r rY r r r rz � s zScanner.scan_line_breakN)Lr r r r r# r$ r& r r r( r' r_ r] r; ra r r= r? rA rC rd rD rE rj rF rG rk rH rJ rL rN rO rP rQ rR rS rr rT rU rt rW r> r@ rB rI rK rM rV r: rc r| r} r� r~ r� r� r ro rp rs r� r� r� r� ru r� r� r� r� r� rv r� r� r� r� rz r r r r r . s� A l ! 8 # $)L( +/%N)�__all__�errorr r r r r r r r r �<module> s