EVOLUTION-NINJA
Edit File: fixdiv.pyc
� � fc @ s� d Z d d l Z d d l Z d d l Z d d l Z d a d � Z d � Z d Z d � Z d � Z d � Z d � Z d d d � � YZ d � Z d � Z e d k r� e j e � � n d S( s( fixdiv - tool to fix division operators. To use this tool, first run `python -Qwarnall yourscript.py 2>warnings'. This runs the script `yourscript.py' while writing warning messages about all uses of the classic division operator to the file `warnings'. The warnings look like this: <file>:<line>: DeprecationWarning: classic <type> division The warnings are written to stderr, so you must use `2>' for the I/O redirect. I know of no way to redirect stderr on Windows in a DOS box, so you will have to modify the script to set sys.stderr to some kind of log file if you want to do this on Windows. The warnings are not limited to the script; modules imported by the script may also trigger warnings. In fact a useful technique is to write a test script specifically intended to exercise all code in a particular module or set of modules. Then run `python fixdiv.py warnings'. This first reads the warnings, looking for classic division warnings, and sorts them by file name and line number. Then, for each file that received at least one warning, it parses the file and tries to match the warnings up to the division operators found in the source code. If it is successful, it writes its findings to stdout, preceded by a line of dashes and a line of the form: Index: <file> If the only findings found are suggestions to change a / operator into a // operator, the output is acceptable input for the Unix 'patch' program. Here are the possible messages on stdout (N stands for a line number): - A plain-diff-style change ('NcN', a line marked by '<', a line containing '---', and a line marked by '>'): A / operator was found that should be changed to //. This is the recommendation when only int and/or long arguments were seen. - 'True division / operator at line N' and a line marked by '=': A / operator was found that can remain unchanged. This is the recommendation when only float and/or complex arguments were seen. - 'Ambiguous / operator (..., ...) at line N', line marked by '?': A / operator was found for which int or long as well as float or complex arguments were seen. This is highly unlikely; if it occurs, you may have to restructure the code to keep the classic semantics, or maybe you don't care about the classic semantics. - 'No conclusive evidence on line N', line marked by '*': A / operator was found for which no warnings were seen. This could be code that was never executed, or code that was only executed with user-defined objects as arguments. You will have to investigate further. Note that // can be overloaded separately from /, using __floordiv__. True division can also be separately overloaded, using __truediv__. Classic division should be the same as either of those. (XXX should I add a warning for division on user-defined objects, to disambiguate this case from code that was never executed?) - 'Phantom ... warnings for line N', line marked by '*': A warning was seen for a line not containing a / operator. The most likely cause is a warning about code executed by 'exec' or eval() (see note below), or an indirect invocation of the / operator, for example via the div() function in the operator module. It could also be caused by a change to the file between the time the test script was run to collect warnings and the time fixdiv was run. - 'More than one / operator in line N'; or 'More than one / operator per statement in lines N-N': The scanner found more than one / operator on a single line, or in a statement split across multiple lines. Because the warnings framework doesn't (and can't) show the offset within the line, and the code generator doesn't always give the correct line number for operations in a multi-line statement, we can't be sure whether all operators in the statement were executed. To be on the safe side, by default a warning is issued about this case. In practice, these cases are usually safe, and the -m option suppresses these warning. - 'Can't find the / operator in line N', line marked by '*': This really shouldn't happen. It means that the tokenize module reported a '/' operator but the line it returns didn't contain a '/' character at the indicated position. - 'Bad warning for line N: XYZ', line marked by '*': This really shouldn't happen. It means that a 'classic XYZ division' warning was read with XYZ being something other than 'int', 'long', 'float', or 'complex'. Notes: - The augmented assignment operator /= is handled the same way as the / operator. - This tool never looks at the // operator; no warnings are ever generated for use of this operator. - This tool never looks at the / operator when a future division statement is in effect; no warnings are generated in this case, and because the tool only looks at files for which at least one classic division warning was seen, it will never look at files containing a future division statement. - Warnings may be issued for code not read from a file, but executed using an exec statement or the eval() function. These may have <string> in the filename position, in which case the fixdiv script will attempt and fail to open a file named '<string>' and issue a warning about this failure; or these may be reported as 'Phantom' warnings (see above). You're on your own to deal with these. You could make all recommended changes and add a future division statement to all affected files, and then re-run the test script; it should not issue any warnings. If there are any, and you have a hard time tracking down where they are generated, you can use the -Werror option to force an error instead of a first warning, generating a traceback. - The tool should be run from the same directory as that from which the original script was run, otherwise it won't be able to open files given by relative pathnames. i����Ni c C sJ y# t j t j d d � \ } } Wn! t j k rF } t | � d SXx> | D]6 \ } } | d k ro t GHd S| d k rN d a qN qN W| s� t d � d S| d r� t j j d t j d � n t | d � } | d k r� d S| j � } | sd G| d GHd S| j � d } x- | D]% } t | | | � } | p?| } q W| S( Ni t hmi s -hs -ms& at least one file argument is requireds! %s: extra file arguments ignored i s& No classic division warnings read from( t getoptt syst argvt errort usaget __doc__t multi_okt stderrt writet readwarningst Nonet keyst sortt process( t optst argst msgt ot at warningst filest exitt filenamet x( ( s, /usr/lib64/python2.7/Tools/scripts/fixdiv.pyt main� s: # c C s[ t j j d t j d | f � t j j d t j d � t j j d t j d � d S( Ns %s: %s i s Usage: %s [-m] warnings s" Try `%s -h' for more information. ( R R R R ( R ( ( s, /usr/lib64/python2.7/Tools/scripts/fixdiv.pyR � s !sL ^(.+?):(\d+): DeprecationWarning: classic (int|long|float|complex) division$c C s" t j t � } y t | � } Wn( t k rI } t j j d | � d SXi } x� | j � } | si Pn | j | � } | s� | j d � d k rS t j j d | � qS qS n | j � \ } } } | j | � } | d k r� g | | <} n | j t | � t | � f � qS W| j � | S( Ns can't open: %s t divisioni s Warning: ignored input ( t ret compilet PATTERNt opent IOErrorR R R t readlinet matcht findt groupst getR t appendt intt internt close( t warningsfilet progt fR R t linet mR t linenot whatt list( ( s, /usr/lib64/python2.7/Tools/scripts/fixdiv.pyR � s. # c C s@ d d GH| s t � y t | � } Wn( t k rO } t j j d | � d SXd G| GHt | � } | j � d } t j | j � } x�t | � \ } } } } | d k r� Pn | | k o� d k n s� t � g } xE | t | � k r"| | d | k r"| j | | � | d 7} q� W| r9t | | � n g } xE | t | � k r�| | d | k r�| j | | � | d 7} qBW| r�| r�q� | r�| r�t | d � q� | r�| r�t | | � q� t | � d k r�t s�g } d } x? | D]7 \ \ } } } | | k r!q�n | j | � | } q�W| sDt � t | � d k rfd G| d GHq�d Gd | d | d f GHq�n g } g } g } xY | D]Q \ } } | d k r�| j | � q�| d k r�| j | � q�| j | � q�Wd } x0| D](\ \ } } } | | k r&qn | } t | � } | | | d !d k rgd | GHd G| GHqn | r�d | G| GHd G| GHq| r�| r�d | | f GHd G| GHd GHd G| | d | | GHq| r�| r�d | GHd G| GHq| r| rd d j | � d j | � | f GHd G| GHqqWq� W| j � d S( Nt -iF s can't open: %s i s Index:i s No conclusive evidences$ *** More than one / operator in lines* *** More than one / operator per statements in lines %d-%di����R& t longt floatt complext /s) *** Can't find the / operator in line %d:t *s *** Bad warning for line %d:s %dc%dt <s ---t >s$ True division / operator at line %d:t =s- *** Ambiguous / operator (%s, %s) at line %d:t |t ?( R& R2 ( R3 R4 ( t AssertionErrorR R R R R t FileContextR t tokenizet generate_tokensR t scanlineR t lenR% t reportphantomwarningst reportR t chopt joinR( ( R R0 t fpR R+ t indext gt startlinenot endlinenot slashest lineinfot orphansR t rowst lastrowt rowt colR, t intlongt floatcomplext badR. R/ ( ( s, /usr/lib64/python2.7/Tools/scripts/fixdiv.pyR � s� ")) !c C s� g } d } d } xF | D]> \ } } | | k rJ | g } | j | � n | j | � q WxM | D]E } | d } d j | d � } d | | f GH| j | d d �qb Wd S( Ni R5 i s$ *** Phantom %s warnings for line %d:t markR6 ( R R% RE RC ( R R+ t blocksRO t lastblockRP R/ t blockt whats( ( s, /usr/lib64/python2.7/Tools/scripts/fixdiv.pyRB $ s c C sZ d } xM | D]E \ \ } } } | | k r d | | f GHd Gt | � GH| } q q Wd S( Ns *** %s on line %d:R6 ( R RD ( RK t messageRO RP RQ R, ( ( s, /usr/lib64/python2.7/Tools/scripts/fixdiv.pyRC 3 s R= c B sA e Z d d d � Z d � Z d � Z d � Z d d d � Z RS( i i c C s: | | _ d | _ d | _ d | _ g | _ g | _ d S( Ni i i ( RF t windowR. t eoflookaheadt lookaheadt buffer( t selfRF R[ R. ( ( s, /usr/lib64/python2.7/Tools/scripts/fixdiv.pyt __init__< s c C s_ xX t | j � | j k rZ | j rZ | j j � } | sG d | _ Pn | j j | � q Wd S( Ni ( RA R] R[ R\ RF R R% ( R_ R, ( ( s, /usr/lib64/python2.7/Tools/scripts/fixdiv.pyt fillC s % c C sL | j � | j s d S| j j d � } | j j | � | j d 7_ | S( Nt i i ( Ra R] t popR^ R% R. ( R_ R, ( ( s, /usr/lib64/python2.7/Tools/scripts/fixdiv.pyR J s c C s� | j � | j t | j � } | j t | j � } | | k oP | j k n rd | j | | S| j | k o~ | k n r� | j | | j St � d S( N( Ra R. RA R^ R] t KeyError( R_ RG t bufstartt lookend( ( s, /usr/lib64/python2.7/Tools/scripts/fixdiv.pyt __getitem__R s R6 c C sn | d k r | } n xR t | | d � D]= } y | | } Wn t k rV d } n X| Gt | � GHq) Wd S( Ni s <missing line>( R t rangeRd RD ( R_ t firstt lastRU t iR, ( ( s, /usr/lib64/python2.7/Tools/scripts/fixdiv.pyRC [ s N( t __name__t __module__R` Ra R Rg R RC ( ( ( s, /usr/lib64/python2.7/Tools/scripts/fixdiv.pyR= ; s c C s� g } d } d } xq | D]i \ } } } } } | d } | d k rM | } n | d k ro | j | | f � n | t j k r Pq q W| | | f S( Ni R5 s /=( R5 s /=( R R% R> t NEWLINE( RH RK RI RJ t typet tokent startt endR, ( ( s, /usr/lib64/python2.7/Tools/scripts/fixdiv.pyR@ e s c C s | j d � r | d S| Sd S( Ns i����( t endswith( R, ( ( s, /usr/lib64/python2.7/Tools/scripts/fixdiv.pyRD s s t __main__( ( R R R R R> R R R R R R RB RC R= R@ RD Rl R ( ( ( s, /usr/lib64/python2.7/Tools/scripts/fixdiv.pyt <module>� s" W *