EVOLUTION-NINJA
Edit File: urllib.pyc
� z fc @ s� d Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l m Z d d d d d d d d d d d d d d d d d d d d d d d d d d d d g Z d Z d! Z e j d" k r� d d# l m Z m Z n: e j d$ k r d d# l m Z m Z n d% � Z d& � Z e a e e e d' � Z e e e e d( � Z d) � Z y d d l Z Wn e Z n Xe Z d* e f d+ � � YZ i Z d f d, � � YZ d e f d- � � YZ e a! d. � Z" e a# d/ � Z$ e a% d0 � Z& e a' d1 � Z( d2 f d3 � � YZ) d4 f d5 � � YZ* d6 e* f d7 � � YZ+ d8 e* f d9 � � YZ, d: e* f d; � � YZ- y e. Wn e/ k r�d< � Z0 n Xd= � Z0 d> � Z1 d? � Z2 e a3 d@ � Z4 e a5 dA � Z6 e a7 dB � Z8 e a9 dC � Z: e a; dD � Z< e a= d dE � Z> e a? dF � Z@ e aA dG � ZB dH � ZC e aD dI � ZE dJ ZF eG dK � eF D� � ZH e jI dL � ZJ dM � ZK dN � ZL dO ZM i ZN xf eO eP dP � eQ eR eP dP � � � � D]= \ ZS ZT eS dQ k o�eT eM k r�eT n dR jU eS � eN eT <q�Wi ZV dS dT � ZW dU dV � ZX dW dX � ZY dY � ZZ e dZ � Z[ e j\ d[ k r�d d\ l] m^ Z^ m_ Z_ d] � Z` d^ � Za d_ � Zb d` � Zc nB e j d" k r�da � Zd db � Zc dc � Ze dd � Zb n eZ Zc e[ Zb de � Zf df � Zg d S(g s� Open an arbitrary URL. See the following document for more info on URLs: "Names and Addresses, URIs, URLs, URNs, URCs", at http://www.w3.org/pub/WWW/Addressing/Overview.html See also the HTTP spec (from which the error codes are derived): "HTTP - Hypertext Transfer Protocol", at http://www.w3.org/pub/WWW/Protocols/ Related standards and specs: - RFC1808: the "relative URL" spec. (authoritative status) - RFC1738 - the "URL standard". (authoritative status) - RFC1630 - the "URI spec". (informational status) The object returned by URLopener().open(file) will differ per protocol. All you know is that is has methods read(), readline(), readlines(), fileno(), close() and info(). The read*(), fileno() and close() methods work like those of open files. The info() method returns a mimetools.Message object which can be used to query various info about the object, if available. (mimetools.Message objects are queried with the getheader() method.) i����N( t urljoint urlopent URLopenert FancyURLopenert urlretrievet urlcleanupt quotet quote_plust unquotet unquote_plust urlencodet url2pathnamet pathname2urlt splittagt localhostt thishostt ftperrorst basejoint unwrapt splittypet splithostt splitusert splitpasswdt splitportt splitnportt splitqueryt splitattrt splitvaluet getproxiess 1.17i t nt( R R t riscosc C s t | � S( s OS-specific conversion from a relative URL of the 'file' scheme to a file system path; not recommended for general use.( R ( t pathname( ( s /usr/lib64/python2.7/urllib.pyR 5 s c C s t | � S( s OS-specific conversion from a file system path to a relative URL of the 'file' scheme; not recommended for general use.( R ( R ( ( s /usr/lib64/python2.7/urllib.pyR : s c C s� d d l m } | d d d �| d k s8 | d k rP t d | d | � } n t sh t � } | a n t } | d k r� | j | � S| j | | � Sd S( s= Create a file-like object for the specified URL to read from.i����( t warnpy3ksM urllib.urlopen() has been removed in Python 3.0 in favor of urllib2.urlopen()t stackleveli t proxiest contextN( t warningsR t NoneR t _urlopenert open( t urlt dataR" R# R t opener( ( s /usr/lib64/python2.7/urllib.pyR H s c C sP | d k r t d | � } n t s4 t � a } n t } | j | | | | � S( NR# ( R% R R& t retrieve( R( t filenamet reporthookR) R# R* ( ( s /usr/lib64/python2.7/urllib.pyR Z s c C s+ t r t j � n t j � t j � d S( N( R& t cleanupt _safe_quoterst cleart ftpcache( ( ( s /usr/lib64/python2.7/urllib.pyR c s t ContentTooShortErrorc B s e Z d � Z RS( c C s t j | | � | | _ d S( N( t IOErrort __init__t content( t selft messageR5 ( ( s /usr/lib64/python2.7/urllib.pyR4 s s ( t __name__t __module__R4 ( ( ( s /usr/lib64/python2.7/urllib.pyR2 r s c B s� e Z d Z d Z d e Z d d d � Z d � Z d � Z d � Z d � Z d d � Z d d � Z d d � Z d d d d � Z d d � Z d d � Z d � Z e r� d d � Z n d � Z d � Z d � Z d d � Z RS( s, Class to open URLs. This is a class rather than just a subroutine because we may need more than one set of global protocol-specific options. Note -- this is a base class for those who don't want the automatic handling of errors type 302 (relocated) and 401 (authorization needed).s Python-urllib/%sc K s� | d k r t � } n t | d � s3 t d � � | | _ | j d � | _ | j d � | _ | | _ d | j f d g | _ g | _ t j | _ d | _ t | _ d S( Nt has_keys proxies must be a mappingt key_filet cert_files User-Agentt Accepts */*( R= s */*( R% R t hasattrt AssertionErrorR" t getR; R< R# t versiont addheaderst _URLopener__tempfilest ost unlinkt _URLopener__unlinkt tempcacheR1 ( R6 R" R# t x509( ( s /usr/lib64/python2.7/urllib.pyR4 � s c C s | j � d S( N( t close( R6 ( ( s /usr/lib64/python2.7/urllib.pyt __del__� s c C s | j � d S( N( R. ( R6 ( ( s /usr/lib64/python2.7/urllib.pyRI � s c C si | j rL x6 | j D]+ } y | j | � Wq t k r= q Xq W| j 2n | j re | j j � n d S( N( RC RF t OSErrorRG R0 ( R6 t file( ( s /usr/lib64/python2.7/urllib.pyR. � s c G s | j j | � d S( sd Add a header to be used by the HTTP interface only e.g. u.addheader('Accept', 'sound/basic')N( RB t append( R6 t args( ( s /usr/lib64/python2.7/urllib.pyt addheader� s c C s� t t | � � } t | d d �} | j rn | | j k rn | j | \ } } t | d � } t | | | � St | � \ } } | s� d } n | | j k r� | j | } t | � \ } } t | � \ } } | | f } n d } d | } | | _ | j d d � } t | | � s%| d k rQ| r>| j | | | � S| j | | � Sn y9 | d k rst | | � | � St | | � | | � SWn2 t j k r�} t d | f t j � d � n Xd S( s6 Use URLopener().open(file) instead of open(file, 'r').t safes %/:=&?~#+!$,;'@()*[]|t rbRL t open_t -t _t open_local_files socket errori N( R t toBytesR RG R' t addinfourlR R" R R% t typet replaceR> t open_unknown_proxyt open_unknownt getattrt sockett errorR3 t syst exc_info( R6 t fullurlR) R, t headerst fpt urltypeR( t proxyt proxyhostt hostt selectort namet msg( ( s /usr/lib64/python2.7/urllib.pyR' � s8 c C s( t | � \ } } t d d | f � d S( s/ Overridable interface to open unknown URL type.s url errors unknown url typeN( R R3 ( R6 Ra R) RX R( ( ( s /usr/lib64/python2.7/urllib.pyR[ � s c C s, t | � \ } } t d d | | f � d S( s/ Overridable interface to open unknown URL type.s url errors invalid proxy for %sN( R R3 ( R6 Re Ra R) RX R( ( ( s /usr/lib64/python2.7/urllib.pyRZ � s c C s� t t | � � } | j r5 | | j k r5 | j | St | � \ } } | d k r� | sf | d k r� yC | j | � } | j � } | j � t t | � d � | f SWq� t k r� q� Xn | j | | � } z�| j � } | r� t | d � } n� d d l } t | � \ } } t | p#d � \ } } t | p;d � \ } } t | pSd � \ } } t j j | � d } | j | � \ } } | j j | � t j | d � } z� | | f } | j d k r�| | j | <n d } d } d } d } d | k rt | d � } n | r*| | | | � n xc | j | � } | d k rLPn | t | � 7} | j | � | d 7} | r-| | | | � q-q-WWd | j � XWd | j � X| d k r�| | k r�t d | | f | � � n | S( st retrieve(url) returns (filename, headers) for a local object or (tempfilename, headers) for a remote object.RL i t wbi����Nt i i i s content-lengths Content-Lengths1 retrieval incomplete: got only %i out of %i bytesi ( R RV RG R R% RU t infoRI R R R3 R' t tempfileR R RD t patht splitextt mkstempRC RM t fdopent intt readt lent writeR2 ( R6 R( R, R- R) RX t url1Rc t hdrsRb t tfpRn t garbageRo t suffixt fdt resultt bst sizeRt t blocknumt block( ( s /usr/lib64/python2.7/urllib.pyR+ � sj c C s= d d l } d } d } t | t � ri t | � \ } } | r` t | � \ } } t | � } n | } n� | \ } } t | � \ } } t | � \ } } | } d } | j � d k r� d } n^ t | � \ } } | r� t | � \ } } n | r d | | | f } n t | � r | } n | s0t d � n | rZt | � } t j | � j � } n d } | r�t | � } t j | � j � } n d } | j | � } | d k r�| j d | � | j d d � | j d d t | � � n | j d | � | r| j d d | � n | r2| j d d | � n | rK| j d | � n x | j D] } | j | � qUW| j | � | j � \ } } } | j � } | d k r�| r�| j � n t d � n d | k o�d k n r�t | | d | | � S| d k r | j | | | | | � S| j | | | | | | � Sd S( s Use HTTP protocol.i����Nt https %s://%s%ss http errors no host givent POSTs Content-Types! application/x-www-form-urlencodeds Content-Lengths %dt GETs Proxy-Authorizations Basic %st Authorizationt Hosts http protocol errori s got a bad status linei� i, s http:( s http errors no host given( s http protocol errori s got a bad status lineN( t httplibR% t isinstancet strR R R R t lowert proxy_bypassR3 t base64t b64encodet stript HTTPt putrequestt putheaderRu RB t endheaderst getreplyt getfileRI RW t http_error( R6 R( R) R� t user_passwdt proxy_passwdRg Rh t realhostRd t restt proxy_autht autht hRN t errcodet errmsgRb Rc ( ( s /usr/lib64/python2.7/urllib.pyt open_http) sx c C s� d | } t | | � rw t | | � } | d k rO | | | | | | � } n | | | | | | | � } | rw | Sn | j | | | | | � S( s� Handle http errors. Derived class can override this, or provide specific handlers named http_error_DDD where DDD is the 3-digit error code.s http_error_%dN( R> R\ R% t http_error_default( R6 R( Rc R� R� Rb R) Ri t methodR} ( ( s /usr/lib64/python2.7/urllib.pyR� r s c C s# | j � t d | | | f � d S( s> Default error handler: close the connection and raise IOError.s http errorN( RI R3 ( R6 R( Rc R� R� Rb ( ( s /usr/lib64/python2.7/urllib.pyR� � s c C sF d d l } d } d } t | t � ri t | � \ } } | r` t | � \ } } t | � } n | } n� | \ } } t | � \ } } t | � \ } } | } d } | j � d k r� d } nI t | � \ } } | r� t | � \ } } n | r d | | | f } n | st d � n | rEt | � } t j | � j � } n d } | rut | � } t j | � j � } n d } | j | d d | j d | j d | j �} | d k r�| j d | � | j d d � | j d d t | � � n | j d | � | r | j d d | � n | r;| j d d | � n | rT| j d | � n x | j D] } | j | � q^W| j | � | j � \ } } } | j � } | d k r�| r�| j � n t d � n d | k o�d k n rt | | d | | � S| d k r&| j | | | | | � S| j | | | | | | � Sd S( s Use HTTPS protocol.i����Nt httpss %s://%s%ss https errors no host giveni R; R< R# R� s Content-Types! application/x-www-form-urlencodeds Content-Lengths %dR� s Proxy-Authorizations Basic %sR� R� s http protocol errors got a bad status linei� i, s https:( s https errors no host given( s http protocol errori s got a bad status lineN( R� R% R� R� R R R R R� R3 R� R� R� t HTTPSR; R< R# R� R� Ru RB R� R� R� RI RW R� ( R6 R( R) R� R� R� Rg Rh R� Rd R� R� R� R� RN R� R� Rb Rc ( ( s /usr/lib64/python2.7/urllib.pyt open_https� s~ c C su t | t � s t d � n | d d k rd | d d !d k rd | d d !j � d k rd | j | � S| j | � Sd S( s/ Use local file or FTP depending on form of URL.s file errors9 proxy support for file protocol currently not implementedi s //i t /i s localhost/N( s file errors9 proxy support for file protocol currently not implemented( R� R� R3 R� t open_ftpRU ( R6 R( ( ( s /usr/lib64/python2.7/urllib.pyt open_file� s < c C s d d l } d d l } d d l } y d d l m } Wn! t k r[ d d l m } n Xt | � \ } } t | � } y t j | � } Wn. t k r� } t | j | j | j � � n X| j } | j j | j d t �} | j | � d } | j | d | p d | | f � � } | s�| } | d d k rFd | } n# | d d k rit d | � � n t t | d � | | � St | � \ } } | r�t j | � t � t � f k r�| } | d d k r�d | } n t t | d � | | � St d � d S( s Use local file.i����N( t StringIOt usegmti s6 Content-Type: %s Content-Length: %d Last-modified: %s s text/plaini R� s file://i s ./sA local file url may start with / or file:. Unknown url of type: %sRQ s local file errors not on local host( s local file errors not on local host( t mimetypest mimetoolst email.utilst cStringIOR� t ImportErrorR R RD t statRK R3 t errnot strerrorR, t st_sizet utilst formatdatet st_mtimet Truet guess_typet Messaget ValueErrorRW R' R R] t gethostbynameR R ( R6 R( R� R� t emailR� Rg RL t localnamet statst eR t modifiedt mtypeRb t urlfilet port( ( s /usr/lib64/python2.7/urllib.pyRU � sD $ ! c C s� t | t � s t d � n d d l } d d l } y d d l m } Wn! t k rj d d l m } n Xt | � \ } } | s� t d � n t | � \ } } t | � \ } } | r� t | � \ } } n d } t | � } | p� d } | p� d } t j | � } | s%d d l } | j } n t | � } t | � \ } } t | � } | j d � } | d | d } } | r�| d r�| d } n | r�| d r�d | d <n | | | d j | � f } t | j � t k r,xJ | j j � D]6 } | | k r�| j | } | j | =| j � q�q�Wn y?| | j k r`t | | | | | � | j | <n | sod } n d } xM | D]E } t | � \ } } | j � d k r|| d k r|| j � } q|q|W| j | j | | � \ } } | j d | � d } d } | r| d | 7} n | d k rA| d k rA| d | 7} n | j! | | � � } t" | | d | � SWn2 t# � k r�} t d | f t$ j% � d � n Xd S( s Use FTP protocol.s ftp errors8 proxy support for ftp protocol currently not implementedi����N( R� s no host givenRl R� i i t Dt IRX t at At it ds ftp:s Content-Type: %s s Content-Length: %d i ( s ftp errors8 proxy support for ftp protocol currently not implemented( s ftp errors no host given( R� R� R� R� R� R� (& R� R� R3 R� R� R� R� R� R R R R R% R R] R� t ftplibt FTP_PORTRs R t splitt joinRu R1 t MAXFTPCACHEt keysRI t ftpwrapperR R� t uppert retrfileR� R� RW R R_ R` ( R6 R( R� R� R� Rg Ro R� t usert passwdR� t attrst dirsRL t keyt kt vRX t attrt valueRc t retrlenR� Rb Rj ( ( s /usr/lib64/python2.7/urllib.pyR� � sx " c C s� t | t � s t d � n d d l } y d d l m } Wn! t k r^ d d l m } n Xy | j d d � \ } } Wn t k r� t d � n X| s� d } n | j d � } | d k r� d | | k r� | | d } | | } n d } g } | j d t j d t j t j � � � � | j d | � | d k rTt j | � } n t | � } | j d t | � � | j d � | j | � d j | � } | | � } | j | d � } t | | | � S( s Use "data" URL.s data errors9 proxy support for data protocol currently not implementedi����N( R� t ,i s bad data URLs text/plain;charset=US-ASCIIt ;i t =Rl s Date: %ss %a, %d %b %Y %H:%M:%S GMTs Content-type: %sR� s Content-Length: %ds ( s data errors9 proxy support for data protocol currently not implemented( s data errors bad data URL( R� R� R3 R� R� R� R� R� R� t rfindRM t timet strftimet gmtimeR� t decodestringR Ru R� R� RW ( R6 R( R) R� R� RX t semit encodingRj t fRb ( ( s /usr/lib64/python2.7/urllib.pyt open_data< s@ N( R8 R9 t __doc__R% RC t __version__RA R4 RJ RI R. RO R' R[ RZ R+ R� R� R� t _have_sslR� R� RU R� R� ( ( ( s /usr/lib64/python2.7/urllib.pyR x s* & A I I % >c B s� e Z d Z d � Z d � Z d d � Z d � Z d d � Z d d � Z d d � Z d d � Z d d � Z d d � Z d d � Z d d � Z d d � Z d d � Z d � Z RS( s? Derived class with handlers for errors we can handle (perhaps).c O s2 t j | | | � i | _ d | _ d | _ d S( Ni i ( R R4 t auth_cachet triest maxtries( R6 RN t kwargs( ( s /usr/lib64/python2.7/urllib.pyR4 m s c C s t | | d | | � S( s3 Default error handling -- don't raise an exception.s http:( RW ( R6 R( Rc R� R� Rb ( ( s /usr/lib64/python2.7/urllib.pyR� s s c C s� | j d 7_ z{ | j rg | j | j k rg t | d � rH | j } n | j } | | | d d | � S| j | | | | | | � } | SWd d | _ Xd S( s% Error 302 -- relocated (temporarily).i t http_error_500i� s) Internal Server Error: Redirect RecursionNi ( R� R� R> R� R� t redirect_internal( R6 R( Rc R� R� Rb R) t methR} ( ( s /usr/lib64/python2.7/urllib.pyt http_error_302w s c C s� d | k r | d } n d | k r2 | d } n d S| j � t | j d | | � } | j � } | j d � p� | j d � p� | j d � s� t d | | d | | � � n | j | � S( Nt locationt urit :s http://s https://s ftp://s redirect errors) - Redirection to url '%s' is not allowed( RI R RX R� t startswithR3 R' ( R6 R( Rc R� R� Rb R) t newurlt newurl_lower( ( s /usr/lib64/python2.7/urllib.pyR� � s c C s | j | | | | | | � S( s* Error 301 -- also relocated (permanently).( R� ( R6 R( Rc R� R� Rb R) ( ( s /usr/lib64/python2.7/urllib.pyt http_error_301� s c C s | j | | | | | | � S( s; Error 303 -- also relocated (essentially identical to 302).( R� ( R6 R( Rc R� R� Rb R) ( ( s /usr/lib64/python2.7/urllib.pyt http_error_303� s c C sE | d k r( | j | | | | | | � S| j | | | | | � Sd S( s1 Error 307 -- relocated, but turn POST into error.N( R% R� R� ( R6 R( Rc R� R� Rb R) ( ( s /usr/lib64/python2.7/urllib.pyt http_error_307� s c C s d | k r+ t j | | | | | | � n | d } d d l } | j d | � } | sx t j | | | | | | � n | j � \ } } | j � d k r� t j | | | | | | � n d | j d } | d k r� t | | � | | � St | | � | | | � Sd S( s_ Error 401 -- authentication required. This function supports Basic authentication only.s www-authenticatei����Ns! [ ]*([^ ]+)[ ]+realm="([^"]*)"t basict retry_t _basic_auth( R R� t ret matcht groupsR� RX R% R\ ( R6 R( Rc R� R� Rb R) t stuffR� R t schemet realmRi ( ( s /usr/lib64/python2.7/urllib.pyt http_error_401� s" c C s d | k r+ t j | | | | | | � n | d } d d l } | j d | � } | sx t j | | | | | | � n | j � \ } } | j � d k r� t j | | | | | | � n d | j d } | d k r� t | | � | | � St | | � | | | � Sd S( se Error 407 -- proxy authentication required. This function supports Basic authentication only.s proxy-authenticatei����Ns! [ ]*([^ ]+)[ ]+realm="([^"]*)"R� t retry_proxy_R� ( R R� R� R R R� RX R% R\ ( R6 R( Rc R� R� Rb R) R R� R R R Ri ( ( s /usr/lib64/python2.7/urllib.pyt http_error_407� s" c C s t | � \ } } d | | } | j d } t | � \ } } t | � \ } } | j d � d } | | } | j | | | � \ } } | p� | s� d St | d d �d t | d d �d | } d | | | j d <| d k r� | j | � S| j | | � Sd S( Ns http://R� t @i RP Rl R� ( R R" R t findt get_user_passwdR% R R' ( R6 R( R R) Rg Rh R� Re Rd Rf t proxyselectorR� R� R� ( ( s /usr/lib64/python2.7/urllib.pyt retry_proxy_http_basic_auth� s . c C s t | � \ } } d | | } | j d } t | � \ } } t | � \ } } | j d � d } | | } | j | | | � \ } } | p� | s� d St | d d �d t | d d �d | } d | | | j d <| d k r� | j | � S| j | | � Sd S( Ns https://R� R i RP Rl R� ( R R" R R R R% R R' ( R6 R( R R) Rg Rh R� Re Rd Rf R R� R� R� ( ( s /usr/lib64/python2.7/urllib.pyt retry_proxy_https_basic_auth� s . c C s� t | � \ } } | j d � d } | | } | j | | | � \ } } | pS | sZ d St | d d �d t | d d �d | } d | | } | d k r� | j | � S| j | | � Sd S( NR i RP Rl R� s http://( R R R R% R R' ( R6 R( R R) Rg Rh R� R� R� R� ( ( s /usr/lib64/python2.7/urllib.pyt retry_http_basic_auth� s . c C s� t | � \ } } | j d � d } | | } | j | | | � \ } } | pS | sZ d St | d d �d t | d d �d | } d | | } | d k r� | j | � S| j | | � Sd S( NR i RP Rl R� s https://( R R R R% R R' ( R6 R( R R) Rg Rh R� R� R� R� ( ( s /usr/lib64/python2.7/urllib.pyt retry_https_basic_auth s . i c C s� | d | j � } | | j k rD | r6 | j | =qD | j | Sn | j | | � \ } } | sh | r~ | | f | j | <n | | f S( NR ( R� R� t prompt_user_passwd( R6 Rg R t clear_cacheR� R� R� ( ( s /usr/lib64/python2.7/urllib.pyR s c C sf d d l } y@ t d | | f � } | j d | | | f � } | | f SWn t k ra Hd SXd S( s# Override this in a GUI environment!i����Ns Enter username for %s at %s: s# Enter password for %s in %s at %s: ( NN( t getpasst raw_inputt KeyboardInterruptR% ( R6 Rg R R R� R� ( ( s /usr/lib64/python2.7/urllib.pyR # s N( R8 R9 R� R4 R� R% R� R� R� R� R� R R R R R R R R ( ( ( s /usr/lib64/python2.7/urllib.pyR j s c C s" t d k r t j d � a n t S( s8 Return the IP address of the magic hostname 'localhost'.R N( t _localhostR% R] R� ( ( ( s /usr/lib64/python2.7/urllib.pyR 4 s c C sR t d k rN y t j t j � � a WqN t j k rJ t j d � a qN Xn t S( s* Return the IP address of the current host.R N( t _thishostR% R] R� t gethostnamet gaierror( ( ( s /usr/lib64/python2.7/urllib.pyR < s c C s( t d k r$ d d l } | j a n t S( s1 Return the set of errors raised by the FTP class.i����N( t _ftperrorsR% R� t all_errors( R� ( ( s /usr/lib64/python2.7/urllib.pyR G s c C sy t d k ru d d l } y d d l m } Wn! t k rO d d l m } n X| j | � d � a t j j � n t S( s) Return an empty mimetools.Message object.i����N( R� i ( t _noheadersR% R� R� R� R� R� Rc RI ( R� R� ( ( s /usr/lib64/python2.7/urllib.pyt noheadersP s R� c B sV e Z d Z e j e d � Z d � Z d � Z d � Z d � Z d � Z d � Z RS( s; Class used by open_ftp() for cache of open FTP connections.c C sq | | _ | | _ | | _ | | _ | | _ | | _ d | _ | | _ y | j � Wn | j � � n Xd S( Ni ( R� R� Rg R� R� t timeoutt refcountt keepalivet initRI ( R6 R� R� Rg R� R� R t persistent( ( s /usr/lib64/python2.7/urllib.pyR4 c s c C s� d d l } d | _ | j � | _ | j j | j | j | j � | j j | j | j � d j | j � } | j j | � d S( Ni����i R� ( R� t busyt FTPt ftpt connectRg R� R t loginR� R� R� R� t cwd( R6 R� t _target( ( s /usr/lib64/python2.7/urllib.pyR t s c C s d d l } | j � | d k r1 d } d } n d | } d } y | j j | � Wn. | j k r� | j � | j j | � n Xd } | r| ry& d | } | j j | � \ } } Wq| j k r } t | � d d k r t d | f t j � d � q qXn | s�| j j d � | r�| j j � } zM y | j j | � Wn2 | j k r�} t d | f t j � d � n XWd | j j | � Xd | } n d } | j j | � \ } } n d | _ t | j d � | j � } | j d 7_ | j � | | f S( Ni����R� R� s TYPE Ai s TYPE i s RETR i t 550s ftp errori s LIST t LISTRQ ( R� R� ( R� t endtransferR$ t voidcmdR R R% t ntransfercmdt error_permR� R3 R_ R` t pwdR' R" t addclosehookt makefilet file_closeR RI ( R6 RL RX R� t cmdt isdirt connR� t reasonR/ t ftpobj( ( s /usr/lib64/python2.7/urllib.pyR� } sL &$ c C sB | j s d Sd | _ y | j j � Wn t � k r= n Xd S( Ni ( R" R$ t voidrespR ( R6 ( ( s /usr/lib64/python2.7/urllib.pyR+ � s c C s) t | _ | j d k r% | j � n d S( Ni ( t FalseR R t real_close( R6 ( ( s /usr/lib64/python2.7/urllib.pyRI � s c C sC | j � | j d 8_ | j d k r? | j r? | j � n d S( Ni i ( R+ R R R: ( R6 ( ( s /usr/lib64/python2.7/urllib.pyR2 � s c C s6 | j � y | j j � Wn t � k r1 n Xd S( N( R+ R$ RI R ( R6 ( ( s /usr/lib64/python2.7/urllib.pyR: � s ( R8 R9 R� R] t _GLOBAL_DEFAULT_TIMEOUTR� R4 R R� R+ RI R2 R: ( ( ( s /usr/lib64/python2.7/urllib.pyR� ` s + t addbasec B s) e Z d Z d � Z d � Z d � Z RS( s( Base class for addinfo and addclosehook.c C s� | | _ | j j | _ | j j | _ t | j d � rK | j j | _ n t | j d � ro | j j | _ n d � | _ t | j d � r� | j j | _ t | j d � r� | j j | _ q� n d S( Nt readlinest filenoc S s d S( N( R% ( ( ( s /usr/lib64/python2.7/urllib.pyt <lambda>� Rl t __iter__t next( Rc Rt t readlineR> R= R> R@ RA ( R6 Rc ( ( s /usr/lib64/python2.7/urllib.pyR4 � s c C s d | j j t | � | j f S( Ns <%s at %r whose fp = %r>( t __class__R8 t idRc ( R6 ( ( s /usr/lib64/python2.7/urllib.pyt __repr__� s c C sJ d | _ d | _ d | _ d | _ | j r= | j j � n d | _ d S( N( R% Rt RB R= R> Rc RI ( R6 ( ( s /usr/lib64/python2.7/urllib.pyRI � s ( R8 R9 R� R4 RE RI ( ( ( s /usr/lib64/python2.7/urllib.pyR< � s R0 c B s e Z d Z d � Z d � Z RS( s* Class to add a close hook to an open file.c G s& t j | | � | | _ | | _ d S( N( R< R4 t closehookt hookargs( R6 Rc RF RG ( ( s /usr/lib64/python2.7/urllib.pyR4 � s c C sP z; | j } | j } | r: d | _ d | _ | | � n Wd t j | � Xd S( N( RF RG R% R< RI ( R6 RF RG ( ( s /usr/lib64/python2.7/urllib.pyRI � s ( R8 R9 R� R4 RI ( ( ( s /usr/lib64/python2.7/urllib.pyR0 � s t addinfoc B s e Z d Z d � Z d � Z RS( s. class to add an info() method to an open file.c C s t j | | � | | _ d S( N( R< R4 Rb ( R6 Rc Rb ( ( s /usr/lib64/python2.7/urllib.pyR4 � s c C s | j S( N( Rb ( R6 ( ( s /usr/lib64/python2.7/urllib.pyRm � s ( R8 R9 R� R4 Rm ( ( ( s /usr/lib64/python2.7/urllib.pyRH � s RW c B s5 e Z d Z d d � Z d � Z d � Z d � Z RS( s9 class to add info() and geturl() methods to an open file.c C s/ t j | | � | | _ | | _ | | _ d S( N( R< R4 Rb R( t code( R6 Rc Rb R( RI ( ( s /usr/lib64/python2.7/urllib.pyR4 s c C s | j S( N( Rb ( R6 ( ( s /usr/lib64/python2.7/urllib.pyRm s c C s | j S( N( RI ( R6 ( ( s /usr/lib64/python2.7/urllib.pyt getcode s c C s | j S( N( R( ( R6 ( ( s /usr/lib64/python2.7/urllib.pyt geturl s N( R8 R9 R� R% R4 Rm RJ RK ( ( ( s /usr/lib64/python2.7/urllib.pyRW � s c C s d S( Ni ( ( t x( ( s /usr/lib64/python2.7/urllib.pyt _is_unicode# s c C s t | t � S( N( R� t unicode( RL ( ( s /usr/lib64/python2.7/urllib.pyRM & s c C sT t | � rP y | j d � } WqP t k rL t d t | � d � � qP Xn | S( s toBytes(u"URL") --> 'URL'.t ASCIIs URL s contains non-ASCII characters( RM t encodet UnicodeErrort repr( R( ( ( s /usr/lib64/python2.7/urllib.pyRV ) s c C si | j � } | d d k rB | d d k rB | d d !j � } n | d d k re | d j � } n | S( s8 unwrap('<URL:type://host/path>') --> 'type://host/path'.i t <i����t >i s URL:( R� ( R( ( ( s /usr/lib64/python2.7/urllib.pyR 5 s c C sv t d k r* d d l } | j d � a n t j | � } | rl | j d � } | j � | t | � d f Sd | f S( s: splittype('type:opaquestring') --> 'type', 'opaquestring'.i����Ns ^([^/:]+):i ( t _typeprogR% R� t compileR t groupR� Ru ( R( R� R R ( ( s /usr/lib64/python2.7/urllib.pyR > s c C s� t d k r$ t j d t j � a n t j | � } | r� | j d � } | j d � } | rz | j d � rz d | } n | | f Sd | f S( s; splithost('//host[:port]/path') --> 'host[:port]', '/path'.s //([^/#?]*)(.*)i i R� N( t _hostprogR% R� RV t DOTALLR RW R� ( R( R t host_portRo ( ( s /usr/lib64/python2.7/urllib.pyR L s c C sY t d k r* d d l } | j d � a n t j | � } | rO | j d d � Sd | f S( sJ splituser('user[:passwd]@host[:port]') --> 'user[:passwd]', 'host[:port]'.i����Ns ^(.*)@(.*)$i i ( t _userprogR% R� RV R RW ( Rg R� R ( ( s /usr/lib64/python2.7/urllib.pyR \ s c C s_ t d k r0 d d l } | j d | j � a n t j | � } | rU | j d d � S| d f S( s/ splitpasswd('user:passwd') -> 'user', 'passwd'.i����Ns ^([^:]*):(.*)$i i ( t _passwdprogR% R� RV t SR RW ( R� R� R ( ( s /usr/lib64/python2.7/urllib.pyR h s c C sn t d k r* d d l } | j d � a n t j | � } | rd | j � \ } } | rd | | f Sn | d f S( s* splitport('host:port') --> 'host', 'port'.i����Ns ^(.*):([0-9]*)$( t _portprogR% R� RV R R ( Rg R� R R� ( ( s /usr/lib64/python2.7/urllib.pyR u s c C s� t d k r* d d l } | j d � a n t j | � } | r� | j d d � \ } } | r� y t | � } Wn t k r� d } n X| | f Sn | | f S( s� Split host and port, returning numeric port. Return given default port if no ':' found; defaults to -1. Return numerical port if a valid number are found after ':'. Return None if ':' but not a valid number.i����Ns ^(.*):(.*)$i i ( t _nportprogR% R� RV R RW Rs R� ( Rg t defportR� R R� t nport( ( s /usr/lib64/python2.7/urllib.pyR � s c C sY t d k r* d d l } | j d � a n t j | � } | rO | j d d � S| d f S( s/ splitquery('/path?query') --> '/path', 'query'.i����Ns ^(.*)\?([^?]*)$i i ( t _queryprogR% R� RV R RW ( R( R� R ( ( s /usr/lib64/python2.7/urllib.pyR � s c C sY t d k r* d d l } | j d � a n t j | � } | rO | j d d � S| d f S( s) splittag('/path#tag') --> '/path', 'tag'.i����Ns ^(.*)#([^#]*)$i i ( t _tagprogR% R� RV R RW ( R( R� R ( ( s /usr/lib64/python2.7/urllib.pyR � s c C s! | j d � } | d | d f S( sk splitattr('/path;attr1=value1;attr2=value2;...') -> '/path', ['attr1=value1', 'attr2=value2', ...].R� i i ( R� ( R( t words( ( s /usr/lib64/python2.7/urllib.pyR � s c C sY t d k r* d d l } | j d � a n t j | � } | rO | j d d � S| d f S( s- splitvalue('attr=value') --> 'attr', 'value'.i����Ns ^([^=]*)=(.*)$i i ( t _valueprogR% R� RV R RW ( R� R� R ( ( s /usr/lib64/python2.7/urllib.pyR � s t 0123456789ABCDEFabcdefc c s? | ]5 } t D]( } | | t t | | d � � f Vq q d S( i N( t _hexdigt chrRs ( t .0R� t b( ( s /usr/lib64/python2.7/urllib.pys <genexpr>� s s ([ -]+)c C sO t | � r� d | k r | St j | � } | d g } | j } xU t d t | � d � D]; } | t t | | � � j d � � | | | d � qZ Wd j | � S| j d � } t | � d k r� | S| d g } | j } x^ | d D]R } y$ | t | d � | | d � Wq� t k r=| d � | | � q� Xq� Wd j | � S( s" unquote('abc%20def') -> 'abc def'.t %i i i t latin1Rl ( RM t _asciireR� RM t rangeRu R R� t decodeR� t _hextochrt KeyError( t st bitst resRM R� t item( ( s /usr/lib64/python2.7/urllib.pyR � s. # c C s | j d d � } t | � S( s% unquote('%7e/abc+def') -> '~/abc def't +t ( RY R ( Rr ( ( s /usr/lib64/python2.7/urllib.pyR � s sA ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789_.-i i� s %{:02X}R� c C s� | s% | d k r! t d � � n | S| t f } y t | \ } } Wnf t k r� t j � } | j g | D] } | | f ^ qn � | j } t | } | | f t | <n X| j | � s� | Sd j t | | � � S( s quote('abc def') -> 'abc%20def' Each part of a URL, e.g. the path info, the query, etc., has a different set of reserved characters that must be quoted. RFC 2396 Uniform Resource Identifiers (URI): Generic Syntax lists the following reserved characters. reserved = ";" | "/" | "?" | ":" | "@" | "&" | "=" | "+" | "$" | "," Each of these characters is reserved in some component of a URL, but not necessarily in all of them. By default, the quote function is intended for quoting the path section of a URL. Thus, it will not encode '/'. This character is reserved, but in typical usage the quote function is being called on a path where the existing slash characters are used as reserved characters. s None object cannot be quotedRl N( R% t TypeErrort always_safeR/ Rq t _safe_mapt copyt updatet __getitem__t rstripR� t map( Rr RP t cachekeyt quotert safe_mapt c( ( s /usr/lib64/python2.7/urllib.pyR � s & Rl c C s< d | k r/ t | | d � } | j d d � St | | � S( s9 Quote the query fragment of a URL; replacing ' ' with '+'Rw Rv ( R RY ( Rr RP ( ( s /usr/lib64/python2.7/urllib.pyR s i c C s� t | d � r | j � } nb y- t | � rJ t | d t � rJ t � n Wn2 t k r t j � \ } } } t d | � n Xg } | s� xc| D]E \ } } t t | � � } t t | � � } | j | d | � q� Wnx| D]\ } } t t | � � } t | t � r7t | � } | j | d | � q� t | � rst | j d d � � } | j | d | � q� y t | � Wn8 t k r�t t | � � } | j | d | � q� Xx/ | D]' } | j | d t t | � � � q�Wq� Wd j | � S( sv Encode a sequence of two-element tuples or dictionary into a URL query string. If any values in the query arg are sequences and doseq is true, each sequence element is converted to a separate parameter. If the query arg is a sequence of two-element tuples, the order of the parameters in the output will match the order of parameters in the input. t itemsi s1 not a valid non-string sequence or mapping objectR� RO RY t &( R> R� Ru R� t tupleRx R_ R` R R� RM RM RP R� ( t queryt doseqt tyt vat tbt lR� R� t elt( ( s /usr/lib64/python2.7/urllib.pyR # s< )c C s� i } xP t j j � D]? \ } } | j � } | r | d d k r | | | d <q q Wd t j k r{ | j d d � n xg t j j � D]V \ } } | d d k r� | j � } | r� | | | d <q� | j | d d � q� q� W| S( s� Return a dictionary of scheme -> proxy server URL mappings. Scan the environment for variables named <scheme>_proxy; this seems to be the standard convention. In order to prefer lowercase variables, we process the environment in two passes, first matches any and second matches only lower case proxies. If you need a different way, you can pass a proxies dictionary to the [Fancy]URLopener constructor. i����t _proxyt REQUEST_METHODR� N( RD t environR� R� t popR% ( R" Ri R� ( ( s /usr/lib64/python2.7/urllib.pyt getproxies_environmentc s c C s� | d k r t � } n y | d } Wn t k r: d SX| d k rK d St | � \ } } g | j d � D] } | j � ^ qm } xs | D]k } | r� | j d � } t j | � } d | } t j | | t j � s� t j | | t j � r� d Sq� q� Wd S( s� Test if proxies should not be used for a particular host. Checks the proxies dict for the value of no_proxy, which should be a list of comma separated DNS suffixes, or '*' for all hosts. t noi t *i R� t .s (.+\.)?%s$N( R% R� Rq R R� R� t lstripR� t escapeR R� ( Rg R" t no_proxyt hostonlyR� Re t no_proxy_listRi t pattern( ( s /usr/lib64/python2.7/urllib.pyt proxy_bypass_environment� s& ( t darwin( t _get_proxy_settingst _get_proxiesc C s� d d l } d d l } d d l m } t | � \ } } d � } t � } d | k ri | d ri t Sn d } x'| j d d � D]} | s� q� n | j d | � } | d k r�| d k r� y | j | � } | | � } Wq� | j k r� q� q� Xn | | j d � � } | j d � } | d k rMd | j d � j d � d } n t | d � } d | } | | ?| | ?k r�t Sq� | | | � r� t Sq� Wt S( s� Return True iff this host shouldn't be accessed using a proxy This function uses the MacOSX framework SystemConfiguration to fetch the proxy information. i����N( t fnmatchc S sy | j d � } t t | � } t | � d k rM | d d d d g d } n | d d >| d d >B| d d >B| d BS( NR� i i i i i i i i ( R� R Rs Ru ( t ipAddrt parts( ( s /usr/lib64/python2.7/urllib.pyt ip2num� s R� t exclude_simplet exceptionss (\d+(?:\.\d+)*)(/\d+)?i i i i ( ( R� R] R� R R� R� R% R@ R R� R^ RW t countRs R9 ( Rg R� R] R� R� R� R� t proxy_settingst hostIPR� t mt baset mask( ( s /usr/lib64/python2.7/urllib.pyt proxy_bypass_macosx_sysconf� s@ # c C s t � S( s� Return a dictionary of scheme -> proxy server URL mappings. This function uses the MacOSX framework SystemConfiguration to fetch the proxy information. ( R� ( ( ( s /usr/lib64/python2.7/urllib.pyt getproxies_macosx_sysconf� s c C s* t � } | r t | | � St | � Sd S( s� Return True, if a host should be bypassed. Checks proxy settings gathered from the environment, if specified, or from the MacOSX framework SystemConfiguration. N( R� R� R� ( Rg R" ( ( s /usr/lib64/python2.7/urllib.pyR� � s c C s t � p t � S( N( R� R� ( ( ( s /usr/lib64/python2.7/urllib.pyR � s c C sm i } y d d l } Wn t k r* | SXy!| j | j d � } | j | d � d } | rAt | j | d � d � } d | k r� x� | j d � D]Y } | j d d � \ } } d d l } | j d | � s� d | | f } n | | | <q� WqA| d d k r| | d <qAd | | d <d | | d <d | | d <n | j � Wn t t t f k rhn X| S( sx Return a dictionary of scheme -> proxy server URL mappings. Win32 uses the registry to store proxies. i����Ns; Software\Microsoft\Windows\CurrentVersion\Internet Settingst ProxyEnablei t ProxyServerR� R� i s ^([^/:]+)://s %s://%si s http:R� s http://%ss https://%sR� s ftp://%sR$ ( t _winregR� t OpenKeyt HKEY_CURRENT_USERt QueryValueExR� R� R� R t Closet WindowsErrorR� Rx ( R" R� t internetSettingst proxyEnablet proxyServert pt protocolt addressR� ( ( s /usr/lib64/python2.7/urllib.pyt getproxies_registry� s: c C s t � p t � S( s� Return a dictionary of scheme -> proxy server URL mappings. Returns settings gathered from the environment, if specified, or the registry. ( R� R� ( ( ( s /usr/lib64/python2.7/urllib.pyR + s c C s� y d d l } d d l } Wn t k r0 d SXyK | j | j d � } | j | d � d } t | j | d � d � } Wn t k r� d SX| s� | r� d St | � \ } } | g } y/ t j | � } | | k r� | j | � n Wn t j k rn Xy/ t j | � } | | k r2| j | � n Wn t j k rIn X| j d � } x� | D]� } | d k r�d | k r�d Sn | j d d � } | j d d � } | j d d � } x* | D]" } | j | | | j � r�d Sq�Wq`Wd S( Ni����i s; Software\Microsoft\Windows\CurrentVersion\Internet SettingsR� t ProxyOverrideR� s <local>R� i s \.R� s .*t ?( R� R� R� R� R� R� R� R� R R] R� RM R^ t getfqdnR� RY R R� ( Rg R� R� R� R� t proxyOverridet rawHostR� t addrt fqdnt testt val( ( s /usr/lib64/python2.7/urllib.pyt proxy_bypass_registry4 sT c C s* t � } | r t | | � St | � Sd S( s� Return True, if the host should be bypassed. Checks proxy settings gathered from the environment, if specified, or the registry. N( R� R� R� ( Rg R" ( ( s /usr/lib64/python2.7/urllib.pyR� h s c C s� d } x$ t d � D] } | t | � } q W| d } t j � } t | � } t | � } t j � } | | k r{ d GHn t | � GHt | � GHt | � GHt | | d � Gd GHd S( NRl i i s Wrong!i t sec( Rn Rh R� R R RR t round( Rr R� t t0t qst uqst t1( ( s /usr/lib64/python2.7/urllib.pyt test1z s c C s d | | | f GHd S( Ns0 Block number: %d, Block size: %d, Total size: %d( ( R� t blocksizet totalsize( ( s /usr/lib64/python2.7/urllib.pyR- � s (h R� t stringR] RD R� R_ R� R� t urlparseR R t __all__R� R� Ri t nturl2pathR R t rourl2pathR% R& R R R t sslR9 R� R� R3 R2 R1 R R R R R R R R R R R� R< R0 RH RW RN t NameErrorRM RV R RU R RX R R[ R R\ R R^ R R_ R Rb R Rc R R Re R Rg t dictRp RV Rm R R Ry Rz t zipt xrangeR� t bytearrayR� R� t formatR/ R R R R� R� t platformt _scproxyR� R� R� R� R� R R� R� R� R- ( ( ( s /usr/lib64/python2.7/urllib.pyt <module> s� � �� c " 45 ' @ $ ; . 4