jji\ddlmZmZmZdZGddZGddeZGddeZy ) )ListTupleUnion)S)Srrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr)Srrrrrrrr rrrrrr r rr rr r rrr r r rrrrr r rrr rr rr r r rr r rrrr rrr r rr r rrrrrr rr r rrrrrr r r r r rrrrrr r)Srrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr)Srrrr rr rrrrr rrr r r r rr r rr rrr rr r rr r rrr r r r r r r r r r r rr r r rr rrr rrrrrrrrr r rr r rrrrrr rrr rr rrrrrrSrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr)Srr rrrr rr rrr rr r r r rr r rrr r rrr rr rrrrr r rrr rr r rr r rrr rr rr r rr r rrrrr r rr r r r r rrrr rrrr rr r rrrrr r)Srrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr)Srrrr rr rrrr rrr rrrrrr rr r r r r rr rr r r r r rrr rr rr r r rrr rrr rrrrrrrrrrrrrr rrrrrr rrrrrr r rr rrrrrr )Srrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr)Srr rr rrrrrrrrrrrrrrrr r r rr r r r rrrr r r rrrrrrrr r rrrrr rrr r rr rrrrrr r rrrrr rrrrr rrrr r rrrr rrrr)Srrrrrrrrrr r rrr rr r rr r rr r rrrrr rrrr r rr rr rrr rr r rrr rrrr rrr r rrr rr rrrr r rr rr rrrrr r rr rrrrrr r)Srr rr rrrr rrrr rr r r rrrrrr rr rr r rrrrr rrr rrrr rrr r rrr rr rrrr r r rr rrrr r rr r r rrrrr rrr rrrr rrrrrr )Srr rrrr rrrr rrr rrrr r rr rr rr rrr rr rr rrrrrr r rrr r r rr rrr rrr rrrrr rrr rrr r r r rr r r r rrrrr rrrr r rrr )Srrrr rrrrrrr rrrrrrrrrr r r rrr rrrrrrrrr r rrr r rrr r rr r rrrrrrrrrrrrrrrrrrr rrrrr r r rrr rrr rrr r )Srrrrrr rr rrrrr r r r r r rr rrrr rr r rrr rr rrrr r r rr r r rrr r r r rr r rrr rr rrr r rrr r r rrrrr rrr rrr r rrrrr r)Srrrr rr rrrrrrrrrrrrr rr rr rrr rrr rr r r rr r r rrr rr rrrr rrrr rrrrrrr rrrrr r rrr rrrrrr rr rr r rr rrrr )Srr rr rrrrrr r rr rr rrrr rrr rr r r rrr rr rr rrr rr r rrrrr r rrrrrrrr rrrrrrrrrrr r r rr rrrrrrr r r rrrrrr)Srrrrrr rr rrr r r rr rrrr rr rr rr rrrrrrrrrrr rrr rrrrr rr rr rrrrrrrrrrrrrrrrrrrr rrrr rrrr rrrrrr r )Srrrrrr rrrrrrrr r rr r r r r r rr rrr rrr r r rrrr rrr r r rr rr r rrrr r rrr rrrrrr rrr r r r rr rrrr rrrr r rr rrrr )Srrrr rr rrrrr rr rrrrrr rr rr rr rrr r rr rr rrr rrrrrrrrrr r rrrrrrrrrrrrrrr rrr r rrrrrrr r rrrrr rrrr )Srrrr rrrrrrrrrr rr r r rrrr rr rrr rr r r r rrrr rrr rrrrr r rrr rrr rrrrrrrr r rrr r r rrrrrrr r r rr rrr rrrr )Srr rrrrrrrrr rrrr rr rrrrrr rrrrrrrrrrrr rrrrrrr rrrrr rrrrrrr rrrrrrrrrrr rrr rrrrrrrr rrrrrrr)Srrrr rr rr rrr rrr r r r rrr rrr r r r rrrrr rr rrrr r rrr r rr r rrrrr r rrr rr rrr r rr rrrrrrr r r rrr r r rrr rrrr)Srrrr rrrrrrr r r rrrr rr rr rrrrrrrrrrr rrrr rrrrrrr rrr r rrrrrrr rrrrrrrrr rrr r rr r r rr r rr r rrrrr r )Srrrrrr rrrrr r r rrrrrrr r rrr rr rr r rrr rr rr rr rrrrrrrr r r rrrrrrrrr rr r rrr r r rrr rrrrrr r r rrr rrr r )Srr rrrr rrrr rr r rr r r rr rr rrr r rrrr rr r rrrrrrr rr r r rr r rr rrrrrrrrr rrrrr r rr r rr rrrrr r rrrrrrrrr )Srrrrrrrr rrr r r rr rrrr rr r rr rrrrrrrr rrrr rr r r rrrrr r r rrrr rrrrrrrrrrrrr r r rr rrrrrrr r rrr r rr r )Srrrr rrrr rrr rrrrrrrr rrr r rrr rrr rrr rrr rrr r rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr rrrrrrrrrr)Srr rr rr rrrrrr rr r rrrr rrrr rrrrrr r rrr rrr rrr r rr r rrr r rrr r rrrrrrrr rrrrr r r rrrrrr rrrr rrr rrr r )Srr rr rr rrrr r r r rr rrrrrrrrrrrrrrrrrrrr rrrr r rrr rrr r rrrrrrrrrrrrrrrr rrr rr rrrr rrrrr rrrrr r )Srrrr rr rr rrr r rr r r rr r r r rr r rrrr rr rr r r rrr rrr rrrr rr r rrr rrr rrr r rrr rrr r r r rr rr rr r r r rrrrrrrr)Srrrrrr rr rr r rrrr rr rr rr rrrr rr r rr r r r rr rrr r r r rrrrrr rrr rrrr rrr rr rrrr rr rrr rrrrrrr rr rrrrr r)Srr rr rr rrrr rrr r r rrrrr r rr r r rrrr rrr rr rrrrrrr rrr rrrr rr r rr r rrrrrrrrr rr r r rr r r rrr r rrrr rrrr)Srrrrrrrrrrrrr rrrrrrrrrrrrrrrrrr rrrrrrr rrrrrrrrrrrrrrrrrrrrrrrrrrrrr rrrrrrrrrrrrrrrr)Srrrrrrrrrrrrrrrrr rrrr rrrr rr rrrr rrrrr rr rr rrrrrrrrrrrr rrr rrr rrrrrrrrrrrrrrrrrrrrrrrrr)Srrrr rr rrrr r rrr rr r r rr r rrr r r r r r rrrr rr rrr rrr rrrr r rr r r rrrr r rr rr r r r r r rr rr rr rr rr rrr rr rrrr )Srrrrrrrrrrrrrrrrrrr rrrrrrrrrrrr rrrrr rrrrrrrrrrrrrrrrrrrrr rrrrr rrrrrrrrrrr rr r rrrrrrr)Srr rr rr rr rr rrr rr r r rrr rr r r rr r rrr rr r rr r rrrr rrrr rr r r rr rrr r rr rrr r rrr r r r rrrr rrrr r r r rr rrrr )Srr rr rr rrrrrr r rrr r r rrrr r r r r r rrrrr r rr r rrrrrr rr r r r r rr rrr rrrrrr rrrrr rr rrrr rrr rr rrrr r rr r )Srrrr rr rrrrr r r r rr r rr rrr r r r r rrrr rrr rrrr rr rrr r rr r r rrr r rr r rr rrrr rrrr rr rrrrrrr rrr r rrrrr r )Srrrrrr rrrr r rrr rrr rr rr rr rr rrrr rr r rrrr rrrrr rrrr r rrrr rrrr rr rrr rrr rrr r rr rrrr rr r rrrrrrrr)Srrrr rrrr rrr r rr r r r r r r r rr r rr r rr rr r rrr rr rr rr r rrrr rr rr r rr rrrr rrrrr rr r r rr rrrrr r r rr rrrrr r )Srr rrrrrrrr rr rr rr r r rr rrr r r r rrrr rrrrr r rrrrrr r rr rr r r r r r r r rrrrr r rrr r r r r rrr r r r rrrrrrr rr r )Srrrr rr rrrrr r r rr r r rr rrrr rr rrrrrrr rrr rrrrrrr r rrr r r rrrrrrrrr rr rrr rrrr rrrrrr r rr r rrrrrr r )Srrrr rrrrrrrr rrr rrrrrr rrr rr rrrrrr rr r r rrrr r r r rrr rrrrrrrrrrrrrrrr rr rrrrrrrrr r r rrrrrrrr )Srr rrrr rrrrr r r r rr r rr rrrrrrr rrrr r r rr r rr r r r rr r rr rr r rrr rr r rr r rrr rrr r r r rrrr rr r r rrr rrrrr r )Srr rr rrrrrrrr rr r r r r r rrrrrrr r r r rrrrr rrr r rrr r r r rr rr r rr r r r rrrrrr r r rrr rrrrr rrrrrrrrrrrrr)Srrrrrr rrrr r rr rr rrrrrr r rrrr r rrrr r r rr r rrr r r r r r r r r rrrrrrrrrrrrrrr rrrr rr rrrrrr r rrrr rrrr)Srrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr rr rr rrrrrrr rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr rrrrrrrrrr)Srrrr rrrrrr r r r rr rr r r r r r rrrrrrrrrr rr r r rrr r rr r r r r r r rrr rrrrrrrrrrrrrrr rr r rr r rrr rrrrrrrr )Srr rr rrrr rrr r r rr rrrr r rr r rr rr rr rrr rrr r rrr r rr rr rr r rrrrrrrrrrrrrrrrrrr rr rrrr r r r r rrrrrr r )Srrrrrrrrrrr rrrr rrrrrrrr rrrrrrrr rr rr rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr rrrrr rrrrrrrrr )Srrrr rr rr rrr r r rr rr rr rr r r rr rrr rrrr r rr r rrrrrr r rrrrr rrr rr rrrr rrrrr r rrr rr rrrr rrrr rrr rrr r )Srrrrrrrrrrr rrrr rrrr rr rr rrrrrrrr r r rr r rrrr rr r rrr rrrrrrrrrrr rrrrrrrrr rr rrrr rrrr rrrrrrr)Srrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr rrrrrrrrr )Srr rr rrrr rr r r rr rrrrr rr r r rr r rrr rr rrrrr rrr rr r rr rr rrrr rrrr rr rrrrrr rr rr rr rrrr r rrrrrrrrrr)Srrrr rrrrrrrrr rrrrrrrr rrrrrrrrrrrrrr r rr rr rr r rrr rr rrrrrrrrrrrrrrrrrr rrrrrr r r rr r rrrrrr )Srrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr )Srrrr rrrr rrrr rrr rrrr rrrr rrrrrr rrrrrr rrr rr r rrrrr rrrr rrrrrrrrr rrrrrr rrrrrrrr r rrr rrrrrr)Srrrrrr rr rrrrrrrrr rr rr rr rr rrrr rr rr rr r rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr r r r rrrrrr)Srrrr rr rrrrr rrrr rrrrrrrrrrrrrrrr rrrrr rrrrr rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr rrrrrrrrr )Srrrr rr rrrr rr r r rr r r rr rrr rr r r rr rr r rrr rr rr rr rrr r rr r rr rrrr rr rrr r rr r rr r rr rr rr rr rr r rrr rr r)Srr rr rr rr rr r rr rr rr rrr r r r r r r r rr rrr r rr rr rrrr rrr rrrrrrrrrrrrrrr rrr r rrr rr rrrrr rrr r rr rrrr)Srr rr rrrrrrrr r rr rr rrrr rrrr rr rrrr r r rr r r rr rrr r rr rr r rrrrrrrrrrr rrr rrrr rrrrrrr r rr r rrrrr r )Srr rrrrrr rr rr r rr r r rr rr r rr r r rrr rr r r rr r rr rr r rr r rr r rrrrrrrrrrrrrrrrrrr rr rrrrrr r r rrrrrr r)Srr rrrr rrrrrr r rr r r rrr rrr r rr rrrrrr rrr rrrrrrr rrr r rrrrrrr rrr rrr rrrr r r rrrrr rrrrr r r rrrrr r)Srrrrrr rrrr r r rrr r rrr r rrr rrrrrr rrrrrr rrrrr r rr rr r rrrrrr rrrrr rrrrr rrr r rr rrrrr r rr r rrrrrr)Srrrrrrrrrr rrr rrrr rr r r rrr rrr rrrr r rrr rrrr rr rr rr r rrrrrrr r rrrrr rrrrr rr rrrrrrrrrrrrr rrr r )Srrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr)Srrrr rrrrrrr r r rrrrrrrrrrrr rrrrrr rrrr r rrrrrrrrrrrrrrrrr rrrrrrrrrrrr rrrrrrrr r rr rrrrr rr)Srrrr rr rrrrrrrrr rrrr rrrrrrrrrrrrrrrrrrrrr rrrrrrrrrrrrrrrrrrrr rrrrrrrrrrrrrrrrr rrrrrr )Srrrr rr rrrrrr r rr rr rr r r rrrrrr rrrrrrrr r rrrrrr rrr r r rrrrrrrrrrrr rrrr rrr rrrrrrr r rrrrr rrrr )Srrrr rrrrrrr rr r rr r r rr rrr r rr rrrrr rr rrrrr r rrr r rrr r r rr r rrr rr r rr r rrr rrrrr rrrr r rrr rrrr rrr )Srrrrrr rr rr rrrr r rrr rr rr r r rr rrrrrrr rr rrrr r r rr rrr rr rr rrr rrr rrr r r r r r r r rrrr r rr r r rrrr rrrr )Srrrr rrrr rr r r r rrr rrr rrrrrrr r r rr r r rr r rrrr r r r r rr r r rrr rrr r rrr rrrrrr rr r rrrr rr rr rr rrrrrrr)Srr rrrrrrrrr r r r r r r rrrr rrrr rrrr rr rr r r r rr r rr r r rr r r r rr rrrrrr rrrrrr rr rrrr rr rrr rr rr rrrrr r)Srr rrrr rr rr rrr rr rr r r r r rrr r rrrr r r r r rr r r rrrr rrrrrrrrr rrrrrrrrrr rr r rr rrr rrrr rr rrrrrrrrrr )Srr rr rrrr rrr rr r r rrrrrr rrrrrr rr rrr rrrr r r rr rr r rr rrr rrr rrrrrrrrrrrrrr rrrrrrr rrrr rrrrrrr )Srrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr rrrr rrrrrrrrrrrrrrrrrrrr rrrrrrrrrrrrrrrrrrrrr)Srrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr rrrrrrrrrrrrrrrrrrr rrrrrrrrrrrrr rrrrrrrrrrrrrrrrrrrr)Srrrrrrrr rrrr rrrr rrrrrr r r r r r rrrrr rr rrrrr rrrr r r rrrr rrr rr rrrrr r r rrr r r rrrr rrr r r rrrr rrr r )Srrrr rr rr rr r r r r r r rr rr r r rrrr r r r rr r r r rr rrr r r r rr rr r r rrr r r r rr r rrr rrr r r rrrrrrrr rrrrrr rrrrceZdZdZdZdZdZdZddZdd Z d e e e fd e ddfd Zdefd ZdefdZde e e fdee e ffdZy)JapaneseContextAnalysisdirreturnNcjd|_g|_d|_d|_d|_|j yNrrF) _total_rel _rel_sample_need_to_skip_char_num_last_char_order_doneresetselfs \/opt/eval-harness/bench_task/venv/lib/python3.12/site-packages/pip/_vendor/chardet/jpcntx.py__init__z JapaneseContextAnalysis.__init__s1&(&'# "  cfd|_dg|jz|_d|_d|_d|_yr)rNUM_OF_CATEGORYrrrrrs rrzJapaneseContextAnalysis.resets93!5!55'(# " rbyte_str num_bytesc|jry|j}||kr|j|||dz\}}||z }||kDr||z |_d|_n||dk7rp|jdk7ra|xjdz c_|j|j kDrd|_y|j t|j|xxdz cc<||_||kryy)NrrrT)rr get_orderrrMAX_REL_THRESHOLDrjp2_char_context)rr!r"iorderchar_lens rfeedzJapaneseContextAnalysis.feeds ::   ' ')m"nnXa!a%-@AOE8 MA9}./)m+(*%RKd&;&;r&AOOq(O)?)??%) $$()>)>?F).%)mrc4|j|jkDSN)rENOUGH_REL_THRESHOLDrs rgot_enough_dataz'JapaneseContextAnalysis.got_enough_datas!:!:::rc|j|jkDr)|j|jdz |jz S|jS)Nr)rMINIMUM_DATA_THRESHOLDr DONT_KNOWrs rget_confidencez&JapaneseContextAnalysis.get_confidencesA ??T88 8OOd&6&6q&99T__L L~~r_cy)Nrr)rr3s rr$z!JapaneseContextAnalysis.get_ordersrrN)__name__ __module__ __qualname__r r1r-r%r0rrrbytes bytearrayintr*boolr.floatr2rr$r6rrr r xsOI .U5)#34...:;; 5 !12uS#Xrr cZeZdZdfd ZedefdZdeee fde e e ffdZ xZ S)SJISContextAnalysisrc0t|d|_y)N SHIFT_JIS)superr _charset_name)r __class__s rrzSJISContextAnalysis.__init__s (rc|jSr,)rErs r charset_namez SJISContextAnalysis.charset_names!!!rr!c|sy|d}d|cxkrdksnd|cxkrdkr nnd}|dk(sd |cxkrdkr nn d |_nd }t|d kDr"|d }|d k(rd|cxkrd kr nd|fS|dz |fSd|fS)Nr5rrCP932rr)rElenrr! first_charr) second_chars rr$zSJISContextAnalysis.get_ordersa[ J &$ &DJ,F$,FHd" (Bd(B%,"H x=1 "1+Kc! (Ct(C8|#T)8338|rr7)r8r9r:rpropertystrrHrr;r<rr=r$ __classcell__)rFs@rrArAsH)"c""%y(8"9eCHorrAc0eZdZdeeefdeeeffdZy)EUCJPContextAnalysisr!rc|sy|d}|dk(sd|cxkrdkrnnd}n |dk(rd}nd }t|d kDr"|d }|d k(rd|cxkrd kr nd |fS|dz |fSd |fS) Nr5rrr rr)rSrTs rr$zEUCJPContextAnalysis.get_ordersa[ $ DJ$>$$>H 4 HH x=1 "1+Kd")D)D8|#T)8338|rN) r8r9r:rr;r<rr=r$r6rrr[r[s'%y(8"9eCHorr[N)typingrrrr&r rAr[r6rrrds>8&%TpBBJ1<2r