h;rSSKrSSKrSSKrSSKrSSKrSSKrSSKJr SSKJr SSK J r J r SSK J r \"S5r\ S;a$\R5(a\R!SS 9 "S S 5r"S S 5r\S:XGa$\"S5 \"S5 \"5r\R-5r\"SSS9 \"S\SSS35 \"S\SSS35 \"S\SSSS35 \"S\SS S!S35 \"S"\SS#SS$35 \"S%\S&S'S(S35 \"S)\S&S*S(S35 \S+(aN\"S,5 \S+R15H.urr\"S-\S.\S/S0S1\S2SS3\S4S5S6\S7S0S83 5 M0 O\"S95 \R6"S:5 GM g!\a \"S;5 gf=f)<N)datetime)Path)MACOSRANK)check_requirementsz train.log>rT) missing_okc\rSrSrSrSrSrSrSrSr Sr S r "S S 5r "S S \ R5rSrg) ConsoleLoggera Console output capture with API/file streaming and deduplication. Captures stdout/stderr output and streams it to either an API endpoint or local file, with intelligent deduplication to reduce noise from repetitive console output. Attributes: destination (str | Path): Target destination for streaming (URL or Path object). is_api (bool): Whether destination is an API endpoint (True) or local file (False). original_stdout: Reference to original sys.stdout for restoration. original_stderr: Reference to original sys.stderr for restoration. log_queue (queue.Queue): Thread-safe queue for buffering log messages. active (bool): Whether console capture is currently active. worker_thread (threading.Thread): Background thread for processing log queue. last_line (str): Last processed line for deduplication. last_time (float): Timestamp of last processed line. last_progress_line (str): Last progress bar line for progress deduplication. last_was_progress (bool): Whether the last line was a progress bar. Examples: Basic file logging: >>> logger = ConsoleLogger("training.log") >>> logger.start_capture() >>> print("This will be logged") >>> logger.stop_capture() API streaming: >>> logger = ConsoleLogger("https://api.example.com/logs") >>> logger.start_capture() >>> # All output streams to API >>> logger.stop_capture() cXl[U[5=(a URS5UlUR(d[ U5Ul[ RUl[ RUl [R"SS9Ul SUlSUlSUlSUlSUlSUlg)z Initialize with API endpoint or local file path. Args: destination (str | Path): API endpoint URL (http/https) or local file path for streaming output. )zhttp://zhttps://)maxsizeFNg) destination isinstancestr startswithis_apirsysstdoutoriginal_stdoutstderroriginal_stderrqueueQueue log_queueactive worker_thread last_line last_timelast_progress_linelast_was_progress)selfrs R/home/james-whalen/.local/lib/python3.13/site-packages/ultralytics/utils/logger.py__init__ConsoleLogger.__init__7s' c2f{7M7MNe7f {{#K0D  #zz"zzT2 !"$!&cUR(agSUlURURUR5[lURUR UR5[lURUR5n[R"S5RU5 [R"URSS9UlUR R#5 g![a NKf=f)zTStart capturing console output and redirect stdout/stderr to custom capture objects.NT ultralytics)targetdaemon)r_ConsoleCapturer _queue_logrrrr _LogHandlerlogging getLogger addHandler Exception threadingThread_stream_workerrstart)r$handlers r% start_captureConsoleLogger.start_capturePs ;;  ))$*>*>P ))$*>*>P  &&t7G   m , 7 7 @'--T5H5HQUV   "    s9AC88 DDcUR(dgSUlUR[lUR[lUR RS5 g)zAStop capturing console output and restore original stdout/stderr.NF)rrrrrrrputr$s r% stop_captureConsoleLogger.stop_capturecs?{{  )) ))  4 r(cHUR(dg[R"5nSU;aURS5SnURS5nU(aUSS:XaUR5 UGH)nUR 5nSU;aMSU;aRURS5SR 5nXPR :XaUR(aMfXPlS UlO(U(dUR(a S UlMS UlX@R:XaX R- S :aMX@lX l URS 5(d,[R"5RS 5nSUSU3nURUS35(aGM)GM, g)z?Queue console text with deduplication and timestamp processing.N r ru─u ━━rTFg?z[20z%Y-%m-%d %H:%M:%S[z] )rtimesplitpoprstripstripr"r#r r!rrnowstrftime _safe_put)r$text current_timelinesline progress_core timestamps r%r.ConsoleLogger._queue_logms_{{ yy{  4<::d#B'D 4  U2Y"_ IIKD;;=D}D $ 9 5a 8 > > @  $;$;;@V@V*7')-& 6 6-2D*).&~~%,*G#*M!N)N??5))$LLN334GH 9+Rv.>>TF"+..Gr(cURRU5 g![RaT URR 5 URRU5 g![R a gf=ff=f)z0Safely put item in queue with overflow handling.TF)r put_nowaitrFull get_nowaitEmpty)r$items r%rKConsoleLogger._safe_putso  NN % %d +zz  ))+))$/;;    s&B5A**B>BBBcUR(aCURRSS9nUcgURU5 UR(aMBgg![R a Mlf=f)z4Background worker for streaming logs to destination.)timeoutN)rrget _write_logrrW)r$log_texts r%r6ConsoleLogger._stream_workers]kk >>--a-8#) kkk ;;  sAAA.-A.cUR(a]SSKn[R"5R 5UR 5S.nUR [UR5USS9 gURRRSSS9 URRSS S 9nURU5 SSS5 g!,(df  g=f![a!n[S U3URS 9 SnAgSnAff=f) z4Write log to API endpoint or local file destination.rN)rQmessage)jsonr\T)parentsexist_okazutf-8)encodingzPlatform logging error: )file)rrequestsrrI isoformatrHpostrrparentmkdiropenwriter3printr)r$rLrjpayloadfes r%r^ConsoleLogger._write_logs M{{(0 (@(@(Btzz|\ c$"2"23'1 M  ''--dT-J%%**3*AQGGDMBAA M ,QC0t7K7K L Ms<A-C0>C.C C CCC D$DDc.\rSrSrSrSrSrSrSrSr g) ConsoleLogger._ConsoleCapturez"Lightweight stdout/stderr capture.originalcallbackcXlX lgNry)r$rzr{s r%r&&ConsoleLogger._ConsoleCapture.__init__s $M$Mr(c\URRU5 URU5 gr})rzrpr{)r$rLs r%rp#ConsoleLogger._ConsoleCapture.writes MM   % MM$ r(c8URR5 gr})rzflushr=s r%r#ConsoleLogger._ConsoleCapture.flushs MM   !r()r{rzN) __name__ __module__ __qualname____firstlineno____doc__ __slots__r&rpr__static_attributes__r(r%r-rws0,  %  "r(r-c6^\rSrSrSrSrU4SjrSrSrU=r $)ConsoleLogger._LogHandlerzLightweight logging handler.)r{c.>[TU]5 Xlgr})superr&r{)r$r{ __class__s r%r&"ConsoleLogger._LogHandler.__init__s G  $Mr(cJURURU5S-5 g)NrB)r{format)r$records r%emitConsoleLogger._LogHandler.emits MM$++f-4 5r() rrrrrrr&rr __classcell__)rs@r%r/rs*!  % 6 6r(r/) rrrr r"r!r#rrrrN)rrrrrr&r9r>r.rKr6r^r-r0Handlerr/rrr(r%r r sGB'2#&!2h   M"" 6goo 6r(r c0\rSrSrSrSrSrSrSrSr g) SystemLoggeru4 Log dynamic system metrics for training monitoring. Captures real-time system metrics including CPU, RAM, disk I/O, network I/O, and NVIDIA GPU statistics for training performance monitoring and analysis. Attributes: pynvml: NVIDIA pynvml module instance if successfully imported, None otherwise. nvidia_initialized (bool): Whether NVIDIA GPU monitoring is available and initialized. net_start: Initial network I/O counters for calculating cumulative usage. disk_start: Initial disk I/O counters for calculating cumulative usage. Examples: Basic usage: >>> logger = SystemLogger() >>> metrics = logger.get_metrics() >>> print(f"CPU: {metrics['cpu']}%, RAM: {metrics['ram']}%") >>> if metrics["gpus"]: ... gpu0 = metrics["gpus"]["0"] ... print(f"GPU0: {gpu0['usage']}% usage, {gpu0['temp']}°C") Training loop integration: >>> system_logger = SystemLogger() >>> for epoch in range(epochs): ... # Training code here ... metrics = system_logger.get_metrics() ... # Log to database/file cSSKnSUlUR5UlUR 5UlUR 5Ulg)zInitialize the system logger.rN)psutilpynvml _init_nvidianvidia_initializednet_io_counters net_startdisk_io_counters disk_start)r$rs r%r&SystemLogger.__init__s= "&"3"3"5//1 113r(c[(ae[S5 [S5UlURR 5 g![ a gf=f)z-Initialize NVIDIA GPU monitoring with pynvml.znvidia-ml-py>=12.0.0rTF)rr __import__rnvmlInitr3r=s r%rSystemLogger._init_nvidia sK u 9 5 6$X.DK KK "  sAA AAcSSKnUR5nUR5nUR5n[R "S5n[ UR5S5[ URS5[ URURR- S- S5[ URURR- S- S5[ URS- S5S.[ URURR- S- S5[ URURR- S- S5S.0S .nUR (a"US R#UR%55 U$) a Get current system metrics. Collects comprehensive system metrics including CPU usage, RAM usage, disk I/O statistics, network I/O statistics, and GPU metrics (if available). Example output: ```python metrics = { "cpu": 45.2, "ram": 78.9, "disk": {"read_mb": 156.7, "write_mb": 89.3, "used_gb": 256.8}, "network": {"recv_mb": 157.2, "sent_mb": 89.1}, "gpus": { 0: {"usage": 95.6, "memory": 85.4, "temp": 72, "power": 285}, 1: {"usage": 94.1, "memory": 82.7, "temp": 70, "power": 278}, }, } ``` - cpu (float): CPU usage percentage (0-100%) - ram (float): RAM usage percentage (0-100%) - disk (dict): - read_mb (float): Cumulative disk read in MB since initialization - write_mb (float): Cumulative disk write in MB since initialization - used_gb (float): Total disk space used in GB - network (dict): - recv_mb (float): Cumulative network received in MB since initialization - sent_mb (float): Cumulative network sent in MB since initialization - gpus (dict): GPU metrics by device index (e.g., 0, 1) containing: - usage (int): GPU utilization percentage (0-100%) - memory (float): CUDA memory usage percentage (0-100%) - temp (int): GPU temperature in degrees Celsius - power (int): GPU power consumption in watts Returns: metrics (dict): System metrics containing 'cpu', 'ram', 'disk', 'network', 'gpus' with respective usage data. rN/ii@)read_mbwrite_mbused_gb)recv_mbsent_mb)cpuramdisknetworkgpusr)rrrvirtual_memoryshutil disk_usageround cpu_percentpercent read_bytesr write_bytesused bytes_recvr bytes_sentrupdate_get_nvidia_metrics)r$rnetrmemoryrmetricss r% get_metricsSystemLogger.get_metricssML $$&&&(&&(&&s+ ++-q1+ $//DOO4N4N"NSZ![]^_!4#3#3doo6Q6Q#QV]"^`ab G!>3L3L"LQX!Y[\] #..4>>3L3L"LQX!Y[\]    " " FO " "4#;#;#= >r(c0nUR(aUR(dU$URR5n[U5HnURR U5nURR U5nURR U5nURRX@RR5nURRU5S-n[URS5[URUR- S-S5UUS.U[U5'M U$![a U$f=f)zMGet NVIDIA GPU metrics including utilization, memory, temperature, and power.rrd)usagertemppower)rrnvmlDeviceGetCountrangenvmlDeviceGetHandleByIndexnvmlDeviceGetUtilizationRatesnvmlDeviceGetMemoryInfonvmlDeviceGetTemperatureNVML_TEMPERATURE_GPUnvmlDeviceGetPowerUsagergpurtotalrr3) r$r device_countihandleutilrrrs r%r SystemLogger._get_nvidia_metricsYs&&dkkK ;;99;L<(??B{{@@H<rsB  )7 $7?'..00t,M6M6`JJZ z 23 "# ^F )((*G .b ) E'%..a0 1 E'%..a0 1 K :4@D E L!