python /home/admin/mtr/script_for_cron.py -j coverage -m 9 -a '' -s coverage -M 0 -S 0 -U 100,100,120
 import MySQLdb succeeded 
root_folder /data_4/data_log/job/2025/December/13122025/coverage/
git_velours : /home/admin/workarea/git/Velours/
out_folder_name htmlcov
output_folder /data_4/data_log/job/2025/December/13122025/coverage/htmlcov
new path :  /data_4/data_log/job/2025/December/13122025/coverage/
 command : coverage3 run  /home/admin/workarea/git/Velours/python/tests/python_tests.py --short_python3 `cat ~/.fotonower_pass/bdd.py.pass` 
cat: /home/admin/.fotonower_pass/bdd.py.pass: Aucun fichier ou dossier de ce type
 import MySQLdb succeeded 
Import error (python version)
python version = 3
warning , we can't find thcl infos in json_data
warning , we can't find pdt infos in json_data
python version used : 3


#&_# BEGIN OF TEST : tests/mask_test #&_# 


/home/admin/workarea/git/Velours/python/tests/mask_test.py
<module 'tests.mask_test' from '/home/admin/workarea/git/Velours/python/tests/mask_test.py'>
Test mask-detection 
python version used : 3

############################### TEST memory used  ################################
free memory at begining : 
begin to check gpu status 
inside check gpu memory
l 3610 free memory gpu now : 417

run mask_detect

Inside batchDatouExec : verbose : False
# VR 17-11-17 : to create in DB !
Here we check the datou graph and we reorder steps !
 Tree builded and cycle checked, now we need to re-order the steps ! 
We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue
We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id)
 DONE and to test : checkNoCycle !
We are managing only one step so we do not consider checkConsistencyNbInputNbOutput !
We are managing only one step so we do not consider checkConsistencyTypeOutputInput !
List Step Type Loaded in datou : mask_detect
list_input_json : []
origin
BFwe have missing 0 photos in the step downloads : 
photo missing : []
try to delete the photos missing in DB
length of list_filenames : 1 ; length of list_pids : 1 ; length of list_args : 1
time to download the photos : 0.16579103469848633
About to test input to load 
we should then remove the video here, and this would fix the bug of datou_current ! 
Calling datou_exec

Inside datou_exec : verbose : False
number of steps :  1
step1:mask_detect
Sat Dec 13 05:20:30 2025
VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next
VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case
VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec 
VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec
Beginning of datou step mask_detect !
save_polygon : True

 begin detect 

begin to check gpu status 
inside check gpu memory
havn't enough memory gpu , need / 3000
l 3632 free memory gpu now : 417
wait 20 seconds
l 3637 free memory gpu now : 417  max_wait_temp :  1 max_wait :  0
 gpu_flag :  0
/home/admin/workarea/git/Velours/python/tests/python_tests.py:11: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp
2025-12-13 05:20:53.682308: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2025-12-13 05:20:53.718441: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493010000 Hz
2025-12-13 05:20:53.719797: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7fda24000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2025-12-13 05:20:53.719816: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
2025-12-13 05:20:53.723144: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
2025-12-13 05:20:53.976985: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x2183f6c0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2025-12-13 05:20:53.977061: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5
2025-12-13 05:20:53.978253: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: 
pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5
coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s
2025-12-13 05:20:53.978755: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2025-12-13 05:20:53.981510: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10
2025-12-13 05:20:53.984385: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10
2025-12-13 05:20:53.984889: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10
2025-12-13 05:20:53.987857: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10
2025-12-13 05:20:53.989245: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10
2025-12-13 05:20:53.994799: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2025-12-13 05:20:53.995921: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0
2025-12-13 05:20:53.996054: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2025-12-13 05:20:53.996608: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix:
2025-12-13 05:20:53.996627: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108]      0 
2025-12-13 05:20:53.996638: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0:   N 
2025-12-13 05:20:53.997558: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 23 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5)
WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead.

Inside mask_sub_process
Inside mask_detect 
About to load cache.load_thcl_param 
To do loadFromThcl(), then load ParamDescType : thcl454
 thcls : [{'id': 454, 'mtr_user_id': 31, 'name': 'mask_coco_origin', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'backgroud,person,bicycle,car,motorcycle,airplane,bus,train,truck,boat,trafficlight,firehydrant,stopsign,parkingmeter,bench,bird,cat,dog,horse,sheep,cow,elephant,bear,zebra,giraffe,backpack,umbrella,handbag,tie,suitcase,frisbee,skis,snowboard,sportsball,kite,baseballbat,baseballglove,skateboard,surfboard,tennisracket,bottle,wineglass,cup,fork,knife,spoon,bowl,banana,apple,sandwich,orange,broccoli,carrot,hotdog,pizza,donut,cake,chair,couch,pottedplant,bed,diningtable,toilet,tv,laptop,mouse,remote,keyboard,cellphone,microwave,oven,toaster,sink,refrigerator,book,clock,vase,scissors,teddybear,hairdrier,toothbrush', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 445, 'photo_desc_type': 3473, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0'}]
 thcl {'id': 454, 'mtr_user_id': 31, 'name': 'mask_coco_origin', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'backgroud,person,bicycle,car,motorcycle,airplane,bus,train,truck,boat,trafficlight,firehydrant,stopsign,parkingmeter,bench,bird,cat,dog,horse,sheep,cow,elephant,bear,zebra,giraffe,backpack,umbrella,handbag,tie,suitcase,frisbee,skis,snowboard,sportsball,kite,baseballbat,baseballglove,skateboard,surfboard,tennisracket,bottle,wineglass,cup,fork,knife,spoon,bowl,banana,apple,sandwich,orange,broccoli,carrot,hotdog,pizza,donut,cake,chair,couch,pottedplant,bed,diningtable,toilet,tv,laptop,mouse,remote,keyboard,cellphone,microwave,oven,toaster,sink,refrigerator,book,clock,vase,scissors,teddybear,hairdrier,toothbrush', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 445, 'photo_desc_type': 3473, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0'}
 Update svm_hashtag_type_desc : 3473
FOUND : 1
 Here is data_from_sql_as_vec to set the ParamDescriptorType : (3473, 'mask_coco_origin', 16384, 25088, 'mask_coco_origin', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2018, 3, 19, 10, 42, 21), datetime.datetime(2018, 3, 19, 10, 42, 21))
{'thcl': {'id': 454, 'mtr_user_id': 31, 'name': 'mask_coco_origin', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'backgroud,person,bicycle,car,motorcycle,airplane,bus,train,truck,boat,trafficlight,firehydrant,stopsign,parkingmeter,bench,bird,cat,dog,horse,sheep,cow,elephant,bear,zebra,giraffe,backpack,umbrella,handbag,tie,suitcase,frisbee,skis,snowboard,sportsball,kite,baseballbat,baseballglove,skateboard,surfboard,tennisracket,bottle,wineglass,cup,fork,knife,spoon,bowl,banana,apple,sandwich,orange,broccoli,carrot,hotdog,pizza,donut,cake,chair,couch,pottedplant,bed,diningtable,toilet,tv,laptop,mouse,remote,keyboard,cellphone,microwave,oven,toaster,sink,refrigerator,book,clock,vase,scissors,teddybear,hairdrier,toothbrush', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 445, 'photo_desc_type': 3473, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['backgroud', 'person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus', 'train', 'truck', 'boat', 'trafficlight', 'firehydrant', 'stopsign', 'parkingmeter', 'bench', 'bird', 'cat', 'dog', 'horse', 'sheep', 'cow', 'elephant', 'bear', 'zebra', 'giraffe', 'backpack', 'umbrella', 'handbag', 'tie', 'suitcase', 'frisbee', 'skis', 'snowboard', 'sportsball', 'kite', 'baseballbat', 'baseballglove', 'skateboard', 'surfboard', 'tennisracket', 'bottle', 'wineglass', 'cup', 'fork', 'knife', 'spoon', 'bowl', 'banana', 'apple', 'sandwich', 'orange', 'broccoli', 'carrot', 'hotdog', 'pizza', 'donut', 'cake', 'chair', 'couch', 'pottedplant', 'bed', 'diningtable', 'toilet', 'tv', 'laptop', 'mouse', 'remote', 'keyboard', 'cellphone', 'microwave', 'oven', 'toaster', 'sink', 'refrigerator', 'book', 'clock', 'vase', 'scissors', 'teddybear', 'hairdrier', 'toothbrush'], 'list_hashtags_csv': 'backgroud,person,bicycle,car,motorcycle,airplane,bus,train,truck,boat,trafficlight,firehydrant,stopsign,parkingmeter,bench,bird,cat,dog,horse,sheep,cow,elephant,bear,zebra,giraffe,backpack,umbrella,handbag,tie,suitcase,frisbee,skis,snowboard,sportsball,kite,baseballbat,baseballglove,skateboard,surfboard,tennisracket,bottle,wineglass,cup,fork,knife,spoon,bowl,banana,apple,sandwich,orange,broccoli,carrot,hotdog,pizza,donut,cake,chair,couch,pottedplant,bed,diningtable,toilet,tv,laptop,mouse,remote,keyboard,cellphone,microwave,oven,toaster,sink,refrigerator,book,clock,vase,scissors,teddybear,hairdrier,toothbrush', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 445, 'svm_hashtag_type_desc': 3473, 'photo_desc_type': 3473, 'pb_hashtag_id_or_classifier': 0}
list_class_names : 
['backgroud', 'person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus', 'train', 'truck', 'boat', 'trafficlight', 'firehydrant', 'stopsign', 'parkingmeter', 'bench', 'bird', 'cat', 'dog', 'horse', 'sheep', 'cow', 'elephant', 'bear', 'zebra', 'giraffe', 'backpack', 'umbrella', 'handbag', 'tie', 'suitcase', 'frisbee', 'skis', 'snowboard', 'sportsball', 'kite', 'baseballbat', 'baseballglove', 'skateboard', 'surfboard', 'tennisracket', 'bottle', 'wineglass', 'cup', 'fork', 'knife', 'spoon', 'bowl', 'banana', 'apple', 'sandwich', 'orange', 'broccoli', 'carrot', 'hotdog', 'pizza', 'donut', 'cake', 'chair', 'couch', 'pottedplant', 'bed', 'diningtable', 'toilet', 'tv', 'laptop', 'mouse', 'remote', 'keyboard', 'cellphone', 'microwave', 'oven', 'toaster', 'sink', 'refrigerator', 'book', 'clock', 'vase', 'scissors', 'teddybear', 'hairdrier', 'toothbrush']

Configurations:
BACKBONE                       resnet101
BACKBONE_SHAPES                [[160 160]
 [ 80  80]
 [ 40  40]
 [ 20  20]
 [ 10  10]]
BACKBONE_STRIDES               [4, 8, 16, 32, 64]
BATCH_SIZE                     1
BBOX_STD_DEV                   [0.1 0.1 0.2 0.2]
DETECTION_MAX_INSTANCES        100
DETECTION_MIN_CONFIDENCE       0.3
DETECTION_NMS_THRESHOLD        0.3
GPU_COUNT                      1
IMAGES_PER_GPU                 1
IMAGE_MAX_DIM                  640
IMAGE_MIN_DIM                  640
IMAGE_PADDING                  True
IMAGE_SHAPE                    [640 640   3]
LEARNING_MOMENTUM              0.9
LEARNING_RATE                  0.001
LOSS_WEIGHTS                   {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0}
MASK_POOL_SIZE                 14
MASK_SHAPE                     [28, 28]
MAX_GT_INSTANCES               100
MEAN_PIXEL                     [123.7 116.8 103.9]
MINI_MASK_SHAPE                (56, 56)
NAME                           mask_coco_origin
NUM_CLASSES                    81
POOL_SIZE                      7
POST_NMS_ROIS_INFERENCE        1000
POST_NMS_ROIS_TRAINING         2000
ROI_POSITIVE_RATIO             0.33
RPN_ANCHOR_RATIOS              [0.5, 1, 2]
RPN_ANCHOR_SCALES              (16, 32, 64, 128, 256)
RPN_ANCHOR_STRIDE              1
2025-12-13 05:20:54.873201: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: 
pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5
coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s
2025-12-13 05:20:54.873283: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2025-12-13 05:20:54.873300: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10
2025-12-13 05:20:54.873318: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10
2025-12-13 05:20:54.873333: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10
2025-12-13 05:20:54.873348: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10
2025-12-13 05:20:54.873364: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10
2025-12-13 05:20:54.873380: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2025-12-13 05:20:54.874014: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0
2025-12-13 05:20:54.874963: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: 
pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5
coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s
2025-12-13 05:20:54.874995: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2025-12-13 05:20:54.875010: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10
2025-12-13 05:20:54.875024: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10
2025-12-13 05:20:54.875038: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10
2025-12-13 05:20:54.875052: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10
2025-12-13 05:20:54.875066: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10
2025-12-13 05:20:54.875080: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2025-12-13 05:20:54.875723: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0
2025-12-13 05:20:54.875756: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix:
2025-12-13 05:20:54.875765: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108]      0 
2025-12-13 05:20:54.875772: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0:   N 
2025-12-13 05:20:54.876424: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 23 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5)
2025-12-13 05:20:54.879670: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.00M (1048576 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.880487: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 921.8K (943872 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.881257: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 829.8K (849664 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.882075: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 747.0K (764928 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.882885: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 672.5K (688640 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.883684: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 605.2K (619776 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.884482: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 544.8K (557824 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.885249: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 490.5K (502272 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.886005: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 441.5K (452096 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.886859: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 397.5K (407040 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.887673: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 357.8K (366336 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.888484: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 322.0K (329728 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.889260: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 290.0K (296960 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.889987: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 261.0K (267264 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.890763: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 235.0K (240640 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.891564: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 211.5K (216576 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.892372: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 190.5K (195072 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.893171: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 171.5K (175616 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.893969: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 154.5K (158208 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.894729: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 139.2K (142592 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.895528: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 125.5K (128512 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.896325: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 113.0K (115712 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.897122: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 101.8K (104192 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.897918: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 91.8K (93952 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.898693: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 82.8K (84736 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.899493: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 74.5K (76288 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.900259: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 67.2K (68864 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.900997: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 60.8K (62208 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.901798: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 54.8K (56064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.902606: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 49.5K (50688 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.903406: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 44.8K (45824 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.904184: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 40.5K (41472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.904909: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 36.5K (37376 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.905633: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 33.0K (33792 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.906448: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 29.8K (30464 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.907250: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 27.0K (27648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.908043: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 24.5K (25088 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.908767: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 22.2K (22784 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.909491: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 20.2K (20736 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.910215: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 18.2K (18688 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.911018: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 16.5K (16896 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.911822: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 15.0K (15360 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.912619: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 13.5K (13824 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.913382: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 12.2K (12544 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.914251: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 11.2K (11520 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.915092: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 10.2K (10496 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.915894: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 9.2K (9472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.916632: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 8.5K (8704 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.917361: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 7.8K (7936 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.918089: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 7.0K (7168 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.918880: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 6.5K (6656 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.919681: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 6.0K (6144 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.920477: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 5.5K (5632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.921264: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 5.0K (5120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.922020: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.5K (4608 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.922846: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.2K (4352 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.923651: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.0K (4096 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.924408: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.8K (3840 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.925139: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.5K (3584 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.925865: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.2K (3328 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.926635: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.0K (3072 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.927438: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.8K (2816 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.928235: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.5K (2560 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.929032: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.929757: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.930506: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.931317: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.932109: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.932906: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.933702: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.934508: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.935309: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.936101: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.936830: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.937626: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.938434: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.939238: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.940035: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.940831: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.941617: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.942427: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.943228: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.944034: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.944766: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.945490: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.946217: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.947022: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.947822: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.948619: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.949391: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.950118: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.950913: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.951725: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.952526: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.953304: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.954033: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.954805: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.955603: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.956372: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.957119: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.957915: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.958701: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.959505: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.960281: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.961010: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.961754: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.962567: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.963367: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.964162: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.964954: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.965751: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.966539: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.967340: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.968133: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.968861: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.969589: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.970323: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.971119: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.971932: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.972732: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.973529: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.974331: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.975130: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.975926: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.976722: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.977486: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.978217: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.979034: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.979772: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.980571: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.981348: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.982150: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.982960: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.983759: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.984515: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.985239: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.985963: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.986726: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.987525: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.988306: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.989068: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.989864: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.990655: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.991477: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.992274: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.993070: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.993866: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.994649: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.995448: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.996236: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.997015: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.997812: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.998603: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:54.999412: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.000213: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.001011: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.001791: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.002601: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.003384: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.004114: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.004911: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.005663: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.006418: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.007218: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.008014: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.008785: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.009582: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.010391: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.011194: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.011953: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.012755: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.013519: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.014247: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.015041: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.015841: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.016617: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.017413: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.018218: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.019032: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.019777: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.020501: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.021226: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.021968: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.022777: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.023550: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.024274: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.024998: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.025796: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.026584: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.027385: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.028181: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.028978: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.029726: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.030473: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.031274: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.032084: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.032886: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.033646: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.034415: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.035215: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.035945: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.036682: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.037475: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.038299: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.039093: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.039890: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.040646: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.041382: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.042194: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.043002: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.043801: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.044567: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.045291: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.046015: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.046792: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.047563: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.048287: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.049057: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.049855: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.050650: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.051450: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.052265: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.053064: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.053812: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.054623: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.055420: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.056149: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.056898: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.057694: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.058487: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.059286: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.060083: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.060879: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.061687: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.062494: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.063295: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.064024: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.064751: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.065475: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.066199: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.066997: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.067780: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.068577: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.069373: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.070170: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.070976: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.071746: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.072544: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.073341: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.074146: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.074950: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.075750: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.076556: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.077358: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.078156: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.078972: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.079772: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.080570: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.081386: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.082190: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.082996: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.083796: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.084594: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.085392: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.086189: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.087010: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.087813: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.088610: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.089406: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.090203: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.091007: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.091829: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.092645: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.093446: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.094243: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.095051: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.095852: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.096649: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.097445: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.098242: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.099056: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.099856: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.100653: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.101470: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.102281: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.103089: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.103894: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.104704: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.105502: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.106319: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.107122: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.107918: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.108682: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.109438: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.110234: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.111056: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.111878: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.112679: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.113471: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.114195: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.114988: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.115753: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.116477: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.117201: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.117926: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.118680: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.119475: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.120200: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.120924: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.121696: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.122521: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.123328: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.124063: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.124787: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.125511: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.126235: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.127041: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.127844: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.128641: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.129436: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.130168: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.130954: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.131746: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.132547: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.133343: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.134140: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.134935: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.135708: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.136496: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.137303: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.138103: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.138938: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.139717: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.140444: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.141173: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.141923: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.142729: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.143528: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.144324: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.145120: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.145877: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.146664: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.147466: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.148263: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.149009: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.149738: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.150497: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.151300: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.152097: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.152896: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.153686: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.154488: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.155292: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.156043: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.156840: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.157624: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.158367: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.159169: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.159924: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.160653: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.161436: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.162249: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.163056: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.163824: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.164549: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.165328: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.166125: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.166928: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.167715: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.168440: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.169164: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.169893: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.170661: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.171462: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.172277: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.173077: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.173962: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.174783: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.175581: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.176306: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.177030: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.177765: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.178580: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.179383: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.180180: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.180937: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.181695: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.182513: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.183341: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.184144: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.184941: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.185727: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.186470: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.187271: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.188020: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.188817: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.189607: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.190356: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.191157: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.191956: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.192758: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.193555: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.194361: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.195162: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.195925: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.196649: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.197427: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.198223: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.199039: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.199815: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.200598: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.201395: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.202181: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.202994: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.203774: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.204564: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.205362: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.206105: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.206888: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.207680: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.208405: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.209182: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.209963: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.210782: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.211592: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.212346: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.213143: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.213932: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.214737: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.215536: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.216333: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.217084: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.217833: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.218643: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.219444: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.220183: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.220908: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.221682: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.222489: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.223297: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.224094: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.224865: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.225642: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.226445: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.227245: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.228055: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.228832: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.229561: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.230307: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.231116: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.231919: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.232715: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.233511: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.234307: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.235106: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.235902: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.236699: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.237495: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.238231: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.239043: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.239827: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.240556: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.241299: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.242083: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.242893: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.243693: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.244489: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.245286: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.246046: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.246824: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.247626: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.248352: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.249076: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.249803: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.250562: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.251379: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.252163: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.252960: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.253757: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.254562: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.255366: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.256165: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.256961: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.257750: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.258491: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.259291: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.260050: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.260782: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.261556: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.262371: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.263173: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.263970: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.264758: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.265492: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.266298: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.267098: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.267876: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.268666: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.269464: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.270209: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.271025: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.271835: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.272634: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.273431: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.274180: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.274961: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.275755: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.276484: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.277212: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.277958: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.278780: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.279580: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.280370: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.281166: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.281979: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.282797: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.283597: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.284335: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.285059: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.285783: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.286522: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.287322: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.288085: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.288869: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.289665: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.290411: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.291210: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.292024: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.292816: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.293592: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.294406: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.295208: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.296003: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.296802: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.297607: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.298419: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.299220: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.300016: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.300808: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.301560: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.302369: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.303169: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.303946: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.304670: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.305394: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.306118: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.306887: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.307685: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.308481: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.309281: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.310086: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.310916: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.311732: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.312534: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.313282: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.314006: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.314765: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.315566: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.316320: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.317045: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.317777: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.318623: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.319431: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.320231: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.321028: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.321836: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.322640: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.323441: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.324235: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.325012: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.325755: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.326562: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.327362: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.328159: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.328944: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.329674: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.330413: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.331206: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.332001: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.332795: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.333591: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.334355: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.335180: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.335984: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.336783: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.337513: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.338246: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.339055: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.339854: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.340583: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.341374: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.342212: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.343034: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.343835: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.344632: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.345376: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.346124: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.346930: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.347737: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.348471: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.349199: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.349928: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.350703: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.351507: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.352325: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.353101: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.353840: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.354601: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.355402: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.356166: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.356890: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.357614: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.358349: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.359148: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.359938: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.360663: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.361409: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.362198: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.363519: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.364331: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.365394: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.366369: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.367172: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.367969: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.368766: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.369501: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.370234: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.371068: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.371875: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.372678: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.373497: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.374307: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.375128: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.375922: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.376691: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.377490: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.378264: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.379063: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.379863: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.380660: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.381473: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.382265: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.383092: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.383891: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.384624: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.385354: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.386088: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.386903: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.387704: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.388452: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.389188: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.389942: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.390763: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.391676: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.392528: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.393332: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.394139: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.394949: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.395753: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.396524: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.397320: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.398117: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.398924: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.399726: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.400556: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.401389: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.402195: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.403003: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.403806: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.404553: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.405325: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.406113: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.406904: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.407722: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.408479: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.409221: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.410005: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.410883: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.411714: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.412520: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.413296: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.414032: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.414828: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.415646: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.416471: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.417288: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.418111: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.418913: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.419720: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.420505: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.421338: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.422176: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.423016: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.423824: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.424628: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.425436: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.426226: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.427044: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.427885: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.428697: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.429526: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.430369: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.431215: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.432063: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.432898: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.433655: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.434420: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.435244: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.436062: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.436810: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.437550: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.438366: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.439206: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.440013: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.440747: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.441578: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.442418: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.443231: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.444037: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.444842: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.445590: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.446348: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.447152: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.447957: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.448697: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.449438: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.450169: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.450967: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.451795: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.452602: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.453373: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.454109: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.454939: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.455748: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.456507: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.457275: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.458025: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.458836: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.459650: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.460448: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.461259: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.462083: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.462925: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.463759: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.464581: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.465368: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.466103: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.466908: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.467726: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.468514: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.469252: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.469997: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.470782: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.471598: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.472434: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.473262: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.474081: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.474877: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.475687: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.476453: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.477254: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.478059: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.478866: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.479680: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.480451: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.481259: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.482095: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.482933: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.483749: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.484513: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.485254: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.486010: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.486799: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.487608: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.488388: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.489120: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.489935: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.490734: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.491546: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.492378: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.493187: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.493956: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.494772: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.495543: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.496284: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.497038: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.497804: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.498624: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.499405: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.500162: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.500969: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.501782: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.502624: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.503454: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.504253: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.505005: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.505754: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.506513: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.507326: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.508130: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.508868: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.509657: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.510475: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.511316: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.512131: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.512940: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.513695: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.514464: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.515273: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.516089: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.516912: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.517681: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.518482: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.519296: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.520109: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.520921: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.521724: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.522557: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.523368: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.524171: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.524924: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.525667: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.526483: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.527290: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.528109: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.528911: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.529666: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.530434: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.531271: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.532094: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.532897: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.533658: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.534413: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.535228: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.536037: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.536791: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.537594: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.538429: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.539258: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.540088: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.540855: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.541629: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.542447: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.543277: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.544058: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.544872: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.545638: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.546390: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.547195: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.547998: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.548756: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.549490: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.550220: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.551039: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.551868: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.552678: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.553484: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.554299: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.555102: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.555921: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.556753: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.557542: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.558290: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.559112: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.559924: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.560682: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.561451: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.562266: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.563103: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.563879: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.564636: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.565445: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.566257: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.567082: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.567891: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.568667: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.569404: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.570159: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.570998: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.571862: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.572634: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.573359: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.574159: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.574971: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.575785: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.576575: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.577329: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.578070: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.578869: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.579674: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.580465: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.581200: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.582036: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.582892: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.583685: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.584505: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.585314: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.586051: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.586845: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.587666: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.588462: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.589201: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.589960: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.590774: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.591607: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.592439: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.593248: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.594059: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.594874: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.595655: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.596393: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.597187: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.597991: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.598834: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.599652: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.600453: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.601187: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.601953: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.602771: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.603577: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.604381: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.605187: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.606016: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.606856: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.607641: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.608388: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.609126: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.609873: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.610703: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.611504: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.612390: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.613213: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.614043: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.614878: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.615679: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.616438: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.617195: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.617946: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.618761: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.619586: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.620335: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.621110: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.621953: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.622831: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.623672: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.624477: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.625222: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.625977: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.626800: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.627604: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.628394: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.629210: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.629961: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.630751: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.631553: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.632313: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.633120: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.633879: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.634631: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.635389: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.636143: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.636917: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.637731: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.638571: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.639405: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.640233: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.641000: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.641777: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.642599: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.643423: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.644180: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.645003: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.645797: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.646571: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.647376: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.648161: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.648968: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.649738: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.650498: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.651329: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.652135: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.652950: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.653725: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.654483: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.655295: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.656060: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.656827: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.657644: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.658479: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.659306: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.660132: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.660926: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.661691: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.662511: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.663326: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.664138: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.664960: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.665774: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.666585: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.667391: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.668131: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.668900: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.669709: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.670537: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.671363: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.672106: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.672911: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.673696: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.674465: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.675280: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.676038: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.676787: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.677561: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.678389: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.679192: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.679996: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.680808: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.681639: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.682457: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.683272: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.684083: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.684888: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.685704: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.686543: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.687376: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.688187: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.689003: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.689825: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.690661: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.691510: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.692340: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.693170: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.693955: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.694774: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.695593: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.696326: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.697061: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.697792: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.698599: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.699409: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.700214: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.701029: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.701856: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.702711: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.703529: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.704334: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.705123: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.705939: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.706763: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.707579: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.708321: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.709064: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.709801: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.710569: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.711399: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.712208: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.712988: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.713785: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.714607: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.715417: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.716222: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.717034: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.717847: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.718662: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.719472: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.720281: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.721042: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.721800: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.722627: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.723441: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.724183: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.724913: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.725647: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.726433: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.727245: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.728052: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.728848: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.729589: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.730338: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.731146: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.731934: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.732757: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.733600: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.734366: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.735178: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.735982: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.736782: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.737584: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.738410: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.739236: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.740006: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.740742: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.741487: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.742326: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.743142: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.743948: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.744761: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.745568: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.746386: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.747208: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.748019: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.748825: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.749628: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.750443: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.751254: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.752086: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.752916: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.753727: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.754553: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.755378: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.756192: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.757004: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.757817: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.758651: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.759463: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.760267: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.761074: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.761903: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.762758: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.763575: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.764399: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.765201: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.766072: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.766922: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.767736: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.768546: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.769370: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.770387: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.771202: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.772023: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.772859: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.773671: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.774489: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.775301: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.776111: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.776912: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.777717: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.778537: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.779352: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.780165: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.780989: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.781831: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.782708: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.783533: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.784348: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.785154: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.785960: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.786778: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.787588: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.788400: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.789208: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.790039: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.790899: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.791737: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.792571: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.793398: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.794227: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.795068: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.795892: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.796717: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.797533: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.798395: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.799225: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.800051: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.800873: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.801691: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.802558: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.803400: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.804226: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.805042: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.805858: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.806683: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.807497: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.808302: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.809109: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.809921: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.810752: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.811614: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.812430: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.813249: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.814057: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.814896: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.815720: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.816532: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.817340: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.818152: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.818989: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.819811: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.820643: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.821475: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.822292: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.823118: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.823945: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.824763: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.825591: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.826433: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.827262: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.828087: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.828913: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.829739: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.830574: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.831502: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.832334: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.833151: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.833964: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.834788: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.835605: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.836420: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.837239: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.838059: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.839001: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.839863: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.840700: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.841541: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.842370: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.843198: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.844019: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.844854: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.845691: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.846880: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.848004: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.849132: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.850280: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.851411: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.852519: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.853760: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.854950: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.856194: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.857600: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.858604: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.859697: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.860766: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.861840: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.862926: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.863959: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.864838: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.865734: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.866637: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.867531: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.868434: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.869319: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.870192: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.871067: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.871935: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.872799: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.873656: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.874731: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.875602: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.876455: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.877310: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.878179: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.879050: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.879910: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.880777: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.881645: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.882529: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.883384: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.884244: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.885104: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.885961: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.886824: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.887681: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.888541: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.889399: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.890268: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.891136: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.891997: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.892854: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.893708: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.894570: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.895426: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.896286: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.897145: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.898012: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.898889: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.899745: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.901384: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.902206: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.903015: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.903827: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.904629: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.905442: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.906251: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.907068: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.907871: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.908669: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.909468: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.910265: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.911084: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:20:55.911907: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.2K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2025-12-13 05:21:06.858725: W tensorflow/core/common_runtime/bfc_allocator.cc:434] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.25MiB (rounded to 2359296)
Current allocation summary follows.
2025-12-13 05:21:06.858778: I tensorflow/core/common_runtime/bfc_allocator.cc:934] BFCAllocator dump for GPU_0_bfc
2025-12-13 05:21:06.858799: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (256): 	Total Chunks: 59, Chunks in use: 59. 14.8KiB allocated for chunks. 14.8KiB in use in bin. 8.8KiB client-requested in use in bin.
2025-12-13 05:21:06.858815: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (512): 	Total Chunks: 40, Chunks in use: 40. 20.0KiB allocated for chunks. 20.0KiB in use in bin. 20.0KiB client-requested in use in bin.
2025-12-13 05:21:06.858830: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (1024): 	Total Chunks: 35, Chunks in use: 35. 35.0KiB allocated for chunks. 35.0KiB in use in bin. 35.0KiB client-requested in use in bin.
2025-12-13 05:21:06.858845: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (2048): 	Total Chunks: 26, Chunks in use: 26. 52.2KiB allocated for chunks. 52.2KiB in use in bin. 51.0KiB client-requested in use in bin.
2025-12-13 05:21:06.858879: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (4096): 	Total Chunks: 10, Chunks in use: 10. 40.0KiB allocated for chunks. 40.0KiB in use in bin. 40.0KiB client-requested in use in bin.
2025-12-13 05:21:06.858894: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (8192): 	Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2025-12-13 05:21:06.858909: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (16384): 	Total Chunks: 1, Chunks in use: 1. 16.0KiB allocated for chunks. 16.0KiB in use in bin. 16.0KiB client-requested in use in bin.
2025-12-13 05:21:06.858924: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (32768): 	Total Chunks: 2, Chunks in use: 1. 79.2KiB allocated for chunks. 36.8KiB in use in bin. 36.8KiB client-requested in use in bin.
2025-12-13 05:21:06.858939: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (65536): 	Total Chunks: 7, Chunks in use: 6. 480.0KiB allocated for chunks. 384.0KiB in use in bin. 384.0KiB client-requested in use in bin.
2025-12-13 05:21:06.858954: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (131072): 	Total Chunks: 6, Chunks in use: 4. 848.0KiB allocated for chunks. 560.0KiB in use in bin. 560.0KiB client-requested in use in bin.
2025-12-13 05:21:06.858969: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (262144): 	Total Chunks: 9, Chunks in use: 7. 2.58MiB allocated for chunks. 2.08MiB in use in bin. 1.75MiB client-requested in use in bin.
2025-12-13 05:21:06.858983: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (524288): 	Total Chunks: 8, Chunks in use: 6. 4.62MiB allocated for chunks. 3.44MiB in use in bin. 3.25MiB client-requested in use in bin.
2025-12-13 05:21:06.858997: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (1048576): 	Total Chunks: 3, Chunks in use: 2. 3.25MiB allocated for chunks. 2.00MiB in use in bin. 2.00MiB client-requested in use in bin.
2025-12-13 05:21:06.859012: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (2097152): 	Total Chunks: 5, Chunks in use: 4. 11.87MiB allocated for chunks. 9.87MiB in use in bin. 8.75MiB client-requested in use in bin.
2025-12-13 05:21:06.859025: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (4194304): 	Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2025-12-13 05:21:06.859038: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (8388608): 	Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2025-12-13 05:21:06.859051: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (16777216): 	Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2025-12-13 05:21:06.859064: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (33554432): 	Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2025-12-13 05:21:06.859077: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (67108864): 	Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2025-12-13 05:21:06.859089: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (134217728): 	Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2025-12-13 05:21:06.859102: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (268435456): 	Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2025-12-13 05:21:06.859116: I tensorflow/core/common_runtime/bfc_allocator.cc:957] Bin for 2.25MiB was 2.00MiB, Chunk State: 
2025-12-13 05:21:06.859139: I tensorflow/core/common_runtime/bfc_allocator.cc:963]   Size: 2.00MiB | Requested Size: 1.00MiB | in_use: 0 | bin_num: 13, prev:   Size: 512.0KiB | Requested Size: 512.0KiB | in_use: 1 | bin_num: -1, next:   Size: 1.00MiB | Requested Size: 1.00MiB | in_use: 1 | bin_num: -1
2025-12-13 05:21:06.859159: I tensorflow/core/common_runtime/bfc_allocator.cc:970] Next region of size 4194304
2025-12-13 05:21:06.859175: I tensorflow/core/common_runtime/bfc_allocator.cc:990] Free  at 7fd844800000 of size 262144 next 105
2025-12-13 05:21:06.859188: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd844840000 of size 262144 next 104
2025-12-13 05:21:06.859199: I tensorflow/core/common_runtime/bfc_allocator.cc:990] Free  at 7fd844880000 of size 655360 next 98
2025-12-13 05:21:06.859211: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd844920000 of size 589824 next 97
2025-12-13 05:21:06.859223: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd8449b0000 of size 262144 next 119
2025-12-13 05:21:06.859234: I tensorflow/core/common_runtime/bfc_allocator.cc:990] Free  at 7fd8449f0000 of size 262144 next 114
2025-12-13 05:21:06.859246: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd844a30000 of size 524288 next 113
2025-12-13 05:21:06.859257: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd844ab0000 of size 262144 next 136
2025-12-13 05:21:06.859269: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd844af0000 of size 327680 next 126
2025-12-13 05:21:06.859281: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd844b40000 of size 786432 next 18446744073709551615
2025-12-13 05:21:06.859292: I tensorflow/core/common_runtime/bfc_allocator.cc:970] Next region of size 8388608
2025-12-13 05:21:06.859303: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd844c00000 of size 262144 next 155
2025-12-13 05:21:06.859315: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd844c40000 of size 327680 next 144
2025-12-13 05:21:06.859326: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd844c90000 of size 589824 next 142
2025-12-13 05:21:06.859338: I tensorflow/core/common_runtime/bfc_allocator.cc:990] Free  at 7fd844d20000 of size 589824 next 162
2025-12-13 05:21:06.859349: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd844db0000 of size 589824 next 161
2025-12-13 05:21:06.859360: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd844e40000 of size 524288 next 173
2025-12-13 05:21:06.859371: I tensorflow/core/common_runtime/bfc_allocator.cc:990] Free  at 7fd844ec0000 of size 2097152 next 192
2025-12-13 05:21:06.859383: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd8450c0000 of size 1048576 next 191
2025-12-13 05:21:06.859394: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd8451c0000 of size 1048576 next 205
2025-12-13 05:21:06.859406: I tensorflow/core/common_runtime/bfc_allocator.cc:990] Free  at 7fd8452c0000 of size 1310720 next 18446744073709551615
2025-12-13 05:21:06.859417: I tensorflow/core/common_runtime/bfc_allocator.cc:970] Next region of size 10352384
2025-12-13 05:21:06.859428: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd845400000 of size 2359296 next 183
2025-12-13 05:21:06.859440: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd845640000 of size 2097152 next 198
2025-12-13 05:21:06.859451: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd845840000 of size 2359296 next 210
2025-12-13 05:21:06.859463: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd845a80000 of size 3536640 next 18446744073709551615
2025-12-13 05:21:06.859474: I tensorflow/core/common_runtime/bfc_allocator.cc:970] Next region of size 2304
2025-12-13 05:21:06.859486: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97ee00000 of size 2304 next 18446744073709551615
2025-12-13 05:21:06.859497: I tensorflow/core/common_runtime/bfc_allocator.cc:970] Next region of size 2097152
2025-12-13 05:21:06.859516: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f000000 of size 256 next 5
2025-12-13 05:21:06.859528: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f000100 of size 256 next 8
2025-12-13 05:21:06.859540: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f000200 of size 256 next 9
2025-12-13 05:21:06.859551: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f000300 of size 256 next 10
2025-12-13 05:21:06.859562: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f000400 of size 256 next 11
2025-12-13 05:21:06.859574: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f000500 of size 256 next 12
2025-12-13 05:21:06.859585: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f000600 of size 256 next 13
2025-12-13 05:21:06.859596: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f000700 of size 256 next 17
2025-12-13 05:21:06.859607: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f000800 of size 256 next 19
2025-12-13 05:21:06.859619: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f000900 of size 256 next 20
2025-12-13 05:21:06.859630: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f000a00 of size 256 next 21
2025-12-13 05:21:06.859641: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f000b00 of size 256 next 22
2025-12-13 05:21:06.859652: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f000c00 of size 256 next 24
2025-12-13 05:21:06.859663: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f000d00 of size 256 next 25
2025-12-13 05:21:06.859675: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f000e00 of size 256 next 23
2025-12-13 05:21:06.859686: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f000f00 of size 256 next 28
2025-12-13 05:21:06.859697: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f001000 of size 256 next 29
2025-12-13 05:21:06.859708: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f001100 of size 256 next 30
2025-12-13 05:21:06.859719: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f001200 of size 256 next 31
2025-12-13 05:21:06.859731: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f001300 of size 256 next 33
2025-12-13 05:21:06.859742: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f001400 of size 256 next 34
2025-12-13 05:21:06.859754: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f001500 of size 1024 next 32
2025-12-13 05:21:06.859765: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f001900 of size 1024 next 37
2025-12-13 05:21:06.859776: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f001d00 of size 1024 next 38
2025-12-13 05:21:06.859787: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f002100 of size 1024 next 39
2025-12-13 05:21:06.859799: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f002500 of size 1024 next 40
2025-12-13 05:21:06.859810: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f002900 of size 1024 next 41
2025-12-13 05:21:06.859821: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f002d00 of size 1024 next 43
2025-12-13 05:21:06.859832: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f003100 of size 1024 next 44
2025-12-13 05:21:06.859844: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f003500 of size 1024 next 45
2025-12-13 05:21:06.859855: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f003900 of size 1024 next 46
2025-12-13 05:21:06.859866: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f003d00 of size 256 next 48
2025-12-13 05:21:06.859878: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f003e00 of size 256 next 49
2025-12-13 05:21:06.859897: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f003f00 of size 256 next 50
2025-12-13 05:21:06.859909: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f004000 of size 256 next 51
2025-12-13 05:21:06.859920: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f004100 of size 256 next 52
2025-12-13 05:21:06.859931: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f004200 of size 256 next 53
2025-12-13 05:21:06.859942: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f004300 of size 256 next 56
2025-12-13 05:21:06.859954: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f004400 of size 256 next 57
2025-12-13 05:21:06.859965: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f004500 of size 256 next 58
2025-12-13 05:21:06.859976: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f004600 of size 256 next 14
2025-12-13 05:21:06.859987: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f004700 of size 256 next 15
2025-12-13 05:21:06.859999: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f004800 of size 256 next 16
2025-12-13 05:21:06.860010: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f004900 of size 1024 next 60
2025-12-13 05:21:06.860021: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f004d00 of size 1024 next 61
2025-12-13 05:21:06.860032: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f005100 of size 1024 next 62
2025-12-13 05:21:06.860044: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f005500 of size 1024 next 63
2025-12-13 05:21:06.860055: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f005900 of size 1024 next 64
2025-12-13 05:21:06.860066: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f005d00 of size 256 next 66
2025-12-13 05:21:06.860077: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f005e00 of size 256 next 67
2025-12-13 05:21:06.860088: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f005f00 of size 256 next 68
2025-12-13 05:21:06.860100: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f006000 of size 256 next 69
2025-12-13 05:21:06.860111: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f006100 of size 256 next 70
2025-12-13 05:21:06.860122: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f006200 of size 256 next 73
2025-12-13 05:21:06.860133: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f006300 of size 256 next 74
2025-12-13 05:21:06.860145: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f006400 of size 256 next 75
2025-12-13 05:21:06.860156: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f006500 of size 256 next 76
2025-12-13 05:21:06.860167: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f006600 of size 256 next 77
2025-12-13 05:21:06.860178: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f006700 of size 1024 next 79
2025-12-13 05:21:06.860190: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f006b00 of size 1024 next 80
2025-12-13 05:21:06.860201: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f006f00 of size 1024 next 81
2025-12-13 05:21:06.860212: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f007300 of size 1024 next 82
2025-12-13 05:21:06.860223: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f007700 of size 1024 next 83
2025-12-13 05:21:06.860235: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f007b00 of size 256 next 84
2025-12-13 05:21:06.860246: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f007c00 of size 256 next 85
2025-12-13 05:21:06.860258: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f007d00 of size 512 next 88
2025-12-13 05:21:06.860276: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f007f00 of size 512 next 89
2025-12-13 05:21:06.860288: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f008100 of size 512 next 90
2025-12-13 05:21:06.860299: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f008300 of size 512 next 91
2025-12-13 05:21:06.860311: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f008500 of size 512 next 92
2025-12-13 05:21:06.860322: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f008700 of size 256 next 95
2025-12-13 05:21:06.860333: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f008800 of size 256 next 96
2025-12-13 05:21:06.860344: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f008900 of size 512 next 94
2025-12-13 05:21:06.860356: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f008b00 of size 512 next 99
2025-12-13 05:21:06.860367: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f008d00 of size 512 next 100
2025-12-13 05:21:06.860378: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f008f00 of size 512 next 101
2025-12-13 05:21:06.860389: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f009100 of size 512 next 2
2025-12-13 05:21:06.860401: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f009300 of size 256 next 3
2025-12-13 05:21:06.860412: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f009400 of size 256 next 4
2025-12-13 05:21:06.860424: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f009500 of size 16384 next 18
2025-12-13 05:21:06.860435: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f00d500 of size 256 next 102
2025-12-13 05:21:06.860446: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f00d600 of size 256 next 103
2025-12-13 05:21:06.860466: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f00d700 of size 2048 next 106
2025-12-13 05:21:06.860478: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f00df00 of size 2048 next 107
2025-12-13 05:21:06.860489: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f00e700 of size 2048 next 108
2025-12-13 05:21:06.860500: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f00ef00 of size 2048 next 109
2025-12-13 05:21:06.860512: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f00f700 of size 2048 next 110
2025-12-13 05:21:06.860523: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f00ff00 of size 256 next 111
2025-12-13 05:21:06.860534: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f010000 of size 256 next 112
2025-12-13 05:21:06.860545: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f010100 of size 2048 next 115
2025-12-13 05:21:06.860556: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f010900 of size 2048 next 116
2025-12-13 05:21:06.860568: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f011100 of size 2048 next 117
2025-12-13 05:21:06.860579: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f011900 of size 2048 next 118
2025-12-13 05:21:06.860590: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f012100 of size 2048 next 6
2025-12-13 05:21:06.860602: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f012900 of size 37632 next 7
2025-12-13 05:21:06.860613: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f01bc00 of size 512 next 120
2025-12-13 05:21:06.860625: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f01be00 of size 512 next 121
2025-12-13 05:21:06.860636: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f01c000 of size 512 next 122
2025-12-13 05:21:06.860657: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f01c200 of size 512 next 123
2025-12-13 05:21:06.860668: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f01c400 of size 512 next 124
2025-12-13 05:21:06.860680: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f01c600 of size 512 next 125
2025-12-13 05:21:06.860691: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f01c800 of size 512 next 127
2025-12-13 05:21:06.860702: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f01ca00 of size 512 next 128
2025-12-13 05:21:06.860713: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f01cc00 of size 512 next 129
2025-12-13 05:21:06.860725: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f01ce00 of size 512 next 130
2025-12-13 05:21:06.860736: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f01d000 of size 2048 next 131
2025-12-13 05:21:06.860747: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f01d800 of size 2048 next 132
2025-12-13 05:21:06.860759: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f01e000 of size 2048 next 133
2025-12-13 05:21:06.860770: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f01e800 of size 2048 next 134
2025-12-13 05:21:06.860781: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f01f000 of size 2048 next 135
2025-12-13 05:21:06.860792: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f01f800 of size 512 next 137
2025-12-13 05:21:06.860804: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f01fa00 of size 512 next 138
2025-12-13 05:21:06.860815: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f01fc00 of size 512 next 139
2025-12-13 05:21:06.860826: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f01fe00 of size 512 next 140
2025-12-13 05:21:06.860838: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f020000 of size 512 next 141
2025-12-13 05:21:06.860849: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f020200 of size 512 next 145
2025-12-13 05:21:06.860860: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f020400 of size 512 next 146
2025-12-13 05:21:06.860871: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f020600 of size 512 next 147
2025-12-13 05:21:06.860883: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f020800 of size 512 next 148
2025-12-13 05:21:06.860894: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f020a00 of size 512 next 149
2025-12-13 05:21:06.860905: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f020c00 of size 2048 next 150
2025-12-13 05:21:06.860917: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f021400 of size 2048 next 151
2025-12-13 05:21:06.860928: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f021c00 of size 2048 next 152
2025-12-13 05:21:06.860939: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f022400 of size 2048 next 153
2025-12-13 05:21:06.860950: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f022c00 of size 2048 next 154
2025-12-13 05:21:06.860962: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f023400 of size 512 next 156
2025-12-13 05:21:06.860973: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f023600 of size 512 next 157
2025-12-13 05:21:06.860984: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f023800 of size 512 next 158
2025-12-13 05:21:06.860995: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f023a00 of size 512 next 159
2025-12-13 05:21:06.861007: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f023c00 of size 512 next 160
2025-12-13 05:21:06.861018: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f023e00 of size 512 next 163
2025-12-13 05:21:06.861037: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f024000 of size 512 next 164
2025-12-13 05:21:06.861048: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f024200 of size 512 next 165
2025-12-13 05:21:06.861060: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f024400 of size 512 next 166
2025-12-13 05:21:06.861071: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f024600 of size 512 next 167
2025-12-13 05:21:06.861082: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f024800 of size 2048 next 168
2025-12-13 05:21:06.861093: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f025000 of size 2048 next 169
2025-12-13 05:21:06.861105: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f025800 of size 2048 next 170
2025-12-13 05:21:06.861116: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f026000 of size 2048 next 171
2025-12-13 05:21:06.861127: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f026800 of size 2048 next 172
2025-12-13 05:21:06.861139: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f027000 of size 1024 next 174
2025-12-13 05:21:06.861150: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f027400 of size 1024 next 175
2025-12-13 05:21:06.861161: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f027800 of size 1024 next 176
2025-12-13 05:21:06.861172: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f027c00 of size 1024 next 177
2025-12-13 05:21:06.861184: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f028000 of size 1024 next 178
2025-12-13 05:21:06.861195: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f028400 of size 256 next 180
2025-12-13 05:21:06.861206: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f028500 of size 256 next 181
2025-12-13 05:21:06.861217: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f028600 of size 1024 next 179
2025-12-13 05:21:06.861229: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f028a00 of size 1024 next 184
2025-12-13 05:21:06.861240: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f028e00 of size 1024 next 185
2025-12-13 05:21:06.861251: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f029200 of size 1024 next 186
2025-12-13 05:21:06.861262: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f029600 of size 1024 next 187
2025-12-13 05:21:06.861274: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f029a00 of size 256 next 189
2025-12-13 05:21:06.861285: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f029b00 of size 256 next 190
2025-12-13 05:21:06.861297: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f029c00 of size 4096 next 188
2025-12-13 05:21:06.861308: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f02ac00 of size 4096 next 42
2025-12-13 05:21:06.861320: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f02bc00 of size 65536 next 36
2025-12-13 05:21:06.861331: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f03bc00 of size 65536 next 35
2025-12-13 05:21:06.861342: I tensorflow/core/common_runtime/bfc_allocator.cc:990] Free  at 7fd97f04bc00 of size 98304 next 27
2025-12-13 05:21:06.861354: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f063c00 of size 147456 next 26
2025-12-13 05:21:06.861365: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f087c00 of size 65536 next 47
2025-12-13 05:21:06.861377: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f097c00 of size 65536 next 59
2025-12-13 05:21:06.861388: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f0a7c00 of size 65536 next 65
2025-12-13 05:21:06.861407: I tensorflow/core/common_runtime/bfc_allocator.cc:990] Free  at 7fd97f0b7c00 of size 163840 next 55
2025-12-13 05:21:06.861419: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f0dfc00 of size 147456 next 54
2025-12-13 05:21:06.861430: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f103c00 of size 65536 next 78
2025-12-13 05:21:06.861441: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f113c00 of size 4096 next 193
2025-12-13 05:21:06.861453: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f114c00 of size 4096 next 194
2025-12-13 05:21:06.861464: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f115c00 of size 4096 next 195
2025-12-13 05:21:06.861475: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f116c00 of size 256 next 196
2025-12-13 05:21:06.861487: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f116d00 of size 256 next 197
2025-12-13 05:21:06.861498: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f116e00 of size 4096 next 199
2025-12-13 05:21:06.861509: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f117e00 of size 4096 next 200
2025-12-13 05:21:06.861520: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f118e00 of size 4096 next 201
2025-12-13 05:21:06.861532: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f119e00 of size 4096 next 202
2025-12-13 05:21:06.861544: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f11ae00 of size 4096 next 203
2025-12-13 05:21:06.861559: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f11be00 of size 1024 next 204
2025-12-13 05:21:06.861574: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f11c200 of size 1024 next 206
2025-12-13 05:21:06.861586: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f11c600 of size 1024 next 207
2025-12-13 05:21:06.861598: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f11ca00 of size 1024 next 208
2025-12-13 05:21:06.861609: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f11ce00 of size 1024 next 209
2025-12-13 05:21:06.861620: I tensorflow/core/common_runtime/bfc_allocator.cc:990] Free  at 7fd97f11d200 of size 43520 next 72
2025-12-13 05:21:06.861631: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f127c00 of size 147456 next 71
2025-12-13 05:21:06.861643: I tensorflow/core/common_runtime/bfc_allocator.cc:990] Free  at 7fd97f14bc00 of size 131072 next 87
2025-12-13 05:21:06.861655: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f16bc00 of size 131072 next 86
2025-12-13 05:21:06.861666: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd97f18bc00 of size 476160 next 18446744073709551615
2025-12-13 05:21:06.861677: I tensorflow/core/common_runtime/bfc_allocator.cc:995]      Summary of in-use Chunks by size: 
2025-12-13 05:21:06.861692: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 59 Chunks of size 256 totalling 14.8KiB
2025-12-13 05:21:06.861705: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 40 Chunks of size 512 totalling 20.0KiB
2025-12-13 05:21:06.861718: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 35 Chunks of size 1024 totalling 35.0KiB
2025-12-13 05:21:06.861731: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 25 Chunks of size 2048 totalling 50.0KiB
2025-12-13 05:21:06.861743: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 1 Chunks of size 2304 totalling 2.2KiB
2025-12-13 05:21:06.861756: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 10 Chunks of size 4096 totalling 40.0KiB
2025-12-13 05:21:06.861768: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 1 Chunks of size 16384 totalling 16.0KiB
2025-12-13 05:21:06.861781: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 1 Chunks of size 37632 totalling 36.8KiB
2025-12-13 05:21:06.861803: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 6 Chunks of size 65536 totalling 384.0KiB
2025-12-13 05:21:06.861817: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 1 Chunks of size 131072 totalling 128.0KiB
2025-12-13 05:21:06.861829: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 3 Chunks of size 147456 totalling 432.0KiB
2025-12-13 05:21:06.861842: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 4 Chunks of size 262144 totalling 1.00MiB
2025-12-13 05:21:06.861854: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 2 Chunks of size 327680 totalling 640.0KiB
2025-12-13 05:21:06.861867: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 1 Chunks of size 476160 totalling 465.0KiB
2025-12-13 05:21:06.861880: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 2 Chunks of size 524288 totalling 1.00MiB
2025-12-13 05:21:06.861892: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 3 Chunks of size 589824 totalling 1.69MiB
2025-12-13 05:21:06.861905: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 1 Chunks of size 786432 totalling 768.0KiB
2025-12-13 05:21:06.861917: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 2 Chunks of size 1048576 totalling 2.00MiB
2025-12-13 05:21:06.861929: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 1 Chunks of size 2097152 totalling 2.00MiB
2025-12-13 05:21:06.861942: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 2 Chunks of size 2359296 totalling 4.50MiB
2025-12-13 05:21:06.861954: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 1 Chunks of size 3536640 totalling 3.37MiB
2025-12-13 05:21:06.861967: I tensorflow/core/common_runtime/bfc_allocator.cc:1002] Sum Total of in-use chunks: 18.52MiB
2025-12-13 05:21:06.861978: I tensorflow/core/common_runtime/bfc_allocator.cc:1004] total_region_allocated_bytes_: 25034752 memory_limit_: 25034752 available bytes: 0 curr_region_allocation_bytes_: 33554432
2025-12-13 05:21:06.861994: I tensorflow/core/common_runtime/bfc_allocator.cc:1010] Stats: 
Limit:                    25034752
InUse:                    19420672
MaxInUse:                 19420928
NumAllocs:                     637
MaxAllocSize:              3536640

2025-12-13 05:21:06.862014: W tensorflow/core/common_runtime/bfc_allocator.cc:439] _**_******************_******_______**********____*************************************xxxx*********
2025-12-13 05:21:06.862051: W tensorflow/core/framework/op_kernel.cc:1753] OP_REQUIRES failed at cwise_ops_common.h:134 : Resource exhausted: OOM when allocating tensor with shape[3,3,256,256] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc
RPN_BBOX_STD_DEV               [0.1 0.1 0.2 0.2]
RPN_NMS_THRESHOLD              0.7
RPN_TRAIN_ANCHORS_PER_IMAGE    256
STEPS_PER_EPOCH                1000
TRAIN_ROIS_PER_IMAGE           200
USE_MINI_MASK                  True
USE_RPN_ROIS                   True
VALIDATION_STEPS               50
WEIGHT_DECAY                   0.0001


 Exception in mask_detect :  OOM when allocating tensor with shape[3,3,256,256] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc [Op:Add]
we want to redo the detection
Using TensorFlow backend.
max_time_sub_proc : 3600
erreur pendant la detection
Useless call to update_current_state
in case -12
caffe_path_current : 
About to save ! 1
Inside saveOutput : final : True verbose : False
eke 12-6-18 : saveMask need to be cleaned for new output !
ERROR : mask output needs to be a dictionnary now !
No output to save, continue without doing anything !

save missing photos in datou_result :
After save, about to update current !
 datou_cur_ids : []
 len(datou.list_steps) : 1
 output : -12
free memory after detection : 
begin to check gpu status 
inside check gpu memory
l 3610 free memory gpu now : 104

############################### TEST detect object  ################################

run mask_detect

Inside batchDatouExec : verbose : False
Catched exception !
Connect or reconnect !
# VR 17-11-17 : to create in DB !
Here we check the datou graph and we reorder steps !
 Tree builded and cycle checked, now we need to re-order the steps ! 
We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue
We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id)
 DONE and to test : checkNoCycle !
We are managing only one step so we do not consider checkConsistencyNbInputNbOutput !
We are managing only one step so we do not consider checkConsistencyTypeOutputInput !
List Step Type Loaded in datou : mask_detect
list_input_json : []
origin
BFwe have missing 0 photos in the step downloads : 
photo missing : []
try to delete the photos missing in DB
length of list_filenames : 1 ; length of list_pids : 1 ; length of list_args : 1
time to download the photos : 0.18932676315307617
About to test input to load 
we should then remove the video here, and this would fix the bug of datou_current ! 
Calling datou_exec

Inside datou_exec : verbose : False
number of steps :  1
step1:mask_detect
Sat Dec 13 05:21:08 2025
VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next
VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case
VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec 
VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec
Beginning of datou step mask_detect !
save_polygon : True

 begin detect 

begin to check gpu status 
inside check gpu memory
havn't enough memory gpu , need / 3000
l 3632 free memory gpu now : 417
wait 20 seconds
l 3637 free memory gpu now : 417  max_wait_temp :  1 max_wait :  0
 gpu_flag :  0
2025-12-13 05:21:31.484082: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2025-12-13 05:21:31.510501: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493010000 Hz
2025-12-13 05:21:31.511992: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7fda28000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2025-12-13 05:21:31.512041: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
2025-12-13 05:21:31.514526: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
2025-12-13 05:21:31.731909: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x2186f860 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2025-12-13 05:21:31.731972: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5
2025-12-13 05:21:31.732675: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: 
pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5
coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s
2025-12-13 05:21:31.733014: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2025-12-13 05:21:31.735079: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10
2025-12-13 05:21:31.737111: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10
2025-12-13 05:21:31.737416: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10
2025-12-13 05:21:31.739663: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10
2025-12-13 05:21:31.740660: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10
2025-12-13 05:21:31.745070: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2025-12-13 05:21:31.745914: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0
2025-12-13 05:21:31.745980: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2025-12-13 05:21:31.746452: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix:
2025-12-13 05:21:31.746469: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108]      0 
2025-12-13 05:21:31.746478: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0:   N 
2025-12-13 05:21:31.747222: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 40 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5)
WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead.

Inside mask_sub_process
Inside mask_detect 
About to load cache.load_thcl_param 
To do loadFromThcl(), then load ParamDescType : thcl454
 thcls : [{'id': 454, 'mtr_user_id': 31, 'name': 'mask_coco_origin', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'backgroud,person,bicycle,car,motorcycle,airplane,bus,train,truck,boat,trafficlight,firehydrant,stopsign,parkingmeter,bench,bird,cat,dog,horse,sheep,cow,elephant,bear,zebra,giraffe,backpack,umbrella,handbag,tie,suitcase,frisbee,skis,snowboard,sportsball,kite,baseballbat,baseballglove,skateboard,surfboard,tennisracket,bottle,wineglass,cup,fork,knife,spoon,bowl,banana,apple,sandwich,orange,broccoli,carrot,hotdog,pizza,donut,cake,chair,couch,pottedplant,bed,diningtable,toilet,tv,laptop,mouse,remote,keyboard,cellphone,microwave,oven,toaster,sink,refrigerator,book,clock,vase,scissors,teddybear,hairdrier,toothbrush', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 445, 'photo_desc_type': 3473, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0'}]
 thcl {'id': 454, 'mtr_user_id': 31, 'name': 'mask_coco_origin', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'backgroud,person,bicycle,car,motorcycle,airplane,bus,train,truck,boat,trafficlight,firehydrant,stopsign,parkingmeter,bench,bird,cat,dog,horse,sheep,cow,elephant,bear,zebra,giraffe,backpack,umbrella,handbag,tie,suitcase,frisbee,skis,snowboard,sportsball,kite,baseballbat,baseballglove,skateboard,surfboard,tennisracket,bottle,wineglass,cup,fork,knife,spoon,bowl,banana,apple,sandwich,orange,broccoli,carrot,hotdog,pizza,donut,cake,chair,couch,pottedplant,bed,diningtable,toilet,tv,laptop,mouse,remote,keyboard,cellphone,microwave,oven,toaster,sink,refrigerator,book,clock,vase,scissors,teddybear,hairdrier,toothbrush', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 445, 'photo_desc_type': 3473, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0'}
 Update svm_hashtag_type_desc : 3473
FOUND : 1
 Here is data_from_sql_as_vec to set the ParamDescriptorType : (3473, 'mask_coco_origin', 16384, 25088, 'mask_coco_origin', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2018, 3, 19, 10, 42, 21), datetime.datetime(2018, 3, 19, 10, 42, 21))
{'thcl': {'id': 454, 'mtr_user_id': 31, 'name': 'mask_coco_origin', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'backgroud,person,bicycle,car,motorcycle,airplane,bus,train,truck,boat,trafficlight,firehydrant,stopsign,parkingmeter,bench,bird,cat,dog,horse,sheep,cow,elephant,bear,zebra,giraffe,backpack,umbrella,handbag,tie,suitcase,frisbee,skis,snowboard,sportsball,kite,baseballbat,baseballglove,skateboard,surfboard,tennisracket,bottle,wineglass,cup,fork,knife,spoon,bowl,banana,apple,sandwich,orange,broccoli,carrot,hotdog,pizza,donut,cake,chair,couch,pottedplant,bed,diningtable,toilet,tv,laptop,mouse,remote,keyboard,cellphone,microwave,oven,toaster,sink,refrigerator,book,clock,vase,scissors,teddybear,hairdrier,toothbrush', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 445, 'photo_desc_type': 3473, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['backgroud', 'person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus', 'train', 'truck', 'boat', 'trafficlight', 'firehydrant', 'stopsign', 'parkingmeter', 'bench', 'bird', 'cat', 'dog', 'horse', 'sheep', 'cow', 'elephant', 'bear', 'zebra', 'giraffe', 'backpack', 'umbrella', 'handbag', 'tie', 'suitcase', 'frisbee', 'skis', 'snowboard', 'sportsball', 'kite', 'baseballbat', 'baseballglove', 'skateboard', 'surfboard', 'tennisracket', 'bottle', 'wineglass', 'cup', 'fork', 'knife', 'spoon', 'bowl', 'banana', 'apple', 'sandwich', 'orange', 'broccoli', 'carrot', 'hotdog', 'pizza', 'donut', 'cake', 'chair', 'couch', 'pottedplant', 'bed', 'diningtable', 'toilet', 'tv', 'laptop', 'mouse', 'remote', 'keyboard', 'cellphone', 'microwave', 'oven', 'toaster', 'sink', 'refrigerator', 'book', 'clock', 'vase', 'scissors', 'teddybear', 'hairdrier', 'toothbrush'], 'list_hashtags_csv': 'backgroud,person,bicycle,car,motorcycle,airplane,bus,train,truck,boat,trafficlight,firehydrant,stopsign,parkingmeter,bench,bird,cat,dog,horse,sheep,cow,elephant,bear,zebra,giraffe,backpack,umbrella,handbag,tie,suitcase,frisbee,skis,snowboard,sportsball,kite,baseballbat,baseballglove,skateboard,surfboard,tennisracket,bottle,wineglass,cup,fork,knife,spoon,bowl,banana,apple,sandwich,orange,broccoli,carrot,hotdog,pizza,donut,cake,chair,couch,pottedplant,bed,diningtable,toilet,tv,laptop,mouse,remote,keyboard,cellphone,microwave,oven,toaster,sink,refrigerator,book,clock,vase,scissors,teddybear,hairdrier,toothbrush', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 445, 'svm_hashtag_type_desc': 3473, 'photo_desc_type': 3473, 'pb_hashtag_id_or_classifier': 0}
list_class_names : 
['backgroud', 'person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus', 'train', 'truck', 'boat', 'trafficlight', 'firehydrant', 'stopsign', 'parkingmeter', 'bench', 'bird', 'cat', 'dog', 'horse', 'sheep', 'cow', 'elephant', 'bear', 'zebra', 'giraffe', 'backpack', 'umbrella', 'handbag', 'tie', 'suitcase', 'frisbee', 'skis', 'snowboard', 'sportsball', 'kite', 'baseballbat', 'baseballglove', 'skateboard', 'surfboard', 'tennisracket', 'bottle', 'wineglass', 'cup', 'fork', 'knife', 'spoon', 'bowl', 'banana', 'apple', 'sandwich', 'orange', 'broccoli', 'carrot', 'hotdog', 'pizza', 'donut', 'cake', 'chair', 'couch', 'pottedplant', 'bed', 'diningtable', 'toilet', 'tv', 'laptop', 'mouse', 'remote', 'keyboard', 'cellphone', 'microwave', 'oven', 'toaster', 'sink', 'refrigerator', 'book', 'clock', 'vase', 'scissors', 'teddybear', 'hairdrier', 'toothbrush']

Configurations:
BACKBONE                       resnet101
BACKBONE_SHAPES                [[160 160]
 [ 80  80]
 [ 40  40]
 [ 20  20]
 [ 10  10]]
BACKBONE_STRIDES               [4, 8, 16, 32, 64]
BATCH_SIZE                     1
BBOX_STD_DEV                   [0.1 0.1 0.2 0.2]
DETECTION_MAX_INSTANCES        100
DETECTION_MIN_CONFIDENCE       0.3
DETECTION_NMS_THRESHOLD        0.3
GPU_COUNT                      1
IMAGES_PER_GPU                 1
IMAGE_MAX_DIM                  640
IMAGE_MIN_DIM                  640
IMAGE_PADDING                  True
IMAGE_SHAPE                    [640 640   3]
LEARNING_MOMENTUM              0.9
LEARNING_RATE                  0.001
LOSS_WEIGHTS                   {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0}
MASK_POOL_SIZE                 14
MASK_SHAPE                     [28, 28]
MAX_GT_INSTANCES               100
MEAN_PIXEL                     [123.7 116.8 103.9]
MINI_MASK_SHAPE                (56, 56)
NAME                           mask_coco_origin
NUM_CLASSES                    81
POOL_SIZE                      7
POST_NMS_ROIS_INFERENCE        1000
POST_NMS_ROIS_TRAINING         2000
ROI_POSITIVE_RATIO             0.33
RPN_ANCHOR_RATIOS              [0.5, 1, 2]
RPN_ANCHOR_SCALES              (16, 32, 64, 128, 256)
RPN_ANCHOR_STRIDE              1
2025-12-13 05:21:32.458110: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: 
pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5
coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s
2025-12-13 05:21:32.458189: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2025-12-13 05:21:32.458207: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10
2025-12-13 05:21:32.458223: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10
2025-12-13 05:21:32.458238: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10
2025-12-13 05:21:32.458252: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10
2025-12-13 05:21:32.458267: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10
2025-12-13 05:21:32.458297: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2025-12-13 05:21:32.458960: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0
2025-12-13 05:21:32.459725: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: 
pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5
coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s
2025-12-13 05:21:32.459757: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2025-12-13 05:21:32.459775: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10
2025-12-13 05:21:32.459791: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10
2025-12-13 05:21:32.459807: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10
2025-12-13 05:21:32.459823: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10
2025-12-13 05:21:32.459839: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10
2025-12-13 05:21:32.459856: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2025-12-13 05:21:32.460507: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0
2025-12-13 05:21:32.460543: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix:
2025-12-13 05:21:32.460553: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108]      0 
2025-12-13 05:21:32.460561: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0:   N 
2025-12-13 05:21:32.461226: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 40 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5)
2025-12-13 05:21:43.641987: W tensorflow/core/common_runtime/bfc_allocator.cc:434] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.25MiB (rounded to 2359296)
Current allocation summary follows.
2025-12-13 05:21:43.642059: I tensorflow/core/common_runtime/bfc_allocator.cc:934] BFCAllocator dump for GPU_0_bfc
2025-12-13 05:21:43.642080: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (256): 	Total Chunks: 59, Chunks in use: 59. 14.8KiB allocated for chunks. 14.8KiB in use in bin. 8.8KiB client-requested in use in bin.
2025-12-13 05:21:43.642097: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (512): 	Total Chunks: 40, Chunks in use: 40. 20.0KiB allocated for chunks. 20.0KiB in use in bin. 20.0KiB client-requested in use in bin.
2025-12-13 05:21:43.642113: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (1024): 	Total Chunks: 76, Chunks in use: 76. 76.2KiB allocated for chunks. 76.2KiB in use in bin. 76.0KiB client-requested in use in bin.
2025-12-13 05:21:43.642128: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (2048): 	Total Chunks: 25, Chunks in use: 25. 50.0KiB allocated for chunks. 50.0KiB in use in bin. 50.0KiB client-requested in use in bin.
2025-12-13 05:21:43.642145: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (4096): 	Total Chunks: 30, Chunks in use: 30. 123.5KiB allocated for chunks. 123.5KiB in use in bin. 120.0KiB client-requested in use in bin.
2025-12-13 05:21:43.642163: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (8192): 	Total Chunks: 1, Chunks in use: 0. 15.0KiB allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2025-12-13 05:21:43.642181: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (16384): 	Total Chunks: 1, Chunks in use: 1. 16.0KiB allocated for chunks. 16.0KiB in use in bin. 16.0KiB client-requested in use in bin.
2025-12-13 05:21:43.642199: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (32768): 	Total Chunks: 1, Chunks in use: 1. 36.8KiB allocated for chunks. 36.8KiB in use in bin. 36.8KiB client-requested in use in bin.
2025-12-13 05:21:43.642216: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (65536): 	Total Chunks: 6, Chunks in use: 6. 384.0KiB allocated for chunks. 384.0KiB in use in bin. 384.0KiB client-requested in use in bin.
2025-12-13 05:21:43.642232: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (131072): 	Total Chunks: 4, Chunks in use: 4. 560.0KiB allocated for chunks. 560.0KiB in use in bin. 560.0KiB client-requested in use in bin.
2025-12-13 05:21:43.642248: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (262144): 	Total Chunks: 9, Chunks in use: 7. 2.41MiB allocated for chunks. 1.91MiB in use in bin. 1.75MiB client-requested in use in bin.
2025-12-13 05:21:43.642264: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (524288): 	Total Chunks: 8, Chunks in use: 6. 4.95MiB allocated for chunks. 3.44MiB in use in bin. 3.25MiB client-requested in use in bin.
2025-12-13 05:21:43.642289: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (1048576): 	Total Chunks: 12, Chunks in use: 10. 13.12MiB allocated for chunks. 10.88MiB in use in bin. 10.00MiB client-requested in use in bin.
2025-12-13 05:21:43.642308: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (2097152): 	Total Chunks: 8, Chunks in use: 8. 19.12MiB allocated for chunks. 19.12MiB in use in bin. 17.75MiB client-requested in use in bin.
2025-12-13 05:21:43.642340: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (4194304): 	Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2025-12-13 05:21:43.642355: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (8388608): 	Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2025-12-13 05:21:43.642371: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (16777216): 	Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2025-12-13 05:21:43.642386: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (33554432): 	Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2025-12-13 05:21:43.642400: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (67108864): 	Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2025-12-13 05:21:43.642413: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (134217728): 	Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2025-12-13 05:21:43.642426: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (268435456): 	Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2025-12-13 05:21:43.642440: I tensorflow/core/common_runtime/bfc_allocator.cc:957] Bin for 2.25MiB was 2.00MiB, Chunk State: 
2025-12-13 05:21:43.642452: I tensorflow/core/common_runtime/bfc_allocator.cc:970] Next region of size 16777216
2025-12-13 05:21:43.642470: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd842000000 of size 2359296 next 181
2025-12-13 05:21:43.642483: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd842240000 of size 2097152 next 196
2025-12-13 05:21:43.642495: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd842440000 of size 1048576 next 216
2025-12-13 05:21:43.642507: I tensorflow/core/common_runtime/bfc_allocator.cc:990] Free  at 7fd842540000 of size 1310720 next 209
2025-12-13 05:21:43.642519: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd842680000 of size 2359296 next 208
2025-12-13 05:21:43.642531: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd8428c0000 of size 1048576 next 234
2025-12-13 05:21:43.642543: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd8429c0000 of size 1310720 next 228
2025-12-13 05:21:43.642555: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd842b00000 of size 2359296 next 227
2025-12-13 05:21:43.642567: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd842d40000 of size 2883584 next 18446744073709551615
2025-12-13 05:21:43.642578: I tensorflow/core/common_runtime/bfc_allocator.cc:970] Next region of size 10354688
2025-12-13 05:21:43.642590: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd843000000 of size 2359296 next 245
2025-12-13 05:21:43.642602: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd843240000 of size 1048576 next 269
2025-12-13 05:21:43.642614: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd843340000 of size 1310720 next 263
2025-12-13 05:21:43.642626: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd843480000 of size 2359296 next 262
2025-12-13 05:21:43.642638: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd8436c0000 of size 3276800 next 18446744073709551615
2025-12-13 05:21:43.642649: I tensorflow/core/common_runtime/bfc_allocator.cc:970] Next region of size 2097152
2025-12-13 05:21:43.642661: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd844600000 of size 147456 next 55
2025-12-13 05:21:43.642673: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd844624000 of size 65536 next 78
2025-12-13 05:21:43.642694: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd844634000 of size 4096 next 191
2025-12-13 05:21:43.642707: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd844635000 of size 4096 next 192
2025-12-13 05:21:43.642719: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd844636000 of size 4096 next 193
2025-12-13 05:21:43.642731: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd844637000 of size 256 next 194
2025-12-13 05:21:43.642742: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd844637100 of size 256 next 195
2025-12-13 05:21:43.642754: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd844637200 of size 4096 next 197
2025-12-13 05:21:43.642766: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd844638200 of size 4096 next 198
2025-12-13 05:21:43.642778: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd844639200 of size 4096 next 199
2025-12-13 05:21:43.642790: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd84463a200 of size 4096 next 200
2025-12-13 05:21:43.642801: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd84463b200 of size 4096 next 201
2025-12-13 05:21:43.642814: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd84463c200 of size 1024 next 202
2025-12-13 05:21:43.642825: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd84463c600 of size 1024 next 204
2025-12-13 05:21:43.642837: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd84463ca00 of size 1024 next 205
2025-12-13 05:21:43.642849: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd84463ce00 of size 1024 next 206
2025-12-13 05:21:43.642860: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd84463d200 of size 1024 next 207
2025-12-13 05:21:43.642872: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd84463d600 of size 1024 next 210
2025-12-13 05:21:43.642884: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd84463da00 of size 1024 next 211
2025-12-13 05:21:43.642896: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd84463de00 of size 1024 next 212
2025-12-13 05:21:43.642907: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd84463e200 of size 1024 next 213
2025-12-13 05:21:43.642919: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd84463e600 of size 1024 next 214
2025-12-13 05:21:43.642931: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd84463ea00 of size 4096 next 215
2025-12-13 05:21:43.642943: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd84463fa00 of size 4096 next 217
2025-12-13 05:21:43.642955: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd844640a00 of size 4096 next 218
2025-12-13 05:21:43.642966: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd844641a00 of size 4096 next 219
2025-12-13 05:21:43.642978: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd844642a00 of size 4096 next 220
2025-12-13 05:21:43.642990: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd844643a00 of size 1024 next 221
2025-12-13 05:21:43.643001: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd844643e00 of size 1024 next 223
2025-12-13 05:21:43.643013: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd844644200 of size 1024 next 224
2025-12-13 05:21:43.643025: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd844644600 of size 1024 next 225
2025-12-13 05:21:43.643036: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd844644a00 of size 1024 next 226
2025-12-13 05:21:43.643048: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd844644e00 of size 1024 next 229
2025-12-13 05:21:43.643060: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd844645200 of size 1024 next 230
2025-12-13 05:21:43.643079: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd844645600 of size 1024 next 231
2025-12-13 05:21:43.643091: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd844645a00 of size 1024 next 232
2025-12-13 05:21:43.643103: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd844645e00 of size 1024 next 233
2025-12-13 05:21:43.643115: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd844646200 of size 7680 next 72
2025-12-13 05:21:43.643127: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd844648000 of size 147456 next 71
2025-12-13 05:21:43.643139: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd84466c000 of size 131072 next 87
2025-12-13 05:21:43.643151: I tensorflow/core/common_runtime/bfc_allocator.cc:990] Free  at 7fd84468c000 of size 262144 next 103
2025-12-13 05:21:43.643163: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd8446cc000 of size 262144 next 102
2025-12-13 05:21:43.643175: I tensorflow/core/common_runtime/bfc_allocator.cc:990] Free  at 7fd84470c000 of size 999424 next 18446744073709551615
2025-12-13 05:21:43.643186: I tensorflow/core/common_runtime/bfc_allocator.cc:970] Next region of size 4194304
2025-12-13 05:21:43.643199: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd844800000 of size 589824 next 96
2025-12-13 05:21:43.643210: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd844890000 of size 262144 next 117
2025-12-13 05:21:43.643222: I tensorflow/core/common_runtime/bfc_allocator.cc:990] Free  at 7fd8448d0000 of size 262144 next 112
2025-12-13 05:21:43.643235: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd844910000 of size 524288 next 111
2025-12-13 05:21:43.643247: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd844990000 of size 262144 next 135
2025-12-13 05:21:43.643259: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd8449d0000 of size 327680 next 124
2025-12-13 05:21:43.643271: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd844a20000 of size 589824 next 123
2025-12-13 05:21:43.643282: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd844ab0000 of size 262144 next 152
2025-12-13 05:21:43.643294: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd844af0000 of size 327680 next 142
2025-12-13 05:21:43.643306: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd844b40000 of size 786432 next 18446744073709551615
2025-12-13 05:21:43.643318: I tensorflow/core/common_runtime/bfc_allocator.cc:970] Next region of size 8388608
2025-12-13 05:21:43.643330: I tensorflow/core/common_runtime/bfc_allocator.cc:990] Free  at 7fd844c00000 of size 589824 next 160
2025-12-13 05:21:43.643341: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd844c90000 of size 589824 next 158
2025-12-13 05:21:43.643353: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd844d20000 of size 524288 next 171
2025-12-13 05:21:43.643365: I tensorflow/core/common_runtime/bfc_allocator.cc:990] Free  at 7fd844da0000 of size 1048576 next 222
2025-12-13 05:21:43.643377: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd844ea0000 of size 1048576 next 190
2025-12-13 05:21:43.643388: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd844fa0000 of size 1048576 next 189
2025-12-13 05:21:43.643400: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd8450a0000 of size 1048576 next 203
2025-12-13 05:21:43.643412: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd8451a0000 of size 1048576 next 251
2025-12-13 05:21:43.643424: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd8452a0000 of size 1441792 next 18446744073709551615
2025-12-13 05:21:43.643435: I tensorflow/core/common_runtime/bfc_allocator.cc:970] Next region of size 1048576
2025-12-13 05:21:43.643447: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e00000 of size 1280 next 1
2025-12-13 05:21:43.643468: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e00500 of size 256 next 5
2025-12-13 05:21:43.643480: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e00600 of size 256 next 8
2025-12-13 05:21:43.643492: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e00700 of size 256 next 9
2025-12-13 05:21:43.643503: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e00800 of size 256 next 10
2025-12-13 05:21:43.643515: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e00900 of size 256 next 11
2025-12-13 05:21:43.643527: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e00a00 of size 256 next 12
2025-12-13 05:21:43.643538: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e00b00 of size 256 next 13
2025-12-13 05:21:43.643550: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e00c00 of size 256 next 17
2025-12-13 05:21:43.643562: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e00d00 of size 256 next 19
2025-12-13 05:21:43.643573: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e00e00 of size 256 next 20
2025-12-13 05:21:43.643585: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e00f00 of size 256 next 21
2025-12-13 05:21:43.643597: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e01000 of size 256 next 22
2025-12-13 05:21:43.643609: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e01100 of size 256 next 24
2025-12-13 05:21:43.643620: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e01200 of size 256 next 25
2025-12-13 05:21:43.643632: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e01300 of size 256 next 23
2025-12-13 05:21:43.643643: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e01400 of size 256 next 28
2025-12-13 05:21:43.643655: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e01500 of size 256 next 29
2025-12-13 05:21:43.643667: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e01600 of size 256 next 30
2025-12-13 05:21:43.643678: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e01700 of size 256 next 31
2025-12-13 05:21:43.643690: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e01800 of size 256 next 33
2025-12-13 05:21:43.643702: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e01900 of size 256 next 34
2025-12-13 05:21:43.643714: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e01a00 of size 1024 next 32
2025-12-13 05:21:43.643725: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e01e00 of size 1024 next 37
2025-12-13 05:21:43.643737: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e02200 of size 1024 next 38
2025-12-13 05:21:43.643749: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e02600 of size 1024 next 39
2025-12-13 05:21:43.643760: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e02a00 of size 1024 next 40
2025-12-13 05:21:43.643772: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e02e00 of size 1024 next 41
2025-12-13 05:21:43.643783: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e03200 of size 1024 next 43
2025-12-13 05:21:43.643795: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e03600 of size 1024 next 44
2025-12-13 05:21:43.643807: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e03a00 of size 1024 next 45
2025-12-13 05:21:43.643819: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e03e00 of size 1024 next 46
2025-12-13 05:21:43.643830: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e04200 of size 256 next 48
2025-12-13 05:21:43.643842: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e04300 of size 256 next 49
2025-12-13 05:21:43.643862: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e04400 of size 256 next 50
2025-12-13 05:21:43.643874: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e04500 of size 256 next 51
2025-12-13 05:21:43.643886: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e04600 of size 256 next 52
2025-12-13 05:21:43.643897: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e04700 of size 256 next 53
2025-12-13 05:21:43.643909: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e04800 of size 256 next 56
2025-12-13 05:21:43.643921: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e04900 of size 256 next 57
2025-12-13 05:21:43.643932: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e04a00 of size 256 next 58
2025-12-13 05:21:43.643944: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e04b00 of size 256 next 14
2025-12-13 05:21:43.643956: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e04c00 of size 256 next 15
2025-12-13 05:21:43.643967: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e04d00 of size 256 next 16
2025-12-13 05:21:43.643979: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e04e00 of size 1024 next 60
2025-12-13 05:21:43.643991: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e05200 of size 1024 next 61
2025-12-13 05:21:43.644003: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e05600 of size 1024 next 62
2025-12-13 05:21:43.644014: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e05a00 of size 1024 next 63
2025-12-13 05:21:43.644026: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e05e00 of size 1024 next 64
2025-12-13 05:21:43.644038: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e06200 of size 256 next 66
2025-12-13 05:21:43.644052: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e06300 of size 256 next 67
2025-12-13 05:21:43.644066: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e06400 of size 256 next 68
2025-12-13 05:21:43.644080: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e06500 of size 256 next 69
2025-12-13 05:21:43.644093: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e06600 of size 256 next 70
2025-12-13 05:21:43.644106: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e06700 of size 256 next 73
2025-12-13 05:21:43.644118: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e06800 of size 256 next 74
2025-12-13 05:21:43.644130: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e06900 of size 256 next 75
2025-12-13 05:21:43.644142: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e06a00 of size 256 next 76
2025-12-13 05:21:43.644155: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e06b00 of size 256 next 77
2025-12-13 05:21:43.644167: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e06c00 of size 1024 next 79
2025-12-13 05:21:43.644179: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e07000 of size 1024 next 80
2025-12-13 05:21:43.644192: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e07400 of size 1024 next 81
2025-12-13 05:21:43.644204: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e07800 of size 1024 next 82
2025-12-13 05:21:43.644216: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e07c00 of size 1024 next 83
2025-12-13 05:21:43.644228: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e08000 of size 256 next 85
2025-12-13 05:21:43.644241: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e08100 of size 256 next 86
2025-12-13 05:21:43.644261: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e08200 of size 512 next 84
2025-12-13 05:21:43.644274: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e08400 of size 512 next 88
2025-12-13 05:21:43.644286: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e08600 of size 512 next 89
2025-12-13 05:21:43.644299: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e08800 of size 512 next 90
2025-12-13 05:21:43.644311: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e08a00 of size 512 next 91
2025-12-13 05:21:43.644323: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e08c00 of size 256 next 93
2025-12-13 05:21:43.644336: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e08d00 of size 256 next 94
2025-12-13 05:21:43.644348: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e08e00 of size 512 next 92
2025-12-13 05:21:43.644360: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e09000 of size 512 next 97
2025-12-13 05:21:43.644372: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e09200 of size 512 next 98
2025-12-13 05:21:43.644385: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e09400 of size 512 next 99
2025-12-13 05:21:43.644397: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e09600 of size 512 next 2
2025-12-13 05:21:43.644409: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e09800 of size 256 next 3
2025-12-13 05:21:43.644422: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e09900 of size 256 next 4
2025-12-13 05:21:43.644434: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e09a00 of size 16384 next 18
2025-12-13 05:21:43.644447: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e0da00 of size 256 next 100
2025-12-13 05:21:43.644459: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e0db00 of size 256 next 101
2025-12-13 05:21:43.644472: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e0dc00 of size 2048 next 104
2025-12-13 05:21:43.644484: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e0e400 of size 2048 next 105
2025-12-13 05:21:43.644496: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e0ec00 of size 2048 next 106
2025-12-13 05:21:43.644509: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e0f400 of size 2048 next 107
2025-12-13 05:21:43.644521: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e0fc00 of size 2048 next 108
2025-12-13 05:21:43.644533: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e10400 of size 256 next 109
2025-12-13 05:21:43.644546: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e10500 of size 256 next 110
2025-12-13 05:21:43.644558: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e10600 of size 2048 next 113
2025-12-13 05:21:43.644571: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e10e00 of size 2048 next 114
2025-12-13 05:21:43.644583: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e11600 of size 2048 next 115
2025-12-13 05:21:43.644595: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e11e00 of size 2048 next 116
2025-12-13 05:21:43.644607: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e12600 of size 2048 next 6
2025-12-13 05:21:43.644620: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e12e00 of size 37632 next 7
2025-12-13 05:21:43.644632: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e1c100 of size 512 next 118
2025-12-13 05:21:43.644645: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e1c300 of size 512 next 119
2025-12-13 05:21:43.644657: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e1c500 of size 512 next 120
2025-12-13 05:21:43.644678: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e1c700 of size 512 next 121
2025-12-13 05:21:43.644690: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e1c900 of size 512 next 122
2025-12-13 05:21:43.644703: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e1cb00 of size 512 next 125
2025-12-13 05:21:43.644715: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e1cd00 of size 512 next 126
2025-12-13 05:21:43.644727: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e1cf00 of size 512 next 127
2025-12-13 05:21:43.644740: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e1d100 of size 512 next 128
2025-12-13 05:21:43.644752: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e1d300 of size 512 next 129
2025-12-13 05:21:43.644764: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e1d500 of size 2048 next 130
2025-12-13 05:21:43.644776: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e1dd00 of size 2048 next 131
2025-12-13 05:21:43.644789: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e1e500 of size 2048 next 132
2025-12-13 05:21:43.644801: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e1ed00 of size 2048 next 133
2025-12-13 05:21:43.644813: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e1f500 of size 2048 next 134
2025-12-13 05:21:43.644825: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e1fd00 of size 512 next 136
2025-12-13 05:21:43.644838: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e1ff00 of size 512 next 137
2025-12-13 05:21:43.644850: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e20100 of size 512 next 138
2025-12-13 05:21:43.644862: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e20300 of size 512 next 139
2025-12-13 05:21:43.644874: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e20500 of size 512 next 140
2025-12-13 05:21:43.644887: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e20700 of size 512 next 141
2025-12-13 05:21:43.644899: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e20900 of size 512 next 143
2025-12-13 05:21:43.644911: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e20b00 of size 512 next 144
2025-12-13 05:21:43.644923: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e20d00 of size 512 next 145
2025-12-13 05:21:43.644936: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e20f00 of size 512 next 146
2025-12-13 05:21:43.644948: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e21100 of size 2048 next 147
2025-12-13 05:21:43.644960: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e21900 of size 2048 next 148
2025-12-13 05:21:43.644972: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e22100 of size 2048 next 149
2025-12-13 05:21:43.644985: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e22900 of size 2048 next 150
2025-12-13 05:21:43.644997: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e23100 of size 2048 next 151
2025-12-13 05:21:43.645009: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e23900 of size 512 next 153
2025-12-13 05:21:43.645021: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e23b00 of size 512 next 154
2025-12-13 05:21:43.645034: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e23d00 of size 512 next 155
2025-12-13 05:21:43.645046: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e23f00 of size 512 next 156
2025-12-13 05:21:43.645058: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e24100 of size 512 next 157
2025-12-13 05:21:43.645079: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e24300 of size 512 next 161
2025-12-13 05:21:43.645091: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e24500 of size 512 next 162
2025-12-13 05:21:43.645104: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e24700 of size 512 next 163
2025-12-13 05:21:43.645116: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e24900 of size 512 next 164
2025-12-13 05:21:43.645128: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e24b00 of size 512 next 165
2025-12-13 05:21:43.645140: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e24d00 of size 2048 next 166
2025-12-13 05:21:43.645153: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e25500 of size 2048 next 167
2025-12-13 05:21:43.645165: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e25d00 of size 2048 next 168
2025-12-13 05:21:43.645177: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e26500 of size 2048 next 169
2025-12-13 05:21:43.645190: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e26d00 of size 2048 next 170
2025-12-13 05:21:43.645202: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e27500 of size 1024 next 172
2025-12-13 05:21:43.645214: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e27900 of size 1024 next 173
2025-12-13 05:21:43.645227: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e27d00 of size 1024 next 174
2025-12-13 05:21:43.645239: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e28100 of size 1024 next 175
2025-12-13 05:21:43.645252: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e28500 of size 1024 next 176
2025-12-13 05:21:43.645264: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e28900 of size 256 next 178
2025-12-13 05:21:43.645276: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e28a00 of size 256 next 179
2025-12-13 05:21:43.645289: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e28b00 of size 1024 next 177
2025-12-13 05:21:43.645301: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e28f00 of size 1024 next 182
2025-12-13 05:21:43.645313: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e29300 of size 1024 next 183
2025-12-13 05:21:43.645325: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e29700 of size 1024 next 184
2025-12-13 05:21:43.645338: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e29b00 of size 1024 next 185
2025-12-13 05:21:43.645350: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e29f00 of size 256 next 187
2025-12-13 05:21:43.645362: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e2a000 of size 256 next 188
2025-12-13 05:21:43.645375: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e2a100 of size 4096 next 186
2025-12-13 05:21:43.645387: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e2b100 of size 4096 next 42
2025-12-13 05:21:43.645400: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e2c100 of size 65536 next 36
2025-12-13 05:21:43.645412: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e3c100 of size 65536 next 35
2025-12-13 05:21:43.645424: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e4c100 of size 4096 next 235
2025-12-13 05:21:43.645436: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e4d100 of size 4096 next 236
2025-12-13 05:21:43.645449: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e4e100 of size 4096 next 237
2025-12-13 05:21:43.645461: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e4f100 of size 4096 next 238
2025-12-13 05:21:43.645473: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e50100 of size 1024 next 239
2025-12-13 05:21:43.645494: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e50500 of size 1024 next 240
2025-12-13 05:21:43.645507: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e50900 of size 1024 next 241
2025-12-13 05:21:43.645519: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e50d00 of size 1024 next 242
2025-12-13 05:21:43.645532: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e51100 of size 1024 next 243
2025-12-13 05:21:43.645544: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e51500 of size 1024 next 246
2025-12-13 05:21:43.645556: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e51900 of size 1024 next 247
2025-12-13 05:21:43.645568: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e51d00 of size 1024 next 248
2025-12-13 05:21:43.645581: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e52100 of size 1024 next 249
2025-12-13 05:21:43.645593: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e52500 of size 1024 next 250
2025-12-13 05:21:43.645605: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e52900 of size 4096 next 252
2025-12-13 05:21:43.645617: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e53900 of size 4096 next 253
2025-12-13 05:21:43.645630: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e54900 of size 4096 next 254
2025-12-13 05:21:43.645642: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e55900 of size 4096 next 255
2025-12-13 05:21:43.645654: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e56900 of size 4096 next 256
2025-12-13 05:21:43.645667: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e57900 of size 1024 next 257
2025-12-13 05:21:43.645679: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e57d00 of size 1024 next 258
2025-12-13 05:21:43.645691: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e58100 of size 1024 next 259
2025-12-13 05:21:43.645703: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e58500 of size 1024 next 260
2025-12-13 05:21:43.645716: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e58900 of size 1024 next 261
2025-12-13 05:21:43.645728: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e58d00 of size 1024 next 264
2025-12-13 05:21:43.645740: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e59100 of size 1024 next 265
2025-12-13 05:21:43.645752: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e59500 of size 1024 next 266
2025-12-13 05:21:43.645765: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e59900 of size 1024 next 267
2025-12-13 05:21:43.645777: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e59d00 of size 1024 next 268
2025-12-13 05:21:43.645789: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e5a100 of size 4096 next 270
2025-12-13 05:21:43.645802: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e5b100 of size 4096 next 271
2025-12-13 05:21:43.645814: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e5c100 of size 4096 next 272
2025-12-13 05:21:43.645826: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e5d100 of size 4096 next 273
2025-12-13 05:21:43.645839: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e5e100 of size 4096 next 274
2025-12-13 05:21:43.645851: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e5f100 of size 1024 next 275
2025-12-13 05:21:43.645863: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e5f500 of size 1024 next 276
2025-12-13 05:21:43.645875: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e5f900 of size 1024 next 277
2025-12-13 05:21:43.645896: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e5fd00 of size 1024 next 278
2025-12-13 05:21:43.645908: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e60100 of size 1024 next 279
2025-12-13 05:21:43.645921: I tensorflow/core/common_runtime/bfc_allocator.cc:990] Free  at 7fd980e60500 of size 15360 next 27
2025-12-13 05:21:43.645933: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e64100 of size 147456 next 26
2025-12-13 05:21:43.645946: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e88100 of size 65536 next 47
2025-12-13 05:21:43.645958: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980e98100 of size 65536 next 59
2025-12-13 05:21:43.645970: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980ea8100 of size 65536 next 65
2025-12-13 05:21:43.645983: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7fd980eb8100 of size 294656 next 18446744073709551615
2025-12-13 05:21:43.645995: I tensorflow/core/common_runtime/bfc_allocator.cc:995]      Summary of in-use Chunks by size: 
2025-12-13 05:21:43.646011: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 59 Chunks of size 256 totalling 14.8KiB
2025-12-13 05:21:43.646025: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 40 Chunks of size 512 totalling 20.0KiB
2025-12-13 05:21:43.646039: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 75 Chunks of size 1024 totalling 75.0KiB
2025-12-13 05:21:43.646053: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 1 Chunks of size 1280 totalling 1.2KiB
2025-12-13 05:21:43.646067: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 25 Chunks of size 2048 totalling 50.0KiB
2025-12-13 05:21:43.646081: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 29 Chunks of size 4096 totalling 116.0KiB
2025-12-13 05:21:43.646094: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 1 Chunks of size 7680 totalling 7.5KiB
2025-12-13 05:21:43.646108: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 1 Chunks of size 16384 totalling 16.0KiB
2025-12-13 05:21:43.646122: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 1 Chunks of size 37632 totalling 36.8KiB
2025-12-13 05:21:43.646135: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 6 Chunks of size 65536 totalling 384.0KiB
2025-12-13 05:21:43.646149: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 1 Chunks of size 131072 totalling 128.0KiB
2025-12-13 05:21:43.646163: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 3 Chunks of size 147456 totalling 432.0KiB
2025-12-13 05:21:43.646177: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 4 Chunks of size 262144 totalling 1.00MiB
2025-12-13 05:21:43.646191: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 1 Chunks of size 294656 totalling 287.8KiB
2025-12-13 05:21:43.646204: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 2 Chunks of size 327680 totalling 640.0KiB
2025-12-13 05:21:43.646218: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 2 Chunks of size 524288 totalling 1.00MiB
2025-12-13 05:21:43.646231: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 3 Chunks of size 589824 totalling 1.69MiB
2025-12-13 05:21:43.646245: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 1 Chunks of size 786432 totalling 768.0KiB
2025-12-13 05:21:43.646259: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 7 Chunks of size 1048576 totalling 7.00MiB
2025-12-13 05:21:43.646304: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 2 Chunks of size 1310720 totalling 2.50MiB
2025-12-13 05:21:43.646324: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 1 Chunks of size 1441792 totalling 1.38MiB
2025-12-13 05:21:43.646337: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 1 Chunks of size 2097152 totalling 2.00MiB
2025-12-13 05:21:43.646352: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 5 Chunks of size 2359296 totalling 11.25MiB
2025-12-13 05:21:43.646378: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 1 Chunks of size 2883584 totalling 2.75MiB
2025-12-13 05:21:43.646393: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 1 Chunks of size 3276800 totalling 3.12MiB
2025-12-13 05:21:43.646407: I tensorflow/core/common_runtime/bfc_allocator.cc:1002] Sum Total of in-use chunks: 36.59MiB
2025-12-13 05:21:43.646419: I tensorflow/core/common_runtime/bfc_allocator.cc:1004] total_region_allocated_bytes_: 42860544 memory_limit_: 42860544 available bytes: 0 curr_region_allocation_bytes_: 67108864
2025-12-13 05:21:43.646435: I tensorflow/core/common_runtime/bfc_allocator.cc:1010] Stats: 
Limit:                    42860544
InUse:                    38372352
MaxInUse:                 38372608
NumAllocs:                     889
MaxAllocSize:              4325376

2025-12-13 05:21:43.646457: W tensorflow/core/common_runtime/bfc_allocator.cc:439] *************__***********************x***********************x***__**********_***__****************
2025-12-13 05:21:43.646498: W tensorflow/core/framework/op_kernel.cc:1753] OP_REQUIRES failed at cwise_ops_common.h:134 : Resource exhausted: OOM when allocating tensor with shape[3,3,256,256] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc
RPN_BBOX_STD_DEV               [0.1 0.1 0.2 0.2]
RPN_NMS_THRESHOLD              0.7
RPN_TRAIN_ANCHORS_PER_IMAGE    256
STEPS_PER_EPOCH                1000
TRAIN_ROIS_PER_IMAGE           200
USE_MINI_MASK                  True
USE_RPN_ROIS                   True
VALIDATION_STEPS               50
WEIGHT_DECAY                   0.0001


 Exception in mask_detect :  OOM when allocating tensor with shape[3,3,256,256] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc [Op:Add]
we want to redo the detection
Using TensorFlow backend.
max_time_sub_proc : 3600
erreur pendant la detection
Useless call to update_current_state
in case -12
caffe_path_current : 
About to save ! 1
Inside saveOutput : final : True verbose : False
eke 12-6-18 : saveMask need to be cleaned for new output !
ERROR : mask output needs to be a dictionnary now !
No output to save, continue without doing anything !

save missing photos in datou_result :
After save, about to update current !
 datou_cur_ids : []
 len(datou.list_steps) : 1
 output : -12
ERROR : 'int' object is not subscriptable
reconnect to base !
warning , we can't find thcl infos in json_data
warning , we can't find pdt infos in json_data


#&_# TEST FAILED #&_# : tests/mask_test #&_# 


Error : invalid literal for int() with base 10: "'int' object is not subscriptable"

#&_# END OF TEST #&_# : tests/mask_test #&_# 





#&_# BEGIN OF TEST : tests/datou_test #&_# 


/home/admin/workarea/git/Velours/python/tests/datou_test.py
<module 'tests.datou_test' from '/home/admin/workarea/git/Velours/python/tests/datou_test.py'>
Datou All Test
python version used : 3

############################### TEST sam  ################################
TEST SAM
Inside batchDatouExec : verbose : False
# VR 17-11-17 : to create in DB !
Here we check the datou graph and we reorder steps !
 Tree builded and cycle checked, now we need to re-order the steps ! 
We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue
We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id)
 DONE and to test : checkNoCycle !
We are managing only one step so we do not consider checkConsistencyNbInputNbOutput !
We are managing only one step so we do not consider checkConsistencyTypeOutputInput !
List Step Type Loaded in datou : sam
list_input_json : []
origin
BFwe have missing 0 photos in the step downloads : 
photo missing : []
try to delete the photos missing in DB
length of list_filenames : 1 ; length of list_pids : 1 ; length of list_args : 1
time to download the photos : 0.2119581699371338
About to test input to load 
we should then remove the video here, and this would fix the bug of datou_current ! 
WARNING : we have an input that is not a photo, we should get rid of it
Calling datou_exec

Inside datou_exec : verbose : False
number of steps :  1
step1:sam
Sat Dec 13 05:21:44 2025
VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next
VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case
VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec 
VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec

Beginning of datou step sam !
Inside sam : nb paths : 1
ERROR in datou_step_exec, will save and exit !
CUDA error: out of memory
CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
  File "/home/admin/workarea/git/Velours/python/mtr/datou/datou_lib.py", line 2339, in datou_exec
    output = datou_step_exec(sNext, args, cache, context, map_info, verbose, mtr_user_id)
  File "/home/admin/workarea/git/Velours/python/mtr/datou/datou_lib.py", line 2440, in datou_step_exec
    return lib_process.datou_step_sam(param, json_param, args, cache, context, map_info, verbose)
  File "/home/admin/workarea/git/Velours/python/mtr/datou/lib_step_exec/lib_step_process.py", line 367, in datou_step_sam
    sam.to(device=device)
  File "/home/admin/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 927, in to
    return self._apply(convert)
  File "/home/admin/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 579, in _apply
    module._apply(fn)
  File "/home/admin/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 579, in _apply
    module._apply(fn)
  File "/home/admin/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 579, in _apply
    module._apply(fn)
  File "/home/admin/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 602, in _apply
    param_applied = fn(param)
  File "/home/admin/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 925, in convert
    return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking)

[1189321094]


begin to insert list_values into mtr_datou_result : 
length of list_values in save_final : 1
time used for this insertion : 0.02034139633178711
 save_final 
ERROR in last step sam, CUDA error: out of memory
CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.

time spend for datou_step_exec : 6.852141857147217
time spend to save output : 0.021741390228271484
total time spend for step 0 : 6.873883247375488
need to delete datou_research and reload, so keep current state 1
need to delete datou_research and reload, so keep current state 1
caffe_path_current : 
About to save ! 2
After save, about to update current !
 datou_cur_ids : []
 len(datou.list_steps) : 1
 output : None
ERROR
nb objects espect :  98
nb_objects detect : 0
ERROR sam FAILED

############################### TEST frcnn  ################################
Inside batchDatouExec : verbose : False
# VR 17-11-17 : to create in DB !
Here we check the datou graph and we reorder steps !
 Tree builded and cycle checked, now we need to re-order the steps ! 
We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue
We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id)
 DONE and to test : checkNoCycle !
We are managing only one step so we do not consider checkConsistencyNbInputNbOutput !
We are managing only one step so we do not consider checkConsistencyTypeOutputInput !
List Step Type Loaded in datou : frcnn
list_input_json : []
origin
BFwe have missing 0 photos in the step downloads : 
photo missing : []
try to delete the photos missing in DB
length of list_filenames : 1 ; length of list_pids : 1 ; length of list_args : 1
time to download the photos : 0.19304347038269043
About to test input to load 
we should then remove the video here, and this would fix the bug of datou_current ! 
Calling datou_exec

Inside datou_exec : verbose : False
number of steps :  1
step1:frcnn
Sat Dec 13 05:21:51 2025
VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next
VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case
VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec 
VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec

Beginning of datou step Faster rcnn !
To loadFromThcl()
model_param file didn't exist 
model_name : detection_plaque_valcor_010622
model_type : caffe_faster_rcnn
list file need :
['caffemodel', 'test.prototxt']
file exist in s3 :
['caffemodel', 'test.prototxt']
file manque in s3 : 
[]
WARNING: Logging before InitGoogleLogging() is written to STDERR
E1213 05:21:52.984022 1361303 common.cpp:114] Cannot create Cublas handle. Cublas won't be available.
E1213 05:21:52.991108 1361303 common.cpp:121] Cannot create Curand generator. Curand won't be available.
F1213 05:21:53.015025 1361303 syncedmem.hpp:22] Check failed: error == cudaSuccess (2 vs. 0)  out of memory
*** Check failure stack trace: ***
Aborted (core dumped)
No data to report.
No data to report.
ret : 34304
 command : coverage3 html -i --omit=/usr/local/lib/python3.8/dist-packages/*,/home/admin/.local/lib/python3.8/site-packages/*,/usr/lib/python3/dist-packages/* -d htmlcov
ret : 256
 command : coverage3 report -i -m 
ret : 256
8.96user 8.27system 1:27.07elapsed 19%CPU (0avgtext+0avgdata 1471884maxresident)k
1488040inputs+3928outputs (3795major+695848minor)pagefaults 0swaps