python /home/admin/mtr/script_for_cron.py -j python_test3 -m 12 -a ' --short_python3 -v ' -s python_test3 -M 0 -S 0 -U 100,100,120 import MySQLdb succeeded Import error (python version) python version = 3 warning , we can't find thcl infos in json_data warning , we can't find pdt infos in json_data list_job_run_as_list : ['mask_detection', 'datou', 'CacheModelData_queries', 'CachePhotoData_queries', 'test_fork', 'prepare_maskdata', 'portfolio_queries', 'sla_mensuel'] python version used : 3 liste_fichiers : [('tests/mask_test', True, 'Test mask-detection ', 'mask_detection'), ('tests/datou_test', True, 'Datou All Test', 'datou', 'all'), ('mtr/database_queries/CacheModelData_queries', True, 'Test Cache Model Data', 'CacheModelData_queries'), ('tests/cache_photo_data_test', True, 'Test local_cache_photo ', 'CachePhotoData_queries'), ('mtr/mask_rcnn/prepare_maskdata', True, 'test prepare mask data', 'prepare_maskdata', 'all'), ('mtr/database_queries/portfolio_queries', True, 'test portfolio queries', 'portfolio_queries'), ('prod/memo/memo', True, 'SLA Mensuel', 'sla_mensuel', 'all')] #&_# BEGIN OF TEST : tests/mask_test #&_# /home/admin/workarea/git/Velours/python/tests/mask_test.py Test mask-detection python version used : 3 ############################### TEST memory used ################################ free memory at begining : begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 401 run mask_detect Inside batchDatouExec : verbose : False # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! List Step Type Loaded in datou : mask_detect list_input_json : [] origin BFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 1 ; length of list_pids : 1 ; length of list_args : 1 time to download the photos : 0.11640572547912598 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : False number of steps : 1 step1:mask_detect Fri Apr 4 06:35:27 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory havn't enough memory gpu , need / 3000 l 3632 free memory gpu now : 401 wait 20 seconds l 3637 free memory gpu now : 401 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 /home/admin/workarea/git/Velours/python/tests/python_tests.py:11: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses import imp 2025-04-04 06:35:50.692050: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-04-04 06:35:50.723217: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-04-04 06:35:50.726428: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f0788000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-04-04 06:35:50.726542: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-04-04 06:35:50.731790: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-04-04 06:35:51.012786: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x12c564a0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-04-04 06:35:51.012850: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-04-04 06:35:51.014110: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-04-04 06:35:51.017371: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-04-04 06:35:51.043522: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-04-04 06:35:51.064523: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-04-04 06:35:51.069024: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-04-04 06:35:51.096991: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-04-04 06:35:51.100735: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-04-04 06:35:51.155864: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-04-04 06:35:51.156954: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-04-04 06:35:51.157285: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-04-04 06:35:51.158105: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-04-04 06:35:51.158126: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-04-04 06:35:51.158138: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-04-04 06:35:51.162710: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 24 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl454 thcls : [{'id': 454, 'mtr_user_id': 31, 'name': 'mask_coco_origin', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'backgroud,person,bicycle,car,motorcycle,airplane,bus,train,truck,boat,trafficlight,firehydrant,stopsign,parkingmeter,bench,bird,cat,dog,horse,sheep,cow,elephant,bear,zebra,giraffe,backpack,umbrella,handbag,tie,suitcase,frisbee,skis,snowboard,sportsball,kite,baseballbat,baseballglove,skateboard,surfboard,tennisracket,bottle,wineglass,cup,fork,knife,spoon,bowl,banana,apple,sandwich,orange,broccoli,carrot,hotdog,pizza,donut,cake,chair,couch,pottedplant,bed,diningtable,toilet,tv,laptop,mouse,remote,keyboard,cellphone,microwave,oven,toaster,sink,refrigerator,book,clock,vase,scissors,teddybear,hairdrier,toothbrush', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 445, 'photo_desc_type': 3473, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 454, 'mtr_user_id': 31, 'name': 'mask_coco_origin', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'backgroud,person,bicycle,car,motorcycle,airplane,bus,train,truck,boat,trafficlight,firehydrant,stopsign,parkingmeter,bench,bird,cat,dog,horse,sheep,cow,elephant,bear,zebra,giraffe,backpack,umbrella,handbag,tie,suitcase,frisbee,skis,snowboard,sportsball,kite,baseballbat,baseballglove,skateboard,surfboard,tennisracket,bottle,wineglass,cup,fork,knife,spoon,bowl,banana,apple,sandwich,orange,broccoli,carrot,hotdog,pizza,donut,cake,chair,couch,pottedplant,bed,diningtable,toilet,tv,laptop,mouse,remote,keyboard,cellphone,microwave,oven,toaster,sink,refrigerator,book,clock,vase,scissors,teddybear,hairdrier,toothbrush', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 445, 'photo_desc_type': 3473, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 3473 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (3473, 'mask_coco_origin', 16384, 25088, 'mask_coco_origin', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2018, 3, 19, 10, 42, 21), datetime.datetime(2018, 3, 19, 10, 42, 21)) {'thcl': {'id': 454, 'mtr_user_id': 31, 'name': 'mask_coco_origin', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'backgroud,person,bicycle,car,motorcycle,airplane,bus,train,truck,boat,trafficlight,firehydrant,stopsign,parkingmeter,bench,bird,cat,dog,horse,sheep,cow,elephant,bear,zebra,giraffe,backpack,umbrella,handbag,tie,suitcase,frisbee,skis,snowboard,sportsball,kite,baseballbat,baseballglove,skateboard,surfboard,tennisracket,bottle,wineglass,cup,fork,knife,spoon,bowl,banana,apple,sandwich,orange,broccoli,carrot,hotdog,pizza,donut,cake,chair,couch,pottedplant,bed,diningtable,toilet,tv,laptop,mouse,remote,keyboard,cellphone,microwave,oven,toaster,sink,refrigerator,book,clock,vase,scissors,teddybear,hairdrier,toothbrush', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 445, 'photo_desc_type': 3473, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['backgroud', 'person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus', 'train', 'truck', 'boat', 'trafficlight', 'firehydrant', 'stopsign', 'parkingmeter', 'bench', 'bird', 'cat', 'dog', 'horse', 'sheep', 'cow', 'elephant', 'bear', 'zebra', 'giraffe', 'backpack', 'umbrella', 'handbag', 'tie', 'suitcase', 'frisbee', 'skis', 'snowboard', 'sportsball', 'kite', 'baseballbat', 'baseballglove', 'skateboard', 'surfboard', 'tennisracket', 'bottle', 'wineglass', 'cup', 'fork', 'knife', 'spoon', 'bowl', 'banana', 'apple', 'sandwich', 'orange', 'broccoli', 'carrot', 'hotdog', 'pizza', 'donut', 'cake', 'chair', 'couch', 'pottedplant', 'bed', 'diningtable', 'toilet', 'tv', 'laptop', 'mouse', 'remote', 'keyboard', 'cellphone', 'microwave', 'oven', 'toaster', 'sink', 'refrigerator', 'book', 'clock', 'vase', 'scissors', 'teddybear', 'hairdrier', 'toothbrush'], 'list_hashtags_csv': 'backgroud,person,bicycle,car,motorcycle,airplane,bus,train,truck,boat,trafficlight,firehydrant,stopsign,parkingmeter,bench,bird,cat,dog,horse,sheep,cow,elephant,bear,zebra,giraffe,backpack,umbrella,handbag,tie,suitcase,frisbee,skis,snowboard,sportsball,kite,baseballbat,baseballglove,skateboard,surfboard,tennisracket,bottle,wineglass,cup,fork,knife,spoon,bowl,banana,apple,sandwich,orange,broccoli,carrot,hotdog,pizza,donut,cake,chair,couch,pottedplant,bed,diningtable,toilet,tv,laptop,mouse,remote,keyboard,cellphone,microwave,oven,toaster,sink,refrigerator,book,clock,vase,scissors,teddybear,hairdrier,toothbrush', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 445, 'svm_hashtag_type_desc': 3473, 'photo_desc_type': 3473, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['backgroud', 'person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus', 'train', 'truck', 'boat', 'trafficlight', 'firehydrant', 'stopsign', 'parkingmeter', 'bench', 'bird', 'cat', 'dog', 'horse', 'sheep', 'cow', 'elephant', 'bear', 'zebra', 'giraffe', 'backpack', 'umbrella', 'handbag', 'tie', 'suitcase', 'frisbee', 'skis', 'snowboard', 'sportsball', 'kite', 'baseballbat', 'baseballglove', 'skateboard', 'surfboard', 'tennisracket', 'bottle', 'wineglass', 'cup', 'fork', 'knife', 'spoon', 'bowl', 'banana', 'apple', 'sandwich', 'orange', 'broccoli', 'carrot', 'hotdog', 'pizza', 'donut', 'cake', 'chair', 'couch', 'pottedplant', 'bed', 'diningtable', 'toilet', 'tv', 'laptop', 'mouse', 'remote', 'keyboard', 'cellphone', 'microwave', 'oven', 'toaster', 'sink', 'refrigerator', 'book', 'clock', 'vase', 'scissors', 'teddybear', 'hairdrier', 'toothbrush'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME mask_coco_origin NUM_CLASSES 81 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 2025-04-04 06:35:51.858130: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-04-04 06:35:51.858216: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-04-04 06:35:51.858243: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-04-04 06:35:51.858269: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-04-04 06:35:51.858294: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-04-04 06:35:51.858319: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-04-04 06:35:51.858357: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-04-04 06:35:51.858384: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-04-04 06:35:51.859466: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-04-04 06:35:51.864993: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-04-04 06:35:51.865036: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-04-04 06:35:51.865056: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-04-04 06:35:51.865074: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-04-04 06:35:51.865091: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-04-04 06:35:51.865109: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-04-04 06:35:51.865127: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-04-04 06:35:51.865145: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-04-04 06:35:51.865923: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-04-04 06:35:51.865961: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-04-04 06:35:51.865972: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-04-04 06:35:51.865981: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-04-04 06:35:51.866815: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 24 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) 2025-04-04 06:36:03.350588: W tensorflow/core/common_runtime/bfc_allocator.cc:434] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.25MiB (rounded to 2359296) Current allocation summary follows. 2025-04-04 06:36:03.350649: I tensorflow/core/common_runtime/bfc_allocator.cc:934] BFCAllocator dump for GPU_0_bfc 2025-04-04 06:36:03.350669: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (256): Total Chunks: 60, Chunks in use: 60. 15.0KiB allocated for chunks. 15.0KiB in use in bin. 8.8KiB client-requested in use in bin. 2025-04-04 06:36:03.350685: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (512): Total Chunks: 40, Chunks in use: 40. 20.0KiB allocated for chunks. 20.0KiB in use in bin. 20.0KiB client-requested in use in bin. 2025-04-04 06:36:03.350701: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (1024): Total Chunks: 46, Chunks in use: 46. 46.2KiB allocated for chunks. 46.2KiB in use in bin. 46.0KiB client-requested in use in bin. 2025-04-04 06:36:03.350716: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (2048): Total Chunks: 25, Chunks in use: 25. 50.0KiB allocated for chunks. 50.0KiB in use in bin. 50.0KiB client-requested in use in bin. 2025-04-04 06:36:03.350731: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (4096): Total Chunks: 15, Chunks in use: 15. 60.0KiB allocated for chunks. 60.0KiB in use in bin. 60.0KiB client-requested in use in bin. 2025-04-04 06:36:03.350745: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (8192): Total Chunks: 1, Chunks in use: 0. 12.2KiB allocated for chunks. 0B in use in bin. 0B client-requested in use in bin. 2025-04-04 06:36:03.350775: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (16384): Total Chunks: 1, Chunks in use: 1. 16.0KiB allocated for chunks. 16.0KiB in use in bin. 16.0KiB client-requested in use in bin. 2025-04-04 06:36:03.350791: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (32768): Total Chunks: 1, Chunks in use: 1. 36.8KiB allocated for chunks. 36.8KiB in use in bin. 36.8KiB client-requested in use in bin. 2025-04-04 06:36:03.350806: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (65536): Total Chunks: 7, Chunks in use: 6. 480.0KiB allocated for chunks. 384.0KiB in use in bin. 384.0KiB client-requested in use in bin. 2025-04-04 06:36:03.350822: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (131072): Total Chunks: 4, Chunks in use: 4. 560.0KiB allocated for chunks. 560.0KiB in use in bin. 560.0KiB client-requested in use in bin. 2025-04-04 06:36:03.350836: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (262144): Total Chunks: 9, Chunks in use: 7. 2.41MiB allocated for chunks. 1.91MiB in use in bin. 1.75MiB client-requested in use in bin. 2025-04-04 06:36:03.350851: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (524288): Total Chunks: 8, Chunks in use: 6. 4.95MiB allocated for chunks. 3.44MiB in use in bin. 3.25MiB client-requested in use in bin. 2025-04-04 06:36:03.350865: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (1048576): Total Chunks: 6, Chunks in use: 4. 6.25MiB allocated for chunks. 4.00MiB in use in bin. 4.00MiB client-requested in use in bin. 2025-04-04 06:36:03.350880: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (2097152): Total Chunks: 4, Chunks in use: 4. 10.00MiB allocated for chunks. 10.00MiB in use in bin. 8.75MiB client-requested in use in bin. 2025-04-04 06:36:03.350893: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (4194304): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin. 2025-04-04 06:36:03.350921: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (8388608): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin. 2025-04-04 06:36:03.350934: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (16777216): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin. 2025-04-04 06:36:03.350948: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (33554432): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin. 2025-04-04 06:36:03.350961: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (67108864): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin. 2025-04-04 06:36:03.350974: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (134217728): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin. 2025-04-04 06:36:03.350987: I tensorflow/core/common_runtime/bfc_allocator.cc:941] Bin (268435456): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin. 2025-04-04 06:36:03.351001: I tensorflow/core/common_runtime/bfc_allocator.cc:957] Bin for 2.25MiB was 2.00MiB, Chunk State: 2025-04-04 06:36:03.351013: I tensorflow/core/common_runtime/bfc_allocator.cc:970] Next region of size 2097152 2025-04-04 06:36:03.351030: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f05aa600000 of size 147456 next 55 2025-04-04 06:36:03.351042: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f05aa624000 of size 65536 next 78 2025-04-04 06:36:03.351054: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f05aa634000 of size 4096 next 191 2025-04-04 06:36:03.351066: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f05aa635000 of size 4096 next 192 2025-04-04 06:36:03.351086: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f05aa636000 of size 4096 next 193 2025-04-04 06:36:03.351099: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f05aa637000 of size 256 next 194 2025-04-04 06:36:03.351110: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f05aa637100 of size 256 next 195 2025-04-04 06:36:03.351122: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f05aa637200 of size 4096 next 197 2025-04-04 06:36:03.351133: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f05aa638200 of size 4096 next 198 2025-04-04 06:36:03.351145: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f05aa639200 of size 4096 next 199 2025-04-04 06:36:03.351157: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f05aa63a200 of size 4096 next 200 2025-04-04 06:36:03.351168: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f05aa63b200 of size 4096 next 201 2025-04-04 06:36:03.351181: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f05aa63c200 of size 1024 next 202 2025-04-04 06:36:03.351192: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f05aa63c600 of size 1024 next 204 2025-04-04 06:36:03.351204: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f05aa63ca00 of size 1024 next 205 2025-04-04 06:36:03.351215: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f05aa63ce00 of size 1024 next 206 2025-04-04 06:36:03.351227: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f05aa63d200 of size 1024 next 207 2025-04-04 06:36:03.351239: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f05aa63d600 of size 1024 next 208 2025-04-04 06:36:03.351250: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f05aa63da00 of size 1024 next 210 2025-04-04 06:36:03.351262: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f05aa63de00 of size 1024 next 211 2025-04-04 06:36:03.351273: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f05aa63e200 of size 1024 next 212 2025-04-04 06:36:03.351285: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f05aa63e600 of size 1024 next 213 2025-04-04 06:36:03.351296: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f05aa63ea00 of size 4096 next 214 2025-04-04 06:36:03.351308: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f05aa63fa00 of size 4096 next 216 2025-04-04 06:36:03.351320: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f05aa640a00 of size 4096 next 217 2025-04-04 06:36:03.351331: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f05aa641a00 of size 4096 next 218 2025-04-04 06:36:03.351343: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f05aa642a00 of size 4096 next 219 2025-04-04 06:36:03.351355: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f05aa643a00 of size 1024 next 220 2025-04-04 06:36:03.351366: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f05aa643e00 of size 1024 next 222 2025-04-04 06:36:03.351378: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f05aa644200 of size 1024 next 223 2025-04-04 06:36:03.351389: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f05aa644600 of size 1024 next 224 2025-04-04 06:36:03.351401: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f05aa644a00 of size 1024 next 225 2025-04-04 06:36:03.351413: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f05aa644e00 of size 256 next 226 2025-04-04 06:36:03.351424: I tensorflow/core/common_runtime/bfc_allocator.cc:990] Free at 7f05aa644f00 of size 12544 next 72 2025-04-04 06:36:03.351436: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f05aa648000 of size 147456 next 71 2025-04-04 06:36:03.351448: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f05aa66c000 of size 131072 next 87 2025-04-04 06:36:03.351467: I tensorflow/core/common_runtime/bfc_allocator.cc:990] Free at 7f05aa68c000 of size 262144 next 103 2025-04-04 06:36:03.351480: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f05aa6cc000 of size 262144 next 102 2025-04-04 06:36:03.351492: I tensorflow/core/common_runtime/bfc_allocator.cc:990] Free at 7f05aa70c000 of size 999424 next 18446744073709551615 2025-04-04 06:36:03.351503: I tensorflow/core/common_runtime/bfc_allocator.cc:970] Next region of size 4194304 2025-04-04 06:36:03.351515: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f05aa800000 of size 589824 next 96 2025-04-04 06:36:03.351527: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f05aa890000 of size 262144 next 117 2025-04-04 06:36:03.351538: I tensorflow/core/common_runtime/bfc_allocator.cc:990] Free at 7f05aa8d0000 of size 262144 next 112 2025-04-04 06:36:03.351550: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f05aa910000 of size 524288 next 111 2025-04-04 06:36:03.351562: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f05aa990000 of size 262144 next 135 2025-04-04 06:36:03.351574: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f05aa9d0000 of size 327680 next 124 2025-04-04 06:36:03.351586: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f05aaa20000 of size 589824 next 123 2025-04-04 06:36:03.351597: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f05aaab0000 of size 262144 next 152 2025-04-04 06:36:03.351609: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f05aaaf0000 of size 327680 next 142 2025-04-04 06:36:03.351621: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f05aab40000 of size 786432 next 18446744073709551615 2025-04-04 06:36:03.351632: I tensorflow/core/common_runtime/bfc_allocator.cc:970] Next region of size 8388608 2025-04-04 06:36:03.351644: I tensorflow/core/common_runtime/bfc_allocator.cc:990] Free at 7f05aac00000 of size 589824 next 160 2025-04-04 06:36:03.351656: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f05aac90000 of size 589824 next 158 2025-04-04 06:36:03.351667: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f05aad20000 of size 524288 next 171 2025-04-04 06:36:03.351679: I tensorflow/core/common_runtime/bfc_allocator.cc:990] Free at 7f05aada0000 of size 1048576 next 221 2025-04-04 06:36:03.351691: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f05aaea0000 of size 1048576 next 190 2025-04-04 06:36:03.351702: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f05aafa0000 of size 1048576 next 189 2025-04-04 06:36:03.351714: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f05ab0a0000 of size 1048576 next 203 2025-04-04 06:36:03.351726: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f05ab1a0000 of size 2490368 next 18446744073709551615 2025-04-04 06:36:03.351737: I tensorflow/core/common_runtime/bfc_allocator.cc:970] Next region of size 10354688 2025-04-04 06:36:03.351749: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f05ab400000 of size 2359296 next 181 2025-04-04 06:36:03.351761: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f05ab640000 of size 2097152 next 196 2025-04-04 06:36:03.351773: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f05ab840000 of size 1048576 next 215 2025-04-04 06:36:03.351784: I tensorflow/core/common_runtime/bfc_allocator.cc:990] Free at 7f05ab940000 of size 1310720 next 209 2025-04-04 06:36:03.351796: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f05aba80000 of size 3538944 next 18446744073709551615 2025-04-04 06:36:03.351808: I tensorflow/core/common_runtime/bfc_allocator.cc:970] Next region of size 1048576 2025-04-04 06:36:03.351820: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae00000 of size 1280 next 1 2025-04-04 06:36:03.351832: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae00500 of size 256 next 5 2025-04-04 06:36:03.351851: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae00600 of size 256 next 8 2025-04-04 06:36:03.351863: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae00700 of size 256 next 9 2025-04-04 06:36:03.351875: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae00800 of size 256 next 10 2025-04-04 06:36:03.351886: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae00900 of size 256 next 11 2025-04-04 06:36:03.351898: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae00a00 of size 256 next 12 2025-04-04 06:36:03.351910: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae00b00 of size 256 next 13 2025-04-04 06:36:03.351921: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae00c00 of size 256 next 17 2025-04-04 06:36:03.351933: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae00d00 of size 256 next 19 2025-04-04 06:36:03.351944: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae00e00 of size 256 next 20 2025-04-04 06:36:03.351956: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae00f00 of size 256 next 21 2025-04-04 06:36:03.351968: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae01000 of size 256 next 22 2025-04-04 06:36:03.351979: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae01100 of size 256 next 24 2025-04-04 06:36:03.351991: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae01200 of size 256 next 25 2025-04-04 06:36:03.352002: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae01300 of size 256 next 23 2025-04-04 06:36:03.352014: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae01400 of size 256 next 28 2025-04-04 06:36:03.352025: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae01500 of size 256 next 29 2025-04-04 06:36:03.352037: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae01600 of size 256 next 30 2025-04-04 06:36:03.352049: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae01700 of size 256 next 31 2025-04-04 06:36:03.352060: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae01800 of size 256 next 33 2025-04-04 06:36:03.352072: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae01900 of size 256 next 34 2025-04-04 06:36:03.352083: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae01a00 of size 1024 next 32 2025-04-04 06:36:03.352095: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae01e00 of size 1024 next 37 2025-04-04 06:36:03.352106: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae02200 of size 1024 next 38 2025-04-04 06:36:03.352118: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae02600 of size 1024 next 39 2025-04-04 06:36:03.352130: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae02a00 of size 1024 next 40 2025-04-04 06:36:03.352141: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae02e00 of size 1024 next 41 2025-04-04 06:36:03.352153: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae03200 of size 1024 next 43 2025-04-04 06:36:03.352164: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae03600 of size 1024 next 44 2025-04-04 06:36:03.352176: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae03a00 of size 1024 next 45 2025-04-04 06:36:03.352187: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae03e00 of size 1024 next 46 2025-04-04 06:36:03.352199: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae04200 of size 256 next 48 2025-04-04 06:36:03.352211: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae04300 of size 256 next 49 2025-04-04 06:36:03.352222: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae04400 of size 256 next 50 2025-04-04 06:36:03.352241: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae04500 of size 256 next 51 2025-04-04 06:36:03.352253: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae04600 of size 256 next 52 2025-04-04 06:36:03.352265: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae04700 of size 256 next 53 2025-04-04 06:36:03.352276: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae04800 of size 256 next 56 2025-04-04 06:36:03.352288: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae04900 of size 256 next 57 2025-04-04 06:36:03.352299: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae04a00 of size 256 next 58 2025-04-04 06:36:03.352311: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae04b00 of size 256 next 14 2025-04-04 06:36:03.352323: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae04c00 of size 256 next 15 2025-04-04 06:36:03.352334: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae04d00 of size 256 next 16 2025-04-04 06:36:03.352346: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae04e00 of size 1024 next 60 2025-04-04 06:36:03.352357: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae05200 of size 1024 next 61 2025-04-04 06:36:03.352369: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae05600 of size 1024 next 62 2025-04-04 06:36:03.352381: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae05a00 of size 1024 next 63 2025-04-04 06:36:03.352392: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae05e00 of size 1024 next 64 2025-04-04 06:36:03.352404: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae06200 of size 256 next 66 2025-04-04 06:36:03.352415: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae06300 of size 256 next 67 2025-04-04 06:36:03.352427: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae06400 of size 256 next 68 2025-04-04 06:36:03.352438: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae06500 of size 256 next 69 2025-04-04 06:36:03.352450: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae06600 of size 256 next 70 2025-04-04 06:36:03.352462: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae06700 of size 256 next 73 2025-04-04 06:36:03.352473: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae06800 of size 256 next 74 2025-04-04 06:36:03.352485: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae06900 of size 256 next 75 2025-04-04 06:36:03.352496: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae06a00 of size 256 next 76 2025-04-04 06:36:03.352508: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae06b00 of size 256 next 77 2025-04-04 06:36:03.352519: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae06c00 of size 1024 next 79 2025-04-04 06:36:03.352531: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae07000 of size 1024 next 80 2025-04-04 06:36:03.352543: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae07400 of size 1024 next 81 2025-04-04 06:36:03.352554: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae07800 of size 1024 next 82 2025-04-04 06:36:03.352566: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae07c00 of size 1024 next 83 2025-04-04 06:36:03.352577: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae08000 of size 256 next 85 2025-04-04 06:36:03.352589: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae08100 of size 256 next 86 2025-04-04 06:36:03.352601: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae08200 of size 512 next 84 2025-04-04 06:36:03.352620: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae08400 of size 512 next 88 2025-04-04 06:36:03.352632: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae08600 of size 512 next 89 2025-04-04 06:36:03.352643: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae08800 of size 512 next 90 2025-04-04 06:36:03.352655: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae08a00 of size 512 next 91 2025-04-04 06:36:03.352666: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae08c00 of size 256 next 93 2025-04-04 06:36:03.352678: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae08d00 of size 256 next 94 2025-04-04 06:36:03.352690: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae08e00 of size 512 next 92 2025-04-04 06:36:03.352701: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae09000 of size 512 next 97 2025-04-04 06:36:03.352713: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae09200 of size 512 next 98 2025-04-04 06:36:03.352724: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae09400 of size 512 next 99 2025-04-04 06:36:03.352736: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae09600 of size 512 next 2 2025-04-04 06:36:03.352747: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae09800 of size 256 next 3 2025-04-04 06:36:03.352759: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae09900 of size 256 next 4 2025-04-04 06:36:03.352771: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae09a00 of size 16384 next 18 2025-04-04 06:36:03.352783: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae0da00 of size 256 next 100 2025-04-04 06:36:03.352794: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae0db00 of size 256 next 101 2025-04-04 06:36:03.352806: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae0dc00 of size 2048 next 104 2025-04-04 06:36:03.352818: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae0e400 of size 2048 next 105 2025-04-04 06:36:03.352829: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae0ec00 of size 2048 next 106 2025-04-04 06:36:03.352841: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae0f400 of size 2048 next 107 2025-04-04 06:36:03.352853: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae0fc00 of size 2048 next 108 2025-04-04 06:36:03.352864: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae10400 of size 256 next 109 2025-04-04 06:36:03.352876: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae10500 of size 256 next 110 2025-04-04 06:36:03.352888: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae10600 of size 2048 next 113 2025-04-04 06:36:03.352899: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae10e00 of size 2048 next 114 2025-04-04 06:36:03.352911: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae11600 of size 2048 next 115 2025-04-04 06:36:03.352922: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae11e00 of size 2048 next 116 2025-04-04 06:36:03.352934: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae12600 of size 2048 next 6 2025-04-04 06:36:03.352946: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae12e00 of size 37632 next 7 2025-04-04 06:36:03.352957: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae1c100 of size 512 next 118 2025-04-04 06:36:03.352969: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae1c300 of size 512 next 119 2025-04-04 06:36:03.352981: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae1c500 of size 512 next 120 2025-04-04 06:36:03.352992: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae1c700 of size 512 next 121 2025-04-04 06:36:03.353011: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae1c900 of size 512 next 122 2025-04-04 06:36:03.353024: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae1cb00 of size 512 next 125 2025-04-04 06:36:03.353035: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae1cd00 of size 512 next 126 2025-04-04 06:36:03.353047: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae1cf00 of size 512 next 127 2025-04-04 06:36:03.353058: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae1d100 of size 512 next 128 2025-04-04 06:36:03.353070: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae1d300 of size 512 next 129 2025-04-04 06:36:03.353081: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae1d500 of size 2048 next 130 2025-04-04 06:36:03.353093: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae1dd00 of size 2048 next 131 2025-04-04 06:36:03.353105: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae1e500 of size 2048 next 132 2025-04-04 06:36:03.353116: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae1ed00 of size 2048 next 133 2025-04-04 06:36:03.353128: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae1f500 of size 2048 next 134 2025-04-04 06:36:03.353139: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae1fd00 of size 512 next 136 2025-04-04 06:36:03.353151: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae1ff00 of size 512 next 137 2025-04-04 06:36:03.353163: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae20100 of size 512 next 138 2025-04-04 06:36:03.353174: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae20300 of size 512 next 139 2025-04-04 06:36:03.353186: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae20500 of size 512 next 140 2025-04-04 06:36:03.353197: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae20700 of size 512 next 141 2025-04-04 06:36:03.353209: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae20900 of size 512 next 143 2025-04-04 06:36:03.353220: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae20b00 of size 512 next 144 2025-04-04 06:36:03.353232: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae20d00 of size 512 next 145 2025-04-04 06:36:03.353244: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae20f00 of size 512 next 146 2025-04-04 06:36:03.353255: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae21100 of size 2048 next 147 2025-04-04 06:36:03.353267: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae21900 of size 2048 next 148 2025-04-04 06:36:03.353278: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae22100 of size 2048 next 149 2025-04-04 06:36:03.353290: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae22900 of size 2048 next 150 2025-04-04 06:36:03.353301: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae23100 of size 2048 next 151 2025-04-04 06:36:03.353313: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae23900 of size 512 next 153 2025-04-04 06:36:03.353325: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae23b00 of size 512 next 154 2025-04-04 06:36:03.353336: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae23d00 of size 512 next 155 2025-04-04 06:36:03.353348: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae23f00 of size 512 next 156 2025-04-04 06:36:03.353359: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae24100 of size 512 next 157 2025-04-04 06:36:03.353371: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae24300 of size 512 next 161 2025-04-04 06:36:03.353391: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae24500 of size 512 next 162 2025-04-04 06:36:03.353406: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae24700 of size 512 next 163 2025-04-04 06:36:03.353418: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae24900 of size 512 next 164 2025-04-04 06:36:03.353430: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae24b00 of size 512 next 165 2025-04-04 06:36:03.353443: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae24d00 of size 2048 next 166 2025-04-04 06:36:03.353455: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae25500 of size 2048 next 167 2025-04-04 06:36:03.353467: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae25d00 of size 2048 next 168 2025-04-04 06:36:03.353479: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae26500 of size 2048 next 169 2025-04-04 06:36:03.353490: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae26d00 of size 2048 next 170 2025-04-04 06:36:03.353502: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae27500 of size 1024 next 172 2025-04-04 06:36:03.353513: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae27900 of size 1024 next 173 2025-04-04 06:36:03.353525: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae27d00 of size 1024 next 174 2025-04-04 06:36:03.353537: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae28100 of size 1024 next 175 2025-04-04 06:36:03.353548: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae28500 of size 1024 next 176 2025-04-04 06:36:03.353560: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae28900 of size 256 next 178 2025-04-04 06:36:03.353572: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae28a00 of size 256 next 179 2025-04-04 06:36:03.353583: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae28b00 of size 1024 next 177 2025-04-04 06:36:03.353595: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae28f00 of size 1024 next 182 2025-04-04 06:36:03.353607: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae29300 of size 1024 next 183 2025-04-04 06:36:03.353618: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae29700 of size 1024 next 184 2025-04-04 06:36:03.353630: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae29b00 of size 1024 next 185 2025-04-04 06:36:03.353641: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae29f00 of size 256 next 187 2025-04-04 06:36:03.353653: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae2a000 of size 256 next 188 2025-04-04 06:36:03.353665: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae2a100 of size 4096 next 186 2025-04-04 06:36:03.353676: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae2b100 of size 4096 next 42 2025-04-04 06:36:03.353688: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae2c100 of size 65536 next 36 2025-04-04 06:36:03.353700: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae3c100 of size 65536 next 35 2025-04-04 06:36:03.353711: I tensorflow/core/common_runtime/bfc_allocator.cc:990] Free at 7f06eae4c100 of size 98304 next 27 2025-04-04 06:36:03.353723: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae64100 of size 147456 next 26 2025-04-04 06:36:03.353735: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae88100 of size 65536 next 47 2025-04-04 06:36:03.353746: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eae98100 of size 65536 next 59 2025-04-04 06:36:03.353758: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eaea8100 of size 65536 next 65 2025-04-04 06:36:03.353770: I tensorflow/core/common_runtime/bfc_allocator.cc:990] InUse at 7f06eaeb8100 of size 294656 next 18446744073709551615 2025-04-04 06:36:03.353790: I tensorflow/core/common_runtime/bfc_allocator.cc:995] Summary of in-use Chunks by size: 2025-04-04 06:36:03.353805: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 60 Chunks of size 256 totalling 15.0KiB 2025-04-04 06:36:03.353819: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 40 Chunks of size 512 totalling 20.0KiB 2025-04-04 06:36:03.353832: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 45 Chunks of size 1024 totalling 45.0KiB 2025-04-04 06:36:03.353845: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 1 Chunks of size 1280 totalling 1.2KiB 2025-04-04 06:36:03.353859: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 25 Chunks of size 2048 totalling 50.0KiB 2025-04-04 06:36:03.353875: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 15 Chunks of size 4096 totalling 60.0KiB 2025-04-04 06:36:03.353888: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 1 Chunks of size 16384 totalling 16.0KiB 2025-04-04 06:36:03.353901: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 1 Chunks of size 37632 totalling 36.8KiB 2025-04-04 06:36:03.353916: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 6 Chunks of size 65536 totalling 384.0KiB 2025-04-04 06:36:03.353929: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 1 Chunks of size 131072 totalling 128.0KiB 2025-04-04 06:36:03.353942: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 3 Chunks of size 147456 totalling 432.0KiB 2025-04-04 06:36:03.353955: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 4 Chunks of size 262144 totalling 1.00MiB 2025-04-04 06:36:03.353968: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 1 Chunks of size 294656 totalling 287.8KiB 2025-04-04 06:36:03.353981: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 2 Chunks of size 327680 totalling 640.0KiB 2025-04-04 06:36:03.353993: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 2 Chunks of size 524288 totalling 1.00MiB 2025-04-04 06:36:03.354006: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 3 Chunks of size 589824 totalling 1.69MiB 2025-04-04 06:36:03.354019: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 1 Chunks of size 786432 totalling 768.0KiB 2025-04-04 06:36:03.354032: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 4 Chunks of size 1048576 totalling 4.00MiB 2025-04-04 06:36:03.354044: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 1 Chunks of size 2097152 totalling 2.00MiB 2025-04-04 06:36:03.354057: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 1 Chunks of size 2359296 totalling 2.25MiB 2025-04-04 06:36:03.354069: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 1 Chunks of size 2490368 totalling 2.38MiB 2025-04-04 06:36:03.354082: I tensorflow/core/common_runtime/bfc_allocator.cc:998] 1 Chunks of size 3538944 totalling 3.38MiB 2025-04-04 06:36:03.354095: I tensorflow/core/common_runtime/bfc_allocator.cc:1002] Sum Total of in-use chunks: 20.50MiB 2025-04-04 06:36:03.354107: I tensorflow/core/common_runtime/bfc_allocator.cc:1004] total_region_allocated_bytes_: 26083328 memory_limit_: 26083328 available bytes: 0 curr_region_allocation_bytes_: 33554432 2025-04-04 06:36:03.354123: I tensorflow/core/common_runtime/bfc_allocator.cc:1010] Stats: Limit: 26083328 InUse: 21499648 MaxInUse: 21730816 NumAllocs: 699 MaxAllocSize: 4325376 2025-04-04 06:36:03.354142: W tensorflow/core/common_runtime/bfc_allocator.cc:439] *****___****************x_*****___********************************************____**********xxx***** 2025-04-04 06:36:03.354582: W tensorflow/core/framework/op_kernel.cc:1753] OP_REQUIRES failed at cwise_ops_common.h:134 : Resource exhausted: OOM when allocating tensor with shape[3,3,256,256] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 Exception in mask_detect : OOM when allocating tensor with shape[3,3,256,256] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc [Op:Mul] we want to redo the detection Using TensorFlow backend. max_time_sub_proc : 3600 erreur pendant la detection Useless call to update_current_state in case -12 caffe_path_current : About to save ! 1 Inside saveOutput : final : True verbose : False eke 12-6-18 : saveMask need to be cleaned for new output ! ERROR : mask output needs to be a dictionnary now ! No output to save, continue without doing anything ! save missing photos in datou_result : After save, about to update current ! datou_cur_ids : [] len(datou.list_steps) : 1 output : -12 free memory after detection : begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 88 ############################### TEST detect object ################################ run mask_detect Inside batchDatouExec : verbose : False Catched exception ! Connect or reconnect ! # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! List Step Type Loaded in datou : mask_detect list_input_json : [] origin BFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 1 ; length of list_pids : 1 ; length of list_args : 1 time to download the photos : 0.17382335662841797 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : False number of steps : 1 step1:mask_detect Fri Apr 4 06:36:04 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory havn't enough memory gpu , need / 3000 l 3632 free memory gpu now : 401 wait 20 seconds l 3637 free memory gpu now : 401 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-04-04 06:36:27.721445: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-04-04 06:36:27.747231: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-04-04 06:36:27.749159: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f0788000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-04-04 06:36:27.749188: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-04-04 06:36:27.752965: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-04-04 06:36:28.016661: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x12c597d0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-04-04 06:36:28.016739: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-04-04 06:36:28.017793: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-04-04 06:36:28.018216: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-04-04 06:36:28.021156: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-04-04 06:36:28.023721: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-04-04 06:36:28.024973: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-04-04 06:36:28.028077: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-04-04 06:36:28.029718: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-04-04 06:36:28.038778: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-04-04 06:36:28.042372: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-04-04 06:36:28.042536: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-04-04 06:36:28.043351: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-04-04 06:36:28.043379: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-04-04 06:36:28.043390: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-04-04 06:36:28.044809: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 3728 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl454 thcls : [{'id': 454, 'mtr_user_id': 31, 'name': 'mask_coco_origin', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'backgroud,person,bicycle,car,motorcycle,airplane,bus,train,truck,boat,trafficlight,firehydrant,stopsign,parkingmeter,bench,bird,cat,dog,horse,sheep,cow,elephant,bear,zebra,giraffe,backpack,umbrella,handbag,tie,suitcase,frisbee,skis,snowboard,sportsball,kite,baseballbat,baseballglove,skateboard,surfboard,tennisracket,bottle,wineglass,cup,fork,knife,spoon,bowl,banana,apple,sandwich,orange,broccoli,carrot,hotdog,pizza,donut,cake,chair,couch,pottedplant,bed,diningtable,toilet,tv,laptop,mouse,remote,keyboard,cellphone,microwave,oven,toaster,sink,refrigerator,book,clock,vase,scissors,teddybear,hairdrier,toothbrush', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 445, 'photo_desc_type': 3473, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 454, 'mtr_user_id': 31, 'name': 'mask_coco_origin', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'backgroud,person,bicycle,car,motorcycle,airplane,bus,train,truck,boat,trafficlight,firehydrant,stopsign,parkingmeter,bench,bird,cat,dog,horse,sheep,cow,elephant,bear,zebra,giraffe,backpack,umbrella,handbag,tie,suitcase,frisbee,skis,snowboard,sportsball,kite,baseballbat,baseballglove,skateboard,surfboard,tennisracket,bottle,wineglass,cup,fork,knife,spoon,bowl,banana,apple,sandwich,orange,broccoli,carrot,hotdog,pizza,donut,cake,chair,couch,pottedplant,bed,diningtable,toilet,tv,laptop,mouse,remote,keyboard,cellphone,microwave,oven,toaster,sink,refrigerator,book,clock,vase,scissors,teddybear,hairdrier,toothbrush', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 445, 'photo_desc_type': 3473, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 3473 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (3473, 'mask_coco_origin', 16384, 25088, 'mask_coco_origin', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2018, 3, 19, 10, 42, 21), datetime.datetime(2018, 3, 19, 10, 42, 21)) {'thcl': {'id': 454, 'mtr_user_id': 31, 'name': 'mask_coco_origin', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'backgroud,person,bicycle,car,motorcycle,airplane,bus,train,truck,boat,trafficlight,firehydrant,stopsign,parkingmeter,bench,bird,cat,dog,horse,sheep,cow,elephant,bear,zebra,giraffe,backpack,umbrella,handbag,tie,suitcase,frisbee,skis,snowboard,sportsball,kite,baseballbat,baseballglove,skateboard,surfboard,tennisracket,bottle,wineglass,cup,fork,knife,spoon,bowl,banana,apple,sandwich,orange,broccoli,carrot,hotdog,pizza,donut,cake,chair,couch,pottedplant,bed,diningtable,toilet,tv,laptop,mouse,remote,keyboard,cellphone,microwave,oven,toaster,sink,refrigerator,book,clock,vase,scissors,teddybear,hairdrier,toothbrush', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 445, 'photo_desc_type': 3473, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['backgroud', 'person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus', 'train', 'truck', 'boat', 'trafficlight', 'firehydrant', 'stopsign', 'parkingmeter', 'bench', 'bird', 'cat', 'dog', 'horse', 'sheep', 'cow', 'elephant', 'bear', 'zebra', 'giraffe', 'backpack', 'umbrella', 'handbag', 'tie', 'suitcase', 'frisbee', 'skis', 'snowboard', 'sportsball', 'kite', 'baseballbat', 'baseballglove', 'skateboard', 'surfboard', 'tennisracket', 'bottle', 'wineglass', 'cup', 'fork', 'knife', 'spoon', 'bowl', 'banana', 'apple', 'sandwich', 'orange', 'broccoli', 'carrot', 'hotdog', 'pizza', 'donut', 'cake', 'chair', 'couch', 'pottedplant', 'bed', 'diningtable', 'toilet', 'tv', 'laptop', 'mouse', 'remote', 'keyboard', 'cellphone', 'microwave', 'oven', 'toaster', 'sink', 'refrigerator', 'book', 'clock', 'vase', 'scissors', 'teddybear', 'hairdrier', 'toothbrush'], 'list_hashtags_csv': 'backgroud,person,bicycle,car,motorcycle,airplane,bus,train,truck,boat,trafficlight,firehydrant,stopsign,parkingmeter,bench,bird,cat,dog,horse,sheep,cow,elephant,bear,zebra,giraffe,backpack,umbrella,handbag,tie,suitcase,frisbee,skis,snowboard,sportsball,kite,baseballbat,baseballglove,skateboard,surfboard,tennisracket,bottle,wineglass,cup,fork,knife,spoon,bowl,banana,apple,sandwich,orange,broccoli,carrot,hotdog,pizza,donut,cake,chair,couch,pottedplant,bed,diningtable,toilet,tv,laptop,mouse,remote,keyboard,cellphone,microwave,oven,toaster,sink,refrigerator,book,clock,vase,scissors,teddybear,hairdrier,toothbrush', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 445, 'svm_hashtag_type_desc': 3473, 'photo_desc_type': 3473, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['backgroud', 'person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus', 'train', 'truck', 'boat', 'trafficlight', 'firehydrant', 'stopsign', 'parkingmeter', 'bench', 'bird', 'cat', 'dog', 'horse', 'sheep', 'cow', 'elephant', 'bear', 'zebra', 'giraffe', 'backpack', 'umbrella', 'handbag', 'tie', 'suitcase', 'frisbee', 'skis', 'snowboard', 'sportsball', 'kite', 'baseballbat', 'baseballglove', 'skateboard', 'surfboard', 'tennisracket', 'bottle', 'wineglass', 'cup', 'fork', 'knife', 'spoon', 'bowl', 'banana', 'apple', 'sandwich', 'orange', 'broccoli', 'carrot', 'hotdog', 'pizza', 'donut', 'cake', 'chair', 'couch', 'pottedplant', 'bed', 'diningtable', 'toilet', 'tv', 'laptop', 'mouse', 'remote', 'keyboard', 'cellphone', 'microwave', 'oven', 'toaster', 'sink', 'refrigerator', 'book', 'clock', 'vase', 'scissors', 'teddybear', 'hairdrier', 'toothbrush'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME mask_coco_origin NUM_CLASSES 81 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 2025-04-04 06:36:28.621235: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-04-04 06:36:28.621334: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-04-04 06:36:28.621367: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-04-04 06:36:28.621397: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-04-04 06:36:28.621427: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-04-04 06:36:28.621455: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-04-04 06:36:28.621483: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-04-04 06:36:28.621512: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-04-04 06:36:28.623027: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-04-04 06:36:28.624341: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-04-04 06:36:28.624389: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-04-04 06:36:28.624405: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-04-04 06:36:28.624418: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-04-04 06:36:28.624432: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-04-04 06:36:28.624445: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-04-04 06:36:28.624459: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-04-04 06:36:28.624472: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-04-04 06:36:28.625299: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-04-04 06:36:28.625326: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-04-04 06:36:28.625334: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-04-04 06:36:28.625341: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-04-04 06:36:28.626216: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 3728 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : mask_coco_origin model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-04-04 06:36:35.931414: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-04-04 06:36:36.255580: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-04-04 06:36:38.161133: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.14G (3374186496 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-04-04 06:36:38.964808: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 3.29GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-04-04 06:36:38.964887: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 3.29GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. local folder : /data/models_weight/mask_coco_origin /data/models_weight/mask_coco_origin/mask_model.h5 size_local : 257557808 size in s3 : 257557808 create time local : 2021-08-09 05:27:17 create time in s3 : 2021-08-06 19:45:17 mask_model.h5 already exist and didn't need to update list_images length : 1 NEW PHOTO Processing 1 images image shape: (720, 1280, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 89) min: 0.00000 max: 1280.00000 nb d'objets trouves : 4 Detection mask done ! Trying to reset tf kernel 2998235 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 89 tf kernel not reseted sub process len(results) : 1 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 1 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 4180 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl454 Catched exception ! Connect or reconnect ! thcls : [{'id': 454, 'mtr_user_id': 31, 'name': 'mask_coco_origin', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'backgroud,person,bicycle,car,motorcycle,airplane,bus,train,truck,boat,trafficlight,firehydrant,stopsign,parkingmeter,bench,bird,cat,dog,horse,sheep,cow,elephant,bear,zebra,giraffe,backpack,umbrella,handbag,tie,suitcase,frisbee,skis,snowboard,sportsball,kite,baseballbat,baseballglove,skateboard,surfboard,tennisracket,bottle,wineglass,cup,fork,knife,spoon,bowl,banana,apple,sandwich,orange,broccoli,carrot,hotdog,pizza,donut,cake,chair,couch,pottedplant,bed,diningtable,toilet,tv,laptop,mouse,remote,keyboard,cellphone,microwave,oven,toaster,sink,refrigerator,book,clock,vase,scissors,teddybear,hairdrier,toothbrush', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 445, 'photo_desc_type': 3473, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 454, 'mtr_user_id': 31, 'name': 'mask_coco_origin', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'backgroud,person,bicycle,car,motorcycle,airplane,bus,train,truck,boat,trafficlight,firehydrant,stopsign,parkingmeter,bench,bird,cat,dog,horse,sheep,cow,elephant,bear,zebra,giraffe,backpack,umbrella,handbag,tie,suitcase,frisbee,skis,snowboard,sportsball,kite,baseballbat,baseballglove,skateboard,surfboard,tennisracket,bottle,wineglass,cup,fork,knife,spoon,bowl,banana,apple,sandwich,orange,broccoli,carrot,hotdog,pizza,donut,cake,chair,couch,pottedplant,bed,diningtable,toilet,tv,laptop,mouse,remote,keyboard,cellphone,microwave,oven,toaster,sink,refrigerator,book,clock,vase,scissors,teddybear,hairdrier,toothbrush', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 445, 'photo_desc_type': 3473, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 3473 ['backgroud', 'person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus', 'train', 'truck', 'boat', 'trafficlight', 'firehydrant', 'stopsign', 'parkingmeter', 'bench', 'bird', 'cat', 'dog', 'horse', 'sheep', 'cow', 'elephant', 'bear', 'zebra', 'giraffe', 'backpack', 'umbrella', 'handbag', 'tie', 'suitcase', 'frisbee', 'skis', 'snowboard', 'sportsball', 'kite', 'baseballbat', 'baseballglove', 'skateboard', 'surfboard', 'tennisracket', 'bottle', 'wineglass', 'cup', 'fork', 'knife', 'spoon', 'bowl', 'banana', 'apple', 'sandwich', 'orange', 'broccoli', 'carrot', 'hotdog', 'pizza', 'donut', 'cake', 'chair', 'couch', 'pottedplant', 'bed', 'diningtable', 'toilet', 'tv', 'laptop', 'mouse', 'remote', 'keyboard', 'cellphone', 'microwave', 'oven', 'toaster', 'sink', 'refrigerator', 'book', 'clock', 'vase', 'scissors', 'teddybear', 'hairdrier', 'toothbrush'] time for calcul the mask position with numpy : 0.00047707557678222656 nb_pixel_total : 16902 time to create 1 rle with old method : 0.020499229431152344 length of segment : 107 time for calcul the mask position with numpy : 0.04471302032470703 nb_pixel_total : 480748 time to create 1 rle with new method : 0.0249941349029541 length of segment : 632 time for calcul the mask position with numpy : 0.0004582405090332031 nb_pixel_total : 36587 time to create 1 rle with old method : 0.042507171630859375 length of segment : 132 time for calcul the mask position with numpy : 0.00010085105895996094 nb_pixel_total : 4794 time to create 1 rle with old method : 0.005888223648071289 length of segment : 51 time spent for convertir_results : 0.9644603729248047 time spend for datou_step_exec : 38.821720123291016 time spend to save output : 3.528594970703125e-05 total time spend for step 1 : 38.82175540924072 caffe_path_current : About to save ! 1 Inside saveOutput : final : True verbose : False eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 412 chid ids of type : 445 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++Number RLEs to save : 0 begin to insert list_values into mtr_datou_result : length of list_values in save_final : 1 time used for this insertion : 0.013938188552856445 save missing photos in datou_result : After save, about to update current ! datou_cur_ids : [] len(datou.list_steps) : 1 output : {'917855882': [[(917855882, 492601069, 445, 1092, 1280, 0, 108, 0.99883705, [(1205, 1, 58), (1165, 2, 105), (1159, 3, 113), (1149, 4, 124), (1113, 5, 161), (1100, 6, 174), (1097, 7, 177), (1095, 8, 179), (1095, 9, 179), (1095, 10, 179), (1095, 11, 179), (1095, 12, 179), (1095, 13, 179), (1095, 14, 178), (1095, 15, 178), (1095, 16, 178), (1095, 17, 178), (1095, 18, 177), (1095, 19, 177), (1095, 20, 177), (1095, 21, 177), (1095, 22, 177), (1095, 23, 178), (1095, 24, 178), (1095, 25, 178), (1095, 26, 179), (1095, 27, 179), (1095, 28, 180), (1095, 29, 181), (1095, 30, 182), (1095, 31, 183), (1095, 32, 183), (1095, 33, 184), (1095, 34, 184), (1096, 35, 183), (1096, 36, 183), (1096, 37, 184), (1097, 38, 183), (1097, 39, 183), (1097, 40, 183), (1098, 41, 182), (1098, 42, 182), (1098, 43, 182), (1099, 44, 181), (1099, 45, 181), (1099, 46, 181), (1100, 47, 180), (1100, 48, 180), (1101, 49, 179), (1101, 50, 179), (1102, 51, 178), (1102, 52, 178), (1103, 53, 177), (1103, 54, 177), (1104, 55, 176), (1104, 56, 176), (1104, 57, 176), (1104, 58, 176), (1105, 59, 175), (1105, 60, 175), (1105, 61, 175), (1105, 62, 175), (1105, 63, 175), (1106, 64, 174), (1106, 65, 174), (1106, 66, 174), (1106, 67, 174), (1106, 68, 174), (1106, 69, 174), (1106, 70, 174), (1106, 71, 174), (1106, 72, 174), (1106, 73, 174), (1107, 74, 173), (1107, 75, 173), (1107, 76, 173), (1107, 77, 173), (1107, 78, 173), (1107, 79, 173), (1108, 80, 172), (1108, 81, 172), (1109, 82, 171), (1110, 83, 170), (1110, 84, 170), (1111, 85, 169), (1112, 86, 168), (1113, 87, 166), (1114, 88, 165), (1115, 89, 164), (1117, 90, 162), (1120, 91, 159), (1138, 92, 141), (1146, 93, 133), (1154, 94, 125), (1167, 95, 112), (1177, 96, 102), (1183, 97, 95), (1185, 98, 93), (1187, 99, 90), (1188, 100, 55), (1264, 100, 12), (1190, 101, 50), (1191, 102, 46), (1194, 103, 40), (1197, 104, 34), (1202, 105, 25), (1207, 106, 16)], ['1222,106,1207,106,1206,105,1197,104,1191,102,1182,96,1176,95,1167,95,1166,94,1154,94,1153,93,1146,93,1145,92,1137,91,1120,91,1115,89,1110,84,1107,79,1106,73,1106,64,1104,55,1099,46,1095,34,1095,8,1100,6,1112,6,1113,5,1148,5,1149,4,1158,4,1165,2,1204,2,1205,1,1262,1,1269,2,1273,5,1273,13,1271,18,1271,22,1273,27,1277,31,1279,37,1279,86,1278,87,1278,96,1275,100,1264,100,1263,99,1243,99,1230,104']), (917855882, 492601069, 445, 52, 1128, 16, 668, 0.9977476, [(711, 22, 22), (925, 22, 47), (608, 23, 146), (894, 23, 103), (598, 24, 234), (850, 24, 158), (590, 25, 427), (582, 26, 444), (575, 27, 458), (569, 28, 466), (565, 29, 472), (560, 30, 480), (556, 31, 486), (550, 32, 495), (544, 33, 503), (538, 34, 512), (532, 35, 520), (527, 36, 527), (523, 37, 534), (518, 38, 541), (514, 39, 548), (510, 40, 554), (506, 41, 561), (503, 42, 566), (499, 43, 572), (496, 44, 577), (493, 45, 582), (491, 46, 585), (489, 47, 589), (487, 48, 592), (485, 49, 595), (483, 50, 598), (482, 51, 600), (481, 52, 602), (480, 53, 603), (479, 54, 605), (478, 55, 606), (476, 56, 608), (475, 57, 610), (474, 58, 611), (473, 59, 613), (472, 60, 614), (470, 61, 616), (469, 62, 618), (468, 63, 619), (466, 64, 621), (465, 65, 623), (464, 66, 624), (462, 67, 626), (461, 68, 628), (459, 69, 630), (458, 70, 631), (456, 71, 633), (455, 72, 635), (453, 73, 637), (452, 74, 638), (451, 75, 639), (449, 76, 641), (448, 77, 642), (447, 78, 643), (446, 79, 644), (445, 80, 645), (444, 81, 646), (442, 82, 648), (441, 83, 649), (440, 84, 650), (439, 85, 651), (438, 86, 652), (437, 87, 653), (436, 88, 654), (435, 89, 655), (434, 90, 656), (433, 91, 657), (432, 92, 658), (431, 93, 659), (430, 94, 660), (429, 95, 661), (428, 96, 662), (427, 97, 663), (425, 98, 665), (423, 99, 667), (421, 100, 669), (419, 101, 671), (417, 102, 673), (413, 103, 677), (410, 104, 680), (405, 105, 685), (401, 106, 689), (397, 107, 693), (392, 108, 698), (387, 109, 703), (382, 110, 708), (377, 111, 713), (373, 112, 717), (368, 113, 722), (365, 114, 725), (362, 115, 728), (358, 116, 732), (356, 117, 734), (353, 118, 737), (351, 119, 739), (348, 120, 742), (346, 121, 744), (344, 122, 746), (341, 123, 749), (338, 124, 752), (335, 125, 755), (331, 126, 759), (327, 127, 763), (323, 128, 767), (319, 129, 770), (314, 130, 775), (308, 131, 781), (303, 132, 786), (294, 133, 795), (286, 134, 803), (279, 135, 810), (273, 136, 816), (267, 137, 822), (262, 138, 827), (258, 139, 831), (255, 140, 834), (252, 141, 837), (250, 142, 839), (247, 143, 842), (245, 144, 844), (242, 145, 847), (240, 146, 849), (237, 147, 852), (233, 148, 856), (230, 149, 859), (226, 150, 863), (220, 151, 869), (213, 152, 876), (206, 153, 883), (200, 154, 889), (193, 155, 896), (187, 156, 902), (183, 157, 906), (181, 158, 908), (178, 159, 911), (176, 160, 913), (174, 161, 915), (172, 162, 917), (170, 163, 919), (168, 164, 921), (167, 165, 922), (165, 166, 924), (164, 167, 925), (162, 168, 927), (161, 169, 928), (159, 170, 930), (157, 171, 932), (155, 172, 934), (153, 173, 935), (151, 174, 937), (148, 175, 940), (146, 176, 942), (144, 177, 944), (142, 178, 946), (140, 179, 948), (139, 180, 949), (137, 181, 951), (136, 182, 952), (134, 183, 954), (133, 184, 955), (132, 185, 956), (131, 186, 957), (130, 187, 958), (129, 188, 959), (128, 189, 960), (127, 190, 960), (126, 191, 961), (126, 192, 961), (125, 193, 962), (124, 194, 963), (123, 195, 964), (122, 196, 965), (122, 197, 965), (121, 198, 966), (120, 199, 967), (119, 200, 968), (118, 201, 969), (117, 202, 970), (116, 203, 971), (114, 204, 973), (113, 205, 973), (112, 206, 974), (111, 207, 975), (109, 208, 977), (108, 209, 978), (107, 210, 979), (106, 211, 980), (105, 212, 981), (104, 213, 982), (103, 214, 983), (102, 215, 984), (101, 216, 985), (101, 217, 984), (100, 218, 985), (100, 219, 985), (99, 220, 986), (98, 221, 987), (98, 222, 987), (97, 223, 988), (97, 224, 987), (96, 225, 988), (96, 226, 988), (95, 227, 989), (95, 228, 989), (94, 229, 990), (94, 230, 990), (94, 231, 990), (93, 232, 990), (93, 233, 990), (92, 234, 991), (92, 235, 991), (92, 236, 991), (91, 237, 992), (91, 238, 991), (91, 239, 991), (91, 240, 990), (91, 241, 990), (90, 242, 991), (90, 243, 990), (90, 244, 990), (90, 245, 989), (90, 246, 989), (89, 247, 990), (89, 248, 989), (89, 249, 989), (89, 250, 988), (89, 251, 988), (88, 252, 988), (88, 253, 988), (88, 254, 987), (88, 255, 986), (88, 256, 986), (87, 257, 986), (87, 258, 985), (87, 259, 985), (87, 260, 984), (87, 261, 983), (86, 262, 983), (86, 263, 982), (86, 264, 982), (86, 265, 981), (85, 266, 981), (85, 267, 980), (85, 268, 980), (84, 269, 980), (84, 270, 979), (84, 271, 979), (84, 272, 978), (83, 273, 979), (83, 274, 978), (83, 275, 977), (82, 276, 978), (82, 277, 977), (82, 278, 977), (81, 279, 977), (81, 280, 977), (81, 281, 977), (80, 282, 977), (80, 283, 977), (80, 284, 976), (79, 285, 977), (79, 286, 976), (79, 287, 976), (78, 288, 976), (78, 289, 976), (78, 290, 975), (77, 291, 976), (77, 292, 975), (77, 293, 975), (76, 294, 975), (76, 295, 975), (76, 296, 974), (75, 297, 975), (75, 298, 974), (74, 299, 975), (74, 300, 974), (74, 301, 974), (73, 302, 974), (73, 303, 974), (72, 304, 974), (72, 305, 974), (71, 306, 974), (71, 307, 973), (71, 308, 972), (70, 309, 972), (70, 310, 971), (70, 311, 970), (70, 312, 968), (69, 313, 968), (69, 314, 966), (69, 315, 964), (69, 316, 962), (68, 317, 961), (68, 318, 959), (68, 319, 958), (68, 320, 956), (67, 321, 955), (67, 322, 954), (67, 323, 952), (67, 324, 951), (66, 325, 951), (66, 326, 950), (66, 327, 948), (66, 328, 947), (65, 329, 947), (65, 330, 946), (65, 331, 946), (65, 332, 945), (65, 333, 944), (65, 334, 942), (65, 335, 941), (65, 336, 940), (65, 337, 939), (65, 338, 938), (64, 339, 937), (64, 340, 936), (64, 341, 934), (64, 342, 932), (64, 343, 930), (64, 344, 928), (64, 345, 927), (64, 346, 925), (64, 347, 923), (64, 348, 922), (64, 349, 920), (64, 350, 919), (63, 351, 919), (63, 352, 918), (63, 353, 917), (63, 354, 916), (63, 355, 915), (63, 356, 914), (63, 357, 912), (63, 358, 911), (63, 359, 910), (63, 360, 909), (63, 361, 908), (63, 362, 906), (63, 363, 905), (63, 364, 904), (63, 365, 902), (63, 366, 901), (63, 367, 899), (63, 368, 898), (62, 369, 897), (62, 370, 895), (62, 371, 893), (62, 372, 891), (62, 373, 890), (62, 374, 888), (62, 375, 887), (62, 376, 886), (62, 377, 885), (62, 378, 884), (62, 379, 883), (63, 380, 880), (63, 381, 879), (63, 382, 878), (63, 383, 877), (63, 384, 876), (63, 385, 875), (63, 386, 874), (63, 387, 873), (63, 388, 872), (64, 389, 870), (64, 390, 869), (64, 391, 868), (64, 392, 867), (64, 393, 866), (64, 394, 864), (64, 395, 863), (65, 396, 861), (65, 397, 860), (65, 398, 859), (65, 399, 858), (65, 400, 857), (65, 401, 856), (65, 402, 854), (65, 403, 853), (65, 404, 851), (65, 405, 850), (65, 406, 848), (66, 407, 846), (66, 408, 844), (66, 409, 843), (66, 410, 842), (66, 411, 841), (66, 412, 840), (66, 413, 838), (66, 414, 837), (66, 415, 836), (66, 416, 835), (66, 417, 835), (66, 418, 834), (66, 419, 833), (67, 420, 831), (67, 421, 830), (67, 422, 829), (67, 423, 829), (67, 424, 828), (67, 425, 827), (67, 426, 826), (67, 427, 825), (67, 428, 824), (68, 429, 822), (68, 430, 820), (68, 431, 819), (68, 432, 818), (68, 433, 816), (68, 434, 815), (68, 435, 813), (68, 436, 811), (69, 437, 809), (69, 438, 807), (69, 439, 806), (69, 440, 804), (69, 441, 803), (69, 442, 802), (69, 443, 800), (70, 444, 798), (70, 445, 797), (70, 446, 796), (70, 447, 796), (71, 448, 794), (71, 449, 794), (72, 450, 792), (72, 451, 792), (73, 452, 790), (73, 453, 789), (74, 454, 788), (74, 455, 787), (75, 456, 786), (75, 457, 785), (76, 458, 784), (76, 459, 783), (77, 460, 782), (77, 461, 781), (77, 462, 781), (78, 463, 779), (78, 464, 779), (79, 465, 777), (79, 466, 777), (79, 467, 776), (80, 468, 775), (80, 469, 774), (80, 470, 774), (81, 471, 772), (81, 472, 771), (82, 473, 770), (82, 474, 769), (83, 475, 767), (83, 476, 766), (83, 477, 766), (84, 478, 764), (84, 479, 763), (85, 480, 761), (85, 481, 760), (85, 482, 759), (86, 483, 757), (86, 484, 755), (87, 485, 753), (87, 486, 752), (87, 487, 751), (88, 488, 748), (88, 489, 747), (88, 490, 746), (89, 491, 744), (89, 492, 743), (90, 493, 741), (90, 494, 741), (91, 495, 739), (91, 496, 738), (92, 497, 737), (93, 498, 735), (94, 499, 733), (94, 500, 733), (95, 501, 731), (96, 502, 729), (97, 503, 728), (98, 504, 726), (99, 505, 724), (99, 506, 724), (100, 507, 722), (101, 508, 721), (102, 509, 719), (104, 510, 717), (105, 511, 715), (106, 512, 714), (107, 513, 712), (108, 514, 711), (110, 515, 708), (111, 516, 707), (113, 517, 704), (114, 518, 703), (115, 519, 701), (117, 520, 698), (118, 521, 697), (119, 522, 695), (121, 523, 693), (122, 524, 691), (124, 525, 689), (125, 526, 687), (126, 527, 685), (128, 528, 683), (129, 529, 681), (131, 530, 678), (132, 531, 676), (134, 532, 673), (135, 533, 672), (137, 534, 669), (138, 535, 667), (140, 536, 664), (141, 537, 662), (143, 538, 659), (144, 539, 657), (146, 540, 654), (148, 541, 651), (149, 542, 649), (151, 543, 645), (153, 544, 642), (154, 545, 640), (156, 546, 638), (158, 547, 635), (159, 548, 633), (161, 549, 630), (162, 550, 628), (164, 551, 626), (166, 552, 623), (167, 553, 621), (169, 554, 618), (170, 555, 617), (171, 556, 615), (173, 557, 613), (174, 558, 611), (176, 559, 608), (177, 560, 607), (178, 561, 605), (180, 562, 603), (181, 563, 601), (183, 564, 599), (184, 565, 598), (186, 566, 595), (189, 567, 592), (192, 568, 589), (195, 569, 585), (198, 570, 582), (201, 571, 579), (204, 572, 575), (206, 573, 573), (209, 574, 569), (212, 575, 566), (215, 576, 563), (218, 577, 559), (221, 578, 556), (223, 579, 553), (226, 580, 550), (228, 581, 547), (230, 582, 545), (232, 583, 542), (234, 584, 540), (235, 585, 538), (237, 586, 536), (238, 587, 534), (240, 588, 531), (242, 589, 528), (243, 590, 526), (245, 591, 523), (247, 592, 520), (249, 593, 516), (251, 594, 513), (253, 595, 509), (256, 596, 505), (258, 597, 501), (261, 598, 497), (263, 599, 493), (267, 600, 488), (271, 601, 482), (274, 602, 478), (278, 603, 473), (281, 604, 468), (284, 605, 464), (287, 606, 460), (290, 607, 456), (292, 608, 453), (295, 609, 449), (298, 610, 445), (300, 611, 442), (303, 612, 438), (305, 613, 434), (308, 614, 430), (310, 615, 427), (312, 616, 423), (315, 617, 418), (317, 618, 415), (320, 619, 410), (322, 620, 406), (325, 621, 401), (328, 622, 395), (330, 623, 390), (333, 624, 384), (335, 625, 379), (338, 626, 374), (341, 627, 369), (345, 628, 362), (349, 629, 356), (353, 630, 350), (357, 631, 344), (360, 632, 340), (364, 633, 334), (368, 634, 328), (373, 635, 320), (378, 636, 313), (384, 637, 304), (389, 638, 295), (395, 639, 282), (401, 640, 270), (408, 641, 256), (416, 642, 240), (432, 643, 216), (449, 644, 192), (465, 645, 169), (480, 646, 148), (495, 647, 126), (511, 648, 104), (526, 649, 82), (565, 650, 9)], ['526,649,416,642,341,627,289,606,263,599,220,577,186,566,144,539,102,509,91,496,70,447,62,379,65,329,86,265,91,237,101,216,134,183,187,156,225,151,262,138,318,130,358,116,416,103,493,45,527,36,608,23,754,24,893,24,925,22,996,23,1032,27,1066,41,1082,52,1089,72,1088,172,1082,237,1045,305,1019,322,1002,338,950,373,895,423,865,446,851,473,822,505,810,528,786,554,772,586,740,612,683,638,607,649']), (917855882, 492601069, 445, 0, 438, 0, 116, 0.99194443, [(127, 1, 141), (94, 2, 206), (59, 3, 273), (338, 3, 59), (22, 4, 380), (19, 5, 386), (16, 6, 391), (15, 7, 393), (14, 8, 395), (14, 9, 396), (13, 10, 398), (12, 11, 399), (12, 12, 399), (11, 13, 401), (10, 14, 402), (11, 15, 402), (11, 16, 403), (12, 17, 403), (12, 18, 404), (12, 19, 405), (12, 20, 405), (12, 21, 405), (12, 22, 406), (12, 23, 406), (12, 24, 406), (12, 25, 407), (12, 26, 407), (12, 27, 407), (12, 28, 408), (12, 29, 408), (12, 30, 408), (12, 31, 408), (12, 32, 408), (12, 33, 408), (12, 34, 409), (12, 35, 409), (12, 36, 409), (12, 37, 409), (12, 38, 409), (12, 39, 409), (12, 40, 409), (12, 41, 409), (12, 42, 409), (12, 43, 410), (12, 44, 410), (12, 45, 409), (12, 46, 409), (12, 47, 409), (12, 48, 409), (12, 49, 409), (12, 50, 409), (12, 51, 409), (12, 52, 408), (12, 53, 408), (12, 54, 407), (12, 55, 406), (12, 56, 405), (12, 57, 404), (12, 58, 402), (11, 59, 402), (11, 60, 401), (11, 61, 401), (11, 62, 400), (11, 63, 399), (11, 64, 398), (11, 65, 398), (11, 66, 397), (11, 67, 396), (11, 68, 395), (11, 69, 395), (11, 70, 394), (11, 71, 394), (11, 72, 394), (11, 73, 393), (11, 74, 393), (11, 75, 393), (11, 76, 393), (11, 77, 393), (11, 78, 393), (11, 79, 392), (11, 80, 392), (10, 81, 393), (10, 82, 394), (10, 83, 394), (9, 84, 396), (9, 85, 263), (284, 85, 121), (9, 86, 75), (96, 86, 30), (141, 86, 120), (292, 86, 112), (9, 87, 71), (151, 87, 105), (294, 87, 110), (8, 88, 67), (160, 88, 92), (295, 88, 108), (8, 89, 63), (175, 89, 74), (296, 89, 107), (7, 90, 61), (204, 90, 41), (297, 90, 105), (7, 91, 57), (298, 91, 104), (7, 92, 53), (299, 92, 103), (6, 93, 50), (300, 93, 101), (7, 94, 46), (303, 94, 96), (7, 95, 44), (305, 95, 93), (7, 96, 42), (308, 96, 88), (7, 97, 40), (310, 97, 85), (7, 98, 38), (312, 98, 82), (8, 99, 34), (314, 99, 79), (8, 100, 32), (316, 100, 75), (8, 101, 29), (319, 101, 71), (13, 102, 19), (324, 102, 63), (20, 103, 6), (331, 103, 51), (337, 104, 37), (344, 105, 22), (351, 106, 2)], ['344,105,330,102,319,101,307,95,300,93,283,84,261,85,244,90,204,90,203,89,175,89,160,88,140,85,125,86,83,86,56,92,36,101,25,103,8,101,6,93,11,80,11,59,12,58,12,17,10,14,16,6,22,4,58,4,59,3,93,3,94,2,126,2,127,1,267,1,268,2,331,3,396,3,406,6,416,19,420,34,420,51,411,60,404,70,402,81,404,85,401,92,392,99,386,102,374,103,365,105']), (917855882, 492601069, 445, 390, 550, 0, 54, 0.9392603, [(414, 0, 7), (441, 0, 60), (508, 0, 28), (402, 1, 142), (401, 2, 146), (402, 3, 145), (404, 4, 143), (406, 5, 140), (408, 6, 137), (410, 7, 134), (411, 8, 132), (412, 9, 130), (413, 10, 127), (414, 11, 125), (415, 12, 123), (415, 13, 122), (416, 14, 120), (417, 15, 117), (417, 16, 116), (418, 17, 114), (418, 18, 113), (418, 19, 111), (418, 20, 109), (419, 21, 107), (419, 22, 105), (419, 23, 103), (419, 24, 102), (419, 25, 100), (420, 26, 97), (420, 27, 95), (420, 28, 94), (421, 29, 91), (421, 30, 90), (422, 31, 88), (422, 32, 88), (422, 33, 87), (423, 34, 84), (423, 35, 82), (423, 36, 81), (424, 37, 79), (424, 38, 77), (424, 39, 75), (424, 40, 73), (424, 41, 71), (425, 42, 67), (425, 43, 66), (426, 44, 62), (426, 45, 6), (433, 45, 52), (443, 46, 30), (450, 47, 1)], ['450,47,449,46,443,46,442,45,426,45,424,41,424,37,423,36,422,31,419,25,419,21,418,20,418,17,417,15,409,6,402,3,402,1,413,1,414,0,420,0,421,1,440,1,441,0,500,0,501,1,507,1,508,0,535,0,536,1,543,1,546,2,546,4,542,8,530,18,527,19,525,21,522,22,520,24,512,28,508,33,505,34,502,37,494,41,492,41,490,43,488,43,484,45,473,45,472,46,451,46'])], 'temp/1743741364_2995536_917855882_da0fa7b7e6b5b551fe26c0ba8713276d.jpg']} ############################### TEST POLYGON ################################ Inside batchDatouExec : verbose : False # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! List Step Type Loaded in datou : mask_detect list_input_json : [] origin BFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 1 ; length of list_pids : 1 ; length of list_args : 1 time to download the photos : 0.16271257400512695 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : False number of steps : 1 step1:mask_detect Fri Apr 4 06:36:55 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 3961 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-04-04 06:36:57.957860: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-04-04 06:36:57.987269: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-04-04 06:36:57.989271: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f078c000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-04-04 06:36:57.989332: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-04-04 06:36:57.993141: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-04-04 06:36:58.179698: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x1394d570 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-04-04 06:36:58.179746: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-04-04 06:36:58.180672: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-04-04 06:36:58.181077: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-04-04 06:36:58.184047: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-04-04 06:36:58.186454: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-04-04 06:36:58.186830: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-04-04 06:36:58.189505: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-04-04 06:36:58.190787: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-04-04 06:36:58.196128: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-04-04 06:36:58.197093: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-04-04 06:36:58.197167: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-04-04 06:36:58.197670: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-04-04 06:36:58.197686: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-04-04 06:36:58.197695: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-04-04 06:36:58.198504: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 1484 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-04-04 06:36:58.404054: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-04-04 06:36:58.404209: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-04-04 06:36:58.404238: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-04-04 06:36:58.404259: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-04-04 06:36:58.404279: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-04-04 06:36:58.404298: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-04-04 06:36:58.404317: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-04-04 06:36:58.404337: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-04-04 06:36:58.405083: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-04-04 06:36:58.405951: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-04-04 06:36:58.405986: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-04-04 06:36:58.406006: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-04-04 06:36:58.406022: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-04-04 06:36:58.406039: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-04-04 06:36:58.406055: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-04-04 06:36:58.406071: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-04-04 06:36:58.406087: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-04-04 06:36:58.406812: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-04-04 06:36:58.406847: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-04-04 06:36:58.406856: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-04-04 06:36:58.406864: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-04-04 06:36:58.407646: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 1484 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (3473, 'mask_coco_origin', 16384, 25088, 'mask_coco_origin', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2018, 3, 19, 10, 42, 21), datetime.datetime(2018, 3, 19, 10, 42, 21)) {'thcl': {'id': 454, 'mtr_user_id': 31, 'name': 'mask_coco_origin', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'backgroud,person,bicycle,car,motorcycle,airplane,bus,train,truck,boat,trafficlight,firehydrant,stopsign,parkingmeter,bench,bird,cat,dog,horse,sheep,cow,elephant,bear,zebra,giraffe,backpack,umbrella,handbag,tie,suitcase,frisbee,skis,snowboard,sportsball,kite,baseballbat,baseballglove,skateboard,surfboard,tennisracket,bottle,wineglass,cup,fork,knife,spoon,bowl,banana,apple,sandwich,orange,broccoli,carrot,hotdog,pizza,donut,cake,chair,couch,pottedplant,bed,diningtable,toilet,tv,laptop,mouse,remote,keyboard,cellphone,microwave,oven,toaster,sink,refrigerator,book,clock,vase,scissors,teddybear,hairdrier,toothbrush', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 445, 'photo_desc_type': 3473, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['backgroud', 'person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus', 'train', 'truck', 'boat', 'trafficlight', 'firehydrant', 'stopsign', 'parkingmeter', 'bench', 'bird', 'cat', 'dog', 'horse', 'sheep', 'cow', 'elephant', 'bear', 'zebra', 'giraffe', 'backpack', 'umbrella', 'handbag', 'tie', 'suitcase', 'frisbee', 'skis', 'snowboard', 'sportsball', 'kite', 'baseballbat', 'baseballglove', 'skateboard', 'surfboard', 'tennisracket', 'bottle', 'wineglass', 'cup', 'fork', 'knife', 'spoon', 'bowl', 'banana', 'apple', 'sandwich', 'orange', 'broccoli', 'carrot', 'hotdog', 'pizza', 'donut', 'cake', 'chair', 'couch', 'pottedplant', 'bed', 'diningtable', 'toilet', 'tv', 'laptop', 'mouse', 'remote', 'keyboard', 'cellphone', 'microwave', 'oven', 'toaster', 'sink', 'refrigerator', 'book', 'clock', 'vase', 'scissors', 'teddybear', 'hairdrier', 'toothbrush'], 'list_hashtags_csv': 'backgroud,person,bicycle,car,motorcycle,airplane,bus,train,truck,boat,trafficlight,firehydrant,stopsign,parkingmeter,bench,bird,cat,dog,horse,sheep,cow,elephant,bear,zebra,giraffe,backpack,umbrella,handbag,tie,suitcase,frisbee,skis,snowboard,sportsball,kite,baseballbat,baseballglove,skateboard,surfboard,tennisracket,bottle,wineglass,cup,fork,knife,spoon,bowl,banana,apple,sandwich,orange,broccoli,carrot,hotdog,pizza,donut,cake,chair,couch,pottedplant,bed,diningtable,toilet,tv,laptop,mouse,remote,keyboard,cellphone,microwave,oven,toaster,sink,refrigerator,book,clock,vase,scissors,teddybear,hairdrier,toothbrush', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 445, 'svm_hashtag_type_desc': 3473, 'photo_desc_type': 3473, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['backgroud', 'person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus', 'train', 'truck', 'boat', 'trafficlight', 'firehydrant', 'stopsign', 'parkingmeter', 'bench', 'bird', 'cat', 'dog', 'horse', 'sheep', 'cow', 'elephant', 'bear', 'zebra', 'giraffe', 'backpack', 'umbrella', 'handbag', 'tie', 'suitcase', 'frisbee', 'skis', 'snowboard', 'sportsball', 'kite', 'baseballbat', 'baseballglove', 'skateboard', 'surfboard', 'tennisracket', 'bottle', 'wineglass', 'cup', 'fork', 'knife', 'spoon', 'bowl', 'banana', 'apple', 'sandwich', 'orange', 'broccoli', 'carrot', 'hotdog', 'pizza', 'donut', 'cake', 'chair', 'couch', 'pottedplant', 'bed', 'diningtable', 'toilet', 'tv', 'laptop', 'mouse', 'remote', 'keyboard', 'cellphone', 'microwave', 'oven', 'toaster', 'sink', 'refrigerator', 'book', 'clock', 'vase', 'scissors', 'teddybear', 'hairdrier', 'toothbrush'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME mask_coco_origin NUM_CLASSES 81 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : mask_coco_origin model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-04-04 06:37:08.221404: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-04-04 06:37:08.407739: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-04-04 06:37:09.784574: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-04-04 06:37:09.784641: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-04-04 06:37:09.791141: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-04-04 06:37:09.791164: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-04-04 06:37:09.840958: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-04-04 06:37:09.841014: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-04-04 06:37:09.882939: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.09GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-04-04 06:37:09.882971: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.09GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-04-04 06:37:09.934225: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.15GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-04-04 06:37:09.934260: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.15GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-04-04 06:37:09.965763: W tensorflow/core/common_runtime/bfc_allocator.cc:311] Garbage collection: deallocate free memory regions (i.e., allocations) so that we can re-allocate a larger region to avoid OOM due to memory fragmentation. If you see this message frequently, you are running near the threshold of the available device memory and re-allocation may incur great performance overhead. You may try smaller batch sizes to observe the performance impact. Set TF_ENABLE_GPU_GARBAGE_COLLECTION=false if you'd like to disable this feature. local folder : /data/models_weight/mask_coco_origin /data/models_weight/mask_coco_origin/mask_model.h5 size_local : 257557808 size in s3 : 257557808 create time local : 2021-08-09 05:27:17 create time in s3 : 2021-08-06 19:45:17 mask_model.h5 already exist and didn't need to update list_images length : 1 NEW PHOTO Processing 1 images image shape: (2448, 2448, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 89) min: 0.00000 max: 2448.00000 nb d'objets trouves : 1 Detection mask done ! Trying to reset tf kernel 2999825 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 1792 tf kernel not reseted sub process len(results) : 1 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 1 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 3959 list_Values should be empty [] ['backgroud', 'person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus', 'train', 'truck', 'boat', 'trafficlight', 'firehydrant', 'stopsign', 'parkingmeter', 'bench', 'bird', 'cat', 'dog', 'horse', 'sheep', 'cow', 'elephant', 'bear', 'zebra', 'giraffe', 'backpack', 'umbrella', 'handbag', 'tie', 'suitcase', 'frisbee', 'skis', 'snowboard', 'sportsball', 'kite', 'baseballbat', 'baseballglove', 'skateboard', 'surfboard', 'tennisracket', 'bottle', 'wineglass', 'cup', 'fork', 'knife', 'spoon', 'bowl', 'banana', 'apple', 'sandwich', 'orange', 'broccoli', 'carrot', 'hotdog', 'pizza', 'donut', 'cake', 'chair', 'couch', 'pottedplant', 'bed', 'diningtable', 'toilet', 'tv', 'laptop', 'mouse', 'remote', 'keyboard', 'cellphone', 'microwave', 'oven', 'toaster', 'sink', 'refrigerator', 'book', 'clock', 'vase', 'scissors', 'teddybear', 'hairdrier', 'toothbrush'] time for calcul the mask position with numpy : 0.2594759464263916 nb_pixel_total : 3697371 time to create 1 rle with new method : 0.33108019828796387 length of segment : 2051 time spent for convertir_results : 1.6518561840057373 time spend for datou_step_exec : 20.376511335372925 time spend to save output : 3.9577484130859375e-05 total time spend for step 1 : 20.376550912857056 caffe_path_current : About to save ! 1 Inside saveOutput : final : True verbose : False eke 12-6-18 : saveMask need to be cleaned for new output ! Catched exception ! Connect or reconnect ! Number saved : None batch 1 Loaded 721 chid ids of type : 445 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++Number RLEs to save : 0 begin to insert list_values into mtr_datou_result : length of list_values in save_final : 1 time used for this insertion : 0.01969599723815918 save missing photos in datou_result : After save, about to update current ! datou_cur_ids : [] len(datou.list_steps) : 1 output : {'917877156': [[(917877156, 492601069, 445, 11, 2268, 107, 2256, 0.98633724, [(638, 113, 267), (1204, 113, 135), (530, 114, 463), (1068, 114, 359), (521, 115, 915), (512, 116, 932), (503, 117, 949), (495, 118, 965), (487, 119, 980), (479, 120, 996), (472, 121, 1010), (465, 122, 1023), (458, 123, 1037), (451, 124, 1050), (445, 125, 1067), (439, 126, 1086), (433, 127, 1104), (427, 128, 1121), (421, 129, 1139), (416, 130, 1155), (410, 131, 1171), (405, 132, 1181), (400, 133, 1190), (396, 134, 1198), (391, 135, 1207), (386, 136, 1216), (382, 137, 1223), (378, 138, 1231), (374, 139, 1238), (372, 140, 1244), (370, 141, 1249), (368, 142, 1254), (367, 143, 1258), (365, 144, 1263), (364, 145, 1267), (362, 146, 1272), (361, 147, 1276), (359, 148, 1281), (358, 149, 1285), (356, 150, 1290), (354, 151, 1295), (353, 152, 1300), (351, 153, 1305), (349, 154, 1311), (347, 155, 1316), (345, 156, 1322), (343, 157, 1328), (341, 158, 1334), (339, 159, 1340), (337, 160, 1346), (335, 161, 1352), (333, 162, 1358), (331, 163, 1365), (329, 164, 1371), (326, 165, 1379), (324, 166, 1386), (321, 167, 1394), (319, 168, 1402), (316, 169, 1410), (313, 170, 1419), (311, 171, 1427), (308, 172, 1436), (305, 173, 1445), (302, 174, 1455), (299, 175, 1465), (295, 176, 1476), (292, 177, 1486), (288, 178, 1498), (285, 179, 1508), (281, 180, 1520), (278, 181, 1531), (275, 182, 1542), (271, 183, 1554), (268, 184, 1563), (265, 185, 1571), (262, 186, 1579), (259, 187, 1587), (256, 188, 1595), (252, 189, 1604), (249, 190, 1611), (247, 191, 1618), (244, 192, 1625), (241, 193, 1632), (238, 194, 1639), (235, 195, 1646), (232, 196, 1653), (230, 197, 1659), (227, 198, 1665), (224, 199, 1672), (222, 200, 1677), (219, 201, 1684), (217, 202, 1689), (214, 203, 1693), (212, 204, 1697), (210, 205, 1700), (209, 206, 1703), (207, 207, 1706), (206, 208, 1708), (204, 209, 1712), (203, 210, 1714), (201, 211, 1717), (200, 212, 1719), (199, 213, 1721), (197, 214, 1724), (196, 215, 1726), (195, 216, 1728), (194, 217, 1730), (193, 218, 1732), (192, 219, 1734), (191, 220, 1736), (190, 221, 1738), (189, 222, 1740), (188, 223, 1742), (187, 224, 1744), (186, 225, 1746), (185, 226, 1747), (184, 227, 1749), (183, 228, 1751), (182, 229, 1753), (180, 230, 1756), (179, 231, 1758), (178, 232, 1760), (177, 233, 1761), (176, 234, 1763), (175, 235, 1765), (174, 236, 1767), (172, 237, 1770), (171, 238, 1772), (170, 239, 1774), (169, 240, 1776), (167, 241, 1780), (166, 242, 1782), (165, 243, 1784), (163, 244, 1787), (162, 245, 1789), (161, 246, 1791), (159, 247, 1794), (158, 248, 1797), (156, 249, 1800), (155, 250, 1802), (153, 251, 1806), (152, 252, 1808), (150, 253, 1811), (148, 254, 1815), (147, 255, 1817), (145, 256, 1821), (143, 257, 1824), (141, 258, 1828), (140, 259, 1830), (138, 260, 1834), (136, 261, 1838), (134, 262, 1841), (132, 263, 1845), (130, 264, 1849), (129, 265, 1852), (128, 266, 1855), (126, 267, 1859), (125, 268, 1862), (124, 269, 1864), (123, 270, 1867), (121, 271, 1870), (120, 272, 1873), (119, 273, 1875), (118, 274, 1877), (117, 275, 1880), (116, 276, 1882), (115, 277, 1884), (114, 278, 1886), (113, 279, 1888), (112, 280, 1890), (111, 281, 1892), (111, 282, 1893), (110, 283, 1895), (109, 284, 1897), (108, 285, 1899), (107, 286, 1901), (107, 287, 1902), (106, 288, 1904), (105, 289, 1906), (105, 290, 1907), (104, 291, 1909), (103, 292, 1910), (102, 293, 1912), (102, 294, 1913), (101, 295, 1915), (101, 296, 1915), (100, 297, 1917), (99, 298, 1919), (99, 299, 1919), (99, 300, 1920), (98, 301, 1921), (98, 302, 1922), (98, 303, 1922), (98, 304, 1923), (97, 305, 1924), (97, 306, 1925), (97, 307, 1925), (96, 308, 1927), (96, 309, 1927), (96, 310, 1928), (95, 311, 1929), (95, 312, 1930), (95, 313, 1930), (95, 314, 1931), (94, 315, 1932), (94, 316, 1933), (94, 317, 1933), (93, 318, 1935), (93, 319, 1935), (93, 320, 1936), (92, 321, 1938), (92, 322, 1938), (92, 323, 1939), (91, 324, 1940), (91, 325, 1941), (91, 326, 1942), (91, 327, 1942), (90, 328, 1944), (90, 329, 1944), (90, 330, 1945), (89, 331, 1947), (89, 332, 1947), (89, 333, 1948), (88, 334, 1949), (88, 335, 1950), (88, 336, 1951), (87, 337, 1952), (87, 338, 1953), (87, 339, 1954), (86, 340, 1956), (86, 341, 1956), (86, 342, 1957), (85, 343, 1959), (85, 344, 1959), (85, 345, 1960), (84, 346, 1962), (84, 347, 1963), (84, 348, 1963), (83, 349, 1965), (83, 350, 1966), (83, 351, 1967), (83, 352, 1968), (82, 353, 1969), (82, 354, 1970), (82, 355, 1971), (81, 356, 1973), (81, 357, 1974), (80, 358, 1975), (80, 359, 1976), (80, 360, 1977), (79, 361, 1979), (79, 362, 1980), (79, 363, 1981), (78, 364, 1983), (78, 365, 1984), (78, 366, 1985), (77, 367, 1987), (77, 368, 1988), (77, 369, 1989), (76, 370, 1991), (76, 371, 1992), (76, 372, 1993), (75, 373, 1995), (75, 374, 1995), (75, 375, 1996), (74, 376, 1998), (74, 377, 1999), (74, 378, 2000), (73, 379, 2001), (73, 380, 2002), (73, 381, 2003), (72, 382, 2005), (72, 383, 2005), (72, 384, 2006), (71, 385, 2008), (71, 386, 2008), (70, 387, 2010), (70, 388, 2011), (70, 389, 2012), (69, 390, 2013), (69, 391, 2014), (69, 392, 2014), (68, 393, 2016), (68, 394, 2017), (67, 395, 2018), (67, 396, 2019), (67, 397, 2020), (66, 398, 2021), (66, 399, 2022), (65, 400, 2023), (65, 401, 2024), (64, 402, 2026), (64, 403, 2026), (64, 404, 2027), (63, 405, 2028), (63, 406, 2029), (62, 407, 2030), (62, 408, 2031), (61, 409, 2032), (61, 410, 2033), (60, 411, 2034), (60, 412, 2035), (59, 413, 2036), (59, 414, 2037), (58, 415, 2038), (58, 416, 2039), (57, 417, 2040), (57, 418, 2041), (56, 419, 2042), (56, 420, 2043), (55, 421, 2044), (55, 422, 2045), (54, 423, 2046), (54, 424, 2047), (53, 425, 2048), (52, 426, 2049), (52, 427, 2050), (51, 428, 2051), (51, 429, 2052), (50, 430, 2053), (50, 431, 2054), (49, 432, 2055), (49, 433, 2055), (48, 434, 2057), (47, 435, 2058), (47, 436, 2059), (46, 437, 2060), (46, 438, 2060), (45, 439, 2062), (45, 440, 2062), (45, 441, 2063), (44, 442, 2064), (44, 443, 2064), (43, 444, 2066), (43, 445, 2066), (42, 446, 2067), (42, 447, 2068), (41, 448, 2069), (41, 449, 2069), (40, 450, 2071), (40, 451, 2071), (40, 452, 2071), (39, 453, 2073), (39, 454, 2073), (39, 455, 2073), (39, 456, 2074), (39, 457, 2074), (39, 458, 2074), (39, 459, 2074), (38, 460, 2076), (38, 461, 2076), (38, 462, 2076), (38, 463, 2077), (38, 464, 2077), (38, 465, 2077), (38, 466, 2078), (37, 467, 2079), (37, 468, 2079), (37, 469, 2080), (37, 470, 2080), (37, 471, 2080), (37, 472, 2081), (36, 473, 2082), (36, 474, 2082), (36, 475, 2083), (36, 476, 2083), (36, 477, 2083), (36, 478, 2084), (36, 479, 2084), (35, 480, 2085), (35, 481, 2086), (35, 482, 2086), (35, 483, 2087), (35, 484, 2087), (35, 485, 2087), (35, 486, 2088), (35, 487, 2088), (34, 488, 2090), (34, 489, 2090), (34, 490, 2090), (34, 491, 2091), (34, 492, 2091), (34, 493, 2092), (34, 494, 2092), (33, 495, 2094), (33, 496, 2094), (33, 497, 2094), (33, 498, 2095), (33, 499, 2095), (33, 500, 2096), (33, 501, 2096), (33, 502, 2097), (32, 503, 2098), (32, 504, 2099), (32, 505, 2099), (32, 506, 2100), (32, 507, 2100), (32, 508, 2101), (32, 509, 2101), (32, 510, 2102), (31, 511, 2103), (31, 512, 2104), (31, 513, 2105), (31, 514, 2105), (31, 515, 2106), (31, 516, 2106), (31, 517, 2107), (31, 518, 2107), (30, 519, 2109), (30, 520, 2110), (30, 521, 2110), (30, 522, 2111), (30, 523, 2112), (30, 524, 2112), (30, 525, 2113), (30, 526, 2113), (30, 527, 2114), (29, 528, 2116), (29, 529, 2116), (29, 530, 2117), (29, 531, 2117), (29, 532, 2118), (29, 533, 2118), (29, 534, 2118), (29, 535, 2119), (29, 536, 2119), (29, 537, 2119), (29, 538, 2120), (29, 539, 2120), (29, 540, 2121), (29, 541, 2121), (29, 542, 2121), (29, 543, 2122), (28, 544, 2123), (28, 545, 2123), (28, 546, 2124), (28, 547, 2124), (28, 548, 2124), (28, 549, 2125), (28, 550, 2125), (28, 551, 2125), (28, 552, 2126), (28, 553, 2126), (28, 554, 2126), (28, 555, 2127), (28, 556, 2127), (28, 557, 2127), (28, 558, 2128), (28, 559, 2128), (28, 560, 2128), (28, 561, 2128), (27, 562, 2130), (27, 563, 2130), (27, 564, 2130), (27, 565, 2131), (27, 566, 2131), (27, 567, 2131), (27, 568, 2131), (27, 569, 2132), (27, 570, 2132), (27, 571, 2132), (27, 572, 2133), (27, 573, 2133), (27, 574, 2133), (27, 575, 2133), (27, 576, 2134), (27, 577, 2134), (27, 578, 2134), (27, 579, 2134), (27, 580, 2135), (26, 581, 2136), (26, 582, 2136), (26, 583, 2137), (26, 584, 2137), (26, 585, 2137), (26, 586, 2137), (26, 587, 2138), (26, 588, 2138), (26, 589, 2138), (26, 590, 2138), (26, 591, 2138), (26, 592, 2139), (26, 593, 2139), (26, 594, 2139), (26, 595, 2139), (26, 596, 2140), (26, 597, 2140), (26, 598, 2140), (26, 599, 2140), (25, 600, 2142), (25, 601, 2142), (25, 602, 2142), (25, 603, 2142), (25, 604, 2142), (25, 605, 2143), (25, 606, 2143), (25, 607, 2143), (25, 608, 2143), (25, 609, 2143), (25, 610, 2143), (25, 611, 2143), (25, 612, 2143), (25, 613, 2143), (25, 614, 2143), (25, 615, 2143), (25, 616, 2143), (25, 617, 2144), (25, 618, 2144), (25, 619, 2144), (25, 620, 2144), (25, 621, 2144), (25, 622, 2144), (25, 623, 2144), (24, 624, 2145), (24, 625, 2145), (24, 626, 2145), (24, 627, 2145), (24, 628, 2145), (24, 629, 2145), (24, 630, 2145), (24, 631, 2146), (24, 632, 2146), (24, 633, 2146), (24, 634, 2146), (24, 635, 2146), (24, 636, 2146), (24, 637, 2146), (24, 638, 2146), (24, 639, 2146), (24, 640, 2146), (24, 641, 2146), (24, 642, 2146), (24, 643, 2146), (24, 644, 2146), (24, 645, 2147), (24, 646, 2147), (24, 647, 2147), (24, 648, 2147), (24, 649, 2147), (24, 650, 2147), (23, 651, 2148), (23, 652, 2148), (23, 653, 2148), (23, 654, 2148), (23, 655, 2148), (23, 656, 2148), (23, 657, 2148), (23, 658, 2148), (23, 659, 2148), (23, 660, 2149), (23, 661, 2149), (23, 662, 2149), (23, 663, 2149), (23, 664, 2149), (23, 665, 2149), (23, 666, 2149), (23, 667, 2149), (23, 668, 2149), (23, 669, 2149), (23, 670, 2149), (23, 671, 2149), (23, 672, 2149), (23, 673, 2149), (23, 674, 2149), (23, 675, 2150), (23, 676, 2150), (23, 677, 2150), (23, 678, 2150), (22, 679, 2151), (22, 680, 2151), (22, 681, 2151), (22, 682, 2151), (22, 683, 2151), (23, 684, 2150), (23, 685, 2150), (23, 686, 2150), (23, 687, 2150), (23, 688, 2150), (23, 689, 2150), (23, 690, 2150), (23, 691, 2150), (23, 692, 2150), (23, 693, 2150), (23, 694, 2149), (23, 695, 2149), (23, 696, 2149), (23, 697, 2149), (23, 698, 2149), (23, 699, 2149), (23, 700, 2149), (24, 701, 2148), (24, 702, 2148), (24, 703, 2148), (24, 704, 2148), (24, 705, 2148), (24, 706, 2148), (24, 707, 2148), (24, 708, 2148), (24, 709, 2148), (24, 710, 2148), (24, 711, 2148), (24, 712, 2148), (24, 713, 2148), (24, 714, 2148), (24, 715, 2148), (24, 716, 2148), (25, 717, 2147), (25, 718, 2146), (25, 719, 2146), (25, 720, 2146), (25, 721, 2146), (25, 722, 2146), (25, 723, 2146), (25, 724, 2146), (25, 725, 2146), (25, 726, 2146), (25, 727, 2146), (25, 728, 2146), (25, 729, 2146), (25, 730, 2146), (25, 731, 2146), (26, 732, 2145), (26, 733, 2145), (26, 734, 2145), (26, 735, 2145), (26, 736, 2145), (26, 737, 2145), (26, 738, 2145), (26, 739, 2145), (26, 740, 2145), (26, 741, 2144), (26, 742, 2144), (26, 743, 2144), (26, 744, 2144), (26, 745, 2144), (26, 746, 2144), (27, 747, 2143), (27, 748, 2143), (27, 749, 2143), (27, 750, 2143), (27, 751, 2143), (27, 752, 2143), (27, 753, 2143), (27, 754, 2143), (27, 755, 2143), (27, 756, 2143), (27, 757, 2143), (27, 758, 2143), (27, 759, 2143), (27, 760, 2143), (27, 761, 2143), (27, 762, 2143), (27, 763, 2142), (27, 764, 2142), (27, 765, 2142), (27, 766, 2142), (27, 767, 2142), (27, 768, 2142), (27, 769, 2142), (27, 770, 2142), (27, 771, 2142), (27, 772, 2142), (27, 773, 2142), (27, 774, 2142), (27, 775, 2142), (27, 776, 2142), (27, 777, 2142), (27, 778, 2142), (27, 779, 2142), (27, 780, 2142), (27, 781, 2142), (27, 782, 2142), (27, 783, 2141), (27, 784, 2141), (27, 785, 2141), (27, 786, 2141), (27, 787, 2141), (27, 788, 2141), (27, 789, 2141), (27, 790, 2141), (27, 791, 2141), (27, 792, 2141), (27, 793, 2141), (27, 794, 2141), (27, 795, 2141), (27, 796, 2141), (27, 797, 2141), (27, 798, 2141), (27, 799, 2141), (27, 800, 2141), (27, 801, 2141), (27, 802, 2140), (27, 803, 2140), (27, 804, 2140), (27, 805, 2140), (27, 806, 2140), (27, 807, 2140), (27, 808, 2140), (27, 809, 2140), (27, 810, 2140), (27, 811, 2140), (26, 812, 2141), (26, 813, 2141), (26, 814, 2141), (26, 815, 2141), (26, 816, 2141), (26, 817, 2141), (26, 818, 2141), (26, 819, 2141), (26, 820, 2140), (26, 821, 2140), (26, 822, 2140), (26, 823, 2140), (26, 824, 2140), (26, 825, 2140), (26, 826, 2140), (26, 827, 2140), (26, 828, 2140), (26, 829, 2140), (26, 830, 2140), (26, 831, 2140), (26, 832, 2140), (26, 833, 2140), (26, 834, 2140), (26, 835, 2140), (26, 836, 2139), (26, 837, 2139), (26, 838, 2139), (26, 839, 2138), (26, 840, 2138), (26, 841, 2138), (26, 842, 2137), (26, 843, 2137), (27, 844, 2135), (27, 845, 2135), (27, 846, 2135), (27, 847, 2134), (27, 848, 2134), (27, 849, 2133), (27, 850, 2133), (27, 851, 2133), (27, 852, 2132), (27, 853, 2132), (27, 854, 2131), (27, 855, 2131), (27, 856, 2130), (27, 857, 2130), (27, 858, 2129), (27, 859, 2129), (27, 860, 2129), (27, 861, 2128), (27, 862, 2128), (27, 863, 2127), (28, 864, 2126), (28, 865, 2125), (28, 866, 2125), (28, 867, 2124), (28, 868, 2123), (28, 869, 2123), (28, 870, 2122), (28, 871, 2122), (28, 872, 2121), (28, 873, 2121), (28, 874, 2120), (28, 875, 2120), (28, 876, 2119), (28, 877, 2118), (28, 878, 2118), (28, 879, 2117), (28, 880, 2117), (28, 881, 2116), (28, 882, 2116), (29, 883, 2114), (29, 884, 2113), (29, 885, 2113), (29, 886, 2112), (29, 887, 2112), (29, 888, 2111), (29, 889, 2111), (29, 890, 2110), (29, 891, 2110), (29, 892, 2109), (29, 893, 2109), (29, 894, 2108), (29, 895, 2108), (29, 896, 2108), (29, 897, 2107), (29, 898, 2107), (29, 899, 2106), (29, 900, 2106), (29, 901, 2105), (30, 902, 2104), (30, 903, 2104), (30, 904, 2103), (30, 905, 2103), (30, 906, 2102), (30, 907, 2102), (30, 908, 2101), (30, 909, 2101), (30, 910, 2101), (30, 911, 2100), (30, 912, 2100), (30, 913, 2100), (30, 914, 2099), (30, 915, 2099), (30, 916, 2099), (30, 917, 2099), (30, 918, 2098), (30, 919, 2098), (30, 920, 2098), (30, 921, 2098), (30, 922, 2097), (30, 923, 2097), (30, 924, 2097), (30, 925, 2097), (30, 926, 2096), (30, 927, 2096), (30, 928, 2096), (30, 929, 2096), (30, 930, 2095), (30, 931, 2095), (29, 932, 2096), (29, 933, 2096), (29, 934, 2096), (29, 935, 2095), (29, 936, 2095), (29, 937, 2095), (29, 938, 2095), (29, 939, 2094), (29, 940, 2094), (29, 941, 2094), (29, 942, 2094), (29, 943, 2094), (29, 944, 2093), (29, 945, 2093), (29, 946, 2093), (29, 947, 2093), (29, 948, 2093), (29, 949, 2092), (29, 950, 2092), (29, 951, 2092), (29, 952, 2092), (29, 953, 2092), (29, 954, 2091), (29, 955, 2091), (29, 956, 2091), (29, 957, 2091), (29, 958, 2091), (29, 959, 2090), (29, 960, 2090), (29, 961, 2090), (29, 962, 2090), (29, 963, 2090), (29, 964, 2089), (28, 965, 2090), (28, 966, 2090), (28, 967, 2090), (28, 968, 2090), (28, 969, 2090), (28, 970, 2089), (28, 971, 2089), (28, 972, 2089), (28, 973, 2089), (28, 974, 2089), (28, 975, 2089), (28, 976, 2088), (28, 977, 2088), (28, 978, 2088), (28, 979, 2088), (28, 980, 2088), (28, 981, 2087), (28, 982, 2087), (28, 983, 2087), (28, 984, 2087), (28, 985, 2087), (28, 986, 2087), (28, 987, 2086), (28, 988, 2086), (28, 989, 2086), (28, 990, 2086), (28, 991, 2086), (28, 992, 2085), (28, 993, 2085), (28, 994, 2085), (28, 995, 2084), (28, 996, 2084), (28, 997, 2084), (28, 998, 2083), (28, 999, 2083), (28, 1000, 2083), (28, 1001, 2082), (28, 1002, 2082), (28, 1003, 2082), (28, 1004, 2081), (28, 1005, 2081), (28, 1006, 2081), (28, 1007, 2080), (28, 1008, 2080), (28, 1009, 2080), (28, 1010, 2079), (28, 1011, 2079), (28, 1012, 2079), (28, 1013, 2078), (28, 1014, 2078), (29, 1015, 2076), (29, 1016, 2076), (29, 1017, 2076), (29, 1018, 2075), (29, 1019, 2075), (29, 1020, 2075), (29, 1021, 2074), (29, 1022, 2074), (29, 1023, 2074), (29, 1024, 2073), (29, 1025, 2073), (29, 1026, 2072), (29, 1027, 2072), (29, 1028, 2072), (29, 1029, 2071), (29, 1030, 2071), (29, 1031, 2070), (29, 1032, 2070), (29, 1033, 2070), (29, 1034, 2069), (29, 1035, 2069), (29, 1036, 2068), (29, 1037, 2068), (29, 1038, 2067), (29, 1039, 2067), (29, 1040, 2067), (29, 1041, 2066), (29, 1042, 2066), (29, 1043, 2065), (29, 1044, 2065), (29, 1045, 2064), (29, 1046, 2064), (29, 1047, 2063), (29, 1048, 2063), (30, 1049, 2061), (30, 1050, 2061), (30, 1051, 2061), (30, 1052, 2060), (30, 1053, 2060), (30, 1054, 2059), (30, 1055, 2059), (30, 1056, 2058), (30, 1057, 2058), (30, 1058, 2057), (30, 1059, 2057), (30, 1060, 2056), (30, 1061, 2056), (30, 1062, 2055), (30, 1063, 2055), (30, 1064, 2054), (30, 1065, 2054), (30, 1066, 2053), (30, 1067, 2052), (30, 1068, 2052), (30, 1069, 2051), (30, 1070, 2050), (30, 1071, 2049), (30, 1072, 2048), (30, 1073, 2048), (30, 1074, 2047), (30, 1075, 2046), (30, 1076, 2045), (30, 1077, 2044), (30, 1078, 2043), (30, 1079, 2042), (30, 1080, 2041), (30, 1081, 2041), (30, 1082, 2040), (30, 1083, 2038), (30, 1084, 2037), (30, 1085, 2036), (30, 1086, 2035), (30, 1087, 2034), (30, 1088, 2033), (30, 1089, 2032), (30, 1090, 2031), (30, 1091, 2030), (30, 1092, 2029), (30, 1093, 2028), (30, 1094, 2027), (30, 1095, 2027), (30, 1096, 2026), (30, 1097, 2025), (30, 1098, 2024), (30, 1099, 2023), (30, 1100, 2022), (29, 1101, 2023), (29, 1102, 2022), (29, 1103, 2021), (29, 1104, 2020), (29, 1105, 2020), (29, 1106, 2019), (29, 1107, 2018), (29, 1108, 2017), (29, 1109, 2017), (29, 1110, 2016), (29, 1111, 2015), (29, 1112, 2015), (29, 1113, 2014), (29, 1114, 2014), (29, 1115, 2013), (29, 1116, 2012), (29, 1117, 2012), (29, 1118, 2011), (29, 1119, 2011), (29, 1120, 2010), (29, 1121, 2009), (29, 1122, 2009), (29, 1123, 2008), (29, 1124, 2008), (29, 1125, 2007), (29, 1126, 2007), (29, 1127, 2006), (29, 1128, 2006), (29, 1129, 2005), (29, 1130, 2005), (29, 1131, 2004), (29, 1132, 2004), (29, 1133, 2003), (29, 1134, 2003), (29, 1135, 2003), (29, 1136, 2002), (29, 1137, 2002), (29, 1138, 2001), (29, 1139, 2001), (29, 1140, 2000), (29, 1141, 2000), (29, 1142, 2000), (29, 1143, 1999), (29, 1144, 1999), (29, 1145, 1999), (29, 1146, 1998), (29, 1147, 1998), (29, 1148, 1998), (29, 1149, 1997), (29, 1150, 1997), (29, 1151, 1997), (29, 1152, 1996), (29, 1153, 1996), (29, 1154, 1996), (29, 1155, 1995), (29, 1156, 1995), (29, 1157, 1995), (29, 1158, 1994), (29, 1159, 1994), (29, 1160, 1994), (29, 1161, 1993), (29, 1162, 1993), (29, 1163, 1992), (29, 1164, 1992), (29, 1165, 1992), (29, 1166, 1991), (29, 1167, 1991), (29, 1168, 1991), (29, 1169, 1990), (29, 1170, 1990), (29, 1171, 1989), (29, 1172, 1989), (29, 1173, 1989), (29, 1174, 1988), (29, 1175, 1988), (29, 1176, 1987), (29, 1177, 1987), (29, 1178, 1986), (29, 1179, 1986), (29, 1180, 1986), (29, 1181, 1985), (29, 1182, 1985), (29, 1183, 1984), (29, 1184, 1984), (29, 1185, 1983), (29, 1186, 1983), (30, 1187, 1982), (30, 1188, 1981), (30, 1189, 1981), (30, 1190, 1980), (30, 1191, 1980), (30, 1192, 1979), (30, 1193, 1979), (30, 1194, 1978), (30, 1195, 1978), (30, 1196, 1977), (30, 1197, 1977), (30, 1198, 1976), (30, 1199, 1976), (30, 1200, 1975), (30, 1201, 1975), (30, 1202, 1974), (30, 1203, 1974), (30, 1204, 1973), (30, 1205, 1972), (30, 1206, 1972), (30, 1207, 1971), (30, 1208, 1971), (30, 1209, 1970), (30, 1210, 1970), (30, 1211, 1969), (30, 1212, 1968), (30, 1213, 1968), (30, 1214, 1967), (30, 1215, 1967), (30, 1216, 1966), (30, 1217, 1965), (30, 1218, 1965), (30, 1219, 1964), (30, 1220, 1963), (30, 1221, 1963), (30, 1222, 1962), (30, 1223, 1961), (30, 1224, 1961), (30, 1225, 1960), (30, 1226, 1959), (30, 1227, 1958), (30, 1228, 1957), (30, 1229, 1957), (30, 1230, 1956), (30, 1231, 1955), (30, 1232, 1954), (30, 1233, 1953), (30, 1234, 1953), (30, 1235, 1952), (30, 1236, 1951), (30, 1237, 1950), (30, 1238, 1950), (30, 1239, 1949), (30, 1240, 1948), (30, 1241, 1947), (30, 1242, 1947), (30, 1243, 1946), (30, 1244, 1945), (30, 1245, 1945), (30, 1246, 1944), (30, 1247, 1943), (30, 1248, 1943), (30, 1249, 1942), (30, 1250, 1942), (30, 1251, 1941), (30, 1252, 1940), (30, 1253, 1940), (30, 1254, 1939), (30, 1255, 1939), (30, 1256, 1938), (30, 1257, 1938), (30, 1258, 1937), (30, 1259, 1937), (30, 1260, 1936), (30, 1261, 1936), (30, 1262, 1935), (30, 1263, 1935), (30, 1264, 1934), (30, 1265, 1934), (30, 1266, 1933), (30, 1267, 1933), (30, 1268, 1932), (30, 1269, 1932), (30, 1270, 1931), (30, 1271, 1931), (30, 1272, 1930), (30, 1273, 1930), (30, 1274, 1930), (30, 1275, 1929), (30, 1276, 1929), (30, 1277, 1928), (30, 1278, 1928), (30, 1279, 1928), (30, 1280, 1927), (30, 1281, 1927), (30, 1282, 1926), (30, 1283, 1926), (30, 1284, 1926), (30, 1285, 1925), (30, 1286, 1925), (30, 1287, 1925), (30, 1288, 1924), (30, 1289, 1924), (30, 1290, 1924), (30, 1291, 1923), (30, 1292, 1923), (30, 1293, 1922), (30, 1294, 1922), (30, 1295, 1922), (30, 1296, 1922), (30, 1297, 1921), (30, 1298, 1921), (30, 1299, 1921), (30, 1300, 1921), (30, 1301, 1920), (30, 1302, 1920), (30, 1303, 1920), (31, 1304, 1919), (31, 1305, 1918), (31, 1306, 1918), (31, 1307, 1918), (31, 1308, 1918), (31, 1309, 1917), (31, 1310, 1917), (31, 1311, 1917), (31, 1312, 1917), (31, 1313, 1916), (31, 1314, 1916), (31, 1315, 1916), (31, 1316, 1916), (31, 1317, 1915), (31, 1318, 1915), (31, 1319, 1915), (31, 1320, 1915), (31, 1321, 1914), (31, 1322, 1914), (31, 1323, 1914), (31, 1324, 1914), (31, 1325, 1913), (31, 1326, 1913), (31, 1327, 1913), (31, 1328, 1913), (31, 1329, 1912), (31, 1330, 1912), (32, 1331, 1911), (32, 1332, 1910), (32, 1333, 1910), (32, 1334, 1910), (32, 1335, 1910), (32, 1336, 1909), (32, 1337, 1909), (32, 1338, 1909), (32, 1339, 1908), (32, 1340, 1908), (32, 1341, 1908), (32, 1342, 1908), (32, 1343, 1907), (32, 1344, 1907), (32, 1345, 1907), (32, 1346, 1906), (32, 1347, 1906), (32, 1348, 1906), (32, 1349, 1905), (32, 1350, 1905), (32, 1351, 1905), (32, 1352, 1905), (32, 1353, 1904), (32, 1354, 1904), (32, 1355, 1904), (32, 1356, 1903), (33, 1357, 1902), (33, 1358, 1902), (33, 1359, 1901), (33, 1360, 1901), (33, 1361, 1901), (33, 1362, 1900), (33, 1363, 1900), (33, 1364, 1900), (33, 1365, 1899), (33, 1366, 1899), (33, 1367, 1899), (33, 1368, 1898), (33, 1369, 1898), (33, 1370, 1898), (33, 1371, 1897), (33, 1372, 1897), (33, 1373, 1897), (33, 1374, 1896), (33, 1375, 1896), (33, 1376, 1895), (33, 1377, 1895), (33, 1378, 1894), (34, 1379, 1892), (34, 1380, 1892), (34, 1381, 1891), (34, 1382, 1891), (34, 1383, 1890), (34, 1384, 1890), (34, 1385, 1889), (34, 1386, 1888), (34, 1387, 1888), (34, 1388, 1887), (34, 1389, 1886), (34, 1390, 1886), (34, 1391, 1885), (34, 1392, 1884), (34, 1393, 1884), (34, 1394, 1883), (34, 1395, 1882), (35, 1396, 1881), (35, 1397, 1880), (35, 1398, 1879), (35, 1399, 1878), (35, 1400, 1877), (35, 1401, 1877), (35, 1402, 1876), (35, 1403, 1875), (35, 1404, 1874), (35, 1405, 1873), (35, 1406, 1872), (35, 1407, 1871), (35, 1408, 1870), (35, 1409, 1870), (35, 1410, 1869), (35, 1411, 1868), (36, 1412, 1866), (36, 1413, 1865), (36, 1414, 1864), (36, 1415, 1863), (36, 1416, 1863), (36, 1417, 1862), (36, 1418, 1861), (36, 1419, 1860), (36, 1420, 1859), (36, 1421, 1859), (36, 1422, 1858), (36, 1423, 1857), (36, 1424, 1857), (36, 1425, 1856), (36, 1426, 1855), (37, 1427, 1854), (37, 1428, 1853), (37, 1429, 1852), (37, 1430, 1852), (37, 1431, 1851), (37, 1432, 1851), (37, 1433, 1850), (37, 1434, 1849), (37, 1435, 1849), (37, 1436, 1848), (37, 1437, 1848), (37, 1438, 1847), (37, 1439, 1847), (37, 1440, 1846), (37, 1441, 1846), (37, 1442, 1845), (38, 1443, 1844), (38, 1444, 1843), (38, 1445, 1843), (38, 1446, 1842), (38, 1447, 1842), (38, 1448, 1841), (38, 1449, 1841), (38, 1450, 1840), (38, 1451, 1840), (38, 1452, 1840), (38, 1453, 1840), (38, 1454, 1840), (38, 1455, 1839), (38, 1456, 1839), (38, 1457, 1839), (38, 1458, 1839), (38, 1459, 1839), (38, 1460, 1838), (38, 1461, 1838), (38, 1462, 1838), (38, 1463, 1838), (38, 1464, 1838), (38, 1465, 1837), (38, 1466, 1837), (38, 1467, 1837), (38, 1468, 1837), (39, 1469, 1836), (39, 1470, 1835), (39, 1471, 1835), (39, 1472, 1835), (39, 1473, 1835), (39, 1474, 1835), (39, 1475, 1835), (39, 1476, 1834), (39, 1477, 1834), (39, 1478, 1834), (39, 1479, 1834), (39, 1480, 1834), (39, 1481, 1833), (39, 1482, 1833), (39, 1483, 1833), (39, 1484, 1833), (39, 1485, 1833), (39, 1486, 1833), (39, 1487, 1832), (39, 1488, 1832), (39, 1489, 1832), (39, 1490, 1832), (39, 1491, 1832), (39, 1492, 1832), (39, 1493, 1831), (39, 1494, 1831), (39, 1495, 1831), (39, 1496, 1831), (39, 1497, 1831), (39, 1498, 1831), (39, 1499, 1830), (39, 1500, 1830), (39, 1501, 1830), (39, 1502, 1830), (39, 1503, 1830), (39, 1504, 1830), (40, 1505, 1828), (40, 1506, 1828), (40, 1507, 1828), (40, 1508, 1828), (40, 1509, 1828), (40, 1510, 1828), (40, 1511, 1828), (40, 1512, 1827), (40, 1513, 1827), (40, 1514, 1827), (40, 1515, 1827), (40, 1516, 1827), (40, 1517, 1827), (40, 1518, 1826), (40, 1519, 1826), (40, 1520, 1826), (40, 1521, 1826), (40, 1522, 1826), (40, 1523, 1826), (40, 1524, 1826), (40, 1525, 1825), (40, 1526, 1825), (40, 1527, 1825), (40, 1528, 1825), (40, 1529, 1825), (40, 1530, 1825), (40, 1531, 1825), (40, 1532, 1824), (40, 1533, 1824), (40, 1534, 1824), (40, 1535, 1824), (40, 1536, 1824), (40, 1537, 1824), (40, 1538, 1824), (40, 1539, 1824), (40, 1540, 1824), (40, 1541, 1823), (40, 1542, 1823), (40, 1543, 1823), (40, 1544, 1823), (40, 1545, 1823), (40, 1546, 1823), (40, 1547, 1823), (40, 1548, 1823), (40, 1549, 1822), (41, 1550, 1821), (41, 1551, 1821), (41, 1552, 1821), (41, 1553, 1821), (41, 1554, 1821), (41, 1555, 1821), (41, 1556, 1821), (41, 1557, 1820), (41, 1558, 1820), (41, 1559, 1820), (41, 1560, 1820), (41, 1561, 1820), (41, 1562, 1820), (41, 1563, 1820), (41, 1564, 1819), (41, 1565, 1819), (41, 1566, 1819), (41, 1567, 1819), (41, 1568, 1819), (41, 1569, 1819), (41, 1570, 1819), (41, 1571, 1819), (41, 1572, 1818), (41, 1573, 1818), (41, 1574, 1818), (41, 1575, 1818), (41, 1576, 1818), (41, 1577, 1818), (41, 1578, 1818), (41, 1579, 1818), (41, 1580, 1817), (41, 1581, 1817), (41, 1582, 1817), (41, 1583, 1817), (41, 1584, 1817), (41, 1585, 1817), (41, 1586, 1817), (41, 1587, 1816), (41, 1588, 1816), (41, 1589, 1816), (41, 1590, 1816), (41, 1591, 1816), (41, 1592, 1816), (41, 1593, 1816), (41, 1594, 1816), (41, 1595, 1815), (41, 1596, 1815), (41, 1597, 1815), (41, 1598, 1815), (41, 1599, 1815), (41, 1600, 1815), (41, 1601, 1815), (41, 1602, 1814), (41, 1603, 1814), (41, 1604, 1814), (41, 1605, 1814), (41, 1606, 1814), (41, 1607, 1814), (41, 1608, 1814), (41, 1609, 1814), (41, 1610, 1814), (41, 1611, 1813), (41, 1612, 1813), (41, 1613, 1813), (41, 1614, 1813), (41, 1615, 1813), (41, 1616, 1813), (41, 1617, 1813), (41, 1618, 1813), (41, 1619, 1812), (41, 1620, 1812), (41, 1621, 1812), (41, 1622, 1812), (41, 1623, 1812), (41, 1624, 1812), (41, 1625, 1812), (41, 1626, 1812), (41, 1627, 1811), (41, 1628, 1811), (41, 1629, 1811), (41, 1630, 1811), (40, 1631, 1812), (40, 1632, 1812), (40, 1633, 1812), (40, 1634, 1812), (40, 1635, 1811), (40, 1636, 1811), (40, 1637, 1811), (40, 1638, 1811), (40, 1639, 1811), (40, 1640, 1811), (40, 1641, 1811), (40, 1642, 1811), (40, 1643, 1810), (40, 1644, 1810), (40, 1645, 1810), (40, 1646, 1810), (40, 1647, 1810), (40, 1648, 1810), (40, 1649, 1810), (40, 1650, 1809), (40, 1651, 1809), (40, 1652, 1809), (40, 1653, 1809), (40, 1654, 1809), (40, 1655, 1809), (40, 1656, 1809), (40, 1657, 1808), (40, 1658, 1808), (40, 1659, 1808), (40, 1660, 1808), (39, 1661, 1809), (39, 1662, 1809), (39, 1663, 1809), (39, 1664, 1808), (39, 1665, 1808), (39, 1666, 1808), (39, 1667, 1808), (39, 1668, 1808), (39, 1669, 1808), (39, 1670, 1808), (39, 1671, 1807), (39, 1672, 1807), (39, 1673, 1807), (39, 1674, 1807), (39, 1675, 1807), (39, 1676, 1807), (39, 1677, 1806), (39, 1678, 1806), (39, 1679, 1806), (39, 1680, 1806), (39, 1681, 1806), (39, 1682, 1805), (39, 1683, 1805), (39, 1684, 1805), (40, 1685, 1804), (40, 1686, 1803), (40, 1687, 1803), (40, 1688, 1803), (40, 1689, 1803), (40, 1690, 1802), (41, 1691, 1801), (41, 1692, 1801), (41, 1693, 1800), (41, 1694, 1800), (41, 1695, 1800), (41, 1696, 1799), (42, 1697, 1798), (42, 1698, 1798), (42, 1699, 1798), (42, 1700, 1797), (42, 1701, 1797), (42, 1702, 1797), (42, 1703, 1796), (43, 1704, 1795), (43, 1705, 1795), (43, 1706, 1794), (43, 1707, 1794), (43, 1708, 1794), (43, 1709, 1793), (44, 1710, 1792), (44, 1711, 1792), (44, 1712, 1791), (44, 1713, 1791), (44, 1714, 1791), (45, 1715, 1789), (45, 1716, 1789), (45, 1717, 1789), (45, 1718, 1788), (45, 1719, 1788), (45, 1720, 1787), (46, 1721, 1786), (46, 1722, 1786), (46, 1723, 1785), (46, 1724, 1785), (46, 1725, 1785), (47, 1726, 1783), (47, 1727, 1783), (47, 1728, 1782), (47, 1729, 1782), (47, 1730, 1782), (47, 1731, 1781), (48, 1732, 1780), (48, 1733, 1779), (48, 1734, 1779), (48, 1735, 1778), (48, 1736, 1778), (49, 1737, 1777), (49, 1738, 1776), (49, 1739, 1776), (49, 1740, 1775), (49, 1741, 1775), (50, 1742, 1773), (50, 1743, 1773), (50, 1744, 1772), (50, 1745, 1772), (50, 1746, 1771), (51, 1747, 1770), (51, 1748, 1769), (51, 1749, 1769), (51, 1750, 1769), (51, 1751, 1768), (52, 1752, 1767), (52, 1753, 1766), (52, 1754, 1766), (52, 1755, 1765), (53, 1756, 1764), (53, 1757, 1764), (53, 1758, 1763), (53, 1759, 1763), (53, 1760, 1763), (53, 1761, 1762), (53, 1762, 1762), (53, 1763, 1762), (53, 1764, 1761), (53, 1765, 1761), (53, 1766, 1760), (53, 1767, 1760), (53, 1768, 1760), (53, 1769, 1759), (53, 1770, 1759), (53, 1771, 1759), (53, 1772, 1758), (53, 1773, 1758), (53, 1774, 1758), (53, 1775, 1757), (53, 1776, 1757), (53, 1777, 1757), (53, 1778, 1757), (53, 1779, 1756), (54, 1780, 1755), (54, 1781, 1755), (54, 1782, 1754), (54, 1783, 1754), (54, 1784, 1754), (54, 1785, 1753), (54, 1786, 1753), (54, 1787, 1753), (54, 1788, 1752), (54, 1789, 1752), (54, 1790, 1752), (54, 1791, 1752), (54, 1792, 1751), (54, 1793, 1751), (54, 1794, 1751), (54, 1795, 1750), (54, 1796, 1750), (54, 1797, 1750), (54, 1798, 1750), (54, 1799, 1749), (54, 1800, 1749), (54, 1801, 1749), (54, 1802, 1748), (54, 1803, 1748), (54, 1804, 1748), (54, 1805, 1748), (54, 1806, 1747), (54, 1807, 1747), (54, 1808, 1747), (55, 1809, 1745), (55, 1810, 1745), (55, 1811, 1745), (55, 1812, 1745), (55, 1813, 1744), (55, 1814, 1744), (55, 1815, 1744), (55, 1816, 1744), (55, 1817, 1743), (55, 1818, 1743), (55, 1819, 1743), (55, 1820, 1743), (55, 1821, 1742), (55, 1822, 1742), (55, 1823, 1742), (55, 1824, 1742), (55, 1825, 1741), (55, 1826, 1741), (55, 1827, 1741), (55, 1828, 1741), (55, 1829, 1740), (55, 1830, 1740), (55, 1831, 1740), (55, 1832, 1740), (55, 1833, 1739), (56, 1834, 1738), (56, 1835, 1738), (57, 1836, 1736), (57, 1837, 1736), (58, 1838, 1735), (58, 1839, 1734), (59, 1840, 1733), (59, 1841, 1733), (60, 1842, 1731), (60, 1843, 1731), (61, 1844, 1730), (61, 1845, 1729), (62, 1846, 1728), (62, 1847, 1728), (63, 1848, 1726), (64, 1849, 1725), (64, 1850, 1724), (65, 1851, 1723), (65, 1852, 1723), (66, 1853, 1721), (66, 1854, 1721), (67, 1855, 1720), (67, 1856, 1719), (68, 1857, 1718), (68, 1858, 1717), (69, 1859, 1716), (69, 1860, 1716), (70, 1861, 1714), (70, 1862, 1714), (71, 1863, 1712), (71, 1864, 1712), (72, 1865, 1711), (72, 1866, 1710), (73, 1867, 1709), (73, 1868, 1708), (74, 1869, 1707), (74, 1870, 1707), (75, 1871, 1705), (75, 1872, 1705), (76, 1873, 1703), (76, 1874, 1703), (76, 1875, 1702), (77, 1876, 1701), (77, 1877, 1700), (78, 1878, 1699), (78, 1879, 1699), (79, 1880, 1697), (79, 1881, 1697), (80, 1882, 1695), (80, 1883, 1695), (81, 1884, 1693), (81, 1885, 1693), (82, 1886, 1691), (82, 1887, 1691), (83, 1888, 1689), (83, 1889, 1689), (84, 1890, 1687), (84, 1891, 1687), (85, 1892, 1685), (85, 1893, 1685), (86, 1894, 1683), (86, 1895, 1683), (86, 1896, 1682), (87, 1897, 1681), (87, 1898, 1680), (88, 1899, 1679), (88, 1900, 1678), (89, 1901, 1677), (89, 1902, 1676), (90, 1903, 1675), (90, 1904, 1674), (91, 1905, 1672), (91, 1906, 1672), (92, 1907, 1670), (92, 1908, 1670), (92, 1909, 1669), (93, 1910, 1668), (93, 1911, 1667), (94, 1912, 1666), (94, 1913, 1665), (95, 1914, 1663), (95, 1915, 1663), (95, 1916, 1662), (96, 1917, 1661), (96, 1918, 1660), (97, 1919, 1659), (97, 1920, 1658), (98, 1921, 1656), (98, 1922, 1656), (98, 1923, 1655), (99, 1924, 1653), (99, 1925, 1653), (100, 1926, 1651), (100, 1927, 1651), (101, 1928, 1649), (101, 1929, 1648), (102, 1930, 1647), (102, 1931, 1646), (103, 1932, 1644), (103, 1933, 1643), (104, 1934, 1642), (104, 1935, 1641), (105, 1936, 1639), (105, 1937, 1639), (106, 1938, 1637), (106, 1939, 1636), (107, 1940, 1634), (108, 1941, 1632), (108, 1942, 1632), (109, 1943, 1630), (109, 1944, 1629), (110, 1945, 1627), (110, 1946, 1626), (111, 1947, 1625), (112, 1948, 1623), (112, 1949, 1622), (113, 1950, 1620), (114, 1951, 1618), (114, 1952, 1618), (115, 1953, 1616), (116, 1954, 1614), (116, 1955, 1613), (117, 1956, 1612), (118, 1957, 1610), (118, 1958, 1609), (119, 1959, 1607), (120, 1960, 1605), (121, 1961, 1604), (121, 1962, 1603), (122, 1963, 1601), (123, 1964, 1599), (124, 1965, 88), (213, 1965, 1509), (124, 1966, 77), (220, 1966, 1501), (125, 1967, 66), (228, 1967, 1492), (126, 1968, 55), (236, 1968, 1483), (127, 1969, 45), (244, 1969, 1475), (128, 1970, 36), (252, 1970, 1466), (129, 1971, 27), (261, 1971, 1456), (130, 1972, 19), (269, 1972, 1447), (130, 1973, 12), (278, 1973, 1437), (131, 1974, 4), (287, 1974, 1428), (296, 1975, 1418), (302, 1976, 1411), (309, 1977, 1403), (315, 1978, 1397), (322, 1979, 1389), (328, 1980, 1382), (334, 1981, 1375), (341, 1982, 1368), (347, 1983, 1361), (353, 1984, 1354), (360, 1985, 1346), (366, 1986, 1340), (372, 1987, 1333), (376, 1988, 1328), (380, 1989, 1323), (383, 1990, 1319), (387, 1991, 1314), (390, 1992, 1310), (394, 1993, 1304), (397, 1994, 1300), (401, 1995, 1295), (405, 1996, 1290), (409, 1997, 1285), (413, 1998, 1279), (417, 1999, 1274), (421, 2000, 1269), (425, 2001, 1263), (430, 2002, 1257), (434, 2003, 1252), (439, 2004, 1245), (443, 2005, 1240), (448, 2006, 1233), (453, 2007, 1227), (457, 2008, 1221), (461, 2009, 1216), (465, 2010, 1210), (469, 2011, 1205), (473, 2012, 1199), (477, 2013, 1193), (482, 2014, 1186), (486, 2015, 1181), (491, 2016, 1174), (495, 2017, 1154), (500, 2018, 1098), (505, 2019, 1074), (510, 2020, 1065), (515, 2021, 1055), (520, 2022, 1046), (526, 2023, 1035), (531, 2024, 1026), (537, 2025, 1015), (542, 2026, 1005), (548, 2027, 995), (554, 2028, 984), (559, 2029, 975), (564, 2030, 965), (570, 2031, 955), (575, 2032, 945), (580, 2033, 936), (586, 2034, 925), (591, 2035, 916), (596, 2036, 905), (601, 2037, 886), (606, 2038, 867), (611, 2039, 847), (616, 2040, 828), (619, 2041, 810), (623, 2042, 796), (626, 2043, 789), (629, 2044, 781), (632, 2045, 773), (635, 2046, 766), (638, 2047, 758), (641, 2048, 751), (643, 2049, 745), (646, 2050, 738), (649, 2051, 731), (651, 2052, 725), (654, 2053, 718), (656, 2054, 713), (659, 2055, 706), (661, 2056, 700), (663, 2057, 695), (666, 2058, 689), (668, 2059, 683), (670, 2060, 678), (672, 2061, 673), (674, 2062, 668), (676, 2063, 663), (678, 2064, 658), (680, 2065, 653), (682, 2066, 648), (683, 2067, 644), (685, 2068, 639), (687, 2069, 634), (689, 2070, 629), (691, 2071, 624), (693, 2072, 618), (695, 2073, 613), (699, 2074, 606), (702, 2075, 599), (706, 2076, 591), (710, 2077, 584), (714, 2078, 576), (718, 2079, 568), (721, 2080, 561), (725, 2081, 553), (729, 2082, 545), (733, 2083, 537), (737, 2084, 529), (741, 2085, 520), (745, 2086, 511), (749, 2087, 502), (753, 2088, 493), (757, 2089, 484), (761, 2090, 475), (765, 2091, 466), (769, 2092, 457), (773, 2093, 448), (777, 2094, 439), (781, 2095, 430), (784, 2096, 422), (787, 2097, 414), (791, 2098, 405), (794, 2099, 397), (798, 2100, 388), (801, 2101, 380), (805, 2102, 373), (808, 2103, 367), (812, 2104, 360), (816, 2105, 353), (819, 2106, 348), (823, 2107, 341), (827, 2108, 334), (830, 2109, 329), (834, 2110, 322), (838, 2111, 316), (842, 2112, 309), (845, 2113, 304), (849, 2114, 297), (853, 2115, 291), (857, 2116, 285), (861, 2117, 279), (865, 2118, 272), (869, 2119, 266), (873, 2120, 260), (877, 2121, 254), (881, 2122, 248), (884, 2123, 243), (888, 2124, 237), (891, 2125, 232), (895, 2126, 226), (898, 2127, 221), (901, 2128, 216), (905, 2129, 210), (908, 2130, 205), (911, 2131, 200), (914, 2132, 195), (917, 2133, 191), (920, 2134, 186), (922, 2135, 182), (925, 2136, 177), (928, 2137, 173), (931, 2138, 168), (933, 2139, 161), (936, 2140, 153), (940, 2141, 144), (946, 2142, 133), (953, 2143, 120), (960, 2144, 107), (967, 2145, 94), (974, 2146, 81), (982, 2147, 67), (990, 2148, 52), (997, 2149, 39), (1006, 2150, 23), (1014, 2151, 8)], ['1021,2151,936,2140,695,2073,615,2039,495,2017,371,1986,201,1965,131,1974,55,1833,39,1684,39,1469,30,1303,30,902,23,700,29,528,39,453,72,384,99,298,130,264,224,199,310,172,374,139,530,114,1426,114,1580,131,1715,168,1905,202,2038,337,2089,402,2146,532,2167,605,2172,675,2165,835,2136,894,2113,991,2081,1068,2051,1100,1951,1293,1927,1377,1877,1450,1844,1681,1794,1832,1743,1937,1664,2016,1579,2018,1500,2036,1419,2041,1273,2082,1181,2100,1098,2138'])], 'temp/1743741415_2995536_917877156_a9c2d4b99270c9302def4ed40606e685.jpg']} nb pixel non reg : 3692295 nb pixel common : 3684531 proportion of common points : 0.997897242771772 [('test release memory', 'SUCCESS', True), ('test detect objet', 'SUCCESS', True), ('test polygone', 'SUCCESS', True)] res_total : True #&_# TEST SUCCEEDED #&_# : tests/mask_test #&_# /home/admin/workarea/git/Velours/python/tests/python_tests.py refs/heads/master_47a3e4a81bc81b59dfbf2b2b102e93cf6aa9db97 SQL :INSERT INTO MTRAdmin.monitor_sys (name, type, server, version_code, result_str, result_bool, lien , test_group ,test_name) VALUES ('python_test3','1','marlene','refs/heads/master_47a3e4a81bc81b59dfbf2b2b102e93cf6aa9db97','{"mask_detection": "success"}','1','http://marlene.fotonower-preprod.com/job/2025/April/04042025/python_test3//data_2/data_log/job/2025/April/04042025/python_test3/log-python3----short_python3--v--marlene-06:35:00.txt','mask_detection','unknown'); #&_# END OF TEST #&_# : tests/mask_test #&_# #&_# BEGIN OF TEST : tests/datou_test #&_# /home/admin/workarea/git/Velours/python/tests/datou_test.py Datou All Test python version used : 3 ############################### TEST sam ################################ TEST SAM Inside batchDatouExec : verbose : True ##### chargement datou SELECT name, created_at,limit_max FROM MTRDatou.mtr_datou WHERE id=4573 SELECT mtd.id, mtdt.`type`, mtd.`param`, mtd.param_json, mtdt.nb_input, mtdt.nb_output, mtdt.prod, mtdt.is_local, mtdt.is_datou_depend, mtdt.is_photo_id_local FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_step_types mtdt WHERE mtdt.`id`=mtd.`type` AND mtd.mtd_id=4573 SELECT mtd.id, mtd.mtd_id, mdsdt.id, mdsdt.name, mdsdt.description, msid.output_or_input, msid.data_order_id, mdsdt.type FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_steptype_io_datatypes msid, MTRDatou.mtr_datou_step_data_types mdsdt WHERE mtd.`type`=msid.`mtr_datou_step_type` AND mtd.mtd_id= 4573 AND msid.data_type=mdsdt.id SELECT mts_id_output, id_output, mts_id_input, id_input FROM MTRDatou.mtr_datou_step_by_step WHERE mtd_id=4573 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! no param json to modify List Step Type Loaded in datou : sam list_input_json : [] ##### fin chargement datou ##### chargement data ##### Call load_data_input : nb_thread : 5 origin SELECT photo_id, url FROM MTRBack.photos ph WHERE photo_id IN (1189321094) Found this number of photos: 1 ##### Call download_photos : nb_thread : 5 begin to download photo : 1189321094 download finish for photo 1189321094 we have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB ##### After download_photos length of list_filenames : 1 ; length of list_pids : 1 ; length of list_args : 1 ##### After load_data_input time to download the photos : 0.25670766830444336 #### fin chargement data Blocking on flush ? No conitnuing About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! WARNING : we have an input that is not a photo, we should get rid of it Calling datou_exec Inside datou_exec : verbose : True number of steps : 1 step1:sam Fri Apr 4 06:37:50 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec After prepare type args : Here we display some param of map_info ! map_filenames : {'temp/1743741470_2995536_1189321094_9626af7f95d010f2a4fd524688d4ea22_76896585.png': 1189321094} map_photo_id_path_extension : {1189321094: {'path': 'temp/1743741470_2995536_1189321094_9626af7f95d010f2a4fd524688d4ea22_76896585.png', 'extension': 'png'}} map_subphoto_mainphoto : {} Beginning of datou step sam ! pht : 4677 Inside sam : nb paths : 1 (640, 960, 3) time for calcul the mask position with numpy : 0.0020132064819335938 nb_pixel_total : 16193 time to create 1 rle with old method : 0.01771831512451172 time for calcul the mask position with numpy : 0.0013747215270996094 nb_pixel_total : 7576 time to create 1 rle with old method : 0.008926868438720703 time for calcul the mask position with numpy : 0.0017948150634765625 nb_pixel_total : 83613 time to create 1 rle with old method : 0.09610509872436523 time for calcul the mask position with numpy : 0.0014462471008300781 nb_pixel_total : 5615 time to create 1 rle with old method : 0.006660938262939453 time for calcul the mask position with numpy : 0.0014340877532958984 nb_pixel_total : 3914 time to create 1 rle with old method : 0.004526376724243164 time for calcul the mask position with numpy : 0.0014181137084960938 nb_pixel_total : 5782 time to create 1 rle with old method : 0.006809711456298828 time for calcul the mask position with numpy : 0.0014846324920654297 nb_pixel_total : 15069 time to create 1 rle with old method : 0.017605304718017578 time for calcul the mask position with numpy : 0.0013535022735595703 nb_pixel_total : 5336 time to create 1 rle with old method : 0.006234884262084961 time for calcul the mask position with numpy : 0.0014982223510742188 nb_pixel_total : 29487 time to create 1 rle with old method : 0.0338597297668457 time for calcul the mask position with numpy : 0.0014944076538085938 nb_pixel_total : 9584 time to create 1 rle with old method : 0.011217832565307617 time for calcul the mask position with numpy : 0.0014004707336425781 nb_pixel_total : 13914 time to create 1 rle with old method : 0.016133785247802734 time for calcul the mask position with numpy : 0.0013670921325683594 nb_pixel_total : 3771 time to create 1 rle with old method : 0.004183292388916016 time for calcul the mask position with numpy : 0.0013852119445800781 nb_pixel_total : 10792 time to create 1 rle with old method : 0.012224674224853516 time for calcul the mask position with numpy : 0.0013113021850585938 nb_pixel_total : 2937 time to create 1 rle with old method : 0.003526926040649414 time for calcul the mask position with numpy : 0.001405477523803711 nb_pixel_total : 2445 time to create 1 rle with old method : 0.002883434295654297 time for calcul the mask position with numpy : 0.0013246536254882812 nb_pixel_total : 4270 time to create 1 rle with old method : 0.004743814468383789 time for calcul the mask position with numpy : 0.0013163089752197266 nb_pixel_total : 1226 time to create 1 rle with old method : 0.0017838478088378906 time for calcul the mask position with numpy : 0.0014336109161376953 nb_pixel_total : 2840 time to create 1 rle with old method : 0.003561735153198242 time for calcul the mask position with numpy : 0.0013048648834228516 nb_pixel_total : 2370 time to create 1 rle with old method : 0.0028955936431884766 time for calcul the mask position with numpy : 0.001332998275756836 nb_pixel_total : 6646 time to create 1 rle with old method : 0.007982492446899414 time for calcul the mask position with numpy : 0.001428365707397461 nb_pixel_total : 592 time to create 1 rle with old method : 0.0007941722869873047 time for calcul the mask position with numpy : 0.0014891624450683594 nb_pixel_total : 38982 time to create 1 rle with old method : 0.04528927803039551 time for calcul the mask position with numpy : 0.0014281272888183594 nb_pixel_total : 693 time to create 1 rle with old method : 0.0008816719055175781 time for calcul the mask position with numpy : 0.0013911724090576172 nb_pixel_total : 2077 time to create 1 rle with old method : 0.002493143081665039 time for calcul the mask position with numpy : 0.0014216899871826172 nb_pixel_total : 3508 time to create 1 rle with old method : 0.004288434982299805 time for calcul the mask position with numpy : 0.001436471939086914 nb_pixel_total : 4272 time to create 1 rle with old method : 0.005466938018798828 time for calcul the mask position with numpy : 0.0014243125915527344 nb_pixel_total : 5483 time to create 1 rle with old method : 0.0066187381744384766 time for calcul the mask position with numpy : 0.0015320777893066406 nb_pixel_total : 16478 time to create 1 rle with old method : 0.0193479061126709 time for calcul the mask position with numpy : 0.0014889240264892578 nb_pixel_total : 13016 time to create 1 rle with old method : 0.015955209732055664 time for calcul the mask position with numpy : 0.0013625621795654297 nb_pixel_total : 8561 time to create 1 rle with old method : 0.010185480117797852 time for calcul the mask position with numpy : 0.0014278888702392578 nb_pixel_total : 3900 time to create 1 rle with old method : 0.004904747009277344 time for calcul the mask position with numpy : 0.0015094280242919922 nb_pixel_total : 11894 time to create 1 rle with old method : 0.014183759689331055 time for calcul the mask position with numpy : 0.0014109611511230469 nb_pixel_total : 970 time to create 1 rle with old method : 0.0012576580047607422 time for calcul the mask position with numpy : 0.0014531612396240234 nb_pixel_total : 9870 time to create 1 rle with old method : 0.01192331314086914 time for calcul the mask position with numpy : 0.0014421939849853516 nb_pixel_total : 2747 time to create 1 rle with old method : 0.0032858848571777344 time for calcul the mask position with numpy : 0.0014121532440185547 nb_pixel_total : 3329 time to create 1 rle with old method : 0.0038738250732421875 time for calcul the mask position with numpy : 0.0013914108276367188 nb_pixel_total : 18474 time to create 1 rle with old method : 0.020977258682250977 time for calcul the mask position with numpy : 0.0013480186462402344 nb_pixel_total : 1025 time to create 1 rle with old method : 0.001199960708618164 time for calcul the mask position with numpy : 0.0013611316680908203 nb_pixel_total : 4173 time to create 1 rle with old method : 0.00822758674621582 time for calcul the mask position with numpy : 0.0014619827270507812 nb_pixel_total : 4102 time to create 1 rle with old method : 0.004908323287963867 time for calcul the mask position with numpy : 0.0014181137084960938 nb_pixel_total : 1657 time to create 1 rle with old method : 0.002048969268798828 time for calcul the mask position with numpy : 0.0014026165008544922 nb_pixel_total : 341 time to create 1 rle with old method : 0.0004570484161376953 time for calcul the mask position with numpy : 0.0013759136199951172 nb_pixel_total : 1243 time to create 1 rle with old method : 0.0014977455139160156 time for calcul the mask position with numpy : 0.0014829635620117188 nb_pixel_total : 10582 time to create 1 rle with old method : 0.012485742568969727 time for calcul the mask position with numpy : 0.0013015270233154297 nb_pixel_total : 1839 time to create 1 rle with old method : 0.0021970272064208984 time for calcul the mask position with numpy : 0.0013713836669921875 nb_pixel_total : 2404 time to create 1 rle with old method : 0.0031075477600097656 time for calcul the mask position with numpy : 0.0014104843139648438 nb_pixel_total : 1195 time to create 1 rle with old method : 0.0014832019805908203 time for calcul the mask position with numpy : 0.0014295578002929688 nb_pixel_total : 2393 time to create 1 rle with old method : 0.0030181407928466797 time for calcul the mask position with numpy : 0.0013682842254638672 nb_pixel_total : 874 time to create 1 rle with old method : 0.0011425018310546875 time for calcul the mask position with numpy : 0.0013966560363769531 nb_pixel_total : 861 time to create 1 rle with old method : 0.0011706352233886719 time for calcul the mask position with numpy : 0.0014255046844482422 nb_pixel_total : 886 time to create 1 rle with old method : 0.001161813735961914 time for calcul the mask position with numpy : 0.0013308525085449219 nb_pixel_total : 1086 time to create 1 rle with old method : 0.0013697147369384766 time for calcul the mask position with numpy : 0.0014460086822509766 nb_pixel_total : 2339 time to create 1 rle with old method : 0.002850770950317383 time for calcul the mask position with numpy : 0.0014317035675048828 nb_pixel_total : 2764 time to create 1 rle with old method : 0.0033032894134521484 time for calcul the mask position with numpy : 0.0014011859893798828 nb_pixel_total : 1673 time to create 1 rle with old method : 0.0020720958709716797 time for calcul the mask position with numpy : 0.00135040283203125 nb_pixel_total : 1065 time to create 1 rle with old method : 0.001285552978515625 time for calcul the mask position with numpy : 0.0013337135314941406 nb_pixel_total : 337 time to create 1 rle with old method : 0.00046896934509277344 time for calcul the mask position with numpy : 0.0013756752014160156 nb_pixel_total : 574 time to create 1 rle with old method : 0.0007891654968261719 time for calcul the mask position with numpy : 0.0014331340789794922 nb_pixel_total : 1677 time to create 1 rle with old method : 0.001962423324584961 time for calcul the mask position with numpy : 0.0015659332275390625 nb_pixel_total : 27548 time to create 1 rle with old method : 0.03197073936462402 time for calcul the mask position with numpy : 0.001405954360961914 nb_pixel_total : 585 time to create 1 rle with old method : 0.0007245540618896484 time for calcul the mask position with numpy : 0.0014536380767822266 nb_pixel_total : 8593 time to create 1 rle with old method : 0.010354280471801758 time for calcul the mask position with numpy : 0.0014557838439941406 nb_pixel_total : 3089 time to create 1 rle with old method : 0.003866434097290039 time for calcul the mask position with numpy : 0.0014600753784179688 nb_pixel_total : 1193 time to create 1 rle with old method : 0.0015506744384765625 time for calcul the mask position with numpy : 0.0014336109161376953 nb_pixel_total : 982 time to create 1 rle with old method : 0.0012564659118652344 time for calcul the mask position with numpy : 0.0014028549194335938 nb_pixel_total : 9119 time to create 1 rle with old method : 0.01083993911743164 time for calcul the mask position with numpy : 0.0016205310821533203 nb_pixel_total : 39175 time to create 1 rle with old method : 0.045496225357055664 time for calcul the mask position with numpy : 0.0014886856079101562 nb_pixel_total : 16682 time to create 1 rle with old method : 0.020404815673828125 time for calcul the mask position with numpy : 0.0014495849609375 nb_pixel_total : 1741 time to create 1 rle with old method : 0.0021560192108154297 time for calcul the mask position with numpy : 0.0013837814331054688 nb_pixel_total : 8441 time to create 1 rle with old method : 0.009778261184692383 time for calcul the mask position with numpy : 0.0014078617095947266 nb_pixel_total : 712 time to create 1 rle with old method : 0.0009288787841796875 time for calcul the mask position with numpy : 0.0013828277587890625 nb_pixel_total : 1519 time to create 1 rle with old method : 0.0018157958984375 time for calcul the mask position with numpy : 0.001386880874633789 nb_pixel_total : 265 time to create 1 rle with old method : 0.00035071372985839844 time for calcul the mask position with numpy : 0.0014121532440185547 nb_pixel_total : 1334 time to create 1 rle with old method : 0.0016777515411376953 time for calcul the mask position with numpy : 0.0013761520385742188 nb_pixel_total : 834 time to create 1 rle with old method : 0.0010595321655273438 time for calcul the mask position with numpy : 0.0014193058013916016 nb_pixel_total : 3172 time to create 1 rle with old method : 0.003906965255737305 time for calcul the mask position with numpy : 0.001383066177368164 nb_pixel_total : 613 time to create 1 rle with old method : 0.0008220672607421875 time for calcul the mask position with numpy : 0.0014176368713378906 nb_pixel_total : 248 time to create 1 rle with old method : 0.00036406517028808594 time for calcul the mask position with numpy : 0.001390695571899414 nb_pixel_total : 4056 time to create 1 rle with old method : 0.004976034164428711 time for calcul the mask position with numpy : 0.0014164447784423828 nb_pixel_total : 1504 time to create 1 rle with old method : 0.0018351078033447266 time for calcul the mask position with numpy : 0.001416921615600586 nb_pixel_total : 976 time to create 1 rle with old method : 0.0012698173522949219 time for calcul the mask position with numpy : 0.0012955665588378906 nb_pixel_total : 221 time to create 1 rle with old method : 0.00030040740966796875 time for calcul the mask position with numpy : 0.0014126300811767578 nb_pixel_total : 734 time to create 1 rle with old method : 0.001003265380859375 time for calcul the mask position with numpy : 0.0012927055358886719 nb_pixel_total : 943 time to create 1 rle with old method : 0.0013461112976074219 time for calcul the mask position with numpy : 0.0014584064483642578 nb_pixel_total : 7498 time to create 1 rle with old method : 0.008603096008300781 time for calcul the mask position with numpy : 0.0012977123260498047 nb_pixel_total : 1438 time to create 1 rle with old method : 0.0018012523651123047 time for calcul the mask position with numpy : 0.0013225078582763672 nb_pixel_total : 300 time to create 1 rle with old method : 0.0004322528839111328 time for calcul the mask position with numpy : 0.0013697147369384766 nb_pixel_total : 3919 time to create 1 rle with old method : 0.004630565643310547 time for calcul the mask position with numpy : 0.001344442367553711 nb_pixel_total : 595 time to create 1 rle with old method : 0.0008111000061035156 time for calcul the mask position with numpy : 0.0014374256134033203 nb_pixel_total : 1463 time to create 1 rle with old method : 0.0018572807312011719 time for calcul the mask position with numpy : 0.0013301372528076172 nb_pixel_total : 1123 time to create 1 rle with old method : 0.0014450550079345703 time for calcul the mask position with numpy : 0.0014297962188720703 nb_pixel_total : 2650 time to create 1 rle with old method : 0.003326892852783203 time for calcul the mask position with numpy : 0.0013287067413330078 nb_pixel_total : 2195 time to create 1 rle with old method : 0.0028090476989746094 time for calcul the mask position with numpy : 0.0013098716735839844 nb_pixel_total : 884 time to create 1 rle with old method : 0.0011696815490722656 time for calcul the mask position with numpy : 0.0014162063598632812 nb_pixel_total : 1609 time to create 1 rle with old method : 0.002043008804321289 time for calcul the mask position with numpy : 0.0013878345489501953 nb_pixel_total : 884 time to create 1 rle with old method : 0.0010991096496582031 time for calcul the mask position with numpy : 0.001432180404663086 nb_pixel_total : 1321 time to create 1 rle with old method : 0.0016057491302490234 time for calcul the mask position with numpy : 0.0013453960418701172 nb_pixel_total : 949 time to create 1 rle with old method : 0.0012784004211425781 insert ignore into MTRPhoto.crop_hashtag_ids (photo_id, hashtag_id, `type`,x0,x1,y0,y1,score) VALUES (%s,%s,%s,%s,%s,%s,%s,%s) batch 1 Loaded 98 chid ids of type : 4677 Number RLEs to save : 8964 INSERT IGNORE INTO MTRPhoto.crop_segments (`crop_hashtag_id`, `x0`, `y0`, `length`) VALUES (%s, %s, %s , %s) first line : ('3747411148', '928', '287', '9') ... last line : ('3747411245', '322', '285', '6') INSERT IGNORE INTO MTRPhoto.crop_sum_segments (`crop_hashtag_id`, `sum_segments`) VALUES (%s, %s) TO DO : save crop sub photo not yet done ! After datou_step_exec type output : map_portfolio_photo : len 0 keys : dict_keys([]) Inside saveOutput : final : True verbose : True saveOutput not yet implemented for datou_step.type : sam we use saveGeneral [1189321094] map_info['map_portfolio_photo'] : {} final : True mtd_id 4573 list_pids : [1189321094] Looping around the photos to save general results len do output : 1 /1189321094Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4573', None, None, None, None, None, None, None, None) ('4573', None, '1189321094', None, None, None, None, None, None) begin to insert list_values into mtr_datou_result : length of list_values in save_final : 3 insert ignore into MTRPhoto.mtr_datou_result (mtd_id, mtr_portfolio_id,mtr_photo_id,result,result_long,result_double,hashtag_id,proba, mtr_current_id) values (%s,%s,%s,%s,%s,%s,%s,%s,%s) on duplicate key update mtr_portfolio_id = mtr_portfolio_id list_values : [('4573', None, '1189321094', 'None', None, None, None, None, None)] time used for this insertion : 0.015771150588989258 save_final save missing photos in datou_result : time spend for datou_step_exec : 25.992315530776978 time spend to save output : 0.016095876693725586 total time spend for step 1 : 26.008411407470703 caffe_path_current : About to save ! 2 After save, about to update current ! datou_cur_ids : [] len(datou.list_steps) : 1 output : {'1189321094': [[, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ], 'temp/1743741470_2995536_1189321094_9626af7f95d010f2a4fd524688d4ea22_76896585.png']} nb_objects detect : 98 ############################### TEST frcnn ################################ test frcnn Inside batchDatouExec : verbose : True ##### chargement datou SELECT name, created_at,limit_max FROM MTRDatou.mtr_datou WHERE id=4184 SELECT mtd.id, mtdt.`type`, mtd.`param`, mtd.param_json, mtdt.nb_input, mtdt.nb_output, mtdt.prod, mtdt.is_local, mtdt.is_datou_depend, mtdt.is_photo_id_local FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_step_types mtdt WHERE mtdt.`id`=mtd.`type` AND mtd.mtd_id=4184 SELECT mtd.id, mtd.mtd_id, mdsdt.id, mdsdt.name, mdsdt.description, msid.output_or_input, msid.data_order_id, mdsdt.type FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_steptype_io_datatypes msid, MTRDatou.mtr_datou_step_data_types mdsdt WHERE mtd.`type`=msid.`mtr_datou_step_type` AND mtd.mtd_id= 4184 AND msid.data_type=mdsdt.id SELECT mts_id_output, id_output, mts_id_input, id_input FROM MTRDatou.mtr_datou_step_by_step WHERE mtd_id=4184 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! no param json to modify List Step Type Loaded in datou : frcnn list_input_json : [] ##### fin chargement datou ##### chargement data ##### Call load_data_input : nb_thread : 5 origin SELECT photo_id, url FROM MTRBack.photos ph WHERE photo_id IN (917754606) Found this number of photos: 1 ##### Call download_photos : nb_thread : 5 begin to download photo : 917754606 download finish for photo 917754606 we have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB ##### After download_photos length of list_filenames : 1 ; length of list_pids : 1 ; length of list_args : 1 ##### After load_data_input time to download the photos : 0.17267847061157227 #### fin chargement data Blocking on flush ? No conitnuing About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : True number of steps : 1 step1:frcnn Fri Apr 4 06:38:17 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec After prepare type args : Here we display some param of map_info ! map_filenames : {'temp/1743741496_2995536_917754606_35f3c9ae49686a6be16030c6ec25c9ee.jpg': 917754606} map_photo_id_path_extension : {917754606: {'path': 'temp/1743741496_2995536_917754606_35f3c9ae49686a6be16030c6ec25c9ee.jpg', 'extension': 'jpg'}} map_subphoto_mainphoto : {} Beginning of datou step Faster rcnn ! classes : ['background', 'plaque'] pht : 4370 caffemodel_name (should be vgg16_immat_307 but not used because net loaded outside in the fonction) : {'id': 3375, 'mtr_user_id': 31, 'name': 'detection_plaque_valcor_010622', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,plaque', 'svm_portfolios_learning': '0,0', 'photo_hashtag_type': 4370, 'photo_desc_type': 5676, 'type_classification': 'caffe_faster_rcnn', 'hashtag_id_list': '0,0'} To loadFromThcl() model_param file didn't exist model_name : detection_plaque_valcor_010622 model_type : caffe_faster_rcnn list file need : ['caffemodel', 'test.prototxt'] file exist in s3 : ['caffemodel', 'test.prototxt'] file manque in s3 : [] WARNING: Logging before InitGoogleLogging() is written to STDERR F0404 06:38:19.175082 2995536 syncedmem.cpp:71] Check failed: error == cudaSuccess (2 vs. 0) out of memory *** Check failure stack trace: *** Command terminated by signal 6 31.83user 24.60system 2:55.12elapsed 32%CPU (0avgtext+0avgdata 3609128maxresident)k 4205192inputs+4680outputs (12663major+2887718minor)pagefaults 0swaps