python /home/admin/mtr/script_for_cron.py -j python_test3 -m 12 -a ' --short_python3 -v ' -s python_test3 -M 0 -S 0 -U 100,100,120 import MySQLdb succeeded Import error (python version) python version = 3 warning , we can't find thcl infos in json_data warning , we can't find pdt infos in json_data list_job_run_as_list : ['mask_detection', 'datou', 'CacheModelData_queries', 'CachePhotoData_queries', 'test_fork', 'prepare_maskdata', 'portfolio_queries', 'sla_mensuel'] python version used : 3 liste_fichiers : [('tests/mask_test', True, 'Test mask-detection ', 'mask_detection'), ('tests/datou_test', True, 'Datou All Test', 'datou', 'all'), ('mtr/database_queries/CacheModelData_queries', True, 'Test Cache Model Data', 'CacheModelData_queries'), ('tests/cache_photo_data_test', True, 'Test local_cache_photo ', 'CachePhotoData_queries'), ('mtr/mask_rcnn/prepare_maskdata', True, 'test prepare mask data', 'prepare_maskdata', 'all'), ('mtr/database_queries/portfolio_queries', True, 'test portfolio queries', 'portfolio_queries'), ('prod/memo/memo', True, 'SLA Mensuel', 'sla_mensuel', 'all')] #&_# BEGIN OF TEST : tests/mask_test #&_# /home/admin/workarea/git/Velours/python/tests/mask_test.py Test mask-detection python version used : 3 ############################### TEST memory used ################################ free memory at begining : begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 1957 run mask_detect Inside batchDatouExec : verbose : False # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! List Step Type Loaded in datou : mask_detect list_input_json : [] origin BFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 1 ; length of list_pids : 1 ; length of list_args : 1 time to download the photos : 0.13669276237487793 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : False number of steps : 1 step1:mask_detect Sun Mar 9 06:35:27 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory havn't enough memory gpu , need / 3000 l 3632 free memory gpu now : 1957 wait 20 seconds l 3637 free memory gpu now : 1957 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 /home/admin/workarea/git/Velours/python/tests/python_tests.py:11: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses import imp 2025-03-09 06:35:50.566670: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-03-09 06:35:50.591176: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-03-09 06:35:50.593267: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7ff1c8000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-03-09 06:35:50.593336: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-03-09 06:35:50.598288: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-03-09 06:35:50.761602: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x29c8d690 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-03-09 06:35:50.761653: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-03-09 06:35:50.762802: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-03-09 06:35:50.763475: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-09 06:35:50.767311: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-09 06:35:50.770098: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-03-09 06:35:50.770554: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-03-09 06:35:50.774177: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-03-09 06:35:50.775372: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-03-09 06:35:50.780920: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-09 06:35:50.782229: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-03-09 06:35:50.782299: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-09 06:35:50.783084: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-03-09 06:35:50.783106: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-03-09 06:35:50.783116: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-03-09 06:35:50.784806: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 4483 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl454 thcls : [{'id': 454, 'mtr_user_id': 31, 'name': 'mask_coco_origin', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'backgroud,person,bicycle,car,motorcycle,airplane,bus,train,truck,boat,trafficlight,firehydrant,stopsign,parkingmeter,bench,bird,cat,dog,horse,sheep,cow,elephant,bear,zebra,giraffe,backpack,umbrella,handbag,tie,suitcase,frisbee,skis,snowboard,sportsball,kite,baseballbat,baseballglove,skateboard,surfboard,tennisracket,bottle,wineglass,cup,fork,knife,spoon,bowl,banana,apple,sandwich,orange,broccoli,carrot,hotdog,pizza,donut,cake,chair,couch,pottedplant,bed,diningtable,toilet,tv,laptop,mouse,remote,keyboard,cellphone,microwave,oven,toaster,sink,refrigerator,book,clock,vase,scissors,teddybear,hairdrier,toothbrush', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 445, 'photo_desc_type': 3473, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 454, 'mtr_user_id': 31, 'name': 'mask_coco_origin', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'backgroud,person,bicycle,car,motorcycle,airplane,bus,train,truck,boat,trafficlight,firehydrant,stopsign,parkingmeter,bench,bird,cat,dog,horse,sheep,cow,elephant,bear,zebra,giraffe,backpack,umbrella,handbag,tie,suitcase,frisbee,skis,snowboard,sportsball,kite,baseballbat,baseballglove,skateboard,surfboard,tennisracket,bottle,wineglass,cup,fork,knife,spoon,bowl,banana,apple,sandwich,orange,broccoli,carrot,hotdog,pizza,donut,cake,chair,couch,pottedplant,bed,diningtable,toilet,tv,laptop,mouse,remote,keyboard,cellphone,microwave,oven,toaster,sink,refrigerator,book,clock,vase,scissors,teddybear,hairdrier,toothbrush', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 445, 'photo_desc_type': 3473, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 3473 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (3473, 'mask_coco_origin', 16384, 25088, 'mask_coco_origin', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2018, 3, 19, 10, 42, 21), datetime.datetime(2018, 3, 19, 10, 42, 21)) {'thcl': {'id': 454, 'mtr_user_id': 31, 'name': 'mask_coco_origin', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'backgroud,person,bicycle,car,motorcycle,airplane,bus,train,truck,boat,trafficlight,firehydrant,stopsign,parkingmeter,bench,bird,cat,dog,horse,sheep,cow,elephant,bear,zebra,giraffe,backpack,umbrella,handbag,tie,suitcase,frisbee,skis,snowboard,sportsball,kite,baseballbat,baseballglove,skateboard,surfboard,tennisracket,bottle,wineglass,cup,fork,knife,spoon,bowl,banana,apple,sandwich,orange,broccoli,carrot,hotdog,pizza,donut,cake,chair,couch,pottedplant,bed,diningtable,toilet,tv,laptop,mouse,remote,keyboard,cellphone,microwave,oven,toaster,sink,refrigerator,book,clock,vase,scissors,teddybear,hairdrier,toothbrush', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 445, 'photo_desc_type': 3473, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['backgroud', 'person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus', 'train', 'truck', 'boat', 'trafficlight', 'firehydrant', 'stopsign', 'parkingmeter', 'bench', 'bird', 'cat', 'dog', 'horse', 'sheep', 'cow', 'elephant', 'bear', 'zebra', 'giraffe', 'backpack', 'umbrella', 'handbag', 'tie', 'suitcase', 'frisbee', 'skis', 'snowboard', 'sportsball', 'kite', 'baseballbat', 'baseballglove', 'skateboard', 'surfboard', 'tennisracket', 'bottle', 'wineglass', 'cup', 'fork', 'knife', 'spoon', 'bowl', 'banana', 'apple', 'sandwich', 'orange', 'broccoli', 'carrot', 'hotdog', 'pizza', 'donut', 'cake', 'chair', 'couch', 'pottedplant', 'bed', 'diningtable', 'toilet', 'tv', 'laptop', 'mouse', 'remote', 'keyboard', 'cellphone', 'microwave', 'oven', 'toaster', 'sink', 'refrigerator', 'book', 'clock', 'vase', 'scissors', 'teddybear', 'hairdrier', 'toothbrush'], 'list_hashtags_csv': 'backgroud,person,bicycle,car,motorcycle,airplane,bus,train,truck,boat,trafficlight,firehydrant,stopsign,parkingmeter,bench,bird,cat,dog,horse,sheep,cow,elephant,bear,zebra,giraffe,backpack,umbrella,handbag,tie,suitcase,frisbee,skis,snowboard,sportsball,kite,baseballbat,baseballglove,skateboard,surfboard,tennisracket,bottle,wineglass,cup,fork,knife,spoon,bowl,banana,apple,sandwich,orange,broccoli,carrot,hotdog,pizza,donut,cake,chair,couch,pottedplant,bed,diningtable,toilet,tv,laptop,mouse,remote,keyboard,cellphone,microwave,oven,toaster,sink,refrigerator,book,clock,vase,scissors,teddybear,hairdrier,toothbrush', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 445, 'svm_hashtag_type_desc': 3473, 'photo_desc_type': 3473, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['backgroud', 'person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus', 'train', 'truck', 'boat', 'trafficlight', 'firehydrant', 'stopsign', 'parkingmeter', 'bench', 'bird', 'cat', 'dog', 'horse', 'sheep', 'cow', 'elephant', 'bear', 'zebra', 'giraffe', 'backpack', 'umbrella', 'handbag', 'tie', 'suitcase', 'frisbee', 'skis', 'snowboard', 'sportsball', 'kite', 'baseballbat', 'baseballglove', 'skateboard', 'surfboard', 'tennisracket', 'bottle', 'wineglass', 'cup', 'fork', 'knife', 'spoon', 'bowl', 'banana', 'apple', 'sandwich', 'orange', 'broccoli', 'carrot', 'hotdog', 'pizza', 'donut', 'cake', 'chair', 'couch', 'pottedplant', 'bed', 'diningtable', 'toilet', 'tv', 'laptop', 'mouse', 'remote', 'keyboard', 'cellphone', 'microwave', 'oven', 'toaster', 'sink', 'refrigerator', 'book', 'clock', 'vase', 'scissors', 'teddybear', 'hairdrier', 'toothbrush'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME mask_coco_origin NUM_CLASSES 81 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 2025-03-09 06:35:51.365672: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-03-09 06:35:51.365771: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-09 06:35:51.365794: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-09 06:35:51.365816: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-03-09 06:35:51.365836: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-03-09 06:35:51.365856: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-03-09 06:35:51.365888: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-03-09 06:35:51.365909: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-09 06:35:51.367109: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-03-09 06:35:51.368344: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-03-09 06:35:51.368381: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-09 06:35:51.368397: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-09 06:35:51.368411: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-03-09 06:35:51.368425: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-03-09 06:35:51.368439: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-03-09 06:35:51.368452: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-03-09 06:35:51.368466: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-09 06:35:51.369401: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-03-09 06:35:51.369435: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-03-09 06:35:51.369443: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-03-09 06:35:51.369451: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-03-09 06:35:51.370484: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 4483 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : mask_coco_origin model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-03-09 06:35:58.907027: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-09 06:35:59.005374: E tensorflow/stream_executor/cuda/cuda_blas.cc:238] failed to create cublas handle: CUBLAS_STATUS_NOT_INITIALIZED 2025-03-09 06:35:59.007683: E tensorflow/stream_executor/cuda/cuda_blas.cc:238] failed to create cublas handle: CUBLAS_STATUS_NOT_INITIALIZED 2025-03-09 06:35:59.021418: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-09 06:35:59.031694: E tensorflow/stream_executor/cuda/cuda_dnn.cc:329] Could not create cudnn handle: CUDNN_STATUS_INTERNAL_ERROR 2025-03-09 06:35:59.033015: E tensorflow/stream_executor/cuda/cuda_dnn.cc:329] Could not create cudnn handle: CUDNN_STATUS_INTERNAL_ERROR local folder : /data/models_weight/mask_coco_origin /data/models_weight/mask_coco_origin/mask_model.h5 size_local : 257557808 size in s3 : 257557808 create time local : 2021-08-09 05:27:17 create time in s3 : 2021-08-06 19:45:17 mask_model.h5 already exist and didn't need to update list_images length : 1 NEW PHOTO Processing 1 images image shape: (480, 640, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 89) min: 0.00000 max: 640.00000 error in detect the image : temp/1741498527_588358_957285035_a42482e51c93c8025d243dd179aee85b.jpg 2 root error(s) found. (0) Unknown: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above. [[node conv1/convolution (defined at usr/local/lib/python3.8/dist-packages/keras/backend/tensorflow_backend.py:3007) ]] [[mrcnn_detection/map/while/body/_1/GatherV2_3/_92]] (1) Unknown: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above. [[node conv1/convolution (defined at usr/local/lib/python3.8/dist-packages/keras/backend/tensorflow_backend.py:3007) ]] 0 successful operations. 0 derived errors ignored. [Op:__inference_keras_scratch_graph_13584] Function call stack: keras_scratch_graph -> keras_scratch_graph Detection mask done ! Trying to reset tf kernel 589708 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 22 tf kernel not reseted sub process len(results) : 0 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 0 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 853 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl454 Catched exception ! Connect or reconnect ! thcls : [{'id': 454, 'mtr_user_id': 31, 'name': 'mask_coco_origin', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'backgroud,person,bicycle,car,motorcycle,airplane,bus,train,truck,boat,trafficlight,firehydrant,stopsign,parkingmeter,bench,bird,cat,dog,horse,sheep,cow,elephant,bear,zebra,giraffe,backpack,umbrella,handbag,tie,suitcase,frisbee,skis,snowboard,sportsball,kite,baseballbat,baseballglove,skateboard,surfboard,tennisracket,bottle,wineglass,cup,fork,knife,spoon,bowl,banana,apple,sandwich,orange,broccoli,carrot,hotdog,pizza,donut,cake,chair,couch,pottedplant,bed,diningtable,toilet,tv,laptop,mouse,remote,keyboard,cellphone,microwave,oven,toaster,sink,refrigerator,book,clock,vase,scissors,teddybear,hairdrier,toothbrush', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 445, 'photo_desc_type': 3473, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 454, 'mtr_user_id': 31, 'name': 'mask_coco_origin', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'backgroud,person,bicycle,car,motorcycle,airplane,bus,train,truck,boat,trafficlight,firehydrant,stopsign,parkingmeter,bench,bird,cat,dog,horse,sheep,cow,elephant,bear,zebra,giraffe,backpack,umbrella,handbag,tie,suitcase,frisbee,skis,snowboard,sportsball,kite,baseballbat,baseballglove,skateboard,surfboard,tennisracket,bottle,wineglass,cup,fork,knife,spoon,bowl,banana,apple,sandwich,orange,broccoli,carrot,hotdog,pizza,donut,cake,chair,couch,pottedplant,bed,diningtable,toilet,tv,laptop,mouse,remote,keyboard,cellphone,microwave,oven,toaster,sink,refrigerator,book,clock,vase,scissors,teddybear,hairdrier,toothbrush', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 445, 'photo_desc_type': 3473, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 3473 ['backgroud', 'person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus', 'train', 'truck', 'boat', 'trafficlight', 'firehydrant', 'stopsign', 'parkingmeter', 'bench', 'bird', 'cat', 'dog', 'horse', 'sheep', 'cow', 'elephant', 'bear', 'zebra', 'giraffe', 'backpack', 'umbrella', 'handbag', 'tie', 'suitcase', 'frisbee', 'skis', 'snowboard', 'sportsball', 'kite', 'baseballbat', 'baseballglove', 'skateboard', 'surfboard', 'tennisracket', 'bottle', 'wineglass', 'cup', 'fork', 'knife', 'spoon', 'bowl', 'banana', 'apple', 'sandwich', 'orange', 'broccoli', 'carrot', 'hotdog', 'pizza', 'donut', 'cake', 'chair', 'couch', 'pottedplant', 'bed', 'diningtable', 'toilet', 'tv', 'laptop', 'mouse', 'remote', 'keyboard', 'cellphone', 'microwave', 'oven', 'toaster', 'sink', 'refrigerator', 'book', 'clock', 'vase', 'scissors', 'teddybear', 'hairdrier', 'toothbrush'] WARNING : results is empty ! time spent for convertir_results : 0.782435417175293 time spend for datou_step_exec : 35.35030651092529 time spend to save output : 7.414817810058594e-05 total time spend for step 1 : 35.350380659103394 caffe_path_current : About to save ! 1 Inside saveOutput : final : True verbose : False eke 12-6-18 : saveMask need to be cleaned for new output ! begin to insert list_values into mtr_datou_result : length of list_values in save_final : 1 time used for this insertion : 0.014914751052856445 save missing photos in datou_result : After save, about to update current ! datou_cur_ids : [] len(datou.list_steps) : 1 output : {'957285035': [[], 'temp/1741498527_588358_957285035_a42482e51c93c8025d243dd179aee85b.jpg']} free memory after detection : begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 853 ############################### TEST detect object ################################ run mask_detect Inside batchDatouExec : verbose : False # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! List Step Type Loaded in datou : mask_detect list_input_json : [] origin BFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 1 ; length of list_pids : 1 ; length of list_args : 1 time to download the photos : 0.12238764762878418 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : False number of steps : 1 step1:mask_detect Sun Mar 9 06:36:04 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory havn't enough memory gpu , need / 3000 l 3632 free memory gpu now : 853 wait 20 seconds l 3637 free memory gpu now : 853 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-03-09 06:36:27.777352: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-03-09 06:36:27.803210: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-03-09 06:36:27.805350: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7ff1c4000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-03-09 06:36:27.805386: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-03-09 06:36:27.809878: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-03-09 06:36:28.009205: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x2a4cabf0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-03-09 06:36:28.009260: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-03-09 06:36:28.010239: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-03-09 06:36:28.010708: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-09 06:36:28.014007: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-09 06:36:28.020261: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-03-09 06:36:28.020696: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-03-09 06:36:28.023834: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-03-09 06:36:28.025206: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-03-09 06:36:28.030380: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-09 06:36:28.031460: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-03-09 06:36:28.031567: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-09 06:36:28.032108: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-03-09 06:36:28.032125: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-03-09 06:36:28.032135: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-03-09 06:36:28.032972: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 476 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-03-09 06:36:28.116825: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-03-09 06:36:28.116959: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-09 06:36:28.116983: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-09 06:36:28.117006: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-03-09 06:36:28.117026: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-03-09 06:36:28.117046: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-03-09 06:36:28.117066: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-03-09 06:36:28.117086: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-09 06:36:28.117941: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-03-09 06:36:28.118993: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-03-09 06:36:28.119038: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-09 06:36:28.119059: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-09 06:36:28.119078: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-03-09 06:36:28.119097: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-03-09 06:36:28.119116: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-03-09 06:36:28.119135: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-03-09 06:36:28.119154: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-09 06:36:28.119987: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-03-09 06:36:28.120027: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-03-09 06:36:28.120038: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-03-09 06:36:28.120048: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-03-09 06:36:28.120947: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 476 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (3473, 'mask_coco_origin', 16384, 25088, 'mask_coco_origin', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2018, 3, 19, 10, 42, 21), datetime.datetime(2018, 3, 19, 10, 42, 21)) {'thcl': {'id': 454, 'mtr_user_id': 31, 'name': 'mask_coco_origin', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'backgroud,person,bicycle,car,motorcycle,airplane,bus,train,truck,boat,trafficlight,firehydrant,stopsign,parkingmeter,bench,bird,cat,dog,horse,sheep,cow,elephant,bear,zebra,giraffe,backpack,umbrella,handbag,tie,suitcase,frisbee,skis,snowboard,sportsball,kite,baseballbat,baseballglove,skateboard,surfboard,tennisracket,bottle,wineglass,cup,fork,knife,spoon,bowl,banana,apple,sandwich,orange,broccoli,carrot,hotdog,pizza,donut,cake,chair,couch,pottedplant,bed,diningtable,toilet,tv,laptop,mouse,remote,keyboard,cellphone,microwave,oven,toaster,sink,refrigerator,book,clock,vase,scissors,teddybear,hairdrier,toothbrush', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 445, 'photo_desc_type': 3473, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['backgroud', 'person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus', 'train', 'truck', 'boat', 'trafficlight', 'firehydrant', 'stopsign', 'parkingmeter', 'bench', 'bird', 'cat', 'dog', 'horse', 'sheep', 'cow', 'elephant', 'bear', 'zebra', 'giraffe', 'backpack', 'umbrella', 'handbag', 'tie', 'suitcase', 'frisbee', 'skis', 'snowboard', 'sportsball', 'kite', 'baseballbat', 'baseballglove', 'skateboard', 'surfboard', 'tennisracket', 'bottle', 'wineglass', 'cup', 'fork', 'knife', 'spoon', 'bowl', 'banana', 'apple', 'sandwich', 'orange', 'broccoli', 'carrot', 'hotdog', 'pizza', 'donut', 'cake', 'chair', 'couch', 'pottedplant', 'bed', 'diningtable', 'toilet', 'tv', 'laptop', 'mouse', 'remote', 'keyboard', 'cellphone', 'microwave', 'oven', 'toaster', 'sink', 'refrigerator', 'book', 'clock', 'vase', 'scissors', 'teddybear', 'hairdrier', 'toothbrush'], 'list_hashtags_csv': 'backgroud,person,bicycle,car,motorcycle,airplane,bus,train,truck,boat,trafficlight,firehydrant,stopsign,parkingmeter,bench,bird,cat,dog,horse,sheep,cow,elephant,bear,zebra,giraffe,backpack,umbrella,handbag,tie,suitcase,frisbee,skis,snowboard,sportsball,kite,baseballbat,baseballglove,skateboard,surfboard,tennisracket,bottle,wineglass,cup,fork,knife,spoon,bowl,banana,apple,sandwich,orange,broccoli,carrot,hotdog,pizza,donut,cake,chair,couch,pottedplant,bed,diningtable,toilet,tv,laptop,mouse,remote,keyboard,cellphone,microwave,oven,toaster,sink,refrigerator,book,clock,vase,scissors,teddybear,hairdrier,toothbrush', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 445, 'svm_hashtag_type_desc': 3473, 'photo_desc_type': 3473, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['backgroud', 'person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus', 'train', 'truck', 'boat', 'trafficlight', 'firehydrant', 'stopsign', 'parkingmeter', 'bench', 'bird', 'cat', 'dog', 'horse', 'sheep', 'cow', 'elephant', 'bear', 'zebra', 'giraffe', 'backpack', 'umbrella', 'handbag', 'tie', 'suitcase', 'frisbee', 'skis', 'snowboard', 'sportsball', 'kite', 'baseballbat', 'baseballglove', 'skateboard', 'surfboard', 'tennisracket', 'bottle', 'wineglass', 'cup', 'fork', 'knife', 'spoon', 'bowl', 'banana', 'apple', 'sandwich', 'orange', 'broccoli', 'carrot', 'hotdog', 'pizza', 'donut', 'cake', 'chair', 'couch', 'pottedplant', 'bed', 'diningtable', 'toilet', 'tv', 'laptop', 'mouse', 'remote', 'keyboard', 'cellphone', 'microwave', 'oven', 'toaster', 'sink', 'refrigerator', 'book', 'clock', 'vase', 'scissors', 'teddybear', 'hairdrier', 'toothbrush'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME mask_coco_origin NUM_CLASSES 81 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : mask_coco_origin model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-03-09 06:36:35.484415: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-09 06:36:35.676621: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-09 06:36:35.707908: E tensorflow/stream_executor/cuda/cuda_dnn.cc:329] Could not create cudnn handle: CUDNN_STATUS_INTERNAL_ERROR 2025-03-09 06:36:35.712645: E tensorflow/stream_executor/cuda/cuda_dnn.cc:329] Could not create cudnn handle: CUDNN_STATUS_INTERNAL_ERROR local folder : /data/models_weight/mask_coco_origin /data/models_weight/mask_coco_origin/mask_model.h5 size_local : 257557808 size in s3 : 257557808 create time local : 2021-08-09 05:27:17 create time in s3 : 2021-08-06 19:45:17 mask_model.h5 already exist and didn't need to update list_images length : 1 NEW PHOTO Processing 1 images image shape: (720, 1280, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 89) min: 0.00000 max: 1280.00000 error in detect the image : temp/1741498564_588358_917855882_da0fa7b7e6b5b551fe26c0ba8713276d.jpg 2 root error(s) found. (0) Unknown: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above. [[node conv1/convolution (defined at usr/local/lib/python3.8/dist-packages/keras/backend/tensorflow_backend.py:3007) ]] [[mrcnn_detection/map/while/body/_1/GatherV2_3/_92]] (1) Unknown: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above. [[node conv1/convolution (defined at usr/local/lib/python3.8/dist-packages/keras/backend/tensorflow_backend.py:3007) ]] 0 successful operations. 0 derived errors ignored. [Op:__inference_keras_scratch_graph_13584] Function call stack: keras_scratch_graph -> keras_scratch_graph Detection mask done ! Trying to reset tf kernel 592022 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 22 tf kernel not reseted sub process len(results) : 0 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 0 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 5734 list_Values should be empty [] ['backgroud', 'person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus', 'train', 'truck', 'boat', 'trafficlight', 'firehydrant', 'stopsign', 'parkingmeter', 'bench', 'bird', 'cat', 'dog', 'horse', 'sheep', 'cow', 'elephant', 'bear', 'zebra', 'giraffe', 'backpack', 'umbrella', 'handbag', 'tie', 'suitcase', 'frisbee', 'skis', 'snowboard', 'sportsball', 'kite', 'baseballbat', 'baseballglove', 'skateboard', 'surfboard', 'tennisracket', 'bottle', 'wineglass', 'cup', 'fork', 'knife', 'spoon', 'bowl', 'banana', 'apple', 'sandwich', 'orange', 'broccoli', 'carrot', 'hotdog', 'pizza', 'donut', 'cake', 'chair', 'couch', 'pottedplant', 'bed', 'diningtable', 'toilet', 'tv', 'laptop', 'mouse', 'remote', 'keyboard', 'cellphone', 'microwave', 'oven', 'toaster', 'sink', 'refrigerator', 'book', 'clock', 'vase', 'scissors', 'teddybear', 'hairdrier', 'toothbrush'] WARNING : results is empty ! time spent for convertir_results : 0.0002052783966064453 time spend for datou_step_exec : 34.669055223464966 time spend to save output : 2.5987625122070312e-05 total time spend for step 1 : 34.66908121109009 caffe_path_current : About to save ! 1 Inside saveOutput : final : True verbose : False eke 12-6-18 : saveMask need to be cleaned for new output ! begin to insert list_values into mtr_datou_result : length of list_values in save_final : 1 Catched exception ! Connect or reconnect ! time used for this insertion : 1.1206154823303223 save missing photos in datou_result : After save, about to update current ! datou_cur_ids : [] len(datou.list_steps) : 1 output : {'917855882': [[], 'temp/1741498564_588358_917855882_da0fa7b7e6b5b551fe26c0ba8713276d.jpg']} number of object is not correct expected : 4 got : 0 ERROR test detect objet FAILED ############################### TEST POLYGON ################################ Inside batchDatouExec : verbose : False # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! List Step Type Loaded in datou : mask_detect list_input_json : [] origin BFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 1 ; length of list_pids : 1 ; length of list_args : 1 time to download the photos : 0.13062119483947754 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : False number of steps : 1 step1:mask_detect Sun Mar 9 06:36:40 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 5955 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-03-09 06:36:42.904544: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-03-09 06:36:42.931387: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-03-09 06:36:42.933423: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7ff1c4000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-03-09 06:36:42.933484: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-03-09 06:36:42.937632: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-03-09 06:36:43.088339: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x2a4cc1d0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-03-09 06:36:43.088401: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-03-09 06:36:43.089350: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-03-09 06:36:43.089727: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-09 06:36:43.091878: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-09 06:36:43.094065: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-03-09 06:36:43.094404: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-03-09 06:36:43.096832: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-03-09 06:36:43.098054: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-03-09 06:36:43.102930: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-09 06:36:43.104208: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-03-09 06:36:43.104297: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-09 06:36:43.104952: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-03-09 06:36:43.104967: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-03-09 06:36:43.104976: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-03-09 06:36:43.106022: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 5249 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-03-09 06:36:43.191083: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-03-09 06:36:43.191239: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-09 06:36:43.191261: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-09 06:36:43.191280: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-03-09 06:36:43.191299: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-03-09 06:36:43.191317: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-03-09 06:36:43.191335: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-03-09 06:36:43.191353: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-09 06:36:43.192299: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-03-09 06:36:43.193283: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-03-09 06:36:43.193316: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-09 06:36:43.193334: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-09 06:36:43.193350: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-03-09 06:36:43.193366: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-03-09 06:36:43.193382: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-03-09 06:36:43.193397: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-03-09 06:36:43.193413: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-09 06:36:43.194401: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-03-09 06:36:43.194440: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-03-09 06:36:43.194449: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-03-09 06:36:43.194457: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-03-09 06:36:43.195524: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 5249 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (3473, 'mask_coco_origin', 16384, 25088, 'mask_coco_origin', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2018, 3, 19, 10, 42, 21), datetime.datetime(2018, 3, 19, 10, 42, 21)) {'thcl': {'id': 454, 'mtr_user_id': 31, 'name': 'mask_coco_origin', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'backgroud,person,bicycle,car,motorcycle,airplane,bus,train,truck,boat,trafficlight,firehydrant,stopsign,parkingmeter,bench,bird,cat,dog,horse,sheep,cow,elephant,bear,zebra,giraffe,backpack,umbrella,handbag,tie,suitcase,frisbee,skis,snowboard,sportsball,kite,baseballbat,baseballglove,skateboard,surfboard,tennisracket,bottle,wineglass,cup,fork,knife,spoon,bowl,banana,apple,sandwich,orange,broccoli,carrot,hotdog,pizza,donut,cake,chair,couch,pottedplant,bed,diningtable,toilet,tv,laptop,mouse,remote,keyboard,cellphone,microwave,oven,toaster,sink,refrigerator,book,clock,vase,scissors,teddybear,hairdrier,toothbrush', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 445, 'photo_desc_type': 3473, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['backgroud', 'person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus', 'train', 'truck', 'boat', 'trafficlight', 'firehydrant', 'stopsign', 'parkingmeter', 'bench', 'bird', 'cat', 'dog', 'horse', 'sheep', 'cow', 'elephant', 'bear', 'zebra', 'giraffe', 'backpack', 'umbrella', 'handbag', 'tie', 'suitcase', 'frisbee', 'skis', 'snowboard', 'sportsball', 'kite', 'baseballbat', 'baseballglove', 'skateboard', 'surfboard', 'tennisracket', 'bottle', 'wineglass', 'cup', 'fork', 'knife', 'spoon', 'bowl', 'banana', 'apple', 'sandwich', 'orange', 'broccoli', 'carrot', 'hotdog', 'pizza', 'donut', 'cake', 'chair', 'couch', 'pottedplant', 'bed', 'diningtable', 'toilet', 'tv', 'laptop', 'mouse', 'remote', 'keyboard', 'cellphone', 'microwave', 'oven', 'toaster', 'sink', 'refrigerator', 'book', 'clock', 'vase', 'scissors', 'teddybear', 'hairdrier', 'toothbrush'], 'list_hashtags_csv': 'backgroud,person,bicycle,car,motorcycle,airplane,bus,train,truck,boat,trafficlight,firehydrant,stopsign,parkingmeter,bench,bird,cat,dog,horse,sheep,cow,elephant,bear,zebra,giraffe,backpack,umbrella,handbag,tie,suitcase,frisbee,skis,snowboard,sportsball,kite,baseballbat,baseballglove,skateboard,surfboard,tennisracket,bottle,wineglass,cup,fork,knife,spoon,bowl,banana,apple,sandwich,orange,broccoli,carrot,hotdog,pizza,donut,cake,chair,couch,pottedplant,bed,diningtable,toilet,tv,laptop,mouse,remote,keyboard,cellphone,microwave,oven,toaster,sink,refrigerator,book,clock,vase,scissors,teddybear,hairdrier,toothbrush', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 445, 'svm_hashtag_type_desc': 3473, 'photo_desc_type': 3473, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['backgroud', 'person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus', 'train', 'truck', 'boat', 'trafficlight', 'firehydrant', 'stopsign', 'parkingmeter', 'bench', 'bird', 'cat', 'dog', 'horse', 'sheep', 'cow', 'elephant', 'bear', 'zebra', 'giraffe', 'backpack', 'umbrella', 'handbag', 'tie', 'suitcase', 'frisbee', 'skis', 'snowboard', 'sportsball', 'kite', 'baseballbat', 'baseballglove', 'skateboard', 'surfboard', 'tennisracket', 'bottle', 'wineglass', 'cup', 'fork', 'knife', 'spoon', 'bowl', 'banana', 'apple', 'sandwich', 'orange', 'broccoli', 'carrot', 'hotdog', 'pizza', 'donut', 'cake', 'chair', 'couch', 'pottedplant', 'bed', 'diningtable', 'toilet', 'tv', 'laptop', 'mouse', 'remote', 'keyboard', 'cellphone', 'microwave', 'oven', 'toaster', 'sink', 'refrigerator', 'book', 'clock', 'vase', 'scissors', 'teddybear', 'hairdrier', 'toothbrush'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME mask_coco_origin NUM_CLASSES 81 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : mask_coco_origin model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-03-09 06:36:53.896130: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-09 06:36:54.071171: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/mask_coco_origin /data/models_weight/mask_coco_origin/mask_model.h5 size_local : 257557808 size in s3 : 257557808 create time local : 2021-08-09 05:27:17 create time in s3 : 2021-08-06 19:45:17 mask_model.h5 already exist and didn't need to update list_images length : 1 NEW PHOTO Processing 1 images image shape: (2448, 2448, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 89) min: 0.00000 max: 2448.00000 nb d'objets trouves : 1 Detection mask done ! Trying to reset tf kernel 592939 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 445 tf kernel not reseted sub process len(results) : 1 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 1 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 5734 list_Values should be empty [] ['backgroud', 'person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus', 'train', 'truck', 'boat', 'trafficlight', 'firehydrant', 'stopsign', 'parkingmeter', 'bench', 'bird', 'cat', 'dog', 'horse', 'sheep', 'cow', 'elephant', 'bear', 'zebra', 'giraffe', 'backpack', 'umbrella', 'handbag', 'tie', 'suitcase', 'frisbee', 'skis', 'snowboard', 'sportsball', 'kite', 'baseballbat', 'baseballglove', 'skateboard', 'surfboard', 'tennisracket', 'bottle', 'wineglass', 'cup', 'fork', 'knife', 'spoon', 'bowl', 'banana', 'apple', 'sandwich', 'orange', 'broccoli', 'carrot', 'hotdog', 'pizza', 'donut', 'cake', 'chair', 'couch', 'pottedplant', 'bed', 'diningtable', 'toilet', 'tv', 'laptop', 'mouse', 'remote', 'keyboard', 'cellphone', 'microwave', 'oven', 'toaster', 'sink', 'refrigerator', 'book', 'clock', 'vase', 'scissors', 'teddybear', 'hairdrier', 'toothbrush'] time for calcul the mask position with numpy : 0.6734075546264648 nb_pixel_total : 3689007 time to create 1 rle with new method : 0.3181498050689697 length of segment : 2038 time spent for convertir_results : 2.1742634773254395 time spend for datou_step_exec : 22.124712467193604 time spend to save output : 3.3855438232421875e-05 total time spend for step 1 : 22.124746322631836 caffe_path_current : About to save ! 1 Inside saveOutput : final : True verbose : False eke 12-6-18 : saveMask need to be cleaned for new output ! Catched exception ! Connect or reconnect ! Number saved : None batch 1 Loaded 721 chid ids of type : 445 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++Number RLEs to save : 0 begin to insert list_values into mtr_datou_result : length of list_values in save_final : 1 time used for this insertion : 0.012915372848510742 save missing photos in datou_result : After save, about to update current ! datou_cur_ids : [] len(datou.list_steps) : 1 output : {'917877156': [[(917877156, 492601069, 445, 7, 2264, 122, 2241, 0.9848451, [(523, 124, 443), (1150, 124, 275), (501, 125, 949), (481, 126, 992), (461, 127, 1035), (443, 128, 1096), (427, 129, 1153), (412, 130, 1176), (397, 131, 1200), (383, 132, 1222), (370, 133, 1243), (367, 134, 1253), (364, 135, 1264), (361, 136, 1274), (359, 137, 1282), (356, 138, 1292), (354, 139, 1300), (352, 140, 1308), (349, 141, 1315), (347, 142, 1322), (345, 143, 1328), (343, 144, 1334), (341, 145, 1340), (339, 146, 1345), (338, 147, 1350), (336, 148, 1356), (334, 149, 1361), (332, 150, 1366), (331, 151, 1370), (329, 152, 1376), (328, 153, 1380), (326, 154, 1385), (325, 155, 1388), (323, 156, 1393), (322, 157, 1397), (321, 158, 1401), (320, 159, 1404), (318, 160, 1409), (317, 161, 1412), (316, 162, 1416), (315, 163, 1419), (313, 164, 1424), (312, 165, 1428), (311, 166, 1431), (309, 167, 1435), (308, 168, 1438), (306, 169, 1442), (305, 170, 1446), (303, 171, 1450), (302, 172, 1453), (300, 173, 1458), (298, 174, 1462), (297, 175, 1466), (295, 176, 1470), (293, 177, 1475), (291, 178, 1480), (289, 179, 1485), (287, 180, 1490), (284, 181, 1496), (282, 182, 1501), (279, 183, 1507), (276, 184, 1514), (274, 185, 1519), (271, 186, 1526), (268, 187, 1533), (265, 188, 1540), (263, 189, 1546), (260, 190, 1553), (257, 191, 1560), (254, 192, 1568), (252, 193, 1577), (249, 194, 1586), (246, 195, 1595), (243, 196, 1604), (240, 197, 1614), (238, 198, 1622), (235, 199, 1631), (232, 200, 1640), (229, 201, 1649), (226, 202, 1658), (223, 203, 1666), (220, 204, 1675), (217, 205, 1684), (214, 206, 1689), (211, 207, 1694), (208, 208, 1699), (206, 209, 1703), (204, 210, 1706), (203, 211, 1709), (201, 212, 1713), (199, 213, 1716), (198, 214, 1719), (196, 215, 1722), (195, 216, 1725), (193, 217, 1728), (192, 218, 1730), (190, 219, 1734), (189, 220, 1736), (188, 221, 1738), (186, 222, 1741), (185, 223, 1743), (184, 224, 1745), (183, 225, 1747), (182, 226, 1749), (181, 227, 1751), (179, 228, 1754), (178, 229, 1756), (177, 230, 1758), (176, 231, 1760), (175, 232, 1762), (174, 233, 1764), (174, 234, 1765), (173, 235, 1766), (172, 236, 1768), (171, 237, 1770), (170, 238, 1772), (169, 239, 1773), (168, 240, 1775), (167, 241, 1777), (166, 242, 1779), (165, 243, 1780), (164, 244, 1782), (163, 245, 1784), (162, 246, 1786), (161, 247, 1788), (160, 248, 1790), (159, 249, 1792), (158, 250, 1794), (157, 251, 1796), (156, 252, 1798), (155, 253, 1800), (153, 254, 1803), (152, 255, 1805), (151, 256, 1807), (150, 257, 1809), (148, 258, 1812), (147, 259, 1814), (146, 260, 1817), (144, 261, 1820), (143, 262, 1822), (141, 263, 1826), (140, 264, 1828), (138, 265, 1831), (136, 266, 1835), (135, 267, 1837), (133, 268, 1841), (131, 269, 1845), (130, 270, 1847), (128, 271, 1851), (126, 272, 1855), (125, 273, 1858), (124, 274, 1860), (122, 275, 1864), (121, 276, 1866), (120, 277, 1869), (118, 278, 1872), (117, 279, 1875), (116, 280, 1877), (115, 281, 1880), (114, 282, 1882), (113, 283, 1884), (112, 284, 1886), (111, 285, 1889), (110, 286, 1891), (109, 287, 1893), (108, 288, 1895), (107, 289, 1897), (106, 290, 1899), (106, 291, 1900), (105, 292, 1902), (104, 293, 1904), (103, 294, 1906), (102, 295, 1908), (102, 296, 1909), (101, 297, 1911), (100, 298, 1913), (100, 299, 1913), (99, 300, 1915), (98, 301, 1917), (98, 302, 1918), (97, 303, 1920), (96, 304, 1921), (96, 305, 1922), (95, 306, 1924), (95, 307, 1924), (94, 308, 1926), (93, 309, 1928), (93, 310, 1928), (92, 311, 1930), (92, 312, 1931), (92, 313, 1931), (92, 314, 1932), (91, 315, 1933), (91, 316, 1934), (91, 317, 1934), (91, 318, 1935), (90, 319, 1936), (90, 320, 1937), (90, 321, 1937), (90, 322, 1938), (89, 323, 1939), (89, 324, 1940), (89, 325, 1941), (89, 326, 1941), (88, 327, 1943), (88, 328, 1943), (88, 329, 1944), (87, 330, 1946), (87, 331, 1946), (87, 332, 1947), (87, 333, 1948), (86, 334, 1949), (86, 335, 1950), (86, 336, 1951), (86, 337, 1952), (85, 338, 1953), (85, 339, 1954), (85, 340, 1955), (84, 341, 1957), (84, 342, 1957), (84, 343, 1958), (84, 344, 1959), (83, 345, 1961), (83, 346, 1962), (83, 347, 1962), (82, 348, 1964), (82, 349, 1965), (82, 350, 1966), (82, 351, 1967), (81, 352, 1969), (81, 353, 1970), (81, 354, 1971), (80, 355, 1973), (80, 356, 1974), (80, 357, 1975), (79, 358, 1977), (79, 359, 1978), (79, 360, 1979), (79, 361, 1980), (78, 362, 1983), (78, 363, 1984), (78, 364, 1985), (77, 365, 1987), (77, 366, 1988), (77, 367, 1989), (76, 368, 1991), (76, 369, 1992), (76, 370, 1993), (76, 371, 1994), (75, 372, 1996), (75, 373, 1997), (75, 374, 1998), (74, 375, 2000), (74, 376, 2001), (74, 377, 2001), (73, 378, 2003), (73, 379, 2004), (73, 380, 2005), (72, 381, 2007), (72, 382, 2007), (72, 383, 2008), (71, 384, 2010), (71, 385, 2010), (71, 386, 2011), (70, 387, 2013), (70, 388, 2013), (70, 389, 2014), (69, 390, 2015), (69, 391, 2016), (69, 392, 2016), (69, 393, 2017), (68, 394, 2018), (68, 395, 2018), (68, 396, 2019), (67, 397, 2020), (67, 398, 2021), (67, 399, 2021), (66, 400, 2023), (66, 401, 2023), (65, 402, 2025), (65, 403, 2025), (65, 404, 2026), (64, 405, 2027), (64, 406, 2027), (64, 407, 2028), (63, 408, 2029), (63, 409, 2030), (63, 410, 2030), (62, 411, 2032), (62, 412, 2032), (61, 413, 2033), (61, 414, 2034), (61, 415, 2034), (60, 416, 2036), (60, 417, 2036), (59, 418, 2037), (59, 419, 2038), (59, 420, 2038), (58, 421, 2040), (58, 422, 2040), (57, 423, 2041), (57, 424, 2042), (56, 425, 2043), (56, 426, 2044), (55, 427, 2045), (55, 428, 2045), (55, 429, 2046), (54, 430, 2047), (54, 431, 2048), (53, 432, 2049), (53, 433, 2049), (52, 434, 2051), (52, 435, 2051), (51, 436, 2053), (51, 437, 2053), (50, 438, 2054), (50, 439, 2055), (49, 440, 2056), (48, 441, 2057), (48, 442, 2058), (47, 443, 2059), (47, 444, 2060), (46, 445, 2061), (46, 446, 2061), (45, 447, 2063), (45, 448, 2063), (44, 449, 2064), (44, 450, 2065), (43, 451, 2066), (43, 452, 2066), (42, 453, 2068), (42, 454, 2068), (41, 455, 2070), (41, 456, 2070), (41, 457, 2070), (40, 458, 2072), (40, 459, 2072), (39, 460, 2073), (39, 461, 2074), (38, 462, 2075), (38, 463, 2075), (38, 464, 2076), (38, 465, 2076), (38, 466, 2076), (38, 467, 2077), (37, 468, 2078), (37, 469, 2078), (37, 470, 2079), (37, 471, 2079), (37, 472, 2079), (37, 473, 2080), (36, 474, 2081), (36, 475, 2082), (36, 476, 2082), (36, 477, 2082), (36, 478, 2083), (36, 479, 2083), (35, 480, 2085), (35, 481, 2085), (35, 482, 2085), (35, 483, 2086), (35, 484, 2086), (35, 485, 2087), (35, 486, 2087), (34, 487, 2089), (34, 488, 2089), (34, 489, 2090), (34, 490, 2090), (34, 491, 2091), (34, 492, 2091), (34, 493, 2092), (33, 494, 2093), (33, 495, 2094), (33, 496, 2094), (33, 497, 2095), (33, 498, 2095), (33, 499, 2096), (33, 500, 2096), (32, 501, 2098), (32, 502, 2098), (32, 503, 2099), (32, 504, 2099), (32, 505, 2100), (32, 506, 2101), (32, 507, 2101), (31, 508, 2103), (31, 509, 2103), (31, 510, 2104), (31, 511, 2105), (31, 512, 2105), (31, 513, 2106), (31, 514, 2107), (31, 515, 2107), (30, 516, 2109), (30, 517, 2110), (30, 518, 2110), (30, 519, 2111), (30, 520, 2112), (30, 521, 2113), (30, 522, 2113), (29, 523, 2115), (29, 524, 2116), (29, 525, 2117), (29, 526, 2117), (29, 527, 2118), (29, 528, 2119), (29, 529, 2119), (29, 530, 2120), (28, 531, 2122), (28, 532, 2122), (28, 533, 2123), (28, 534, 2124), (28, 535, 2124), (28, 536, 2125), (28, 537, 2125), (28, 538, 2126), (28, 539, 2126), (28, 540, 2127), (27, 541, 2128), (27, 542, 2128), (27, 543, 2128), (27, 544, 2129), (27, 545, 2129), (27, 546, 2129), (27, 547, 2129), (27, 548, 2129), (27, 549, 2130), (27, 550, 2130), (27, 551, 2130), (27, 552, 2130), (27, 553, 2131), (27, 554, 2131), (27, 555, 2131), (27, 556, 2131), (27, 557, 2132), (27, 558, 2132), (27, 559, 2132), (26, 560, 2133), (26, 561, 2133), (26, 562, 2134), (26, 563, 2134), (26, 564, 2134), (26, 565, 2134), (26, 566, 2135), (26, 567, 2135), (26, 568, 2135), (26, 569, 2135), (26, 570, 2135), (26, 571, 2136), (26, 572, 2136), (26, 573, 2136), (26, 574, 2136), (26, 575, 2136), (26, 576, 2137), (26, 577, 2137), (26, 578, 2137), (26, 579, 2137), (25, 580, 2138), (25, 581, 2139), (25, 582, 2139), (25, 583, 2139), (25, 584, 2139), (25, 585, 2139), (25, 586, 2140), (25, 587, 2140), (25, 588, 2140), (25, 589, 2140), (25, 590, 2140), (25, 591, 2141), (25, 592, 2141), (25, 593, 2141), (25, 594, 2141), (25, 595, 2141), (25, 596, 2141), (25, 597, 2142), (25, 598, 2142), (25, 599, 2142), (25, 600, 2142), (24, 601, 2143), (24, 602, 2143), (24, 603, 2144), (24, 604, 2144), (24, 605, 2144), (24, 606, 2144), (24, 607, 2144), (24, 608, 2144), (24, 609, 2145), (24, 610, 2145), (24, 611, 2145), (24, 612, 2145), (24, 613, 2145), (24, 614, 2145), (24, 615, 2145), (24, 616, 2145), (24, 617, 2145), (24, 618, 2146), (24, 619, 2146), (24, 620, 2146), (24, 621, 2146), (24, 622, 2146), (23, 623, 2147), (23, 624, 2147), (23, 625, 2147), (23, 626, 2147), (23, 627, 2147), (23, 628, 2147), (23, 629, 2147), (23, 630, 2147), (23, 631, 2147), (23, 632, 2147), (23, 633, 2147), (23, 634, 2147), (23, 635, 2147), (23, 636, 2147), (23, 637, 2147), (23, 638, 2147), (23, 639, 2148), (23, 640, 2148), (23, 641, 2148), (23, 642, 2148), (23, 643, 2148), (23, 644, 2148), (23, 645, 2148), (23, 646, 2148), (22, 647, 2149), (22, 648, 2149), (22, 649, 2149), (22, 650, 2149), (22, 651, 2149), (22, 652, 2149), (22, 653, 2149), (22, 654, 2149), (22, 655, 2149), (22, 656, 2149), (22, 657, 2149), (22, 658, 2149), (22, 659, 2149), (22, 660, 2150), (22, 661, 2150), (22, 662, 2150), (22, 663, 2150), (22, 664, 2150), (22, 665, 2150), (22, 666, 2150), (22, 667, 2150), (22, 668, 2150), (22, 669, 2150), (22, 670, 2150), (22, 671, 2150), (21, 672, 2151), (21, 673, 2151), (21, 674, 2151), (21, 675, 2151), (21, 676, 2151), (21, 677, 2151), (21, 678, 2151), (21, 679, 2151), (21, 680, 2151), (21, 681, 2151), (21, 682, 2151), (21, 683, 2152), (21, 684, 2152), (21, 685, 2152), (21, 686, 2152), (21, 687, 2152), (21, 688, 2152), (21, 689, 2152), (21, 690, 2152), (21, 691, 2152), (21, 692, 2152), (21, 693, 2152), (21, 694, 2152), (21, 695, 2152), (21, 696, 2152), (21, 697, 2151), (21, 698, 2151), (22, 699, 2150), (22, 700, 2150), (22, 701, 2150), (22, 702, 2150), (22, 703, 2150), (22, 704, 2150), (22, 705, 2150), (22, 706, 2150), (22, 707, 2150), (22, 708, 2150), (22, 709, 2150), (22, 710, 2150), (22, 711, 2150), (23, 712, 2149), (23, 713, 2149), (23, 714, 2149), (23, 715, 2149), (23, 716, 2149), (23, 717, 2149), (23, 718, 2149), (23, 719, 2149), (23, 720, 2149), (23, 721, 2149), (23, 722, 2149), (23, 723, 2148), (23, 724, 2148), (24, 725, 2147), (24, 726, 2147), (24, 727, 2147), (24, 728, 2147), (24, 729, 2147), (24, 730, 2147), (24, 731, 2147), (24, 732, 2147), (24, 733, 2147), (24, 734, 2147), (24, 735, 2147), (24, 736, 2147), (25, 737, 2146), (25, 738, 2146), (25, 739, 2146), (25, 740, 2146), (25, 741, 2146), (25, 742, 2146), (25, 743, 2146), (25, 744, 2146), (25, 745, 2146), (25, 746, 2146), (25, 747, 2146), (26, 748, 2144), (26, 749, 2144), (26, 750, 2144), (26, 751, 2144), (26, 752, 2144), (26, 753, 2144), (26, 754, 2144), (26, 755, 2144), (26, 756, 2144), (26, 757, 2144), (26, 758, 2144), (26, 759, 2144), (27, 760, 2143), (27, 761, 2143), (27, 762, 2143), (27, 763, 2143), (27, 764, 2143), (27, 765, 2143), (27, 766, 2143), (27, 767, 2143), (27, 768, 2143), (27, 769, 2143), (27, 770, 2143), (27, 771, 2143), (27, 772, 2142), (27, 773, 2142), (27, 774, 2142), (27, 775, 2142), (27, 776, 2142), (27, 777, 2142), (27, 778, 2142), (27, 779, 2142), (27, 780, 2142), (27, 781, 2142), (27, 782, 2142), (27, 783, 2142), (27, 784, 2142), (27, 785, 2142), (27, 786, 2142), (27, 787, 2142), (27, 788, 2142), (27, 789, 2142), (27, 790, 2142), (27, 791, 2142), (27, 792, 2142), (27, 793, 2142), (27, 794, 2142), (26, 795, 2142), (26, 796, 2142), (26, 797, 2142), (26, 798, 2142), (26, 799, 2142), (26, 800, 2142), (26, 801, 2142), (26, 802, 2142), (26, 803, 2142), (26, 804, 2142), (26, 805, 2142), (26, 806, 2142), (26, 807, 2142), (26, 808, 2142), (26, 809, 2142), (26, 810, 2142), (26, 811, 2142), (26, 812, 2142), (26, 813, 2142), (26, 814, 2142), (26, 815, 2142), (26, 816, 2142), (26, 817, 2142), (26, 818, 2141), (26, 819, 2141), (26, 820, 2141), (26, 821, 2141), (26, 822, 2141), (26, 823, 2141), (26, 824, 2141), (26, 825, 2141), (26, 826, 2141), (26, 827, 2141), (26, 828, 2141), (26, 829, 2141), (26, 830, 2141), (26, 831, 2141), (26, 832, 2141), (26, 833, 2141), (26, 834, 2141), (26, 835, 2141), (26, 836, 2141), (26, 837, 2141), (26, 838, 2141), (26, 839, 2141), (26, 840, 2140), (26, 841, 2140), (26, 842, 2140), (26, 843, 2140), (26, 844, 2139), (26, 845, 2139), (26, 846, 2139), (26, 847, 2138), (26, 848, 2138), (26, 849, 2138), (26, 850, 2137), (26, 851, 2137), (26, 852, 2137), (26, 853, 2136), (26, 854, 2136), (26, 855, 2136), (27, 856, 2134), (27, 857, 2134), (27, 858, 2134), (27, 859, 2133), (27, 860, 2133), (27, 861, 2132), (27, 862, 2132), (27, 863, 2132), (27, 864, 2131), (27, 865, 2131), (27, 866, 2130), (27, 867, 2130), (27, 868, 2130), (27, 869, 2129), (27, 870, 2129), (27, 871, 2128), (27, 872, 2128), (27, 873, 2128), (27, 874, 2127), (27, 875, 2127), (28, 876, 2125), (28, 877, 2125), (28, 878, 2124), (28, 879, 2124), (28, 880, 2123), (28, 881, 2123), (28, 882, 2122), (28, 883, 2122), (28, 884, 2121), (28, 885, 2121), (28, 886, 2120), (28, 887, 2120), (28, 888, 2119), (28, 889, 2119), (28, 890, 2118), (28, 891, 2118), (28, 892, 2117), (28, 893, 2117), (29, 894, 2115), (29, 895, 2114), (29, 896, 2114), (29, 897, 2113), (29, 898, 2113), (29, 899, 2112), (29, 900, 2111), (29, 901, 2111), (29, 902, 2110), (29, 903, 2110), (29, 904, 2109), (29, 905, 2109), (29, 906, 2108), (29, 907, 2108), (29, 908, 2107), (29, 909, 2107), (29, 910, 2106), (29, 911, 2106), (29, 912, 2105), (30, 913, 2104), (30, 914, 2103), (30, 915, 2103), (30, 916, 2102), (30, 917, 2102), (30, 918, 2102), (30, 919, 2101), (30, 920, 2101), (30, 921, 2101), (30, 922, 2100), (29, 923, 2101), (29, 924, 2101), (29, 925, 2101), (29, 926, 2100), (29, 927, 2100), (29, 928, 2100), (29, 929, 2099), (29, 930, 2099), (29, 931, 2099), (29, 932, 2098), (29, 933, 2098), (29, 934, 2098), (29, 935, 2097), (29, 936, 2097), (29, 937, 2097), (29, 938, 2097), (29, 939, 2096), (29, 940, 2096), (29, 941, 2096), (29, 942, 2095), (29, 943, 2095), (29, 944, 2095), (29, 945, 2095), (29, 946, 2094), (29, 947, 2094), (29, 948, 2094), (29, 949, 2094), (29, 950, 2093), (29, 951, 2093), (28, 952, 2094), (28, 953, 2094), (28, 954, 2093), (28, 955, 2093), (28, 956, 2093), (28, 957, 2093), (28, 958, 2092), (28, 959, 2092), (28, 960, 2092), (28, 961, 2092), (28, 962, 2091), (28, 963, 2091), (28, 964, 2091), (28, 965, 2091), (28, 966, 2090), (28, 967, 2090), (28, 968, 2090), (28, 969, 2090), (28, 970, 2089), (28, 971, 2089), (28, 972, 2089), (28, 973, 2089), (28, 974, 2089), (28, 975, 2088), (28, 976, 2088), (28, 977, 2088), (28, 978, 2088), (28, 979, 2088), (28, 980, 2087), (28, 981, 2087), (27, 982, 2088), (27, 983, 2088), (27, 984, 2087), (27, 985, 2087), (27, 986, 2087), (27, 987, 2087), (27, 988, 2087), (27, 989, 2086), (27, 990, 2086), (27, 991, 2086), (27, 992, 2086), (27, 993, 2086), (27, 994, 2085), (27, 995, 2085), (27, 996, 2085), (27, 997, 2084), (27, 998, 2084), (27, 999, 2083), (27, 1000, 2083), (27, 1001, 2083), (27, 1002, 2082), (28, 1003, 2081), (28, 1004, 2081), (28, 1005, 2080), (28, 1006, 2080), (28, 1007, 2080), (28, 1008, 2079), (28, 1009, 2079), (28, 1010, 2079), (28, 1011, 2078), (28, 1012, 2078), (28, 1013, 2078), (28, 1014, 2077), (28, 1015, 2077), (28, 1016, 2076), (28, 1017, 2076), (28, 1018, 2076), (28, 1019, 2075), (28, 1020, 2075), (28, 1021, 2074), (28, 1022, 2074), (28, 1023, 2074), (28, 1024, 2073), (28, 1025, 2073), (28, 1026, 2073), (28, 1027, 2072), (28, 1028, 2072), (28, 1029, 2071), (28, 1030, 2071), (28, 1031, 2071), (29, 1032, 2069), (29, 1033, 2069), (29, 1034, 2068), (29, 1035, 2068), (29, 1036, 2068), (29, 1037, 2067), (29, 1038, 2067), (29, 1039, 2066), (29, 1040, 2066), (29, 1041, 2065), (29, 1042, 2065), (29, 1043, 2065), (29, 1044, 2064), (29, 1045, 2064), (29, 1046, 2063), (29, 1047, 2063), (29, 1048, 2062), (29, 1049, 2062), (29, 1050, 2062), (29, 1051, 2061), (29, 1052, 2061), (29, 1053, 2060), (29, 1054, 2060), (29, 1055, 2059), (29, 1056, 2059), (29, 1057, 2058), (29, 1058, 2058), (29, 1059, 2057), (30, 1060, 2056), (30, 1061, 2055), (30, 1062, 2055), (30, 1063, 2055), (30, 1064, 2054), (30, 1065, 2054), (30, 1066, 2053), (30, 1067, 2053), (30, 1068, 2052), (30, 1069, 2051), (30, 1070, 2051), (30, 1071, 2050), (30, 1072, 2049), (30, 1073, 2048), (30, 1074, 2048), (30, 1075, 2047), (30, 1076, 2046), (30, 1077, 2045), (30, 1078, 2044), (30, 1079, 2044), (30, 1080, 2043), (30, 1081, 2042), (30, 1082, 2041), (29, 1083, 2041), (29, 1084, 2040), (29, 1085, 2039), (29, 1086, 2038), (29, 1087, 2037), (29, 1088, 2036), (29, 1089, 2035), (29, 1090, 2034), (29, 1091, 2033), (29, 1092, 2032), (29, 1093, 2030), (29, 1094, 2029), (29, 1095, 2028), (29, 1096, 2027), (29, 1097, 2026), (29, 1098, 2025), (29, 1099, 2024), (29, 1100, 2024), (29, 1101, 2023), (29, 1102, 2022), (29, 1103, 2021), (29, 1104, 2020), (29, 1105, 2019), (29, 1106, 2018), (29, 1107, 2018), (29, 1108, 2017), (29, 1109, 2016), (29, 1110, 2015), (29, 1111, 2015), (29, 1112, 2014), (29, 1113, 2013), (29, 1114, 2013), (29, 1115, 2012), (29, 1116, 2011), (29, 1117, 2011), (29, 1118, 2010), (29, 1119, 2009), (29, 1120, 2009), (29, 1121, 2008), (29, 1122, 2007), (29, 1123, 2007), (29, 1124, 2006), (29, 1125, 2006), (29, 1126, 2005), (29, 1127, 2005), (29, 1128, 2004), (29, 1129, 2003), (29, 1130, 2003), (29, 1131, 2002), (29, 1132, 2002), (29, 1133, 2001), (29, 1134, 2001), (29, 1135, 2000), (28, 1136, 2001), (28, 1137, 2001), (28, 1138, 2000), (28, 1139, 2000), (28, 1140, 1999), (28, 1141, 1999), (28, 1142, 1998), (28, 1143, 1998), (28, 1144, 1998), (28, 1145, 1997), (28, 1146, 1997), (28, 1147, 1997), (28, 1148, 1996), (28, 1149, 1996), (28, 1150, 1996), (28, 1151, 1995), (28, 1152, 1995), (28, 1153, 1995), (29, 1154, 1993), (29, 1155, 1993), (29, 1156, 1993), (29, 1157, 1992), (29, 1158, 1992), (29, 1159, 1992), (29, 1160, 1991), (29, 1161, 1991), (29, 1162, 1991), (29, 1163, 1990), (29, 1164, 1990), (29, 1165, 1990), (29, 1166, 1989), (29, 1167, 1989), (29, 1168, 1988), (29, 1169, 1988), (29, 1170, 1988), (29, 1171, 1987), (29, 1172, 1987), (29, 1173, 1987), (29, 1174, 1986), (29, 1175, 1986), (29, 1176, 1985), (29, 1177, 1985), (29, 1178, 1985), (29, 1179, 1984), (29, 1180, 1984), (29, 1181, 1983), (29, 1182, 1983), (29, 1183, 1983), (29, 1184, 1982), (29, 1185, 1982), (29, 1186, 1981), (29, 1187, 1981), (29, 1188, 1981), (29, 1189, 1980), (29, 1190, 1980), (29, 1191, 1979), (29, 1192, 1979), (29, 1193, 1978), (29, 1194, 1978), (29, 1195, 1977), (29, 1196, 1977), (29, 1197, 1977), (29, 1198, 1976), (29, 1199, 1976), (29, 1200, 1975), (29, 1201, 1975), (29, 1202, 1974), (29, 1203, 1974), (29, 1204, 1973), (29, 1205, 1973), (29, 1206, 1972), (29, 1207, 1972), (29, 1208, 1971), (29, 1209, 1971), (29, 1210, 1970), (29, 1211, 1970), (29, 1212, 1969), (29, 1213, 1969), (29, 1214, 1968), (29, 1215, 1968), (29, 1216, 1967), (29, 1217, 1967), (29, 1218, 1966), (29, 1219, 1965), (29, 1220, 1965), (29, 1221, 1964), (29, 1222, 1964), (29, 1223, 1963), (29, 1224, 1962), (29, 1225, 1962), (29, 1226, 1961), (29, 1227, 1960), (29, 1228, 1959), (29, 1229, 1959), (29, 1230, 1958), (29, 1231, 1957), (29, 1232, 1956), (29, 1233, 1955), (29, 1234, 1955), (29, 1235, 1954), (29, 1236, 1953), (29, 1237, 1952), (29, 1238, 1951), (29, 1239, 1950), (29, 1240, 1950), (29, 1241, 1949), (29, 1242, 1948), (29, 1243, 1947), (30, 1244, 1945), (30, 1245, 1945), (30, 1246, 1944), (30, 1247, 1943), (30, 1248, 1942), (30, 1249, 1942), (30, 1250, 1941), (30, 1251, 1940), (30, 1252, 1940), (30, 1253, 1939), (30, 1254, 1938), (30, 1255, 1938), (30, 1256, 1937), (30, 1257, 1937), (30, 1258, 1936), (30, 1259, 1935), (30, 1260, 1935), (30, 1261, 1934), (30, 1262, 1934), (30, 1263, 1933), (30, 1264, 1933), (30, 1265, 1932), (30, 1266, 1932), (30, 1267, 1931), (30, 1268, 1931), (30, 1269, 1930), (30, 1270, 1930), (30, 1271, 1929), (30, 1272, 1929), (30, 1273, 1928), (30, 1274, 1928), (30, 1275, 1927), (30, 1276, 1927), (30, 1277, 1926), (30, 1278, 1926), (30, 1279, 1925), (30, 1280, 1925), (30, 1281, 1925), (30, 1282, 1924), (30, 1283, 1924), (30, 1284, 1923), (30, 1285, 1923), (30, 1286, 1923), (30, 1287, 1922), (30, 1288, 1922), (30, 1289, 1921), (30, 1290, 1921), (30, 1291, 1921), (30, 1292, 1920), (30, 1293, 1920), (30, 1294, 1920), (30, 1295, 1919), (30, 1296, 1919), (30, 1297, 1919), (30, 1298, 1919), (30, 1299, 1918), (30, 1300, 1918), (30, 1301, 1918), (30, 1302, 1918), (30, 1303, 1918), (30, 1304, 1917), (30, 1305, 1917), (30, 1306, 1917), (30, 1307, 1917), (30, 1308, 1916), (30, 1309, 1916), (30, 1310, 1916), (30, 1311, 1916), (30, 1312, 1915), (31, 1313, 1914), (31, 1314, 1914), (31, 1315, 1914), (31, 1316, 1913), (31, 1317, 1913), (31, 1318, 1913), (31, 1319, 1913), (31, 1320, 1912), (31, 1321, 1912), (31, 1322, 1912), (31, 1323, 1912), (31, 1324, 1911), (31, 1325, 1911), (31, 1326, 1911), (31, 1327, 1911), (31, 1328, 1910), (31, 1329, 1910), (31, 1330, 1910), (31, 1331, 1910), (31, 1332, 1909), (31, 1333, 1909), (31, 1334, 1909), (31, 1335, 1909), (31, 1336, 1908), (31, 1337, 1908), (31, 1338, 1908), (31, 1339, 1908), (32, 1340, 1906), (32, 1341, 1906), (32, 1342, 1906), (32, 1343, 1906), (32, 1344, 1905), (32, 1345, 1905), (32, 1346, 1905), (32, 1347, 1904), (32, 1348, 1904), (32, 1349, 1904), (32, 1350, 1904), (32, 1351, 1903), (32, 1352, 1903), (32, 1353, 1903), (32, 1354, 1903), (32, 1355, 1902), (32, 1356, 1902), (32, 1357, 1902), (32, 1358, 1901), (32, 1359, 1901), (32, 1360, 1901), (32, 1361, 1901), (32, 1362, 1900), (32, 1363, 1900), (32, 1364, 1900), (32, 1365, 1899), (32, 1366, 1899), (33, 1367, 1898), (33, 1368, 1897), (33, 1369, 1897), (33, 1370, 1897), (33, 1371, 1896), (33, 1372, 1896), (33, 1373, 1896), (33, 1374, 1895), (33, 1375, 1895), (33, 1376, 1894), (33, 1377, 1894), (33, 1378, 1893), (33, 1379, 1893), (33, 1380, 1892), (33, 1381, 1892), (33, 1382, 1891), (34, 1383, 1890), (34, 1384, 1889), (34, 1385, 1888), (34, 1386, 1888), (34, 1387, 1887), (34, 1388, 1887), (34, 1389, 1886), (34, 1390, 1886), (34, 1391, 1885), (34, 1392, 1884), (34, 1393, 1884), (34, 1394, 1883), (34, 1395, 1882), (34, 1396, 1882), (35, 1397, 1880), (35, 1398, 1879), (35, 1399, 1879), (35, 1400, 1878), (35, 1401, 1877), (35, 1402, 1877), (35, 1403, 1876), (35, 1404, 1875), (35, 1405, 1874), (35, 1406, 1873), (35, 1407, 1873), (35, 1408, 1872), (35, 1409, 1871), (36, 1410, 1869), (36, 1411, 1868), (36, 1412, 1867), (36, 1413, 1866), (36, 1414, 1865), (36, 1415, 1865), (36, 1416, 1864), (36, 1417, 1863), (36, 1418, 1862), (36, 1419, 1861), (36, 1420, 1860), (36, 1421, 1859), (36, 1422, 1859), (37, 1423, 1857), (37, 1424, 1856), (37, 1425, 1855), (37, 1426, 1855), (37, 1427, 1854), (37, 1428, 1853), (37, 1429, 1853), (37, 1430, 1852), (37, 1431, 1851), (37, 1432, 1850), (37, 1433, 1850), (37, 1434, 1849), (37, 1435, 1849), (38, 1436, 1847), (38, 1437, 1846), (38, 1438, 1846), (38, 1439, 1845), (38, 1440, 1845), (38, 1441, 1844), (38, 1442, 1843), (38, 1443, 1843), (38, 1444, 1842), (38, 1445, 1842), (38, 1446, 1841), (38, 1447, 1841), (38, 1448, 1841), (38, 1449, 1841), (38, 1450, 1840), (38, 1451, 1840), (38, 1452, 1840), (38, 1453, 1840), (39, 1454, 1838), (39, 1455, 1838), (39, 1456, 1838), (39, 1457, 1838), (39, 1458, 1837), (39, 1459, 1837), (39, 1460, 1837), (39, 1461, 1837), (39, 1462, 1836), (39, 1463, 1836), (39, 1464, 1836), (39, 1465, 1836), (39, 1466, 1835), (39, 1467, 1835), (39, 1468, 1835), (39, 1469, 1835), (39, 1470, 1835), (39, 1471, 1834), (39, 1472, 1834), (39, 1473, 1834), (39, 1474, 1834), (39, 1475, 1833), (39, 1476, 1833), (39, 1477, 1833), (39, 1478, 1833), (39, 1479, 1833), (39, 1480, 1832), (39, 1481, 1832), (39, 1482, 1832), (39, 1483, 1832), (39, 1484, 1832), (39, 1485, 1831), (39, 1486, 1831), (39, 1487, 1831), (39, 1488, 1831), (39, 1489, 1831), (39, 1490, 1830), (39, 1491, 1830), (39, 1492, 1830), (39, 1493, 1830), (39, 1494, 1830), (39, 1495, 1829), (39, 1496, 1829), (39, 1497, 1829), (40, 1498, 1828), (40, 1499, 1828), (40, 1500, 1827), (40, 1501, 1827), (40, 1502, 1827), (40, 1503, 1827), (40, 1504, 1827), (40, 1505, 1826), (40, 1506, 1826), (40, 1507, 1826), (40, 1508, 1826), (40, 1509, 1826), (40, 1510, 1825), (40, 1511, 1825), (40, 1512, 1825), (40, 1513, 1825), (40, 1514, 1825), (40, 1515, 1825), (40, 1516, 1824), (40, 1517, 1824), (40, 1518, 1824), (40, 1519, 1824), (40, 1520, 1824), (40, 1521, 1824), (40, 1522, 1823), (40, 1523, 1823), (40, 1524, 1823), (40, 1525, 1823), (40, 1526, 1823), (40, 1527, 1823), (40, 1528, 1823), (40, 1529, 1823), (40, 1530, 1822), (40, 1531, 1822), (40, 1532, 1822), (40, 1533, 1822), (40, 1534, 1822), (40, 1535, 1822), (40, 1536, 1822), (40, 1537, 1822), (40, 1538, 1821), (40, 1539, 1821), (40, 1540, 1821), (40, 1541, 1821), (41, 1542, 1820), (41, 1543, 1820), (41, 1544, 1820), (41, 1545, 1820), (41, 1546, 1820), (41, 1547, 1819), (41, 1548, 1819), (41, 1549, 1819), (41, 1550, 1819), (41, 1551, 1819), (41, 1552, 1819), (41, 1553, 1819), (41, 1554, 1819), (41, 1555, 1818), (41, 1556, 1818), (41, 1557, 1818), (41, 1558, 1818), (41, 1559, 1818), (41, 1560, 1818), (41, 1561, 1818), (41, 1562, 1818), (41, 1563, 1817), (41, 1564, 1817), (41, 1565, 1817), (41, 1566, 1817), (41, 1567, 1817), (41, 1568, 1817), (41, 1569, 1817), (41, 1570, 1817), (41, 1571, 1816), (41, 1572, 1816), (41, 1573, 1816), (41, 1574, 1816), (41, 1575, 1816), (41, 1576, 1816), (41, 1577, 1816), (41, 1578, 1816), (41, 1579, 1815), (41, 1580, 1815), (41, 1581, 1815), (41, 1582, 1815), (41, 1583, 1815), (41, 1584, 1815), (41, 1585, 1815), (41, 1586, 1815), (42, 1587, 1813), (42, 1588, 1813), (42, 1589, 1813), (42, 1590, 1813), (42, 1591, 1813), (42, 1592, 1813), (42, 1593, 1813), (42, 1594, 1813), (42, 1595, 1812), (42, 1596, 1812), (42, 1597, 1812), (42, 1598, 1812), (42, 1599, 1812), (42, 1600, 1812), (42, 1601, 1812), (42, 1602, 1812), (41, 1603, 1813), (41, 1604, 1813), (41, 1605, 1812), (41, 1606, 1812), (41, 1607, 1812), (41, 1608, 1812), (41, 1609, 1812), (41, 1610, 1812), (41, 1611, 1812), (41, 1612, 1812), (41, 1613, 1812), (41, 1614, 1811), (41, 1615, 1811), (41, 1616, 1811), (41, 1617, 1811), (41, 1618, 1811), (41, 1619, 1811), (41, 1620, 1811), (41, 1621, 1811), (41, 1622, 1811), (41, 1623, 1811), (41, 1624, 1810), (41, 1625, 1810), (41, 1626, 1810), (40, 1627, 1811), (40, 1628, 1811), (40, 1629, 1811), (40, 1630, 1811), (40, 1631, 1811), (40, 1632, 1811), (40, 1633, 1810), (40, 1634, 1810), (40, 1635, 1810), (40, 1636, 1810), (40, 1637, 1810), (40, 1638, 1810), (40, 1639, 1810), (40, 1640, 1810), (40, 1641, 1809), (40, 1642, 1809), (40, 1643, 1809), (40, 1644, 1809), (40, 1645, 1809), (40, 1646, 1809), (40, 1647, 1809), (40, 1648, 1809), (40, 1649, 1809), (40, 1650, 1808), (39, 1651, 1809), (39, 1652, 1809), (39, 1653, 1809), (39, 1654, 1809), (39, 1655, 1809), (39, 1656, 1809), (39, 1657, 1809), (39, 1658, 1808), (39, 1659, 1808), (39, 1660, 1808), (39, 1661, 1808), (39, 1662, 1808), (39, 1663, 1808), (39, 1664, 1808), (39, 1665, 1808), (39, 1666, 1807), (39, 1667, 1807), (39, 1668, 1807), (39, 1669, 1807), (39, 1670, 1807), (39, 1671, 1807), (39, 1672, 1807), (39, 1673, 1807), (39, 1674, 1806), (39, 1675, 1806), (39, 1676, 1806), (39, 1677, 1806), (39, 1678, 1805), (40, 1679, 1804), (40, 1680, 1804), (40, 1681, 1804), (40, 1682, 1804), (40, 1683, 1803), (40, 1684, 1803), (40, 1685, 1803), (41, 1686, 1802), (41, 1687, 1801), (41, 1688, 1801), (41, 1689, 1801), (41, 1690, 1801), (41, 1691, 1800), (42, 1692, 1799), (42, 1693, 1799), (42, 1694, 1799), (42, 1695, 1798), (42, 1696, 1798), (42, 1697, 1798), (43, 1698, 1797), (43, 1699, 1796), (43, 1700, 1796), (43, 1701, 1796), (43, 1702, 1795), (44, 1703, 1794), (44, 1704, 1794), (44, 1705, 1794), (44, 1706, 1793), (44, 1707, 1793), (44, 1708, 1793), (45, 1709, 1791), (45, 1710, 1791), (45, 1711, 1791), (45, 1712, 1791), (45, 1713, 1790), (45, 1714, 1790), (46, 1715, 1789), (46, 1716, 1788), (46, 1717, 1788), (46, 1718, 1788), (46, 1719, 1787), (47, 1720, 1786), (47, 1721, 1786), (47, 1722, 1786), (47, 1723, 1785), (47, 1724, 1785), (48, 1725, 1784), (48, 1726, 1783), (48, 1727, 1783), (48, 1728, 1783), (48, 1729, 1782), (49, 1730, 1781), (49, 1731, 1781), (49, 1732, 1780), (49, 1733, 1780), (50, 1734, 1778), (50, 1735, 1778), (50, 1736, 1778), (50, 1737, 1777), (50, 1738, 1777), (51, 1739, 1776), (51, 1740, 1775), (51, 1741, 1775), (51, 1742, 1775), (51, 1743, 1774), (52, 1744, 1773), (52, 1745, 1772), (52, 1746, 1772), (52, 1747, 1772), (52, 1748, 1771), (53, 1749, 1770), (53, 1750, 1769), (53, 1751, 1769), (53, 1752, 1769), (53, 1753, 1768), (53, 1754, 1768), (53, 1755, 1767), (53, 1756, 1767), (53, 1757, 1767), (53, 1758, 1766), (53, 1759, 1766), (53, 1760, 1765), (53, 1761, 1765), (53, 1762, 1764), (53, 1763, 1764), (53, 1764, 1764), (53, 1765, 1763), (53, 1766, 1763), (53, 1767, 1762), (53, 1768, 1762), (53, 1769, 1762), (53, 1770, 1761), (53, 1771, 1761), (53, 1772, 1760), (53, 1773, 1760), (53, 1774, 1760), (53, 1775, 1759), (53, 1776, 1759), (53, 1777, 1759), (53, 1778, 1758), (53, 1779, 1758), (53, 1780, 1757), (53, 1781, 1757), (53, 1782, 1757), (53, 1783, 1756), (53, 1784, 1756), (53, 1785, 1756), (53, 1786, 1755), (53, 1787, 1755), (53, 1788, 1755), (53, 1789, 1754), (53, 1790, 1754), (53, 1791, 1754), (53, 1792, 1753), (53, 1793, 1753), (53, 1794, 1753), (53, 1795, 1752), (53, 1796, 1752), (53, 1797, 1752), (53, 1798, 1751), (53, 1799, 1751), (53, 1800, 1751), (53, 1801, 1750), (53, 1802, 1750), (53, 1803, 1750), (53, 1804, 1749), (53, 1805, 1749), (54, 1806, 1748), (54, 1807, 1747), (54, 1808, 1747), (54, 1809, 1747), (54, 1810, 1747), (54, 1811, 1746), (54, 1812, 1746), (54, 1813, 1746), (54, 1814, 1745), (54, 1815, 1745), (54, 1816, 1745), (54, 1817, 1744), (54, 1818, 1744), (54, 1819, 1744), (54, 1820, 1744), (54, 1821, 1743), (54, 1822, 1743), (54, 1823, 1743), (54, 1824, 1742), (54, 1825, 1742), (55, 1826, 1741), (55, 1827, 1740), (56, 1828, 1739), (56, 1829, 1739), (57, 1830, 1737), (57, 1831, 1737), (58, 1832, 1736), (58, 1833, 1735), (59, 1834, 1734), (59, 1835, 1734), (59, 1836, 1733), (60, 1837, 1732), (60, 1838, 1732), (61, 1839, 1730), (61, 1840, 1730), (62, 1841, 1729), (62, 1842, 1728), (63, 1843, 1727), (63, 1844, 1726), (64, 1845, 1725), (64, 1846, 1725), (65, 1847, 1723), (65, 1848, 1723), (66, 1849, 1722), (66, 1850, 1721), (66, 1851, 1721), (67, 1852, 1720), (67, 1853, 1719), (68, 1854, 1718), (68, 1855, 1717), (69, 1856, 1716), (69, 1857, 1716), (70, 1858, 1714), (70, 1859, 1714), (70, 1860, 1714), (71, 1861, 1712), (71, 1862, 1712), (72, 1863, 1710), (72, 1864, 1710), (73, 1865, 1709), (73, 1866, 1708), (74, 1867, 1707), (74, 1868, 1706), (75, 1869, 1705), (75, 1870, 1705), (75, 1871, 1704), (76, 1872, 1703), (76, 1873, 1702), (77, 1874, 1701), (77, 1875, 1701), (78, 1876, 1699), (78, 1877, 1699), (78, 1878, 1698), (79, 1879, 1697), (79, 1880, 1697), (80, 1881, 1695), (80, 1882, 1695), (81, 1883, 1693), (81, 1884, 1693), (81, 1885, 1692), (82, 1886, 1691), (82, 1887, 1691), (83, 1888, 1689), (83, 1889, 1689), (84, 1890, 1687), (84, 1891, 1687), (84, 1892, 1687), (85, 1893, 1685), (85, 1894, 1685), (86, 1895, 1683), (86, 1896, 1683), (87, 1897, 1681), (87, 1898, 1681), (87, 1899, 1680), (88, 1900, 1679), (88, 1901, 1679), (88, 1902, 1678), (89, 1903, 1677), (89, 1904, 1676), (90, 1905, 1675), (90, 1906, 1674), (90, 1907, 1674), (91, 1908, 1672), (91, 1909, 1672), (91, 1910, 1671), (92, 1911, 1670), (92, 1912, 1669), (93, 1913, 1668), (93, 1914, 1667), (94, 1915, 1666), (94, 1916, 1665), (94, 1917, 1665), (95, 1918, 1663), (95, 1919, 1662), (96, 1920, 1661), (96, 1921, 1660), (97, 1922, 1659), (97, 1923, 1658), (97, 1924, 1658), (98, 1925, 1656), (98, 1926, 1655), (99, 1927, 1654), (99, 1928, 1653), (100, 1929, 1652), (100, 1930, 1651), (101, 1931, 1649), (101, 1932, 1649), (102, 1933, 1647), (102, 1934, 1647), (103, 1935, 1645), (103, 1936, 1644), (104, 1937, 1643), (104, 1938, 1642), (105, 1939, 1640), (106, 1940, 1639), (106, 1941, 1638), (107, 1942, 1636), (107, 1943, 1636), (108, 1944, 1634), (108, 1945, 1633), (109, 1946, 1632), (110, 1947, 1630), (110, 1948, 1629), (111, 1949, 1627), (112, 1950, 1625), (112, 1951, 1624), (113, 1952, 1623), (114, 1953, 1621), (114, 1954, 1620), (115, 1955, 1618), (116, 1956, 1616), (116, 1957, 1616), (117, 1958, 1614), (118, 1959, 1612), (118, 1960, 1611), (119, 1961, 1609), (120, 1962, 1608), (121, 1963, 81), (212, 1963, 1515), (122, 1964, 69), (219, 1964, 1507), (122, 1965, 59), (226, 1965, 1499), (123, 1966, 48), (233, 1966, 1492), (124, 1967, 38), (241, 1967, 1483), (125, 1968, 29), (248, 1968, 1475), (126, 1969, 20), (256, 1969, 1466), (127, 1970, 11), (264, 1970, 1458), (127, 1971, 5), (271, 1971, 1450), (279, 1972, 1441), (287, 1973, 1433), (293, 1974, 1426), (298, 1975, 1420), (303, 1976, 1414), (308, 1977, 1408), (313, 1978, 1402), (318, 1979, 1396), (324, 1980, 1389), (330, 1981, 1382), (336, 1982, 1375), (343, 1983, 1367), (350, 1984, 1359), (357, 1985, 1350), (364, 1986, 1342), (371, 1987, 1334), (375, 1988, 1329), (379, 1989, 1324), (383, 1990, 1318), (387, 1991, 1313), (391, 1992, 1308), (396, 1993, 1301), (400, 1994, 1296), (404, 1995, 1291), (409, 1996, 1284), (413, 1997, 1279), (418, 1998, 1272), (423, 1999, 1266), (428, 2000, 1259), (433, 2001, 1253), (438, 2002, 1246), (443, 2003, 1240), (449, 2004, 1232), (453, 2005, 1226), (458, 2006, 1220), (463, 2007, 1213), (468, 2008, 1206), (473, 2009, 1200), (478, 2010, 1193), (483, 2011, 1186), (489, 2012, 1178), (494, 2013, 1171), (499, 2014, 1164), (505, 2015, 1156), (510, 2016, 1099), (516, 2017, 1059), (522, 2018, 1049), (528, 2019, 1039), (533, 2020, 1030), (538, 2021, 1021), (543, 2022, 1012), (548, 2023, 1003), (553, 2024, 995), (558, 2025, 986), (563, 2026, 977), (567, 2027, 970), (572, 2028, 961), (576, 2029, 953), (581, 2030, 945), (585, 2031, 938), (589, 2032, 930), (593, 2033, 923), (597, 2034, 916), (602, 2035, 907), (606, 2036, 900), (610, 2037, 893), (613, 2038, 887), (616, 2039, 877), (618, 2040, 865), (621, 2041, 853), (623, 2042, 841), (626, 2043, 828), (628, 2044, 817), (630, 2045, 806), (633, 2046, 794), (635, 2047, 783), (637, 2048, 777), (639, 2049, 772), (641, 2050, 767), (643, 2051, 762), (645, 2052, 757), (647, 2053, 752), (648, 2054, 748), (650, 2055, 743), (652, 2056, 737), (654, 2057, 732), (656, 2058, 727), (658, 2059, 722), (659, 2060, 717), (661, 2061, 712), (663, 2062, 706), (666, 2063, 700), (668, 2064, 694), (670, 2065, 688), (672, 2066, 682), (674, 2067, 676), (677, 2068, 669), (679, 2069, 663), (682, 2070, 656), (684, 2071, 649), (687, 2072, 641), (689, 2073, 634), (692, 2074, 625), (697, 2075, 615), (701, 2076, 606), (705, 2077, 596), (710, 2078, 585), (714, 2079, 576), (718, 2080, 566), (723, 2081, 555), (727, 2082, 545), (731, 2083, 535), (735, 2084, 525), (740, 2085, 514), (744, 2086, 505), (748, 2087, 496), (752, 2088, 487), (756, 2089, 478), (760, 2090, 469), (764, 2091, 460), (768, 2092, 451), (772, 2093, 443), (776, 2094, 434), (779, 2095, 427), (782, 2096, 419), (785, 2097, 412), (789, 2098, 404), (792, 2099, 397), (795, 2100, 390), (798, 2101, 383), (801, 2102, 376), (804, 2103, 370), (808, 2104, 364), (811, 2105, 358), (814, 2106, 353), (817, 2107, 348), (820, 2108, 343), (823, 2109, 338), (826, 2110, 333), (829, 2111, 327), (832, 2112, 322), (835, 2113, 317), (838, 2114, 313), (842, 2115, 307), (845, 2116, 302), (848, 2117, 297), (851, 2118, 292), (854, 2119, 287), (857, 2120, 283), (860, 2121, 278), (863, 2122, 273), (865, 2123, 270), (868, 2124, 265), (871, 2125, 260), (874, 2126, 256), (876, 2127, 252), (879, 2128, 247), (882, 2129, 243), (885, 2130, 238), (888, 2131, 233), (891, 2132, 228), (894, 2133, 223), (898, 2134, 217), (901, 2135, 212), (904, 2136, 207), (908, 2137, 200), (911, 2138, 195), (915, 2139, 189), (918, 2140, 183), (922, 2141, 177), (926, 2142, 171), (930, 2143, 162), (933, 2144, 151), (942, 2145, 134), (952, 2146, 115), (961, 2147, 98), (972, 2148, 78), (982, 2149, 59), (993, 2150, 39), (1004, 2151, 19)], ['1022,2151,933,2144,763,2090,692,2074,609,2036,286,1972,211,1962,127,1971,54,1825,53,1749,39,1678,39,1454,29,1243,29,894,21,698,27,541,38,462,63,410,92,311,118,278,204,210,296,176,370,133,523,124,1424,124,1579,129,1691,148,1822,193,1900,205,2012,298,2082,387,2119,482,2154,540,2169,618,2166,839,2131,916,2112,993,2005,1197,1954,1279,1928,1373,1878,1446,1860,1538,1845,1673,1786,1852,1719,1973,1660,2015,1575,2016,1492,2039,1169,2104,1096,2142'])], 'temp/1741498599_588358_917877156_a9c2d4b99270c9302def4ed40606e685.jpg']} nb pixel non reg : 3692295 nb pixel common : 3683960 proportion of common points : 0.9977425964068418 [('test release memory', 'SUCCESS', True), ('test detect objet', 'ERROR in the number of the objects', False), ('test polygone', 'SUCCESS', True)] res_total : False #&_# TEST FAILED #&_# : tests/mask_test #&_# /home/admin/workarea/git/Velours/python/tests/python_tests.py refs/heads/master_7b90246f7285e983362b27a932a2f1f627b3aad9 SQL :INSERT INTO MTRAdmin.monitor_sys (name, type, server, version_code, result_str, result_bool, lien , test_group ,test_name) VALUES ('python_test3','1','marlene','refs/heads/master_7b90246f7285e983362b27a932a2f1f627b3aad9','{"mask_detection": "fail"}','0','http://marlene.fotonower-preprod.com/job/2025/March/09032025/python_test3//data_2/data_log/job/2025/March/09032025/python_test3/log-python3----short_python3--v--marlene-06:35:00.txt','mask_detection','unknown'); #&_# END OF TEST #&_# : tests/mask_test #&_# #&_# BEGIN OF TEST : tests/datou_test #&_# /home/admin/workarea/git/Velours/python/tests/datou_test.py Datou All Test python version used : 3 ############################### TEST sam ################################ TEST SAM Inside batchDatouExec : verbose : True ##### chargement datou SELECT name, created_at,limit_max FROM MTRDatou.mtr_datou WHERE id=4573 SELECT mtd.id, mtdt.`type`, mtd.`param`, mtd.param_json, mtdt.nb_input, mtdt.nb_output, mtdt.prod, mtdt.is_local, mtdt.is_datou_depend, mtdt.is_photo_id_local FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_step_types mtdt WHERE mtdt.`id`=mtd.`type` AND mtd.mtd_id=4573 SELECT mtd.id, mtd.mtd_id, mdsdt.id, mdsdt.name, mdsdt.description, msid.output_or_input, msid.data_order_id, mdsdt.type FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_steptype_io_datatypes msid, MTRDatou.mtr_datou_step_data_types mdsdt WHERE mtd.`type`=msid.`mtr_datou_step_type` AND mtd.mtd_id= 4573 AND msid.data_type=mdsdt.id SELECT mts_id_output, id_output, mts_id_input, id_input FROM MTRDatou.mtr_datou_step_by_step WHERE mtd_id=4573 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! no param json to modify List Step Type Loaded in datou : sam list_input_json : [] ##### fin chargement datou ##### chargement data ##### Call load_data_input : nb_thread : 5 origin SELECT photo_id, url FROM MTRBack.photos ph WHERE photo_id IN (1189321094) Found this number of photos: 1 ##### Call download_photos : nb_thread : 5 begin to download photo : 1189321094 download finish for photo 1189321094 we have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB ##### After download_photos length of list_filenames : 1 ; length of list_pids : 1 ; length of list_args : 1 ##### After load_data_input time to download the photos : 0.21112465858459473 #### fin chargement data Blocking on flush ? No conitnuing About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! WARNING : we have an input that is not a photo, we should get rid of it Calling datou_exec Inside datou_exec : verbose : True number of steps : 1 step1:sam Sun Mar 9 06:37:09 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec After prepare type args : Here we display some param of map_info ! map_filenames : {'temp/1741498629_588358_1189321094_9626af7f95d010f2a4fd524688d4ea22_76896585.png': 1189321094} map_photo_id_path_extension : {1189321094: {'path': 'temp/1741498629_588358_1189321094_9626af7f95d010f2a4fd524688d4ea22_76896585.png', 'extension': 'png'}} map_subphoto_mainphoto : {} Beginning of datou step sam ! pht : 4677 Inside sam : nb paths : 1 (640, 960, 3) time for calcul the mask position with numpy : 0.001978635787963867 nb_pixel_total : 3753 time to create 1 rle with old method : 0.0043256282806396484 time for calcul the mask position with numpy : 0.00147247314453125 nb_pixel_total : 8640 time to create 1 rle with old method : 0.00983738899230957 time for calcul the mask position with numpy : 0.0015118122100830078 nb_pixel_total : 13921 time to create 1 rle with old method : 0.015835285186767578 time for calcul the mask position with numpy : 0.0014386177062988281 nb_pixel_total : 2938 time to create 1 rle with old method : 0.0035576820373535156 time for calcul the mask position with numpy : 0.001432657241821289 nb_pixel_total : 2316 time to create 1 rle with old method : 0.0030210018157958984 time for calcul the mask position with numpy : 0.0014748573303222656 nb_pixel_total : 10819 time to create 1 rle with old method : 0.013063669204711914 time for calcul the mask position with numpy : 0.0014464855194091797 nb_pixel_total : 4164 time to create 1 rle with old method : 0.0051422119140625 time for calcul the mask position with numpy : 0.0014734268188476562 nb_pixel_total : 7635 time to create 1 rle with old method : 0.009243249893188477 time for calcul the mask position with numpy : 0.0014538764953613281 nb_pixel_total : 5630 time to create 1 rle with old method : 0.007300138473510742 time for calcul the mask position with numpy : 0.0014429092407226562 nb_pixel_total : 2330 time to create 1 rle with old method : 0.0029468536376953125 time for calcul the mask position with numpy : 0.0018815994262695312 nb_pixel_total : 83664 time to create 1 rle with old method : 0.09748721122741699 time for calcul the mask position with numpy : 0.0015947818756103516 nb_pixel_total : 29423 time to create 1 rle with old method : 0.035567522048950195 time for calcul the mask position with numpy : 0.0014400482177734375 nb_pixel_total : 1226 time to create 1 rle with old method : 0.0016388893127441406 time for calcul the mask position with numpy : 0.0014374256134033203 nb_pixel_total : 3954 time to create 1 rle with old method : 0.004933595657348633 time for calcul the mask position with numpy : 0.0014488697052001953 nb_pixel_total : 6615 time to create 1 rle with old method : 0.008414030075073242 time for calcul the mask position with numpy : 0.001451253890991211 nb_pixel_total : 2332 time to create 1 rle with old method : 0.0029408931732177734 time for calcul the mask position with numpy : 0.0014472007751464844 nb_pixel_total : 1735 time to create 1 rle with old method : 0.0022139549255371094 time for calcul the mask position with numpy : 0.0014514923095703125 nb_pixel_total : 2078 time to create 1 rle with old method : 0.0025734901428222656 time for calcul the mask position with numpy : 0.0015141963958740234 nb_pixel_total : 13088 time to create 1 rle with old method : 0.016021251678466797 time for calcul the mask position with numpy : 0.0015265941619873047 nb_pixel_total : 12629 time to create 1 rle with old method : 0.018481969833374023 time for calcul the mask position with numpy : 0.0015053749084472656 nb_pixel_total : 2444 time to create 1 rle with old method : 0.003070831298828125 time for calcul the mask position with numpy : 0.0015139579772949219 nb_pixel_total : 16192 time to create 1 rle with old method : 0.019227266311645508 time for calcul the mask position with numpy : 0.0014600753784179688 nb_pixel_total : 861 time to create 1 rle with old method : 0.0012314319610595703 time for calcul the mask position with numpy : 0.0014123916625976562 nb_pixel_total : 586 time to create 1 rle with old method : 0.0008206367492675781 time for calcul the mask position with numpy : 0.0014333724975585938 nb_pixel_total : 4284 time to create 1 rle with old method : 0.005765676498413086 time for calcul the mask position with numpy : 0.0014655590057373047 nb_pixel_total : 8251 time to create 1 rle with old method : 0.00984811782836914 time for calcul the mask position with numpy : 0.0014462471008300781 nb_pixel_total : 5484 time to create 1 rle with old method : 0.006597757339477539 time for calcul the mask position with numpy : 0.0014719963073730469 nb_pixel_total : 9870 time to create 1 rle with old method : 0.012066125869750977 time for calcul the mask position with numpy : 0.001508474349975586 nb_pixel_total : 16571 time to create 1 rle with old method : 0.019673585891723633 time for calcul the mask position with numpy : 0.0014681816101074219 nb_pixel_total : 7764 time to create 1 rle with old method : 0.009224414825439453 time for calcul the mask position with numpy : 0.0014221668243408203 nb_pixel_total : 1686 time to create 1 rle with old method : 0.002154111862182617 time for calcul the mask position with numpy : 0.0014181137084960938 nb_pixel_total : 735 time to create 1 rle with old method : 0.0010771751403808594 time for calcul the mask position with numpy : 0.001806020736694336 nb_pixel_total : 11923 time to create 1 rle with old method : 0.015089035034179688 time for calcul the mask position with numpy : 0.0017724037170410156 nb_pixel_total : 2727 time to create 1 rle with old method : 0.0034999847412109375 time for calcul the mask position with numpy : 0.0018949508666992188 nb_pixel_total : 38831 time to create 1 rle with old method : 0.046269893646240234 time for calcul the mask position with numpy : 0.0014495849609375 nb_pixel_total : 3561 time to create 1 rle with old method : 0.004456043243408203 time for calcul the mask position with numpy : 0.0014400482177734375 nb_pixel_total : 1325 time to create 1 rle with old method : 0.0017423629760742188 time for calcul the mask position with numpy : 0.0014481544494628906 nb_pixel_total : 3898 time to create 1 rle with old method : 0.0055084228515625 time for calcul the mask position with numpy : 0.0016748905181884766 nb_pixel_total : 5585 time to create 1 rle with old method : 0.007995367050170898 time for calcul the mask position with numpy : 0.0017018318176269531 nb_pixel_total : 2785 time to create 1 rle with old method : 0.003907918930053711 time for calcul the mask position with numpy : 0.001741170883178711 nb_pixel_total : 1022 time to create 1 rle with old method : 0.0015826225280761719 time for calcul the mask position with numpy : 0.0016493797302246094 nb_pixel_total : 3349 time to create 1 rle with old method : 0.004445314407348633 time for calcul the mask position with numpy : 0.0015282630920410156 nb_pixel_total : 812 time to create 1 rle with old method : 0.0011637210845947266 time for calcul the mask position with numpy : 0.0018498897552490234 nb_pixel_total : 10631 time to create 1 rle with old method : 0.013462066650390625 time for calcul the mask position with numpy : 0.001444101333618164 nb_pixel_total : 347 time to create 1 rle with old method : 0.0004794597625732422 time for calcul the mask position with numpy : 0.0014498233795166016 nb_pixel_total : 4125 time to create 1 rle with old method : 0.005217313766479492 time for calcul the mask position with numpy : 0.0014657974243164062 nb_pixel_total : 1648 time to create 1 rle with old method : 0.0021295547485351562 time for calcul the mask position with numpy : 0.0014276504516601562 nb_pixel_total : 1253 time to create 1 rle with old method : 0.0015659332275390625 time for calcul the mask position with numpy : 0.0014417171478271484 nb_pixel_total : 4180 time to create 1 rle with old method : 0.005292415618896484 time for calcul the mask position with numpy : 0.0014445781707763672 nb_pixel_total : 2385 time to create 1 rle with old method : 0.0031251907348632812 time for calcul the mask position with numpy : 0.0015048980712890625 nb_pixel_total : 14694 time to create 1 rle with old method : 0.018137693405151367 time for calcul the mask position with numpy : 0.001436471939086914 nb_pixel_total : 860 time to create 1 rle with old method : 0.0012149810791015625 time for calcul the mask position with numpy : 0.0014150142669677734 nb_pixel_total : 594 time to create 1 rle with old method : 0.0008511543273925781 time for calcul the mask position with numpy : 0.0014164447784423828 nb_pixel_total : 888 time to create 1 rle with old method : 0.0011730194091796875 time for calcul the mask position with numpy : 0.0014269351959228516 nb_pixel_total : 576 time to create 1 rle with old method : 0.0007789134979248047 time for calcul the mask position with numpy : 0.0014262199401855469 nb_pixel_total : 1672 time to create 1 rle with old method : 0.0022470951080322266 time for calcul the mask position with numpy : 0.0014314651489257812 nb_pixel_total : 2408 time to create 1 rle with old method : 0.003275632858276367 time for calcul the mask position with numpy : 0.0014178752899169922 nb_pixel_total : 692 time to create 1 rle with old method : 0.0009658336639404297 time for calcul the mask position with numpy : 0.001416921615600586 nb_pixel_total : 338 time to create 1 rle with old method : 0.00048661231994628906 time for calcul the mask position with numpy : 0.001430511474609375 nb_pixel_total : 2767 time to create 1 rle with old method : 0.0036172866821289062 time for calcul the mask position with numpy : 0.0014355182647705078 nb_pixel_total : 1206 time to create 1 rle with old method : 0.0015933513641357422 time for calcul the mask position with numpy : 0.0014774799346923828 nb_pixel_total : 8666 time to create 1 rle with old method : 0.010767459869384766 time for calcul the mask position with numpy : 0.001443624496459961 nb_pixel_total : 1054 time to create 1 rle with old method : 0.0014607906341552734 time for calcul the mask position with numpy : 0.0014297962188720703 nb_pixel_total : 1660 time to create 1 rle with old method : 0.0022280216217041016 time for calcul the mask position with numpy : 0.0014197826385498047 nb_pixel_total : 1074 time to create 1 rle with old method : 0.0014333724975585938 time for calcul the mask position with numpy : 0.0014789104461669922 nb_pixel_total : 875 time to create 1 rle with old method : 0.0010786056518554688 time for calcul the mask position with numpy : 0.0015347003936767578 nb_pixel_total : 16679 time to create 1 rle with old method : 0.02049422264099121 time for calcul the mask position with numpy : 0.0014672279357910156 nb_pixel_total : 3112 time to create 1 rle with old method : 0.0041234493255615234 time for calcul the mask position with numpy : 0.0016238689422607422 nb_pixel_total : 27666 time to create 1 rle with old method : 0.04040241241455078 time for calcul the mask position with numpy : 0.0018086433410644531 nb_pixel_total : 1500 time to create 1 rle with old method : 0.0018551349639892578 time for calcul the mask position with numpy : 0.0014882087707519531 nb_pixel_total : 3165 time to create 1 rle with old method : 0.0038194656372070312 time for calcul the mask position with numpy : 0.0015554428100585938 nb_pixel_total : 1512 time to create 1 rle with old method : 0.002010822296142578 time for calcul the mask position with numpy : 0.0014786720275878906 nb_pixel_total : 713 time to create 1 rle with old method : 0.0009717941284179688 time for calcul the mask position with numpy : 0.0014240741729736328 nb_pixel_total : 267 time to create 1 rle with old method : 0.000347137451171875 time for calcul the mask position with numpy : 0.0014867782592773438 nb_pixel_total : 9501 time to create 1 rle with old method : 0.011038064956665039 time for calcul the mask position with numpy : 0.0015010833740234375 nb_pixel_total : 887 time to create 1 rle with old method : 0.0011131763458251953 time for calcul the mask position with numpy : 0.0014519691467285156 nb_pixel_total : 971 time to create 1 rle with old method : 0.0012080669403076172 time for calcul the mask position with numpy : 0.0015354156494140625 nb_pixel_total : 18474 time to create 1 rle with old method : 0.020972490310668945 time for calcul the mask position with numpy : 0.001550436019897461 nb_pixel_total : 615 time to create 1 rle with old method : 0.0008206367492675781 time for calcul the mask position with numpy : 0.0014562606811523438 nb_pixel_total : 248 time to create 1 rle with old method : 0.00037407875061035156 time for calcul the mask position with numpy : 0.0015330314636230469 nb_pixel_total : 975 time to create 1 rle with old method : 0.0013034343719482422 time for calcul the mask position with numpy : 0.0014827251434326172 nb_pixel_total : 221 time to create 1 rle with old method : 0.00033545494079589844 time for calcul the mask position with numpy : 0.0014910697937011719 nb_pixel_total : 9071 time to create 1 rle with old method : 0.010471820831298828 time for calcul the mask position with numpy : 0.0015096664428710938 nb_pixel_total : 1609 time to create 1 rle with old method : 0.00206756591796875 time for calcul the mask position with numpy : 0.0017094612121582031 nb_pixel_total : 39044 time to create 1 rle with old method : 0.044519662857055664 time for calcul the mask position with numpy : 0.0015065670013427734 nb_pixel_total : 1635 time to create 1 rle with old method : 0.0021004676818847656 time for calcul the mask position with numpy : 0.0015120506286621094 nb_pixel_total : 7503 time to create 1 rle with old method : 0.009266376495361328 time for calcul the mask position with numpy : 0.0014717578887939453 nb_pixel_total : 299 time to create 1 rle with old method : 0.0005145072937011719 time for calcul the mask position with numpy : 0.001455545425415039 nb_pixel_total : 595 time to create 1 rle with old method : 0.0008499622344970703 time for calcul the mask position with numpy : 0.0014574527740478516 nb_pixel_total : 5012 time to create 1 rle with old method : 0.006475925445556641 time for calcul the mask position with numpy : 0.001683950424194336 nb_pixel_total : 1123 time to create 1 rle with old method : 0.0017931461334228516 time for calcul the mask position with numpy : 0.0014548301696777344 nb_pixel_total : 941 time to create 1 rle with old method : 0.0014312267303466797 time for calcul the mask position with numpy : 0.0014603137969970703 nb_pixel_total : 2198 time to create 1 rle with old method : 0.002949953079223633 time for calcul the mask position with numpy : 0.0014901161193847656 nb_pixel_total : 947 time to create 1 rle with old method : 0.0013027191162109375 time for calcul the mask position with numpy : 0.0014827251434326172 nb_pixel_total : 886 time to create 1 rle with old method : 0.0011906623840332031 time for calcul the mask position with numpy : 0.0014505386352539062 nb_pixel_total : 358 time to create 1 rle with old method : 0.0005095005035400391 time for calcul the mask position with numpy : 0.0014400482177734375 nb_pixel_total : 476 time to create 1 rle with old method : 0.0006778240203857422 time for calcul the mask position with numpy : 0.0014464855194091797 nb_pixel_total : 1615 time to create 1 rle with old method : 0.002088308334350586 time for calcul the mask position with numpy : 0.001550912857055664 nb_pixel_total : 11121 time to create 1 rle with old method : 0.017539262771606445 time for calcul the mask position with numpy : 0.0014967918395996094 nb_pixel_total : 1323 time to create 1 rle with old method : 0.0017437934875488281 time for calcul the mask position with numpy : 0.0014674663543701172 nb_pixel_total : 279 time to create 1 rle with old method : 0.0004210472106933594 time for calcul the mask position with numpy : 0.0014657974243164062 nb_pixel_total : 1449 time to create 1 rle with old method : 0.002004861831665039 time for calcul the mask position with numpy : 0.0015094280242919922 nb_pixel_total : 7404 time to create 1 rle with old method : 0.009601593017578125 insert ignore into MTRPhoto.crop_hashtag_ids (photo_id, hashtag_id, `type`,x0,x1,y0,y1,score) VALUES (%s,%s,%s,%s,%s,%s,%s,%s) insert ignore into MTRPhoto.crop_hashtag_ids (photo_id, hashtag_id, `type`,x0,x1,y0,y1,score) VALUES (%s,%s,%s,%s,%s,%s,%s,%s) batch 1 Loaded 103 chid ids of type : 4677 Number RLEs to save : 9587 INSERT IGNORE INTO MTRPhoto.crop_segments (`crop_hashtag_id`, `x0`, `y0`, `length`) VALUES (%s, %s, %s , %s) first line : ('3701310224', '202', '535', '8') ... last line : ('3701310326', '705', '66', '5') INSERT IGNORE INTO MTRPhoto.crop_sum_segments (`crop_hashtag_id`, `sum_segments`) VALUES (%s, %s) TO DO : save crop sub photo not yet done ! After datou_step_exec type output : map_portfolio_photo : len 0 keys : dict_keys([]) Inside saveOutput : final : True verbose : True saveOutput not yet implemented for datou_step.type : sam we use saveGeneral [1189321094] map_info['map_portfolio_photo'] : {} final : True mtd_id 4573 list_pids : [1189321094] Looping around the photos to save general results len do output : 1 /1189321094Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4573', None, None, None, None, None, None, None, None) ('4573', None, '1189321094', None, None, None, None, None, None) begin to insert list_values into mtr_datou_result : length of list_values in save_final : 3 insert ignore into MTRPhoto.mtr_datou_result (mtd_id, mtr_portfolio_id,mtr_photo_id,result,result_long,result_double,hashtag_id,proba, mtr_current_id) values (%s,%s,%s,%s,%s,%s,%s,%s,%s) on duplicate key update mtr_portfolio_id = mtr_portfolio_id list_values : [('4573', None, '1189321094', 'None', None, None, None, None, None)] time used for this insertion : 0.11339139938354492 save_final save missing photos in datou_result : time spend for datou_step_exec : 11.542450904846191 time spend to save output : 0.11369705200195312 total time spend for step 1 : 11.656147956848145 caffe_path_current : About to save ! 2 After save, about to update current ! datou_cur_ids : [] len(datou.list_steps) : 1 output : {'1189321094': [[, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ], 'temp/1741498629_588358_1189321094_9626af7f95d010f2a4fd524688d4ea22_76896585.png']} nb_objects detect : 103 ############################### TEST frcnn ################################ test frcnn Inside batchDatouExec : verbose : True ##### chargement datou SELECT name, created_at,limit_max FROM MTRDatou.mtr_datou WHERE id=4184 SELECT mtd.id, mtdt.`type`, mtd.`param`, mtd.param_json, mtdt.nb_input, mtdt.nb_output, mtdt.prod, mtdt.is_local, mtdt.is_datou_depend, mtdt.is_photo_id_local FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_step_types mtdt WHERE mtdt.`id`=mtd.`type` AND mtd.mtd_id=4184 SELECT mtd.id, mtd.mtd_id, mdsdt.id, mdsdt.name, mdsdt.description, msid.output_or_input, msid.data_order_id, mdsdt.type FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_steptype_io_datatypes msid, MTRDatou.mtr_datou_step_data_types mdsdt WHERE mtd.`type`=msid.`mtr_datou_step_type` AND mtd.mtd_id= 4184 AND msid.data_type=mdsdt.id SELECT mts_id_output, id_output, mts_id_input, id_input FROM MTRDatou.mtr_datou_step_by_step WHERE mtd_id=4184 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! no param json to modify List Step Type Loaded in datou : frcnn list_input_json : [] ##### fin chargement datou ##### chargement data ##### Call load_data_input : nb_thread : 5 origin SELECT photo_id, url FROM MTRBack.photos ph WHERE photo_id IN (917754606) Found this number of photos: 1 ##### Call download_photos : nb_thread : 5 begin to download photo : 917754606 download finish for photo 917754606 we have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB ##### After download_photos length of list_filenames : 1 ; length of list_pids : 1 ; length of list_args : 1 ##### After load_data_input time to download the photos : 0.14650511741638184 #### fin chargement data Blocking on flush ? No conitnuing About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : True number of steps : 1 step1:frcnn Sun Mar 9 06:37:21 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec After prepare type args : Here we display some param of map_info ! map_filenames : {'temp/1741498641_588358_917754606_35f3c9ae49686a6be16030c6ec25c9ee.jpg': 917754606} map_photo_id_path_extension : {917754606: {'path': 'temp/1741498641_588358_917754606_35f3c9ae49686a6be16030c6ec25c9ee.jpg', 'extension': 'jpg'}} map_subphoto_mainphoto : {} Beginning of datou step Faster rcnn ! classes : ['background', 'plaque'] pht : 4370 caffemodel_name (should be vgg16_immat_307 but not used because net loaded outside in the fonction) : {'id': 3375, 'mtr_user_id': 31, 'name': 'detection_plaque_valcor_010622', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,plaque', 'svm_portfolios_learning': '0,0', 'photo_hashtag_type': 4370, 'photo_desc_type': 5676, 'type_classification': 'caffe_faster_rcnn', 'hashtag_id_list': '0,0'} To loadFromThcl() model_param file didn't exist model_name : detection_plaque_valcor_010622 model_type : caffe_faster_rcnn list file need : ['caffemodel', 'test.prototxt'] file exist in s3 : ['caffemodel', 'test.prototxt'] file manque in s3 : [] local folder : /data/models_weight/detection_plaque_valcor_010622 /data/models_weight/detection_plaque_valcor_010622/caffemodel size_local : 349723073 size in s3 : 349723073 create time local : 2022-07-12 14:12:27 create time in s3 : 2022-06-01 15:05:56 caffemodel already exist and didn't need to update /data/models_weight/detection_plaque_valcor_010622/test.prototxt size_local : 7163 size in s3 : 7163 create time local : 2022-07-12 14:12:27 create time in s3 : 2022-06-01 15:05:55 test.prototxt already exist and didn't need to update prototxt : /data/models_weight/detection_plaque_valcor_010622/test.prototxt caffemodel : /data/models_weight/detection_plaque_valcor_010622/caffemodel Loaded network /data/models_weight/detection_plaque_valcor_010622/caffemodel About to compute detect_faster_rcnn : len(args) : 1 Inside frcnn step exec : nb paths : 1 image_path : temp/1741498641_588358_917754606_35f3c9ae49686a6be16030c6ec25c9ee.jpg image_size (600, 800, 3) [[[ 4 6 6] [ 5 7 7] [ 6 8 8] ... [207 215 214] [206 214 213] [206 214 213]] [[ 4 6 6] [ 5 7 7] [ 6 8 8] ... [207 215 214] [206 214 213] [206 214 213]] [[ 4 6 6] [ 5 7 7] [ 6 8 8] ... [207 215 214] [206 214 213] [206 214 213]] ... [[ 14 16 16] [ 13 15 15] [ 11 13 13] ... [198 206 205] [198 206 205] [198 206 205]] [[ 16 18 18] [ 14 16 16] [ 11 13 13] ... [206 214 213] [206 214 213] [206 214 213]] [[ 13 15 15] [ 12 14 14] [ 9 11 11] ... [210 218 217] [210 218 217] [210 218 217]]] Detection took 0.076s for 300 object proposals c : plaque list_crops.shape (72, 5) proba : 0.06383991 (374.12686, 293.91937, 430.81003, 317.8086) proba : 0.052222025 (382.1776, 297.1887, 552.3576, 344.658) proba : 0.012270994 (345.35666, 272.4297, 468.85776, 320.72418) We are managing local photo_id len de result frcnn : 1 After datou_step_exec type output : time spend for datou_step_exec : 2.3898708820343018 time spend to save output : 6.866455078125e-05 total time spend for step 1 : 2.389939546585083 caffe_path_current : About to save ! 1 Inside saveOutput : final : True verbose : True Inside saveFrcnn : final : True verbose : True threshold to save the result : 0.1 output flattener : [(0, 493029425, 4370, 374, 430, 293, 317, 0.06383991, None), (0, 493029425, 4370, 382, 552, 297, 344, 0.052222025, None), (0, 493029425, 4370, 345, 468, 272, 320, 0.012270994, None)] Warning : no hashtag_ids to insert in the database final : True begin to insert list_values into mtr_datou_result : length of list_values in save_final : 1 insert ignore into MTRPhoto.mtr_datou_result (mtd_id, mtr_portfolio_id,mtr_photo_id,result,result_long,result_double,hashtag_id,proba, mtr_current_id) values (%s,%s,%s,%s,%s,%s,%s,%s,%s) on duplicate key update mtr_portfolio_id = mtr_portfolio_id list_values : [('4184', None, '917754606', '0', 0, '0', 493061979, '0', None)] time used for this insertion : 0.014064788818359375 [917754606] map_info['map_portfolio_photo'] : {} final : True mtd_id 4184 list_pids : [917754606] Looping around the photos to save general results len do output : 1 /0 before output type Managing all output in save final without adding information in the mtr_datou_result ('4184', None, None, None, None, None, None, None, None) ('4184', None, '917754606', None, None, None, None, None, None) begin to insert list_values into mtr_datou_result : length of list_values in save_final : 1 insert ignore into MTRPhoto.mtr_datou_result (mtd_id, mtr_portfolio_id,mtr_photo_id,result,result_long,result_double,hashtag_id,proba, mtr_current_id) values (%s,%s,%s,%s,%s,%s,%s,%s,%s) on duplicate key update mtr_portfolio_id = mtr_portfolio_id list_values : [('4184', None, '917754606', None, None, None, None, None, None)] time used for this insertion : 0.012830018997192383 save_final save missing photos in datou_result : After save, about to update current ! datou_cur_ids : [] len(datou.list_steps) : 1 output : {0: [[(0, 493029425, 4370, 374, 430, 293, 317, 0.06383991, None), (0, 493029425, 4370, 382, 552, 297, 344, 0.052222025, None), (0, 493029425, 4370, 345, 468, 272, 320, 0.012270994, None)], 'temp/1741498641_588358_917754606_35f3c9ae49686a6be16030c6ec25c9ee.jpg']} ############################### TEST thcl ################################ TEST THCL Inside batchDatouExec : verbose : True ##### chargement datou SELECT name, created_at,limit_max FROM MTRDatou.mtr_datou WHERE id=2 SELECT mtd.id, mtdt.`type`, mtd.`param`, mtd.param_json, mtdt.nb_input, mtdt.nb_output, mtdt.prod, mtdt.is_local, mtdt.is_datou_depend, mtdt.is_photo_id_local FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_step_types mtdt WHERE mtdt.`id`=mtd.`type` AND mtd.mtd_id=2 SELECT mtd.id, mtd.mtd_id, mdsdt.id, mdsdt.name, mdsdt.description, msid.output_or_input, msid.data_order_id, mdsdt.type FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_steptype_io_datatypes msid, MTRDatou.mtr_datou_step_data_types mdsdt WHERE mtd.`type`=msid.`mtr_datou_step_type` AND mtd.mtd_id= 2 AND msid.data_type=mdsdt.id SELECT mts_id_output, id_output, mts_id_input, id_input FROM MTRDatou.mtr_datou_step_by_step WHERE mtd_id=2 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : step 1 thcl is not linked in the step_by_step architecture ! WARNING : step 2 argmax is not linked in the step_by_step architecture ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! DataTypes for each output/input checked ! no param json to modify List Step Type Loaded in datou : thcl, argmax list_input_json : [] ##### fin chargement datou ##### chargement data ##### Call load_data_input : nb_thread : 5 origin SELECT photo_id, url FROM MTRBack.photos ph WHERE photo_id IN (916235064) Found this number of photos: 1 ##### Call download_photos : nb_thread : 5 begin to download photo : 916235064 download finish for photo 916235064 we have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB ##### After download_photos length of list_filenames : 1 ; length of list_pids : 1 ; length of list_args : 1 ##### After load_data_input time to download the photos : 0.09055948257446289 #### fin chargement data Blocking on flush ? No conitnuing About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : True number of steps : 2 step1:thcl Sun Mar 9 06:37:24 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec After prepare type args : Here we display some param of map_info ! map_filenames : {'temp/1741498643_588358_916235064_6293d1bb790dc6902450e7c572b7d10b.jpg': 916235064} map_photo_id_path_extension : {916235064: {'path': 'temp/1741498643_588358_916235064_6293d1bb790dc6902450e7c572b7d10b.jpg', 'extension': 'jpg'}} map_subphoto_mainphoto : {} Beginning of datou step Thcl ! multi_thcl or not :False multi_thcl_cond or not :False dic_thcl : {'355': 1} we are using the classfication for only one thcl 355 In convert_file_to_np l 337 : 1 l343 1 l357 after caffe.io.load_image dimension du image : (3, (66, 66, 3)) dimension displayed ! time to import caffe and check if the image exist : 0.008196353912353516 time to convert the images to numpy array : 0.0011203289031982422 total time to convert the images to numpy array : 0.009659290313720703 list photo_ids error: [] list photo_ids correct : [916235064] number of photos to traite : 1 try to delete the photos incorrect in DB tagging for thcl : 355 To do loadFromThcl(), then load ParamDescType : thcl355 get_desc_type_from_thcl : type of cat SELECT id, mtr_user_id, name, pb_hashtag_id, hashtag_id_list, button_legend_list, portfolio_id_lists, photo_hashtag_type, photo_desc_type, svm_limit, limit_tagging, is_public, live, created_at, updated_at, type_classification FROM MTRDatou.classification_theme WHERE `id` IN (355) thcls : [{'id': 355, 'mtr_user_id': 31, 'name': 'car_360_1027', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'c_elysee_1027_gao__port_506302,mokka_1027_gao__port_506374,captur_1027_gao__port_506399,sorento_1027_gao__port_506192,navara_1027_gao__port_506205,xc90_1027_gao__port_506350,saxo_1027_gao__port_506052,trafic_1027_gao__port_506295,punto_evo_1027_gao__port_506066,5_1027_gao__port_506117,250_1027_gao__port_506065,d_max_1027_gao__port_506125,panamera_1027_gao__port_506387,alhambra_1027_gao__port_506381,x6_1027_gao__port_506349,vitara_1027_gao__port_506328,fiesta_1027_gao__port_506377,qashqai_1027_gao__port_506286,147_1027_gao__port_506124,c5_1027_gao__port_506172,q5_1027_gao__port_506206,giulia_1027_gao__port_506178,karl_1027_gao__port_506371,mehari_1027_gao__port_506076,911_1027_gao__port_506114,508_1027_gao__port_506329,idea_1027_gao__port_506122,megane_1027_gao__port_506220,ghibli_1027_gao__port_506174,touareg_1027_gao__port_506224,i10_1027_gao__port_506232,jumper_1027_gao__port_506234,classe_clk_1027_gao__port_506173,kuga_1027_gao__port_506181,ct_1027_gao__port_506323,leon_1027_gao__port_506326,ds5_1027_gao__port_506376,cordoba_1027_gao__port_506048,classe_cla_1027_gao__port_506400,jumpy_1027_gao__port_506179,avensis_1027_gao__port_506311,juke_1027_gao__port_506325,4008_1027_gao__port_506402,190_series_1027_gao__port_506051,serie_3_1027_gao__port_506294,q7_1027_gao__port_506318,glc_1027_gao__port_506303,grand_vitara_1027_gao__port_506175,s40_1027_gao__port_506099,toledo_1027_gao__port_506061,5008_1027_gao__port_506337,continental_1027_gao__port_506250,coupe_1027_gao__port_506082,iq_1027_gao__port_506166,407_1027_gao__port_506133,touran_1027_gao__port_506308,300c_1027_gao__port_506078,classe_gl_1027_gao__port_506340,vivaro_1027_gao__port_506310,sl_1027_gao__port_506100,elise_1027_gao__port_506121,1007_1027_gao__port_506070,i40_1027_gao__port_506218,bipper_tepee_1027_gao__port_506227,focus_1027_gao__port_506272,primera_1027_gao__port_506147,r4_1027_gao__port_506160,a8_1027_gao__port_506265,boxer_1027_gao__port_506202,s5_1027_gao__port_506222,r21_1027_gao__port_506093,c3_1027_gao__port_506257,santa_fe_1027_gao__port_506208,m4_1027_gao__port_506344,safrane_1027_gao__port_506077,classe_gle_1027_gao__port_506395,0_1027_gao__port_506094,ix35_1027_gao__port_506219,carens_1027_gao__port_506298,classe_a_1027_gao__port_506339,ix20_1027_gao__port_506343,note_1027_gao__port_506365,a5_1027_gao__port_506200,sx4_1027_gao__port_506348,sandero_1027_gao__port_506198,3008_1027_gao__port_506385,q50_1027_gao__port_506239,latitude_1027_gao__port_506236,v40_1027_gao__port_506391,xsara_1027_gao__port_506087,grand_c_max_1027_gao__port_506342,swift_1027_gao__port_506149,serie_1_1027_gao__port_506184,xc70_1027_gao__port_506393,master_1027_gao__port_506203,clio_1027_gao__port_506280,duster_1027_gao__port_506216,traveller_1027_gao__port_506403,tipo_1027_gao__port_506355,rav_4_1027_gao__port_506332,coccinelle_1027_gao__port_506259,spacetourer_1027_gao__port_506401,xe_1027_gao__port_506357,ds3_1027_gao__port_506324,mx_5_1027_gao__port_506098,land_cruiser_1027_gao__port_506315,classe_b_1027_gao__port_506335,806_1027_gao__port_506088,rx_8_1027_gao__port_506046,spark_1027_gao__port_506185,6_1027_gao__port_506171,bravo_1027_gao__port_506080,nx_1027_gao__port_506345,sharan_1027_gao__port_506347,x_type_1027_gao__port_506067,jimny_1027_gao__port_506233,wrangler_1027_gao__port_506225,c_crosser_1027_gao__port_506312,v70_1027_gao__port_506278,classe_e_1027_gao__port_506300,classe_v_1027_gao__port_506258,m3_1027_gao__port_506182,abarth_500_1027_gao__port_506226,serie_6_1027_gao__port_506262,modus_1027_gao__port_506146,3_1027_gao__port_506113,405_1027_gao__port_506108,allroad_1027_gao__port_506297,auris_1027_gao__port_506322,galaxy_1027_gao__port_506143,giulietta_1027_gao__port_506363,106_1027_gao__port_506073,classe_m_1027_gao__port_506154,espace_1027_gao__port_506313,panda_1027_gao__port_506189,rcz_1027_gao__port_506197,4007_1027_gao__port_506162,classe_cl_1027_gao__port_506249,leaf_1027_gao__port_506139,octavia_1027_gao__port_506237,ds4_1027_gao__port_506336,freelander_1027_gao__port_506084,evasion_1027_gao__port_506109,punto_1027_gao__port_506106,2cv_1027_gao__port_506045,x4_1027_gao__port_506392,antara_1027_gao__port_506247,murano_1027_gao__port_506316,alto_1027_gao__port_506201,meriva_1027_gao__port_506353,orlando_1027_gao__port_506305,new_beetle_1027_gao__port_506050,306_1027_gao__port_506145,tiguan_1027_gao__port_506362,s_type_1027_gao__port_506101,c1_1027_gao__port_506128,vectra_1027_gao__port_506044,outlander_1027_gao__port_506317,307_1027_gao__port_506074,a6_s6_1027_gao__port_506134,nemo_combi_1027_gao__port_506196,berlingo_1027_gao__port_506194,partner_1027_gao__port_506285,cayenne_1027_gao__port_506177,quattroporte_1027_gao__port_506240,c_max_1027_gao__port_506282,fabia_1027_gao__port_506396,cx_3_1027_gao__port_506281,x_trail_1027_gao__port_506264,scirocco_1027_gao__port_506276,matiz_1027_gao__port_506144,tigra_1027_gao__port_506069,escort_1027_gao__port_506091,c2_1027_gao__port_506081,mini_1027_gao__port_506168,i30_1027_gao__port_506291,picanto_1027_gao__port_506238,mito_1027_gao__port_506072,impreza_1027_gao__port_506085,kangoo_1027_gao__port_506235,a4_1027_gao__port_506193,cayman_1027_gao__port_506268,sportage_1027_gao__port_506148,up_1027_gao__port_506356,optima_1027_gao__port_506386,defender_1027_gao__port_506229,serie_2_1027_gao__port_506256,edge_1027_gao__port_506187,r19_1027_gao__port_506110,jetta_1027_gao__port_506304,eos_1027_gao__port_506115,accord_1027_gao__port_506214,yaris_1027_gao__port_506334,classe_cls_1027_gao__port_506289,polo_1027_gao__port_506361,serie_4_1027_gao__port_506366,mini_cabriolet_1027_gao__port_506204,prius_1027_gao__port_506190,lodgy_1027_gao__port_506188,serie_7_1027_gao__port_506307,c15_1027_gao__port_506055,kadjar_1027_gao__port_506389,insignia_1027_gao__port_506364,308_1027_gao__port_506279,roomster_1027_gao__port_506241,80_1027_gao__port_506057,309_1027_gao__port_506063,tucson_1027_gao__port_506320,x3_1027_gao__port_506212,xf_1027_gao__port_506263,2008_1027_gao__port_506394,passat_1027_gao__port_506306,compass_1027_gao__port_506260,twingo_1027_gao__port_506309,micra_1027_gao__port_506221,golf_1027_gao__port_506155,soul_1027_gao__port_506176,rapid_1027_gao__port_506398,forester_1027_gao__port_506360,slk_1027_gao__port_506210,forfour_1027_gao__port_506341,serie_5_1027_gao__port_506209,xj_1027_gao__port_506170,pajero_1027_gao__port_506097,agila_1027_gao__port_506119,a6_1027_gao__port_506163,fox_1027_gao__port_506092,boxster_1027_gao__port_506267,altea_1027_gao__port_506246,samurai_1027_gao__port_506047,trax_1027_gao__port_506296,getz_1027_gao__port_506058,cherokee_1027_gao__port_506269,koleos_1027_gao__port_506378,z_series_1027_gao__port_506123,ecosport_1027_gao__port_506271,space_star_1027_gao__port_506277,rs3_sportback_1027_gao__port_506207,civic_1027_gao__port_506141,talisman_1027_gao__port_506390,f_pace_1027_gao__port_506314,classe_c_1027_gao__port_506299,tt_1027_gao__port_506075,pathfinder_1027_gao__port_506183,156_1027_gao__port_506157,cx_5_1027_gao__port_506228,scenic_1027_gao__port_506255,yeti_1027_gao__port_506358,mustang_1027_gao__port_506053,stilo_1027_gao__port_506060,ateca_1027_gao__port_506382,fiorino_1027_gao__port_506217,classe_glk_1027_gao__port_506290,fortwo_1027_gao__port_506230,cruze_1027_gao__port_506186,107_1027_gao__port_506213,aygo_1027_gao__port_506248,rx_1027_gao__port_506354,500_1027_gao__port_506245,bora_1027_gao__port_506104,transit_1027_gao__port_506111,pt_cruiser_1027_gao__port_506054,patrol_1027_gao__port_506068,r8_1027_gao__port_506156,xm_1027_gao__port_506102,s60_1027_gao__port_506191,aveo_1027_gao__port_506158,captiva_1027_gao__port_506159,ax_1027_gao__port_506153,rexton_1027_gao__port_506107,camaro_1027_gao__port_506056,ypsilon_1027_gao__port_506131,delta_1027_gao__port_506165,c4_1027_gao__port_506370,zx_1027_gao__port_506161,verso_1027_gao__port_506242,superb_1027_gao__port_506327,r5_1027_gao__port_506253,caddy_1027_gao__port_506330,x5_1027_gao__port_506243,f_type_1027_gao__port_506231,fusion_1027_gao__port_506096,dokker_1027_gao__port_506331,205_1027_gao__port_506062,macan_1027_gao__port_506195,tourneo_1027_gao__port_506369,108_1027_gao__port_506384,9_3_1027_gao__port_506071,mondeo_1027_gao__port_506116,cr_v_1027_gao__port_506164,c30_1027_gao__port_506090,pulsar_1027_gao__port_506397,ibiza_1027_gao__port_506273,a1_1027_gao__port_506338,matrix_1027_gao__port_506140,carnival_1027_gao__port_506136,xantia_1027_gao__port_506086,terrano_1027_gao__port_506083,q3_1027_gao__port_506275,hr_v_1027_gao__port_506283,expert_1027_gao__port_506142,multivan_1027_gao__port_506383,venga_1027_gao__port_506380,scudo_1027_gao__port_506129,laguna_1027_gao__port_506368,vel_satis_1027_gao__port_506130,b_max_1027_gao__port_506367,ignis_1027_gao__port_506292,159_1027_gao__port_506064,grande_punto_1027_gao__port_506138,logan_1027_gao__port_506167,s_max_1027_gao__port_506223,caravelle_1027_gao__port_506351,adam_1027_gao__port_506079,406_1027_gao__port_506132,q30_1027_gao__port_506293,almera_1027_gao__port_506089,corsa_1027_gao__port_506095,corolla_1027_gao__port_506120,xc60_1027_gao__port_506388,viano_1027_gao__port_506211,pro_cee_d_1027_gao__port_506274,a3_1027_gao__port_506321,v50_1027_gao__port_506150,voyager_1027_gao__port_506169,corvette_1027_gao__port_506049,rio_1027_gao__port_506379,jazz_1027_gao__port_506252,200_1027_gao__port_506112,tts_1027_gao__port_506199,zafira_1027_gao__port_506287,asx_1027_gao__port_506266,607_1027_gao__port_506118,207_1027_gao__port_506103,classe_s_1027_gao__port_506301,c6_1027_gao__port_506105,express_1027_gao__port_506137,classe_gla_1027_gao__port_506352,v60_1027_gao__port_506333,ka_1027_gao__port_506180,range_rover_1027_gao__port_506254,discovery_1027_gao__port_506375,classe_r_1027_gao__port_506270,transporter_1027_gao__port_506319,cee_d_1027_gao__port_506288,zoe_1027_gao__port_506244,i20_1027_gao__port_506284,gtv_1027_gao__port_506059,s4_avant_1027_gao__port_506261,x1_1027_gao__port_506372,autres_1027_gao__port_506127,208_1027_gao__port_506359,c8_1027_gao__port_506135,astra_1027_gao__port_506215,2_1027_gao__port_506151,doblo_1027_gao__port_506251,807_1027_gao__port_506152,206_1027_gao__port_506126,a7_1027_gao__port_506373,renegade_1027_gao__port_506346', 'svm_portfolios_learning': '506302,506374,506399,506192,506205,506350,506052,506295,506066,506117,506065,506125,506387,506381,506349,506328,506377,506286,506124,506172,506206,506178,506371,506076,506114,506329,506122,506220,506174,506224,506232,506234,506173,506181,506323,506326,506376,506048,506400,506179,506311,506325,506402,506051,506294,506318,506303,506175,506099,506061,506337,506250,506082,506166,506133,506308,506078,506340,506310,506100,506121,506070,506218,506227,506272,506147,506160,506265,506202,506222,506093,506257,506208,506344,506077,506395,506094,506219,506298,506339,506343,506365,506200,506348,506198,506385,506239,506236,506391,506087,506342,506149,506184,506393,506203,506280,506216,506403,506355,506332,506259,506401,506357,506324,506098,506315,506335,506088,506046,506185,506171,506080,506345,506347,506067,506233,506225,506312,506278,506300,506258,506182,506226,506262,506146,506113,506108,506297,506322,506143,506363,506073,506154,506313,506189,506197,506162,506249,506139,506237,506336,506084,506109,506106,506045,506392,506247,506316,506201,506353,506305,506050,506145,506362,506101,506128,506044,506317,506074,506134,506196,506194,506285,506177,506240,506282,506396,506281,506264,506276,506144,506069,506091,506081,506168,506291,506238,506072,506085,506235,506193,506268,506148,506356,506386,506229,506256,506187,506110,506304,506115,506214,506334,506289,506361,506366,506204,506190,506188,506307,506055,506389,506364,506279,506241,506057,506063,506320,506212,506263,506394,506306,506260,506309,506221,506155,506176,506398,506360,506210,506341,506209,506170,506097,506119,506163,506092,506267,506246,506047,506296,506058,506269,506378,506123,506271,506277,506207,506141,506390,506314,506299,506075,506183,506157,506228,506255,506358,506053,506060,506382,506217,506290,506230,506186,506213,506248,506354,506245,506104,506111,506054,506068,506156,506102,506191,506158,506159,506153,506107,506056,506131,506165,506370,506161,506242,506327,506253,506330,506243,506231,506096,506331,506062,506195,506369,506384,506071,506116,506164,506090,506397,506273,506338,506140,506136,506086,506083,506275,506283,506142,506383,506380,506129,506368,506130,506367,506292,506064,506138,506167,506223,506351,506079,506132,506293,506089,506095,506120,506388,506211,506274,506321,506150,506169,506049,506379,506252,506112,506199,506287,506266,506118,506103,506301,506105,506137,506352,506333,506180,506254,506375,506270,506319,506288,506244,506284,506059,506261,506372,506127,506359,506135,506215,506151,506251,506152,506126,506373,506346', 'photo_hashtag_type': 332, 'photo_desc_type': 3390, 'type_classification': 'caffe', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 355, 'mtr_user_id': 31, 'name': 'car_360_1027', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'c_elysee_1027_gao__port_506302,mokka_1027_gao__port_506374,captur_1027_gao__port_506399,sorento_1027_gao__port_506192,navara_1027_gao__port_506205,xc90_1027_gao__port_506350,saxo_1027_gao__port_506052,trafic_1027_gao__port_506295,punto_evo_1027_gao__port_506066,5_1027_gao__port_506117,250_1027_gao__port_506065,d_max_1027_gao__port_506125,panamera_1027_gao__port_506387,alhambra_1027_gao__port_506381,x6_1027_gao__port_506349,vitara_1027_gao__port_506328,fiesta_1027_gao__port_506377,qashqai_1027_gao__port_506286,147_1027_gao__port_506124,c5_1027_gao__port_506172,q5_1027_gao__port_506206,giulia_1027_gao__port_506178,karl_1027_gao__port_506371,mehari_1027_gao__port_506076,911_1027_gao__port_506114,508_1027_gao__port_506329,idea_1027_gao__port_506122,megane_1027_gao__port_506220,ghibli_1027_gao__port_506174,touareg_1027_gao__port_506224,i10_1027_gao__port_506232,jumper_1027_gao__port_506234,classe_clk_1027_gao__port_506173,kuga_1027_gao__port_506181,ct_1027_gao__port_506323,leon_1027_gao__port_506326,ds5_1027_gao__port_506376,cordoba_1027_gao__port_506048,classe_cla_1027_gao__port_506400,jumpy_1027_gao__port_506179,avensis_1027_gao__port_506311,juke_1027_gao__port_506325,4008_1027_gao__port_506402,190_series_1027_gao__port_506051,serie_3_1027_gao__port_506294,q7_1027_gao__port_506318,glc_1027_gao__port_506303,grand_vitara_1027_gao__port_506175,s40_1027_gao__port_506099,toledo_1027_gao__port_506061,5008_1027_gao__port_506337,continental_1027_gao__port_506250,coupe_1027_gao__port_506082,iq_1027_gao__port_506166,407_1027_gao__port_506133,touran_1027_gao__port_506308,300c_1027_gao__port_506078,classe_gl_1027_gao__port_506340,vivaro_1027_gao__port_506310,sl_1027_gao__port_506100,elise_1027_gao__port_506121,1007_1027_gao__port_506070,i40_1027_gao__port_506218,bipper_tepee_1027_gao__port_506227,focus_1027_gao__port_506272,primera_1027_gao__port_506147,r4_1027_gao__port_506160,a8_1027_gao__port_506265,boxer_1027_gao__port_506202,s5_1027_gao__port_506222,r21_1027_gao__port_506093,c3_1027_gao__port_506257,santa_fe_1027_gao__port_506208,m4_1027_gao__port_506344,safrane_1027_gao__port_506077,classe_gle_1027_gao__port_506395,0_1027_gao__port_506094,ix35_1027_gao__port_506219,carens_1027_gao__port_506298,classe_a_1027_gao__port_506339,ix20_1027_gao__port_506343,note_1027_gao__port_506365,a5_1027_gao__port_506200,sx4_1027_gao__port_506348,sandero_1027_gao__port_506198,3008_1027_gao__port_506385,q50_1027_gao__port_506239,latitude_1027_gao__port_506236,v40_1027_gao__port_506391,xsara_1027_gao__port_506087,grand_c_max_1027_gao__port_506342,swift_1027_gao__port_506149,serie_1_1027_gao__port_506184,xc70_1027_gao__port_506393,master_1027_gao__port_506203,clio_1027_gao__port_506280,duster_1027_gao__port_506216,traveller_1027_gao__port_506403,tipo_1027_gao__port_506355,rav_4_1027_gao__port_506332,coccinelle_1027_gao__port_506259,spacetourer_1027_gao__port_506401,xe_1027_gao__port_506357,ds3_1027_gao__port_506324,mx_5_1027_gao__port_506098,land_cruiser_1027_gao__port_506315,classe_b_1027_gao__port_506335,806_1027_gao__port_506088,rx_8_1027_gao__port_506046,spark_1027_gao__port_506185,6_1027_gao__port_506171,bravo_1027_gao__port_506080,nx_1027_gao__port_506345,sharan_1027_gao__port_506347,x_type_1027_gao__port_506067,jimny_1027_gao__port_506233,wrangler_1027_gao__port_506225,c_crosser_1027_gao__port_506312,v70_1027_gao__port_506278,classe_e_1027_gao__port_506300,classe_v_1027_gao__port_506258,m3_1027_gao__port_506182,abarth_500_1027_gao__port_506226,serie_6_1027_gao__port_506262,modus_1027_gao__port_506146,3_1027_gao__port_506113,405_1027_gao__port_506108,allroad_1027_gao__port_506297,auris_1027_gao__port_506322,galaxy_1027_gao__port_506143,giulietta_1027_gao__port_506363,106_1027_gao__port_506073,classe_m_1027_gao__port_506154,espace_1027_gao__port_506313,panda_1027_gao__port_506189,rcz_1027_gao__port_506197,4007_1027_gao__port_506162,classe_cl_1027_gao__port_506249,leaf_1027_gao__port_506139,octavia_1027_gao__port_506237,ds4_1027_gao__port_506336,freelander_1027_gao__port_506084,evasion_1027_gao__port_506109,punto_1027_gao__port_506106,2cv_1027_gao__port_506045,x4_1027_gao__port_506392,antara_1027_gao__port_506247,murano_1027_gao__port_506316,alto_1027_gao__port_506201,meriva_1027_gao__port_506353,orlando_1027_gao__port_506305,new_beetle_1027_gao__port_506050,306_1027_gao__port_506145,tiguan_1027_gao__port_506362,s_type_1027_gao__port_506101,c1_1027_gao__port_506128,vectra_1027_gao__port_506044,outlander_1027_gao__port_506317,307_1027_gao__port_506074,a6_s6_1027_gao__port_506134,nemo_combi_1027_gao__port_506196,berlingo_1027_gao__port_506194,partner_1027_gao__port_506285,cayenne_1027_gao__port_506177,quattroporte_1027_gao__port_506240,c_max_1027_gao__port_506282,fabia_1027_gao__port_506396,cx_3_1027_gao__port_506281,x_trail_1027_gao__port_506264,scirocco_1027_gao__port_506276,matiz_1027_gao__port_506144,tigra_1027_gao__port_506069,escort_1027_gao__port_506091,c2_1027_gao__port_506081,mini_1027_gao__port_506168,i30_1027_gao__port_506291,picanto_1027_gao__port_506238,mito_1027_gao__port_506072,impreza_1027_gao__port_506085,kangoo_1027_gao__port_506235,a4_1027_gao__port_506193,cayman_1027_gao__port_506268,sportage_1027_gao__port_506148,up_1027_gao__port_506356,optima_1027_gao__port_506386,defender_1027_gao__port_506229,serie_2_1027_gao__port_506256,edge_1027_gao__port_506187,r19_1027_gao__port_506110,jetta_1027_gao__port_506304,eos_1027_gao__port_506115,accord_1027_gao__port_506214,yaris_1027_gao__port_506334,classe_cls_1027_gao__port_506289,polo_1027_gao__port_506361,serie_4_1027_gao__port_506366,mini_cabriolet_1027_gao__port_506204,prius_1027_gao__port_506190,lodgy_1027_gao__port_506188,serie_7_1027_gao__port_506307,c15_1027_gao__port_506055,kadjar_1027_gao__port_506389,insignia_1027_gao__port_506364,308_1027_gao__port_506279,roomster_1027_gao__port_506241,80_1027_gao__port_506057,309_1027_gao__port_506063,tucson_1027_gao__port_506320,x3_1027_gao__port_506212,xf_1027_gao__port_506263,2008_1027_gao__port_506394,passat_1027_gao__port_506306,compass_1027_gao__port_506260,twingo_1027_gao__port_506309,micra_1027_gao__port_506221,golf_1027_gao__port_506155,soul_1027_gao__port_506176,rapid_1027_gao__port_506398,forester_1027_gao__port_506360,slk_1027_gao__port_506210,forfour_1027_gao__port_506341,serie_5_1027_gao__port_506209,xj_1027_gao__port_506170,pajero_1027_gao__port_506097,agila_1027_gao__port_506119,a6_1027_gao__port_506163,fox_1027_gao__port_506092,boxster_1027_gao__port_506267,altea_1027_gao__port_506246,samurai_1027_gao__port_506047,trax_1027_gao__port_506296,getz_1027_gao__port_506058,cherokee_1027_gao__port_506269,koleos_1027_gao__port_506378,z_series_1027_gao__port_506123,ecosport_1027_gao__port_506271,space_star_1027_gao__port_506277,rs3_sportback_1027_gao__port_506207,civic_1027_gao__port_506141,talisman_1027_gao__port_506390,f_pace_1027_gao__port_506314,classe_c_1027_gao__port_506299,tt_1027_gao__port_506075,pathfinder_1027_gao__port_506183,156_1027_gao__port_506157,cx_5_1027_gao__port_506228,scenic_1027_gao__port_506255,yeti_1027_gao__port_506358,mustang_1027_gao__port_506053,stilo_1027_gao__port_506060,ateca_1027_gao__port_506382,fiorino_1027_gao__port_506217,classe_glk_1027_gao__port_506290,fortwo_1027_gao__port_506230,cruze_1027_gao__port_506186,107_1027_gao__port_506213,aygo_1027_gao__port_506248,rx_1027_gao__port_506354,500_1027_gao__port_506245,bora_1027_gao__port_506104,transit_1027_gao__port_506111,pt_cruiser_1027_gao__port_506054,patrol_1027_gao__port_506068,r8_1027_gao__port_506156,xm_1027_gao__port_506102,s60_1027_gao__port_506191,aveo_1027_gao__port_506158,captiva_1027_gao__port_506159,ax_1027_gao__port_506153,rexton_1027_gao__port_506107,camaro_1027_gao__port_506056,ypsilon_1027_gao__port_506131,delta_1027_gao__port_506165,c4_1027_gao__port_506370,zx_1027_gao__port_506161,verso_1027_gao__port_506242,superb_1027_gao__port_506327,r5_1027_gao__port_506253,caddy_1027_gao__port_506330,x5_1027_gao__port_506243,f_type_1027_gao__port_506231,fusion_1027_gao__port_506096,dokker_1027_gao__port_506331,205_1027_gao__port_506062,macan_1027_gao__port_506195,tourneo_1027_gao__port_506369,108_1027_gao__port_506384,9_3_1027_gao__port_506071,mondeo_1027_gao__port_506116,cr_v_1027_gao__port_506164,c30_1027_gao__port_506090,pulsar_1027_gao__port_506397,ibiza_1027_gao__port_506273,a1_1027_gao__port_506338,matrix_1027_gao__port_506140,carnival_1027_gao__port_506136,xantia_1027_gao__port_506086,terrano_1027_gao__port_506083,q3_1027_gao__port_506275,hr_v_1027_gao__port_506283,expert_1027_gao__port_506142,multivan_1027_gao__port_506383,venga_1027_gao__port_506380,scudo_1027_gao__port_506129,laguna_1027_gao__port_506368,vel_satis_1027_gao__port_506130,b_max_1027_gao__port_506367,ignis_1027_gao__port_506292,159_1027_gao__port_506064,grande_punto_1027_gao__port_506138,logan_1027_gao__port_506167,s_max_1027_gao__port_506223,caravelle_1027_gao__port_506351,adam_1027_gao__port_506079,406_1027_gao__port_506132,q30_1027_gao__port_506293,almera_1027_gao__port_506089,corsa_1027_gao__port_506095,corolla_1027_gao__port_506120,xc60_1027_gao__port_506388,viano_1027_gao__port_506211,pro_cee_d_1027_gao__port_506274,a3_1027_gao__port_506321,v50_1027_gao__port_506150,voyager_1027_gao__port_506169,corvette_1027_gao__port_506049,rio_1027_gao__port_506379,jazz_1027_gao__port_506252,200_1027_gao__port_506112,tts_1027_gao__port_506199,zafira_1027_gao__port_506287,asx_1027_gao__port_506266,607_1027_gao__port_506118,207_1027_gao__port_506103,classe_s_1027_gao__port_506301,c6_1027_gao__port_506105,express_1027_gao__port_506137,classe_gla_1027_gao__port_506352,v60_1027_gao__port_506333,ka_1027_gao__port_506180,range_rover_1027_gao__port_506254,discovery_1027_gao__port_506375,classe_r_1027_gao__port_506270,transporter_1027_gao__port_506319,cee_d_1027_gao__port_506288,zoe_1027_gao__port_506244,i20_1027_gao__port_506284,gtv_1027_gao__port_506059,s4_avant_1027_gao__port_506261,x1_1027_gao__port_506372,autres_1027_gao__port_506127,208_1027_gao__port_506359,c8_1027_gao__port_506135,astra_1027_gao__port_506215,2_1027_gao__port_506151,doblo_1027_gao__port_506251,807_1027_gao__port_506152,206_1027_gao__port_506126,a7_1027_gao__port_506373,renegade_1027_gao__port_506346', 'svm_portfolios_learning': '506302,506374,506399,506192,506205,506350,506052,506295,506066,506117,506065,506125,506387,506381,506349,506328,506377,506286,506124,506172,506206,506178,506371,506076,506114,506329,506122,506220,506174,506224,506232,506234,506173,506181,506323,506326,506376,506048,506400,506179,506311,506325,506402,506051,506294,506318,506303,506175,506099,506061,506337,506250,506082,506166,506133,506308,506078,506340,506310,506100,506121,506070,506218,506227,506272,506147,506160,506265,506202,506222,506093,506257,506208,506344,506077,506395,506094,506219,506298,506339,506343,506365,506200,506348,506198,506385,506239,506236,506391,506087,506342,506149,506184,506393,506203,506280,506216,506403,506355,506332,506259,506401,506357,506324,506098,506315,506335,506088,506046,506185,506171,506080,506345,506347,506067,506233,506225,506312,506278,506300,506258,506182,506226,506262,506146,506113,506108,506297,506322,506143,506363,506073,506154,506313,506189,506197,506162,506249,506139,506237,506336,506084,506109,506106,506045,506392,506247,506316,506201,506353,506305,506050,506145,506362,506101,506128,506044,506317,506074,506134,506196,506194,506285,506177,506240,506282,506396,506281,506264,506276,506144,506069,506091,506081,506168,506291,506238,506072,506085,506235,506193,506268,506148,506356,506386,506229,506256,506187,506110,506304,506115,506214,506334,506289,506361,506366,506204,506190,506188,506307,506055,506389,506364,506279,506241,506057,506063,506320,506212,506263,506394,506306,506260,506309,506221,506155,506176,506398,506360,506210,506341,506209,506170,506097,506119,506163,506092,506267,506246,506047,506296,506058,506269,506378,506123,506271,506277,506207,506141,506390,506314,506299,506075,506183,506157,506228,506255,506358,506053,506060,506382,506217,506290,506230,506186,506213,506248,506354,506245,506104,506111,506054,506068,506156,506102,506191,506158,506159,506153,506107,506056,506131,506165,506370,506161,506242,506327,506253,506330,506243,506231,506096,506331,506062,506195,506369,506384,506071,506116,506164,506090,506397,506273,506338,506140,506136,506086,506083,506275,506283,506142,506383,506380,506129,506368,506130,506367,506292,506064,506138,506167,506223,506351,506079,506132,506293,506089,506095,506120,506388,506211,506274,506321,506150,506169,506049,506379,506252,506112,506199,506287,506266,506118,506103,506301,506105,506137,506352,506333,506180,506254,506375,506270,506319,506288,506244,506284,506059,506261,506372,506127,506359,506135,506215,506151,506251,506152,506126,506373,506346', 'photo_hashtag_type': 332, 'photo_desc_type': 3390, 'type_classification': 'caffe', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 3390 SELECT * FROM MTRDatou.photo_desc_type_params WHERE id in (3390) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (3390, 'car_360_1027', 16384, 25088, 'car_360_1027', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2017, 10, 28, 12, 29, 27), datetime.datetime(2017, 10, 28, 12, 29, 27)) To loadFromThcl() : net_3390 begin to check gpu status inside check gpu memory havn't enough memory gpu , need / 2500 l 3632 free memory gpu now : 1637 wait 20 seconds l 3637 free memory gpu now : 1637 max_wait_temp : 1 max_wait : 0 SELECT * FROM MTRDatou.photo_desc_type_params WHERE id in (3390) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (3390, 'car_360_1027', 16384, 25088, 'car_360_1027', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2017, 10, 28, 12, 29, 27), datetime.datetime(2017, 10, 28, 12, 29, 27)) param : , param.caffemodel : car_360_1027 None mean_file_type : mean_file_path : prototxt_file_path : model : car_360_1027 Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : car_360_1027 model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : [] local folder : /data/models_weight/car_360_1027 /data/models_weight/car_360_1027/caffemodel size_local : 542944640 size in s3 : 542944640 create time local : 2021-08-09 05:28:34 create time in s3 : 2021-08-06 17:57:43 caffemodel already exist and didn't need to update /data/models_weight/car_360_1027/deploy_conv_normal.prototxt size_local : 4626 size in s3 : 4626 create time local : 2021-08-09 05:28:34 create time in s3 : 2021-08-06 17:57:42 deploy_conv_normal.prototxt already exist and didn't need to update /data/models_weight/car_360_1027/deploy_fc.prototxt size_local : 1132 size in s3 : 1132 create time local : 2021-08-09 05:28:34 create time in s3 : 2021-08-06 17:57:43 deploy_fc.prototxt already exist and didn't need to update /data/models_weight/car_360_1027/deploy.prototxt size_local : 5654 size in s3 : 5654 create time local : 2021-08-09 05:28:34 create time in s3 : 2021-08-06 17:57:42 deploy.prototxt already exist and didn't need to update /data/models_weight/car_360_1027/mean.npy size_local : 1572944 size in s3 : 1572944 create time local : 2021-08-09 05:28:34 create time in s3 : 2021-08-06 17:57:55 mean.npy already exist and didn't need to update /data/models_weight/car_360_1027/synset_words.txt size_local : 13687 size in s3 : 13687 create time local : 2021-08-09 05:28:34 create time in s3 : 2021-08-06 17:57:43 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/car_360_1027/deploy.prototxt caffemodel_filename : /data/models_weight/car_360_1027/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory havn't enough memory gpu , need / 2500 l 3632 free memory gpu now : 1637 wait 20 seconds l 3637 free memory gpu now : 1637 max_wait_temp : 1 max_wait : 0 dict_keys(['prob', 'pool5']) time used to do the prepocess of the images : 0.015654563903808594 time used to do the prediction : 0.16445326805114746 save descriptor for thcl : 355 (1, 512, 7, 7) Got the blobs of the net to insert : [0, 0, 0, 0, 0, 0, 0, 0, 0, 0] code_as_byte_string:b'0000000000'| time to traite the descriptors : 0.05224132537841797 Testing : ['916235064'] In select_photos_meta_from_ids: SELECT photo_id, url, FROM_UNIXTIME(uploaded_at), latitude, longitude, text FROM MTRBack.photos WHERE photo_id IN (916235064) Catched exception ! Connect or reconnect ! result : {916235064: {'photo_id': 916235064, 'url': 'https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/2017/10/14/6293d1bb790dc6902450e7c572b7d10b.jpg', 'latitude': 0.0, 'longitude': 0.0, 'text': None}} list_photo_exists : [916235064] storage_type for insertDescriptorsMulti : 1 To insert : 916235064 time to insert the descriptors : 0.8107748031616211 After datou_step_exec type output : map_portfolio_photo : len 0 keys : dict_keys([]) Inside saveOutput : final : False verbose : True time used to find the portfolios of the photos select button_legend_list from MTRDatou.classification_theme where id = 355 SAVE THCL, output : {'916235064': [[('916235064', 'c_elysee_1027_gao__port_506302', 0.001881488, 332, '355'), ('916235064', 'mokka_1027_gao__port_506374', 0.0011635546, 332, '355'), ('916235064', 'captur_1027_gao__port_506399', 0.0008156919, 332, '355'), ('916235064', 'sorento_1027_gao__port_506192', 0.0011772468, 332, '355'), ('916235064', 'navara_1027_gao__port_506205', 0.0025851124, 332, '355'), ('916235064', 'xc90_1027_gao__port_506350', 0.0041701584, 332, '355'), ('916235064', 'saxo_1027_gao__port_506052', 0.0034807639, 332, '355'), ('916235064', 'trafic_1027_gao__port_506295', 0.007367062, 332, '355'), ('916235064', 'punto_evo_1027_gao__port_506066', 0.002188681, 332, '355'), ('916235064', '5_1027_gao__port_506117', 0.00057978893, 332, '355'), ('916235064', '250_1027_gao__port_506065', 0.0045914417, 332, '355'), ('916235064', 'd_max_1027_gao__port_506125', 0.0031586885, 332, '355'), ('916235064', 'panamera_1027_gao__port_506387', 0.0022507631, 332, '355'), ('916235064', 'alhambra_1027_gao__port_506381', 0.0053202934, 332, '355'), ('916235064', 'x6_1027_gao__port_506349', 0.0010999305, 332, '355'), ('916235064', 'vitara_1027_gao__port_506328', 0.0054027988, 332, '355'), ('916235064', 'fiesta_1027_gao__port_506377', 0.003918938, 332, '355'), ('916235064', 'qashqai_1027_gao__port_506286', 0.0014786645, 332, '355'), ('916235064', '147_1027_gao__port_506124', 0.001977866, 332, '355'), ('916235064', 'c5_1027_gao__port_506172', 0.0012441645, 332, '355'), ('916235064', 'q5_1027_gao__port_506206', 0.0015049673, 332, '355'), ('916235064', 'giulia_1027_gao__port_506178', 0.0021693234, 332, '355'), ('916235064', 'karl_1027_gao__port_506371', 0.0027080441, 332, '355'), ('916235064', 'mehari_1027_gao__port_506076', 0.0047035282, 332, '355'), ('916235064', '911_1027_gao__port_506114', 0.0019417771, 332, '355'), ('916235064', '508_1027_gao__port_506329', 0.0009585605, 332, '355'), ('916235064', 'idea_1027_gao__port_506122', 0.0007700267, 332, '355'), ('916235064', 'megane_1027_gao__port_506220', 0.0019467189, 332, '355'), ('916235064', 'ghibli_1027_gao__port_506174', 0.0013724914, 332, '355'), ('916235064', 'touareg_1027_gao__port_506224', 0.0016201943, 332, '355'), ('916235064', 'i10_1027_gao__port_506232', 0.0013924459, 332, '355'), ('916235064', 'jumper_1027_gao__port_506234', 0.010045431, 332, '355'), ('916235064', 'classe_clk_1027_gao__port_506173', 0.0010793079, 332, '355'), ('916235064', 'kuga_1027_gao__port_506181', 0.00084472646, 332, '355'), ('916235064', 'ct_1027_gao__port_506323', 0.0012521575, 332, '355'), ('916235064', 'leon_1027_gao__port_506326', 0.0025844, 332, '355'), ('916235064', 'ds5_1027_gao__port_506376', 0.0012429417, 332, '355'), ('916235064', 'cordoba_1027_gao__port_506048', 0.0028649825, 332, '355'), ('916235064', 'classe_cla_1027_gao__port_506400', 0.0012949253, 332, '355'), ('916235064', 'jumpy_1027_gao__port_506179', 0.01033922, 332, '355'), ('916235064', 'avensis_1027_gao__port_506311', 0.0018767515, 332, '355'), ('916235064', 'juke_1027_gao__port_506325', 0.0011343167, 332, '355'), ('916235064', '4008_1027_gao__port_506402', 0.0015757406, 332, '355'), ('916235064', '190_series_1027_gao__port_506051', 0.0039805383, 332, '355'), ('916235064', 'serie_3_1027_gao__port_506294', 0.0028741586, 332, '355'), ('916235064', 'q7_1027_gao__port_506318', 0.0023356173, 332, '355'), ('916235064', 'glc_1027_gao__port_506303', 0.0012106822, 332, '355'), ('916235064', 'grand_vitara_1027_gao__port_506175', 0.0011447587, 332, '355'), ('916235064', 's40_1027_gao__port_506099', 0.00223385, 332, '355'), ('916235064', 'toledo_1027_gao__port_506061', 0.0017465311, 332, '355'), ('916235064', '5008_1027_gao__port_506337', 0.004699091, 332, '355'), ('916235064', 'continental_1027_gao__port_506250', 0.0021912393, 332, '355'), ('916235064', 'coupe_1027_gao__port_506082', 0.0022631562, 332, '355'), ('916235064', 'iq_1027_gao__port_506166', 0.0018174681, 332, '355'), ('916235064', '407_1027_gao__port_506133', 0.0009056061, 332, '355'), ('916235064', 'touran_1027_gao__port_506308', 0.0020400889, 332, '355'), ('916235064', '300c_1027_gao__port_506078', 0.002533472, 332, '355'), ('916235064', 'classe_gl_1027_gao__port_506340', 0.0044892007, 332, '355'), ('916235064', 'vivaro_1027_gao__port_506310', 0.0034251995, 332, '355'), ('916235064', 'sl_1027_gao__port_506100', 0.0031355706, 332, '355'), ('916235064', 'elise_1027_gao__port_506121', 0.001025579, 332, '355'), ('916235064', '1007_1027_gao__port_506070', 0.0015354921, 332, '355'), ('916235064', 'i40_1027_gao__port_506218', 0.00059148675, 332, '355'), ('916235064', 'bipper_tepee_1027_gao__port_506227', 0.004029605, 332, '355'), ('916235064', 'focus_1027_gao__port_506272', 0.0011585662, 332, '355'), ('916235064', 'primera_1027_gao__port_506147', 0.0012158051, 332, '355'), ('916235064', 'r4_1027_gao__port_506160', 0.014966717, 332, '355'), ('916235064', 'a8_1027_gao__port_506265', 0.0011320816, 332, '355'), ('916235064', 'boxer_1027_gao__port_506202', 0.010546196, 332, '355'), ('916235064', 's5_1027_gao__port_506222', 0.001198511, 332, '355'), ('916235064', 'r21_1027_gao__port_506093', 0.0041855313, 332, '355'), ('916235064', 'c3_1027_gao__port_506257', 0.0023633684, 332, '355'), ('916235064', 'santa_fe_1027_gao__port_506208', 0.0016322288, 332, '355'), ('916235064', 'm4_1027_gao__port_506344', 0.0015568405, 332, '355'), ('916235064', 'safrane_1027_gao__port_506077', 0.0013958503, 332, '355'), ('916235064', 'classe_gle_1027_gao__port_506395', 0.0021979548, 332, '355'), ('916235064', '0_1027_gao__port_506094', 0.008827779, 332, '355'), ('916235064', 'ix35_1027_gao__port_506219', 0.0014614465, 332, '355'), ('916235064', 'carens_1027_gao__port_506298', 0.0008824748, 332, '355'), ('916235064', 'classe_a_1027_gao__port_506339', 0.0024713709, 332, '355'), ('916235064', 'ix20_1027_gao__port_506343', 0.0010093511, 332, '355'), ('916235064', 'note_1027_gao__port_506365', 0.0015962347, 332, '355'), ('916235064', 'a5_1027_gao__port_506200', 0.0015330826, 332, '355'), ('916235064', 'sx4_1027_gao__port_506348', 0.0014916564, 332, '355'), ('916235064', 'sandero_1027_gao__port_506198', 0.0014585463, 332, '355'), ('916235064', '3008_1027_gao__port_506385', 0.0056457324, 332, '355'), ('916235064', 'q50_1027_gao__port_506239', 0.0011165234, 332, '355'), ('916235064', 'latitude_1027_gao__port_506236', 0.00080191577, 332, '355'), ('916235064', 'v40_1027_gao__port_506391', 0.0017145084, 332, '355'), ('916235064', 'xsara_1027_gao__port_506087', 0.0009823112, 332, '355'), ('916235064', 'grand_c_max_1027_gao__port_506342', 0.0017958524, 332, '355'), ('916235064', 'swift_1027_gao__port_506149', 0.0015020471, 332, '355'), ('916235064', 'serie_1_1027_gao__port_506184', 0.0015140205, 332, '355'), ('916235064', 'xc70_1027_gao__port_506393', 0.003619153, 332, '355'), ('916235064', 'master_1027_gao__port_506203', 0.007958301, 332, '355'), ('916235064', 'clio_1027_gao__port_506280', 0.0029573, 332, '355'), ('916235064', 'duster_1027_gao__port_506216', 0.00074439595, 332, '355'), ('916235064', 'traveller_1027_gao__port_506403', 0.0042937207, 332, '355'), ('916235064', 'tipo_1027_gao__port_506355', 0.0010929329, 332, '355'), ('916235064', 'rav_4_1027_gao__port_506332', 0.0013603923, 332, '355'), ('916235064', 'coccinelle_1027_gao__port_506259', 0.003494449, 332, '355'), ('916235064', 'spacetourer_1027_gao__port_506401', 0.0030971812, 332, '355'), ('916235064', 'xe_1027_gao__port_506357', 0.0014470823, 332, '355'), ('916235064', 'ds3_1027_gao__port_506324', 0.0013093649, 332, '355'), ('916235064', 'mx_5_1027_gao__port_506098', 0.002588586, 332, '355'), ('916235064', 'land_cruiser_1027_gao__port_506315', 0.009532302, 332, '355'), ('916235064', 'classe_b_1027_gao__port_506335', 0.0017214976, 332, '355'), ('916235064', '806_1027_gao__port_506088', 0.002561946, 332, '355'), ('916235064', 'rx_8_1027_gao__port_506046', 0.0036215407, 332, '355'), ('916235064', 'spark_1027_gao__port_506185', 0.0010076852, 332, '355'), ('916235064', '6_1027_gao__port_506171', 0.0011182154, 332, '355'), ('916235064', 'bravo_1027_gao__port_506080', 0.0014649665, 332, '355'), ('916235064', 'nx_1027_gao__port_506345', 0.0013683186, 332, '355'), ('916235064', 'sharan_1027_gao__port_506347', 0.005092317, 332, '355'), ('916235064', 'x_type_1027_gao__port_506067', 0.00078023056, 332, '355'), ('916235064', 'jimny_1027_gao__port_506233', 0.004606392, 332, '355'), ('916235064', 'wrangler_1027_gao__port_506225', 0.0017998576, 332, '355'), ('916235064', 'c_crosser_1027_gao__port_506312', 0.0015926793, 332, '355'), ('916235064', 'v70_1027_gao__port_506278', 0.0019675388, 332, '355'), ('916235064', 'classe_e_1027_gao__port_506300', 0.0017368982, 332, '355'), ('916235064', 'classe_v_1027_gao__port_506258', 0.012732168, 332, '355'), ('916235064', 'm3_1027_gao__port_506182', 0.002337174, 332, '355'), ('916235064', 'abarth_500_1027_gao__port_506226', 0.004043396, 332, '355'), ('916235064', 'serie_6_1027_gao__port_506262', 0.0011314655, 332, '355'), ('916235064', 'modus_1027_gao__port_506146', 0.0018293033, 332, '355'), ('916235064', '3_1027_gao__port_506113', 0.0015082445, 332, '355'), ('916235064', '405_1027_gao__port_506108', 0.008054989, 332, '355'), ('916235064', 'allroad_1027_gao__port_506297', 0.0010597616, 332, '355'), ('916235064', 'auris_1027_gao__port_506322', 0.0011525457, 332, '355'), ('916235064', 'galaxy_1027_gao__port_506143', 0.0025148138, 332, '355'), ('916235064', 'giulietta_1027_gao__port_506363', 0.00086506235, 332, '355'), ('916235064', '106_1027_gao__port_506073', 0.008270344, 332, '355'), ('916235064', 'classe_m_1027_gao__port_506154', 0.0030019982, 332, '355'), ('916235064', 'espace_1027_gao__port_506313', 0.001064563, 332, '355'), ('916235064', 'panda_1027_gao__port_506189', 0.009030028, 332, '355'), ('916235064', 'rcz_1027_gao__port_506197', 0.0011293247, 332, '355'), ('916235064', '4007_1027_gao__port_506162', 0.00067927694, 332, '355'), ('916235064', 'classe_cl_1027_gao__port_506249', 0.0010860265, 332, '355'), ('916235064', 'leaf_1027_gao__port_506139', 0.0018036704, 332, '355'), ('916235064', 'octavia_1027_gao__port_506237', 0.0018602483, 332, '355'), ('916235064', 'ds4_1027_gao__port_506336', 0.002415702, 332, '355'), ('916235064', 'freelander_1027_gao__port_506084', 0.0023474407, 332, '355'), ('916235064', 'evasion_1027_gao__port_506109', 0.0031141757, 332, '355'), ('916235064', 'punto_1027_gao__port_506106', 0.0019494628, 332, '355'), ('916235064', '2cv_1027_gao__port_506045', 0.007973876, 332, '355'), ('916235064', 'x4_1027_gao__port_506392', 0.0017948952, 332, '355'), ('916235064', 'antara_1027_gao__port_506247', 0.001246858, 332, '355'), ('916235064', 'murano_1027_gao__port_506316', 0.00060890167, 332, '355'), ('916235064', 'alto_1027_gao__port_506201', 0.009231071, 332, '355'), ('916235064', 'meriva_1027_gao__port_506353', 0.0013763967, 332, '355'), ('916235064', 'orlando_1027_gao__port_506305', 0.0018460603, 332, '355'), ('916235064', 'new_beetle_1027_gao__port_506050', 0.0011636717, 332, '355'), ('916235064', '306_1027_gao__port_506145', 0.0035070025, 332, '355'), ('916235064', 'tiguan_1027_gao__port_506362', 0.002682482, 332, '355'), ('916235064', 's_type_1027_gao__port_506101', 0.0011380889, 332, '355'), ('916235064', 'c1_1027_gao__port_506128', 0.0027513842, 332, '355'), ('916235064', 'vectra_1027_gao__port_506044', 0.0011989388, 332, '355'), ('916235064', 'outlander_1027_gao__port_506317', 0.0017122021, 332, '355'), ('916235064', '307_1027_gao__port_506074', 0.0020011372, 332, '355'), ('916235064', 'a6_s6_1027_gao__port_506134', 0.0016569874, 332, '355'), ('916235064', 'nemo_combi_1027_gao__port_506196', 0.0022663234, 332, '355'), ('916235064', 'berlingo_1027_gao__port_506194', 0.0046629403, 332, '355'), ('916235064', 'partner_1027_gao__port_506285', 0.0039412454, 332, '355'), ('916235064', 'cayenne_1027_gao__port_506177', 0.0037978268, 332, '355'), ('916235064', 'quattroporte_1027_gao__port_506240', 0.0024439634, 332, '355'), ('916235064', 'c_max_1027_gao__port_506282', 0.0013124, 332, '355'), ('916235064', 'fabia_1027_gao__port_506396', 0.005299431, 332, '355'), ('916235064', 'cx_3_1027_gao__port_506281', 0.0014462644, 332, '355'), ('916235064', 'x_trail_1027_gao__port_506264', 0.0018315479, 332, '355'), ('916235064', 'scirocco_1027_gao__port_506276', 0.0047903727, 332, '355'), ('916235064', 'matiz_1027_gao__port_506144', 0.0017558546, 332, '355'), ('916235064', 'tigra_1027_gao__port_506069', 0.00085419125, 332, '355'), ('916235064', 'escort_1027_gao__port_506091', 0.004840265, 332, '355'), ('916235064', 'c2_1027_gao__port_506081', 0.0014904203, 332, '355'), ('916235064', 'mini_1027_gao__port_506168', 0.001192115, 332, '355'), ('916235064', 'i30_1027_gao__port_506291', 0.00063251634, 332, '355'), ('916235064', 'picanto_1027_gao__port_506238', 0.0029746988, 332, '355'), ('916235064', 'mito_1027_gao__port_506072', 0.001507529, 332, '355'), ('916235064', 'impreza_1027_gao__port_506085', 0.0020192014, 332, '355'), ('916235064', 'kangoo_1027_gao__port_506235', 0.006583688, 332, '355'), ('916235064', 'a4_1027_gao__port_506193', 0.0019874866, 332, '355'), ('916235064', 'cayman_1027_gao__port_506268', 0.0018140925, 332, '355'), ('916235064', 'sportage_1027_gao__port_506148', 0.0014275507, 332, '355'), ('916235064', 'up_1027_gao__port_506356', 0.006863067, 332, '355'), ('916235064', 'optima_1027_gao__port_506386', 0.00089179486, 332, '355'), ('916235064', 'defender_1027_gao__port_506229', 0.006721728, 332, '355'), ('916235064', 'serie_2_1027_gao__port_506256', 0.0022275012, 332, '355'), ('916235064', 'edge_1027_gao__port_506187', 0.0008747733, 332, '355'), ('916235064', 'r19_1027_gao__port_506110', 0.0049424553, 332, '355'), ('916235064', 'jetta_1027_gao__port_506304', 0.003619459, 332, '355'), ('916235064', 'eos_1027_gao__port_506115', 0.0038921533, 332, '355'), ('916235064', 'accord_1027_gao__port_506214', 0.002012729, 332, '355'), ('916235064', 'yaris_1027_gao__port_506334', 0.0032333, 332, '355'), ('916235064', 'classe_cls_1027_gao__port_506289', 0.0007851733, 332, '355'), ('916235064', 'polo_1027_gao__port_506361', 0.004310651, 332, '355'), ('916235064', 'serie_4_1027_gao__port_506366', 0.001147668, 332, '355'), ('916235064', 'mini_cabriolet_1027_gao__port_506204', 0.00083775906, 332, '355'), ('916235064', 'prius_1027_gao__port_506190', 0.0011489986, 332, '355'), ('916235064', 'lodgy_1027_gao__port_506188', 0.0020172296, 332, '355'), ('916235064', 'serie_7_1027_gao__port_506307', 0.0012477048, 332, '355'), ('916235064', 'c15_1027_gao__port_506055', 0.017713884, 332, '355'), ('916235064', 'kadjar_1027_gao__port_506389', 0.0012504495, 332, '355'), ('916235064', 'insignia_1027_gao__port_506364', 0.0016433554, 332, '355'), ('916235064', '308_1027_gao__port_506279', 0.002123678, 332, '355'), ('916235064', 'roomster_1027_gao__port_506241', 0.0018012763, 332, '355'), ('916235064', '80_1027_gao__port_506057', 0.0046133194, 332, '355'), ('916235064', '309_1027_gao__port_506063', 0.013525401, 332, '355'), ('916235064', 'tucson_1027_gao__port_506320', 0.0021235931, 332, '355'), ('916235064', 'x3_1027_gao__port_506212', 0.00089770573, 332, '355'), ('916235064', 'xf_1027_gao__port_506263', 0.0011165391, 332, '355'), ('916235064', '2008_1027_gao__port_506394', 0.0026408099, 332, '355'), ('916235064', 'passat_1027_gao__port_506306', 0.0014975085, 332, '355'), ('916235064', 'compass_1027_gao__port_506260', 0.003256092, 332, '355'), ('916235064', 'twingo_1027_gao__port_506309', 0.0064986134, 332, '355'), ('916235064', 'micra_1027_gao__port_506221', 0.003585671, 332, '355'), ('916235064', 'golf_1027_gao__port_506155', 0.0031929684, 332, '355'), ('916235064', 'soul_1027_gao__port_506176', 0.0012889478, 332, '355'), ('916235064', 'rapid_1027_gao__port_506398', 0.0025915354, 332, '355'), ('916235064', 'forester_1027_gao__port_506360', 0.0022763622, 332, '355'), ('916235064', 'slk_1027_gao__port_506210', 0.001584473, 332, '355'), ('916235064', 'forfour_1027_gao__port_506341', 0.0021795337, 332, '355'), ('916235064', 'serie_5_1027_gao__port_506209', 0.0013695366, 332, '355'), ('916235064', 'xj_1027_gao__port_506170', 0.002600794, 332, '355'), ('916235064', 'pajero_1027_gao__port_506097', 0.005211568, 332, '355'), ('916235064', 'agila_1027_gao__port_506119', 0.0048493613, 332, '355'), ('916235064', 'a6_1027_gao__port_506163', 0.0018911085, 332, '355'), ('916235064', 'fox_1027_gao__port_506092', 0.00084649236, 332, '355'), ('916235064', 'boxster_1027_gao__port_506267', 0.0015941493, 332, '355'), ('916235064', 'altea_1027_gao__port_506246', 0.002139043, 332, '355'), ('916235064', 'samurai_1027_gao__port_506047', 0.006251702, 332, '355'), ('916235064', 'trax_1027_gao__port_506296', 0.0019438587, 332, '355'), ('916235064', 'getz_1027_gao__port_506058', 0.0016384881, 332, '355'), ('916235064', 'cherokee_1027_gao__port_506269', 0.0029811051, 332, '355'), ('916235064', 'koleos_1027_gao__port_506378', 0.0015590829, 332, '355'), ('916235064', 'z_series_1027_gao__port_506123', 0.0016564039, 332, '355'), ('916235064', 'ecosport_1027_gao__port_506271', 0.0013230306, 332, '355'), ('916235064', 'space_star_1027_gao__port_506277', 0.0021142373, 332, '355'), ('916235064', 'rs3_sportback_1027_gao__port_506207', 0.0019116476, 332, '355'), ('916235064', 'civic_1027_gao__port_506141', 0.0026898917, 332, '355'), ('916235064', 'talisman_1027_gao__port_506390', 0.0007612977, 332, '355'), ('916235064', 'f_pace_1027_gao__port_506314', 0.0016165122, 332, '355'), ('916235064', 'classe_c_1027_gao__port_506299', 0.0017941886, 332, '355'), ('916235064', 'tt_1027_gao__port_506075', 0.0013934884, 332, '355'), ('916235064', 'pathfinder_1027_gao__port_506183', 0.0016515092, 332, '355'), ('916235064', '156_1027_gao__port_506157', 0.0015443906, 332, '355'), ('916235064', 'cx_5_1027_gao__port_506228', 0.0014414431, 332, '355'), ('916235064', 'scenic_1027_gao__port_506255', 0.0016085034, 332, '355'), ('916235064', 'yeti_1027_gao__port_506358', 0.0020914818, 332, '355'), ('916235064', 'mustang_1027_gao__port_506053', 0.010050118, 332, '355'), ('916235064', 'stilo_1027_gao__port_506060', 0.001083286, 332, '355'), ('916235064', 'ateca_1027_gao__port_506382', 0.0017010622, 332, '355'), ('916235064', 'fiorino_1027_gao__port_506217', 0.009198821, 332, '355'), ('916235064', 'classe_glk_1027_gao__port_506290', 0.0017017875, 332, '355'), ('916235064', 'fortwo_1027_gao__port_506230', 0.0016010429, 332, '355'), ('916235064', 'cruze_1027_gao__port_506186', 0.0010052221, 332, '355'), ('916235064', '107_1027_gao__port_506213', 0.0016274898, 332, '355'), ('916235064', 'aygo_1027_gao__port_506248', 0.003243105, 332, '355'), ('916235064', 'rx_1027_gao__port_506354', 0.0010632935, 332, '355'), ('916235064', '500_1027_gao__port_506245', 0.0016354092, 332, '355'), ('916235064', 'bora_1027_gao__port_506104', 0.0038164337, 332, '355'), ('916235064', 'transit_1027_gao__port_506111', 0.00486192, 332, '355'), ('916235064', 'pt_cruiser_1027_gao__port_506054', 0.0019164018, 332, '355'), ('916235064', 'patrol_1027_gao__port_506068', 0.004240243, 332, '355'), ('916235064', 'r8_1027_gao__port_506156', 0.0012723003, 332, '355'), ('916235064', 'xm_1027_gao__port_506102', 0.002267937, 332, '355'), ('916235064', 's60_1027_gao__port_506191', 0.0031990022, 332, '355'), ('916235064', 'aveo_1027_gao__port_506158', 0.0038397212, 332, '355'), ('916235064', 'captiva_1027_gao__port_506159', 0.0017187928, 332, '355'), ('916235064', 'ax_1027_gao__port_506153', 0.006896364, 332, '355'), ('916235064', 'rexton_1027_gao__port_506107', 0.0013020239, 332, '355'), ('916235064', 'camaro_1027_gao__port_506056', 0.0024901906, 332, '355'), ('916235064', 'ypsilon_1027_gao__port_506131', 0.0019540447, 332, '355'), ('916235064', 'delta_1027_gao__port_506165', 0.001400219, 332, '355'), ('916235064', 'c4_1027_gao__port_506370', 0.0013008723, 332, '355'), ('916235064', 'zx_1027_gao__port_506161', 0.0045938133, 332, '355'), ('916235064', 'verso_1027_gao__port_506242', 0.0007721059, 332, '355'), ('916235064', 'superb_1027_gao__port_506327', 0.001994159, 332, '355'), ('916235064', 'r5_1027_gao__port_506253', 0.009546494, 332, '355'), ('916235064', 'caddy_1027_gao__port_506330', 0.013826299, 332, '355'), ('916235064', 'x5_1027_gao__port_506243', 0.0011204281, 332, '355'), ('916235064', 'f_type_1027_gao__port_506231', 0.00082992815, 332, '355'), ('916235064', 'fusion_1027_gao__port_506096', 0.0012669299, 332, '355'), ('916235064', 'dokker_1027_gao__port_506331', 0.005357435, 332, '355'), ('916235064', '205_1027_gao__port_506062', 0.0066853757, 332, '355'), ('916235064', 'macan_1027_gao__port_506195', 0.0015573651, 332, '355'), ('916235064', 'tourneo_1027_gao__port_506369', 0.0064015957, 332, '355'), ('916235064', '108_1027_gao__port_506384', 0.0052640038, 332, '355'), ('916235064', '9_3_1027_gao__port_506071', 0.00083744066, 332, '355'), ('916235064', 'mondeo_1027_gao__port_506116', 0.0014395545, 332, '355'), ('916235064', 'cr_v_1027_gao__port_506164', 0.0016414316, 332, '355'), ('916235064', 'c30_1027_gao__port_506090', 0.0017483974, 332, '355'), ('916235064', 'pulsar_1027_gao__port_506397', 0.0012018875, 332, '355'), ('916235064', 'ibiza_1027_gao__port_506273', 0.003723022, 332, '355'), ('916235064', 'a1_1027_gao__port_506338', 0.0012346468, 332, '355'), ('916235064', 'matrix_1027_gao__port_506140', 0.00070773775, 332, '355'), ('916235064', 'carnival_1027_gao__port_506136', 0.0022812125, 332, '355'), ('916235064', 'xantia_1027_gao__port_506086', 0.0021960482, 332, '355'), ('916235064', 'terrano_1027_gao__port_506083', 0.0020294879, 332, '355'), ('916235064', 'q3_1027_gao__port_506275', 0.0011264886, 332, '355'), ('916235064', 'hr_v_1027_gao__port_506283', 0.001780512, 332, '355'), ('916235064', 'expert_1027_gao__port_506142', 0.0073696454, 332, '355'), ('916235064', 'multivan_1027_gao__port_506383', 0.0065042027, 332, '355'), ('916235064', 'venga_1027_gao__port_506380', 0.0008004925, 332, '355'), ('916235064', 'scudo_1027_gao__port_506129', 0.00559271, 332, '355'), ('916235064', 'laguna_1027_gao__port_506368', 0.0007134644, 332, '355'), ('916235064', 'vel_satis_1027_gao__port_506130', 0.0027263658, 332, '355'), ('916235064', 'b_max_1027_gao__port_506367', 0.001724623, 332, '355'), ('916235064', 'ignis_1027_gao__port_506292', 0.004356205, 332, '355'), ('916235064', '159_1027_gao__port_506064', 0.0010783394, 332, '355'), ('916235064', 'grande_punto_1027_gao__port_506138', 0.002363293, 332, '355'), ('916235064', 'logan_1027_gao__port_506167', 0.004397185, 332, '355'), ('916235064', 's_max_1027_gao__port_506223', 0.0012527297, 332, '355'), ('916235064', 'caravelle_1027_gao__port_506351', 0.0030293767, 332, '355'), ('916235064', 'adam_1027_gao__port_506079', 0.0010537043, 332, '355'), ('916235064', '406_1027_gao__port_506132', 0.0013574009, 332, '355'), ('916235064', 'q30_1027_gao__port_506293', 0.00097155076, 332, '355'), ('916235064', 'almera_1027_gao__port_506089', 0.001023923, 332, '355'), ('916235064', 'corsa_1027_gao__port_506095', 0.0025202187, 332, '355'), ('916235064', 'corolla_1027_gao__port_506120', 0.0026821934, 332, '355'), ('916235064', 'xc60_1027_gao__port_506388', 0.0018984337, 332, '355'), ('916235064', 'viano_1027_gao__port_506211', 0.002694355, 332, '355'), ('916235064', 'pro_cee_d_1027_gao__port_506274', 0.0008319267, 332, '355'), ('916235064', 'a3_1027_gao__port_506321', 0.0037380003, 332, '355'), ('916235064', 'v50_1027_gao__port_506150', 0.0007919421, 332, '355'), ('916235064', 'voyager_1027_gao__port_506169', 0.0030524868, 332, '355'), ('916235064', 'corvette_1027_gao__port_506049', 0.0037227466, 332, '355'), ('916235064', 'rio_1027_gao__port_506379', 0.0017739313, 332, '355'), ('916235064', 'jazz_1027_gao__port_506252', 0.0015306006, 332, '355'), ('916235064', '200_1027_gao__port_506112', 0.0040873, 332, '355'), ('916235064', 'tts_1027_gao__port_506199', 0.001186331, 332, '355'), ('916235064', 'zafira_1027_gao__port_506287', 0.0026953255, 332, '355'), ('916235064', 'asx_1027_gao__port_506266', 0.0011406954, 332, '355'), ('916235064', '607_1027_gao__port_506118', 0.0012528016, 332, '355'), ('916235064', '207_1027_gao__port_506103', 0.0015148654, 332, '355'), ('916235064', 'classe_s_1027_gao__port_506301', 0.0031655438, 332, '355'), ('916235064', 'c6_1027_gao__port_506105', 0.0017347562, 332, '355'), ('916235064', 'express_1027_gao__port_506137', 0.016727803, 332, '355'), ('916235064', 'classe_gla_1027_gao__port_506352', 0.0018254973, 332, '355'), ('916235064', 'v60_1027_gao__port_506333', 0.0021458182, 332, '355'), ('916235064', 'ka_1027_gao__port_506180', 0.0014151047, 332, '355'), ('916235064', 'range_rover_1027_gao__port_506254', 0.0020553232, 332, '355'), ('916235064', 'discovery_1027_gao__port_506375', 0.0022965097, 332, '355'), ('916235064', 'classe_r_1027_gao__port_506270', 0.0013944509, 332, '355'), ('916235064', 'transporter_1027_gao__port_506319', 0.011968875, 332, '355'), ('916235064', 'cee_d_1027_gao__port_506288', 0.0010547898, 332, '355'), ('916235064', 'zoe_1027_gao__port_506244', 0.002071429, 332, '355'), ('916235064', 'i20_1027_gao__port_506284', 0.0017869647, 332, '355'), ('916235064', 'gtv_1027_gao__port_506059', 0.0057222107, 332, '355'), ('916235064', 's4_avant_1027_gao__port_506261', 0.0027666183, 332, '355'), ('916235064', 'x1_1027_gao__port_506372', 0.0017146241, 332, '355'), ('916235064', 'autres_1027_gao__port_506127', 0.004825338, 332, '355'), ('916235064', '208_1027_gao__port_506359', 0.0018686247, 332, '355'), ('916235064', 'c8_1027_gao__port_506135', 0.0012579451, 332, '355'), ('916235064', 'astra_1027_gao__port_506215', 0.0012625353, 332, '355'), ('916235064', '2_1027_gao__port_506151', 0.0009244881, 332, '355'), ('916235064', 'doblo_1027_gao__port_506251', 0.007466719, 332, '355'), ('916235064', '807_1027_gao__port_506152', 0.0007290021, 332, '355'), ('916235064', '206_1027_gao__port_506126', 0.0010385315, 332, '355'), ('916235064', 'a7_1027_gao__port_506373', 0.0006911813, 332, '355'), ('916235064', 'renegade_1027_gao__port_506346', 0.002141798, 332, '355')]]} begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 0 insert into MTRPhoto.class_photo_score (thcl, photo_id, hashtag_id, score) values (%s,%s,%s,%s) on duplicate key update score = values(score) time used for this insertion : 1.049041748046875e-05 save missing photos in datou_result : time spend for datou_step_exec : 46.30065989494324 time spend to save output : 1.4861817359924316 total time spend for step 1 : 47.78684163093567 step2:argmax Sun Mar 9 06:38:11 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec After prepare type args : Here we display some param of map_info ! map_filenames : {'temp/1741498643_588358_916235064_6293d1bb790dc6902450e7c572b7d10b.jpg': 916235064} map_photo_id_path_extension : {916235064: {'path': 'temp/1741498643_588358_916235064_6293d1bb790dc6902450e7c572b7d10b.jpg', 'extension': 'jpg'}} map_subphoto_mainphoto : {} Beginning of datou_step Argmax ! calculate argmax for thcl : 355 After datou_step_exec type output : map_portfolio_photo : len 0 keys : dict_keys([]) Inside saveOutput : final : True verbose : True photo_id : 916235064 output[photo_id] : [('916235064', 'c15_1027_gao__port_506055', 0.017713884, 332, '355'), 'temp/1741498643_588358_916235064_6293d1bb790dc6902450e7c572b7d10b.jpg'] begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 1 insert ignore into MTRBack.photo_hashtag_ids (photo_id, hashtag_id, type) values (%s,%s,%s) insert ignore into MTRBack.photo_hashtag_ids (photo_id, hashtag_id, type) values (%s,%s,%s) first line : ('916235064', '2049863950', '332') ... last line : ('916235064', '2049863950', '332') time used for this insertion : 0.00810694694519043 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 1 insert into MTRPhoto.class_photo_score (thcl, photo_id, hashtag_id, score) values (%s,%s,%s,%s) on duplicate key update score = values(score) time used for this insertion : 0.011272430419921875 len list_finale : 1, len picture : 1 begin to insert list_values into mtr_datou_result : length of list_values in save_final : 1 insert ignore into MTRPhoto.mtr_datou_result (mtd_id, mtr_portfolio_id,mtr_photo_id,result,result_long,result_double,hashtag_id,proba, mtr_current_id) values (%s,%s,%s,%s,%s,%s,%s,%s,%s) on duplicate key update mtr_portfolio_id = mtr_portfolio_id list_values : [('2', None, '916235064', 'c15_1027_gao__port_506055', None, None, '2049863950', '0.017713884', None)] time used for this insertion : 0.01636219024658203 saving photo_ids in datou_result photo id not in port begin to insert list_values into mtr_datou_result : length of list_values in save_final : 0 insert ignore into MTRPhoto.mtr_datou_result (mtd_id, mtr_portfolio_id,mtr_photo_id,result,result_long,result_double,hashtag_id,proba, mtr_current_id) values (%s,%s,%s,%s,%s,%s,%s,%s,%s) on duplicate key update mtr_portfolio_id = mtr_portfolio_id list_values : [] time used for this insertion : 8.106231689453125e-06 save missing photos in datou_result : time spend for datou_step_exec : 0.000530242919921875 time spend to save output : 0.036362409591674805 total time spend for step 2 : 0.03689265251159668 caffe_path_current : About to save ! 2 After save, about to update current ! datou_cur_ids : [] len(datou.list_steps) : 2 output : {'916235064': [('916235064', 'c15_1027_gao__port_506055', 0.017713884, 332, '355'), 'temp/1741498643_588358_916235064_6293d1bb790dc6902450e7c572b7d10b.jpg']} ############################### TEST tfhub2 ################################ TEST TFHUB2 ######################## test with use_multi_inputs=0 ######################## Inside batchDatouExec : verbose : True ##### chargement datou SELECT name, created_at,limit_max FROM MTRDatou.mtr_datou WHERE id=4567 SELECT mtd.id, mtdt.`type`, mtd.`param`, mtd.param_json, mtdt.nb_input, mtdt.nb_output, mtdt.prod, mtdt.is_local, mtdt.is_datou_depend, mtdt.is_photo_id_local FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_step_types mtdt WHERE mtdt.`id`=mtd.`type` AND mtd.mtd_id=4567 SELECT mtd.id, mtd.mtd_id, mdsdt.id, mdsdt.name, mdsdt.description, msid.output_or_input, msid.data_order_id, mdsdt.type FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_steptype_io_datatypes msid, MTRDatou.mtr_datou_step_data_types mdsdt WHERE mtd.`type`=msid.`mtr_datou_step_type` AND mtd.mtd_id= 4567 AND msid.data_type=mdsdt.id SELECT mts_id_output, id_output, mts_id_input, id_input FROM MTRDatou.mtr_datou_step_by_step WHERE mtd_id=4567 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : step 12835 tfhub_classification2 is not linked in the step_by_step architecture ! WARNING : step 12836 argmax is not linked in the step_by_step architecture ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! DataTypes for each output/input checked ! no param json to modify List Step Type Loaded in datou : tfhub_classification2, argmax list_input_json : [] ##### fin chargement datou ##### chargement data ##### Call load_data_input : nb_thread : 5 origin SELECT photo_id, url FROM MTRBack.photos ph WHERE photo_id IN (1171252784,1171252764,1171252487) Found this number of photos: 3 ##### Call download_photos : nb_thread : 5 begin to download photo : 1171252487 begin to download photo : 1171252764 begin to download photo : 1171252784 download finish for photo 1171252784 download finish for photo 1171252487 download finish for photo 1171252764 we have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB ##### After download_photos length of list_filenames : 3 ; length of list_pids : 3 ; length of list_args : 3 ##### After load_data_input time to download the photos : 0.19728899002075195 #### fin chargement data Blocking on flush ? No conitnuing About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : True number of steps : 2 step1:tfhub_classification2 Sun Mar 9 06:38:12 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec After prepare type args : Here we display some param of map_info ! map_filenames : {'temp/1741498691_588358_1171252784_5a3c5d3bb155a7a116f67ded51bffb59.jpg': 1171252784, 'temp/1741498691_588358_1171252487_5ebdd6b0a6bb39942a3808ed114806de.jpg': 1171252487, 'temp/1741498691_588358_1171252764_29d5179a892cc50aadc9d67245534b59.jpg': 1171252764} map_photo_id_path_extension : {1171252784: {'path': 'temp/1741498691_588358_1171252784_5a3c5d3bb155a7a116f67ded51bffb59.jpg', 'extension': 'jpg'}, 1171252487: {'path': 'temp/1741498691_588358_1171252487_5ebdd6b0a6bb39942a3808ed114806de.jpg', 'extension': 'jpg'}, 1171252764: {'path': 'temp/1741498691_588358_1171252764_29d5179a892cc50aadc9d67245534b59.jpg', 'extension': 'jpg'}} map_subphoto_mainphoto : {} Beginning of datou_step TFHub with tf2 ! multi_thcl or not :False multi_thcl_cond or not :False dic_thcl : {'3609': 1} we are using the classfication for only one thcl 3609 begin to check gpu status inside check gpu memory havn't enough memory gpu , need / 3096 l 3632 free memory gpu now : 32 wait 20 seconds inside check gpu memory havn't enough memory gpu , need / 3096 l 3632 free memory gpu now : 1637 wait 20 seconds inside check gpu memory havn't enough memory gpu , need / 3096 l 3632 free memory gpu now : 1637 wait 20 seconds inside check gpu memory havn't enough memory gpu , need / 3096 l 3632 free memory gpu now : 1637 wait 20 seconds inside check gpu memory havn't enough memory gpu , need / 3096 l 3632 free memory gpu now : 1484 wait 20 seconds inside check gpu memory havn't enough memory gpu , need / 3096 l 3632 free memory gpu now : 1637 wait 20 seconds 2025-03-09 06:40:21.081526: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-03-09 06:40:21.082343: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-03-09 06:40:21.082469: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-09 06:40:21.082534: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-09 06:40:21.085317: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-03-09 06:40:21.085436: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-03-09 06:40:21.088569: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-03-09 06:40:21.089827: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-03-09 06:40:21.094859: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-09 06:40:21.095934: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-03-09 06:40:21.096648: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-03-09 06:40:21.131282: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-03-09 06:40:21.133500: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7fef28000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-03-09 06:40:21.133531: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-03-09 06:40:21.137135: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x2d99dbb0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-03-09 06:40:21.137191: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-03-09 06:40:21.138048: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-03-09 06:40:21.138224: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-09 06:40:21.138257: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-09 06:40:21.138392: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-03-09 06:40:21.138433: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-03-09 06:40:21.138484: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-03-09 06:40:21.138537: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-03-09 06:40:21.138591: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-09 06:40:21.139781: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-03-09 06:40:21.139868: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-09 06:40:21.139944: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-03-09 06:40:21.139961: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-03-09 06:40:21.139973: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-03-09 06:40:21.141202: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 3096 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) l 3637 free memory gpu now : 1637 max_wait_temp : 6 max_wait : 5 1 Physical GPUs, 1 Logical GPUs tagging for thcl : 3609 To do loadFromThcl(), then load ParamDescType : thcl3609 get_desc_type_from_thcl : type of cat SELECT id, mtr_user_id, name, pb_hashtag_id, hashtag_id_list, button_legend_list, portfolio_id_lists, photo_hashtag_type, photo_desc_type, svm_limit, limit_tagging, is_public, live, created_at, updated_at, type_classification FROM MTRDatou.classification_theme WHERE `id` IN (3609) thcls : [{'id': 3609, 'mtr_user_id': 31, 'name': 'tfhub_19_06_2023', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'jrm,pcm,pcnc,pehd,tapis_vide', 'svm_portfolios_learning': '9336903,9336904,9336905,9336906,9336909', 'photo_hashtag_type': 4674, 'photo_desc_type': 5832, 'type_classification': 'tf_classification2', 'hashtag_id_list': '495916461,560181804,1284539308,628944319,2107748999'}] thcl {'id': 3609, 'mtr_user_id': 31, 'name': 'tfhub_19_06_2023', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'jrm,pcm,pcnc,pehd,tapis_vide', 'svm_portfolios_learning': '9336903,9336904,9336905,9336906,9336909', 'photo_hashtag_type': 4674, 'photo_desc_type': 5832, 'type_classification': 'tf_classification2', 'hashtag_id_list': '495916461,560181804,1284539308,628944319,2107748999'} Update svm_hashtag_type_desc : 5832 SELECT * FROM MTRDatou.photo_desc_type_params WHERE id in (5832) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5832, 'tfhub_19_06_2023', 1280, 1280, 'tfhub_19_06_2023', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2023, 6, 19, 12, 55, 22), datetime.datetime(2023, 6, 19, 12, 55, 22)) model_name : tfhub_19_06_2023 model_param file didn't exist model_name : tfhub_19_06_2023 model_type : tf_classification2 list file need : ['Confusion_Matrix.png', 'Precision_Recall_jrm.jpg', 'Precision_Recall_pcm.jpg', 'Precision_Recall_pcnc.jpg', 'Precision_Recall_pehd.jpg', 'Precision_Recall_tapis_vide.jpg', 'Result_Summary.txt', 'checkpoint', 'model_checkpoint.ckpt.data-00000-of-00002', 'model_checkpoint.ckpt.data-00001-of-00002', 'model_checkpoint.ckpt.index', 'model_weights.h5'] file exist in s3 : ['Confusion_Matrix.png', 'Precision_Recall_jrm.jpg', 'Precision_Recall_pcm.jpg', 'Precision_Recall_pcnc.jpg', 'Precision_Recall_pehd.jpg', 'Precision_Recall_tapis_vide.jpg', 'Result_Summary.txt', 'checkpoint', 'model_checkpoint.ckpt.data-00000-of-00002', 'model_checkpoint.ckpt.data-00001-of-00002', 'model_checkpoint.ckpt.index', 'model_weights.h5'] file manque in s3 : [] 2025-03-09 06:40:29.790602: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.02G (3246391296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-09 06:40:29.791280: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.72G (2921752064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-09 06:40:29.791857: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.45G (2629576704 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-09 06:40:29.792430: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.20G (2366618880 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-09 06:40:29.793034: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.98G (2129957120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-09 06:40:29.793599: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.79G (1916961536 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-09 06:40:29.794194: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.61G (1725265408 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory /home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python/../../tools/../lib/rpn/proposal_layer.py:28: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details. layer_params = yaml.load(self.param_str_) local folder : /data/models_weight/tfhub_19_06_2023 /data/models_weight/tfhub_19_06_2023/Confusion_Matrix.png size_local : 57753 size in s3 : 57753 create time local : 2023-06-22 17:09:38 create time in s3 : 2023-06-19 10:55:15 Confusion_Matrix.png already exist and didn't need to update /data/models_weight/tfhub_19_06_2023/Precision_Recall_jrm.jpg size_local : 79724 size in s3 : 79724 create time local : 2023-06-22 17:09:38 create time in s3 : 2023-06-19 10:55:20 Precision_Recall_jrm.jpg already exist and didn't need to update /data/models_weight/tfhub_19_06_2023/Precision_Recall_pcm.jpg size_local : 83556 size in s3 : 83556 create time local : 2023-06-22 17:09:38 create time in s3 : 2023-06-19 10:55:15 Precision_Recall_pcm.jpg already exist and didn't need to update /data/models_weight/tfhub_19_06_2023/Precision_Recall_pcnc.jpg size_local : 74107 size in s3 : 74107 create time local : 2023-06-22 17:09:38 create time in s3 : 2023-06-19 10:55:20 Precision_Recall_pcnc.jpg already exist and didn't need to update /data/models_weight/tfhub_19_06_2023/Precision_Recall_pehd.jpg size_local : 72705 size in s3 : 72705 create time local : 2023-06-22 17:09:39 create time in s3 : 2023-06-19 10:55:20 Precision_Recall_pehd.jpg already exist and didn't need to update /data/models_weight/tfhub_19_06_2023/Precision_Recall_tapis_vide.jpg size_local : 70874 size in s3 : 70874 create time local : 2023-06-22 17:09:39 create time in s3 : 2023-06-19 10:55:15 Precision_Recall_tapis_vide.jpg already exist and didn't need to update /data/models_weight/tfhub_19_06_2023/Result_Summary.txt size_local : 642 size in s3 : 642 create time local : 2023-06-22 17:09:39 create time in s3 : 2023-06-19 10:55:22 Result_Summary.txt already exist and didn't need to update /data/models_weight/tfhub_19_06_2023/checkpoint size_local : 99 size in s3 : 99 create time local : 2023-06-22 17:09:39 create time in s3 : 2023-06-19 10:55:22 checkpoint already exist and didn't need to update /data/models_weight/tfhub_19_06_2023/model_checkpoint.ckpt.data-00000-of-00002 size_local : 216488 size in s3 : 216488 create time local : 2023-06-22 17:09:39 create time in s3 : 2023-06-19 10:55:22 model_checkpoint.ckpt.data-00000-of-00002 already exist and didn't need to update /data/models_weight/tfhub_19_06_2023/model_checkpoint.ckpt.data-00001-of-00002 size_local : 32279708 size in s3 : 32279708 create time local : 2023-06-22 17:09:40 create time in s3 : 2023-06-19 10:55:21 model_checkpoint.ckpt.data-00001-of-00002 already exist and didn't need to update /data/models_weight/tfhub_19_06_2023/model_checkpoint.ckpt.index size_local : 43546 size in s3 : 43546 create time local : 2023-06-22 17:09:40 create time in s3 : 2023-06-19 10:55:22 model_checkpoint.ckpt.index already exist and didn't need to update /data/models_weight/tfhub_19_06_2023/model_weights.h5 size_local : 16499144 size in s3 : 16499144 create time local : 2023-06-22 17:09:40 create time in s3 : 2023-06-19 10:55:15 model_weights.h5 already exist and didn't need to update desc size : 1280 Model: "sequential" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= module (KerasLayer) (None, 1280) 4049564 _________________________________________________________________ tfhub_19_06_2023dense (Dense (None, 5) 6405 ================================================================= Total params: 4,055,969 Trainable params: 6,405 Non-trainable params: 4,049,564 _________________________________________________________________ Loading Weights... time used to create the model : 10.378074407577515 time used to load_weights : 0.13579773902893066 0it [00:00, ?it/s] 3it [00:00, 443.56it/s]2025-03-09 06:40:33.917731: E tensorflow/stream_executor/cuda/cuda_blas.cc:238] failed to create cublas handle: CUBLAS_STATUS_ALLOC_FAILED 2025-03-09 06:40:34.060392: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-09 06:40:34.083268: E tensorflow/stream_executor/cuda/cuda_dnn.cc:329] Could not create cudnn handle: CUDNN_STATUS_INTERNAL_ERROR 2025-03-09 06:40:34.090192: E tensorflow/stream_executor/cuda/cuda_dnn.cc:329] Could not create cudnn handle: CUDNN_STATUS_INTERNAL_ERROR temp/1741498691_588358_1171252784_5a3c5d3bb155a7a116f67ded51bffb59.jpg temp/1741498691_588358_1171252487_5ebdd6b0a6bb39942a3808ed114806de.jpg temp/1741498691_588358_1171252764_29d5179a892cc50aadc9d67245534b59.jpg Found 3 images belonging to 1 classes. begin to do the prediction : ERROR in datou_step_exec, will save and exit ! Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above. [[{{node model/module/StatefulPartitionedCall/StatefulPartitionedCall/StatefulPartitionedCall/stem_conv2d/StatefulPartitionedCall/Conv2D}}]] [Op:__inference_predict_function_34620] Function call stack: predict_function File "/home/admin/workarea/git/Velours/python/mtr/datou/datou_lib.py", line 2329, in datou_exec output = datou_step_exec(sNext, args, cache, context, map_info, verbose, mtr_user_id) File "/home/admin/workarea/git/Velours/python/mtr/datou/datou_lib.py", line 2523, in datou_step_exec return lib_process.datou_step_tfhub2(param, json_param, args, cache, context, map_info, verbose) File "/home/admin/workarea/git/Velours/python/mtr/datou/lib_step_exec/lib_step_process.py", line 3146, in datou_step_tfhub2 classes, outputs, features = this_model.predict_image_paths(list_paths, keep_aspect_ratio=keep_aspect_ratio, File "/home/admin/workarea/git/Velours/python/mtr/tfhub2/evaluate.py", line 288, in predict_image_paths Y_pred, F_pred = self.model.predict(valid_generator, validation_steps) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/keras/engine/training.py", line 88, in _method_wrapper return method(self, *args, **kwargs) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/keras/engine/training.py", line 1268, in predict tmp_batch_outputs = predict_function(iterator) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/def_function.py", line 580, in __call__ result = self._call(*args, **kwds) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/def_function.py", line 650, in _call return self._concrete_stateful_fn._filtered_call(canon_args, canon_kwds) # pylint: disable=protected-access File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/function.py", line 1661, in _filtered_call return self._call_flat( File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/function.py", line 1745, in _call_flat return self._build_call_outputs(self._inference_function.call( File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/function.py", line 593, in call outputs = execute.execute( File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/execute.py", line 59, in quick_execute tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name, [1171252784, 1171252487, 1171252764] map_info['map_portfolio_photo'] : {} final : True mtd_id 4567 list_pids : [1171252784, 1171252487, 1171252764] begin to insert list_values into mtr_datou_result : length of list_values in save_final : 3 insert ignore into MTRPhoto.mtr_datou_result (mtd_id, mtr_portfolio_id,mtr_photo_id,result,result_long,result_double,hashtag_id,proba, mtr_current_id) values (%s,%s,%s,%s,%s,%s,%s,%s,%s) on duplicate key update mtr_portfolio_id = mtr_portfolio_id list_values : [('4567', None, '1171252784', "[>, , , , , ' Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.\\n\\t [[{{node model/module/StatefulPartitionedCall/StatefulPartitionedCall/StatefulPartitionedCall/stem_conv2d/StatefulPartitionedCall/Conv2D}}]] [Op:__inference_predict_function_34620]\\n\\nFunction call stack:\\npredict_function\\n']", '-1', '-1.0', '501120777', '1.0', None), ('4567', None, '1171252487', "[>, , , , , ' Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.\\n\\t [[{{node model/module/StatefulPartitionedCall/StatefulPartitionedCall/StatefulPartitionedCall/stem_conv2d/StatefulPartitionedCall/Conv2D}}]] [Op:__inference_predict_function_34620]\\n\\nFunction call stack:\\npredict_function\\n']", '-1', '-1.0', '501120777', '1.0', None), ('4567', None, '1171252764', "[>, , , , , ' Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.\\n\\t [[{{node model/module/StatefulPartitionedCall/StatefulPartitionedCall/StatefulPartitionedCall/stem_conv2d/StatefulPartitionedCall/Conv2D}}]] [Op:__inference_predict_function_34620]\\n\\nFunction call stack:\\npredict_function\\n']", '-1', '-1.0', '501120777', '1.0', None)] time used for this insertion : 0.0205080509185791 save_final ERROR in last step tfhub_classification2, Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above. [[{{node model/module/StatefulPartitionedCall/StatefulPartitionedCall/StatefulPartitionedCall/stem_conv2d/StatefulPartitionedCall/Conv2D}}]] [Op:__inference_predict_function_34620] Function call stack: predict_function time spend for datou_step_exec : 142.00522470474243 time spend to save output : 0.023289918899536133 total time spend for step 0 : 142.02851462364197 need to delete datou_research and reload, so keep current state 1 caffe_path_current : About to save ! 2 After save, about to update current ! datou_cur_ids : [] len(datou.list_steps) : 2 output : None probably due to empty image bug ERROR expected : {'1171252784': [(1171252784, 'jrm', 0.9677492, 4674, '3609'), 'temp/1687511175_1882837_1171252784_5a3c5d3bb155a7a116f67ded51bffb59.jpg'], '1171252764': [(1171252764, 'jrm', 0.9853587, 4674, '3609'), 'temp/1687511175_1882837_1171252764_29d5179a892cc50aadc9d67245534b59.jpg'], '1171252487': [(1171252487, 'jrm', 0.9262757, 4674, '3609'), 'temp/1687511175_1882837_1171252487_5ebdd6b0a6bb39942a3808ed114806de.jpg']} got : None ######################## test with use_multi_inputs=1 ######################## Inside batchDatouExec : verbose : True ##### chargement datou SELECT name, created_at,limit_max FROM MTRDatou.mtr_datou WHERE id=4621 SELECT mtd.id, mtdt.`type`, mtd.`param`, mtd.param_json, mtdt.nb_input, mtdt.nb_output, mtdt.prod, mtdt.is_local, mtdt.is_datou_depend, mtdt.is_photo_id_local FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_step_types mtdt WHERE mtdt.`id`=mtd.`type` AND mtd.mtd_id=4621 SELECT mtd.id, mtd.mtd_id, mdsdt.id, mdsdt.name, mdsdt.description, msid.output_or_input, msid.data_order_id, mdsdt.type FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_steptype_io_datatypes msid, MTRDatou.mtr_datou_step_data_types mdsdt WHERE mtd.`type`=msid.`mtr_datou_step_type` AND mtd.mtd_id= 4621 AND msid.data_type=mdsdt.id SELECT mts_id_output, id_output, mts_id_input, id_input FROM MTRDatou.mtr_datou_step_by_step WHERE mtd_id=4621 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : step 12927 tfhub_classification2 is not linked in the step_by_step architecture ! WARNING : step 12928 argmax is not linked in the step_by_step architecture ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! DataTypes for each output/input checked ! no param json to modify List Step Type Loaded in datou : tfhub_classification2, argmax list_input_json : [] ##### fin chargement datou ##### chargement data ##### Call load_data_input : nb_thread : 5 origin SELECT photo_id, url FROM MTRBack.photos ph WHERE photo_id IN (1171291875,1171275372,1171275314) Found this number of photos: 3 ##### Call download_photos : nb_thread : 5 begin to download photo : 1171275314 begin to download photo : 1171275372 begin to download photo : 1171291875 download finish for photo 1171275372 download finish for photo 1171275314 download finish for photo 1171291875 we have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB ##### After download_photos length of list_filenames : 3 ; length of list_pids : 3 ; length of list_args : 3 ##### After load_data_input time to download the photos : 0.17251253128051758 #### fin chargement data Blocking on flush ? No conitnuing About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : True number of steps : 2 step1:tfhub_classification2 Sun Mar 9 06:40:34 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec After prepare type args : Here we display some param of map_info ! map_filenames : {'temp/1741498834_588358_1171275372_76d81364ff7df843bff095f45c07ba35.jpg': 1171275372, 'temp/1741498834_588358_1171275314_6e0a72c8fa00d5e4b018bd689b547133.jpg': 1171275314, 'temp/1741498834_588358_1171291875_b62cd9e0d976b143f86fe82d072798c0.jpg': 1171291875} map_photo_id_path_extension : {1171275372: {'path': 'temp/1741498834_588358_1171275372_76d81364ff7df843bff095f45c07ba35.jpg', 'extension': 'jpg'}, 1171275314: {'path': 'temp/1741498834_588358_1171275314_6e0a72c8fa00d5e4b018bd689b547133.jpg', 'extension': 'jpg'}, 1171291875: {'path': 'temp/1741498834_588358_1171291875_b62cd9e0d976b143f86fe82d072798c0.jpg', 'extension': 'jpg'}} map_subphoto_mainphoto : {} Beginning of datou_step TFHub with tf2 ! multi_thcl or not :False multi_thcl_cond or not :False dic_thcl : {'3655': 1} we are using the classfication for only one thcl 3655 begin to check gpu status inside check gpu memory havn't enough memory gpu , need / 3096 l 3632 free memory gpu now : 21 wait 20 seconds inside check gpu memory havn't enough memory gpu , need / 3096 l 3632 free memory gpu now : 21 wait 20 seconds inside check gpu memory havn't enough memory gpu , need / 3096 l 3632 free memory gpu now : 21 wait 20 seconds inside check gpu memory havn't enough memory gpu , need / 3096 l 3632 free memory gpu now : 21 wait 20 seconds inside check gpu memory havn't enough memory gpu , need / 3096 l 3632 free memory gpu now : 21 wait 20 seconds inside check gpu memory havn't enough memory gpu , need / 3096 l 3632 free memory gpu now : 21 wait 20 seconds l 3637 free memory gpu now : 21 max_wait_temp : 6 max_wait : 5 1 Physical GPUs, 1 Logical GPUs tagging for thcl : 3655 To do loadFromThcl(), then load ParamDescType : thcl3655 get_desc_type_from_thcl : type of cat SELECT id, mtr_user_id, name, pb_hashtag_id, hashtag_id_list, button_legend_list, portfolio_id_lists, photo_hashtag_type, photo_desc_type, svm_limit, limit_tagging, is_public, live, created_at, updated_at, type_classification FROM MTRDatou.classification_theme WHERE `id` IN (3655) thcls : [{'id': 3655, 'mtr_user_id': 31, 'name': 'tfhub_18_7_2023', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'pcm,pcnc,jrm,pehd,tapis_vide', 'svm_portfolios_learning': '9336904,9336905,9336903,9336906,9336909', 'photo_hashtag_type': 4723, 'photo_desc_type': 5862, 'type_classification': 'tf_classification2', 'hashtag_id_list': '560181804,1284539308,495916461,628944319,2107748999'}] thcl {'id': 3655, 'mtr_user_id': 31, 'name': 'tfhub_18_7_2023', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'pcm,pcnc,jrm,pehd,tapis_vide', 'svm_portfolios_learning': '9336904,9336905,9336903,9336906,9336909', 'photo_hashtag_type': 4723, 'photo_desc_type': 5862, 'type_classification': 'tf_classification2', 'hashtag_id_list': '560181804,1284539308,495916461,628944319,2107748999'} Update svm_hashtag_type_desc : 5862 SELECT * FROM MTRDatou.photo_desc_type_params WHERE id in (5862) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5862, 'tfhub_18_7_2023', 1280, 1280, 'tfhub_18_7_2023', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2023, 7, 18, 22, 46, 29), datetime.datetime(2023, 7, 18, 22, 46, 29)) model_name : tfhub_18_7_2023 model_param file didn't exist model_name : tfhub_18_7_2023 model_type : tf_classification2 list file need : ['Confusion_Matrix.png', 'Precision_Recall_jrm.jpg', 'Precision_Recall_pcm.jpg', 'Precision_Recall_pcnc.jpg', 'Precision_Recall_pehd.jpg', 'Precision_Recall_tapis_vide.jpg', 'Result_Summary.txt', 'checkpoint', 'model_checkpoint.ckpt.data-00000-of-00002', 'model_checkpoint.ckpt.data-00001-of-00002', 'model_checkpoint.ckpt.index', 'model_weights.h5'] file exist in s3 : ['Confusion_Matrix.png', 'Precision_Recall_jrm.jpg', 'Precision_Recall_pcm.jpg', 'Precision_Recall_pcnc.jpg', 'Precision_Recall_pehd.jpg', 'Precision_Recall_tapis_vide.jpg', 'Result_Summary.txt', 'checkpoint', 'model_checkpoint.ckpt.data-00000-of-00002', 'model_checkpoint.ckpt.data-00001-of-00002', 'model_checkpoint.ckpt.index', 'model_weights.h5'] file manque in s3 : [] 2025-03-09 06:42:52.258382: E tensorflow/stream_executor/cuda/cuda_blas.cc:238] failed to create cublas handle: CUBLAS_STATUS_ALLOC_FAILED 2025-03-09 06:42:52.342081: E tensorflow/stream_executor/cuda/cuda_dnn.cc:329] Could not create cudnn handle: CUDNN_STATUS_INTERNAL_ERROR 2025-03-09 06:42:52.343478: E tensorflow/stream_executor/cuda/cuda_dnn.cc:329] Could not create cudnn handle: CUDNN_STATUS_INTERNAL_ERROR local folder : /data/models_weight/tfhub_18_7_2023 /data/models_weight/tfhub_18_7_2023/Confusion_Matrix.png size_local : 54360 size in s3 : 54360 create time local : 2023-08-11 11:22:56 create time in s3 : 2023-07-18 20:46:28 Confusion_Matrix.png already exist and didn't need to update /data/models_weight/tfhub_18_7_2023/Precision_Recall_jrm.jpg size_local : 72583 size in s3 : 72583 create time local : 2023-08-11 11:22:56 create time in s3 : 2023-07-18 20:46:23 Precision_Recall_jrm.jpg already exist and didn't need to update /data/models_weight/tfhub_18_7_2023/Precision_Recall_pcm.jpg size_local : 81681 size in s3 : 81681 create time local : 2023-08-11 11:22:56 create time in s3 : 2023-07-18 20:46:17 Precision_Recall_pcm.jpg already exist and didn't need to update /data/models_weight/tfhub_18_7_2023/Precision_Recall_pcnc.jpg size_local : 79510 size in s3 : 79510 create time local : 2023-08-11 11:22:56 create time in s3 : 2023-07-18 20:46:23 Precision_Recall_pcnc.jpg already exist and didn't need to update /data/models_weight/tfhub_18_7_2023/Precision_Recall_pehd.jpg size_local : 59936 size in s3 : 59936 create time local : 2023-08-11 11:22:57 create time in s3 : 2023-07-18 20:46:23 Precision_Recall_pehd.jpg already exist and didn't need to update /data/models_weight/tfhub_18_7_2023/Precision_Recall_tapis_vide.jpg size_local : 78974 size in s3 : 78974 create time local : 2023-08-11 11:22:57 create time in s3 : 2023-07-18 20:46:17 Precision_Recall_tapis_vide.jpg already exist and didn't need to update /data/models_weight/tfhub_18_7_2023/Result_Summary.txt size_local : 642 size in s3 : 642 create time local : 2023-08-11 11:22:57 create time in s3 : 2023-07-18 20:46:23 Result_Summary.txt already exist and didn't need to update /data/models_weight/tfhub_18_7_2023/checkpoint size_local : 99 size in s3 : 99 create time local : 2023-08-11 11:22:57 create time in s3 : 2023-07-18 20:46:23 checkpoint already exist and didn't need to update /data/models_weight/tfhub_18_7_2023/model_checkpoint.ckpt.data-00000-of-00002 size_local : 216529 size in s3 : 216529 create time local : 2023-08-11 11:22:57 create time in s3 : 2023-07-18 20:46:17 model_checkpoint.ckpt.data-00000-of-00002 already exist and didn't need to update /data/models_weight/tfhub_18_7_2023/model_checkpoint.ckpt.data-00001-of-00002 size_local : 32279748 size in s3 : 32279748 create time local : 2023-08-11 11:22:58 create time in s3 : 2023-07-18 20:46:19 model_checkpoint.ckpt.data-00001-of-00002 already exist and didn't need to update /data/models_weight/tfhub_18_7_2023/model_checkpoint.ckpt.index size_local : 43546 size in s3 : 43546 create time local : 2023-08-11 11:22:58 create time in s3 : 2023-07-18 20:46:19 model_checkpoint.ckpt.index already exist and didn't need to update /data/models_weight/tfhub_18_7_2023/model_weights.h5 size_local : 16500868 size in s3 : 16500868 create time local : 2023-08-11 11:22:58 create time in s3 : 2023-07-18 20:46:18 model_weights.h5 already exist and didn't need to update desc size : 1280 Model: "model_1" __________________________________________________________________________________________________ Layer (type) Output Shape Param # Connected to ================================================================================================== input_2 (InputLayer) [(None, 224, 224, 3) 0 __________________________________________________________________________________________________ input_3 (InputLayer) [(None, 1)] 0 __________________________________________________________________________________________________ module (KerasLayer) (None, 1280) 4049564 input_2[0][0] __________________________________________________________________________________________________ concatenate (Concatenate) (None, 1281) 0 input_3[0][0] module[0][0] __________________________________________________________________________________________________ tfhub_18_7_2023dense (Dense) (None, 5) 6410 concatenate[0][0] ================================================================================================== Total params: 4,055,974 Trainable params: 0 Non-trainable params: 4,055,974 __________________________________________________________________________________________________ Loading Weights... time used to create the model : 8.294269561767578 time used to load_weights : 0.15512990951538086 found 3 data found 0 labels begin to do the prediction : ERROR in datou_step_exec, will save and exit ! Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above. [[{{node model_2/module/StatefulPartitionedCall/StatefulPartitionedCall/StatefulPartitionedCall/stem_conv2d/StatefulPartitionedCall/Conv2D}}]] [Op:__inference_predict_function_69245] Function call stack: predict_function File "/home/admin/workarea/git/Velours/python/mtr/datou/datou_lib.py", line 2329, in datou_exec output = datou_step_exec(sNext, args, cache, context, map_info, verbose, mtr_user_id) File "/home/admin/workarea/git/Velours/python/mtr/datou/datou_lib.py", line 2523, in datou_step_exec return lib_process.datou_step_tfhub2(param, json_param, args, cache, context, map_info, verbose) File "/home/admin/workarea/git/Velours/python/mtr/datou/lib_step_exec/lib_step_process.py", line 3146, in datou_step_tfhub2 classes, outputs, features = this_model.predict_image_paths(list_paths, keep_aspect_ratio=keep_aspect_ratio, File "/home/admin/workarea/git/Velours/python/mtr/tfhub2/evaluate.py", line 288, in predict_image_paths Y_pred, F_pred = self.model.predict(valid_generator, validation_steps) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/keras/engine/training.py", line 88, in _method_wrapper return method(self, *args, **kwargs) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/keras/engine/training.py", line 1268, in predict tmp_batch_outputs = predict_function(iterator) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/def_function.py", line 580, in __call__ result = self._call(*args, **kwds) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/def_function.py", line 650, in _call return self._concrete_stateful_fn._filtered_call(canon_args, canon_kwds) # pylint: disable=protected-access File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/function.py", line 1661, in _filtered_call return self._call_flat( File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/function.py", line 1745, in _call_flat return self._build_call_outputs(self._inference_function.call( File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/function.py", line 593, in call outputs = execute.execute( File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/execute.py", line 59, in quick_execute tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name, [1171275372, 1171275314, 1171291875] map_info['map_portfolio_photo'] : {} final : True mtd_id 4621 list_pids : [1171275372, 1171275314, 1171291875] begin to insert list_values into mtr_datou_result : length of list_values in save_final : 3 insert ignore into MTRPhoto.mtr_datou_result (mtd_id, mtr_portfolio_id,mtr_photo_id,result,result_long,result_double,hashtag_id,proba, mtr_current_id) values (%s,%s,%s,%s,%s,%s,%s,%s,%s) on duplicate key update mtr_portfolio_id = mtr_portfolio_id list_values : [('4621', None, '1171275372', "[>, , , , , ' Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.\\n\\t [[{{node model_2/module/StatefulPartitionedCall/StatefulPartitionedCall/StatefulPartitionedCall/stem_conv2d/StatefulPartitionedCall/Conv2D}}]] [Op:__inference_predict_function_69245]\\n\\nFunction call stack:\\npredict_function\\n']", '-1', '-1.0', '501120777', '1.0', None), ('4621', None, '1171275314', "[>, , , , , ' Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.\\n\\t [[{{node model_2/module/StatefulPartitionedCall/StatefulPartitionedCall/StatefulPartitionedCall/stem_conv2d/StatefulPartitionedCall/Conv2D}}]] [Op:__inference_predict_function_69245]\\n\\nFunction call stack:\\npredict_function\\n']", '-1', '-1.0', '501120777', '1.0', None), ('4621', None, '1171291875', "[>, , , , , ' Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.\\n\\t [[{{node model_2/module/StatefulPartitionedCall/StatefulPartitionedCall/StatefulPartitionedCall/stem_conv2d/StatefulPartitionedCall/Conv2D}}]] [Op:__inference_predict_function_69245]\\n\\nFunction call stack:\\npredict_function\\n']", '-1', '-1.0', '501120777', '1.0', None)] time used for this insertion : 0.015561103820800781 save_final ERROR in last step tfhub_classification2, Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above. [[{{node model_2/module/StatefulPartitionedCall/StatefulPartitionedCall/StatefulPartitionedCall/stem_conv2d/StatefulPartitionedCall/Conv2D}}]] [Op:__inference_predict_function_69245] Function call stack: predict_function time spend for datou_step_exec : 138.03829908370972 time spend to save output : 0.01619863510131836 total time spend for step 0 : 138.05449771881104 need to delete datou_research and reload, so keep current state 1 caffe_path_current : About to save ! 2 After save, about to update current ! datou_cur_ids : [] len(datou.list_steps) : 2 output : None probably due to empty image bug ERROR expected : {'1171291875': [(1171291875, 'tapis_vide', 0.97062814, 4723, '3655'), 'temp/1691745841_1143057_1171291875_b62cd9e0d976b143f86fe82d072798c0.jpg'], '1171275372': [(1171275372, 'tapis_vide', 0.9674145, 4723, '3655'), 'temp/1691745841_1143057_1171275372_76d81364ff7df843bff095f45c07ba35.jpg'], '1171275314': [(1171275314, 'tapis_vide', 0.96509415, 4723, '3655'), 'temp/1691745841_1143057_1171275314_6e0a72c8fa00d5e4b018bd689b547133.jpg']} got : None ERROR tfhub2 FAILED ############################### TEST ordonner ################################ To do loadFromThcl(), then load ParamDescType : thcl358 get_desc_type_from_thcl : type of cat SELECT id, mtr_user_id, name, pb_hashtag_id, hashtag_id_list, button_legend_list, portfolio_id_lists, photo_hashtag_type, photo_desc_type, svm_limit, limit_tagging, is_public, live, created_at, updated_at, type_classification FROM MTRDatou.classification_theme WHERE `id` IN (358) thcls : [{'id': 358, 'mtr_user_id': 31, 'name': 'car_orientation_0111', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'FirstUploadExperveo_vignette__port_505674,CAR_EXTERIEUR_Roue__port_503398,FirstUploadExperveo_carrosseriegrosplan_VIndanslamoquette__port_506486,FirstUploadExperveo_carrosseriegrosplan_siegegrosplan__port_506485,CAR_EXTERIEUR_Cote_droit_axe_avant__port_504465,CAR_EXTERIEUR_Cote_gauche_axe_arriere__port_504198,CAR_EXTERIEUR_Face_avant_axe_droit__port_504451,CAR_EXTERIEUR_angle_avant_gauche_axe_avant__port_504235,FirstUploadExperveo_vin__port_505675,CAR_EXTERIEUR_cote_droite__port_504108,CAR_INTERIEUR_avant_volant_class_6_levierdevitesse__port_506565,FirstUploadExperveo_carrosseriegrosplan_carrosserie__port_506483,CAR_EXTERIEUR_Angle_arriere_gauche_axe_arriere__port_504201,cartegrise_orientation__port_505064,CAR_EXTERIEUR_Angle_arriere_droit_axe_arriere__port_504217,CAR_INTERIEUR_avant_vue-arriere_class_1__port_506531,CAR_EXTERIEUR_Face_arriere_axe_droit__port_504218,CAR_EXTERIEUR_Cote_droit_axe_arriere__port_504214,CAR_EXTERIEUR_Angle_avant_droit__port_504087,FirstUploadExperveo_carrosseriegrosplan_morceauderoue__port_506484,CAR_INTERIEUR_avant_volant_class_6_class_2__port_506563,CAR_EXTERIEUR_Angle_arriere_droit__port_504160,CAR_EXTERIEUR_arriere__port_504184,CAR_INTERIEUR_avant_volant_class_6_boutonrond__port_506562,INTERIEUR_Compteur_kilometrique__port_503644,CAR_INTERIEUR_avant_vue_gauche_habitacle_class_1__port_506494,CAR_EXTERIEUR_Angle_arriere_gauche__port_504170,CAR_EXTERIEUR_Angle_avant_droit_axe_arriere__port_504226,CAR_EXTERIEUR_Face_arriere_axe_gauche__port_504202,CAR_EXTERIEUR_moteur__port_503704,FirstUploadExperveo_carrosseriegrosplan_class_6__port_506487,CAR_INTERIEUR_siege_arriere_class_1__port_506551,CAR_EXTERIEUR_avant__port_504146,CAR_EXTERIEUR_Angle_arriere_droit_axe_droit__port_504215,CAR_EXTERIEUR_Angle_avant_droit_axe_droit__port_504225,CAR_INTERIEUR_avant_volant_class_6_ecrangrosplan__port_506564,FirstUploadExperveo_carrosseriegrosplan_moteurgrosplanetdegat__port_506482,CAR_INTERIEUR_coffre__port_503412,FirstUploadExperveo_rouetranche__port_505677,UploadPhotoImmatBest_class_1__port_505051,CAR_INTERIEUR_avant_vue-arriere_class_2__port_506532,CAR_EXTERIEUR_angle_avant_gauche__port_504098,CAR_EXTERIEUR_face_avant_axe_gauche__port_504236,CAR_INTERIEUR_avant_vue_droite_habitacle_class_1__port_506540,CAR_EXTERIEUR_cote_gauche_axe_avant__port_504233,CAR_EXTERIEUR_roue_de_secour__port_503763,CAR_EXTERIEUR_Angle_arriere_gauche_axe_gauche__port_504199,CAR_EXTERIEUR_cote_gauche__port_504017,CAR_INTERIEUR_avant_volant_class_1__port_506503,CAR_INTERIEUR_avant_volant_class_2__port_506504,CAR_EXTERIEUR_angle_avant_gauche_axe_gauche__port_504234', 'svm_portfolios_learning': '505674,503398,506486,506485,504465,504198,504451,504235,505675,504108,506565,506483,504201,505064,504217,506531,504218,504214,504087,506484,506563,504160,504184,506562,503644,506494,504170,504226,504202,503704,506487,506551,504146,504215,504225,506564,506482,503412,505677,505051,506532,504098,504236,506540,504233,503763,504199,504017,506503,506504,504234', 'photo_hashtag_type': 337, 'photo_desc_type': 3392, 'type_classification': 'caffe', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 358, 'mtr_user_id': 31, 'name': 'car_orientation_0111', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'FirstUploadExperveo_vignette__port_505674,CAR_EXTERIEUR_Roue__port_503398,FirstUploadExperveo_carrosseriegrosplan_VIndanslamoquette__port_506486,FirstUploadExperveo_carrosseriegrosplan_siegegrosplan__port_506485,CAR_EXTERIEUR_Cote_droit_axe_avant__port_504465,CAR_EXTERIEUR_Cote_gauche_axe_arriere__port_504198,CAR_EXTERIEUR_Face_avant_axe_droit__port_504451,CAR_EXTERIEUR_angle_avant_gauche_axe_avant__port_504235,FirstUploadExperveo_vin__port_505675,CAR_EXTERIEUR_cote_droite__port_504108,CAR_INTERIEUR_avant_volant_class_6_levierdevitesse__port_506565,FirstUploadExperveo_carrosseriegrosplan_carrosserie__port_506483,CAR_EXTERIEUR_Angle_arriere_gauche_axe_arriere__port_504201,cartegrise_orientation__port_505064,CAR_EXTERIEUR_Angle_arriere_droit_axe_arriere__port_504217,CAR_INTERIEUR_avant_vue-arriere_class_1__port_506531,CAR_EXTERIEUR_Face_arriere_axe_droit__port_504218,CAR_EXTERIEUR_Cote_droit_axe_arriere__port_504214,CAR_EXTERIEUR_Angle_avant_droit__port_504087,FirstUploadExperveo_carrosseriegrosplan_morceauderoue__port_506484,CAR_INTERIEUR_avant_volant_class_6_class_2__port_506563,CAR_EXTERIEUR_Angle_arriere_droit__port_504160,CAR_EXTERIEUR_arriere__port_504184,CAR_INTERIEUR_avant_volant_class_6_boutonrond__port_506562,INTERIEUR_Compteur_kilometrique__port_503644,CAR_INTERIEUR_avant_vue_gauche_habitacle_class_1__port_506494,CAR_EXTERIEUR_Angle_arriere_gauche__port_504170,CAR_EXTERIEUR_Angle_avant_droit_axe_arriere__port_504226,CAR_EXTERIEUR_Face_arriere_axe_gauche__port_504202,CAR_EXTERIEUR_moteur__port_503704,FirstUploadExperveo_carrosseriegrosplan_class_6__port_506487,CAR_INTERIEUR_siege_arriere_class_1__port_506551,CAR_EXTERIEUR_avant__port_504146,CAR_EXTERIEUR_Angle_arriere_droit_axe_droit__port_504215,CAR_EXTERIEUR_Angle_avant_droit_axe_droit__port_504225,CAR_INTERIEUR_avant_volant_class_6_ecrangrosplan__port_506564,FirstUploadExperveo_carrosseriegrosplan_moteurgrosplanetdegat__port_506482,CAR_INTERIEUR_coffre__port_503412,FirstUploadExperveo_rouetranche__port_505677,UploadPhotoImmatBest_class_1__port_505051,CAR_INTERIEUR_avant_vue-arriere_class_2__port_506532,CAR_EXTERIEUR_angle_avant_gauche__port_504098,CAR_EXTERIEUR_face_avant_axe_gauche__port_504236,CAR_INTERIEUR_avant_vue_droite_habitacle_class_1__port_506540,CAR_EXTERIEUR_cote_gauche_axe_avant__port_504233,CAR_EXTERIEUR_roue_de_secour__port_503763,CAR_EXTERIEUR_Angle_arriere_gauche_axe_gauche__port_504199,CAR_EXTERIEUR_cote_gauche__port_504017,CAR_INTERIEUR_avant_volant_class_1__port_506503,CAR_INTERIEUR_avant_volant_class_2__port_506504,CAR_EXTERIEUR_angle_avant_gauche_axe_gauche__port_504234', 'svm_portfolios_learning': '505674,503398,506486,506485,504465,504198,504451,504235,505675,504108,506565,506483,504201,505064,504217,506531,504218,504214,504087,506484,506563,504160,504184,506562,503644,506494,504170,504226,504202,503704,506487,506551,504146,504215,504225,506564,506482,503412,505677,505051,506532,504098,504236,506540,504233,503763,504199,504017,506503,506504,504234', 'photo_hashtag_type': 337, 'photo_desc_type': 3392, 'type_classification': 'caffe', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 3392 ['FirstUploadExperveo_vignette__port_505674', 'CAR_EXTERIEUR_Roue__port_503398', 'FirstUploadExperveo_carrosseriegrosplan_VIndanslamoquette__port_506486', 'FirstUploadExperveo_carrosseriegrosplan_siegegrosplan__port_506485', 'CAR_EXTERIEUR_Cote_droit_axe_avant__port_504465', 'CAR_EXTERIEUR_Cote_gauche_axe_arriere__port_504198', 'CAR_EXTERIEUR_Face_avant_axe_droit__port_504451', 'CAR_EXTERIEUR_angle_avant_gauche_axe_avant__port_504235', 'FirstUploadExperveo_vin__port_505675', 'CAR_EXTERIEUR_cote_droite__port_504108', 'CAR_INTERIEUR_avant_volant_class_6_levierdevitesse__port_506565', 'FirstUploadExperveo_carrosseriegrosplan_carrosserie__port_506483', 'CAR_EXTERIEUR_Angle_arriere_gauche_axe_arriere__port_504201', 'cartegrise_orientation__port_505064', 'CAR_EXTERIEUR_Angle_arriere_droit_axe_arriere__port_504217', 'CAR_INTERIEUR_avant_vue-arriere_class_1__port_506531', 'CAR_EXTERIEUR_Face_arriere_axe_droit__port_504218', 'CAR_EXTERIEUR_Cote_droit_axe_arriere__port_504214', 'CAR_EXTERIEUR_Angle_avant_droit__port_504087', 'FirstUploadExperveo_carrosseriegrosplan_morceauderoue__port_506484', 'CAR_INTERIEUR_avant_volant_class_6_class_2__port_506563', 'CAR_EXTERIEUR_Angle_arriere_droit__port_504160', 'CAR_EXTERIEUR_arriere__port_504184', 'CAR_INTERIEUR_avant_volant_class_6_boutonrond__port_506562', 'INTERIEUR_Compteur_kilometrique__port_503644', 'CAR_INTERIEUR_avant_vue_gauche_habitacle_class_1__port_506494', 'CAR_EXTERIEUR_Angle_arriere_gauche__port_504170', 'CAR_EXTERIEUR_Angle_avant_droit_axe_arriere__port_504226', 'CAR_EXTERIEUR_Face_arriere_axe_gauche__port_504202', 'CAR_EXTERIEUR_moteur__port_503704', 'FirstUploadExperveo_carrosseriegrosplan_class_6__port_506487', 'CAR_INTERIEUR_siege_arriere_class_1__port_506551', 'CAR_EXTERIEUR_avant__port_504146', 'CAR_EXTERIEUR_Angle_arriere_droit_axe_droit__port_504215', 'CAR_EXTERIEUR_Angle_avant_droit_axe_droit__port_504225', 'CAR_INTERIEUR_avant_volant_class_6_ecrangrosplan__port_506564', 'FirstUploadExperveo_carrosseriegrosplan_moteurgrosplanetdegat__port_506482', 'CAR_INTERIEUR_coffre__port_503412', 'FirstUploadExperveo_rouetranche__port_505677', 'UploadPhotoImmatBest_class_1__port_505051', 'CAR_INTERIEUR_avant_vue-arriere_class_2__port_506532', 'CAR_EXTERIEUR_angle_avant_gauche__port_504098', 'CAR_EXTERIEUR_face_avant_axe_gauche__port_504236', 'CAR_INTERIEUR_avant_vue_droite_habitacle_class_1__port_506540', 'CAR_EXTERIEUR_cote_gauche_axe_avant__port_504233', 'CAR_EXTERIEUR_roue_de_secour__port_503763', 'CAR_EXTERIEUR_Angle_arriere_gauche_axe_gauche__port_504199', 'CAR_EXTERIEUR_cote_gauche__port_504017', 'CAR_INTERIEUR_avant_volant_class_1__port_506503', 'CAR_INTERIEUR_avant_volant_class_2__port_506504', 'CAR_EXTERIEUR_angle_avant_gauche_axe_gauche__port_504234'] 51 SELECT hashtag_id,hashtag FROM MTRBack.hashtags where hashtag in ('FirstUploadExperveo_vignette__port_505674','CAR_EXTERIEUR_Roue__port_503398','FirstUploadExperveo_carrosseriegrosplan_VIndanslamoquette__port_506486','FirstUploadExperveo_carrosseriegrosplan_siegegrosplan__port_506485','CAR_EXTERIEUR_Cote_droit_axe_avant__port_504465','CAR_EXTERIEUR_Cote_gauche_axe_arriere__port_504198','CAR_EXTERIEUR_Face_avant_axe_droit__port_504451','CAR_EXTERIEUR_angle_avant_gauche_axe_avant__port_504235','FirstUploadExperveo_vin__port_505675','CAR_EXTERIEUR_cote_droite__port_504108','CAR_INTERIEUR_avant_volant_class_6_levierdevitesse__port_506565','FirstUploadExperveo_carrosseriegrosplan_carrosserie__port_506483','CAR_EXTERIEUR_Angle_arriere_gauche_axe_arriere__port_504201','cartegrise_orientation__port_505064','CAR_EXTERIEUR_Angle_arriere_droit_axe_arriere__port_504217','CAR_INTERIEUR_avant_vue-arriere_class_1__port_506531','CAR_EXTERIEUR_Face_arriere_axe_droit__port_504218','CAR_EXTERIEUR_Cote_droit_axe_arriere__port_504214','CAR_EXTERIEUR_Angle_avant_droit__port_504087','FirstUploadExperveo_carrosseriegrosplan_morceauderoue__port_506484','CAR_INTERIEUR_avant_volant_class_6_class_2__port_506563','CAR_EXTERIEUR_Angle_arriere_droit__port_504160','CAR_EXTERIEUR_arriere__port_504184','CAR_INTERIEUR_avant_volant_class_6_boutonrond__port_506562','INTERIEUR_Compteur_kilometrique__port_503644','CAR_INTERIEUR_avant_vue_gauche_habitacle_class_1__port_506494','CAR_EXTERIEUR_Angle_arriere_gauche__port_504170','CAR_EXTERIEUR_Angle_avant_droit_axe_arriere__port_504226','CAR_EXTERIEUR_Face_arriere_axe_gauche__port_504202','CAR_EXTERIEUR_moteur__port_503704','FirstUploadExperveo_carrosseriegrosplan_class_6__port_506487','CAR_INTERIEUR_siege_arriere_class_1__port_506551','CAR_EXTERIEUR_avant__port_504146','CAR_EXTERIEUR_Angle_arriere_droit_axe_droit__port_504215','CAR_EXTERIEUR_Angle_avant_droit_axe_droit__port_504225','CAR_INTERIEUR_avant_volant_class_6_ecrangrosplan__port_506564','FirstUploadExperveo_carrosseriegrosplan_moteurgrosplanetdegat__port_506482','CAR_INTERIEUR_coffre__port_503412','FirstUploadExperveo_rouetranche__port_505677','UploadPhotoImmatBest_class_1__port_505051','CAR_INTERIEUR_avant_vue-arriere_class_2__port_506532','CAR_EXTERIEUR_angle_avant_gauche__port_504098','CAR_EXTERIEUR_face_avant_axe_gauche__port_504236','CAR_INTERIEUR_avant_vue_droite_habitacle_class_1__port_506540','CAR_EXTERIEUR_cote_gauche_axe_avant__port_504233','CAR_EXTERIEUR_roue_de_secour__port_503763','CAR_EXTERIEUR_Angle_arriere_gauche_axe_gauche__port_504199','CAR_EXTERIEUR_cote_gauche__port_504017','CAR_INTERIEUR_avant_volant_class_1__port_506503','CAR_INTERIEUR_avant_volant_class_2__port_506504','CAR_EXTERIEUR_angle_avant_gauche_axe_gauche__port_504234'); 51 dict_keys(['cartegrise_orientation__port_505064', 'car_exterieur_angle_arriere_droit_axe_arriere__port_504217', 'car_exterieur_angle_arriere_droit_axe_droit__port_504215', 'car_exterieur_angle_arriere_droit__port_504160', 'car_exterieur_angle_arriere_gauche_axe_arriere__port_504201', 'car_exterieur_angle_arriere_gauche_axe_gauche__port_504199', 'car_exterieur_angle_arriere_gauche__port_504170', 'car_exterieur_angle_avant_droit_axe_arriere__port_504226', 'car_exterieur_angle_avant_droit_axe_droit__port_504225', 'car_exterieur_angle_avant_droit__port_504087', 'car_exterieur_angle_avant_gauche_axe_avant__port_504235', 'car_exterieur_angle_avant_gauche_axe_gauche__port_504234', 'car_exterieur_angle_avant_gauche__port_504098', 'car_exterieur_arriere__port_504184', 'car_exterieur_avant__port_504146', 'car_exterieur_cote_droite__port_504108', 'car_exterieur_cote_droit_axe_arriere__port_504214', 'car_exterieur_cote_droit_axe_avant__port_504465', 'car_exterieur_cote_gauche_axe_arriere__port_504198', 'car_exterieur_cote_gauche_axe_avant__port_504233', 'car_exterieur_cote_gauche__port_504017', 'car_exterieur_face_arriere_axe_droit__port_504218', 'car_exterieur_face_arriere_axe_gauche__port_504202', 'car_exterieur_face_avant_axe_droit__port_504451', 'car_exterieur_face_avant_axe_gauche__port_504236', 'car_exterieur_moteur__port_503704', 'car_exterieur_roue_de_secour__port_503763', 'car_exterieur_roue__port_503398', 'car_interieur_avant_volant_class_1__port_506503', 'car_interieur_avant_volant_class_2__port_506504', 'car_interieur_avant_volant_class_6_boutonrond__port_506562', 'car_interieur_avant_volant_class_6_class_2__port_506563', 'car_interieur_avant_volant_class_6_ecrangrosplan__port_506564', 'car_interieur_avant_volant_class_6_levierdevitesse__port_506565', 'car_interieur_avant_vue-arriere_class_1__port_506531', 'car_interieur_avant_vue-arriere_class_2__port_506532', 'car_interieur_avant_vue_droite_habitacle_class_1__port_506540', 'car_interieur_avant_vue_gauche_habitacle_class_1__port_506494', 'car_interieur_coffre__port_503412', 'car_interieur_siege_arriere_class_1__port_506551', 'firstuploadexperveo_carrosseriegrosplan_carrosserie__port_506483', 'firstuploadexperveo_carrosseriegrosplan_class_6__port_506487', 'firstuploadexperveo_carrosseriegrosplan_morceauderoue__port_506484', 'firstuploadexperveo_carrosseriegrosplan_moteurgrosplanetdegat__port_506482', 'firstuploadexperveo_carrosseriegrosplan_siegegrosplan__port_506485', 'firstuploadexperveo_carrosseriegrosplan_vindanslamoquette__port_506486', 'firstuploadexperveo_rouetranche__port_505677', 'firstuploadexperveo_vignette__port_505674', 'firstuploadexperveo_vin__port_505675', 'interieur_compteur_kilometrique__port_503644', 'uploadphotoimmatbest_class_1__port_505051']) select photo_hashtag_type from MTRDatou.classification_theme where id = 358 thcl : 358 photo_hashtag_type : 337 SELECT phi.hashtag_id , phi.photo_id FROM MTRBack.photo_hashtag_ids phi, MTRUser.mtr_portfolio_photos mp where phi.type = 337 and phi.photo_id = mp.mtr_photo_id and mp.mtr_portfolio_id =510365; {510365: [(917973295, 1), (917973297, 1), (917973302, 1), (917973293, 7), (917973296, 11), (917973300, 11), (917973286, 13), (917973289, 13), (917973301, 24), (917973285, 29), (917973290, 29), (917973299, 29), (917973304, 35), (917973287, 36), (917973298, 36), (917973305, 36), (917973292, 37), (917973291, 41), (917973303, 41), (917973294, 42), (917973288, 46)]} ############################### TEST rotate ################################ test rotate only Inside batchDatouExec : verbose : True ##### chargement datou SELECT name, created_at,limit_max FROM MTRDatou.mtr_datou WHERE id=230 SELECT mtd.id, mtdt.`type`, mtd.`param`, mtd.param_json, mtdt.nb_input, mtdt.nb_output, mtdt.prod, mtdt.is_local, mtdt.is_datou_depend, mtdt.is_photo_id_local FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_step_types mtdt WHERE mtdt.`id`=mtd.`type` AND mtd.mtd_id=230 SELECT mtd.id, mtd.mtd_id, mdsdt.id, mdsdt.name, mdsdt.description, msid.output_or_input, msid.data_order_id, mdsdt.type FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_steptype_io_datatypes msid, MTRDatou.mtr_datou_step_data_types mdsdt WHERE mtd.`type`=msid.`mtr_datou_step_type` AND mtd.mtd_id= 230 AND msid.data_type=mdsdt.id SELECT mts_id_output, id_output, mts_id_input, id_input FROM MTRDatou.mtr_datou_step_by_step WHERE mtd_id=230 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! no param json to modify List Step Type Loaded in datou : rotate list_input_json : [] ##### fin chargement datou ##### chargement data ##### Call load_data_input : nb_thread : 5 origin SELECT photo_id, url FROM MTRBack.photos ph WHERE photo_id IN (917849322) Found this number of photos: 1 ##### Call download_photos : nb_thread : 5 begin to download photo : 917849322 download finish for photo 917849322 we have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB ##### After download_photos length of list_filenames : 1 ; length of list_pids : 1 ; length of list_args : 1 ##### After load_data_input time to download the photos : 0.17310094833374023 #### fin chargement data Blocking on flush ? No conitnuing About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : True number of steps : 1 step1:rotate Sun Mar 9 06:42:54 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec After prepare type args : Here we display some param of map_info ! map_filenames : {'temp/1741498974_588358_917849322_2bd260e91e91df8378dde8bb8b8c4548.jpg': 917849322} map_photo_id_path_extension : {917849322: {'path': 'temp/1741498974_588358_917849322_2bd260e91e91df8378dde8bb8b8c4548.jpg', 'extension': 'jpg'}} map_subphoto_mainphoto : {} Beginning of datou_step_rotate ! We are in a linear step without datou_depend ! rotate photos of 90,180,270 degres batch 1 select photo_id, hashtag_id, `type`, x0, x1, y0, y1, score, id from MTRPhoto.crop_hashtag_ids where photo_id in ( 917849322) and `type` in (0) Loaded 0 chid ids of type : 0 SELECT crop_hashtag_id, points FROM MTRPhoto.crop_polygon_points WHERE crop_hashtag_id in () map_chi : {} photo_id in download_rotate_and_save : 917849322 list_chi_loc : 0 Use all angle ! Rotation of photo 917849322 of 90 degree temp/1741498974_588358_917849322_2bd260e91e91df8378dde8bb8b8c4548.jpg [] 90 remove_crop_border : False version de PIL : 9.5.0 Needs to change image size ! [[ 6.123234e-17 1.000000e+00] [-1.000000e+00 6.123234e-17]] 90 [[ 6.123234e-17 1.000000e+00] [-1.000000e+00 6.123234e-17]] shrink_image : False image_rotate : image_path : temp/1741498974_588358_917849322_2bd260e91e91df8378dde8bb8b8c4548.jpg path_name_rotate : temp/1741498974_588358_917849322_2bd260e91e91df8378dde8bb8b8c454890.jpg image_rotate.mode : RGB Rotation of photo 917849322 of 180 degree temp/1741498974_588358_917849322_2bd260e91e91df8378dde8bb8b8c4548.jpg [] 180 remove_crop_border : False version de PIL : 9.5.0 Needs to change image size ! [[-1.0000000e+00 1.2246468e-16] [-1.2246468e-16 -1.0000000e+00]] 180 [[-1.0000000e+00 1.2246468e-16] [-1.2246468e-16 -1.0000000e+00]] shrink_image : False image_rotate : image_path : temp/1741498974_588358_917849322_2bd260e91e91df8378dde8bb8b8c4548.jpg path_name_rotate : temp/1741498974_588358_917849322_2bd260e91e91df8378dde8bb8b8c4548180.jpg image_rotate.mode : RGB Rotation of photo 917849322 of 270 degree temp/1741498974_588358_917849322_2bd260e91e91df8378dde8bb8b8c4548.jpg [] 270 remove_crop_border : False version de PIL : 9.5.0 Needs to change image size ! [[-1.8369702e-16 -1.0000000e+00] [ 1.0000000e+00 -1.8369702e-16]] 270 [[-1.8369702e-16 -1.0000000e+00] [ 1.0000000e+00 -1.8369702e-16]] shrink_image : False image_rotate : image_path : temp/1741498974_588358_917849322_2bd260e91e91df8378dde8bb8b8c4548.jpg path_name_rotate : temp/1741498974_588358_917849322_2bd260e91e91df8378dde8bb8b8c4548270.jpg image_rotate.mode : RGB About to upload 3 photos upload in portfolio : 551782 init cache_photo without model_param we have 3 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741498974_588358 we have uploaded 3 photos in the portfolio 551782 time of upload the photos Elapsed time : 1.2800462245941162 map_filename_photo_id : 3 map_filename_photo_id : {'temp/1741498974_588358_917849322_2bd260e91e91df8378dde8bb8b8c454890.jpg': 1342189038, 'temp/1741498974_588358_917849322_2bd260e91e91df8378dde8bb8b8c4548180.jpg': 1342189039, 'temp/1741498974_588358_917849322_2bd260e91e91df8378dde8bb8b8c4548270.jpg': 1342189040} Len new_chis : 3 Len list_new_chi_with_photo_id : 0 of type : 0 list_new_chi_with_photo_id : [] After datou_step_exec type output : time spend for datou_step_exec : 1.5219039916992188 time spend to save output : 6.914138793945312e-05 total time spend for step 1 : 1.5219731330871582 caffe_path_current : About to save ! 1 Inside saveOutput : final : True verbose : True saveOutput not yet implemented for datou_step.type : rotate we use saveGeneral [917849322] map_info['map_portfolio_photo'] : {} final : True mtd_id 230 list_pids : [917849322] Looping around the photos to save general results len do output : 3 /1342189038Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342189039Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342189040Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('230', None, None, None, None, None, None, None, None) ('230', None, '917849322', None, None, None, None, None, None) begin to insert list_values into mtr_datou_result : length of list_values in save_final : 10 insert ignore into MTRPhoto.mtr_datou_result (mtd_id, mtr_portfolio_id,mtr_photo_id,result,result_long,result_double,hashtag_id,proba, mtr_current_id) values (%s,%s,%s,%s,%s,%s,%s,%s,%s) on duplicate key update mtr_portfolio_id = mtr_portfolio_id list_values : [('230', None, '1342189038', 'None', None, None, None, None, None), ('230', None, '1342189039', 'None', None, None, None, None, None), ('230', None, '1342189040', 'None', None, None, None, None, None), ('230', None, '917849322', None, None, None, None, None, None)] time used for this insertion : 0.01220703125 save_final save missing photos in datou_result : After save, about to update current ! datou_cur_ids : [] len(datou.list_steps) : 1 output : {1342189038: ['917849322', 'temp/1741498974_588358_917849322_2bd260e91e91df8378dde8bb8b8c454890.jpg', []], 1342189039: ['917849322', 'temp/1741498974_588358_917849322_2bd260e91e91df8378dde8bb8b8c4548180.jpg', []], 1342189040: ['917849322', 'temp/1741498974_588358_917849322_2bd260e91e91df8378dde8bb8b8c4548270.jpg', []]} test rotate only is a success ! test rotate conditionnel Inside batchDatouExec : verbose : True ##### chargement datou SELECT name, created_at,limit_max FROM MTRDatou.mtr_datou WHERE id=233 SELECT mtd.id, mtdt.`type`, mtd.`param`, mtd.param_json, mtdt.nb_input, mtdt.nb_output, mtdt.prod, mtdt.is_local, mtdt.is_datou_depend, mtdt.is_photo_id_local FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_step_types mtdt WHERE mtdt.`id`=mtd.`type` AND mtd.mtd_id=233 SELECT mtd.id, mtd.mtd_id, mdsdt.id, mdsdt.name, mdsdt.description, msid.output_or_input, msid.data_order_id, mdsdt.type FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_steptype_io_datatypes msid, MTRDatou.mtr_datou_step_data_types mdsdt WHERE mtd.`type`=msid.`mtr_datou_step_type` AND mtd.mtd_id= 233 AND msid.data_type=mdsdt.id SELECT mts_id_output, id_output, mts_id_input, id_input FROM MTRDatou.mtr_datou_step_by_step WHERE mtd_id=233 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! DataTypes for each output/input checked ! no param json to modify List Step Type Loaded in datou : thcl, argmax, rotate list_input_json : [] ##### fin chargement datou ##### chargement data ##### Call load_data_input : nb_thread : 5 origin SELECT photo_id, url FROM MTRBack.photos ph WHERE photo_id IN (917849322) Found this number of photos: 1 ##### Call download_photos : nb_thread : 5 begin to download photo : 917849322 download finish for photo 917849322 we have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB ##### After download_photos length of list_filenames : 1 ; length of list_pids : 1 ; length of list_args : 1 ##### After load_data_input time to download the photos : 0.16278624534606934 #### fin chargement data Blocking on flush ? No conitnuing About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : True number of steps : 3 step1:thcl Sun Mar 9 06:42:56 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec After prepare type args : Here we display some param of map_info ! map_filenames : {'temp/1741498975_588358_917849322_2bd260e91e91df8378dde8bb8b8c4548.jpg': 917849322} map_photo_id_path_extension : {917849322: {'path': 'temp/1741498975_588358_917849322_2bd260e91e91df8378dde8bb8b8c4548.jpg', 'extension': 'jpg'}} map_subphoto_mainphoto : {} Beginning of datou step Thcl ! multi_thcl or not :False multi_thcl_cond or not :False dic_thcl : {'500': 1} we are using the classfication for only one thcl 500 In convert_file_to_np l 337 : 1 l343 1 l357 after caffe.io.load_image dimension du image : (3, (2448, 3264, 3)) dimension displayed ! time to import caffe and check if the image exist : 0.00023055076599121094 time to convert the images to numpy array : 2.214833974838257 total time to convert the images to numpy array : 2.2155613899230957 list photo_ids error: [] list photo_ids correct : [917849322] number of photos to traite : 1 try to delete the photos incorrect in DB tagging for thcl : 500 To do loadFromThcl(), then load ParamDescType : thcl500 get_desc_type_from_thcl : type of cat SELECT id, mtr_user_id, name, pb_hashtag_id, hashtag_id_list, button_legend_list, portfolio_id_lists, photo_hashtag_type, photo_desc_type, svm_limit, limit_tagging, is_public, live, created_at, updated_at, type_classification FROM MTRDatou.classification_theme WHERE `id` IN (500) thcls : [{'id': 500, 'mtr_user_id': 31, 'name': 'orientation_carte_grise_all_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'carteGrisesVerticales__port_549774,cartegrise_90deg__port_550987,cartesGrisesEnvers__port_549765,portfolio_270deg__port_550988', 'svm_portfolios_learning': '549774,550987,549765,550988', 'photo_hashtag_type': 507, 'photo_desc_type': 3517, 'type_classification': 'caffe', 'hashtag_id_list': '0,0,0,0'}] thcl {'id': 500, 'mtr_user_id': 31, 'name': 'orientation_carte_grise_all_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'carteGrisesVerticales__port_549774,cartegrise_90deg__port_550987,cartesGrisesEnvers__port_549765,portfolio_270deg__port_550988', 'svm_portfolios_learning': '549774,550987,549765,550988', 'photo_hashtag_type': 507, 'photo_desc_type': 3517, 'type_classification': 'caffe', 'hashtag_id_list': '0,0,0,0'} Update svm_hashtag_type_desc : 3517 SELECT * FROM MTRDatou.photo_desc_type_params WHERE id in (3517) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (3517, 'orientation_carte_grise_all_2', 16384, 25088, 'orientation_carte_grise_all_2', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2018, 4, 18, 20, 4, 34), datetime.datetime(2018, 4, 18, 20, 4, 34)) To loadFromThcl() : net_3517 begin to check gpu status inside check gpu memory havn't enough memory gpu , need / 2500 l 3632 free memory gpu now : 21 wait 20 seconds l 3637 free memory gpu now : 21 max_wait_temp : 1 max_wait : 0 SELECT * FROM MTRDatou.photo_desc_type_params WHERE id in (3517) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (3517, 'orientation_carte_grise_all_2', 16384, 25088, 'orientation_carte_grise_all_2', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2018, 4, 18, 20, 4, 34), datetime.datetime(2018, 4, 18, 20, 4, 34)) param : , param.caffemodel : orientation_carte_grise_all_2 None mean_file_type : mean_file_path : prototxt_file_path : model : orientation_carte_grise_all_2 Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : orientation_carte_grise_all_2 model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : [] local folder : /data/models_weight/orientation_carte_grise_all_2 /data/models_weight/orientation_carte_grise_all_2/caffemodel size_local : 537110520 size in s3 : 537110520 create time local : 2021-08-09 05:29:00 create time in s3 : 2021-08-06 20:07:17 caffemodel already exist and didn't need to update /data/models_weight/orientation_carte_grise_all_2/deploy_conv_normal.prototxt size_local : 4626 size in s3 : 4626 create time local : 2021-08-09 05:29:00 create time in s3 : 2021-08-06 20:07:16 deploy_conv_normal.prototxt already exist and didn't need to update /data/models_weight/orientation_carte_grise_all_2/deploy_fc.prototxt size_local : 1130 size in s3 : 1130 create time local : 2021-08-09 05:29:00 create time in s3 : 2021-08-06 20:07:16 deploy_fc.prototxt already exist and didn't need to update /data/models_weight/orientation_carte_grise_all_2/deploy.prototxt size_local : 5653 size in s3 : 5653 create time local : 2021-08-09 05:29:00 create time in s3 : 2021-08-06 20:07:16 deploy.prototxt already exist and didn't need to update /data/models_weight/orientation_carte_grise_all_2/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:29:00 create time in s3 : 2021-08-06 20:07:31 mean.npy already exist and didn't need to update /data/models_weight/orientation_carte_grise_all_2/synset_words.txt size_local : 159 size in s3 : 159 create time local : 2021-08-09 05:29:00 create time in s3 : 2021-08-06 20:07:16 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/orientation_carte_grise_all_2/deploy.prototxt caffemodel_filename : /data/models_weight/orientation_carte_grise_all_2/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory havn't enough memory gpu , need / 2500 l 3632 free memory gpu now : 21 wait 20 seconds WARNING: Logging before InitGoogleLogging() is written to STDERR F0309 06:43:45.779811 588358 syncedmem.cpp:78] Check failed: error == cudaSuccess (2 vs. 0) out of memory *** Check failure stack trace: *** Command terminated by signal 6 60.56user 38.44system 8:21.58elapsed 19%CPU (0avgtext+0avgdata 5704984maxresident)k 5328480inputs+27624outputs (6091major+4583988minor)pagefaults 0swaps