python /home/admin/mtr/script_for_cron.py -j python_test3 -m 12 -a ' --short_python3 -v ' -s python_test3 -M 0 -S 0 -U 100,100,120 import MySQLdb succeeded Import error (python version) python version = 3 warning , we can't find thcl infos in json_data warning , we can't find pdt infos in json_data list_job_run_as_list : ['mask_detection', 'datou', 'CacheModelData_queries', 'CachePhotoData_queries', 'test_fork', 'prepare_maskdata', 'portfolio_queries', 'sla_mensuel'] python version used : 3 liste_fichiers : [('tests/mask_test', True, 'Test mask-detection ', 'mask_detection'), ('tests/datou_test', True, 'Datou All Test', 'datou', 'all'), ('mtr/database_queries/CacheModelData_queries', True, 'Test Cache Model Data', 'CacheModelData_queries'), ('tests/cache_photo_data_test', True, 'Test local_cache_photo ', 'CachePhotoData_queries'), ('mtr/mask_rcnn/prepare_maskdata', True, 'test prepare mask data', 'prepare_maskdata', 'all'), ('mtr/database_queries/portfolio_queries', True, 'test portfolio queries', 'portfolio_queries'), ('prod/memo/memo', True, 'SLA Mensuel', 'sla_mensuel', 'all')] #&_# BEGIN OF TEST : tests/mask_test #&_# /home/admin/workarea/git/Velours/python/tests/mask_test.py Test mask-detection python version used : 3 ############################### TEST memory used ################################ free memory at begining : begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 4862 run mask_detect Inside batchDatouExec : verbose : False # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! List Step Type Loaded in datou : mask_detect list_input_json : [] origin BFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 1 ; length of list_pids : 1 ; length of list_args : 1 time to download the photos : 0.36889028549194336 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : False number of steps : 1 step1:mask_detect Fri Feb 28 03:35:27 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 4862 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 /home/admin/workarea/git/Velours/python/tests/python_tests.py:11: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses import imp 2025-02-28 03:35:30.951462: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-02-28 03:35:30.987144: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-02-28 03:35:30.989780: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f5ef8000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-02-28 03:35:30.989861: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-02-28 03:35:30.995340: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-02-28 03:35:31.226034: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x23d59720 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-02-28 03:35:31.226090: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-02-28 03:35:31.227605: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-28 03:35:31.228059: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-28 03:35:31.232416: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-28 03:35:31.241949: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-28 03:35:31.242415: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-28 03:35:31.262998: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-28 03:35:31.266335: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-28 03:35:31.298348: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-28 03:35:31.299874: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-28 03:35:31.300275: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-28 03:35:31.301049: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-28 03:35:31.301070: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-28 03:35:31.301082: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-28 03:35:31.302566: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 4410 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl454 thcls : [{'id': 454, 'mtr_user_id': 31, 'name': 'mask_coco_origin', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'backgroud,person,bicycle,car,motorcycle,airplane,bus,train,truck,boat,trafficlight,firehydrant,stopsign,parkingmeter,bench,bird,cat,dog,horse,sheep,cow,elephant,bear,zebra,giraffe,backpack,umbrella,handbag,tie,suitcase,frisbee,skis,snowboard,sportsball,kite,baseballbat,baseballglove,skateboard,surfboard,tennisracket,bottle,wineglass,cup,fork,knife,spoon,bowl,banana,apple,sandwich,orange,broccoli,carrot,hotdog,pizza,donut,cake,chair,couch,pottedplant,bed,diningtable,toilet,tv,laptop,mouse,remote,keyboard,cellphone,microwave,oven,toaster,sink,refrigerator,book,clock,vase,scissors,teddybear,hairdrier,toothbrush', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 445, 'photo_desc_type': 3473, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 454, 'mtr_user_id': 31, 'name': 'mask_coco_origin', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'backgroud,person,bicycle,car,motorcycle,airplane,bus,train,truck,boat,trafficlight,firehydrant,stopsign,parkingmeter,bench,bird,cat,dog,horse,sheep,cow,elephant,bear,zebra,giraffe,backpack,umbrella,handbag,tie,suitcase,frisbee,skis,snowboard,sportsball,kite,baseballbat,baseballglove,skateboard,surfboard,tennisracket,bottle,wineglass,cup,fork,knife,spoon,bowl,banana,apple,sandwich,orange,broccoli,carrot,hotdog,pizza,donut,cake,chair,couch,pottedplant,bed,diningtable,toilet,tv,laptop,mouse,remote,keyboard,cellphone,microwave,oven,toaster,sink,refrigerator,book,clock,vase,scissors,teddybear,hairdrier,toothbrush', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 445, 'photo_desc_type': 3473, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 3473 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (3473, 'mask_coco_origin', 16384, 25088, 'mask_coco_origin', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2018, 3, 19, 10, 42, 21), datetime.datetime(2018, 3, 19, 10, 42, 21)) {'thcl': {'id': 454, 'mtr_user_id': 31, 'name': 'mask_coco_origin', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'backgroud,person,bicycle,car,motorcycle,airplane,bus,train,truck,boat,trafficlight,firehydrant,stopsign,parkingmeter,bench,bird,cat,dog,horse,sheep,cow,elephant,bear,zebra,giraffe,backpack,umbrella,handbag,tie,suitcase,frisbee,skis,snowboard,sportsball,kite,baseballbat,baseballglove,skateboard,surfboard,tennisracket,bottle,wineglass,cup,fork,knife,spoon,bowl,banana,apple,sandwich,orange,broccoli,carrot,hotdog,pizza,donut,cake,chair,couch,pottedplant,bed,diningtable,toilet,tv,laptop,mouse,remote,keyboard,cellphone,microwave,oven,toaster,sink,refrigerator,book,clock,vase,scissors,teddybear,hairdrier,toothbrush', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 445, 'photo_desc_type': 3473, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['backgroud', 'person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus', 'train', 'truck', 'boat', 'trafficlight', 'firehydrant', 'stopsign', 'parkingmeter', 'bench', 'bird', 'cat', 'dog', 'horse', 'sheep', 'cow', 'elephant', 'bear', 'zebra', 'giraffe', 'backpack', 'umbrella', 'handbag', 'tie', 'suitcase', 'frisbee', 'skis', 'snowboard', 'sportsball', 'kite', 'baseballbat', 'baseballglove', 'skateboard', 'surfboard', 'tennisracket', 'bottle', 'wineglass', 'cup', 'fork', 'knife', 'spoon', 'bowl', 'banana', 'apple', 'sandwich', 'orange', 'broccoli', 'carrot', 'hotdog', 'pizza', 'donut', 'cake', 'chair', 'couch', 'pottedplant', 'bed', 'diningtable', 'toilet', 'tv', 'laptop', 'mouse', 'remote', 'keyboard', 'cellphone', 'microwave', 'oven', 'toaster', 'sink', 'refrigerator', 'book', 'clock', 'vase', 'scissors', 'teddybear', 'hairdrier', 'toothbrush'], 'list_hashtags_csv': 'backgroud,person,bicycle,car,motorcycle,airplane,bus,train,truck,boat,trafficlight,firehydrant,stopsign,parkingmeter,bench,bird,cat,dog,horse,sheep,cow,elephant,bear,zebra,giraffe,backpack,umbrella,handbag,tie,suitcase,frisbee,skis,snowboard,sportsball,kite,baseballbat,baseballglove,skateboard,surfboard,tennisracket,bottle,wineglass,cup,fork,knife,spoon,bowl,banana,apple,sandwich,orange,broccoli,carrot,hotdog,pizza,donut,cake,chair,couch,pottedplant,bed,diningtable,toilet,tv,laptop,mouse,remote,keyboard,cellphone,microwave,oven,toaster,sink,refrigerator,book,clock,vase,scissors,teddybear,hairdrier,toothbrush', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 445, 'svm_hashtag_type_desc': 3473, 'photo_desc_type': 3473, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['backgroud', 'person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus', 'train', 'truck', 'boat', 'trafficlight', 'firehydrant', 'stopsign', 'parkingmeter', 'bench', 'bird', 'cat', 'dog', 'horse', 'sheep', 'cow', 'elephant', 'bear', 'zebra', 'giraffe', 'backpack', 'umbrella', 'handbag', 'tie', 'suitcase', 'frisbee', 'skis', 'snowboard', 'sportsball', 'kite', 'baseballbat', 'baseballglove', 'skateboard', 'surfboard', 'tennisracket', 'bottle', 'wineglass', 'cup', 'fork', 'knife', 'spoon', 'bowl', 'banana', 'apple', 'sandwich', 'orange', 'broccoli', 'carrot', 'hotdog', 'pizza', 'donut', 'cake', 'chair', 'couch', 'pottedplant', 'bed', 'diningtable', 'toilet', 'tv', 'laptop', 'mouse', 'remote', 'keyboard', 'cellphone', 'microwave', 'oven', 'toaster', 'sink', 'refrigerator', 'book', 'clock', 'vase', 'scissors', 'teddybear', 'hairdrier', 'toothbrush'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME mask_coco_origin NUM_CLASSES 81 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 2025-02-28 03:35:31.904753: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-28 03:35:31.904869: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-28 03:35:31.904899: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-28 03:35:31.904917: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-28 03:35:31.904932: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-28 03:35:31.904946: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-28 03:35:31.904972: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-28 03:35:31.904988: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-28 03:35:31.905998: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-28 03:35:31.907141: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-28 03:35:31.907184: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-28 03:35:31.907200: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-28 03:35:31.907214: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-28 03:35:31.907228: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-28 03:35:31.907242: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-28 03:35:31.907256: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-28 03:35:31.907271: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-28 03:35:31.908229: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-28 03:35:31.908264: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-28 03:35:31.908273: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-28 03:35:31.908280: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-28 03:35:31.909261: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 4410 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : mask_coco_origin model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-02-28 03:35:40.312504: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-28 03:35:40.530020: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-28 03:35:42.369794: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.81G (4089315328 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory local folder : /data/models_weight/mask_coco_origin /data/models_weight/mask_coco_origin/mask_model.h5 size_local : 257557808 size in s3 : 257557808 create time local : 2021-08-09 05:27:17 create time in s3 : 2021-08-06 19:45:17 mask_model.h5 already exist and didn't need to update list_images length : 1 NEW PHOTO Processing 1 images image shape: (480, 640, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 89) min: 0.00000 max: 640.00000 nb d'objets trouves : 5 Detection mask done ! Trying to reset tf kernel 2762506 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 159 tf kernel not reseted sub process len(results) : 1 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 1 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 4862 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl454 Catched exception ! Connect or reconnect ! thcls : [{'id': 454, 'mtr_user_id': 31, 'name': 'mask_coco_origin', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'backgroud,person,bicycle,car,motorcycle,airplane,bus,train,truck,boat,trafficlight,firehydrant,stopsign,parkingmeter,bench,bird,cat,dog,horse,sheep,cow,elephant,bear,zebra,giraffe,backpack,umbrella,handbag,tie,suitcase,frisbee,skis,snowboard,sportsball,kite,baseballbat,baseballglove,skateboard,surfboard,tennisracket,bottle,wineglass,cup,fork,knife,spoon,bowl,banana,apple,sandwich,orange,broccoli,carrot,hotdog,pizza,donut,cake,chair,couch,pottedplant,bed,diningtable,toilet,tv,laptop,mouse,remote,keyboard,cellphone,microwave,oven,toaster,sink,refrigerator,book,clock,vase,scissors,teddybear,hairdrier,toothbrush', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 445, 'photo_desc_type': 3473, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 454, 'mtr_user_id': 31, 'name': 'mask_coco_origin', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'backgroud,person,bicycle,car,motorcycle,airplane,bus,train,truck,boat,trafficlight,firehydrant,stopsign,parkingmeter,bench,bird,cat,dog,horse,sheep,cow,elephant,bear,zebra,giraffe,backpack,umbrella,handbag,tie,suitcase,frisbee,skis,snowboard,sportsball,kite,baseballbat,baseballglove,skateboard,surfboard,tennisracket,bottle,wineglass,cup,fork,knife,spoon,bowl,banana,apple,sandwich,orange,broccoli,carrot,hotdog,pizza,donut,cake,chair,couch,pottedplant,bed,diningtable,toilet,tv,laptop,mouse,remote,keyboard,cellphone,microwave,oven,toaster,sink,refrigerator,book,clock,vase,scissors,teddybear,hairdrier,toothbrush', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 445, 'photo_desc_type': 3473, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 3473 ['backgroud', 'person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus', 'train', 'truck', 'boat', 'trafficlight', 'firehydrant', 'stopsign', 'parkingmeter', 'bench', 'bird', 'cat', 'dog', 'horse', 'sheep', 'cow', 'elephant', 'bear', 'zebra', 'giraffe', 'backpack', 'umbrella', 'handbag', 'tie', 'suitcase', 'frisbee', 'skis', 'snowboard', 'sportsball', 'kite', 'baseballbat', 'baseballglove', 'skateboard', 'surfboard', 'tennisracket', 'bottle', 'wineglass', 'cup', 'fork', 'knife', 'spoon', 'bowl', 'banana', 'apple', 'sandwich', 'orange', 'broccoli', 'carrot', 'hotdog', 'pizza', 'donut', 'cake', 'chair', 'couch', 'pottedplant', 'bed', 'diningtable', 'toilet', 'tv', 'laptop', 'mouse', 'remote', 'keyboard', 'cellphone', 'microwave', 'oven', 'toaster', 'sink', 'refrigerator', 'book', 'clock', 'vase', 'scissors', 'teddybear', 'hairdrier', 'toothbrush'] time for calcul the mask position with numpy : 0.0008540153503417969 nb_pixel_total : 15553 time to create 1 rle with old method : 0.019060611724853516 length of segment : 256 time for calcul the mask position with numpy : 0.0031042098999023438 nb_pixel_total : 145321 time to create 1 rle with old method : 0.16649770736694336 length of segment : 371 time for calcul the mask position with numpy : 0.0002856254577636719 nb_pixel_total : 14254 time to create 1 rle with old method : 0.01711249351501465 length of segment : 151 time for calcul the mask position with numpy : 0.00010609626770019531 nb_pixel_total : 5613 time to create 1 rle with old method : 0.0075664520263671875 length of segment : 48 time for calcul the mask position with numpy : 6.29425048828125e-05 nb_pixel_total : 1825 time to create 1 rle with old method : 0.002532958984375 length of segment : 39 time spent for convertir_results : 1.002573013305664 time spend for datou_step_exec : 20.0562686920166 time spend to save output : 3.886222839355469e-05 total time spend for step 1 : 20.056307554244995 caffe_path_current : About to save ! 1 Inside saveOutput : final : True verbose : False eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 3296 chid ids of type : 445 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++Number RLEs to save : 0 begin to insert list_values into mtr_datou_result : length of list_values in save_final : 1 time used for this insertion : 0.014894723892211914 save missing photos in datou_result : After save, about to update current ! datou_cur_ids : [] len(datou.list_steps) : 1 output : {'957285035': [[(957285035, 492601069, 445, 0, 186, 22, 282, 0.9954862, [(140, 26, 6), (135, 27, 15), (133, 28, 18), (131, 29, 22), (126, 30, 28), (10, 31, 1), (120, 31, 35), (8, 32, 13), (26, 32, 4), (115, 32, 41), (7, 33, 52), (109, 33, 48), (6, 34, 70), (103, 34, 55), (5, 35, 154), (4, 36, 155), (3, 37, 156), (3, 38, 156), (3, 39, 156), (2, 40, 157), (2, 41, 157), (2, 42, 157), (2, 43, 157), (2, 44, 157), (2, 45, 157), (1, 46, 158), (1, 47, 158), (1, 48, 158), (1, 49, 157), (1, 50, 157), (1, 51, 156), (1, 52, 156), (1, 53, 155), (1, 54, 154), (1, 55, 152), (1, 56, 149), (1, 57, 145), (1, 58, 141), (1, 59, 136), (1, 60, 133), (1, 61, 130), (1, 62, 127), (1, 63, 126), (1, 64, 124), (1, 65, 123), (1, 66, 121), (1, 67, 120), (1, 68, 118), (1, 69, 117), (1, 70, 116), (1, 71, 115), (1, 72, 114), (1, 73, 113), (1, 74, 112), (1, 75, 111), (1, 76, 110), (1, 77, 108), (1, 78, 108), (1, 79, 107), (1, 80, 106), (1, 81, 105), (2, 82, 104), (2, 83, 103), (2, 84, 103), (2, 85, 102), (2, 86, 102), (2, 87, 101), (2, 88, 100), (2, 89, 99), (2, 90, 99), (2, 91, 98), (2, 92, 97), (2, 93, 96), (2, 94, 95), (2, 95, 93), (2, 96, 91), (2, 97, 90), (2, 98, 89), (2, 99, 87), (2, 100, 86), (2, 101, 86), (2, 102, 85), (2, 103, 84), (2, 104, 83), (2, 105, 83), (2, 106, 82), (2, 107, 81), (2, 108, 80), (2, 109, 80), (2, 110, 79), (2, 111, 78), (2, 112, 77), (2, 113, 76), (1, 114, 76), (1, 115, 75), (1, 116, 74), (1, 117, 73), (1, 118, 72), (1, 119, 71), (1, 120, 71), (1, 121, 70), (1, 122, 69), (1, 123, 69), (1, 124, 68), (1, 125, 68), (1, 126, 67), (1, 127, 67), (1, 128, 66), (1, 129, 66), (1, 130, 66), (1, 131, 65), (1, 132, 65), (1, 133, 64), (1, 134, 63), (1, 135, 63), (1, 136, 62), (1, 137, 61), (1, 138, 60), (1, 139, 60), (1, 140, 59), (1, 141, 58), (1, 142, 58), (1, 143, 57), (1, 144, 56), (1, 145, 56), (1, 146, 55), (1, 147, 54), (1, 148, 54), (1, 149, 53), (1, 150, 52), (1, 151, 52), (1, 152, 51), (1, 153, 50), (1, 154, 49), (1, 155, 48), (1, 156, 47), (1, 157, 46), (1, 158, 45), (1, 159, 45), (1, 160, 44), (1, 161, 43), (1, 162, 42), (1, 163, 41), (1, 164, 41), (1, 165, 40), (1, 166, 40), (1, 167, 39), (1, 168, 38), (1, 169, 37), (1, 170, 36), (1, 171, 35), (1, 172, 34), (1, 173, 34), (1, 174, 33), (1, 175, 33), (1, 176, 32), (1, 177, 32), (1, 178, 32), (1, 179, 32), (1, 180, 31), (1, 181, 31), (1, 182, 31), (1, 183, 30), (1, 184, 30), (1, 185, 30), (1, 186, 29), (1, 187, 29), (1, 188, 29), (1, 189, 28), (1, 190, 28), (1, 191, 27), (1, 192, 27), (1, 193, 26), (1, 194, 26), (1, 195, 26), (1, 196, 26), (1, 197, 26), (1, 198, 26), (1, 199, 26), (1, 200, 25), (1, 201, 25), (1, 202, 25), (1, 203, 25), (1, 204, 25), (1, 205, 25), (1, 206, 25), (1, 207, 25), (1, 208, 25), (1, 209, 25), (1, 210, 25), (1, 211, 25), (1, 212, 25), (1, 213, 25), (1, 214, 25), (1, 215, 25), (1, 216, 25), (1, 217, 25), (1, 218, 25), (1, 219, 25), (1, 220, 24), (1, 221, 24), (1, 222, 24), (1, 223, 24), (1, 224, 24), (1, 225, 24), (1, 226, 25), (1, 227, 25), (1, 228, 25), (2, 229, 24), (2, 230, 24), (2, 231, 24), (2, 232, 23), (2, 233, 23), (2, 234, 23), (2, 235, 23), (2, 236, 23), (2, 237, 23), (2, 238, 23), (2, 239, 23), (2, 240, 23), (2, 241, 23), (2, 242, 23), (2, 243, 23), (2, 244, 23), (2, 245, 23), (2, 246, 23), (2, 247, 23), (2, 248, 23), (2, 249, 24), (2, 250, 24), (2, 251, 23), (2, 252, 23), (2, 253, 23), (2, 254, 23), (2, 255, 23), (2, 256, 23), (2, 257, 23), (2, 258, 23), (2, 259, 23), (2, 260, 23), (2, 261, 23), (3, 262, 22), (3, 263, 22), (3, 264, 22), (3, 265, 22), (4, 266, 21), (4, 267, 21), (5, 268, 20), (5, 269, 20), (6, 270, 19), (7, 271, 17), (8, 272, 16), (8, 273, 16), (9, 274, 13), (11, 275, 9), (15, 276, 2)], ['16,276,8,273,2,261,2,229,1,228,1,114,2,113,2,82,1,81,1,46,3,37,8,32,29,32,30,33,58,33,59,34,75,34,76,35,102,35,114,33,120,31,130,30,135,27,145,26,152,29,158,35,158,48,154,54,141,58,128,61,119,67,105,81,103,86,96,94,89,98,81,109,71,119,65,132,60,138,52,150,41,163,40,166,34,172,29,188,26,193,25,200,25,219,24,232,24,270,23,273']), (957285035, 492601069, 445, 29, 591, 24, 419, 0.9923496, [(315, 37, 25), (272, 38, 86), (253, 39, 130), (238, 40, 151), (199, 41, 196), (189, 42, 213), (180, 43, 238), (175, 44, 250), (172, 45, 257), (169, 46, 265), (166, 47, 274), (162, 48, 284), (159, 49, 294), (157, 50, 304), (155, 51, 310), (153, 52, 317), (151, 53, 323), (149, 54, 330), (148, 55, 334), (146, 56, 337), (144, 57, 341), (142, 58, 344), (140, 59, 347), (138, 60, 350), (136, 61, 353), (134, 62, 356), (132, 63, 358), (130, 64, 361), (128, 65, 364), (126, 66, 367), (124, 67, 370), (122, 68, 373), (120, 69, 376), (118, 70, 379), (117, 71, 381), (115, 72, 385), (114, 73, 387), (113, 74, 389), (112, 75, 391), (112, 76, 393), (111, 77, 395), (110, 78, 397), (109, 79, 399), (109, 80, 400), (108, 81, 402), (107, 82, 404), (107, 83, 404), (106, 84, 406), (105, 85, 408), (105, 86, 409), (104, 87, 410), (104, 88, 411), (103, 89, 413), (102, 90, 415), (101, 91, 417), (100, 92, 420), (98, 93, 423), (97, 94, 426), (96, 95, 428), (94, 96, 431), (93, 97, 433), (92, 98, 435), (91, 99, 437), (90, 100, 439), (89, 101, 441), (89, 102, 441), (89, 103, 442), (89, 104, 443), (89, 105, 444), (89, 106, 444), (89, 107, 445), (89, 108, 446), (89, 109, 447), (89, 110, 448), (89, 111, 449), (89, 112, 450), (89, 113, 451), (89, 114, 453), (89, 115, 454), (89, 116, 455), (88, 117, 456), (88, 118, 457), (87, 119, 459), (87, 120, 459), (86, 121, 461), (85, 122, 462), (85, 123, 463), (84, 124, 464), (84, 125, 465), (83, 126, 466), (82, 127, 468), (82, 128, 468), (81, 129, 470), (80, 130, 471), (78, 131, 473), (76, 132, 476), (75, 133, 477), (73, 134, 480), (71, 135, 482), (70, 136, 484), (68, 137, 486), (67, 138, 488), (65, 139, 490), (64, 140, 492), (63, 141, 493), (61, 142, 496), (60, 143, 497), (59, 144, 499), (58, 145, 501), (58, 146, 501), (57, 147, 503), (57, 148, 504), (56, 149, 506), (56, 150, 507), (56, 151, 507), (55, 152, 509), (55, 153, 510), (54, 154, 511), (54, 155, 512), (54, 156, 513), (53, 157, 514), (53, 158, 514), (52, 159, 515), (52, 160, 516), (51, 161, 517), (51, 162, 517), (51, 163, 517), (50, 164, 518), (50, 165, 518), (49, 166, 519), (49, 167, 520), (48, 168, 521), (48, 169, 521), (47, 170, 522), (47, 171, 522), (46, 172, 523), (46, 173, 523), (46, 174, 523), (45, 175, 524), (45, 176, 523), (44, 177, 524), (44, 178, 524), (44, 179, 524), (43, 180, 525), (43, 181, 525), (42, 182, 525), (42, 183, 525), (41, 184, 526), (41, 185, 526), (41, 186, 526), (40, 187, 526), (39, 188, 526), (39, 189, 525), (38, 190, 526), (38, 191, 525), (37, 192, 525), (37, 193, 523), (36, 194, 523), (36, 195, 523), (36, 196, 522), (35, 197, 522), (35, 198, 521), (34, 199, 521), (34, 200, 521), (34, 201, 520), (34, 202, 520), (34, 203, 520), (34, 204, 519), (33, 205, 520), (33, 206, 520), (33, 207, 519), (33, 208, 519), (33, 209, 519), (33, 210, 518), (33, 211, 518), (33, 212, 518), (33, 213, 517), (32, 214, 518), (32, 215, 517), (32, 216, 517), (32, 217, 516), (32, 218, 515), (32, 219, 514), (32, 220, 513), (32, 221, 512), (32, 222, 511), (32, 223, 510), (32, 224, 508), (32, 225, 507), (32, 226, 505), (32, 227, 504), (32, 228, 503), (32, 229, 502), (32, 230, 502), (32, 231, 501), (32, 232, 500), (32, 233, 499), (32, 234, 498), (32, 235, 497), (31, 236, 496), (31, 237, 495), (31, 238, 494), (31, 239, 493), (31, 240, 491), (31, 241, 490), (31, 242, 488), (31, 243, 487), (31, 244, 486), (31, 245, 485), (31, 246, 483), (31, 247, 482), (31, 248, 480), (31, 249, 479), (31, 250, 477), (31, 251, 475), (31, 252, 474), (31, 253, 472), (31, 254, 470), (31, 255, 468), (31, 256, 467), (31, 257, 465), (31, 258, 464), (31, 259, 463), (31, 260, 462), (31, 261, 461), (31, 262, 459), (31, 263, 458), (31, 264, 456), (31, 265, 455), (31, 266, 453), (31, 267, 451), (31, 268, 449), (31, 269, 448), (31, 270, 447), (31, 271, 445), (31, 272, 444), (31, 273, 443), (32, 274, 441), (32, 275, 440), (32, 276, 438), (32, 277, 437), (32, 278, 435), (32, 279, 434), (32, 280, 432), (33, 281, 429), (33, 282, 427), (33, 283, 426), (33, 284, 424), (33, 285, 423), (34, 286, 421), (34, 287, 420), (34, 288, 419), (35, 289, 416), (35, 290, 415), (35, 291, 414), (36, 292, 411), (36, 293, 410), (37, 294, 407), (37, 295, 406), (38, 296, 403), (38, 297, 401), (39, 298, 399), (39, 299, 397), (41, 300, 394), (42, 301, 392), (43, 302, 389), (44, 303, 387), (45, 304, 385), (46, 305, 382), (47, 306, 380), (47, 307, 378), (48, 308, 376), (49, 309, 373), (50, 310, 370), (51, 311, 368), (51, 312, 367), (52, 313, 365), (54, 314, 362), (55, 315, 360), (56, 316, 359), (58, 317, 356), (61, 318, 352), (64, 319, 349), (67, 320, 345), (70, 321, 341), (73, 322, 338), (76, 323, 334), (78, 324, 332), (80, 325, 329), (82, 326, 327), (84, 327, 324), (86, 328, 322), (88, 329, 320), (90, 330, 317), (93, 331, 314), (96, 332, 311), (99, 333, 307), (103, 334, 303), (106, 335, 299), (108, 336, 297), (111, 337, 294), (113, 338, 291), (115, 339, 289), (117, 340, 286), (119, 341, 283), (121, 342, 281), (123, 343, 278), (125, 344, 275), (127, 345, 272), (130, 346, 268), (132, 347, 266), (135, 348, 262), (138, 349, 258), (141, 350, 255), (143, 351, 252), (146, 352, 249), (148, 353, 246), (150, 354, 244), (151, 355, 242), (153, 356, 240), (154, 357, 239), (156, 358, 237), (159, 359, 233), (161, 360, 231), (163, 361, 229), (165, 362, 227), (167, 363, 224), (169, 364, 222), (171, 365, 220), (172, 366, 219), (173, 367, 218), (175, 368, 215), (176, 369, 214), (177, 370, 213), (178, 371, 212), (180, 372, 209), (183, 373, 206), (186, 374, 203), (188, 375, 200), (191, 376, 197), (194, 377, 193), (196, 378, 191), (199, 379, 188), (201, 380, 185), (203, 381, 183), (205, 382, 180), (207, 383, 178), (208, 384, 176), (210, 385, 174), (212, 386, 171), (213, 387, 169), (216, 388, 165), (218, 389, 162), (221, 390, 158), (225, 391, 153), (228, 392, 149), (232, 393, 144), (235, 394, 140), (238, 395, 136), (241, 396, 133), (245, 397, 128), (248, 398, 124), (252, 399, 119), (257, 400, 113), (263, 401, 105), (272, 402, 94), (283, 403, 82), (297, 404, 65), (306, 405, 53), (313, 406, 38), (321, 407, 23)], ['321,407,296,403,263,401,218,389,198,378,178,371,170,364,140,349,127,345,105,334,90,330,75,322,56,316,39,299,31,273,32,214,36,194,58,145,79,131,89,116,89,101,104,88,115,72,159,49,180,43,199,41,237,41,272,38,339,37,382,39,402,43,417,43,481,55,543,116,556,143,566,156,568,167,566,186,554,199,548,216,528,235,471,275,460,281,414,315,403,339,392,355,389,371,383,385,369,400,358,405']), (957285035, 492601069, 445, 485, 636, 23, 174, 0.97109216, [(540, 24, 21), (626, 24, 3), (531, 25, 49), (594, 25, 40), (527, 26, 107), (523, 27, 111), (520, 28, 114), (517, 29, 118), (516, 30, 119), (515, 31, 120), (513, 32, 122), (512, 33, 123), (510, 34, 125), (509, 35, 126), (507, 36, 128), (506, 37, 129), (504, 38, 131), (503, 39, 132), (501, 40, 134), (500, 41, 135), (499, 42, 136), (498, 43, 137), (497, 44, 138), (496, 45, 139), (496, 46, 139), (495, 47, 140), (495, 48, 140), (494, 49, 141), (493, 50, 142), (492, 51, 143), (491, 52, 144), (491, 53, 144), (490, 54, 145), (490, 55, 145), (490, 56, 145), (490, 57, 146), (490, 58, 146), (490, 59, 146), (491, 60, 145), (491, 61, 145), (491, 62, 145), (492, 63, 144), (493, 64, 143), (494, 65, 142), (495, 66, 141), (496, 67, 140), (497, 68, 138), (498, 69, 138), (499, 70, 137), (500, 71, 136), (501, 72, 135), (503, 73, 133), (503, 74, 133), (505, 75, 131), (506, 76, 130), (507, 77, 129), (508, 78, 128), (509, 79, 127), (510, 80, 126), (511, 81, 125), (512, 82, 124), (513, 83, 123), (514, 84, 122), (515, 85, 121), (516, 86, 120), (517, 87, 119), (518, 88, 118), (519, 89, 117), (521, 90, 115), (521, 91, 115), (522, 92, 114), (523, 93, 113), (524, 94, 112), (525, 95, 111), (526, 96, 110), (527, 97, 109), (529, 98, 107), (530, 99, 106), (532, 100, 104), (533, 101, 103), (534, 102, 102), (535, 103, 101), (536, 104, 100), (538, 105, 98), (540, 106, 96), (541, 107, 95), (543, 108, 93), (546, 109, 90), (548, 110, 88), (549, 111, 87), (551, 112, 84), (552, 113, 83), (553, 114, 82), (555, 115, 80), (556, 116, 79), (556, 117, 79), (557, 118, 78), (558, 119, 77), (559, 120, 76), (560, 121, 75), (560, 122, 75), (561, 123, 74), (561, 124, 74), (561, 125, 74), (562, 126, 73), (562, 127, 73), (563, 128, 72), (563, 129, 72), (564, 130, 70), (564, 131, 70), (565, 132, 69), (565, 133, 68), (565, 134, 68), (565, 135, 67), (566, 136, 65), (566, 137, 64), (566, 138, 64), (566, 139, 62), (566, 140, 61), (566, 141, 59), (566, 142, 57), (566, 143, 56), (566, 144, 55), (566, 145, 54), (567, 146, 53), (567, 147, 52), (567, 148, 51), (568, 149, 50), (568, 150, 49), (568, 151, 48), (568, 152, 47), (569, 153, 45), (569, 154, 44), (570, 155, 42), (570, 156, 42), (570, 157, 41), (571, 158, 39), (571, 159, 39), (572, 160, 37), (572, 161, 37), (573, 162, 35), (573, 163, 34), (573, 164, 34), (574, 165, 32), (575, 166, 30), (577, 167, 28), (579, 168, 25), (581, 169, 22), (584, 170, 19), (587, 171, 15), (591, 172, 8)], ['598,172,591,172,590,171,581,169,575,166,570,157,568,152,568,149,566,145,566,136,565,132,561,125,560,121,556,116,547,109,543,108,536,104,531,99,527,97,491,62,490,54,495,48,496,45,501,40,514,32,517,29,531,25,539,25,540,24,560,24,561,25,579,25,580,26,593,26,594,25,633,25,634,29,634,56,635,57,635,111,634,112,634,129,632,134,629,138,623,141,619,145,617,149,611,155,608,161,604,166']), (957285035, 492601069, 445, 280, 481, 2, 55, 0.82984585, [(292, 3, 128), (284, 4, 146), (282, 5, 151), (281, 6, 154), (281, 7, 156), (281, 8, 157), (281, 9, 158), (281, 10, 160), (281, 11, 162), (281, 12, 165), (281, 13, 167), (281, 14, 169), (281, 15, 171), (281, 16, 173), (281, 17, 174), (281, 18, 175), (281, 19, 177), (281, 20, 178), (281, 21, 179), (281, 22, 180), (281, 23, 181), (281, 24, 182), (281, 25, 183), (281, 26, 184), (281, 27, 185), (281, 28, 185), (281, 29, 185), (282, 30, 185), (283, 31, 27), (337, 31, 131), (371, 32, 97), (401, 33, 68), (409, 34, 61), (419, 35, 52), (424, 36, 48), (429, 37, 44), (432, 38, 41), (434, 39, 40), (436, 40, 39), (438, 41, 37), (441, 42, 35), (444, 43, 32), (448, 44, 29), (452, 45, 25), (454, 46, 23), (459, 47, 17), (463, 48, 12), (468, 49, 5)], ['472,49,468,49,467,48,459,47,458,46,454,46,451,44,448,44,447,43,444,43,440,41,438,41,428,36,424,36,423,35,419,35,418,34,409,34,408,33,401,33,400,32,371,32,370,31,337,31,336,30,283,31,281,29,281,6,284,4,291,4,292,3,419,3,420,4,429,4,430,5,432,5,436,7,441,11,445,12,453,16,456,19,457,19,465,27,465,29,472,37,476,44,476,46']), (957285035, 492601069, 445, 456, 547, 6, 45, 0.74058264, [(482, 8, 19), (463, 9, 4), (481, 9, 44), (457, 10, 12), (479, 10, 50), (457, 11, 13), (476, 11, 56), (457, 12, 15), (475, 12, 65), (457, 13, 84), (457, 14, 85), (457, 15, 89), (457, 16, 89), (458, 17, 88), (459, 18, 87), (460, 19, 86), (461, 20, 80), (464, 21, 71), (466, 22, 63), (467, 23, 59), (468, 24, 55), (469, 25, 52), (469, 26, 51), (470, 27, 48), (471, 28, 46), (471, 29, 44), (472, 30, 42), (473, 31, 39), (473, 32, 38), (474, 33, 36), (475, 34, 33), (475, 35, 32), (476, 36, 30), (476, 37, 29), (477, 38, 26), (478, 39, 23), (479, 40, 20), (480, 41, 17), (488, 42, 5)], ['492,42,488,42,487,41,480,41,476,37,475,34,473,32,469,25,465,21,461,20,457,16,457,10,466,9,470,12,474,13,476,11,480,10,482,8,500,8,501,9,524,9,525,10,528,10,532,12,539,12,542,15,545,15,545,19,535,20,534,21,529,21,525,23,523,23,513,30,512,30,504,37,496,41,493,41'])], 'temp/1740710127_2762130_957285035_a42482e51c93c8025d243dd179aee85b.jpg']} free memory after detection : begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 4862 ############################### TEST detect object ################################ run mask_detect Inside batchDatouExec : verbose : False # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! List Step Type Loaded in datou : mask_detect list_input_json : [] origin BFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 1 ; length of list_pids : 1 ; length of list_args : 1 time to download the photos : 0.1762244701385498 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : False number of steps : 1 step1:mask_detect Fri Feb 28 03:35:54 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 4862 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-02-28 03:35:57.050023: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-02-28 03:35:57.083282: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-02-28 03:35:57.085560: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f5efc000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-02-28 03:35:57.085647: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-02-28 03:35:57.089444: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-02-28 03:35:57.330634: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x23f9bea0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-02-28 03:35:57.330698: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-02-28 03:35:57.331921: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-28 03:35:57.332732: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-28 03:35:57.337670: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-28 03:35:57.340506: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-28 03:35:57.341727: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-28 03:35:57.345063: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-28 03:35:57.346166: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-28 03:35:57.353209: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-28 03:35:57.354677: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-28 03:35:57.354771: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-28 03:35:57.355519: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-28 03:35:57.355540: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-28 03:35:57.355551: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-28 03:35:57.356518: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 4410 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-02-28 03:35:57.461880: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-28 03:35:57.462021: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-28 03:35:57.462048: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-28 03:35:57.462069: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-28 03:35:57.462090: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-28 03:35:57.462110: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-28 03:35:57.462130: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-28 03:35:57.462179: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-28 03:35:57.463339: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-28 03:35:57.464500: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-28 03:35:57.464551: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-28 03:35:57.464571: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-28 03:35:57.464596: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-28 03:35:57.464616: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-28 03:35:57.464636: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-28 03:35:57.464655: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-28 03:35:57.464675: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-28 03:35:57.465805: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-28 03:35:57.465846: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-28 03:35:57.465857: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-28 03:35:57.465867: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-28 03:35:57.467061: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 4410 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (3473, 'mask_coco_origin', 16384, 25088, 'mask_coco_origin', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2018, 3, 19, 10, 42, 21), datetime.datetime(2018, 3, 19, 10, 42, 21)) {'thcl': {'id': 454, 'mtr_user_id': 31, 'name': 'mask_coco_origin', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'backgroud,person,bicycle,car,motorcycle,airplane,bus,train,truck,boat,trafficlight,firehydrant,stopsign,parkingmeter,bench,bird,cat,dog,horse,sheep,cow,elephant,bear,zebra,giraffe,backpack,umbrella,handbag,tie,suitcase,frisbee,skis,snowboard,sportsball,kite,baseballbat,baseballglove,skateboard,surfboard,tennisracket,bottle,wineglass,cup,fork,knife,spoon,bowl,banana,apple,sandwich,orange,broccoli,carrot,hotdog,pizza,donut,cake,chair,couch,pottedplant,bed,diningtable,toilet,tv,laptop,mouse,remote,keyboard,cellphone,microwave,oven,toaster,sink,refrigerator,book,clock,vase,scissors,teddybear,hairdrier,toothbrush', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 445, 'photo_desc_type': 3473, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['backgroud', 'person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus', 'train', 'truck', 'boat', 'trafficlight', 'firehydrant', 'stopsign', 'parkingmeter', 'bench', 'bird', 'cat', 'dog', 'horse', 'sheep', 'cow', 'elephant', 'bear', 'zebra', 'giraffe', 'backpack', 'umbrella', 'handbag', 'tie', 'suitcase', 'frisbee', 'skis', 'snowboard', 'sportsball', 'kite', 'baseballbat', 'baseballglove', 'skateboard', 'surfboard', 'tennisracket', 'bottle', 'wineglass', 'cup', 'fork', 'knife', 'spoon', 'bowl', 'banana', 'apple', 'sandwich', 'orange', 'broccoli', 'carrot', 'hotdog', 'pizza', 'donut', 'cake', 'chair', 'couch', 'pottedplant', 'bed', 'diningtable', 'toilet', 'tv', 'laptop', 'mouse', 'remote', 'keyboard', 'cellphone', 'microwave', 'oven', 'toaster', 'sink', 'refrigerator', 'book', 'clock', 'vase', 'scissors', 'teddybear', 'hairdrier', 'toothbrush'], 'list_hashtags_csv': 'backgroud,person,bicycle,car,motorcycle,airplane,bus,train,truck,boat,trafficlight,firehydrant,stopsign,parkingmeter,bench,bird,cat,dog,horse,sheep,cow,elephant,bear,zebra,giraffe,backpack,umbrella,handbag,tie,suitcase,frisbee,skis,snowboard,sportsball,kite,baseballbat,baseballglove,skateboard,surfboard,tennisracket,bottle,wineglass,cup,fork,knife,spoon,bowl,banana,apple,sandwich,orange,broccoli,carrot,hotdog,pizza,donut,cake,chair,couch,pottedplant,bed,diningtable,toilet,tv,laptop,mouse,remote,keyboard,cellphone,microwave,oven,toaster,sink,refrigerator,book,clock,vase,scissors,teddybear,hairdrier,toothbrush', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 445, 'svm_hashtag_type_desc': 3473, 'photo_desc_type': 3473, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['backgroud', 'person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus', 'train', 'truck', 'boat', 'trafficlight', 'firehydrant', 'stopsign', 'parkingmeter', 'bench', 'bird', 'cat', 'dog', 'horse', 'sheep', 'cow', 'elephant', 'bear', 'zebra', 'giraffe', 'backpack', 'umbrella', 'handbag', 'tie', 'suitcase', 'frisbee', 'skis', 'snowboard', 'sportsball', 'kite', 'baseballbat', 'baseballglove', 'skateboard', 'surfboard', 'tennisracket', 'bottle', 'wineglass', 'cup', 'fork', 'knife', 'spoon', 'bowl', 'banana', 'apple', 'sandwich', 'orange', 'broccoli', 'carrot', 'hotdog', 'pizza', 'donut', 'cake', 'chair', 'couch', 'pottedplant', 'bed', 'diningtable', 'toilet', 'tv', 'laptop', 'mouse', 'remote', 'keyboard', 'cellphone', 'microwave', 'oven', 'toaster', 'sink', 'refrigerator', 'book', 'clock', 'vase', 'scissors', 'teddybear', 'hairdrier', 'toothbrush'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME mask_coco_origin NUM_CLASSES 81 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : mask_coco_origin model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-02-28 03:36:04.513821: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-28 03:36:04.705691: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-28 03:36:06.140275: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.81G (4089315328 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory local folder : /data/models_weight/mask_coco_origin /data/models_weight/mask_coco_origin/mask_model.h5 size_local : 257557808 size in s3 : 257557808 create time local : 2021-08-09 05:27:17 create time in s3 : 2021-08-06 19:45:17 mask_model.h5 already exist and didn't need to update list_images length : 1 NEW PHOTO Processing 1 images image shape: (720, 1280, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 89) min: 0.00000 max: 1280.00000 nb d'objets trouves : 4 Detection mask done ! Trying to reset tf kernel 2763772 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 159 tf kernel not reseted sub process len(results) : 1 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 1 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 4862 list_Values should be empty [] ['backgroud', 'person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus', 'train', 'truck', 'boat', 'trafficlight', 'firehydrant', 'stopsign', 'parkingmeter', 'bench', 'bird', 'cat', 'dog', 'horse', 'sheep', 'cow', 'elephant', 'bear', 'zebra', 'giraffe', 'backpack', 'umbrella', 'handbag', 'tie', 'suitcase', 'frisbee', 'skis', 'snowboard', 'sportsball', 'kite', 'baseballbat', 'baseballglove', 'skateboard', 'surfboard', 'tennisracket', 'bottle', 'wineglass', 'cup', 'fork', 'knife', 'spoon', 'bowl', 'banana', 'apple', 'sandwich', 'orange', 'broccoli', 'carrot', 'hotdog', 'pizza', 'donut', 'cake', 'chair', 'couch', 'pottedplant', 'bed', 'diningtable', 'toilet', 'tv', 'laptop', 'mouse', 'remote', 'keyboard', 'cellphone', 'microwave', 'oven', 'toaster', 'sink', 'refrigerator', 'book', 'clock', 'vase', 'scissors', 'teddybear', 'hairdrier', 'toothbrush'] time for calcul the mask position with numpy : 0.0008044242858886719 nb_pixel_total : 16902 time to create 1 rle with old method : 0.022394418716430664 length of segment : 107 time for calcul the mask position with numpy : 0.02991652488708496 nb_pixel_total : 480755 time to create 1 rle with new method : 0.18766093254089355 length of segment : 632 time for calcul the mask position with numpy : 0.0005035400390625 nb_pixel_total : 36641 time to create 1 rle with old method : 0.04433441162109375 length of segment : 133 time for calcul the mask position with numpy : 0.00012183189392089844 nb_pixel_total : 4794 time to create 1 rle with old method : 0.006679534912109375 length of segment : 51 time spent for convertir_results : 0.4994997978210449 time spend for datou_step_exec : 16.832083463668823 time spend to save output : 6.413459777832031e-05 total time spend for step 1 : 16.8321475982666 caffe_path_current : About to save ! 1 Inside saveOutput : final : True verbose : False eke 12-6-18 : saveMask need to be cleaned for new output ! Catched exception ! Connect or reconnect ! Number saved : None batch 1 Loaded 400 chid ids of type : 445 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++Number RLEs to save : 0 begin to insert list_values into mtr_datou_result : length of list_values in save_final : 1 time used for this insertion : 0.016117334365844727 save missing photos in datou_result : After save, about to update current ! datou_cur_ids : [] len(datou.list_steps) : 1 output : {'917855882': [[(917855882, 492601069, 445, 1092, 1280, 0, 108, 0.9988385, [(1205, 1, 58), (1165, 2, 105), (1159, 3, 113), (1149, 4, 124), (1113, 5, 161), (1100, 6, 174), (1097, 7, 177), (1095, 8, 179), (1095, 9, 179), (1095, 10, 179), (1095, 11, 179), (1095, 12, 179), (1095, 13, 179), (1095, 14, 178), (1095, 15, 178), (1095, 16, 178), (1095, 17, 178), (1095, 18, 177), (1095, 19, 177), (1095, 20, 177), (1095, 21, 177), (1095, 22, 177), (1095, 23, 178), (1095, 24, 178), (1095, 25, 178), (1095, 26, 179), (1095, 27, 179), (1095, 28, 180), (1095, 29, 181), (1095, 30, 182), (1095, 31, 183), (1095, 32, 183), (1095, 33, 184), (1095, 34, 184), (1096, 35, 183), (1096, 36, 183), (1096, 37, 184), (1097, 38, 183), (1097, 39, 183), (1097, 40, 183), (1098, 41, 182), (1098, 42, 182), (1098, 43, 182), (1099, 44, 181), (1099, 45, 181), (1099, 46, 181), (1100, 47, 180), (1100, 48, 180), (1101, 49, 179), (1101, 50, 179), (1102, 51, 178), (1102, 52, 178), (1103, 53, 177), (1103, 54, 177), (1104, 55, 176), (1104, 56, 176), (1104, 57, 176), (1104, 58, 176), (1105, 59, 175), (1105, 60, 175), (1105, 61, 175), (1105, 62, 175), (1105, 63, 175), (1106, 64, 174), (1106, 65, 174), (1106, 66, 174), (1106, 67, 174), (1106, 68, 174), (1106, 69, 174), (1106, 70, 174), (1106, 71, 174), (1106, 72, 174), (1106, 73, 174), (1107, 74, 173), (1107, 75, 173), (1107, 76, 173), (1107, 77, 173), (1107, 78, 173), (1107, 79, 173), (1108, 80, 172), (1108, 81, 172), (1109, 82, 171), (1110, 83, 170), (1110, 84, 170), (1111, 85, 169), (1112, 86, 168), (1113, 87, 166), (1114, 88, 165), (1115, 89, 164), (1117, 90, 162), (1120, 91, 159), (1138, 92, 141), (1146, 93, 133), (1154, 94, 125), (1167, 95, 112), (1177, 96, 102), (1183, 97, 95), (1185, 98, 93), (1187, 99, 90), (1188, 100, 55), (1264, 100, 12), (1190, 101, 50), (1191, 102, 46), (1194, 103, 40), (1197, 104, 34), (1202, 105, 25), (1207, 106, 16)], ['1222,106,1207,106,1206,105,1197,104,1191,102,1182,96,1176,95,1167,95,1166,94,1154,94,1153,93,1146,93,1145,92,1137,91,1120,91,1115,89,1110,84,1107,79,1106,73,1106,64,1104,55,1099,46,1095,34,1095,8,1100,6,1112,6,1113,5,1148,5,1149,4,1158,4,1165,2,1204,2,1205,1,1262,1,1269,2,1273,5,1273,13,1271,18,1271,22,1273,27,1277,31,1279,37,1279,86,1278,87,1278,96,1275,100,1264,100,1263,99,1243,99,1230,104']), (917855882, 492601069, 445, 52, 1128, 16, 668, 0.9977508, [(710, 22, 23), (925, 22, 47), (608, 23, 146), (893, 23, 104), (598, 24, 235), (849, 24, 159), (589, 25, 428), (582, 26, 444), (574, 27, 459), (569, 28, 466), (565, 29, 472), (560, 30, 480), (555, 31, 487), (550, 32, 495), (544, 33, 503), (538, 34, 512), (532, 35, 520), (527, 36, 527), (523, 37, 534), (518, 38, 541), (514, 39, 548), (510, 40, 554), (506, 41, 561), (503, 42, 566), (499, 43, 572), (496, 44, 577), (493, 45, 582), (490, 46, 586), (488, 47, 590), (487, 48, 592), (485, 49, 595), (483, 50, 598), (482, 51, 600), (481, 52, 602), (480, 53, 603), (479, 54, 605), (478, 55, 606), (476, 56, 608), (475, 57, 610), (474, 58, 611), (473, 59, 613), (472, 60, 614), (470, 61, 616), (469, 62, 618), (468, 63, 619), (466, 64, 621), (465, 65, 623), (464, 66, 624), (462, 67, 626), (461, 68, 628), (459, 69, 630), (458, 70, 631), (456, 71, 633), (455, 72, 635), (453, 73, 637), (452, 74, 638), (451, 75, 639), (449, 76, 641), (448, 77, 642), (447, 78, 643), (446, 79, 644), (445, 80, 645), (444, 81, 646), (442, 82, 648), (441, 83, 649), (440, 84, 650), (439, 85, 651), (438, 86, 652), (437, 87, 653), (436, 88, 654), (435, 89, 655), (434, 90, 656), (433, 91, 657), (432, 92, 658), (431, 93, 659), (430, 94, 660), (429, 95, 661), (428, 96, 662), (427, 97, 663), (425, 98, 665), (423, 99, 667), (421, 100, 669), (419, 101, 671), (417, 102, 673), (413, 103, 677), (409, 104, 681), (405, 105, 685), (401, 106, 689), (397, 107, 693), (392, 108, 698), (387, 109, 703), (382, 110, 708), (377, 111, 713), (373, 112, 717), (369, 113, 721), (365, 114, 725), (362, 115, 728), (359, 116, 731), (356, 117, 734), (353, 118, 737), (351, 119, 739), (349, 120, 741), (346, 121, 744), (344, 122, 746), (342, 123, 748), (339, 124, 751), (335, 125, 755), (331, 126, 759), (327, 127, 763), (323, 128, 767), (319, 129, 770), (314, 130, 775), (308, 131, 781), (303, 132, 786), (294, 133, 795), (287, 134, 802), (279, 135, 810), (273, 136, 816), (267, 137, 822), (262, 138, 827), (258, 139, 831), (255, 140, 834), (252, 141, 837), (250, 142, 839), (247, 143, 842), (245, 144, 844), (242, 145, 847), (240, 146, 849), (237, 147, 852), (233, 148, 856), (230, 149, 859), (226, 150, 863), (220, 151, 869), (213, 152, 876), (206, 153, 883), (200, 154, 889), (193, 155, 896), (187, 156, 902), (183, 157, 906), (181, 158, 908), (178, 159, 911), (176, 160, 913), (174, 161, 915), (172, 162, 917), (170, 163, 919), (168, 164, 921), (166, 165, 923), (165, 166, 924), (164, 167, 925), (162, 168, 927), (161, 169, 928), (159, 170, 930), (157, 171, 932), (155, 172, 934), (153, 173, 935), (151, 174, 937), (148, 175, 940), (146, 176, 942), (144, 177, 944), (142, 178, 946), (140, 179, 948), (139, 180, 949), (137, 181, 951), (136, 182, 952), (134, 183, 954), (133, 184, 955), (132, 185, 956), (131, 186, 957), (130, 187, 958), (129, 188, 959), (128, 189, 960), (127, 190, 960), (126, 191, 961), (126, 192, 961), (125, 193, 962), (124, 194, 963), (123, 195, 964), (122, 196, 965), (122, 197, 965), (121, 198, 966), (120, 199, 967), (119, 200, 968), (118, 201, 969), (117, 202, 970), (116, 203, 971), (114, 204, 973), (113, 205, 973), (112, 206, 974), (111, 207, 975), (109, 208, 977), (108, 209, 978), (107, 210, 979), (106, 211, 980), (105, 212, 981), (104, 213, 982), (103, 214, 983), (102, 215, 984), (101, 216, 985), (101, 217, 984), (100, 218, 985), (99, 219, 986), (99, 220, 986), (98, 221, 987), (98, 222, 987), (97, 223, 988), (97, 224, 987), (96, 225, 988), (96, 226, 988), (95, 227, 989), (95, 228, 989), (94, 229, 990), (94, 230, 990), (94, 231, 990), (93, 232, 990), (93, 233, 990), (92, 234, 991), (92, 235, 991), (92, 236, 991), (91, 237, 992), (91, 238, 991), (91, 239, 991), (91, 240, 990), (91, 241, 990), (90, 242, 991), (90, 243, 990), (90, 244, 990), (90, 245, 989), (90, 246, 989), (89, 247, 990), (89, 248, 989), (89, 249, 989), (89, 250, 988), (89, 251, 988), (88, 252, 988), (88, 253, 988), (88, 254, 987), (88, 255, 986), (88, 256, 986), (87, 257, 986), (87, 258, 986), (87, 259, 985), (87, 260, 984), (87, 261, 983), (86, 262, 983), (86, 263, 983), (86, 264, 982), (86, 265, 981), (85, 266, 981), (85, 267, 980), (85, 268, 980), (85, 269, 979), (84, 270, 979), (84, 271, 979), (84, 272, 978), (83, 273, 979), (83, 274, 978), (83, 275, 977), (82, 276, 978), (82, 277, 977), (82, 278, 977), (81, 279, 977), (81, 280, 977), (81, 281, 977), (80, 282, 977), (80, 283, 977), (80, 284, 976), (79, 285, 977), (79, 286, 976), (79, 287, 976), (78, 288, 976), (78, 289, 976), (78, 290, 975), (77, 291, 976), (77, 292, 975), (77, 293, 975), (76, 294, 975), (76, 295, 975), (76, 296, 974), (75, 297, 975), (75, 298, 974), (74, 299, 975), (74, 300, 974), (74, 301, 974), (73, 302, 974), (73, 303, 974), (72, 304, 974), (72, 305, 974), (71, 306, 974), (71, 307, 973), (71, 308, 972), (70, 309, 972), (70, 310, 971), (70, 311, 970), (70, 312, 968), (69, 313, 968), (69, 314, 966), (69, 315, 964), (69, 316, 962), (68, 317, 961), (68, 318, 959), (68, 319, 958), (68, 320, 956), (67, 321, 955), (67, 322, 954), (67, 323, 952), (67, 324, 951), (66, 325, 951), (66, 326, 950), (66, 327, 948), (66, 328, 947), (65, 329, 947), (65, 330, 946), (65, 331, 946), (65, 332, 945), (65, 333, 944), (65, 334, 942), (65, 335, 941), (65, 336, 940), (65, 337, 939), (65, 338, 938), (64, 339, 937), (64, 340, 936), (64, 341, 934), (64, 342, 932), (64, 343, 930), (64, 344, 928), (64, 345, 926), (64, 346, 925), (64, 347, 923), (64, 348, 922), (64, 349, 920), (64, 350, 919), (63, 351, 919), (63, 352, 918), (63, 353, 917), (63, 354, 916), (63, 355, 915), (63, 356, 914), (63, 357, 912), (63, 358, 911), (63, 359, 910), (63, 360, 909), (63, 361, 908), (63, 362, 906), (63, 363, 905), (63, 364, 904), (63, 365, 902), (63, 366, 901), (63, 367, 899), (63, 368, 898), (62, 369, 897), (62, 370, 895), (62, 371, 893), (62, 372, 891), (62, 373, 890), (62, 374, 888), (62, 375, 887), (62, 376, 886), (62, 377, 885), (62, 378, 884), (62, 379, 883), (63, 380, 880), (63, 381, 879), (63, 382, 878), (63, 383, 877), (63, 384, 876), (63, 385, 875), (63, 386, 874), (63, 387, 873), (63, 388, 872), (64, 389, 870), (64, 390, 869), (64, 391, 868), (64, 392, 867), (64, 393, 865), (64, 394, 864), (64, 395, 863), (65, 396, 861), (65, 397, 860), (65, 398, 859), (65, 399, 858), (65, 400, 857), (65, 401, 856), (65, 402, 854), (65, 403, 853), (65, 404, 851), (65, 405, 850), (65, 406, 848), (66, 407, 846), (66, 408, 844), (66, 409, 843), (66, 410, 842), (66, 411, 841), (66, 412, 840), (66, 413, 838), (66, 414, 837), (66, 415, 836), (66, 416, 835), (66, 417, 835), (66, 418, 834), (66, 419, 833), (67, 420, 831), (67, 421, 830), (67, 422, 829), (67, 423, 829), (67, 424, 828), (67, 425, 827), (67, 426, 826), (67, 427, 825), (67, 428, 824), (68, 429, 822), (68, 430, 820), (68, 431, 819), (68, 432, 818), (68, 433, 816), (68, 434, 815), (68, 435, 813), (68, 436, 811), (69, 437, 809), (69, 438, 807), (69, 439, 806), (69, 440, 804), (69, 441, 803), (69, 442, 802), (69, 443, 800), (70, 444, 798), (70, 445, 797), (70, 446, 796), (70, 447, 796), (71, 448, 794), (71, 449, 794), (72, 450, 792), (72, 451, 792), (73, 452, 790), (73, 453, 789), (74, 454, 788), (74, 455, 787), (75, 456, 786), (75, 457, 785), (76, 458, 784), (76, 459, 783), (77, 460, 782), (77, 461, 781), (77, 462, 781), (78, 463, 779), (78, 464, 779), (79, 465, 777), (79, 466, 777), (79, 467, 776), (80, 468, 775), (80, 469, 774), (80, 470, 774), (81, 471, 772), (81, 472, 771), (82, 473, 770), (82, 474, 769), (83, 475, 767), (83, 476, 766), (83, 477, 766), (84, 478, 764), (84, 479, 763), (85, 480, 761), (85, 481, 760), (85, 482, 759), (86, 483, 757), (86, 484, 755), (87, 485, 753), (87, 486, 752), (87, 487, 751), (88, 488, 748), (88, 489, 747), (88, 490, 746), (89, 491, 744), (89, 492, 743), (90, 493, 741), (90, 494, 741), (91, 495, 739), (91, 496, 738), (92, 497, 737), (93, 498, 735), (94, 499, 733), (94, 500, 733), (95, 501, 731), (96, 502, 729), (97, 503, 728), (98, 504, 726), (99, 505, 724), (99, 506, 724), (100, 507, 722), (101, 508, 721), (102, 509, 719), (104, 510, 717), (105, 511, 715), (106, 512, 714), (107, 513, 712), (108, 514, 711), (110, 515, 708), (111, 516, 707), (113, 517, 704), (114, 518, 703), (115, 519, 701), (117, 520, 698), (118, 521, 697), (119, 522, 695), (121, 523, 693), (122, 524, 691), (124, 525, 689), (125, 526, 687), (126, 527, 685), (128, 528, 683), (129, 529, 681), (131, 530, 678), (132, 531, 676), (134, 532, 674), (135, 533, 672), (137, 534, 669), (138, 535, 667), (140, 536, 664), (141, 537, 662), (143, 538, 659), (144, 539, 657), (146, 540, 654), (148, 541, 651), (149, 542, 649), (151, 543, 645), (153, 544, 642), (154, 545, 640), (156, 546, 638), (158, 547, 635), (159, 548, 633), (161, 549, 630), (162, 550, 628), (164, 551, 626), (166, 552, 623), (167, 553, 621), (169, 554, 618), (170, 555, 617), (171, 556, 615), (173, 557, 613), (174, 558, 611), (176, 559, 608), (177, 560, 607), (179, 561, 604), (180, 562, 603), (181, 563, 601), (183, 564, 599), (185, 565, 597), (186, 566, 595), (189, 567, 592), (192, 568, 589), (195, 569, 585), (198, 570, 582), (201, 571, 579), (204, 572, 575), (206, 573, 573), (209, 574, 569), (212, 575, 566), (215, 576, 563), (218, 577, 559), (221, 578, 556), (224, 579, 552), (226, 580, 550), (228, 581, 547), (230, 582, 545), (232, 583, 542), (234, 584, 540), (235, 585, 539), (237, 586, 536), (239, 587, 533), (240, 588, 531), (242, 589, 528), (244, 590, 525), (245, 591, 523), (247, 592, 520), (249, 593, 516), (251, 594, 513), (253, 595, 510), (256, 596, 505), (258, 597, 501), (261, 598, 497), (263, 599, 493), (267, 600, 488), (271, 601, 482), (274, 602, 478), (278, 603, 473), (281, 604, 468), (284, 605, 464), (287, 606, 460), (290, 607, 456), (292, 608, 453), (295, 609, 449), (298, 610, 445), (300, 611, 442), (303, 612, 438), (305, 613, 434), (308, 614, 430), (310, 615, 427), (312, 616, 423), (315, 617, 418), (317, 618, 415), (320, 619, 410), (322, 620, 406), (325, 621, 401), (327, 622, 396), (330, 623, 390), (333, 624, 384), (335, 625, 379), (338, 626, 374), (341, 627, 369), (345, 628, 362), (349, 629, 356), (353, 630, 350), (357, 631, 344), (360, 632, 340), (364, 633, 334), (368, 634, 328), (373, 635, 320), (378, 636, 313), (383, 637, 305), (389, 638, 295), (395, 639, 282), (401, 640, 270), (408, 641, 256), (416, 642, 240), (431, 643, 217), (448, 644, 193), (465, 645, 169), (480, 646, 148), (495, 647, 126), (511, 648, 104), (526, 649, 82), (565, 650, 9)], ['526,649,431,643,368,634,253,595,223,578,186,566,102,509,91,496,70,447,62,379,64,339,70,309,85,269,91,237,101,216,134,183,187,156,225,151,279,135,302,133,353,118,416,103,490,46,506,41,608,23,754,24,892,24,925,22,996,23,1032,27,1066,41,1082,52,1089,72,1089,128,1085,216,1082,237,1045,305,1019,322,1002,338,950,373,920,401,865,446,851,473,822,505,810,528,786,554,773,585,740,612,683,638,607,649']), (917855882, 492601069, 445, 0, 440, 0, 116, 0.9919624, [(127, 1, 141), (94, 2, 206), (384, 2, 2), (59, 3, 272), (340, 3, 57), (22, 4, 381), (19, 5, 387), (16, 6, 392), (15, 7, 394), (14, 8, 396), (14, 9, 397), (13, 10, 399), (12, 11, 400), (12, 12, 400), (11, 13, 402), (10, 14, 403), (11, 15, 403), (11, 16, 404), (12, 17, 403), (12, 18, 404), (12, 19, 405), (12, 20, 405), (12, 21, 406), (12, 22, 406), (12, 23, 407), (12, 24, 407), (12, 25, 408), (12, 26, 408), (12, 27, 408), (12, 28, 408), (12, 29, 409), (12, 30, 409), (12, 31, 409), (12, 32, 409), (12, 33, 409), (12, 34, 410), (12, 35, 410), (12, 36, 410), (12, 37, 410), (12, 38, 410), (12, 39, 410), (12, 40, 410), (12, 41, 411), (12, 42, 411), (12, 43, 411), (12, 44, 411), (12, 45, 411), (12, 46, 410), (12, 47, 410), (12, 48, 410), (12, 49, 410), (12, 50, 410), (12, 51, 410), (12, 52, 409), (12, 53, 408), (12, 54, 408), (12, 55, 407), (12, 56, 406), (12, 57, 404), (12, 58, 403), (11, 59, 403), (11, 60, 402), (11, 61, 401), (11, 62, 400), (11, 63, 400), (11, 64, 399), (11, 65, 398), (11, 66, 397), (11, 67, 397), (11, 68, 396), (11, 69, 395), (11, 70, 395), (11, 71, 394), (11, 72, 394), (11, 73, 394), (11, 74, 393), (11, 75, 393), (11, 76, 393), (11, 77, 393), (11, 78, 393), (11, 79, 393), (11, 80, 392), (10, 81, 394), (10, 82, 394), (10, 83, 395), (9, 84, 396), (9, 85, 262), (279, 85, 126), (9, 86, 75), (98, 86, 28), (142, 86, 117), (292, 86, 112), (9, 87, 71), (152, 87, 103), (294, 87, 110), (8, 88, 68), (161, 88, 91), (296, 88, 107), (8, 89, 63), (176, 89, 73), (297, 89, 106), (7, 90, 61), (205, 90, 40), (298, 90, 104), (7, 91, 57), (299, 91, 103), (6, 92, 54), (300, 92, 102), (6, 93, 50), (301, 93, 100), (7, 94, 46), (303, 94, 97), (7, 95, 44), (306, 95, 92), (7, 96, 42), (308, 96, 89), (7, 97, 40), (310, 97, 86), (7, 98, 38), (312, 98, 83), (8, 99, 34), (314, 99, 79), (8, 100, 32), (317, 100, 75), (8, 101, 29), (319, 101, 71), (13, 102, 19), (324, 102, 63), (20, 103, 6), (330, 103, 51), (337, 104, 37), (344, 105, 22), (352, 106, 3)], ['344,105,319,101,301,93,291,85,259,85,244,90,205,90,204,89,176,89,161,88,141,85,126,85,125,86,98,86,84,85,56,92,36,101,26,102,8,101,6,92,11,80,11,59,12,58,12,17,10,14,16,6,22,4,58,4,59,3,93,3,94,2,126,2,127,1,267,1,268,2,299,2,300,3,396,3,407,6,419,25,421,34,421,51,410,62,404,71,402,80,404,85,401,92,394,98,386,102,365,105']), (917855882, 492601069, 445, 390, 550, 0, 54, 0.9392398, [(414, 0, 7), (441, 0, 60), (508, 0, 28), (402, 1, 142), (401, 2, 146), (402, 3, 145), (404, 4, 143), (406, 5, 140), (408, 6, 137), (410, 7, 134), (411, 8, 132), (412, 9, 130), (413, 10, 127), (414, 11, 125), (415, 12, 123), (415, 13, 122), (416, 14, 120), (417, 15, 117), (417, 16, 116), (418, 17, 114), (418, 18, 113), (418, 19, 111), (418, 20, 109), (419, 21, 107), (419, 22, 105), (419, 23, 103), (419, 24, 102), (419, 25, 100), (420, 26, 97), (420, 27, 95), (420, 28, 94), (421, 29, 91), (421, 30, 90), (422, 31, 88), (422, 32, 88), (422, 33, 87), (423, 34, 84), (423, 35, 82), (423, 36, 81), (424, 37, 79), (424, 38, 77), (424, 39, 75), (424, 40, 73), (424, 41, 71), (425, 42, 67), (425, 43, 66), (426, 44, 62), (426, 45, 6), (433, 45, 52), (443, 46, 30), (450, 47, 1)], ['450,47,449,46,443,46,442,45,426,45,424,41,424,37,423,36,422,31,419,25,419,21,418,20,418,17,417,15,409,6,402,3,402,1,413,1,414,0,420,0,421,1,440,1,441,0,500,0,501,1,507,1,508,0,535,0,536,1,543,1,546,2,546,4,542,8,530,18,527,19,525,21,522,22,520,24,512,28,508,33,505,34,502,37,494,41,492,41,490,43,488,43,484,45,473,45,472,46,451,46'])], 'temp/1740710154_2762130_917855882_da0fa7b7e6b5b551fe26c0ba8713276d.jpg']} ############################### TEST POLYGON ################################ Inside batchDatouExec : verbose : False # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! List Step Type Loaded in datou : mask_detect list_input_json : [] origin BFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 1 ; length of list_pids : 1 ; length of list_args : 1 time to download the photos : 0.3040645122528076 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : False number of steps : 1 step1:mask_detect Fri Feb 28 03:36:13 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 4862 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-02-28 03:36:16.045875: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-02-28 03:36:16.071392: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-02-28 03:36:16.073568: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f5efc000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-02-28 03:36:16.073615: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-02-28 03:36:16.077281: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-02-28 03:36:16.306295: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x245ea6e0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-02-28 03:36:16.306347: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-02-28 03:36:16.307526: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-28 03:36:16.307888: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-28 03:36:16.310623: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-28 03:36:16.313026: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-28 03:36:16.313424: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-28 03:36:16.316089: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-28 03:36:16.317392: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-28 03:36:16.322238: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-28 03:36:16.323418: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-28 03:36:16.323496: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-28 03:36:16.324087: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-28 03:36:16.324102: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-28 03:36:16.324110: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-28 03:36:16.325148: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 4410 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-02-28 03:36:16.403299: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-28 03:36:16.403385: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-28 03:36:16.403407: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-28 03:36:16.403428: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-28 03:36:16.403448: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-28 03:36:16.403467: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-28 03:36:16.403487: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-28 03:36:16.403507: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-28 03:36:16.404664: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-28 03:36:16.405705: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-28 03:36:16.405742: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-28 03:36:16.405762: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-28 03:36:16.405782: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-28 03:36:16.405801: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-28 03:36:16.405820: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-28 03:36:16.405840: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-28 03:36:16.405859: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-28 03:36:16.407020: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-28 03:36:16.407053: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-28 03:36:16.407063: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-28 03:36:16.407072: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-28 03:36:16.408193: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 4410 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (3473, 'mask_coco_origin', 16384, 25088, 'mask_coco_origin', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2018, 3, 19, 10, 42, 21), datetime.datetime(2018, 3, 19, 10, 42, 21)) {'thcl': {'id': 454, 'mtr_user_id': 31, 'name': 'mask_coco_origin', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'backgroud,person,bicycle,car,motorcycle,airplane,bus,train,truck,boat,trafficlight,firehydrant,stopsign,parkingmeter,bench,bird,cat,dog,horse,sheep,cow,elephant,bear,zebra,giraffe,backpack,umbrella,handbag,tie,suitcase,frisbee,skis,snowboard,sportsball,kite,baseballbat,baseballglove,skateboard,surfboard,tennisracket,bottle,wineglass,cup,fork,knife,spoon,bowl,banana,apple,sandwich,orange,broccoli,carrot,hotdog,pizza,donut,cake,chair,couch,pottedplant,bed,diningtable,toilet,tv,laptop,mouse,remote,keyboard,cellphone,microwave,oven,toaster,sink,refrigerator,book,clock,vase,scissors,teddybear,hairdrier,toothbrush', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 445, 'photo_desc_type': 3473, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['backgroud', 'person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus', 'train', 'truck', 'boat', 'trafficlight', 'firehydrant', 'stopsign', 'parkingmeter', 'bench', 'bird', 'cat', 'dog', 'horse', 'sheep', 'cow', 'elephant', 'bear', 'zebra', 'giraffe', 'backpack', 'umbrella', 'handbag', 'tie', 'suitcase', 'frisbee', 'skis', 'snowboard', 'sportsball', 'kite', 'baseballbat', 'baseballglove', 'skateboard', 'surfboard', 'tennisracket', 'bottle', 'wineglass', 'cup', 'fork', 'knife', 'spoon', 'bowl', 'banana', 'apple', 'sandwich', 'orange', 'broccoli', 'carrot', 'hotdog', 'pizza', 'donut', 'cake', 'chair', 'couch', 'pottedplant', 'bed', 'diningtable', 'toilet', 'tv', 'laptop', 'mouse', 'remote', 'keyboard', 'cellphone', 'microwave', 'oven', 'toaster', 'sink', 'refrigerator', 'book', 'clock', 'vase', 'scissors', 'teddybear', 'hairdrier', 'toothbrush'], 'list_hashtags_csv': 'backgroud,person,bicycle,car,motorcycle,airplane,bus,train,truck,boat,trafficlight,firehydrant,stopsign,parkingmeter,bench,bird,cat,dog,horse,sheep,cow,elephant,bear,zebra,giraffe,backpack,umbrella,handbag,tie,suitcase,frisbee,skis,snowboard,sportsball,kite,baseballbat,baseballglove,skateboard,surfboard,tennisracket,bottle,wineglass,cup,fork,knife,spoon,bowl,banana,apple,sandwich,orange,broccoli,carrot,hotdog,pizza,donut,cake,chair,couch,pottedplant,bed,diningtable,toilet,tv,laptop,mouse,remote,keyboard,cellphone,microwave,oven,toaster,sink,refrigerator,book,clock,vase,scissors,teddybear,hairdrier,toothbrush', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 445, 'svm_hashtag_type_desc': 3473, 'photo_desc_type': 3473, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['backgroud', 'person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus', 'train', 'truck', 'boat', 'trafficlight', 'firehydrant', 'stopsign', 'parkingmeter', 'bench', 'bird', 'cat', 'dog', 'horse', 'sheep', 'cow', 'elephant', 'bear', 'zebra', 'giraffe', 'backpack', 'umbrella', 'handbag', 'tie', 'suitcase', 'frisbee', 'skis', 'snowboard', 'sportsball', 'kite', 'baseballbat', 'baseballglove', 'skateboard', 'surfboard', 'tennisracket', 'bottle', 'wineglass', 'cup', 'fork', 'knife', 'spoon', 'bowl', 'banana', 'apple', 'sandwich', 'orange', 'broccoli', 'carrot', 'hotdog', 'pizza', 'donut', 'cake', 'chair', 'couch', 'pottedplant', 'bed', 'diningtable', 'toilet', 'tv', 'laptop', 'mouse', 'remote', 'keyboard', 'cellphone', 'microwave', 'oven', 'toaster', 'sink', 'refrigerator', 'book', 'clock', 'vase', 'scissors', 'teddybear', 'hairdrier', 'toothbrush'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME mask_coco_origin NUM_CLASSES 81 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : mask_coco_origin model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-02-28 03:36:25.978730: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-28 03:36:26.162087: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-28 03:36:27.699068: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.81G (4089315328 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory local folder : /data/models_weight/mask_coco_origin /data/models_weight/mask_coco_origin/mask_model.h5 size_local : 257557808 size in s3 : 257557808 create time local : 2021-08-09 05:27:17 create time in s3 : 2021-08-06 19:45:17 mask_model.h5 already exist and didn't need to update list_images length : 1 NEW PHOTO Processing 1 images image shape: (2448, 2448, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 89) min: 0.00000 max: 2448.00000 nb d'objets trouves : 1 Detection mask done ! Trying to reset tf kernel 2764941 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 159 tf kernel not reseted sub process len(results) : 1 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 1 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 4862 list_Values should be empty [] ['backgroud', 'person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus', 'train', 'truck', 'boat', 'trafficlight', 'firehydrant', 'stopsign', 'parkingmeter', 'bench', 'bird', 'cat', 'dog', 'horse', 'sheep', 'cow', 'elephant', 'bear', 'zebra', 'giraffe', 'backpack', 'umbrella', 'handbag', 'tie', 'suitcase', 'frisbee', 'skis', 'snowboard', 'sportsball', 'kite', 'baseballbat', 'baseballglove', 'skateboard', 'surfboard', 'tennisracket', 'bottle', 'wineglass', 'cup', 'fork', 'knife', 'spoon', 'bowl', 'banana', 'apple', 'sandwich', 'orange', 'broccoli', 'carrot', 'hotdog', 'pizza', 'donut', 'cake', 'chair', 'couch', 'pottedplant', 'bed', 'diningtable', 'toilet', 'tv', 'laptop', 'mouse', 'remote', 'keyboard', 'cellphone', 'microwave', 'oven', 'toaster', 'sink', 'refrigerator', 'book', 'clock', 'vase', 'scissors', 'teddybear', 'hairdrier', 'toothbrush'] time for calcul the mask position with numpy : 0.49260401725769043 nb_pixel_total : 3697068 time to create 1 rle with new method : 0.6179347038269043 length of segment : 2045 time spent for convertir_results : 2.0495412349700928 time spend for datou_step_exec : 21.21785283088684 time spend to save output : 2.574920654296875e-05 total time spend for step 1 : 21.217878580093384 caffe_path_current : About to save ! 1 Inside saveOutput : final : True verbose : False eke 12-6-18 : saveMask need to be cleaned for new output ! Catched exception ! Connect or reconnect ! Number saved : None batch 1 Loaded 719 chid ids of type : 445 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++Number RLEs to save : 0 begin to insert list_values into mtr_datou_result : length of list_values in save_final : 1 time used for this insertion : 0.020975112915039062 save missing photos in datou_result : After save, about to update current ! datou_cur_ids : [] len(datou.list_steps) : 1 output : {'917877156': [[(917877156, 492601069, 445, 0, 2279, 103, 2222, 0.98193616, [(692, 110, 3), (1250, 110, 27), (652, 111, 289), (1202, 111, 135), (614, 112, 377), (1080, 112, 347), (525, 113, 909), (518, 114, 922), (511, 115, 935), (505, 116, 948), (498, 117, 960), (492, 118, 972), (487, 119, 983), (481, 120, 994), (476, 121, 1004), (470, 122, 1015), (465, 123, 1024), (460, 124, 1034), (456, 125, 1042), (451, 126, 1052), (447, 127, 1061), (442, 128, 1076), (438, 129, 1089), (434, 130, 1102), (430, 131, 1115), (427, 132, 1126), (423, 133, 1139), (419, 134, 1151), (416, 135, 1161), (413, 136, 1172), (409, 137, 1181), (406, 138, 1187), (403, 139, 1193), (400, 140, 1200), (397, 141, 1206), (394, 142, 1212), (391, 143, 1218), (388, 144, 1225), (385, 145, 1231), (382, 146, 1238), (379, 147, 1245), (375, 148, 1253), (372, 149, 1259), (368, 150, 1268), (365, 151, 1275), (363, 152, 1281), (360, 153, 1288), (358, 154, 1295), (355, 155, 1303), (353, 156, 1310), (350, 157, 1318), (348, 158, 1325), (345, 159, 1333), (342, 160, 1341), (339, 161, 1350), (336, 162, 1358), (333, 163, 1367), (330, 164, 1376), (327, 165, 1385), (324, 166, 1395), (321, 167, 1404), (317, 168, 1415), (314, 169, 1426), (311, 170, 1436), (307, 171, 1448), (303, 172, 1461), (299, 173, 1473), (296, 174, 1485), (292, 175, 1497), (288, 176, 1510), (284, 177, 1523), (280, 178, 1536), (277, 179, 1548), (274, 180, 1559), (271, 181, 1567), (268, 182, 1574), (266, 183, 1580), (263, 184, 1588), (260, 185, 1595), (258, 186, 1600), (255, 187, 1607), (253, 188, 1613), (250, 189, 1619), (248, 190, 1625), (245, 191, 1631), (243, 192, 1636), (241, 193, 1641), (239, 194, 1646), (237, 195, 1651), (235, 196, 1656), (233, 197, 1661), (231, 198, 1665), (229, 199, 1670), (227, 200, 1674), (225, 201, 1679), (223, 202, 1683), (222, 203, 1686), (220, 204, 1691), (218, 205, 1695), (217, 206, 1697), (215, 207, 1700), (213, 208, 1704), (212, 209, 1706), (210, 210, 1709), (209, 211, 1711), (207, 212, 1715), (206, 213, 1717), (205, 214, 1719), (203, 215, 1722), (202, 216, 1724), (201, 217, 1726), (199, 218, 1729), (198, 219, 1731), (197, 220, 1732), (195, 221, 1735), (194, 222, 1737), (193, 223, 1739), (192, 224, 1741), (190, 225, 1744), (189, 226, 1746), (187, 227, 1749), (186, 228, 1751), (185, 229, 1753), (183, 230, 1756), (182, 231, 1758), (180, 232, 1762), (179, 233, 1764), (178, 234, 1766), (176, 235, 1769), (175, 236, 1771), (173, 237, 1774), (172, 238, 1776), (170, 239, 1780), (169, 240, 1782), (167, 241, 1785), (166, 242, 1787), (164, 243, 1791), (163, 244, 1793), (161, 245, 1796), (160, 246, 1799), (158, 247, 1802), (156, 248, 1805), (155, 249, 1808), (153, 250, 1811), (151, 251, 1814), (150, 252, 1817), (148, 253, 1820), (146, 254, 1824), (145, 255, 1826), (143, 256, 1830), (141, 257, 1834), (140, 258, 1836), (138, 259, 1840), (136, 260, 1843), (134, 261, 1847), (132, 262, 1851), (131, 263, 1854), (129, 264, 1857), (127, 265, 1861), (125, 266, 1865), (123, 267, 1869), (122, 268, 1872), (120, 269, 1875), (119, 270, 1878), (118, 271, 1880), (117, 272, 1882), (115, 273, 1886), (114, 274, 1888), (113, 275, 1890), (112, 276, 1892), (111, 277, 1894), (110, 278, 1896), (109, 279, 1899), (108, 280, 1901), (107, 281, 1903), (106, 282, 1905), (106, 283, 1905), (105, 284, 1907), (104, 285, 1909), (103, 286, 1911), (102, 287, 1913), (101, 288, 1915), (101, 289, 1916), (100, 290, 1918), (99, 291, 1919), (99, 292, 1920), (98, 293, 1921), (98, 294, 1922), (97, 295, 1923), (97, 296, 1924), (97, 297, 1924), (96, 298, 1926), (96, 299, 1926), (95, 300, 1928), (95, 301, 1928), (95, 302, 1929), (94, 303, 1930), (94, 304, 1931), (94, 305, 1931), (93, 306, 1933), (93, 307, 1933), (92, 308, 1935), (92, 309, 1935), (92, 310, 1936), (91, 311, 1937), (91, 312, 1938), (91, 313, 1938), (90, 314, 1940), (90, 315, 1940), (89, 316, 1942), (89, 317, 1942), (89, 318, 1943), (88, 319, 1944), (88, 320, 1945), (88, 321, 1945), (87, 322, 1947), (87, 323, 1948), (87, 324, 1948), (86, 325, 1950), (86, 326, 1950), (86, 327, 1951), (85, 328, 1952), (85, 329, 1953), (84, 330, 1954), (84, 331, 1955), (84, 332, 1955), (83, 333, 1957), (83, 334, 1957), (83, 335, 1958), (82, 336, 1959), (82, 337, 1960), (82, 338, 1961), (81, 339, 1962), (81, 340, 1963), (81, 341, 1963), (80, 342, 1965), (80, 343, 1965), (80, 344, 1966), (79, 345, 1967), (79, 346, 1968), (79, 347, 1968), (78, 348, 1970), (78, 349, 1971), (78, 350, 1971), (77, 351, 1973), (77, 352, 1973), (77, 353, 1974), (76, 354, 1975), (76, 355, 1976), (76, 356, 1977), (75, 357, 1978), (75, 358, 1979), (75, 359, 1979), (75, 360, 1980), (74, 361, 1981), (74, 362, 1982), (74, 363, 1983), (73, 364, 1984), (73, 365, 1985), (73, 366, 1985), (72, 367, 1987), (72, 368, 1988), (72, 369, 1988), (72, 370, 1989), (71, 371, 1991), (71, 372, 1992), (71, 373, 1993), (71, 374, 1993), (70, 375, 1995), (70, 376, 1996), (70, 377, 1997), (70, 378, 1998), (69, 379, 2000), (69, 380, 2001), (69, 381, 2002), (69, 382, 2003), (68, 383, 2005), (68, 384, 2006), (68, 385, 2007), (67, 386, 2009), (67, 387, 2011), (67, 388, 2012), (67, 389, 2013), (66, 390, 2015), (66, 391, 2016), (66, 392, 2017), (65, 393, 2019), (65, 394, 2020), (65, 395, 2021), (65, 396, 2022), (64, 397, 2023), (64, 398, 2024), (64, 399, 2025), (63, 400, 2027), (63, 401, 2028), (63, 402, 2029), (62, 403, 2030), (62, 404, 2031), (62, 405, 2032), (62, 406, 2033), (61, 407, 2034), (61, 408, 2035), (61, 409, 2036), (60, 410, 2037), (60, 411, 2038), (60, 412, 2039), (59, 413, 2040), (59, 414, 2041), (59, 415, 2042), (58, 416, 2043), (58, 417, 2044), (57, 418, 2046), (57, 419, 2046), (57, 420, 2047), (56, 421, 2048), (56, 422, 2049), (56, 423, 2049), (55, 424, 2051), (55, 425, 2051), (55, 426, 2052), (54, 427, 2053), (54, 428, 2054), (53, 429, 2055), (53, 430, 2056), (53, 431, 2056), (52, 432, 2058), (52, 433, 2058), (51, 434, 2060), (51, 435, 2060), (51, 436, 2061), (50, 437, 2062), (50, 438, 2062), (49, 439, 2064), (49, 440, 2064), (48, 441, 2066), (48, 442, 2066), (48, 443, 2067), (47, 444, 2068), (47, 445, 2068), (47, 446, 2068), (47, 447, 2068), (46, 448, 2070), (46, 449, 2070), (46, 450, 2070), (46, 451, 2070), (45, 452, 2071), (45, 453, 2072), (45, 454, 2072), (45, 455, 2072), (45, 456, 2072), (44, 457, 2074), (44, 458, 2074), (44, 459, 2074), (44, 460, 2074), (43, 461, 2075), (43, 462, 2076), (43, 463, 2076), (43, 464, 2076), (42, 465, 2077), (42, 466, 2078), (42, 467, 2078), (42, 468, 2078), (41, 469, 2079), (41, 470, 2080), (41, 471, 2080), (41, 472, 2080), (40, 473, 2081), (40, 474, 2082), (40, 475, 2082), (39, 476, 2083), (39, 477, 2083), (39, 478, 2084), (39, 479, 2084), (39, 480, 2084), (38, 481, 2085), (38, 482, 2086), (38, 483, 2086), (38, 484, 2086), (37, 485, 2087), (37, 486, 2088), (37, 487, 2088), (37, 488, 2088), (36, 489, 2090), (36, 490, 2090), (36, 491, 2090), (36, 492, 2090), (35, 493, 2092), (35, 494, 2092), (35, 495, 2092), (35, 496, 2092), (35, 497, 2093), (34, 498, 2094), (34, 499, 2094), (34, 500, 2095), (34, 501, 2095), (34, 502, 2095), (33, 503, 2097), (33, 504, 2097), (33, 505, 2097), (33, 506, 2097), (32, 507, 2099), (32, 508, 2099), (32, 509, 2099), (32, 510, 2100), (32, 511, 2100), (31, 512, 2101), (31, 513, 2102), (31, 514, 2102), (31, 515, 2102), (31, 516, 2103), (30, 517, 2104), (30, 518, 2104), (30, 519, 2105), (30, 520, 2105), (30, 521, 2105), (30, 522, 2105), (30, 523, 2105), (30, 524, 2106), (29, 525, 2107), (29, 526, 2107), (29, 527, 2107), (29, 528, 2107), (29, 529, 2108), (29, 530, 2108), (29, 531, 2108), (29, 532, 2108), (29, 533, 2108), (28, 534, 2110), (28, 535, 2110), (28, 536, 2110), (28, 537, 2110), (28, 538, 2111), (28, 539, 2111), (28, 540, 2111), (28, 541, 2111), (28, 542, 2112), (27, 543, 2113), (27, 544, 2113), (27, 545, 2113), (27, 546, 2114), (27, 547, 2114), (27, 548, 2114), (27, 549, 2114), (27, 550, 2114), (27, 551, 2115), (27, 552, 2115), (26, 553, 2116), (26, 554, 2116), (26, 555, 2117), (26, 556, 2117), (26, 557, 2117), (26, 558, 2117), (26, 559, 2118), (26, 560, 2118), (26, 561, 2118), (26, 562, 2119), (25, 563, 2120), (25, 564, 2120), (25, 565, 2120), (25, 566, 2121), (25, 567, 2121), (25, 568, 2121), (25, 569, 2121), (25, 570, 2122), (25, 571, 2122), (25, 572, 2122), (24, 573, 2124), (24, 574, 2124), (24, 575, 2124), (24, 576, 2124), (24, 577, 2125), (24, 578, 2125), (24, 579, 2125), (24, 580, 2126), (24, 581, 2126), (24, 582, 2126), (23, 583, 2128), (23, 584, 2128), (23, 585, 2128), (23, 586, 2128), (23, 587, 2129), (23, 588, 2129), (23, 589, 2129), (23, 590, 2130), (23, 591, 2130), (23, 592, 2130), (23, 593, 2131), (22, 594, 2132), (22, 595, 2132), (22, 596, 2132), (22, 597, 2133), (22, 598, 2133), (22, 599, 2133), (22, 600, 2133), (22, 601, 2133), (22, 602, 2134), (22, 603, 2134), (22, 604, 2134), (22, 605, 2134), (22, 606, 2134), (22, 607, 2134), (22, 608, 2135), (22, 609, 2135), (22, 610, 2135), (21, 611, 2136), (21, 612, 2136), (21, 613, 2137), (21, 614, 2137), (21, 615, 2137), (21, 616, 2137), (21, 617, 2137), (21, 618, 2138), (21, 619, 2138), (21, 620, 2138), (21, 621, 2138), (21, 622, 2138), (21, 623, 2139), (21, 624, 2139), (21, 625, 2139), (21, 626, 2139), (21, 627, 2139), (21, 628, 2139), (20, 629, 2141), (20, 630, 2141), (20, 631, 2141), (20, 632, 2141), (20, 633, 2141), (20, 634, 2141), (20, 635, 2142), (20, 636, 2142), (20, 637, 2142), (20, 638, 2142), (20, 639, 2142), (20, 640, 2143), (20, 641, 2143), (20, 642, 2143), (20, 643, 2143), (20, 644, 2143), (20, 645, 2143), (20, 646, 2144), (19, 647, 2145), (19, 648, 2145), (19, 649, 2145), (19, 650, 2145), (19, 651, 2145), (19, 652, 2146), (19, 653, 2146), (19, 654, 2146), (19, 655, 2146), (19, 656, 2146), (19, 657, 2146), (19, 658, 2146), (19, 659, 2147), (19, 660, 2147), (19, 661, 2147), (19, 662, 2147), (19, 663, 2147), (19, 664, 2147), (19, 665, 2148), (18, 666, 2149), (18, 667, 2149), (18, 668, 2149), (18, 669, 2149), (18, 670, 2149), (18, 671, 2149), (18, 672, 2149), (19, 673, 2148), (19, 674, 2148), (19, 675, 2148), (19, 676, 2148), (19, 677, 2148), (19, 678, 2148), (19, 679, 2148), (19, 680, 2148), (19, 681, 2147), (19, 682, 2147), (20, 683, 2146), (20, 684, 2146), (20, 685, 2146), (20, 686, 2146), (20, 687, 2146), (20, 688, 2146), (20, 689, 2146), (20, 690, 2146), (20, 691, 2146), (20, 692, 2146), (21, 693, 2144), (21, 694, 2144), (21, 695, 2144), (21, 696, 2144), (21, 697, 2144), (21, 698, 2144), (21, 699, 2144), (21, 700, 2144), (21, 701, 2144), (21, 702, 2144), (22, 703, 2143), (22, 704, 2142), (22, 705, 2142), (22, 706, 2142), (22, 707, 2142), (22, 708, 2142), (22, 709, 2142), (22, 710, 2142), (22, 711, 2142), (23, 712, 2141), (23, 713, 2141), (23, 714, 2141), (23, 715, 2141), (23, 716, 2140), (23, 717, 2140), (23, 718, 2140), (23, 719, 2140), (23, 720, 2140), (23, 721, 2140), (24, 722, 2139), (24, 723, 2139), (24, 724, 2139), (24, 725, 2139), (24, 726, 2139), (24, 727, 2138), (24, 728, 2138), (24, 729, 2138), (25, 730, 2137), (25, 731, 2137), (25, 732, 2137), (25, 733, 2137), (25, 734, 2137), (25, 735, 2137), (25, 736, 2137), (25, 737, 2137), (25, 738, 2136), (26, 739, 2135), (26, 740, 2135), (26, 741, 2135), (26, 742, 2135), (26, 743, 2135), (26, 744, 2135), (26, 745, 2135), (26, 746, 2135), (26, 747, 2135), (26, 748, 2134), (26, 749, 2134), (26, 750, 2134), (26, 751, 2134), (26, 752, 2134), (26, 753, 2134), (26, 754, 2134), (26, 755, 2134), (26, 756, 2134), (26, 757, 2134), (26, 758, 2133), (26, 759, 2133), (26, 760, 2133), (26, 761, 2133), (26, 762, 2133), (26, 763, 2133), (26, 764, 2133), (26, 765, 2133), (26, 766, 2133), (26, 767, 2133), (26, 768, 2132), (26, 769, 2132), (26, 770, 2132), (26, 771, 2132), (26, 772, 2132), (26, 773, 2132), (26, 774, 2132), (26, 775, 2132), (26, 776, 2132), (26, 777, 2131), (26, 778, 2131), (26, 779, 2131), (26, 780, 2131), (26, 781, 2131), (26, 782, 2131), (26, 783, 2131), (26, 784, 2131), (26, 785, 2131), (26, 786, 2130), (26, 787, 2130), (26, 788, 2130), (25, 789, 2131), (25, 790, 2131), (25, 791, 2131), (25, 792, 2131), (25, 793, 2131), (25, 794, 2131), (25, 795, 2130), (25, 796, 2130), (25, 797, 2130), (25, 798, 2130), (25, 799, 2130), (25, 800, 2130), (25, 801, 2130), (25, 802, 2130), (25, 803, 2130), (25, 804, 2130), (25, 805, 2129), (25, 806, 2129), (25, 807, 2129), (25, 808, 2129), (25, 809, 2129), (25, 810, 2129), (25, 811, 2129), (25, 812, 2129), (25, 813, 2129), (25, 814, 2129), (25, 815, 2128), (25, 816, 2128), (25, 817, 2128), (25, 818, 2128), (25, 819, 2128), (25, 820, 2128), (25, 821, 2128), (25, 822, 2128), (25, 823, 2127), (25, 824, 2127), (25, 825, 2126), (25, 826, 2126), (25, 827, 2125), (25, 828, 2125), (25, 829, 2125), (25, 830, 2124), (25, 831, 2124), (26, 832, 2122), (26, 833, 2122), (26, 834, 2122), (26, 835, 2121), (26, 836, 2121), (26, 837, 2120), (26, 838, 2120), (26, 839, 2120), (26, 840, 2119), (26, 841, 2119), (26, 842, 2118), (26, 843, 2118), (26, 844, 2118), (26, 845, 2117), (26, 846, 2117), (27, 847, 2116), (27, 848, 2115), (27, 849, 2115), (27, 850, 2115), (27, 851, 2114), (27, 852, 2114), (27, 853, 2114), (27, 854, 2113), (27, 855, 2113), (27, 856, 2113), (27, 857, 2112), (27, 858, 2112), (27, 859, 2112), (27, 860, 2111), (27, 861, 2111), (28, 862, 2110), (28, 863, 2110), (28, 864, 2109), (28, 865, 2109), (28, 866, 2109), (28, 867, 2108), (28, 868, 2108), (28, 869, 2108), (28, 870, 2107), (28, 871, 2107), (28, 872, 2107), (28, 873, 2107), (28, 874, 2106), (28, 875, 2106), (29, 876, 2105), (29, 877, 2105), (29, 878, 2104), (29, 879, 2104), (29, 880, 2104), (29, 881, 2103), (29, 882, 2103), (29, 883, 2103), (29, 884, 2103), (29, 885, 2102), (29, 886, 2102), (29, 887, 2102), (29, 888, 2102), (30, 889, 2100), (30, 890, 2100), (30, 891, 2100), (30, 892, 2100), (30, 893, 2100), (30, 894, 2099), (30, 895, 2099), (30, 896, 2099), (30, 897, 2099), (30, 898, 2098), (30, 899, 2098), (30, 900, 2098), (30, 901, 2098), (30, 902, 2098), (30, 903, 2098), (30, 904, 2098), (30, 905, 2098), (30, 906, 2097), (30, 907, 2097), (30, 908, 2097), (30, 909, 2097), (30, 910, 2097), (29, 911, 2098), (29, 912, 2098), (29, 913, 2098), (29, 914, 2097), (29, 915, 2097), (29, 916, 2097), (29, 917, 2097), (29, 918, 2097), (29, 919, 2097), (29, 920, 2097), (29, 921, 2097), (29, 922, 2096), (29, 923, 2096), (29, 924, 2096), (29, 925, 2096), (29, 926, 2096), (29, 927, 2096), (29, 928, 2096), (29, 929, 2096), (29, 930, 2096), (29, 931, 2095), (28, 932, 2096), (28, 933, 2096), (28, 934, 2096), (28, 935, 2096), (28, 936, 2096), (28, 937, 2096), (28, 938, 2096), (28, 939, 2096), (28, 940, 2095), (28, 941, 2095), (28, 942, 2095), (28, 943, 2095), (28, 944, 2095), (28, 945, 2095), (28, 946, 2095), (28, 947, 2095), (28, 948, 2095), (28, 949, 2094), (28, 950, 2094), (28, 951, 2094), (28, 952, 2094), (27, 953, 2095), (27, 954, 2095), (27, 955, 2095), (27, 956, 2095), (27, 957, 2095), (27, 958, 2094), (27, 959, 2094), (27, 960, 2094), (27, 961, 2094), (27, 962, 2094), (27, 963, 2094), (27, 964, 2094), (27, 965, 2094), (27, 966, 2094), (27, 967, 2093), (27, 968, 2093), (27, 969, 2093), (27, 970, 2093), (27, 971, 2093), (27, 972, 2093), (27, 973, 2093), (27, 974, 2093), (27, 975, 2092), (27, 976, 2092), (27, 977, 2092), (27, 978, 2092), (27, 979, 2091), (27, 980, 2091), (27, 981, 2091), (27, 982, 2091), (27, 983, 2090), (27, 984, 2090), (27, 985, 2090), (27, 986, 2089), (27, 987, 2089), (27, 988, 2089), (27, 989, 2089), (27, 990, 2088), (27, 991, 2088), (27, 992, 2088), (27, 993, 2088), (27, 994, 2087), (28, 995, 2086), (28, 996, 2086), (28, 997, 2085), (28, 998, 2085), (28, 999, 2085), (28, 1000, 2084), (28, 1001, 2084), (28, 1002, 2084), (28, 1003, 2084), (28, 1004, 2083), (28, 1005, 2083), (28, 1006, 2083), (28, 1007, 2082), (28, 1008, 2082), (28, 1009, 2082), (28, 1010, 2081), (28, 1011, 2081), (28, 1012, 2081), (28, 1013, 2080), (28, 1014, 2080), (28, 1015, 2080), (28, 1016, 2079), (28, 1017, 2079), (28, 1018, 2079), (29, 1019, 2077), (29, 1020, 2077), (29, 1021, 2077), (29, 1022, 2076), (29, 1023, 2076), (29, 1024, 2076), (29, 1025, 2075), (29, 1026, 2075), (29, 1027, 2074), (29, 1028, 2074), (29, 1029, 2074), (29, 1030, 2073), (29, 1031, 2073), (29, 1032, 2073), (29, 1033, 2072), (29, 1034, 2072), (29, 1035, 2071), (29, 1036, 2071), (29, 1037, 2071), (29, 1038, 2070), (29, 1039, 2070), (29, 1040, 2069), (29, 1041, 2069), (30, 1042, 2067), (30, 1043, 2067), (30, 1044, 2067), (30, 1045, 2066), (30, 1046, 2066), (30, 1047, 2065), (30, 1048, 2065), (30, 1049, 2064), (30, 1050, 2064), (30, 1051, 2063), (30, 1052, 2062), (30, 1053, 2062), (30, 1054, 2061), (30, 1055, 2060), (30, 1056, 2059), (30, 1057, 2059), (30, 1058, 2058), (30, 1059, 2057), (30, 1060, 2056), (30, 1061, 2055), (30, 1062, 2054), (30, 1063, 2054), (30, 1064, 2053), (30, 1065, 2052), (30, 1066, 2051), (30, 1067, 2050), (29, 1068, 2050), (29, 1069, 2049), (29, 1070, 2048), (29, 1071, 2047), (29, 1072, 2046), (29, 1073, 2045), (29, 1074, 2044), (29, 1075, 2043), (29, 1076, 2042), (29, 1077, 2041), (29, 1078, 2040), (29, 1079, 2039), (29, 1080, 2038), (29, 1081, 2037), (29, 1082, 2036), (29, 1083, 2035), (29, 1084, 2034), (29, 1085, 2034), (29, 1086, 2033), (29, 1087, 2032), (29, 1088, 2031), (29, 1089, 2031), (29, 1090, 2030), (29, 1091, 2029), (29, 1092, 2028), (29, 1093, 2028), (29, 1094, 2027), (29, 1095, 2026), (29, 1096, 2026), (29, 1097, 2025), (29, 1098, 2024), (29, 1099, 2024), (29, 1100, 2023), (29, 1101, 2022), (29, 1102, 2022), (29, 1103, 2021), (29, 1104, 2021), (29, 1105, 2020), (29, 1106, 2020), (29, 1107, 2019), (29, 1108, 2018), (29, 1109, 2018), (29, 1110, 2017), (29, 1111, 2017), (29, 1112, 2016), (29, 1113, 2016), (29, 1114, 2015), (29, 1115, 2015), (29, 1116, 2014), (29, 1117, 2014), (29, 1118, 2014), (29, 1119, 2013), (29, 1120, 2013), (29, 1121, 2012), (29, 1122, 2012), (29, 1123, 2011), (29, 1124, 2011), (29, 1125, 2011), (29, 1126, 2010), (29, 1127, 2010), (29, 1128, 2010), (29, 1129, 2009), (29, 1130, 2009), (29, 1131, 2009), (29, 1132, 2008), (29, 1133, 2008), (29, 1134, 2007), (29, 1135, 2007), (29, 1136, 2007), (29, 1137, 2006), (29, 1138, 2006), (29, 1139, 2006), (29, 1140, 2005), (29, 1141, 2005), (29, 1142, 2005), (29, 1143, 2004), (29, 1144, 2004), (29, 1145, 2004), (29, 1146, 2003), (29, 1147, 2003), (29, 1148, 2002), (29, 1149, 2002), (29, 1150, 2002), (29, 1151, 2001), (29, 1152, 2001), (29, 1153, 2000), (29, 1154, 2000), (29, 1155, 2000), (29, 1156, 1999), (29, 1157, 1999), (29, 1158, 1998), (29, 1159, 1998), (29, 1160, 1998), (29, 1161, 1997), (29, 1162, 1997), (29, 1163, 1996), (29, 1164, 1996), (29, 1165, 1995), (29, 1166, 1995), (29, 1167, 1995), (29, 1168, 1994), (29, 1169, 1994), (29, 1170, 1993), (29, 1171, 1993), (29, 1172, 1992), (29, 1173, 1992), (29, 1174, 1991), (29, 1175, 1991), (29, 1176, 1990), (29, 1177, 1990), (29, 1178, 1990), (29, 1179, 1989), (29, 1180, 1989), (29, 1181, 1988), (29, 1182, 1988), (29, 1183, 1987), (29, 1184, 1987), (29, 1185, 1986), (29, 1186, 1986), (29, 1187, 1985), (29, 1188, 1985), (29, 1189, 1984), (29, 1190, 1984), (29, 1191, 1983), (29, 1192, 1983), (29, 1193, 1982), (29, 1194, 1981), (29, 1195, 1981), (29, 1196, 1980), (29, 1197, 1980), (29, 1198, 1979), (29, 1199, 1979), (29, 1200, 1978), (29, 1201, 1977), (29, 1202, 1977), (29, 1203, 1976), (29, 1204, 1975), (29, 1205, 1975), (29, 1206, 1974), (29, 1207, 1973), (29, 1208, 1972), (29, 1209, 1972), (29, 1210, 1971), (29, 1211, 1970), (29, 1212, 1969), (29, 1213, 1968), (29, 1214, 1967), (29, 1215, 1967), (29, 1216, 1966), (29, 1217, 1965), (29, 1218, 1964), (29, 1219, 1963), (29, 1220, 1962), (29, 1221, 1961), (29, 1222, 1960), (29, 1223, 1960), (29, 1224, 1959), (29, 1225, 1958), (29, 1226, 1957), (29, 1227, 1956), (29, 1228, 1956), (29, 1229, 1955), (29, 1230, 1954), (29, 1231, 1953), (29, 1232, 1953), (29, 1233, 1952), (29, 1234, 1951), (29, 1235, 1951), (29, 1236, 1950), (29, 1237, 1949), (29, 1238, 1949), (29, 1239, 1948), (29, 1240, 1948), (29, 1241, 1947), (29, 1242, 1946), (29, 1243, 1946), (29, 1244, 1945), (29, 1245, 1945), (29, 1246, 1944), (29, 1247, 1944), (29, 1248, 1943), (29, 1249, 1943), (29, 1250, 1942), (29, 1251, 1942), (29, 1252, 1941), (29, 1253, 1941), (29, 1254, 1940), (29, 1255, 1940), (29, 1256, 1939), (29, 1257, 1939), (29, 1258, 1938), (29, 1259, 1938), (29, 1260, 1937), (29, 1261, 1937), (29, 1262, 1936), (29, 1263, 1936), (29, 1264, 1936), (29, 1265, 1935), (29, 1266, 1935), (29, 1267, 1934), (29, 1268, 1934), (29, 1269, 1934), (29, 1270, 1933), (29, 1271, 1933), (29, 1272, 1933), (29, 1273, 1932), (29, 1274, 1932), (29, 1275, 1931), (29, 1276, 1931), (29, 1277, 1931), (30, 1278, 1930), (30, 1279, 1929), (30, 1280, 1929), (30, 1281, 1929), (30, 1282, 1929), (30, 1283, 1928), (30, 1284, 1928), (30, 1285, 1928), (30, 1286, 1928), (30, 1287, 1928), (30, 1288, 1927), (30, 1289, 1927), (30, 1290, 1927), (30, 1291, 1927), (30, 1292, 1926), (30, 1293, 1926), (30, 1294, 1926), (30, 1295, 1926), (30, 1296, 1925), (30, 1297, 1925), (30, 1298, 1925), (30, 1299, 1924), (30, 1300, 1924), (30, 1301, 1924), (31, 1302, 1923), (31, 1303, 1922), (31, 1304, 1922), (31, 1305, 1922), (31, 1306, 1922), (31, 1307, 1921), (31, 1308, 1921), (31, 1309, 1921), (31, 1310, 1921), (31, 1311, 1920), (31, 1312, 1920), (31, 1313, 1920), (31, 1314, 1919), (31, 1315, 1919), (31, 1316, 1919), (31, 1317, 1919), (31, 1318, 1918), (31, 1319, 1918), (31, 1320, 1918), (31, 1321, 1918), (31, 1322, 1917), (31, 1323, 1917), (31, 1324, 1917), (31, 1325, 1916), (31, 1326, 1916), (32, 1327, 1915), (32, 1328, 1914), (32, 1329, 1914), (32, 1330, 1914), (32, 1331, 1914), (32, 1332, 1913), (32, 1333, 1913), (32, 1334, 1913), (32, 1335, 1912), (32, 1336, 1912), (32, 1337, 1912), (32, 1338, 1911), (32, 1339, 1911), (32, 1340, 1911), (32, 1341, 1911), (32, 1342, 1910), (32, 1343, 1910), (32, 1344, 1910), (32, 1345, 1909), (32, 1346, 1909), (32, 1347, 1909), (32, 1348, 1908), (32, 1349, 1908), (33, 1350, 1907), (33, 1351, 1906), (33, 1352, 1906), (33, 1353, 1905), (33, 1354, 1905), (33, 1355, 1904), (33, 1356, 1904), (33, 1357, 1903), (33, 1358, 1903), (33, 1359, 1902), (33, 1360, 1902), (33, 1361, 1901), (33, 1362, 1900), (34, 1363, 1899), (34, 1364, 1898), (34, 1365, 1898), (34, 1366, 1897), (34, 1367, 1896), (34, 1368, 1896), (34, 1369, 1895), (34, 1370, 1894), (34, 1371, 1894), (34, 1372, 1893), (34, 1373, 1892), (34, 1374, 1892), (35, 1375, 1890), (35, 1376, 1889), (35, 1377, 1888), (35, 1378, 1888), (35, 1379, 1887), (35, 1380, 1886), (35, 1381, 1885), (35, 1382, 1884), (35, 1383, 1883), (35, 1384, 1883), (35, 1385, 1882), (36, 1386, 1880), (36, 1387, 1879), (36, 1388, 1878), (36, 1389, 1877), (36, 1390, 1876), (36, 1391, 1875), (36, 1392, 1874), (36, 1393, 1873), (36, 1394, 1872), (36, 1395, 1871), (36, 1396, 1871), (36, 1397, 1870), (37, 1398, 1868), (37, 1399, 1867), (37, 1400, 1866), (37, 1401, 1866), (37, 1402, 1865), (37, 1403, 1864), (37, 1404, 1863), (37, 1405, 1863), (37, 1406, 1862), (37, 1407, 1861), (37, 1408, 1861), (38, 1409, 1859), (38, 1410, 1858), (38, 1411, 1858), (38, 1412, 1857), (38, 1413, 1856), (38, 1414, 1856), (38, 1415, 1855), (38, 1416, 1855), (38, 1417, 1854), (38, 1418, 1853), (39, 1419, 1852), (39, 1420, 1851), (39, 1421, 1851), (39, 1422, 1850), (39, 1423, 1850), (39, 1424, 1849), (39, 1425, 1849), (39, 1426, 1848), (39, 1427, 1848), (39, 1428, 1847), (39, 1429, 1847), (39, 1430, 1847), (39, 1431, 1847), (39, 1432, 1847), (39, 1433, 1846), (39, 1434, 1846), (39, 1435, 1846), (39, 1436, 1846), (39, 1437, 1845), (40, 1438, 1844), (40, 1439, 1844), (40, 1440, 1844), (40, 1441, 1844), (40, 1442, 1843), (40, 1443, 1843), (40, 1444, 1843), (40, 1445, 1843), (40, 1446, 1843), (40, 1447, 1842), (40, 1448, 1842), (40, 1449, 1842), (40, 1450, 1842), (40, 1451, 1842), (40, 1452, 1841), (40, 1453, 1841), (40, 1454, 1841), (40, 1455, 1841), (40, 1456, 1841), (40, 1457, 1840), (40, 1458, 1840), (40, 1459, 1840), (40, 1460, 1840), (40, 1461, 1840), (40, 1462, 1839), (40, 1463, 1839), (40, 1464, 1839), (40, 1465, 1839), (40, 1466, 1839), (40, 1467, 1838), (40, 1468, 1838), (40, 1469, 1838), (40, 1470, 1838), (40, 1471, 1838), (40, 1472, 1837), (40, 1473, 1837), (40, 1474, 1837), (40, 1475, 1837), (40, 1476, 1837), (40, 1477, 1836), (40, 1478, 1836), (40, 1479, 1836), (40, 1480, 1836), (40, 1481, 1836), (40, 1482, 1836), (40, 1483, 1835), (41, 1484, 1834), (41, 1485, 1834), (41, 1486, 1834), (41, 1487, 1834), (41, 1488, 1834), (41, 1489, 1833), (41, 1490, 1833), (41, 1491, 1833), (41, 1492, 1833), (41, 1493, 1833), (41, 1494, 1833), (41, 1495, 1832), (41, 1496, 1832), (41, 1497, 1832), (41, 1498, 1832), (41, 1499, 1832), (41, 1500, 1832), (41, 1501, 1831), (41, 1502, 1831), (41, 1503, 1831), (41, 1504, 1831), (41, 1505, 1831), (41, 1506, 1831), (41, 1507, 1831), (41, 1508, 1830), (41, 1509, 1830), (41, 1510, 1830), (41, 1511, 1830), (41, 1512, 1830), (41, 1513, 1830), (41, 1514, 1830), (41, 1515, 1829), (42, 1516, 1828), (42, 1517, 1828), (42, 1518, 1828), (42, 1519, 1828), (42, 1520, 1828), (42, 1521, 1828), (42, 1522, 1827), (42, 1523, 1827), (42, 1524, 1827), (42, 1525, 1827), (42, 1526, 1827), (42, 1527, 1827), (42, 1528, 1827), (42, 1529, 1827), (42, 1530, 1826), (42, 1531, 1826), (42, 1532, 1826), (42, 1533, 1826), (42, 1534, 1826), (42, 1535, 1826), (42, 1536, 1826), (42, 1537, 1825), (43, 1538, 1824), (43, 1539, 1824), (43, 1540, 1824), (43, 1541, 1824), (43, 1542, 1824), (43, 1543, 1824), (43, 1544, 1823), (43, 1545, 1823), (43, 1546, 1823), (43, 1547, 1823), (43, 1548, 1823), (43, 1549, 1823), (43, 1550, 1823), (43, 1551, 1822), (43, 1552, 1822), (43, 1553, 1822), (43, 1554, 1822), (43, 1555, 1822), (43, 1556, 1822), (43, 1557, 1822), (43, 1558, 1821), (43, 1559, 1821), (43, 1560, 1821), (43, 1561, 1821), (44, 1562, 1820), (44, 1563, 1820), (44, 1564, 1819), (44, 1565, 1819), (44, 1566, 1819), (44, 1567, 1819), (44, 1568, 1819), (44, 1569, 1819), (44, 1570, 1819), (44, 1571, 1818), (44, 1572, 1818), (44, 1573, 1818), (44, 1574, 1818), (44, 1575, 1818), (44, 1576, 1818), (44, 1577, 1818), (44, 1578, 1817), (44, 1579, 1817), (44, 1580, 1817), (44, 1581, 1817), (44, 1582, 1817), (44, 1583, 1817), (44, 1584, 1817), (44, 1585, 1816), (44, 1586, 1816), (44, 1587, 1816), (44, 1588, 1816), (44, 1589, 1816), (44, 1590, 1816), (43, 1591, 1817), (43, 1592, 1816), (43, 1593, 1816), (43, 1594, 1816), (43, 1595, 1816), (43, 1596, 1816), (43, 1597, 1816), (43, 1598, 1815), (43, 1599, 1815), (43, 1600, 1815), (43, 1601, 1815), (43, 1602, 1815), (43, 1603, 1815), (43, 1604, 1815), (43, 1605, 1814), (43, 1606, 1814), (43, 1607, 1814), (42, 1608, 1815), (42, 1609, 1815), (42, 1610, 1815), (42, 1611, 1814), (42, 1612, 1814), (42, 1613, 1814), (42, 1614, 1814), (42, 1615, 1814), (42, 1616, 1814), (42, 1617, 1813), (42, 1618, 1813), (42, 1619, 1813), (42, 1620, 1813), (42, 1621, 1813), (42, 1622, 1813), (42, 1623, 1812), (41, 1624, 1813), (41, 1625, 1813), (41, 1626, 1813), (41, 1627, 1813), (41, 1628, 1813), (41, 1629, 1812), (41, 1630, 1812), (41, 1631, 1812), (41, 1632, 1812), (41, 1633, 1812), (41, 1634, 1811), (41, 1635, 1811), (41, 1636, 1811), (41, 1637, 1811), (41, 1638, 1811), (41, 1639, 1811), (41, 1640, 1810), (40, 1641, 1811), (40, 1642, 1811), (40, 1643, 1811), (40, 1644, 1811), (40, 1645, 1810), (40, 1646, 1810), (40, 1647, 1810), (40, 1648, 1810), (40, 1649, 1810), (40, 1650, 1809), (40, 1651, 1809), (40, 1652, 1809), (40, 1653, 1809), (40, 1654, 1809), (40, 1655, 1808), (40, 1656, 1808), (40, 1657, 1808), (40, 1658, 1808), (40, 1659, 1807), (41, 1660, 1806), (41, 1661, 1806), (41, 1662, 1805), (41, 1663, 1805), (41, 1664, 1805), (41, 1665, 1804), (42, 1666, 1803), (42, 1667, 1803), (42, 1668, 1803), (42, 1669, 1802), (42, 1670, 1802), (42, 1671, 1802), (43, 1672, 1800), (43, 1673, 1800), (43, 1674, 1800), (43, 1675, 1799), (43, 1676, 1799), (43, 1677, 1799), (44, 1678, 1797), (44, 1679, 1797), (44, 1680, 1797), (44, 1681, 1796), (44, 1682, 1796), (44, 1683, 1796), (45, 1684, 1794), (45, 1685, 1794), (45, 1686, 1794), (45, 1687, 1793), (45, 1688, 1793), (45, 1689, 1793), (45, 1690, 1792), (46, 1691, 1791), (46, 1692, 1791), (46, 1693, 1790), (46, 1694, 1790), (46, 1695, 1789), (46, 1696, 1789), (47, 1697, 1788), (47, 1698, 1787), (47, 1699, 1787), (47, 1700, 1786), (47, 1701, 1786), (47, 1702, 1786), (47, 1703, 1785), (48, 1704, 1784), (48, 1705, 1783), (48, 1706, 1783), (48, 1707, 1782), (48, 1708, 1782), (48, 1709, 1782), (48, 1710, 1781), (49, 1711, 1780), (49, 1712, 1779), (49, 1713, 1779), (49, 1714, 1778), (49, 1715, 1778), (49, 1716, 1778), (49, 1717, 1777), (50, 1718, 1776), (50, 1719, 1775), (50, 1720, 1775), (50, 1721, 1775), (50, 1722, 1774), (50, 1723, 1774), (50, 1724, 1773), (51, 1725, 1772), (51, 1726, 1772), (51, 1727, 1771), (51, 1728, 1771), (51, 1729, 1771), (51, 1730, 1770), (51, 1731, 1770), (51, 1732, 1770), (51, 1733, 1769), (51, 1734, 1769), (51, 1735, 1769), (51, 1736, 1768), (51, 1737, 1768), (51, 1738, 1768), (51, 1739, 1767), (51, 1740, 1767), (51, 1741, 1767), (51, 1742, 1766), (51, 1743, 1766), (51, 1744, 1766), (51, 1745, 1766), (51, 1746, 1765), (51, 1747, 1765), (51, 1748, 1765), (51, 1749, 1764), (51, 1750, 1764), (51, 1751, 1764), (51, 1752, 1763), (51, 1753, 1763), (51, 1754, 1763), (51, 1755, 1763), (51, 1756, 1762), (51, 1757, 1762), (51, 1758, 1762), (51, 1759, 1761), (51, 1760, 1761), (51, 1761, 1761), (51, 1762, 1761), (51, 1763, 1760), (51, 1764, 1760), (51, 1765, 1760), (51, 1766, 1759), (51, 1767, 1759), (51, 1768, 1759), (51, 1769, 1759), (51, 1770, 1758), (51, 1771, 1758), (51, 1772, 1758), (51, 1773, 1758), (51, 1774, 1757), (51, 1775, 1757), (51, 1776, 1757), (51, 1777, 1757), (51, 1778, 1756), (51, 1779, 1756), (51, 1780, 1756), (51, 1781, 1756), (51, 1782, 1755), (51, 1783, 1755), (51, 1784, 1755), (51, 1785, 1755), (51, 1786, 1754), (51, 1787, 1754), (51, 1788, 1754), (51, 1789, 1754), (51, 1790, 1753), (51, 1791, 1753), (51, 1792, 1753), (51, 1793, 1753), (51, 1794, 1752), (51, 1795, 1752), (51, 1796, 1752), (51, 1797, 1752), (51, 1798, 1751), (51, 1799, 1751), (51, 1800, 1751), (51, 1801, 1751), (51, 1802, 1750), (51, 1803, 1750), (51, 1804, 1750), (51, 1805, 1750), (51, 1806, 1750), (52, 1807, 1748), (52, 1808, 1748), (53, 1809, 1747), (53, 1810, 1746), (53, 1811, 1746), (54, 1812, 1745), (54, 1813, 1744), (55, 1814, 1743), (55, 1815, 1743), (55, 1816, 1743), (56, 1817, 1741), (56, 1818, 1741), (56, 1819, 1741), (57, 1820, 1739), (57, 1821, 1739), (58, 1822, 1738), (58, 1823, 1737), (58, 1824, 1737), (59, 1825, 1736), (59, 1826, 1735), (60, 1827, 1734), (60, 1828, 1734), (60, 1829, 1734), (61, 1830, 1732), (61, 1831, 1732), (61, 1832, 1732), (62, 1833, 1730), (62, 1834, 1730), (62, 1835, 1730), (63, 1836, 1728), (63, 1837, 1728), (64, 1838, 1727), (64, 1839, 1726), (64, 1840, 1726), (65, 1841, 1725), (65, 1842, 1724), (65, 1843, 1724), (66, 1844, 1723), (66, 1845, 1722), (67, 1846, 1721), (67, 1847, 1721), (67, 1848, 1720), (68, 1849, 1719), (68, 1850, 1719), (68, 1851, 1718), (69, 1852, 1717), (69, 1853, 1717), (69, 1854, 1716), (70, 1855, 1715), (70, 1856, 1714), (70, 1857, 1714), (71, 1858, 1713), (71, 1859, 1712), (71, 1860, 1712), (72, 1861, 1711), (72, 1862, 1710), (73, 1863, 1709), (73, 1864, 1709), (73, 1865, 1708), (74, 1866, 1707), (74, 1867, 1706), (74, 1868, 1706), (75, 1869, 1705), (75, 1870, 1704), (75, 1871, 1704), (76, 1872, 1703), (76, 1873, 1702), (76, 1874, 1702), (77, 1875, 1700), (77, 1876, 1700), (77, 1877, 1700), (78, 1878, 1698), (78, 1879, 1698), (78, 1880, 1698), (79, 1881, 1696), (79, 1882, 1696), (79, 1883, 1695), (80, 1884, 1694), (80, 1885, 1693), (80, 1886, 1693), (80, 1887, 1693), (81, 1888, 1691), (81, 1889, 1691), (81, 1890, 1690), (82, 1891, 1689), (82, 1892, 1688), (82, 1893, 1688), (83, 1894, 1686), (83, 1895, 1686), (83, 1896, 1685), (84, 1897, 1684), (84, 1898, 1683), (84, 1899, 1683), (85, 1900, 1681), (85, 1901, 1681), (85, 1902, 1680), (86, 1903, 1679), (86, 1904, 1678), (86, 1905, 1678), (87, 1906, 1676), (87, 1907, 1676), (87, 1908, 1675), (88, 1909, 1674), (88, 1910, 1673), (88, 1911, 1673), (89, 1912, 1671), (89, 1913, 1671), (90, 1914, 1669), (90, 1915, 1668), (90, 1916, 1668), (91, 1917, 1666), (91, 1918, 1666), (92, 1919, 1664), (92, 1920, 1663), (92, 1921, 1663), (93, 1922, 1661), (93, 1923, 1661), (94, 1924, 1659), (94, 1925, 1658), (95, 1926, 1657), (95, 1927, 1656), (95, 1928, 1655), (96, 1929, 1654), (96, 1930, 1653), (97, 1931, 1651), (97, 1932, 1651), (98, 1933, 1649), (98, 1934, 1648), (99, 1935, 1647), (99, 1936, 1646), (100, 1937, 1644), (100, 1938, 1644), (101, 1939, 1642), (101, 1940, 1641), (102, 1941, 1640), (102, 1942, 1639), (103, 1943, 1637), (103, 1944, 1637), (104, 1945, 1635), (105, 1946, 1633), (105, 1947, 1633), (106, 1948, 1631), (106, 1949, 1631), (107, 1950, 1629), (108, 1951, 1627), (108, 1952, 1627), (109, 1953, 1625), (109, 1954, 1625), (110, 1955, 1623), (111, 1956, 1621), (111, 1957, 1621), (112, 1958, 1619), (113, 1959, 1617), (114, 1960, 1615), (114, 1961, 1614), (115, 1962, 1612), (116, 1963, 1611), (117, 1964, 1609), (118, 1965, 79), (205, 1965, 1520), (118, 1966, 62), (212, 1966, 1512), (119, 1967, 45), (218, 1967, 1505), (120, 1968, 30), (225, 1968, 1497), (121, 1969, 16), (232, 1969, 1489), (122, 1970, 3), (239, 1970, 1481), (246, 1971, 1473), (254, 1972, 1464), (262, 1973, 1455), (270, 1974, 1446), (278, 1975, 1437), (287, 1976, 1427), (296, 1977, 1417), (306, 1978, 1406), (317, 1979, 1394), (329, 1980, 1381), (342, 1981, 1367), (356, 1982, 1352), (368, 1983, 1339), (373, 1984, 1332), (378, 1985, 1326), (384, 1986, 1319), (389, 1987, 1313), (395, 1988, 1306), (400, 1989, 1300), (406, 1990, 1292), (412, 1991, 1285), (418, 1992, 1278), (423, 1993, 1272), (429, 1994, 1264), (435, 1995, 1257), (441, 1996, 1250), (448, 1997, 1242), (453, 1998, 1235), (458, 1999, 1229), (463, 2000, 1222), (468, 2001, 1216), (472, 2002, 1211), (477, 2003, 1204), (482, 2004, 1198), (487, 2005, 1191), (492, 2006, 1185), (496, 2007, 1179), (501, 2008, 1173), (506, 2009, 1166), (510, 2010, 1161), (515, 2011, 1154), (519, 2012, 1131), (524, 2013, 1099), (528, 2014, 1066), (532, 2015, 1053), (536, 2016, 1046), (540, 2017, 1039), (544, 2018, 1033), (547, 2019, 1027), (551, 2020, 1020), (554, 2021, 1015), (558, 2022, 1008), (561, 2023, 1003), (564, 2024, 997), (568, 2025, 991), (571, 2026, 986), (574, 2027, 980), (577, 2028, 975), (580, 2029, 970), (583, 2030, 965), (585, 2031, 961), (588, 2032, 956), (591, 2033, 951), (593, 2034, 947), (595, 2035, 943), (597, 2036, 940), (600, 2037, 935), (602, 2038, 931), (605, 2039, 926), (607, 2040, 922), (610, 2041, 916), (612, 2042, 912), (614, 2043, 908), (616, 2044, 904), (618, 2045, 899), (620, 2046, 895), (623, 2047, 890), (625, 2048, 885), (627, 2049, 880), (630, 2050, 870), (632, 2051, 855), (634, 2052, 839), (637, 2053, 822), (639, 2054, 805), (642, 2055, 786), (645, 2056, 775), (647, 2057, 768), (650, 2058, 759), (653, 2059, 750), (656, 2060, 741), (659, 2061, 733), (662, 2062, 724), (665, 2063, 715), (668, 2064, 706), (672, 2065, 696), (675, 2066, 686), (678, 2067, 677), (682, 2068, 667), (685, 2069, 657), (689, 2070, 646), (693, 2071, 635), (697, 2072, 624), (702, 2073, 611), (706, 2074, 600), (711, 2075, 588), (715, 2076, 577), (719, 2077, 566), (723, 2078, 555), (727, 2079, 544), (731, 2080, 532), (734, 2081, 525), (738, 2082, 516), (742, 2083, 509), (745, 2084, 502), (749, 2085, 494), (752, 2086, 487), (756, 2087, 480), (759, 2088, 473), (763, 2089, 466), (766, 2090, 460), (769, 2091, 453), (772, 2092, 447), (775, 2093, 441), (778, 2094, 435), (780, 2095, 430), (783, 2096, 425), (786, 2097, 419), (788, 2098, 414), (791, 2099, 409), (793, 2100, 404), (796, 2101, 398), (798, 2102, 394), (800, 2103, 390), (803, 2104, 384), (805, 2105, 380), (807, 2106, 376), (810, 2107, 370), (812, 2108, 366), (814, 2109, 362), (817, 2110, 357), (819, 2111, 353), (822, 2112, 347), (825, 2113, 342), (827, 2114, 337), (830, 2115, 332), (833, 2116, 327), (836, 2117, 321), (839, 2118, 315), (842, 2119, 310), (845, 2120, 304), (848, 2121, 298), (852, 2122, 292), (855, 2123, 286), (859, 2124, 279), (863, 2125, 272), (867, 2126, 265), (872, 2127, 257), (876, 2128, 250), (880, 2129, 243), (885, 2130, 235), (889, 2131, 228), (894, 2132, 220), (898, 2133, 213), (903, 2134, 204), (907, 2135, 197), (912, 2136, 188), (917, 2137, 177), (922, 2138, 163), (926, 2139, 150), (931, 2140, 136), (937, 2141, 120), (955, 2142, 93), (973, 2143, 66), (992, 2144, 37), (1012, 2145, 8)], ['937,2141,855,2123,777,2093,668,2064,467,2000,367,1982,217,1966,122,1970,95,1928,51,1806,51,1725,40,1659,40,1438,29,1277,30,910,22,594,47,444,97,295,120,269,245,191,419,134,525,113,652,111,1426,112,1569,134,1648,154,1832,180,1910,204,2017,290,2059,369,2114,443,2166,665,2152,822,2127,898,2119,974,2093,1050,2046,1108,2007,1199,1959,1275,1939,1350,1885,1428,1847,1658,1772,1887,1726,1963,1668,2011,1585,2014,1506,2049,1428,2054,1222,2090,1099,2136'])], 'temp/1740710173_2762130_917877156_a9c2d4b99270c9302def4ed40606e685.jpg']} nb pixel non reg : 3692295 nb pixel common : 3678042 proportion of common points : 0.9961397992305598 [('test release memory', 'SUCCESS', True), ('test detect objet', 'SUCCESS', True), ('test polygone', 'SUCCESS', True)] res_total : True #&_# TEST SUCCEEDED #&_# : tests/mask_test #&_# /home/admin/workarea/git/Velours/python/tests/python_tests.py refs/heads/master_3bb8fc9eb89f6e73213a93d2f1429765ec1e113a SQL :INSERT INTO MTRAdmin.monitor_sys (name, type, server, version_code, result_str, result_bool, lien , test_group ,test_name) VALUES ('python_test3','1','marlene','refs/heads/master_3bb8fc9eb89f6e73213a93d2f1429765ec1e113a','{"mask_detection": "success"}','1','http://marlene.fotonower-preprod.com/job/2025/February/28022025/python_test3//data_2/data_log/job/2025/February/28022025/python_test3/log-python3----short_python3--v--marlene-03:35:00.txt','mask_detection','unknown'); #&_# END OF TEST #&_# : tests/mask_test #&_# #&_# BEGIN OF TEST : tests/datou_test #&_# /home/admin/workarea/git/Velours/python/tests/datou_test.py Datou All Test python version used : 3 ############################### TEST sam ################################ TEST SAM Inside batchDatouExec : verbose : True ##### chargement datou SELECT name, created_at,limit_max FROM MTRDatou.mtr_datou WHERE id=4573 SELECT mtd.id, mtdt.`type`, mtd.`param`, mtd.param_json, mtdt.nb_input, mtdt.nb_output, mtdt.prod, mtdt.is_local, mtdt.is_datou_depend, mtdt.is_photo_id_local FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_step_types mtdt WHERE mtdt.`id`=mtd.`type` AND mtd.mtd_id=4573 SELECT mtd.id, mtd.mtd_id, mdsdt.id, mdsdt.name, mdsdt.description, msid.output_or_input, msid.data_order_id, mdsdt.type FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_steptype_io_datatypes msid, MTRDatou.mtr_datou_step_data_types mdsdt WHERE mtd.`type`=msid.`mtr_datou_step_type` AND mtd.mtd_id= 4573 AND msid.data_type=mdsdt.id SELECT mts_id_output, id_output, mts_id_input, id_input FROM MTRDatou.mtr_datou_step_by_step WHERE mtd_id=4573 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! no param json to modify List Step Type Loaded in datou : sam list_input_json : [] ##### fin chargement datou ##### chargement data ##### Call load_data_input : nb_thread : 5 origin SELECT photo_id, url FROM MTRBack.photos ph WHERE photo_id IN (1189321094) Found this number of photos: 1 ##### Call download_photos : nb_thread : 5 begin to download photo : 1189321094 download finish for photo 1189321094 we have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB ##### After download_photos length of list_filenames : 1 ; length of list_pids : 1 ; length of list_args : 1 ##### After load_data_input time to download the photos : 0.44260501861572266 #### fin chargement data Blocking on flush ? No conitnuing About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! WARNING : we have an input that is not a photo, we should get rid of it Calling datou_exec Inside datou_exec : verbose : True number of steps : 1 step1:sam Fri Feb 28 03:36:41 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec After prepare type args : Here we display some param of map_info ! map_filenames : {'temp/1740710201_2762130_1189321094_9626af7f95d010f2a4fd524688d4ea22_76896585.png': 1189321094} map_photo_id_path_extension : {1189321094: {'path': 'temp/1740710201_2762130_1189321094_9626af7f95d010f2a4fd524688d4ea22_76896585.png', 'extension': 'png'}} map_subphoto_mainphoto : {} Beginning of datou step sam ! pht : 4677 Inside sam : nb paths : 1 (640, 960, 3) time for calcul the mask position with numpy : 0.002562284469604492 nb_pixel_total : 14745 time to create 1 rle with old method : 0.021999120712280273 time for calcul the mask position with numpy : 0.002122640609741211 nb_pixel_total : 10777 time to create 1 rle with old method : 0.016869544982910156 time for calcul the mask position with numpy : 0.0018482208251953125 nb_pixel_total : 5613 time to create 1 rle with old method : 0.008159399032592773 time for calcul the mask position with numpy : 0.001886129379272461 nb_pixel_total : 13966 time to create 1 rle with old method : 0.02036881446838379 time for calcul the mask position with numpy : 0.002621173858642578 nb_pixel_total : 83494 time to create 1 rle with old method : 0.10358858108520508 time for calcul the mask position with numpy : 0.0014765262603759766 nb_pixel_total : 3769 time to create 1 rle with old method : 0.004481077194213867 time for calcul the mask position with numpy : 0.0014386177062988281 nb_pixel_total : 2949 time to create 1 rle with old method : 0.0035851001739501953 time for calcul the mask position with numpy : 0.0016083717346191406 nb_pixel_total : 29434 time to create 1 rle with old method : 0.035780906677246094 time for calcul the mask position with numpy : 0.0014543533325195312 nb_pixel_total : 4273 time to create 1 rle with old method : 0.0050847530364990234 time for calcul the mask position with numpy : 0.0015192031860351562 nb_pixel_total : 9877 time to create 1 rle with old method : 0.015613317489624023 time for calcul the mask position with numpy : 0.0019116401672363281 nb_pixel_total : 1227 time to create 1 rle with old method : 0.002243518829345703 time for calcul the mask position with numpy : 0.002107381820678711 nb_pixel_total : 577 time to create 1 rle with old method : 0.001066446304321289 time for calcul the mask position with numpy : 0.0017917156219482422 nb_pixel_total : 2452 time to create 1 rle with old method : 0.003919363021850586 time for calcul the mask position with numpy : 0.0021588802337646484 nb_pixel_total : 27823 time to create 1 rle with old method : 0.041326045989990234 time for calcul the mask position with numpy : 0.0015437602996826172 nb_pixel_total : 6634 time to create 1 rle with old method : 0.007917165756225586 time for calcul the mask position with numpy : 0.001584768295288086 nb_pixel_total : 13116 time to create 1 rle with old method : 0.023275375366210938 time for calcul the mask position with numpy : 0.0018157958984375 nb_pixel_total : 3952 time to create 1 rle with old method : 0.005630016326904297 time for calcul the mask position with numpy : 0.001791238784790039 nb_pixel_total : 16361 time to create 1 rle with old method : 0.02103447914123535 time for calcul the mask position with numpy : 0.0015354156494140625 nb_pixel_total : 4464 time to create 1 rle with old method : 0.005732059478759766 time for calcul the mask position with numpy : 0.0014493465423583984 nb_pixel_total : 2080 time to create 1 rle with old method : 0.0024566650390625 time for calcul the mask position with numpy : 0.0014963150024414062 nb_pixel_total : 7505 time to create 1 rle with old method : 0.008765220642089844 time for calcul the mask position with numpy : 0.0014450550079345703 nb_pixel_total : 1686 time to create 1 rle with old method : 0.0020961761474609375 time for calcul the mask position with numpy : 0.0014576911926269531 nb_pixel_total : 4272 time to create 1 rle with old method : 0.005324363708496094 time for calcul the mask position with numpy : 0.0014758110046386719 nb_pixel_total : 5472 time to create 1 rle with old method : 0.006750822067260742 time for calcul the mask position with numpy : 0.0015149116516113281 nb_pixel_total : 12750 time to create 1 rle with old method : 0.015544652938842773 time for calcul the mask position with numpy : 0.00150299072265625 nb_pixel_total : 8571 time to create 1 rle with old method : 0.010016918182373047 time for calcul the mask position with numpy : 0.0014271736145019531 nb_pixel_total : 1121 time to create 1 rle with old method : 0.0014014244079589844 time for calcul the mask position with numpy : 0.0014574527740478516 nb_pixel_total : 2765 time to create 1 rle with old method : 0.003401041030883789 time for calcul the mask position with numpy : 0.001417398452758789 nb_pixel_total : 1083 time to create 1 rle with old method : 0.001422882080078125 time for calcul the mask position with numpy : 0.001428365707397461 nb_pixel_total : 1320 time to create 1 rle with old method : 0.0017282962799072266 time for calcul the mask position with numpy : 0.0014543533325195312 nb_pixel_total : 3509 time to create 1 rle with old method : 0.0041751861572265625 time for calcul the mask position with numpy : 0.0014390945434570312 nb_pixel_total : 5551 time to create 1 rle with old method : 0.0067691802978515625 time for calcul the mask position with numpy : 0.0016243457794189453 nb_pixel_total : 38980 time to create 1 rle with old method : 0.046373844146728516 time for calcul the mask position with numpy : 0.0014476776123046875 nb_pixel_total : 1548 time to create 1 rle with old method : 0.001979351043701172 time for calcul the mask position with numpy : 0.0014376640319824219 nb_pixel_total : 2811 time to create 1 rle with old method : 0.0034339427947998047 time for calcul the mask position with numpy : 0.0015244483947753906 nb_pixel_total : 16407 time to create 1 rle with old method : 0.018879175186157227 time for calcul the mask position with numpy : 0.0014705657958984375 nb_pixel_total : 2381 time to create 1 rle with old method : 0.0029573440551757812 time for calcul the mask position with numpy : 0.0014429092407226562 nb_pixel_total : 3335 time to create 1 rle with old method : 0.003942251205444336 time for calcul the mask position with numpy : 0.001444101333618164 nb_pixel_total : 2731 time to create 1 rle with old method : 0.003358602523803711 time for calcul the mask position with numpy : 0.0014595985412597656 nb_pixel_total : 7783 time to create 1 rle with old method : 0.009034872055053711 time for calcul the mask position with numpy : 0.0014469623565673828 nb_pixel_total : 2451 time to create 1 rle with old method : 0.0029647350311279297 time for calcul the mask position with numpy : 0.0014951229095458984 nb_pixel_total : 11958 time to create 1 rle with old method : 0.014002084732055664 time for calcul the mask position with numpy : 0.001443624496459961 nb_pixel_total : 4193 time to create 1 rle with old method : 0.005049228668212891 time for calcul the mask position with numpy : 0.0014698505401611328 nb_pixel_total : 10625 time to create 1 rle with old method : 0.012347936630249023 time for calcul the mask position with numpy : 0.0016262531280517578 nb_pixel_total : 39058 time to create 1 rle with old method : 0.04589104652404785 time for calcul the mask position with numpy : 0.001489877700805664 nb_pixel_total : 1035 time to create 1 rle with old method : 0.0013194084167480469 time for calcul the mask position with numpy : 0.0014567375183105469 nb_pixel_total : 4037 time to create 1 rle with old method : 0.006056785583496094 time for calcul the mask position with numpy : 0.0022001266479492188 nb_pixel_total : 1648 time to create 1 rle with old method : 0.0038564205169677734 time for calcul the mask position with numpy : 0.002620220184326172 nb_pixel_total : 344 time to create 1 rle with old method : 0.0009517669677734375 time for calcul the mask position with numpy : 0.002533435821533203 nb_pixel_total : 1244 time to create 1 rle with old method : 0.0036764144897460938 time for calcul the mask position with numpy : 0.002364635467529297 nb_pixel_total : 3860 time to create 1 rle with old method : 0.008262395858764648 time for calcul the mask position with numpy : 0.0019004344940185547 nb_pixel_total : 875 time to create 1 rle with old method : 0.001756906509399414 time for calcul the mask position with numpy : 0.0018961429595947266 nb_pixel_total : 874 time to create 1 rle with old method : 0.0016734600067138672 time for calcul the mask position with numpy : 0.002177715301513672 nb_pixel_total : 2322 time to create 1 rle with old method : 0.0043146610260009766 time for calcul the mask position with numpy : 0.0018231868743896484 nb_pixel_total : 1208 time to create 1 rle with old method : 0.0020351409912109375 time for calcul the mask position with numpy : 0.0019254684448242188 nb_pixel_total : 598 time to create 1 rle with old method : 0.0009527206420898438 time for calcul the mask position with numpy : 0.0016868114471435547 nb_pixel_total : 889 time to create 1 rle with old method : 0.0012955665588378906 time for calcul the mask position with numpy : 0.001979351043701172 nb_pixel_total : 713 time to create 1 rle with old method : 0.00121307373046875 time for calcul the mask position with numpy : 0.002022981643676758 nb_pixel_total : 695 time to create 1 rle with old method : 0.0012004375457763672 time for calcul the mask position with numpy : 0.0016639232635498047 nb_pixel_total : 337 time to create 1 rle with old method : 0.0004787445068359375 time for calcul the mask position with numpy : 0.001508474349975586 nb_pixel_total : 13365 time to create 1 rle with old method : 0.01894545555114746 time for calcul the mask position with numpy : 0.0018813610076904297 nb_pixel_total : 2410 time to create 1 rle with old method : 0.0033872127532958984 time for calcul the mask position with numpy : 0.001676797866821289 nb_pixel_total : 1672 time to create 1 rle with old method : 0.0025620460510253906 time for calcul the mask position with numpy : 0.0016756057739257812 nb_pixel_total : 3105 time to create 1 rle with old method : 0.004991292953491211 time for calcul the mask position with numpy : 0.0019516944885253906 nb_pixel_total : 1207 time to create 1 rle with old method : 0.0018973350524902344 time for calcul the mask position with numpy : 0.001840353012084961 nb_pixel_total : 584 time to create 1 rle with old method : 0.0009534358978271484 time for calcul the mask position with numpy : 0.0016293525695800781 nb_pixel_total : 1078 time to create 1 rle with old method : 0.0016508102416992188 time for calcul the mask position with numpy : 0.0018031597137451172 nb_pixel_total : 8605 time to create 1 rle with old method : 0.014135360717773438 time for calcul the mask position with numpy : 0.0015840530395507812 nb_pixel_total : 3205 time to create 1 rle with old method : 0.004406452178955078 time for calcul the mask position with numpy : 0.0015382766723632812 nb_pixel_total : 913 time to create 1 rle with old method : 0.001247406005859375 time for calcul the mask position with numpy : 0.0017216205596923828 nb_pixel_total : 16687 time to create 1 rle with old method : 0.0230104923248291 time for calcul the mask position with numpy : 0.0016815662384033203 nb_pixel_total : 1750 time to create 1 rle with old method : 0.0024743080139160156 time for calcul the mask position with numpy : 0.0017535686492919922 nb_pixel_total : 18608 time to create 1 rle with old method : 0.02319645881652832 time for calcul the mask position with numpy : 0.0014848709106445312 nb_pixel_total : 8448 time to create 1 rle with old method : 0.009703397750854492 time for calcul the mask position with numpy : 0.0013256072998046875 nb_pixel_total : 267 time to create 1 rle with old method : 0.00033092498779296875 time for calcul the mask position with numpy : 0.0013616085052490234 nb_pixel_total : 9085 time to create 1 rle with old method : 0.010503768920898438 time for calcul the mask position with numpy : 0.0014309883117675781 nb_pixel_total : 1526 time to create 1 rle with old method : 0.0018863677978515625 time for calcul the mask position with numpy : 0.00142669677734375 nb_pixel_total : 963 time to create 1 rle with old method : 0.0012769699096679688 time for calcul the mask position with numpy : 0.0014564990997314453 nb_pixel_total : 7509 time to create 1 rle with old method : 0.008763551712036133 time for calcul the mask position with numpy : 0.0014336109161376953 nb_pixel_total : 830 time to create 1 rle with old method : 0.0010445117950439453 time for calcul the mask position with numpy : 0.0014693737030029297 nb_pixel_total : 9507 time to create 1 rle with old method : 0.011022090911865234 time for calcul the mask position with numpy : 0.001322031021118164 nb_pixel_total : 270 time to create 1 rle with old method : 0.00045800209045410156 time for calcul the mask position with numpy : 0.0014238357543945312 nb_pixel_total : 2262 time to create 1 rle with old method : 0.0026400089263916016 time for calcul the mask position with numpy : 0.0014233589172363281 nb_pixel_total : 970 time to create 1 rle with old method : 0.001220703125 time for calcul the mask position with numpy : 0.0014319419860839844 nb_pixel_total : 615 time to create 1 rle with old method : 0.0008037090301513672 time for calcul the mask position with numpy : 0.0014369487762451172 nb_pixel_total : 4271 time to create 1 rle with old method : 0.005350351333618164 time for calcul the mask position with numpy : 0.00142669677734375 nb_pixel_total : 220 time to create 1 rle with old method : 0.00032210350036621094 time for calcul the mask position with numpy : 0.001474142074584961 nb_pixel_total : 1502 time to create 1 rle with old method : 0.001859426498413086 time for calcul the mask position with numpy : 0.001428365707397461 nb_pixel_total : 1421 time to create 1 rle with old method : 0.0018358230590820312 time for calcul the mask position with numpy : 0.0014338493347167969 nb_pixel_total : 1631 time to create 1 rle with old method : 0.002065420150756836 time for calcul the mask position with numpy : 0.0013458728790283203 nb_pixel_total : 735 time to create 1 rle with old method : 0.0010221004486083984 time for calcul the mask position with numpy : 0.0014181137084960938 nb_pixel_total : 595 time to create 1 rle with old method : 0.0008077621459960938 time for calcul the mask position with numpy : 0.001497507095336914 nb_pixel_total : 11128 time to create 1 rle with old method : 0.01299738883972168 time for calcul the mask position with numpy : 0.0014042854309082031 nb_pixel_total : 293 time to create 1 rle with old method : 0.0004165172576904297 time for calcul the mask position with numpy : 0.0013720989227294922 nb_pixel_total : 914 time to create 1 rle with old method : 0.0013191699981689453 time for calcul the mask position with numpy : 0.0013854503631591797 nb_pixel_total : 2201 time to create 1 rle with old method : 0.0027730464935302734 time for calcul the mask position with numpy : 0.0014235973358154297 nb_pixel_total : 193 time to create 1 rle with old method : 0.00032019615173339844 time for calcul the mask position with numpy : 0.001415252685546875 nb_pixel_total : 886 time to create 1 rle with old method : 0.0011441707611083984 time for calcul the mask position with numpy : 0.0014414787292480469 nb_pixel_total : 946 time to create 1 rle with old method : 0.001176595687866211 time for calcul the mask position with numpy : 0.001394033432006836 nb_pixel_total : 480 time to create 1 rle with old method : 0.0006580352783203125 time for calcul the mask position with numpy : 0.001447916030883789 nb_pixel_total : 889 time to create 1 rle with old method : 0.001146554946899414 time for calcul the mask position with numpy : 0.0014510154724121094 nb_pixel_total : 1323 time to create 1 rle with old method : 0.0017075538635253906 time for calcul the mask position with numpy : 0.001438140869140625 nb_pixel_total : 1424 time to create 1 rle with old method : 0.0018749237060546875 time for calcul the mask position with numpy : 0.0014214515686035156 nb_pixel_total : 333 time to create 1 rle with old method : 0.0004642009735107422 time for calcul the mask position with numpy : 0.0014193058013916016 nb_pixel_total : 1206 time to create 1 rle with old method : 0.0015435218811035156 time for calcul the mask position with numpy : 0.0014400482177734375 nb_pixel_total : 828 time to create 1 rle with old method : 0.0010900497436523438 insert ignore into MTRPhoto.crop_hashtag_ids (photo_id, hashtag_id, `type`,x0,x1,y0,y1,score) VALUES (%s,%s,%s,%s,%s,%s,%s,%s) insert ignore into MTRPhoto.crop_hashtag_ids (photo_id, hashtag_id, `type`,x0,x1,y0,y1,score) VALUES (%s,%s,%s,%s,%s,%s,%s,%s) batch 1 Loaded 106 chid ids of type : 4677 Number RLEs to save : 9871 INSERT IGNORE INTO MTRPhoto.crop_segments (`crop_hashtag_id`, `x0`, `y0`, `length`) VALUES (%s, %s, %s , %s) first line : ('3689476778', '642', '86', '6') ... last line : ('3689476883', '815', '44', '5') INSERT IGNORE INTO MTRPhoto.crop_sum_segments (`crop_hashtag_id`, `sum_segments`) VALUES (%s, %s) TO DO : save crop sub photo not yet done ! After datou_step_exec type output : map_portfolio_photo : len 0 keys : dict_keys([]) Inside saveOutput : final : True verbose : True saveOutput not yet implemented for datou_step.type : sam we use saveGeneral [1189321094] map_info['map_portfolio_photo'] : {} final : True mtd_id 4573 list_pids : [1189321094] Looping around the photos to save general results len do output : 1 /1189321094Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4573', None, None, None, None, None, None, None, None) ('4573', None, '1189321094', None, None, None, None, None, None) begin to insert list_values into mtr_datou_result : length of list_values in save_final : 3 insert ignore into MTRPhoto.mtr_datou_result (mtd_id, mtr_portfolio_id,mtr_photo_id,result,result_long,result_double,hashtag_id,proba, mtr_current_id) values (%s,%s,%s,%s,%s,%s,%s,%s,%s) on duplicate key update mtr_portfolio_id = mtr_portfolio_id list_values : [('4573', None, '1189321094', 'None', None, None, None, None, None)] time used for this insertion : 0.0168612003326416 save_final save missing photos in datou_result : time spend for datou_step_exec : 11.389883041381836 time spend to save output : 0.01723480224609375 total time spend for step 1 : 11.40711784362793 caffe_path_current : About to save ! 2 After save, about to update current ! datou_cur_ids : [] len(datou.list_steps) : 1 output : {'1189321094': [[, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ], 'temp/1740710201_2762130_1189321094_9626af7f95d010f2a4fd524688d4ea22_76896585.png']} nb_objects detect : 106 ############################### TEST frcnn ################################ test frcnn Inside batchDatouExec : verbose : True ##### chargement datou SELECT name, created_at,limit_max FROM MTRDatou.mtr_datou WHERE id=4184 SELECT mtd.id, mtdt.`type`, mtd.`param`, mtd.param_json, mtdt.nb_input, mtdt.nb_output, mtdt.prod, mtdt.is_local, mtdt.is_datou_depend, mtdt.is_photo_id_local FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_step_types mtdt WHERE mtdt.`id`=mtd.`type` AND mtd.mtd_id=4184 SELECT mtd.id, mtd.mtd_id, mdsdt.id, mdsdt.name, mdsdt.description, msid.output_or_input, msid.data_order_id, mdsdt.type FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_steptype_io_datatypes msid, MTRDatou.mtr_datou_step_data_types mdsdt WHERE mtd.`type`=msid.`mtr_datou_step_type` AND mtd.mtd_id= 4184 AND msid.data_type=mdsdt.id SELECT mts_id_output, id_output, mts_id_input, id_input FROM MTRDatou.mtr_datou_step_by_step WHERE mtd_id=4184 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! no param json to modify List Step Type Loaded in datou : frcnn list_input_json : [] ##### fin chargement datou ##### chargement data ##### Call load_data_input : nb_thread : 5 origin SELECT photo_id, url FROM MTRBack.photos ph WHERE photo_id IN (917754606) Found this number of photos: 1 ##### Call download_photos : nb_thread : 5 begin to download photo : 917754606 download finish for photo 917754606 we have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB ##### After download_photos length of list_filenames : 1 ; length of list_pids : 1 ; length of list_args : 1 ##### After load_data_input time to download the photos : 0.1535797119140625 #### fin chargement data Blocking on flush ? No conitnuing About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : True number of steps : 1 step1:frcnn Fri Feb 28 03:36:53 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec After prepare type args : Here we display some param of map_info ! map_filenames : {'temp/1740710213_2762130_917754606_35f3c9ae49686a6be16030c6ec25c9ee.jpg': 917754606} map_photo_id_path_extension : {917754606: {'path': 'temp/1740710213_2762130_917754606_35f3c9ae49686a6be16030c6ec25c9ee.jpg', 'extension': 'jpg'}} map_subphoto_mainphoto : {} Beginning of datou step Faster rcnn ! classes : ['background', 'plaque'] pht : 4370 caffemodel_name (should be vgg16_immat_307 but not used because net loaded outside in the fonction) : {'id': 3375, 'mtr_user_id': 31, 'name': 'detection_plaque_valcor_010622', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,plaque', 'svm_portfolios_learning': '0,0', 'photo_hashtag_type': 4370, 'photo_desc_type': 5676, 'type_classification': 'caffe_faster_rcnn', 'hashtag_id_list': '0,0'} To loadFromThcl() model_param file didn't exist model_name : detection_plaque_valcor_010622 model_type : caffe_faster_rcnn list file need : ['caffemodel', 'test.prototxt'] file exist in s3 : ['caffemodel', 'test.prototxt'] file manque in s3 : [] local folder : /data/models_weight/detection_plaque_valcor_010622 /data/models_weight/detection_plaque_valcor_010622/caffemodel size_local : 349723073 size in s3 : 349723073 create time local : 2022-07-12 14:12:27 create time in s3 : 2022-06-01 15:05:56 caffemodel already exist and didn't need to update /data/models_weight/detection_plaque_valcor_010622/test.prototxt size_local : 7163 size in s3 : 7163 create time local : 2022-07-12 14:12:27 create time in s3 : 2022-06-01 15:05:55 test.prototxt already exist and didn't need to update prototxt : /data/models_weight/detection_plaque_valcor_010622/test.prototxt caffemodel : /data/models_weight/detection_plaque_valcor_010622/caffemodel Loaded network /data/models_weight/detection_plaque_valcor_010622/caffemodel About to compute detect_faster_rcnn : len(args) : 1 Inside frcnn step exec : nb paths : 1 image_path : temp/1740710213_2762130_917754606_35f3c9ae49686a6be16030c6ec25c9ee.jpg image_size (600, 800, 3) [[[ 4 6 6] [ 5 7 7] [ 6 8 8] ... [207 215 214] [206 214 213] [206 214 213]] [[ 4 6 6] [ 5 7 7] [ 6 8 8] ... [207 215 214] [206 214 213] [206 214 213]] [[ 4 6 6] [ 5 7 7] [ 6 8 8] ... [207 215 214] [206 214 213] [206 214 213]] ... [[ 14 16 16] [ 13 15 15] [ 11 13 13] ... [198 206 205] [198 206 205] [198 206 205]] [[ 16 18 18] [ 14 16 16] [ 11 13 13] ... [206 214 213] [206 214 213] [206 214 213]] [[ 13 15 15] [ 12 14 14] [ 9 11 11] ... [210 218 217] [210 218 217] [210 218 217]]] Detection took 0.111s for 300 object proposals c : plaque list_crops.shape (72, 5) proba : 0.06384736 (374.12915, 293.91763, 430.81635, 317.80783) proba : 0.052204903 (382.1774, 297.18784, 552.35486, 344.65823) proba : 0.012277189 (345.3615, 272.43155, 468.8557, 320.72464) We are managing local photo_id len de result frcnn : 1 After datou_step_exec type output : time spend for datou_step_exec : 2.4451475143432617 time spend to save output : 8.797645568847656e-05 total time spend for step 1 : 2.44523549079895 caffe_path_current : About to save ! 1 Inside saveOutput : final : True verbose : True Inside saveFrcnn : final : True verbose : True threshold to save the result : 0.1 output flattener : [(0, 493029425, 4370, 374, 430, 293, 317, 0.06384736, None), (0, 493029425, 4370, 382, 552, 297, 344, 0.052204903, None), (0, 493029425, 4370, 345, 468, 272, 320, 0.012277189, None)] Warning : no hashtag_ids to insert in the database final : True begin to insert list_values into mtr_datou_result : length of list_values in save_final : 1 insert ignore into MTRPhoto.mtr_datou_result (mtd_id, mtr_portfolio_id,mtr_photo_id,result,result_long,result_double,hashtag_id,proba, mtr_current_id) values (%s,%s,%s,%s,%s,%s,%s,%s,%s) on duplicate key update mtr_portfolio_id = mtr_portfolio_id list_values : [('4184', None, '917754606', '0', 0, '0', 493061979, '0', None)] time used for this insertion : 0.016643285751342773 [917754606] map_info['map_portfolio_photo'] : {} final : True mtd_id 4184 list_pids : [917754606] Looping around the photos to save general results len do output : 1 /0 before output type Managing all output in save final without adding information in the mtr_datou_result ('4184', None, None, None, None, None, None, None, None) ('4184', None, '917754606', None, None, None, None, None, None) begin to insert list_values into mtr_datou_result : length of list_values in save_final : 1 insert ignore into MTRPhoto.mtr_datou_result (mtd_id, mtr_portfolio_id,mtr_photo_id,result,result_long,result_double,hashtag_id,proba, mtr_current_id) values (%s,%s,%s,%s,%s,%s,%s,%s,%s) on duplicate key update mtr_portfolio_id = mtr_portfolio_id list_values : [('4184', None, '917754606', None, None, None, None, None, None)] time used for this insertion : 0.0166165828704834 save_final save missing photos in datou_result : After save, about to update current ! datou_cur_ids : [] len(datou.list_steps) : 1 output : {0: [[(0, 493029425, 4370, 374, 430, 293, 317, 0.06384736, None), (0, 493029425, 4370, 382, 552, 297, 344, 0.052204903, None), (0, 493029425, 4370, 345, 468, 272, 320, 0.012277189, None)], 'temp/1740710213_2762130_917754606_35f3c9ae49686a6be16030c6ec25c9ee.jpg']} ############################### TEST thcl ################################ TEST THCL Inside batchDatouExec : verbose : True ##### chargement datou SELECT name, created_at,limit_max FROM MTRDatou.mtr_datou WHERE id=2 SELECT mtd.id, mtdt.`type`, mtd.`param`, mtd.param_json, mtdt.nb_input, mtdt.nb_output, mtdt.prod, mtdt.is_local, mtdt.is_datou_depend, mtdt.is_photo_id_local FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_step_types mtdt WHERE mtdt.`id`=mtd.`type` AND mtd.mtd_id=2 SELECT mtd.id, mtd.mtd_id, mdsdt.id, mdsdt.name, mdsdt.description, msid.output_or_input, msid.data_order_id, mdsdt.type FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_steptype_io_datatypes msid, MTRDatou.mtr_datou_step_data_types mdsdt WHERE mtd.`type`=msid.`mtr_datou_step_type` AND mtd.mtd_id= 2 AND msid.data_type=mdsdt.id SELECT mts_id_output, id_output, mts_id_input, id_input FROM MTRDatou.mtr_datou_step_by_step WHERE mtd_id=2 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : step 1 thcl is not linked in the step_by_step architecture ! WARNING : step 2 argmax is not linked in the step_by_step architecture ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! DataTypes for each output/input checked ! no param json to modify List Step Type Loaded in datou : thcl, argmax list_input_json : [] ##### fin chargement datou ##### chargement data ##### Call load_data_input : nb_thread : 5 origin SELECT photo_id, url FROM MTRBack.photos ph WHERE photo_id IN (916235064) Found this number of photos: 1 ##### Call download_photos : nb_thread : 5 begin to download photo : 916235064 download finish for photo 916235064 we have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB ##### After download_photos length of list_filenames : 1 ; length of list_pids : 1 ; length of list_args : 1 ##### After load_data_input time to download the photos : 0.21775603294372559 #### fin chargement data Blocking on flush ? No conitnuing About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : True number of steps : 2 step1:thcl Fri Feb 28 03:36:55 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec After prepare type args : Here we display some param of map_info ! map_filenames : {'temp/1740710215_2762130_916235064_6293d1bb790dc6902450e7c572b7d10b.jpg': 916235064} map_photo_id_path_extension : {916235064: {'path': 'temp/1740710215_2762130_916235064_6293d1bb790dc6902450e7c572b7d10b.jpg', 'extension': 'jpg'}} map_subphoto_mainphoto : {} Beginning of datou step Thcl ! multi_thcl or not :False multi_thcl_cond or not :False dic_thcl : {'355': 1} we are using the classfication for only one thcl 355 In convert_file_to_np l 337 : 1 l343 1 l357 after caffe.io.load_image dimension du image : (3, (66, 66, 3)) dimension displayed ! time to import caffe and check if the image exist : 0.0035653114318847656 time to convert the images to numpy array : 0.0005323886871337891 total time to convert the images to numpy array : 0.004486799240112305 list photo_ids error: [] list photo_ids correct : [916235064] number of photos to traite : 1 try to delete the photos incorrect in DB tagging for thcl : 355 To do loadFromThcl(), then load ParamDescType : thcl355 get_desc_type_from_thcl : type of cat SELECT id, mtr_user_id, name, pb_hashtag_id, hashtag_id_list, button_legend_list, portfolio_id_lists, photo_hashtag_type, photo_desc_type, svm_limit, limit_tagging, is_public, live, created_at, updated_at, type_classification FROM MTRDatou.classification_theme WHERE `id` IN (355) thcls : [{'id': 355, 'mtr_user_id': 31, 'name': 'car_360_1027', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'c_elysee_1027_gao__port_506302,mokka_1027_gao__port_506374,captur_1027_gao__port_506399,sorento_1027_gao__port_506192,navara_1027_gao__port_506205,xc90_1027_gao__port_506350,saxo_1027_gao__port_506052,trafic_1027_gao__port_506295,punto_evo_1027_gao__port_506066,5_1027_gao__port_506117,250_1027_gao__port_506065,d_max_1027_gao__port_506125,panamera_1027_gao__port_506387,alhambra_1027_gao__port_506381,x6_1027_gao__port_506349,vitara_1027_gao__port_506328,fiesta_1027_gao__port_506377,qashqai_1027_gao__port_506286,147_1027_gao__port_506124,c5_1027_gao__port_506172,q5_1027_gao__port_506206,giulia_1027_gao__port_506178,karl_1027_gao__port_506371,mehari_1027_gao__port_506076,911_1027_gao__port_506114,508_1027_gao__port_506329,idea_1027_gao__port_506122,megane_1027_gao__port_506220,ghibli_1027_gao__port_506174,touareg_1027_gao__port_506224,i10_1027_gao__port_506232,jumper_1027_gao__port_506234,classe_clk_1027_gao__port_506173,kuga_1027_gao__port_506181,ct_1027_gao__port_506323,leon_1027_gao__port_506326,ds5_1027_gao__port_506376,cordoba_1027_gao__port_506048,classe_cla_1027_gao__port_506400,jumpy_1027_gao__port_506179,avensis_1027_gao__port_506311,juke_1027_gao__port_506325,4008_1027_gao__port_506402,190_series_1027_gao__port_506051,serie_3_1027_gao__port_506294,q7_1027_gao__port_506318,glc_1027_gao__port_506303,grand_vitara_1027_gao__port_506175,s40_1027_gao__port_506099,toledo_1027_gao__port_506061,5008_1027_gao__port_506337,continental_1027_gao__port_506250,coupe_1027_gao__port_506082,iq_1027_gao__port_506166,407_1027_gao__port_506133,touran_1027_gao__port_506308,300c_1027_gao__port_506078,classe_gl_1027_gao__port_506340,vivaro_1027_gao__port_506310,sl_1027_gao__port_506100,elise_1027_gao__port_506121,1007_1027_gao__port_506070,i40_1027_gao__port_506218,bipper_tepee_1027_gao__port_506227,focus_1027_gao__port_506272,primera_1027_gao__port_506147,r4_1027_gao__port_506160,a8_1027_gao__port_506265,boxer_1027_gao__port_506202,s5_1027_gao__port_506222,r21_1027_gao__port_506093,c3_1027_gao__port_506257,santa_fe_1027_gao__port_506208,m4_1027_gao__port_506344,safrane_1027_gao__port_506077,classe_gle_1027_gao__port_506395,0_1027_gao__port_506094,ix35_1027_gao__port_506219,carens_1027_gao__port_506298,classe_a_1027_gao__port_506339,ix20_1027_gao__port_506343,note_1027_gao__port_506365,a5_1027_gao__port_506200,sx4_1027_gao__port_506348,sandero_1027_gao__port_506198,3008_1027_gao__port_506385,q50_1027_gao__port_506239,latitude_1027_gao__port_506236,v40_1027_gao__port_506391,xsara_1027_gao__port_506087,grand_c_max_1027_gao__port_506342,swift_1027_gao__port_506149,serie_1_1027_gao__port_506184,xc70_1027_gao__port_506393,master_1027_gao__port_506203,clio_1027_gao__port_506280,duster_1027_gao__port_506216,traveller_1027_gao__port_506403,tipo_1027_gao__port_506355,rav_4_1027_gao__port_506332,coccinelle_1027_gao__port_506259,spacetourer_1027_gao__port_506401,xe_1027_gao__port_506357,ds3_1027_gao__port_506324,mx_5_1027_gao__port_506098,land_cruiser_1027_gao__port_506315,classe_b_1027_gao__port_506335,806_1027_gao__port_506088,rx_8_1027_gao__port_506046,spark_1027_gao__port_506185,6_1027_gao__port_506171,bravo_1027_gao__port_506080,nx_1027_gao__port_506345,sharan_1027_gao__port_506347,x_type_1027_gao__port_506067,jimny_1027_gao__port_506233,wrangler_1027_gao__port_506225,c_crosser_1027_gao__port_506312,v70_1027_gao__port_506278,classe_e_1027_gao__port_506300,classe_v_1027_gao__port_506258,m3_1027_gao__port_506182,abarth_500_1027_gao__port_506226,serie_6_1027_gao__port_506262,modus_1027_gao__port_506146,3_1027_gao__port_506113,405_1027_gao__port_506108,allroad_1027_gao__port_506297,auris_1027_gao__port_506322,galaxy_1027_gao__port_506143,giulietta_1027_gao__port_506363,106_1027_gao__port_506073,classe_m_1027_gao__port_506154,espace_1027_gao__port_506313,panda_1027_gao__port_506189,rcz_1027_gao__port_506197,4007_1027_gao__port_506162,classe_cl_1027_gao__port_506249,leaf_1027_gao__port_506139,octavia_1027_gao__port_506237,ds4_1027_gao__port_506336,freelander_1027_gao__port_506084,evasion_1027_gao__port_506109,punto_1027_gao__port_506106,2cv_1027_gao__port_506045,x4_1027_gao__port_506392,antara_1027_gao__port_506247,murano_1027_gao__port_506316,alto_1027_gao__port_506201,meriva_1027_gao__port_506353,orlando_1027_gao__port_506305,new_beetle_1027_gao__port_506050,306_1027_gao__port_506145,tiguan_1027_gao__port_506362,s_type_1027_gao__port_506101,c1_1027_gao__port_506128,vectra_1027_gao__port_506044,outlander_1027_gao__port_506317,307_1027_gao__port_506074,a6_s6_1027_gao__port_506134,nemo_combi_1027_gao__port_506196,berlingo_1027_gao__port_506194,partner_1027_gao__port_506285,cayenne_1027_gao__port_506177,quattroporte_1027_gao__port_506240,c_max_1027_gao__port_506282,fabia_1027_gao__port_506396,cx_3_1027_gao__port_506281,x_trail_1027_gao__port_506264,scirocco_1027_gao__port_506276,matiz_1027_gao__port_506144,tigra_1027_gao__port_506069,escort_1027_gao__port_506091,c2_1027_gao__port_506081,mini_1027_gao__port_506168,i30_1027_gao__port_506291,picanto_1027_gao__port_506238,mito_1027_gao__port_506072,impreza_1027_gao__port_506085,kangoo_1027_gao__port_506235,a4_1027_gao__port_506193,cayman_1027_gao__port_506268,sportage_1027_gao__port_506148,up_1027_gao__port_506356,optima_1027_gao__port_506386,defender_1027_gao__port_506229,serie_2_1027_gao__port_506256,edge_1027_gao__port_506187,r19_1027_gao__port_506110,jetta_1027_gao__port_506304,eos_1027_gao__port_506115,accord_1027_gao__port_506214,yaris_1027_gao__port_506334,classe_cls_1027_gao__port_506289,polo_1027_gao__port_506361,serie_4_1027_gao__port_506366,mini_cabriolet_1027_gao__port_506204,prius_1027_gao__port_506190,lodgy_1027_gao__port_506188,serie_7_1027_gao__port_506307,c15_1027_gao__port_506055,kadjar_1027_gao__port_506389,insignia_1027_gao__port_506364,308_1027_gao__port_506279,roomster_1027_gao__port_506241,80_1027_gao__port_506057,309_1027_gao__port_506063,tucson_1027_gao__port_506320,x3_1027_gao__port_506212,xf_1027_gao__port_506263,2008_1027_gao__port_506394,passat_1027_gao__port_506306,compass_1027_gao__port_506260,twingo_1027_gao__port_506309,micra_1027_gao__port_506221,golf_1027_gao__port_506155,soul_1027_gao__port_506176,rapid_1027_gao__port_506398,forester_1027_gao__port_506360,slk_1027_gao__port_506210,forfour_1027_gao__port_506341,serie_5_1027_gao__port_506209,xj_1027_gao__port_506170,pajero_1027_gao__port_506097,agila_1027_gao__port_506119,a6_1027_gao__port_506163,fox_1027_gao__port_506092,boxster_1027_gao__port_506267,altea_1027_gao__port_506246,samurai_1027_gao__port_506047,trax_1027_gao__port_506296,getz_1027_gao__port_506058,cherokee_1027_gao__port_506269,koleos_1027_gao__port_506378,z_series_1027_gao__port_506123,ecosport_1027_gao__port_506271,space_star_1027_gao__port_506277,rs3_sportback_1027_gao__port_506207,civic_1027_gao__port_506141,talisman_1027_gao__port_506390,f_pace_1027_gao__port_506314,classe_c_1027_gao__port_506299,tt_1027_gao__port_506075,pathfinder_1027_gao__port_506183,156_1027_gao__port_506157,cx_5_1027_gao__port_506228,scenic_1027_gao__port_506255,yeti_1027_gao__port_506358,mustang_1027_gao__port_506053,stilo_1027_gao__port_506060,ateca_1027_gao__port_506382,fiorino_1027_gao__port_506217,classe_glk_1027_gao__port_506290,fortwo_1027_gao__port_506230,cruze_1027_gao__port_506186,107_1027_gao__port_506213,aygo_1027_gao__port_506248,rx_1027_gao__port_506354,500_1027_gao__port_506245,bora_1027_gao__port_506104,transit_1027_gao__port_506111,pt_cruiser_1027_gao__port_506054,patrol_1027_gao__port_506068,r8_1027_gao__port_506156,xm_1027_gao__port_506102,s60_1027_gao__port_506191,aveo_1027_gao__port_506158,captiva_1027_gao__port_506159,ax_1027_gao__port_506153,rexton_1027_gao__port_506107,camaro_1027_gao__port_506056,ypsilon_1027_gao__port_506131,delta_1027_gao__port_506165,c4_1027_gao__port_506370,zx_1027_gao__port_506161,verso_1027_gao__port_506242,superb_1027_gao__port_506327,r5_1027_gao__port_506253,caddy_1027_gao__port_506330,x5_1027_gao__port_506243,f_type_1027_gao__port_506231,fusion_1027_gao__port_506096,dokker_1027_gao__port_506331,205_1027_gao__port_506062,macan_1027_gao__port_506195,tourneo_1027_gao__port_506369,108_1027_gao__port_506384,9_3_1027_gao__port_506071,mondeo_1027_gao__port_506116,cr_v_1027_gao__port_506164,c30_1027_gao__port_506090,pulsar_1027_gao__port_506397,ibiza_1027_gao__port_506273,a1_1027_gao__port_506338,matrix_1027_gao__port_506140,carnival_1027_gao__port_506136,xantia_1027_gao__port_506086,terrano_1027_gao__port_506083,q3_1027_gao__port_506275,hr_v_1027_gao__port_506283,expert_1027_gao__port_506142,multivan_1027_gao__port_506383,venga_1027_gao__port_506380,scudo_1027_gao__port_506129,laguna_1027_gao__port_506368,vel_satis_1027_gao__port_506130,b_max_1027_gao__port_506367,ignis_1027_gao__port_506292,159_1027_gao__port_506064,grande_punto_1027_gao__port_506138,logan_1027_gao__port_506167,s_max_1027_gao__port_506223,caravelle_1027_gao__port_506351,adam_1027_gao__port_506079,406_1027_gao__port_506132,q30_1027_gao__port_506293,almera_1027_gao__port_506089,corsa_1027_gao__port_506095,corolla_1027_gao__port_506120,xc60_1027_gao__port_506388,viano_1027_gao__port_506211,pro_cee_d_1027_gao__port_506274,a3_1027_gao__port_506321,v50_1027_gao__port_506150,voyager_1027_gao__port_506169,corvette_1027_gao__port_506049,rio_1027_gao__port_506379,jazz_1027_gao__port_506252,200_1027_gao__port_506112,tts_1027_gao__port_506199,zafira_1027_gao__port_506287,asx_1027_gao__port_506266,607_1027_gao__port_506118,207_1027_gao__port_506103,classe_s_1027_gao__port_506301,c6_1027_gao__port_506105,express_1027_gao__port_506137,classe_gla_1027_gao__port_506352,v60_1027_gao__port_506333,ka_1027_gao__port_506180,range_rover_1027_gao__port_506254,discovery_1027_gao__port_506375,classe_r_1027_gao__port_506270,transporter_1027_gao__port_506319,cee_d_1027_gao__port_506288,zoe_1027_gao__port_506244,i20_1027_gao__port_506284,gtv_1027_gao__port_506059,s4_avant_1027_gao__port_506261,x1_1027_gao__port_506372,autres_1027_gao__port_506127,208_1027_gao__port_506359,c8_1027_gao__port_506135,astra_1027_gao__port_506215,2_1027_gao__port_506151,doblo_1027_gao__port_506251,807_1027_gao__port_506152,206_1027_gao__port_506126,a7_1027_gao__port_506373,renegade_1027_gao__port_506346', 'svm_portfolios_learning': '506302,506374,506399,506192,506205,506350,506052,506295,506066,506117,506065,506125,506387,506381,506349,506328,506377,506286,506124,506172,506206,506178,506371,506076,506114,506329,506122,506220,506174,506224,506232,506234,506173,506181,506323,506326,506376,506048,506400,506179,506311,506325,506402,506051,506294,506318,506303,506175,506099,506061,506337,506250,506082,506166,506133,506308,506078,506340,506310,506100,506121,506070,506218,506227,506272,506147,506160,506265,506202,506222,506093,506257,506208,506344,506077,506395,506094,506219,506298,506339,506343,506365,506200,506348,506198,506385,506239,506236,506391,506087,506342,506149,506184,506393,506203,506280,506216,506403,506355,506332,506259,506401,506357,506324,506098,506315,506335,506088,506046,506185,506171,506080,506345,506347,506067,506233,506225,506312,506278,506300,506258,506182,506226,506262,506146,506113,506108,506297,506322,506143,506363,506073,506154,506313,506189,506197,506162,506249,506139,506237,506336,506084,506109,506106,506045,506392,506247,506316,506201,506353,506305,506050,506145,506362,506101,506128,506044,506317,506074,506134,506196,506194,506285,506177,506240,506282,506396,506281,506264,506276,506144,506069,506091,506081,506168,506291,506238,506072,506085,506235,506193,506268,506148,506356,506386,506229,506256,506187,506110,506304,506115,506214,506334,506289,506361,506366,506204,506190,506188,506307,506055,506389,506364,506279,506241,506057,506063,506320,506212,506263,506394,506306,506260,506309,506221,506155,506176,506398,506360,506210,506341,506209,506170,506097,506119,506163,506092,506267,506246,506047,506296,506058,506269,506378,506123,506271,506277,506207,506141,506390,506314,506299,506075,506183,506157,506228,506255,506358,506053,506060,506382,506217,506290,506230,506186,506213,506248,506354,506245,506104,506111,506054,506068,506156,506102,506191,506158,506159,506153,506107,506056,506131,506165,506370,506161,506242,506327,506253,506330,506243,506231,506096,506331,506062,506195,506369,506384,506071,506116,506164,506090,506397,506273,506338,506140,506136,506086,506083,506275,506283,506142,506383,506380,506129,506368,506130,506367,506292,506064,506138,506167,506223,506351,506079,506132,506293,506089,506095,506120,506388,506211,506274,506321,506150,506169,506049,506379,506252,506112,506199,506287,506266,506118,506103,506301,506105,506137,506352,506333,506180,506254,506375,506270,506319,506288,506244,506284,506059,506261,506372,506127,506359,506135,506215,506151,506251,506152,506126,506373,506346', 'photo_hashtag_type': 332, 'photo_desc_type': 3390, 'type_classification': 'caffe', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 355, 'mtr_user_id': 31, 'name': 'car_360_1027', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'c_elysee_1027_gao__port_506302,mokka_1027_gao__port_506374,captur_1027_gao__port_506399,sorento_1027_gao__port_506192,navara_1027_gao__port_506205,xc90_1027_gao__port_506350,saxo_1027_gao__port_506052,trafic_1027_gao__port_506295,punto_evo_1027_gao__port_506066,5_1027_gao__port_506117,250_1027_gao__port_506065,d_max_1027_gao__port_506125,panamera_1027_gao__port_506387,alhambra_1027_gao__port_506381,x6_1027_gao__port_506349,vitara_1027_gao__port_506328,fiesta_1027_gao__port_506377,qashqai_1027_gao__port_506286,147_1027_gao__port_506124,c5_1027_gao__port_506172,q5_1027_gao__port_506206,giulia_1027_gao__port_506178,karl_1027_gao__port_506371,mehari_1027_gao__port_506076,911_1027_gao__port_506114,508_1027_gao__port_506329,idea_1027_gao__port_506122,megane_1027_gao__port_506220,ghibli_1027_gao__port_506174,touareg_1027_gao__port_506224,i10_1027_gao__port_506232,jumper_1027_gao__port_506234,classe_clk_1027_gao__port_506173,kuga_1027_gao__port_506181,ct_1027_gao__port_506323,leon_1027_gao__port_506326,ds5_1027_gao__port_506376,cordoba_1027_gao__port_506048,classe_cla_1027_gao__port_506400,jumpy_1027_gao__port_506179,avensis_1027_gao__port_506311,juke_1027_gao__port_506325,4008_1027_gao__port_506402,190_series_1027_gao__port_506051,serie_3_1027_gao__port_506294,q7_1027_gao__port_506318,glc_1027_gao__port_506303,grand_vitara_1027_gao__port_506175,s40_1027_gao__port_506099,toledo_1027_gao__port_506061,5008_1027_gao__port_506337,continental_1027_gao__port_506250,coupe_1027_gao__port_506082,iq_1027_gao__port_506166,407_1027_gao__port_506133,touran_1027_gao__port_506308,300c_1027_gao__port_506078,classe_gl_1027_gao__port_506340,vivaro_1027_gao__port_506310,sl_1027_gao__port_506100,elise_1027_gao__port_506121,1007_1027_gao__port_506070,i40_1027_gao__port_506218,bipper_tepee_1027_gao__port_506227,focus_1027_gao__port_506272,primera_1027_gao__port_506147,r4_1027_gao__port_506160,a8_1027_gao__port_506265,boxer_1027_gao__port_506202,s5_1027_gao__port_506222,r21_1027_gao__port_506093,c3_1027_gao__port_506257,santa_fe_1027_gao__port_506208,m4_1027_gao__port_506344,safrane_1027_gao__port_506077,classe_gle_1027_gao__port_506395,0_1027_gao__port_506094,ix35_1027_gao__port_506219,carens_1027_gao__port_506298,classe_a_1027_gao__port_506339,ix20_1027_gao__port_506343,note_1027_gao__port_506365,a5_1027_gao__port_506200,sx4_1027_gao__port_506348,sandero_1027_gao__port_506198,3008_1027_gao__port_506385,q50_1027_gao__port_506239,latitude_1027_gao__port_506236,v40_1027_gao__port_506391,xsara_1027_gao__port_506087,grand_c_max_1027_gao__port_506342,swift_1027_gao__port_506149,serie_1_1027_gao__port_506184,xc70_1027_gao__port_506393,master_1027_gao__port_506203,clio_1027_gao__port_506280,duster_1027_gao__port_506216,traveller_1027_gao__port_506403,tipo_1027_gao__port_506355,rav_4_1027_gao__port_506332,coccinelle_1027_gao__port_506259,spacetourer_1027_gao__port_506401,xe_1027_gao__port_506357,ds3_1027_gao__port_506324,mx_5_1027_gao__port_506098,land_cruiser_1027_gao__port_506315,classe_b_1027_gao__port_506335,806_1027_gao__port_506088,rx_8_1027_gao__port_506046,spark_1027_gao__port_506185,6_1027_gao__port_506171,bravo_1027_gao__port_506080,nx_1027_gao__port_506345,sharan_1027_gao__port_506347,x_type_1027_gao__port_506067,jimny_1027_gao__port_506233,wrangler_1027_gao__port_506225,c_crosser_1027_gao__port_506312,v70_1027_gao__port_506278,classe_e_1027_gao__port_506300,classe_v_1027_gao__port_506258,m3_1027_gao__port_506182,abarth_500_1027_gao__port_506226,serie_6_1027_gao__port_506262,modus_1027_gao__port_506146,3_1027_gao__port_506113,405_1027_gao__port_506108,allroad_1027_gao__port_506297,auris_1027_gao__port_506322,galaxy_1027_gao__port_506143,giulietta_1027_gao__port_506363,106_1027_gao__port_506073,classe_m_1027_gao__port_506154,espace_1027_gao__port_506313,panda_1027_gao__port_506189,rcz_1027_gao__port_506197,4007_1027_gao__port_506162,classe_cl_1027_gao__port_506249,leaf_1027_gao__port_506139,octavia_1027_gao__port_506237,ds4_1027_gao__port_506336,freelander_1027_gao__port_506084,evasion_1027_gao__port_506109,punto_1027_gao__port_506106,2cv_1027_gao__port_506045,x4_1027_gao__port_506392,antara_1027_gao__port_506247,murano_1027_gao__port_506316,alto_1027_gao__port_506201,meriva_1027_gao__port_506353,orlando_1027_gao__port_506305,new_beetle_1027_gao__port_506050,306_1027_gao__port_506145,tiguan_1027_gao__port_506362,s_type_1027_gao__port_506101,c1_1027_gao__port_506128,vectra_1027_gao__port_506044,outlander_1027_gao__port_506317,307_1027_gao__port_506074,a6_s6_1027_gao__port_506134,nemo_combi_1027_gao__port_506196,berlingo_1027_gao__port_506194,partner_1027_gao__port_506285,cayenne_1027_gao__port_506177,quattroporte_1027_gao__port_506240,c_max_1027_gao__port_506282,fabia_1027_gao__port_506396,cx_3_1027_gao__port_506281,x_trail_1027_gao__port_506264,scirocco_1027_gao__port_506276,matiz_1027_gao__port_506144,tigra_1027_gao__port_506069,escort_1027_gao__port_506091,c2_1027_gao__port_506081,mini_1027_gao__port_506168,i30_1027_gao__port_506291,picanto_1027_gao__port_506238,mito_1027_gao__port_506072,impreza_1027_gao__port_506085,kangoo_1027_gao__port_506235,a4_1027_gao__port_506193,cayman_1027_gao__port_506268,sportage_1027_gao__port_506148,up_1027_gao__port_506356,optima_1027_gao__port_506386,defender_1027_gao__port_506229,serie_2_1027_gao__port_506256,edge_1027_gao__port_506187,r19_1027_gao__port_506110,jetta_1027_gao__port_506304,eos_1027_gao__port_506115,accord_1027_gao__port_506214,yaris_1027_gao__port_506334,classe_cls_1027_gao__port_506289,polo_1027_gao__port_506361,serie_4_1027_gao__port_506366,mini_cabriolet_1027_gao__port_506204,prius_1027_gao__port_506190,lodgy_1027_gao__port_506188,serie_7_1027_gao__port_506307,c15_1027_gao__port_506055,kadjar_1027_gao__port_506389,insignia_1027_gao__port_506364,308_1027_gao__port_506279,roomster_1027_gao__port_506241,80_1027_gao__port_506057,309_1027_gao__port_506063,tucson_1027_gao__port_506320,x3_1027_gao__port_506212,xf_1027_gao__port_506263,2008_1027_gao__port_506394,passat_1027_gao__port_506306,compass_1027_gao__port_506260,twingo_1027_gao__port_506309,micra_1027_gao__port_506221,golf_1027_gao__port_506155,soul_1027_gao__port_506176,rapid_1027_gao__port_506398,forester_1027_gao__port_506360,slk_1027_gao__port_506210,forfour_1027_gao__port_506341,serie_5_1027_gao__port_506209,xj_1027_gao__port_506170,pajero_1027_gao__port_506097,agila_1027_gao__port_506119,a6_1027_gao__port_506163,fox_1027_gao__port_506092,boxster_1027_gao__port_506267,altea_1027_gao__port_506246,samurai_1027_gao__port_506047,trax_1027_gao__port_506296,getz_1027_gao__port_506058,cherokee_1027_gao__port_506269,koleos_1027_gao__port_506378,z_series_1027_gao__port_506123,ecosport_1027_gao__port_506271,space_star_1027_gao__port_506277,rs3_sportback_1027_gao__port_506207,civic_1027_gao__port_506141,talisman_1027_gao__port_506390,f_pace_1027_gao__port_506314,classe_c_1027_gao__port_506299,tt_1027_gao__port_506075,pathfinder_1027_gao__port_506183,156_1027_gao__port_506157,cx_5_1027_gao__port_506228,scenic_1027_gao__port_506255,yeti_1027_gao__port_506358,mustang_1027_gao__port_506053,stilo_1027_gao__port_506060,ateca_1027_gao__port_506382,fiorino_1027_gao__port_506217,classe_glk_1027_gao__port_506290,fortwo_1027_gao__port_506230,cruze_1027_gao__port_506186,107_1027_gao__port_506213,aygo_1027_gao__port_506248,rx_1027_gao__port_506354,500_1027_gao__port_506245,bora_1027_gao__port_506104,transit_1027_gao__port_506111,pt_cruiser_1027_gao__port_506054,patrol_1027_gao__port_506068,r8_1027_gao__port_506156,xm_1027_gao__port_506102,s60_1027_gao__port_506191,aveo_1027_gao__port_506158,captiva_1027_gao__port_506159,ax_1027_gao__port_506153,rexton_1027_gao__port_506107,camaro_1027_gao__port_506056,ypsilon_1027_gao__port_506131,delta_1027_gao__port_506165,c4_1027_gao__port_506370,zx_1027_gao__port_506161,verso_1027_gao__port_506242,superb_1027_gao__port_506327,r5_1027_gao__port_506253,caddy_1027_gao__port_506330,x5_1027_gao__port_506243,f_type_1027_gao__port_506231,fusion_1027_gao__port_506096,dokker_1027_gao__port_506331,205_1027_gao__port_506062,macan_1027_gao__port_506195,tourneo_1027_gao__port_506369,108_1027_gao__port_506384,9_3_1027_gao__port_506071,mondeo_1027_gao__port_506116,cr_v_1027_gao__port_506164,c30_1027_gao__port_506090,pulsar_1027_gao__port_506397,ibiza_1027_gao__port_506273,a1_1027_gao__port_506338,matrix_1027_gao__port_506140,carnival_1027_gao__port_506136,xantia_1027_gao__port_506086,terrano_1027_gao__port_506083,q3_1027_gao__port_506275,hr_v_1027_gao__port_506283,expert_1027_gao__port_506142,multivan_1027_gao__port_506383,venga_1027_gao__port_506380,scudo_1027_gao__port_506129,laguna_1027_gao__port_506368,vel_satis_1027_gao__port_506130,b_max_1027_gao__port_506367,ignis_1027_gao__port_506292,159_1027_gao__port_506064,grande_punto_1027_gao__port_506138,logan_1027_gao__port_506167,s_max_1027_gao__port_506223,caravelle_1027_gao__port_506351,adam_1027_gao__port_506079,406_1027_gao__port_506132,q30_1027_gao__port_506293,almera_1027_gao__port_506089,corsa_1027_gao__port_506095,corolla_1027_gao__port_506120,xc60_1027_gao__port_506388,viano_1027_gao__port_506211,pro_cee_d_1027_gao__port_506274,a3_1027_gao__port_506321,v50_1027_gao__port_506150,voyager_1027_gao__port_506169,corvette_1027_gao__port_506049,rio_1027_gao__port_506379,jazz_1027_gao__port_506252,200_1027_gao__port_506112,tts_1027_gao__port_506199,zafira_1027_gao__port_506287,asx_1027_gao__port_506266,607_1027_gao__port_506118,207_1027_gao__port_506103,classe_s_1027_gao__port_506301,c6_1027_gao__port_506105,express_1027_gao__port_506137,classe_gla_1027_gao__port_506352,v60_1027_gao__port_506333,ka_1027_gao__port_506180,range_rover_1027_gao__port_506254,discovery_1027_gao__port_506375,classe_r_1027_gao__port_506270,transporter_1027_gao__port_506319,cee_d_1027_gao__port_506288,zoe_1027_gao__port_506244,i20_1027_gao__port_506284,gtv_1027_gao__port_506059,s4_avant_1027_gao__port_506261,x1_1027_gao__port_506372,autres_1027_gao__port_506127,208_1027_gao__port_506359,c8_1027_gao__port_506135,astra_1027_gao__port_506215,2_1027_gao__port_506151,doblo_1027_gao__port_506251,807_1027_gao__port_506152,206_1027_gao__port_506126,a7_1027_gao__port_506373,renegade_1027_gao__port_506346', 'svm_portfolios_learning': '506302,506374,506399,506192,506205,506350,506052,506295,506066,506117,506065,506125,506387,506381,506349,506328,506377,506286,506124,506172,506206,506178,506371,506076,506114,506329,506122,506220,506174,506224,506232,506234,506173,506181,506323,506326,506376,506048,506400,506179,506311,506325,506402,506051,506294,506318,506303,506175,506099,506061,506337,506250,506082,506166,506133,506308,506078,506340,506310,506100,506121,506070,506218,506227,506272,506147,506160,506265,506202,506222,506093,506257,506208,506344,506077,506395,506094,506219,506298,506339,506343,506365,506200,506348,506198,506385,506239,506236,506391,506087,506342,506149,506184,506393,506203,506280,506216,506403,506355,506332,506259,506401,506357,506324,506098,506315,506335,506088,506046,506185,506171,506080,506345,506347,506067,506233,506225,506312,506278,506300,506258,506182,506226,506262,506146,506113,506108,506297,506322,506143,506363,506073,506154,506313,506189,506197,506162,506249,506139,506237,506336,506084,506109,506106,506045,506392,506247,506316,506201,506353,506305,506050,506145,506362,506101,506128,506044,506317,506074,506134,506196,506194,506285,506177,506240,506282,506396,506281,506264,506276,506144,506069,506091,506081,506168,506291,506238,506072,506085,506235,506193,506268,506148,506356,506386,506229,506256,506187,506110,506304,506115,506214,506334,506289,506361,506366,506204,506190,506188,506307,506055,506389,506364,506279,506241,506057,506063,506320,506212,506263,506394,506306,506260,506309,506221,506155,506176,506398,506360,506210,506341,506209,506170,506097,506119,506163,506092,506267,506246,506047,506296,506058,506269,506378,506123,506271,506277,506207,506141,506390,506314,506299,506075,506183,506157,506228,506255,506358,506053,506060,506382,506217,506290,506230,506186,506213,506248,506354,506245,506104,506111,506054,506068,506156,506102,506191,506158,506159,506153,506107,506056,506131,506165,506370,506161,506242,506327,506253,506330,506243,506231,506096,506331,506062,506195,506369,506384,506071,506116,506164,506090,506397,506273,506338,506140,506136,506086,506083,506275,506283,506142,506383,506380,506129,506368,506130,506367,506292,506064,506138,506167,506223,506351,506079,506132,506293,506089,506095,506120,506388,506211,506274,506321,506150,506169,506049,506379,506252,506112,506199,506287,506266,506118,506103,506301,506105,506137,506352,506333,506180,506254,506375,506270,506319,506288,506244,506284,506059,506261,506372,506127,506359,506135,506215,506151,506251,506152,506126,506373,506346', 'photo_hashtag_type': 332, 'photo_desc_type': 3390, 'type_classification': 'caffe', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 3390 SELECT * FROM MTRDatou.photo_desc_type_params WHERE id in (3390) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (3390, 'car_360_1027', 16384, 25088, 'car_360_1027', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2017, 10, 28, 12, 29, 27), datetime.datetime(2017, 10, 28, 12, 29, 27)) To loadFromThcl() : net_3390 begin to check gpu status inside check gpu memory havn't enough memory gpu , need / 2500 l 3632 free memory gpu now : 986 wait 20 seconds l 3637 free memory gpu now : 986 max_wait_temp : 1 max_wait : 0 SELECT * FROM MTRDatou.photo_desc_type_params WHERE id in (3390) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (3390, 'car_360_1027', 16384, 25088, 'car_360_1027', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2017, 10, 28, 12, 29, 27), datetime.datetime(2017, 10, 28, 12, 29, 27)) param : , param.caffemodel : car_360_1027 None mean_file_type : mean_file_path : prototxt_file_path : model : car_360_1027 Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : car_360_1027 model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : [] local folder : /data/models_weight/car_360_1027 /data/models_weight/car_360_1027/caffemodel size_local : 542944640 size in s3 : 542944640 create time local : 2021-08-09 05:28:34 create time in s3 : 2021-08-06 17:57:43 caffemodel already exist and didn't need to update /data/models_weight/car_360_1027/deploy_conv_normal.prototxt size_local : 4626 size in s3 : 4626 create time local : 2021-08-09 05:28:34 create time in s3 : 2021-08-06 17:57:42 deploy_conv_normal.prototxt already exist and didn't need to update /data/models_weight/car_360_1027/deploy_fc.prototxt size_local : 1132 size in s3 : 1132 create time local : 2021-08-09 05:28:34 create time in s3 : 2021-08-06 17:57:43 deploy_fc.prototxt already exist and didn't need to update /data/models_weight/car_360_1027/deploy.prototxt size_local : 5654 size in s3 : 5654 create time local : 2021-08-09 05:28:34 create time in s3 : 2021-08-06 17:57:42 deploy.prototxt already exist and didn't need to update /data/models_weight/car_360_1027/mean.npy size_local : 1572944 size in s3 : 1572944 create time local : 2021-08-09 05:28:34 create time in s3 : 2021-08-06 17:57:55 mean.npy already exist and didn't need to update /data/models_weight/car_360_1027/synset_words.txt size_local : 13687 size in s3 : 13687 create time local : 2021-08-09 05:28:34 create time in s3 : 2021-08-06 17:57:43 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/car_360_1027/deploy.prototxt caffemodel_filename : /data/models_weight/car_360_1027/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory havn't enough memory gpu , need / 2500 l 3632 free memory gpu now : 984 wait 20 seconds l 3637 free memory gpu now : 984 max_wait_temp : 1 max_wait : 0 dict_keys(['pool5', 'prob']) time used to do the prepocess of the images : 0.02390575408935547 time used to do the prediction : 0.19367480278015137 save descriptor for thcl : 355 (1, 512, 7, 7) Got the blobs of the net to insert : [0, 0, 0, 0, 0, 0, 0, 0, 0, 0] code_as_byte_string:b'0000000000'| time to traite the descriptors : 0.054053544998168945 Testing : ['916235064'] In select_photos_meta_from_ids: SELECT photo_id, url, FROM_UNIXTIME(uploaded_at), latitude, longitude, text FROM MTRBack.photos WHERE photo_id IN (916235064) Catched exception ! Connect or reconnect ! result : {916235064: {'photo_id': 916235064, 'url': 'https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/2017/10/14/6293d1bb790dc6902450e7c572b7d10b.jpg', 'latitude': 0.0, 'longitude': 0.0, 'text': None}} list_photo_exists : [916235064] storage_type for insertDescriptorsMulti : 1 To insert : 916235064 time to insert the descriptors : 2.1199021339416504 After datou_step_exec type output : map_portfolio_photo : len 0 keys : dict_keys([]) Inside saveOutput : final : False verbose : True time used to find the portfolios of the photos select button_legend_list from MTRDatou.classification_theme where id = 355 SAVE THCL, output : {'916235064': [[('916235064', 'c_elysee_1027_gao__port_506302', 0.0018819829, 332, '355'), ('916235064', 'mokka_1027_gao__port_506374', 0.0011636707, 332, '355'), ('916235064', 'captur_1027_gao__port_506399', 0.000816072, 332, '355'), ('916235064', 'sorento_1027_gao__port_506192', 0.0011776943, 332, '355'), ('916235064', 'navara_1027_gao__port_506205', 0.0025853885, 332, '355'), ('916235064', 'xc90_1027_gao__port_506350', 0.004171521, 332, '355'), ('916235064', 'saxo_1027_gao__port_506052', 0.003480227, 332, '355'), ('916235064', 'trafic_1027_gao__port_506295', 0.0073649143, 332, '355'), ('916235064', 'punto_evo_1027_gao__port_506066', 0.002188672, 332, '355'), ('916235064', '5_1027_gao__port_506117', 0.0005799453, 332, '355'), ('916235064', '250_1027_gao__port_506065', 0.0045910133, 332, '355'), ('916235064', 'd_max_1027_gao__port_506125', 0.0031583295, 332, '355'), ('916235064', 'panamera_1027_gao__port_506387', 0.002251267, 332, '355'), ('916235064', 'alhambra_1027_gao__port_506381', 0.0053206445, 332, '355'), ('916235064', 'x6_1027_gao__port_506349', 0.0011000738, 332, '355'), ('916235064', 'vitara_1027_gao__port_506328', 0.0054023494, 332, '355'), ('916235064', 'fiesta_1027_gao__port_506377', 0.003919149, 332, '355'), ('916235064', 'qashqai_1027_gao__port_506286', 0.0014790139, 332, '355'), ('916235064', '147_1027_gao__port_506124', 0.0019776083, 332, '355'), ('916235064', 'c5_1027_gao__port_506172', 0.0012443245, 332, '355'), ('916235064', 'q5_1027_gao__port_506206', 0.0015054578, 332, '355'), ('916235064', 'giulia_1027_gao__port_506178', 0.0021697117, 332, '355'), ('916235064', 'karl_1027_gao__port_506371', 0.0027088688, 332, '355'), ('916235064', 'mehari_1027_gao__port_506076', 0.0047011473, 332, '355'), ('916235064', '911_1027_gao__port_506114', 0.0019418563, 332, '355'), ('916235064', '508_1027_gao__port_506329', 0.0009588494, 332, '355'), ('916235064', 'idea_1027_gao__port_506122', 0.00077015924, 332, '355'), ('916235064', 'megane_1027_gao__port_506220', 0.0019470402, 332, '355'), ('916235064', 'ghibli_1027_gao__port_506174', 0.0013730422, 332, '355'), ('916235064', 'touareg_1027_gao__port_506224', 0.0016204282, 332, '355'), ('916235064', 'i10_1027_gao__port_506232', 0.0013926253, 332, '355'), ('916235064', 'jumper_1027_gao__port_506234', 0.01004369, 332, '355'), ('916235064', 'classe_clk_1027_gao__port_506173', 0.0010793956, 332, '355'), ('916235064', 'kuga_1027_gao__port_506181', 0.0008448386, 332, '355'), ('916235064', 'ct_1027_gao__port_506323', 0.0012521527, 332, '355'), ('916235064', 'leon_1027_gao__port_506326', 0.002584827, 332, '355'), ('916235064', 'ds5_1027_gao__port_506376', 0.0012434311, 332, '355'), ('916235064', 'cordoba_1027_gao__port_506048', 0.0028649028, 332, '355'), ('916235064', 'classe_cla_1027_gao__port_506400', 0.0012953416, 332, '355'), ('916235064', 'jumpy_1027_gao__port_506179', 0.010337308, 332, '355'), ('916235064', 'avensis_1027_gao__port_506311', 0.0018766425, 332, '355'), ('916235064', 'juke_1027_gao__port_506325', 0.0011346631, 332, '355'), ('916235064', '4008_1027_gao__port_506402', 0.0015762425, 332, '355'), ('916235064', '190_series_1027_gao__port_506051', 0.0039790645, 332, '355'), ('916235064', 'serie_3_1027_gao__port_506294', 0.0028742536, 332, '355'), ('916235064', 'q7_1027_gao__port_506318', 0.0023362506, 332, '355'), ('916235064', 'glc_1027_gao__port_506303', 0.0012110298, 332, '355'), ('916235064', 'grand_vitara_1027_gao__port_506175', 0.001144975, 332, '355'), ('916235064', 's40_1027_gao__port_506099', 0.0022341395, 332, '355'), ('916235064', 'toledo_1027_gao__port_506061', 0.0017467097, 332, '355'), ('916235064', '5008_1027_gao__port_506337', 0.004700288, 332, '355'), ('916235064', 'continental_1027_gao__port_506250', 0.002191987, 332, '355'), ('916235064', 'coupe_1027_gao__port_506082', 0.0022627634, 332, '355'), ('916235064', 'iq_1027_gao__port_506166', 0.0018173229, 332, '355'), ('916235064', '407_1027_gao__port_506133', 0.00090583804, 332, '355'), ('916235064', 'touran_1027_gao__port_506308', 0.002040651, 332, '355'), ('916235064', '300c_1027_gao__port_506078', 0.0025341655, 332, '355'), ('916235064', 'classe_gl_1027_gao__port_506340', 0.0044900165, 332, '355'), ('916235064', 'vivaro_1027_gao__port_506310', 0.0034249064, 332, '355'), ('916235064', 'sl_1027_gao__port_506100', 0.0031345373, 332, '355'), ('916235064', 'elise_1027_gao__port_506121', 0.0010255522, 332, '355'), ('916235064', '1007_1027_gao__port_506070', 0.0015355606, 332, '355'), ('916235064', 'i40_1027_gao__port_506218', 0.0005916346, 332, '355'), ('916235064', 'bipper_tepee_1027_gao__port_506227', 0.0040307576, 332, '355'), ('916235064', 'focus_1027_gao__port_506272', 0.0011589169, 332, '355'), ('916235064', 'primera_1027_gao__port_506147', 0.0012156558, 332, '355'), ('916235064', 'r4_1027_gao__port_506160', 0.014965488, 332, '355'), ('916235064', 'a8_1027_gao__port_506265', 0.0011321153, 332, '355'), ('916235064', 'boxer_1027_gao__port_506202', 0.010542957, 332, '355'), ('916235064', 's5_1027_gao__port_506222', 0.0011985744, 332, '355'), ('916235064', 'r21_1027_gao__port_506093', 0.0041845837, 332, '355'), ('916235064', 'c3_1027_gao__port_506257', 0.0023642792, 332, '355'), ('916235064', 'santa_fe_1027_gao__port_506208', 0.0016328013, 332, '355'), ('916235064', 'm4_1027_gao__port_506344', 0.0015568332, 332, '355'), ('916235064', 'safrane_1027_gao__port_506077', 0.0013959111, 332, '355'), ('916235064', 'classe_gle_1027_gao__port_506395', 0.0021984705, 332, '355'), ('916235064', '0_1027_gao__port_506094', 0.008825756, 332, '355'), ('916235064', 'ix35_1027_gao__port_506219', 0.0014618912, 332, '355'), ('916235064', 'carens_1027_gao__port_506298', 0.0008827356, 332, '355'), ('916235064', 'classe_a_1027_gao__port_506339', 0.0024717397, 332, '355'), ('916235064', 'ix20_1027_gao__port_506343', 0.0010095265, 332, '355'), ('916235064', 'note_1027_gao__port_506365', 0.0015965191, 332, '355'), ('916235064', 'a5_1027_gao__port_506200', 0.0015331907, 332, '355'), ('916235064', 'sx4_1027_gao__port_506348', 0.001492191, 332, '355'), ('916235064', 'sandero_1027_gao__port_506198', 0.001458861, 332, '355'), ('916235064', '3008_1027_gao__port_506385', 0.0056464295, 332, '355'), ('916235064', 'q50_1027_gao__port_506239', 0.0011167356, 332, '355'), ('916235064', 'latitude_1027_gao__port_506236', 0.0008021934, 332, '355'), ('916235064', 'v40_1027_gao__port_506391', 0.0017150971, 332, '355'), ('916235064', 'xsara_1027_gao__port_506087', 0.0009824217, 332, '355'), ('916235064', 'grand_c_max_1027_gao__port_506342', 0.0017963758, 332, '355'), ('916235064', 'swift_1027_gao__port_506149', 0.0015022043, 332, '355'), ('916235064', 'serie_1_1027_gao__port_506184', 0.0015142196, 332, '355'), ('916235064', 'xc70_1027_gao__port_506393', 0.0036210518, 332, '355'), ('916235064', 'master_1027_gao__port_506203', 0.007955813, 332, '355'), ('916235064', 'clio_1027_gao__port_506280', 0.0029576942, 332, '355'), ('916235064', 'duster_1027_gao__port_506216', 0.0007445999, 332, '355'), ('916235064', 'traveller_1027_gao__port_506403', 0.004294809, 332, '355'), ('916235064', 'tipo_1027_gao__port_506355', 0.001093204, 332, '355'), ('916235064', 'rav_4_1027_gao__port_506332', 0.0013606928, 332, '355'), ('916235064', 'coccinelle_1027_gao__port_506259', 0.0034949167, 332, '355'), ('916235064', 'spacetourer_1027_gao__port_506401', 0.003097593, 332, '355'), ('916235064', 'xe_1027_gao__port_506357', 0.0014473131, 332, '355'), ('916235064', 'ds3_1027_gao__port_506324', 0.0013095489, 332, '355'), ('916235064', 'mx_5_1027_gao__port_506098', 0.0025886188, 332, '355'), ('916235064', 'land_cruiser_1027_gao__port_506315', 0.009529189, 332, '355'), ('916235064', 'classe_b_1027_gao__port_506335', 0.0017217693, 332, '355'), ('916235064', '806_1027_gao__port_506088', 0.002561674, 332, '355'), ('916235064', 'rx_8_1027_gao__port_506046', 0.0036221757, 332, '355'), ('916235064', 'spark_1027_gao__port_506185', 0.0010079186, 332, '355'), ('916235064', '6_1027_gao__port_506171', 0.0011184748, 332, '355'), ('916235064', 'bravo_1027_gao__port_506080', 0.0014648373, 332, '355'), ('916235064', 'nx_1027_gao__port_506345', 0.0013685864, 332, '355'), ('916235064', 'sharan_1027_gao__port_506347', 0.0050927233, 332, '355'), ('916235064', 'x_type_1027_gao__port_506067', 0.0007805179, 332, '355'), ('916235064', 'jimny_1027_gao__port_506233', 0.004606294, 332, '355'), ('916235064', 'wrangler_1027_gao__port_506225', 0.0017996663, 332, '355'), ('916235064', 'c_crosser_1027_gao__port_506312', 0.0015929984, 332, '355'), ('916235064', 'v70_1027_gao__port_506278', 0.0019680876, 332, '355'), ('916235064', 'classe_e_1027_gao__port_506300', 0.001736928, 332, '355'), ('916235064', 'classe_v_1027_gao__port_506258', 0.012730503, 332, '355'), ('916235064', 'm3_1027_gao__port_506182', 0.002336998, 332, '355'), ('916235064', 'abarth_500_1027_gao__port_506226', 0.0040435675, 332, '355'), ('916235064', 'serie_6_1027_gao__port_506262', 0.0011316562, 332, '355'), ('916235064', 'modus_1027_gao__port_506146', 0.0018296492, 332, '355'), ('916235064', '3_1027_gao__port_506113', 0.001508383, 332, '355'), ('916235064', '405_1027_gao__port_506108', 0.008053445, 332, '355'), ('916235064', 'allroad_1027_gao__port_506297', 0.0010600166, 332, '355'), ('916235064', 'auris_1027_gao__port_506322', 0.0011524231, 332, '355'), ('916235064', 'galaxy_1027_gao__port_506143', 0.0025148888, 332, '355'), ('916235064', 'giulietta_1027_gao__port_506363', 0.0008652059, 332, '355'), ('916235064', '106_1027_gao__port_506073', 0.008268182, 332, '355'), ('916235064', 'classe_m_1027_gao__port_506154', 0.0030025651, 332, '355'), ('916235064', 'espace_1027_gao__port_506313', 0.0010649903, 332, '355'), ('916235064', 'panda_1027_gao__port_506189', 0.009028092, 332, '355'), ('916235064', 'rcz_1027_gao__port_506197', 0.0011296879, 332, '355'), ('916235064', '4007_1027_gao__port_506162', 0.00067946065, 332, '355'), ('916235064', 'classe_cl_1027_gao__port_506249', 0.0010861263, 332, '355'), ('916235064', 'leaf_1027_gao__port_506139', 0.0018038137, 332, '355'), ('916235064', 'octavia_1027_gao__port_506237', 0.0018602641, 332, '355'), ('916235064', 'ds4_1027_gao__port_506336', 0.002416746, 332, '355'), ('916235064', 'freelander_1027_gao__port_506084', 0.002347608, 332, '355'), ('916235064', 'evasion_1027_gao__port_506109', 0.0031143324, 332, '355'), ('916235064', 'punto_1027_gao__port_506106', 0.0019496921, 332, '355'), ('916235064', '2cv_1027_gao__port_506045', 0.0079725515, 332, '355'), ('916235064', 'x4_1027_gao__port_506392', 0.0017952994, 332, '355'), ('916235064', 'antara_1027_gao__port_506247', 0.0012470431, 332, '355'), ('916235064', 'murano_1027_gao__port_506316', 0.0006089692, 332, '355'), ('916235064', 'alto_1027_gao__port_506201', 0.009231133, 332, '355'), ('916235064', 'meriva_1027_gao__port_506353', 0.0013769487, 332, '355'), ('916235064', 'orlando_1027_gao__port_506305', 0.0018463959, 332, '355'), ('916235064', 'new_beetle_1027_gao__port_506050', 0.001163804, 332, '355'), ('916235064', '306_1027_gao__port_506145', 0.0035073452, 332, '355'), ('916235064', 'tiguan_1027_gao__port_506362', 0.0026828654, 332, '355'), ('916235064', 's_type_1027_gao__port_506101', 0.0011383498, 332, '355'), ('916235064', 'c1_1027_gao__port_506128', 0.0027513776, 332, '355'), ('916235064', 'vectra_1027_gao__port_506044', 0.0011988985, 332, '355'), ('916235064', 'outlander_1027_gao__port_506317', 0.0017126058, 332, '355'), ('916235064', '307_1027_gao__port_506074', 0.002001521, 332, '355'), ('916235064', 'a6_s6_1027_gao__port_506134', 0.001656968, 332, '355'), ('916235064', 'nemo_combi_1027_gao__port_506196', 0.0022667288, 332, '355'), ('916235064', 'berlingo_1027_gao__port_506194', 0.004663057, 332, '355'), ('916235064', 'partner_1027_gao__port_506285', 0.0039416086, 332, '355'), ('916235064', 'cayenne_1027_gao__port_506177', 0.0037987842, 332, '355'), ('916235064', 'quattroporte_1027_gao__port_506240', 0.002444546, 332, '355'), ('916235064', 'c_max_1027_gao__port_506282', 0.0013126658, 332, '355'), ('916235064', 'fabia_1027_gao__port_506396', 0.0052987942, 332, '355'), ('916235064', 'cx_3_1027_gao__port_506281', 0.0014464337, 332, '355'), ('916235064', 'x_trail_1027_gao__port_506264', 0.0018316876, 332, '355'), ('916235064', 'scirocco_1027_gao__port_506276', 0.0047913417, 332, '355'), ('916235064', 'matiz_1027_gao__port_506144', 0.0017560254, 332, '355'), ('916235064', 'tigra_1027_gao__port_506069', 0.0008542283, 332, '355'), ('916235064', 'escort_1027_gao__port_506091', 0.0048384164, 332, '355'), ('916235064', 'c2_1027_gao__port_506081', 0.0014904608, 332, '355'), ('916235064', 'mini_1027_gao__port_506168', 0.0011920558, 332, '355'), ('916235064', 'i30_1027_gao__port_506291', 0.0006327676, 332, '355'), ('916235064', 'picanto_1027_gao__port_506238', 0.0029750087, 332, '355'), ('916235064', 'mito_1027_gao__port_506072', 0.0015079068, 332, '355'), ('916235064', 'impreza_1027_gao__port_506085', 0.002018906, 332, '355'), ('916235064', 'kangoo_1027_gao__port_506235', 0.0065840106, 332, '355'), ('916235064', 'a4_1027_gao__port_506193', 0.0019874212, 332, '355'), ('916235064', 'cayman_1027_gao__port_506268', 0.0018141825, 332, '355'), ('916235064', 'sportage_1027_gao__port_506148', 0.0014278581, 332, '355'), ('916235064', 'up_1027_gao__port_506356', 0.0068628825, 332, '355'), ('916235064', 'optima_1027_gao__port_506386', 0.0008920425, 332, '355'), ('916235064', 'defender_1027_gao__port_506229', 0.006720554, 332, '355'), ('916235064', 'serie_2_1027_gao__port_506256', 0.002227904, 332, '355'), ('916235064', 'edge_1027_gao__port_506187', 0.0008750407, 332, '355'), ('916235064', 'r19_1027_gao__port_506110', 0.0049412195, 332, '355'), ('916235064', 'jetta_1027_gao__port_506304', 0.0036190315, 332, '355'), ('916235064', 'eos_1027_gao__port_506115', 0.0038919395, 332, '355'), ('916235064', 'accord_1027_gao__port_506214', 0.0020127334, 332, '355'), ('916235064', 'yaris_1027_gao__port_506334', 0.0032335576, 332, '355'), ('916235064', 'classe_cls_1027_gao__port_506289', 0.00078531896, 332, '355'), ('916235064', 'polo_1027_gao__port_506361', 0.004310373, 332, '355'), ('916235064', 'serie_4_1027_gao__port_506366', 0.0011476325, 332, '355'), ('916235064', 'mini_cabriolet_1027_gao__port_506204', 0.00083777454, 332, '355'), ('916235064', 'prius_1027_gao__port_506190', 0.001148931, 332, '355'), ('916235064', 'lodgy_1027_gao__port_506188', 0.0020176605, 332, '355'), ('916235064', 'serie_7_1027_gao__port_506307', 0.0012479239, 332, '355'), ('916235064', 'c15_1027_gao__port_506055', 0.017707583, 332, '355'), ('916235064', 'kadjar_1027_gao__port_506389', 0.0012508628, 332, '355'), ('916235064', 'insignia_1027_gao__port_506364', 0.0016439934, 332, '355'), ('916235064', '308_1027_gao__port_506279', 0.0021241026, 332, '355'), ('916235064', 'roomster_1027_gao__port_506241', 0.0018016247, 332, '355'), ('916235064', '80_1027_gao__port_506057', 0.00461312, 332, '355'), ('916235064', '309_1027_gao__port_506063', 0.013521381, 332, '355'), ('916235064', 'tucson_1027_gao__port_506320', 0.0021239803, 332, '355'), ('916235064', 'x3_1027_gao__port_506212', 0.00089785014, 332, '355'), ('916235064', 'xf_1027_gao__port_506263', 0.0011169681, 332, '355'), ('916235064', '2008_1027_gao__port_506394', 0.0026413908, 332, '355'), ('916235064', 'passat_1027_gao__port_506306', 0.0014977059, 332, '355'), ('916235064', 'compass_1027_gao__port_506260', 0.0032565005, 332, '355'), ('916235064', 'twingo_1027_gao__port_506309', 0.006499429, 332, '355'), ('916235064', 'micra_1027_gao__port_506221', 0.003586026, 332, '355'), ('916235064', 'golf_1027_gao__port_506155', 0.0031925703, 332, '355'), ('916235064', 'soul_1027_gao__port_506176', 0.0012892126, 332, '355'), ('916235064', 'rapid_1027_gao__port_506398', 0.0025912889, 332, '355'), ('916235064', 'forester_1027_gao__port_506360', 0.0022761947, 332, '355'), ('916235064', 'slk_1027_gao__port_506210', 0.0015845025, 332, '355'), ('916235064', 'forfour_1027_gao__port_506341', 0.002180045, 332, '355'), ('916235064', 'serie_5_1027_gao__port_506209', 0.0013697062, 332, '355'), ('916235064', 'xj_1027_gao__port_506170', 0.002601112, 332, '355'), ('916235064', 'pajero_1027_gao__port_506097', 0.005211489, 332, '355'), ('916235064', 'agila_1027_gao__port_506119', 0.0048497473, 332, '355'), ('916235064', 'a6_1027_gao__port_506163', 0.0018911762, 332, '355'), ('916235064', 'fox_1027_gao__port_506092', 0.0008464736, 332, '355'), ('916235064', 'boxster_1027_gao__port_506267', 0.0015943095, 332, '355'), ('916235064', 'altea_1027_gao__port_506246', 0.0021396047, 332, '355'), ('916235064', 'samurai_1027_gao__port_506047', 0.0062490194, 332, '355'), ('916235064', 'trax_1027_gao__port_506296', 0.0019445211, 332, '355'), ('916235064', 'getz_1027_gao__port_506058', 0.0016386382, 332, '355'), ('916235064', 'cherokee_1027_gao__port_506269', 0.0029814462, 332, '355'), ('916235064', 'koleos_1027_gao__port_506378', 0.0015592856, 332, '355'), ('916235064', 'z_series_1027_gao__port_506123', 0.0016563424, 332, '355'), ('916235064', 'ecosport_1027_gao__port_506271', 0.0013233194, 332, '355'), ('916235064', 'space_star_1027_gao__port_506277', 0.0021146685, 332, '355'), ('916235064', 'rs3_sportback_1027_gao__port_506207', 0.0019117384, 332, '355'), ('916235064', 'civic_1027_gao__port_506141', 0.0026894875, 332, '355'), ('916235064', 'talisman_1027_gao__port_506390', 0.0007615137, 332, '355'), ('916235064', 'f_pace_1027_gao__port_506314', 0.0016168407, 332, '355'), ('916235064', 'classe_c_1027_gao__port_506299', 0.0017942205, 332, '355'), ('916235064', 'tt_1027_gao__port_506075', 0.0013935706, 332, '355'), ('916235064', 'pathfinder_1027_gao__port_506183', 0.0016518055, 332, '355'), ('916235064', '156_1027_gao__port_506157', 0.0015445983, 332, '355'), ('916235064', 'cx_5_1027_gao__port_506228', 0.0014417169, 332, '355'), ('916235064', 'scenic_1027_gao__port_506255', 0.0016088252, 332, '355'), ('916235064', 'yeti_1027_gao__port_506358', 0.0020917866, 332, '355'), ('916235064', 'mustang_1027_gao__port_506053', 0.010049084, 332, '355'), ('916235064', 'stilo_1027_gao__port_506060', 0.001083067, 332, '355'), ('916235064', 'ateca_1027_gao__port_506382', 0.001701416, 332, '355'), ('916235064', 'fiorino_1027_gao__port_506217', 0.009198153, 332, '355'), ('916235064', 'classe_glk_1027_gao__port_506290', 0.0017019588, 332, '355'), ('916235064', 'fortwo_1027_gao__port_506230', 0.0016010761, 332, '355'), ('916235064', 'cruze_1027_gao__port_506186', 0.0010055227, 332, '355'), ('916235064', '107_1027_gao__port_506213', 0.0016278058, 332, '355'), ('916235064', 'aygo_1027_gao__port_506248', 0.0032430028, 332, '355'), ('916235064', 'rx_1027_gao__port_506354', 0.0010633959, 332, '355'), ('916235064', '500_1027_gao__port_506245', 0.0016357987, 332, '355'), ('916235064', 'bora_1027_gao__port_506104', 0.0038158388, 332, '355'), ('916235064', 'transit_1027_gao__port_506111', 0.0048614433, 332, '355'), ('916235064', 'pt_cruiser_1027_gao__port_506054', 0.0019167394, 332, '355'), ('916235064', 'patrol_1027_gao__port_506068', 0.004239265, 332, '355'), ('916235064', 'r8_1027_gao__port_506156', 0.0012723109, 332, '355'), ('916235064', 'xm_1027_gao__port_506102', 0.0022679009, 332, '355'), ('916235064', 's60_1027_gao__port_506191', 0.0031999652, 332, '355'), ('916235064', 'aveo_1027_gao__port_506158', 0.0038401852, 332, '355'), ('916235064', 'captiva_1027_gao__port_506159', 0.0017195303, 332, '355'), ('916235064', 'ax_1027_gao__port_506153', 0.006895004, 332, '355'), ('916235064', 'rexton_1027_gao__port_506107', 0.0013023567, 332, '355'), ('916235064', 'camaro_1027_gao__port_506056', 0.002490206, 332, '355'), ('916235064', 'ypsilon_1027_gao__port_506131', 0.0019548559, 332, '355'), ('916235064', 'delta_1027_gao__port_506165', 0.001400206, 332, '355'), ('916235064', 'c4_1027_gao__port_506370', 0.0013011897, 332, '355'), ('916235064', 'zx_1027_gao__port_506161', 0.004593175, 332, '355'), ('916235064', 'verso_1027_gao__port_506242', 0.00077223626, 332, '355'), ('916235064', 'superb_1027_gao__port_506327', 0.0019940822, 332, '355'), ('916235064', 'r5_1027_gao__port_506253', 0.009545209, 332, '355'), ('916235064', 'caddy_1027_gao__port_506330', 0.0138224, 332, '355'), ('916235064', 'x5_1027_gao__port_506243', 0.0011205815, 332, '355'), ('916235064', 'f_type_1027_gao__port_506231', 0.0008301577, 332, '355'), ('916235064', 'fusion_1027_gao__port_506096', 0.0012672874, 332, '355'), ('916235064', 'dokker_1027_gao__port_506331', 0.0053587947, 332, '355'), ('916235064', '205_1027_gao__port_506062', 0.00668314, 332, '355'), ('916235064', 'macan_1027_gao__port_506195', 0.0015577503, 332, '355'), ('916235064', 'tourneo_1027_gao__port_506369', 0.0064018476, 332, '355'), ('916235064', '108_1027_gao__port_506384', 0.005264762, 332, '355'), ('916235064', '9_3_1027_gao__port_506071', 0.00083760766, 332, '355'), ('916235064', 'mondeo_1027_gao__port_506116', 0.0014400047, 332, '355'), ('916235064', 'cr_v_1027_gao__port_506164', 0.0016417459, 332, '355'), ('916235064', 'c30_1027_gao__port_506090', 0.0017485756, 332, '355'), ('916235064', 'pulsar_1027_gao__port_506397', 0.0012021667, 332, '355'), ('916235064', 'ibiza_1027_gao__port_506273', 0.0037229566, 332, '355'), ('916235064', 'a1_1027_gao__port_506338', 0.0012348281, 332, '355'), ('916235064', 'matrix_1027_gao__port_506140', 0.0007078527, 332, '355'), ('916235064', 'carnival_1027_gao__port_506136', 0.0022816118, 332, '355'), ('916235064', 'xantia_1027_gao__port_506086', 0.0021959587, 332, '355'), ('916235064', 'terrano_1027_gao__port_506083', 0.0020296997, 332, '355'), ('916235064', 'q3_1027_gao__port_506275', 0.0011267024, 332, '355'), ('916235064', 'hr_v_1027_gao__port_506283', 0.0017808204, 332, '355'), ('916235064', 'expert_1027_gao__port_506142', 0.0073685693, 332, '355'), ('916235064', 'multivan_1027_gao__port_506383', 0.006503073, 332, '355'), ('916235064', 'venga_1027_gao__port_506380', 0.0008005335, 332, '355'), ('916235064', 'scudo_1027_gao__port_506129', 0.005592788, 332, '355'), ('916235064', 'laguna_1027_gao__port_506368', 0.00071358937, 332, '355'), ('916235064', 'vel_satis_1027_gao__port_506130', 0.002726631, 332, '355'), ('916235064', 'b_max_1027_gao__port_506367', 0.0017251571, 332, '355'), ('916235064', 'ignis_1027_gao__port_506292', 0.0043558395, 332, '355'), ('916235064', '159_1027_gao__port_506064', 0.0010783295, 332, '355'), ('916235064', 'grande_punto_1027_gao__port_506138', 0.002363524, 332, '355'), ('916235064', 'logan_1027_gao__port_506167', 0.004397477, 332, '355'), ('916235064', 's_max_1027_gao__port_506223', 0.0012530426, 332, '355'), ('916235064', 'caravelle_1027_gao__port_506351', 0.0030294454, 332, '355'), ('916235064', 'adam_1027_gao__port_506079', 0.001053976, 332, '355'), ('916235064', '406_1027_gao__port_506132', 0.0013575103, 332, '355'), ('916235064', 'q30_1027_gao__port_506293', 0.000971782, 332, '355'), ('916235064', 'almera_1027_gao__port_506089', 0.00102415, 332, '355'), ('916235064', 'corsa_1027_gao__port_506095', 0.0025206185, 332, '355'), ('916235064', 'corolla_1027_gao__port_506120', 0.002682078, 332, '355'), ('916235064', 'xc60_1027_gao__port_506388', 0.001899344, 332, '355'), ('916235064', 'viano_1027_gao__port_506211', 0.0026945882, 332, '355'), ('916235064', 'pro_cee_d_1027_gao__port_506274', 0.000832092, 332, '355'), ('916235064', 'a3_1027_gao__port_506321', 0.0037377726, 332, '355'), ('916235064', 'v50_1027_gao__port_506150', 0.00079212553, 332, '355'), ('916235064', 'voyager_1027_gao__port_506169', 0.0030528062, 332, '355'), ('916235064', 'corvette_1027_gao__port_506049', 0.0037226465, 332, '355'), ('916235064', 'rio_1027_gao__port_506379', 0.0017744392, 332, '355'), ('916235064', 'jazz_1027_gao__port_506252', 0.0015309273, 332, '355'), ('916235064', '200_1027_gao__port_506112', 0.004086386, 332, '355'), ('916235064', 'tts_1027_gao__port_506199', 0.0011863346, 332, '355'), ('916235064', 'zafira_1027_gao__port_506287', 0.002695633, 332, '355'), ('916235064', 'asx_1027_gao__port_506266', 0.0011409748, 332, '355'), ('916235064', '607_1027_gao__port_506118', 0.0012533511, 332, '355'), ('916235064', '207_1027_gao__port_506103', 0.0015150409, 332, '355'), ('916235064', 'classe_s_1027_gao__port_506301', 0.0031656933, 332, '355'), ('916235064', 'c6_1027_gao__port_506105', 0.0017350123, 332, '355'), ('916235064', 'express_1027_gao__port_506137', 0.016720155, 332, '355'), ('916235064', 'classe_gla_1027_gao__port_506352', 0.0018259264, 332, '355'), ('916235064', 'v60_1027_gao__port_506333', 0.002146718, 332, '355'), ('916235064', 'ka_1027_gao__port_506180', 0.0014153257, 332, '355'), ('916235064', 'range_rover_1027_gao__port_506254', 0.0020553614, 332, '355'), ('916235064', 'discovery_1027_gao__port_506375', 0.0022965262, 332, '355'), ('916235064', 'classe_r_1027_gao__port_506270', 0.00139479, 332, '355'), ('916235064', 'transporter_1027_gao__port_506319', 0.01196677, 332, '355'), ('916235064', 'cee_d_1027_gao__port_506288', 0.0010550352, 332, '355'), ('916235064', 'zoe_1027_gao__port_506244', 0.0020716116, 332, '355'), ('916235064', 'i20_1027_gao__port_506284', 0.001787332, 332, '355'), ('916235064', 'gtv_1027_gao__port_506059', 0.0057210173, 332, '355'), ('916235064', 's4_avant_1027_gao__port_506261', 0.0027662837, 332, '355'), ('916235064', 'x1_1027_gao__port_506372', 0.0017148767, 332, '355'), ('916235064', 'autres_1027_gao__port_506127', 0.0048248083, 332, '355'), ('916235064', '208_1027_gao__port_506359', 0.001869147, 332, '355'), ('916235064', 'c8_1027_gao__port_506135', 0.0012582964, 332, '355'), ('916235064', 'astra_1027_gao__port_506215', 0.0012628183, 332, '355'), ('916235064', '2_1027_gao__port_506151', 0.000924605, 332, '355'), ('916235064', 'doblo_1027_gao__port_506251', 0.007466486, 332, '355'), ('916235064', '807_1027_gao__port_506152', 0.00072915806, 332, '355'), ('916235064', '206_1027_gao__port_506126', 0.0010386561, 332, '355'), ('916235064', 'a7_1027_gao__port_506373', 0.0006912201, 332, '355'), ('916235064', 'renegade_1027_gao__port_506346', 0.002142143, 332, '355')]]} begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 0 insert into MTRPhoto.class_photo_score (thcl, photo_id, hashtag_id, score) values (%s,%s,%s,%s) on duplicate key update score = values(score) time used for this insertion : 8.344650268554688e-06 save missing photos in datou_result : time spend for datou_step_exec : 47.494118213653564 time spend to save output : 14.057841539382935 total time spend for step 1 : 61.5519597530365 step2:argmax Fri Feb 28 03:37:57 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec After prepare type args : Here we display some param of map_info ! map_filenames : {'temp/1740710215_2762130_916235064_6293d1bb790dc6902450e7c572b7d10b.jpg': 916235064} map_photo_id_path_extension : {916235064: {'path': 'temp/1740710215_2762130_916235064_6293d1bb790dc6902450e7c572b7d10b.jpg', 'extension': 'jpg'}} map_subphoto_mainphoto : {} Beginning of datou_step Argmax ! calculate argmax for thcl : 355 After datou_step_exec type output : map_portfolio_photo : len 0 keys : dict_keys([]) Inside saveOutput : final : True verbose : True photo_id : 916235064 output[photo_id] : [('916235064', 'c15_1027_gao__port_506055', 0.017707583, 332, '355'), 'temp/1740710215_2762130_916235064_6293d1bb790dc6902450e7c572b7d10b.jpg'] begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 1 insert ignore into MTRBack.photo_hashtag_ids (photo_id, hashtag_id, type) values (%s,%s,%s) insert ignore into MTRBack.photo_hashtag_ids (photo_id, hashtag_id, type) values (%s,%s,%s) first line : ('916235064', '2049863950', '332') ... last line : ('916235064', '2049863950', '332') time used for this insertion : 0.6360764503479004 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 1 insert into MTRPhoto.class_photo_score (thcl, photo_id, hashtag_id, score) values (%s,%s,%s,%s) on duplicate key update score = values(score) time used for this insertion : 0.01742076873779297 len list_finale : 1, len picture : 1 begin to insert list_values into mtr_datou_result : length of list_values in save_final : 1 insert ignore into MTRPhoto.mtr_datou_result (mtd_id, mtr_portfolio_id,mtr_photo_id,result,result_long,result_double,hashtag_id,proba, mtr_current_id) values (%s,%s,%s,%s,%s,%s,%s,%s,%s) on duplicate key update mtr_portfolio_id = mtr_portfolio_id list_values : [('2', None, '916235064', 'c15_1027_gao__port_506055', None, None, '2049863950', '0.017707583', None)] time used for this insertion : 0.11373424530029297 saving photo_ids in datou_result photo id not in port begin to insert list_values into mtr_datou_result : length of list_values in save_final : 0 insert ignore into MTRPhoto.mtr_datou_result (mtd_id, mtr_portfolio_id,mtr_photo_id,result,result_long,result_double,hashtag_id,proba, mtr_current_id) values (%s,%s,%s,%s,%s,%s,%s,%s,%s) on duplicate key update mtr_portfolio_id = mtr_portfolio_id list_values : [] time used for this insertion : 4.291534423828125e-06 save missing photos in datou_result : time spend for datou_step_exec : 0.00025963783264160156 time spend to save output : 0.7675325870513916 total time spend for step 2 : 0.7677922248840332 caffe_path_current : About to save ! 2 After save, about to update current ! datou_cur_ids : [] len(datou.list_steps) : 2 output : {'916235064': [('916235064', 'c15_1027_gao__port_506055', 0.017707583, 332, '355'), 'temp/1740710215_2762130_916235064_6293d1bb790dc6902450e7c572b7d10b.jpg']} ############################### TEST tfhub2 ################################ TEST TFHUB2 ######################## test with use_multi_inputs=0 ######################## Inside batchDatouExec : verbose : True ##### chargement datou SELECT name, created_at,limit_max FROM MTRDatou.mtr_datou WHERE id=4567 SELECT mtd.id, mtdt.`type`, mtd.`param`, mtd.param_json, mtdt.nb_input, mtdt.nb_output, mtdt.prod, mtdt.is_local, mtdt.is_datou_depend, mtdt.is_photo_id_local FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_step_types mtdt WHERE mtdt.`id`=mtd.`type` AND mtd.mtd_id=4567 SELECT mtd.id, mtd.mtd_id, mdsdt.id, mdsdt.name, mdsdt.description, msid.output_or_input, msid.data_order_id, mdsdt.type FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_steptype_io_datatypes msid, MTRDatou.mtr_datou_step_data_types mdsdt WHERE mtd.`type`=msid.`mtr_datou_step_type` AND mtd.mtd_id= 4567 AND msid.data_type=mdsdt.id SELECT mts_id_output, id_output, mts_id_input, id_input FROM MTRDatou.mtr_datou_step_by_step WHERE mtd_id=4567 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : step 12835 tfhub_classification2 is not linked in the step_by_step architecture ! WARNING : step 12836 argmax is not linked in the step_by_step architecture ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! DataTypes for each output/input checked ! no param json to modify List Step Type Loaded in datou : tfhub_classification2, argmax list_input_json : [] ##### fin chargement datou ##### chargement data ##### Call load_data_input : nb_thread : 5 origin SELECT photo_id, url FROM MTRBack.photos ph WHERE photo_id IN (1171252784,1171252764,1171252487) Found this number of photos: 3 ##### Call download_photos : nb_thread : 5 begin to download photo : 1171252487 begin to download photo : 1171252764 begin to download photo : 1171252784 download finish for photo 1171252784 download finish for photo 1171252764 download finish for photo 1171252487 we have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB ##### After download_photos length of list_filenames : 3 ; length of list_pids : 3 ; length of list_args : 3 ##### After load_data_input time to download the photos : 0.20179152488708496 #### fin chargement data Blocking on flush ? No conitnuing About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : True number of steps : 2 step1:tfhub_classification2 Fri Feb 28 03:37:58 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec After prepare type args : Here we display some param of map_info ! map_filenames : {'temp/1740710278_2762130_1171252784_5a3c5d3bb155a7a116f67ded51bffb59.jpg': 1171252784, 'temp/1740710278_2762130_1171252764_29d5179a892cc50aadc9d67245534b59.jpg': 1171252764, 'temp/1740710278_2762130_1171252487_5ebdd6b0a6bb39942a3808ed114806de.jpg': 1171252487} map_photo_id_path_extension : {1171252784: {'path': 'temp/1740710278_2762130_1171252784_5a3c5d3bb155a7a116f67ded51bffb59.jpg', 'extension': 'jpg'}, 1171252764: {'path': 'temp/1740710278_2762130_1171252764_29d5179a892cc50aadc9d67245534b59.jpg', 'extension': 'jpg'}, 1171252487: {'path': 'temp/1740710278_2762130_1171252487_5ebdd6b0a6bb39942a3808ed114806de.jpg', 'extension': 'jpg'}} map_subphoto_mainphoto : {} Beginning of datou_step TFHub with tf2 ! multi_thcl or not :False multi_thcl_cond or not :False dic_thcl : {'3609': 1} we are using the classfication for only one thcl 3609 begin to check gpu status inside check gpu memory havn't enough memory gpu , need / 3096 l 3632 free memory gpu now : 986 wait 20 seconds inside check gpu memory havn't enough memory gpu , need / 3096 l 3632 free memory gpu now : 986 wait 20 seconds inside check gpu memory havn't enough memory gpu , need / 3096 l 3632 free memory gpu now : 1207 wait 20 seconds inside check gpu memory havn't enough memory gpu , need / 3096 l 3632 free memory gpu now : 1207 wait 20 seconds inside check gpu memory havn't enough memory gpu , need / 3096 l 3632 free memory gpu now : 1207 wait 20 seconds inside check gpu memory havn't enough memory gpu , need / 3096 l 3632 free memory gpu now : 1207 wait 20 seconds 2025-02-28 03:40:07.622618: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-02-28 03:40:07.623544: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-28 03:40:07.623619: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-28 03:40:07.623664: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-28 03:40:07.626172: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-28 03:40:07.626234: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-28 03:40:07.630992: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-28 03:40:07.632438: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-28 03:40:07.643402: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-28 03:40:07.644440: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-28 03:40:07.645974: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-02-28 03:40:07.675321: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-02-28 03:40:07.677163: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f5c60000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-02-28 03:40:07.677194: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-02-28 03:40:07.680736: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x2c1cf2c0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-02-28 03:40:07.680768: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-02-28 03:40:07.681763: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-28 03:40:07.681905: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-28 03:40:07.681941: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-28 03:40:07.682043: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-28 03:40:07.682088: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-28 03:40:07.682145: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-28 03:40:07.682207: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-28 03:40:07.682270: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-28 03:40:07.683477: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-28 03:40:07.683562: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-28 03:40:07.683616: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-28 03:40:07.683633: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-28 03:40:07.683646: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-28 03:40:07.684930: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 3096 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) l 3637 free memory gpu now : 1207 max_wait_temp : 6 max_wait : 5 1 Physical GPUs, 1 Logical GPUs tagging for thcl : 3609 To do loadFromThcl(), then load ParamDescType : thcl3609 get_desc_type_from_thcl : type of cat SELECT id, mtr_user_id, name, pb_hashtag_id, hashtag_id_list, button_legend_list, portfolio_id_lists, photo_hashtag_type, photo_desc_type, svm_limit, limit_tagging, is_public, live, created_at, updated_at, type_classification FROM MTRDatou.classification_theme WHERE `id` IN (3609) thcls : [{'id': 3609, 'mtr_user_id': 31, 'name': 'tfhub_19_06_2023', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'jrm,pcm,pcnc,pehd,tapis_vide', 'svm_portfolios_learning': '9336903,9336904,9336905,9336906,9336909', 'photo_hashtag_type': 4674, 'photo_desc_type': 5832, 'type_classification': 'tf_classification2', 'hashtag_id_list': '495916461,560181804,1284539308,628944319,2107748999'}] thcl {'id': 3609, 'mtr_user_id': 31, 'name': 'tfhub_19_06_2023', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'jrm,pcm,pcnc,pehd,tapis_vide', 'svm_portfolios_learning': '9336903,9336904,9336905,9336906,9336909', 'photo_hashtag_type': 4674, 'photo_desc_type': 5832, 'type_classification': 'tf_classification2', 'hashtag_id_list': '495916461,560181804,1284539308,628944319,2107748999'} Update svm_hashtag_type_desc : 5832 SELECT * FROM MTRDatou.photo_desc_type_params WHERE id in (5832) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5832, 'tfhub_19_06_2023', 1280, 1280, 'tfhub_19_06_2023', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2023, 6, 19, 12, 55, 22), datetime.datetime(2023, 6, 19, 12, 55, 22)) model_name : tfhub_19_06_2023 model_param file didn't exist model_name : tfhub_19_06_2023 model_type : tf_classification2 list file need : ['Confusion_Matrix.png', 'Precision_Recall_jrm.jpg', 'Precision_Recall_pcm.jpg', 'Precision_Recall_pcnc.jpg', 'Precision_Recall_pehd.jpg', 'Precision_Recall_tapis_vide.jpg', 'Result_Summary.txt', 'checkpoint', 'model_checkpoint.ckpt.data-00000-of-00002', 'model_checkpoint.ckpt.data-00001-of-00002', 'model_checkpoint.ckpt.index', 'model_weights.h5'] file exist in s3 : ['Confusion_Matrix.png', 'Precision_Recall_jrm.jpg', 'Precision_Recall_pcm.jpg', 'Precision_Recall_pcnc.jpg', 'Precision_Recall_pehd.jpg', 'Precision_Recall_tapis_vide.jpg', 'Result_Summary.txt', 'checkpoint', 'model_checkpoint.ckpt.data-00000-of-00002', 'model_checkpoint.ckpt.data-00001-of-00002', 'model_checkpoint.ckpt.index', 'model_weights.h5'] file manque in s3 : [] 2025-02-28 03:40:14.953752: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.02G (3246391296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-28 03:40:14.954296: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.72G (2921752064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-28 03:40:14.954794: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.45G (2629576704 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-28 03:40:14.955346: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.20G (2366618880 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-28 03:40:14.955918: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.98G (2129957120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-28 03:40:14.956455: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.79G (1916961536 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-28 03:40:14.956990: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.61G (1725265408 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-28 03:40:14.957546: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.45G (1552738816 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-28 03:40:14.958093: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.30G (1397465088 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory local folder : /data/models_weight/tfhub_19_06_2023 /data/models_weight/tfhub_19_06_2023/Confusion_Matrix.png size_local : 57753 size in s3 : 57753 create time local : 2023-06-22 17:09:38 create time in s3 : 2023-06-19 10:55:15 Confusion_Matrix.png already exist and didn't need to update /data/models_weight/tfhub_19_06_2023/Precision_Recall_jrm.jpg size_local : 79724 size in s3 : 79724 create time local : 2023-06-22 17:09:38 create time in s3 : 2023-06-19 10:55:20 Precision_Recall_jrm.jpg already exist and didn't need to update /data/models_weight/tfhub_19_06_2023/Precision_Recall_pcm.jpg size_local : 83556 size in s3 : 83556 create time local : 2023-06-22 17:09:38 create time in s3 : 2023-06-19 10:55:15 Precision_Recall_pcm.jpg already exist and didn't need to update /data/models_weight/tfhub_19_06_2023/Precision_Recall_pcnc.jpg size_local : 74107 size in s3 : 74107 create time local : 2023-06-22 17:09:38 create time in s3 : 2023-06-19 10:55:20 Precision_Recall_pcnc.jpg already exist and didn't need to update /data/models_weight/tfhub_19_06_2023/Precision_Recall_pehd.jpg size_local : 72705 size in s3 : 72705 create time local : 2023-06-22 17:09:39 create time in s3 : 2023-06-19 10:55:20 Precision_Recall_pehd.jpg already exist and didn't need to update /data/models_weight/tfhub_19_06_2023/Precision_Recall_tapis_vide.jpg size_local : 70874 size in s3 : 70874 create time local : 2023-06-22 17:09:39 create time in s3 : 2023-06-19 10:55:15 Precision_Recall_tapis_vide.jpg already exist and didn't need to update /data/models_weight/tfhub_19_06_2023/Result_Summary.txt size_local : 642 size in s3 : 642 create time local : 2023-06-22 17:09:39 create time in s3 : 2023-06-19 10:55:22 Result_Summary.txt already exist and didn't need to update /data/models_weight/tfhub_19_06_2023/checkpoint size_local : 99 size in s3 : 99 create time local : 2023-06-22 17:09:39 create time in s3 : 2023-06-19 10:55:22 checkpoint already exist and didn't need to update /data/models_weight/tfhub_19_06_2023/model_checkpoint.ckpt.data-00000-of-00002 size_local : 216488 size in s3 : 216488 create time local : 2023-06-22 17:09:39 create time in s3 : 2023-06-19 10:55:22 model_checkpoint.ckpt.data-00000-of-00002 already exist and didn't need to update /data/models_weight/tfhub_19_06_2023/model_checkpoint.ckpt.data-00001-of-00002 size_local : 32279708 size in s3 : 32279708 create time local : 2023-06-22 17:09:40 create time in s3 : 2023-06-19 10:55:21 model_checkpoint.ckpt.data-00001-of-00002 already exist and didn't need to update /data/models_weight/tfhub_19_06_2023/model_checkpoint.ckpt.index size_local : 43546 size in s3 : 43546 create time local : 2023-06-22 17:09:40 create time in s3 : 2023-06-19 10:55:22 model_checkpoint.ckpt.index already exist and didn't need to update /data/models_weight/tfhub_19_06_2023/model_weights.h5 size_local : 16499144 size in s3 : 16499144 create time local : 2023-06-22 17:09:40 create time in s3 : 2023-06-19 10:55:15 model_weights.h5 already exist and didn't need to update ERROR in datou_step_exec, will save and exit ! assertion failed: [0] [Op:Assert] name: EagerVariableNameReuse File "/home/admin/workarea/git/Velours/python/mtr/datou/datou_lib.py", line 2329, in datou_exec output = datou_step_exec(sNext, args, cache, context, map_info, verbose, mtr_user_id) File "/home/admin/workarea/git/Velours/python/mtr/datou/datou_lib.py", line 2523, in datou_step_exec return lib_process.datou_step_tfhub2(param, json_param, args, cache, context, map_info, verbose) File "/home/admin/workarea/git/Velours/python/mtr/datou/lib_step_exec/lib_step_process.py", line 3138, in datou_step_tfhub2 this_model = model_evaluator(model_name, model_type=model_type, fc_size=fc_size,use_multi_inputs=use_multi_inputs) File "/home/admin/workarea/git/Velours/python/mtr/tfhub2/evaluate.py", line 156, in __init__ self.model, _, _ = create_tfhub_model(module_handle=self.tfhub_module, File "/home/admin/workarea/git/Velours/python/mtr/tfhub2/evaluate.py", line 77, in create_tfhub_model hub.KerasLayer(module_handle, trainable=do_fine_tuning, name="module"), File "/home/admin/.local/lib/python3.8/site-packages/tensorflow_hub/keras_layer.py", line 152, in __init__ self._func = load_module(handle, tags, self._load_options) File "/home/admin/.local/lib/python3.8/site-packages/tensorflow_hub/keras_layer.py", line 421, in load_module return module_v2.load(handle, tags=tags, options=set_load_options) File "/home/admin/.local/lib/python3.8/site-packages/tensorflow_hub/module_v2.py", line 106, in load obj = tf.compat.v1.saved_model.load_v2(module_path, tags=tags) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/saved_model/load.py", line 578, in load return load_internal(export_dir, tags) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/saved_model/load.py", line 602, in load_internal loader = loader_cls(object_graph_proto, File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/saved_model/load.py", line 123, in __init__ self._load_all() File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/saved_model/load.py", line 134, in _load_all self._load_nodes() File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/saved_model/load.py", line 264, in _load_nodes node, setter = self._recreate(proto, node_id) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/saved_model/load.py", line 370, in _recreate return factory[kind]() File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/saved_model/load.py", line 363, in "variable": lambda: self._recreate_variable(proto.variable), File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/saved_model/load.py", line 426, in _recreate_variable return variables.Variable( File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/variables.py", line 261, in __call__ return cls._variable_v2_call(*args, **kwargs) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/variables.py", line 243, in _variable_v2_call return previous_getter( File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/variables.py", line 66, in getter return captured_getter(captured_previous, **kwargs) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/saved_model/load.py", line 418, in uninitialized_variable_creator return resource_variable_ops.UninitializedVariable(**kwargs) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/variables.py", line 263, in __call__ return super(VariableMetaclass, cls).__call__(*args, **kwargs) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/resource_variable_ops.py", line 1795, in __init__ handle = _variable_handle_from_shape_and_dtype( File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/resource_variable_ops.py", line 174, in _variable_handle_from_shape_and_dtype gen_logging_ops._assert( # pylint: disable=protected-access File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/gen_logging_ops.py", line 55, in _assert _ops.raise_from_not_ok_status(e, name) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/ops.py", line 6653, in raise_from_not_ok_status six.raise_from(core._status_to_exception(e.code, message), None) File "", line 3, in raise_from [1171252784, 1171252764, 1171252487] map_info['map_portfolio_photo'] : {} final : True mtd_id 4567 list_pids : [1171252784, 1171252764, 1171252487] begin to insert list_values into mtr_datou_result : length of list_values in save_final : 3 insert ignore into MTRPhoto.mtr_datou_result (mtd_id, mtr_portfolio_id,mtr_photo_id,result,result_long,result_double,hashtag_id,proba, mtr_current_id) values (%s,%s,%s,%s,%s,%s,%s,%s,%s) on duplicate key update mtr_portfolio_id = mtr_portfolio_id list_values : [('4567', None, '1171252784', "[>, , , , , 'assertion failed: [0] [Op:Assert] name: EagerVariableNameReuse']", '-1', '-1.0', '501120777', '1.0', None), ('4567', None, '1171252764', "[>, , , , , 'assertion failed: [0] [Op:Assert] name: EagerVariableNameReuse']", '-1', '-1.0', '501120777', '1.0', None), ('4567', None, '1171252487', "[>, , , , , 'assertion failed: [0] [Op:Assert] name: EagerVariableNameReuse']", '-1', '-1.0', '501120777', '1.0', None)] time used for this insertion : 0.02858138084411621 save_final ERROR in last step tfhub_classification2, assertion failed: [0] [Op:Assert] name: EagerVariableNameReuse time spend for datou_step_exec : 136.62033557891846 time spend to save output : 0.03406333923339844 total time spend for step 0 : 136.65439891815186 need to delete datou_research and reload, so keep current state 1 caffe_path_current : About to save ! 2 After save, about to update current ! datou_cur_ids : [] len(datou.list_steps) : 2 output : None probably due to empty image bug ERROR expected : {'1171252784': [(1171252784, 'jrm', 0.9677492, 4674, '3609'), 'temp/1687511175_1882837_1171252784_5a3c5d3bb155a7a116f67ded51bffb59.jpg'], '1171252764': [(1171252764, 'jrm', 0.9853587, 4674, '3609'), 'temp/1687511175_1882837_1171252764_29d5179a892cc50aadc9d67245534b59.jpg'], '1171252487': [(1171252487, 'jrm', 0.9262757, 4674, '3609'), 'temp/1687511175_1882837_1171252487_5ebdd6b0a6bb39942a3808ed114806de.jpg']} got : None ######################## test with use_multi_inputs=1 ######################## Inside batchDatouExec : verbose : True ##### chargement datou SELECT name, created_at,limit_max FROM MTRDatou.mtr_datou WHERE id=4621 SELECT mtd.id, mtdt.`type`, mtd.`param`, mtd.param_json, mtdt.nb_input, mtdt.nb_output, mtdt.prod, mtdt.is_local, mtdt.is_datou_depend, mtdt.is_photo_id_local FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_step_types mtdt WHERE mtdt.`id`=mtd.`type` AND mtd.mtd_id=4621 SELECT mtd.id, mtd.mtd_id, mdsdt.id, mdsdt.name, mdsdt.description, msid.output_or_input, msid.data_order_id, mdsdt.type FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_steptype_io_datatypes msid, MTRDatou.mtr_datou_step_data_types mdsdt WHERE mtd.`type`=msid.`mtr_datou_step_type` AND mtd.mtd_id= 4621 AND msid.data_type=mdsdt.id SELECT mts_id_output, id_output, mts_id_input, id_input FROM MTRDatou.mtr_datou_step_by_step WHERE mtd_id=4621 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : step 12927 tfhub_classification2 is not linked in the step_by_step architecture ! WARNING : step 12928 argmax is not linked in the step_by_step architecture ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! DataTypes for each output/input checked ! no param json to modify List Step Type Loaded in datou : tfhub_classification2, argmax list_input_json : [] ##### fin chargement datou ##### chargement data ##### Call load_data_input : nb_thread : 5 origin SELECT photo_id, url FROM MTRBack.photos ph WHERE photo_id IN (1171291875,1171275372,1171275314) Found this number of photos: 3 ##### Call download_photos : nb_thread : 5 begin to download photo : 1171275314 begin to download photo : 1171275372 begin to download photo : 1171291875 download finish for photo 1171275314 download finish for photo 1171275372 download finish for photo 1171291875 we have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB ##### After download_photos length of list_filenames : 3 ; length of list_pids : 3 ; length of list_args : 3 ##### After load_data_input time to download the photos : 0.6028883457183838 #### fin chargement data Blocking on flush ? No conitnuing About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : True number of steps : 2 step1:tfhub_classification2 Fri Feb 28 03:40:15 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec After prepare type args : Here we display some param of map_info ! map_filenames : {'temp/1740710415_2762130_1171275314_6e0a72c8fa00d5e4b018bd689b547133.jpg': 1171275314, 'temp/1740710415_2762130_1171275372_76d81364ff7df843bff095f45c07ba35.jpg': 1171275372, 'temp/1740710415_2762130_1171291875_b62cd9e0d976b143f86fe82d072798c0.jpg': 1171291875} map_photo_id_path_extension : {1171275314: {'path': 'temp/1740710415_2762130_1171275314_6e0a72c8fa00d5e4b018bd689b547133.jpg', 'extension': 'jpg'}, 1171275372: {'path': 'temp/1740710415_2762130_1171275372_76d81364ff7df843bff095f45c07ba35.jpg', 'extension': 'jpg'}, 1171291875: {'path': 'temp/1740710415_2762130_1171291875_b62cd9e0d976b143f86fe82d072798c0.jpg', 'extension': 'jpg'}} map_subphoto_mainphoto : {} Beginning of datou_step TFHub with tf2 ! multi_thcl or not :False multi_thcl_cond or not :False dic_thcl : {'3655': 1} we are using the classfication for only one thcl 3655 begin to check gpu status inside check gpu memory havn't enough memory gpu , need / 3096 l 3632 free memory gpu now : 7 wait 20 seconds inside check gpu memory havn't enough memory gpu , need / 3096 l 3632 free memory gpu now : 7 wait 20 seconds inside check gpu memory havn't enough memory gpu , need / 3096 l 3632 free memory gpu now : 7 wait 20 seconds inside check gpu memory havn't enough memory gpu , need / 3096 l 3632 free memory gpu now : 7 wait 20 seconds inside check gpu memory havn't enough memory gpu , need / 3096 l 3632 free memory gpu now : 7 wait 20 seconds inside check gpu memory havn't enough memory gpu , need / 3096 l 3632 free memory gpu now : 7 wait 20 seconds l 3637 free memory gpu now : 7 max_wait_temp : 6 max_wait : 5 1 Physical GPUs, 1 Logical GPUs tagging for thcl : 3655 To do loadFromThcl(), then load ParamDescType : thcl3655 get_desc_type_from_thcl : type of cat SELECT id, mtr_user_id, name, pb_hashtag_id, hashtag_id_list, button_legend_list, portfolio_id_lists, photo_hashtag_type, photo_desc_type, svm_limit, limit_tagging, is_public, live, created_at, updated_at, type_classification FROM MTRDatou.classification_theme WHERE `id` IN (3655) thcls : [{'id': 3655, 'mtr_user_id': 31, 'name': 'tfhub_18_7_2023', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'pcm,pcnc,jrm,pehd,tapis_vide', 'svm_portfolios_learning': '9336904,9336905,9336903,9336906,9336909', 'photo_hashtag_type': 4723, 'photo_desc_type': 5862, 'type_classification': 'tf_classification2', 'hashtag_id_list': '560181804,1284539308,495916461,628944319,2107748999'}] thcl {'id': 3655, 'mtr_user_id': 31, 'name': 'tfhub_18_7_2023', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'pcm,pcnc,jrm,pehd,tapis_vide', 'svm_portfolios_learning': '9336904,9336905,9336903,9336906,9336909', 'photo_hashtag_type': 4723, 'photo_desc_type': 5862, 'type_classification': 'tf_classification2', 'hashtag_id_list': '560181804,1284539308,495916461,628944319,2107748999'} Update svm_hashtag_type_desc : 5862 SELECT * FROM MTRDatou.photo_desc_type_params WHERE id in (5862) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5862, 'tfhub_18_7_2023', 1280, 1280, 'tfhub_18_7_2023', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2023, 7, 18, 22, 46, 29), datetime.datetime(2023, 7, 18, 22, 46, 29)) model_name : tfhub_18_7_2023 model_param file didn't exist model_name : tfhub_18_7_2023 model_type : tf_classification2 list file need : ['Confusion_Matrix.png', 'Precision_Recall_jrm.jpg', 'Precision_Recall_pcm.jpg', 'Precision_Recall_pcnc.jpg', 'Precision_Recall_pehd.jpg', 'Precision_Recall_tapis_vide.jpg', 'Result_Summary.txt', 'checkpoint', 'model_checkpoint.ckpt.data-00000-of-00002', 'model_checkpoint.ckpt.data-00001-of-00002', 'model_checkpoint.ckpt.index', 'model_weights.h5'] file exist in s3 : ['Confusion_Matrix.png', 'Precision_Recall_jrm.jpg', 'Precision_Recall_pcm.jpg', 'Precision_Recall_pcnc.jpg', 'Precision_Recall_pehd.jpg', 'Precision_Recall_tapis_vide.jpg', 'Result_Summary.txt', 'checkpoint', 'model_checkpoint.ckpt.data-00000-of-00002', 'model_checkpoint.ckpt.data-00001-of-00002', 'model_checkpoint.ckpt.index', 'model_weights.h5'] file manque in s3 : [] local folder : /data/models_weight/tfhub_18_7_2023 /data/models_weight/tfhub_18_7_2023/Confusion_Matrix.png size_local : 54360 size in s3 : 54360 create time local : 2023-08-11 11:22:56 create time in s3 : 2023-07-18 20:46:28 Confusion_Matrix.png already exist and didn't need to update /data/models_weight/tfhub_18_7_2023/Precision_Recall_jrm.jpg size_local : 72583 size in s3 : 72583 create time local : 2023-08-11 11:22:56 create time in s3 : 2023-07-18 20:46:23 Precision_Recall_jrm.jpg already exist and didn't need to update /data/models_weight/tfhub_18_7_2023/Precision_Recall_pcm.jpg size_local : 81681 size in s3 : 81681 create time local : 2023-08-11 11:22:56 create time in s3 : 2023-07-18 20:46:17 Precision_Recall_pcm.jpg already exist and didn't need to update /data/models_weight/tfhub_18_7_2023/Precision_Recall_pcnc.jpg size_local : 79510 size in s3 : 79510 create time local : 2023-08-11 11:22:56 create time in s3 : 2023-07-18 20:46:23 Precision_Recall_pcnc.jpg already exist and didn't need to update /data/models_weight/tfhub_18_7_2023/Precision_Recall_pehd.jpg size_local : 59936 size in s3 : 59936 create time local : 2023-08-11 11:22:57 create time in s3 : 2023-07-18 20:46:23 Precision_Recall_pehd.jpg already exist and didn't need to update /data/models_weight/tfhub_18_7_2023/Precision_Recall_tapis_vide.jpg size_local : 78974 size in s3 : 78974 create time local : 2023-08-11 11:22:57 create time in s3 : 2023-07-18 20:46:17 Precision_Recall_tapis_vide.jpg already exist and didn't need to update /data/models_weight/tfhub_18_7_2023/Result_Summary.txt size_local : 642 size in s3 : 642 create time local : 2023-08-11 11:22:57 create time in s3 : 2023-07-18 20:46:23 Result_Summary.txt already exist and didn't need to update /data/models_weight/tfhub_18_7_2023/checkpoint size_local : 99 size in s3 : 99 create time local : 2023-08-11 11:22:57 create time in s3 : 2023-07-18 20:46:23 checkpoint already exist and didn't need to update /data/models_weight/tfhub_18_7_2023/model_checkpoint.ckpt.data-00000-of-00002 size_local : 216529 size in s3 : 216529 create time local : 2023-08-11 11:22:57 create time in s3 : 2023-07-18 20:46:17 model_checkpoint.ckpt.data-00000-of-00002 already exist and didn't need to update /data/models_weight/tfhub_18_7_2023/model_checkpoint.ckpt.data-00001-of-00002 size_local : 32279748 size in s3 : 32279748 create time local : 2023-08-11 11:22:58 create time in s3 : 2023-07-18 20:46:19 model_checkpoint.ckpt.data-00001-of-00002 already exist and didn't need to update /data/models_weight/tfhub_18_7_2023/model_checkpoint.ckpt.index size_local : 43546 size in s3 : 43546 create time local : 2023-08-11 11:22:58 create time in s3 : 2023-07-18 20:46:19 model_checkpoint.ckpt.index already exist and didn't need to update /data/models_weight/tfhub_18_7_2023/model_weights.h5 size_local : 16500868 size in s3 : 16500868 create time local : 2023-08-11 11:22:58 create time in s3 : 2023-07-18 20:46:18 model_weights.h5 already exist and didn't need to update ERROR in datou_step_exec, will save and exit ! assertion failed: [0] [Op:Assert] name: EagerVariableNameReuse File "/home/admin/workarea/git/Velours/python/mtr/datou/datou_lib.py", line 2329, in datou_exec output = datou_step_exec(sNext, args, cache, context, map_info, verbose, mtr_user_id) File "/home/admin/workarea/git/Velours/python/mtr/datou/datou_lib.py", line 2523, in datou_step_exec return lib_process.datou_step_tfhub2(param, json_param, args, cache, context, map_info, verbose) File "/home/admin/workarea/git/Velours/python/mtr/datou/lib_step_exec/lib_step_process.py", line 3138, in datou_step_tfhub2 this_model = model_evaluator(model_name, model_type=model_type, fc_size=fc_size,use_multi_inputs=use_multi_inputs) File "/home/admin/workarea/git/Velours/python/mtr/tfhub2/evaluate.py", line 156, in __init__ self.model, _, _ = create_tfhub_model(module_handle=self.tfhub_module, File "/home/admin/workarea/git/Velours/python/mtr/tfhub2/evaluate.py", line 62, in create_tfhub_model fe_layer = hub.KerasLayer(module_handle, trainable=do_fine_tuning, name="module", File "/home/admin/.local/lib/python3.8/site-packages/tensorflow_hub/keras_layer.py", line 152, in __init__ self._func = load_module(handle, tags, self._load_options) File "/home/admin/.local/lib/python3.8/site-packages/tensorflow_hub/keras_layer.py", line 421, in load_module return module_v2.load(handle, tags=tags, options=set_load_options) File "/home/admin/.local/lib/python3.8/site-packages/tensorflow_hub/module_v2.py", line 106, in load obj = tf.compat.v1.saved_model.load_v2(module_path, tags=tags) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/saved_model/load.py", line 578, in load return load_internal(export_dir, tags) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/saved_model/load.py", line 602, in load_internal loader = loader_cls(object_graph_proto, File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/saved_model/load.py", line 123, in __init__ self._load_all() File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/saved_model/load.py", line 134, in _load_all self._load_nodes() File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/saved_model/load.py", line 264, in _load_nodes node, setter = self._recreate(proto, node_id) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/saved_model/load.py", line 370, in _recreate return factory[kind]() File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/saved_model/load.py", line 363, in "variable": lambda: self._recreate_variable(proto.variable), File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/saved_model/load.py", line 426, in _recreate_variable return variables.Variable( File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/variables.py", line 261, in __call__ return cls._variable_v2_call(*args, **kwargs) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/variables.py", line 243, in _variable_v2_call return previous_getter( File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/variables.py", line 66, in getter return captured_getter(captured_previous, **kwargs) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/saved_model/load.py", line 418, in uninitialized_variable_creator return resource_variable_ops.UninitializedVariable(**kwargs) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/variables.py", line 263, in __call__ return super(VariableMetaclass, cls).__call__(*args, **kwargs) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/resource_variable_ops.py", line 1795, in __init__ handle = _variable_handle_from_shape_and_dtype( File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/resource_variable_ops.py", line 174, in _variable_handle_from_shape_and_dtype gen_logging_ops._assert( # pylint: disable=protected-access File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/gen_logging_ops.py", line 55, in _assert _ops.raise_from_not_ok_status(e, name) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/ops.py", line 6653, in raise_from_not_ok_status six.raise_from(core._status_to_exception(e.code, message), None) File "", line 3, in raise_from [1171275314, 1171275372, 1171291875] map_info['map_portfolio_photo'] : {} final : True mtd_id 4621 list_pids : [1171275314, 1171275372, 1171291875] begin to insert list_values into mtr_datou_result : length of list_values in save_final : 3 insert ignore into MTRPhoto.mtr_datou_result (mtd_id, mtr_portfolio_id,mtr_photo_id,result,result_long,result_double,hashtag_id,proba, mtr_current_id) values (%s,%s,%s,%s,%s,%s,%s,%s,%s) on duplicate key update mtr_portfolio_id = mtr_portfolio_id list_values : [('4621', None, '1171275314', "[>, , , , , 'assertion failed: [0] [Op:Assert] name: EagerVariableNameReuse']", '-1', '-1.0', '501120777', '1.0', None), ('4621', None, '1171275372', "[>, , , , , 'assertion failed: [0] [Op:Assert] name: EagerVariableNameReuse']", '-1', '-1.0', '501120777', '1.0', None), ('4621', None, '1171291875', "[>, , , , , 'assertion failed: [0] [Op:Assert] name: EagerVariableNameReuse']", '-1', '-1.0', '501120777', '1.0', None)] time used for this insertion : 0.22445940971374512 save_final ERROR in last step tfhub_classification2, assertion failed: [0] [Op:Assert] name: EagerVariableNameReuse time spend for datou_step_exec : 134.92890214920044 time spend to save output : 0.2250983715057373 total time spend for step 0 : 135.15400052070618 need to delete datou_research and reload, so keep current state 1 caffe_path_current : About to save ! 2 After save, about to update current ! datou_cur_ids : [] len(datou.list_steps) : 2 output : None probably due to empty image bug ERROR expected : {'1171291875': [(1171291875, 'tapis_vide', 0.97062814, 4723, '3655'), 'temp/1691745841_1143057_1171291875_b62cd9e0d976b143f86fe82d072798c0.jpg'], '1171275372': [(1171275372, 'tapis_vide', 0.9674145, 4723, '3655'), 'temp/1691745841_1143057_1171275372_76d81364ff7df843bff095f45c07ba35.jpg'], '1171275314': [(1171275314, 'tapis_vide', 0.96509415, 4723, '3655'), 'temp/1691745841_1143057_1171275314_6e0a72c8fa00d5e4b018bd689b547133.jpg']} got : None ERROR tfhub2 FAILED ############################### TEST ordonner ################################ To do loadFromThcl(), then load ParamDescType : thcl358 get_desc_type_from_thcl : type of cat SELECT id, mtr_user_id, name, pb_hashtag_id, hashtag_id_list, button_legend_list, portfolio_id_lists, photo_hashtag_type, photo_desc_type, svm_limit, limit_tagging, is_public, live, created_at, updated_at, type_classification FROM MTRDatou.classification_theme WHERE `id` IN (358) thcls : [{'id': 358, 'mtr_user_id': 31, 'name': 'car_orientation_0111', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'FirstUploadExperveo_vignette__port_505674,CAR_EXTERIEUR_Roue__port_503398,FirstUploadExperveo_carrosseriegrosplan_VIndanslamoquette__port_506486,FirstUploadExperveo_carrosseriegrosplan_siegegrosplan__port_506485,CAR_EXTERIEUR_Cote_droit_axe_avant__port_504465,CAR_EXTERIEUR_Cote_gauche_axe_arriere__port_504198,CAR_EXTERIEUR_Face_avant_axe_droit__port_504451,CAR_EXTERIEUR_angle_avant_gauche_axe_avant__port_504235,FirstUploadExperveo_vin__port_505675,CAR_EXTERIEUR_cote_droite__port_504108,CAR_INTERIEUR_avant_volant_class_6_levierdevitesse__port_506565,FirstUploadExperveo_carrosseriegrosplan_carrosserie__port_506483,CAR_EXTERIEUR_Angle_arriere_gauche_axe_arriere__port_504201,cartegrise_orientation__port_505064,CAR_EXTERIEUR_Angle_arriere_droit_axe_arriere__port_504217,CAR_INTERIEUR_avant_vue-arriere_class_1__port_506531,CAR_EXTERIEUR_Face_arriere_axe_droit__port_504218,CAR_EXTERIEUR_Cote_droit_axe_arriere__port_504214,CAR_EXTERIEUR_Angle_avant_droit__port_504087,FirstUploadExperveo_carrosseriegrosplan_morceauderoue__port_506484,CAR_INTERIEUR_avant_volant_class_6_class_2__port_506563,CAR_EXTERIEUR_Angle_arriere_droit__port_504160,CAR_EXTERIEUR_arriere__port_504184,CAR_INTERIEUR_avant_volant_class_6_boutonrond__port_506562,INTERIEUR_Compteur_kilometrique__port_503644,CAR_INTERIEUR_avant_vue_gauche_habitacle_class_1__port_506494,CAR_EXTERIEUR_Angle_arriere_gauche__port_504170,CAR_EXTERIEUR_Angle_avant_droit_axe_arriere__port_504226,CAR_EXTERIEUR_Face_arriere_axe_gauche__port_504202,CAR_EXTERIEUR_moteur__port_503704,FirstUploadExperveo_carrosseriegrosplan_class_6__port_506487,CAR_INTERIEUR_siege_arriere_class_1__port_506551,CAR_EXTERIEUR_avant__port_504146,CAR_EXTERIEUR_Angle_arriere_droit_axe_droit__port_504215,CAR_EXTERIEUR_Angle_avant_droit_axe_droit__port_504225,CAR_INTERIEUR_avant_volant_class_6_ecrangrosplan__port_506564,FirstUploadExperveo_carrosseriegrosplan_moteurgrosplanetdegat__port_506482,CAR_INTERIEUR_coffre__port_503412,FirstUploadExperveo_rouetranche__port_505677,UploadPhotoImmatBest_class_1__port_505051,CAR_INTERIEUR_avant_vue-arriere_class_2__port_506532,CAR_EXTERIEUR_angle_avant_gauche__port_504098,CAR_EXTERIEUR_face_avant_axe_gauche__port_504236,CAR_INTERIEUR_avant_vue_droite_habitacle_class_1__port_506540,CAR_EXTERIEUR_cote_gauche_axe_avant__port_504233,CAR_EXTERIEUR_roue_de_secour__port_503763,CAR_EXTERIEUR_Angle_arriere_gauche_axe_gauche__port_504199,CAR_EXTERIEUR_cote_gauche__port_504017,CAR_INTERIEUR_avant_volant_class_1__port_506503,CAR_INTERIEUR_avant_volant_class_2__port_506504,CAR_EXTERIEUR_angle_avant_gauche_axe_gauche__port_504234', 'svm_portfolios_learning': '505674,503398,506486,506485,504465,504198,504451,504235,505675,504108,506565,506483,504201,505064,504217,506531,504218,504214,504087,506484,506563,504160,504184,506562,503644,506494,504170,504226,504202,503704,506487,506551,504146,504215,504225,506564,506482,503412,505677,505051,506532,504098,504236,506540,504233,503763,504199,504017,506503,506504,504234', 'photo_hashtag_type': 337, 'photo_desc_type': 3392, 'type_classification': 'caffe', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 358, 'mtr_user_id': 31, 'name': 'car_orientation_0111', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'FirstUploadExperveo_vignette__port_505674,CAR_EXTERIEUR_Roue__port_503398,FirstUploadExperveo_carrosseriegrosplan_VIndanslamoquette__port_506486,FirstUploadExperveo_carrosseriegrosplan_siegegrosplan__port_506485,CAR_EXTERIEUR_Cote_droit_axe_avant__port_504465,CAR_EXTERIEUR_Cote_gauche_axe_arriere__port_504198,CAR_EXTERIEUR_Face_avant_axe_droit__port_504451,CAR_EXTERIEUR_angle_avant_gauche_axe_avant__port_504235,FirstUploadExperveo_vin__port_505675,CAR_EXTERIEUR_cote_droite__port_504108,CAR_INTERIEUR_avant_volant_class_6_levierdevitesse__port_506565,FirstUploadExperveo_carrosseriegrosplan_carrosserie__port_506483,CAR_EXTERIEUR_Angle_arriere_gauche_axe_arriere__port_504201,cartegrise_orientation__port_505064,CAR_EXTERIEUR_Angle_arriere_droit_axe_arriere__port_504217,CAR_INTERIEUR_avant_vue-arriere_class_1__port_506531,CAR_EXTERIEUR_Face_arriere_axe_droit__port_504218,CAR_EXTERIEUR_Cote_droit_axe_arriere__port_504214,CAR_EXTERIEUR_Angle_avant_droit__port_504087,FirstUploadExperveo_carrosseriegrosplan_morceauderoue__port_506484,CAR_INTERIEUR_avant_volant_class_6_class_2__port_506563,CAR_EXTERIEUR_Angle_arriere_droit__port_504160,CAR_EXTERIEUR_arriere__port_504184,CAR_INTERIEUR_avant_volant_class_6_boutonrond__port_506562,INTERIEUR_Compteur_kilometrique__port_503644,CAR_INTERIEUR_avant_vue_gauche_habitacle_class_1__port_506494,CAR_EXTERIEUR_Angle_arriere_gauche__port_504170,CAR_EXTERIEUR_Angle_avant_droit_axe_arriere__port_504226,CAR_EXTERIEUR_Face_arriere_axe_gauche__port_504202,CAR_EXTERIEUR_moteur__port_503704,FirstUploadExperveo_carrosseriegrosplan_class_6__port_506487,CAR_INTERIEUR_siege_arriere_class_1__port_506551,CAR_EXTERIEUR_avant__port_504146,CAR_EXTERIEUR_Angle_arriere_droit_axe_droit__port_504215,CAR_EXTERIEUR_Angle_avant_droit_axe_droit__port_504225,CAR_INTERIEUR_avant_volant_class_6_ecrangrosplan__port_506564,FirstUploadExperveo_carrosseriegrosplan_moteurgrosplanetdegat__port_506482,CAR_INTERIEUR_coffre__port_503412,FirstUploadExperveo_rouetranche__port_505677,UploadPhotoImmatBest_class_1__port_505051,CAR_INTERIEUR_avant_vue-arriere_class_2__port_506532,CAR_EXTERIEUR_angle_avant_gauche__port_504098,CAR_EXTERIEUR_face_avant_axe_gauche__port_504236,CAR_INTERIEUR_avant_vue_droite_habitacle_class_1__port_506540,CAR_EXTERIEUR_cote_gauche_axe_avant__port_504233,CAR_EXTERIEUR_roue_de_secour__port_503763,CAR_EXTERIEUR_Angle_arriere_gauche_axe_gauche__port_504199,CAR_EXTERIEUR_cote_gauche__port_504017,CAR_INTERIEUR_avant_volant_class_1__port_506503,CAR_INTERIEUR_avant_volant_class_2__port_506504,CAR_EXTERIEUR_angle_avant_gauche_axe_gauche__port_504234', 'svm_portfolios_learning': '505674,503398,506486,506485,504465,504198,504451,504235,505675,504108,506565,506483,504201,505064,504217,506531,504218,504214,504087,506484,506563,504160,504184,506562,503644,506494,504170,504226,504202,503704,506487,506551,504146,504215,504225,506564,506482,503412,505677,505051,506532,504098,504236,506540,504233,503763,504199,504017,506503,506504,504234', 'photo_hashtag_type': 337, 'photo_desc_type': 3392, 'type_classification': 'caffe', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 3392 ['FirstUploadExperveo_vignette__port_505674', 'CAR_EXTERIEUR_Roue__port_503398', 'FirstUploadExperveo_carrosseriegrosplan_VIndanslamoquette__port_506486', 'FirstUploadExperveo_carrosseriegrosplan_siegegrosplan__port_506485', 'CAR_EXTERIEUR_Cote_droit_axe_avant__port_504465', 'CAR_EXTERIEUR_Cote_gauche_axe_arriere__port_504198', 'CAR_EXTERIEUR_Face_avant_axe_droit__port_504451', 'CAR_EXTERIEUR_angle_avant_gauche_axe_avant__port_504235', 'FirstUploadExperveo_vin__port_505675', 'CAR_EXTERIEUR_cote_droite__port_504108', 'CAR_INTERIEUR_avant_volant_class_6_levierdevitesse__port_506565', 'FirstUploadExperveo_carrosseriegrosplan_carrosserie__port_506483', 'CAR_EXTERIEUR_Angle_arriere_gauche_axe_arriere__port_504201', 'cartegrise_orientation__port_505064', 'CAR_EXTERIEUR_Angle_arriere_droit_axe_arriere__port_504217', 'CAR_INTERIEUR_avant_vue-arriere_class_1__port_506531', 'CAR_EXTERIEUR_Face_arriere_axe_droit__port_504218', 'CAR_EXTERIEUR_Cote_droit_axe_arriere__port_504214', 'CAR_EXTERIEUR_Angle_avant_droit__port_504087', 'FirstUploadExperveo_carrosseriegrosplan_morceauderoue__port_506484', 'CAR_INTERIEUR_avant_volant_class_6_class_2__port_506563', 'CAR_EXTERIEUR_Angle_arriere_droit__port_504160', 'CAR_EXTERIEUR_arriere__port_504184', 'CAR_INTERIEUR_avant_volant_class_6_boutonrond__port_506562', 'INTERIEUR_Compteur_kilometrique__port_503644', 'CAR_INTERIEUR_avant_vue_gauche_habitacle_class_1__port_506494', 'CAR_EXTERIEUR_Angle_arriere_gauche__port_504170', 'CAR_EXTERIEUR_Angle_avant_droit_axe_arriere__port_504226', 'CAR_EXTERIEUR_Face_arriere_axe_gauche__port_504202', 'CAR_EXTERIEUR_moteur__port_503704', 'FirstUploadExperveo_carrosseriegrosplan_class_6__port_506487', 'CAR_INTERIEUR_siege_arriere_class_1__port_506551', 'CAR_EXTERIEUR_avant__port_504146', 'CAR_EXTERIEUR_Angle_arriere_droit_axe_droit__port_504215', 'CAR_EXTERIEUR_Angle_avant_droit_axe_droit__port_504225', 'CAR_INTERIEUR_avant_volant_class_6_ecrangrosplan__port_506564', 'FirstUploadExperveo_carrosseriegrosplan_moteurgrosplanetdegat__port_506482', 'CAR_INTERIEUR_coffre__port_503412', 'FirstUploadExperveo_rouetranche__port_505677', 'UploadPhotoImmatBest_class_1__port_505051', 'CAR_INTERIEUR_avant_vue-arriere_class_2__port_506532', 'CAR_EXTERIEUR_angle_avant_gauche__port_504098', 'CAR_EXTERIEUR_face_avant_axe_gauche__port_504236', 'CAR_INTERIEUR_avant_vue_droite_habitacle_class_1__port_506540', 'CAR_EXTERIEUR_cote_gauche_axe_avant__port_504233', 'CAR_EXTERIEUR_roue_de_secour__port_503763', 'CAR_EXTERIEUR_Angle_arriere_gauche_axe_gauche__port_504199', 'CAR_EXTERIEUR_cote_gauche__port_504017', 'CAR_INTERIEUR_avant_volant_class_1__port_506503', 'CAR_INTERIEUR_avant_volant_class_2__port_506504', 'CAR_EXTERIEUR_angle_avant_gauche_axe_gauche__port_504234'] 51 SELECT hashtag_id,hashtag FROM MTRBack.hashtags where hashtag in ('FirstUploadExperveo_vignette__port_505674','CAR_EXTERIEUR_Roue__port_503398','FirstUploadExperveo_carrosseriegrosplan_VIndanslamoquette__port_506486','FirstUploadExperveo_carrosseriegrosplan_siegegrosplan__port_506485','CAR_EXTERIEUR_Cote_droit_axe_avant__port_504465','CAR_EXTERIEUR_Cote_gauche_axe_arriere__port_504198','CAR_EXTERIEUR_Face_avant_axe_droit__port_504451','CAR_EXTERIEUR_angle_avant_gauche_axe_avant__port_504235','FirstUploadExperveo_vin__port_505675','CAR_EXTERIEUR_cote_droite__port_504108','CAR_INTERIEUR_avant_volant_class_6_levierdevitesse__port_506565','FirstUploadExperveo_carrosseriegrosplan_carrosserie__port_506483','CAR_EXTERIEUR_Angle_arriere_gauche_axe_arriere__port_504201','cartegrise_orientation__port_505064','CAR_EXTERIEUR_Angle_arriere_droit_axe_arriere__port_504217','CAR_INTERIEUR_avant_vue-arriere_class_1__port_506531','CAR_EXTERIEUR_Face_arriere_axe_droit__port_504218','CAR_EXTERIEUR_Cote_droit_axe_arriere__port_504214','CAR_EXTERIEUR_Angle_avant_droit__port_504087','FirstUploadExperveo_carrosseriegrosplan_morceauderoue__port_506484','CAR_INTERIEUR_avant_volant_class_6_class_2__port_506563','CAR_EXTERIEUR_Angle_arriere_droit__port_504160','CAR_EXTERIEUR_arriere__port_504184','CAR_INTERIEUR_avant_volant_class_6_boutonrond__port_506562','INTERIEUR_Compteur_kilometrique__port_503644','CAR_INTERIEUR_avant_vue_gauche_habitacle_class_1__port_506494','CAR_EXTERIEUR_Angle_arriere_gauche__port_504170','CAR_EXTERIEUR_Angle_avant_droit_axe_arriere__port_504226','CAR_EXTERIEUR_Face_arriere_axe_gauche__port_504202','CAR_EXTERIEUR_moteur__port_503704','FirstUploadExperveo_carrosseriegrosplan_class_6__port_506487','CAR_INTERIEUR_siege_arriere_class_1__port_506551','CAR_EXTERIEUR_avant__port_504146','CAR_EXTERIEUR_Angle_arriere_droit_axe_droit__port_504215','CAR_EXTERIEUR_Angle_avant_droit_axe_droit__port_504225','CAR_INTERIEUR_avant_volant_class_6_ecrangrosplan__port_506564','FirstUploadExperveo_carrosseriegrosplan_moteurgrosplanetdegat__port_506482','CAR_INTERIEUR_coffre__port_503412','FirstUploadExperveo_rouetranche__port_505677','UploadPhotoImmatBest_class_1__port_505051','CAR_INTERIEUR_avant_vue-arriere_class_2__port_506532','CAR_EXTERIEUR_angle_avant_gauche__port_504098','CAR_EXTERIEUR_face_avant_axe_gauche__port_504236','CAR_INTERIEUR_avant_vue_droite_habitacle_class_1__port_506540','CAR_EXTERIEUR_cote_gauche_axe_avant__port_504233','CAR_EXTERIEUR_roue_de_secour__port_503763','CAR_EXTERIEUR_Angle_arriere_gauche_axe_gauche__port_504199','CAR_EXTERIEUR_cote_gauche__port_504017','CAR_INTERIEUR_avant_volant_class_1__port_506503','CAR_INTERIEUR_avant_volant_class_2__port_506504','CAR_EXTERIEUR_angle_avant_gauche_axe_gauche__port_504234'); 51 dict_keys(['cartegrise_orientation__port_505064', 'car_exterieur_angle_arriere_droit_axe_arriere__port_504217', 'car_exterieur_angle_arriere_droit_axe_droit__port_504215', 'car_exterieur_angle_arriere_droit__port_504160', 'car_exterieur_angle_arriere_gauche_axe_arriere__port_504201', 'car_exterieur_angle_arriere_gauche_axe_gauche__port_504199', 'car_exterieur_angle_arriere_gauche__port_504170', 'car_exterieur_angle_avant_droit_axe_arriere__port_504226', 'car_exterieur_angle_avant_droit_axe_droit__port_504225', 'car_exterieur_angle_avant_droit__port_504087', 'car_exterieur_angle_avant_gauche_axe_avant__port_504235', 'car_exterieur_angle_avant_gauche_axe_gauche__port_504234', 'car_exterieur_angle_avant_gauche__port_504098', 'car_exterieur_arriere__port_504184', 'car_exterieur_avant__port_504146', 'car_exterieur_cote_droite__port_504108', 'car_exterieur_cote_droit_axe_arriere__port_504214', 'car_exterieur_cote_droit_axe_avant__port_504465', 'car_exterieur_cote_gauche_axe_arriere__port_504198', 'car_exterieur_cote_gauche_axe_avant__port_504233', 'car_exterieur_cote_gauche__port_504017', 'car_exterieur_face_arriere_axe_droit__port_504218', 'car_exterieur_face_arriere_axe_gauche__port_504202', 'car_exterieur_face_avant_axe_droit__port_504451', 'car_exterieur_face_avant_axe_gauche__port_504236', 'car_exterieur_moteur__port_503704', 'car_exterieur_roue_de_secour__port_503763', 'car_exterieur_roue__port_503398', 'car_interieur_avant_volant_class_1__port_506503', 'car_interieur_avant_volant_class_2__port_506504', 'car_interieur_avant_volant_class_6_boutonrond__port_506562', 'car_interieur_avant_volant_class_6_class_2__port_506563', 'car_interieur_avant_volant_class_6_ecrangrosplan__port_506564', 'car_interieur_avant_volant_class_6_levierdevitesse__port_506565', 'car_interieur_avant_vue-arriere_class_1__port_506531', 'car_interieur_avant_vue-arriere_class_2__port_506532', 'car_interieur_avant_vue_droite_habitacle_class_1__port_506540', 'car_interieur_avant_vue_gauche_habitacle_class_1__port_506494', 'car_interieur_coffre__port_503412', 'car_interieur_siege_arriere_class_1__port_506551', 'firstuploadexperveo_carrosseriegrosplan_carrosserie__port_506483', 'firstuploadexperveo_carrosseriegrosplan_class_6__port_506487', 'firstuploadexperveo_carrosseriegrosplan_morceauderoue__port_506484', 'firstuploadexperveo_carrosseriegrosplan_moteurgrosplanetdegat__port_506482', 'firstuploadexperveo_carrosseriegrosplan_siegegrosplan__port_506485', 'firstuploadexperveo_carrosseriegrosplan_vindanslamoquette__port_506486', 'firstuploadexperveo_rouetranche__port_505677', 'firstuploadexperveo_vignette__port_505674', 'firstuploadexperveo_vin__port_505675', 'interieur_compteur_kilometrique__port_503644', 'uploadphotoimmatbest_class_1__port_505051']) select photo_hashtag_type from MTRDatou.classification_theme where id = 358 thcl : 358 photo_hashtag_type : 337 SELECT phi.hashtag_id , phi.photo_id FROM MTRBack.photo_hashtag_ids phi, MTRUser.mtr_portfolio_photos mp where phi.type = 337 and phi.photo_id = mp.mtr_photo_id and mp.mtr_portfolio_id =510365; {510365: [(917973295, 1), (917973297, 1), (917973302, 1), (917973293, 7), (917973296, 11), (917973300, 11), (917973286, 13), (917973289, 13), (917973301, 24), (917973285, 29), (917973290, 29), (917973299, 29), (917973304, 35), (917973287, 36), (917973298, 36), (917973305, 36), (917973292, 37), (917973291, 41), (917973303, 41), (917973294, 42), (917973288, 46)]} ############################### TEST rotate ################################ test rotate only Inside batchDatouExec : verbose : True ##### chargement datou SELECT name, created_at,limit_max FROM MTRDatou.mtr_datou WHERE id=230 SELECT mtd.id, mtdt.`type`, mtd.`param`, mtd.param_json, mtdt.nb_input, mtdt.nb_output, mtdt.prod, mtdt.is_local, mtdt.is_datou_depend, mtdt.is_photo_id_local FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_step_types mtdt WHERE mtdt.`id`=mtd.`type` AND mtd.mtd_id=230 SELECT mtd.id, mtd.mtd_id, mdsdt.id, mdsdt.name, mdsdt.description, msid.output_or_input, msid.data_order_id, mdsdt.type FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_steptype_io_datatypes msid, MTRDatou.mtr_datou_step_data_types mdsdt WHERE mtd.`type`=msid.`mtr_datou_step_type` AND mtd.mtd_id= 230 AND msid.data_type=mdsdt.id SELECT mts_id_output, id_output, mts_id_input, id_input FROM MTRDatou.mtr_datou_step_by_step WHERE mtd_id=230 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! no param json to modify List Step Type Loaded in datou : rotate list_input_json : [] ##### fin chargement datou ##### chargement data ##### Call load_data_input : nb_thread : 5 origin SELECT photo_id, url FROM MTRBack.photos ph WHERE photo_id IN (917849322) Found this number of photos: 1 ##### Call download_photos : nb_thread : 5 begin to download photo : 917849322 download finish for photo 917849322 we have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB ##### After download_photos length of list_filenames : 1 ; length of list_pids : 1 ; length of list_args : 1 ##### After load_data_input time to download the photos : 0.2132253646850586 #### fin chargement data Blocking on flush ? No conitnuing About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : True number of steps : 1 step1:rotate Fri Feb 28 03:42:32 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec After prepare type args : Here we display some param of map_info ! map_filenames : {'temp/1740710552_2762130_917849322_2bd260e91e91df8378dde8bb8b8c4548.jpg': 917849322} map_photo_id_path_extension : {917849322: {'path': 'temp/1740710552_2762130_917849322_2bd260e91e91df8378dde8bb8b8c4548.jpg', 'extension': 'jpg'}} map_subphoto_mainphoto : {} Beginning of datou_step_rotate ! We are in a linear step without datou_depend ! rotate photos of 90,180,270 degres batch 1 select photo_id, hashtag_id, `type`, x0, x1, y0, y1, score, id from MTRPhoto.crop_hashtag_ids where photo_id in ( 917849322) and `type` in (0) Loaded 0 chid ids of type : 0 SELECT crop_hashtag_id, points FROM MTRPhoto.crop_polygon_points WHERE crop_hashtag_id in () map_chi : {} photo_id in download_rotate_and_save : 917849322 list_chi_loc : 0 Use all angle ! Rotation of photo 917849322 of 90 degree temp/1740710552_2762130_917849322_2bd260e91e91df8378dde8bb8b8c4548.jpg [] 90 remove_crop_border : False version de PIL : 9.5.0 Needs to change image size ! [[ 6.123234e-17 1.000000e+00] [-1.000000e+00 6.123234e-17]] 90 [[ 6.123234e-17 1.000000e+00] [-1.000000e+00 6.123234e-17]] shrink_image : False image_rotate : image_path : temp/1740710552_2762130_917849322_2bd260e91e91df8378dde8bb8b8c4548.jpg path_name_rotate : temp/1740710552_2762130_917849322_2bd260e91e91df8378dde8bb8b8c454890.jpg image_rotate.mode : RGB Rotation of photo 917849322 of 180 degree temp/1740710552_2762130_917849322_2bd260e91e91df8378dde8bb8b8c4548.jpg [] 180 remove_crop_border : False version de PIL : 9.5.0 Needs to change image size ! [[-1.0000000e+00 1.2246468e-16] [-1.2246468e-16 -1.0000000e+00]] 180 [[-1.0000000e+00 1.2246468e-16] [-1.2246468e-16 -1.0000000e+00]] shrink_image : False image_rotate : image_path : temp/1740710552_2762130_917849322_2bd260e91e91df8378dde8bb8b8c4548.jpg path_name_rotate : temp/1740710552_2762130_917849322_2bd260e91e91df8378dde8bb8b8c4548180.jpg image_rotate.mode : RGB Rotation of photo 917849322 of 270 degree temp/1740710552_2762130_917849322_2bd260e91e91df8378dde8bb8b8c4548.jpg [] 270 remove_crop_border : False version de PIL : 9.5.0 Needs to change image size ! [[-1.8369702e-16 -1.0000000e+00] [ 1.0000000e+00 -1.8369702e-16]] 270 [[-1.8369702e-16 -1.0000000e+00] [ 1.0000000e+00 -1.8369702e-16]] shrink_image : False image_rotate : image_path : temp/1740710552_2762130_917849322_2bd260e91e91df8378dde8bb8b8c4548.jpg path_name_rotate : temp/1740710552_2762130_917849322_2bd260e91e91df8378dde8bb8b8c4548270.jpg image_rotate.mode : RGB About to upload 3 photos upload in portfolio : 551782 init cache_photo without model_param we have 3 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740710553_2762130 we have uploaded 3 photos in the portfolio 551782 time of upload the photos Elapsed time : 1.2588872909545898 map_filename_photo_id : 3 map_filename_photo_id : {'temp/1740710552_2762130_917849322_2bd260e91e91df8378dde8bb8b8c454890.jpg': 1340052638, 'temp/1740710552_2762130_917849322_2bd260e91e91df8378dde8bb8b8c4548180.jpg': 1340052639, 'temp/1740710552_2762130_917849322_2bd260e91e91df8378dde8bb8b8c4548270.jpg': 1340052640} Len new_chis : 3 Len list_new_chi_with_photo_id : 0 of type : 0 list_new_chi_with_photo_id : [] After datou_step_exec type output : time spend for datou_step_exec : 2.14817476272583 time spend to save output : 4.482269287109375e-05 total time spend for step 1 : 2.148219585418701 caffe_path_current : About to save ! 1 Inside saveOutput : final : True verbose : True saveOutput not yet implemented for datou_step.type : rotate we use saveGeneral [917849322] map_info['map_portfolio_photo'] : {} final : True mtd_id 230 list_pids : [917849322] Looping around the photos to save general results len do output : 3 /1340052638Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340052639Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340052640Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('230', None, None, None, None, None, None, None, None) ('230', None, '917849322', None, None, None, None, None, None) begin to insert list_values into mtr_datou_result : length of list_values in save_final : 10 insert ignore into MTRPhoto.mtr_datou_result (mtd_id, mtr_portfolio_id,mtr_photo_id,result,result_long,result_double,hashtag_id,proba, mtr_current_id) values (%s,%s,%s,%s,%s,%s,%s,%s,%s) on duplicate key update mtr_portfolio_id = mtr_portfolio_id list_values : [('230', None, '1340052638', 'None', None, None, None, None, None), ('230', None, '1340052639', 'None', None, None, None, None, None), ('230', None, '1340052640', 'None', None, None, None, None, None), ('230', None, '917849322', None, None, None, None, None, None)] time used for this insertion : 0.013829946517944336 save_final save missing photos in datou_result : After save, about to update current ! datou_cur_ids : [] len(datou.list_steps) : 1 output : {1340052638: ['917849322', 'temp/1740710552_2762130_917849322_2bd260e91e91df8378dde8bb8b8c454890.jpg', []], 1340052639: ['917849322', 'temp/1740710552_2762130_917849322_2bd260e91e91df8378dde8bb8b8c4548180.jpg', []], 1340052640: ['917849322', 'temp/1740710552_2762130_917849322_2bd260e91e91df8378dde8bb8b8c4548270.jpg', []]} test rotate only is a success ! test rotate conditionnel Inside batchDatouExec : verbose : True ##### chargement datou SELECT name, created_at,limit_max FROM MTRDatou.mtr_datou WHERE id=233 SELECT mtd.id, mtdt.`type`, mtd.`param`, mtd.param_json, mtdt.nb_input, mtdt.nb_output, mtdt.prod, mtdt.is_local, mtdt.is_datou_depend, mtdt.is_photo_id_local FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_step_types mtdt WHERE mtdt.`id`=mtd.`type` AND mtd.mtd_id=233 SELECT mtd.id, mtd.mtd_id, mdsdt.id, mdsdt.name, mdsdt.description, msid.output_or_input, msid.data_order_id, mdsdt.type FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_steptype_io_datatypes msid, MTRDatou.mtr_datou_step_data_types mdsdt WHERE mtd.`type`=msid.`mtr_datou_step_type` AND mtd.mtd_id= 233 AND msid.data_type=mdsdt.id SELECT mts_id_output, id_output, mts_id_input, id_input FROM MTRDatou.mtr_datou_step_by_step WHERE mtd_id=233 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! DataTypes for each output/input checked ! no param json to modify List Step Type Loaded in datou : thcl, argmax, rotate list_input_json : [] ##### fin chargement datou ##### chargement data ##### Call load_data_input : nb_thread : 5 origin SELECT photo_id, url FROM MTRBack.photos ph WHERE photo_id IN (917849322) Found this number of photos: 1 ##### Call download_photos : nb_thread : 5 begin to download photo : 917849322 download finish for photo 917849322 we have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB ##### After download_photos length of list_filenames : 1 ; length of list_pids : 1 ; length of list_args : 1 ##### After load_data_input time to download the photos : 0.1194913387298584 #### fin chargement data Blocking on flush ? No conitnuing About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : True number of steps : 3 step1:thcl Fri Feb 28 03:42:34 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec After prepare type args : Here we display some param of map_info ! map_filenames : {'temp/1740710554_2762130_917849322_2bd260e91e91df8378dde8bb8b8c4548.jpg': 917849322} map_photo_id_path_extension : {917849322: {'path': 'temp/1740710554_2762130_917849322_2bd260e91e91df8378dde8bb8b8c4548.jpg', 'extension': 'jpg'}} map_subphoto_mainphoto : {} Beginning of datou step Thcl ! multi_thcl or not :False multi_thcl_cond or not :False dic_thcl : {'500': 1} we are using the classfication for only one thcl 500 In convert_file_to_np l 337 : 1 l343 1 l357 after caffe.io.load_image dimension du image : (3, (2448, 3264, 3)) dimension displayed ! time to import caffe and check if the image exist : 0.00027441978454589844 time to convert the images to numpy array : 2.0095479488372803 total time to convert the images to numpy array : 2.0101194381713867 list photo_ids error: [] list photo_ids correct : [917849322] number of photos to traite : 1 try to delete the photos incorrect in DB tagging for thcl : 500 To do loadFromThcl(), then load ParamDescType : thcl500 get_desc_type_from_thcl : type of cat SELECT id, mtr_user_id, name, pb_hashtag_id, hashtag_id_list, button_legend_list, portfolio_id_lists, photo_hashtag_type, photo_desc_type, svm_limit, limit_tagging, is_public, live, created_at, updated_at, type_classification FROM MTRDatou.classification_theme WHERE `id` IN (500) thcls : [{'id': 500, 'mtr_user_id': 31, 'name': 'orientation_carte_grise_all_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'carteGrisesVerticales__port_549774,cartegrise_90deg__port_550987,cartesGrisesEnvers__port_549765,portfolio_270deg__port_550988', 'svm_portfolios_learning': '549774,550987,549765,550988', 'photo_hashtag_type': 507, 'photo_desc_type': 3517, 'type_classification': 'caffe', 'hashtag_id_list': '0,0,0,0'}] thcl {'id': 500, 'mtr_user_id': 31, 'name': 'orientation_carte_grise_all_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'carteGrisesVerticales__port_549774,cartegrise_90deg__port_550987,cartesGrisesEnvers__port_549765,portfolio_270deg__port_550988', 'svm_portfolios_learning': '549774,550987,549765,550988', 'photo_hashtag_type': 507, 'photo_desc_type': 3517, 'type_classification': 'caffe', 'hashtag_id_list': '0,0,0,0'} Update svm_hashtag_type_desc : 3517 SELECT * FROM MTRDatou.photo_desc_type_params WHERE id in (3517) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (3517, 'orientation_carte_grise_all_2', 16384, 25088, 'orientation_carte_grise_all_2', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2018, 4, 18, 20, 4, 34), datetime.datetime(2018, 4, 18, 20, 4, 34)) To loadFromThcl() : net_3517 begin to check gpu status inside check gpu memory havn't enough memory gpu , need / 2500 l 3632 free memory gpu now : 7 wait 20 seconds l 3637 free memory gpu now : 7 max_wait_temp : 1 max_wait : 0 SELECT * FROM MTRDatou.photo_desc_type_params WHERE id in (3517) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (3517, 'orientation_carte_grise_all_2', 16384, 25088, 'orientation_carte_grise_all_2', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2018, 4, 18, 20, 4, 34), datetime.datetime(2018, 4, 18, 20, 4, 34)) param : , param.caffemodel : orientation_carte_grise_all_2 None mean_file_type : mean_file_path : prototxt_file_path : model : orientation_carte_grise_all_2 Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : orientation_carte_grise_all_2 model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : [] local folder : /data/models_weight/orientation_carte_grise_all_2 /data/models_weight/orientation_carte_grise_all_2/caffemodel size_local : 537110520 size in s3 : 537110520 create time local : 2021-08-09 05:29:00 create time in s3 : 2021-08-06 20:07:17 caffemodel already exist and didn't need to update /data/models_weight/orientation_carte_grise_all_2/deploy_conv_normal.prototxt size_local : 4626 size in s3 : 4626 create time local : 2021-08-09 05:29:00 create time in s3 : 2021-08-06 20:07:16 deploy_conv_normal.prototxt already exist and didn't need to update /data/models_weight/orientation_carte_grise_all_2/deploy_fc.prototxt size_local : 1130 size in s3 : 1130 create time local : 2021-08-09 05:29:00 create time in s3 : 2021-08-06 20:07:16 deploy_fc.prototxt already exist and didn't need to update /data/models_weight/orientation_carte_grise_all_2/deploy.prototxt size_local : 5653 size in s3 : 5653 create time local : 2021-08-09 05:29:00 create time in s3 : 2021-08-06 20:07:16 deploy.prototxt already exist and didn't need to update /data/models_weight/orientation_carte_grise_all_2/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:29:00 create time in s3 : 2021-08-06 20:07:31 mean.npy already exist and didn't need to update /data/models_weight/orientation_carte_grise_all_2/synset_words.txt size_local : 159 size in s3 : 159 create time local : 2021-08-09 05:29:00 create time in s3 : 2021-08-06 20:07:16 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/orientation_carte_grise_all_2/deploy.prototxt caffemodel_filename : /data/models_weight/orientation_carte_grise_all_2/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory havn't enough memory gpu , need / 2500 l 3632 free memory gpu now : 7 wait 20 seconds WARNING: Logging before InitGoogleLogging() is written to STDERR F0228 03:43:24.087610 2762130 syncedmem.cpp:78] Check failed: error == cudaSuccess (2 vs. 0) out of memory *** Check failure stack trace: *** Command terminated by signal 6 57.65user 39.27system 8:00.01elapsed 20%CPU (0avgtext+0avgdata 4885908maxresident)k 6301272inputs+23168outputs (12099major+4644478minor)pagefaults 0swaps