python /home/admin/mtr/script_for_cron.py -j default -m 20 -a 'python3 ~/workarea/git/Velours/python/prod/datou.py -j batch_current -a 3995 -P 5486001' -s traitement_sts -M 0 -S 0 -U 100,80,95 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/home/admin/workarea/git/apy', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 4069223 load datou : 3995 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11524 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11533 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11532 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11528 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11528 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11531 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11531 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11525 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11525 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11578 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11578 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11527 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11526 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11526 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11535 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11527 doesn't seem to be define in the database( WARNING : type of input 3 of step 11526 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11524 doesn't seem to be define in the database( WARNING : type of input 2 of step 11528 doesn't seem to be define in the database( WARNING : output 1 of step 11524 have datatype=2 whereas input 1 of step 11531 have datatype=7 WARNING : type of output 2 of step 11531 doesn't seem to be define in the database( WARNING : type of input 1 of step 11525 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11527 have datatype=10 whereas input 3 of step 11534 have datatype=6 WARNING : type of input 2 of step 11578 doesn't seem to be define in the database( WARNING : output 1 of step 11525 have datatype=7 whereas input 2 of step 11578 have datatype=None WARNING : type of output 3 of step 11578 doesn't seem to be define in the database( WARNING : type of input 1 of step 11527 doesn't seem to be define in the database( WARNING : output 0 of step 11527 have datatype=10 whereas input 0 of step 11584 have datatype=18 WARNING : type of input 5 of step 11534 doesn't seem to be define in the database( WARNING : output 0 of step 11584 have datatype=11 whereas input 5 of step 11534 have datatype=None WARNING : type of output 1 of step 11533 doesn't seem to be define in the database( WARNING : type of input 3 of step 11528 doesn't seem to be define in the database( WARNING : type of output 1 of step 11532 doesn't seem to be define in the database( WARNING : type of input 3 of step 11528 doesn't seem to be define in the database( WARNING : output 0 of step 11531 have datatype=1 whereas input 0 of step 11525 have datatype=2 DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? output optionnel thcl was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5359, 'Mask_Limeil_Label_PEHD_080621', 16384, 25088, 'Mask_Limeil_Label_PEHD_080621', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 6, 9, 7, 22, 34), datetime.datetime(2021, 6, 9, 7, 22, 34)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5557, 'learn_generique_01122021_6000_v2', 2048, 2048, 'learn_generique_01122021_6000_v2', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2021, 12, 1, 19, 59, 17), datetime.datetime(2021, 12, 1, 19, 59, 17)) load thcls load THCL from format json or kwargs add thcl : 2976 in CacheModelConfig load THCL from format json or kwargs add thcl : 3233 in CacheModelConfig load pdts add pdt : 5359 in CacheModelConfig add pdt : 5557 in CacheModelConfig Running datou job : batch_current updating current state to 1 list_input_json: [] Current got : datou_id : 3995, datou_cur_ids : ['2596639'] with mtr_portfolio_ids : ['5486001'] and first list_photo_ids : [] new path : /proc/4069223/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11524 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11533 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11532 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11528 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11528 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11531 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11531 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11525 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11525 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11578 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11578 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11527 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11526 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11526 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11535 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11527 doesn't seem to be define in the database( WARNING : type of input 3 of step 11526 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11524 doesn't seem to be define in the database( WARNING : type of input 2 of step 11528 doesn't seem to be define in the database( WARNING : output 1 of step 11524 have datatype=2 whereas input 1 of step 11531 have datatype=7 WARNING : type of output 2 of step 11531 doesn't seem to be define in the database( WARNING : type of input 1 of step 11525 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11527 have datatype=10 whereas input 3 of step 11534 have datatype=6 WARNING : type of input 2 of step 11578 doesn't seem to be define in the database( WARNING : output 1 of step 11525 have datatype=7 whereas input 2 of step 11578 have datatype=None WARNING : type of output 3 of step 11578 doesn't seem to be define in the database( WARNING : type of input 1 of step 11527 doesn't seem to be define in the database( WARNING : output 0 of step 11527 have datatype=10 whereas input 0 of step 11584 have datatype=18 WARNING : type of input 5 of step 11534 doesn't seem to be define in the database( WARNING : output 0 of step 11584 have datatype=11 whereas input 5 of step 11534 have datatype=None WARNING : type of output 1 of step 11533 doesn't seem to be define in the database( WARNING : type of input 3 of step 11528 doesn't seem to be define in the database( WARNING : type of output 1 of step 11532 doesn't seem to be define in the database( WARNING : type of input 3 of step 11528 doesn't seem to be define in the database( WARNING : output 0 of step 11531 have datatype=1 whereas input 0 of step 11525 have datatype=2 DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, blur_detection, brightness, crop_condition, thcl, merge_mask_thcl_custom, rle_unique_nms_with_priority, crop_condition, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 20 list_input_json : [] origin We have 1 , BFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 5 ; length of list_pids : 5 ; length of list_args : 5 time to download the photos : 0.894862174987793 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 13 step1:mask_detect Tue Feb 18 15:06:42 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10593 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-02-18 15:06:45.112732: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-02-18 15:06:45.120318: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-02-18 15:06:45.121828: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f32ec000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-02-18 15:06:45.121870: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-02-18 15:06:45.124836: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-02-18 15:06:45.350557: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x243c16f0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-02-18 15:06:45.350596: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-02-18 15:06:45.351685: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-18 15:06:45.352001: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-18 15:06:45.358185: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-18 15:06:45.360804: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-18 15:06:45.361187: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-18 15:06:45.363701: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-18 15:06:45.364968: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-18 15:06:45.370061: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-18 15:06:45.371891: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-18 15:06:45.371973: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-18 15:06:45.372781: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-18 15:06:45.372797: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-18 15:06:45.372805: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-18 15:06:45.374124: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9815 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-02-18 15:06:45.664115: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-18 15:06:45.664203: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-18 15:06:45.664226: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-18 15:06:45.664247: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-18 15:06:45.664267: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-18 15:06:45.664287: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-18 15:06:45.664307: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-18 15:06:45.664327: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-18 15:06:45.666013: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-18 15:06:45.667331: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-18 15:06:45.667371: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-18 15:06:45.667392: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-18 15:06:45.667412: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-18 15:06:45.667431: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-18 15:06:45.667451: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-18 15:06:45.667470: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-18 15:06:45.667490: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-18 15:06:45.669044: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-18 15:06:45.669074: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-18 15:06:45.669084: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-18 15:06:45.669094: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-18 15:06:45.670717: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9815 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2976 thcls : [{'id': 2976, 'mtr_user_id': 31, 'name': 'Mask_Limeil_Label_PEHD_080621', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,environnement,fibreux_cont,metal_cont,pet_clair_cont,pet_fonce_cont,pet_opaque_cont,barquette,film_plastique,autre_contaminant,etiquette_detachee', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3760, 'photo_desc_type': 5359, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2976, 'mtr_user_id': 31, 'name': 'Mask_Limeil_Label_PEHD_080621', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,environnement,fibreux_cont,metal_cont,pet_clair_cont,pet_fonce_cont,pet_opaque_cont,barquette,film_plastique,autre_contaminant,etiquette_detachee', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3760, 'photo_desc_type': 5359, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5359 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5359, 'Mask_Limeil_Label_PEHD_080621', 16384, 25088, 'Mask_Limeil_Label_PEHD_080621', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 6, 9, 7, 22, 34), datetime.datetime(2021, 6, 9, 7, 22, 34)) {'thcl': {'id': 2976, 'mtr_user_id': 31, 'name': 'Mask_Limeil_Label_PEHD_080621', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,environnement,fibreux_cont,metal_cont,pet_clair_cont,pet_fonce_cont,pet_opaque_cont,barquette,film_plastique,autre_contaminant,etiquette_detachee', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3760, 'photo_desc_type': 5359, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'environnement', 'fibreux_cont', 'metal_cont', 'pet_clair_cont', 'pet_fonce_cont', 'pet_opaque_cont', 'barquette', 'film_plastique', 'autre_contaminant', 'etiquette_detachee'], 'list_hashtags_csv': 'background,environnement,fibreux_cont,metal_cont,pet_clair_cont,pet_fonce_cont,pet_opaque_cont,barquette,film_plastique,autre_contaminant,etiquette_detachee', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3760, 'svm_hashtag_type_desc': 5359, 'photo_desc_type': 5359, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'environnement', 'fibreux_cont', 'metal_cont', 'pet_clair_cont', 'pet_fonce_cont', 'pet_opaque_cont', 'barquette', 'film_plastique', 'autre_contaminant', 'etiquette_detachee'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME Mask_Limeil_Label_PEHD_080621 NUM_CLASSES 11 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : Mask_Limeil_Label_PEHD_080621 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-02-18 15:06:56.187756: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-18 15:06:56.366242: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/Mask_Limeil_Label_PEHD_080621 /data/models_weight/Mask_Limeil_Label_PEHD_080621/mask_model.h5 size_local : 256052544 size in s3 : 256052544 create time local : 2021-08-11 19:43:15 create time in s3 : 2021-08-06 17:21:30 mask_model.h5 already exist and didn't need to update list_images length : 5 NEW PHOTO Processing 1 images image shape: (2464, 3280, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 19) min: 0.00000 max: 3280.00000 nb d'objets trouves : 79 NEW PHOTO Processing 1 images image shape: (2464, 3280, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 19) min: 0.00000 max: 3280.00000 nb d'objets trouves : 73 NEW PHOTO Processing 1 images image shape: (2464, 3280, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 19) min: 0.00000 max: 3280.00000 nb d'objets trouves : 71 NEW PHOTO Processing 1 images image shape: (2464, 3280, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 19) min: 0.00000 max: 3280.00000 nb d'objets trouves : 74 NEW PHOTO Processing 1 images image shape: (2464, 3280, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 19) min: 0.00000 max: 3280.00000 nb d'objets trouves : 79 Detection mask done ! Trying to reset tf kernel 4069547 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 5304 tf kernel not reseted sub process len(results) : 5 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 5 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 10372 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2976 Catched exception ! Connect or reconnect ! thcls : [{'id': 2976, 'mtr_user_id': 31, 'name': 'Mask_Limeil_Label_PEHD_080621', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,environnement,fibreux_cont,metal_cont,pet_clair_cont,pet_fonce_cont,pet_opaque_cont,barquette,film_plastique,autre_contaminant,etiquette_detachee', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3760, 'photo_desc_type': 5359, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2976, 'mtr_user_id': 31, 'name': 'Mask_Limeil_Label_PEHD_080621', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,environnement,fibreux_cont,metal_cont,pet_clair_cont,pet_fonce_cont,pet_opaque_cont,barquette,film_plastique,autre_contaminant,etiquette_detachee', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3760, 'photo_desc_type': 5359, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5359 ['background', 'environnement', 'fibreux_cont', 'metal_cont', 'pet_clair_cont', 'pet_fonce_cont', 'pet_opaque_cont', 'barquette', 'film_plastique', 'autre_contaminant', 'etiquette_detachee'] time for calcul the mask position with numpy : 0.02075934410095215 nb_pixel_total : 263100 time to create 1 rle with new method : 0.014598608016967773 length of segment : 539 time for calcul the mask position with numpy : 0.010059833526611328 nb_pixel_total : 169385 time to create 1 rle with new method : 0.016454458236694336 length of segment : 819 time for calcul the mask position with numpy : 0.0041005611419677734 nb_pixel_total : 75583 time to create 1 rle with old method : 0.08257937431335449 length of segment : 259 time for calcul the mask position with numpy : 0.004437923431396484 nb_pixel_total : 72907 time to create 1 rle with old method : 0.08443760871887207 length of segment : 391 time for calcul the mask position with numpy : 0.0015993118286132812 nb_pixel_total : 27196 time to create 1 rle with old method : 0.030732393264770508 length of segment : 182 time for calcul the mask position with numpy : 0.026965856552124023 nb_pixel_total : 336878 time to create 1 rle with new method : 0.03529644012451172 length of segment : 823 time for calcul the mask position with numpy : 0.014116048812866211 nb_pixel_total : 314078 time to create 1 rle with new method : 0.018938302993774414 length of segment : 660 time for calcul the mask position with numpy : 0.009272098541259766 nb_pixel_total : 239812 time to create 1 rle with new method : 0.014548540115356445 length of segment : 666 time for calcul the mask position with numpy : 0.005922555923461914 nb_pixel_total : 96352 time to create 1 rle with old method : 0.10918068885803223 length of segment : 520 time for calcul the mask position with numpy : 0.00782918930053711 nb_pixel_total : 163613 time to create 1 rle with new method : 0.010391473770141602 length of segment : 479 time for calcul the mask position with numpy : 0.016628265380859375 nb_pixel_total : 315085 time to create 1 rle with new method : 0.03916811943054199 length of segment : 852 time for calcul the mask position with numpy : 0.013112783432006836 nb_pixel_total : 308819 time to create 1 rle with new method : 0.0155792236328125 length of segment : 653 time for calcul the mask position with numpy : 0.0023686885833740234 nb_pixel_total : 40529 time to create 1 rle with old method : 0.045304059982299805 length of segment : 343 time for calcul the mask position with numpy : 0.001100778579711914 nb_pixel_total : 21107 time to create 1 rle with old method : 0.027940750122070312 length of segment : 118 time for calcul the mask position with numpy : 0.01270294189453125 nb_pixel_total : 251378 time to create 1 rle with new method : 0.02081751823425293 length of segment : 511 time for calcul the mask position with numpy : 0.005197048187255859 nb_pixel_total : 74746 time to create 1 rle with old method : 0.12262892723083496 length of segment : 357 time for calcul the mask position with numpy : 0.013547658920288086 nb_pixel_total : 318893 time to create 1 rle with new method : 1.504606008529663 length of segment : 743 time for calcul the mask position with numpy : 0.0070590972900390625 nb_pixel_total : 207776 time to create 1 rle with new method : 0.01938152313232422 length of segment : 565 time for calcul the mask position with numpy : 0.0014030933380126953 nb_pixel_total : 36402 time to create 1 rle with old method : 0.040242910385131836 length of segment : 332 time for calcul the mask position with numpy : 0.0006706714630126953 nb_pixel_total : 19610 time to create 1 rle with old method : 0.02236652374267578 length of segment : 174 time for calcul the mask position with numpy : 0.0002963542938232422 nb_pixel_total : 6953 time to create 1 rle with old method : 0.008465051651000977 length of segment : 69 time for calcul the mask position with numpy : 0.009924650192260742 nb_pixel_total : 326541 time to create 1 rle with new method : 0.03059101104736328 length of segment : 1597 time for calcul the mask position with numpy : 0.001293182373046875 nb_pixel_total : 36158 time to create 1 rle with old method : 0.0430448055267334 length of segment : 308 time for calcul the mask position with numpy : 0.001644134521484375 nb_pixel_total : 51830 time to create 1 rle with old method : 0.06104326248168945 length of segment : 458 time for calcul the mask position with numpy : 0.0036165714263916016 nb_pixel_total : 87535 time to create 1 rle with old method : 0.10309553146362305 length of segment : 598 time for calcul the mask position with numpy : 0.0005545616149902344 nb_pixel_total : 10604 time to create 1 rle with old method : 0.012311458587646484 length of segment : 184 time for calcul the mask position with numpy : 0.0017845630645751953 nb_pixel_total : 40109 time to create 1 rle with old method : 0.04571080207824707 length of segment : 323 time for calcul the mask position with numpy : 0.0010006427764892578 nb_pixel_total : 20893 time to create 1 rle with old method : 0.023648500442504883 length of segment : 214 time for calcul the mask position with numpy : 0.00025916099548339844 nb_pixel_total : 4506 time to create 1 rle with old method : 0.005751371383666992 length of segment : 62 time for calcul the mask position with numpy : 0.0011212825775146484 nb_pixel_total : 24962 time to create 1 rle with old method : 0.028166532516479492 length of segment : 255 time for calcul the mask position with numpy : 0.0012807846069335938 nb_pixel_total : 39890 time to create 1 rle with old method : 0.04877591133117676 length of segment : 281 time for calcul the mask position with numpy : 0.0006361007690429688 nb_pixel_total : 17844 time to create 1 rle with old method : 0.020534038543701172 length of segment : 198 time for calcul the mask position with numpy : 0.0017085075378417969 nb_pixel_total : 40490 time to create 1 rle with old method : 0.04782366752624512 length of segment : 211 time for calcul the mask position with numpy : 0.0007131099700927734 nb_pixel_total : 10991 time to create 1 rle with old method : 0.012469291687011719 length of segment : 81 time for calcul the mask position with numpy : 0.0058574676513671875 nb_pixel_total : 250670 time to create 1 rle with new method : 0.01628732681274414 length of segment : 584 time for calcul the mask position with numpy : 0.0007345676422119141 nb_pixel_total : 8256 time to create 1 rle with old method : 0.009239435195922852 length of segment : 138 time for calcul the mask position with numpy : 0.0029532909393310547 nb_pixel_total : 37394 time to create 1 rle with old method : 0.04882407188415527 length of segment : 423 time for calcul the mask position with numpy : 0.0106658935546875 nb_pixel_total : 246389 time to create 1 rle with new method : 0.015367984771728516 length of segment : 907 time for calcul the mask position with numpy : 0.01344156265258789 nb_pixel_total : 263245 time to create 1 rle with new method : 0.02064204216003418 length of segment : 1043 time for calcul the mask position with numpy : 0.0016121864318847656 nb_pixel_total : 29753 time to create 1 rle with old method : 0.03340601921081543 length of segment : 216 time for calcul the mask position with numpy : 0.013092517852783203 nb_pixel_total : 111378 time to create 1 rle with old method : 0.12572932243347168 length of segment : 561 time for calcul the mask position with numpy : 0.009911298751831055 nb_pixel_total : 163901 time to create 1 rle with new method : 0.015347480773925781 length of segment : 786 time for calcul the mask position with numpy : 0.005365610122680664 nb_pixel_total : 105034 time to create 1 rle with old method : 0.12215352058410645 length of segment : 413 time for calcul the mask position with numpy : 0.004091501235961914 nb_pixel_total : 91847 time to create 1 rle with old method : 0.101531982421875 length of segment : 338 time for calcul the mask position with numpy : 0.0016531944274902344 nb_pixel_total : 27221 time to create 1 rle with old method : 0.03138279914855957 length of segment : 292 time for calcul the mask position with numpy : 0.0023305416107177734 nb_pixel_total : 47081 time to create 1 rle with old method : 0.056374311447143555 length of segment : 256 time for calcul the mask position with numpy : 0.0003705024719238281 nb_pixel_total : 12611 time to create 1 rle with old method : 0.01857757568359375 length of segment : 144 time for calcul the mask position with numpy : 0.030187129974365234 nb_pixel_total : 368702 time to create 1 rle with new method : 0.5721211433410645 length of segment : 1470 time for calcul the mask position with numpy : 0.006584882736206055 nb_pixel_total : 191973 time to create 1 rle with new method : 0.011920928955078125 length of segment : 818 time for calcul the mask position with numpy : 0.0017061233520507812 nb_pixel_total : 22609 time to create 1 rle with old method : 0.02651357650756836 length of segment : 391 time for calcul the mask position with numpy : 0.0012822151184082031 nb_pixel_total : 19674 time to create 1 rle with old method : 0.02241206169128418 length of segment : 247 time for calcul the mask position with numpy : 0.00496673583984375 nb_pixel_total : 65374 time to create 1 rle with old method : 0.07023215293884277 length of segment : 402 time for calcul the mask position with numpy : 0.006942033767700195 nb_pixel_total : 88463 time to create 1 rle with old method : 0.09493613243103027 length of segment : 459 time for calcul the mask position with numpy : 0.001756429672241211 nb_pixel_total : 34729 time to create 1 rle with old method : 0.03825950622558594 length of segment : 271 time for calcul the mask position with numpy : 0.002650737762451172 nb_pixel_total : 46043 time to create 1 rle with old method : 0.050978899002075195 length of segment : 498 time for calcul the mask position with numpy : 0.0005135536193847656 nb_pixel_total : 19714 time to create 1 rle with old method : 0.0230863094329834 length of segment : 255 time for calcul the mask position with numpy : 0.0021216869354248047 nb_pixel_total : 38043 time to create 1 rle with old method : 0.04274868965148926 length of segment : 286 time for calcul the mask position with numpy : 0.0027518272399902344 nb_pixel_total : 64232 time to create 1 rle with old method : 0.07571029663085938 length of segment : 368 time for calcul the mask position with numpy : 0.002355813980102539 nb_pixel_total : 79525 time to create 1 rle with old method : 0.09686923027038574 length of segment : 489 time for calcul the mask position with numpy : 0.0011434555053710938 nb_pixel_total : 17019 time to create 1 rle with old method : 0.019567251205444336 length of segment : 236 time for calcul the mask position with numpy : 0.0021474361419677734 nb_pixel_total : 38574 time to create 1 rle with old method : 0.04422807693481445 length of segment : 503 time for calcul the mask position with numpy : 0.002676725387573242 nb_pixel_total : 43648 time to create 1 rle with old method : 0.0493471622467041 length of segment : 312 time for calcul the mask position with numpy : 0.0008835792541503906 nb_pixel_total : 13503 time to create 1 rle with old method : 0.015695571899414062 length of segment : 170 time for calcul the mask position with numpy : 0.001344442367553711 nb_pixel_total : 17881 time to create 1 rle with old method : 0.02253580093383789 length of segment : 180 time for calcul the mask position with numpy : 0.010655641555786133 nb_pixel_total : 218984 time to create 1 rle with new method : 0.01619577407836914 length of segment : 766 time for calcul the mask position with numpy : 0.0015645027160644531 nb_pixel_total : 25990 time to create 1 rle with old method : 0.029474496841430664 length of segment : 159 time for calcul the mask position with numpy : 0.013135671615600586 nb_pixel_total : 240668 time to create 1 rle with new method : 0.022364377975463867 length of segment : 781 time for calcul the mask position with numpy : 0.0028960704803466797 nb_pixel_total : 50185 time to create 1 rle with old method : 0.07205677032470703 length of segment : 237 time for calcul the mask position with numpy : 0.0021920204162597656 nb_pixel_total : 25004 time to create 1 rle with old method : 0.04082083702087402 length of segment : 270 time for calcul the mask position with numpy : 0.013613462448120117 nb_pixel_total : 300900 time to create 1 rle with new method : 0.0513148307800293 length of segment : 1146 time for calcul the mask position with numpy : 0.0018951892852783203 nb_pixel_total : 22510 time to create 1 rle with old method : 0.025941133499145508 length of segment : 167 time for calcul the mask position with numpy : 0.009004592895507812 nb_pixel_total : 233143 time to create 1 rle with new method : 0.015744924545288086 length of segment : 1120 time for calcul the mask position with numpy : 0.00879669189453125 nb_pixel_total : 180742 time to create 1 rle with new method : 0.015885591506958008 length of segment : 722 time for calcul the mask position with numpy : 0.0025420188903808594 nb_pixel_total : 28699 time to create 1 rle with old method : 0.03690838813781738 length of segment : 274 time for calcul the mask position with numpy : 0.002124786376953125 nb_pixel_total : 30214 time to create 1 rle with old method : 0.03859972953796387 length of segment : 270 time for calcul the mask position with numpy : 0.0067157745361328125 nb_pixel_total : 135792 time to create 1 rle with old method : 0.15612101554870605 length of segment : 461 time for calcul the mask position with numpy : 0.0014379024505615234 nb_pixel_total : 21893 time to create 1 rle with old method : 0.028568029403686523 length of segment : 181 time for calcul the mask position with numpy : 0.0021615028381347656 nb_pixel_total : 32670 time to create 1 rle with old method : 0.05499863624572754 length of segment : 334 time for calcul the mask position with numpy : 0.0006115436553955078 nb_pixel_total : 9235 time to create 1 rle with old method : 0.012740850448608398 length of segment : 112 time for calcul the mask position with numpy : 0.0007655620574951172 nb_pixel_total : 11757 time to create 1 rle with old method : 0.014206171035766602 length of segment : 109 time for calcul the mask position with numpy : 0.0007572174072265625 nb_pixel_total : 10597 time to create 1 rle with old method : 0.013872861862182617 length of segment : 143 time for calcul the mask position with numpy : 0.0020639896392822266 nb_pixel_total : 35619 time to create 1 rle with old method : 0.04091238975524902 length of segment : 249 time for calcul the mask position with numpy : 0.0026483535766601562 nb_pixel_total : 28262 time to create 1 rle with old method : 0.035811662673950195 length of segment : 282 time for calcul the mask position with numpy : 0.000993967056274414 nb_pixel_total : 31652 time to create 1 rle with old method : 0.04112815856933594 length of segment : 268 time for calcul the mask position with numpy : 0.004435539245605469 nb_pixel_total : 50690 time to create 1 rle with old method : 0.06373000144958496 length of segment : 327 time for calcul the mask position with numpy : 0.008042097091674805 nb_pixel_total : 135222 time to create 1 rle with old method : 0.16640305519104004 length of segment : 541 time for calcul the mask position with numpy : 0.008778572082519531 nb_pixel_total : 98770 time to create 1 rle with old method : 0.1378321647644043 length of segment : 487 time for calcul the mask position with numpy : 0.0022628307342529297 nb_pixel_total : 36160 time to create 1 rle with old method : 0.04565620422363281 length of segment : 254 time for calcul the mask position with numpy : 0.002054452896118164 nb_pixel_total : 33619 time to create 1 rle with old method : 0.04014754295349121 length of segment : 238 time for calcul the mask position with numpy : 0.0013728141784667969 nb_pixel_total : 24974 time to create 1 rle with old method : 0.028418779373168945 length of segment : 175 time for calcul the mask position with numpy : 0.0065593719482421875 nb_pixel_total : 132568 time to create 1 rle with old method : 0.1501293182373047 length of segment : 359 time for calcul the mask position with numpy : 0.0004382133483886719 nb_pixel_total : 5512 time to create 1 rle with old method : 0.0065386295318603516 length of segment : 108 time for calcul the mask position with numpy : 0.0010008811950683594 nb_pixel_total : 11935 time to create 1 rle with old method : 0.013944149017333984 length of segment : 138 time for calcul the mask position with numpy : 0.0014750957489013672 nb_pixel_total : 20702 time to create 1 rle with old method : 0.02515435218811035 length of segment : 287 time for calcul the mask position with numpy : 0.0016109943389892578 nb_pixel_total : 39176 time to create 1 rle with old method : 0.04477834701538086 length of segment : 204 time for calcul the mask position with numpy : 0.0199124813079834 nb_pixel_total : 132486 time to create 1 rle with old method : 0.1487410068511963 length of segment : 1181 time for calcul the mask position with numpy : 0.009706735610961914 nb_pixel_total : 240913 time to create 1 rle with new method : 0.014758825302124023 length of segment : 838 time for calcul the mask position with numpy : 0.009911537170410156 nb_pixel_total : 203053 time to create 1 rle with new method : 0.013565540313720703 length of segment : 785 time for calcul the mask position with numpy : 0.008893966674804688 nb_pixel_total : 173221 time to create 1 rle with new method : 0.010104179382324219 length of segment : 871 time for calcul the mask position with numpy : 0.008438825607299805 nb_pixel_total : 206852 time to create 1 rle with new method : 0.013292074203491211 length of segment : 580 time for calcul the mask position with numpy : 0.004415035247802734 nb_pixel_total : 96126 time to create 1 rle with old method : 0.10451841354370117 length of segment : 296 time for calcul the mask position with numpy : 0.0010640621185302734 nb_pixel_total : 12873 time to create 1 rle with old method : 0.014555215835571289 length of segment : 166 time for calcul the mask position with numpy : 0.0026667118072509766 nb_pixel_total : 45470 time to create 1 rle with old method : 0.05163121223449707 length of segment : 259 time for calcul the mask position with numpy : 0.0017240047454833984 nb_pixel_total : 56075 time to create 1 rle with old method : 0.06183052062988281 length of segment : 304 time for calcul the mask position with numpy : 0.003645658493041992 nb_pixel_total : 45305 time to create 1 rle with old method : 0.0486752986907959 length of segment : 395 time for calcul the mask position with numpy : 0.0024614334106445312 nb_pixel_total : 33841 time to create 1 rle with old method : 0.05520510673522949 length of segment : 265 time for calcul the mask position with numpy : 0.004163265228271484 nb_pixel_total : 46962 time to create 1 rle with old method : 0.053655385971069336 length of segment : 384 time for calcul the mask position with numpy : 0.0013327598571777344 nb_pixel_total : 30939 time to create 1 rle with old method : 0.03551745414733887 length of segment : 242 time for calcul the mask position with numpy : 0.0022521018981933594 nb_pixel_total : 29682 time to create 1 rle with old method : 0.033431053161621094 length of segment : 324 time for calcul the mask position with numpy : 0.0035696029663085938 nb_pixel_total : 65171 time to create 1 rle with old method : 0.06931257247924805 length of segment : 436 time for calcul the mask position with numpy : 0.003147602081298828 nb_pixel_total : 58857 time to create 1 rle with old method : 0.06422233581542969 length of segment : 323 time for calcul the mask position with numpy : 0.0026984214782714844 nb_pixel_total : 26204 time to create 1 rle with old method : 0.03098154067993164 length of segment : 421 time for calcul the mask position with numpy : 0.0013191699981689453 nb_pixel_total : 20856 time to create 1 rle with old method : 0.023731708526611328 length of segment : 174 time for calcul the mask position with numpy : 0.0022644996643066406 nb_pixel_total : 83548 time to create 1 rle with old method : 0.08934307098388672 length of segment : 614 time for calcul the mask position with numpy : 0.0009982585906982422 nb_pixel_total : 14524 time to create 1 rle with old method : 0.01713085174560547 length of segment : 126 time for calcul the mask position with numpy : 0.0024416446685791016 nb_pixel_total : 38681 time to create 1 rle with old method : 0.04604482650756836 length of segment : 340 time for calcul the mask position with numpy : 0.010045528411865234 nb_pixel_total : 173082 time to create 1 rle with new method : 0.015332698822021484 length of segment : 1105 time for calcul the mask position with numpy : 0.0021905899047851562 nb_pixel_total : 45846 time to create 1 rle with old method : 0.0514528751373291 length of segment : 187 time for calcul the mask position with numpy : 0.0018572807312011719 nb_pixel_total : 28713 time to create 1 rle with old method : 0.03231692314147949 length of segment : 343 time for calcul the mask position with numpy : 0.0008080005645751953 nb_pixel_total : 11021 time to create 1 rle with old method : 0.013206720352172852 length of segment : 141 time for calcul the mask position with numpy : 0.00468897819519043 nb_pixel_total : 97675 time to create 1 rle with old method : 0.10583305358886719 length of segment : 428 time for calcul the mask position with numpy : 0.004789113998413086 nb_pixel_total : 83561 time to create 1 rle with old method : 0.09422445297241211 length of segment : 256 time for calcul the mask position with numpy : 0.0022478103637695312 nb_pixel_total : 14505 time to create 1 rle with old method : 0.01697826385498047 length of segment : 320 time for calcul the mask position with numpy : 0.0012156963348388672 nb_pixel_total : 24342 time to create 1 rle with old method : 0.02757096290588379 length of segment : 218 time for calcul the mask position with numpy : 0.0013897418975830078 nb_pixel_total : 30008 time to create 1 rle with old method : 0.04222583770751953 length of segment : 170 time for calcul the mask position with numpy : 0.0016131401062011719 nb_pixel_total : 9871 time to create 1 rle with old method : 0.01633477210998535 length of segment : 186 time for calcul the mask position with numpy : 0.00246429443359375 nb_pixel_total : 49091 time to create 1 rle with old method : 0.05437064170837402 length of segment : 307 time for calcul the mask position with numpy : 0.002437114715576172 nb_pixel_total : 42082 time to create 1 rle with old method : 0.04815173149108887 length of segment : 225 time for calcul the mask position with numpy : 0.0015649795532226562 nb_pixel_total : 15437 time to create 1 rle with old method : 0.025329113006591797 length of segment : 140 time for calcul the mask position with numpy : 0.010533332824707031 nb_pixel_total : 224445 time to create 1 rle with new method : 0.01175379753112793 length of segment : 687 time for calcul the mask position with numpy : 0.0017139911651611328 nb_pixel_total : 35378 time to create 1 rle with old method : 0.0398252010345459 length of segment : 167 time for calcul the mask position with numpy : 0.0034706592559814453 nb_pixel_total : 56026 time to create 1 rle with old method : 0.06502199172973633 length of segment : 304 time for calcul the mask position with numpy : 0.003276348114013672 nb_pixel_total : 67646 time to create 1 rle with old method : 0.07768774032592773 length of segment : 234 time for calcul the mask position with numpy : 0.015700340270996094 nb_pixel_total : 390090 time to create 1 rle with new method : 0.02004551887512207 length of segment : 1205 time for calcul the mask position with numpy : 0.015569925308227539 nb_pixel_total : 281466 time to create 1 rle with new method : 0.019129276275634766 length of segment : 1135 time for calcul the mask position with numpy : 0.011667490005493164 nb_pixel_total : 136493 time to create 1 rle with old method : 0.14563918113708496 length of segment : 775 time for calcul the mask position with numpy : 0.0017931461334228516 nb_pixel_total : 29812 time to create 1 rle with old method : 0.0373990535736084 length of segment : 264 time for calcul the mask position with numpy : 0.0008499622344970703 nb_pixel_total : 9187 time to create 1 rle with old method : 0.011193990707397461 length of segment : 93 time for calcul the mask position with numpy : 0.012420654296875 nb_pixel_total : 186886 time to create 1 rle with new method : 0.018059253692626953 length of segment : 996 time for calcul the mask position with numpy : 0.0016758441925048828 nb_pixel_total : 23599 time to create 1 rle with old method : 0.027013778686523438 length of segment : 389 time for calcul the mask position with numpy : 0.0077593326568603516 nb_pixel_total : 193676 time to create 1 rle with new method : 0.017199993133544922 length of segment : 644 time for calcul the mask position with numpy : 0.0016498565673828125 nb_pixel_total : 50336 time to create 1 rle with old method : 0.05846261978149414 length of segment : 320 time for calcul the mask position with numpy : 0.011178731918334961 nb_pixel_total : 151366 time to create 1 rle with new method : 0.018321514129638672 length of segment : 705 time for calcul the mask position with numpy : 0.0018286705017089844 nb_pixel_total : 27063 time to create 1 rle with old method : 0.030330419540405273 length of segment : 300 time for calcul the mask position with numpy : 0.0015530586242675781 nb_pixel_total : 56553 time to create 1 rle with old method : 0.0641779899597168 length of segment : 276 time for calcul the mask position with numpy : 0.002923250198364258 nb_pixel_total : 38477 time to create 1 rle with old method : 0.04204082489013672 length of segment : 506 time for calcul the mask position with numpy : 0.0016617774963378906 nb_pixel_total : 39761 time to create 1 rle with old method : 0.0440058708190918 length of segment : 165 time for calcul the mask position with numpy : 0.005929231643676758 nb_pixel_total : 130397 time to create 1 rle with old method : 0.1405644416809082 length of segment : 446 time for calcul the mask position with numpy : 0.0032668113708496094 nb_pixel_total : 73194 time to create 1 rle with old method : 0.07775545120239258 length of segment : 364 time for calcul the mask position with numpy : 0.00193023681640625 nb_pixel_total : 41344 time to create 1 rle with old method : 0.0466463565826416 length of segment : 220 time for calcul the mask position with numpy : 0.0009624958038330078 nb_pixel_total : 13763 time to create 1 rle with old method : 0.015092611312866211 length of segment : 229 time for calcul the mask position with numpy : 0.0013396739959716797 nb_pixel_total : 16157 time to create 1 rle with old method : 0.018972396850585938 length of segment : 164 time for calcul the mask position with numpy : 0.0010838508605957031 nb_pixel_total : 14582 time to create 1 rle with old method : 0.01711583137512207 length of segment : 196 time for calcul the mask position with numpy : 0.006838560104370117 nb_pixel_total : 121228 time to create 1 rle with old method : 0.12966203689575195 length of segment : 572 time for calcul the mask position with numpy : 0.0046842098236083984 nb_pixel_total : 32758 time to create 1 rle with old method : 0.03831195831298828 length of segment : 319 time for calcul the mask position with numpy : 0.0008015632629394531 nb_pixel_total : 8974 time to create 1 rle with old method : 0.010727643966674805 length of segment : 91 time for calcul the mask position with numpy : 0.001867055892944336 nb_pixel_total : 28827 time to create 1 rle with old method : 0.03426051139831543 length of segment : 191 time for calcul the mask position with numpy : 0.0041539669036865234 nb_pixel_total : 12978 time to create 1 rle with old method : 0.015508174896240234 length of segment : 336 time for calcul the mask position with numpy : 0.004094362258911133 nb_pixel_total : 54812 time to create 1 rle with old method : 0.0669093132019043 length of segment : 318 time for calcul the mask position with numpy : 0.0009074211120605469 nb_pixel_total : 21600 time to create 1 rle with old method : 0.025898456573486328 length of segment : 204 time for calcul the mask position with numpy : 0.0018002986907958984 nb_pixel_total : 26091 time to create 1 rle with old method : 0.03010392189025879 length of segment : 174 time for calcul the mask position with numpy : 0.0038461685180664062 nb_pixel_total : 40217 time to create 1 rle with old method : 0.04454183578491211 length of segment : 523 time for calcul the mask position with numpy : 0.0003681182861328125 nb_pixel_total : 4070 time to create 1 rle with old method : 0.005090951919555664 length of segment : 67 time for calcul the mask position with numpy : 0.0016448497772216797 nb_pixel_total : 27695 time to create 1 rle with old method : 0.03317427635192871 length of segment : 158 time spent for convertir_results : 19.31187868118286 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 164 chid ids of type : 3760 Number RLEs to save : 66537 save missing photos in datou_result : time spend for datou_step_exec : 101.23009014129639 time spend to save output : 3.8435099124908447 total time spend for step 1 : 105.07360005378723 step2:blur_detection Tue Feb 18 15:08:27 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection toutes les photos sont déjà traitées, on saute les calculs Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 5 time used for this insertion : 0.008865833282470703 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 5 time used for this insertion : 0.008929967880249023 save missing photos in datou_result : time spend for datou_step_exec : 0.021689414978027344 time spend to save output : 0.022581815719604492 total time spend for step 2 : 0.044271230697631836 step3:brightness Tue Feb 18 15:08:27 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness toutes les photos sont déjà traitées, on saute les calculs Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 5 time used for this insertion : 0.009582757949829102 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 5 time used for this insertion : 0.009115219116210938 save missing photos in datou_result : time spend for datou_step_exec : 0.026445627212524414 time spend to save output : 0.02362060546875 total time spend for step 3 : 0.050066232681274414 step4:crop_condition Tue Feb 18 15:08:27 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3760 Loading chi in step crop for list_pids : 5 ! batch 1 Loaded 164 chid ids of type : 3760 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : barquette param for this class : {'min_score': 0.7} filtre for class : barquette hashtag_id of this class : 492787675 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 12 About to insert : list_path_to_insert length 12 new photo from crops ! we have finished the crop for the class : barquette begin to crop the class : fibreux_cont param for this class : {'min_score': 0.7} filtre for class : fibreux_cont hashtag_id of this class : 2107756748 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 16 About to insert : list_path_to_insert length 16 new photo from crops ! we have finished the crop for the class : fibreux_cont begin to crop the class : film_plastique param for this class : {'min_score': 0.7} filtre for class : film_plastique hashtag_id of this class : 2107756122 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 25 About to insert : list_path_to_insert length 25 new photo from crops ! we have finished the crop for the class : film_plastique begin to crop the class : autre_contaminant param for this class : {'min_score': 0.7} filtre for class : autre_contaminant hashtag_id of this class : 2107756781 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 36 About to insert : list_path_to_insert length 36 new photo from crops ! we have finished the crop for the class : autre_contaminant begin to crop the class : etiquette_detachee param for this class : {'min_score': 0.7} filtre for class : etiquette_detachee hashtag_id of this class : 2107756860 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 22 About to insert : list_path_to_insert length 22 new photo from crops ! we have finished the crop for the class : etiquette_detachee begin to crop the class : pet_clair_cont param for this class : {'min_score': 0.7} filtre for class : pet_clair_cont hashtag_id of this class : 2107758154 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 11 About to insert : list_path_to_insert length 11 new photo from crops ! we have finished the crop for the class : pet_clair_cont begin to crop the class : metal_cont param for this class : {'min_score': 0.7} filtre for class : metal_cont hashtag_id of this class : 2107756749 begin to crop the class : pet_fonce_cont param for this class : {'min_score': 0.7} filtre for class : pet_fonce_cont hashtag_id of this class : 2107758155 begin to crop the class : pet_opaque_cont param for this class : {'min_score': 0.7} filtre for class : pet_opaque_cont hashtag_id of this class : 2107758156 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 4 About to insert : list_path_to_insert length 4 new photo from crops ! we have finished the crop for the class : pet_opaque_cont delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1065568816, 1065568708, 1065568705, 1065568698, 1065568694] Looping around the photos to save general results len do output : 126 /-3679040266Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040245Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040239Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040256Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040254Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040273Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040288Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040305Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040351Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040343Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040341Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040377Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040253Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040265Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040272Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040284Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040286Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040291Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040292Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040290Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040323Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040300Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040312Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040347Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040392Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040376Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040360Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040380Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040252Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040238Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040261Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040242Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040233Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040281Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040289Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040282Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040318Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040316Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040314Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040322Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040310Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040307Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040295Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040313Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040339Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040338Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040353Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040352Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040371Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040361Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040383Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040379Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040374Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040232Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040260Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040249Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040243Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040280Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040287Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040276Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040293Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040311Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040308Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040321Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040317Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040303Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040298Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040315Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040319Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040320Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040309Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040304Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040348Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040350Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040334Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040355Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040330Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040337Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040358Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040378Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040388Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040375Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040365Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040386Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040385Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040362Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040369Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040393Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040373Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040248Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040259Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040234Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040250Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040263Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040258Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040285Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040275Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040279Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040274Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040324Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040297Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040335Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040357Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040345Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040349Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040331Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040391Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040381Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040390Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040382Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040389Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040257Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040262Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040306Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040356Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040344Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040340Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040342Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040336Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040354Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040333Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040366Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040255Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040283Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040269Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3679040332Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3995', None, None, None, None, None, None, None, '2596639') ('3995', '5486001', '1065568816', None, None, None, None, None, '2596639') ('3995', None, None, None, None, None, None, None, '2596639') ('3995', '5486001', '1065568708', None, None, None, None, None, '2596639') ('3995', None, None, None, None, None, None, None, '2596639') ('3995', '5486001', '1065568705', None, None, None, None, None, '2596639') ('3995', None, None, None, None, None, None, None, '2596639') ('3995', '5486001', '1065568698', None, None, None, None, None, '2596639') ('3995', None, None, None, None, None, None, None, '2596639') ('3995', '5486001', '1065568694', None, None, None, None, None, '2596639') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 383 time used for this insertion : 0.03902268409729004 save_final save missing photos in datou_result : time spend for datou_step_exec : 32.76854157447815 time spend to save output : 0.043215274810791016 total time spend for step 4 : 32.81175684928894 step5:thcl Tue Feb 18 15:09:00 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step Thcl ! we are using the classfication for only one thcl 3233 time to import caffe and check if the image exist : 0.011695146560668945 time to convert the images to numpy array : 0.07320690155029297 time to import caffe and check if the image exist : 0.01996779441833496 time to convert the images to numpy array : 0.0836946964263916 time to import caffe and check if the image exist : 0.01315927505493164 time to convert the images to numpy array : 0.09482574462890625 time to import caffe and check if the image exist : 0.014159917831420898 time to convert the images to numpy array : 0.09937238693237305 time to import caffe and check if the image exist : 0.016611099243164062 time to convert the images to numpy array : 0.10008406639099121 time to import caffe and check if the image exist : 0.014497756958007812 time to convert the images to numpy array : 0.10563778877258301 time to import caffe and check if the image exist : 0.01588916778564453 time to convert the images to numpy array : 0.11070823669433594 time to import caffe and check if the image exist : 0.016332387924194336 time to convert the images to numpy array : 0.11199760437011719 time to import caffe and check if the image exist : 0.013494491577148438 time to convert the images to numpy array : 0.11672019958496094 time to import caffe and check if the image exist : 0.00506901741027832 time to convert the images to numpy array : 0.14183950424194336 total time to convert the images to numpy array : 0.4080522060394287 list photo_ids error: [] list photo_ids correct : [-3679040342, -3679040336, -3679040354, -3679040333, -3679040366, -3679040255, -3679040283, -3679040269, -3679040332, -3679040349, -3679040331, -3679040391, -3679040381, -3679040390, -3679040382, -3679040389, -3679040257, -3679040262, -3679040306, -3679040356, -3679040344, -3679040340, -3679040374, -3679040232, -3679040260, -3679040249, -3679040243, -3679040280, -3679040287, -3679040276, -3679040293, -3679040311, -3679040308, -3679040321, -3679040317, -3679040234, -3679040250, -3679040263, -3679040258, -3679040285, -3679040275, -3679040279, -3679040274, -3679040324, -3679040297, -3679040335, -3679040357, -3679040345, -3679040265, -3679040272, -3679040284, -3679040286, -3679040291, -3679040292, -3679040290, -3679040323, -3679040300, -3679040312, -3679040347, -3679040392, -3679040376, -3679040322, -3679040310, -3679040307, -3679040295, -3679040313, -3679040339, -3679040338, -3679040353, -3679040352, -3679040371, -3679040361, -3679040383, -3679040379, -3679040358, -3679040378, -3679040388, -3679040375, -3679040365, -3679040386, -3679040385, -3679040362, -3679040369, -3679040393, -3679040373, -3679040248, -3679040259, -3679040303, -3679040298, -3679040315, -3679040319, -3679040320, -3679040309, -3679040304, -3679040348, -3679040350, -3679040334, -3679040355, -3679040330, -3679040337, -3679040360, -3679040380, -3679040252, -3679040238, -3679040261, -3679040242, -3679040233, -3679040281, -3679040289, -3679040282, -3679040318, -3679040316, -3679040314, -3679040266, -3679040245, -3679040239, -3679040256, -3679040254, -3679040273, -3679040288, -3679040305, -3679040351, -3679040343, -3679040341, -3679040377, -3679040253] number of photos to traite : 126 try to delete the photos incorrect in DB tagging for thcl : 3233 To do loadFromThcl(), then load ParamDescType : thcl3233 thcls : [{'id': 3233, 'mtr_user_id': 31, 'name': 'learn_generique_01122021_6000_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'barquette_opaque,carton,ela,environnement,etiquette,film_plastique,kraft,metal,papier,pehd,pet_clair,pet_fonce,pet_opaque,textiles_sanitaires', 'svm_portfolios_learning': '4825680,4829009,4825557,4824709,4824714,4824733,4825676,4824737,4824753,4829002,4824848,4824863,4825667,4825677', 'photo_hashtag_type': 4151, 'photo_desc_type': 5557, 'type_classification': 'caffe', 'hashtag_id_list': '2107760128,492774966,492741797,493012381,492636447,2107756122,493202403,492628673,492668766,628944319,2107755846,2107755900,2107759152,2107760129'}] thcl {'id': 3233, 'mtr_user_id': 31, 'name': 'learn_generique_01122021_6000_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'barquette_opaque,carton,ela,environnement,etiquette,film_plastique,kraft,metal,papier,pehd,pet_clair,pet_fonce,pet_opaque,textiles_sanitaires', 'svm_portfolios_learning': '4825680,4829009,4825557,4824709,4824714,4824733,4825676,4824737,4824753,4829002,4824848,4824863,4825667,4825677', 'photo_hashtag_type': 4151, 'photo_desc_type': 5557, 'type_classification': 'caffe', 'hashtag_id_list': '2107760128,492774966,492741797,493012381,492636447,2107756122,493202403,492628673,492668766,628944319,2107755846,2107755900,2107759152,2107760129'} Update svm_hashtag_type_desc : 5557 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5557, 'learn_generique_01122021_6000_v2', 2048, 2048, 'learn_generique_01122021_6000_v2', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2021, 12, 1, 19, 59, 17), datetime.datetime(2021, 12, 1, 19, 59, 17)) To loadFromThcl() : net_5557 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10372 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5557, 'learn_generique_01122021_6000_v2', 2048, 2048, 'learn_generique_01122021_6000_v2', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2021, 12, 1, 19, 59, 17), datetime.datetime(2021, 12, 1, 19, 59, 17)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_generique_01122021_6000_v2 Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_generique_01122021_6000_v2 model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_generique_01122021_6000_v2 /data/models_weight/learn_generique_01122021_6000_v2/caffemodel size_local : 94383085 size in s3 : 94383085 create time local : 2021-12-09 17:03:46 create time in s3 : 2021-12-01 18:45:54 caffemodel already exist and didn't need to update /data/models_weight/learn_generique_01122021_6000_v2/deploy.prototxt size_local : 32544 size in s3 : 32544 create time local : 2021-12-09 17:03:46 create time in s3 : 2021-12-01 18:45:53 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_generique_01122021_6000_v2/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-12-09 17:03:47 create time in s3 : 2021-12-01 18:59:02 mean.npy already exist and didn't need to update /data/models_weight/learn_generique_01122021_6000_v2/synset_words.txt size_local : 454 size in s3 : 454 create time local : 2021-12-09 17:03:47 create time in s3 : 2021-12-01 18:59:15 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/:/home/admin/workarea/git/apy/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_generique_01122021_6000_v2/deploy.prototxt caffemodel_filename : /data/models_weight/learn_generique_01122021_6000_v2/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10153 max_wait_temp : 1 max_wait : 0 dict_keys(['pool5', 'prob']) time used to do the prepocess of the images : 0.9735159873962402 time used to do the prediction : 0.8591337203979492 save descriptor for thcl : 3233 time to traite the descriptors : 0.7737288475036621 Catched exception ! Connect or reconnect ! storage_type for insertDescriptorsMulti : 3 Missing photo l117 : -3679040342 Missing photo l117 : -3679040336 Missing photo l117 : -3679040354 Missing photo l117 : -3679040333 Missing photo l117 : -3679040366 Missing photo l117 : -3679040255 Missing photo l117 : -3679040283 Missing photo l117 : -3679040269 Missing photo l117 : -3679040332 Missing photo l117 : -3679040349 Missing photo l117 : -3679040331 Missing photo l117 : -3679040391 Missing photo l117 : -3679040381 Missing photo l117 : -3679040390 Missing photo l117 : -3679040382 Missing photo l117 : -3679040389 Missing photo l117 : -3679040257 Missing photo l117 : -3679040262 Missing photo l117 : -3679040306 Missing photo l117 : -3679040356 Missing photo l117 : -3679040344 Missing photo l117 : -3679040340 Missing photo l117 : -3679040374 Missing photo l117 : -3679040232 Missing photo l117 : -3679040260 Missing photo l117 : -3679040249 Missing photo l117 : -3679040243 Missing photo l117 : -3679040280 Missing photo l117 : -3679040287 Missing photo l117 : -3679040276 Missing photo l117 : -3679040293 Missing photo l117 : -3679040311 Missing photo l117 : -3679040308 Missing photo l117 : -3679040321 Missing photo l117 : -3679040317 Missing photo l117 : -3679040234 Missing photo l117 : -3679040250 Missing photo l117 : -3679040263 Missing photo l117 : -3679040258 Missing photo l117 : -3679040285 Missing photo l117 : -3679040275 Missing photo l117 : -3679040279 Missing photo l117 : -3679040274 Missing photo l117 : -3679040324 Missing photo l117 : -3679040297 Missing photo l117 : -3679040335 Missing photo l117 : -3679040357 Missing photo l117 : -3679040345 Missing photo l117 : -3679040265 Missing photo l117 : -3679040272 Missing photo l117 : -3679040284 Missing photo l117 : -3679040286 Missing photo l117 : -3679040291 Missing photo l117 : -3679040292 Missing photo l117 : -3679040290 Missing photo l117 : -3679040323 Missing photo l117 : -3679040300 Missing photo l117 : -3679040312 Missing photo l117 : -3679040347 Missing photo l117 : -3679040392 Missing photo l117 : -3679040376 Missing photo l117 : -3679040322 Missing photo l117 : -3679040310 Missing photo l117 : -3679040307 Missing photo l117 : -3679040295 Missing photo l117 : -3679040313 Missing photo l117 : -3679040339 Missing photo l117 : -3679040338 Missing photo l117 : -3679040353 Missing photo l117 : -3679040352 Missing photo l117 : -3679040371 Missing photo l117 : -3679040361 Missing photo l117 : -3679040383 Missing photo l117 : -3679040379 Missing photo l117 : -3679040358 Missing photo l117 : -3679040378 Missing photo l117 : -3679040388 Missing photo l117 : -3679040375 Missing photo l117 : -3679040365 Missing photo l117 : -3679040386 Missing photo l117 : -3679040385 Missing photo l117 : -3679040362 Missing photo l117 : -3679040369 Missing photo l117 : -3679040393 Missing photo l117 : -3679040373 Missing photo l117 : -3679040248 Missing photo l117 : -3679040259 Missing photo l117 : -3679040303 Missing photo l117 : -3679040298 Missing photo l117 : -3679040315 Missing photo l117 : -3679040319 Missing photo l117 : -3679040320 Missing photo l117 : -3679040309 Missing photo l117 : -3679040304 Missing photo l117 : -3679040348 Missing photo l117 : -3679040350 Missing photo l117 : -3679040334 Missing photo l117 : -3679040355 Missing photo l117 : -3679040330 Missing photo l117 : -3679040337 Missing photo l117 : -3679040360 Missing photo l117 : -3679040380 Missing photo l117 : -3679040252 Missing photo l117 : -3679040238 Missing photo l117 : -3679040261 Missing photo l117 : -3679040242 Missing photo l117 : -3679040233 Missing photo l117 : -3679040281 Missing photo l117 : -3679040289 Missing photo l117 : -3679040282 Missing photo l117 : -3679040318 Missing photo l117 : -3679040316 Missing photo l117 : -3679040314 Missing photo l117 : -3679040266 Missing photo l117 : -3679040245 Missing photo l117 : -3679040239 Missing photo l117 : -3679040256 Missing photo l117 : -3679040254 Missing photo l117 : -3679040273 Missing photo l117 : -3679040288 Missing photo l117 : -3679040305 Missing photo l117 : -3679040351 Missing photo l117 : -3679040343 Missing photo l117 : -3679040341 Missing photo l117 : -3679040377 Missing photo l117 : -3679040253 To insert : -3679040342 Missing photo l134 : -3679040342 To insert : -3679040336 Missing photo l134 : -3679040336 To insert : -3679040354 Missing photo l134 : -3679040354 To insert : -3679040333 Missing photo l134 : -3679040333 To insert : -3679040366 Missing photo l134 : -3679040366 To insert : -3679040255 Missing photo l134 : -3679040255 To insert : -3679040283 Missing photo l134 : -3679040283 To insert : -3679040269 Missing photo l134 : -3679040269 To insert : -3679040332 Missing photo l134 : -3679040332 To insert : -3679040349 Missing photo l134 : -3679040349 To insert : -3679040331 Missing photo l134 : -3679040331 To insert : -3679040391 Missing photo l134 : -3679040391 To insert : -3679040381 Missing photo l134 : -3679040381 To insert : -3679040390 Missing photo l134 : -3679040390 To insert : -3679040382 Missing photo l134 : -3679040382 To insert : -3679040389 Missing photo l134 : -3679040389 To insert : -3679040257 Missing photo l134 : -3679040257 To insert : -3679040262 Missing photo l134 : -3679040262 To insert : -3679040306 Missing photo l134 : -3679040306 To insert : -3679040356 Missing photo l134 : -3679040356 To insert : -3679040344 Missing photo l134 : -3679040344 To insert : -3679040340 Missing photo l134 : -3679040340 To insert : -3679040374 Missing photo l134 : -3679040374 To insert : -3679040232 Missing photo l134 : -3679040232 To insert : -3679040260 Missing photo l134 : -3679040260 To insert : -3679040249 Missing photo l134 : -3679040249 To insert : -3679040243 Missing photo l134 : -3679040243 To insert : -3679040280 Missing photo l134 : -3679040280 To insert : -3679040287 Missing photo l134 : -3679040287 To insert : -3679040276 Missing photo l134 : -3679040276 To insert : -3679040293 Missing photo l134 : -3679040293 To insert : -3679040311 Missing photo l134 : -3679040311 To insert : -3679040308 Missing photo l134 : -3679040308 To insert : -3679040321 Missing photo l134 : -3679040321 To insert : -3679040317 Missing photo l134 : -3679040317 To insert : -3679040234 Missing photo l134 : -3679040234 To insert : -3679040250 Missing photo l134 : -3679040250 To insert : -3679040263 Missing photo l134 : -3679040263 To insert : -3679040258 Missing photo l134 : -3679040258 To insert : -3679040285 Missing photo l134 : -3679040285 To insert : -3679040275 Missing photo l134 : -3679040275 To insert : -3679040279 Missing photo l134 : -3679040279 To insert : -3679040274 Missing photo l134 : -3679040274 To insert : -3679040324 Missing photo l134 : -3679040324 To insert : -3679040297 Missing photo l134 : -3679040297 To insert : -3679040335 Missing photo l134 : -3679040335 To insert : -3679040357 Missing photo l134 : -3679040357 To insert : -3679040345 Missing photo l134 : -3679040345 To insert : -3679040265 Missing photo l134 : -3679040265 To insert : -3679040272 Missing photo l134 : -3679040272 To insert : -3679040284 Missing photo l134 : -3679040284 To insert : -3679040286 Missing photo l134 : -3679040286 To insert : -3679040291 Missing photo l134 : -3679040291 To insert : -3679040292 Missing photo l134 : -3679040292 To insert : -3679040290 Missing photo l134 : -3679040290 To insert : -3679040323 Missing photo l134 : -3679040323 To insert : -3679040300 Missing photo l134 : -3679040300 To insert : -3679040312 Missing photo l134 : -3679040312 To insert : -3679040347 Missing photo l134 : -3679040347 To insert : -3679040392 Missing photo l134 : -3679040392 To insert : -3679040376 Missing photo l134 : -3679040376 To insert : -3679040322 Missing photo l134 : -3679040322 To insert : -3679040310 Missing photo l134 : -3679040310 To insert : -3679040307 Missing photo l134 : -3679040307 To insert : -3679040295 Missing photo l134 : -3679040295 To insert : -3679040313 Missing photo l134 : -3679040313 To insert : -3679040339 Missing photo l134 : -3679040339 To insert : -3679040338 Missing photo l134 : -3679040338 To insert : -3679040353 Missing photo l134 : -3679040353 To insert : -3679040352 Missing photo l134 : -3679040352 To insert : -3679040371 Missing photo l134 : -3679040371 To insert : -3679040361 Missing photo l134 : -3679040361 To insert : -3679040383 Missing photo l134 : -3679040383 To insert : -3679040379 Missing photo l134 : -3679040379 To insert : -3679040358 Missing photo l134 : -3679040358 To insert : -3679040378 Missing photo l134 : -3679040378 To insert : -3679040388 Missing photo l134 : -3679040388 To insert : -3679040375 Missing photo l134 : -3679040375 To insert : -3679040365 Missing photo l134 : -3679040365 To insert : -3679040386 Missing photo l134 : -3679040386 To insert : -3679040385 Missing photo l134 : -3679040385 To insert : -3679040362 Missing photo l134 : -3679040362 To insert : -3679040369 Missing photo l134 : -3679040369 To insert : -3679040393 Missing photo l134 : -3679040393 To insert : -3679040373 Missing photo l134 : -3679040373 To insert : -3679040248 Missing photo l134 : -3679040248 To insert : -3679040259 Missing photo l134 : -3679040259 To insert : -3679040303 Missing photo l134 : -3679040303 To insert : -3679040298 Missing photo l134 : -3679040298 To insert : -3679040315 Missing photo l134 : -3679040315 To insert : -3679040319 Missing photo l134 : -3679040319 To insert : -3679040320 Missing photo l134 : -3679040320 To insert : -3679040309 Missing photo l134 : -3679040309 To insert : -3679040304 Missing photo l134 : -3679040304 To insert : -3679040348 Missing photo l134 : -3679040348 To insert : -3679040350 Missing photo l134 : -3679040350 To insert : -3679040334 Missing photo l134 : -3679040334 To insert : -3679040355 Missing photo l134 : -3679040355 To insert : -3679040330 Missing photo l134 : -3679040330 To insert : -3679040337 Missing photo l134 : -3679040337 To insert : -3679040360 Missing photo l134 : -3679040360 To insert : -3679040380 Missing photo l134 : -3679040380 To insert : -3679040252 Missing photo l134 : -3679040252 To insert : -3679040238 Missing photo l134 : -3679040238 To insert : -3679040261 Missing photo l134 : -3679040261 To insert : -3679040242 Missing photo l134 : -3679040242 To insert : -3679040233 Missing photo l134 : -3679040233 To insert : -3679040281 Missing photo l134 : -3679040281 To insert : -3679040289 Missing photo l134 : -3679040289 To insert : -3679040282 Missing photo l134 : -3679040282 To insert : -3679040318 Missing photo l134 : -3679040318 To insert : -3679040316 Missing photo l134 : -3679040316 To insert : -3679040314 Missing photo l134 : -3679040314 To insert : -3679040266 Missing photo l134 : -3679040266 To insert : -3679040245 Missing photo l134 : -3679040245 To insert : -3679040239 Missing photo l134 : -3679040239 To insert : -3679040256 Missing photo l134 : -3679040256 To insert : -3679040254 Missing photo l134 : -3679040254 To insert : -3679040273 Missing photo l134 : -3679040273 To insert : -3679040288 Missing photo l134 : -3679040288 To insert : -3679040305 Missing photo l134 : -3679040305 To insert : -3679040351 Missing photo l134 : -3679040351 To insert : -3679040343 Missing photo l134 : -3679040343 To insert : -3679040341 Missing photo l134 : -3679040341 To insert : -3679040377 Missing photo l134 : -3679040377 To insert : -3679040253 Missing photo l134 : -3679040253 time to insert the descriptors : 26.408145427703857 Inside saveOutput : final : False verbose : 0 time used to find the portfolios of the photos SAVE THCL : begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 0 time used for this insertion : 5.7220458984375e-06 save missing photos in datou_result : time spend for datou_step_exec : 33.124242544174194 time spend to save output : 0.05274319648742676 total time spend for step 5 : 33.17698574066162 step6:merge_mask_thcl_custom Tue Feb 18 15:09:33 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step merge_mask_thcl_custom batch 1 Loaded 164 chid ids of type : 3760 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++As expected we have just one thcl present End of step merge_mask_thcl_custom Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : merge_mask_thcl_custom we use saveGeneral [1065568816, 1065568708, 1065568705, 1065568698, 1065568694] Looping around the photos to save general results len do output : 5 /1065568816Didn't retrieve data .Didn't retrieve data . /1065568708Didn't retrieve data .Didn't retrieve data . /1065568705Didn't retrieve data .Didn't retrieve data . /1065568698Didn't retrieve data .Didn't retrieve data . /1065568694Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3995', None, None, None, None, None, None, None, '2596639') ('3995', '5486001', '1065568816', None, None, None, None, None, '2596639') ('3995', None, None, None, None, None, None, None, '2596639') ('3995', '5486001', '1065568708', None, None, None, None, None, '2596639') ('3995', None, None, None, None, None, None, None, '2596639') ('3995', '5486001', '1065568705', None, None, None, None, None, '2596639') ('3995', None, None, None, None, None, None, None, '2596639') ('3995', '5486001', '1065568698', None, None, None, None, None, '2596639') ('3995', None, None, None, None, None, None, None, '2596639') ('3995', '5486001', '1065568694', None, None, None, None, None, '2596639') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 15 time used for this insertion : 0.012635946273803711 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.055871009826660156 time spend to save output : 0.013015270233154297 total time spend for step 6 : 0.06888628005981445 step7:rle_unique_nms_with_priority Tue Feb 18 15:09:33 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms nb_obj : 25 nb_hashtags : 8 time to prepare the origin masks : 20.767236948013306 time for calcul the mask position with numpy : 0.47080254554748535 nb_pixel_total : 7030294 time to create 1 rle with new method : 1.433046579360962 time for calcul the mask position with numpy : 0.039972543716430664 nb_pixel_total : 40498 time to create 1 rle with old method : 0.04422473907470703 time for calcul the mask position with numpy : 0.03922867774963379 nb_pixel_total : 21072 time to create 1 rle with old method : 0.024063587188720703 time for calcul the mask position with numpy : 0.03931689262390137 nb_pixel_total : 39820 time to create 1 rle with old method : 0.04364013671875 time for calcul the mask position with numpy : 0.04822134971618652 nb_pixel_total : 39893 time to create 1 rle with old method : 0.0420835018157959 time for calcul the mask position with numpy : 0.03925371170043945 nb_pixel_total : 8238 time to create 1 rle with old method : 0.009201288223266602 time for calcul the mask position with numpy : 0.03866434097290039 nb_pixel_total : 20264 time to create 1 rle with old method : 0.021971464157104492 time for calcul the mask position with numpy : 0.038596391677856445 nb_pixel_total : 10563 time to create 1 rle with old method : 0.011356830596923828 time for calcul the mask position with numpy : 0.04312849044799805 nb_pixel_total : 75070 time to create 1 rle with old method : 0.10119819641113281 time for calcul the mask position with numpy : 0.041104793548583984 nb_pixel_total : 37347 time to create 1 rle with old method : 0.039381980895996094 time for calcul the mask position with numpy : 0.039395809173583984 nb_pixel_total : 36096 time to create 1 rle with old method : 0.03889966011047363 time for calcul the mask position with numpy : 0.03928804397583008 nb_pixel_total : 40380 time to create 1 rle with old method : 0.04508066177368164 time for calcul the mask position with numpy : 0.04116082191467285 nb_pixel_total : 60445 time to create 1 rle with old method : 0.06504011154174805 time for calcul the mask position with numpy : 0.03882288932800293 nb_pixel_total : 56331 time to create 1 rle with old method : 0.060568809509277344 time for calcul the mask position with numpy : 0.040947675704956055 nb_pixel_total : 163362 time to create 1 rle with new method : 1.0256969928741455 time for calcul the mask position with numpy : 0.040786027908325195 nb_pixel_total : 72706 time to create 1 rle with old method : 0.07593750953674316 time for calcul the mask position with numpy : 0.03773307800292969 nb_pixel_total : 20853 time to create 1 rle with old method : 0.021127939224243164 time for calcul the mask position with numpy : 0.03754544258117676 nb_pixel_total : 36340 time to create 1 rle with old method : 0.03844761848449707 time for calcul the mask position with numpy : 0.03829550743103027 nb_pixel_total : 105566 time to create 1 rle with old method : 0.11148285865783691 time for calcul the mask position with numpy : 0.03954172134399414 nb_pixel_total : 27129 time to create 1 rle with old method : 0.029211997985839844 time for calcul the mask position with numpy : 0.0399785041809082 nb_pixel_total : 17793 time to create 1 rle with old method : 0.0191805362701416 time for calcul the mask position with numpy : 0.03863358497619629 nb_pixel_total : 4467 time to create 1 rle with old method : 0.0047342777252197266 time for calcul the mask position with numpy : 0.03905320167541504 nb_pixel_total : 74644 time to create 1 rle with old method : 0.07965993881225586 time for calcul the mask position with numpy : 0.038718223571777344 nb_pixel_total : 24918 time to create 1 rle with old method : 0.028055667877197266 time for calcul the mask position with numpy : 0.034941673278808594 nb_pixel_total : 10934 time to create 1 rle with old method : 0.012063026428222656 time for calcul the mask position with numpy : 0.031590938568115234 nb_pixel_total : 6897 time to create 1 rle with old method : 0.007783174514770508 create new chi : 4.947105407714844 time to delete rle : 0.06128716468811035 batch 1 Loaded 26 chid ids of type : 4211 Number RLEs to save : 15225 TO DO : save crop sub photo not yet done ! save time : 2.45051908493042 nb_obj : 21 nb_hashtags : 5 time to prepare the origin masks : 18.643624782562256 time for calcul the mask position with numpy : 0.5094950199127197 nb_pixel_total : 7217762 time to create 1 rle with new method : 1.1683628559112549 time for calcul the mask position with numpy : 0.024172306060791016 nb_pixel_total : 2881 time to create 1 rle with old method : 0.005329608917236328 time for calcul the mask position with numpy : 0.026477813720703125 nb_pixel_total : 20802 time to create 1 rle with old method : 0.022618532180786133 time for calcul the mask position with numpy : 0.025094270706176758 nb_pixel_total : 65259 time to create 1 rle with old method : 0.07111954689025879 time for calcul the mask position with numpy : 0.024624347686767578 nb_pixel_total : 19645 time to create 1 rle with old method : 0.0221560001373291 time for calcul the mask position with numpy : 0.027384281158447266 nb_pixel_total : 34684 time to create 1 rle with old method : 0.053794145584106445 time for calcul the mask position with numpy : 0.024953842163085938 nb_pixel_total : 91693 time to create 1 rle with old method : 0.10168147087097168 time for calcul the mask position with numpy : 0.028467178344726562 nb_pixel_total : 48459 time to create 1 rle with old method : 0.05111432075500488 time for calcul the mask position with numpy : 0.024927616119384766 nb_pixel_total : 88287 time to create 1 rle with old method : 0.09340190887451172 time for calcul the mask position with numpy : 0.025144338607788086 nb_pixel_total : 27140 time to create 1 rle with old method : 0.029094457626342773 time for calcul the mask position with numpy : 0.024982452392578125 nb_pixel_total : 46990 time to create 1 rle with old method : 0.04964804649353027 time for calcul the mask position with numpy : 0.025438785552978516 nb_pixel_total : 104810 time to create 1 rle with old method : 0.10982871055603027 time for calcul the mask position with numpy : 0.02391648292541504 nb_pixel_total : 29731 time to create 1 rle with old method : 0.03137660026550293 time for calcul the mask position with numpy : 0.02402019500732422 nb_pixel_total : 17810 time to create 1 rle with old method : 0.020076751708984375 time for calcul the mask position with numpy : 0.023891448974609375 nb_pixel_total : 37980 time to create 1 rle with old method : 0.03964567184448242 time for calcul the mask position with numpy : 0.024932384490966797 nb_pixel_total : 13434 time to create 1 rle with old method : 0.01387643814086914 time for calcul the mask position with numpy : 0.02483057975769043 nb_pixel_total : 16849 time to create 1 rle with old method : 0.018797874450683594 time for calcul the mask position with numpy : 0.026090383529663086 nb_pixel_total : 43536 time to create 1 rle with old method : 0.05158114433288574 time for calcul the mask position with numpy : 0.02702045440673828 nb_pixel_total : 39073 time to create 1 rle with old method : 0.041586875915527344 time for calcul the mask position with numpy : 0.024953842163085938 nb_pixel_total : 23167 time to create 1 rle with old method : 0.025304794311523438 time for calcul the mask position with numpy : 0.02550363540649414 nb_pixel_total : 12604 time to create 1 rle with old method : 0.013880729675292969 time for calcul the mask position with numpy : 0.02513599395751953 nb_pixel_total : 79324 time to create 1 rle with old method : 0.08494687080383301 create new chi : 3.2035770416259766 time to delete rle : 0.0013394355773925781 batch 1 Loaded 22 chid ids of type : 4211 Number RLEs to save : 14740 TO DO : save crop sub photo not yet done ! save time : 0.8846924304962158 nb_obj : 26 nb_hashtags : 8 time to prepare the origin masks : 13.407380819320679 time for calcul the mask position with numpy : 0.34313440322875977 nb_pixel_total : 6993680 time to create 1 rle with new method : 0.7189393043518066 time for calcul the mask position with numpy : 0.03157854080200195 nb_pixel_total : 25810 time to create 1 rle with old method : 0.025934934616088867 time for calcul the mask position with numpy : 0.031089067459106445 nb_pixel_total : 26046 time to create 1 rle with old method : 0.026265621185302734 time for calcul the mask position with numpy : 0.030671358108520508 nb_pixel_total : 32525 time to create 1 rle with old method : 0.03317070007324219 time for calcul the mask position with numpy : 0.0317234992980957 nb_pixel_total : 31657 time to create 1 rle with old method : 0.03441190719604492 time for calcul the mask position with numpy : 0.032151222229003906 nb_pixel_total : 11846 time to create 1 rle with old method : 0.012619495391845703 time for calcul the mask position with numpy : 0.03259110450744629 nb_pixel_total : 9201 time to create 1 rle with old method : 0.009938478469848633 time for calcul the mask position with numpy : 0.03246331214904785 nb_pixel_total : 24915 time to create 1 rle with old method : 0.026058197021484375 time for calcul the mask position with numpy : 0.0317232608795166 nb_pixel_total : 5500 time to create 1 rle with old method : 0.005835533142089844 time for calcul the mask position with numpy : 0.0320286750793457 nb_pixel_total : 33493 time to create 1 rle with old method : 0.03689312934875488 time for calcul the mask position with numpy : 0.03323006629943848 nb_pixel_total : 21811 time to create 1 rle with old method : 0.023915529251098633 time for calcul the mask position with numpy : 0.03315305709838867 nb_pixel_total : 10575 time to create 1 rle with old method : 0.012001991271972656 time for calcul the mask position with numpy : 0.03340721130371094 nb_pixel_total : 39128 time to create 1 rle with old method : 0.04160451889038086 time for calcul the mask position with numpy : 0.03248476982116699 nb_pixel_total : 28654 time to create 1 rle with old method : 0.030431509017944336 time for calcul the mask position with numpy : 0.03330540657043457 nb_pixel_total : 98730 time to create 1 rle with old method : 0.10437965393066406 time for calcul the mask position with numpy : 0.03331303596496582 nb_pixel_total : 50082 time to create 1 rle with old method : 0.0527951717376709 time for calcul the mask position with numpy : 0.03254580497741699 nb_pixel_total : 36005 time to create 1 rle with old method : 0.038883209228515625 time for calcul the mask position with numpy : 0.03218841552734375 nb_pixel_total : 50660 time to create 1 rle with old method : 0.0545964241027832 time for calcul the mask position with numpy : 0.03289628028869629 nb_pixel_total : 28182 time to create 1 rle with old method : 0.03118276596069336 time for calcul the mask position with numpy : 0.03537416458129883 nb_pixel_total : 135717 time to create 1 rle with old method : 0.14577603340148926 time for calcul the mask position with numpy : 0.032405853271484375 nb_pixel_total : 134981 time to create 1 rle with old method : 0.1406097412109375 time for calcul the mask position with numpy : 0.03216695785522461 nb_pixel_total : 132270 time to create 1 rle with old method : 0.14165043830871582 time for calcul the mask position with numpy : 0.03283357620239258 nb_pixel_total : 35586 time to create 1 rle with old method : 0.039576053619384766 time for calcul the mask position with numpy : 0.0322878360748291 nb_pixel_total : 22432 time to create 1 rle with old method : 0.024382829666137695 time for calcul the mask position with numpy : 0.032591819763183594 nb_pixel_total : 20641 time to create 1 rle with old method : 0.022523880004882812 time for calcul the mask position with numpy : 0.03278183937072754 nb_pixel_total : 30052 time to create 1 rle with old method : 0.032416582107543945 time for calcul the mask position with numpy : 0.03256869316101074 nb_pixel_total : 11741 time to create 1 rle with old method : 0.012724161148071289 create new chi : 3.1028854846954346 time to delete rle : 0.001478433609008789 batch 1 Loaded 27 chid ids of type : 4211 Number RLEs to save : 15699 TO DO : save crop sub photo not yet done ! save time : 1.2747628688812256 nb_obj : 28 nb_hashtags : 6 time to prepare the origin masks : 18.215323448181152 time for calcul the mask position with numpy : 0.4158148765563965 nb_pixel_total : 6989764 time to create 1 rle with new method : 1.0439436435699463 time for calcul the mask position with numpy : 0.03258323669433594 nb_pixel_total : 29592 time to create 1 rle with old method : 0.031854867935180664 time for calcul the mask position with numpy : 0.03318142890930176 nb_pixel_total : 15397 time to create 1 rle with old method : 0.016993284225463867 time for calcul the mask position with numpy : 0.03260183334350586 nb_pixel_total : 14455 time to create 1 rle with old method : 0.015500783920288086 time for calcul the mask position with numpy : 0.03293561935424805 nb_pixel_total : 20761 time to create 1 rle with old method : 0.023061275482177734 time for calcul the mask position with numpy : 0.034841299057006836 nb_pixel_total : 24272 time to create 1 rle with old method : 0.0263671875 time for calcul the mask position with numpy : 0.03283953666687012 nb_pixel_total : 9847 time to create 1 rle with old method : 0.010895490646362305 time for calcul the mask position with numpy : 0.03300762176513672 nb_pixel_total : 48914 time to create 1 rle with old method : 0.05361580848693848 time for calcul the mask position with numpy : 0.03795027732849121 nb_pixel_total : 30825 time to create 1 rle with old method : 0.04276585578918457 time for calcul the mask position with numpy : 0.03329205513000488 nb_pixel_total : 46903 time to create 1 rle with old method : 0.06299853324890137 time for calcul the mask position with numpy : 0.03263998031616211 nb_pixel_total : 14479 time to create 1 rle with old method : 0.01630854606628418 time for calcul the mask position with numpy : 0.03426837921142578 nb_pixel_total : 95909 time to create 1 rle with old method : 0.10179734230041504 time for calcul the mask position with numpy : 0.033582448959350586 nb_pixel_total : 11007 time to create 1 rle with old method : 0.012734174728393555 time for calcul the mask position with numpy : 0.03407692909240723 nb_pixel_total : 29578 time to create 1 rle with old method : 0.033161163330078125 time for calcul the mask position with numpy : 0.03770160675048828 nb_pixel_total : 41921 time to create 1 rle with old method : 0.04548954963684082 time for calcul the mask position with numpy : 0.033605337142944336 nb_pixel_total : 33813 time to create 1 rle with old method : 0.05300402641296387 time for calcul the mask position with numpy : 0.036529541015625 nb_pixel_total : 59725 time to create 1 rle with old method : 0.06440591812133789 time for calcul the mask position with numpy : 0.03523373603820801 nb_pixel_total : 38632 time to create 1 rle with old method : 0.04046773910522461 time for calcul the mask position with numpy : 0.03298592567443848 nb_pixel_total : 55950 time to create 1 rle with old method : 0.06000328063964844 time for calcul the mask position with numpy : 0.03296232223510742 nb_pixel_total : 45221 time to create 1 rle with old method : 0.047492027282714844 time for calcul the mask position with numpy : 0.03291440010070801 nb_pixel_total : 97574 time to create 1 rle with old method : 0.10357666015625 time for calcul the mask position with numpy : 0.032549381256103516 nb_pixel_total : 83267 time to create 1 rle with old method : 0.09322237968444824 time for calcul the mask position with numpy : 0.03609752655029297 nb_pixel_total : 928 time to create 1 rle with old method : 0.0012600421905517578 time for calcul the mask position with numpy : 0.03460574150085449 nb_pixel_total : 45383 time to create 1 rle with old method : 0.05116391181945801 time for calcul the mask position with numpy : 0.03433346748352051 nb_pixel_total : 12752 time to create 1 rle with old method : 0.014653682708740234 time for calcul the mask position with numpy : 0.0341181755065918 nb_pixel_total : 45757 time to create 1 rle with old method : 0.04916024208068848 time for calcul the mask position with numpy : 0.035346031188964844 nb_pixel_total : 26085 time to create 1 rle with old method : 0.029306888580322266 time for calcul the mask position with numpy : 0.03621983528137207 nb_pixel_total : 83266 time to create 1 rle with old method : 0.09393644332885742 time for calcul the mask position with numpy : 0.03270125389099121 nb_pixel_total : 29943 time to create 1 rle with old method : 0.03331398963928223 create new chi : 3.685518503189087 time to delete rle : 0.0015883445739746094 batch 1 Loaded 29 chid ids of type : 4211 Number RLEs to save : 17545 TO DO : save crop sub photo not yet done ! save time : 1.092787504196167 nb_obj : 26 nb_hashtags : 7 time to prepare the origin masks : 15.466997623443604 time for calcul the mask position with numpy : 1.0802736282348633 nb_pixel_total : 6894572 time to create 1 rle with new method : 1.3094851970672607 time for calcul the mask position with numpy : 0.0332181453704834 nb_pixel_total : 35291 time to create 1 rle with old method : 0.043771982192993164 time for calcul the mask position with numpy : 0.03661680221557617 nb_pixel_total : 55157 time to create 1 rle with old method : 0.061458587646484375 time for calcul the mask position with numpy : 0.03303694725036621 nb_pixel_total : 13714 time to create 1 rle with old method : 0.01510930061340332 time for calcul the mask position with numpy : 0.033597707748413086 nb_pixel_total : 48923 time to create 1 rle with old method : 0.06000804901123047 time for calcul the mask position with numpy : 0.03322887420654297 nb_pixel_total : 8938 time to create 1 rle with old method : 0.01006317138671875 time for calcul the mask position with numpy : 0.03246712684631348 nb_pixel_total : 23369 time to create 1 rle with old method : 0.02448582649230957 time for calcul the mask position with numpy : 0.03239178657531738 nb_pixel_total : 26919 time to create 1 rle with old method : 0.027822494506835938 time for calcul the mask position with numpy : 0.033319711685180664 nb_pixel_total : 28736 time to create 1 rle with old method : 0.031610965728759766 time for calcul the mask position with numpy : 0.03516697883605957 nb_pixel_total : 136379 time to create 1 rle with old method : 0.1788332462310791 time for calcul the mask position with numpy : 0.03344082832336426 nb_pixel_total : 4055 time to create 1 rle with old method : 0.00477910041809082 time for calcul the mask position with numpy : 0.03391265869140625 nb_pixel_total : 39560 time to create 1 rle with old method : 0.04320025444030762 time for calcul the mask position with numpy : 0.03462076187133789 nb_pixel_total : 67466 time to create 1 rle with old method : 0.07528138160705566 time for calcul the mask position with numpy : 0.03412294387817383 nb_pixel_total : 55942 time to create 1 rle with old method : 0.059921979904174805 time for calcul the mask position with numpy : 0.03589463233947754 nb_pixel_total : 56388 time to create 1 rle with old method : 0.06255364418029785 time for calcul the mask position with numpy : 0.03362321853637695 nb_pixel_total : 14498 time to create 1 rle with old method : 0.015845775604248047 time for calcul the mask position with numpy : 0.03341817855834961 nb_pixel_total : 29783 time to create 1 rle with old method : 0.0322265625 time for calcul the mask position with numpy : 0.03448629379272461 nb_pixel_total : 16085 time to create 1 rle with old method : 0.01808953285217285 time for calcul the mask position with numpy : 0.03561258316040039 nb_pixel_total : 47201 time to create 1 rle with old method : 0.05120658874511719 time for calcul the mask position with numpy : 0.0334014892578125 nb_pixel_total : 27655 time to create 1 rle with old method : 0.03099203109741211 time for calcul the mask position with numpy : 0.03352856636047363 nb_pixel_total : 41163 time to create 1 rle with old method : 0.04497981071472168 time for calcul the mask position with numpy : 0.03412604331970215 nb_pixel_total : 130109 time to create 1 rle with old method : 0.14196085929870605 time for calcul the mask position with numpy : 0.03278088569641113 nb_pixel_total : 21550 time to create 1 rle with old method : 0.023473024368286133 time for calcul the mask position with numpy : 0.034119606018066406 nb_pixel_total : 121013 time to create 1 rle with old method : 0.13231301307678223 time for calcul the mask position with numpy : 0.03359532356262207 nb_pixel_total : 38360 time to create 1 rle with old method : 0.04042696952819824 time for calcul the mask position with numpy : 0.03289341926574707 nb_pixel_total : 73122 time to create 1 rle with old method : 0.0758047103881836 time for calcul the mask position with numpy : 0.03251791000366211 nb_pixel_total : 25972 time to create 1 rle with old method : 0.02817535400390625 create new chi : 4.642413377761841 time to delete rle : 0.0017778873443603516 batch 1 Loaded 27 chid ids of type : 4211 Number RLEs to save : 17287 TO DO : save crop sub photo not yet done ! save time : 1.0754904747009277 map_output_result : {1065568816: (0.0, 'Should be the crop_list due to order', 0.0), 1065568708: (0.0, 'Should be the crop_list due to order', 0.0), 1065568705: (0.0, 'Should be the crop_list due to order', 0.0), 1065568698: (0.0, 'Should be the crop_list due to order', 0.0), 1065568694: (0.0, 'Should be the crop_list due to order', 0.0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1065568816, 1065568708, 1065568705, 1065568698, 1065568694] Looping around the photos to save general results len do output : 5 /1065568816.Didn't retrieve data . /1065568708.Didn't retrieve data . /1065568705.Didn't retrieve data . /1065568698.Didn't retrieve data . /1065568694.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3995', None, None, None, None, None, None, None, '2596639') ('3995', '5486001', '1065568816', None, None, None, None, None, '2596639') ('3995', None, None, None, None, None, None, None, '2596639') ('3995', '5486001', '1065568708', None, None, None, None, None, '2596639') ('3995', None, None, None, None, None, None, None, '2596639') ('3995', '5486001', '1065568705', None, None, None, None, None, '2596639') ('3995', None, None, None, None, None, None, None, '2596639') ('3995', '5486001', '1065568698', None, None, None, None, None, '2596639') ('3995', None, None, None, None, None, None, None, '2596639') ('3995', '5486001', '1065568694', None, None, None, None, None, '2596639') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 15 time used for this insertion : 0.012108325958251953 save_final save missing photos in datou_result : time spend for datou_step_exec : 113.96301031112671 time spend to save output : 0.012591838836669922 total time spend for step 7 : 113.97560214996338 step8:crop_condition Tue Feb 18 15:11:27 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure some photos are not treated, begin crop_condition Loading chi in step crop with photo_hashtag_type : 4211 Loading chi in step crop for list_pids : 5 ! batch 1 Loaded 131 chid ids of type : 4211 begin to crop the class : barquette_opaque param for this class : {'min_score': 0.5} filtre for class : barquette_opaque hashtag_id of this class : 2107760128 Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 5 About to insert : list_path_to_insert length 5 new photo from crops ! About to upload 5 photos upload in portfolio : 4869462 init cache_photo without model_param we have 5 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739887889_4069223 we have uploaded 5 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.8726885318756104 we have finished the crop for the class : barquette_opaque begin to crop the class : carton param for this class : {'min_score': 0.5} filtre for class : carton hashtag_id of this class : 492774966 Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 4 About to insert : list_path_to_insert length 4 new photo from crops ! About to upload 4 photos upload in portfolio : 4869462 init cache_photo without model_param we have 4 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739887893_4069223 we have uploaded 4 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.8171722888946533 we have finished the crop for the class : carton begin to crop the class : ela param for this class : {'min_score': 0.5} filtre for class : ela hashtag_id of this class : 492741797 Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 5 About to insert : list_path_to_insert length 5 new photo from crops ! About to upload 5 photos upload in portfolio : 4869462 init cache_photo without model_param we have 5 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739887897_4069223 we have uploaded 5 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.8880712985992432 we have finished the crop for the class : ela begin to crop the class : environnement param for this class : {'min_score': 0.5} filtre for class : environnement hashtag_id of this class : 493012381 Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 4869462 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739887900_4069223 we have uploaded 1 photos in the portfolio 4869462 time of upload the photos Elapsed time : 0.6628360748291016 we have finished the crop for the class : environnement begin to crop the class : etiquette param for this class : {'min_score': 0.5} filtre for class : etiquette hashtag_id of this class : 492636447 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 21 About to insert : list_path_to_insert length 21 new photo from crops ! About to upload 21 photos upload in portfolio : 4869462 init cache_photo without model_param we have 21 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739887904_4069223 we have uploaded 21 photos in the portfolio 4869462 time of upload the photos Elapsed time : 5.3132030963897705 we have finished the crop for the class : etiquette begin to crop the class : film_plastique param for this class : {'min_score': 0.5} filtre for class : film_plastique hashtag_id of this class : 2107756122 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 10 About to insert : list_path_to_insert length 10 new photo from crops ! About to upload 10 photos upload in portfolio : 4869462 init cache_photo without model_param we have 10 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739887914_4069223 we have uploaded 10 photos in the portfolio 4869462 time of upload the photos Elapsed time : 2.5723702907562256 we have finished the crop for the class : film_plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 begin to crop the class : pehd param for this class : {'min_score': 0.5} filtre for class : pehd hashtag_id of this class : 628944319 Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 4869462 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739887917_4069223 we have uploaded 1 photos in the portfolio 4869462 time of upload the photos Elapsed time : 0.6972002983093262 we have finished the crop for the class : pehd begin to crop the class : pet_clair param for this class : {'min_score': 0.5} filtre for class : pet_clair hashtag_id of this class : 2107755846 Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 4869462 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739887920_4069223 we have uploaded 1 photos in the portfolio 4869462 time of upload the photos Elapsed time : 0.5776574611663818 we have finished the crop for the class : pet_clair begin to crop the class : pet_opaque param for this class : {'min_score': 0.5} filtre for class : pet_opaque hashtag_id of this class : 2107759152 Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! About to upload 3 photos upload in portfolio : 4869462 init cache_photo without model_param we have 3 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739887922_4069223 we have uploaded 3 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.2017242908477783 we have finished the crop for the class : pet_opaque begin to crop the class : textiles_sanitaires param for this class : {'min_score': 0.5} filtre for class : textiles_sanitaires hashtag_id of this class : 2107760129 begin to crop the class : pet_fonce param for this class : {'min_score': 0.5} filtre for class : pet_fonce hashtag_id of this class : 2107755900 begin to crop the class : papier param for this class : {'min_score': 0.5} filtre for class : papier hashtag_id of this class : 492668766 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 39 About to insert : list_path_to_insert length 39 new photo from crops ! About to upload 39 photos upload in portfolio : 4869462 init cache_photo without model_param we have 39 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739887932_4069223 we have uploaded 39 photos in the portfolio 4869462 time of upload the photos Elapsed time : 10.07307481765747 we have finished the crop for the class : papier delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1065568816, 1065568708, 1065568705, 1065568698, 1065568694] Looping around the photos to save general results len do output : 90 /1338347752Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347753Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347754Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347755Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347756Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347758Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347759Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347760Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347761Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347765Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347766Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347767Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347768Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347769Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347771Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347774Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347775Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347776Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347777Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347778Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347779Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347780Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347781Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347782Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347783Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347784Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347785Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347786Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347787Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347788Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347789Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347790Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347791Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347792Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347793Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347794Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347798Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347800Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347801Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347802Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347803Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347804Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347805Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347806Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347807Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347808Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347810Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347815Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347816Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347817Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347818Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347826Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347827Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347828Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347829Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347830Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347831Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347832Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347833Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347834Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347835Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347836Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347837Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347838Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347839Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347840Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347841Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347842Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347843Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347844Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347845Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347846Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347847Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347848Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347849Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347850Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347852Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347853Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347854Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347855Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347856Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347857Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347858Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347859Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347860Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347861Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347862Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347863Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347864Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338347865Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3995', None, None, None, None, None, None, None, '2596639') ('3995', '5486001', '1065568816', None, None, None, None, None, '2596639') ('3995', None, None, None, None, None, None, None, '2596639') ('3995', '5486001', '1065568708', None, None, None, None, None, '2596639') ('3995', None, None, None, None, None, None, None, '2596639') ('3995', '5486001', '1065568705', None, None, None, None, None, '2596639') ('3995', None, None, None, None, None, None, None, '2596639') ('3995', '5486001', '1065568698', None, None, None, None, None, '2596639') ('3995', None, None, None, None, None, None, None, '2596639') ('3995', '5486001', '1065568694', None, None, None, None, None, '2596639') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 275 time used for this insertion : 0.036208152770996094 save_final save missing photos in datou_result : time spend for datou_step_exec : 55.00908827781677 time spend to save output : 0.03892064094543457 total time spend for step 8 : 55.04800891876221 step9:ventilate_hashtags_in_portfolio Tue Feb 18 15:12:22 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 5486001 get user id for portfolio 5486001 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=5486001 AND mptpi.`type`=4211 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pehd','ela','flou','metal','kraft','papier','film_plastique','mal_croppe','pet_clair','pet_opaque','environnement','textiles_sanitaires','barquette_opaque','pet_fonce','autre','carton','etiquette')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=5486001 AND mptpi.`type`=4211 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pehd','ela','flou','metal','kraft','papier','film_plastique','mal_croppe','pet_clair','pet_opaque','environnement','textiles_sanitaires','barquette_opaque','pet_fonce','autre','carton','etiquette')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=5486001 AND mptpi.`type`=4211 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pehd','ela','flou','metal','kraft','papier','film_plastique','mal_croppe','pet_clair','pet_opaque','environnement','textiles_sanitaires','barquette_opaque','pet_fonce','autre','carton','etiquette')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/20640186,20640187,20640188,20640189,20640190,20640191,20640192,20640193,20640194,20640195,20640196,20640197,20640198,20640199,20640200,20640201,20640202?tags=pehd,ela,flou,metal,kraft,papier,film_plastique,mal_croppe,pet_clair,pet_opaque,environnement,textiles_sanitaires,barquette_opaque,pet_fonce,autre,carton,etiquette Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1065568816, 1065568708, 1065568705, 1065568698, 1065568694] Looping around the photos to save general results len do output : 1 /5486001. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3995', None, None, None, None, None, None, None, '2596639') ('3995', '5486001', '1065568816', None, None, None, None, None, '2596639') ('3995', None, None, None, None, None, None, None, '2596639') ('3995', '5486001', '1065568708', None, None, None, None, None, '2596639') ('3995', None, None, None, None, None, None, None, '2596639') ('3995', '5486001', '1065568705', None, None, None, None, None, '2596639') ('3995', None, None, None, None, None, None, None, '2596639') ('3995', '5486001', '1065568698', None, None, None, None, None, '2596639') ('3995', None, None, None, None, None, None, None, '2596639') ('3995', '5486001', '1065568694', None, None, None, None, None, '2596639') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 6 time used for this insertion : 0.014511346817016602 save_final save missing photos in datou_result : time spend for datou_step_exec : 2.3281707763671875 time spend to save output : 0.014704704284667969 total time spend for step 9 : 2.3428754806518555 step10:final Tue Feb 18 15:12:24 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1065568816: ('0.04017411948578654',), 1065568708: ('0.04017411948578654',), 1065568705: ('0.04017411948578654',), 1065568698: ('0.04017411948578654',), 1065568694: ('0.04017411948578654',)} new output for save of step final : {1065568816: ('0.04017411948578654',), 1065568708: ('0.04017411948578654',), 1065568705: ('0.04017411948578654',), 1065568698: ('0.04017411948578654',), 1065568694: ('0.04017411948578654',)} [1065568816, 1065568708, 1065568705, 1065568698, 1065568694] Looping around the photos to save general results len do output : 5 /1065568816.Didn't retrieve data . /1065568708.Didn't retrieve data . /1065568705.Didn't retrieve data . /1065568698.Didn't retrieve data . /1065568694.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3995', None, None, None, None, None, None, None, '2596639') ('3995', '5486001', '1065568816', None, None, None, None, None, '2596639') ('3995', None, None, None, None, None, None, None, '2596639') ('3995', '5486001', '1065568708', None, None, None, None, None, '2596639') ('3995', None, None, None, None, None, None, None, '2596639') ('3995', '5486001', '1065568705', None, None, None, None, None, '2596639') ('3995', None, None, None, None, None, None, None, '2596639') ('3995', '5486001', '1065568698', None, None, None, None, None, '2596639') ('3995', None, None, None, None, None, None, None, '2596639') ('3995', '5486001', '1065568694', None, None, None, None, None, '2596639') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 15 time used for this insertion : 0.012901067733764648 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.10406613349914551 time spend to save output : 0.01334691047668457 total time spend for step 10 : 0.11741304397583008 step11:velours_tree Tue Feb 18 15:12:25 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.1369180679321289 time spend to save output : 8.130073547363281e-05 total time spend for step 11 : 0.13699936866760254 step12:send_mail_cod Tue Feb 18 15:12:25 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin in order to get the selector url, please entre the license of selector results_Auto_P5486001_18-02-2025_15_12_25.pdf 20640187 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette206401871739887945 20640188 imagette206401881739887945 20640189 imagette206401891739887945 20640190 imagette206401901739887945 20640191 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette206401911739887945 20640192 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette206401921739887946 20640193 imagette206401931739887947 20640194 change filename to text .imagette206401941739887947 20640195 change filename to text .change filename to text .change filename to text .imagette206401951739887947 20640197 imagette206401971739887947 20640198 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette206401981739887947 20640199 imagette206401991739887948 20640200 imagette206402001739887948 20640201 change filename to text .change filename to text .change filename to text .change filename to text .imagette206402011739887948 20640202 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette206402021739887948 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=5486001 and hashtag_type = 4211 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/20640186,20640187,20640188,20640189,20640190,20640191,20640192,20640193,20640194,20640195,20640196,20640197,20640198,20640199,20640200,20640201,20640202?tags=pehd,ela,flou,metal,kraft,papier,film_plastique,mal_croppe,pet_clair,pet_opaque,environnement,textiles_sanitaires,barquette_opaque,pet_fonce,autre,carton,etiquette your option no_mail is active, we will not send the real mail to your client args[1065568816] : ((1065568816, -0.8402069540962303, 492688767), (1065568816, -0.5797699732161256, 501862349), '0.04017411948578654') We are sending mail with results at report@fotonower.com args[1065568708] : ((1065568708, -4.070521924534813, 492609224), (1065568708, -0.15785097409432802, 496442774), '0.04017411948578654') We are sending mail with results at report@fotonower.com args[1065568705] : ((1065568705, -2.9011072176121386, 492609224), (1065568705, -0.06353982647497829, 2107752395), '0.04017411948578654') We are sending mail with results at report@fotonower.com args[1065568698] : ((1065568698, -1.706155447051669, 492688767), (1065568698, 0.008376777865801007, 2107752395), '0.04017411948578654') We are sending mail with results at report@fotonower.com args[1065568694] : ((1065568694, -2.374330514925937, 492609224), (1065568694, -0.31270551135448865, 496442774), '0.04017411948578654') We are sending mail with results at report@fotonower.com refus_total : 0.04017411948578654 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=5486001 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P5486001_18-02-2025_15_12_25.pdf results_Auto_P5486001_18-02-2025_15_12_25.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P5486001_18-02-2025_15_12_25.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3995','5486001','results_Auto_P5486001_18-02-2025_15_12_25.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P5486001_18-02-2025_15_12_25.pdf','pdf','','0.6','0.04017411948578654') Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1065568816, 1065568708, 1065568705, 1065568698, 1065568694] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3995', None, None, None, None, None, None, None, '2596639') ('3995', '5486001', '1065568816', None, None, None, None, None, '2596639') ('3995', None, None, None, None, None, None, None, '2596639') ('3995', '5486001', '1065568708', None, None, None, None, None, '2596639') ('3995', None, None, None, None, None, None, None, '2596639') ('3995', '5486001', '1065568705', None, None, None, None, None, '2596639') ('3995', None, None, None, None, None, None, None, '2596639') ('3995', '5486001', '1065568698', None, None, None, None, None, '2596639') ('3995', None, None, None, None, None, None, None, '2596639') ('3995', '5486001', '1065568694', None, None, None, None, None, '2596639') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 5 time used for this insertion : 0.012252569198608398 save_final save missing photos in datou_result : time spend for datou_step_exec : 5.9544360637664795 time spend to save output : 0.01247715950012207 total time spend for step 12 : 5.966913223266602 step13:split_time_score Tue Feb 18 15:12:31 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('19', 5),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 25112021 5486001 Nombre de photos uploadées : 5 / 23040 (0%) 25112021 5486001 Nombre de photos taguées (types de déchets): 0 / 5 (0%) 25112021 5486001 Nombre de photos taguées (volume) : 0 / 5 (0%) elapsed_time : load_data_split_time_score 1.9073486328125e-06 elapsed_time : order_list_meta_photo_and_scores 5.4836273193359375e-06 ????? elapsed_time : fill_and_build_computed_from_old_data 0.00035190582275390625 elapsed_time : insert_dashboard_record_day_entry 0.02319478988647461 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P5181046_02-02-2022_12_15_38.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 5181046 order by id desc limit 1 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P5181067_01-02-2022_17_14_55.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 5181067 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 5485994 order by id desc limit 1 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P5485999_19-02-2022_20_45_15.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 5485999 order by id desc limit 1 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P5181107_01-02-2022_18_24_31.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 5181107 order by id desc limit 1 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P5181110_01-02-2022_04_17_35.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 5181110 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 5486000 order by id desc limit 1 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P5181144_01-02-2022_05_00_33.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 5181144 order by id desc limit 1 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P5181167_31-01-2022_21_17_04.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 5181167 order by id desc limit 1 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P5486004_19-02-2022_08_14_08.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 5486004 order by id desc limit 1 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P5486005_19-02-2022_19_34_46.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 5486005 order by id desc limit 1 NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'25112021': {'nb_upload': 5, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1065568816, 1065568708, 1065568705, 1065568698, 1065568694] Looping around the photos to save general results len do output : 1 /5486001Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3995', None, None, None, None, None, None, None, '2596639') ('3995', '5486001', '1065568816', None, None, None, None, None, '2596639') ('3995', None, None, None, None, None, None, None, '2596639') ('3995', '5486001', '1065568708', None, None, None, None, None, '2596639') ('3995', None, None, None, None, None, None, None, '2596639') ('3995', '5486001', '1065568705', None, None, None, None, None, '2596639') ('3995', None, None, None, None, None, None, None, '2596639') ('3995', '5486001', '1065568698', None, None, None, None, None, '2596639') ('3995', None, None, None, None, None, None, None, '2596639') ('3995', '5486001', '1065568694', None, None, None, None, None, '2596639') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 6 time used for this insertion : 0.013760566711425781 save_final save missing photos in datou_result : time spend for datou_step_exec : 8.269029140472412 time spend to save output : 0.013951778411865234 total time spend for step 13 : 8.282980918884277 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 5 set_done_treatment 178.61user 120.24system 6:01.96elapsed 82%CPU (0avgtext+0avgdata 6664988maxresident)k 751168inputs+110536outputs (879major+19305957minor)pagefaults 0swaps