python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 20 -a ' -a 4189' -s datou_current_4189 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 2498080 load datou : 4189 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? load thcls load pdts Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 4189, datou_cur_ids : ['2719729'] with mtr_portfolio_ids : ['21836103'] and first list_photo_ids : [] new path : /proc/2498080/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! List Step Type Loaded in datou : split_time_score over limit max, limiting to limit_max 100 list_input_json : [] origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! we have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB time to download the photos : 0.24993658065795898 About to test input to load Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 1 step1:split_time_score Thu Apr 3 09:05:30 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec begin split time score 2022-04-13 10:29:59 0 TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 3379, 'mtr_user_id': 31, 'name': 'learn_classif_flux_maj_generique_effnet_v2_s_02062022', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'aluminium,ela,film_pedb,flux_dev,jrm,pcm,pcnc,pehd_pp,pet_clair,refus,tapis_vide', 'svm_portfolios_learning': '5515864,5515840,5515844,5515850,6244400,6237996,6237998,5515847,5515841,5515868,5515866', 'photo_hashtag_type': 4374, 'photo_desc_type': 5680, 'type_classification': 'tf_classification2', 'hashtag_id_list': '493546845,492741797,2107760237,2107760238,495916461,560181804,1284539308,2107760239,2107755846,538914404,2107748999'}] thcls : [{'id': 3513, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2_tf', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 4557, 'photo_desc_type': 5767, 'type_classification': 'tf_classification2', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('10', 366), ('14', 363)) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {1: 729} 28032025 21836103 Nombre de photos uploadées : 729 / 23040 (3%) 28032025 21836103 Nombre de photos taguées (types de déchets): 729 / 729 (100%) 28032025 21836103 Nombre de photos taguées (volume) : 729 / 729 (100%) elapsed_time : load_data_split_time_score 3.5762786865234375e-06 elapsed_time : order_list_meta_photo_and_scores 0.0003185272216796875 LLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLL elapsed_time : fill_and_build_computed_from_old_data 0.06757307052612305 elapsed_time : insert_dashboard_record_day_entry 0.026795148849487305 LLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLL Creating list_photo_by_hashtags elapsed_time : list_photo_by_hashtags 0.07407617568969727 Creating list_photo_total elapsed_time : select_descriptors 30.719800233840942 28032025 21836103 Nombre de photos avec descriptors (type 5680) : 313 / 313 (100%) ERROR : Hum hum, what can we do for different size of descriptors (ignore the difference ) : 0 vs 1280 photo_id : 1348220164 photo_id_prec : 0 0:00:00|ON:LMissing descriptors for photos 1348222402 and 1348222426 LLMissing descriptors for photos 1348223451 and 1348223464 LMissing descriptors for photos 1348223464 and 1348223508 LLLMissing descriptors for photos 1348225598 and 1348225610 LLMissing descriptors for photos 1348227531 and 1348227565 LMissing descriptors for photos 1348227565 and 1348227605 LMissing descriptors for photos 1348227605 and 1348227672 LMissing descriptors for photos 1348227672 and 1348227759 LLMissing descriptors for photos 1348228743 and 1348228754 LMissing descriptors for photos 1348228754 and 1348228775 LMissing descriptors for photos 1348228775 and 1348228779 LMissing descriptors for photos 1348228779 and 1348228786 LMissing descriptors for photos 1348228786 and 1348228802 LMissing descriptors for photos 1348228802 and 1348229773 LLMissing descriptors for photos 1348229780 and 1348229791 LMissing descriptors for photos 1348229791 and 1348230702 LLMissing descriptors for photos 1348230773 and 1348230813 LMissing descriptors for photos 1348230813 and 1348230834 LMissing descriptors for photos 1348230834 and 1348230872 LLMissing descriptors for photos 1348233466 and 1348233482 LMissing descriptors for photos 1348233482 and 1348233575 LMissing descriptors for photos 1348233575 and 1348233580 LMissing descriptors for photos 1348233580 and 1348233623 LMissing descriptors for photos 1348233623 and 1348234811 LLMissing descriptors for photos 1348234820 and 1348234828 LMissing descriptors for photos 1348234828 and 1348234858 LMissing descriptors for photos 1348234858 and 1348234865 LMissing descriptors for photos 1348234865 and 1348235776 LMissing descriptors for photos 1348235776 and 1348235779 LMissing descriptors for photos 1348235779 and 1348235797 LMissing descriptors for photos 1348235797 and 1348235814 LMissing descriptors for photos 1348235814 and 1348235830 LMissing descriptors for photos 1348235830 and 1348235861 LMissing descriptors for photos 1348235861 and 1348236528 LMissing descriptors for photos 1348236528 and 1348236531 LMissing descriptors for photos 1348236531 and 1348236536 LMissing descriptors for photos 1348236536 and 1348236549 LMissing descriptors for photos 1348236549 and 1348236604 LMissing descriptors for photos 1348236604 and 1348236650 LMissing descriptors for photos 1348236650 and 1348237990 LLL0:16:00.463622|OFF:0:00:59.867234|ON:LL0:01:00.340046|OFF:LL0:01:00.476114|ON:LL0:00:00|OFF:0:00:00|ON:LL0:03:02.233039|OFF:0:00:00|ON:LL0:08:57.207073|OFF:LLLLLLLLLL0:01:01.884774|ON:LL0:02:58.745555|OFF:LLLL0:00:58.926850|ON:LL0:03:59.881032|OFF:0:00:00|ON:LL0:01:00.917680|OFF:0:00:00|ON:LLMissing descriptors for photos 1348305497 and 1348305504 LLL0:08:59.503161|OFF:0:00:00|ON:LL0:00:59.918326|OFF:LL0:00:00|ON:LLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLL0:07:59.640482|OFF:L0:00:00|ON:LLLLMissing descriptors for photos 1349293945 and 1349293950 LMissing descriptors for photos 1349293950 and 1349293968 LMissing descriptors for photos 1349293968 and 1349293972 LMissing descriptors for photos 1349293972 and 1349293980 LMissing descriptors for photos 1349293980 and 1349293984 LMissing descriptors for photos 1349293984 and 1349293988 LMissing descriptors for photos 1349293988 and 1349293992 LMissing descriptors for photos 1349293992 and 1349294011 LMissing descriptors for photos 1349294011 and 1349294015 LMissing descriptors for photos 1349294015 and 1349294020 LMissing descriptors for photos 1349294020 and 1349294023 LMissing descriptors for photos 1349294023 and 1349294026 LMissing descriptors for photos 1349294026 and 1349294032 LMissing descriptors for photos 1349294032 and 1349294051 LMissing descriptors for photos 1349294051 and 1349294056 LMissing descriptors for photos 1349294056 and 1349294060 LMissing descriptors for photos 1349294060 and 1349294062 LMissing descriptors for photos 1349294062 and 1349294066 LMissing descriptors for photos 1349294066 and 1349294072 LMissing descriptors for photos 1349294072 and 1349294099 LMissing descriptors for photos 1349294099 and 1349294108 LMissing descriptors for photos 1349294108 and 1349294111 LMissing descriptors for photos 1349294111 and 1349294114 LMissing descriptors for photos 1349294114 and 1349294120 LMissing descriptors for photos 1349294120 and 1349294129 LMissing descriptors for photos 1349294129 and 1349294225 LMissing descriptors for photos 1349294225 and 1349294232 LMissing descriptors for photos 1349294232 and 1349294236 LMissing descriptors for photos 1349294236 and 1349294240 LMissing descriptors for photos 1349294240 and 1349294244 LMissing descriptors for photos 1349294244 and 1349294249 LMissing descriptors for photos 1349294249 and 1349294280 LMissing descriptors for photos 1349294280 and 1349294289 LMissing descriptors for photos 1349294289 and 1349294292 LMissing descriptors for photos 1349294292 and 1349294310 LMissing descriptors for photos 1349294310 and 1349294315 LMissing descriptors for photos 1349294315 and 1349294324 LMissing descriptors for photos 1349294324 and 1349294343 LMissing descriptors for photos 1349294343 and 1349294354 LMissing descriptors for photos 1349294354 and 1349294420 LMissing descriptors for photos 1349294420 and 1349294485 LMissing descriptors for photos 1349294485 and 1349294570 LMissing descriptors for photos 1349294570 and 1349294588 LMissing descriptors for photos 1349294588 and 1349294633 LMissing descriptors for photos 1349294633 and 1349294639 LMissing descriptors for photos 1349294639 and 1349294645 LMissing descriptors for photos 1349294645 and 1349294650 LMissing descriptors for photos 1349294650 and 1349294654 LMissing descriptors for photos 1349294654 and 1349294667 LMissing descriptors for photos 1349294667 and 1349294718 LMissing descriptors for photos 1349294718 and 1349294726 LMissing descriptors for photos 1349294726 and 1349294732 LMissing descriptors for photos 1349294732 and 1349294737 LMissing descriptors for photos 1349294737 and 1349294743 LMissing descriptors for photos 1349294743 and 1349294748 LMissing descriptors for photos 1349294748 and 1349294758 LMissing descriptors for photos 1349294758 and 1349294763 LMissing descriptors for photos 1349294763 and 1349294770 LMissing descriptors for photos 1349294770 and 1349294778 LMissing descriptors for photos 1349294778 and 1349294785 LMissing descriptors for photos 1349294785 and 1349294789 LMissing descriptors for photos 1349294789 and 1349294822 LMissing descriptors for photos 1349294822 and 1349294849 LMissing descriptors for photos 1349294849 and 1349294856 LMissing descriptors for photos 1349294856 and 1349294862 LMissing descriptors for photos 1349294862 and 1349294869 LMissing descriptors for photos 1349294869 and 1349294872 LMissing descriptors for photos 1349294872 and 1349294889 LMissing descriptors for photos 1349294889 and 1349294896 LMissing descriptors for photos 1349294896 and 1349294901 LMissing descriptors for photos 1349294901 and 1349294907 LMissing descriptors for photos 1349294907 and 1349294914 LMissing descriptors for photos 1349294914 and 1349294918 LMissing descriptors for photos 1349294918 and 1349294944 LMissing descriptors for photos 1349294944 and 1349294949 LMissing descriptors for photos 1349294949 and 1349294958 LMissing descriptors for photos 1349294958 and 1349294973 LMissing descriptors for photos 1349294973 and 1349294976 LMissing descriptors for photos 1349294976 and 1349294979 LMissing descriptors for photos 1349294979 and 1349294996 LMissing descriptors for photos 1349294996 and 1349294998 LMissing descriptors for photos 1349294998 and 1349295003 LMissing descriptors for photos 1349295003 and 1349295008 LMissing descriptors for photos 1349295008 and 1349295012 LMissing descriptors for photos 1349295012 and 1349295020 LMissing descriptors for photos 1349295020 and 1349295098 LMissing descriptors for photos 1349295098 and 1349295105 LMissing descriptors for photos 1349295105 and 1349295111 LMissing descriptors for photos 1349295111 and 1349295121 LMissing descriptors for photos 1349295121 and 1349295125 LMissing descriptors for photos 1349295125 and 1349295131 LMissing descriptors for photos 1349295131 and 1349295148 LMissing descriptors for photos 1349295148 and 1349295152 LMissing descriptors for photos 1349295152 and 1349295156 LMissing descriptors for photos 1349295156 and 1349295161 LMissing descriptors for photos 1349295161 and 1349295165 LMissing descriptors for photos 1349295165 and 1349295169 LMissing descriptors for photos 1349295169 and 1349295266 LMissing descriptors for photos 1349295266 and 1349295305 LMissing descriptors for photos 1349295305 and 1349295342 LMissing descriptors for photos 1349295342 and 1349295385 LMissing descriptors for photos 1349295385 and 1349295432 LMissing descriptors for photos 1349295432 and 1349295473 LMissing descriptors for photos 1349295473 and 1349295611 LMissing descriptors for photos 1349295611 and 1349295620 LMissing descriptors for photos 1349295620 and 1349295626 LMissing descriptors for photos 1349295626 and 1349295660 LMissing descriptors for photos 1349295660 and 1349295673 LMissing descriptors for photos 1349295673 and 1349295709 LMissing descriptors for photos 1349295709 and 1349295743 LMissing descriptors for photos 1349295743 and 1349295751 LMissing descriptors for photos 1349295751 and 1349295755 LMissing descriptors for photos 1349295755 and 1349295760 LMissing descriptors for photos 1349295760 and 1349295762 LMissing descriptors for photos 1349295762 and 1349295767 LMissing descriptors for photos 1349295767 and 1349295794 LMissing descriptors for photos 1349295794 and 1349295805 LMissing descriptors for photos 1349295805 and 1349295809 LMissing descriptors for photos 1349295809 and 1349295812 LMissing descriptors for photos 1349295812 and 1349295817 LMissing descriptors for photos 1349295817 and 1349295845 LMissing descriptors for photos 1349295845 and 1349295952 LMissing descriptors for photos 1349295952 and 1349295960 LMissing descriptors for photos 1349295960 and 1349295999 LMissing descriptors for photos 1349295999 and 1349296004 LMissing descriptors for photos 1349296004 and 1349296010 LMissing descriptors for photos 1349296010 and 1349296015 LMissing descriptors for photos 1349296015 and 1349296041 LMissing descriptors for photos 1349296041 and 1349296046 LMissing descriptors for photos 1349296046 and 1349296048 LMissing descriptors for photos 1349296048 and 1349296055 LMissing descriptors for photos 1349296055 and 1349296062 LMissing descriptors for photos 1349296062 and 1349296069 LMissing descriptors for photos 1349296069 and 1349296090 LMissing descriptors for photos 1349296090 and 1349296095 LMissing descriptors for photos 1349296095 and 1349296098 LMissing descriptors for photos 1349296098 and 1349296107 LMissing descriptors for photos 1349296107 and 1349296115 LMissing descriptors for photos 1349296115 and 1349296121 LMissing descriptors for photos 1349296121 and 1349296139 LMissing descriptors for photos 1349296139 and 1349296141 LMissing descriptors for photos 1349296141 and 1349296145 LMissing descriptors for photos 1349296145 and 1349296149 LMissing descriptors for photos 1349296149 and 1349296154 LMissing descriptors for photos 1349296154 and 1349296164 LMissing descriptors for photos 1349296164 and 1349296179 LMissing descriptors for photos 1349296179 and 1349296181 LMissing descriptors for photos 1349296181 and 1349296183 LMissing descriptors for photos 1349296183 and 1349296185 LMissing descriptors for photos 1349296185 and 1349296193 LMissing descriptors for photos 1349296193 and 1349296199 LMissing descriptors for photos 1349296199 and 1349296224 LMissing descriptors for photos 1349296224 and 1349296226 LMissing descriptors for photos 1349296226 and 1349296235 LMissing descriptors for photos 1349296235 and 1349296237 LMissing descriptors for photos 1349296237 and 1349296241 LMissing descriptors for photos 1349296241 and 1349296254 LMissing descriptors for photos 1349296254 and 1349296281 LMissing descriptors for photos 1349296281 and 1349296284 LMissing descriptors for photos 1349296284 and 1349296288 LMissing descriptors for photos 1349296288 and 1349296292 LMissing descriptors for photos 1349296292 and 1349296298 LMissing descriptors for photos 1349296298 and 1349296301 LMissing descriptors for photos 1349296301 and 1349296320 LMissing descriptors for photos 1349296320 and 1349296324 LMissing descriptors for photos 1349296324 and 1349296330 LMissing descriptors for photos 1349296330 and 1349296338 LMissing descriptors for photos 1349296338 and 1349296347 LMissing descriptors for photos 1349296347 and 1349296355 LMissing descriptors for photos 1349296355 and 1349296375 LMissing descriptors for photos 1349296375 and 1349296382 LMissing descriptors for photos 1349296382 and 1349296390 LMissing descriptors for photos 1349296390 and 1349296395 LMissing descriptors for photos 1349296395 and 1349296402 LMissing descriptors for photos 1349296402 and 1349296406 LMissing descriptors for photos 1349296406 and 1349296483 LMissing descriptors for photos 1349296483 and 1349296520 LMissing descriptors for photos 1349296520 and 1349296559 LMissing descriptors for photos 1349296559 and 1349296601 LMissing descriptors for photos 1349296601 and 1349296652 LMissing descriptors for photos 1349296652 and 1349296692 LMissing descriptors for photos 1349296692 and 1349297437 LMissing descriptors for photos 1349297437 and 1349297577 LMissing descriptors for photos 1349297577 and 1349297655 LMissing descriptors for photos 1349297655 and 1349297737 LMissing descriptors for photos 1349297737 and 1349297799 LMissing descriptors for photos 1349297799 and 1349297871 LMissing descriptors for photos 1349297871 and 1349298165 LMissing descriptors for photos 1349298165 and 1349298302 LMissing descriptors for photos 1349298302 and 1349298396 LMissing descriptors for photos 1349298396 and 1349298455 LMissing descriptors for photos 1349298455 and 1349298496 LMissing descriptors for photos 1349298496 and 1349298503 LMissing descriptors for photos 1349298503 and 1349298649 LMissing descriptors for photos 1349298649 and 1349298672 LMissing descriptors for photos 1349298672 and 1349298713 LMissing descriptors for photos 1349298713 and 1349298799 LMissing descriptors for photos 1349298799 and 1349298860 LMissing descriptors for photos 1349298860 and 1349298937 LMissing descriptors for photos 1349298937 and 1349299114 LMissing descriptors for photos 1349299114 and 1349299180 LMissing descriptors for photos 1349299180 and 1349299215 LMissing descriptors for photos 1349299215 and 1349299250 LMissing descriptors for photos 1349299250 and 1349299253 LMissing descriptors for photos 1349299253 and 1349299277 LMissing descriptors for photos 1349299277 and 1349299515 LMissing descriptors for photos 1349299515 and 1349299548 LMissing descriptors for photos 1349299548 and 1349299623 LMissing descriptors for photos 1349299623 and 1349299643 LMissing descriptors for photos 1349299643 and 1349299656 LMissing descriptors for photos 1349299656 and 1349299662 LMissing descriptors for photos 1349299662 and 1349299728 LMissing descriptors for photos 1349299728 and 1349299736 LMissing descriptors for photos 1349299736 and 1349299749 LMissing descriptors for photos 1349299749 and 1349299758 LMissing descriptors for photos 1349299758 and 1349299767 LMissing descriptors for photos 1349299767 and 1349299784 LMissing descriptors for photos 1349299784 and 1349299805 LMissing descriptors for photos 1349299805 and 1349299807 LMissing descriptors for photos 1349299807 and 1349299814 LMissing descriptors for photos 1349299814 and 1349299824 LMissing descriptors for photos 1349299824 and 1349299839 LMissing descriptors for photos 1349299839 and 1349299856 LMissing descriptors for photos 1349299856 and 1349299903 LMissing descriptors for photos 1349299903 and 1349299932 LMissing descriptors for photos 1349299932 and 1349299954 LMissing descriptors for photos 1349299954 and 1349299965 LMissing descriptors for photos 1349299965 and 1349299972 LMissing descriptors for photos 1349299972 and 1349299992 LMissing descriptors for photos 1349299992 and 1349300038 LMissing descriptors for photos 1349300038 and 1349300043 LMissing descriptors for photos 1349300043 and 1349300047 LMissing descriptors for photos 1349300047 and 1349300055 LMissing descriptors for photos 1349300055 and 1349300062 LMissing descriptors for photos 1349300062 and 1349300071 LMissing descriptors for photos 1349300071 and 1349300091 LMissing descriptors for photos 1349300091 and 1349300098 LMissing descriptors for photos 1349300098 and 1349300112 LMissing descriptors for photos 1349300112 and 1349300118 LMissing descriptors for photos 1349300118 and 1349300125 LMissing descriptors for photos 1349300125 and 1349300129 LMissing descriptors for photos 1349300129 and 1349300264 LMissing descriptors for photos 1349300264 and 1349300268 LMissing descriptors for photos 1349300268 and 1349300271 LMissing descriptors for photos 1349300271 and 1349300277 LMissing descriptors for photos 1349300277 and 1349300284 LMissing descriptors for photos 1349300284 and 1349300290 LMissing descriptors for photos 1349300290 and 1349300349 LMissing descriptors for photos 1349300349 and 1349300387 LMissing descriptors for photos 1349300387 and 1349300393 LMissing descriptors for photos 1349300393 and 1349300397 LMissing descriptors for photos 1349300397 and 1349300444 LMissing descriptors for photos 1349300444 and 1349300450 LMissing descriptors for photos 1349300450 and 1349300491 LMissing descriptors for photos 1349300491 and 1349300499 LMissing descriptors for photos 1349300499 and 1349300509 LMissing descriptors for photos 1349300509 and 1349300514 LMissing descriptors for photos 1349300514 and 1349300518 LMissing descriptors for photos 1349300518 and 1349300527 LMissing descriptors for photos 1349300527 and 1349300563 LMissing descriptors for photos 1349300563 and 1349300589 LMissing descriptors for photos 1349300589 and 1349300599 LMissing descriptors for photos 1349300599 and 1349300611 LMissing descriptors for photos 1349300611 and 1349300639 LMissing descriptors for photos 1349300639 and 1349300644 LMissing descriptors for photos 1349300644 and 1349300670 LMissing descriptors for photos 1349300670 and 1349300674 LMissing descriptors for photos 1349300674 and 1349300679 LMissing descriptors for photos 1349300679 and 1349300682 LMissing descriptors for photos 1349300682 and 1349300693 LMissing descriptors for photos 1349300693 and 1349300709 LMissing descriptors for photos 1349300709 and 1349300794 LMissing descriptors for photos 1349300794 and 1349300834 LMissing descriptors for photos 1349300834 and 1349300837 LMissing descriptors for photos 1349300837 and 1349300841 LMissing descriptors for photos 1349300841 and 1349300844 LMissing descriptors for photos 1349300844 and 1349300847 LMissing descriptors for photos 1349300847 and 1349300861 LMissing descriptors for photos 1349300861 and 1349300868 LMissing descriptors for photos 1349300868 and 1349300878 LMissing descriptors for photos 1349300878 and 1349300882 LMissing descriptors for photos 1349300882 and 1349300886 LMissing descriptors for photos 1349300886 and 1349300893 LMissing descriptors for photos 1349300893 and 1349300927 LMissing descriptors for photos 1349300927 and 1349300947 LMissing descriptors for photos 1349300947 and 1349300971 LMissing descriptors for photos 1349300971 and 1349300980 LMissing descriptors for photos 1349300980 and 1349300994 LMissing descriptors for photos 1349300994 and 1349300995 LMissing descriptors for photos 1349300995 and 1349301004 LMissing descriptors for photos 1349301004 and 1349301013 LMissing descriptors for photos 1349301013 and 1349301020 LMissing descriptors for photos 1349301020 and 1349301023 LMissing descriptors for photos 1349301023 and 1349301026 LMissing descriptors for photos 1349301026 and 1349301029 LMissing descriptors for photos 1349301029 and 1349301050 LMissing descriptors for photos 1349301050 and 1349301076 LMissing descriptors for photos 1349301076 and 1349301101 LMissing descriptors for photos 1349301101 and 1349301112 LMissing descriptors for photos 1349301112 and 1349301117 LMissing descriptors for photos 1349301117 and 1349301119 LMissing descriptors for photos 1349301119 and 1349301136 LMissing descriptors for photos 1349301136 and 1349301144 LMissing descriptors for photos 1349301144 and 1349301158 LMissing descriptors for photos 1349301158 and 1349301167 LMissing descriptors for photos 1349301167 and 1349301175 LMissing descriptors for photos 1349301175 and 1349301176 LMissing descriptors for photos 1349301176 and 1349301205 LMissing descriptors for photos 1349301205 and 1349301213 LMissing descriptors for photos 1349301213 and 1349301217 LMissing descriptors for photos 1349301217 and 1349301218 LMissing descriptors for photos 1349301218 and 1349301219 LMissing descriptors for photos 1349301219 and 1349301223 LMissing descriptors for photos 1349301223 and 1349301345 LMissing descriptors for photos 1349301345 and 1349301368 LMissing descriptors for photos 1349301368 and 1349467143 LMissing descriptors for photos 1349467143 and 1349467147 LMissing descriptors for photos 1349467147 and 1349301373 LMissing descriptors for photos 1349301373 and 1349301399 LMissing descriptors for photos 1349301399 and 1349301408 LMissing descriptors for photos 1349301408 and 1349301411 LMissing descriptors for photos 1349301411 and 1349467149 LMissing descriptors for photos 1349467149 and 1349467152 LMissing descriptors for photos 1349467152 and 1349467153 LMissing descriptors for photos 1349467153 and 1349467154 LMissing descriptors for photos 1349467154 and 1349467183 LMissing descriptors for photos 1349467183 and 1349467192 LMissing descriptors for photos 1349467192 and 1349467204 LMissing descriptors for photos 1349467204 and 1349467211 LMissing descriptors for photos 1349467211 and 1349467215 LMissing descriptors for photos 1349467215 and 1349467221 LMissing descriptors for photos 1349467221 and 1349467256 LMissing descriptors for photos 1349467256 and 1349467267 LMissing descriptors for photos 1349467267 and 1349467273 LMissing descriptors for photos 1349467273 and 1349467282 LMissing descriptors for photos 1349467282 and 1349467285 LMissing descriptors for photos 1349467285 and 1349467287 LMissing descriptors for photos 1349467287 and 1349467318 LMissing descriptors for photos 1349467318 and 1349467337 LMissing descriptors for photos 1349467337 and 1349467349 LMissing descriptors for photos 1349467349 and 1349467352 LMissing descriptors for photos 1349467352 and 1349467367 LMissing descriptors for photos 1349467367 and 1349467388 LMissing descriptors for photos 1349467388 and 1349467452 LMissing descriptors for photos 1349467452 and 1349467454 LMissing descriptors for photos 1349467454 and 1349467459 LMissing descriptors for photos 1349467459 and 1349467466 LMissing descriptors for photos 1349467466 and 1349467490 LMissing descriptors for photos 1349467490 and 1349467495 LMissing descriptors for photos 1349467495 and 1349467536 LMissing descriptors for photos 1349467536 and 1349467541 LMissing descriptors for photos 1349467541 and 1349467559 LMissing descriptors for photos 1349467559 and 1349467623 LMissing descriptors for photos 1349467623 and 1349467685 LMissing descriptors for photos 1349467685 and 1349467736 LMissing descriptors for photos 1349467736 and 1349467807 LMissing descriptors for photos 1349467807 and 1349467813 LMissing descriptors for photos 1349467813 and 1349467820 LMissing descriptors for photos 1349467820 and 1349467826 LMissing descriptors for photos 1349467826 and 1349467832 LMissing descriptors for photos 1349467832 and 1349467839 LMissing descriptors for photos 1349467839 and 1349467852 L 28032025 Removing 20 photos because of the 'same image' condition Total on : 3298.8500160000003 list_time_on Total off : 241.15497200000001 list_time_off dist_desc begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 729 time used for this insertion : 0.3954751491546631 photos_removed : len 20 elapsed_time : remove_photo_duplicate 0.45911121368408203 To do, maybe not at the correct place ! ................L.L..L.L.L.....L..L.L..L.L.L.L.L..L.L.L.L.L.L.L..L.L.L..L.L.L.L..L.L.L.L.L.L..L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L..............................L...........L.L....L.................L.......................................................L.L.L.L.L.L....................L.L.L.........................L........L.................................L..L.L...............L........L.L......L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.Lforce hashtag to JRM elapsed_time : CREATE_PORT_BATCH_BY_HOUR 0.029088973999023438 NUMBER BATCH : 4 list_ponderation used : [1e-05, 1e-05, 1e-05, 1e-05, 1e-05] , list_hashtag_class_create_as_list : ['jrm'] result_one_balle_Type_JRM:{'day': '28032025', 'map_nb_amount': {0: 5, 1: 37, 2: 1, 3: 0, 4: 0}, 'map_time_amount': {0: 0, 1: 0, 2: 0, 3: 0, 4: 0}, 'duration': 839.9720571041107, 'nb_balles_papier': 0.0004300000000000005, 'begin_time_port': 'image_28032025_10_00_02_299201m0.jpg 1e-05 for time 1, id_amount 2 this amount prod time diff : 1e-05'} Production hashtag (incorrect ponderation at 20-10-18) : 0.0004300000000000005 result_one_balle_Type_JRM:{'day': '28032025', 'map_nb_amount': {0: 9, 1: 77, 2: 0, 3: 1, 4: 0}, 'map_time_amount': {0: 0, 1: 0, 2: 0, 3: 0, 4: 0}, 'duration': 839.3213129043579, 'nb_balles_papier': 0.0008700000000000016, 'begin_time_port': 'image_28032025_10_15_02_920659m0.jpg 1e-05 for time 1, id_amount 2 this amount prod time diff : 1e-05'} Production hashtag (incorrect ponderation at 20-10-18) : 0.0008700000000000016 result_one_balle_Type_JRM:{'day': '28032025', 'map_nb_amount': {0: 6, 1: 70, 2: 2, 3: 1, 4: 0}, 'map_time_amount': {0: 0, 1: 0, 2: 0, 3: 0, 4: 0}, 'duration': 839.7173590660095, 'nb_balles_papier': 0.0007900000000000014, 'begin_time_port': 'image_28032025_10_30_03_092150m0.jpg 1e-05 for time 1, id_amount 2 this amount prod time diff : 1e-05'} Production hashtag (incorrect ponderation at 20-10-18) : 0.0007900000000000014 LLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLresult_one_balle_Type_JRM:{'day': '28032025', 'map_nb_amount': {0: 13, 1: 69, 2: 0, 3: 2, 4: 0}, 'map_time_amount': {0: 0, 1: 0, 2: 0, 3: 0, 4: 0}, 'duration': 839.7072219848633, 'nb_balles_papier': 0.0008400000000000016, 'begin_time_port': 'image_28032025_10_45_02_596967m0.jpg 1e-05 for time 1, id_amount 2 this amount prod time diff : 1e-05'} Production hashtag (incorrect ponderation at 20-10-18) : 0.0008400000000000016 We have rejected 0 photos because of the batch_size condition ! NUMBER BATCH list_of_portfolios_to_create : 4 list_same_port_ids : [21843349] find same portfolio which already exist 21843349 , we will use it list_same_port_ids : [21844258] find same portfolio which already exist 21844258 , we will use it list_same_port_ids : [21871103] find same portfolio which already exist 21871103 , we will use it list_same_port_ids : [] https://marlene.fotonower.com/api/v1/secured/portfolio/new?name=JRM_diff_batch__28032025_10_45_02_596967&access_token=b05576c56a0e42ad0cb9b46155f68f82 Qualite : 0.0627576097786807 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21843349 AND mptpi.`type`=3726 To do Qualite : 0.12265962776947602 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21844258 AND mptpi.`type`=3726 To do Qualite : 0.10023969089530624 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21871103 AND mptpi.`type`=3726 To do # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22015297 AND mptpi.`type`=3726 To do elapsed_time : count_nb_balles_and_create_portfolio 12.54265546798706 # DISPLAY ALL COLLECTED DATA : {'28032025': {'nb_upload': 729, 'nb_taggue_class': 729, 'nb_taggue_densite': 729, 'nb_descriptors': 313}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1349467852, 1349467839, 1349467832, 1349467826, 1349467820, 1349467813, 1349467807, 1349467736, 1349467685, 1349467623, 1349467559, 1349467541, 1349467536, 1349467495, 1349467490, 1349467466, 1349467459, 1349467454, 1349467452, 1349467388, 1349467367, 1349467352, 1349467349, 1349467337, 1349467318, 1349467287, 1349467285, 1349467282, 1349467273, 1349467267, 1349467256, 1349467221, 1349467215, 1349467211, 1349467204, 1349467192, 1349467183, 1349467154, 1349467153, 1349467152, 1349467149, 1349467147, 1349467143, 1349301411, 1349301408, 1349301399, 1349301373, 1349301368, 1349301345, 1349301223, 1349301219, 1349301218, 1349301217, 1349301213, 1349301205, 1349301176, 1349301175, 1349301167, 1349301158, 1349301144, 1349301136, 1349301119, 1349301117, 1349301112, 1349301101, 1349301076, 1349301050, 1349301029, 1349301026, 1349301023, 1349301020, 1349301013, 1349301004, 1349300995, 1349300994, 1349300980, 1349300971, 1349300947, 1349300927, 1349300893, 1349300886, 1349300882, 1349300878, 1349300868, 1349300861, 1349300847, 1349300844, 1349300841, 1349300837, 1349300834, 1349300794, 1349300709, 1349300693, 1349300682, 1349300679, 1349300674, 1349300670, 1349300644, 1349300639, 1349300611] Looping around the photos to save general results len do output : 1 /21836103Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349467852', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349467839', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349467832', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349467826', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349467820', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349467813', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349467807', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349467736', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349467685', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349467623', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349467559', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349467541', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349467536', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349467495', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349467490', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349467466', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349467459', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349467454', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349467452', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349467388', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349467367', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349467352', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349467349', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349467337', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349467318', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349467287', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349467285', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349467282', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349467273', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349467267', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349467256', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349467221', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349467215', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349467211', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349467204', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349467192', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349467183', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349467154', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349467153', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349467152', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349467149', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349467147', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349467143', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349301411', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349301408', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349301399', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349301373', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349301368', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349301345', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349301223', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349301219', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349301218', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349301217', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349301213', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349301205', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349301176', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349301175', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349301167', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349301158', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349301144', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349301136', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349301119', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349301117', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349301112', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349301101', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349301076', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349301050', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349301029', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349301026', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349301023', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349301020', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349301013', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349301004', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349300995', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349300994', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349300980', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349300971', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349300947', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349300927', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349300893', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349300886', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349300882', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349300878', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349300868', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349300861', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349300847', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349300844', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349300841', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349300837', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349300834', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349300794', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349300709', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349300693', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349300682', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349300679', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349300674', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349300670', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349300644', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349300639', None, None, None, None, None, '2719729') ('4189', None, None, None, None, None, None, None, '2719729') ('4189', '21836103', '1349300611', None, None, None, None, None, '2719729') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 101 time used for this insertion : 1.7035064697265625 save_final save missing photos in datou_result : time spend for datou_step_exec : 51.357091426849365 time spend to save output : 1.704390048980713 total time spend for step 1 : 53.06148147583008 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 1 set_done_treatment 2.10user 0.65system 0:57.79elapsed 4%CPU (0avgtext+0avgdata 106596maxresident)k 312inputs+2680outputs (12major+50351minor)pagefaults 0swaps