python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 20 -a ' -a 4189' -s datou_current_4189 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 3932034 load datou : 4189 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? load thcls load pdts Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 4189, datou_cur_ids : ['4213539'] with mtr_portfolio_ids : ['29220554'] and first list_photo_ids : [] new path : /proc/3932034/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! List Step Type Loaded in datou : split_time_score over limit max, limiting to limit_max 100 list_input_json : [] origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! we have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB time to download the photos : 0.032732486724853516 About to test input to load Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 1 step1:split_time_score Fri Dec 12 08:35:28 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec begin split time score 2022-04-13 10:29:59 0 TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 3379, 'mtr_user_id': 31, 'name': 'learn_classif_flux_maj_generique_effnet_v2_s_02062022', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'aluminium,ela,film_pedb,flux_dev,jrm,pcm,pcnc,pehd_pp,pet_clair,refus,tapis_vide', 'svm_portfolios_learning': '5515864,5515840,5515844,5515850,6244400,6237996,6237998,5515847,5515841,5515868,5515866', 'photo_hashtag_type': 4374, 'photo_desc_type': 5680, 'type_classification': 'tf_classification2', 'hashtag_id_list': '493546845,492741797,2107760237,2107760238,495916461,560181804,1284539308,2107760239,2107755846,538914404,2107748999'}] thcls : [{'id': 3513, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2_tf', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 4557, 'photo_desc_type': 5767, 'type_classification': 'tf_classification2', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('10', 377), ('14', 243)) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {1: 620} 05122025 29220554 Nombre de photos uploadées : 620 / 23040 (2%) 05122025 29220554 Nombre de photos taguées (types de déchets): 620 / 620 (100%) 05122025 29220554 Nombre de photos taguées (volume) : 620 / 620 (100%) elapsed_time : load_data_split_time_score 3.0994415283203125e-06 elapsed_time : order_list_meta_photo_and_scores 0.0002276897430419922 LLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLL elapsed_time : fill_and_build_computed_from_old_data 0.028529882431030273 Catched exception ! Connect or reconnect ! Catched exception ! Connect or reconnect ! elapsed_time : insert_dashboard_record_day_entry 0.260784387588501 LLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLL Creating list_photo_by_hashtags elapsed_time : list_photo_by_hashtags 0.0519716739654541 Creating list_photo_total elapsed_time : select_descriptors 34.54554605484009 05122025 29220554 Nombre de photos avec descriptors (type 5680) : 373 / 373 (100%) ERROR : Hum hum, what can we do for different size of descriptors (ignore the difference ) : 0 vs 1280 photo_id : 1398065761 photo_id_prec : 0 0:00:00|ON:LL0:03:59.891024|OFF:LLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLL0:10:00.545733|ON:LL0:00:00|OFF:LLLLLL0:00:00|ON:LL0:08:59.893135|OFF:LLLLLL0:00:00|ON:LL0:06:00.731673|OFF:0:00:00|ON:LL0:02:59.415904|OFF:0:00:00|ON:LMissing descriptors for photos 1398092766 and 1398092942 LLMissing descriptors for photos 1398092948 and 1398092951 LLL0:08:00.277121|OFF:0:00:00|ON:LL0:13:59.675380|OFF:0:00:59.837468|ON:LMissing descriptors for photos 1398458406 and 1398458685 LMissing descriptors for photos 1398458685 and 1398458686 LMissing descriptors for photos 1398458686 and 1398461105 LMissing descriptors for photos 1398461105 and 1398461121 LMissing descriptors for photos 1398461121 and 1398461124 LMissing descriptors for photos 1398461124 and 1398461140 LMissing descriptors for photos 1398461140 and 1398461143 LMissing descriptors for photos 1398461143 and 1398461145 LMissing descriptors for photos 1398461145 and 1398462602 LMissing descriptors for photos 1398462602 and 1398462603 LMissing descriptors for photos 1398462603 and 1398462604 LMissing descriptors for photos 1398462604 and 1398462608 LMissing descriptors for photos 1398462608 and 1398462610 LMissing descriptors for photos 1398462610 and 1398462616 LMissing descriptors for photos 1398462616 and 1398463123 LMissing descriptors for photos 1398463123 and 1398463167 LMissing descriptors for photos 1398463167 and 1398463183 LMissing descriptors for photos 1398463183 and 1398463185 LMissing descriptors for photos 1398463185 and 1398463215 LMissing descriptors for photos 1398463215 and 1398463218 LMissing descriptors for photos 1398463218 and 1398463770 LMissing descriptors for photos 1398463770 and 1398463773 LMissing descriptors for photos 1398463773 and 1398463775 LMissing descriptors for photos 1398463775 and 1398463778 LMissing descriptors for photos 1398463778 and 1398463781 LMissing descriptors for photos 1398463781 and 1398463787 LMissing descriptors for photos 1398463787 and 1398464796 LMissing descriptors for photos 1398464796 and 1398464801 LMissing descriptors for photos 1398464801 and 1398464803 LMissing descriptors for photos 1398464803 and 1398464806 LMissing descriptors for photos 1398464806 and 1398464810 LMissing descriptors for photos 1398464810 and 1398464812 LMissing descriptors for photos 1398464812 and 1398465117 LMissing descriptors for photos 1398465117 and 1398465121 LMissing descriptors for photos 1398465121 and 1398465125 LMissing descriptors for photos 1398465125 and 1398465129 LMissing descriptors for photos 1398465129 and 1398465131 LMissing descriptors for photos 1398465131 and 1398465135 LMissing descriptors for photos 1398465135 and 1398465458 LMissing descriptors for photos 1398465458 and 1398465459 LMissing descriptors for photos 1398465459 and 1398465461 LMissing descriptors for photos 1398465461 and 1398465464 LMissing descriptors for photos 1398465464 and 1398465468 LMissing descriptors for photos 1398465468 and 1398465473 LMissing descriptors for photos 1398465473 and 1398465535 LMissing descriptors for photos 1398465535 and 1398465539 LMissing descriptors for photos 1398465539 and 1398466653 LMissing descriptors for photos 1398466653 and 1398466657 LMissing descriptors for photos 1398466657 and 1398466663 LMissing descriptors for photos 1398466663 and 1398466667 LMissing descriptors for photos 1398466667 and 1398466670 LMissing descriptors for photos 1398466670 and 1398466700 LMissing descriptors for photos 1398466700 and 1398828545 LMissing descriptors for photos 1398828545 and 1398828550 LMissing descriptors for photos 1398828550 and 1398828555 LMissing descriptors for photos 1398828555 and 1398828557 LMissing descriptors for photos 1398828557 and 1398828560 LMissing descriptors for photos 1398828560 and 1398828564 LMissing descriptors for photos 1398828564 and 1398828795 LMissing descriptors for photos 1398828795 and 1398828801 LMissing descriptors for photos 1398828801 and 1398828803 LMissing descriptors for photos 1398828803 and 1398828805 LMissing descriptors for photos 1398828805 and 1398828808 LMissing descriptors for photos 1398828808 and 1398828812 LMissing descriptors for photos 1398828812 and 1398829024 LMissing descriptors for photos 1398829024 and 1398829030 LMissing descriptors for photos 1398829030 and 1398829033 LMissing descriptors for photos 1398829033 and 1398829038 LMissing descriptors for photos 1398829038 and 1398829049 LMissing descriptors for photos 1398829049 and 1398829052 LMissing descriptors for photos 1398829052 and 1398829366 LMissing descriptors for photos 1398829366 and 1398829369 LMissing descriptors for photos 1398829369 and 1398829372 LMissing descriptors for photos 1398829372 and 1398829379 LMissing descriptors for photos 1398829379 and 1398829380 LMissing descriptors for photos 1398829380 and 1398829382 LMissing descriptors for photos 1398829382 and 1398829669 LMissing descriptors for photos 1398829669 and 1398829671 LMissing descriptors for photos 1398829671 and 1398829673 LMissing descriptors for photos 1398829673 and 1398829676 LMissing descriptors for photos 1398829676 and 1398829677 LMissing descriptors for photos 1398829677 and 1398829678 LMissing descriptors for photos 1398829678 and 1398829894 LMissing descriptors for photos 1398829894 and 1398829895 LMissing descriptors for photos 1398829895 and 1398829898 LMissing descriptors for photos 1398829898 and 1398829900 LMissing descriptors for photos 1398829900 and 1398829923 LMissing descriptors for photos 1398829923 and 1398830138 LMissing descriptors for photos 1398830138 and 1398830142 LMissing descriptors for photos 1398830142 and 1398830147 LMissing descriptors for photos 1398830147 and 1398830151 LMissing descriptors for photos 1398830151 and 1398830153 LMissing descriptors for photos 1398830153 and 1398830155 LMissing descriptors for photos 1398830155 and 1398830370 LMissing descriptors for photos 1398830370 and 1398830375 LMissing descriptors for photos 1398830375 and 1398830380 LMissing descriptors for photos 1398830380 and 1398830385 LMissing descriptors for photos 1398830385 and 1398830391 LMissing descriptors for photos 1398830391 and 1398830398 LMissing descriptors for photos 1398830398 and 1398830596 LMissing descriptors for photos 1398830596 and 1398830606 LMissing descriptors for photos 1398830606 and 1398830610 LMissing descriptors for photos 1398830610 and 1398830613 LMissing descriptors for photos 1398830613 and 1398830616 LMissing descriptors for photos 1398830616 and 1398830618 LMissing descriptors for photos 1398830618 and 1398830830 LMissing descriptors for photos 1398830830 and 1398830835 LMissing descriptors for photos 1398830835 and 1398830837 LMissing descriptors for photos 1398830837 and 1398830839 LMissing descriptors for photos 1398830839 and 1398830840 LMissing descriptors for photos 1398830840 and 1398830841 LMissing descriptors for photos 1398830841 and 1398831107 LMissing descriptors for photos 1398831107 and 1398831110 LMissing descriptors for photos 1398831110 and 1398831116 LMissing descriptors for photos 1398831116 and 1398831120 LMissing descriptors for photos 1398831120 and 1398831122 LMissing descriptors for photos 1398831122 and 1398831124 LMissing descriptors for photos 1398831124 and 1398831355 LMissing descriptors for photos 1398831355 and 1398831356 LMissing descriptors for photos 1398831356 and 1398831358 LMissing descriptors for photos 1398831358 and 1398831360 LMissing descriptors for photos 1398831360 and 1398831363 LMissing descriptors for photos 1398831363 and 1398831365 LMissing descriptors for photos 1398831365 and 1398832313 LMissing descriptors for photos 1398832313 and 1398832315 LMissing descriptors for photos 1398832315 and 1398832324 LMissing descriptors for photos 1398832324 and 1398832338 LMissing descriptors for photos 1398832338 and 1398832344 LMissing descriptors for photos 1398832344 and 1398832345 LMissing descriptors for photos 1398832345 and 1398833849 LMissing descriptors for photos 1398833849 and 1398833989 LMissing descriptors for photos 1398833989 and 1398834154 LMissing descriptors for photos 1398834154 and 1398837230 LMissing descriptors for photos 1398837230 and 1398837281 LMissing descriptors for photos 1398837281 and 1398837358 LMissing descriptors for photos 1398837358 and 1398837431 LMissing descriptors for photos 1398837431 and 1398837481 LMissing descriptors for photos 1398837481 and 1398837530 LMissing descriptors for photos 1398837530 and 1398840762 LMissing descriptors for photos 1398840762 and 1398840765 LMissing descriptors for photos 1398840765 and 1398840768 LMissing descriptors for photos 1398840768 and 1398840771 LMissing descriptors for photos 1398840771 and 1398840776 LMissing descriptors for photos 1398840776 and 1398840821 LMissing descriptors for photos 1398840821 and 1398843385 LMissing descriptors for photos 1398843385 and 1398843436 LMissing descriptors for photos 1398843436 and 1398843495 LMissing descriptors for photos 1398843495 and 1398843528 LMissing descriptors for photos 1398843528 and 1398843564 LMissing descriptors for photos 1398843564 and 1398843630 LMissing descriptors for photos 1398843630 and 1398847475 LMissing descriptors for photos 1398847475 and 1398847486 LMissing descriptors for photos 1398847486 and 1398847488 LMissing descriptors for photos 1398847488 and 1398847490 LMissing descriptors for photos 1398847490 and 1398847492 LMissing descriptors for photos 1398847492 and 1398847494 LMissing descriptors for photos 1398847494 and 1398849201 LMissing descriptors for photos 1398849201 and 1398849205 LMissing descriptors for photos 1398849205 and 1398849211 LMissing descriptors for photos 1398849211 and 1398849215 LMissing descriptors for photos 1398849215 and 1398849218 LMissing descriptors for photos 1398849218 and 1398849223 LMissing descriptors for photos 1398849223 and 1398850672 LMissing descriptors for photos 1398850672 and 1398850675 LMissing descriptors for photos 1398850675 and 1398850681 LMissing descriptors for photos 1398850681 and 1398850685 LMissing descriptors for photos 1398850685 and 1398850687 LMissing descriptors for photos 1398850687 and 1398850691 LMissing descriptors for photos 1398850691 and 1398852555 LMissing descriptors for photos 1398852555 and 1398852600 LMissing descriptors for photos 1398852600 and 1398852632 LMissing descriptors for photos 1398852632 and 1398852639 LMissing descriptors for photos 1398852639 and 1398852642 LMissing descriptors for photos 1398852642 and 1398852655 LMissing descriptors for photos 1398852655 and 1398853239 LMissing descriptors for photos 1398853239 and 1398853243 LMissing descriptors for photos 1398853243 and 1398853252 LMissing descriptors for photos 1398853252 and 1398855079 LMissing descriptors for photos 1398855079 and 1398855085 LMissing descriptors for photos 1398855085 and 1398855112 LMissing descriptors for photos 1398855112 and 1398855247 LMissing descriptors for photos 1398855247 and 1398855286 LMissing descriptors for photos 1398855286 and 1398855330 LMissing descriptors for photos 1398855330 and 1399005689 LMissing descriptors for photos 1399005689 and 1399005698 LMissing descriptors for photos 1399005698 and 1399005700 LMissing descriptors for photos 1399005700 and 1399005701 LMissing descriptors for photos 1399005701 and 1399005703 LMissing descriptors for photos 1399005703 and 1399005706 LMissing descriptors for photos 1399005706 and 1399006162 LMissing descriptors for photos 1399006162 and 1399006176 LMissing descriptors for photos 1399006176 and 1399006180 LMissing descriptors for photos 1399006180 and 1399006189 LMissing descriptors for photos 1399006189 and 1399006194 LMissing descriptors for photos 1399006194 and 1399006197 LMissing descriptors for photos 1399006197 and 1399006614 LMissing descriptors for photos 1399006614 and 1399006617 LMissing descriptors for photos 1399006617 and 1399006620 LMissing descriptors for photos 1399006620 and 1399006622 LMissing descriptors for photos 1399006622 and 1399006626 LMissing descriptors for photos 1399006626 and 1399006671 LMissing descriptors for photos 1399006671 and 1399007556 LMissing descriptors for photos 1399007556 and 1399007569 LMissing descriptors for photos 1399007569 and 1399007575 LMissing descriptors for photos 1399007575 and 1399007578 LMissing descriptors for photos 1399007578 and 1399007591 LMissing descriptors for photos 1399007591 and 1399007641 LMissing descriptors for photos 1399007641 and 1399008084 LMissing descriptors for photos 1399008084 and 1399008088 LMissing descriptors for photos 1399008088 and 1399008094 LMissing descriptors for photos 1399008094 and 1399008104 LMissing descriptors for photos 1399008104 and 1399008118 LMissing descriptors for photos 1399008118 and 1399008122 LMissing descriptors for photos 1399008122 and 1399008741 LMissing descriptors for photos 1399008741 and 1399008742 LMissing descriptors for photos 1399008742 and 1399008747 LMissing descriptors for photos 1399008747 and 1399008752 LMissing descriptors for photos 1399008752 and 1399008763 LMissing descriptors for photos 1399008763 and 1399008965 LMissing descriptors for photos 1399008965 and 1399008969 LMissing descriptors for photos 1399008969 and 1399008974 LMissing descriptors for photos 1399008974 and 1399008977 LMissing descriptors for photos 1399008977 and 1399008979 LMissing descriptors for photos 1399008979 and 1399008981 LMissing descriptors for photos 1399008981 and 1399009578 LMissing descriptors for photos 1399009578 and 1399009582 LMissing descriptors for photos 1399009582 and 1399009586 LMissing descriptors for photos 1399009586 and 1399009591 LMissing descriptors for photos 1399009591 and 1399009593 LMissing descriptors for photos 1399009593 and 1399009595 LMissing descriptors for photos 1399009595 and 1399012272 LMissing descriptors for photos 1399012272 and 1399012321 LMissing descriptors for photos 1399012321 and 1399012370 LMissing descriptors for photos 1399012370 and 1399012418 LMissing descriptors for photos 1399012418 and 1399012448 LMissing descriptors for photos 1399012448 and 1399012450 LMissing descriptors for photos 1399012450 and 1399015055 LMissing descriptors for photos 1399015055 and 1399015058 LMissing descriptors for photos 1399015058 and 1399015063 LMissing descriptors for photos 1399015063 and 1399015067 LMissing descriptors for photos 1399015067 and 1399015068 LMissing descriptors for photos 1399015068 and 1399015069 L 05122025 Removing 76 photos because of the 'same image' condition Total on : 2639.884237 list_time_on Total off : 660.3832010000001 list_time_off dist_desc begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 620 time used for this insertion : 0.36040568351745605 photos_removed : len 76 elapsed_time : remove_photo_duplicate 0.419752836227417 To do, maybe not at the correct place ! ..........................L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L...L.L.L.L...................................................L.L.L.L....................................L................L.................L.L..L.L.............................L.................................................................................................L..............................L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.Lforce hashtag to JRM elapsed_time : CREATE_PORT_BATCH_BY_HOUR 0.022954225540161133 NUMBER BATCH : 4 list_ponderation used : [1e-05, 1e-05, 1e-05, 1e-05, 1e-05] , list_hashtag_class_create_as_list : ['jrm'] result_one_balle_Type_JRM:{'day': '05122025', 'map_nb_amount': {0: 3, 1: 23, 2: 1, 3: 0, 4: 0}, 'map_time_amount': {0: 0, 1: 0, 2: 0, 3: 0, 4: 0}, 'duration': 840.4367568492889, 'nb_balles_papier': 0.0002800000000000001, 'begin_time_port': 'image_05122025_10_00_02_411787m0.jpg 1e-05 for time 1, id_amount 2 this amount prod time diff : 1e-05'} Production hashtag (incorrect ponderation at 20-10-18) : 0.0002800000000000001 result_one_balle_Type_JRM:{'day': '05122025', 'map_nb_amount': {0: 13, 1: 69, 2: 4, 3: 0, 4: 0}, 'map_time_amount': {0: 0, 1: 0, 2: 0, 3: 0, 4: 0}, 'duration': 840.984915971756, 'nb_balles_papier': 0.0008600000000000016, 'begin_time_port': 'image_05122025_10_15_02_488436m0.jpg 1e-05 for time 1, id_amount 2 this amount prod time diff : 1e-05'} Production hashtag (incorrect ponderation at 20-10-18) : 0.0008600000000000016 result_one_balle_Type_JRM:{'day': '05122025', 'map_nb_amount': {0: 14, 1: 69, 2: 2, 3: 0, 4: 0}, 'map_time_amount': {0: 0, 1: 0, 2: 0, 3: 0, 4: 0}, 'duration': 839.7304759025574, 'nb_balles_papier': 0.0008500000000000016, 'begin_time_port': 'image_05122025_10_30_03_248012m0.jpg 1e-05 for time 1, id_amount 2 this amount prod time diff : 1e-05'} Production hashtag (incorrect ponderation at 20-10-18) : 0.0008500000000000016 result_one_balle_Type_JRM:{'day': '05122025', 'map_nb_amount': {0: 14, 1: 82, 2: 2, 3: 0, 4: 0}, 'map_time_amount': {0: 0, 1: 0, 2: 0, 3: 0, 4: 0}, 'duration': 840.0472350120544, 'nb_balles_papier': 0.000980000000000002, 'begin_time_port': 'image_05122025_10_45_02_652463m0.jpg 1e-05 for time 1, id_amount 2 this amount prod time diff : 1e-05'} Production hashtag (incorrect ponderation at 20-10-18) : 0.000980000000000002 We have rejected 0 photos because of the batch_size condition ! NUMBER BATCH list_of_portfolios_to_create : 4 list_same_port_ids : [29225244] find same portfolio which already exist 29225244 , we will use it list_same_port_ids : [29225960] find same portfolio which already exist 29225960 , we will use it list_same_port_ids : [29228464] find same portfolio which already exist 29228464 , we will use it list_same_port_ids : [29262639] find same portfolio which already exist 29262639 , we will use it Qualite : 0.12941184624967841 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=29225244 AND mptpi.`type`=3726 To do Qualite : 0.10165417644987536 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=29225960 AND mptpi.`type`=3726 To do Qualite : 0.0890542220714585 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=29228464 AND mptpi.`type`=3726 To do Qualite : 0.09937930843692273 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=29262639 AND mptpi.`type`=3726 To do elapsed_time : count_nb_balles_and_create_portfolio 1.4517707824707031 # DISPLAY ALL COLLECTED DATA : {'05122025': {'nb_upload': 620, 'nb_taggue_class': 620, 'nb_taggue_densite': 620, 'nb_descriptors': 373}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1399015069, 1399015068, 1399015067, 1399015063, 1399015058, 1399015055, 1399012450, 1399012448, 1399012418, 1399012370, 1399012321, 1399012272, 1399009595, 1399009593, 1399009591, 1399009586, 1399009582, 1399009578, 1399008981, 1399008979, 1399008977, 1399008974, 1399008969, 1399008965, 1399008763, 1399008752, 1399008747, 1399008742, 1399008741, 1399008122, 1399008118, 1399008104, 1399008094, 1399008088, 1399008084, 1399007641, 1399007591, 1399007578, 1399007575, 1399007569, 1399007556, 1399006671, 1399006626, 1399006622, 1399006620, 1399006617, 1399006614, 1399006197, 1399006194, 1399006189, 1399006180, 1399006176, 1399006162, 1399005706, 1399005703, 1399005701, 1399005700, 1399005698, 1399005689, 1398855330, 1398855286, 1398855247, 1398855112, 1398855085, 1398855079, 1398853252, 1398853243, 1398853239, 1398852655, 1398852642, 1398852639, 1398852632, 1398852600, 1398852555, 1398850691, 1398850687, 1398850685, 1398850681, 1398850675, 1398850672, 1398849223, 1398849218, 1398849215, 1398849211, 1398849205, 1398849201, 1398847494, 1398847492, 1398847490, 1398847488, 1398847486, 1398847475, 1398843630, 1398843564, 1398843528, 1398843495, 1398843436, 1398843385, 1398840821, 1398840776] Looping around the photos to save general results len do output : 1 /29220554Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1399015069', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1399015068', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1399015067', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1399015063', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1399015058', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1399015055', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1399012450', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1399012448', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1399012418', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1399012370', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1399012321', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1399012272', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1399009595', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1399009593', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1399009591', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1399009586', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1399009582', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1399009578', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1399008981', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1399008979', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1399008977', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1399008974', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1399008969', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1399008965', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1399008763', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1399008752', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1399008747', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1399008742', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1399008741', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1399008122', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1399008118', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1399008104', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1399008094', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1399008088', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1399008084', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1399007641', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1399007591', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1399007578', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1399007575', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1399007569', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1399007556', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1399006671', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1399006626', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1399006622', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1399006620', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1399006617', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1399006614', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1399006197', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1399006194', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1399006189', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1399006180', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1399006176', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1399006162', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1399005706', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1399005703', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1399005701', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1399005700', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1399005698', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1399005689', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1398855330', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1398855286', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1398855247', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1398855112', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1398855085', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1398855079', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1398853252', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1398853243', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1398853239', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1398852655', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1398852642', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1398852639', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1398852632', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1398852600', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1398852555', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1398850691', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1398850687', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1398850685', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1398850681', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1398850675', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1398850672', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1398849223', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1398849218', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1398849215', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1398849211', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1398849205', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1398849201', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1398847494', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1398847492', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1398847490', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1398847488', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1398847486', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1398847475', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1398843630', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1398843564', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1398843528', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1398843495', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1398843436', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1398843385', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1398840821', None, None, None, None, None, '4213539') ('4189', None, None, None, None, None, None, None, '4213539') ('4189', '29220554', '1398840776', None, None, None, None, None, '4213539') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 101 time used for this insertion : 0.13314127922058105 save_final save missing photos in datou_result : time spend for datou_step_exec : 37.162920236587524 time spend to save output : 0.1339585781097412 total time spend for step 1 : 37.296878814697266 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 1 set_done_treatment 2.11user 0.61system 0:40.00elapsed 6%CPU (0avgtext+0avgdata 113532maxresident)k 128inputs+3120outputs (10major+53513minor)pagefaults 0swaps