python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 20 -a ' -a 4883' -s datou_current_4883 -M 0 -S 0 -U 95,95,120
 import MySQLdb succeeded 
Import error (python version)
['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages']
process id : 2478799
load datou : 4883
# VR 17-11-17 : to create in DB !
Here we check the datou graph and we reorder steps !
 Tree builded and cycle checked, now we need to re-order the steps ! 
We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue
We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id)
 DONE and to test : checkNoCycle !
We are managing only one step so we do not consider checkConsistencyNbInputNbOutput !
We are managing only one step so we do not consider checkConsistencyTypeOutputInput !
Unexpected type for variable list_input_json
ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0)
 Tried to parse : 
None was removed should we ?
donnée sous forme de texte was removed should we ?
[ptf_id0,ptf_id1...] was removed should we ?
load thcls
load pdts
Running datou job : batch_current
 TODO datou_current to load to do maybe to take outside batchDatouExec 
no input labels
no input values
updating current state to 1
we have a portfolio with more photos than limit : 2519>1000
please execute split_portfolio.py -i 22190399 -l 1000
size over we load limit photo not treated
list_input_json: {}
Current got : datou_id : 4883, datou_cur_ids : ['2739598'] with mtr_portfolio_ids : ['22190399'] and first list_photo_ids : []
new path :  /proc/2478799/
Inside batchDatouExec : verbose : 0
# VR 17-11-17 : to create in DB !
Here we check the datou graph and we reorder steps !
 Tree builded and cycle checked, now we need to re-order the steps ! 
We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue
We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id)
 DONE and to test : checkNoCycle !
We are managing only one step so we do not consider checkConsistencyNbInputNbOutput !
We are managing only one step so we do not consider checkConsistencyTypeOutputInput !
List Step Type Loaded in datou : split_time_score
over limit max, limiting to limit_max 100
list_input_json : {}
origin
We have 1 , 
WARNING: data may be incomplete, need to offset and complete !
we have missing 0 photos in the step downloads : 
photo missing : []
try to delete the photos missing in DB
time to download the photos : 0.03637576103210449
About to test input to load 
Calling datou_exec

Inside datou_exec : verbose : 0
number of steps :  1
step1:split_time_score
Thu Apr 10 13:22:25 2025
VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next
VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case
VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec 
VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec
begin split time score
2022-04-13 10:29:59 0
TODO : Insert select and so on
Begin split_port_in_batch_balle
thcls : [{'id': 3379, 'mtr_user_id': 31, 'name': 'learn_classif_flux_maj_generique_effnet_v2_s_02062022', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'aluminium,ela,film_pedb,flux_dev,jrm,pcm,pcnc,pehd_pp,pet_clair,refus,tapis_vide', 'svm_portfolios_learning': '5515864,5515840,5515844,5515850,6244400,6237996,6237998,5515847,5515841,5515868,5515866', 'photo_hashtag_type': 4374, 'photo_desc_type': 5680, 'type_classification': 'tf_classification2', 'hashtag_id_list': '493546845,492741797,2107760237,2107760238,495916461,560181804,1284539308,2107760239,2107755846,538914404,2107748999'}]
thcls : [{'id': 3513, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2_tf', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 4557, 'photo_desc_type': 5767, 'type_classification': 'tf_classification2', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}]
(('00', 276), ('01', 73), ('02', 247), ('03', 254), ('04', 53), ('05', 246), ('06', 267), ('07', 272), ('08', 234), ('09', 110), ('10', 243), ('11', 194), ('12', 50))
ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {1: 2493}
10042025 22190399 Nombre de photos uploadées : 161 / 23040 (0%)
10042025 22190399 Nombre de photos taguées (types de déchets): 2493 / 161 (1548%)
10042025 22190399 Nombre de photos taguées (volume) : 2493 / 161 (1548%)
 elapsed_time : load_data_split_time_score 2.384185791015625e-06
 elapsed_time : order_list_meta_photo_and_scores 0.000997781753540039
???? elapsed_time : fill_and_build_computed_from_old_data 0.03585457801818848
 elapsed_time : insert_dashboard_record_day_entry 0.030011653900146484

 Creating list_photo_by_hashtags 

Hashtag is None 
Hashtag is None 
Hashtag is None 
Hashtag is None 


 elapsed_time : list_photo_by_hashtags 0.013550043106079102
***** BEGIN SPLIT TIME *****
1744243200.0
`1744276824.010135 1744276800.0
`1744276924.009144 1744276920.0
`1744276934.010139 1744276920.0
`1744276985.010155 1744276980.0
`1744277014.010136 1744276980.0
`1744277034.010137 1744276980.0
`1744277064.009124 1744277040.0
`1744277074.010138 1744277040.0
`1744277084.010139 1744277040.0
`1744277094.010133 1744277040.0
`1744277104.010151 1744277100.0
`1744277114.010135 1744277100.0
`1744277124.010134 1744277100.0
`1744277135.010134 1744277100.0
`1744277144.009124 1744277100.0
`1744277154.010135 1744277100.0
`1744277164.010138 1744277160.0
`1744277174.010138 1744277160.0
`1744277214.009125 1744277160.0
`1744277234.010135 1744277220.0
`1744277244.010132 1744277220.0
`1744277254.010139 1744277220.0
`1744277264.010137 1744277220.0
`1744277275.010136 1744277220.0
`1744277284.009147 1744277280.0
`1744277294.010135 1744277280.0
`1744277304.010133 1744277280.0
`1744277344.010153 1744277340.0
`1744277364.010137 1744277340.0
`1744277374.010139 1744277340.0
`1744277384.01014 1744277340.0
`1744277394.01014 1744277340.0
`1744277404.010145 1744277400.0
`1744277414.010133 1744277400.0
`1744277425.010134 1744277400.0
`1744277434.009124 1744277400.0
`1744277444.010136 1744277400.0
`1744277684.010138 1744277640.0
`1744277694.010134 1744277640.0
`1744277944.009139 1744277940.0
`1744277954.010137 1744277940.0
`1744277964.010155 1744277940.0
`1744277984.01014 1744277940.0
`1744277994.010135 1744277940.0
`1744278005.010147 1744278000.0
`1744278014.009129 1744278000.0
`1744278024.010135 1744278000.0
`1744278034.010135 1744278000.0
`1744278044.010137 1744278000.0
`1744278054.010138 1744278000.0
`1744278075.010137 1744278060.0
`1744278084.009124 1744278060.0
`1744278094.010135 1744278060.0
`1744278104.010133 1744278060.0
`1744278114.010134 1744278060.0
`1744278174.010134 1744278120.0
`1744278184.01016 1744278180.0
`1744278194.010133 1744278180.0
`1744278204.010135 1744278180.0
`1744278214.010142 1744278180.0
`1744278225.010136 1744278180.0
`1744278234.009127 1744278180.0
`1744278244.010142 1744278240.0
`1744278264.010136 1744278240.0
`1744278274.010141 1744278240.0
`1744278284.010135 1744278240.0
`1744278304.009131 1744278300.0
`1744278334.010133 1744278300.0
`1744278365.010144 1744278360.0
`1744278374.009125 1744278360.0
`1744278384.010142 1744278360.0
`1744278424.01015 1744278420.0
`1744278434.010134 1744278420.0
`1744278504.010131 1744278480.0
`1744278514.010144 1744278480.0
`1744278554.009121 1744278540.0
`1744278564.010139 1744278540.0
`1744278574.010135 1744278540.0
`1744278584.010133 1744278540.0
`1744278594.010137 1744278540.0
`1744278604.010142 1744278600.0
`1744278625.010135 1744278600.0
`1744278634.009124 1744278600.0
`1744278644.010134 1744278600.0
`1744278684.010134 1744278660.0
`1744278695.010133 1744278660.0
`1744278714.010139 1744278660.0
`1744278754.010147 1744278720.0
`1744278774.009125 1744278720.0
`1744278784.010138 1744278780.0
`1744278794.01014 1744278780.0
`1744278804.010136 1744278780.0
`1744278824.010144 1744278780.0
`1744278834.010134 1744278780.0
`1744278845.010046 1744278840.0
`1744278854.009124 1744278840.0
`1744278904.01015 1744278900.0
`1744278934.009132 1744278900.0
`1744278944.010135 1744278900.0
`1744278954.010136 1744278900.0
`1744278964.010149 1744278960.0
`1744278974.010139 1744278960.0
`1744278995.010146 1744278960.0
`1744279004.009127 1744278960.0
`1744279024.010184 1744279020.0
`1744279054.01014 1744279020.0
`1744279074.009126 1744279020.0
`1744279094.010134 1744279080.0
`1744279104.010141 1744279080.0
`1744279124.010136 1744279080.0
`1744279135.010135 1744279080.0
`1744279214.010187 1744279200.0
`1744279224.010143 1744279200.0
`1744279234.010141 1744279200.0
`1744279244.01014 1744279200.0
`1744279255.010137 1744279200.0
`1744279274.009125 1744279260.0
`1744279294.010135 1744279260.0
`1744279304.010136 1744279260.0
`1744279314.010136 1744279260.0
`1744279335.010136 1744279320.0
`1744279345.010134 1744279320.0
`1744279354.009124 1744279320.0
`1744279364.010134 1744279320.0
`1744279374.010135 1744279320.0
`1744279405.010133 1744279380.0
`1744279415.010133 1744279380.0
`1744279424.009124 1744279380.0
`1744279434.010137 1744279380.0
`1744279444.010138 1744279440.0
`1744279454.010134 1744279440.0
`1744279464.010135 1744279440.0
`1744279475.010134 1744279440.0
`1744279494.009126 1744279440.0
`1744279504.010165 1744279500.0
`1744279514.010135 1744279500.0
`1744279524.010135 1744279500.0
`1744279534.010135 1744279500.0
`1744279555.010139 1744279500.0
`1744279564.00914 1744279560.0
`1744279604.010135 1744279560.0
`1744279614.010136 1744279560.0
`1744279635.010135 1744279620.0
`1744279644.009125 1744279620.0
`1744279664.010142 1744279620.0
`1744279684.010157 1744279680.0
`1744279695.010136 1744279680.0
`1744279705.010134 1744279680.0
`1744279714.009126 1744279680.0
`1744279724.010139 1744279680.0
`1744279734.010138 1744279680.0
`1744279744.010153 1744279740.0
`1744279784.009123 1744279740.0
`1744279804.010148 1744279800.0
`1744279814.010138 1744279800.0
`1744279824.010135 1744279800.0
`1744279835.010146 1744279800.0
`1744279845.010139 1744279800.0
`1744280224.010169 1744280220.0
`1744282024.01016 1744282020.0
`1744282574.010138 1744282560.0
list printed:  [[0], [1, 2], [3, 4, 5], [6, 7, 8, 9], [10, 11, 12, 13, 14, 15], [16, 17, 18], [19, 20, 21, 22, 23], [24, 25, 26], [27, 28, 29, 30, 31], [32, 33, 34, 35, 36], [37, 38], [39, 40, 41, 42, 43], [44, 45, 46, 47, 48, 49], [50, 51, 52, 53, 54], [55], [56, 57, 58, 59, 60, 61], [62, 63, 64, 65], [66, 67], [68, 69, 70], [71, 72], [73, 74], [75, 76, 77, 78, 79], [80, 81, 82, 83], [84, 85, 86], [87, 88], [89, 90, 91, 92, 93], [94, 95], [96, 97, 98, 99], [100, 101, 102, 103], [104, 105, 106], [107, 108, 109, 110], [111, 112, 113, 114, 115], [116, 117, 118, 119], [120, 121, 122, 123, 124], [125, 126, 127, 128], [129, 130, 131, 132, 133], [134, 135, 136, 137, 138], [139, 140, 141], [142, 143, 144], [145, 146, 147, 148, 149, 150], [151, 152], [153, 154, 155, 156, 157], [158], [159], [160]]
forced_hashtag:  JRM
force hashtag to JRM
 elapsed_time : SPLIT_TIME 0.00967097282409668
***** END SPLIT TIME *****
 NUMBER BATCH : 45
 list_ponderation used : [1e-05, 1e-05, 1e-05, 1e-05, 1e-05] , list_hashtag_class_create_as_list : ['jrm']
Missing time_diff : 2107751015 : jrm should be duplicate or empty carpet ! 
Missing time_diff : 2107751015 : jrm should be duplicate or empty carpet ! 
result_one_balle_Type_JRM:{'day': '10042025', 'map_nb_amount': {0: 0, 1: 2, 2: 0, 3: 0, 4: 0}, 'map_time_amount': {0: 0, 1: 0, 2: 0, 3: 0, 4: 0}, 'duration': 0, 'nb_balles_papier': 2e-05, 'begin_time_port': 'image'}
 Production hashtag (incorrect ponderation at 20-10-18) : 2e-05
We reject this portfolio because of the condition_save_data : 2025-04-10 11:22:04.009144
Missing time_diff : 2107751015 : jrm should be duplicate or empty carpet ! 
Missing time_diff : 2107751016 : jrm should be duplicate or empty carpet ! 
Missing time_diff : 2107751015 : jrm should be duplicate or empty carpet ! 
result_one_balle_Type_JRM:{'day': '10042025', 'map_nb_amount': {0: 0, 1: 2, 2: 1, 3: 0, 4: 0}, 'map_time_amount': {0: 0, 1: 0, 2: 0, 3: 0, 4: 0}, 'duration': 48.99998211860657, 'nb_balles_papier': 3.0000000000000004e-05, 'begin_time_port': 'image_10042025_11_23_05_010155m0.jpg 7.000000000000001e-05 for time 1, id_amount 2 this amount prod time diff : 1e-05'}
 Production hashtag (incorrect ponderation at 20-10-18) : 3.0000000000000004e-05
We reject this portfolio because of the condition_save_data : 2025-04-10 11:23:05.010155
Missing time_diff : 2107751014 : jrm should be duplicate or empty carpet ! 
Missing time_diff : 2107751014 : jrm should be duplicate or empty carpet ! 
Missing time_diff : 2107751015 : jrm should be duplicate or empty carpet ! 
Missing time_diff : 2107751015 : jrm should be duplicate or empty carpet ! 
result_one_balle_Type_JRM:{'day': '10042025', 'map_nb_amount': {0: 2, 1: 2, 2: 0, 3: 0, 4: 0}, 'map_time_amount': {0: 0, 1: 0, 2: 0, 3: 0, 4: 0}, 'duration': 30.001008987426758, 'nb_balles_papier': 4e-05, 'begin_time_port': 'image_10042025_11_24_24_009124m0.jpg 0.0001 for time 1, id_amount 1 this amount prod time diff : 1e-05'}
 Production hashtag (incorrect ponderation at 20-10-18) : 4e-05
We reject this portfolio because of the condition_save_data : 2025-04-10 11:24:24.009124
Missing time_diff : 2107751015 : jrm should be duplicate or empty carpet ! 
Missing time_diff : 2107751015 : jrm should be duplicate or empty carpet ! 
Missing time_diff : 2107751015 : jrm should be duplicate or empty carpet ! 
Missing time_diff : 2107751015 : jrm should be duplicate or empty carpet ! 
Missing time_diff : 2107751015 : jrm should be duplicate or empty carpet ! 
Missing time_diff : 2107751015 : jrm should be duplicate or empty carpet ! 
result_one_balle_Type_JRM:{'day': '10042025', 'map_nb_amount': {0: 0, 1: 6, 2: 0, 3: 0, 4: 0}, 'map_time_amount': {0: 0, 1: 0, 2: 0, 3: 0, 4: 0}, 'duration': 49.9999840259552, 'nb_balles_papier': 6e-05, 'begin_time_port': 'image_10042025_11_25_04_010151m0.jpg 0.00014000000000000001 for time 1, id_amount 2 this amount prod time diff : 1e-05'}
 Production hashtag (incorrect ponderation at 20-10-18) : 6e-05
We reject this portfolio because of the condition_save_data : 2025-04-10 11:25:04.010151
Missing time_diff : 2107751014 : jrm should be duplicate or empty carpet ! 
Missing time_diff : 2107751015 : jrm should be duplicate or empty carpet ! 
Missing time_diff : 2107751015 : jrm should be duplicate or empty carpet ! 
result_one_balle_Type_JRM:{'day': '10042025', 'map_nb_amount': {0: 1, 1: 2, 2: 0, 3: 0, 4: 0}, 'map_time_amount': {0: 0, 1: 0, 2: 0, 3: 0, 4: 0}, 'duration': 49.9989869594574, 'nb_balles_papier': 3.0000000000000004e-05, 'begin_time_port': 'image_10042025_11_26_04_010138m0.jpg 0.0002 for time 1, id_amount 1 this amount prod time diff : 1e-05'}
 Production hashtag (incorrect ponderation at 20-10-18) : 3.0000000000000004e-05
We reject this portfolio because of the condition_save_data : 2025-04-10 11:26:04.010138
Missing time_diff : 2107751015 : jrm should be duplicate or empty carpet ! 
Missing time_diff : 2107751015 : jrm should be duplicate or empty carpet ! 
Missing time_diff : 2107751015 : jrm should be duplicate or empty carpet ! 
Missing time_diff : 2107751015 : jrm should be duplicate or empty carpet ! 
Missing time_diff : 2107751015 : jrm should be duplicate or empty carpet ! 
result_one_balle_Type_JRM:{'day': '10042025', 'map_nb_amount': {0: 0, 1: 5, 2: 0, 3: 0, 4: 0}, 'map_time_amount': {0: 0, 1: 0, 2: 0, 3: 0, 4: 0}, 'duration': 41.000000953674316, 'nb_balles_papier': 5e-05, 'begin_time_port': 'image_10042025_11_27_14_010135m0.jpg 0.00023 for time 1, id_amount 2 this amount prod time diff : 1e-05'}
 Production hashtag (incorrect ponderation at 20-10-18) : 5e-05
We reject this portfolio because of the condition_save_data : 2025-04-10 11:27:14.010135
Missing time_diff : 2107751015 : jrm should be duplicate or empty carpet ! 
Missing time_diff : 2107751015 : jrm should be duplicate or empty carpet ! 
Missing time_diff : 2107751015 : jrm should be duplicate or empty carpet ! 
result_one_balle_Type_JRM:{'day': '10042025', 'map_nb_amount': {0: 0, 1: 3, 2: 0, 3: 0, 4: 0}, 'map_time_amount': {0: 0, 1: 0, 2: 0, 3: 0, 4: 0}, 'duration': 20.000986099243164, 'nb_balles_papier': 3.0000000000000004e-05, 'begin_time_port': 'image_10042025_11_28_04_009147m0.jpg 0.0002800000000000001 for time 1, id_amount 2 this amount prod time diff : 1e-05'}
 Production hashtag (incorrect ponderation at 20-10-18) : 3.0000000000000004e-05
We reject this portfolio because of the condition_save_data : 2025-04-10 11:28:04.009147
Missing time_diff : 2107751015 : jrm should be duplicate or empty carpet ! 
Missing time_diff : 2107751014 : jrm should be duplicate or empty carpet ! 
Missing time_diff : 2107751014 : jrm should be duplicate or empty carpet ! 
Missing time_diff : 2107751015 : jrm should be duplicate or empty carpet ! 
Missing time_diff : 2107751015 : jrm should be duplicate or empty carpet ! 
result_one_balle_Type_JRM:{'day': '10042025', 'map_nb_amount': {0: 2, 1: 3, 2: 0, 3: 0, 4: 0}, 'map_time_amount': {0: 0, 1: 0, 2: 0, 3: 0, 4: 0}, 'duration': 49.99998688697815, 'nb_balles_papier': 5e-05, 'begin_time_port': 'image_10042025_11_29_04_010153m0.jpg 0.00031000000000000016 for time 1, id_amount 2 this amount prod time diff : 1e-05'}
 Production hashtag (incorrect ponderation at 20-10-18) : 5e-05
We reject this portfolio because of the condition_save_data : 2025-04-10 11:29:04.010153
Missing time_diff : 2107751015 : jrm should be duplicate or empty carpet ! 
Missing time_diff : 2107751015 : jrm should be duplicate or empty carpet ! 
Missing time_diff : 2107751015 : jrm should be duplicate or empty carpet ! 
Missing time_diff : 2107751015 : jrm should be duplicate or empty carpet ! 
Missing time_diff : 2107751015 : jrm should be duplicate or empty carpet ! 
result_one_balle_Type_JRM:{'day': '10042025', 'map_nb_amount': {0: 0, 1: 5, 2: 0, 3: 0, 4: 0}, 'map_time_amount': {0: 0, 1: 0, 2: 0, 3: 0, 4: 0}, 'duration': 39.999990940093994, 'nb_balles_papier': 5e-05, 'begin_time_port': 'image_10042025_11_30_04_010145m0.jpg 1e-05 for time 1, id_amount 2 this amount prod time diff : 1e-05'}
 Production hashtag (incorrect ponderation at 20-10-18) : 5e-05
We reject this portfolio because of the condition_save_data : 2025-04-10 11:30:04.010145
Missing time_diff : 2107751015 : jrm should be duplicate or empty carpet ! 
Missing time_diff : 2107751015 : jrm should be duplicate or empty carpet ! 
result_one_balle_Type_JRM:{'day': '10042025', 'map_nb_amount': {0: 0, 1: 2, 2: 0, 3: 0, 4: 0}, 'map_time_amount': {0: 0, 1: 0, 2: 0, 3: 0, 4: 0}, 'duration': 0, 'nb_balles_papier': 2e-05, 'begin_time_port': 'image'}
 Production hashtag (incorrect ponderation at 20-10-18) : 2e-05
We reject this portfolio because of the condition_save_data : 2025-04-10 11:34:44.010138
Missing time_diff : 2107751015 : jrm should be duplicate or empty carpet ! 
Missing time_diff : 2107751015 : jrm should be duplicate or empty carpet ! 
Missing time_diff : 2107751015 : jrm should be duplicate or empty ca