python /home/admin/mtr/script_for_cron.py -j default -m 20 -a 'python3 /home/admin/workarea/git/Velours/python/prod/memo/memo.py --job=sla_mensuel --raspi=20084 --dashboard_places=dechet_Valcor_v2 --rootfolder_sla=/data_4/data_log/job/memo/dechet_Valcor_v2 -v --mtr_id=1055 --date=`date -d"$nowdate 15 day ago" +%Y-%m` ' -s sla_mensuel -M 0 -S 0 -U 90,80,95 import MySQLdb succeeded Import error (python version) Need env var : LOGRASPI Raspi : 20084, Date : 2025-07 _cr_cnf_ : /home/admin/mtr/.credentials/credentials.py _cr_cnf_ : /home/admin/mtr/.credentials/credentials.py warning , we can't find thcl infos in json_data warning , we can't find pdt infos in json_data nb_day : (1, 31) VR TODO TOCHECK : due to this bug it shouldn't have being able to work, or maybe it was due to a change directory not done ! SELECT name, created_at,limit_max FROM MTRDatou.mtr_datou WHERE id=4311 SELECT mtd.id, mtdt.`type`, mtd.`param`, mtd.param_json, mtdt.nb_input, mtdt.nb_output, mtdt.prod, mtdt.is_local, mtdt.is_datou_depend, mtdt.is_photo_id_local FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_step_types mtdt WHERE mtdt.`id`=mtd.`type` AND mtd.mtd_id=4311 SELECT mtd.id, mtd.mtd_id, mdsdt.id, mdsdt.name, mdsdt.description, msid.output_or_input, msid.data_order_id, mdsdt.type FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_steptype_io_datatypes msid, MTRDatou.mtr_datou_step_data_types mdsdt WHERE mtd.`type`=msid.`mtr_datou_step_type` AND mtd.mtd_id= 4311 AND msid.data_type=mdsdt.id SELECT mts_id_output, id_output, mts_id_input, id_input FROM MTRDatou.mtr_datou_step_by_step WHERE mtd_id=4311 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11939 matching_dashboard is not consistent : 1 used against 0 in the step definition ! Step 11940 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 0 of step 11939 doesn't seem to be define in the database( WARNING : type of input 2 of step 11940 doesn't seem to be define in the database( DataTypes for each output/input checked ! {"input_values":[]} We expected an STS datou with a camera_place_name, so there is a PB, datou_id: 4311 ! None was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? {'id': '4311', 'name': 'sts_dechet_valcor_matching_v2_2022Q3', 'limit_max': 100, 'nb_steps': 2, 'estimated_time': 8640, 'list_steps': [{'id': 11939, 'type': 'matching_dashboard', 'param': '0', 'json_param_as_string': "{'camera_matching': 'plaque_Valcor', 'max_time_diff': 120, 'matching_method': '1_vs_n', 'input_default_values': []}", 'json_param': {'camera_matching': 'plaque_Valcor', 'max_time_diff': 120, 'matching_method': '1_vs_n', 'input_default_values': []}, 'nb_input': 1, 'nb_output': 0, 'prod': False, 'is_local': False, 'is_datou_depend': False, 'is_photo_id_local': False, 'list_data': [{'step_id': 11939, 'datou_id': 4311, 'type_id': 18, 'name': 'portfolio_id', 'output_or_input': False, 'data_order_id': 0, 'type_python': 'int'}]}, {'id': 11940, 'type': 'split_time_score', 'param': '0', 'json_param_as_string': "{'camera_place_name': 'dechet_Valcor_v2', 'ETA': 8640, 'param_moyenne_mobile': 5, 'thcl': 3442, 'list_task_complete': [''], 'list_tasks': 'LOAD,ORDER,CONVENTION,SPLIT_TRUCK,merge_port_according_to_plate_info,remove_no_plate_batch,CREATE_PORT_LAUNCH_CARAC_FILL_DASHBOARD', 'forced_hashtag': 'truck', 'list_hashtag_class_create': 'truck', 'ponderation': '0.001,0.001,0.001,0.001,0.001', 'host': 'www.fotonower.com', 'datou_proportion_papier': {'test': 370, 'truck': 4234}, 'datou_matching_id': 4311, 'cod': 1, 'min_number_photo_per_batch': 2, 'datou_truck_for_info': 4234}", 'json_param': {'camera_place_name': 'dechet_Valcor_v2', 'ETA': 8640, 'param_moyenne_mobile': 5, 'thcl': 3442, 'list_task_complete': [''], 'list_tasks': 'LOAD,ORDER,CONVENTION,SPLIT_TRUCK,merge_port_according_to_plate_info,remove_no_plate_batch,CREATE_PORT_LAUNCH_CARAC_FILL_DASHBOARD', 'forced_hashtag': 'truck', 'list_hashtag_class_create': 'truck', 'ponderation': '0.001,0.001,0.001,0.001,0.001', 'host': 'www.fotonower.com', 'datou_proportion_papier': {'test': 370, 'truck': 4234}, 'datou_matching_id': 4311, 'cod': 1, 'min_number_photo_per_batch': 2, 'datou_truck_for_info': 4234}, 'nb_input': 2, 'nb_output': 1, 'prod': False, 'is_local': False, 'is_datou_depend': False, 'is_photo_id_local': False, 'list_data': [{'step_id': 11940, 'datou_id': 4311, 'type_id': 18, 'name': 'portfolio_id', 'output_or_input': False, 'data_order_id': 0, 'type_python': 'int'}, {'step_id': 11940, 'datou_id': 4311, 'type_id': 10, 'name': 'data_string', 'output_or_input': False, 'data_order_id': 1, 'type_python': 'string'}, {'step_id': 11940, 'datou_id': 4311, 'type_id': 22, 'name': 'portfolios_ids_list', 'output_or_input': True, 'data_order_id': 0, 'type_python': 'liste_de_int'}]}], 'depends_step': [{'output': 11939, 'output_id': 0, 'input': 11940, 'input_id': 2}]} check if it is the case where we have multiple sts in one datou no, it is not this case SELECT name, created_at,limit_max FROM MTRDatou.mtr_datou WHERE id=370 SELECT mtd.id, mtdt.`type`, mtd.`param`, mtd.param_json, mtdt.nb_input, mtdt.nb_output, mtdt.prod, mtdt.is_local, mtdt.is_datou_depend, mtdt.is_photo_id_local FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_step_types mtdt WHERE mtdt.`id`=mtd.`type` AND mtd.mtd_id=370 SELECT mtd.id, mtd.mtd_id, mdsdt.id, mdsdt.name, mdsdt.description, msid.output_or_input, msid.data_order_id, mdsdt.type FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_steptype_io_datatypes msid, MTRDatou.mtr_datou_step_data_types mdsdt WHERE mtd.`type`=msid.`mtr_datou_step_type` AND mtd.mtd_id= 370 AND msid.data_type=mdsdt.id SELECT mts_id_output, id_output, mts_id_input, id_input FROM MTRDatou.mtr_datou_step_by_step WHERE mtd_id=370 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! DataTypes for each output/input checked ! {"input_values":[]} Unexpected error treated as WARNING for now expected given param in final step {'min_score': 0.7, 'name_pipeline': 'proportion_papier_tapis', 'hashtag_type': 705, 'hashtag_proportion': 492668766, 'hashtag_parmi': 538914404, 'hashtag_to_consider': 'papier_only', 'hashtag_background': 493012381} Found hashtag_parmi in final step, will be used and useful if different from matiere_majoritaire in classifier ventilate_hashtags_in_portfolio step not found ! SELECT name, created_at,limit_max FROM MTRDatou.mtr_datou WHERE id=4234 SELECT mtd.id, mtdt.`type`, mtd.`param`, mtd.param_json, mtdt.nb_input, mtdt.nb_output, mtdt.prod, mtdt.is_local, mtdt.is_datou_depend, mtdt.is_photo_id_local FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_step_types mtdt WHERE mtdt.`id`=mtd.`type` AND mtd.mtd_id=4234 SELECT mtd.id, mtd.mtd_id, mdsdt.id, mdsdt.name, mdsdt.description, msid.output_or_input, msid.data_order_id, mdsdt.type FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_steptype_io_datatypes msid, MTRDatou.mtr_datou_step_data_types mdsdt WHERE mtd.`type`=msid.`mtr_datou_step_type` AND mtd.mtd_id= 4234 AND msid.data_type=mdsdt.id SELECT mts_id_output, id_output, mts_id_input, id_input FROM MTRDatou.mtr_datou_step_by_step WHERE mtd_id=4234 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! {"input_values":[]} Unexpected error treated as WARNING for now expected given param in final step {'name_pipeline': 'aggregate_carac_ratio', 'hashtag_type': 4230, 'hashtag_proportion': 'papier,carton,metal,pet_clair,autre,pehd,pet_fonce', 'hashtag_parmi': 'refus', 'hashtag_weights': {'autre': 3.0, 'barquette_avec_film': 3.0, 'carton': 3.0, 'metal': 3.0, 'papier': 3.0, 'pehd': 3.0, 'pet_fonce': 3.0, 'pet_clair': 1.0, 'flou': 1.0, 'mal_croppe': 1.0}, 'option_sccu': 1} Found hashtag_parmi in final step, will be used and useful if different from matiere_majoritaire in classifier TODO : Insert select and so on SELECT id, name, misc_info, datou_id_for_right FROM MTRPhoto.dashboard_places WHERE name="dechet_Valcor_v2"; SELECT name, created_at,limit_max FROM MTRDatou.mtr_datou WHERE id=4311 SELECT mtd.id, mtdt.`type`, mtd.`param`, mtd.param_json, mtdt.nb_input, mtdt.nb_output, mtdt.prod, mtdt.is_local, mtdt.is_datou_depend, mtdt.is_photo_id_local FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_step_types mtdt WHERE mtdt.`id`=mtd.`type` AND mtd.mtd_id=4311 SELECT mtd.id, mtd.mtd_id, mdsdt.id, mdsdt.name, mdsdt.description, msid.output_or_input, msid.data_order_id, mdsdt.type FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_steptype_io_datatypes msid, MTRDatou.mtr_datou_step_data_types mdsdt WHERE mtd.`type`=msid.`mtr_datou_step_type` AND mtd.mtd_id= 4311 AND msid.data_type=mdsdt.id SELECT mts_id_output, id_output, mts_id_input, id_input FROM MTRDatou.mtr_datou_step_by_step WHERE mtd_id=4311 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11939 matching_dashboard is not consistent : 1 used against 0 in the step definition ! Step 11940 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 0 of step 11939 doesn't seem to be define in the database( WARNING : type of input 2 of step 11940 doesn't seem to be define in the database( DataTypes for each output/input checked ! {"input_values":[]} We expected an STS datou with a camera_place_name, so there is a PB, datou_id: 4311 ! None was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? {'id': 4311, 'name': 'sts_dechet_valcor_matching_v2_2022Q3', 'limit_max': 100, 'nb_steps': 2, 'estimated_time': 8640, 'list_steps': [{'id': 11939, 'type': 'matching_dashboard', 'param': '0', 'json_param_as_string': "{'camera_matching': 'plaque_Valcor', 'max_time_diff': 120, 'matching_method': '1_vs_n', 'input_default_values': []}", 'json_param': {'camera_matching': 'plaque_Valcor', 'max_time_diff': 120, 'matching_method': '1_vs_n', 'input_default_values': []}, 'nb_input': 1, 'nb_output': 0, 'prod': False, 'is_local': False, 'is_datou_depend': False, 'is_photo_id_local': False, 'list_data': [{'step_id': 11939, 'datou_id': 4311, 'type_id': 18, 'name': 'portfolio_id', 'output_or_input': False, 'data_order_id': 0, 'type_python': 'int'}]}, {'id': 11940, 'type': 'split_time_score', 'param': '0', 'json_param_as_string': "{'camera_place_name': 'dechet_Valcor_v2', 'ETA': 8640, 'param_moyenne_mobile': 5, 'thcl': 3442, 'list_task_complete': [''], 'list_tasks': 'LOAD,ORDER,CONVENTION,SPLIT_TRUCK,merge_port_according_to_plate_info,remove_no_plate_batch,CREATE_PORT_LAUNCH_CARAC_FILL_DASHBOARD', 'forced_hashtag': 'truck', 'list_hashtag_class_create': 'truck', 'ponderation': '0.001,0.001,0.001,0.001,0.001', 'host': 'www.fotonower.com', 'datou_proportion_papier': {'test': 370, 'truck': 4234}, 'datou_matching_id': 4311, 'cod': 1, 'min_number_photo_per_batch': 2, 'datou_truck_for_info': 4234}", 'json_param': {'camera_place_name': 'dechet_Valcor_v2', 'ETA': 8640, 'param_moyenne_mobile': 5, 'thcl': 3442, 'list_task_complete': [''], 'list_tasks': 'LOAD,ORDER,CONVENTION,SPLIT_TRUCK,merge_port_according_to_plate_info,remove_no_plate_batch,CREATE_PORT_LAUNCH_CARAC_FILL_DASHBOARD', 'forced_hashtag': 'truck', 'list_hashtag_class_create': 'truck', 'ponderation': '0.001,0.001,0.001,0.001,0.001', 'host': 'www.fotonower.com', 'datou_proportion_papier': {'test': 370, 'truck': 4234}, 'datou_matching_id': 4311, 'cod': 1, 'min_number_photo_per_batch': 2, 'datou_truck_for_info': 4234}, 'nb_input': 2, 'nb_output': 1, 'prod': False, 'is_local': False, 'is_datou_depend': False, 'is_photo_id_local': False, 'list_data': [{'step_id': 11940, 'datou_id': 4311, 'type_id': 18, 'name': 'portfolio_id', 'output_or_input': False, 'data_order_id': 0, 'type_python': 'int'}, {'step_id': 11940, 'datou_id': 4311, 'type_id': 10, 'name': 'data_string', 'output_or_input': False, 'data_order_id': 1, 'type_python': 'string'}, {'step_id': 11940, 'datou_id': 4311, 'type_id': 22, 'name': 'portfolios_ids_list', 'output_or_input': True, 'data_order_id': 0, 'type_python': 'liste_de_int'}]}], 'depends_step': [{'output': 11939, 'output_id': 0, 'input': 11940, 'input_id': 2}]} check if it is the case where we have multiple sts in one datou no, it is not this case SELECT name, created_at,limit_max FROM MTRDatou.mtr_datou WHERE id=370 SELECT mtd.id, mtdt.`type`, mtd.`param`, mtd.param_json, mtdt.nb_input, mtdt.nb_output, mtdt.prod, mtdt.is_local, mtdt.is_datou_depend, mtdt.is_photo_id_local FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_step_types mtdt WHERE mtdt.`id`=mtd.`type` AND mtd.mtd_id=370 SELECT mtd.id, mtd.mtd_id, mdsdt.id, mdsdt.name, mdsdt.description, msid.output_or_input, msid.data_order_id, mdsdt.type FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_steptype_io_datatypes msid, MTRDatou.mtr_datou_step_data_types mdsdt WHERE mtd.`type`=msid.`mtr_datou_step_type` AND mtd.mtd_id= 370 AND msid.data_type=mdsdt.id SELECT mts_id_output, id_output, mts_id_input, id_input FROM MTRDatou.mtr_datou_step_by_step WHERE mtd_id=370 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! DataTypes for each output/input checked ! {"input_values":[]} Unexpected error treated as WARNING for now expected given param in final step {'min_score': 0.7, 'name_pipeline': 'proportion_papier_tapis', 'hashtag_type': 705, 'hashtag_proportion': 492668766, 'hashtag_parmi': 538914404, 'hashtag_to_consider': 'papier_only', 'hashtag_background': 493012381} Found hashtag_parmi in final step, will be used and useful if different from matiere_majoritaire in classifier ventilate_hashtags_in_portfolio step not found ! SELECT name, created_at,limit_max FROM MTRDatou.mtr_datou WHERE id=4234 SELECT mtd.id, mtdt.`type`, mtd.`param`, mtd.param_json, mtdt.nb_input, mtdt.nb_output, mtdt.prod, mtdt.is_local, mtdt.is_datou_depend, mtdt.is_photo_id_local FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_step_types mtdt WHERE mtdt.`id`=mtd.`type` AND mtd.mtd_id=4234 SELECT mtd.id, mtd.mtd_id, mdsdt.id, mdsdt.name, mdsdt.description, msid.output_or_input, msid.data_order_id, mdsdt.type FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_steptype_io_datatypes msid, MTRDatou.mtr_datou_step_data_types mdsdt WHERE mtd.`type`=msid.`mtr_datou_step_type` AND mtd.mtd_id= 4234 AND msid.data_type=mdsdt.id SELECT mts_id_output, id_output, mts_id_input, id_input FROM MTRDatou.mtr_datou_step_by_step WHERE mtd_id=4234 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! {"input_values":[]} Unexpected error treated as WARNING for now expected given param in final step {'name_pipeline': 'aggregate_carac_ratio', 'hashtag_type': 4230, 'hashtag_proportion': 'papier,carton,metal,pet_clair,autre,pehd,pet_fonce', 'hashtag_parmi': 'refus', 'hashtag_weights': {'autre': 3.0, 'barquette_avec_film': 3.0, 'carton': 3.0, 'metal': 3.0, 'papier': 3.0, 'pehd': 3.0, 'pet_fonce': 3.0, 'pet_clair': 1.0, 'flou': 1.0, 'mal_croppe': 1.0}, 'option_sccu': 1} Found hashtag_parmi in final step, will be used and useful if different from matiere_majoritaire in classifier Fonction SLA_MENSUEL nb_day : (1, 31) VR TODO TOCHECK : due to this bug it shouldn't have being able to work, or maybe it was due to a change directory not done ! SELECT name, created_at,limit_max FROM MTRDatou.mtr_datou WHERE id=4311 SELECT mtd.id, mtdt.`type`, mtd.`param`, mtd.param_json, mtdt.nb_input, mtdt.nb_output, mtdt.prod, mtdt.is_local, mtdt.is_datou_depend, mtdt.is_photo_id_local FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_step_types mtdt WHERE mtdt.`id`=mtd.`type` AND mtd.mtd_id=4311 SELECT mtd.id, mtd.mtd_id, mdsdt.id, mdsdt.name, mdsdt.description, msid.output_or_input, msid.data_order_id, mdsdt.type FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_steptype_io_datatypes msid, MTRDatou.mtr_datou_step_data_types mdsdt WHERE mtd.`type`=msid.`mtr_datou_step_type` AND mtd.mtd_id= 4311 AND msid.data_type=mdsdt.id SELECT mts_id_output, id_output, mts_id_input, id_input FROM MTRDatou.mtr_datou_step_by_step WHERE mtd_id=4311 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11939 matching_dashboard is not consistent : 1 used against 0 in the step definition ! Step 11940 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 0 of step 11939 doesn't seem to be define in the database( WARNING : type of input 2 of step 11940 doesn't seem to be define in the database( DataTypes for each output/input checked ! {"input_values":[]} We expected an STS datou with a camera_place_name, so there is a PB, datou_id: 4311 ! None was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? {'id': '4311', 'name': 'sts_dechet_valcor_matching_v2_2022Q3', 'limit_max': 100, 'nb_steps': 2, 'estimated_time': 8640, 'list_steps': [{'id': 11939, 'type': 'matching_dashboard', 'param': '0', 'json_param_as_string': "{'camera_matching': 'plaque_Valcor', 'max_time_diff': 120, 'matching_method': '1_vs_n', 'input_default_values': []}", 'json_param': {'camera_matching': 'plaque_Valcor', 'max_time_diff': 120, 'matching_method': '1_vs_n', 'input_default_values': []}, 'nb_input': 1, 'nb_output': 0, 'prod': False, 'is_local': False, 'is_datou_depend': False, 'is_photo_id_local': False, 'list_data': [{'step_id': 11939, 'datou_id': 4311, 'type_id': 18, 'name': 'portfolio_id', 'output_or_input': False, 'data_order_id': 0, 'type_python': 'int'}]}, {'id': 11940, 'type': 'split_time_score', 'param': '0', 'json_param_as_string': "{'camera_place_name': 'dechet_Valcor_v2', 'ETA': 8640, 'param_moyenne_mobile': 5, 'thcl': 3442, 'list_task_complete': [''], 'list_tasks': 'LOAD,ORDER,CONVENTION,SPLIT_TRUCK,merge_port_according_to_plate_info,remove_no_plate_batch,CREATE_PORT_LAUNCH_CARAC_FILL_DASHBOARD', 'forced_hashtag': 'truck', 'list_hashtag_class_create': 'truck', 'ponderation': '0.001,0.001,0.001,0.001,0.001', 'host': 'www.fotonower.com', 'datou_proportion_papier': {'test': 370, 'truck': 4234}, 'datou_matching_id': 4311, 'cod': 1, 'min_number_photo_per_batch': 2, 'datou_truck_for_info': 4234}", 'json_param': {'camera_place_name': 'dechet_Valcor_v2', 'ETA': 8640, 'param_moyenne_mobile': 5, 'thcl': 3442, 'list_task_complete': [''], 'list_tasks': 'LOAD,ORDER,CONVENTION,SPLIT_TRUCK,merge_port_according_to_plate_info,remove_no_plate_batch,CREATE_PORT_LAUNCH_CARAC_FILL_DASHBOARD', 'forced_hashtag': 'truck', 'list_hashtag_class_create': 'truck', 'ponderation': '0.001,0.001,0.001,0.001,0.001', 'host': 'www.fotonower.com', 'datou_proportion_papier': {'test': 370, 'truck': 4234}, 'datou_matching_id': 4311, 'cod': 1, 'min_number_photo_per_batch': 2, 'datou_truck_for_info': 4234}, 'nb_input': 2, 'nb_output': 1, 'prod': False, 'is_local': False, 'is_datou_depend': False, 'is_photo_id_local': False, 'list_data': [{'step_id': 11940, 'datou_id': 4311, 'type_id': 18, 'name': 'portfolio_id', 'output_or_input': False, 'data_order_id': 0, 'type_python': 'int'}, {'step_id': 11940, 'datou_id': 4311, 'type_id': 10, 'name': 'data_string', 'output_or_input': False, 'data_order_id': 1, 'type_python': 'string'}, {'step_id': 11940, 'datou_id': 4311, 'type_id': 22, 'name': 'portfolios_ids_list', 'output_or_input': True, 'data_order_id': 0, 'type_python': 'liste_de_int'}]}], 'depends_step': [{'output': 11939, 'output_id': 0, 'input': 11940, 'input_id': 2}]} check if it is the case where we have multiple sts in one datou no, it is not this case SELECT name, created_at,limit_max FROM MTRDatou.mtr_datou WHERE id=370 SELECT mtd.id, mtdt.`type`, mtd.`param`, mtd.param_json, mtdt.nb_input, mtdt.nb_output, mtdt.prod, mtdt.is_local, mtdt.is_datou_depend, mtdt.is_photo_id_local FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_step_types mtdt WHERE mtdt.`id`=mtd.`type` AND mtd.mtd_id=370 SELECT mtd.id, mtd.mtd_id, mdsdt.id, mdsdt.name, mdsdt.description, msid.output_or_input, msid.data_order_id, mdsdt.type FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_steptype_io_datatypes msid, MTRDatou.mtr_datou_step_data_types mdsdt WHERE mtd.`type`=msid.`mtr_datou_step_type` AND mtd.mtd_id= 370 AND msid.data_type=mdsdt.id SELECT mts_id_output, id_output, mts_id_input, id_input FROM MTRDatou.mtr_datou_step_by_step WHERE mtd_id=370 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! DataTypes for each output/input checked ! {"input_values":[]} Unexpected error treated as WARNING for now expected given param in final step {'min_score': 0.7, 'name_pipeline': 'proportion_papier_tapis', 'hashtag_type': 705, 'hashtag_proportion': 492668766, 'hashtag_parmi': 538914404, 'hashtag_to_consider': 'papier_only', 'hashtag_background': 493012381} Found hashtag_parmi in final step, will be used and useful if different from matiere_majoritaire in classifier ventilate_hashtags_in_portfolio step not found ! SELECT name, created_at,limit_max FROM MTRDatou.mtr_datou WHERE id=4234 SELECT mtd.id, mtdt.`type`, mtd.`param`, mtd.param_json, mtdt.nb_input, mtdt.nb_output, mtdt.prod, mtdt.is_local, mtdt.is_datou_depend, mtdt.is_photo_id_local FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_step_types mtdt WHERE mtdt.`id`=mtd.`type` AND mtd.mtd_id=4234 SELECT mtd.id, mtd.mtd_id, mdsdt.id, mdsdt.name, mdsdt.description, msid.output_or_input, msid.data_order_id, mdsdt.type FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_steptype_io_datatypes msid, MTRDatou.mtr_datou_step_data_types mdsdt WHERE mtd.`type`=msid.`mtr_datou_step_type` AND mtd.mtd_id= 4234 AND msid.data_type=mdsdt.id SELECT mts_id_output, id_output, mts_id_input, id_input FROM MTRDatou.mtr_datou_step_by_step WHERE mtd_id=4234 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! {"input_values":[]} Unexpected error treated as WARNING for now expected given param in final step {'name_pipeline': 'aggregate_carac_ratio', 'hashtag_type': 4230, 'hashtag_proportion': 'papier,carton,metal,pet_clair,autre,pehd,pet_fonce', 'hashtag_parmi': 'refus', 'hashtag_weights': {'autre': 3.0, 'barquette_avec_film': 3.0, 'carton': 3.0, 'metal': 3.0, 'papier': 3.0, 'pehd': 3.0, 'pet_fonce': 3.0, 'pet_clair': 1.0, 'flou': 1.0, 'mal_croppe': 1.0}, 'option_sccu': 1} Found hashtag_parmi in final step, will be used and useful if different from matiere_majoritaire in classifier TODO : Insert select and so on SELECT id, name, misc_info, datou_id_for_right FROM MTRPhoto.dashboard_places WHERE name="dechet_Valcor_v2"; SELECT name, created_at,limit_max FROM MTRDatou.mtr_datou WHERE id=4311 SELECT mtd.id, mtdt.`type`, mtd.`param`, mtd.param_json, mtdt.nb_input, mtdt.nb_output, mtdt.prod, mtdt.is_local, mtdt.is_datou_depend, mtdt.is_photo_id_local FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_step_types mtdt WHERE mtdt.`id`=mtd.`type` AND mtd.mtd_id=4311 SELECT mtd.id, mtd.mtd_id, mdsdt.id, mdsdt.name, mdsdt.description, msid.output_or_input, msid.data_order_id, mdsdt.type FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_steptype_io_datatypes msid, MTRDatou.mtr_datou_step_data_types mdsdt WHERE mtd.`type`=msid.`mtr_datou_step_type` AND mtd.mtd_id= 4311 AND msid.data_type=mdsdt.id SELECT mts_id_output, id_output, mts_id_input, id_input FROM MTRDatou.mtr_datou_step_by_step WHERE mtd_id=4311 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11939 matching_dashboard is not consistent : 1 used against 0 in the step definition ! Step 11940 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 0 of step 11939 doesn't seem to be define in the database( WARNING : type of input 2 of step 11940 doesn't seem to be define in the database( DataTypes for each output/input checked ! {"input_values":[]} We expected an STS datou with a camera_place_name, so there is a PB, datou_id: 4311 ! None was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? {'id': 4311, 'name': 'sts_dechet_valcor_matching_v2_2022Q3', 'limit_max': 100, 'nb_steps': 2, 'estimated_time': 8640, 'list_steps': [{'id': 11939, 'type': 'matching_dashboard', 'param': '0', 'json_param_as_string': "{'camera_matching': 'plaque_Valcor', 'max_time_diff': 120, 'matching_method': '1_vs_n', 'input_default_values': []}", 'json_param': {'camera_matching': 'plaque_Valcor', 'max_time_diff': 120, 'matching_method': '1_vs_n', 'input_default_values': []}, 'nb_input': 1, 'nb_output': 0, 'prod': False, 'is_local': False, 'is_datou_depend': False, 'is_photo_id_local': False, 'list_data': [{'step_id': 11939, 'datou_id': 4311, 'type_id': 18, 'name': 'portfolio_id', 'output_or_input': False, 'data_order_id': 0, 'type_python': 'int'}]}, {'id': 11940, 'type': 'split_time_score', 'param': '0', 'json_param_as_string': "{'camera_place_name': 'dechet_Valcor_v2', 'ETA': 8640, 'param_moyenne_mobile': 5, 'thcl': 3442, 'list_task_complete': [''], 'list_tasks': 'LOAD,ORDER,CONVENTION,SPLIT_TRUCK,merge_port_according_to_plate_info,remove_no_plate_batch,CREATE_PORT_LAUNCH_CARAC_FILL_DASHBOARD', 'forced_hashtag': 'truck', 'list_hashtag_class_create': 'truck', 'ponderation': '0.001,0.001,0.001,0.001,0.001', 'host': 'www.fotonower.com', 'datou_proportion_papier': {'test': 370, 'truck': 4234}, 'datou_matching_id': 4311, 'cod': 1, 'min_number_photo_per_batch': 2, 'datou_truck_for_info': 4234}", 'json_param': {'camera_place_name': 'dechet_Valcor_v2', 'ETA': 8640, 'param_moyenne_mobile': 5, 'thcl': 3442, 'list_task_complete': [''], 'list_tasks': 'LOAD,ORDER,CONVENTION,SPLIT_TRUCK,merge_port_according_to_plate_info,remove_no_plate_batch,CREATE_PORT_LAUNCH_CARAC_FILL_DASHBOARD', 'forced_hashtag': 'truck', 'list_hashtag_class_create': 'truck', 'ponderation': '0.001,0.001,0.001,0.001,0.001', 'host': 'www.fotonower.com', 'datou_proportion_papier': {'test': 370, 'truck': 4234}, 'datou_matching_id': 4311, 'cod': 1, 'min_number_photo_per_batch': 2, 'datou_truck_for_info': 4234}, 'nb_input': 2, 'nb_output': 1, 'prod': False, 'is_local': False, 'is_datou_depend': False, 'is_photo_id_local': False, 'list_data': [{'step_id': 11940, 'datou_id': 4311, 'type_id': 18, 'name': 'portfolio_id', 'output_or_input': False, 'data_order_id': 0, 'type_python': 'int'}, {'step_id': 11940, 'datou_id': 4311, 'type_id': 10, 'name': 'data_string', 'output_or_input': False, 'data_order_id': 1, 'type_python': 'string'}, {'step_id': 11940, 'datou_id': 4311, 'type_id': 22, 'name': 'portfolios_ids_list', 'output_or_input': True, 'data_order_id': 0, 'type_python': 'liste_de_int'}]}], 'depends_step': [{'output': 11939, 'output_id': 0, 'input': 11940, 'input_id': 2}]} check if it is the case where we have multiple sts in one datou no, it is not this case SELECT name, created_at,limit_max FROM MTRDatou.mtr_datou WHERE id=370 SELECT mtd.id, mtdt.`type`, mtd.`param`, mtd.param_json, mtdt.nb_input, mtdt.nb_output, mtdt.prod, mtdt.is_local, mtdt.is_datou_depend, mtdt.is_photo_id_local FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_step_types mtdt WHERE mtdt.`id`=mtd.`type` AND mtd.mtd_id=370 SELECT mtd.id, mtd.mtd_id, mdsdt.id, mdsdt.name, mdsdt.description, msid.output_or_input, msid.data_order_id, mdsdt.type FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_steptype_io_datatypes msid, MTRDatou.mtr_datou_step_data_types mdsdt WHERE mtd.`type`=msid.`mtr_datou_step_type` AND mtd.mtd_id= 370 AND msid.data_type=mdsdt.id SELECT mts_id_output, id_output, mts_id_input, id_input FROM MTRDatou.mtr_datou_step_by_step WHERE mtd_id=370 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! DataTypes for each output/input checked ! {"input_values":[]} Unexpected error treated as WARNING for now expected given param in final step {'min_score': 0.7, 'name_pipeline': 'proportion_papier_tapis', 'hashtag_type': 705, 'hashtag_proportion': 492668766, 'hashtag_parmi': 538914404, 'hashtag_to_consider': 'papier_only', 'hashtag_background': 493012381} Found hashtag_parmi in final step, will be used and useful if different from matiere_majoritaire in classifier ventilate_hashtags_in_portfolio step not found ! SELECT name, created_at,limit_max FROM MTRDatou.mtr_datou WHERE id=4234 SELECT mtd.id, mtdt.`type`, mtd.`param`, mtd.param_json, mtdt.nb_input, mtdt.nb_output, mtdt.prod, mtdt.is_local, mtdt.is_datou_depend, mtdt.is_photo_id_local FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_step_types mtdt WHERE mtdt.`id`=mtd.`type` AND mtd.mtd_id=4234 SELECT mtd.id, mtd.mtd_id, mdsdt.id, mdsdt.name, mdsdt.description, msid.output_or_input, msid.data_order_id, mdsdt.type FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_steptype_io_datatypes msid, MTRDatou.mtr_datou_step_data_types mdsdt WHERE mtd.`type`=msid.`mtr_datou_step_type` AND mtd.mtd_id= 4234 AND msid.data_type=mdsdt.id SELECT mts_id_output, id_output, mts_id_input, id_input FROM MTRDatou.mtr_datou_step_by_step WHERE mtd_id=4234 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! {"input_values":[]} Unexpected error treated as WARNING for now expected given param in final step {'name_pipeline': 'aggregate_carac_ratio', 'hashtag_type': 4230, 'hashtag_proportion': 'papier,carton,metal,pet_clair,autre,pehd,pet_fonce', 'hashtag_parmi': 'refus', 'hashtag_weights': {'autre': 3.0, 'barquette_avec_film': 3.0, 'carton': 3.0, 'metal': 3.0, 'papier': 3.0, 'pehd': 3.0, 'pet_fonce': 3.0, 'pet_clair': 1.0, 'flou': 1.0, 'mal_croppe': 1.0}, 'option_sccu': 1} Found hashtag_parmi in final step, will be used and useful if different from matiere_majoritaire in classifier place_id : 132 month : 07 mtr_id : 1055 dashboard_places : dechet_Valcor_v2 date : 2025-07 year : 2025 raspi : 20084 select count(distinct mtr_photo_id) from MTRUser.mtr_portfolio_photos where mtr_portfolio_id in (select id from MTRUser.mtr_portfolios where id in (select mtr_portfolio_id from MTRPhoto.dashboard_results where dashboard_run_id in(select last_run_id from MTRPhoto.dashboard_entry_day where dashboard_place_id in (select id from MTRPhoto.dashboard_places where name = 'dechet_Valcor_v2' and date like '%2025-07%') and created_at like '%2025-07%'))); nb_day : (1, 31) after unwanted_material_data nb_day : (1, 31) SQL: SELECT ms.id, ms.name, mst.name as type, ms.test_group, ms.test_name, ms.server, ms.version_code, ms.result_str, ms.result_bool, ms.result_float, ms.lien, ms.created_at FROM MTRAdmin.monitor_sys ms INNER JOIN MTRAdmin.monitor_sys_type mst on mst.id = ms.type WHERE ms.test_name like '%20084%' AND ms.result_str like '%2025/07%'ORDER BY ms.id desc LIMIT 0, 1000000; No error ? {'log_dict': {'total_nb_images': 2119, 'max_time_between_images': '0 days 140 seconds', 'coverage_pause': '0 days 21190 seconds', 'coverage_times_pause': '0 days 42380 seconds', 'nb_photos_decision': 2120, 'nb_gray_level_ok': 2120, 'nb_dedup_ok': 2030, 'nb_two_criteria_ok': 2030, 'nb_forced_upload': 0, 'end_of_day_photos_duplicated': 90, 'end_of_day_photos_nothing_done': 1, 'end_of_day_photos_ok_for_neural_classification': 93, 'end_of_day_photos_uploaded': 1935, 'ratio_coverage': 0.24525746825773445}, 'dashboard_dict': {'nb_ptf_finished': 71, 'nb_images_finished': 306, 'nb_ptf_finished_for_older': 0, 'nb_images_finished_for_older': 0, 'nb_ptf_started': 0, 'nb_images_started': 0, 'nb_selected_images': 306, 'last_update': '2025/07/02', 'portfolio_id': 24450604, 'last_run_id': 1952836, 'total_number_of_photos': 2049}} No error ? {'log_dict': {'total_nb_images': 2119, 'max_time_between_images': '0 days 140 seconds', 'coverage_pause': '0 days 21190 seconds', 'coverage_times_pause': '0 days 42380 seconds', 'nb_photos_decision': 2120, 'nb_gray_level_ok': 2120, 'nb_dedup_ok': 2030, 'nb_two_criteria_ok': 2030, 'nb_forced_upload': 0, 'end_of_day_photos_duplicated': 90, 'end_of_day_photos_nothing_done': 1, 'end_of_day_photos_ok_for_neural_classification': 93, 'end_of_day_photos_uploaded': 1935, 'ratio_coverage': 0.24525746825773445}, 'dashboard_dict': {'nb_ptf_finished': 71, 'nb_images_finished': 306, 'nb_ptf_finished_for_older': 0, 'nb_images_finished_for_older': 0, 'nb_ptf_started': 0, 'nb_images_started': 0, 'nb_selected_images': 306, 'last_update': '2025/07/02', 'portfolio_id': 24450604, 'last_run_id': 1952836, 'total_number_of_photos': 2049}} Since we don't have condition on date we need to verify this lately ! No error ? {'log_dict': {'total_nb_images': 2119, 'max_time_between_images': '0 days 140 seconds', 'coverage_pause': '0 days 21190 seconds', 'coverage_times_pause': '0 days 42380 seconds', 'nb_photos_decision': 2120, 'nb_gray_level_ok': 2120, 'nb_dedup_ok': 2030, 'nb_two_criteria_ok': 2030, 'nb_forced_upload': 0, 'end_of_day_photos_duplicated': 90, 'end_of_day_photos_nothing_done': 1, 'end_of_day_photos_ok_for_neural_classification': 93, 'end_of_day_photos_uploaded': 1935, 'ratio_coverage': 0.24525746825773445}, 'dashboard_dict': {'nb_ptf_finished': 71, 'nb_images_finished': 306, 'nb_ptf_finished_for_older': 0, 'nb_images_finished_for_older': 0, 'nb_ptf_started': 0, 'nb_images_started': 0, 'nb_selected_images': 306, 'last_update': '2025/07/02', 'portfolio_id': 24450604, 'last_run_id': 1952836, 'total_number_of_photos': 2049}} Since we don't have condition on date we need to verify this lately ! No error ? {'log_dict': {'total_nb_images': 2119, 'max_time_between_images': '0 days 140 seconds', 'coverage_pause': '0 days 21190 seconds', 'coverage_times_pause': '0 days 42380 seconds', 'nb_photos_decision': 2120, 'nb_gray_level_ok': 2120, 'nb_dedup_ok': 2030, 'nb_two_criteria_ok': 2030, 'nb_forced_upload': 0, 'end_of_day_photos_duplicated': 90, 'end_of_day_photos_nothing_done': 1, 'end_of_day_photos_ok_for_neural_classification': 93, 'end_of_day_photos_uploaded': 1935, 'ratio_coverage': 0.24525746825773445}, 'dashboard_dict': {'nb_ptf_finished': 71, 'nb_images_finished': 306, 'nb_ptf_finished_for_older': 0, 'nb_images_finished_for_older': 0, 'nb_ptf_started': 0, 'nb_images_started': 0, 'nb_selected_images': 306, 'last_update': '2025/07/02', 'portfolio_id': 24450604, 'last_run_id': 1952836, 'total_number_of_photos': 2049}} Since we don't have condition on date we need to verify this lately ! No error ? {'log_dict': {'total_nb_images': 2119, 'max_time_between_images': '0 days 140 seconds', 'coverage_pause': '0 days 21190 seconds', 'coverage_times_pause': '0 days 42380 seconds', 'nb_photos_decision': 2120, 'nb_gray_level_ok': 2120, 'nb_dedup_ok': 2030, 'nb_two_criteria_ok': 2030, 'nb_forced_upload': 0, 'end_of_day_photos_duplicated': 90, 'end_of_day_photos_nothing_done': 1, 'end_of_day_photos_ok_for_neural_classification': 93, 'end_of_day_photos_uploaded': 1935, 'ratio_coverage': 0.24525746825773445}, 'dashboard_dict': {'nb_ptf_finished': 71, 'nb_images_finished': 306, 'nb_ptf_finished_for_older': 0, 'nb_images_finished_for_older': 0, 'nb_ptf_started': 0, 'nb_images_started': 0, 'nb_selected_images': 306, 'last_update': '2025/07/02', 'portfolio_id': 24450604, 'last_run_id': 1952836, 'total_number_of_photos': 2049}} Since we don't have condition on date we need to verify this lately ! No error ? {'log_dict': {'total_nb_images': 2119, 'max_time_between_images': '0 days 140 seconds', 'coverage_pause': '0 days 21190 seconds', 'coverage_times_pause': '0 days 42380 seconds', 'nb_photos_decision': 2120, 'nb_gray_level_ok': 2120, 'nb_dedup_ok': 2030, 'nb_two_criteria_ok': 2030, 'nb_forced_upload': 0, 'end_of_day_photos_duplicated': 90, 'end_of_day_photos_nothing_done': 1, 'end_of_day_photos_ok_for_neural_classification': 93, 'end_of_day_photos_uploaded': 1935, 'ratio_coverage': 0.24525746825773445}, 'dashboard_dict': {'nb_ptf_finished': 19, 'nb_images_finished': 98, 'nb_ptf_finished_for_older': 0, 'nb_images_finished_for_older': 0, 'nb_ptf_started': 0, 'nb_images_started': 0, 'nb_selected_images': 98, 'last_update': '2025/07/02', 'portfolio_id': 24450604, 'last_run_id': 1947480, 'total_number_of_photos': 1947}} Since we don't have condition on date we need to verify this lately ! Since we don't have condition on date we need to verify this lately ! Found day : 2025-07-01 00:00:00 Missing day sla_per_day needs to be called ! Missing day sla_per_day needs to be called ! Missing day sla_per_day needs to be called ! Missing day sla_per_day needs to be called ! Missing day sla_per_day needs to be called ! Missing day sla_per_day needs to be called ! Missing day sla_per_day needs to be called ! Missing day sla_per_day needs to be called ! Missing day sla_per_day needs to be called ! Missing day sla_per_day needs to be called ! Missing day sla_per_day needs to be called ! Missing day sla_per_day needs to be called ! Missing day sla_per_day needs to be called ! Missing day sla_per_day needs to be called ! Missing day sla_per_day needs to be called ! Missing day sla_per_day needs to be called ! Missing day sla_per_day needs to be called ! Missing day sla_per_day needs to be called ! Missing day sla_per_day needs to be called ! Missing day sla_per_day needs to be called ! Missing day sla_per_day needs to be called ! Missing day sla_per_day needs to be called ! Missing day sla_per_day needs to be called ! Missing day sla_per_day needs to be called ! Missing day sla_per_day needs to be called ! Missing day sla_per_day needs to be called ! Missing day sla_per_day needs to be called ! Missing day sla_per_day needs to be called ! Missing day sla_per_day needs to be called ! Missing day sla_per_day needs to be called ! We will need to call the sla_per_day for these missing days : 2025-07-02 00:00:00,2025-07-03 00:00:00,2025-07-04 00:00:00,2025-07-05 00:00:00,2025-07-06 00:00:00,2025-07-07 00:00:00,2025-07-08 00:00:00,2025-07-09 00:00:00,2025-07-10 00:00:00,2025-07-11 00:00:00,2025-07-12 00:00:00,2025-07-13 00:00:00,2025-07-14 00:00:00,2025-07-15 00:00:00,2025-07-16 00:00:00,2025-07-17 00:00:00,2025-07-18 00:00:00,2025-07-19 00:00:00,2025-07-20 00:00:00,2025-07-21 00:00:00,2025-07-22 00:00:00,2025-07-23 00:00:00,2025-07-24 00:00:00,2025-07-25 00:00:00,2025-07-26 00:00:00,2025-07-27 00:00:00,2025-07-28 00:00:00,2025-07-29 00:00:00,2025-07-30 00:00:00,2025-07-31 00:00:00 after coverage_data SELECT DATE(date_debut) as date, sum(qualite*nombre_balle)/sum(nombre_balle), count(distinct mtr_photo_id) FROM MTRPhoto.dashboard_results dr, MTRUser.mtr_portfolio_photos mpp, MTRPhoto.dashboard_entry_day ded where mpp.mtr_portfolio_id=dr.mtr_portfolio_id and ded.dashboard_place_id = 132 and month(ded.date) = 07 and year(ded.date) = 2025 and dr.dashboard_run_id=ded.last_run_id and dr.qualite>=0 GROUP BY ded.date order by dr.date_debut; SELECT DATE(date_debut) as date, sum(qualite*nombre_balle)/sum(nombre_balle), count(distinct mtr_photo_id) FROM MTRPhoto.dashboard_results dr, MTRUser.mtr_portfolio_photos mpp, MTRPhoto.dashboard_entry_day ded where mpp.mtr_portfolio_id=dr.mtr_portfolio_id and dr.hashtag = '_______truck' and ded.dashboard_place_id = 132 and month(ded.date) = 07 and year(ded.date) = 2025 and dr.dashboard_run_id=ded.last_run_id and dr.qualite>=0 GROUP BY ded.date order by dr.date_debut; select avg(cps.score) as score, SUBSTRING_INDEX(SUBSTRING_INDEX(mp.name,'__', -1), '_', 1) as sbstr, SUBSTRING_INDEX(SUBSTRING_INDEX(mp.name,'_', -2), '_', 1) as sbstr2, SUBSTRING_INDEX(mp.name,'_', -1) as sbstr3 FROM MTRUser.mtr_portfolio_photos mpp, MTRUser.mtr_portfolios mp,MTRPhoto.class_photo_score cps,MTRBack.hashtags h,MTRPhoto.dashboard_results dr,MTRPhoto.dashboard_entry_day ded, MTRPhoto.dashboard_places dp WHERE mp.id=mpp.mtr_portfolio_id and cps.photo_id=mpp.mtr_photo_id and h.hashtag_id=cps.hashtag_id and dr.mtr_portfolio_id = mpp.mtr_portfolio_id and dr.dashboard_run_id = ded.last_run_id and ded.dashboard_place_id = dp.id and h.hashtag in ('blur', 'mean_blur', 'clear') and dp.id = 132 and mp.name like '%072025%' and cps.thcl = 1533 GROUP BY sbstr ; after number_of_batch date_start : 2025-07-01 : dt_date_just_month_year : 2025-07-01 00:00:00 : VR TODO TOCHECK : due to this bug it shouldn't have being able to work, or maybe it was due to a change directory not done ! after pl.get_datou_sts_from_crontab : verbose : True no sts found, try to find from database SELECT dri.id FROM MTRPhoto.dashboard_run_ids dri, MTRPhoto.dashboard_entry_day ded, MTRPhoto.dashboard_places dp WHERE dp.name= "dechet_Valcor_v2" AND ded.dashboard_place_id=dp.id AND dri.dashboard_entry_day=ded.id AND dri.id=ded.last_run_id AND ded.date >= "2025-07-01" AND ded.date <= "2025-07-31" apple3 {'test': {'mat': 'test', 'pht': 0, 'datou_carac_id': 370, 'unwanted_material': [], 'hashtag_majoritaire_from_carac': 538914404}, 'truck': {'mat': 'truck', 'pht': 4230, 'datou_carac_id': 4234, 'unwanted_material': [], 'hashtag_majoritaire_from_carac': 'refus'}} SELECT h.hashtag as unwanted_material, substr(dr.hashtag,8) as main_material, ptp.type as pht_type, sum(pcr.value*dr.nombre_balle)/sum(dr.nombre_balle) as ratio, count(distinct mpp.mtr_photo_id) as nb_photo, group_concat(distinct ptp.mtr_portfolio_id_2) as list_port_cont, group_concat(distinct concat(cast(ptp.mtr_portfolio_id_1 as char), ":", cast(ptp.mtr_portfolio_id_2 as char))) as assoc_port, group_concat(distinct concat(cast(ptp.mtr_portfolio_id_1 as char), ":", h.hashtag, ":", cast(ptp.type as char), ":", cast(ptp.mtr_portfolio_id_2 as char))) as assoc_mat FROM MTRPhoto.dashboard_results dr, MTRPhoto.mtr_port_to_port_ids ptp, MTRUser.mtr_portfolio_photos mpp, MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h WHERE dr.dashboard_run_id IN (1952836,1956272,1959630,1962546,1964187,1964629,1969138,1970494,1973520,1980704,1980340,1982048,1982489,1986798,1990898,1995131,1997924,1999399,1999814,2004090,2007653,2015152,2015179,2018811,2020535,2021275,2029913,2043881,2042903,2051745) AND dr.mtr_portfolio_id=ptp.mtr_portfolio_id_1 AND dr.qualite >= 0 AND mpp.mtr_portfolio_id=ptp.mtr_portfolio_id_2 AND pcr.portfolio_id=ptp.mtr_portfolio_id_1 AND h.hashtag_id = pcr.hashtag_id AND ptp.type = pcr.hashtag_type AND mpp.hide_status = 0 AND ptp.hashtag_id=h.hashtag_id AND ptp.type IN (0,4230) group by h.hashtag, dr.hashtag, ptp.type; VR TODO TO BETTER PARSE ! ({'unwanted_material': 'autre', 'main_material': 'truck', 'pht_type': 4230, 'ratio': 0.02141215561237392, 'nb_photo': 89507, 'list_port_cont': '24452073,24452107,24452158,24452266,24452318,24452357,24452459,24452921,24452974,24453019,24453063,24453088,24453125,24453174,24453179,24453263,24453303,24453339,24453497,24491845,24491888,24491922,24491950,24491983,24492023,24492033,24492091,24492132,24492163,24492200,24492242,24492282,24492312,24492345,24492396,24492426,24492451,24492494,24492534,24492562,24492608,24492627,24492634,24492721,24492728,24492780,24492825,24492850,24492885,24492922,24492947,24493007,24493017,24493079,24493115,24493150,24493191,24493198,24493255,24493291,24493312,24493371,24493403,24493443,24493444,24493508,24493536,24493582,24493607,24493659,24493688,24500172,24500294,24500390,24502680,24503839,24503904,24503949,24503994,24504119,24504194,24504311,24504420,24504452,24510008,24510035,24510092,24510599,24510641,24510741,24510868,24510950,24510976,24511121,24511180,24511196,24511284,24511372,24511429,24511659,24511718,24511855,24512039,24517153,24517218,24517557,24517644,24517897,24518045,24518070,24518321,24518460,24521395,2452144', 'assoc_port': '24452044:24452266,24452047:24452318,24452048:24452459,24452049:24452158,24452050:24452357,24452051:24452107,24452052:24452073,24452770:24453497,24452789:24453339,24452808:24453088,24452817:24453174,24452828:24453303,24452834:24453063,24452841:24453179,24452843:24453019,24452844:24453125,24452845:24453263,24452846:24452974,24452847:24452921,24491721:24493688,24491722:24493444,24491723:24493508,24491724:24493659,24491725:24493191,24491726:24493255,24491727:24493536,24491728:24493403,24491729:24493582,24491730:24493198,24491731:24492922,24491732:24492947,24491733:24493291,24491734:24493371,24491735:24493017,24491736:24493007,24491737:24493150,24491738:24493079,24491739:24493115,24491740:24492721,24491741:24492728,24491742:24492608,24491743:24493607,24491744:24492634,24491745:24492627,24491746:24492451,24491747:24492426,24491748:24492885,24491749:24492396,24491750:24492562,24491751:24493443,24491752:24492825,24491753:24492534,24491754:24492494,24491755:24493312,24491756:24492345,24491757:24492780,24491758:2449228', 'assoc_mat': '24452044:autre:4230:24452266,24452047:autre:4230:24452318,24452048:autre:4230:24452459,24452049:autre:4230:24452158,24452050:autre:4230:24452357,24452051:autre:4230:24452107,24452052:autre:4230:24452073,24452770:autre:4230:24453497,24452789:autre:4230:24453339,24452808:autre:4230:24453088,24452817:autre:4230:24453174,24452828:autre:4230:24453303,24452834:autre:4230:24453063,24452841:autre:4230:24453179,24452843:autre:4230:24453019,24452844:autre:4230:24453125,24452845:autre:4230:24453263,24452846:autre:4230:24452974,24452847:autre:4230:24452921,24491721:autre:4230:24493688,24491722:autre:4230:24493444,24491723:autre:4230:24493508,24491724:autre:4230:24493659,24491725:autre:4230:24493191,24491726:autre:4230:24493255,24491727:autre:4230:24493536,24491728:autre:4230:24493403,24491729:autre:4230:24493582,24491730:autre:4230:24493198,24491731:autre:4230:24492922,24491732:autre:4230:24492947,24491733:autre:4230:24493291,24491734:autre:4230:24493371,24491735:autre:4230:24493017,24491736:autre:4230:24493007,24491737:'}, {'unwanted_material': 'carton', 'main_material': 'truck', 'pht_type': 4230, 'ratio': 0.029727232594059218, 'nb_photo': 117018, 'list_port_cont': '24452076,24452119,24452164,24452272,24452325,24452374,24452449,24452930,24452985,24453026,24453048,24453102,24453142,24453177,24453196,24453264,24453306,24453346,24453502,24491854,24491884,24491923,24491962,24491986,24492032,24492053,24492103,24492133,24492174,24492194,24492252,24492278,24492322,24492359,24492388,24492429,24492450,24492490,24492536,24492574,24492604,24492638,24492647,24492707,24492737,24492784,24492820,24492855,24492902,24492934,24492950,24493002,24493026,24493077,24493116,24493152,24493185,24493207,24493254,24493296,24493309,24493370,24493399,24493435,24493453,24493500,24493551,24493583,24493615,24493654,24493692,24500179,24500296,24500395,24502681,24503840,24503906,24503951,24503999,24504129,24504210,24504320,24504426,24504459,24510002,24510048,24510110,24510613,24510653,24510750,24510865,24510948,24510990,24511118,24511177,24511194,24511297,24511384,24511437,24511656,24511727,24511856,24512032,24517157,24517208,24517570,24517660,24517899,24518046,24518080,24518312,24518466,24521403,2452143', 'assoc_port': '24452044:24452272,24452047:24452325,24452048:24452449,24452049:24452164,24452050:24452374,24452051:24452119,24452052:24452076,24452770:24453502,24452789:24453346,24452808:24453102,24452817:24453177,24452828:24453306,24452834:24453048,24452841:24453196,24452843:24453026,24452844:24453142,24452845:24453264,24452846:24452985,24452847:24452930,24491721:24493692,24491722:24493453,24491723:24493500,24491724:24493654,24491725:24493185,24491726:24493254,24491727:24493551,24491728:24493399,24491729:24493583,24491730:24493207,24491731:24492934,24491732:24492950,24491733:24493296,24491734:24493370,24491735:24493026,24491736:24493002,24491737:24493152,24491738:24493077,24491739:24493116,24491740:24492707,24491741:24492737,24491742:24492604,24491743:24493615,24491744:24492638,24491745:24492647,24491746:24492450,24491747:24492429,24491748:24492902,24491749:24492388,24491750:24492574,24491751:24493435,24491752:24492820,24491753:24492536,24491754:24492490,24491755:24493309,24491756:24492359,24491757:24492784,24491758:2449227', 'assoc_mat': '24452044:carton:4230:24452272,24452047:carton:4230:24452325,24452048:carton:4230:24452449,24452049:carton:4230:24452164,24452050:carton:4230:24452374,24452051:carton:4230:24452119,24452052:carton:4230:24452076,24452770:carton:4230:24453502,24452789:carton:4230:24453346,24452808:carton:4230:24453102,24452817:carton:4230:24453177,24452828:carton:4230:24453306,24452834:carton:4230:24453048,24452841:carton:4230:24453196,24452843:carton:4230:24453026,24452844:carton:4230:24453142,24452845:carton:4230:24453264,24452846:carton:4230:24452985,24452847:carton:4230:24452930,24491721:carton:4230:24493692,24491722:carton:4230:24493453,24491723:carton:4230:24493500,24491724:carton:4230:24493654,24491725:carton:4230:24493185,24491726:carton:4230:24493254,24491727:carton:4230:24493551,24491728:carton:4230:24493399,24491729:carton:4230:24493583,24491730:carton:4230:24493207,24491731:carton:4230:24492934,24491732:carton:4230:24492950,24491733:carton:4230:24493296,24491734:carton:4230:24493370,24491735:carton:4230:24493026,2449'}, {'unwanted_material': 'metal', 'main_material': 'truck', 'pht_type': 4230, 'ratio': 0.008217845252983105, 'nb_photo': 4223, 'list_port_cont': '24452087,24452124,24452155,24452330,24452364,24452448,24452913,24452973,24453024,24453049,24453093,24453135,24453178,24453195,24453265,24453304,24453343,24491840,24491876,24491992,24492052,24492167,24492272,24492308,24492360,24492384,24492443,24492501,24492532,24492576,24492629,24492632,24492791,24492819,24492852,24492886,24492948,24493006,24493080,24493293,24493307,24493359,24493461,24493580,24493622,24493655,24493690,24500181,24500306,24500381,24502684,24503851,24503916,24503953,24503995,24504319,24504411,24504462,24510642,24510746,24510862,24510946,24511106,24511178,24511288,24511375,24511432,24511644,24511719,24511843,24512035,24517215,24517571,24517659,24517895,24518090,24518315,24521427,24525622,24526572,24526686,24527125,24567355,24567453,24567582,24567598,24567722,24568066,24573520,24573582,24573677,24573736,24573817,24573818,24573950,24574029,24574099,24574166,24574283,24574427,24574600,24574659,24574729,24574779,24574844,24575508,24575656,24575726,24575736,24575937,24576168,24576436,24576754,2458118', 'assoc_port': '24452047:24452330,24452048:24452448,24452049:24452155,24452050:24452364,24452051:24452124,24452052:24452087,24452789:24453343,24452808:24453093,24452817:24453178,24452828:24453304,24452834:24453049,24452841:24453195,24452843:24453024,24452844:24453135,24452845:24453265,24452846:24452973,24452847:24452913,24491721:24493690,24491722:24493461,24491724:24493655,24491729:24493580,24491732:24492948,24491733:24493293,24491734:24493359,24491736:24493006,24491738:24493080,24491743:24493622,24491744:24492632,24491745:24492629,24491747:24492443,24491748:24492886,24491749:24492384,24491750:24492576,24491752:24492819,24491753:24492532,24491754:24492501,24491755:24493307,24491756:24492360,24491757:24492791,24491758:24492272,24491759:24492308,24491760:24492167,24491763:24491840,24491767:24491992,24491770:24492852,24491771:24491876,24491772:24492052,24499974:24500381,24499989:24500306,24500004:24502684,24500008:24500181,24503600:24504411,24503602:24504462,24503606:24503995,24503607:24504319,24503609:24503953,24503611:2450385', 'assoc_mat': '24452047:metal:4230:24452330,24452048:metal:4230:24452448,24452049:metal:4230:24452155,24452050:metal:4230:24452364,24452051:metal:4230:24452124,24452052:metal:4230:24452087,24452789:metal:4230:24453343,24452808:metal:4230:24453093,24452817:metal:4230:24453178,24452828:metal:4230:24453304,24452834:metal:4230:24453049,24452841:metal:4230:24453195,24452843:metal:4230:24453024,24452844:metal:4230:24453135,24452845:metal:4230:24453265,24452846:metal:4230:24452973,24452847:metal:4230:24452913,24491721:metal:4230:24493690,24491722:metal:4230:24493461,24491724:metal:4230:24493655,24491729:metal:4230:24493580,24491732:metal:4230:24492948,24491733:metal:4230:24493293,24491734:metal:4230:24493359,24491736:metal:4230:24493006,24491738:metal:4230:24493080,24491743:metal:4230:24493622,24491744:metal:4230:24492632,24491745:metal:4230:24492629,24491747:metal:4230:24492443,24491748:metal:4230:24492886,24491749:metal:4230:24492384,24491750:metal:4230:24492576,24491752:metal:4230:24492819,24491753:metal:4230:24492532,24491754:'}, {'unwanted_material': 'papier', 'main_material': 'truck', 'pht_type': 4230, 'ratio': 0.15884656879218992, 'nb_photo': 306144, 'list_port_cont': '24452083,24452117,24452162,24452264,24452334,24452372,24452456,24452923,24452983,24453022,24453055,24453103,24453131,24453168,24453194,24453271,24453300,24453354,24453506,24491856,24491891,24491921,24491961,24491990,24492034,24492048,24492107,24492137,24492178,24492211,24492247,24492281,24492323,24492351,24492391,24492441,24492442,24492504,24492540,24492565,24492610,24492636,24492641,24492711,24492730,24492793,24492824,24492856,24492894,24492935,24492955,24493010,24493028,24493076,24493114,24493149,24493183,24493206,24493258,24493295,24493306,24493365,24493401,24493441,24493458,24493504,24493548,24493578,24493623,24493651,24493685,24500178,24500305,24500389,24502674,24503847,24503924,24503956,24503988,24504133,24504205,24504314,24504423,24504458,24510006,24510047,24510107,24510615,24510651,24510751,24510869,24510943,24510980,24511117,24511176,24511189,24511293,24511383,24511431,24511662,24511721,24511849,24512038,24517160,24517207,24517572,24517647,24517904,24518034,24518082,24518314,24518469,24521401,2452143', 'assoc_port': '24452044:24452264,24452047:24452334,24452048:24452456,24452049:24452162,24452050:24452372,24452051:24452117,24452052:24452083,24452770:24453506,24452789:24453354,24452808:24453103,24452817:24453168,24452828:24453300,24452834:24453055,24452841:24453194,24452843:24453022,24452844:24453131,24452845:24453271,24452846:24452983,24452847:24452923,24491721:24493685,24491722:24493458,24491723:24493504,24491724:24493651,24491725:24493183,24491726:24493258,24491727:24493548,24491728:24493401,24491729:24493578,24491730:24493206,24491731:24492935,24491732:24492955,24491733:24493295,24491734:24493365,24491735:24493028,24491736:24493010,24491737:24493149,24491738:24493076,24491739:24493114,24491740:24492711,24491741:24492730,24491742:24492610,24491743:24493623,24491744:24492636,24491745:24492641,24491746:24492442,24491747:24492441,24491748:24492894,24491749:24492391,24491750:24492565,24491751:24493441,24491752:24492824,24491753:24492540,24491754:24492504,24491755:24493306,24491756:24492351,24491757:24492793,24491758:2449228', 'assoc_mat': '24452044:papier:4230:24452264,24452047:papier:4230:24452334,24452048:papier:4230:24452456,24452049:papier:4230:24452162,24452050:papier:4230:24452372,24452051:papier:4230:24452117,24452052:papier:4230:24452083,24452770:papier:4230:24453506,24452789:papier:4230:24453354,24452808:papier:4230:24453103,24452817:papier:4230:24453168,24452828:papier:4230:24453300,24452834:papier:4230:24453055,24452841:papier:4230:24453194,24452843:papier:4230:24453022,24452844:papier:4230:24453131,24452845:papier:4230:24453271,24452846:papier:4230:24452983,24452847:papier:4230:24452923,24491721:papier:4230:24493685,24491722:papier:4230:24493458,24491723:papier:4230:24493504,24491724:papier:4230:24493651,24491725:papier:4230:24493183,24491726:papier:4230:24493258,24491727:papier:4230:24493548,24491728:papier:4230:24493401,24491729:papier:4230:24493578,24491730:papier:4230:24493206,24491731:papier:4230:24492935,24491732:papier:4230:24492955,24491733:papier:4230:24493295,24491734:papier:4230:24493365,24491735:papier:4230:24493028,2449'}, {'unwanted_material': 'pehd', 'main_material': 'truck', 'pht_type': 4230, 'ratio': 0.00121672865586385, 'nb_photo': 1319, 'list_port_cont': '24453140,24453171,24453186,24453315,24453505,24491892,24491963,24491993,24492179,24492355,24492389,24492505,24492538,24492612,24492788,24492826,24492863,24492946,24493189,24493205,24493315,24493429,24493494,24493619,24493657,24502682,24503848,24503910,24503961,24510639,24510739,24510873,24510987,24511119,24511179,24511203,24511298,24511380,24511434,24511663,24511725,24512033,24517149,24517573,24518473,24526698,24567451,24574165,24574788,24574852,24576438,24581180,24581383,24581954,24582113,24595010,24595062,24595967,24596294,24596383,24596819,24596856,24596893,24596957,24597249,24597284,24597381,24597425,24597516,24597535,24597721,24598368,24598824,24599382,24603645,24603800,24604385,24604512,24605508,24605976,24606200,24606346,24610871,24611080,24611212,24611317,24611385,24615381,24615421,24615540,24615672,24615802,24615909,24616159,24616417,24616603,24616722,24616894,24616967,24617088,24620087,24640957,24641933,24641968,24645871,24646222,24717794,24718476,24721008,24721885,24722001,24722686,24722817,2472373', 'assoc_port': '24452770:24453505,24452817:24453171,24452828:24453315,24452841:24453186,24452844:24453140,24491723:24493494,24491724:24493657,24491725:24493189,24491730:24493205,24491732:24492946,24491742:24492612,24491743:24493619,24491749:24492389,24491751:24493429,24491752:24492826,24491753:24492538,24491754:24492505,24491755:24493315,24491756:24492355,24491757:24492788,24491760:24492179,24491767:24491993,24491769:24491963,24491770:24492863,24491771:24491892,24500004:24502682,24503609:24503961,24503611:24503848,24503613:24503910,24510372:24511663,24510374:24512033,24510375:24511725,24510376:24511380,24510389:24511434,24510402:24511298,24510403:24511119,24510404:24511179,24510405:24511203,24510406:24510739,24510422:24510987,24510437:24510873,24510452:24510639,24516260:24518473,24516340:24517149,24516344:24517573,24526363:24526698,24567050:24567451,24573197:24576438,24573209:24574788,24573211:24574852,24573283:24574165,24580565:24581954,24580570:24581383,24580574:24581180,24580579:24582113,24594212:24599382,24594215:2459836', 'assoc_mat': '24452770:pehd:4230:24453505,24452817:pehd:4230:24453171,24452828:pehd:4230:24453315,24452841:pehd:4230:24453186,24452844:pehd:4230:24453140,24491723:pehd:4230:24493494,24491724:pehd:4230:24493657,24491725:pehd:4230:24493189,24491730:pehd:4230:24493205,24491732:pehd:4230:24492946,24491742:pehd:4230:24492612,24491743:pehd:4230:24493619,24491749:pehd:4230:24492389,24491751:pehd:4230:24493429,24491752:pehd:4230:24492826,24491753:pehd:4230:24492538,24491754:pehd:4230:24492505,24491755:pehd:4230:24493315,24491756:pehd:4230:24492355,24491757:pehd:4230:24492788,24491760:pehd:4230:24492179,24491767:pehd:4230:24491993,24491769:pehd:4230:24491963,24491770:pehd:4230:24492863,24491771:pehd:4230:24491892,24500004:pehd:4230:24502682,24503609:pehd:4230:24503961,24503611:pehd:4230:24503848,24503613:pehd:4230:24503910,24510372:pehd:4230:24511663,24510374:pehd:4230:24512033,24510375:pehd:4230:24511725,24510376:pehd:4230:24511380,24510389:pehd:4230:24511434,24510402:pehd:4230:24511298,24510403:pehd:4230:24511119,24510404:pehd:42'}, {'unwanted_material': 'pet_clair', 'main_material': 'truck', 'pht_type': 4230, 'ratio': 0.011905527100546353, 'nb_photo': 53012, 'list_port_cont': '24452078,24452118,24452160,24452270,24452329,24452369,24452457,24452929,24452989,24453016,24453057,24453098,24453132,24453175,24453191,24453268,24453307,24453348,24453501,24491846,24491881,24491927,24491959,24491997,24492036,24492047,24492104,24492140,24492176,24492214,24492251,24492279,24492317,24492356,24492392,24492439,24492445,24492502,24492541,24492575,24492614,24492624,24492650,24492713,24492738,24492785,24492830,24492864,24492891,24492936,24492944,24493008,24493023,24493082,24493117,24493146,24493190,24493203,24493261,24493292,24493317,24493367,24493394,24493438,24493456,24493510,24493545,24493585,24493614,24493653,24493687,24500188,24500301,24500391,24502678,24503841,24503900,24503962,24503993,24504132,24504211,24504313,24504424,24504460,24510003,24510049,24510084,24510610,24510655,24510743,24510871,24510951,24510988,24511120,24511174,24511202,24511295,24511382,24511433,24511664,24511715,24511848,24512037,24517141,24517210,24517567,24517655,24517896,24518032,24518094,24518318,24518471,24521400,2452143', 'assoc_port': '24452044:24452270,24452047:24452329,24452048:24452457,24452049:24452160,24452050:24452369,24452051:24452118,24452052:24452078,24452770:24453501,24452789:24453348,24452808:24453098,24452817:24453175,24452828:24453307,24452834:24453057,24452841:24453191,24452843:24453016,24452844:24453132,24452845:24453268,24452846:24452989,24452847:24452929,24491721:24493687,24491722:24493456,24491723:24493510,24491724:24493653,24491725:24493190,24491726:24493261,24491727:24493545,24491728:24493394,24491729:24493585,24491730:24493203,24491731:24492936,24491732:24492944,24491733:24493292,24491734:24493367,24491735:24493023,24491736:24493008,24491737:24493146,24491738:24493082,24491739:24493117,24491740:24492713,24491741:24492738,24491742:24492614,24491743:24493614,24491744:24492624,24491745:24492650,24491746:24492445,24491747:24492439,24491748:24492891,24491749:24492392,24491750:24492575,24491751:24493438,24491752:24492830,24491753:24492541,24491754:24492502,24491755:24493317,24491756:24492356,24491757:24492785,24491758:2449227', 'assoc_mat': '24452044:pet_clair:4230:24452270,24452047:pet_clair:4230:24452329,24452048:pet_clair:4230:24452457,24452049:pet_clair:4230:24452160,24452050:pet_clair:4230:24452369,24452051:pet_clair:4230:24452118,24452052:pet_clair:4230:24452078,24452770:pet_clair:4230:24453501,24452789:pet_clair:4230:24453348,24452808:pet_clair:4230:24453098,24452817:pet_clair:4230:24453175,24452828:pet_clair:4230:24453307,24452834:pet_clair:4230:24453057,24452841:pet_clair:4230:24453191,24452843:pet_clair:4230:24453016,24452844:pet_clair:4230:24453132,24452845:pet_clair:4230:24453268,24452846:pet_clair:4230:24452989,24452847:pet_clair:4230:24452929,24491721:pet_clair:4230:24493687,24491722:pet_clair:4230:24493456,24491723:pet_clair:4230:24493510,24491724:pet_clair:4230:24493653,24491725:pet_clair:4230:24493190,24491726:pet_clair:4230:24493261,24491727:pet_clair:4230:24493545,24491728:pet_clair:4230:24493394,24491729:pet_clair:4230:24493585,24491730:pet_clair:4230:24493203,24491731:pet_clair:4230:24492936,24491732:pet_clair:4230:24492944,2'}, {'unwanted_material': 'pet_fonce', 'main_material': 'truck', 'pht_type': 4230, 'ratio': 0.0012378756266419198, 'nb_photo': 1495, 'list_port_cont': '24452123,24452371,24452918,24452990,24492027,24492142,24492577,24492792,24492865,24492897,24493004,24493154,24493361,24493434,24503845,24503958,24510052,24510611,24510982,24511183,24511376,24511436,24511654,24511726,24517151,24517213,24517569,24517658,24517887,24518040,24518088,24518310,24518472,24521402,24527122,24573529,24573689,24573808,24574850,24574970,24581190,24581953,24595016,24595063,24595885,24595965,24596386,24596956,24597307,24597512,24597529,24597726,24598239,24598371,24598831,24599371,24604015,24604301,24605838,24605952,24605991,24606340,24606440,24606997,24610247,24610402,24610621,24610994,24611073,24611224,24611395,24611499,24615808,24615860,24615906,24616063,24616388,24616425,24616604,24616721,24616904,24639527,24640967,24645870,24718470,24720845,24722358,24722777,24723730,24723903,24723964,24724048,24724842,24725253,24725382,24725430,24725733,24725741,24725770,24726023,24726131,24726268,24726763,24726864,24727384,24727604,24727733,24728260,24728980,24729070,24730224,24731782,24731816,2473194', 'assoc_port': '24452050:24452371,24452051:24452123,24452846:24452990,24452847:24452918,24491734:24493361,24491736:24493004,24491737:24493154,24491748:24492897,24491750:24492577,24491751:24493434,24491757:24492792,24491764:24492142,24491766:24492027,24491770:24492865,24503599:24510052,24503609:24503958,24503611:24503845,24510372:24511654,24510375:24511726,24510376:24511376,24510389:24511436,24510404:24511183,24510422:24510982,24510457:24510611,24516257:24518310,24516260:24518472,24516264:24518040,24516268:24517887,24516317:24518088,24516333:24517658,24516340:24517151,24516344:24517569,24516347:24517213,24521123:24521402,24526361:24527122,24573203:24574970,24573211:24574850,24573253:24573808,24573299:24573689,24573300:24573529,24580565:24581953,24580574:24581190,24594212:24599371,24594214:24598239,24594215:24598371,24594216:24597726,24594217:24598831,24594218:24597512,24594219:24597529,24594233:24597307,24594267:24596386,24594292:24595063,24594318:24595016,24594322:24596956,24594394:24595965,24594496:24595885,24603092:2460699', 'assoc_mat': '24452050:pet_fonce:4230:24452371,24452051:pet_fonce:4230:24452123,24452846:pet_fonce:4230:24452990,24452847:pet_fonce:4230:24452918,24491734:pet_fonce:4230:24493361,24491736:pet_fonce:4230:24493004,24491737:pet_fonce:4230:24493154,24491748:pet_fonce:4230:24492897,24491750:pet_fonce:4230:24492577,24491751:pet_fonce:4230:24493434,24491757:pet_fonce:4230:24492792,24491764:pet_fonce:4230:24492142,24491766:pet_fonce:4230:24492027,24491770:pet_fonce:4230:24492865,24503599:pet_fonce:4230:24510052,24503609:pet_fonce:4230:24503958,24503611:pet_fonce:4230:24503845,24510372:pet_fonce:4230:24511654,24510375:pet_fonce:4230:24511726,24510376:pet_fonce:4230:24511376,24510389:pet_fonce:4230:24511436,24510404:pet_fonce:4230:24511183,24510422:pet_fonce:4230:24510982,24510457:pet_fonce:4230:24510611,24516257:pet_fonce:4230:24518310,24516260:pet_fonce:4230:24518472,24516264:pet_fonce:4230:24518040,24516268:pet_fonce:4230:24517887,24516317:pet_fonce:4230:24518088,24516333:pet_fonce:4230:24517658,24516340:pet_fonce:4230:24517151,2'}) select count(distinct mtr_photo_id) from MTRUser.mtr_portfolio_photos where mtr_portfolio_id in (select mtr_portfolio_id from MTRPhoto.dashboard_results where dashboard_run_id in(1952836,1956272,1959630,1962546,1964187,1964629,1969138,1970494,1973520,1980704,1980340,1982048,1982489,1986798,1990898,1995131,1997924,1999399,1999814,2004090,2007653,2015152,2015179,2018811,2020535,2021275,2029913,2043881,2042903,2051745)); after get_hostname_from_raspi hasthag : truck hasthag that could be used but not yet : _______refus SELECT substr(hashtag,8), substr(date_debut,1,7) as month, (sum(qualite*nombre_balle)/sum(nombre_balle)) FROM MTRPhoto.dashboard_results WHERE hashtag = '_______truck' and mtr_portfolio_id in (select id FROM MTRUser.mtr_portfolios where mtr_user_id=1055) and qualite >=0 and dashboard_run_id in (select last_run_id from MTRPhoto.dashboard_entry_day where dashboard_place_id=132 and date >='2025-07-01' and date <= '2025-07-31') GROUP BY month order by month; after impurety_average_per_hashtag {'number_of_analysed_photos': ((6397,),), 'unwanted_material_monthly_percentage': 0, 'material_repartition': (('truck', 6240),), 'material_dict': {'truck': 6240}, 'coverage_data': (('2025/07/01/', 0.245257),), 'impurety_average_data': ((datetime.date(2025, 7, 1), 0.21617695082265517, 306), (datetime.date(2025, 7, 2), 0.23391639519104637, 156), (datetime.date(2025, 7, 3), 0.2309142747675992, 345), (datetime.date(2025, 7, 4), 0.23377389854969996, 305), (datetime.date(2025, 7, 5), 0.20825018262090078, 81), (datetime.date(2025, 7, 6), 0.14708950562269027, 2), (datetime.date(2025, 7, 7), 0.22716136139430107, 261), (datetime.date(2025, 7, 8), 0.23984058294550722, 262), (datetime.date(2025, 7, 9), 0.2179782781873437, 266), (datetime.date(2025, 7, 10), 0.22977184483864538, 934), (datetime.date(2025, 7, 11), 0.2313726246794592, 291), (datetime.date(2025, 7, 12), 0.23038289261790657, 175), (datetime.date(2025, 7, 13), 0.19411187829688958, 67), (datetime.date(2025, 7, 14), 0.20748544251853415, 91), (datetime.date(2025, 7, 15), 0.22295557977417005, 279), (datetime.date(2025, 7, 17), 0.2212728525503619, 676), (datetime.date(2025, 7, 18), 0.24303894269617388, 96), (datetime.date(2025, 7, 19), 0.22593637946870926, 47), (datetime.date(2025, 7, 20), 0.1876579659229267, 76), (datetime.date(2025, 7, 21), 0.20516658996852352, 172), (datetime.date(2025, 7, 22), 0.24395799178730843, 93), (datetime.date(2025, 7, 23), 0.2325178322778279, 102), (datetime.date(2025, 7, 24), 0.22025147373288498, 229), (datetime.date(2025, 7, 25), 0.2133161588895479, 198), (datetime.date(2025, 7, 26), 0.24557527830036452, 8), (datetime.date(2025, 7, 27), 0.16258781082288543, 4), (datetime.date(2025, 7, 28), 0.2183676731043412, 237), (datetime.date(2025, 7, 29), 0.21482399029732788, 166), (datetime.date(2025, 7, 30), 0.2601588364040241, 315), (datetime.date(2025, 7, 31), 0.2025969936733917, 153)), 'impurety_average_data_per_hashtag': {'truck': ((datetime.date(2025, 7, 1), 0.21617695082265517, 306), (datetime.date(2025, 7, 2), 0.23391639519104637, 156), (datetime.date(2025, 7, 3), 0.2309142747675992, 345), (datetime.date(2025, 7, 4), 0.23377389854969996, 305), (datetime.date(2025, 7, 5), 0.20825018262090078, 81), (datetime.date(2025, 7, 6), 0.14708950562269027, 2), (datetime.date(2025, 7, 7), 0.22716136139430107, 261), (datetime.date(2025, 7, 8), 0.23984058294550722, 262), (datetime.date(2025, 7, 9), 0.2179782781873437, 266), (datetime.date(2025, 7, 10), 0.22977184483864538, 934), (datetime.date(2025, 7, 11), 0.2313726246794592, 291), (datetime.date(2025, 7, 12), 0.23038289261790657, 175), (datetime.date(2025, 7, 13), 0.19411187829688958, 67), (datetime.date(2025, 7, 14), 0.20748544251853415, 91), (datetime.date(2025, 7, 15), 0.22295557977417005, 279), (datetime.date(2025, 7, 17), 0.2212728525503619, 676), (datetime.date(2025, 7, 18), 0.24303894269617388, 96), (datetime.date(2025, 7, 19), 0.22593637946870926, 47), (datetime.date(2025, 7, 20), 0.1876579659229267, 76), (datetime.date(2025, 7, 21), 0.20516658996852352, 172), (datetime.date(2025, 7, 22), 0.24395799178730843, 93), (datetime.date(2025, 7, 23), 0.2325178322778279, 102), (datetime.date(2025, 7, 24), 0.22025147373288498, 229), (datetime.date(2025, 7, 25), 0.2133161588895479, 198), (datetime.date(2025, 7, 26), 0.24557527830036452, 8), (datetime.date(2025, 7, 27), 0.16258781082288543, 4), (datetime.date(2025, 7, 28), 0.2183676731043412, 237), (datetime.date(2025, 7, 29), 0.21482399029732788, 166), (datetime.date(2025, 7, 30), 0.2601588364040241, 315), (datetime.date(2025, 7, 31), 0.2025969936733917, 153))}, 'quality_array': [['-7.243963914921785', '01072025'], ['-7.158229466013983', '02072025'], ['-7.351509236787634', '03072025'], ['-7.246940222132847', '04072025'], ['-7.494813739703017', '05072025'], ['-7.65769630499743', '06072025'], ['-7.245878843938361', '07072025'], ['-7.221207053936817', '08072025'], ['-7.283650887240919', '09072025'], ['-7.28104286724847', '10072025'], ['-7.122696334505309', '11072025'], ['-7.3318713532800865', '12072025'], ['-7.327776823878171', '13072025'], ['-7.30786609843308', '14072025'], ['-7.43840772199447', '15072025'], ['-7.410263417047756', '17072025'], ['-7.410791131970419', '18072025'], ['-7.654342920923421', '19072025'], ['-7.408633716195079', '20072025'], ['-7.452248678811687', '21072025'], ['-7.461326122735549', '22072025'], ['-7.458392277112584', '23072025'], ['-7.2605141655753', '24072025'], ['-7.3306846500318175', '25072025'], ['-6.9764200474128675', '26072025'], ['-7.625860989331908', '27072025'], ['-7.400760148365198', '28072025'], ['-7.12615724623945', '29072025'], ['-7.363784901034534', '30072025'], ['-7.709085751580858', '31072025']], 'number_of_batch': ((1182,),), 'datou_id_sts': 4311, 'map_hashtag_number_mat': {'hash_to_mat_nb': ({'unwanted_material': 'autre', 'main_material': 'truck', 'pht_type': 4230, 'ratio': 0.02141215561237392, 'nb_photo': 89507, 'list_port_cont': '24452073,24452107,24452158,24452266,24452318,24452357,24452459,24452921,24452974,24453019,24453063,24453088,24453125,24453174,24453179,24453263,24453303,24453339,24453497,24491845,24491888,24491922,24491950,24491983,24492023,24492033,24492091,24492132,24492163,24492200,24492242,24492282,24492312,24492345,24492396,24492426,24492451,24492494,24492534,24492562,24492608,24492627,24492634,24492721,24492728,24492780,24492825,24492850,24492885,24492922,24492947,24493007,24493017,24493079,24493115,24493150,24493191,24493198,24493255,24493291,24493312,24493371,24493403,24493443,24493444,24493508,24493536,24493582,24493607,24493659,24493688,24500172,24500294,24500390,24502680,24503839,24503904,24503949,24503994,24504119,24504194,24504311,24504420,24504452,24510008,24510035,24510092,24510599,24510641,24510741,24510868,24510950,24510976,24511121,24511180,24511196,24511284,24511372,24511429,24511659,24511718,24511855,24512039,24517153,24517218,24517557,24517644,24517897,24518045,24518070,24518321,24518460,24521395,2452144', 'assoc_port': '24452044:24452266,24452047:24452318,24452048:24452459,24452049:24452158,24452050:24452357,24452051:24452107,24452052:24452073,24452770:24453497,24452789:24453339,24452808:24453088,24452817:24453174,24452828:24453303,24452834:24453063,24452841:24453179,24452843:24453019,24452844:24453125,24452845:24453263,24452846:24452974,24452847:24452921,24491721:24493688,24491722:24493444,24491723:24493508,24491724:24493659,24491725:24493191,24491726:24493255,24491727:24493536,24491728:24493403,24491729:24493582,24491730:24493198,24491731:24492922,24491732:24492947,24491733:24493291,24491734:24493371,24491735:24493017,24491736:24493007,24491737:24493150,24491738:24493079,24491739:24493115,24491740:24492721,24491741:24492728,24491742:24492608,24491743:24493607,24491744:24492634,24491745:24492627,24491746:24492451,24491747:24492426,24491748:24492885,24491749:24492396,24491750:24492562,24491751:24493443,24491752:24492825,24491753:24492534,24491754:24492494,24491755:24493312,24491756:24492345,24491757:24492780,24491758:2449228', 'assoc_mat': '24452044:autre:4230:24452266,24452047:autre:4230:24452318,24452048:autre:4230:24452459,24452049:autre:4230:24452158,24452050:autre:4230:24452357,24452051:autre:4230:24452107,24452052:autre:4230:24452073,24452770:autre:4230:24453497,24452789:autre:4230:24453339,24452808:autre:4230:24453088,24452817:autre:4230:24453174,24452828:autre:4230:24453303,24452834:autre:4230:24453063,24452841:autre:4230:24453179,24452843:autre:4230:24453019,24452844:autre:4230:24453125,24452845:autre:4230:24453263,24452846:autre:4230:24452974,24452847:autre:4230:24452921,24491721:autre:4230:24493688,24491722:autre:4230:24493444,24491723:autre:4230:24493508,24491724:autre:4230:24493659,24491725:autre:4230:24493191,24491726:autre:4230:24493255,24491727:autre:4230:24493536,24491728:autre:4230:24493403,24491729:autre:4230:24493582,24491730:autre:4230:24493198,24491731:autre:4230:24492922,24491732:autre:4230:24492947,24491733:autre:4230:24493291,24491734:autre:4230:24493371,24491735:autre:4230:24493017,24491736:autre:4230:24493007,24491737:'}, {'unwanted_material': 'carton', 'main_material': 'truck', 'pht_type': 4230, 'ratio': 0.029727232594059218, 'nb_photo': 117018, 'list_port_cont': '24452076,24452119,24452164,24452272,24452325,24452374,24452449,24452930,24452985,24453026,24453048,24453102,24453142,24453177,24453196,24453264,24453306,24453346,24453502,24491854,24491884,24491923,24491962,24491986,24492032,24492053,24492103,24492133,24492174,24492194,24492252,24492278,24492322,24492359,24492388,24492429,24492450,24492490,24492536,24492574,24492604,24492638,24492647,24492707,24492737,24492784,24492820,24492855,24492902,24492934,24492950,24493002,24493026,24493077,24493116,24493152,24493185,24493207,24493254,24493296,24493309,24493370,24493399,24493435,24493453,24493500,24493551,24493583,24493615,24493654,24493692,24500179,24500296,24500395,24502681,24503840,24503906,24503951,24503999,24504129,24504210,24504320,24504426,24504459,24510002,24510048,24510110,24510613,24510653,24510750,24510865,24510948,24510990,24511118,24511177,24511194,24511297,24511384,24511437,24511656,24511727,24511856,24512032,24517157,24517208,24517570,24517660,24517899,24518046,24518080,24518312,24518466,24521403,2452143', 'assoc_port': '24452044:24452272,24452047:24452325,24452048:24452449,24452049:24452164,24452050:24452374,24452051:24452119,24452052:24452076,24452770:24453502,24452789:24453346,24452808:24453102,24452817:24453177,24452828:24453306,24452834:24453048,24452841:24453196,24452843:24453026,24452844:24453142,24452845:24453264,24452846:24452985,24452847:24452930,24491721:24493692,24491722:24493453,24491723:24493500,24491724:24493654,24491725:24493185,24491726:24493254,24491727:24493551,24491728:24493399,24491729:24493583,24491730:24493207,24491731:24492934,24491732:24492950,24491733:24493296,24491734:24493370,24491735:24493026,24491736:24493002,24491737:24493152,24491738:24493077,24491739:24493116,24491740:24492707,24491741:24492737,24491742:24492604,24491743:24493615,24491744:24492638,24491745:24492647,24491746:24492450,24491747:24492429,24491748:24492902,24491749:24492388,24491750:24492574,24491751:24493435,24491752:24492820,24491753:24492536,24491754:24492490,24491755:24493309,24491756:24492359,24491757:24492784,24491758:2449227', 'assoc_mat': '24452044:carton:4230:24452272,24452047:carton:4230:24452325,24452048:carton:4230:24452449,24452049:carton:4230:24452164,24452050:carton:4230:24452374,24452051:carton:4230:24452119,24452052:carton:4230:24452076,24452770:carton:4230:24453502,24452789:carton:4230:24453346,24452808:carton:4230:24453102,24452817:carton:4230:24453177,24452828:carton:4230:24453306,24452834:carton:4230:24453048,24452841:carton:4230:24453196,24452843:carton:4230:24453026,24452844:carton:4230:24453142,24452845:carton:4230:24453264,24452846:carton:4230:24452985,24452847:carton:4230:24452930,24491721:carton:4230:24493692,24491722:carton:4230:24493453,24491723:carton:4230:24493500,24491724:carton:4230:24493654,24491725:carton:4230:24493185,24491726:carton:4230:24493254,24491727:carton:4230:24493551,24491728:carton:4230:24493399,24491729:carton:4230:24493583,24491730:carton:4230:24493207,24491731:carton:4230:24492934,24491732:carton:4230:24492950,24491733:carton:4230:24493296,24491734:carton:4230:24493370,24491735:carton:4230:24493026,2449'}, {'unwanted_material': 'metal', 'main_material': 'truck', 'pht_type': 4230, 'ratio': 0.008217845252983105, 'nb_photo': 4223, 'list_port_cont': '24452087,24452124,24452155,24452330,24452364,24452448,24452913,24452973,24453024,24453049,24453093,24453135,24453178,24453195,24453265,24453304,24453343,24491840,24491876,24491992,24492052,24492167,24492272,24492308,24492360,24492384,24492443,24492501,24492532,24492576,24492629,24492632,24492791,24492819,24492852,24492886,24492948,24493006,24493080,24493293,24493307,24493359,24493461,24493580,24493622,24493655,24493690,24500181,24500306,24500381,24502684,24503851,24503916,24503953,24503995,24504319,24504411,24504462,24510642,24510746,24510862,24510946,24511106,24511178,24511288,24511375,24511432,24511644,24511719,24511843,24512035,24517215,24517571,24517659,24517895,24518090,24518315,24521427,24525622,24526572,24526686,24527125,24567355,24567453,24567582,24567598,24567722,24568066,24573520,24573582,24573677,24573736,24573817,24573818,24573950,24574029,24574099,24574166,24574283,24574427,24574600,24574659,24574729,24574779,24574844,24575508,24575656,24575726,24575736,24575937,24576168,24576436,24576754,2458118', 'assoc_port': '24452047:24452330,24452048:24452448,24452049:24452155,24452050:24452364,24452051:24452124,24452052:24452087,24452789:24453343,24452808:24453093,24452817:24453178,24452828:24453304,24452834:24453049,24452841:24453195,24452843:24453024,24452844:24453135,24452845:24453265,24452846:24452973,24452847:24452913,24491721:24493690,24491722:24493461,24491724:24493655,24491729:24493580,24491732:24492948,24491733:24493293,24491734:24493359,24491736:24493006,24491738:24493080,24491743:24493622,24491744:24492632,24491745:24492629,24491747:24492443,24491748:24492886,24491749:24492384,24491750:24492576,24491752:24492819,24491753:24492532,24491754:24492501,24491755:24493307,24491756:24492360,24491757:24492791,24491758:24492272,24491759:24492308,24491760:24492167,24491763:24491840,24491767:24491992,24491770:24492852,24491771:24491876,24491772:24492052,24499974:24500381,24499989:24500306,24500004:24502684,24500008:24500181,24503600:24504411,24503602:24504462,24503606:24503995,24503607:24504319,24503609:24503953,24503611:2450385', 'assoc_mat': '24452047:metal:4230:24452330,24452048:metal:4230:24452448,24452049:metal:4230:24452155,24452050:metal:4230:24452364,24452051:metal:4230:24452124,24452052:metal:4230:24452087,24452789:metal:4230:24453343,24452808:metal:4230:24453093,24452817:metal:4230:24453178,24452828:metal:4230:24453304,24452834:metal:4230:24453049,24452841:metal:4230:24453195,24452843:metal:4230:24453024,24452844:metal:4230:24453135,24452845:metal:4230:24453265,24452846:metal:4230:24452973,24452847:metal:4230:24452913,24491721:metal:4230:24493690,24491722:metal:4230:24493461,24491724:metal:4230:24493655,24491729:metal:4230:24493580,24491732:metal:4230:24492948,24491733:metal:4230:24493293,24491734:metal:4230:24493359,24491736:metal:4230:24493006,24491738:metal:4230:24493080,24491743:metal:4230:24493622,24491744:metal:4230:24492632,24491745:metal:4230:24492629,24491747:metal:4230:24492443,24491748:metal:4230:24492886,24491749:metal:4230:24492384,24491750:metal:4230:24492576,24491752:metal:4230:24492819,24491753:metal:4230:24492532,24491754:'}, {'unwanted_material': 'papier', 'main_material': 'truck', 'pht_type': 4230, 'ratio': 0.15884656879218992, 'nb_photo': 306144, 'list_port_cont': '24452083,24452117,24452162,24452264,24452334,24452372,24452456,24452923,24452983,24453022,24453055,24453103,24453131,24453168,24453194,24453271,24453300,24453354,24453506,24491856,24491891,24491921,24491961,24491990,24492034,24492048,24492107,24492137,24492178,24492211,24492247,24492281,24492323,24492351,24492391,24492441,24492442,24492504,24492540,24492565,24492610,24492636,24492641,24492711,24492730,24492793,24492824,24492856,24492894,24492935,24492955,24493010,24493028,24493076,24493114,24493149,24493183,24493206,24493258,24493295,24493306,24493365,24493401,24493441,24493458,24493504,24493548,24493578,24493623,24493651,24493685,24500178,24500305,24500389,24502674,24503847,24503924,24503956,24503988,24504133,24504205,24504314,24504423,24504458,24510006,24510047,24510107,24510615,24510651,24510751,24510869,24510943,24510980,24511117,24511176,24511189,24511293,24511383,24511431,24511662,24511721,24511849,24512038,24517160,24517207,24517572,24517647,24517904,24518034,24518082,24518314,24518469,24521401,2452143', 'assoc_port': '24452044:24452264,24452047:24452334,24452048:24452456,24452049:24452162,24452050:24452372,24452051:24452117,24452052:24452083,24452770:24453506,24452789:24453354,24452808:24453103,24452817:24453168,24452828:24453300,24452834:24453055,24452841:24453194,24452843:24453022,24452844:24453131,24452845:24453271,24452846:24452983,24452847:24452923,24491721:24493685,24491722:24493458,24491723:24493504,24491724:24493651,24491725:24493183,24491726:24493258,24491727:24493548,24491728:24493401,24491729:24493578,24491730:24493206,24491731:24492935,24491732:24492955,24491733:24493295,24491734:24493365,24491735:24493028,24491736:24493010,24491737:24493149,24491738:24493076,24491739:24493114,24491740:24492711,24491741:24492730,24491742:24492610,24491743:24493623,24491744:24492636,24491745:24492641,24491746:24492442,24491747:24492441,24491748:24492894,24491749:24492391,24491750:24492565,24491751:24493441,24491752:24492824,24491753:24492540,24491754:24492504,24491755:24493306,24491756:24492351,24491757:24492793,24491758:2449228', 'assoc_mat': '24452044:papier:4230:24452264,24452047:papier:4230:24452334,24452048:papier:4230:24452456,24452049:papier:4230:24452162,24452050:papier:4230:24452372,24452051:papier:4230:24452117,24452052:papier:4230:24452083,24452770:papier:4230:24453506,24452789:papier:4230:24453354,24452808:papier:4230:24453103,24452817:papier:4230:24453168,24452828:papier:4230:24453300,24452834:papier:4230:24453055,24452841:papier:4230:24453194,24452843:papier:4230:24453022,24452844:papier:4230:24453131,24452845:papier:4230:24453271,24452846:papier:4230:24452983,24452847:papier:4230:24452923,24491721:papier:4230:24493685,24491722:papier:4230:24493458,24491723:papier:4230:24493504,24491724:papier:4230:24493651,24491725:papier:4230:24493183,24491726:papier:4230:24493258,24491727:papier:4230:24493548,24491728:papier:4230:24493401,24491729:papier:4230:24493578,24491730:papier:4230:24493206,24491731:papier:4230:24492935,24491732:papier:4230:24492955,24491733:papier:4230:24493295,24491734:papier:4230:24493365,24491735:papier:4230:24493028,2449'}, {'unwanted_material': 'pehd', 'main_material': 'truck', 'pht_type': 4230, 'ratio': 0.00121672865586385, 'nb_photo': 1319, 'list_port_cont': '24453140,24453171,24453186,24453315,24453505,24491892,24491963,24491993,24492179,24492355,24492389,24492505,24492538,24492612,24492788,24492826,24492863,24492946,24493189,24493205,24493315,24493429,24493494,24493619,24493657,24502682,24503848,24503910,24503961,24510639,24510739,24510873,24510987,24511119,24511179,24511203,24511298,24511380,24511434,24511663,24511725,24512033,24517149,24517573,24518473,24526698,24567451,24574165,24574788,24574852,24576438,24581180,24581383,24581954,24582113,24595010,24595062,24595967,24596294,24596383,24596819,24596856,24596893,24596957,24597249,24597284,24597381,24597425,24597516,24597535,24597721,24598368,24598824,24599382,24603645,24603800,24604385,24604512,24605508,24605976,24606200,24606346,24610871,24611080,24611212,24611317,24611385,24615381,24615421,24615540,24615672,24615802,24615909,24616159,24616417,24616603,24616722,24616894,24616967,24617088,24620087,24640957,24641933,24641968,24645871,24646222,24717794,24718476,24721008,24721885,24722001,24722686,24722817,2472373', 'assoc_port': '24452770:24453505,24452817:24453171,24452828:24453315,24452841:24453186,24452844:24453140,24491723:24493494,24491724:24493657,24491725:24493189,24491730:24493205,24491732:24492946,24491742:24492612,24491743:24493619,24491749:24492389,24491751:24493429,24491752:24492826,24491753:24492538,24491754:24492505,24491755:24493315,24491756:24492355,24491757:24492788,24491760:24492179,24491767:24491993,24491769:24491963,24491770:24492863,24491771:24491892,24500004:24502682,24503609:24503961,24503611:24503848,24503613:24503910,24510372:24511663,24510374:24512033,24510375:24511725,24510376:24511380,24510389:24511434,24510402:24511298,24510403:24511119,24510404:24511179,24510405:24511203,24510406:24510739,24510422:24510987,24510437:24510873,24510452:24510639,24516260:24518473,24516340:24517149,24516344:24517573,24526363:24526698,24567050:24567451,24573197:24576438,24573209:24574788,24573211:24574852,24573283:24574165,24580565:24581954,24580570:24581383,24580574:24581180,24580579:24582113,24594212:24599382,24594215:2459836', 'assoc_mat': '24452770:pehd:4230:24453505,24452817:pehd:4230:24453171,24452828:pehd:4230:24453315,24452841:pehd:4230:24453186,24452844:pehd:4230:24453140,24491723:pehd:4230:24493494,24491724:pehd:4230:24493657,24491725:pehd:4230:24493189,24491730:pehd:4230:24493205,24491732:pehd:4230:24492946,24491742:pehd:4230:24492612,24491743:pehd:4230:24493619,24491749:pehd:4230:24492389,24491751:pehd:4230:24493429,24491752:pehd:4230:24492826,24491753:pehd:4230:24492538,24491754:pehd:4230:24492505,24491755:pehd:4230:24493315,24491756:pehd:4230:24492355,24491757:pehd:4230:24492788,24491760:pehd:4230:24492179,24491767:pehd:4230:24491993,24491769:pehd:4230:24491963,24491770:pehd:4230:24492863,24491771:pehd:4230:24491892,24500004:pehd:4230:24502682,24503609:pehd:4230:24503961,24503611:pehd:4230:24503848,24503613:pehd:4230:24503910,24510372:pehd:4230:24511663,24510374:pehd:4230:24512033,24510375:pehd:4230:24511725,24510376:pehd:4230:24511380,24510389:pehd:4230:24511434,24510402:pehd:4230:24511298,24510403:pehd:4230:24511119,24510404:pehd:42'}, {'unwanted_material': 'pet_clair', 'main_material': 'truck', 'pht_type': 4230, 'ratio': 0.011905527100546353, 'nb_photo': 53012, 'list_port_cont': '24452078,24452118,24452160,24452270,24452329,24452369,24452457,24452929,24452989,24453016,24453057,24453098,24453132,24453175,24453191,24453268,24453307,24453348,24453501,24491846,24491881,24491927,24491959,24491997,24492036,24492047,24492104,24492140,24492176,24492214,24492251,24492279,24492317,24492356,24492392,24492439,24492445,24492502,24492541,24492575,24492614,24492624,24492650,24492713,24492738,24492785,24492830,24492864,24492891,24492936,24492944,24493008,24493023,24493082,24493117,24493146,24493190,24493203,24493261,24493292,24493317,24493367,24493394,24493438,24493456,24493510,24493545,24493585,24493614,24493653,24493687,24500188,24500301,24500391,24502678,24503841,24503900,24503962,24503993,24504132,24504211,24504313,24504424,24504460,24510003,24510049,24510084,24510610,24510655,24510743,24510871,24510951,24510988,24511120,24511174,24511202,24511295,24511382,24511433,24511664,24511715,24511848,24512037,24517141,24517210,24517567,24517655,24517896,24518032,24518094,24518318,24518471,24521400,2452143', 'assoc_port': '24452044:24452270,24452047:24452329,24452048:24452457,24452049:24452160,24452050:24452369,24452051:24452118,24452052:24452078,24452770:24453501,24452789:24453348,24452808:24453098,24452817:24453175,24452828:24453307,24452834:24453057,24452841:24453191,24452843:24453016,24452844:24453132,24452845:24453268,24452846:24452989,24452847:24452929,24491721:24493687,24491722:24493456,24491723:24493510,24491724:24493653,24491725:24493190,24491726:24493261,24491727:24493545,24491728:24493394,24491729:24493585,24491730:24493203,24491731:24492936,24491732:24492944,24491733:24493292,24491734:24493367,24491735:24493023,24491736:24493008,24491737:24493146,24491738:24493082,24491739:24493117,24491740:24492713,24491741:24492738,24491742:24492614,24491743:24493614,24491744:24492624,24491745:24492650,24491746:24492445,24491747:24492439,24491748:24492891,24491749:24492392,24491750:24492575,24491751:24493438,24491752:24492830,24491753:24492541,24491754:24492502,24491755:24493317,24491756:24492356,24491757:24492785,24491758:2449227', 'assoc_mat': '24452044:pet_clair:4230:24452270,24452047:pet_clair:4230:24452329,24452048:pet_clair:4230:24452457,24452049:pet_clair:4230:24452160,24452050:pet_clair:4230:24452369,24452051:pet_clair:4230:24452118,24452052:pet_clair:4230:24452078,24452770:pet_clair:4230:24453501,24452789:pet_clair:4230:24453348,24452808:pet_clair:4230:24453098,24452817:pet_clair:4230:24453175,24452828:pet_clair:4230:24453307,24452834:pet_clair:4230:24453057,24452841:pet_clair:4230:24453191,24452843:pet_clair:4230:24453016,24452844:pet_clair:4230:24453132,24452845:pet_clair:4230:24453268,24452846:pet_clair:4230:24452989,24452847:pet_clair:4230:24452929,24491721:pet_clair:4230:24493687,24491722:pet_clair:4230:24493456,24491723:pet_clair:4230:24493510,24491724:pet_clair:4230:24493653,24491725:pet_clair:4230:24493190,24491726:pet_clair:4230:24493261,24491727:pet_clair:4230:24493545,24491728:pet_clair:4230:24493394,24491729:pet_clair:4230:24493585,24491730:pet_clair:4230:24493203,24491731:pet_clair:4230:24492936,24491732:pet_clair:4230:24492944,2'}, {'unwanted_material': 'pet_fonce', 'main_material': 'truck', 'pht_type': 4230, 'ratio': 0.0012378756266419198, 'nb_photo': 1495, 'list_port_cont': '24452123,24452371,24452918,24452990,24492027,24492142,24492577,24492792,24492865,24492897,24493004,24493154,24493361,24493434,24503845,24503958,24510052,24510611,24510982,24511183,24511376,24511436,24511654,24511726,24517151,24517213,24517569,24517658,24517887,24518040,24518088,24518310,24518472,24521402,24527122,24573529,24573689,24573808,24574850,24574970,24581190,24581953,24595016,24595063,24595885,24595965,24596386,24596956,24597307,24597512,24597529,24597726,24598239,24598371,24598831,24599371,24604015,24604301,24605838,24605952,24605991,24606340,24606440,24606997,24610247,24610402,24610621,24610994,24611073,24611224,24611395,24611499,24615808,24615860,24615906,24616063,24616388,24616425,24616604,24616721,24616904,24639527,24640967,24645870,24718470,24720845,24722358,24722777,24723730,24723903,24723964,24724048,24724842,24725253,24725382,24725430,24725733,24725741,24725770,24726023,24726131,24726268,24726763,24726864,24727384,24727604,24727733,24728260,24728980,24729070,24730224,24731782,24731816,2473194', 'assoc_port': '24452050:24452371,24452051:24452123,24452846:24452990,24452847:24452918,24491734:24493361,24491736:24493004,24491737:24493154,24491748:24492897,24491750:24492577,24491751:24493434,24491757:24492792,24491764:24492142,24491766:24492027,24491770:24492865,24503599:24510052,24503609:24503958,24503611:24503845,24510372:24511654,24510375:24511726,24510376:24511376,24510389:24511436,24510404:24511183,24510422:24510982,24510457:24510611,24516257:24518310,24516260:24518472,24516264:24518040,24516268:24517887,24516317:24518088,24516333:24517658,24516340:24517151,24516344:24517569,24516347:24517213,24521123:24521402,24526361:24527122,24573203:24574970,24573211:24574850,24573253:24573808,24573299:24573689,24573300:24573529,24580565:24581953,24580574:24581190,24594212:24599371,24594214:24598239,24594215:24598371,24594216:24597726,24594217:24598831,24594218:24597512,24594219:24597529,24594233:24597307,24594267:24596386,24594292:24595063,24594318:24595016,24594322:24596956,24594394:24595965,24594496:24595885,24603092:2460699', 'assoc_mat': '24452050:pet_fonce:4230:24452371,24452051:pet_fonce:4230:24452123,24452846:pet_fonce:4230:24452990,24452847:pet_fonce:4230:24452918,24491734:pet_fonce:4230:24493361,24491736:pet_fonce:4230:24493004,24491737:pet_fonce:4230:24493154,24491748:pet_fonce:4230:24492897,24491750:pet_fonce:4230:24492577,24491751:pet_fonce:4230:24493434,24491757:pet_fonce:4230:24492792,24491764:pet_fonce:4230:24492142,24491766:pet_fonce:4230:24492027,24491770:pet_fonce:4230:24492865,24503599:pet_fonce:4230:24510052,24503609:pet_fonce:4230:24503958,24503611:pet_fonce:4230:24503845,24510372:pet_fonce:4230:24511654,24510375:pet_fonce:4230:24511726,24510376:pet_fonce:4230:24511376,24510389:pet_fonce:4230:24511436,24510404:pet_fonce:4230:24511183,24510422:pet_fonce:4230:24510982,24510457:pet_fonce:4230:24510611,24516257:pet_fonce:4230:24518310,24516260:pet_fonce:4230:24518472,24516264:pet_fonce:4230:24518040,24516268:pet_fonce:4230:24517887,24516317:pet_fonce:4230:24518088,24516333:pet_fonce:4230:24517658,24516340:pet_fonce:4230:24517151,2'}), 'main_stat': {'nb_photos': 6397}}, 'hostname': (('raspi-84',),), 'impurety_average_per_hashtag': []} VR TODO TOCHECK : due to this bug it shouldn't have being able to work, or maybe it was due to a change directory not done ! SELECT name, created_at,limit_max FROM MTRDatou.mtr_datou WHERE id=4311 SELECT mtd.id, mtdt.`type`, mtd.`param`, mtd.param_json, mtdt.nb_input, mtdt.nb_output, mtdt.prod, mtdt.is_local, mtdt.is_datou_depend, mtdt.is_photo_id_local FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_step_types mtdt WHERE mtdt.`id`=mtd.`type` AND mtd.mtd_id=4311 SELECT mtd.id, mtd.mtd_id, mdsdt.id, mdsdt.name, mdsdt.description, msid.output_or_input, msid.data_order_id, mdsdt.type FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_steptype_io_datatypes msid, MTRDatou.mtr_datou_step_data_types mdsdt WHERE mtd.`type`=msid.`mtr_datou_step_type` AND mtd.mtd_id= 4311 AND msid.data_type=mdsdt.id SELECT mts_id_output, id_output, mts_id_input, id_input FROM MTRDatou.mtr_datou_step_by_step WHERE mtd_id=4311 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11939 matching_dashboard is not consistent : 1 used against 0 in the step definition ! Step 11940 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 0 of step 11939 doesn't seem to be define in the database( WARNING : type of input 2 of step 11940 doesn't seem to be define in the database( DataTypes for each output/input checked ! {"input_values":[]} We expected an STS datou with a camera_place_name, so there is a PB, datou_id: 4311 ! None was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? {'id': '4311', 'name': 'sts_dechet_valcor_matching_v2_2022Q3', 'limit_max': 100, 'nb_steps': 2, 'estimated_time': 8640, 'list_steps': [{'id': 11939, 'type': 'matching_dashboard', 'param': '0', 'json_param_as_string': "{'camera_matching': 'plaque_Valcor', 'max_time_diff': 120, 'matching_method': '1_vs_n', 'input_default_values': []}", 'json_param': {'camera_matching': 'plaque_Valcor', 'max_time_diff': 120, 'matching_method': '1_vs_n', 'input_default_values': []}, 'nb_input': 1, 'nb_output': 0, 'prod': False, 'is_local': False, 'is_datou_depend': False, 'is_photo_id_local': False, 'list_data': [{'step_id': 11939, 'datou_id': 4311, 'type_id': 18, 'name': 'portfolio_id', 'output_or_input': False, 'data_order_id': 0, 'type_python': 'int'}]}, {'id': 11940, 'type': 'split_time_score', 'param': '0', 'json_param_as_string': "{'camera_place_name': 'dechet_Valcor_v2', 'ETA': 8640, 'param_moyenne_mobile': 5, 'thcl': 3442, 'list_task_complete': [''], 'list_tasks': 'LOAD,ORDER,CONVENTION,SPLIT_TRUCK,merge_port_according_to_plate_info,remove_no_plate_batch,CREATE_PORT_LAUNCH_CARAC_FILL_DASHBOARD', 'forced_hashtag': 'truck', 'list_hashtag_class_create': 'truck', 'ponderation': '0.001,0.001,0.001,0.001,0.001', 'host': 'www.fotonower.com', 'datou_proportion_papier': {'test': 370, 'truck': 4234}, 'datou_matching_id': 4311, 'cod': 1, 'min_number_photo_per_batch': 2, 'datou_truck_for_info': 4234}", 'json_param': {'camera_place_name': 'dechet_Valcor_v2', 'ETA': 8640, 'param_moyenne_mobile': 5, 'thcl': 3442, 'list_task_complete': [''], 'list_tasks': 'LOAD,ORDER,CONVENTION,SPLIT_TRUCK,merge_port_according_to_plate_info,remove_no_plate_batch,CREATE_PORT_LAUNCH_CARAC_FILL_DASHBOARD', 'forced_hashtag': 'truck', 'list_hashtag_class_create': 'truck', 'ponderation': '0.001,0.001,0.001,0.001,0.001', 'host': 'www.fotonower.com', 'datou_proportion_papier': {'test': 370, 'truck': 4234}, 'datou_matching_id': 4311, 'cod': 1, 'min_number_photo_per_batch': 2, 'datou_truck_for_info': 4234}, 'nb_input': 2, 'nb_output': 1, 'prod': False, 'is_local': False, 'is_datou_depend': False, 'is_photo_id_local': False, 'list_data': [{'step_id': 11940, 'datou_id': 4311, 'type_id': 18, 'name': 'portfolio_id', 'output_or_input': False, 'data_order_id': 0, 'type_python': 'int'}, {'step_id': 11940, 'datou_id': 4311, 'type_id': 10, 'name': 'data_string', 'output_or_input': False, 'data_order_id': 1, 'type_python': 'string'}, {'step_id': 11940, 'datou_id': 4311, 'type_id': 22, 'name': 'portfolios_ids_list', 'output_or_input': True, 'data_order_id': 0, 'type_python': 'liste_de_int'}]}], 'depends_step': [{'output': 11939, 'output_id': 0, 'input': 11940, 'input_id': 2}]} check if it is the case where we have multiple sts in one datou no, it is not this case SELECT name, created_at,limit_max FROM MTRDatou.mtr_datou WHERE id=370 SELECT mtd.id, mtdt.`type`, mtd.`param`, mtd.param_json, mtdt.nb_input, mtdt.nb_output, mtdt.prod, mtdt.is_local, mtdt.is_datou_depend, mtdt.is_photo_id_local FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_step_types mtdt WHERE mtdt.`id`=mtd.`type` AND mtd.mtd_id=370 SELECT mtd.id, mtd.mtd_id, mdsdt.id, mdsdt.name, mdsdt.description, msid.output_or_input, msid.data_order_id, mdsdt.type FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_steptype_io_datatypes msid, MTRDatou.mtr_datou_step_data_types mdsdt WHERE mtd.`type`=msid.`mtr_datou_step_type` AND mtd.mtd_id= 370 AND msid.data_type=mdsdt.id SELECT mts_id_output, id_output, mts_id_input, id_input FROM MTRDatou.mtr_datou_step_by_step WHERE mtd_id=370 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! DataTypes for each output/input checked ! {"input_values":[]} Unexpected error treated as WARNING for now expected given param in final step {'min_score': 0.7, 'name_pipeline': 'proportion_papier_tapis', 'hashtag_type': 705, 'hashtag_proportion': 492668766, 'hashtag_parmi': 538914404, 'hashtag_to_consider': 'papier_only', 'hashtag_background': 493012381} Found hashtag_parmi in final step, will be used and useful if different from matiere_majoritaire in classifier ventilate_hashtags_in_portfolio step not found ! SELECT name, created_at,limit_max FROM MTRDatou.mtr_datou WHERE id=4234 SELECT mtd.id, mtdt.`type`, mtd.`param`, mtd.param_json, mtdt.nb_input, mtdt.nb_output, mtdt.prod, mtdt.is_local, mtdt.is_datou_depend, mtdt.is_photo_id_local FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_step_types mtdt WHERE mtdt.`id`=mtd.`type` AND mtd.mtd_id=4234 SELECT mtd.id, mtd.mtd_id, mdsdt.id, mdsdt.name, mdsdt.description, msid.output_or_input, msid.data_order_id, mdsdt.type FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_steptype_io_datatypes msid, MTRDatou.mtr_datou_step_data_types mdsdt WHERE mtd.`type`=msid.`mtr_datou_step_type` AND mtd.mtd_id= 4234 AND msid.data_type=mdsdt.id SELECT mts_id_output, id_output, mts_id_input, id_input FROM MTRDatou.mtr_datou_step_by_step WHERE mtd_id=4234 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! {"input_values":[]} Unexpected error treated as WARNING for now expected given param in final step {'name_pipeline': 'aggregate_carac_ratio', 'hashtag_type': 4230, 'hashtag_proportion': 'papier,carton,metal,pet_clair,autre,pehd,pet_fonce', 'hashtag_parmi': 'refus', 'hashtag_weights': {'autre': 3.0, 'barquette_avec_film': 3.0, 'carton': 3.0, 'metal': 3.0, 'papier': 3.0, 'pehd': 3.0, 'pet_fonce': 3.0, 'pet_clair': 1.0, 'flou': 1.0, 'mal_croppe': 1.0}, 'option_sccu': 1} Found hashtag_parmi in final step, will be used and useful if different from matiere_majoritaire in classifier TODO : Insert select and so on SELECT id, name, misc_info, datou_id_for_right FROM MTRPhoto.dashboard_places WHERE name="dechet_Valcor_v2"; SELECT name, created_at,limit_max FROM MTRDatou.mtr_datou WHERE id=4311 SELECT mtd.id, mtdt.`type`, mtd.`param`, mtd.param_json, mtdt.nb_input, mtdt.nb_output, mtdt.prod, mtdt.is_local, mtdt.is_datou_depend, mtdt.is_photo_id_local FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_step_types mtdt WHERE mtdt.`id`=mtd.`type` AND mtd.mtd_id=4311 SELECT mtd.id, mtd.mtd_id, mdsdt.id, mdsdt.name, mdsdt.description, msid.output_or_input, msid.data_order_id, mdsdt.type FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_steptype_io_datatypes msid, MTRDatou.mtr_datou_step_data_types mdsdt WHERE mtd.`type`=msid.`mtr_datou_step_type` AND mtd.mtd_id= 4311 AND msid.data_type=mdsdt.id SELECT mts_id_output, id_output, mts_id_input, id_input FROM MTRDatou.mtr_datou_step_by_step WHERE mtd_id=4311 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11939 matching_dashboard is not consistent : 1 used against 0 in the step definition ! Step 11940 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 0 of step 11939 doesn't seem to be define in the database( WARNING : type of input 2 of step 11940 doesn't seem to be define in the database( DataTypes for each output/input checked ! {"input_values":[]} We expected an STS datou with a camera_place_name, so there is a PB, datou_id: 4311 ! None was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? {'id': 4311, 'name': 'sts_dechet_valcor_matching_v2_2022Q3', 'limit_max': 100, 'nb_steps': 2, 'estimated_time': 8640, 'list_steps': [{'id': 11939, 'type': 'matching_dashboard', 'param': '0', 'json_param_as_string': "{'camera_matching': 'plaque_Valcor', 'max_time_diff': 120, 'matching_method': '1_vs_n', 'input_default_values': []}", 'json_param': {'camera_matching': 'plaque_Valcor', 'max_time_diff': 120, 'matching_method': '1_vs_n', 'input_default_values': []}, 'nb_input': 1, 'nb_output': 0, 'prod': False, 'is_local': False, 'is_datou_depend': False, 'is_photo_id_local': False, 'list_data': [{'step_id': 11939, 'datou_id': 4311, 'type_id': 18, 'name': 'portfolio_id', 'output_or_input': False, 'data_order_id': 0, 'type_python': 'int'}]}, {'id': 11940, 'type': 'split_time_score', 'param': '0', 'json_param_as_string': "{'camera_place_name': 'dechet_Valcor_v2', 'ETA': 8640, 'param_moyenne_mobile': 5, 'thcl': 3442, 'list_task_complete': [''], 'list_tasks': 'LOAD,ORDER,CONVENTION,SPLIT_TRUCK,merge_port_according_to_plate_info,remove_no_plate_batch,CREATE_PORT_LAUNCH_CARAC_FILL_DASHBOARD', 'forced_hashtag': 'truck', 'list_hashtag_class_create': 'truck', 'ponderation': '0.001,0.001,0.001,0.001,0.001', 'host': 'www.fotonower.com', 'datou_proportion_papier': {'test': 370, 'truck': 4234}, 'datou_matching_id': 4311, 'cod': 1, 'min_number_photo_per_batch': 2, 'datou_truck_for_info': 4234}", 'json_param': {'camera_place_name': 'dechet_Valcor_v2', 'ETA': 8640, 'param_moyenne_mobile': 5, 'thcl': 3442, 'list_task_complete': [''], 'list_tasks': 'LOAD,ORDER,CONVENTION,SPLIT_TRUCK,merge_port_according_to_plate_info,remove_no_plate_batch,CREATE_PORT_LAUNCH_CARAC_FILL_DASHBOARD', 'forced_hashtag': 'truck', 'list_hashtag_class_create': 'truck', 'ponderation': '0.001,0.001,0.001,0.001,0.001', 'host': 'www.fotonower.com', 'datou_proportion_papier': {'test': 370, 'truck': 4234}, 'datou_matching_id': 4311, 'cod': 1, 'min_number_photo_per_batch': 2, 'datou_truck_for_info': 4234}, 'nb_input': 2, 'nb_output': 1, 'prod': False, 'is_local': False, 'is_datou_depend': False, 'is_photo_id_local': False, 'list_data': [{'step_id': 11940, 'datou_id': 4311, 'type_id': 18, 'name': 'portfolio_id', 'output_or_input': False, 'data_order_id': 0, 'type_python': 'int'}, {'step_id': 11940, 'datou_id': 4311, 'type_id': 10, 'name': 'data_string', 'output_or_input': False, 'data_order_id': 1, 'type_python': 'string'}, {'step_id': 11940, 'datou_id': 4311, 'type_id': 22, 'name': 'portfolios_ids_list', 'output_or_input': True, 'data_order_id': 0, 'type_python': 'liste_de_int'}]}], 'depends_step': [{'output': 11939, 'output_id': 0, 'input': 11940, 'input_id': 2}]} check if it is the case where we have multiple sts in one datou no, it is not this case SELECT name, created_at,limit_max FROM MTRDatou.mtr_datou WHERE id=370 SELECT mtd.id, mtdt.`type`, mtd.`param`, mtd.param_json, mtdt.nb_input, mtdt.nb_output, mtdt.prod, mtdt.is_local, mtdt.is_datou_depend, mtdt.is_photo_id_local FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_step_types mtdt WHERE mtdt.`id`=mtd.`type` AND mtd.mtd_id=370 SELECT mtd.id, mtd.mtd_id, mdsdt.id, mdsdt.name, mdsdt.description, msid.output_or_input, msid.data_order_id, mdsdt.type FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_steptype_io_datatypes msid, MTRDatou.mtr_datou_step_data_types mdsdt WHERE mtd.`type`=msid.`mtr_datou_step_type` AND mtd.mtd_id= 370 AND msid.data_type=mdsdt.id SELECT mts_id_output, id_output, mts_id_input, id_input FROM MTRDatou.mtr_datou_step_by_step WHERE mtd_id=370 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! DataTypes for each output/input checked ! {"input_values":[]} Unexpected error treated as WARNING for now expected given param in final step {'min_score': 0.7, 'name_pipeline': 'proportion_papier_tapis', 'hashtag_type': 705, 'hashtag_proportion': 492668766, 'hashtag_parmi': 538914404, 'hashtag_to_consider': 'papier_only', 'hashtag_background': 493012381} Found hashtag_parmi in final step, will be used and useful if different from matiere_majoritaire in classifier ventilate_hashtags_in_portfolio step not found ! SELECT name, created_at,limit_max FROM MTRDatou.mtr_datou WHERE id=4234 SELECT mtd.id, mtdt.`type`, mtd.`param`, mtd.param_json, mtdt.nb_input, mtdt.nb_output, mtdt.prod, mtdt.is_local, mtdt.is_datou_depend, mtdt.is_photo_id_local FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_step_types mtdt WHERE mtdt.`id`=mtd.`type` AND mtd.mtd_id=4234 SELECT mtd.id, mtd.mtd_id, mdsdt.id, mdsdt.name, mdsdt.description, msid.output_or_input, msid.data_order_id, mdsdt.type FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_steptype_io_datatypes msid, MTRDatou.mtr_datou_step_data_types mdsdt WHERE mtd.`type`=msid.`mtr_datou_step_type` AND mtd.mtd_id= 4234 AND msid.data_type=mdsdt.id SELECT mts_id_output, id_output, mts_id_input, id_input FROM MTRDatou.mtr_datou_step_by_step WHERE mtd_id=4234 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! {"input_values":[]} Unexpected error treated as WARNING for now expected given param in final step {'name_pipeline': 'aggregate_carac_ratio', 'hashtag_type': 4230, 'hashtag_proportion': 'papier,carton,metal,pet_clair,autre,pehd,pet_fonce', 'hashtag_parmi': 'refus', 'hashtag_weights': {'autre': 3.0, 'barquette_avec_film': 3.0, 'carton': 3.0, 'metal': 3.0, 'papier': 3.0, 'pehd': 3.0, 'pet_fonce': 3.0, 'pet_clair': 1.0, 'flou': 1.0, 'mal_croppe': 1.0}, 'option_sccu': 1} Found hashtag_parmi in final step, will be used and useful if different from matiere_majoritaire in classifier {'number_of_analysed_photos': ((6397,),), 'unwanted_material_monthly_percentage': 0, 'material_repartition': (('truck', 6240),), 'material_dict': {'truck': 6240}, 'coverage_data': (('2025/07/01/', 0.245257),), 'impurety_average_data': ((datetime.date(2025, 7, 1), 0.21617695082265517, 306), (datetime.date(2025, 7, 2), 0.23391639519104637, 156), (datetime.date(2025, 7, 3), 0.2309142747675992, 345), (datetime.date(2025, 7, 4), 0.23377389854969996, 305), (datetime.date(2025, 7, 5), 0.20825018262090078, 81), (datetime.date(2025, 7, 6), 0.14708950562269027, 2), (datetime.date(2025, 7, 7), 0.22716136139430107, 261), (datetime.date(2025, 7, 8), 0.23984058294550722, 262), (datetime.date(2025, 7, 9), 0.2179782781873437, 266), (datetime.date(2025, 7, 10), 0.22977184483864538, 934), (datetime.date(2025, 7, 11), 0.2313726246794592, 291), (datetime.date(2025, 7, 12), 0.23038289261790657, 175), (datetime.date(2025, 7, 13), 0.19411187829688958, 67), (datetime.date(2025, 7, 14), 0.20748544251853415, 91), (datetime.date(2025, 7, 15), 0.22295557977417005, 279), (datetime.date(2025, 7, 17), 0.2212728525503619, 676), (datetime.date(2025, 7, 18), 0.24303894269617388, 96), (datetime.date(2025, 7, 19), 0.22593637946870926, 47), (datetime.date(2025, 7, 20), 0.1876579659229267, 76), (datetime.date(2025, 7, 21), 0.20516658996852352, 172), (datetime.date(2025, 7, 22), 0.24395799178730843, 93), (datetime.date(2025, 7, 23), 0.2325178322778279, 102), (datetime.date(2025, 7, 24), 0.22025147373288498, 229), (datetime.date(2025, 7, 25), 0.2133161588895479, 198), (datetime.date(2025, 7, 26), 0.24557527830036452, 8), (datetime.date(2025, 7, 27), 0.16258781082288543, 4), (datetime.date(2025, 7, 28), 0.2183676731043412, 237), (datetime.date(2025, 7, 29), 0.21482399029732788, 166), (datetime.date(2025, 7, 30), 0.2601588364040241, 315), (datetime.date(2025, 7, 31), 0.2025969936733917, 153)), 'impurety_average_data_per_hashtag': {'truck': ((datetime.date(2025, 7, 1), 0.21617695082265517, 306), (datetime.date(2025, 7, 2), 0.23391639519104637, 156), (datetime.date(2025, 7, 3), 0.2309142747675992, 345), (datetime.date(2025, 7, 4), 0.23377389854969996, 305), (datetime.date(2025, 7, 5), 0.20825018262090078, 81), (datetime.date(2025, 7, 6), 0.14708950562269027, 2), (datetime.date(2025, 7, 7), 0.22716136139430107, 261), (datetime.date(2025, 7, 8), 0.23984058294550722, 262), (datetime.date(2025, 7, 9), 0.2179782781873437, 266), (datetime.date(2025, 7, 10), 0.22977184483864538, 934), (datetime.date(2025, 7, 11), 0.2313726246794592, 291), (datetime.date(2025, 7, 12), 0.23038289261790657, 175), (datetime.date(2025, 7, 13), 0.19411187829688958, 67), (datetime.date(2025, 7, 14), 0.20748544251853415, 91), (datetime.date(2025, 7, 15), 0.22295557977417005, 279), (datetime.date(2025, 7, 17), 0.2212728525503619, 676), (datetime.date(2025, 7, 18), 0.24303894269617388, 96), (datetime.date(2025, 7, 19), 0.22593637946870926, 47), (datetime.date(2025, 7, 20), 0.1876579659229267, 76), (datetime.date(2025, 7, 21), 0.20516658996852352, 172), (datetime.date(2025, 7, 22), 0.24395799178730843, 93), (datetime.date(2025, 7, 23), 0.2325178322778279, 102), (datetime.date(2025, 7, 24), 0.22025147373288498, 229), (datetime.date(2025, 7, 25), 0.2133161588895479, 198), (datetime.date(2025, 7, 26), 0.24557527830036452, 8), (datetime.date(2025, 7, 27), 0.16258781082288543, 4), (datetime.date(2025, 7, 28), 0.2183676731043412, 237), (datetime.date(2025, 7, 29), 0.21482399029732788, 166), (datetime.date(2025, 7, 30), 0.2601588364040241, 315), (datetime.date(2025, 7, 31), 0.2025969936733917, 153))}, 'quality_array': [['-7.243963914921785', '01072025'], ['-7.158229466013983', '02072025'], ['-7.351509236787634', '03072025'], ['-7.246940222132847', '04072025'], ['-7.494813739703017', '05072025'], ['-7.65769630499743', '06072025'], ['-7.245878843938361', '07072025'], ['-7.221207053936817', '08072025'], ['-7.283650887240919', '09072025'], ['-7.28104286724847', '10072025'], ['-7.122696334505309', '11072025'], ['-7.3318713532800865', '12072025'], ['-7.327776823878171', '13072025'], ['-7.30786609843308', '14072025'], ['-7.43840772199447', '15072025'], ['-7.410263417047756', '17072025'], ['-7.410791131970419', '18072025'], ['-7.654342920923421', '19072025'], ['-7.408633716195079', '20072025'], ['-7.452248678811687', '21072025'], ['-7.461326122735549', '22072025'], ['-7.458392277112584', '23072025'], ['-7.2605141655753', '24072025'], ['-7.3306846500318175', '25072025'], ['-6.9764200474128675', '26072025'], ['-7.625860989331908', '27072025'], ['-7.400760148365198', '28072025'], ['-7.12615724623945', '29072025'], ['-7.363784901034534', '30072025'], ['-7.709085751580858', '31072025']], 'number_of_batch': ((1182,),), 'datou_id_sts': 4311, 'map_hashtag_number_mat': {'hash_to_mat_nb': ({'unwanted_material': 'autre', 'main_material': 'truck', 'pht_type': 4230, 'ratio': 0.02141215561237392, 'nb_photo': 89507, 'list_port_cont': '24452073,24452107,24452158,24452266,24452318,24452357,24452459,24452921,24452974,24453019,24453063,24453088,24453125,24453174,24453179,24453263,24453303,24453339,24453497,24491845,24491888,24491922,24491950,24491983,24492023,24492033,24492091,24492132,24492163,24492200,24492242,24492282,24492312,24492345,24492396,24492426,24492451,24492494,24492534,24492562,24492608,24492627,24492634,24492721,24492728,24492780,24492825,24492850,24492885,24492922,24492947,24493007,24493017,24493079,24493115,24493150,24493191,24493198,24493255,24493291,24493312,24493371,24493403,24493443,24493444,24493508,24493536,24493582,24493607,24493659,24493688,24500172,24500294,24500390,24502680,24503839,24503904,24503949,24503994,24504119,24504194,24504311,24504420,24504452,24510008,24510035,24510092,24510599,24510641,24510741,24510868,24510950,24510976,24511121,24511180,24511196,24511284,24511372,24511429,24511659,24511718,24511855,24512039,24517153,24517218,24517557,24517644,24517897,24518045,24518070,24518321,24518460,24521395,2452144', 'assoc_port': '24452044:24452266,24452047:24452318,24452048:24452459,24452049:24452158,24452050:24452357,24452051:24452107,24452052:24452073,24452770:24453497,24452789:24453339,24452808:24453088,24452817:24453174,24452828:24453303,24452834:24453063,24452841:24453179,24452843:24453019,24452844:24453125,24452845:24453263,24452846:24452974,24452847:24452921,24491721:24493688,24491722:24493444,24491723:24493508,24491724:24493659,24491725:24493191,24491726:24493255,24491727:24493536,24491728:24493403,24491729:24493582,24491730:24493198,24491731:24492922,24491732:24492947,24491733:24493291,24491734:24493371,24491735:24493017,24491736:24493007,24491737:24493150,24491738:24493079,24491739:24493115,24491740:24492721,24491741:24492728,24491742:24492608,24491743:24493607,24491744:24492634,24491745:24492627,24491746:24492451,24491747:24492426,24491748:24492885,24491749:24492396,24491750:24492562,24491751:24493443,24491752:24492825,24491753:24492534,24491754:24492494,24491755:24493312,24491756:24492345,24491757:24492780,24491758:2449228', 'assoc_mat': '24452044:autre:4230:24452266,24452047:autre:4230:24452318,24452048:autre:4230:24452459,24452049:autre:4230:24452158,24452050:autre:4230:24452357,24452051:autre:4230:24452107,24452052:autre:4230:24452073,24452770:autre:4230:24453497,24452789:autre:4230:24453339,24452808:autre:4230:24453088,24452817:autre:4230:24453174,24452828:autre:4230:24453303,24452834:autre:4230:24453063,24452841:autre:4230:24453179,24452843:autre:4230:24453019,24452844:autre:4230:24453125,24452845:autre:4230:24453263,24452846:autre:4230:24452974,24452847:autre:4230:24452921,24491721:autre:4230:24493688,24491722:autre:4230:24493444,24491723:autre:4230:24493508,24491724:autre:4230:24493659,24491725:autre:4230:24493191,24491726:autre:4230:24493255,24491727:autre:4230:24493536,24491728:autre:4230:24493403,24491729:autre:4230:24493582,24491730:autre:4230:24493198,24491731:autre:4230:24492922,24491732:autre:4230:24492947,24491733:autre:4230:24493291,24491734:autre:4230:24493371,24491735:autre:4230:24493017,24491736:autre:4230:24493007,24491737:'}, {'unwanted_material': 'carton', 'main_material': 'truck', 'pht_type': 4230, 'ratio': 0.029727232594059218, 'nb_photo': 117018, 'list_port_cont': '24452076,24452119,24452164,24452272,24452325,24452374,24452449,24452930,24452985,24453026,24453048,24453102,24453142,24453177,24453196,24453264,24453306,24453346,24453502,24491854,24491884,24491923,24491962,24491986,24492032,24492053,24492103,24492133,24492174,24492194,24492252,24492278,24492322,24492359,24492388,24492429,24492450,24492490,24492536,24492574,24492604,24492638,24492647,24492707,24492737,24492784,24492820,24492855,24492902,24492934,24492950,24493002,24493026,24493077,24493116,24493152,24493185,24493207,24493254,24493296,24493309,24493370,24493399,24493435,24493453,24493500,24493551,24493583,24493615,24493654,24493692,24500179,24500296,24500395,24502681,24503840,24503906,24503951,24503999,24504129,24504210,24504320,24504426,24504459,24510002,24510048,24510110,24510613,24510653,24510750,24510865,24510948,24510990,24511118,24511177,24511194,24511297,24511384,24511437,24511656,24511727,24511856,24512032,24517157,24517208,24517570,24517660,24517899,24518046,24518080,24518312,24518466,24521403,2452143', 'assoc_port': '24452044:24452272,24452047:24452325,24452048:24452449,24452049:24452164,24452050:24452374,24452051:24452119,24452052:24452076,24452770:24453502,24452789:24453346,24452808:24453102,24452817:24453177,24452828:24453306,24452834:24453048,24452841:24453196,24452843:24453026,24452844:24453142,24452845:24453264,24452846:24452985,24452847:24452930,24491721:24493692,24491722:24493453,24491723:24493500,24491724:24493654,24491725:24493185,24491726:24493254,24491727:24493551,24491728:24493399,24491729:24493583,24491730:24493207,24491731:24492934,24491732:24492950,24491733:24493296,24491734:24493370,24491735:24493026,24491736:24493002,24491737:24493152,24491738:24493077,24491739:24493116,24491740:24492707,24491741:24492737,24491742:24492604,24491743:24493615,24491744:24492638,24491745:24492647,24491746:24492450,24491747:24492429,24491748:24492902,24491749:24492388,24491750:24492574,24491751:24493435,24491752:24492820,24491753:24492536,24491754:24492490,24491755:24493309,24491756:24492359,24491757:24492784,24491758:2449227', 'assoc_mat': '24452044:carton:4230:24452272,24452047:carton:4230:24452325,24452048:carton:4230:24452449,24452049:carton:4230:24452164,24452050:carton:4230:24452374,24452051:carton:4230:24452119,24452052:carton:4230:24452076,24452770:carton:4230:24453502,24452789:carton:4230:24453346,24452808:carton:4230:24453102,24452817:carton:4230:24453177,24452828:carton:4230:24453306,24452834:carton:4230:24453048,24452841:carton:4230:24453196,24452843:carton:4230:24453026,24452844:carton:4230:24453142,24452845:carton:4230:24453264,24452846:carton:4230:24452985,24452847:carton:4230:24452930,24491721:carton:4230:24493692,24491722:carton:4230:24493453,24491723:carton:4230:24493500,24491724:carton:4230:24493654,24491725:carton:4230:24493185,24491726:carton:4230:24493254,24491727:carton:4230:24493551,24491728:carton:4230:24493399,24491729:carton:4230:24493583,24491730:carton:4230:24493207,24491731:carton:4230:24492934,24491732:carton:4230:24492950,24491733:carton:4230:24493296,24491734:carton:4230:24493370,24491735:carton:4230:24493026,2449'}, {'unwanted_material': 'metal', 'main_material': 'truck', 'pht_type': 4230, 'ratio': 0.008217845252983105, 'nb_photo': 4223, 'list_port_cont': '24452087,24452124,24452155,24452330,24452364,24452448,24452913,24452973,24453024,24453049,24453093,24453135,24453178,24453195,24453265,24453304,24453343,24491840,24491876,24491992,24492052,24492167,24492272,24492308,24492360,24492384,24492443,24492501,24492532,24492576,24492629,24492632,24492791,24492819,24492852,24492886,24492948,24493006,24493080,24493293,24493307,24493359,24493461,24493580,24493622,24493655,24493690,24500181,24500306,24500381,24502684,24503851,24503916,24503953,24503995,24504319,24504411,24504462,24510642,24510746,24510862,24510946,24511106,24511178,24511288,24511375,24511432,24511644,24511719,24511843,24512035,24517215,24517571,24517659,24517895,24518090,24518315,24521427,24525622,24526572,24526686,24527125,24567355,24567453,24567582,24567598,24567722,24568066,24573520,24573582,24573677,24573736,24573817,24573818,24573950,24574029,24574099,24574166,24574283,24574427,24574600,24574659,24574729,24574779,24574844,24575508,24575656,24575726,24575736,24575937,24576168,24576436,24576754,2458118', 'assoc_port': '24452047:24452330,24452048:24452448,24452049:24452155,24452050:24452364,24452051:24452124,24452052:24452087,24452789:24453343,24452808:24453093,24452817:24453178,24452828:24453304,24452834:24453049,24452841:24453195,24452843:24453024,24452844:24453135,24452845:24453265,24452846:24452973,24452847:24452913,24491721:24493690,24491722:24493461,24491724:24493655,24491729:24493580,24491732:24492948,24491733:24493293,24491734:24493359,24491736:24493006,24491738:24493080,24491743:24493622,24491744:24492632,24491745:24492629,24491747:24492443,24491748:24492886,24491749:24492384,24491750:24492576,24491752:24492819,24491753:24492532,24491754:24492501,24491755:24493307,24491756:24492360,24491757:24492791,24491758:24492272,24491759:24492308,24491760:24492167,24491763:24491840,24491767:24491992,24491770:24492852,24491771:24491876,24491772:24492052,24499974:24500381,24499989:24500306,24500004:24502684,24500008:24500181,24503600:24504411,24503602:24504462,24503606:24503995,24503607:24504319,24503609:24503953,24503611:2450385', 'assoc_mat': '24452047:metal:4230:24452330,24452048:metal:4230:24452448,24452049:metal:4230:24452155,24452050:metal:4230:24452364,24452051:metal:4230:24452124,24452052:metal:4230:24452087,24452789:metal:4230:24453343,24452808:metal:4230:24453093,24452817:metal:4230:24453178,24452828:metal:4230:24453304,24452834:metal:4230:24453049,24452841:metal:4230:24453195,24452843:metal:4230:24453024,24452844:metal:4230:24453135,24452845:metal:4230:24453265,24452846:metal:4230:24452973,24452847:metal:4230:24452913,24491721:metal:4230:24493690,24491722:metal:4230:24493461,24491724:metal:4230:24493655,24491729:metal:4230:24493580,24491732:metal:4230:24492948,24491733:metal:4230:24493293,24491734:metal:4230:24493359,24491736:metal:4230:24493006,24491738:metal:4230:24493080,24491743:metal:4230:24493622,24491744:metal:4230:24492632,24491745:metal:4230:24492629,24491747:metal:4230:24492443,24491748:metal:4230:24492886,24491749:metal:4230:24492384,24491750:metal:4230:24492576,24491752:metal:4230:24492819,24491753:metal:4230:24492532,24491754:'}, {'unwanted_material': 'papier', 'main_material': 'truck', 'pht_type': 4230, 'ratio': 0.15884656879218992, 'nb_photo': 306144, 'list_port_cont': '24452083,24452117,24452162,24452264,24452334,24452372,24452456,24452923,24452983,24453022,24453055,24453103,24453131,24453168,24453194,24453271,24453300,24453354,24453506,24491856,24491891,24491921,24491961,24491990,24492034,24492048,24492107,24492137,24492178,24492211,24492247,24492281,24492323,24492351,24492391,24492441,24492442,24492504,24492540,24492565,24492610,24492636,24492641,24492711,24492730,24492793,24492824,24492856,24492894,24492935,24492955,24493010,24493028,24493076,24493114,24493149,24493183,24493206,24493258,24493295,24493306,24493365,24493401,24493441,24493458,24493504,24493548,24493578,24493623,24493651,24493685,24500178,24500305,24500389,24502674,24503847,24503924,24503956,24503988,24504133,24504205,24504314,24504423,24504458,24510006,24510047,24510107,24510615,24510651,24510751,24510869,24510943,24510980,24511117,24511176,24511189,24511293,24511383,24511431,24511662,24511721,24511849,24512038,24517160,24517207,24517572,24517647,24517904,24518034,24518082,24518314,24518469,24521401,2452143', 'assoc_port': '24452044:24452264,24452047:24452334,24452048:24452456,24452049:24452162,24452050:24452372,24452051:24452117,24452052:24452083,24452770:24453506,24452789:24453354,24452808:24453103,24452817:24453168,24452828:24453300,24452834:24453055,24452841:24453194,24452843:24453022,24452844:24453131,24452845:24453271,24452846:24452983,24452847:24452923,24491721:24493685,24491722:24493458,24491723:24493504,24491724:24493651,24491725:24493183,24491726:24493258,24491727:24493548,24491728:24493401,24491729:24493578,24491730:24493206,24491731:24492935,24491732:24492955,24491733:24493295,24491734:24493365,24491735:24493028,24491736:24493010,24491737:24493149,24491738:24493076,24491739:24493114,24491740:24492711,24491741:24492730,24491742:24492610,24491743:24493623,24491744:24492636,24491745:24492641,24491746:24492442,24491747:24492441,24491748:24492894,24491749:24492391,24491750:24492565,24491751:24493441,24491752:24492824,24491753:24492540,24491754:24492504,24491755:24493306,24491756:24492351,24491757:24492793,24491758:2449228', 'assoc_mat': '24452044:papier:4230:24452264,24452047:papier:4230:24452334,24452048:papier:4230:24452456,24452049:papier:4230:24452162,24452050:papier:4230:24452372,24452051:papier:4230:24452117,24452052:papier:4230:24452083,24452770:papier:4230:24453506,24452789:papier:4230:24453354,24452808:papier:4230:24453103,24452817:papier:4230:24453168,24452828:papier:4230:24453300,24452834:papier:4230:24453055,24452841:papier:4230:24453194,24452843:papier:4230:24453022,24452844:papier:4230:24453131,24452845:papier:4230:24453271,24452846:papier:4230:24452983,24452847:papier:4230:24452923,24491721:papier:4230:24493685,24491722:papier:4230:24493458,24491723:papier:4230:24493504,24491724:papier:4230:24493651,24491725:papier:4230:24493183,24491726:papier:4230:24493258,24491727:papier:4230:24493548,24491728:papier:4230:24493401,24491729:papier:4230:24493578,24491730:papier:4230:24493206,24491731:papier:4230:24492935,24491732:papier:4230:24492955,24491733:papier:4230:24493295,24491734:papier:4230:24493365,24491735:papier:4230:24493028,2449'}, {'unwanted_material': 'pehd', 'main_material': 'truck', 'pht_type': 4230, 'ratio': 0.00121672865586385, 'nb_photo': 1319, 'list_port_cont': '24453140,24453171,24453186,24453315,24453505,24491892,24491963,24491993,24492179,24492355,24492389,24492505,24492538,24492612,24492788,24492826,24492863,24492946,24493189,24493205,24493315,24493429,24493494,24493619,24493657,24502682,24503848,24503910,24503961,24510639,24510739,24510873,24510987,24511119,24511179,24511203,24511298,24511380,24511434,24511663,24511725,24512033,24517149,24517573,24518473,24526698,24567451,24574165,24574788,24574852,24576438,24581180,24581383,24581954,24582113,24595010,24595062,24595967,24596294,24596383,24596819,24596856,24596893,24596957,24597249,24597284,24597381,24597425,24597516,24597535,24597721,24598368,24598824,24599382,24603645,24603800,24604385,24604512,24605508,24605976,24606200,24606346,24610871,24611080,24611212,24611317,24611385,24615381,24615421,24615540,24615672,24615802,24615909,24616159,24616417,24616603,24616722,24616894,24616967,24617088,24620087,24640957,24641933,24641968,24645871,24646222,24717794,24718476,24721008,24721885,24722001,24722686,24722817,2472373', 'assoc_port': '24452770:24453505,24452817:24453171,24452828:24453315,24452841:24453186,24452844:24453140,24491723:24493494,24491724:24493657,24491725:24493189,24491730:24493205,24491732:24492946,24491742:24492612,24491743:24493619,24491749:24492389,24491751:24493429,24491752:24492826,24491753:24492538,24491754:24492505,24491755:24493315,24491756:24492355,24491757:24492788,24491760:24492179,24491767:24491993,24491769:24491963,24491770:24492863,24491771:24491892,24500004:24502682,24503609:24503961,24503611:24503848,24503613:24503910,24510372:24511663,24510374:24512033,24510375:24511725,24510376:24511380,24510389:24511434,24510402:24511298,24510403:24511119,24510404:24511179,24510405:24511203,24510406:24510739,24510422:24510987,24510437:24510873,24510452:24510639,24516260:24518473,24516340:24517149,24516344:24517573,24526363:24526698,24567050:24567451,24573197:24576438,24573209:24574788,24573211:24574852,24573283:24574165,24580565:24581954,24580570:24581383,24580574:24581180,24580579:24582113,24594212:24599382,24594215:2459836', 'assoc_mat': '24452770:pehd:4230:24453505,24452817:pehd:4230:24453171,24452828:pehd:4230:24453315,24452841:pehd:4230:24453186,24452844:pehd:4230:24453140,24491723:pehd:4230:24493494,24491724:pehd:4230:24493657,24491725:pehd:4230:24493189,24491730:pehd:4230:24493205,24491732:pehd:4230:24492946,24491742:pehd:4230:24492612,24491743:pehd:4230:24493619,24491749:pehd:4230:24492389,24491751:pehd:4230:24493429,24491752:pehd:4230:24492826,24491753:pehd:4230:24492538,24491754:pehd:4230:24492505,24491755:pehd:4230:24493315,24491756:pehd:4230:24492355,24491757:pehd:4230:24492788,24491760:pehd:4230:24492179,24491767:pehd:4230:24491993,24491769:pehd:4230:24491963,24491770:pehd:4230:24492863,24491771:pehd:4230:24491892,24500004:pehd:4230:24502682,24503609:pehd:4230:24503961,24503611:pehd:4230:24503848,24503613:pehd:4230:24503910,24510372:pehd:4230:24511663,24510374:pehd:4230:24512033,24510375:pehd:4230:24511725,24510376:pehd:4230:24511380,24510389:pehd:4230:24511434,24510402:pehd:4230:24511298,24510403:pehd:4230:24511119,24510404:pehd:42'}, {'unwanted_material': 'pet_clair', 'main_material': 'truck', 'pht_type': 4230, 'ratio': 0.011905527100546353, 'nb_photo': 53012, 'list_port_cont': '24452078,24452118,24452160,24452270,24452329,24452369,24452457,24452929,24452989,24453016,24453057,24453098,24453132,24453175,24453191,24453268,24453307,24453348,24453501,24491846,24491881,24491927,24491959,24491997,24492036,24492047,24492104,24492140,24492176,24492214,24492251,24492279,24492317,24492356,24492392,24492439,24492445,24492502,24492541,24492575,24492614,24492624,24492650,24492713,24492738,24492785,24492830,24492864,24492891,24492936,24492944,24493008,24493023,24493082,24493117,24493146,24493190,24493203,24493261,24493292,24493317,24493367,24493394,24493438,24493456,24493510,24493545,24493585,24493614,24493653,24493687,24500188,24500301,24500391,24502678,24503841,24503900,24503962,24503993,24504132,24504211,24504313,24504424,24504460,24510003,24510049,24510084,24510610,24510655,24510743,24510871,24510951,24510988,24511120,24511174,24511202,24511295,24511382,24511433,24511664,24511715,24511848,24512037,24517141,24517210,24517567,24517655,24517896,24518032,24518094,24518318,24518471,24521400,2452143', 'assoc_port': '24452044:24452270,24452047:24452329,24452048:24452457,24452049:24452160,24452050:24452369,24452051:24452118,24452052:24452078,24452770:24453501,24452789:24453348,24452808:24453098,24452817:24453175,24452828:24453307,24452834:24453057,24452841:24453191,24452843:24453016,24452844:24453132,24452845:24453268,24452846:24452989,24452847:24452929,24491721:24493687,24491722:24493456,24491723:24493510,24491724:24493653,24491725:24493190,24491726:24493261,24491727:24493545,24491728:24493394,24491729:24493585,24491730:24493203,24491731:24492936,24491732:24492944,24491733:24493292,24491734:24493367,24491735:24493023,24491736:24493008,24491737:24493146,24491738:24493082,24491739:24493117,24491740:24492713,24491741:24492738,24491742:24492614,24491743:24493614,24491744:24492624,24491745:24492650,24491746:24492445,24491747:24492439,24491748:24492891,24491749:24492392,24491750:24492575,24491751:24493438,24491752:24492830,24491753:24492541,24491754:24492502,24491755:24493317,24491756:24492356,24491757:24492785,24491758:2449227', 'assoc_mat': '24452044:pet_clair:4230:24452270,24452047:pet_clair:4230:24452329,24452048:pet_clair:4230:24452457,24452049:pet_clair:4230:24452160,24452050:pet_clair:4230:24452369,24452051:pet_clair:4230:24452118,24452052:pet_clair:4230:24452078,24452770:pet_clair:4230:24453501,24452789:pet_clair:4230:24453348,24452808:pet_clair:4230:24453098,24452817:pet_clair:4230:24453175,24452828:pet_clair:4230:24453307,24452834:pet_clair:4230:24453057,24452841:pet_clair:4230:24453191,24452843:pet_clair:4230:24453016,24452844:pet_clair:4230:24453132,24452845:pet_clair:4230:24453268,24452846:pet_clair:4230:24452989,24452847:pet_clair:4230:24452929,24491721:pet_clair:4230:24493687,24491722:pet_clair:4230:24493456,24491723:pet_clair:4230:24493510,24491724:pet_clair:4230:24493653,24491725:pet_clair:4230:24493190,24491726:pet_clair:4230:24493261,24491727:pet_clair:4230:24493545,24491728:pet_clair:4230:24493394,24491729:pet_clair:4230:24493585,24491730:pet_clair:4230:24493203,24491731:pet_clair:4230:24492936,24491732:pet_clair:4230:24492944,2'}, {'unwanted_material': 'pet_fonce', 'main_material': 'truck', 'pht_type': 4230, 'ratio': 0.0012378756266419198, 'nb_photo': 1495, 'list_port_cont': '24452123,24452371,24452918,24452990,24492027,24492142,24492577,24492792,24492865,24492897,24493004,24493154,24493361,24493434,24503845,24503958,24510052,24510611,24510982,24511183,24511376,24511436,24511654,24511726,24517151,24517213,24517569,24517658,24517887,24518040,24518088,24518310,24518472,24521402,24527122,24573529,24573689,24573808,24574850,24574970,24581190,24581953,24595016,24595063,24595885,24595965,24596386,24596956,24597307,24597512,24597529,24597726,24598239,24598371,24598831,24599371,24604015,24604301,24605838,24605952,24605991,24606340,24606440,24606997,24610247,24610402,24610621,24610994,24611073,24611224,24611395,24611499,24615808,24615860,24615906,24616063,24616388,24616425,24616604,24616721,24616904,24639527,24640967,24645870,24718470,24720845,24722358,24722777,24723730,24723903,24723964,24724048,24724842,24725253,24725382,24725430,24725733,24725741,24725770,24726023,24726131,24726268,24726763,24726864,24727384,24727604,24727733,24728260,24728980,24729070,24730224,24731782,24731816,2473194', 'assoc_port': '24452050:24452371,24452051:24452123,24452846:24452990,24452847:24452918,24491734:24493361,24491736:24493004,24491737:24493154,24491748:24492897,24491750:24492577,24491751:24493434,24491757:24492792,24491764:24492142,24491766:24492027,24491770:24492865,24503599:24510052,24503609:24503958,24503611:24503845,24510372:24511654,24510375:24511726,24510376:24511376,24510389:24511436,24510404:24511183,24510422:24510982,24510457:24510611,24516257:24518310,24516260:24518472,24516264:24518040,24516268:24517887,24516317:24518088,24516333:24517658,24516340:24517151,24516344:24517569,24516347:24517213,24521123:24521402,24526361:24527122,24573203:24574970,24573211:24574850,24573253:24573808,24573299:24573689,24573300:24573529,24580565:24581953,24580574:24581190,24594212:24599371,24594214:24598239,24594215:24598371,24594216:24597726,24594217:24598831,24594218:24597512,24594219:24597529,24594233:24597307,24594267:24596386,24594292:24595063,24594318:24595016,24594322:24596956,24594394:24595965,24594496:24595885,24603092:2460699', 'assoc_mat': '24452050:pet_fonce:4230:24452371,24452051:pet_fonce:4230:24452123,24452846:pet_fonce:4230:24452990,24452847:pet_fonce:4230:24452918,24491734:pet_fonce:4230:24493361,24491736:pet_fonce:4230:24493004,24491737:pet_fonce:4230:24493154,24491748:pet_fonce:4230:24492897,24491750:pet_fonce:4230:24492577,24491751:pet_fonce:4230:24493434,24491757:pet_fonce:4230:24492792,24491764:pet_fonce:4230:24492142,24491766:pet_fonce:4230:24492027,24491770:pet_fonce:4230:24492865,24503599:pet_fonce:4230:24510052,24503609:pet_fonce:4230:24503958,24503611:pet_fonce:4230:24503845,24510372:pet_fonce:4230:24511654,24510375:pet_fonce:4230:24511726,24510376:pet_fonce:4230:24511376,24510389:pet_fonce:4230:24511436,24510404:pet_fonce:4230:24511183,24510422:pet_fonce:4230:24510982,24510457:pet_fonce:4230:24510611,24516257:pet_fonce:4230:24518310,24516260:pet_fonce:4230:24518472,24516264:pet_fonce:4230:24518040,24516268:pet_fonce:4230:24517887,24516317:pet_fonce:4230:24518088,24516333:pet_fonce:4230:24517658,24516340:pet_fonce:4230:24517151,2'}), 'main_stat': {'nb_photos': 6397}}, 'hostname': (('raspi-84',),), 'impurety_average_per_hashtag': []} filepath : /data_4/data_log/job/memo/dechet_Valcor_v2/sla_mensuel_dechet_Valcor_v2_mois_07_annee_2025.pdf Before plot_photos_and_unwanted_material_summary # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11939 matching_dashboard is not consistent : 1 used against 0 in the step definition ! Step 11940 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 0 of step 11939 doesn't seem to be define in the database( WARNING : type of input 2 of step 11940 doesn't seem to be define in the database( DataTypes for each output/input checked ! None was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? Before organize_and_plot_unwanted_material_stats {'truck': {'autre': ['autre', '2.14%', 89507], 'carton': ['carton', '2.97%', 117018], 'metal': ['metal', '0.82%', 4223], 'papier': ['papier', '15.88%', 306144], 'pehd': ['pehd', '0.12%', 1319], 'pet_clair': ['pet_clair', '1.19%', 53012], 'pet_fonce': ['pet_fonce', '0.12%', 1495]}} before organize_and_plot_coverage_data before organize_and_plot_impurety_average_data before organize_and_plot_quality_photo_data finish generating filepath : /data_4/data_log/job/memo/dechet_Valcor_v2/sla_mensuel_dechet_Valcor_v2_mois_07_annee_2025.pdf le fichier se trouve ici : /data_4/data_log/job/memo/dechet_Valcor_v2 2.78user 2.66system 0:12.36elapsed 44%CPU (0avgtext+0avgdata 175520maxresident)k 56inputs+424outputs (0major+44519minor)pagefaults 0swaps