python /home/admin/mtr/script_for_cron.py -j python_test3 -m 12 -a ' --short_python3 -v ' -s python_test3 -M 0 -S 0 -U 100,100,120 import MySQLdb succeeded Import error (python version) python version = 3 warning , we can't find thcl infos in json_data warning , we can't find pdt infos in json_data list_job_run_as_list : ['mask_detection', 'datou', 'CacheModelData_queries', 'CachePhotoData_queries', 'test_fork', 'prepare_maskdata', 'portfolio_queries', 'sla_mensuel'] python version used : 3 liste_fichiers : [('tests/mask_test', True, 'Test mask-detection ', 'mask_detection'), ('tests/datou_test', True, 'Datou All Test', 'datou', 'all'), ('mtr/database_queries/CacheModelData_queries', True, 'Test Cache Model Data', 'CacheModelData_queries'), ('tests/cache_photo_data_test', True, 'Test local_cache_photo ', 'CachePhotoData_queries'), ('mtr/mask_rcnn/prepare_maskdata', True, 'test prepare mask data', 'prepare_maskdata', 'all'), ('mtr/database_queries/portfolio_queries', True, 'test portfolio queries', 'portfolio_queries'), ('prod/memo/memo', True, 'SLA Mensuel', 'sla_mensuel', 'all')] #&_# BEGIN OF TEST : tests/mask_test #&_# /home/admin/workarea/git/Velours/python/tests/mask_test.py Test mask-detection python version used : 3 ############################### TEST memory used ################################ free memory at begining : begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 10582 run mask_detect Inside batchDatouExec : verbose : False # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! List Step Type Loaded in datou : mask_detect list_input_json : [] origin we have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 0 ; length of list_pids : 0 ; length of list_args : 0 time to download the photos : 0.0064084529876708984 About to test input to load empty input or other problem ret : 0 no datou_current to update free memory after detection : begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 10582 ############################### TEST detect object ################################ run mask_detect Inside batchDatouExec : verbose : False # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! List Step Type Loaded in datou : mask_detect list_input_json : [] origin we have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 0 ; length of list_pids : 0 ; length of list_args : 0 time to download the photos : 0.005845785140991211 About to test input to load empty input or other problem ret : 0 no datou_current to update ERROR : cannot unpack non-iterable NoneType object reconnect to base ! warning , we can't find thcl infos in json_data warning , we can't find pdt infos in json_data #&_# TEST FAILED #&_# : tests/mask_test #&_# Error : invalid literal for int() with base 10: 'cannot unpack non-iterable NoneType object' /home/admin/workarea/git/Velours/python/tests/python_tests.py refs/heads/migration_mbp_22_09_25_6b97d44a6efa1f1691f192e531c2c17a72e983b6 SQL :INSERT INTO MTRAdmin.monitor_sys (name, type, server, version_code, result_str, result_bool, lien , test_group ,test_name) VALUES ('python_test3','1','marlene','refs/heads/migration_mbp_22_09_25_6b97d44a6efa1f1691f192e531c2c17a72e983b6','{"mask_detection": "fail"}','0','http://marlene.fotonower-preprod.com/job/2025/September/24092025/python_test3/','mask_detection','unknown'); #&_# END OF TEST #&_# : tests/mask_test #&_# #&_# BEGIN OF TEST : tests/datou_test #&_# /home/admin/workarea/git/Velours/python/tests/datou_test.py Datou All Test python version used : 3 ############################### TEST sam ################################ TEST SAM Inside batchDatouExec : verbose : True ##### chargement datou SELECT name, created_at,limit_max FROM MTRDatou.mtr_datou WHERE id=4573 SELECT mtd.id, mtdt.`type`, mtd.`param`, mtd.param_json, mtdt.nb_input, mtdt.nb_output, mtdt.prod, mtdt.is_local, mtdt.is_datou_depend, mtdt.is_photo_id_local FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_step_types mtdt WHERE mtdt.`id`=mtd.`type` AND mtd.mtd_id=4573 SELECT mtd.id, mtd.mtd_id, mdsdt.id, mdsdt.name, mdsdt.description, msid.output_or_input, msid.data_order_id, mdsdt.type FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_steptype_io_datatypes msid, MTRDatou.mtr_datou_step_data_types mdsdt WHERE mtd.`type`=msid.`mtr_datou_step_type` AND mtd.mtd_id= 4573 AND msid.data_type=mdsdt.id SELECT mts_id_output, id_output, mts_id_input, id_input FROM MTRDatou.mtr_datou_step_by_step WHERE mtd_id=4573 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! no param json to modify List Step Type Loaded in datou : sam list_input_json : [] ##### fin chargement datou ##### chargement data ##### Call load_data_input : nb_thread : 5 origin SELECT photo_id, url FROM MTRBack.view_photos_with_urls_250912 ph WHERE photo_id IN (1189321094) Found this number of photos: 1 ##### Call download_photos : nb_thread : 5 begin to download photo : 1189321094 download finish for photo 1189321094 we have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB ##### After download_photos length of list_filenames : 1 ; length of list_pids : 1 ; length of list_args : 1 ##### After load_data_input time to download the photos : 0.19774556159973145 #### fin chargement data Blocking on flush ? No conitnuing About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! WARNING : we have an input that is not a photo, we should get rid of it Calling datou_exec Inside datou_exec : verbose : True number of steps : 1 step1:sam Wed Sep 24 17:48:10 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec After prepare type args : Here we display some param of map_info ! map_filenames : {'temp/1758728890_1864669_1189321094_9626af7f95d010f2a4fd524688d4ea22_76896585.png': 1189321094} map_photo_id_path_extension : {1189321094: {'path': 'temp/1758728890_1864669_1189321094_9626af7f95d010f2a4fd524688d4ea22_76896585.png', 'extension': 'png'}} map_subphoto_mainphoto : {} Beginning of datou step sam ! pht : 4677 Inside sam : nb paths : 1 (640, 960, 3) time for calcul the mask position with numpy : 0.0016205310821533203 nb_pixel_total : 5631 time to create 1 rle with old method : 0.006330728530883789 time for calcul the mask position with numpy : 0.0013318061828613281 nb_pixel_total : 3774 time to create 1 rle with old method : 0.004288673400878906 time for calcul the mask position with numpy : 0.00142669677734375 nb_pixel_total : 16194 time to create 1 rle with old method : 0.017151832580566406 time for calcul the mask position with numpy : 0.002207040786743164 nb_pixel_total : 83754 time to create 1 rle with old method : 0.0887908935546875 time for calcul the mask position with numpy : 0.001344442367553711 nb_pixel_total : 6628 time to create 1 rle with old method : 0.0075550079345703125 time for calcul the mask position with numpy : 0.0012998580932617188 nb_pixel_total : 5983 time to create 1 rle with old method : 0.0066525936126708984 time for calcul the mask position with numpy : 0.0012836456298828125 nb_pixel_total : 4158 time to create 1 rle with old method : 0.004521846771240234 time for calcul the mask position with numpy : 0.001306295394897461 nb_pixel_total : 8620 time to create 1 rle with old method : 0.009628772735595703 time for calcul the mask position with numpy : 0.0014579296112060547 nb_pixel_total : 14696 time to create 1 rle with old method : 0.016899824142456055 time for calcul the mask position with numpy : 0.001340627670288086 nb_pixel_total : 13977 time to create 1 rle with old method : 0.015366554260253906 time for calcul the mask position with numpy : 0.001279592514038086 nb_pixel_total : 2937 time to create 1 rle with old method : 0.0032906532287597656 time for calcul the mask position with numpy : 0.0012950897216796875 nb_pixel_total : 5301 time to create 1 rle with old method : 0.005925178527832031 time for calcul the mask position with numpy : 0.0013074874877929688 nb_pixel_total : 10824 time to create 1 rle with old method : 0.011760473251342773 time for calcul the mask position with numpy : 0.0014111995697021484 nb_pixel_total : 29427 time to create 1 rle with old method : 0.03223013877868652 time for calcul the mask position with numpy : 0.0012826919555664062 nb_pixel_total : 2452 time to create 1 rle with old method : 0.002801179885864258 time for calcul the mask position with numpy : 0.0012803077697753906 nb_pixel_total : 3376 time to create 1 rle with old method : 0.00377655029296875 time for calcul the mask position with numpy : 0.0012705326080322266 nb_pixel_total : 1227 time to create 1 rle with old method : 0.0014557838439941406 time for calcul the mask position with numpy : 0.0013630390167236328 nb_pixel_total : 3956 time to create 1 rle with old method : 0.004345417022705078 time for calcul the mask position with numpy : 0.001310110092163086 nb_pixel_total : 7637 time to create 1 rle with old method : 0.008293390274047852 time for calcul the mask position with numpy : 0.0013802051544189453 nb_pixel_total : 4247 time to create 1 rle with old method : 0.004851818084716797 time for calcul the mask position with numpy : 0.0014586448669433594 nb_pixel_total : 38705 time to create 1 rle with old method : 0.04120969772338867 time for calcul the mask position with numpy : 0.0012691020965576172 nb_pixel_total : 1564 time to create 1 rle with old method : 0.0018482208251953125 time for calcul the mask position with numpy : 0.0012676715850830078 nb_pixel_total : 2079 time to create 1 rle with old method : 0.002252340316772461 time for calcul the mask position with numpy : 0.0012638568878173828 nb_pixel_total : 1730 time to create 1 rle with old method : 0.0019297599792480469 time for calcul the mask position with numpy : 0.0012731552124023438 nb_pixel_total : 2426 time to create 1 rle with old method : 0.0028257369995117188 time for calcul the mask position with numpy : 0.0013668537139892578 nb_pixel_total : 3514 time to create 1 rle with old method : 0.0039632320404052734 time for calcul the mask position with numpy : 0.0012927055358886719 nb_pixel_total : 5501 time to create 1 rle with old method : 0.0063245296478271484 time for calcul the mask position with numpy : 0.0014078617095947266 nb_pixel_total : 27948 time to create 1 rle with old method : 0.03157615661621094 time for calcul the mask position with numpy : 0.0012853145599365234 nb_pixel_total : 1512 time to create 1 rle with old method : 0.0018925666809082031 time for calcul the mask position with numpy : 0.0013909339904785156 nb_pixel_total : 12689 time to create 1 rle with old method : 0.014858245849609375 time for calcul the mask position with numpy : 0.0012736320495605469 nb_pixel_total : 1322 time to create 1 rle with old method : 0.0016391277313232422 time for calcul the mask position with numpy : 0.0013310909271240234 nb_pixel_total : 4178 time to create 1 rle with old method : 0.004914522171020508 time for calcul the mask position with numpy : 0.001316070556640625 nb_pixel_total : 2775 time to create 1 rle with old method : 0.0033426284790039062 time for calcul the mask position with numpy : 0.0014460086822509766 nb_pixel_total : 577 time to create 1 rle with old method : 0.0006875991821289062 time for calcul the mask position with numpy : 0.0012824535369873047 nb_pixel_total : 888 time to create 1 rle with old method : 0.0011365413665771484 time for calcul the mask position with numpy : 0.0014660358428955078 nb_pixel_total : 11968 time to create 1 rle with old method : 0.013859748840332031 time for calcul the mask position with numpy : 0.0015532970428466797 nb_pixel_total : 16429 time to create 1 rle with old method : 0.01856708526611328 time for calcul the mask position with numpy : 0.0014309883117675781 nb_pixel_total : 10784 time to create 1 rle with old method : 0.012448787689208984 time for calcul the mask position with numpy : 0.0013246536254882812 nb_pixel_total : 9894 time to create 1 rle with old method : 0.011554718017578125 time for calcul the mask position with numpy : 0.0013234615325927734 nb_pixel_total : 2727 time to create 1 rle with old method : 0.003359556198120117 time for calcul the mask position with numpy : 0.0013921260833740234 nb_pixel_total : 2449 time to create 1 rle with old method : 0.0029392242431640625 time for calcul the mask position with numpy : 0.0014274120330810547 nb_pixel_total : 2783 time to create 1 rle with old method : 0.0033931732177734375 time for calcul the mask position with numpy : 0.0013365745544433594 nb_pixel_total : 1028 time to create 1 rle with old method : 0.0011670589447021484 time for calcul the mask position with numpy : 0.0012969970703125 nb_pixel_total : 2334 time to create 1 rle with old method : 0.0026397705078125 time for calcul the mask position with numpy : 0.0014123916625976562 nb_pixel_total : 2385 time to create 1 rle with old method : 0.0028040409088134766 time for calcul the mask position with numpy : 0.0013866424560546875 nb_pixel_total : 342 time to create 1 rle with old method : 0.000423431396484375 time for calcul the mask position with numpy : 0.0013031959533691406 nb_pixel_total : 1652 time to create 1 rle with old method : 0.002032756805419922 time for calcul the mask position with numpy : 0.0012869834899902344 nb_pixel_total : 1253 time to create 1 rle with old method : 0.001424551010131836 time for calcul the mask position with numpy : 0.0014472007751464844 nb_pixel_total : 8506 time to create 1 rle with old method : 0.009507417678833008 time for calcul the mask position with numpy : 0.0012946128845214844 nb_pixel_total : 868 time to create 1 rle with old method : 0.0012278556823730469 time for calcul the mask position with numpy : 0.0013701915740966797 nb_pixel_total : 4180 time to create 1 rle with old method : 0.004873991012573242 time for calcul the mask position with numpy : 0.0012905597686767578 nb_pixel_total : 3820 time to create 1 rle with old method : 0.004597187042236328 time for calcul the mask position with numpy : 0.0012769699096679688 nb_pixel_total : 2033 time to create 1 rle with old method : 0.002323627471923828 time for calcul the mask position with numpy : 0.0012698173522949219 nb_pixel_total : 1206 time to create 1 rle with old method : 0.001425027847290039 time for calcul the mask position with numpy : 0.0012683868408203125 nb_pixel_total : 875 time to create 1 rle with old method : 0.0010716915130615234 time for calcul the mask position with numpy : 0.0012574195861816406 nb_pixel_total : 590 time to create 1 rle with old method : 0.0007114410400390625 time for calcul the mask position with numpy : 0.0013363361358642578 nb_pixel_total : 16701 time to create 1 rle with old method : 0.019017457962036133 time for calcul the mask position with numpy : 0.0012729167938232422 nb_pixel_total : 1709 time to create 1 rle with old method : 0.0019328594207763672 time for calcul the mask position with numpy : 0.0012669563293457031 nb_pixel_total : 692 time to create 1 rle with old method : 0.0008378028869628906 time for calcul the mask position with numpy : 0.001363992691040039 nb_pixel_total : 1055 time to create 1 rle with old method : 0.0012786388397216797 time for calcul the mask position with numpy : 0.0012638568878173828 nb_pixel_total : 337 time to create 1 rle with old method : 0.00043272972106933594 time for calcul the mask position with numpy : 0.0012848377227783203 nb_pixel_total : 1198 time to create 1 rle with old method : 0.0013790130615234375 time for calcul the mask position with numpy : 0.0012662410736083984 nb_pixel_total : 735 time to create 1 rle with old method : 0.0009527206420898438 time for calcul the mask position with numpy : 0.0012974739074707031 nb_pixel_total : 602 time to create 1 rle with old method : 0.0007271766662597656 time for calcul the mask position with numpy : 0.0012679100036621094 nb_pixel_total : 1076 time to create 1 rle with old method : 0.0012941360473632812 time for calcul the mask position with numpy : 0.0013740062713623047 nb_pixel_total : 3094 time to create 1 rle with old method : 0.003512859344482422 time for calcul the mask position with numpy : 0.0013055801391601562 nb_pixel_total : 8613 time to create 1 rle with old method : 0.009830474853515625 time for calcul the mask position with numpy : 0.001337289810180664 nb_pixel_total : 12978 time to create 1 rle with old method : 0.014111995697021484 time for calcul the mask position with numpy : 0.0015959739685058594 nb_pixel_total : 39198 time to create 1 rle with old method : 0.04234600067138672 time for calcul the mask position with numpy : 0.0012729167938232422 nb_pixel_total : 971 time to create 1 rle with old method : 0.0011360645294189453 time for calcul the mask position with numpy : 0.0013043880462646484 nb_pixel_total : 8478 time to create 1 rle with old method : 0.009143590927124023 time for calcul the mask position with numpy : 0.0012617111206054688 nb_pixel_total : 1052 time to create 1 rle with old method : 0.001211404800415039 time for calcul the mask position with numpy : 0.0012793540954589844 nb_pixel_total : 2203 time to create 1 rle with old method : 0.002579927444458008 time for calcul the mask position with numpy : 0.0013890266418457031 nb_pixel_total : 7519 time to create 1 rle with old method : 0.008234500885009766 time for calcul the mask position with numpy : 0.0012650489807128906 nb_pixel_total : 267 time to create 1 rle with old method : 0.0003228187561035156 time for calcul the mask position with numpy : 0.001270294189453125 nb_pixel_total : 716 time to create 1 rle with old method : 0.0008900165557861328 time for calcul the mask position with numpy : 0.0012671947479248047 nb_pixel_total : 888 time to create 1 rle with old method : 0.0010309219360351562 time for calcul the mask position with numpy : 0.0013735294342041016 nb_pixel_total : 9075 time to create 1 rle with old method : 0.009962797164916992 time for calcul the mask position with numpy : 0.0012879371643066406 nb_pixel_total : 3165 time to create 1 rle with old method : 0.003668069839477539 time for calcul the mask position with numpy : 0.0014319419860839844 nb_pixel_total : 617 time to create 1 rle with old method : 0.0008265972137451172 time for calcul the mask position with numpy : 0.0014688968658447266 nb_pixel_total : 9503 time to create 1 rle with old method : 0.010759115219116211 time for calcul the mask position with numpy : 0.0014662742614746094 nb_pixel_total : 971 time to create 1 rle with old method : 0.0012464523315429688 time for calcul the mask position with numpy : 0.0012860298156738281 nb_pixel_total : 1503 time to create 1 rle with old method : 0.0017380714416503906 time for calcul the mask position with numpy : 0.0012636184692382812 nb_pixel_total : 221 time to create 1 rle with old method : 0.00032806396484375 time for calcul the mask position with numpy : 0.0014498233795166016 nb_pixel_total : 38906 time to create 1 rle with old method : 0.04201865196228027 time for calcul the mask position with numpy : 0.0012652873992919922 nb_pixel_total : 1446 time to create 1 rle with old method : 0.0017004013061523438 time for calcul the mask position with numpy : 0.0012652873992919922 nb_pixel_total : 842 time to create 1 rle with old method : 0.0011043548583984375 time for calcul the mask position with numpy : 0.0013947486877441406 nb_pixel_total : 934 time to create 1 rle with old method : 0.0012598037719726562 time for calcul the mask position with numpy : 0.00127410888671875 nb_pixel_total : 1666 time to create 1 rle with old method : 0.0019295215606689453 time for calcul the mask position with numpy : 0.0012714862823486328 nb_pixel_total : 292 time to create 1 rle with old method : 0.0004162788391113281 time for calcul the mask position with numpy : 0.001276254653930664 nb_pixel_total : 1438 time to create 1 rle with old method : 0.0017261505126953125 time for calcul the mask position with numpy : 0.001272439956665039 nb_pixel_total : 596 time to create 1 rle with old method : 0.0007262229919433594 time for calcul the mask position with numpy : 0.001308441162109375 nb_pixel_total : 5014 time to create 1 rle with old method : 0.005615949630737305 time for calcul the mask position with numpy : 0.0012829303741455078 nb_pixel_total : 1353 time to create 1 rle with old method : 0.0016465187072753906 time for calcul the mask position with numpy : 0.001268148422241211 nb_pixel_total : 1122 time to create 1 rle with old method : 0.0013120174407958984 time for calcul the mask position with numpy : 0.0014164447784423828 nb_pixel_total : 946 time to create 1 rle with old method : 0.0011153221130371094 time for calcul the mask position with numpy : 0.0012695789337158203 nb_pixel_total : 475 time to create 1 rle with old method : 0.0005819797515869141 time for calcul the mask position with numpy : 0.0012919902801513672 nb_pixel_total : 1615 time to create 1 rle with old method : 0.0018842220306396484 time for calcul the mask position with numpy : 0.0012731552124023438 nb_pixel_total : 1340 time to create 1 rle with old method : 0.0015845298767089844 time for calcul the mask position with numpy : 0.0012848377227783203 nb_pixel_total : 3372 time to create 1 rle with old method : 0.004027605056762695 time for calcul the mask position with numpy : 0.0014290809631347656 nb_pixel_total : 829 time to create 1 rle with old method : 0.0009975433349609375 insert ignore into MTRPhoto.crop_hashtag_ids (photo_id, hashtag_id, `type`,x0,x1,y0,y1,score) VALUES (%s,%s,%s,%s,%s,%s,%s,%s) insert ignore into MTRPhoto.crop_hashtag_ids (photo_id, hashtag_id, `type`,x0,x1,y0,y1,score) VALUES (%s,%s,%s,%s,%s,%s,%s,%s) batch 1 Loaded 101 chid ids of type : 4677 Number RLEs to save : 9338 INSERT IGNORE INTO MTRPhoto.crop_segments (`crop_hashtag_id`, `x0`, `y0`, `length`) VALUES (%s, %s, %s , %s) first line : ('3973480828', '464', '201', '4') ... last line : ('3973480928', '815', '44', '5') INSERT IGNORE INTO MTRPhoto.crop_sum_segments (`crop_hashtag_id`, `sum_segments`) VALUES (%s, %s) TO DO : save crop sub photo not yet done ! After datou_step_exec type output : map_portfolio_photo : len 0 keys : dict_keys([]) Inside saveOutput : final : True verbose : True saveOutput not yet implemented for datou_step.type : sam we use saveGeneral [1189321094] map_info['map_portfolio_photo'] : {} final : True mtd_id 4573 list_pids : [1189321094] Looping around the photos to save general results len do output : 1 /1189321094Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4573', None, None, None, None, None, None, None, None) ('4573', None, '1189321094', None, None, None, None, None, None) begin to insert list_values into mtr_datou_result : length of list_values in save_final : 3 insert ignore into MTRPhoto.mtr_datou_result (mtd_id, mtr_portfolio_id,mtr_photo_id,result,result_long,result_double,hashtag_id,proba, mtr_current_id) values (%s,%s,%s,%s,%s,%s,%s,%s,%s) on duplicate key update mtr_portfolio_id = mtr_portfolio_id list_values : [('4573', None, '1189321094', 'None', None, None, None, None, None)] time used for this insertion : 0.014179706573486328 save_final save missing photos in datou_result : time spend for datou_step_exec : 12.109817743301392 time spend to save output : 0.014468669891357422 total time spend for step 1 : 12.124286413192749 caffe_path_current : About to save ! 2 After save, about to update current ! datou_cur_ids : [] len(datou.list_steps) : 1 output : {'1189321094': [[, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ], 'temp/1758728890_1864669_1189321094_9626af7f95d010f2a4fd524688d4ea22_76896585.png']} nb_objects detect : 101 ############################### TEST frcnn ################################ test frcnn Inside batchDatouExec : verbose : True ##### chargement datou SELECT name, created_at,limit_max FROM MTRDatou.mtr_datou WHERE id=4184 SELECT mtd.id, mtdt.`type`, mtd.`param`, mtd.param_json, mtdt.nb_input, mtdt.nb_output, mtdt.prod, mtdt.is_local, mtdt.is_datou_depend, mtdt.is_photo_id_local FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_step_types mtdt WHERE mtdt.`id`=mtd.`type` AND mtd.mtd_id=4184 SELECT mtd.id, mtd.mtd_id, mdsdt.id, mdsdt.name, mdsdt.description, msid.output_or_input, msid.data_order_id, mdsdt.type FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_steptype_io_datatypes msid, MTRDatou.mtr_datou_step_data_types mdsdt WHERE mtd.`type`=msid.`mtr_datou_step_type` AND mtd.mtd_id= 4184 AND msid.data_type=mdsdt.id SELECT mts_id_output, id_output, mts_id_input, id_input FROM MTRDatou.mtr_datou_step_by_step WHERE mtd_id=4184 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! no param json to modify List Step Type Loaded in datou : frcnn list_input_json : [] ##### fin chargement datou ##### chargement data ##### Call load_data_input : nb_thread : 5 origin SELECT photo_id, url FROM MTRBack.view_photos_with_urls_250912 ph WHERE photo_id IN (917754606) Found this number of photos: 1 ##### Call download_photos : nb_thread : 5 begin to download photo : 917754606 download finish for photo 917754606 we have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB ##### After download_photos length of list_filenames : 1 ; length of list_pids : 1 ; length of list_args : 1 ##### After load_data_input time to download the photos : 0.14730501174926758 #### fin chargement data Blocking on flush ? No conitnuing About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : True number of steps : 1 step1:frcnn Wed Sep 24 17:48:22 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec After prepare type args : Here we display some param of map_info ! map_filenames : {'temp/1758728902_1864669_917754606_35f3c9ae49686a6be16030c6ec25c9ee.jpg': 917754606} map_photo_id_path_extension : {917754606: {'path': 'temp/1758728902_1864669_917754606_35f3c9ae49686a6be16030c6ec25c9ee.jpg', 'extension': 'jpg'}} map_subphoto_mainphoto : {} Beginning of datou step Faster rcnn ! classes : ['background', 'plaque'] pht : 4370 caffemodel_name (should be vgg16_immat_307 but not used because net loaded outside in the fonction) : {'id': 3375, 'mtr_user_id': 31, 'name': 'detection_plaque_valcor_010622', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,plaque', 'svm_portfolios_learning': '0,0', 'photo_hashtag_type': 4370, 'photo_desc_type': 5676, 'type_classification': 'caffe_faster_rcnn', 'hashtag_id_list': '0,0'} To loadFromThcl() model_param file didn't exist model_name : detection_plaque_valcor_010622 model_type : caffe_faster_rcnn list file need : ['caffemodel', 'test.prototxt'] file exist in s3 : ['caffemodel', 'test.prototxt'] file manque in s3 : [] local folder : /data/models_weight/detection_plaque_valcor_010622 /data/models_weight/detection_plaque_valcor_010622/caffemodel size_local : 349723073 size in s3 : 349723073 create time local : 2022-07-12 14:12:27 create time in s3 : 2022-06-01 15:05:56 caffemodel already exist and didn't need to update /data/models_weight/detection_plaque_valcor_010622/test.prototxt size_local : 7163 size in s3 : 7163 create time local : 2022-07-12 14:12:27 create time in s3 : 2022-06-01 15:05:55 test.prototxt already exist and didn't need to update prototxt : /data/models_weight/detection_plaque_valcor_010622/test.prototxt caffemodel : /data/models_weight/detection_plaque_valcor_010622/caffemodel Loaded network /data/models_weight/detection_plaque_valcor_010622/caffemodel About to compute detect_faster_rcnn : len(args) : 1 Inside frcnn step exec : nb paths : 1 image_path : temp/1758728902_1864669_917754606_35f3c9ae49686a6be16030c6ec25c9ee.jpg image_size (600, 800, 3) [[[ 4 6 6] [ 5 7 7] [ 6 8 8] ... [207 215 214] [206 214 213] [206 214 213]] [[ 4 6 6] [ 5 7 7] [ 6 8 8] ... [207 215 214] [206 214 213] [206 214 213]] [[ 4 6 6] [ 5 7 7] [ 6 8 8] ... [207 215 214] [206 214 213] [206 214 213]] ... [[ 14 16 16] [ 13 15 15] [ 11 13 13] ... [198 206 205] [198 206 205] [198 206 205]] [[ 16 18 18] [ 14 16 16] [ 11 13 13] ... [206 214 213] [206 214 213] [206 214 213]] [[ 13 15 15] [ 12 14 14] [ 9 11 11] ... [210 218 217] [210 218 217] [210 218 217]]] Detection took 0.070s for 300 object proposals c : plaque list_crops.shape (72, 5) proba : 0.063844025 (374.12805, 293.92123, 430.8133, 317.81076) proba : 0.05222673 (382.18085, 297.1903, 552.35376, 344.6588) proba : 0.01227401 (345.35724, 272.4308, 468.86072, 320.72668) We are managing local photo_id len de result frcnn : 1 After datou_step_exec type output : time spend for datou_step_exec : 2.3562254905700684 time spend to save output : 0.00012230873107910156 total time spend for step 1 : 2.3563477993011475 caffe_path_current : About to save ! 1 Inside saveOutput : final : True verbose : True Inside saveFrcnn : final : True verbose : True threshold to save the result : 0.1 output flattener : [(0, 493029425, 4370, 374, 430, 293, 317, 0.063844025, None), (0, 493029425, 4370, 382, 552, 297, 344, 0.05222673, None), (0, 493029425, 4370, 345, 468, 272, 320, 0.01227401, None)] Warning : no hashtag_ids to insert in the database final : True begin to insert list_values into mtr_datou_result : length of list_values in save_final : 1 insert ignore into MTRPhoto.mtr_datou_result (mtd_id, mtr_portfolio_id,mtr_photo_id,result,result_long,result_double,hashtag_id,proba, mtr_current_id) values (%s,%s,%s,%s,%s,%s,%s,%s,%s) on duplicate key update mtr_portfolio_id = mtr_portfolio_id list_values : [('4184', None, '917754606', '0', 0, '0', 493061979, '0', None)] time used for this insertion : 0.016937971115112305 [917754606] map_info['map_portfolio_photo'] : {} final : True mtd_id 4184 list_pids : [917754606] Looping around the photos to save general results len do output : 1 /0 before output type Managing all output in save final without adding information in the mtr_datou_result ('4184', None, None, None, None, None, None, None, None) ('4184', None, '917754606', None, None, None, None, None, None) begin to insert list_values into mtr_datou_result : length of list_values in save_final : 1 insert ignore into MTRPhoto.mtr_datou_result (mtd_id, mtr_portfolio_id,mtr_photo_id,result,result_long,result_double,hashtag_id,proba, mtr_current_id) values (%s,%s,%s,%s,%s,%s,%s,%s,%s) on duplicate key update mtr_portfolio_id = mtr_portfolio_id list_values : [('4184', None, '917754606', None, None, None, None, None, None)] time used for this insertion : 0.013954401016235352 save_final save missing photos in datou_result : After save, about to update current ! datou_cur_ids : [] len(datou.list_steps) : 1 output : {0: [[(0, 493029425, 4370, 374, 430, 293, 317, 0.063844025, None), (0, 493029425, 4370, 382, 552, 297, 344, 0.05222673, None), (0, 493029425, 4370, 345, 468, 272, 320, 0.01227401, None)], 'temp/1758728902_1864669_917754606_35f3c9ae49686a6be16030c6ec25c9ee.jpg']} ############################### TEST thcl ################################ TEST THCL Inside batchDatouExec : verbose : True ##### chargement datou SELECT name, created_at,limit_max FROM MTRDatou.mtr_datou WHERE id=2 SELECT mtd.id, mtdt.`type`, mtd.`param`, mtd.param_json, mtdt.nb_input, mtdt.nb_output, mtdt.prod, mtdt.is_local, mtdt.is_datou_depend, mtdt.is_photo_id_local FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_step_types mtdt WHERE mtdt.`id`=mtd.`type` AND mtd.mtd_id=2 SELECT mtd.id, mtd.mtd_id, mdsdt.id, mdsdt.name, mdsdt.description, msid.output_or_input, msid.data_order_id, mdsdt.type FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_steptype_io_datatypes msid, MTRDatou.mtr_datou_step_data_types mdsdt WHERE mtd.`type`=msid.`mtr_datou_step_type` AND mtd.mtd_id= 2 AND msid.data_type=mdsdt.id SELECT mts_id_output, id_output, mts_id_input, id_input FROM MTRDatou.mtr_datou_step_by_step WHERE mtd_id=2 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : step 1 thcl is not linked in the step_by_step architecture ! WARNING : step 2 argmax is not linked in the step_by_step architecture ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! DataTypes for each output/input checked ! no param json to modify List Step Type Loaded in datou : thcl, argmax list_input_json : [] ##### fin chargement datou ##### chargement data ##### Call load_data_input : nb_thread : 5 origin SELECT photo_id, url FROM MTRBack.view_photos_with_urls_250912 ph WHERE photo_id IN (916235064) Found this number of photos: 1 ##### Call download_photos : nb_thread : 5 begin to download photo : 916235064 download finish for photo 916235064 we have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB ##### After download_photos length of list_filenames : 1 ; length of list_pids : 1 ; length of list_args : 1 ##### After load_data_input time to download the photos : 0.1662731170654297 #### fin chargement data Blocking on flush ? No conitnuing About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : True number of steps : 2 step1:thcl Wed Sep 24 17:48:25 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec After prepare type args : Here we display some param of map_info ! map_filenames : {'temp/1758728904_1864669_916235064_6293d1bb790dc6902450e7c572b7d10b.jpg': 916235064} map_photo_id_path_extension : {916235064: {'path': 'temp/1758728904_1864669_916235064_6293d1bb790dc6902450e7c572b7d10b.jpg', 'extension': 'jpg'}} map_subphoto_mainphoto : {} Beginning of datou step Thcl ! multi_thcl or not :False multi_thcl_cond or not :False dic_thcl : {'355': 1} we are using the classfication for only one thcl 355 In convert_file_to_np l 337 : 1 l343 1 l357 after caffe.io.load_image dimension du image : (3, (66, 66, 3)) dimension displayed ! time to import caffe and check if the image exist : 0.011922836303710938 time to convert the images to numpy array : 0.002833127975463867 total time to convert the images to numpy array : 0.015055179595947266 list photo_ids error: [] list photo_ids correct : [916235064] number of photos to traite : 1 try to delete the photos incorrect in DB tagging for thcl : 355 To do loadFromThcl(), then load ParamDescType : thcl355 get_desc_type_from_thcl : type of cat SELECT id, mtr_user_id, name, pb_hashtag_id, hashtag_id_list, button_legend_list, portfolio_id_lists, photo_hashtag_type, photo_desc_type, svm_limit, limit_tagging, is_public, live, created_at, updated_at, type_classification FROM MTRDatou.classification_theme WHERE `id` IN (355) thcls : [{'id': 355, 'mtr_user_id': 31, 'name': 'car_360_1027', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'c_elysee_1027_gao__port_506302,mokka_1027_gao__port_506374,captur_1027_gao__port_506399,sorento_1027_gao__port_506192,navara_1027_gao__port_506205,xc90_1027_gao__port_506350,saxo_1027_gao__port_506052,trafic_1027_gao__port_506295,punto_evo_1027_gao__port_506066,5_1027_gao__port_506117,250_1027_gao__port_506065,d_max_1027_gao__port_506125,panamera_1027_gao__port_506387,alhambra_1027_gao__port_506381,x6_1027_gao__port_506349,vitara_1027_gao__port_506328,fiesta_1027_gao__port_506377,qashqai_1027_gao__port_506286,147_1027_gao__port_506124,c5_1027_gao__port_506172,q5_1027_gao__port_506206,giulia_1027_gao__port_506178,karl_1027_gao__port_506371,mehari_1027_gao__port_506076,911_1027_gao__port_506114,508_1027_gao__port_506329,idea_1027_gao__port_506122,megane_1027_gao__port_506220,ghibli_1027_gao__port_506174,touareg_1027_gao__port_506224,i10_1027_gao__port_506232,jumper_1027_gao__port_506234,classe_clk_1027_gao__port_506173,kuga_1027_gao__port_506181,ct_1027_gao__port_506323,leon_1027_gao__port_506326,ds5_1027_gao__port_506376,cordoba_1027_gao__port_506048,classe_cla_1027_gao__port_506400,jumpy_1027_gao__port_506179,avensis_1027_gao__port_506311,juke_1027_gao__port_506325,4008_1027_gao__port_506402,190_series_1027_gao__port_506051,serie_3_1027_gao__port_506294,q7_1027_gao__port_506318,glc_1027_gao__port_506303,grand_vitara_1027_gao__port_506175,s40_1027_gao__port_506099,toledo_1027_gao__port_506061,5008_1027_gao__port_506337,continental_1027_gao__port_506250,coupe_1027_gao__port_506082,iq_1027_gao__port_506166,407_1027_gao__port_506133,touran_1027_gao__port_506308,300c_1027_gao__port_506078,classe_gl_1027_gao__port_506340,vivaro_1027_gao__port_506310,sl_1027_gao__port_506100,elise_1027_gao__port_506121,1007_1027_gao__port_506070,i40_1027_gao__port_506218,bipper_tepee_1027_gao__port_506227,focus_1027_gao__port_506272,primera_1027_gao__port_506147,r4_1027_gao__port_506160,a8_1027_gao__port_506265,boxer_1027_gao__port_506202,s5_1027_gao__port_506222,r21_1027_gao__port_506093,c3_1027_gao__port_506257,santa_fe_1027_gao__port_506208,m4_1027_gao__port_506344,safrane_1027_gao__port_506077,classe_gle_1027_gao__port_506395,0_1027_gao__port_506094,ix35_1027_gao__port_506219,carens_1027_gao__port_506298,classe_a_1027_gao__port_506339,ix20_1027_gao__port_506343,note_1027_gao__port_506365,a5_1027_gao__port_506200,sx4_1027_gao__port_506348,sandero_1027_gao__port_506198,3008_1027_gao__port_506385,q50_1027_gao__port_506239,latitude_1027_gao__port_506236,v40_1027_gao__port_506391,xsara_1027_gao__port_506087,grand_c_max_1027_gao__port_506342,swift_1027_gao__port_506149,serie_1_1027_gao__port_506184,xc70_1027_gao__port_506393,master_1027_gao__port_506203,clio_1027_gao__port_506280,duster_1027_gao__port_506216,traveller_1027_gao__port_506403,tipo_1027_gao__port_506355,rav_4_1027_gao__port_506332,coccinelle_1027_gao__port_506259,spacetourer_1027_gao__port_506401,xe_1027_gao__port_506357,ds3_1027_gao__port_506324,mx_5_1027_gao__port_506098,land_cruiser_1027_gao__port_506315,classe_b_1027_gao__port_506335,806_1027_gao__port_506088,rx_8_1027_gao__port_506046,spark_1027_gao__port_506185,6_1027_gao__port_506171,bravo_1027_gao__port_506080,nx_1027_gao__port_506345,sharan_1027_gao__port_506347,x_type_1027_gao__port_506067,jimny_1027_gao__port_506233,wrangler_1027_gao__port_506225,c_crosser_1027_gao__port_506312,v70_1027_gao__port_506278,classe_e_1027_gao__port_506300,classe_v_1027_gao__port_506258,m3_1027_gao__port_506182,abarth_500_1027_gao__port_506226,serie_6_1027_gao__port_506262,modus_1027_gao__port_506146,3_1027_gao__port_506113,405_1027_gao__port_506108,allroad_1027_gao__port_506297,auris_1027_gao__port_506322,galaxy_1027_gao__port_506143,giulietta_1027_gao__port_506363,106_1027_gao__port_506073,classe_m_1027_gao__port_506154,espace_1027_gao__port_506313,panda_1027_gao__port_506189,rcz_1027_gao__port_506197,4007_1027_gao__port_506162,classe_cl_1027_gao__port_506249,leaf_1027_gao__port_506139,octavia_1027_gao__port_506237,ds4_1027_gao__port_506336,freelander_1027_gao__port_506084,evasion_1027_gao__port_506109,punto_1027_gao__port_506106,2cv_1027_gao__port_506045,x4_1027_gao__port_506392,antara_1027_gao__port_506247,murano_1027_gao__port_506316,alto_1027_gao__port_506201,meriva_1027_gao__port_506353,orlando_1027_gao__port_506305,new_beetle_1027_gao__port_506050,306_1027_gao__port_506145,tiguan_1027_gao__port_506362,s_type_1027_gao__port_506101,c1_1027_gao__port_506128,vectra_1027_gao__port_506044,outlander_1027_gao__port_506317,307_1027_gao__port_506074,a6_s6_1027_gao__port_506134,nemo_combi_1027_gao__port_506196,berlingo_1027_gao__port_506194,partner_1027_gao__port_506285,cayenne_1027_gao__port_506177,quattroporte_1027_gao__port_506240,c_max_1027_gao__port_506282,fabia_1027_gao__port_506396,cx_3_1027_gao__port_506281,x_trail_1027_gao__port_506264,scirocco_1027_gao__port_506276,matiz_1027_gao__port_506144,tigra_1027_gao__port_506069,escort_1027_gao__port_506091,c2_1027_gao__port_506081,mini_1027_gao__port_506168,i30_1027_gao__port_506291,picanto_1027_gao__port_506238,mito_1027_gao__port_506072,impreza_1027_gao__port_506085,kangoo_1027_gao__port_506235,a4_1027_gao__port_506193,cayman_1027_gao__port_506268,sportage_1027_gao__port_506148,up_1027_gao__port_506356,optima_1027_gao__port_506386,defender_1027_gao__port_506229,serie_2_1027_gao__port_506256,edge_1027_gao__port_506187,r19_1027_gao__port_506110,jetta_1027_gao__port_506304,eos_1027_gao__port_506115,accord_1027_gao__port_506214,yaris_1027_gao__port_506334,classe_cls_1027_gao__port_506289,polo_1027_gao__port_506361,serie_4_1027_gao__port_506366,mini_cabriolet_1027_gao__port_506204,prius_1027_gao__port_506190,lodgy_1027_gao__port_506188,serie_7_1027_gao__port_506307,c15_1027_gao__port_506055,kadjar_1027_gao__port_506389,insignia_1027_gao__port_506364,308_1027_gao__port_506279,roomster_1027_gao__port_506241,80_1027_gao__port_506057,309_1027_gao__port_506063,tucson_1027_gao__port_506320,x3_1027_gao__port_506212,xf_1027_gao__port_506263,2008_1027_gao__port_506394,passat_1027_gao__port_506306,compass_1027_gao__port_506260,twingo_1027_gao__port_506309,micra_1027_gao__port_506221,golf_1027_gao__port_506155,soul_1027_gao__port_506176,rapid_1027_gao__port_506398,forester_1027_gao__port_506360,slk_1027_gao__port_506210,forfour_1027_gao__port_506341,serie_5_1027_gao__port_506209,xj_1027_gao__port_506170,pajero_1027_gao__port_506097,agila_1027_gao__port_506119,a6_1027_gao__port_506163,fox_1027_gao__port_506092,boxster_1027_gao__port_506267,altea_1027_gao__port_506246,samurai_1027_gao__port_506047,trax_1027_gao__port_506296,getz_1027_gao__port_506058,cherokee_1027_gao__port_506269,koleos_1027_gao__port_506378,z_series_1027_gao__port_506123,ecosport_1027_gao__port_506271,space_star_1027_gao__port_506277,rs3_sportback_1027_gao__port_506207,civic_1027_gao__port_506141,talisman_1027_gao__port_506390,f_pace_1027_gao__port_506314,classe_c_1027_gao__port_506299,tt_1027_gao__port_506075,pathfinder_1027_gao__port_506183,156_1027_gao__port_506157,cx_5_1027_gao__port_506228,scenic_1027_gao__port_506255,yeti_1027_gao__port_506358,mustang_1027_gao__port_506053,stilo_1027_gao__port_506060,ateca_1027_gao__port_506382,fiorino_1027_gao__port_506217,classe_glk_1027_gao__port_506290,fortwo_1027_gao__port_506230,cruze_1027_gao__port_506186,107_1027_gao__port_506213,aygo_1027_gao__port_506248,rx_1027_gao__port_506354,500_1027_gao__port_506245,bora_1027_gao__port_506104,transit_1027_gao__port_506111,pt_cruiser_1027_gao__port_506054,patrol_1027_gao__port_506068,r8_1027_gao__port_506156,xm_1027_gao__port_506102,s60_1027_gao__port_506191,aveo_1027_gao__port_506158,captiva_1027_gao__port_506159,ax_1027_gao__port_506153,rexton_1027_gao__port_506107,camaro_1027_gao__port_506056,ypsilon_1027_gao__port_506131,delta_1027_gao__port_506165,c4_1027_gao__port_506370,zx_1027_gao__port_506161,verso_1027_gao__port_506242,superb_1027_gao__port_506327,r5_1027_gao__port_506253,caddy_1027_gao__port_506330,x5_1027_gao__port_506243,f_type_1027_gao__port_506231,fusion_1027_gao__port_506096,dokker_1027_gao__port_506331,205_1027_gao__port_506062,macan_1027_gao__port_506195,tourneo_1027_gao__port_506369,108_1027_gao__port_506384,9_3_1027_gao__port_506071,mondeo_1027_gao__port_506116,cr_v_1027_gao__port_506164,c30_1027_gao__port_506090,pulsar_1027_gao__port_506397,ibiza_1027_gao__port_506273,a1_1027_gao__port_506338,matrix_1027_gao__port_506140,carnival_1027_gao__port_506136,xantia_1027_gao__port_506086,terrano_1027_gao__port_506083,q3_1027_gao__port_506275,hr_v_1027_gao__port_506283,expert_1027_gao__port_506142,multivan_1027_gao__port_506383,venga_1027_gao__port_506380,scudo_1027_gao__port_506129,laguna_1027_gao__port_506368,vel_satis_1027_gao__port_506130,b_max_1027_gao__port_506367,ignis_1027_gao__port_506292,159_1027_gao__port_506064,grande_punto_1027_gao__port_506138,logan_1027_gao__port_506167,s_max_1027_gao__port_506223,caravelle_1027_gao__port_506351,adam_1027_gao__port_506079,406_1027_gao__port_506132,q30_1027_gao__port_506293,almera_1027_gao__port_506089,corsa_1027_gao__port_506095,corolla_1027_gao__port_506120,xc60_1027_gao__port_506388,viano_1027_gao__port_506211,pro_cee_d_1027_gao__port_506274,a3_1027_gao__port_506321,v50_1027_gao__port_506150,voyager_1027_gao__port_506169,corvette_1027_gao__port_506049,rio_1027_gao__port_506379,jazz_1027_gao__port_506252,200_1027_gao__port_506112,tts_1027_gao__port_506199,zafira_1027_gao__port_506287,asx_1027_gao__port_506266,607_1027_gao__port_506118,207_1027_gao__port_506103,classe_s_1027_gao__port_506301,c6_1027_gao__port_506105,express_1027_gao__port_506137,classe_gla_1027_gao__port_506352,v60_1027_gao__port_506333,ka_1027_gao__port_506180,range_rover_1027_gao__port_506254,discovery_1027_gao__port_506375,classe_r_1027_gao__port_506270,transporter_1027_gao__port_506319,cee_d_1027_gao__port_506288,zoe_1027_gao__port_506244,i20_1027_gao__port_506284,gtv_1027_gao__port_506059,s4_avant_1027_gao__port_506261,x1_1027_gao__port_506372,autres_1027_gao__port_506127,208_1027_gao__port_506359,c8_1027_gao__port_506135,astra_1027_gao__port_506215,2_1027_gao__port_506151,doblo_1027_gao__port_506251,807_1027_gao__port_506152,206_1027_gao__port_506126,a7_1027_gao__port_506373,renegade_1027_gao__port_506346', 'svm_portfolios_learning': '506302,506374,506399,506192,506205,506350,506052,506295,506066,506117,506065,506125,506387,506381,506349,506328,506377,506286,506124,506172,506206,506178,506371,506076,506114,506329,506122,506220,506174,506224,506232,506234,506173,506181,506323,506326,506376,506048,506400,506179,506311,506325,506402,506051,506294,506318,506303,506175,506099,506061,506337,506250,506082,506166,506133,506308,506078,506340,506310,506100,506121,506070,506218,506227,506272,506147,506160,506265,506202,506222,506093,506257,506208,506344,506077,506395,506094,506219,506298,506339,506343,506365,506200,506348,506198,506385,506239,506236,506391,506087,506342,506149,506184,506393,506203,506280,506216,506403,506355,506332,506259,506401,506357,506324,506098,506315,506335,506088,506046,506185,506171,506080,506345,506347,506067,506233,506225,506312,506278,506300,506258,506182,506226,506262,506146,506113,506108,506297,506322,506143,506363,506073,506154,506313,506189,506197,506162,506249,506139,506237,506336,506084,506109,506106,506045,506392,506247,506316,506201,506353,506305,506050,506145,506362,506101,506128,506044,506317,506074,506134,506196,506194,506285,506177,506240,506282,506396,506281,506264,506276,506144,506069,506091,506081,506168,506291,506238,506072,506085,506235,506193,506268,506148,506356,506386,506229,506256,506187,506110,506304,506115,506214,506334,506289,506361,506366,506204,506190,506188,506307,506055,506389,506364,506279,506241,506057,506063,506320,506212,506263,506394,506306,506260,506309,506221,506155,506176,506398,506360,506210,506341,506209,506170,506097,506119,506163,506092,506267,506246,506047,506296,506058,506269,506378,506123,506271,506277,506207,506141,506390,506314,506299,506075,506183,506157,506228,506255,506358,506053,506060,506382,506217,506290,506230,506186,506213,506248,506354,506245,506104,506111,506054,506068,506156,506102,506191,506158,506159,506153,506107,506056,506131,506165,506370,506161,506242,506327,506253,506330,506243,506231,506096,506331,506062,506195,506369,506384,506071,506116,506164,506090,506397,506273,506338,506140,506136,506086,506083,506275,506283,506142,506383,506380,506129,506368,506130,506367,506292,506064,506138,506167,506223,506351,506079,506132,506293,506089,506095,506120,506388,506211,506274,506321,506150,506169,506049,506379,506252,506112,506199,506287,506266,506118,506103,506301,506105,506137,506352,506333,506180,506254,506375,506270,506319,506288,506244,506284,506059,506261,506372,506127,506359,506135,506215,506151,506251,506152,506126,506373,506346', 'photo_hashtag_type': 332, 'photo_desc_type': 3390, 'type_classification': 'caffe', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 355, 'mtr_user_id': 31, 'name': 'car_360_1027', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'c_elysee_1027_gao__port_506302,mokka_1027_gao__port_506374,captur_1027_gao__port_506399,sorento_1027_gao__port_506192,navara_1027_gao__port_506205,xc90_1027_gao__port_506350,saxo_1027_gao__port_506052,trafic_1027_gao__port_506295,punto_evo_1027_gao__port_506066,5_1027_gao__port_506117,250_1027_gao__port_506065,d_max_1027_gao__port_506125,panamera_1027_gao__port_506387,alhambra_1027_gao__port_506381,x6_1027_gao__port_506349,vitara_1027_gao__port_506328,fiesta_1027_gao__port_506377,qashqai_1027_gao__port_506286,147_1027_gao__port_506124,c5_1027_gao__port_506172,q5_1027_gao__port_506206,giulia_1027_gao__port_506178,karl_1027_gao__port_506371,mehari_1027_gao__port_506076,911_1027_gao__port_506114,508_1027_gao__port_506329,idea_1027_gao__port_506122,megane_1027_gao__port_506220,ghibli_1027_gao__port_506174,touareg_1027_gao__port_506224,i10_1027_gao__port_506232,jumper_1027_gao__port_506234,classe_clk_1027_gao__port_506173,kuga_1027_gao__port_506181,ct_1027_gao__port_506323,leon_1027_gao__port_506326,ds5_1027_gao__port_506376,cordoba_1027_gao__port_506048,classe_cla_1027_gao__port_506400,jumpy_1027_gao__port_506179,avensis_1027_gao__port_506311,juke_1027_gao__port_506325,4008_1027_gao__port_506402,190_series_1027_gao__port_506051,serie_3_1027_gao__port_506294,q7_1027_gao__port_506318,glc_1027_gao__port_506303,grand_vitara_1027_gao__port_506175,s40_1027_gao__port_506099,toledo_1027_gao__port_506061,5008_1027_gao__port_506337,continental_1027_gao__port_506250,coupe_1027_gao__port_506082,iq_1027_gao__port_506166,407_1027_gao__port_506133,touran_1027_gao__port_506308,300c_1027_gao__port_506078,classe_gl_1027_gao__port_506340,vivaro_1027_gao__port_506310,sl_1027_gao__port_506100,elise_1027_gao__port_506121,1007_1027_gao__port_506070,i40_1027_gao__port_506218,bipper_tepee_1027_gao__port_506227,focus_1027_gao__port_506272,primera_1027_gao__port_506147,r4_1027_gao__port_506160,a8_1027_gao__port_506265,boxer_1027_gao__port_506202,s5_1027_gao__port_506222,r21_1027_gao__port_506093,c3_1027_gao__port_506257,santa_fe_1027_gao__port_506208,m4_1027_gao__port_506344,safrane_1027_gao__port_506077,classe_gle_1027_gao__port_506395,0_1027_gao__port_506094,ix35_1027_gao__port_506219,carens_1027_gao__port_506298,classe_a_1027_gao__port_506339,ix20_1027_gao__port_506343,note_1027_gao__port_506365,a5_1027_gao__port_506200,sx4_1027_gao__port_506348,sandero_1027_gao__port_506198,3008_1027_gao__port_506385,q50_1027_gao__port_506239,latitude_1027_gao__port_506236,v40_1027_gao__port_506391,xsara_1027_gao__port_506087,grand_c_max_1027_gao__port_506342,swift_1027_gao__port_506149,serie_1_1027_gao__port_506184,xc70_1027_gao__port_506393,master_1027_gao__port_506203,clio_1027_gao__port_506280,duster_1027_gao__port_506216,traveller_1027_gao__port_506403,tipo_1027_gao__port_506355,rav_4_1027_gao__port_506332,coccinelle_1027_gao__port_506259,spacetourer_1027_gao__port_506401,xe_1027_gao__port_506357,ds3_1027_gao__port_506324,mx_5_1027_gao__port_506098,land_cruiser_1027_gao__port_506315,classe_b_1027_gao__port_506335,806_1027_gao__port_506088,rx_8_1027_gao__port_506046,spark_1027_gao__port_506185,6_1027_gao__port_506171,bravo_1027_gao__port_506080,nx_1027_gao__port_506345,sharan_1027_gao__port_506347,x_type_1027_gao__port_506067,jimny_1027_gao__port_506233,wrangler_1027_gao__port_506225,c_crosser_1027_gao__port_506312,v70_1027_gao__port_506278,classe_e_1027_gao__port_506300,classe_v_1027_gao__port_506258,m3_1027_gao__port_506182,abarth_500_1027_gao__port_506226,serie_6_1027_gao__port_506262,modus_1027_gao__port_506146,3_1027_gao__port_506113,405_1027_gao__port_506108,allroad_1027_gao__port_506297,auris_1027_gao__port_506322,galaxy_1027_gao__port_506143,giulietta_1027_gao__port_506363,106_1027_gao__port_506073,classe_m_1027_gao__port_506154,espace_1027_gao__port_506313,panda_1027_gao__port_506189,rcz_1027_gao__port_506197,4007_1027_gao__port_506162,classe_cl_1027_gao__port_506249,leaf_1027_gao__port_506139,octavia_1027_gao__port_506237,ds4_1027_gao__port_506336,freelander_1027_gao__port_506084,evasion_1027_gao__port_506109,punto_1027_gao__port_506106,2cv_1027_gao__port_506045,x4_1027_gao__port_506392,antara_1027_gao__port_506247,murano_1027_gao__port_506316,alto_1027_gao__port_506201,meriva_1027_gao__port_506353,orlando_1027_gao__port_506305,new_beetle_1027_gao__port_506050,306_1027_gao__port_506145,tiguan_1027_gao__port_506362,s_type_1027_gao__port_506101,c1_1027_gao__port_506128,vectra_1027_gao__port_506044,outlander_1027_gao__port_506317,307_1027_gao__port_506074,a6_s6_1027_gao__port_506134,nemo_combi_1027_gao__port_506196,berlingo_1027_gao__port_506194,partner_1027_gao__port_506285,cayenne_1027_gao__port_506177,quattroporte_1027_gao__port_506240,c_max_1027_gao__port_506282,fabia_1027_gao__port_506396,cx_3_1027_gao__port_506281,x_trail_1027_gao__port_506264,scirocco_1027_gao__port_506276,matiz_1027_gao__port_506144,tigra_1027_gao__port_506069,escort_1027_gao__port_506091,c2_1027_gao__port_506081,mini_1027_gao__port_506168,i30_1027_gao__port_506291,picanto_1027_gao__port_506238,mito_1027_gao__port_506072,impreza_1027_gao__port_506085,kangoo_1027_gao__port_506235,a4_1027_gao__port_506193,cayman_1027_gao__port_506268,sportage_1027_gao__port_506148,up_1027_gao__port_506356,optima_1027_gao__port_506386,defender_1027_gao__port_506229,serie_2_1027_gao__port_506256,edge_1027_gao__port_506187,r19_1027_gao__port_506110,jetta_1027_gao__port_506304,eos_1027_gao__port_506115,accord_1027_gao__port_506214,yaris_1027_gao__port_506334,classe_cls_1027_gao__port_506289,polo_1027_gao__port_506361,serie_4_1027_gao__port_506366,mini_cabriolet_1027_gao__port_506204,prius_1027_gao__port_506190,lodgy_1027_gao__port_506188,serie_7_1027_gao__port_506307,c15_1027_gao__port_506055,kadjar_1027_gao__port_506389,insignia_1027_gao__port_506364,308_1027_gao__port_506279,roomster_1027_gao__port_506241,80_1027_gao__port_506057,309_1027_gao__port_506063,tucson_1027_gao__port_506320,x3_1027_gao__port_506212,xf_1027_gao__port_506263,2008_1027_gao__port_506394,passat_1027_gao__port_506306,compass_1027_gao__port_506260,twingo_1027_gao__port_506309,micra_1027_gao__port_506221,golf_1027_gao__port_506155,soul_1027_gao__port_506176,rapid_1027_gao__port_506398,forester_1027_gao__port_506360,slk_1027_gao__port_506210,forfour_1027_gao__port_506341,serie_5_1027_gao__port_506209,xj_1027_gao__port_506170,pajero_1027_gao__port_506097,agila_1027_gao__port_506119,a6_1027_gao__port_506163,fox_1027_gao__port_506092,boxster_1027_gao__port_506267,altea_1027_gao__port_506246,samurai_1027_gao__port_506047,trax_1027_gao__port_506296,getz_1027_gao__port_506058,cherokee_1027_gao__port_506269,koleos_1027_gao__port_506378,z_series_1027_gao__port_506123,ecosport_1027_gao__port_506271,space_star_1027_gao__port_506277,rs3_sportback_1027_gao__port_506207,civic_1027_gao__port_506141,talisman_1027_gao__port_506390,f_pace_1027_gao__port_506314,classe_c_1027_gao__port_506299,tt_1027_gao__port_506075,pathfinder_1027_gao__port_506183,156_1027_gao__port_506157,cx_5_1027_gao__port_506228,scenic_1027_gao__port_506255,yeti_1027_gao__port_506358,mustang_1027_gao__port_506053,stilo_1027_gao__port_506060,ateca_1027_gao__port_506382,fiorino_1027_gao__port_506217,classe_glk_1027_gao__port_506290,fortwo_1027_gao__port_506230,cruze_1027_gao__port_506186,107_1027_gao__port_506213,aygo_1027_gao__port_506248,rx_1027_gao__port_506354,500_1027_gao__port_506245,bora_1027_gao__port_506104,transit_1027_gao__port_506111,pt_cruiser_1027_gao__port_506054,patrol_1027_gao__port_506068,r8_1027_gao__port_506156,xm_1027_gao__port_506102,s60_1027_gao__port_506191,aveo_1027_gao__port_506158,captiva_1027_gao__port_506159,ax_1027_gao__port_506153,rexton_1027_gao__port_506107,camaro_1027_gao__port_506056,ypsilon_1027_gao__port_506131,delta_1027_gao__port_506165,c4_1027_gao__port_506370,zx_1027_gao__port_506161,verso_1027_gao__port_506242,superb_1027_gao__port_506327,r5_1027_gao__port_506253,caddy_1027_gao__port_506330,x5_1027_gao__port_506243,f_type_1027_gao__port_506231,fusion_1027_gao__port_506096,dokker_1027_gao__port_506331,205_1027_gao__port_506062,macan_1027_gao__port_506195,tourneo_1027_gao__port_506369,108_1027_gao__port_506384,9_3_1027_gao__port_506071,mondeo_1027_gao__port_506116,cr_v_1027_gao__port_506164,c30_1027_gao__port_506090,pulsar_1027_gao__port_506397,ibiza_1027_gao__port_506273,a1_1027_gao__port_506338,matrix_1027_gao__port_506140,carnival_1027_gao__port_506136,xantia_1027_gao__port_506086,terrano_1027_gao__port_506083,q3_1027_gao__port_506275,hr_v_1027_gao__port_506283,expert_1027_gao__port_506142,multivan_1027_gao__port_506383,venga_1027_gao__port_506380,scudo_1027_gao__port_506129,laguna_1027_gao__port_506368,vel_satis_1027_gao__port_506130,b_max_1027_gao__port_506367,ignis_1027_gao__port_506292,159_1027_gao__port_506064,grande_punto_1027_gao__port_506138,logan_1027_gao__port_506167,s_max_1027_gao__port_506223,caravelle_1027_gao__port_506351,adam_1027_gao__port_506079,406_1027_gao__port_506132,q30_1027_gao__port_506293,almera_1027_gao__port_506089,corsa_1027_gao__port_506095,corolla_1027_gao__port_506120,xc60_1027_gao__port_506388,viano_1027_gao__port_506211,pro_cee_d_1027_gao__port_506274,a3_1027_gao__port_506321,v50_1027_gao__port_506150,voyager_1027_gao__port_506169,corvette_1027_gao__port_506049,rio_1027_gao__port_506379,jazz_1027_gao__port_506252,200_1027_gao__port_506112,tts_1027_gao__port_506199,zafira_1027_gao__port_506287,asx_1027_gao__port_506266,607_1027_gao__port_506118,207_1027_gao__port_506103,classe_s_1027_gao__port_506301,c6_1027_gao__port_506105,express_1027_gao__port_506137,classe_gla_1027_gao__port_506352,v60_1027_gao__port_506333,ka_1027_gao__port_506180,range_rover_1027_gao__port_506254,discovery_1027_gao__port_506375,classe_r_1027_gao__port_506270,transporter_1027_gao__port_506319,cee_d_1027_gao__port_506288,zoe_1027_gao__port_506244,i20_1027_gao__port_506284,gtv_1027_gao__port_506059,s4_avant_1027_gao__port_506261,x1_1027_gao__port_506372,autres_1027_gao__port_506127,208_1027_gao__port_506359,c8_1027_gao__port_506135,astra_1027_gao__port_506215,2_1027_gao__port_506151,doblo_1027_gao__port_506251,807_1027_gao__port_506152,206_1027_gao__port_506126,a7_1027_gao__port_506373,renegade_1027_gao__port_506346', 'svm_portfolios_learning': '506302,506374,506399,506192,506205,506350,506052,506295,506066,506117,506065,506125,506387,506381,506349,506328,506377,506286,506124,506172,506206,506178,506371,506076,506114,506329,506122,506220,506174,506224,506232,506234,506173,506181,506323,506326,506376,506048,506400,506179,506311,506325,506402,506051,506294,506318,506303,506175,506099,506061,506337,506250,506082,506166,506133,506308,506078,506340,506310,506100,506121,506070,506218,506227,506272,506147,506160,506265,506202,506222,506093,506257,506208,506344,506077,506395,506094,506219,506298,506339,506343,506365,506200,506348,506198,506385,506239,506236,506391,506087,506342,506149,506184,506393,506203,506280,506216,506403,506355,506332,506259,506401,506357,506324,506098,506315,506335,506088,506046,506185,506171,506080,506345,506347,506067,506233,506225,506312,506278,506300,506258,506182,506226,506262,506146,506113,506108,506297,506322,506143,506363,506073,506154,506313,506189,506197,506162,506249,506139,506237,506336,506084,506109,506106,506045,506392,506247,506316,506201,506353,506305,506050,506145,506362,506101,506128,506044,506317,506074,506134,506196,506194,506285,506177,506240,506282,506396,506281,506264,506276,506144,506069,506091,506081,506168,506291,506238,506072,506085,506235,506193,506268,506148,506356,506386,506229,506256,506187,506110,506304,506115,506214,506334,506289,506361,506366,506204,506190,506188,506307,506055,506389,506364,506279,506241,506057,506063,506320,506212,506263,506394,506306,506260,506309,506221,506155,506176,506398,506360,506210,506341,506209,506170,506097,506119,506163,506092,506267,506246,506047,506296,506058,506269,506378,506123,506271,506277,506207,506141,506390,506314,506299,506075,506183,506157,506228,506255,506358,506053,506060,506382,506217,506290,506230,506186,506213,506248,506354,506245,506104,506111,506054,506068,506156,506102,506191,506158,506159,506153,506107,506056,506131,506165,506370,506161,506242,506327,506253,506330,506243,506231,506096,506331,506062,506195,506369,506384,506071,506116,506164,506090,506397,506273,506338,506140,506136,506086,506083,506275,506283,506142,506383,506380,506129,506368,506130,506367,506292,506064,506138,506167,506223,506351,506079,506132,506293,506089,506095,506120,506388,506211,506274,506321,506150,506169,506049,506379,506252,506112,506199,506287,506266,506118,506103,506301,506105,506137,506352,506333,506180,506254,506375,506270,506319,506288,506244,506284,506059,506261,506372,506127,506359,506135,506215,506151,506251,506152,506126,506373,506346', 'photo_hashtag_type': 332, 'photo_desc_type': 3390, 'type_classification': 'caffe', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 3390 SELECT * FROM MTRDatou.photo_desc_type_params WHERE id in (3390) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (3390, 'car_360_1027', 16384, 25088, 'car_360_1027', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2017, 10, 28, 12, 29, 27), datetime.datetime(2017, 10, 28, 12, 29, 27)) To loadFromThcl() : net_3390 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 6482 max_wait_temp : 1 max_wait : 0 SELECT * FROM MTRDatou.photo_desc_type_params WHERE id in (3390) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (3390, 'car_360_1027', 16384, 25088, 'car_360_1027', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2017, 10, 28, 12, 29, 27), datetime.datetime(2017, 10, 28, 12, 29, 27)) param : , param.caffemodel : car_360_1027 None mean_file_type : mean_file_path : prototxt_file_path : model : car_360_1027 Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : car_360_1027 model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : [] local folder : /data/models_weight/car_360_1027 /data/models_weight/car_360_1027/caffemodel size_local : 542944640 size in s3 : 542944640 create time local : 2021-08-09 05:28:34 create time in s3 : 2021-08-06 17:57:43 caffemodel already exist and didn't need to update /data/models_weight/car_360_1027/deploy_conv_normal.prototxt size_local : 4626 size in s3 : 4626 create time local : 2021-08-09 05:28:34 create time in s3 : 2021-08-06 17:57:42 deploy_conv_normal.prototxt already exist and didn't need to update /data/models_weight/car_360_1027/deploy_fc.prototxt size_local : 1132 size in s3 : 1132 create time local : 2021-08-09 05:28:34 create time in s3 : 2021-08-06 17:57:43 deploy_fc.prototxt already exist and didn't need to update /data/models_weight/car_360_1027/deploy.prototxt size_local : 5654 size in s3 : 5654 create time local : 2021-08-09 05:28:34 create time in s3 : 2021-08-06 17:57:42 deploy.prototxt already exist and didn't need to update /data/models_weight/car_360_1027/mean.npy size_local : 1572944 size in s3 : 1572944 create time local : 2021-08-09 05:28:34 create time in s3 : 2021-08-06 17:57:55 mean.npy already exist and didn't need to update /data/models_weight/car_360_1027/synset_words.txt size_local : 13687 size in s3 : 13687 create time local : 2021-08-09 05:28:34 create time in s3 : 2021-08-06 17:57:43 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/:/home/admin/workarea/git/apy/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/car_360_1027/deploy.prototxt caffemodel_filename : /data/models_weight/car_360_1027/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 6482 max_wait_temp : 1 max_wait : 0 dict_keys(['pool5', 'prob']) time used to do the prepocess of the images : 0.01124715805053711 time used to do the prediction : 0.0572812557220459 save descriptor for thcl : 355 (1, 512, 7, 7) Got the blobs of the net to insert : [0, 0, 0, 0, 0, 0, 0, 0, 0, 0] code_as_byte_string:b'0000000000'| time to traite the descriptors : 0.050583839416503906 Testing : ['916235064'] In select_photos_meta_from_ids: SELECT photo_id, url, FROM_UNIXTIME(uploaded_at), latitude, longitude, text FROM MTRBack.view_photos_with_urls_250912 WHERE photo_id IN (916235064) result : {916235064: {'photo_id': 916235064, 'url': 'https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/2017/10/14/6293d1bb790dc6902450e7c572b7d10b.jpg', 'latitude': None, 'longitude': None, 'text': None}} list_photo_exists : [916235064] storage_type for insertDescriptorsMulti : 1 To insert : 916235064 time to insert the descriptors : 0.5178940296173096 After datou_step_exec type output : map_portfolio_photo : len 0 keys : dict_keys([]) Inside saveOutput : final : False verbose : True time used to find the portfolios of the photos select button_legend_list from MTRDatou.classification_theme where id = 355 SAVE THCL, output : {'916235064': [[('916235064', 'c_elysee_1027_gao__port_506302', 0.0018817618, 332, '355'), ('916235064', 'mokka_1027_gao__port_506374', 0.0011637942, 332, '355'), ('916235064', 'captur_1027_gao__port_506399', 0.0008158403, 332, '355'), ('916235064', 'sorento_1027_gao__port_506192', 0.001177487, 332, '355'), ('916235064', 'navara_1027_gao__port_506205', 0.0025849529, 332, '355'), ('916235064', 'xc90_1027_gao__port_506350', 0.0041703535, 332, '355'), ('916235064', 'saxo_1027_gao__port_506052', 0.0034804884, 332, '355'), ('916235064', 'trafic_1027_gao__port_506295', 0.007365642, 332, '355'), ('916235064', 'punto_evo_1027_gao__port_506066', 0.0021890248, 332, '355'), ('916235064', '5_1027_gao__port_506117', 0.0005798624, 332, '355'), ('916235064', '250_1027_gao__port_506065', 0.0045911963, 332, '355'), ('916235064', 'd_max_1027_gao__port_506125', 0.003158559, 332, '355'), ('916235064', 'panamera_1027_gao__port_506387', 0.0022511254, 332, '355'), ('916235064', 'alhambra_1027_gao__port_506381', 0.0053211725, 332, '355'), ('916235064', 'x6_1027_gao__port_506349', 0.0011001274, 332, '355'), ('916235064', 'vitara_1027_gao__port_506328', 0.0054025697, 332, '355'), ('916235064', 'fiesta_1027_gao__port_506377', 0.003918864, 332, '355'), ('916235064', 'qashqai_1027_gao__port_506286', 0.0014788493, 332, '355'), ('916235064', '147_1027_gao__port_506124', 0.0019777154, 332, '355'), ('916235064', 'c5_1027_gao__port_506172', 0.0012445294, 332, '355'), ('916235064', 'q5_1027_gao__port_506206', 0.0015052112, 332, '355'), ('916235064', 'giulia_1027_gao__port_506178', 0.0021694666, 332, '355'), ('916235064', 'karl_1027_gao__port_506371', 0.0027080069, 332, '355'), ('916235064', 'mehari_1027_gao__port_506076', 0.0047028693, 332, '355'), ('916235064', '911_1027_gao__port_506114', 0.0019418759, 332, '355'), ('916235064', '508_1027_gao__port_506329', 0.00095878704, 332, '355'), ('916235064', 'idea_1027_gao__port_506122', 0.0007701506, 332, '355'), ('916235064', 'megane_1027_gao__port_506220', 0.0019471889, 332, '355'), ('916235064', 'ghibli_1027_gao__port_506174', 0.001372601, 332, '355'), ('916235064', 'touareg_1027_gao__port_506224', 0.0016204873, 332, '355'), ('916235064', 'i10_1027_gao__port_506232', 0.0013925376, 332, '355'), ('916235064', 'jumper_1027_gao__port_506234', 0.0100444835, 332, '355'), ('916235064', 'classe_clk_1027_gao__port_506173', 0.0010793903, 332, '355'), ('916235064', 'kuga_1027_gao__port_506181', 0.00084482884, 332, '355'), ('916235064', 'ct_1027_gao__port_506323', 0.0012523653, 332, '355'), ('916235064', 'leon_1027_gao__port_506326', 0.002584786, 332, '355'), ('916235064', 'ds5_1027_gao__port_506376', 0.0012432766, 332, '355'), ('916235064', 'cordoba_1027_gao__port_506048', 0.0028654227, 332, '355'), ('916235064', 'classe_cla_1027_gao__port_506400', 0.0012950996, 332, '355'), ('916235064', 'jumpy_1027_gao__port_506179', 0.010339205, 332, '355'), ('916235064', 'avensis_1027_gao__port_506311', 0.0018770488, 332, '355'), ('916235064', 'juke_1027_gao__port_506325', 0.0011343176, 332, '355'), ('916235064', '4008_1027_gao__port_506402', 0.0015760058, 332, '355'), ('916235064', '190_series_1027_gao__port_506051', 0.003980508, 332, '355'), ('916235064', 'serie_3_1027_gao__port_506294', 0.0028743919, 332, '355'), ('916235064', 'q7_1027_gao__port_506318', 0.0023359605, 332, '355'), ('916235064', 'glc_1027_gao__port_506303', 0.0012107468, 332, '355'), ('916235064', 'grand_vitara_1027_gao__port_506175', 0.0011447786, 332, '355'), ('916235064', 's40_1027_gao__port_506099', 0.0022346452, 332, '355'), ('916235064', 'toledo_1027_gao__port_506061', 0.0017465387, 332, '355'), ('916235064', '5008_1027_gao__port_506337', 0.004699124, 332, '355'), ('916235064', 'continental_1027_gao__port_506250', 0.00219136, 332, '355'), ('916235064', 'coupe_1027_gao__port_506082', 0.002263447, 332, '355'), ('916235064', 'iq_1027_gao__port_506166', 0.0018176099, 332, '355'), ('916235064', '407_1027_gao__port_506133', 0.0009057978, 332, '355'), ('916235064', 'touran_1027_gao__port_506308', 0.0020403562, 332, '355'), ('916235064', '300c_1027_gao__port_506078', 0.002533732, 332, '355'), ('916235064', 'classe_gl_1027_gao__port_506340', 0.004489175, 332, '355'), ('916235064', 'vivaro_1027_gao__port_506310', 0.0034243749, 332, '355'), ('916235064', 'sl_1027_gao__port_506100', 0.0031352206, 332, '355'), ('916235064', 'elise_1027_gao__port_506121', 0.0010256486, 332, '355'), ('916235064', '1007_1027_gao__port_506070', 0.0015354877, 332, '355'), ('916235064', 'i40_1027_gao__port_506218', 0.0005916002, 332, '355'), ('916235064', 'bipper_tepee_1027_gao__port_506227', 0.004028857, 332, '355'), ('916235064', 'focus_1027_gao__port_506272', 0.0011588798, 332, '355'), ('916235064', 'primera_1027_gao__port_506147', 0.0012159661, 332, '355'), ('916235064', 'r4_1027_gao__port_506160', 0.014962202, 332, '355'), ('916235064', 'a8_1027_gao__port_506265', 0.0011323408, 332, '355'), ('916235064', 'boxer_1027_gao__port_506202', 0.010544506, 332, '355'), ('916235064', 's5_1027_gao__port_506222', 0.0011986456, 332, '355'), ('916235064', 'r21_1027_gao__port_506093', 0.0041863243, 332, '355'), ('916235064', 'c3_1027_gao__port_506257', 0.0023639083, 332, '355'), ('916235064', 'santa_fe_1027_gao__port_506208', 0.0016324506, 332, '355'), ('916235064', 'm4_1027_gao__port_506344', 0.0015570896, 332, '355'), ('916235064', 'safrane_1027_gao__port_506077', 0.001396135, 332, '355'), ('916235064', 'classe_gle_1027_gao__port_506395', 0.0021982722, 332, '355'), ('916235064', '0_1027_gao__port_506094', 0.008826612, 332, '355'), ('916235064', 'ix35_1027_gao__port_506219', 0.0014613832, 332, '355'), ('916235064', 'carens_1027_gao__port_506298', 0.0008826003, 332, '355'), ('916235064', 'classe_a_1027_gao__port_506339', 0.0024715352, 332, '355'), ('916235064', 'ix20_1027_gao__port_506343', 0.0010094561, 332, '355'), ('916235064', 'note_1027_gao__port_506365', 0.0015962139, 332, '355'), ('916235064', 'a5_1027_gao__port_506200', 0.001533257, 332, '355'), ('916235064', 'sx4_1027_gao__port_506348', 0.0014919864, 332, '355'), ('916235064', 'sandero_1027_gao__port_506198', 0.0014586772, 332, '355'), ('916235064', '3008_1027_gao__port_506385', 0.0056460253, 332, '355'), ('916235064', 'q50_1027_gao__port_506239', 0.0011167575, 332, '355'), ('916235064', 'latitude_1027_gao__port_506236', 0.0008020807, 332, '355'), ('916235064', 'v40_1027_gao__port_506391', 0.0017147971, 332, '355'), ('916235064', 'xsara_1027_gao__port_506087', 0.0009823972, 332, '355'), ('916235064', 'grand_c_max_1027_gao__port_506342', 0.0017959469, 332, '355'), ('916235064', 'swift_1027_gao__port_506149', 0.0015022326, 332, '355'), ('916235064', 'serie_1_1027_gao__port_506184', 0.0015141575, 332, '355'), ('916235064', 'xc70_1027_gao__port_506393', 0.0036196674, 332, '355'), ('916235064', 'master_1027_gao__port_506203', 0.007956093, 332, '355'), ('916235064', 'clio_1027_gao__port_506280', 0.0029574628, 332, '355'), ('916235064', 'duster_1027_gao__port_506216', 0.0007443862, 332, '355'), ('916235064', 'traveller_1027_gao__port_506403', 0.0042940513, 332, '355'), ('916235064', 'tipo_1027_gao__port_506355', 0.0010929768, 332, '355'), ('916235064', 'rav_4_1027_gao__port_506332', 0.0013606499, 332, '355'), ('916235064', 'coccinelle_1027_gao__port_506259', 0.0034943665, 332, '355'), ('916235064', 'spacetourer_1027_gao__port_506401', 0.003097474, 332, '355'), ('916235064', 'xe_1027_gao__port_506357', 0.0014471704, 332, '355'), ('916235064', 'ds3_1027_gao__port_506324', 0.001309695, 332, '355'), ('916235064', 'mx_5_1027_gao__port_506098', 0.0025883208, 332, '355'), ('916235064', 'land_cruiser_1027_gao__port_506315', 0.009531049, 332, '355'), ('916235064', 'classe_b_1027_gao__port_506335', 0.0017216664, 332, '355'), ('916235064', '806_1027_gao__port_506088', 0.0025619408, 332, '355'), ('916235064', 'rx_8_1027_gao__port_506046', 0.0036215486, 332, '355'), ('916235064', 'spark_1027_gao__port_506185', 0.0010077814, 332, '355'), ('916235064', '6_1027_gao__port_506171', 0.0011184077, 332, '355'), ('916235064', 'bravo_1027_gao__port_506080', 0.0014650801, 332, '355'), ('916235064', 'nx_1027_gao__port_506345', 0.0013685111, 332, '355'), ('916235064', 'sharan_1027_gao__port_506347', 0.005091906, 332, '355'), ('916235064', 'x_type_1027_gao__port_506067', 0.0007802656, 332, '355'), ('916235064', 'jimny_1027_gao__port_506233', 0.004605661, 332, '355'), ('916235064', 'wrangler_1027_gao__port_506225', 0.0017996392, 332, '355'), ('916235064', 'c_crosser_1027_gao__port_506312', 0.0015927051, 332, '355'), ('916235064', 'v70_1027_gao__port_506278', 0.0019678029, 332, '355'), ('916235064', 'classe_e_1027_gao__port_506300', 0.0017370107, 332, '355'), ('916235064', 'classe_v_1027_gao__port_506258', 0.012729445, 332, '355'), ('916235064', 'm3_1027_gao__port_506182', 0.002337339, 332, '355'), ('916235064', 'abarth_500_1027_gao__port_506226', 0.0040435544, 332, '355'), ('916235064', 'serie_6_1027_gao__port_506262', 0.0011315476, 332, '355'), ('916235064', 'modus_1027_gao__port_506146', 0.0018291214, 332, '355'), ('916235064', '3_1027_gao__port_506113', 0.0015083712, 332, '355'), ('916235064', '405_1027_gao__port_506108', 0.008054639, 332, '355'), ('916235064', 'allroad_1027_gao__port_506297', 0.0010599724, 332, '355'), ('916235064', 'auris_1027_gao__port_506322', 0.0011526877, 332, '355'), ('916235064', 'galaxy_1027_gao__port_506143', 0.002514887, 332, '355'), ('916235064', 'giulietta_1027_gao__port_506363', 0.00086521, 332, '355'), ('916235064', '106_1027_gao__port_506073', 0.008271093, 332, '355'), ('916235064', 'classe_m_1027_gao__port_506154', 0.003001904, 332, '355'), ('916235064', 'espace_1027_gao__port_506313', 0.0010646257, 332, '355'), ('916235064', 'panda_1027_gao__port_506189', 0.009030095, 332, '355'), ('916235064', 'rcz_1027_gao__port_506197', 0.0011295427, 332, '355'), ('916235064', '4007_1027_gao__port_506162', 0.0006793045, 332, '355'), ('916235064', 'classe_cl_1027_gao__port_506249', 0.0010862235, 332, '355'), ('916235064', 'leaf_1027_gao__port_506139', 0.0018039574, 332, '355'), ('916235064', 'octavia_1027_gao__port_506237', 0.0018605762, 332, '355'), ('916235064', 'ds4_1027_gao__port_506336', 0.002416371, 332, '355'), ('916235064', 'freelander_1027_gao__port_506084', 0.0023475946, 332, '355'), ('916235064', 'evasion_1027_gao__port_506109', 0.0031140866, 332, '355'), ('916235064', 'punto_1027_gao__port_506106', 0.0019498018, 332, '355'), ('916235064', '2cv_1027_gao__port_506045', 0.007972635, 332, '355'), ('916235064', 'x4_1027_gao__port_506392', 0.0017952245, 332, '355'), ('916235064', 'antara_1027_gao__port_506247', 0.0012467961, 332, '355'), ('916235064', 'murano_1027_gao__port_506316', 0.00060904556, 332, '355'), ('916235064', 'alto_1027_gao__port_506201', 0.009231179, 332, '355'), ('916235064', 'meriva_1027_gao__port_506353', 0.0013765461, 332, '355'), ('916235064', 'orlando_1027_gao__port_506305', 0.0018462765, 332, '355'), ('916235064', 'new_beetle_1027_gao__port_506050', 0.0011638136, 332, '355'), ('916235064', '306_1027_gao__port_506145', 0.0035070505, 332, '355'), ('916235064', 'tiguan_1027_gao__port_506362', 0.0026826167, 332, '355'), ('916235064', 's_type_1027_gao__port_506101', 0.0011381481, 332, '355'), ('916235064', 'c1_1027_gao__port_506128', 0.0027515215, 332, '355'), ('916235064', 'vectra_1027_gao__port_506044', 0.0011991289, 332, '355'), ('916235064', 'outlander_1027_gao__port_506317', 0.0017122234, 332, '355'), ('916235064', '307_1027_gao__port_506074', 0.0020013654, 332, '355'), ('916235064', 'a6_s6_1027_gao__port_506134', 0.0016573346, 332, '355'), ('916235064', 'nemo_combi_1027_gao__port_506196', 0.0022660242, 332, '355'), ('916235064', 'berlingo_1027_gao__port_506194', 0.004661617, 332, '355'), ('916235064', 'partner_1027_gao__port_506285', 0.0039406996, 332, '355'), ('916235064', 'cayenne_1027_gao__port_506177', 0.0037982247, 332, '355'), ('916235064', 'quattroporte_1027_gao__port_506240', 0.0024442656, 332, '355'), ('916235064', 'c_max_1027_gao__port_506282', 0.0013125541, 332, '355'), ('916235064', 'fabia_1027_gao__port_506396', 0.005300178, 332, '355'), ('916235064', 'cx_3_1027_gao__port_506281', 0.0014463922, 332, '355'), ('916235064', 'x_trail_1027_gao__port_506264', 0.0018318305, 332, '355'), ('916235064', 'scirocco_1027_gao__port_506276', 0.004790767, 332, '355'), ('916235064', 'matiz_1027_gao__port_506144', 0.001755978, 332, '355'), ('916235064', 'tigra_1027_gao__port_506069', 0.0008543848, 332, '355'), ('916235064', 'escort_1027_gao__port_506091', 0.0048406105, 332, '355'), ('916235064', 'c2_1027_gao__port_506081', 0.0014905174, 332, '355'), ('916235064', 'mini_1027_gao__port_506168', 0.0011921304, 332, '355'), ('916235064', 'i30_1027_gao__port_506291', 0.0006326676, 332, '355'), ('916235064', 'picanto_1027_gao__port_506238', 0.002975352, 332, '355'), ('916235064', 'mito_1027_gao__port_506072', 0.0015076511, 332, '355'), ('916235064', 'impreza_1027_gao__port_506085', 0.0020191863, 332, '355'), ('916235064', 'kangoo_1027_gao__port_506235', 0.006581657, 332, '355'), ('916235064', 'a4_1027_gao__port_506193', 0.001987892, 332, '355'), ('916235064', 'cayman_1027_gao__port_506268', 0.0018142147, 332, '355'), ('916235064', 'sportage_1027_gao__port_506148', 0.0014277227, 332, '355'), ('916235064', 'up_1027_gao__port_506356', 0.0068632783, 332, '355'), ('916235064', 'optima_1027_gao__port_506386', 0.00089204096, 332, '355'), ('916235064', 'defender_1027_gao__port_506229', 0.006720834, 332, '355'), ('916235064', 'serie_2_1027_gao__port_506256', 0.0022278228, 332, '355'), ('916235064', 'edge_1027_gao__port_506187', 0.0008749559, 332, '355'), ('916235064', 'r19_1027_gao__port_506110', 0.004942464, 332, '355'), ('916235064', 'jetta_1027_gao__port_506304', 0.0036193547, 332, '355'), ('916235064', 'eos_1027_gao__port_506115', 0.0038924068, 332, '355'), ('916235064', 'accord_1027_gao__port_506214', 0.0020127355, 332, '355'), ('916235064', 'yaris_1027_gao__port_506334', 0.003233582, 332, '355'), ('916235064', 'classe_cls_1027_gao__port_506289', 0.0007852971, 332, '355'), ('916235064', 'polo_1027_gao__port_506361', 0.004310878, 332, '355'), ('916235064', 'serie_4_1027_gao__port_506366', 0.0011477313, 332, '355'), ('916235064', 'mini_cabriolet_1027_gao__port_506204', 0.00083779753, 332, '355'), ('916235064', 'prius_1027_gao__port_506190', 0.0011492136, 332, '355'), ('916235064', 'lodgy_1027_gao__port_506188', 0.002017142, 332, '355'), ('916235064', 'serie_7_1027_gao__port_506307', 0.001247741, 332, '355'), ('916235064', 'c15_1027_gao__port_506055', 0.01771089, 332, '355'), ('916235064', 'kadjar_1027_gao__port_506389', 0.0012506138, 332, '355'), ('916235064', 'insignia_1027_gao__port_506364', 0.0016436157, 332, '355'), ('916235064', '308_1027_gao__port_506279', 0.0021239314, 332, '355'), ('916235064', 'roomster_1027_gao__port_506241', 0.0018013328, 332, '355'), ('916235064', '80_1027_gao__port_506057', 0.0046138, 332, '355'), ('916235064', '309_1027_gao__port_506063', 0.013523588, 332, '355'), ('916235064', 'tucson_1027_gao__port_506320', 0.002123825, 332, '355'), ('916235064', 'x3_1027_gao__port_506212', 0.0008976747, 332, '355'), ('916235064', 'xf_1027_gao__port_506263', 0.0011167275, 332, '355'), ('916235064', '2008_1027_gao__port_506394', 0.0026409887, 332, '355'), ('916235064', 'passat_1027_gao__port_506306', 0.0014978591, 332, '355'), ('916235064', 'compass_1027_gao__port_506260', 0.0032562115, 332, '355'), ('916235064', 'twingo_1027_gao__port_506309', 0.0064983936, 332, '355'), ('916235064', 'micra_1027_gao__port_506221', 0.0035853232, 332, '355'), ('916235064', 'golf_1027_gao__port_506155', 0.0031929885, 332, '355'), ('916235064', 'soul_1027_gao__port_506176', 0.0012889912, 332, '355'), ('916235064', 'rapid_1027_gao__port_506398', 0.0025919084, 332, '355'), ('916235064', 'forester_1027_gao__port_506360', 0.002276351, 332, '355'), ('916235064', 'slk_1027_gao__port_506210', 0.0015845272, 332, '355'), ('916235064', 'forfour_1027_gao__port_506341', 0.0021796897, 332, '355'), ('916235064', 'serie_5_1027_gao__port_506209', 0.0013697526, 332, '355'), ('916235064', 'xj_1027_gao__port_506170', 0.0026005297, 332, '355'), ('916235064', 'pajero_1027_gao__port_506097', 0.005210996, 332, '355'), ('916235064', 'agila_1027_gao__port_506119', 0.0048494367, 332, '355'), ('916235064', 'a6_1027_gao__port_506163', 0.0018915117, 332, '355'), ('916235064', 'fox_1027_gao__port_506092', 0.0008466561, 332, '355'), ('916235064', 'boxster_1027_gao__port_506267', 0.0015942677, 332, '355'), ('916235064', 'altea_1027_gao__port_506246', 0.002139255, 332, '355'), ('916235064', 'samurai_1027_gao__port_506047', 0.0062502758, 332, '355'), ('916235064', 'trax_1027_gao__port_506296', 0.0019441537, 332, '355'), ('916235064', 'getz_1027_gao__port_506058', 0.0016388722, 332, '355'), ('916235064', 'cherokee_1027_gao__port_506269', 0.0029812048, 332, '355'), ('916235064', 'koleos_1027_gao__port_506378', 0.0015592619, 332, '355'), ('916235064', 'z_series_1027_gao__port_506123', 0.0016565504, 332, '355'), ('916235064', 'ecosport_1027_gao__port_506271', 0.0013230937, 332, '355'), ('916235064', 'space_star_1027_gao__port_506277', 0.0021144215, 332, '355'), ('916235064', 'rs3_sportback_1027_gao__port_506207', 0.001912017, 332, '355'), ('916235064', 'civic_1027_gao__port_506141', 0.002690429, 332, '355'), ('916235064', 'talisman_1027_gao__port_506390', 0.00076135405, 332, '355'), ('916235064', 'f_pace_1027_gao__port_506314', 0.0016166993, 332, '355'), ('916235064', 'classe_c_1027_gao__port_506299', 0.0017943447, 332, '355'), ('916235064', 'tt_1027_gao__port_506075', 0.001393763, 332, '355'), ('916235064', 'pathfinder_1027_gao__port_506183', 0.0016514915, 332, '355'), ('916235064', '156_1027_gao__port_506157', 0.0015442749, 332, '355'), ('916235064', 'cx_5_1027_gao__port_506228', 0.0014415624, 332, '355'), ('916235064', 'scenic_1027_gao__port_506255', 0.001608709, 332, '355'), ('916235064', 'yeti_1027_gao__port_506358', 0.0020914655, 332, '355'), ('916235064', 'mustang_1027_gao__port_506053', 0.010049834, 332, '355'), ('916235064', 'stilo_1027_gao__port_506060', 0.0010834113, 332, '355'), ('916235064', 'ateca_1027_gao__port_506382', 0.0017015386, 332, '355'), ('916235064', 'fiorino_1027_gao__port_506217', 0.009196521, 332, '355'), ('916235064', 'classe_glk_1027_gao__port_506290', 0.0017017386, 332, '355'), ('916235064', 'fortwo_1027_gao__port_506230', 0.0016011387, 332, '355'), ('916235064', 'cruze_1027_gao__port_506186', 0.0010053845, 332, '355'), ('916235064', '107_1027_gao__port_506213', 0.0016275591, 332, '355'), ('916235064', 'aygo_1027_gao__port_506248', 0.0032434892, 332, '355'), ('916235064', 'rx_1027_gao__port_506354', 0.0010635207, 332, '355'), ('916235064', '500_1027_gao__port_506245', 0.0016353122, 332, '355'), ('916235064', 'bora_1027_gao__port_506104', 0.0038163252, 332, '355'), ('916235064', 'transit_1027_gao__port_506111', 0.0048613315, 332, '355'), ('916235064', 'pt_cruiser_1027_gao__port_506054', 0.0019160203, 332, '355'), ('916235064', 'patrol_1027_gao__port_506068', 0.0042399843, 332, '355'), ('916235064', 'r8_1027_gao__port_506156', 0.0012724173, 332, '355'), ('916235064', 'xm_1027_gao__port_506102', 0.002268145, 332, '355'), ('916235064', 's60_1027_gao__port_506191', 0.0031998279, 332, '355'), ('916235064', 'aveo_1027_gao__port_506158', 0.0038401186, 332, '355'), ('916235064', 'captiva_1027_gao__port_506159', 0.0017191364, 332, '355'), ('916235064', 'ax_1027_gao__port_506153', 0.0068971966, 332, '355'), ('916235064', 'rexton_1027_gao__port_506107', 0.0013020947, 332, '355'), ('916235064', 'camaro_1027_gao__port_506056', 0.0024903184, 332, '355'), ('916235064', 'ypsilon_1027_gao__port_506131', 0.0019541774, 332, '355'), ('916235064', 'delta_1027_gao__port_506165', 0.001400294, 332, '355'), ('916235064', 'c4_1027_gao__port_506370', 0.0013012717, 332, '355'), ('916235064', 'zx_1027_gao__port_506161', 0.0045944564, 332, '355'), ('916235064', 'verso_1027_gao__port_506242', 0.00077214825, 332, '355'), ('916235064', 'superb_1027_gao__port_506327', 0.0019943845, 332, '355'), ('916235064', 'r5_1027_gao__port_506253', 0.009546877, 332, '355'), ('916235064', 'caddy_1027_gao__port_506330', 0.013822283, 332, '355'), ('916235064', 'x5_1027_gao__port_506243', 0.0011205002, 332, '355'), ('916235064', 'f_type_1027_gao__port_506231', 0.0008300645, 332, '355'), ('916235064', 'fusion_1027_gao__port_506096', 0.0012670856, 332, '355'), ('916235064', 'dokker_1027_gao__port_506331', 0.00535567, 332, '355'), ('916235064', '205_1027_gao__port_506062', 0.0066850972, 332, '355'), ('916235064', 'macan_1027_gao__port_506195', 0.0015575542, 332, '355'), ('916235064', 'tourneo_1027_gao__port_506369', 0.0064004734, 332, '355'), ('916235064', '108_1027_gao__port_506384', 0.005263806, 332, '355'), ('916235064', '9_3_1027_gao__port_506071', 0.0008374908, 332, '355'), ('916235064', 'mondeo_1027_gao__port_506116', 0.0014398164, 332, '355'), ('916235064', 'cr_v_1027_gao__port_506164', 0.0016416339, 332, '355'), ('916235064', 'c30_1027_gao__port_506090', 0.0017488975, 332, '355'), ('916235064', 'pulsar_1027_gao__port_506397', 0.0012020799, 332, '355'), ('916235064', 'ibiza_1027_gao__port_506273', 0.003723433, 332, '355'), ('916235064', 'a1_1027_gao__port_506338', 0.0012348009, 332, '355'), ('916235064', 'matrix_1027_gao__port_506140', 0.0007078068, 332, '355'), ('916235064', 'carnival_1027_gao__port_506136', 0.00228112, 332, '355'), ('916235064', 'xantia_1027_gao__port_506086', 0.0021962242, 332, '355'), ('916235064', 'terrano_1027_gao__port_506083', 0.002029231, 332, '355'), ('916235064', 'q3_1027_gao__port_506275', 0.0011266725, 332, '355'), ('916235064', 'hr_v_1027_gao__port_506283', 0.0017807393, 332, '355'), ('916235064', 'expert_1027_gao__port_506142', 0.0073690056, 332, '355'), ('916235064', 'multivan_1027_gao__port_506383', 0.00650313, 332, '355'), ('916235064', 'venga_1027_gao__port_506380', 0.00080062094, 332, '355'), ('916235064', 'scudo_1027_gao__port_506129', 0.0055920184, 332, '355'), ('916235064', 'laguna_1027_gao__port_506368', 0.000713595, 332, '355'), ('916235064', 'vel_satis_1027_gao__port_506130', 0.0027267027, 332, '355'), ('916235064', 'b_max_1027_gao__port_506367', 0.0017247844, 332, '355'), ('916235064', 'ignis_1027_gao__port_506292', 0.00435634, 332, '355'), ('916235064', '159_1027_gao__port_506064', 0.0010783084, 332, '355'), ('916235064', 'grande_punto_1027_gao__port_506138', 0.0023638434, 332, '355'), ('916235064', 'logan_1027_gao__port_506167', 0.004396972, 332, '355'), ('916235064', 's_max_1027_gao__port_506223', 0.0012528334, 332, '355'), ('916235064', 'caravelle_1027_gao__port_506351', 0.0030291986, 332, '355'), ('916235064', 'adam_1027_gao__port_506079', 0.0010538257, 332, '355'), ('916235064', '406_1027_gao__port_506132', 0.0013576014, 332, '355'), ('916235064', 'q30_1027_gao__port_506293', 0.0009717255, 332, '355'), ('916235064', 'almera_1027_gao__port_506089', 0.0010240069, 332, '355'), ('916235064', 'corsa_1027_gao__port_506095', 0.002520736, 332, '355'), ('916235064', 'corolla_1027_gao__port_506120', 0.0026823047, 332, '355'), ('916235064', 'xc60_1027_gao__port_506388', 0.0018985949, 332, '355'), ('916235064', 'viano_1027_gao__port_506211', 0.00269417, 332, '355'), ('916235064', 'pro_cee_d_1027_gao__port_506274', 0.0008320698, 332, '355'), ('916235064', 'a3_1027_gao__port_506321', 0.0037381991, 332, '355'), ('916235064', 'v50_1027_gao__port_506150', 0.00079222204, 332, '355'), ('916235064', 'voyager_1027_gao__port_506169', 0.0030519376, 332, '355'), ('916235064', 'corvette_1027_gao__port_506049', 0.0037224414, 332, '355'), ('916235064', 'rio_1027_gao__port_506379', 0.0017745172, 332, '355'), ('916235064', 'jazz_1027_gao__port_506252', 0.0015306934, 332, '355'), ('916235064', '200_1027_gao__port_506112', 0.0040874607, 332, '355'), ('916235064', 'tts_1027_gao__port_506199', 0.0011864078, 332, '355'), ('916235064', 'zafira_1027_gao__port_506287', 0.0026957379, 332, '355'), ('916235064', 'asx_1027_gao__port_506266', 0.0011408447, 332, '355'), ('916235064', '607_1027_gao__port_506118', 0.0012530944, 332, '355'), ('916235064', '207_1027_gao__port_506103', 0.001515015, 332, '355'), ('916235064', 'classe_s_1027_gao__port_506301', 0.0031657107, 332, '355'), ('916235064', 'c6_1027_gao__port_506105', 0.0017351882, 332, '355'), ('916235064', 'express_1027_gao__port_506137', 0.016725285, 332, '355'), ('916235064', 'classe_gla_1027_gao__port_506352', 0.001825665, 332, '355'), ('916235064', 'v60_1027_gao__port_506333', 0.002145971, 332, '355'), ('916235064', 'ka_1027_gao__port_506180', 0.0014152594, 332, '355'), ('916235064', 'range_rover_1027_gao__port_506254', 0.0020555519, 332, '355'), ('916235064', 'discovery_1027_gao__port_506375', 0.0022962121, 332, '355'), ('916235064', 'classe_r_1027_gao__port_506270', 0.0013944157, 332, '355'), ('916235064', 'transporter_1027_gao__port_506319', 0.011967105, 332, '355'), ('916235064', 'cee_d_1027_gao__port_506288', 0.0010549369, 332, '355'), ('916235064', 'zoe_1027_gao__port_506244', 0.0020717208, 332, '355'), ('916235064', 'i20_1027_gao__port_506284', 0.0017872903, 332, '355'), ('916235064', 'gtv_1027_gao__port_506059', 0.0057219453, 332, '355'), ('916235064', 's4_avant_1027_gao__port_506261', 0.0027673496, 332, '355'), ('916235064', 'x1_1027_gao__port_506372', 0.0017146787, 332, '355'), ('916235064', 'autres_1027_gao__port_506127', 0.004824621, 332, '355'), ('916235064', '208_1027_gao__port_506359', 0.0018688363, 332, '355'), ('916235064', 'c8_1027_gao__port_506135', 0.0012579048, 332, '355'), ('916235064', 'astra_1027_gao__port_506215', 0.0012627443, 332, '355'), ('916235064', '2_1027_gao__port_506151', 0.0009246284, 332, '355'), ('916235064', 'doblo_1027_gao__port_506251', 0.007466123, 332, '355'), ('916235064', '807_1027_gao__port_506152', 0.0007291065, 332, '355'), ('916235064', '206_1027_gao__port_506126', 0.0010386725, 332, '355'), ('916235064', 'a7_1027_gao__port_506373', 0.0006913146, 332, '355'), ('916235064', 'renegade_1027_gao__port_506346', 0.0021415534, 332, '355')]]} begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 0 insert into MTRPhoto.class_photo_score (thcl, photo_id, hashtag_id, score) values (%s,%s,%s,%s) on duplicate key update score = values(score) time used for this insertion : 5.4836273193359375e-06 save missing photos in datou_result : time spend for datou_step_exec : 5.316884517669678 time spend to save output : 2.156909227371216 total time spend for step 1 : 7.4737937450408936 step2:argmax Wed Sep 24 17:48:32 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec After prepare type args : Here we display some param of map_info ! map_filenames : {'temp/1758728904_1864669_916235064_6293d1bb790dc6902450e7c572b7d10b.jpg': 916235064} map_photo_id_path_extension : {916235064: {'path': 'temp/1758728904_1864669_916235064_6293d1bb790dc6902450e7c572b7d10b.jpg', 'extension': 'jpg'}} map_subphoto_mainphoto : {} Beginning of datou_step Argmax ! calculate argmax for thcl : 355 After datou_step_exec type output : map_portfolio_photo : len 0 keys : dict_keys([]) Inside saveOutput : final : True verbose : True photo_id : 916235064 output[photo_id] : [('916235064', 'c15_1027_gao__port_506055', 0.01771089, 332, '355'), 'temp/1758728904_1864669_916235064_6293d1bb790dc6902450e7c572b7d10b.jpg'] begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 1 insert ignore into MTRBack.photo_hashtag_ids (photo_id, hashtag_id, type) values (%s,%s,%s) insert ignore into MTRBack.photo_hashtag_ids (photo_id, hashtag_id, type) values (%s,%s,%s) first line : ('916235064', '2049863950', '332') ... last line : ('916235064', '2049863950', '332') time used for this insertion : 0.022551774978637695 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 1 insert into MTRPhoto.class_photo_score (thcl, photo_id, hashtag_id, score) values (%s,%s,%s,%s) on duplicate key update score = values(score) time used for this insertion : 0.015980243682861328 len list_finale : 1, len picture : 1 begin to insert list_values into mtr_datou_result : length of list_values in save_final : 1 insert ignore into MTRPhoto.mtr_datou_result (mtd_id, mtr_portfolio_id,mtr_photo_id,result,result_long,result_double,hashtag_id,proba, mtr_current_id) values (%s,%s,%s,%s,%s,%s,%s,%s,%s) on duplicate key update mtr_portfolio_id = mtr_portfolio_id list_values : [('2', None, '916235064', 'c15_1027_gao__port_506055', None, None, '2049863950', '0.01771089', None)] time used for this insertion : 0.013082027435302734 saving photo_ids in datou_result photo id not in port begin to insert list_values into mtr_datou_result : length of list_values in save_final : 0 insert ignore into MTRPhoto.mtr_datou_result (mtd_id, mtr_portfolio_id,mtr_photo_id,result,result_long,result_double,hashtag_id,proba, mtr_current_id) values (%s,%s,%s,%s,%s,%s,%s,%s,%s) on duplicate key update mtr_portfolio_id = mtr_portfolio_id list_values : [] time used for this insertion : 3.814697265625e-06 save missing photos in datou_result : time spend for datou_step_exec : 0.00018930435180664062 time spend to save output : 0.05185580253601074 total time spend for step 2 : 0.05204510688781738 caffe_path_current : About to save ! 2 After save, about to update current ! datou_cur_ids : [] len(datou.list_steps) : 2 output : {'916235064': [('916235064', 'c15_1027_gao__port_506055', 0.01771089, 332, '355'), 'temp/1758728904_1864669_916235064_6293d1bb790dc6902450e7c572b7d10b.jpg']} ############################### TEST tfhub2 ################################ TEST TFHUB2 ######################## test with use_multi_inputs=0 ######################## Inside batchDatouExec : verbose : True ##### chargement datou SELECT name, created_at,limit_max FROM MTRDatou.mtr_datou WHERE id=4567 SELECT mtd.id, mtdt.`type`, mtd.`param`, mtd.param_json, mtdt.nb_input, mtdt.nb_output, mtdt.prod, mtdt.is_local, mtdt.is_datou_depend, mtdt.is_photo_id_local FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_step_types mtdt WHERE mtdt.`id`=mtd.`type` AND mtd.mtd_id=4567 SELECT mtd.id, mtd.mtd_id, mdsdt.id, mdsdt.name, mdsdt.description, msid.output_or_input, msid.data_order_id, mdsdt.type FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_steptype_io_datatypes msid, MTRDatou.mtr_datou_step_data_types mdsdt WHERE mtd.`type`=msid.`mtr_datou_step_type` AND mtd.mtd_id= 4567 AND msid.data_type=mdsdt.id SELECT mts_id_output, id_output, mts_id_input, id_input FROM MTRDatou.mtr_datou_step_by_step WHERE mtd_id=4567 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : step 12835 tfhub_classification2 is not linked in the step_by_step architecture ! WARNING : step 12836 argmax is not linked in the step_by_step architecture ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! DataTypes for each output/input checked ! no param json to modify List Step Type Loaded in datou : tfhub_classification2, argmax list_input_json : [] ##### fin chargement datou ##### chargement data ##### Call load_data_input : nb_thread : 5 origin SELECT photo_id, url FROM MTRBack.view_photos_with_urls_250912 ph WHERE photo_id IN (1171252784,1171252764,1171252487) Found this number of photos: 3 ##### Call download_photos : nb_thread : 5 begin to download photo : 1171252487 begin to download photo : 1171252764 begin to download photo : 1171252784 download finish for photo 1171252764 download finish for photo 1171252487 download finish for photo 1171252784 we have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB ##### After download_photos length of list_filenames : 3 ; length of list_pids : 3 ; length of list_args : 3 ##### After load_data_input time to download the photos : 0.19075512886047363 #### fin chargement data Blocking on flush ? No conitnuing About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : True number of steps : 2 step1:tfhub_classification2 Wed Sep 24 17:48:32 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec After prepare type args : Here we display some param of map_info ! map_filenames : {'temp/1758728912_1864669_1171252764_29d5179a892cc50aadc9d67245534b59.jpg': 1171252764, 'temp/1758728912_1864669_1171252487_5ebdd6b0a6bb39942a3808ed114806de.jpg': 1171252487, 'temp/1758728912_1864669_1171252784_5a3c5d3bb155a7a116f67ded51bffb59.jpg': 1171252784} map_photo_id_path_extension : {1171252764: {'path': 'temp/1758728912_1864669_1171252764_29d5179a892cc50aadc9d67245534b59.jpg', 'extension': 'jpg'}, 1171252487: {'path': 'temp/1758728912_1864669_1171252487_5ebdd6b0a6bb39942a3808ed114806de.jpg', 'extension': 'jpg'}, 1171252784: {'path': 'temp/1758728912_1864669_1171252784_5a3c5d3bb155a7a116f67ded51bffb59.jpg', 'extension': 'jpg'}} map_subphoto_mainphoto : {} Beginning of datou_step TFHub with tf2 ! multi_thcl or not :False multi_thcl_cond or not :False dic_thcl : {'3609': 1} we are using the classfication for only one thcl 3609 begin to check gpu status inside check gpu memory 2025-09-24 17:48:36.429626: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-09-24 17:48:36.430715: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-24 17:48:36.430816: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-24 17:48:36.430872: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-24 17:48:36.445729: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-24 17:48:36.445951: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-24 17:48:36.479675: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-24 17:48:36.484977: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-24 17:48:36.545782: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-24 17:48:36.547182: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-24 17:48:36.547600: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-09-24 17:48:36.580571: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3492910000 Hz 2025-09-24 17:48:36.582524: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7fe234000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-09-24 17:48:36.582568: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-09-24 17:48:36.586278: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0xb3370bc0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-09-24 17:48:36.586309: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-09-24 17:48:36.587506: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-24 17:48:36.587636: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-24 17:48:36.587660: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-24 17:48:36.587758: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-24 17:48:36.587791: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-24 17:48:36.587834: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-24 17:48:36.587884: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-24 17:48:36.587947: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-24 17:48:36.589197: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-24 17:48:36.589269: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-24 17:48:36.589322: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-09-24 17:48:36.589335: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-09-24 17:48:36.589344: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-09-24 17:48:36.590622: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 3096 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) l 3637 free memory gpu now : 6482 max_wait_temp : 1 max_wait : 5 1 Physical GPUs, 1 Logical GPUs tagging for thcl : 3609 To do loadFromThcl(), then load ParamDescType : thcl3609 get_desc_type_from_thcl : type of cat SELECT id, mtr_user_id, name, pb_hashtag_id, hashtag_id_list, button_legend_list, portfolio_id_lists, photo_hashtag_type, photo_desc_type, svm_limit, limit_tagging, is_public, live, created_at, updated_at, type_classification FROM MTRDatou.classification_theme WHERE `id` IN (3609) thcls : [{'id': 3609, 'mtr_user_id': 31, 'name': 'tfhub_19_06_2023', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'jrm,pcm,pcnc,pehd,tapis_vide', 'svm_portfolios_learning': '9336903,9336904,9336905,9336906,9336909', 'photo_hashtag_type': 4674, 'photo_desc_type': 5832, 'type_classification': 'tf_classification2', 'hashtag_id_list': '495916461,560181804,1284539308,628944319,2107748999'}] thcl {'id': 3609, 'mtr_user_id': 31, 'name': 'tfhub_19_06_2023', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'jrm,pcm,pcnc,pehd,tapis_vide', 'svm_portfolios_learning': '9336903,9336904,9336905,9336906,9336909', 'photo_hashtag_type': 4674, 'photo_desc_type': 5832, 'type_classification': 'tf_classification2', 'hashtag_id_list': '495916461,560181804,1284539308,628944319,2107748999'} Update svm_hashtag_type_desc : 5832 SELECT * FROM MTRDatou.photo_desc_type_params WHERE id in (5832) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5832, 'tfhub_19_06_2023', 1280, 1280, 'tfhub_19_06_2023', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2023, 6, 19, 12, 55, 22), datetime.datetime(2023, 6, 19, 12, 55, 22)) model_name : tfhub_19_06_2023 model_param file didn't exist model_name : tfhub_19_06_2023 model_type : tf_classification2 list file need : ['Confusion_Matrix.png', 'Precision_Recall_jrm.jpg', 'Precision_Recall_pcm.jpg', 'Precision_Recall_pcnc.jpg', 'Precision_Recall_pehd.jpg', 'Precision_Recall_tapis_vide.jpg', 'Result_Summary.txt', 'checkpoint', 'model_checkpoint.ckpt.data-00000-of-00002', 'model_checkpoint.ckpt.data-00001-of-00002', 'model_checkpoint.ckpt.index', 'model_weights.h5'] file exist in s3 : ['Confusion_Matrix.png', 'Precision_Recall_jrm.jpg', 'Precision_Recall_pcm.jpg', 'Precision_Recall_pcnc.jpg', 'Precision_Recall_pehd.jpg', 'Precision_Recall_tapis_vide.jpg', 'Result_Summary.txt', 'checkpoint', 'model_checkpoint.ckpt.data-00000-of-00002', 'model_checkpoint.ckpt.data-00001-of-00002', 'model_checkpoint.ckpt.index', 'model_weights.h5'] file manque in s3 : [] /home/admin/workarea/git/Velours/python/tests/python_tests.py:11: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses import imp Traceback (most recent call last): File "/home/admin/workarea/git/Velours/python/tests/python_tests.py", line 245, in v, r = module_name.test(verbose=verbose, context=context) File "/home/admin/workarea/git/Velours/python/tests/mask_test.py", line 264, in test title2, result2, res_bool2 = test_detect_objet(context, verbose) TypeError: cannot unpack non-iterable NoneType object File "/home/admin/workarea/git/Velours/python/tests/python_tests.py", line 251, in traceback.print_stack() /home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python/../../tools/../lib/rpn/proposal_layer.py:28: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details. layer_params = yaml.load(self.param_str_) local folder : /data/models_weight/tfhub_19_06_2023 /data/models_weight/tfhub_19_06_2023/Confusion_Matrix.png size_local : 57753 size in s3 : 57753 create time local : 2023-06-22 17:09:38 create time in s3 : 2023-06-19 10:55:15 Confusion_Matrix.png already exist and didn't need to update /data/models_weight/tfhub_19_06_2023/Precision_Recall_jrm.jpg size_local : 79724 size in s3 : 79724 create time local : 2023-06-22 17:09:38 create time in s3 : 2023-06-19 10:55:20 Precision_Recall_jrm.jpg already exist and didn't need to update /data/models_weight/tfhub_19_06_2023/Precision_Recall_pcm.jpg size_local : 83556 size in s3 : 83556 create time local : 2023-06-22 17:09:38 create time in s3 : 2023-06-19 10:55:15 Precision_Recall_pcm.jpg already exist and didn't need to update /data/models_weight/tfhub_19_06_2023/Precision_Recall_pcnc.jpg size_local : 74107 size in s3 : 74107 create time local : 2023-06-22 17:09:38 create time in s3 : 2023-06-19 10:55:20 Precision_Recall_pcnc.jpg already exist and didn't need to update /data/models_weight/tfhub_19_06_2023/Precision_Recall_pehd.jpg size_local : 72705 size in s3 : 72705 create time local : 2023-06-22 17:09:39 create time in s3 : 2023-06-19 10:55:20 Precision_Recall_pehd.jpg already exist and didn't need to update /data/models_weight/tfhub_19_06_2023/Precision_Recall_tapis_vide.jpg size_local : 70874 size in s3 : 70874 create time local : 2023-06-22 17:09:39 create time in s3 : 2023-06-19 10:55:15 Precision_Recall_tapis_vide.jpg already exist and didn't need to update /data/models_weight/tfhub_19_06_2023/Result_Summary.txt size_local : 642 size in s3 : 642 create time local : 2023-06-22 17:09:39 create time in s3 : 2023-06-19 10:55:22 Result_Summary.txt already exist and didn't need to update /data/models_weight/tfhub_19_06_2023/checkpoint size_local : 99 size in s3 : 99 create time local : 2023-06-22 17:09:39 create time in s3 : 2023-06-19 10:55:22 checkpoint already exist and didn't need to update /data/models_weight/tfhub_19_06_2023/model_checkpoint.ckpt.data-00000-of-00002 size_local : 216488 size in s3 : 216488 create time local : 2023-06-22 17:09:39 create time in s3 : 2023-06-19 10:55:22 model_checkpoint.ckpt.data-00000-of-00002 already exist and didn't need to update /data/models_weight/tfhub_19_06_2023/model_checkpoint.ckpt.data-00001-of-00002 size_local : 32279708 size in s3 : 32279708 create time local : 2023-06-22 17:09:40 create time in s3 : 2023-06-19 10:55:21 model_checkpoint.ckpt.data-00001-of-00002 already exist and didn't need to update /data/models_weight/tfhub_19_06_2023/model_checkpoint.ckpt.index size_local : 43546 size in s3 : 43546 create time local : 2023-06-22 17:09:40 create time in s3 : 2023-06-19 10:55:22 model_checkpoint.ckpt.index already exist and didn't need to update /data/models_weight/tfhub_19_06_2023/model_weights.h5 size_local : 16499144 size in s3 : 16499144 create time local : 2023-06-22 17:09:40 create time in s3 : 2023-06-19 10:55:15 model_weights.h5 already exist and didn't need to update desc size : 1280 Model: "sequential" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= module (KerasLayer) (None, 1280) 4049564 _________________________________________________________________ tfhub_19_06_2023dense (Dense (None, 5) 6405 ================================================================= Total params: 4,055,969 Trainable params: 6,405 Non-trainable params: 4,049,564 _________________________________________________________________ Loading Weights... time used to create the model : 8.298471450805664 time used to load_weights : 0.12676000595092773 0it [00:00, ?it/s] 3it [00:00, 1068.52it/s]2025-09-24 17:48:47.464893: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 temp/1758728912_1864669_1171252764_29d5179a892cc50aadc9d67245534b59.jpg temp/1758728912_1864669_1171252487_5ebdd6b0a6bb39942a3808ed114806de.jpg temp/1758728912_1864669_1171252784_5a3c5d3bb155a7a116f67ded51bffb59.jpg Found 3 images belonging to 1 classes. begin to do the prediction : time used to do the prediction : 3.284085512161255 ['temp/image000000000_1758728912_1864669_1171252764_29d5179a892cc50aadc9d67245534b59.jpg', 'temp/image000000001_1758728912_1864669_1171252487_5ebdd6b0a6bb39942a3808ed114806de.jpg', 'temp/image000000002_1758728912_1864669_1171252784_5a3c5d3bb155a7a116f67ded51bffb59.jpg'] (3,) (3, 5) (3, 1280) shape of features : (3, 1280) shape of new features : (1, 3, 1280) save descriptor for thcl : 3609 (3, 1280) Got the blobs of the net to insert : [0, 6, 0, 1, 0, 0, 0, 1, 0, 0] code_as_byte_string:b'0006000100'| Got the blobs of the net to insert : [0, 9, 0, 0, 0, 0, 1, 0, 0, 0] code_as_byte_string:b'0009000000'| Got the blobs of the net to insert : [0, 6, 0, 0, 1, 0, 0, 1, 0, 0] code_as_byte_string:b'0006000001'| time to traite the descriptors : 0.023665428161621094 Testing : ['1171252764', '1171252487', '1171252784'] In select_photos_meta_from_ids: SELECT photo_id, url, FROM_UNIXTIME(uploaded_at), latitude, longitude, text FROM MTRBack.view_photos_with_urls_250912 WHERE photo_id IN (1171252764,1171252487,1171252784) result : {1171252487: {'photo_id': 1171252487, 'url': 'https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/2023/2/22/5ebdd6b0a6bb39942a3808ed114806de.jpg', 'latitude': None, 'longitude': None, 'text': 'image_22022023_21_55_35_005998m0.jpg 0.4259977941513062 for time 6.000020980834961, id_amount 3 this amount prod time diff : 0.006000020980834961'}, 1171252764: {'photo_id': 1171252764, 'url': 'https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/2023/2/22/29d5179a892cc50aadc9d67245534b59.jpg', 'latitude': None, 'longitude': None, 'text': 'image_22022023_21_55_41_005998m0.jpg 0.4319977941513062 for time 6.0, id_amount 3 this amount prod time diff : 0.006'}, 1171252784: {'photo_id': 1171252784, 'url': 'https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/2023/2/22/5a3c5d3bb155a7a116f67ded51bffb59.jpg', 'latitude': None, 'longitude': None, 'text': 'image_22022023_21_55_47_006033m0.jpg 0.4379978291988373 for time 6.000035047531128, id_amount 4 this amount prod time diff : 0.006000035047531128'}} list_photo_exists : [1171252487, 1171252764, 1171252784] storage_type for insertDescriptorsMulti : 3 To insert : 1171252764 To insert : 1171252487 To insert : 1171252784 time to insert the descriptors : 1.0634679794311523 After datou_step_exec type output : map_portfolio_photo : len 0 keys : dict_keys([]) Inside saveOutput : final : False verbose : True saveOutput not yet implemented for datou_step.type : tfhub_classification2 we use saveGeneral [1171252764, 1171252487, 1171252784] map_info['map_portfolio_photo'] : {} final : False mtd_id 4567 list_pids : [1171252764, 1171252487, 1171252784] Looping around the photos to save general results len do output : 3 /1171252764Didn't retrieve data . /1171252487Didn't retrieve data . /1171252784Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4567', None, None, None, None, None, None, None, None) ('4567', None, '1171252764', None, None, None, None, None, None) ('4567', None, None, None, None, None, None, None, None) ('4567', None, '1171252487', None, None, None, None, None, None) ('4567', None, None, None, None, None, None, None, None) ('4567', None, '1171252784', None, None, None, None, None, None) begin to insert list_values into mtr_datou_result : length of list_values in save_final : 6 insert ignore into MTRPhoto.mtr_datou_result (mtd_id, mtr_portfolio_id,mtr_photo_id,result,result_long,result_double,hashtag_id,proba, mtr_current_id) values (%s,%s,%s,%s,%s,%s,%s,%s,%s) on duplicate key update mtr_portfolio_id = mtr_portfolio_id list_values : [('4567', None, '1171252764', 'None', None, None, None, None, None), ('4567', None, '1171252487', 'None', None, None, None, None, None), ('4567', None, '1171252784', 'None', None, None, None, None, None)] time used for this insertion : 0.015656232833862305 save_final save missing photos in datou_result : time spend for datou_step_exec : 18.895084381103516 time spend to save output : 0.015932083129882812 total time spend for step 1 : 18.9110164642334 step2:argmax Wed Sep 24 17:48:51 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec After prepare type args : Here we display some param of map_info ! map_filenames : {'temp/1758728912_1864669_1171252764_29d5179a892cc50aadc9d67245534b59.jpg': 1171252764, 'temp/1758728912_1864669_1171252487_5ebdd6b0a6bb39942a3808ed114806de.jpg': 1171252487, 'temp/1758728912_1864669_1171252784_5a3c5d3bb155a7a116f67ded51bffb59.jpg': 1171252784} map_photo_id_path_extension : {1171252764: {'path': 'temp/1758728912_1864669_1171252764_29d5179a892cc50aadc9d67245534b59.jpg', 'extension': 'jpg'}, 1171252487: {'path': 'temp/1758728912_1864669_1171252487_5ebdd6b0a6bb39942a3808ed114806de.jpg', 'extension': 'jpg'}, 1171252784: {'path': 'temp/1758728912_1864669_1171252784_5a3c5d3bb155a7a116f67ded51bffb59.jpg', 'extension': 'jpg'}} map_subphoto_mainphoto : {} Beginning of datou_step Argmax ! calculate argmax for thcl : 3609 After datou_step_exec type output : map_portfolio_photo : len 0 keys : dict_keys([]) Inside saveOutput : final : True verbose : True photo_id : 1171252764 output[photo_id] : [(1171252764, 'jrm', 0.9853954, 4674, '3609'), 'temp/1758728912_1864669_1171252764_29d5179a892cc50aadc9d67245534b59.jpg'] photo_id : 1171252487 output[photo_id] : [(1171252487, 'jrm', 0.9262383, 4674, '3609'), 'temp/1758728912_1864669_1171252487_5ebdd6b0a6bb39942a3808ed114806de.jpg'] photo_id : 1171252784 output[photo_id] : [(1171252784, 'jrm', 0.9677775, 4674, '3609'), 'temp/1758728912_1864669_1171252784_5a3c5d3bb155a7a116f67ded51bffb59.jpg'] begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 3 insert ignore into MTRBack.photo_hashtag_ids (photo_id, hashtag_id, type) values (%s,%s,%s) insert ignore into MTRBack.photo_hashtag_ids (photo_id, hashtag_id, type) values (%s,%s,%s) first line : ('1171252764', '495916461', '4674') ... last line : ('1171252784', '495916461', '4674') time used for this insertion : 0.049849510192871094 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 3 insert into MTRPhoto.class_photo_score (thcl, photo_id, hashtag_id, score) values (%s,%s,%s,%s) on duplicate key update score = values(score) time used for this insertion : 0.05595064163208008 len list_finale : 3, len picture : 3 begin to insert list_values into mtr_datou_result : length of list_values in save_final : 3 insert ignore into MTRPhoto.mtr_datou_result (mtd_id, mtr_portfolio_id,mtr_photo_id,result,result_long,result_double,hashtag_id,proba, mtr_current_id) values (%s,%s,%s,%s,%s,%s,%s,%s,%s) on duplicate key update mtr_portfolio_id = mtr_portfolio_id list_values : [('4567', None, '1171252764', 'jrm', None, None, '495916461', '0.9853954', None), ('4567', None, '1171252487', 'jrm', None, None, '495916461', '0.9262383', None), ('4567', None, '1171252784', 'jrm', None, None, '495916461', '0.9677775', None)] time used for this insertion : 0.012916326522827148 saving photo_ids in datou_result photo id not in port photo id not in port photo id not in port begin to insert list_values into mtr_datou_result : length of list_values in save_final : 0 insert ignore into MTRPhoto.mtr_datou_result (mtd_id, mtr_portfolio_id,mtr_photo_id,result,result_long,result_double,hashtag_id,proba, mtr_current_id) values (%s,%s,%s,%s,%s,%s,%s,%s,%s) on duplicate key update mtr_portfolio_id = mtr_portfolio_id list_values : [] time used for this insertion : 3.337860107421875e-06 save missing photos in datou_result : time spend for datou_step_exec : 0.00013828277587890625 time spend to save output : 0.12288928031921387 total time spend for step 2 : 0.12302756309509277 caffe_path_current : About to save ! 2 After save, about to update current ! datou_cur_ids : [] len(datou.list_steps) : 2 output : {'1171252764': [(1171252764, 'jrm', 0.9853954, 4674, '3609'), 'temp/1758728912_1864669_1171252764_29d5179a892cc50aadc9d67245534b59.jpg'], '1171252487': [(1171252487, 'jrm', 0.9262383, 4674, '3609'), 'temp/1758728912_1864669_1171252487_5ebdd6b0a6bb39942a3808ed114806de.jpg'], '1171252784': [(1171252784, 'jrm', 0.9677775, 4674, '3609'), 'temp/1758728912_1864669_1171252784_5a3c5d3bb155a7a116f67ded51bffb59.jpg']} --------------------- test with use_multi_inputs=0 is succeded ------------------- ######################## test with use_multi_inputs=1 ######################## Inside batchDatouExec : verbose : True ##### chargement datou SELECT name, created_at,limit_max FROM MTRDatou.mtr_datou WHERE id=4621 SELECT mtd.id, mtdt.`type`, mtd.`param`, mtd.param_json, mtdt.nb_input, mtdt.nb_output, mtdt.prod, mtdt.is_local, mtdt.is_datou_depend, mtdt.is_photo_id_local FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_step_types mtdt WHERE mtdt.`id`=mtd.`type` AND mtd.mtd_id=4621 SELECT mtd.id, mtd.mtd_id, mdsdt.id, mdsdt.name, mdsdt.description, msid.output_or_input, msid.data_order_id, mdsdt.type FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_steptype_io_datatypes msid, MTRDatou.mtr_datou_step_data_types mdsdt WHERE mtd.`type`=msid.`mtr_datou_step_type` AND mtd.mtd_id= 4621 AND msid.data_type=mdsdt.id SELECT mts_id_output, id_output, mts_id_input, id_input FROM MTRDatou.mtr_datou_step_by_step WHERE mtd_id=4621 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : step 12927 tfhub_classification2 is not linked in the step_by_step architecture ! WARNING : step 12928 argmax is not linked in the step_by_step architecture ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! DataTypes for each output/input checked ! no param json to modify List Step Type Loaded in datou : tfhub_classification2, argmax list_input_json : [] ##### fin chargement datou ##### chargement data ##### Call load_data_input : nb_thread : 5 origin SELECT photo_id, url FROM MTRBack.view_photos_with_urls_250912 ph WHERE photo_id IN (1171291875,1171275372,1171275314) Found this number of photos: 3 ##### Call download_photos : nb_thread : 5 begin to download photo : 1171275314 begin to download photo : 1171275372 begin to download photo : 1171291875 download finish for photo 1171291875 download finish for photo 1171275372 download finish for photo 1171275314 we have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB ##### After download_photos length of list_filenames : 3 ; length of list_pids : 3 ; length of list_args : 3 ##### After load_data_input time to download the photos : 0.2006852626800537 #### fin chargement data Blocking on flush ? No conitnuing About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : True number of steps : 2 step1:tfhub_classification2 Wed Sep 24 17:48:52 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec After prepare type args : Here we display some param of map_info ! map_filenames : {'temp/1758728931_1864669_1171291875_b62cd9e0d976b143f86fe82d072798c0.jpg': 1171291875, 'temp/1758728931_1864669_1171275372_76d81364ff7df843bff095f45c07ba35.jpg': 1171275372, 'temp/1758728931_1864669_1171275314_6e0a72c8fa00d5e4b018bd689b547133.jpg': 1171275314} map_photo_id_path_extension : {1171291875: {'path': 'temp/1758728931_1864669_1171291875_b62cd9e0d976b143f86fe82d072798c0.jpg', 'extension': 'jpg'}, 1171275372: {'path': 'temp/1758728931_1864669_1171275372_76d81364ff7df843bff095f45c07ba35.jpg', 'extension': 'jpg'}, 1171275314: {'path': 'temp/1758728931_1864669_1171275314_6e0a72c8fa00d5e4b018bd689b547133.jpg', 'extension': 'jpg'}} map_subphoto_mainphoto : {} Beginning of datou_step TFHub with tf2 ! multi_thcl or not :False multi_thcl_cond or not :False dic_thcl : {'3655': 1} we are using the classfication for only one thcl 3655 begin to check gpu status inside check gpu memory inside check gpu memory inside check gpu memory inside check gpu memory inside check gpu memory inside check gpu memory l 3637 free memory gpu now : 2930 max_wait_temp : 6 max_wait : 5 1 Physical GPUs, 1 Logical GPUs tagging for thcl : 3655 To do loadFromThcl(), then load ParamDescType : thcl3655 get_desc_type_from_thcl : type of cat SELECT id, mtr_user_id, name, pb_hashtag_id, hashtag_id_list, button_legend_list, portfolio_id_lists, photo_hashtag_type, photo_desc_type, svm_limit, limit_tagging, is_public, live, created_at, updated_at, type_classification FROM MTRDatou.classification_theme WHERE `id` IN (3655) thcls : [{'id': 3655, 'mtr_user_id': 31, 'name': 'tfhub_18_7_2023', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'pcm,pcnc,jrm,pehd,tapis_vide', 'svm_portfolios_learning': '9336904,9336905,9336903,9336906,9336909', 'photo_hashtag_type': 4723, 'photo_desc_type': 5862, 'type_classification': 'tf_classification2', 'hashtag_id_list': '560181804,1284539308,495916461,628944319,2107748999'}] thcl {'id': 3655, 'mtr_user_id': 31, 'name': 'tfhub_18_7_2023', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'pcm,pcnc,jrm,pehd,tapis_vide', 'svm_portfolios_learning': '9336904,9336905,9336903,9336906,9336909', 'photo_hashtag_type': 4723, 'photo_desc_type': 5862, 'type_classification': 'tf_classification2', 'hashtag_id_list': '560181804,1284539308,495916461,628944319,2107748999'} Update svm_hashtag_type_desc : 5862 SELECT * FROM MTRDatou.photo_desc_type_params WHERE id in (5862) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5862, 'tfhub_18_7_2023', 1280, 1280, 'tfhub_18_7_2023', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2023, 7, 18, 22, 46, 29), datetime.datetime(2023, 7, 18, 22, 46, 29)) model_name : tfhub_18_7_2023 model_param file didn't exist model_name : tfhub_18_7_2023 model_type : tf_classification2 list file need : ['Confusion_Matrix.png', 'Precision_Recall_jrm.jpg', 'Precision_Recall_pcm.jpg', 'Precision_Recall_pcnc.jpg', 'Precision_Recall_pehd.jpg', 'Precision_Recall_tapis_vide.jpg', 'Result_Summary.txt', 'checkpoint', 'model_checkpoint.ckpt.data-00000-of-00002', 'model_checkpoint.ckpt.data-00001-of-00002', 'model_checkpoint.ckpt.index', 'model_weights.h5'] file exist in s3 : ['Confusion_Matrix.png', 'Precision_Recall_jrm.jpg', 'Precision_Recall_pcm.jpg', 'Precision_Recall_pcnc.jpg', 'Precision_Recall_pehd.jpg', 'Precision_Recall_tapis_vide.jpg', 'Result_Summary.txt', 'checkpoint', 'model_checkpoint.ckpt.data-00000-of-00002', 'model_checkpoint.ckpt.data-00001-of-00002', 'model_checkpoint.ckpt.index', 'model_weights.h5'] file manque in s3 : [] local folder : /data/models_weight/tfhub_18_7_2023 /data/models_weight/tfhub_18_7_2023/Confusion_Matrix.png size_local : 54360 size in s3 : 54360 create time local : 2023-08-11 11:22:56 create time in s3 : 2023-07-18 20:46:28 Confusion_Matrix.png already exist and didn't need to update /data/models_weight/tfhub_18_7_2023/Precision_Recall_jrm.jpg size_local : 72583 size in s3 : 72583 create time local : 2023-08-11 11:22:56 create time in s3 : 2023-07-18 20:46:23 Precision_Recall_jrm.jpg already exist and didn't need to update /data/models_weight/tfhub_18_7_2023/Precision_Recall_pcm.jpg size_local : 81681 size in s3 : 81681 create time local : 2023-08-11 11:22:56 create time in s3 : 2023-07-18 20:46:17 Precision_Recall_pcm.jpg already exist and didn't need to update /data/models_weight/tfhub_18_7_2023/Precision_Recall_pcnc.jpg size_local : 79510 size in s3 : 79510 create time local : 2023-08-11 11:22:56 create time in s3 : 2023-07-18 20:46:23 Precision_Recall_pcnc.jpg already exist and didn't need to update /data/models_weight/tfhub_18_7_2023/Precision_Recall_pehd.jpg size_local : 59936 size in s3 : 59936 create time local : 2023-08-11 11:22:57 create time in s3 : 2023-07-18 20:46:23 Precision_Recall_pehd.jpg already exist and didn't need to update /data/models_weight/tfhub_18_7_2023/Precision_Recall_tapis_vide.jpg size_local : 78974 size in s3 : 78974 create time local : 2023-08-11 11:22:57 create time in s3 : 2023-07-18 20:46:17 Precision_Recall_tapis_vide.jpg already exist and didn't need to update /data/models_weight/tfhub_18_7_2023/Result_Summary.txt size_local : 642 size in s3 : 642 create time local : 2023-08-11 11:22:57 create time in s3 : 2023-07-18 20:46:23 Result_Summary.txt already exist and didn't need to update /data/models_weight/tfhub_18_7_2023/checkpoint size_local : 99 size in s3 : 99 create time local : 2023-08-11 11:22:57 create time in s3 : 2023-07-18 20:46:23 checkpoint already exist and didn't need to update /data/models_weight/tfhub_18_7_2023/model_checkpoint.ckpt.data-00000-of-00002 size_local : 216529 size in s3 : 216529 create time local : 2023-08-11 11:22:57 create time in s3 : 2023-07-18 20:46:17 model_checkpoint.ckpt.data-00000-of-00002 already exist and didn't need to update /data/models_weight/tfhub_18_7_2023/model_checkpoint.ckpt.data-00001-of-00002 size_local : 32279748 size in s3 : 32279748 create time local : 2023-08-11 11:22:58 create time in s3 : 2023-07-18 20:46:19 model_checkpoint.ckpt.data-00001-of-00002 already exist and didn't need to update /data/models_weight/tfhub_18_7_2023/model_checkpoint.ckpt.index size_local : 43546 size in s3 : 43546 create time local : 2023-08-11 11:22:58 create time in s3 : 2023-07-18 20:46:19 model_checkpoint.ckpt.index already exist and didn't need to update /data/models_weight/tfhub_18_7_2023/model_weights.h5 size_local : 16500868 size in s3 : 16500868 create time local : 2023-08-11 11:22:58 create time in s3 : 2023-07-18 20:46:18 model_weights.h5 already exist and didn't need to update desc size : 1280 Model: "model" __________________________________________________________________________________________________ Layer (type) Output Shape Param # Connected to ================================================================================================== input_1 (InputLayer) [(None, 224, 224, 3) 0 __________________________________________________________________________________________________ input_2 (InputLayer) [(None, 1)] 0 __________________________________________________________________________________________________ module (KerasLayer) (None, 1280) 4049564 input_1[0][0] __________________________________________________________________________________________________ concatenate (Concatenate) (None, 1281) 0 input_2[0][0] module[0][0] __________________________________________________________________________________________________ tfhub_18_7_2023dense (Dense) (None, 5) 6410 concatenate[0][0] ================================================================================================== Total params: 4,055,974 Trainable params: 0 Non-trainable params: 4,055,974 __________________________________________________________________________________________________ Loading Weights... time used to create the model : 7.191299676895142 time used to load_weights : 0.1258227825164795 found 3 data found 0 labels begin to do the prediction : time used to do the prediction : 0.9764394760131836 ['temp/1758728931_1864669_1171291875_b62cd9e0d976b143f86fe82d072798c0.jpg', 'temp/1758728931_1864669_1171275372_76d81364ff7df843bff095f45c07ba35.jpg', 'temp/1758728931_1864669_1171275314_6e0a72c8fa00d5e4b018bd689b547133.jpg'] (3,) (3, 5) (3, 1280) shape of features : (3, 1280) shape of new features : (1, 3, 1280) save descriptor for thcl : 3655 (3, 1280) Got the blobs of the net to insert : [0, 1, 0, 0, 11, 0, 2, 2, 0, 0] code_as_byte_string:b'000100000b'| Got the blobs of the net to insert : [0, 0, 0, 0, 14, 0, 1, 4, 0, 0] code_as_byte_string:b'000000000e'| Got the blobs of the net to insert : [0, 0, 0, 0, 8, 0, 0, 0, 3, 0] code_as_byte_string:b'0000000008'| time to traite the descriptors : 0.035140275955200195 Testing : ['1171291875', '1171275372', '1171275314'] In select_photos_meta_from_ids: SELECT photo_id, url, FROM_UNIXTIME(uploaded_at), latitude, longitude, text FROM MTRBack.view_photos_with_urls_250912 WHERE photo_id IN (1171291875,1171275372,1171275314) result : {1171275314: {'photo_id': 1171275314, 'url': 'https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/2023/2/23/6e0a72c8fa00d5e4b018bd689b547133.jpg', 'latitude': None, 'longitude': None, 'text': 'image_22022023_23_54_22_6187.jpg'}, 1171275372: {'photo_id': 1171275372, 'url': 'https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/2023/2/23/76d81364ff7df843bff095f45c07ba35.jpg', 'latitude': None, 'longitude': None, 'text': 'image_22022023_23_56_46_6098.jpg'}, 1171291875: {'photo_id': 1171291875, 'url': 'https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/2023/2/23/b62cd9e0d976b143f86fe82d072798c0.jpg', 'latitude': None, 'longitude': None, 'text': 'image_22022023_23_59_04_5803.jpg'}} list_photo_exists : [1171275314, 1171275372, 1171291875] storage_type for insertDescriptorsMulti : 3 To insert : 1171291875 To insert : 1171275372 To insert : 1171275314 time to insert the descriptors : 0.9170694351196289 After datou_step_exec type output : map_portfolio_photo : len 0 keys : dict_keys([]) Inside saveOutput : final : False verbose : True saveOutput not yet implemented for datou_step.type : tfhub_classification2 we use saveGeneral [1171291875, 1171275372, 1171275314] map_info['map_portfolio_photo'] : {} final : False mtd_id 4621 list_pids : [1171291875, 1171275372, 1171275314] Looping around the photos to save general results len do output : 3 /1171291875Didn't retrieve data . /1171275372Didn't retrieve data . /1171275314Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4621', None, None, None, None, None, None, None, None) ('4621', None, '1171291875', None, None, None, None, None, None) ('4621', None, None, None, None, None, None, None, None) ('4621', None, '1171275372', None, None, None, None, None, None) ('4621', None, None, None, None, None, None, None, None) ('4621', None, '1171275314', None, None, None, None, None, None) begin to insert list_values into mtr_datou_result : length of list_values in save_final : 6 insert ignore into MTRPhoto.mtr_datou_result (mtd_id, mtr_portfolio_id,mtr_photo_id,result,result_long,result_double,hashtag_id,proba, mtr_current_id) values (%s,%s,%s,%s,%s,%s,%s,%s,%s) on duplicate key update mtr_portfolio_id = mtr_portfolio_id list_values : [('4621', None, '1171291875', 'None', None, None, None, None, None), ('4621', None, '1171275372', 'None', None, None, None, None, None), ('4621', None, '1171275314', 'None', None, None, None, None, None)] time used for this insertion : 0.015532732009887695 save_final save missing photos in datou_result : time spend for datou_step_exec : 18.316528797149658 time spend to save output : 0.01589059829711914 total time spend for step 1 : 18.332419395446777 step2:argmax Wed Sep 24 17:49:10 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec After prepare type args : Here we display some param of map_info ! map_filenames : {'temp/1758728931_1864669_1171291875_b62cd9e0d976b143f86fe82d072798c0.jpg': 1171291875, 'temp/1758728931_1864669_1171275372_76d81364ff7df843bff095f45c07ba35.jpg': 1171275372, 'temp/1758728931_1864669_1171275314_6e0a72c8fa00d5e4b018bd689b547133.jpg': 1171275314} map_photo_id_path_extension : {1171291875: {'path': 'temp/1758728931_1864669_1171291875_b62cd9e0d976b143f86fe82d072798c0.jpg', 'extension': 'jpg'}, 1171275372: {'path': 'temp/1758728931_1864669_1171275372_76d81364ff7df843bff095f45c07ba35.jpg', 'extension': 'jpg'}, 1171275314: {'path': 'temp/1758728931_1864669_1171275314_6e0a72c8fa00d5e4b018bd689b547133.jpg', 'extension': 'jpg'}} map_subphoto_mainphoto : {} Beginning of datou_step Argmax ! calculate argmax for thcl : 3655 After datou_step_exec type output : map_portfolio_photo : len 0 keys : dict_keys([]) Inside saveOutput : final : True verbose : True photo_id : 1171291875 output[photo_id] : [(1171291875, 'tapis_vide', 0.9707762, 4723, '3655'), 'temp/1758728931_1864669_1171291875_b62cd9e0d976b143f86fe82d072798c0.jpg'] photo_id : 1171275372 output[photo_id] : [(1171275372, 'tapis_vide', 0.967402, 4723, '3655'), 'temp/1758728931_1864669_1171275372_76d81364ff7df843bff095f45c07ba35.jpg'] photo_id : 1171275314 output[photo_id] : [(1171275314, 'tapis_vide', 0.9651185, 4723, '3655'), 'temp/1758728931_1864669_1171275314_6e0a72c8fa00d5e4b018bd689b547133.jpg'] begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 3 insert ignore into MTRBack.photo_hashtag_ids (photo_id, hashtag_id, type) values (%s,%s,%s) insert ignore into MTRBack.photo_hashtag_ids (photo_id, hashtag_id, type) values (%s,%s,%s) first line : ('1171291875', '2107748999', '4723') ... last line : ('1171275314', '2107748999', '4723') time used for this insertion : 0.024682283401489258 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 3 insert into MTRPhoto.class_photo_score (thcl, photo_id, hashtag_id, score) values (%s,%s,%s,%s) on duplicate key update score = values(score) time used for this insertion : 0.020175933837890625 len list_finale : 3, len picture : 3 begin to insert list_values into mtr_datou_result : length of list_values in save_final : 3 insert ignore into MTRPhoto.mtr_datou_result (mtd_id, mtr_portfolio_id,mtr_photo_id,result,result_long,result_double,hashtag_id,proba, mtr_current_id) values (%s,%s,%s,%s,%s,%s,%s,%s,%s) on duplicate key update mtr_portfolio_id = mtr_portfolio_id list_values : [('4621', None, '1171291875', 'tapis_vide', None, None, '2107748999', '0.9707762', None), ('4621', None, '1171275372', 'tapis_vide', None, None, '2107748999', '0.967402', None), ('4621', None, '1171275314', 'tapis_vide', None, None, '2107748999', '0.9651185', None)] time used for this insertion : 0.01565694808959961 saving photo_ids in datou_result photo id not in port photo id not in port photo id not in port begin to insert list_values into mtr_datou_result : length of list_values in save_final : 0 insert ignore into MTRPhoto.mtr_datou_result (mtd_id, mtr_portfolio_id,mtr_photo_id,result,result_long,result_double,hashtag_id,proba, mtr_current_id) values (%s,%s,%s,%s,%s,%s,%s,%s,%s) on duplicate key update mtr_portfolio_id = mtr_portfolio_id list_values : [] time used for this insertion : 3.814697265625e-06 save missing photos in datou_result : time spend for datou_step_exec : 0.0001418590545654297 time spend to save output : 0.0648345947265625 total time spend for step 2 : 0.06497645378112793 caffe_path_current : About to save ! 2 After save, about to update current ! datou_cur_ids : [] len(datou.list_steps) : 2 output : {'1171291875': [(1171291875, 'tapis_vide', 0.9707762, 4723, '3655'), 'temp/1758728931_1864669_1171291875_b62cd9e0d976b143f86fe82d072798c0.jpg'], '1171275372': [(1171275372, 'tapis_vide', 0.967402, 4723, '3655'), 'temp/1758728931_1864669_1171275372_76d81364ff7df843bff095f45c07ba35.jpg'], '1171275314': [(1171275314, 'tapis_vide', 0.9651185, 4723, '3655'), 'temp/1758728931_1864669_1171275314_6e0a72c8fa00d5e4b018bd689b547133.jpg']} --------------------- test with use_multi_inputs=1 is succeded ------------------- ############################### TEST ordonner ################################ To do loadFromThcl(), then load ParamDescType : thcl358 get_desc_type_from_thcl : type of cat SELECT id, mtr_user_id, name, pb_hashtag_id, hashtag_id_list, button_legend_list, portfolio_id_lists, photo_hashtag_type, photo_desc_type, svm_limit, limit_tagging, is_public, live, created_at, updated_at, type_classification FROM MTRDatou.classification_theme WHERE `id` IN (358) thcls : [{'id': 358, 'mtr_user_id': 31, 'name': 'car_orientation_0111', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'FirstUploadExperveo_vignette__port_505674,CAR_EXTERIEUR_Roue__port_503398,FirstUploadExperveo_carrosseriegrosplan_VIndanslamoquette__port_506486,FirstUploadExperveo_carrosseriegrosplan_siegegrosplan__port_506485,CAR_EXTERIEUR_Cote_droit_axe_avant__port_504465,CAR_EXTERIEUR_Cote_gauche_axe_arriere__port_504198,CAR_EXTERIEUR_Face_avant_axe_droit__port_504451,CAR_EXTERIEUR_angle_avant_gauche_axe_avant__port_504235,FirstUploadExperveo_vin__port_505675,CAR_EXTERIEUR_cote_droite__port_504108,CAR_INTERIEUR_avant_volant_class_6_levierdevitesse__port_506565,FirstUploadExperveo_carrosseriegrosplan_carrosserie__port_506483,CAR_EXTERIEUR_Angle_arriere_gauche_axe_arriere__port_504201,cartegrise_orientation__port_505064,CAR_EXTERIEUR_Angle_arriere_droit_axe_arriere__port_504217,CAR_INTERIEUR_avant_vue-arriere_class_1__port_506531,CAR_EXTERIEUR_Face_arriere_axe_droit__port_504218,CAR_EXTERIEUR_Cote_droit_axe_arriere__port_504214,CAR_EXTERIEUR_Angle_avant_droit__port_504087,FirstUploadExperveo_carrosseriegrosplan_morceauderoue__port_506484,CAR_INTERIEUR_avant_volant_class_6_class_2__port_506563,CAR_EXTERIEUR_Angle_arriere_droit__port_504160,CAR_EXTERIEUR_arriere__port_504184,CAR_INTERIEUR_avant_volant_class_6_boutonrond__port_506562,INTERIEUR_Compteur_kilometrique__port_503644,CAR_INTERIEUR_avant_vue_gauche_habitacle_class_1__port_506494,CAR_EXTERIEUR_Angle_arriere_gauche__port_504170,CAR_EXTERIEUR_Angle_avant_droit_axe_arriere__port_504226,CAR_EXTERIEUR_Face_arriere_axe_gauche__port_504202,CAR_EXTERIEUR_moteur__port_503704,FirstUploadExperveo_carrosseriegrosplan_class_6__port_506487,CAR_INTERIEUR_siege_arriere_class_1__port_506551,CAR_EXTERIEUR_avant__port_504146,CAR_EXTERIEUR_Angle_arriere_droit_axe_droit__port_504215,CAR_EXTERIEUR_Angle_avant_droit_axe_droit__port_504225,CAR_INTERIEUR_avant_volant_class_6_ecrangrosplan__port_506564,FirstUploadExperveo_carrosseriegrosplan_moteurgrosplanetdegat__port_506482,CAR_INTERIEUR_coffre__port_503412,FirstUploadExperveo_rouetranche__port_505677,UploadPhotoImmatBest_class_1__port_505051,CAR_INTERIEUR_avant_vue-arriere_class_2__port_506532,CAR_EXTERIEUR_angle_avant_gauche__port_504098,CAR_EXTERIEUR_face_avant_axe_gauche__port_504236,CAR_INTERIEUR_avant_vue_droite_habitacle_class_1__port_506540,CAR_EXTERIEUR_cote_gauche_axe_avant__port_504233,CAR_EXTERIEUR_roue_de_secour__port_503763,CAR_EXTERIEUR_Angle_arriere_gauche_axe_gauche__port_504199,CAR_EXTERIEUR_cote_gauche__port_504017,CAR_INTERIEUR_avant_volant_class_1__port_506503,CAR_INTERIEUR_avant_volant_class_2__port_506504,CAR_EXTERIEUR_angle_avant_gauche_axe_gauche__port_504234', 'svm_portfolios_learning': '505674,503398,506486,506485,504465,504198,504451,504235,505675,504108,506565,506483,504201,505064,504217,506531,504218,504214,504087,506484,506563,504160,504184,506562,503644,506494,504170,504226,504202,503704,506487,506551,504146,504215,504225,506564,506482,503412,505677,505051,506532,504098,504236,506540,504233,503763,504199,504017,506503,506504,504234', 'photo_hashtag_type': 337, 'photo_desc_type': 3392, 'type_classification': 'caffe', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 358, 'mtr_user_id': 31, 'name': 'car_orientation_0111', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'FirstUploadExperveo_vignette__port_505674,CAR_EXTERIEUR_Roue__port_503398,FirstUploadExperveo_carrosseriegrosplan_VIndanslamoquette__port_506486,FirstUploadExperveo_carrosseriegrosplan_siegegrosplan__port_506485,CAR_EXTERIEUR_Cote_droit_axe_avant__port_504465,CAR_EXTERIEUR_Cote_gauche_axe_arriere__port_504198,CAR_EXTERIEUR_Face_avant_axe_droit__port_504451,CAR_EXTERIEUR_angle_avant_gauche_axe_avant__port_504235,FirstUploadExperveo_vin__port_505675,CAR_EXTERIEUR_cote_droite__port_504108,CAR_INTERIEUR_avant_volant_class_6_levierdevitesse__port_506565,FirstUploadExperveo_carrosseriegrosplan_carrosserie__port_506483,CAR_EXTERIEUR_Angle_arriere_gauche_axe_arriere__port_504201,cartegrise_orientation__port_505064,CAR_EXTERIEUR_Angle_arriere_droit_axe_arriere__port_504217,CAR_INTERIEUR_avant_vue-arriere_class_1__port_506531,CAR_EXTERIEUR_Face_arriere_axe_droit__port_504218,CAR_EXTERIEUR_Cote_droit_axe_arriere__port_504214,CAR_EXTERIEUR_Angle_avant_droit__port_504087,FirstUploadExperveo_carrosseriegrosplan_morceauderoue__port_506484,CAR_INTERIEUR_avant_volant_class_6_class_2__port_506563,CAR_EXTERIEUR_Angle_arriere_droit__port_504160,CAR_EXTERIEUR_arriere__port_504184,CAR_INTERIEUR_avant_volant_class_6_boutonrond__port_506562,INTERIEUR_Compteur_kilometrique__port_503644,CAR_INTERIEUR_avant_vue_gauche_habitacle_class_1__port_506494,CAR_EXTERIEUR_Angle_arriere_gauche__port_504170,CAR_EXTERIEUR_Angle_avant_droit_axe_arriere__port_504226,CAR_EXTERIEUR_Face_arriere_axe_gauche__port_504202,CAR_EXTERIEUR_moteur__port_503704,FirstUploadExperveo_carrosseriegrosplan_class_6__port_506487,CAR_INTERIEUR_siege_arriere_class_1__port_506551,CAR_EXTERIEUR_avant__port_504146,CAR_EXTERIEUR_Angle_arriere_droit_axe_droit__port_504215,CAR_EXTERIEUR_Angle_avant_droit_axe_droit__port_504225,CAR_INTERIEUR_avant_volant_class_6_ecrangrosplan__port_506564,FirstUploadExperveo_carrosseriegrosplan_moteurgrosplanetdegat__port_506482,CAR_INTERIEUR_coffre__port_503412,FirstUploadExperveo_rouetranche__port_505677,UploadPhotoImmatBest_class_1__port_505051,CAR_INTERIEUR_avant_vue-arriere_class_2__port_506532,CAR_EXTERIEUR_angle_avant_gauche__port_504098,CAR_EXTERIEUR_face_avant_axe_gauche__port_504236,CAR_INTERIEUR_avant_vue_droite_habitacle_class_1__port_506540,CAR_EXTERIEUR_cote_gauche_axe_avant__port_504233,CAR_EXTERIEUR_roue_de_secour__port_503763,CAR_EXTERIEUR_Angle_arriere_gauche_axe_gauche__port_504199,CAR_EXTERIEUR_cote_gauche__port_504017,CAR_INTERIEUR_avant_volant_class_1__port_506503,CAR_INTERIEUR_avant_volant_class_2__port_506504,CAR_EXTERIEUR_angle_avant_gauche_axe_gauche__port_504234', 'svm_portfolios_learning': '505674,503398,506486,506485,504465,504198,504451,504235,505675,504108,506565,506483,504201,505064,504217,506531,504218,504214,504087,506484,506563,504160,504184,506562,503644,506494,504170,504226,504202,503704,506487,506551,504146,504215,504225,506564,506482,503412,505677,505051,506532,504098,504236,506540,504233,503763,504199,504017,506503,506504,504234', 'photo_hashtag_type': 337, 'photo_desc_type': 3392, 'type_classification': 'caffe', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 3392 ['FirstUploadExperveo_vignette__port_505674', 'CAR_EXTERIEUR_Roue__port_503398', 'FirstUploadExperveo_carrosseriegrosplan_VIndanslamoquette__port_506486', 'FirstUploadExperveo_carrosseriegrosplan_siegegrosplan__port_506485', 'CAR_EXTERIEUR_Cote_droit_axe_avant__port_504465', 'CAR_EXTERIEUR_Cote_gauche_axe_arriere__port_504198', 'CAR_EXTERIEUR_Face_avant_axe_droit__port_504451', 'CAR_EXTERIEUR_angle_avant_gauche_axe_avant__port_504235', 'FirstUploadExperveo_vin__port_505675', 'CAR_EXTERIEUR_cote_droite__port_504108', 'CAR_INTERIEUR_avant_volant_class_6_levierdevitesse__port_506565', 'FirstUploadExperveo_carrosseriegrosplan_carrosserie__port_506483', 'CAR_EXTERIEUR_Angle_arriere_gauche_axe_arriere__port_504201', 'cartegrise_orientation__port_505064', 'CAR_EXTERIEUR_Angle_arriere_droit_axe_arriere__port_504217', 'CAR_INTERIEUR_avant_vue-arriere_class_1__port_506531', 'CAR_EXTERIEUR_Face_arriere_axe_droit__port_504218', 'CAR_EXTERIEUR_Cote_droit_axe_arriere__port_504214', 'CAR_EXTERIEUR_Angle_avant_droit__port_504087', 'FirstUploadExperveo_carrosseriegrosplan_morceauderoue__port_506484', 'CAR_INTERIEUR_avant_volant_class_6_class_2__port_506563', 'CAR_EXTERIEUR_Angle_arriere_droit__port_504160', 'CAR_EXTERIEUR_arriere__port_504184', 'CAR_INTERIEUR_avant_volant_class_6_boutonrond__port_506562', 'INTERIEUR_Compteur_kilometrique__port_503644', 'CAR_INTERIEUR_avant_vue_gauche_habitacle_class_1__port_506494', 'CAR_EXTERIEUR_Angle_arriere_gauche__port_504170', 'CAR_EXTERIEUR_Angle_avant_droit_axe_arriere__port_504226', 'CAR_EXTERIEUR_Face_arriere_axe_gauche__port_504202', 'CAR_EXTERIEUR_moteur__port_503704', 'FirstUploadExperveo_carrosseriegrosplan_class_6__port_506487', 'CAR_INTERIEUR_siege_arriere_class_1__port_506551', 'CAR_EXTERIEUR_avant__port_504146', 'CAR_EXTERIEUR_Angle_arriere_droit_axe_droit__port_504215', 'CAR_EXTERIEUR_Angle_avant_droit_axe_droit__port_504225', 'CAR_INTERIEUR_avant_volant_class_6_ecrangrosplan__port_506564', 'FirstUploadExperveo_carrosseriegrosplan_moteurgrosplanetdegat__port_506482', 'CAR_INTERIEUR_coffre__port_503412', 'FirstUploadExperveo_rouetranche__port_505677', 'UploadPhotoImmatBest_class_1__port_505051', 'CAR_INTERIEUR_avant_vue-arriere_class_2__port_506532', 'CAR_EXTERIEUR_angle_avant_gauche__port_504098', 'CAR_EXTERIEUR_face_avant_axe_gauche__port_504236', 'CAR_INTERIEUR_avant_vue_droite_habitacle_class_1__port_506540', 'CAR_EXTERIEUR_cote_gauche_axe_avant__port_504233', 'CAR_EXTERIEUR_roue_de_secour__port_503763', 'CAR_EXTERIEUR_Angle_arriere_gauche_axe_gauche__port_504199', 'CAR_EXTERIEUR_cote_gauche__port_504017', 'CAR_INTERIEUR_avant_volant_class_1__port_506503', 'CAR_INTERIEUR_avant_volant_class_2__port_506504', 'CAR_EXTERIEUR_angle_avant_gauche_axe_gauche__port_504234'] 51 SELECT hashtag_id,hashtag FROM MTRBack.hashtags where hashtag in ('FirstUploadExperveo_vignette__port_505674','CAR_EXTERIEUR_Roue__port_503398','FirstUploadExperveo_carrosseriegrosplan_VIndanslamoquette__port_506486','FirstUploadExperveo_carrosseriegrosplan_siegegrosplan__port_506485','CAR_EXTERIEUR_Cote_droit_axe_avant__port_504465','CAR_EXTERIEUR_Cote_gauche_axe_arriere__port_504198','CAR_EXTERIEUR_Face_avant_axe_droit__port_504451','CAR_EXTERIEUR_angle_avant_gauche_axe_avant__port_504235','FirstUploadExperveo_vin__port_505675','CAR_EXTERIEUR_cote_droite__port_504108','CAR_INTERIEUR_avant_volant_class_6_levierdevitesse__port_506565','FirstUploadExperveo_carrosseriegrosplan_carrosserie__port_506483','CAR_EXTERIEUR_Angle_arriere_gauche_axe_arriere__port_504201','cartegrise_orientation__port_505064','CAR_EXTERIEUR_Angle_arriere_droit_axe_arriere__port_504217','CAR_INTERIEUR_avant_vue-arriere_class_1__port_506531','CAR_EXTERIEUR_Face_arriere_axe_droit__port_504218','CAR_EXTERIEUR_Cote_droit_axe_arriere__port_504214','CAR_EXTERIEUR_Angle_avant_droit__port_504087','FirstUploadExperveo_carrosseriegrosplan_morceauderoue__port_506484','CAR_INTERIEUR_avant_volant_class_6_class_2__port_506563','CAR_EXTERIEUR_Angle_arriere_droit__port_504160','CAR_EXTERIEUR_arriere__port_504184','CAR_INTERIEUR_avant_volant_class_6_boutonrond__port_506562','INTERIEUR_Compteur_kilometrique__port_503644','CAR_INTERIEUR_avant_vue_gauche_habitacle_class_1__port_506494','CAR_EXTERIEUR_Angle_arriere_gauche__port_504170','CAR_EXTERIEUR_Angle_avant_droit_axe_arriere__port_504226','CAR_EXTERIEUR_Face_arriere_axe_gauche__port_504202','CAR_EXTERIEUR_moteur__port_503704','FirstUploadExperveo_carrosseriegrosplan_class_6__port_506487','CAR_INTERIEUR_siege_arriere_class_1__port_506551','CAR_EXTERIEUR_avant__port_504146','CAR_EXTERIEUR_Angle_arriere_droit_axe_droit__port_504215','CAR_EXTERIEUR_Angle_avant_droit_axe_droit__port_504225','CAR_INTERIEUR_avant_volant_class_6_ecrangrosplan__port_506564','FirstUploadExperveo_carrosseriegrosplan_moteurgrosplanetdegat__port_506482','CAR_INTERIEUR_coffre__port_503412','FirstUploadExperveo_rouetranche__port_505677','UploadPhotoImmatBest_class_1__port_505051','CAR_INTERIEUR_avant_vue-arriere_class_2__port_506532','CAR_EXTERIEUR_angle_avant_gauche__port_504098','CAR_EXTERIEUR_face_avant_axe_gauche__port_504236','CAR_INTERIEUR_avant_vue_droite_habitacle_class_1__port_506540','CAR_EXTERIEUR_cote_gauche_axe_avant__port_504233','CAR_EXTERIEUR_roue_de_secour__port_503763','CAR_EXTERIEUR_Angle_arriere_gauche_axe_gauche__port_504199','CAR_EXTERIEUR_cote_gauche__port_504017','CAR_INTERIEUR_avant_volant_class_1__port_506503','CAR_INTERIEUR_avant_volant_class_2__port_506504','CAR_EXTERIEUR_angle_avant_gauche_axe_gauche__port_504234'); 51 dict_keys(['cartegrise_orientation__port_505064', 'car_exterieur_angle_arriere_droit_axe_arriere__port_504217', 'car_exterieur_angle_arriere_droit_axe_droit__port_504215', 'car_exterieur_angle_arriere_droit__port_504160', 'car_exterieur_angle_arriere_gauche_axe_arriere__port_504201', 'car_exterieur_angle_arriere_gauche_axe_gauche__port_504199', 'car_exterieur_angle_arriere_gauche__port_504170', 'car_exterieur_angle_avant_droit_axe_arriere__port_504226', 'car_exterieur_angle_avant_droit_axe_droit__port_504225', 'car_exterieur_angle_avant_droit__port_504087', 'car_exterieur_angle_avant_gauche_axe_avant__port_504235', 'car_exterieur_angle_avant_gauche_axe_gauche__port_504234', 'car_exterieur_angle_avant_gauche__port_504098', 'car_exterieur_arriere__port_504184', 'car_exterieur_avant__port_504146', 'car_exterieur_cote_droite__port_504108', 'car_exterieur_cote_droit_axe_arriere__port_504214', 'car_exterieur_cote_droit_axe_avant__port_504465', 'car_exterieur_cote_gauche_axe_arriere__port_504198', 'car_exterieur_cote_gauche_axe_avant__port_504233', 'car_exterieur_cote_gauche__port_504017', 'car_exterieur_face_arriere_axe_droit__port_504218', 'car_exterieur_face_arriere_axe_gauche__port_504202', 'car_exterieur_face_avant_axe_droit__port_504451', 'car_exterieur_face_avant_axe_gauche__port_504236', 'car_exterieur_moteur__port_503704', 'car_exterieur_roue_de_secour__port_503763', 'car_exterieur_roue__port_503398', 'car_interieur_avant_volant_class_1__port_506503', 'car_interieur_avant_volant_class_2__port_506504', 'car_interieur_avant_volant_class_6_boutonrond__port_506562', 'car_interieur_avant_volant_class_6_class_2__port_506563', 'car_interieur_avant_volant_class_6_ecrangrosplan__port_506564', 'car_interieur_avant_volant_class_6_levierdevitesse__port_506565', 'car_interieur_avant_vue-arriere_class_1__port_506531', 'car_interieur_avant_vue-arriere_class_2__port_506532', 'car_interieur_avant_vue_droite_habitacle_class_1__port_506540', 'car_interieur_avant_vue_gauche_habitacle_class_1__port_506494', 'car_interieur_coffre__port_503412', 'car_interieur_siege_arriere_class_1__port_506551', 'firstuploadexperveo_carrosseriegrosplan_carrosserie__port_506483', 'firstuploadexperveo_carrosseriegrosplan_class_6__port_506487', 'firstuploadexperveo_carrosseriegrosplan_morceauderoue__port_506484', 'firstuploadexperveo_carrosseriegrosplan_moteurgrosplanetdegat__port_506482', 'firstuploadexperveo_carrosseriegrosplan_siegegrosplan__port_506485', 'firstuploadexperveo_carrosseriegrosplan_vindanslamoquette__port_506486', 'firstuploadexperveo_rouetranche__port_505677', 'firstuploadexperveo_vignette__port_505674', 'firstuploadexperveo_vin__port_505675', 'interieur_compteur_kilometrique__port_503644', 'uploadphotoimmatbest_class_1__port_505051']) select photo_hashtag_type from MTRDatou.classification_theme where id = 358 thcl : 358 photo_hashtag_type : 337 SELECT phi.hashtag_id , phi.photo_id FROM MTRBack.photo_hashtag_ids phi, MTRUser.mtr_portfolio_photos mp where phi.type = 337 and phi.photo_id = mp.mtr_photo_id and mp.mtr_portfolio_id =510365; {510365: [(917973295, 1), (917973297, 1), (917973302, 1), (917973293, 7), (917973296, 11), (917973300, 11), (917973286, 13), (917973289, 13), (917973301, 24), (917973285, 29), (917973290, 29), (917973299, 29), (917973304, 35), (917973287, 36), (917973298, 36), (917973305, 36), (917973292, 37), (917973291, 41), (917973303, 41), (917973294, 42), (917973288, 46)]} ############################### TEST rotate ################################ test rotate only Inside batchDatouExec : verbose : True ##### chargement datou SELECT name, created_at,limit_max FROM MTRDatou.mtr_datou WHERE id=230 SELECT mtd.id, mtdt.`type`, mtd.`param`, mtd.param_json, mtdt.nb_input, mtdt.nb_output, mtdt.prod, mtdt.is_local, mtdt.is_datou_depend, mtdt.is_photo_id_local FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_step_types mtdt WHERE mtdt.`id`=mtd.`type` AND mtd.mtd_id=230 SELECT mtd.id, mtd.mtd_id, mdsdt.id, mdsdt.name, mdsdt.description, msid.output_or_input, msid.data_order_id, mdsdt.type FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_steptype_io_datatypes msid, MTRDatou.mtr_datou_step_data_types mdsdt WHERE mtd.`type`=msid.`mtr_datou_step_type` AND mtd.mtd_id= 230 AND msid.data_type=mdsdt.id SELECT mts_id_output, id_output, mts_id_input, id_input FROM MTRDatou.mtr_datou_step_by_step WHERE mtd_id=230 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! no param json to modify List Step Type Loaded in datou : rotate list_input_json : [] ##### fin chargement datou ##### chargement data ##### Call load_data_input : nb_thread : 5 origin SELECT photo_id, url FROM MTRBack.view_photos_with_urls_250912 ph WHERE photo_id IN (917849322) Found this number of photos: 1 ##### Call download_photos : nb_thread : 5 begin to download photo : 917849322 download finish for photo 917849322 we have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB ##### After download_photos length of list_filenames : 1 ; length of list_pids : 1 ; length of list_args : 1 ##### After load_data_input time to download the photos : 0.17842507362365723 #### fin chargement data Blocking on flush ? No conitnuing About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : True number of steps : 1 step1:rotate Wed Sep 24 17:49:10 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec After prepare type args : Here we display some param of map_info ! map_filenames : {'temp/1758728950_1864669_917849322_2bd260e91e91df8378dde8bb8b8c4548.jpg': 917849322} map_photo_id_path_extension : {917849322: {'path': 'temp/1758728950_1864669_917849322_2bd260e91e91df8378dde8bb8b8c4548.jpg', 'extension': 'jpg'}} map_subphoto_mainphoto : {} Beginning of datou_step_rotate ! We are in a linear step without datou_depend ! rotate photos of 90,180,270 degres batch 1 select photo_id, hashtag_id, `type`, x0, x1, y0, y1, score, id from MTRPhoto.crop_hashtag_ids where photo_id in ( 917849322) and `type` in (0) Loaded 0 chid ids of type : 0 SELECT crop_hashtag_id, points FROM MTRPhoto.crop_polygon_points WHERE crop_hashtag_id in () map_chi : {} photo_id in download_rotate_and_save : 917849322 list_chi_loc : 0 Use all angle ! Rotation of photo 917849322 of 90 degree temp/1758728950_1864669_917849322_2bd260e91e91df8378dde8bb8b8c4548.jpg [] 90 remove_crop_border : False version de PIL : 9.5.0 Needs to change image size ! [[ 6.123234e-17 1.000000e+00] [-1.000000e+00 6.123234e-17]] 90 [[ 6.123234e-17 1.000000e+00] [-1.000000e+00 6.123234e-17]] shrink_image : False image_rotate : image_path : temp/1758728950_1864669_917849322_2bd260e91e91df8378dde8bb8b8c4548.jpg path_name_rotate : temp/1758728950_1864669_917849322_2bd260e91e91df8378dde8bb8b8c454890.jpg image_rotate.mode : RGB Rotation of photo 917849322 of 180 degree temp/1758728950_1864669_917849322_2bd260e91e91df8378dde8bb8b8c4548.jpg [] 180 remove_crop_border : False version de PIL : 9.5.0 Needs to change image size ! [[-1.0000000e+00 1.2246468e-16] [-1.2246468e-16 -1.0000000e+00]] 180 [[-1.0000000e+00 1.2246468e-16] [-1.2246468e-16 -1.0000000e+00]] shrink_image : False image_rotate : image_path : temp/1758728950_1864669_917849322_2bd260e91e91df8378dde8bb8b8c4548.jpg path_name_rotate : temp/1758728950_1864669_917849322_2bd260e91e91df8378dde8bb8b8c4548180.jpg image_rotate.mode : RGB Rotation of photo 917849322 of 270 degree temp/1758728950_1864669_917849322_2bd260e91e91df8378dde8bb8b8c4548.jpg [] 270 remove_crop_border : False version de PIL : 9.5.0 Needs to change image size ! [[-1.8369702e-16 -1.0000000e+00] [ 1.0000000e+00 -1.8369702e-16]] 270 [[-1.8369702e-16 -1.0000000e+00] [ 1.0000000e+00 -1.8369702e-16]] shrink_image : False image_rotate : image_path : temp/1758728950_1864669_917849322_2bd260e91e91df8378dde8bb8b8c4548.jpg path_name_rotate : temp/1758728950_1864669_917849322_2bd260e91e91df8378dde8bb8b8c4548270.jpg image_rotate.mode : RGB About to upload 3 photos upload in portfolio : 551782 init cache_photo without model_param we have 3 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1758728951_1864669 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first Unexecpected behavior in 07/2025 that can be generalized l287 : type_extension .jpg This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first Unexecpected behavior in 07/2025 that can be generalized l287 : type_extension .jpg This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first Unexecpected behavior in 07/2025 that can be generalized l287 : type_extension .jpg This is a hack ! we have uploaded 3 photos in the portfolio 551782 time of upload the photos Elapsed time : 1.1079144477844238 map_filename_photo_id : 3 map_filename_photo_id : {'temp/1758728950_1864669_917849322_2bd260e91e91df8378dde8bb8b8c454890.jpg': 1385928371, 'temp/1758728950_1864669_917849322_2bd260e91e91df8378dde8bb8b8c4548180.jpg': 1385928372, 'temp/1758728950_1864669_917849322_2bd260e91e91df8378dde8bb8b8c4548270.jpg': 1385928373} Len new_chis : 3 Len list_new_chi_with_photo_id : 0 of type : 0 list_new_chi_with_photo_id : [] After datou_step_exec type output : time spend for datou_step_exec : 1.3258357048034668 time spend to save output : 2.1696090698242188e-05 total time spend for step 1 : 1.325857400894165 caffe_path_current : About to save ! 1 Inside saveOutput : final : True verbose : True saveOutput not yet implemented for datou_step.type : rotate we use saveGeneral [917849322] map_info['map_portfolio_photo'] : {} final : True mtd_id 230 list_pids : [917849322] Looping around the photos to save general results len do output : 3 /1385928371Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385928372Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385928373Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('230', None, None, None, None, None, None, None, None) ('230', None, '917849322', None, None, None, None, None, None) begin to insert list_values into mtr_datou_result : length of list_values in save_final : 10 insert ignore into MTRPhoto.mtr_datou_result (mtd_id, mtr_portfolio_id,mtr_photo_id,result,result_long,result_double,hashtag_id,proba, mtr_current_id) values (%s,%s,%s,%s,%s,%s,%s,%s,%s) on duplicate key update mtr_portfolio_id = mtr_portfolio_id list_values : [('230', None, '1385928371', 'None', None, None, None, None, None), ('230', None, '1385928372', 'None', None, None, None, None, None), ('230', None, '1385928373', 'None', None, None, None, None, None), ('230', None, '917849322', None, None, None, None, None, None)] time used for this insertion : 0.015902280807495117 save_final save missing photos in datou_result : After save, about to update current ! datou_cur_ids : [] len(datou.list_steps) : 1 output : {1385928371: ['917849322', 'temp/1758728950_1864669_917849322_2bd260e91e91df8378dde8bb8b8c454890.jpg', []], 1385928372: ['917849322', 'temp/1758728950_1864669_917849322_2bd260e91e91df8378dde8bb8b8c4548180.jpg', []], 1385928373: ['917849322', 'temp/1758728950_1864669_917849322_2bd260e91e91df8378dde8bb8b8c4548270.jpg', []]} test rotate only is a success ! test rotate conditionnel Inside batchDatouExec : verbose : True ##### chargement datou SELECT name, created_at,limit_max FROM MTRDatou.mtr_datou WHERE id=233 SELECT mtd.id, mtdt.`type`, mtd.`param`, mtd.param_json, mtdt.nb_input, mtdt.nb_output, mtdt.prod, mtdt.is_local, mtdt.is_datou_depend, mtdt.is_photo_id_local FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_step_types mtdt WHERE mtdt.`id`=mtd.`type` AND mtd.mtd_id=233 SELECT mtd.id, mtd.mtd_id, mdsdt.id, mdsdt.name, mdsdt.description, msid.output_or_input, msid.data_order_id, mdsdt.type FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_steptype_io_datatypes msid, MTRDatou.mtr_datou_step_data_types mdsdt WHERE mtd.`type`=msid.`mtr_datou_step_type` AND mtd.mtd_id= 233 AND msid.data_type=mdsdt.id SELECT mts_id_output, id_output, mts_id_input, id_input FROM MTRDatou.mtr_datou_step_by_step WHERE mtd_id=233 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! DataTypes for each output/input checked ! no param json to modify List Step Type Loaded in datou : thcl, argmax, rotate list_input_json : [] ##### fin chargement datou ##### chargement data ##### Call load_data_input : nb_thread : 5 origin SELECT photo_id, url FROM MTRBack.view_photos_with_urls_250912 ph WHERE photo_id IN (917849322) Found this number of photos: 1 ##### Call download_photos : nb_thread : 5 begin to download photo : 917849322 download finish for photo 917849322 we have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB ##### After download_photos length of list_filenames : 1 ; length of list_pids : 1 ; length of list_args : 1 ##### After load_data_input time to download the photos : 0.1412975788116455 #### fin chargement data Blocking on flush ? No conitnuing About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : True number of steps : 3 step1:thcl Wed Sep 24 17:49:12 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec After prepare type args : Here we display some param of map_info ! map_filenames : {'temp/1758728952_1864669_917849322_2bd260e91e91df8378dde8bb8b8c4548.jpg': 917849322} map_photo_id_path_extension : {917849322: {'path': 'temp/1758728952_1864669_917849322_2bd260e91e91df8378dde8bb8b8c4548.jpg', 'extension': 'jpg'}} map_subphoto_mainphoto : {} Beginning of datou step Thcl ! multi_thcl or not :False multi_thcl_cond or not :False dic_thcl : {'500': 1} we are using the classfication for only one thcl 500 In convert_file_to_np l 337 : 1 l343 1 l357 after caffe.io.load_image dimension du image : (3, (2448, 3264, 3)) dimension displayed ! time to import caffe and check if the image exist : 0.00022602081298828125 time to convert the images to numpy array : 1.1347525119781494 total time to convert the images to numpy array : 1.1353216171264648 list photo_ids error: [] list photo_ids correct : [917849322] number of photos to traite : 1 try to delete the photos incorrect in DB tagging for thcl : 500 To do loadFromThcl(), then load ParamDescType : thcl500 get_desc_type_from_thcl : type of cat SELECT id, mtr_user_id, name, pb_hashtag_id, hashtag_id_list, button_legend_list, portfolio_id_lists, photo_hashtag_type, photo_desc_type, svm_limit, limit_tagging, is_public, live, created_at, updated_at, type_classification FROM MTRDatou.classification_theme WHERE `id` IN (500) thcls : [{'id': 500, 'mtr_user_id': 31, 'name': 'orientation_carte_grise_all_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'carteGrisesVerticales__port_549774,cartegrise_90deg__port_550987,cartesGrisesEnvers__port_549765,portfolio_270deg__port_550988', 'svm_portfolios_learning': '549774,550987,549765,550988', 'photo_hashtag_type': 507, 'photo_desc_type': 3517, 'type_classification': 'caffe', 'hashtag_id_list': '0,0,0,0'}] thcl {'id': 500, 'mtr_user_id': 31, 'name': 'orientation_carte_grise_all_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'carteGrisesVerticales__port_549774,cartegrise_90deg__port_550987,cartesGrisesEnvers__port_549765,portfolio_270deg__port_550988', 'svm_portfolios_learning': '549774,550987,549765,550988', 'photo_hashtag_type': 507, 'photo_desc_type': 3517, 'type_classification': 'caffe', 'hashtag_id_list': '0,0,0,0'} Update svm_hashtag_type_desc : 3517 SELECT * FROM MTRDatou.photo_desc_type_params WHERE id in (3517) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (3517, 'orientation_carte_grise_all_2', 16384, 25088, 'orientation_carte_grise_all_2', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2018, 4, 18, 20, 4, 34), datetime.datetime(2018, 4, 18, 20, 4, 34)) To loadFromThcl() : net_3517 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 2930 max_wait_temp : 1 max_wait : 0 SELECT * FROM MTRDatou.photo_desc_type_params WHERE id in (3517) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (3517, 'orientation_carte_grise_all_2', 16384, 25088, 'orientation_carte_grise_all_2', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2018, 4, 18, 20, 4, 34), datetime.datetime(2018, 4, 18, 20, 4, 34)) param : , param.caffemodel : orientation_carte_grise_all_2 None mean_file_type : mean_file_path : prototxt_file_path : model : orientation_carte_grise_all_2 Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : orientation_carte_grise_all_2 model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : [] local folder : /data/models_weight/orientation_carte_grise_all_2 /data/models_weight/orientation_carte_grise_all_2/caffemodel size_local : 537110520 size in s3 : 537110520 create time local : 2021-08-09 05:29:00 create time in s3 : 2021-08-06 20:07:17 caffemodel already exist and didn't need to update /data/models_weight/orientation_carte_grise_all_2/deploy_conv_normal.prototxt size_local : 4626 size in s3 : 4626 create time local : 2021-08-09 05:29:00 create time in s3 : 2021-08-06 20:07:16 deploy_conv_normal.prototxt already exist and didn't need to update /data/models_weight/orientation_carte_grise_all_2/deploy_fc.prototxt size_local : 1130 size in s3 : 1130 create time local : 2021-08-09 05:29:00 create time in s3 : 2021-08-06 20:07:16 deploy_fc.prototxt already exist and didn't need to update /data/models_weight/orientation_carte_grise_all_2/deploy.prototxt size_local : 5653 size in s3 : 5653 create time local : 2021-08-09 05:29:00 create time in s3 : 2021-08-06 20:07:16 deploy.prototxt already exist and didn't need to update /data/models_weight/orientation_carte_grise_all_2/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:29:00 create time in s3 : 2021-08-06 20:07:31 mean.npy already exist and didn't need to update /data/models_weight/orientation_carte_grise_all_2/synset_words.txt size_local : 159 size in s3 : 159 create time local : 2021-08-09 05:29:00 create time in s3 : 2021-08-06 20:07:16 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/:/home/admin/workarea/git/apy/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/orientation_carte_grise_all_2/deploy.prototxt caffemodel_filename : /data/models_weight/orientation_carte_grise_all_2/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 2930 max_wait_temp : 1 max_wait : 0 dict_keys(['pool5', 'prob']) time used to do the prepocess of the images : 1.5548603534698486 time used to do the prediction : 0.10913586616516113 save descriptor for thcl : 500 (1, 512, 7, 7) Got the blobs of the net to insert : [0, 0, 0, 0, 0, 0, 0, 0, 0, 0] code_as_byte_string:b'0000000000'| time to traite the descriptors : 0.04782414436340332 Testing : ['917849322'] In select_photos_meta_from_ids: SELECT photo_id, url, FROM_UNIXTIME(uploaded_at), latitude, longitude, text FROM MTRBack.view_photos_with_urls_250912 WHERE photo_id IN (917849322) result : {917849322: {'photo_id': 917849322, 'url': 'https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/2022/9/13/2bd260e91e91df8378dde8bb8b8c4548.jpg', 'latitude': None, 'longitude': None, 'text': 'image_13092022_12_32_19_5566.jpg'}} list_photo_exists : [917849322] storage_type for insertDescriptorsMulti : 1 To insert : 917849322 time to insert the descriptors : 0.6743607521057129 After datou_step_exec type output : time spend for datou_step_exec : 8.951798915863037 time spend to save output : 7.319450378417969e-05 total time spend for step 1 : 8.951872110366821 step2:argmax Wed Sep 24 17:49:21 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 : {'917849322': [[('917849322', 'carteGrisesVerticales__port_549774', 0.99764854, 507, '500'), ('917849322', 'cartegrise_90deg__port_550987', 0.0005037934, 507, '500'), ('917849322', 'cartesGrisesEnvers__port_549765', 0.0003663915, 507, '500'), ('917849322', 'portfolio_270deg__port_550988', 0.001481267, 507, '500')]]} input_args_next_step : {'917849322': ()} output_args : {'917849322': [[('917849322', 'carteGrisesVerticales__port_549774', 0.99764854, 507, '500'), ('917849322', 'cartegrise_90deg__port_550987', 0.0005037934, 507, '500'), ('917849322', 'cartesGrisesEnvers__port_549765', 0.0003663915, 507, '500'), ('917849322', 'portfolio_270deg__port_550988', 0.001481267, 507, '500')]]} args : 917849322 depend.output_id : 0 VR 22-3-18 : For now we do not clean correctly the datou structure input_args_next_step, len :1, first value : ([('917849322', 'carteGrisesVerticales__port_549774', 0.99764854, 507, '500'), ('917849322', 'cartegrise_90deg__port_550987', 0.0005037934, 507, '500'), ('917849322', 'cartesGrisesEnvers__port_549765', 0.0003663915, 507, '500'), ('917849322', 'portfolio_270deg__port_550988', 0.001481267, 507, '500')],) After prepare type args : Here we display some param of map_info ! map_filenames : {'temp/1758728952_1864669_917849322_2bd260e91e91df8378dde8bb8b8c4548.jpg': 917849322} map_photo_id_path_extension : {917849322: {'path': 'temp/1758728952_1864669_917849322_2bd260e91e91df8378dde8bb8b8c4548.jpg', 'extension': 'jpg'}} map_subphoto_mainphoto : {} Beginning of datou_step Argmax ! calculate argmax for thcl : 500 After datou_step_exec type output : time spend for datou_step_exec : 0.00023698806762695312 time spend to save output : 3.647804260253906e-05 total time spend for step 2 : 0.0002734661102294922 step3:rotate Wed Sep 24 17:49:21 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 : {'917849322': [('917849322', 'carteGrisesVerticales__port_549774', 0.99764854, 507, '500'), 'temp/1758728952_1864669_917849322_2bd260e91e91df8378dde8bb8b8c4548.jpg']} input_args_next_step : {'917849322': ()} output_args : {'917849322': [('917849322', 'carteGrisesVerticales__port_549774', 0.99764854, 507, '500'), 'temp/1758728952_1864669_917849322_2bd260e91e91df8378dde8bb8b8c4548.jpg']} args : 917849322 depend.output_id : 1 complete output_args for input 1 : {'917849322': [('917849322', 'carteGrisesVerticales__port_549774', 0.99764854, 507, '500'), 'temp/1758728952_1864669_917849322_2bd260e91e91df8378dde8bb8b8c4548.jpg']} input_args_next_step : {'917849322': ('temp/1758728952_1864669_917849322_2bd260e91e91df8378dde8bb8b8c4548.jpg',)} output_args : {'917849322': [('917849322', 'carteGrisesVerticales__port_549774', 0.99764854, 507, '500'), 'temp/1758728952_1864669_917849322_2bd260e91e91df8378dde8bb8b8c4548.jpg']} args : 917849322 depend.output_id : 0 VR 22-3-18 : For now we do not clean correctly the datou structure input_args_next_step, len :1, first value : ('temp/1758728952_1864669_917849322_2bd260e91e91df8378dde8bb8b8c4548.jpg', ('917849322', 'carteGrisesVerticales__port_549774', 0.99764854, 507, '500')) After prepare type args : Here we display some param of map_info ! map_filenames : {'temp/1758728952_1864669_917849322_2bd260e91e91df8378dde8bb8b8c4548.jpg': 917849322} map_photo_id_path_extension : {917849322: {'path': 'temp/1758728952_1864669_917849322_2bd260e91e91df8378dde8bb8b8c4548.jpg', 'extension': 'jpg'}} map_subphoto_mainphoto : {} Beginning of datou_step_rotate ! We are in a datou with depends ! angle_condi : {'carteGrisesVerticales__port_549774': 0, 'cartegrise_90deg__port_550987': 270, 'portfolio_270deg__port_550988': 90, 'cartesGrisesEnvers__port_549765': 180} rotate photos for hashtag carteGrisesVerticales__port_549774 of 0 degres 1 photos founded : [917849322] batch 1 select photo_id, hashtag_id, `type`, x0, x1, y0, y1, score, id from MTRPhoto.crop_hashtag_ids where photo_id in ( 917849322) and `type` in (0) Loaded 0 chid ids of type : 0 SELECT crop_hashtag_id, points FROM MTRPhoto.crop_polygon_points WHERE crop_hashtag_id in () map_chi : {} photo_id in download_rotate_and_save : 917849322 list_chi_loc : 0 Use all angle ! Rotation of photo 917849322 of 0 degree temp/1758728952_1864669_917849322_2bd260e91e91df8378dde8bb8b8c4548.jpg [] 0 remove_crop_border : False version de PIL : 9.5.0 Needs to change image size ! [[ 1. 0.] [-0. 1.]] 0 [[ 1. 0.] [-0. 1.]] shrink_image : False image_rotate : image_path : temp/1758728952_1864669_917849322_2bd260e91e91df8378dde8bb8b8c4548.jpg path_name_rotate : temp/1758728952_1864669_917849322_2bd260e91e91df8378dde8bb8b8c45480.jpg image_rotate.mode : RGB About to upload 1 photos upload in portfolio : 551782 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1758728961_1864669 we have uploaded 1 photos in the portfolio 551782 time of upload the photos Elapsed time : 0.9328508377075195 map_filename_photo_id : 1 map_filename_photo_id : {'temp/1758728952_1864669_917849322_2bd260e91e91df8378dde8bb8b8c45480.jpg': 1385928376} Len new_chis : 1 Len list_new_chi_with_photo_id : 0 of type : 0 list_new_chi_with_photo_id : [] rotate photos for hashtag cartegrise_90deg__port_550987 of 270 degres 0 photos founded : [] rotate photos for hashtag portfolio_270deg__port_550988 of 90 degres 0 photos founded : [] rotate photos for hashtag cartesGrisesEnvers__port_549765 of 180 degres 0 photos founded : [] After datou_step_exec type output : time spend for datou_step_exec : 1.0256409645080566 time spend to save output : 4.100799560546875e-05 total time spend for step 3 : 1.025681972503662 caffe_path_current : About to save ! 1 Inside saveOutput : final : True verbose : True saveOutput not yet implemented for datou_step.type : rotate we use saveGeneral [917849322] map_info['map_portfolio_photo'] : {} final : True mtd_id 233 list_pids : [917849322] Looping around the photos to save general results len do output : 1 /1385928376Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('233', None, None, None, None, None, None, None, None) ('233', None, '917849322', None, None, None, None, None, None) begin to insert list_values into mtr_datou_result : length of list_values in save_final : 4 insert ignore into MTRPhoto.mtr_datou_result (mtd_id, mtr_portfolio_id,mtr_photo_id,result,result_long,result_double,hashtag_id,proba, mtr_current_id) values (%s,%s,%s,%s,%s,%s,%s,%s,%s) on duplicate key update mtr_portfolio_id = mtr_portfolio_id list_values : [('233', None, '1385928376', 'None', None, None, None, None, None), ('233', None, '917849322', None, None, None, None, None, None)] time used for this insertion : 0.01236724853515625 save_final save missing photos in datou_result : After save, about to update current ! datou_cur_ids : [] len(datou.list_steps) : 3 output : {1385928376: ['917849322', 'temp/1758728952_1864669_917849322_2bd260e91e91df8378dde8bb8b8c45480.jpg', []]} ############################### TEST data_augmentation_ellipse_varroa_tile_rotate ################################ SELECT id FROM MTRPhoto.crop_hashtag_ids WHERE photo_id=937852786 AND `type`=520 list_chi_ids : [] # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : step 316 crop is not linked in the step_by_step architecture ! Step 318 rotate have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 318 rotate have less outputs used (0) than in the step definition (3) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! DataTypes for each output/input checked ! Unexpected type seems boolean for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : DATA AUGMENTATION ELLIPSE VARROA TILE ROTATE Inside batchDatouExec : verbose : True ##### chargement datou SELECT name, created_at,limit_max FROM MTRDatou.mtr_datou WHERE id=243 SELECT mtd.id, mtdt.`type`, mtd.`param`, mtd.param_json, mtdt.nb_input, mtdt.nb_output, mtdt.prod, mtdt.is_local, mtdt.is_datou_depend, mtdt.is_photo_id_local FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_step_types mtdt WHERE mtdt.`id`=mtd.`type` AND mtd.mtd_id=243 SELECT mtd.id, mtd.mtd_id, mdsdt.id, mdsdt.name, mdsdt.description, msid.output_or_input, msid.data_order_id, mdsdt.type FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_steptype_io_datatypes msid, MTRDatou.mtr_datou_step_data_types mdsdt WHERE mtd.`type`=msid.`mtr_datou_step_type` AND mtd.mtd_id= 243 AND msid.data_type=mdsdt.id SELECT mts_id_output, id_output, mts_id_input, id_input FROM MTRDatou.mtr_datou_step_by_step WHERE mtd_id=243 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : step 316 crop is not linked in the step_by_step architecture ! Step 318 rotate have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 318 rotate have less outputs used (0) than in the step definition (3) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! DataTypes for each output/input checked ! no param json to modify List Step Type Loaded in datou : crop, tile, rotate list_input_json : [] ##### fin chargement datou ##### chargement data ##### Call load_data_input : nb_thread : 5 origin SELECT photo_id, url FROM MTRBack.view_photos_with_urls_250912 ph WHERE photo_id IN (937852786) Found this number of photos: 1 ##### Call download_photos : nb_thread : 5 begin to download photo : 937852786 download finish for photo 937852786 we have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB ##### After download_photos length of list_filenames : 1 ; length of list_pids : 1 ; length of list_args : 1 ##### After load_data_input time to download the photos : 0.38816022872924805 #### fin chargement data Blocking on flush ? No conitnuing About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : True number of steps : 3 step1:crop Wed Sep 24 17:49:22 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec After prepare type args : Here we display some param of map_info ! map_filenames : {'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67.jpg': 937852786} map_photo_id_path_extension : {937852786: {'path': 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67.jpg', 'extension': 'jpg'}} map_subphoto_mainphoto : {} Beginning of datou_step Crop ! param_json : {'hashtag_id_ellipse': 2087736828, 'photo_hashtag_type_from_ellipse': 520, 'token': '78d09a0790ec6ecbf119343125a81fdc', 'portfolio_name': 'crop_detect_varroa', 'photo_hashtag_type': 407, 'feed_id_new_photos_not_used': 549103, 'host': 'www.fotonower.com', 'margin': 8, 'upload_type': 'python'} margin_type : margin margin_value : [8, 8, 8, 8] Loading chi in step crop with photo_hashtag_type : 407 Loading chi in step crop for list_pids : 1 ! batch 1 select photo_id, hashtag_id, `type`, x0, x1, y0, y1, score, id from MTRPhoto.crop_hashtag_ids where photo_id in ( 937852786) and `type` in (407) Loaded 4 chid ids of type : 407 SELECT crop_hashtag_id, points FROM MTRPhoto.crop_polygon_points WHERE crop_hashtag_id in (8165075,8165076,8165077,8165078) +WARNING : Unexpected points, we should remove this data for chi_id : 8165075, for now we just ignore these empty polygon points +WARNING : Unexpected points, we should remove this data for chi_id : 8165076, for now we just ignore these empty polygon points +WARNING : Unexpected points, we should remove this data for chi_id : 8165077, for now we just ignore these empty polygon points +WARNING : Unexpected points, we should remove this data for chi_id : 8165078, for now we just ignore these empty polygon points SELECT * FROM MTRPhoto.crop_segments WHERE crop_hashtag_id in (8165075,8165076,8165077,8165078) SELECT * FROM MTRPhoto.crop_sum_segments WHERE crop_hashtag_id in (8165075,8165076,8165077,8165078) select photo_id, sub_photo_id, x0, x1, y0, y1, resize_coeff_x, resize_coeff_y, crop_type, id from MTRPhoto.photo_sub_photos where photo_id in ( 937852786) WARNING : margin is only used for type bib ! type of cropped photo chosen : we resize croppped photo by 1 on x axis and by 1 on y axis new_file_path_bib_crop : temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_bib_crop_8165075_0.jpg new_file_path_bib_crop : temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_bib_crop_8165076_0.jpg new_file_path_bib_crop : temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_bib_crop_8165077_0.jpg new_file_path_bib_crop : temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_bib_crop_8165078_0.jpg map_result returned by crop_photo_return_map_crop : length : 4 map_result after crop : {8165075: {'crop': 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_bib_crop_8165075_0.jpg', 'photo_id': 937852786, 'sub_photo_id': -1, 'coordonates': (426, 467, 312, 347), 'sub_photo_infos': (418, 475, 304, 355, 1, 1), 'same_chi': False}, 8165076: {'crop': 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_bib_crop_8165076_0.jpg', 'photo_id': 937852786, 'sub_photo_id': -1, 'coordonates': (411, 445, 443, 480), 'sub_photo_infos': (403, 453, 435, 480, 1, 1), 'same_chi': False}, 8165077: {'crop': 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_bib_crop_8165077_0.jpg', 'photo_id': 937852786, 'sub_photo_id': -1, 'coordonates': (103, 138, 358, 396), 'sub_photo_infos': (95, 146, 350, 404, 1, 1), 'same_chi': False}, 8165078: {'crop': 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_bib_crop_8165078_0.jpg', 'photo_id': 937852786, 'sub_photo_id': -1, 'coordonates': (104, 131, 256, 292), 'sub_photo_infos': (96, 139, 248, 300, 1, 1), 'same_chi': False}} Here we crop with rles About to insert : list_path_to_insert length 4 new photo from crops ! About to upload 4 photos upload in portfolio : 27193559 init cache_photo without model_param we have 4 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1758728966_1864669 batch_size : 0, verbose : True, strat_bulk_insert : ignore_different_from_first Unexecpected behavior in 07/2025 that can be generalized l287 : type_extension .jpg INSERT IGNORE INTO MTRUser.mtr_fotonower_photos_opt (`height`, `width`, `size_object`, `photo_id`, `base_url`, `date`, `chi_id`, `type_extension`, `hash`) VALUES (51, 57, 1794, 1385928380, 'https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com', '2025/9/24', 35344272, 'jpg', UNHEX('c2d4b7e9a87b733b24c212df7b55bc09')) ON DUPLICATE KEY UPDATE `height`=values(`height`), `width`=values(`width`), `size_object`=values(`size_object`), `photo_id`=values(`photo_id`), `base_url`=values(`base_url`), `date`=values(`date`), `chi_id`=values(`chi_id`), `type_extension`=values(`type_extension`), `hash`=values(`hash`) This is a hack ! batch_size : 0, verbose : True, strat_bulk_insert : ignore_different_from_first Unexecpected behavior in 07/2025 that can be generalized l287 : type_extension .jpg INSERT IGNORE INTO MTRUser.mtr_fotonower_photos_opt (`height`, `width`, `size_object`, `photo_id`, `base_url`, `date`, `chi_id`, `type_extension`, `hash`) VALUES (45, 50, 1790, 1385928381, 'https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com', '2025/9/24', 18938475, 'jpg', UNHEX('73bb947187a7cccf402d3c49d52f0e04')) ON DUPLICATE KEY UPDATE `height`=values(`height`), `width`=values(`width`), `size_object`=values(`size_object`), `photo_id`=values(`photo_id`), `base_url`=values(`base_url`), `date`=values(`date`), `chi_id`=values(`chi_id`), `type_extension`=values(`type_extension`), `hash`=values(`hash`) This is a hack ! batch_size : 0, verbose : True, strat_bulk_insert : ignore_different_from_first Unexecpected behavior in 07/2025 that can be generalized l287 : type_extension .jpg INSERT IGNORE INTO MTRUser.mtr_fotonower_photos_opt (`height`, `width`, `size_object`, `photo_id`, `base_url`, `date`, `chi_id`, `type_extension`, `hash`) VALUES (54, 51, 1867, 1385928382, 'https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com', '2025/9/24', 12242232, 'jpg', UNHEX('cef221ccc8163d341ac8369fab7a6492')) ON DUPLICATE KEY UPDATE `height`=values(`height`), `width`=values(`width`), `size_object`=values(`size_object`), `photo_id`=values(`photo_id`), `base_url`=values(`base_url`), `date`=values(`date`), `chi_id`=values(`chi_id`), `type_extension`=values(`type_extension`), `hash`=values(`hash`) This is a hack ! batch_size : 0, verbose : True, strat_bulk_insert : ignore_different_from_first Unexecpected behavior in 07/2025 that can be generalized l287 : type_extension .jpg INSERT IGNORE INTO MTRUser.mtr_fotonower_photos_opt (`height`, `width`, `size_object`, `photo_id`, `base_url`, `date`, `chi_id`, `type_extension`, `hash`) VALUES (52, 43, 1418, 1385928383, 'https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com', '2025/9/24', 47193970, 'jpg', UNHEX('036d93e6c08ecafaae7820b02e96abeb')) ON DUPLICATE KEY UPDATE `height`=values(`height`), `width`=values(`width`), `size_object`=values(`size_object`), `photo_id`=values(`photo_id`), `base_url`=values(`base_url`), `date`=values(`date`), `chi_id`=values(`chi_id`), `type_extension`=values(`type_extension`), `hash`=values(`hash`) This is a hack ! INSERT ignore into MTRUser.mtr_portfolio_photos (`mtr_portfolio_id`, `mtr_photo_id`, `mtr_user_id`, `created_at`) VALUES (27193559, 1385928380, 0, NOW()),(27193559, 1385928381, 0, NOW()),(27193559, 1385928382, 0, NOW()),(27193559, 1385928383, 0, NOW()) 4 we have uploaded 4 photos in the portfolio 27193559 time of upload the photos Elapsed time : 4.927336931228638 {'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_bib_crop_8165075_0.jpg': 1385928380, 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_bib_crop_8165076_0.jpg': 1385928381, 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_bib_crop_8165077_0.jpg': 1385928382, 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_bib_crop_8165078_0.jpg': 1385928383} list_errors : [] map_result_insert : {'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_bib_crop_8165075_0.jpg': 1385928380, 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_bib_crop_8165076_0.jpg': 1385928381, 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_bib_crop_8165077_0.jpg': 1385928382, 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_bib_crop_8165078_0.jpg': 1385928383} Now we prepare data that will be used for ellipse search ! chi_id found to be used 8165075 path of cropped varroa found to be used to match on an ellipse temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_bib_crop_8165075_0.jpg sub_photo_id found to be used 1385928380 chi_id found to be used 8165076 path of cropped varroa found to be used to match on an ellipse temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_bib_crop_8165076_0.jpg sub_photo_id found to be used 1385928381 chi_id found to be used 8165077 path of cropped varroa found to be used to match on an ellipse temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_bib_crop_8165077_0.jpg sub_photo_id found to be used 1385928382 chi_id found to be used 8165078 path of cropped varroa found to be used to match on an ellipse temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_bib_crop_8165078_0.jpg sub_photo_id found to be used 1385928383 insert ignore into MTRPhoto.crop_sub_photo_ids (crop_hashtag_id, sub_photo_id, mtr_user_id) VALUES (%s,%s,%s) : [(8165075, '1385928380', 31), (8165076, '1385928381', 31), (8165077, '1385928382', 31), (8165078, '1385928383', 31)] map of cropped photos with some data : {'1385928380': ['937852786', 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_bib_crop_8165075_0.jpg', (426, 467, 312, 347)], '1385928381': ['937852786', 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_bib_crop_8165076_0.jpg', (411, 445, 443, 480)], '1385928382': ['937852786', 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_bib_crop_8165077_0.jpg', (103, 138, 358, 396)], '1385928383': ['937852786', 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_bib_crop_8165078_0.jpg', (104, 131, 256, 292)]} About to compute ellipse and record with type : 520 (54, 57) (51, 57) [54, 57] (54, 57) score : 5120 strategy_opt : 5| strategy_opt : [('excentricity', [0.5, 2.0, 0.05]), ('angle', [-90.0, 90.0, 5.0]), ('xc', [14.25, 42.75, 1.78125]), ('yc', [12.75, 38.25, 1.59375]), ('radius', [14.25, 42.75, 1.78125])] {0.5: 18351, 0.55: 16004, 0.6000000000000001: 14127, 0.65: 12575, 0.7000000000000001: 11300, 0.75: 10158, 0.8: 9208, 0.8500000000000001: 8418, 0.9: 7642, 0.9500000000000001: 6925, 1.0: 6364, 1.05: 5843, 1.1: 5353, 1.1500000000000001: 4868, 1.2000000000000002: 4553, 1.25: 4250, 1.3: 3947, 1.35: 3698, 1.4000000000000001: 3438, 1.4500000000000002: 3230, 1.5: 3012, 1.55: 2829, 1.6: 2636, 1.6500000000000001: 2535, 1.7000000000000002: 2460, 1.75: 2388, 1.8: 2372, 1.85: 2354, 1.9000000000000001: 2332, 1.9500000000000002: 2311} [(1.9500000000000002, 2311), (1.9000000000000001, 2332), (1.85, 2354), (1.8, 2372), (1.75, 2388), (1.7000000000000002, 2460), (1.6500000000000001, 2535), (1.6, 2636), (1.55, 2829), (1.5, 3012), (1.4500000000000002, 3230), (1.4000000000000001, 3438), (1.35, 3698), (1.3, 3947), (1.25, 4250), (1.2000000000000002, 4553), (1.1500000000000001, 4868), (1.1, 5353), (1.05, 5843), (1.0, 6364), (0.9500000000000001, 6925), (0.9, 7642), (0.8500000000000001, 8418), (0.8, 9208), (0.75, 10158), (0.7000000000000001, 11300), (0.65, 12575), (0.6000000000000001, 14127), (0.55, 16004), (0.5, 18351)] arg_min reach at : 1.9500000000000002 with value = 2311 | arg_min : 1.9500000000000002 min_score : 2311{-90.0: 3359, -85.0: 3372, -80.0: 3395, -75.0: 3380, -70.0: 3304, -65.0: 3180, -60.0: 2991, -55.0: 2793, -50.0: 2601, -45.0: 2377, -40.0: 2183, -35.0: 2078, -30.0: 1968, -25.0: 1968, -20.0: 2021, -15.0: 2036, -10.0: 2102, -5.0: 2188, 0.0: 2208, 5.0: 2265, 10.0: 2311, 15.0: 2333, 20.0: 2362, 25.0: 2375, 30.0: 2375, 35.0: 2397, 40.0: 2370, 45.0: 2421, 50.0: 2491, 55.0: 2661, 60.0: 2826, 65.0: 2993, 70.0: 3106, 75.0: 3182, 80.0: 3263, 85.0: 3306} [(-30.0, 1968), (-25.0, 1968), (-20.0, 2021), (-15.0, 2036), (-35.0, 2078), (-10.0, 2102), (-40.0, 2183), (-5.0, 2188), (0.0, 2208), (5.0, 2265), (10.0, 2311), (15.0, 2333), (20.0, 2362), (40.0, 2370), (25.0, 2375), (30.0, 2375), (-45.0, 2377), (35.0, 2397), (45.0, 2421), (50.0, 2491), (-50.0, 2601), (55.0, 2661), (-55.0, 2793), (60.0, 2826), (-60.0, 2991), (65.0, 2993), (70.0, 3106), (-65.0, 3180), (75.0, 3182), (80.0, 3263), (-70.0, 3304), (85.0, 3306), (-90.0, 3359), (-85.0, 3372), (-75.0, 3380), (-80.0, 3395)] arg_min reach at : -30.0 with value = 1968 | arg_min : -30.0 min_score : 1968{14.25: 1703, 16.03125: 1654, 17.8125: 1614, 19.59375: 1679, 21.375: 1702, 23.15625: 1712, 24.9375: 1730, 26.71875: 1847, 28.5: 1968, 30.28125: 2177, 32.0625: 2456, 33.84375: 2827, 35.625: 3294, 37.40625: 3723, 39.1875: 4161, 40.96875: 4643} [(17.8125, 1614), (16.03125, 1654), (19.59375, 1679), (21.375, 1702), (14.25, 1703), (23.15625, 1712), (24.9375, 1730), (26.71875, 1847), (28.5, 1968), (30.28125, 2177), (32.0625, 2456), (33.84375, 2827), (35.625, 3294), (37.40625, 3723), (39.1875, 4161), (40.96875, 4643)] arg_min reach at : 17.8125 with value = 1614 | arg_min : 17.8125 min_score : 1614{12.75: 5534, 14.34375: 5352, 15.9375: 5116, 17.53125: 4723, 19.125: 4111, 20.71875: 3506, 22.3125: 2799, 23.90625: 2194, 25.5: 1614, 27.09375: 1339, 28.6875: 1206, 30.28125: 1137, 31.875: 1105, 33.46875: 1339, 35.0625: 1659, 36.65625: 2022} [(31.875, 1105), (30.28125, 1137), (28.6875, 1206), (27.09375, 1339), (33.46875, 1339), (25.5, 1614), (35.0625, 1659), (36.65625, 2022), (23.90625, 2194), (22.3125, 2799), (20.71875, 3506), (19.125, 4111), (17.53125, 4723), (15.9375, 5116), (14.34375, 5352), (12.75, 5534)] arg_min reach at : 31.875 with value = 1105 | arg_min : 31.875 min_score : 1105{14.25: 1942, 16.03125: 1853, 17.8125: 1756, 19.59375: 1651, 21.375: 1533, 23.15625: 1424, 24.9375: 1309, 26.71875: 1199, 28.5: 1105, 30.28125: 1219, 32.0625: 1332, 33.84375: 1547, 35.625: 1881, 37.40625: 2378, 39.1875: 3042, 40.96875: 3824} [(28.5, 1105), (26.71875, 1199), (30.28125, 1219), (24.9375, 1309), (32.0625, 1332), (23.15625, 1424), (21.375, 1533), (33.84375, 1547), (19.59375, 1651), (17.8125, 1756), (16.03125, 1853), (35.625, 1881), (14.25, 1942), (37.40625, 2378), (39.1875, 3042), (40.96875, 3824)] arg_min reach at : 28.5 with value = 1105 | arg_min : 28.5 min_score : 1105{0.5: 17397, 0.55: 15138, 0.6000000000000001: 13352, 0.65: 11859, 0.7000000000000001: 10523, 0.75: 9463, 0.8: 8469, 0.8500000000000001: 7565, 0.9: 6551, 0.9500000000000001: 5592, 1.0: 4856, 1.05: 4291, 1.1: 3654, 1.1500000000000001: 3099, 1.2000000000000002: 2647, 1.25: 2309, 1.3: 1983, 1.35: 1755, 1.4000000000000001: 1594, 1.4500000000000002: 1447, 1.5: 1358, 1.55: 1323, 1.6: 1219, 1.6500000000000001: 1215, 1.7000000000000002: 1201, 1.75: 1131, 1.8: 1134, 1.85: 1142, 1.9000000000000001: 1137, 1.9500000000000002: 1105} [(1.9500000000000002, 1105), (1.75, 1131), (1.8, 1134), (1.9000000000000001, 1137), (1.85, 1142), (1.7000000000000002, 1201), (1.6500000000000001, 1215), (1.6, 1219), (1.55, 1323), (1.5, 1358), (1.4500000000000002, 1447), (1.4000000000000001, 1594), (1.35, 1755), (1.3, 1983), (1.25, 2309), (1.2000000000000002, 2647), (1.1500000000000001, 3099), (1.1, 3654), (1.05, 4291), (1.0, 4856), (0.9500000000000001, 5592), (0.9, 6551), (0.8500000000000001, 7565), (0.8, 8469), (0.75, 9463), (0.7000000000000001, 10523), (0.65, 11859), (0.6000000000000001, 13352), (0.55, 15138), (0.5, 17397)] arg_min reach at : 1.9500000000000002 with value = 1105 arg_min : 1.9500000000000002 min_score : 1105{-90.0: 3514, -85.0: 3396, -80.0: 3238, -75.0: 3017, -70.0: 2764, -65.0: 2549, -60.0: 2339, -55.0: 2105, -50.0: 1912, -45.0: 1701, -40.0: 1467, -35.0: 1266, -30.0: 1105, -25.0: 1110, -20.0: 1114, -15.0: 1111, -10.0: 1110, -5.0: 1121, 0.0: 1120, 5.0: 1118, 10.0: 1109, 15.0: 1109, 20.0: 1099, 25.0: 1088, 30.0: 1108, 35.0: 1355, 40.0: 1649, 45.0: 1925, 50.0: 2257, 55.0: 2542, 60.0: 2845, 65.0: 3137, 70.0: 3370, 75.0: 3500, 80.0: 3578, 85.0: 3540} [(25.0, 1088), (20.0, 1099), (-30.0, 1105), (30.0, 1108), (10.0, 1109), (15.0, 1109), (-25.0, 1110), (-10.0, 1110), (-15.0, 1111), (-20.0, 1114), (5.0, 1118), (0.0, 1120), (-5.0, 1121), (-35.0, 1266), (35.0, 1355), (-40.0, 1467), (40.0, 1649), (-45.0, 1701), (-50.0, 1912), (45.0, 1925), (-55.0, 2105), (50.0, 2257), (-60.0, 2339), (55.0, 2542), (-65.0, 2549), (-70.0, 2764), (60.0, 2845), (-75.0, 3017), (65.0, 3137), (-80.0, 3238), (70.0, 3370), (-85.0, 3396), (75.0, 3500), (-90.0, 3514), (85.0, 3540), (80.0, 3578)] arg_min reach at : 25.0 with value = 1088 arg_min : 25.0 min_score : 1088{14.25: 1174, 16.03125: 1133, 17.8125: 1088, 19.59375: 1051, 21.375: 1026, 23.15625: 997, 24.9375: 979, 26.71875: 1139, 28.5: 1344, 30.28125: 1564, 32.0625: 1949, 33.84375: 2422, 35.625: 2930, 37.40625: 3453, 39.1875: 4006, 40.96875: 4592} [(24.9375, 979), (23.15625, 997), (21.375, 1026), (19.59375, 1051), (17.8125, 1088), (16.03125, 1133), (26.71875, 1139), (14.25, 1174), (28.5, 1344), (30.28125, 1564), (32.0625, 1949), (33.84375, 2422), (35.625, 2930), (37.40625, 3453), (39.1875, 4006), (40.96875, 4592)] arg_min reach at : 24.9375 with value = 979 arg_min : 24.9375 min_score : 979{12.75: 5539, 14.34375: 5239, 15.9375: 4911, 17.53125: 4425, 19.125: 3749, 20.71875: 3181, 22.3125: 2575, 23.90625: 2124, 25.5: 1808, 27.09375: 1478, 28.6875: 1290, 30.28125: 1104, 31.875: 979, 33.46875: 1070, 35.0625: 1368, 36.65625: 1872} [(31.875, 979), (33.46875, 1070), (30.28125, 1104), (28.6875, 1290), (35.0625, 1368), (27.09375, 1478), (25.5, 1808), (36.65625, 1872), (23.90625, 2124), (22.3125, 2575), (20.71875, 3181), (19.125, 3749), (17.53125, 4425), (15.9375, 4911), (14.34375, 5239), (12.75, 5539)] arg_min reach at : 31.875 with value = 979 arg_min : 31.875 min_score : 979{14.25: 1939, 16.03125: 1852, 17.8125: 1755, 19.59375: 1643, 21.375: 1531, 23.15625: 1397, 24.9375: 1264, 26.71875: 1119, 28.5: 979, 30.28125: 1023, 32.0625: 1319, 33.84375: 1771, 35.625: 2432, 37.40625: 3324, 39.1875: 4299, 40.96875: 5405} [(28.5, 979), (30.28125, 1023), (26.71875, 1119), (24.9375, 1264), (32.0625, 1319), (23.15625, 1397), (21.375, 1531), (19.59375, 1643), (17.8125, 1755), (33.84375, 1771), (16.03125, 1852), (14.25, 1939), (35.625, 2432), (37.40625, 3324), (39.1875, 4299), (40.96875, 5405)] arg_min reach at : 28.5 with value = 979 arg_min : 28.5 min_score : 979 yc : 31.875 xc : 24.9375 angle : 25.0 radius : 28.5 excentricity : 1.9500000000000002 yc : 31.875 xc : 24.9375 angle : 25.0 radius : 28.5 excentricity : 1.9500000000000002 x0 : 426 y1 : 347 width : 41, height : 35, area : 1435, score : 1.0 x0 : 432 y1 : 355 width : 35, height : 52, area : 1820, score : 1.0 Now saving polygons points : 1| batch 1 Loaded 1 chid ids of type : 520 CHI and polygons saved ! (47, 50) (45, 50) [47, 50] (47, 50) score : 5362 strategy_opt : 5| strategy_opt : [('excentricity', [0.5, 2.0, 0.05]), ('angle', [-90.0, 90.0, 5.0]), ('xc', [12.5, 37.5, 1.5625]), ('yc', [11.25, 33.75, 1.40625]), ('radius', [12.5, 37.5, 1.5625])] {0.5: 15776, 0.55: 13889, 0.6000000000000001: 12446, 0.65: 11147, 0.7000000000000001: 10181, 0.75: 9228, 0.8: 8441, 0.8500000000000001: 7765, 0.9: 7258, 0.9500000000000001: 6684, 1.0: 6195, 1.05: 5920, 1.1: 5487, 1.1500000000000001: 5099, 1.2000000000000002: 4795, 1.25: 4511, 1.3: 4245, 1.35: 4042, 1.4000000000000001: 3828, 1.4500000000000002: 3629, 1.5: 3467, 1.55: 3235, 1.6: 3126, 1.6500000000000001: 2946, 1.7000000000000002: 2826, 1.75: 2700, 1.8: 2549, 1.85: 2472, 1.9000000000000001: 2381, 1.9500000000000002: 2281} [(1.9500000000000002, 2281), (1.9000000000000001, 2381), (1.85, 2472), (1.8, 2549), (1.75, 2700), (1.7000000000000002, 2826), (1.6500000000000001, 2946), (1.6, 3126), (1.55, 3235), (1.5, 3467), (1.4500000000000002, 3629), (1.4000000000000001, 3828), (1.35, 4042), (1.3, 4245), (1.25, 4511), (1.2000000000000002, 4795), (1.1500000000000001, 5099), (1.1, 5487), (1.05, 5920), (1.0, 6195), (0.9500000000000001, 6684), (0.9, 7258), (0.8500000000000001, 7765), (0.8, 8441), (0.75, 9228), (0.7000000000000001, 10181), (0.65, 11147), (0.6000000000000001, 12446), (0.55, 13889), (0.5, 15776)] arg_min reach at : 1.9500000000000002 with value = 2281 | arg_min : 1.9500000000000002 min_score : 2281{-90.0: 3295, -85.0: 3240, -80.0: 3290, -75.0: 3269, -70.0: 3287, -65.0: 3223, -60.0: 3095, -55.0: 3043, -50.0: 2985, -45.0: 2906, -40.0: 2796, -35.0: 2724, -30.0: 2625, -25.0: 2489, -20.0: 2330, -15.0: 2220, -10.0: 2127, -5.0: 2127, 0.0: 2143, 5.0: 2182, 10.0: 2281, 15.0: 2385, 20.0: 2506, 25.0: 2665, 30.0: 2768, 35.0: 2900, 40.0: 2939, 45.0: 2994, 50.0: 3007, 55.0: 3076, 60.0: 3095, 65.0: 3201, 70.0: 3243, 75.0: 3247, 80.0: 3279, 85.0: 3240} [(-10.0, 2127), (-5.0, 2127), (0.0, 2143), (5.0, 2182), (-15.0, 2220), (10.0, 2281), (-20.0, 2330), (15.0, 2385), (-25.0, 2489), (20.0, 2506), (-30.0, 2625), (25.0, 2665), (-35.0, 2724), (30.0, 2768), (-40.0, 2796), (35.0, 2900), (-45.0, 2906), (40.0, 2939), (-50.0, 2985), (45.0, 2994), (50.0, 3007), (-55.0, 3043), (55.0, 3076), (-60.0, 3095), (60.0, 3095), (65.0, 3201), (-65.0, 3223), (-85.0, 3240), (85.0, 3240), (70.0, 3243), (75.0, 3247), (-75.0, 3269), (80.0, 3279), (-70.0, 3287), (-80.0, 3290), (-90.0, 3295)] arg_min reach at : -10.0 with value = 2127 | arg_min : -10.0 min_score : 2127{12.5: 2171, 14.0625: 2288, 15.625: 2314, 17.1875: 2360, 18.75: 2347, 20.3125: 2300, 21.875: 2249, 23.4375: 2188, 25.0: 2127, 26.5625: 2163, 28.125: 2315, 29.6875: 2476, 31.25: 2756, 32.8125: 2992, 34.375: 3319, 35.9375: 3672} [(25.0, 2127), (26.5625, 2163), (12.5, 2171), (23.4375, 2188), (21.875, 2249), (14.0625, 2288), (20.3125, 2300), (15.625, 2314), (28.125, 2315), (18.75, 2347), (17.1875, 2360), (29.6875, 2476), (31.25, 2756), (32.8125, 2992), (34.375, 3319), (35.9375, 3672)] arg_min reach at : 25.0 with value = 2127 | arg_min : 25.0 min_score : 2127{11.25: 6815, 12.65625: 6505, 14.0625: 5928, 15.46875: 5326, 16.875: 4772, 18.28125: 4095, 19.6875: 3413, 21.09375: 2728, 22.5: 2127, 23.90625: 1694, 25.3125: 1345, 26.71875: 1048, 28.125: 823, 29.53125: 728, 30.9375: 714, 32.34375: 1027} [(30.9375, 714), (29.53125, 728), (28.125, 823), (32.34375, 1027), (26.71875, 1048), (25.3125, 1345), (23.90625, 1694), (22.5, 2127), (21.09375, 2728), (19.6875, 3413), (18.28125, 4095), (16.875, 4772), (15.46875, 5326), (14.0625, 5928), (12.65625, 6505), (11.25, 6815)] arg_min reach at : 30.9375 with value = 714 | arg_min : 30.9375 min_score : 714{12.5: 1275, 14.0625: 1207, 15.625: 1132, 17.1875: 1049, 18.75: 966, 20.3125: 866, 21.875: 784, 23.4375: 729, 25.0: 714, 26.5625: 976, 28.125: 1546, 29.6875: 2147, 31.25: 2981, 32.8125: 3767, 34.375: 4761, 35.9375: 5732} [(25.0, 714), (23.4375, 729), (21.875, 784), (20.3125, 866), (18.75, 966), (26.5625, 976), (17.1875, 1049), (15.625, 1132), (14.0625, 1207), (12.5, 1275), (28.125, 1546), (29.6875, 2147), (31.25, 2981), (32.8125, 3767), (34.375, 4761), (35.9375, 5732)] arg_min reach at : 25.0 with value = 714 | arg_min : 25.0 min_score : 714{0.5: 20107, 0.55: 18155, 0.6000000000000001: 16503, 0.65: 15058, 0.7000000000000001: 13673, 0.75: 12474, 0.8: 11100, 0.8500000000000001: 9667, 0.9: 8374, 0.9500000000000001: 7344, 1.0: 6274, 1.05: 5570, 1.1: 4846, 1.1500000000000001: 4170, 1.2000000000000002: 3623, 1.25: 3211, 1.3: 2832, 1.35: 2475, 1.4000000000000001: 2182, 1.4500000000000002: 1949, 1.5: 1655, 1.55: 1513, 1.6: 1333, 1.6500000000000001: 1182, 1.7000000000000002: 1038, 1.75: 921, 1.8: 841, 1.85: 732, 1.9000000000000001: 733, 1.9500000000000002: 714} [(1.9500000000000002, 714), (1.85, 732), (1.9000000000000001, 733), (1.8, 841), (1.75, 921), (1.7000000000000002, 1038), (1.6500000000000001, 1182), (1.6, 1333), (1.55, 1513), (1.5, 1655), (1.4500000000000002, 1949), (1.4000000000000001, 2182), (1.35, 2475), (1.3, 2832), (1.25, 3211), (1.2000000000000002, 3623), (1.1500000000000001, 4170), (1.1, 4846), (1.05, 5570), (1.0, 6274), (0.9500000000000001, 7344), (0.9, 8374), (0.8500000000000001, 9667), (0.8, 11100), (0.75, 12474), (0.7000000000000001, 13673), (0.65, 15058), (0.6000000000000001, 16503), (0.55, 18155), (0.5, 20107)] arg_min reach at : 1.9500000000000002 with value = 714 arg_min : 1.9500000000000002 min_score : 714{-90.0: 2866, -85.0: 3006, -80.0: 3063, -75.0: 3168, -70.0: 3193, -65.0: 3193, -60.0: 3149, -55.0: 3022, -50.0: 2885, -45.0: 2656, -40.0: 2413, -35.0: 2119, -30.0: 1819, -25.0: 1500, -20.0: 1171, -15.0: 865, -10.0: 714, -5.0: 668, 0.0: 689, 5.0: 734, 10.0: 824, 15.0: 898, 20.0: 1116, 25.0: 1368, 30.0: 1610, 35.0: 1866, 40.0: 2105, 45.0: 2304, 50.0: 2467, 55.0: 2604, 60.0: 2731, 65.0: 2753, 70.0: 2753, 75.0: 2761, 80.0: 2722, 85.0: 2786} [(-5.0, 668), (0.0, 689), (-10.0, 714), (5.0, 734), (10.0, 824), (-15.0, 865), (15.0, 898), (20.0, 1116), (-20.0, 1171), (25.0, 1368), (-25.0, 1500), (30.0, 1610), (-30.0, 1819), (35.0, 1866), (40.0, 2105), (-35.0, 2119), (45.0, 2304), (-40.0, 2413), (50.0, 2467), (55.0, 2604), (-45.0, 2656), (80.0, 2722), (60.0, 2731), (65.0, 2753), (70.0, 2753), (75.0, 2761), (85.0, 2786), (-90.0, 2866), (-50.0, 2885), (-85.0, 3006), (-55.0, 3022), (-80.0, 3063), (-60.0, 3149), (-75.0, 3168), (-70.0, 3193), (-65.0, 3193)] arg_min reach at : -5.0 with value = 668 arg_min : -5.0 min_score : 668{12.5: 1164, 14.0625: 1082, 15.625: 1019, 17.1875: 933, 18.75: 866, 20.3125: 765, 21.875: 713, 23.4375: 655, 25.0: 668, 26.5625: 845, 28.125: 1107, 29.6875: 1374, 31.25: 1691, 32.8125: 2036, 34.375: 2375, 35.9375: 2731} [(23.4375, 655), (25.0, 668), (21.875, 713), (20.3125, 765), (26.5625, 845), (18.75, 866), (17.1875, 933), (15.625, 1019), (14.0625, 1082), (28.125, 1107), (12.5, 1164), (29.6875, 1374), (31.25, 1691), (32.8125, 2036), (34.375, 2375), (35.9375, 2731)] arg_min reach at : 23.4375 with value = 655 arg_min : 23.4375 min_score : 655{11.25: 7209, 12.65625: 6826, 14.0625: 6195, 15.46875: 5638, 16.875: 4941, 18.28125: 4177, 19.6875: 3484, 21.09375: 2750, 22.5: 2183, 23.90625: 1709, 25.3125: 1327, 26.71875: 1008, 28.125: 779, 29.53125: 631, 30.9375: 655, 32.34375: 920} [(29.53125, 631), (30.9375, 655), (28.125, 779), (32.34375, 920), (26.71875, 1008), (25.3125, 1327), (23.90625, 1709), (22.5, 2183), (21.09375, 2750), (19.6875, 3484), (18.28125, 4177), (16.875, 4941), (15.46875, 5638), (14.0625, 6195), (12.65625, 6826), (11.25, 7209)] arg_min reach at : 29.53125 with value = 631 arg_min : 29.53125 min_score : 631{12.5: 1282, 14.0625: 1211, 15.625: 1130, 17.1875: 1050, 18.75: 960, 20.3125: 871, 21.875: 772, 23.4375: 672, 25.0: 631, 26.5625: 666, 28.125: 947, 29.6875: 1472, 31.25: 2184, 32.8125: 3032, 34.375: 3932, 35.9375: 5005} [(25.0, 631), (26.5625, 666), (23.4375, 672), (21.875, 772), (20.3125, 871), (28.125, 947), (18.75, 960), (17.1875, 1050), (15.625, 1130), (14.0625, 1211), (12.5, 1282), (29.6875, 1472), (31.25, 2184), (32.8125, 3032), (34.375, 3932), (35.9375, 5005)] arg_min reach at : 25.0 with value = 631 arg_min : 25.0 min_score : 631 yc : 29.53125 xc : 23.4375 angle : -5.0 radius : 25.0 excentricity : 1.9500000000000002 yc : 29.53125 xc : 23.4375 angle : -5.0 radius : 25.0 excentricity : 1.9500000000000002 x0 : 411 y1 : 480 width : 34, height : 37, area : 1258, score : 1.0 x0 : 419 y1 : 483 width : 26, height : 49, area : 1274, score : 1.0 Now saving polygons points : 1| batch 1 Loaded 2 chid ids of type : 520 + CHI and polygons saved ! (55, 54) (54, 51) [55, 54] (55, 54) score : 4603 strategy_opt : 5| strategy_opt : [('excentricity', [0.5, 2.0, 0.05]), ('angle', [-90.0, 90.0, 5.0]), ('xc', [12.75, 38.25, 1.59375]), ('yc', [13.5, 40.5, 1.6875]), ('radius', [13.5, 40.5, 1.6875])] {0.5: 14831, 0.55: 12756, 0.6000000000000001: 10989, 0.65: 9469, 0.7000000000000001: 8158, 0.75: 7251, 0.8: 6254, 0.8500000000000001: 5559, 0.9: 5119, 0.9500000000000001: 4564, 1.0: 4072, 1.05: 3783, 1.1: 3496, 1.1500000000000001: 3342, 1.2000000000000002: 3228, 1.25: 3161, 1.3: 3127, 1.35: 3087, 1.4000000000000001: 3047, 1.4500000000000002: 3050, 1.5: 2990, 1.55: 2992, 1.6: 2992, 1.6500000000000001: 2985, 1.7000000000000002: 2988, 1.75: 2999, 1.8: 2994, 1.85: 2981, 1.9000000000000001: 3002, 1.9500000000000002: 2995} [(1.85, 2981), (1.6500000000000001, 2985), (1.7000000000000002, 2988), (1.5, 2990), (1.55, 2992), (1.6, 2992), (1.8, 2994), (1.9500000000000002, 2995), (1.75, 2999), (1.9000000000000001, 3002), (1.4000000000000001, 3047), (1.4500000000000002, 3050), (1.35, 3087), (1.3, 3127), (1.25, 3161), (1.2000000000000002, 3228), (1.1500000000000001, 3342), (1.1, 3496), (1.05, 3783), (1.0, 4072), (0.9500000000000001, 4564), (0.9, 5119), (0.8500000000000001, 5559), (0.8, 6254), (0.75, 7251), (0.7000000000000001, 8158), (0.65, 9469), (0.6000000000000001, 10989), (0.55, 12756), (0.5, 14831)] arg_min reach at : 1.85 with value = 2981 | arg_min : 1.85 min_score : 2981{-90.0: 1442, -85.0: 1411, -80.0: 1411, -75.0: 1423, -70.0: 1409, -65.0: 1428, -60.0: 1417, -55.0: 1385, -50.0: 1356, -45.0: 1374, -40.0: 1401, -35.0: 1501, -30.0: 1635, -25.0: 1772, -20.0: 1988, -15.0: 2183, -10.0: 2376, -5.0: 2552, 0.0: 2691, 5.0: 2860, 10.0: 2981, 15.0: 3063, 20.0: 3044, 25.0: 2938, 30.0: 2889, 35.0: 2865, 40.0: 2787, 45.0: 2694, 50.0: 2566, 55.0: 2452, 60.0: 2275, 65.0: 2055, 70.0: 1849, 75.0: 1731, 80.0: 1532, 85.0: 1466} [(-50.0, 1356), (-45.0, 1374), (-55.0, 1385), (-40.0, 1401), (-70.0, 1409), (-85.0, 1411), (-80.0, 1411), (-60.0, 1417), (-75.0, 1423), (-65.0, 1428), (-90.0, 1442), (85.0, 1466), (-35.0, 1501), (80.0, 1532), (-30.0, 1635), (75.0, 1731), (-25.0, 1772), (70.0, 1849), (-20.0, 1988), (65.0, 2055), (-15.0, 2183), (60.0, 2275), (-10.0, 2376), (55.0, 2452), (-5.0, 2552), (50.0, 2566), (0.0, 2691), (45.0, 2694), (40.0, 2787), (5.0, 2860), (35.0, 2865), (30.0, 2889), (25.0, 2938), (10.0, 2981), (20.0, 3044), (15.0, 3063)] arg_min reach at : -50.0 with value = 1356 | arg_min : -50.0 min_score : 1356{12.75: 3483, 14.34375: 3154, 15.9375: 2818, 17.53125: 2499, 19.125: 2212, 20.71875: 1992, 22.3125: 1750, 23.90625: 1554, 25.5: 1356, 27.09375: 1213, 28.6875: 1123, 30.28125: 1079, 31.875: 1312, 33.46875: 1586, 35.0625: 1971, 36.65625: 2464} [(30.28125, 1079), (28.6875, 1123), (27.09375, 1213), (31.875, 1312), (25.5, 1356), (23.90625, 1554), (33.46875, 1586), (22.3125, 1750), (35.0625, 1971), (20.71875, 1992), (19.125, 2212), (36.65625, 2464), (17.53125, 2499), (15.9375, 2818), (14.34375, 3154), (12.75, 3483)] arg_min reach at : 30.28125 with value = 1079 | arg_min : 30.28125 min_score : 1079{13.5: 1299, 15.1875: 1190, 16.875: 1129, 18.5625: 1073, 20.25: 1041, 21.9375: 1025, 23.625: 995, 25.3125: 1002, 27.0: 1079, 28.6875: 1280, 30.375: 1506, 32.0625: 1818, 33.75: 2218, 35.4375: 2606, 37.125: 3013, 38.8125: 3545} [(23.625, 995), (25.3125, 1002), (21.9375, 1025), (20.25, 1041), (18.5625, 1073), (27.0, 1079), (16.875, 1129), (15.1875, 1190), (28.6875, 1280), (13.5, 1299), (30.375, 1506), (32.0625, 1818), (33.75, 2218), (35.4375, 2606), (37.125, 3013), (38.8125, 3545)] arg_min reach at : 23.625 with value = 995 | arg_min : 23.625 min_score : 995{13.5: 1907, 15.1875: 1827, 16.875: 1734, 18.5625: 1635, 20.25: 1520, 21.9375: 1400, 23.625: 1268, 25.3125: 1139, 27.0: 995, 28.6875: 1081, 30.375: 1376, 32.0625: 1837, 33.75: 2365, 35.4375: 3034, 37.125: 3644, 38.8125: 4482} [(27.0, 995), (28.6875, 1081), (25.3125, 1139), (23.625, 1268), (30.375, 1376), (21.9375, 1400), (20.25, 1520), (18.5625, 1635), (16.875, 1734), (15.1875, 1827), (32.0625, 1837), (13.5, 1907), (33.75, 2365), (35.4375, 3034), (37.125, 3644), (38.8125, 4482)] arg_min reach at : 27.0 with value = 995 | arg_min : 27.0 min_score : 995{0.5: 16358, 0.55: 14291, 0.6000000000000001: 12544, 0.65: 11117, 0.7000000000000001: 9800, 0.75: 8586, 0.8: 7533, 0.8500000000000001: 6488, 0.9: 5599, 0.9500000000000001: 4779, 1.0: 4075, 1.05: 3520, 1.1: 3021, 1.1500000000000001: 2582, 1.2000000000000002: 2150, 1.25: 1820, 1.3: 1534, 1.35: 1347, 1.4000000000000001: 1219, 1.4500000000000002: 1130, 1.5: 1073, 1.55: 1037, 1.6: 998, 1.6500000000000001: 961, 1.7000000000000002: 981, 1.75: 984, 1.8: 995, 1.85: 995, 1.9000000000000001: 1023, 1.9500000000000002: 1058} [(1.6500000000000001, 961), (1.7000000000000002, 981), (1.75, 984), (1.8, 995), (1.85, 995), (1.6, 998), (1.9000000000000001, 1023), (1.55, 1037), (1.9500000000000002, 1058), (1.5, 1073), (1.4500000000000002, 1130), (1.4000000000000001, 1219), (1.35, 1347), (1.3, 1534), (1.25, 1820), (1.2000000000000002, 2150), (1.1500000000000001, 2582), (1.1, 3021), (1.05, 3520), (1.0, 4075), (0.9500000000000001, 4779), (0.9, 5599), (0.8500000000000001, 6488), (0.8, 7533), (0.75, 8586), (0.7000000000000001, 9800), (0.65, 11117), (0.6000000000000001, 12544), (0.55, 14291), (0.5, 16358)] arg_min reach at : 1.6500000000000001 with value = 961 arg_min : 1.6500000000000001 min_score : 961{-90.0: 906, -85.0: 871, -80.0: 858, -75.0: 866, -70.0: 852, -65.0: 864, -60.0: 855, -55.0: 853, -50.0: 961, -45.0: 1180, -40.0: 1412, -35.0: 1690, -30.0: 1959, -25.0: 2243, -20.0: 2484, -15.0: 2677, -10.0: 2820, -5.0: 2976, 0.0: 3039, 5.0: 3049, 10.0: 3098, 15.0: 3122, 20.0: 3061, 25.0: 2954, 30.0: 2845, 35.0: 2631, 40.0: 2417, 45.0: 2130, 50.0: 1863, 55.0: 1564, 60.0: 1439, 65.0: 1326, 70.0: 1220, 75.0: 1142, 80.0: 1056, 85.0: 986} [(-70.0, 852), (-55.0, 853), (-60.0, 855), (-80.0, 858), (-65.0, 864), (-75.0, 866), (-85.0, 871), (-90.0, 906), (-50.0, 961), (85.0, 986), (80.0, 1056), (75.0, 1142), (-45.0, 1180), (70.0, 1220), (65.0, 1326), (-40.0, 1412), (60.0, 1439), (55.0, 1564), (-35.0, 1690), (50.0, 1863), (-30.0, 1959), (45.0, 2130), (-25.0, 2243), (40.0, 2417), (-20.0, 2484), (35.0, 2631), (-15.0, 2677), (-10.0, 2820), (30.0, 2845), (25.0, 2954), (-5.0, 2976), (0.0, 3039), (5.0, 3049), (20.0, 3061), (10.0, 3098), (15.0, 3122)] arg_min reach at : -70.0 with value = 852 arg_min : -70.0 min_score : 852{12.75: 4966, 14.34375: 4695, 15.9375: 4364, 17.53125: 3945, 19.125: 3465, 20.71875: 2958, 22.3125: 2415, 23.90625: 1841, 25.5: 1295, 27.09375: 936, 28.6875: 847, 30.28125: 852, 31.875: 883, 33.46875: 997, 35.0625: 1322, 36.65625: 1859} [(28.6875, 847), (30.28125, 852), (31.875, 883), (27.09375, 936), (33.46875, 997), (25.5, 1295), (35.0625, 1322), (23.90625, 1841), (36.65625, 1859), (22.3125, 2415), (20.71875, 2958), (19.125, 3465), (17.53125, 3945), (15.9375, 4364), (14.34375, 4695), (12.75, 4966)] arg_min reach at : 28.6875 with value = 847 arg_min : 28.6875 min_score : 847{13.5: 1489, 15.1875: 1374, 16.875: 1244, 18.5625: 1126, 20.25: 996, 21.9375: 893, 23.625: 847, 25.3125: 933, 27.0: 1153, 28.6875: 1446, 30.375: 1827, 32.0625: 2217, 33.75: 2658, 35.4375: 3137, 37.125: 3653, 38.8125: 4142} [(23.625, 847), (21.9375, 893), (25.3125, 933), (20.25, 996), (18.5625, 1126), (27.0, 1153), (16.875, 1244), (15.1875, 1374), (28.6875, 1446), (13.5, 1489), (30.375, 1827), (32.0625, 2217), (33.75, 2658), (35.4375, 3137), (37.125, 3653), (38.8125, 4142)] arg_min reach at : 23.625 with value = 847 arg_min : 23.625 min_score : 847{13.5: 1870, 15.1875: 1783, 16.875: 1676, 18.5625: 1564, 20.25: 1442, 21.9375: 1302, 23.625: 1153, 25.3125: 997, 27.0: 847, 28.6875: 864, 30.375: 1163, 32.0625: 1613, 33.75: 2300, 35.4375: 3191, 37.125: 4251, 38.8125: 5356} [(27.0, 847), (28.6875, 864), (25.3125, 997), (23.625, 1153), (30.375, 1163), (21.9375, 1302), (20.25, 1442), (18.5625, 1564), (32.0625, 1613), (16.875, 1676), (15.1875, 1783), (13.5, 1870), (33.75, 2300), (35.4375, 3191), (37.125, 4251), (38.8125, 5356)] arg_min reach at : 27.0 with value = 847 arg_min : 27.0 min_score : 847 yc : 23.625 xc : 28.6875 angle : -70.0 radius : 27.0 excentricity : 1.6500000000000001 yc : 23.625 xc : 28.6875 angle : -70.0 radius : 27.0 excentricity : 1.6500000000000001 x0 : 103 y1 : 396 width : 35, height : 38, area : 1330, score : 1.0 x0 : 93 y1 : 396 width : 51, height : 36, area : 1836, score : 1.0 Now saving polygons points : 1| batch 1 Loaded 3 chid ids of type : 520 ++ CHI and polygons saved ! (57, 52) (52, 43) [57, 52] (57, 52) score : 7970 strategy_opt : 5| strategy_opt : [('excentricity', [0.5, 2.0, 0.05]), ('angle', [-90.0, 90.0, 5.0]), ('xc', [10.75, 32.25, 1.34375]), ('yc', [13.0, 39.0, 1.625]), ('radius', [13.0, 39.0, 1.625])] {0.5: 16167, 0.55: 14430, 0.6000000000000001: 12864, 0.65: 11529, 0.7000000000000001: 10346, 0.75: 9256, 0.8: 8418, 0.8500000000000001: 7609, 0.9: 6862, 0.9500000000000001: 6277, 1.0: 5482, 1.05: 4840, 1.1: 4264, 1.1500000000000001: 3773, 1.2000000000000002: 3275, 1.25: 2948, 1.3: 2727, 1.35: 2514, 1.4000000000000001: 2254, 1.4500000000000002: 2155, 1.5: 1986, 1.55: 1892, 1.6: 1846, 1.6500000000000001: 1782, 1.7000000000000002: 1717, 1.75: 1656, 1.8: 1641, 1.85: 1602, 1.9000000000000001: 1587, 1.9500000000000002: 1576} [(1.9500000000000002, 1576), (1.9000000000000001, 1587), (1.85, 1602), (1.8, 1641), (1.75, 1656), (1.7000000000000002, 1717), (1.6500000000000001, 1782), (1.6, 1846), (1.55, 1892), (1.5, 1986), (1.4500000000000002, 2155), (1.4000000000000001, 2254), (1.35, 2514), (1.3, 2727), (1.25, 2948), (1.2000000000000002, 3275), (1.1500000000000001, 3773), (1.1, 4264), (1.05, 4840), (1.0, 5482), (0.9500000000000001, 6277), (0.9, 6862), (0.8500000000000001, 7609), (0.8, 8418), (0.75, 9256), (0.7000000000000001, 10346), (0.65, 11529), (0.6000000000000001, 12864), (0.55, 14430), (0.5, 16167)] arg_min reach at : 1.9500000000000002 with value = 1576 | arg_min : 1.9500000000000002 min_score : 1576{-90.0: 2291, -85.0: 2369, -80.0: 2390, -75.0: 2401, -70.0: 2341, -65.0: 2390, -60.0: 2418, -55.0: 2445, -50.0: 2599, -45.0: 2701, -40.0: 2755, -35.0: 2787, -30.0: 2753, -25.0: 2678, -20.0: 2611, -15.0: 2478, -10.0: 2313, -5.0: 2098, 0.0: 1870, 5.0: 1702, 10.0: 1576, 15.0: 1422, 20.0: 1258, 25.0: 1061, 30.0: 872, 35.0: 752, 40.0: 632, 45.0: 677, 50.0: 872, 55.0: 1070, 60.0: 1274, 65.0: 1477, 70.0: 1637, 75.0: 1895, 80.0: 2049, 85.0: 2193} [(40.0, 632), (45.0, 677), (35.0, 752), (30.0, 872), (50.0, 872), (25.0, 1061), (55.0, 1070), (20.0, 1258), (60.0, 1274), (15.0, 1422), (65.0, 1477), (10.0, 1576), (70.0, 1637), (5.0, 1702), (0.0, 1870), (75.0, 1895), (80.0, 2049), (-5.0, 2098), (85.0, 2193), (-90.0, 2291), (-10.0, 2313), (-70.0, 2341), (-85.0, 2369), (-80.0, 2390), (-65.0, 2390), (-75.0, 2401), (-60.0, 2418), (-55.0, 2445), (-15.0, 2478), (-50.0, 2599), (-20.0, 2611), (-25.0, 2678), (-45.0, 2701), (-30.0, 2753), (-40.0, 2755), (-35.0, 2787)] arg_min reach at : 40.0 with value = 632 | arg_min : 40.0 min_score : 632{10.75: 982, 12.09375: 841, 13.4375: 751, 14.78125: 672, 16.125: 624, 17.46875: 599, 18.8125: 575, 20.15625: 561, 21.5: 632, 22.84375: 895, 24.1875: 1257, 25.53125: 1708, 26.875: 2192, 28.21875: 2686, 29.5625: 3202, 30.90625: 3701} [(20.15625, 561), (18.8125, 575), (17.46875, 599), (16.125, 624), (21.5, 632), (14.78125, 672), (13.4375, 751), (12.09375, 841), (22.84375, 895), (10.75, 982), (24.1875, 1257), (25.53125, 1708), (26.875, 2192), (28.21875, 2686), (29.5625, 3202), (30.90625, 3701)] arg_min reach at : 20.15625 with value = 561 | arg_min : 20.15625 min_score : 561{13.0: 3254, 14.625: 2871, 16.25: 2436, 17.875: 1977, 19.5: 1474, 21.125: 1071, 22.75: 835, 24.375: 629, 26.0: 561, 27.625: 716, 29.25: 937, 30.875: 1286, 32.5: 1738, 34.125: 2226, 35.75: 2782, 37.375: 3291} [(26.0, 561), (24.375, 629), (27.625, 716), (22.75, 835), (29.25, 937), (21.125, 1071), (30.875, 1286), (19.5, 1474), (32.5, 1738), (17.875, 1977), (34.125, 2226), (16.25, 2436), (35.75, 2782), (14.625, 2871), (13.0, 3254), (37.375, 3291)] arg_min reach at : 26.0 with value = 561 | arg_min : 26.0 min_score : 561{13.0: 1371, 14.625: 1301, 16.25: 1217, 17.875: 1127, 19.5: 1028, 21.125: 926, 22.75: 811, 24.375: 683, 26.0: 561, 27.625: 624, 29.25: 1026, 30.875: 1508, 32.5: 2147, 34.125: 2981, 35.75: 3949, 37.375: 5069} [(26.0, 561), (27.625, 624), (24.375, 683), (22.75, 811), (21.125, 926), (29.25, 1026), (19.5, 1028), (17.875, 1127), (16.25, 1217), (14.625, 1301), (13.0, 1371), (30.875, 1508), (32.5, 2147), (34.125, 2981), (35.75, 3949), (37.375, 5069)] arg_min reach at : 26.0 with value = 561 | arg_min : 26.0 min_score : 561{0.5: 13327, 0.55: 12420, 0.6000000000000001: 11540, 0.65: 10560, 0.7000000000000001: 9652, 0.75: 8782, 0.8: 7834, 0.8500000000000001: 7061, 0.9: 6294, 0.9500000000000001: 5633, 1.0: 4916, 1.05: 4424, 1.1: 3865, 1.1500000000000001: 3323, 1.2000000000000002: 2817, 1.25: 2401, 1.3: 1935, 1.35: 1600, 1.4000000000000001: 1218, 1.4500000000000002: 1019, 1.5: 834, 1.55: 693, 1.6: 614, 1.6500000000000001: 568, 1.7000000000000002: 549, 1.75: 529, 1.8: 520, 1.85: 521, 1.9000000000000001: 534, 1.9500000000000002: 561} [(1.8, 520), (1.85, 521), (1.75, 529), (1.9000000000000001, 534), (1.7000000000000002, 549), (1.9500000000000002, 561), (1.6500000000000001, 568), (1.6, 614), (1.55, 693), (1.5, 834), (1.4500000000000002, 1019), (1.4000000000000001, 1218), (1.35, 1600), (1.3, 1935), (1.25, 2401), (1.2000000000000002, 2817), (1.1500000000000001, 3323), (1.1, 3865), (1.05, 4424), (1.0, 4916), (0.9500000000000001, 5633), (0.9, 6294), (0.8500000000000001, 7061), (0.8, 7834), (0.75, 8782), (0.7000000000000001, 9652), (0.65, 10560), (0.6000000000000001, 11540), (0.55, 12420), (0.5, 13327)] arg_min reach at : 1.8 with value = 520 arg_min : 1.8 min_score : 520{-90.0: 2114, -85.0: 2268, -80.0: 2343, -75.0: 2405, -70.0: 2409, -65.0: 2435, -60.0: 2467, -55.0: 2514, -50.0: 2590, -45.0: 2654, -40.0: 2632, -35.0: 2610, -30.0: 2565, -25.0: 2526, -20.0: 2435, -15.0: 2330, -10.0: 2181, -5.0: 2013, 0.0: 1792, 5.0: 1584, 10.0: 1389, 15.0: 1208, 20.0: 1038, 25.0: 843, 30.0: 673, 35.0: 564, 40.0: 520, 45.0: 597, 50.0: 764, 55.0: 963, 60.0: 1158, 65.0: 1390, 70.0: 1573, 75.0: 1800, 80.0: 1969, 85.0: 2070} [(40.0, 520), (35.0, 564), (45.0, 597), (30.0, 673), (50.0, 764), (25.0, 843), (55.0, 963), (20.0, 1038), (60.0, 1158), (15.0, 1208), (10.0, 1389), (65.0, 1390), (70.0, 1573), (5.0, 1584), (0.0, 1792), (75.0, 1800), (80.0, 1969), (-5.0, 2013), (85.0, 2070), (-90.0, 2114), (-10.0, 2181), (-85.0, 2268), (-15.0, 2330), (-80.0, 2343), (-75.0, 2405), (-70.0, 2409), (-65.0, 2435), (-20.0, 2435), (-60.0, 2467), (-55.0, 2514), (-25.0, 2526), (-30.0, 2565), (-50.0, 2590), (-35.0, 2610), (-40.0, 2632), (-45.0, 2654)] arg_min reach at : 40.0 with value = 520 arg_min : 40.0 min_score : 520{10.75: 1004, 12.09375: 902, 13.4375: 775, 14.78125: 655, 16.125: 580, 17.46875: 515, 18.8125: 494, 20.15625: 520, 21.5: 692, 22.84375: 1028, 24.1875: 1466, 25.53125: 2001, 26.875: 2490, 28.21875: 3027, 29.5625: 3629, 30.90625: 4163} [(18.8125, 494), (17.46875, 515), (20.15625, 520), (16.125, 580), (14.78125, 655), (21.5, 692), (13.4375, 775), (12.09375, 902), (10.75, 1004), (22.84375, 1028), (24.1875, 1466), (25.53125, 2001), (26.875, 2490), (28.21875, 3027), (29.5625, 3629), (30.90625, 4163)] arg_min reach at : 18.8125 with value = 494 arg_min : 18.8125 min_score : 494{13.0: 3121, 14.625: 2763, 16.25: 2397, 17.875: 1962, 19.5: 1409, 21.125: 1056, 22.75: 809, 24.375: 589, 26.0: 494, 27.625: 635, 29.25: 987, 30.875: 1406, 32.5: 1937, 34.125: 2442, 35.75: 2987, 37.375: 3537} [(26.0, 494), (24.375, 589), (27.625, 635), (22.75, 809), (29.25, 987), (21.125, 1056), (30.875, 1406), (19.5, 1409), (32.5, 1937), (17.875, 1962), (16.25, 2397), (34.125, 2442), (14.625, 2763), (35.75, 2987), (13.0, 3121), (37.375, 3537)] arg_min reach at : 26.0 with value = 494 arg_min : 26.0 min_score : 494{13.0: 1347, 14.625: 1266, 16.25: 1182, 17.875: 1086, 19.5: 979, 21.125: 865, 22.75: 737, 24.375: 616, 26.0: 494, 27.625: 559, 29.25: 954, 30.875: 1524, 32.5: 2347, 34.125: 3350, 35.75: 4468, 37.375: 5635} [(26.0, 494), (27.625, 559), (24.375, 616), (22.75, 737), (21.125, 865), (29.25, 954), (19.5, 979), (17.875, 1086), (16.25, 1182), (14.625, 1266), (13.0, 1347), (30.875, 1524), (32.5, 2347), (34.125, 3350), (35.75, 4468), (37.375, 5635)] arg_min reach at : 26.0 with value = 494 arg_min : 26.0 min_score : 494 yc : 26.0 xc : 18.8125 angle : 40.0 radius : 26.0 excentricity : 1.8 yc : 26.0 xc : 18.8125 angle : 40.0 radius : 26.0 excentricity : 1.8 x0 : 104 y1 : 292 width : 27, height : 36, area : 972, score : 1.0 x0 : 102 y1 : 288 width : 39, height : 43, area : 1677, score : 1.0 Now saving polygons points : 1| batch 1 Loaded 4 chid ids of type : 520 +++ CHI and polygons saved ! ['temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_bib_crop_8165075_0_ellipsebest.jpg', 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_bib_crop_8165075_0_varroa_with_ellipsebest.jpg', 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_bib_crop_8165076_0_ellipsebest.jpg', 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_bib_crop_8165076_0_varroa_with_ellipsebest.jpg', 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_bib_crop_8165077_0_ellipsebest.jpg', 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_bib_crop_8165077_0_varroa_with_ellipsebest.jpg', 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_bib_crop_8165078_0_ellipsebest.jpg', 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_bib_crop_8165078_0_varroa_with_ellipsebest.jpg'] About to upload 8 photos upload in portfolio : 27193561 Result OK ! uploaded one batch 0 Elapsed time : 23.828872203826904 After datou_step_exec type output : time spend for datou_step_exec : 30.177649974822998 time spend to save output : 1.7881393432617188e-05 total time spend for step 1 : 30.17766785621643 step2:tile Wed Sep 24 17:49:53 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 : ['temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67.jpg'] We expect there is only one output and this part is used while all output are not tuple or array We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure input_args_next_step, len :1, first value : [('temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67.jpg',)] After prepare type args : Here we display some param of map_info ! map_filenames : {'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67.jpg': 937852786, 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_bib_crop_8165075_0.jpg': 1385928380, 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_bib_crop_8165076_0.jpg': 1385928381, 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_bib_crop_8165077_0.jpg': 1385928382, 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_bib_crop_8165078_0.jpg': 1385928383} map_photo_id_path_extension : {937852786: {'path': 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67.jpg', 'extension': 'jpg'}, 1385928380: {'path': 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_bib_crop_8165075_0.jpg'}, 1385928381: {'path': 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_bib_crop_8165076_0.jpg'}, 1385928382: {'path': 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_bib_crop_8165077_0.jpg'}, 1385928383: {'path': 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_bib_crop_8165078_0.jpg'}} map_subphoto_mainphoto : {1385928380: 937852786, 1385928381: 937852786, 1385928382: 937852786, 1385928383: 937852786} verbose : True param_json : {'photo_tile_type': 17, 'whiten': True, 'remove_crop_border': True, 'minimal_size_crop_border': 900, 'stride': 240, 'crop_hashtag_type_tiled': 521, 'ETA': 86400, 'new_width': 480, 'new_height': 480, 'token': '78d09a0790ec6ecbf119343125a81fdc', 'portfolio_name': 'tile_taggage_varroa', 'crop_hashtag_type': 520, 'host': 'www.fotonower.com', 'arg_aux_upload': {'type_upload': 'python'}} type(crop_hashtag_type) : type(crop_hashtag_type_tiled) : We consider crop_hashtag_type is an integer ! map_chi_type_to_chi_type_cropped : {520: 521} TO DEPRECATE VR 14-6-18 map_filenames : {937852786: 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67.jpg'} list_pids : 1 list_pids : 2 list_subpids to replace list_pids : 0 batch 1 select photo_id, hashtag_id, `type`, x0, x1, y0, y1, score, id from MTRPhoto.crop_hashtag_ids where photo_id in ( 937852786,937852786) and `type` in (520) Loaded 4 chid ids of type : 520 SELECT crop_hashtag_id, points FROM MTRPhoto.crop_polygon_points WHERE crop_hashtag_id in (3973481199,3973481200,3973481201,3973481202) ++++SELECT * FROM MTRPhoto.crop_segments WHERE crop_hashtag_id in (3973481199,3973481200,3973481201,3973481202) SELECT * FROM MTRPhoto.crop_sum_segments WHERE crop_hashtag_id in (3973481199,3973481200,3973481201,3973481202) created feed_id_new_photos : 27193593 with name tile_taggage_varroa feed_id_new_photos : 27193593 filename : temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67.jpg photo_id : 937852786 height_image_input : 480 width_image_input : 480 new_width : 480 new_height : 480 stride : 240 stride_relative : 0.1 chi to copy from the main photo to the tiled photo input_chi_for_this_image_as_chi : 4 list_bib_to_crops : 1 [(0, 480, 0, 480, 0)] calcul des nouveaux crops pour le tile x0:0,x1:480,y0:0,y1:480 calcul avec la methode originale calcul avec la methode originale calcul avec la methode originale calcul avec la methode originale chi selectionnes : [, , , ] new_crops_tiles : 1 crop_transformed : 4 insert ignore into MTRPhoto.crop_hashtag_ids (photo_id, hashtag_id, `type`,x0,x1,y0,y1,score) VALUES (%s,%s,%s,%s,%s,%s,%s,%s) [(937852786, 2090988864, 17, 0, 480, 0, 480, 1.0)] list_photo_ids_cropped : [937852786] batch 1 select photo_id, hashtag_id, `type`, x0, x1, y0, y1, score, id from MTRPhoto.crop_hashtag_ids where photo_id in ( 937852786) and `type` in (17) Loaded 1 chid ids of type : 17 SELECT crop_hashtag_id, points FROM MTRPhoto.crop_polygon_points WHERE crop_hashtag_id in (8165084) SELECT * FROM MTRPhoto.crop_segments WHERE crop_hashtag_id in (8165084) SELECT * FROM MTRPhoto.crop_sum_segments WHERE crop_hashtag_id in (8165084) treat the image : temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67.jpg , 0 before upload mediasElapsed time : 0.008472919464111328 on upload les photos avec python init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1758729001_1864669 batch_size : 0, verbose : True, strat_bulk_insert : ignore_different_from_first Unexecpected behavior in 07/2025 that can be generalized l287 : type_extension .jpg INSERT IGNORE INTO MTRUser.mtr_fotonower_photos_opt (`height`, `width`, `size_object`, `photo_id`, `base_url`, `date`, `chi_id`, `type_extension`, `hash`) VALUES (480, 480, 50498, 1385928409, 'https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com', '2025/9/24', 60595147, 'jpg', UNHEX('345879d5145933efcce16bd1d96b81a8')) ON DUPLICATE KEY UPDATE `height`=values(`height`), `width`=values(`width`), `size_object`=values(`size_object`), `photo_id`=values(`photo_id`), `base_url`=values(`base_url`), `date`=values(`date`), `chi_id`=values(`chi_id`), `type_extension`=values(`type_extension`), `hash`=values(`hash`) This is a hack ! INSERT ignore into MTRUser.mtr_portfolio_photos (`mtr_portfolio_id`, `mtr_photo_id`, `mtr_user_id`, `created_at`) VALUES (27193593, 1385928409, 0, NOW()) 1 we have uploaded 1 photos in the portfolio 27193593 Importing ! upload mediasElapsed time : 0.6507360935211182 , 0insert ignore into MTRPhoto.crop_sub_photo_ids (crop_hashtag_id, sub_photo_id, mtr_user_id) VALUES (%s,%s,%s) : [(8165084, 1385928409, 0)] Saving 4 CHIs. list_chi_tile : [": {'photo_id': 1385928409, 'hashtag_id': 2087736828, 'type': 521, 'x0': 432, 'x1': 467, 'y0': 303, 'y1': 355, 'score': 1.0, 'id': 0, 'points': ['463,352,452,353,439,350,426,342,418,333,417,325,422,319,433,318,446,321,459,328,467,338,469,346', '463,352,452,353,439,350,426,342,418,333,417,325,422,319,433,318,446,321,459,328,467,338,469,346'], 'sub_photo_id': 0, 'rles': [], 'hashtag': '', 'sum_segment': 0}", ": {'photo_id': 1385928409, 'hashtag_id': 2087736828, 'type': 521, 'x0': 419, 'x1': 445, 'y0': 434, 'y1': 480, 'score': 1.0, 'id': 0, 'points': ['451,461,449,467,441,473,429,477,416,477,406,473,402,467,403,461,411,455,423,451,435,452,446,455', '451,461,449,467,441,473,429,477,416,477,406,473,402,467,403,461,411,455,423,451,435,452,446,455'], 'sub_photo_id': 0, 'rles': [], 'hashtag': '', 'sum_segment': 0}", ": {'photo_id': 1385928409, 'hashtag_id': 2087736828, 'type': 521, 'x0': 93, 'x1': 144, 'y0': 360, 'y1': 396, 'score': 1.0, 'id': 0, 'points': ['112,359,120,350,130,348,137,351,141,361,140,374,134,387,126,396,117,399,109,395,105,385,106,372', '112,359,120,350,130,348,137,351,141,361,140,374,134,387,126,396,117,399,109,395,105,385,106,372'], 'sub_photo_id': 0, 'rles': [], 'hashtag': '', 'sum_segment': 0}", ": {'photo_id': 1385928409, 'hashtag_id': 2087736828, 'type': 521, 'x0': 102, 'x1': 141, 'y0': 245, 'y1': 288, 'score': 1.0, 'id': 0, 'points': ['124,293,112,290,102,281,95,271,93,262,96,255,105,254,116,257,127,266,134,276,136,285,132,292', '124,293,112,290,102,281,95,271,93,262,96,255,105,254,116,257,127,266,134,276,136,285,132,292'], 'sub_photo_id': 0, 'rles': [], 'hashtag': '', 'sum_segment': 0}"] insert ignore into MTRPhoto.crop_hashtag_ids (photo_id, hashtag_id, `type`,x0,x1,y0,y1,score) VALUES (%s,%s,%s,%s,%s,%s,%s,%s) batch 1 Loaded 4 chid ids of type : 521 INSERT IGNORE INTO MTRPhoto.crop_polygon_points (`crop_hashtag_id`, `points`) VALUES (%s, %s) Number RLEs to save : 0 INSERT IGNORE INTO MTRPhoto.crop_sum_segments (`crop_hashtag_id`, `sum_segments`) VALUES (%s, %s) TO DO : save crop sub photo not yet done ! end of tileElapsed time : 0.7246801853179932 map_pid_results : {'1385928409': ['temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0.jpg']} After datou_step_exec type output : time spend for datou_step_exec : 8.709296941757202 time spend to save output : 5.698204040527344e-05 total time spend for step 2 : 8.709353923797607 step3:rotate Wed Sep 24 17:50:01 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 : {'1385928409': ['temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0.jpg']} input_args_next_step : {'1385928409': ()} output_args : {'1385928409': ['temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0.jpg']} args : 1385928409 depend.output_id : 0 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure input_args_next_step, len :1, first value : ('temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0.jpg',) After prepare type args : Here we display some param of map_info ! map_filenames : {'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67.jpg': 937852786, 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_bib_crop_8165075_0.jpg': 1385928380, 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_bib_crop_8165076_0.jpg': 1385928381, 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_bib_crop_8165077_0.jpg': 1385928382, 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_bib_crop_8165078_0.jpg': 1385928383, 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0.jpg': 1385928409} map_photo_id_path_extension : {937852786: {'path': 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67.jpg', 'extension': 'jpg'}, 1385928380: {'path': 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_bib_crop_8165075_0.jpg'}, 1385928381: {'path': 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_bib_crop_8165076_0.jpg'}, 1385928382: {'path': 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_bib_crop_8165077_0.jpg'}, 1385928383: {'path': 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_bib_crop_8165078_0.jpg'}, 1385928409: {'path': 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0.jpg'}} map_subphoto_mainphoto : {1385928380: 937852786, 1385928381: 937852786, 1385928382: 937852786, 1385928383: 937852786, 1385928409: 937852786} Beginning of datou_step_rotate ! Warning, new_feed_id is empty ! We are in a datou with depends ! rotate photos of 0,15,30,45,60,75,90,105,120,135,150,165,180,195,210,225,240,255,270,285,300,315,330,345 degres batch 1 select photo_id, hashtag_id, `type`, x0, x1, y0, y1, score, id from MTRPhoto.crop_hashtag_ids where photo_id in ( 1385928409) and `type` in (521) Loaded 4 chid ids of type : 521 SELECT crop_hashtag_id, points FROM MTRPhoto.crop_polygon_points WHERE crop_hashtag_id in (3973481277,3973481278,3973481276,3973481275) ++WARNING : duplicated polygon, we should remove this data for chi_id : 3973481275. Ignored now ++WARNING : duplicated polygon, we should remove this data for chi_id : 3973481276. Ignored now ++WARNING : duplicated polygon, we should remove this data for chi_id : 3973481277. Ignored now ++WARNING : duplicated polygon, we should remove this data for chi_id : 3973481278. Ignored now SELECT * FROM MTRPhoto.crop_segments WHERE crop_hashtag_id in (3973481277,3973481278,3973481276,3973481275) SELECT * FROM MTRPhoto.crop_sum_segments WHERE crop_hashtag_id in (3973481277,3973481278,3973481276,3973481275) map_chi : {1385928409: [, , , ]} feed_id_new_photos : 27193598 photo_id in download_rotate_and_save : 1385928409 list_chi_loc : 4 Use all angle ! Rotation of photo 1385928409 of 0 degree temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0.jpg [, , , ] 0 remove_crop_border : True version de PIL : 9.5.0 Needs to change image size ! [[ 1. 0.] [-0. 1.]] 0 [[ 1. 0.] [-0. 1.]] shrink_image : True len(list_crops) : 4 time for calcul the mask position with numpy : 0.0006346702575683594 nb_pixel_total : 1389 time to create 1 rle with old method : 0.0029222965240478516 .time for calcul the mask position with numpy : 0.00036215782165527344 nb_pixel_total : 1157 time to create 1 rle with old method : 0.0013985633850097656 . crop are not in the shrunk photo ! crop are not in the shrunk photo ! len(list_crops_rotate) : 2 list_crops_rotate : : {'photo_id': -1, 'hashtag_id': 2087736828, 'type': 529, 'x0': 25, 'x1': 61, 'y0': 268, 'y1': 319, 'score': 1.0, 'id': None, 'points': ['32,279,40,270,50,268,57,271,61,281,60,294,54,307,46,316,37,319,29,315,25,305,26,292'], 'sub_photo_id': 0, 'rles': [(-1, 48, 268, 4), (-1, 43, 269, 11), (-1, 40, 270, 16), (-1, 39, 271, 19), (-1, 38, 272, 20), (-1, 37, 273, 22), (-1, 36, 274, 23), (-1, 36, 275, 24), (-1, 35, 276, 25), (-1, 34, 277, 26), (-1, 33, 278, 28), (-1, 32, 279, 29), (-1, 32, 280, 30), (-1, 31, 281, 31), (-1, 31, 282, 31), (-1, 30, 283, 32), (-1, 30, 284, 32), (-1, 29, 285, 33), (-1, 29, 286, 33), (-1, 28, 287, 34), (-1, 28, 288, 33), (-1, 27, 289, 34), (-1, 27, 290, 34), (-1, 26, 291, 35), (-1, 26, 292, 35), (-1, 26, 293, 35), (-1, 26, 294, 35), (-1, 26, 295, 35), (-1, 26, 296, 34), (-1, 26, 297, 34), (-1, 26, 298, 33), (-1, 25, 299, 34), (-1, 25, 300, 33), (-1, 25, 301, 33), (-1, 25, 302, 32), (-1, 25, 303, 32), (-1, 25, 304, 31), (-1, 25, 305, 31), (-1, 25, 306, 30), (-1, 26, 307, 29), (-1, 26, 308, 28), (-1, 27, 309, 26), (-1, 27, 310, 25), (-1, 27, 311, 24), (-1, 28, 312, 23), (-1, 28, 313, 22), (-1, 29, 314, 20), (-1, 29, 315, 19), (-1, 31, 316, 16), (-1, 33, 317, 12), (-1, 35, 318, 7), (-1, 37, 319, 2)], 'hashtag': '', 'sum_segment': 0},: {'photo_id': -1, 'hashtag_id': 2087736828, 'type': 529, 'x0': 13, 'x1': 56, 'y0': 174, 'y1': 213, 'score': 1.0, 'id': None, 'points': ['44,213,32,210,22,201,15,191,13,182,16,175,25,174,36,177,47,186,54,196,56,205,52,212'], 'sub_photo_id': 0, 'rles': [(-1, 21, 174, 6), (-1, 16, 175, 15), (-1, 16, 176, 19), (-1, 15, 177, 22), (-1, 15, 178, 23), (-1, 14, 179, 26), (-1, 14, 180, 27), (-1, 13, 181, 29), (-1, 13, 182, 30), (-1, 13, 183, 31), (-1, 13, 184, 33), (-1, 14, 185, 33), (-1, 14, 186, 34), (-1, 14, 187, 35), (-1, 14, 188, 35), (-1, 15, 189, 35), (-1, 15, 190, 36), (-1, 15, 191, 36), (-1, 16, 192, 36), (-1, 16, 193, 37), (-1, 17, 194, 37), (-1, 18, 195, 36), (-1, 18, 196, 37), (-1, 19, 197, 36), (-1, 20, 198, 35), (-1, 21, 199, 35), (-1, 21, 200, 35), (-1, 22, 201, 34), (-1, 23, 202, 33), (-1, 24, 203, 33), (-1, 25, 204, 32), (-1, 26, 205, 31), (-1, 28, 206, 28), (-1, 29, 207, 27), (-1, 30, 208, 25), (-1, 31, 209, 24), (-1, 32, 210, 22), (-1, 35, 211, 19), (-1, 39, 212, 14), (-1, 43, 213, 6)], 'hashtag': '', 'sum_segment': 0} Rotation of photo 1385928409 of 15 degree temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0.jpg [, , , ] 15 remove_crop_border : True version de PIL : 9.5.0 Needs to change image size ! [[ 0.96592583 0.25881905] [-0.25881905 0.96592583]] 15 [[ 0.96592583 0.25881905] [-0.25881905 0.96592583]] shrink_image : True len(list_crops) : 4 time for calcul the mask position with numpy : 0.00045943260192871094 nb_pixel_total : 694 time to create 1 rle with old method : 0.0014264583587646484 On the border Smaller than minimal size ! time for calcul the mask position with numpy : 0.0004258155822753906 nb_pixel_total : 1162 time to create 1 rle with old method : 0.002381563186645508 . crop are not in the shrunk photo ! crop are not in the shrunk photo ! len(list_crops_rotate) : 1 list_crops_rotate : : {'photo_id': -2, 'hashtag_id': 2087736828, 'type': 529, 'x0': 24, 'x1': 72, 'y0': 209, 'y1': 242, 'score': 1.0, 'id': None, 'points': ['62,241,50,242,38,236,28,228,24,220,25,212,34,209,45,209,58,215,67,222,72,231,70,238'], 'sub_photo_id': 0, 'rles': [(-1, 33, 209, 3), (-1, 37, 209, 9), (-1, 30, 210, 18), (-1, 49, 210, 1), (-1, 30, 211, 21), (-1, 26, 212, 27), (-1, 26, 213, 29), (-1, 26, 214, 32), (-1, 25, 215, 35), (-1, 25, 216, 36), (-1, 25, 217, 37), (-1, 25, 218, 38), (-1, 24, 219, 40), (-1, 25, 220, 40), (-1, 25, 221, 42), (-1, 25, 222, 43), (-1, 26, 223, 43), (-1, 27, 224, 42), (-1, 27, 225, 43), (-1, 28, 226, 43), (-1, 29, 227, 42), (-1, 29, 228, 42), (-1, 30, 229, 43), (-1, 30, 230, 43), (-1, 32, 231, 41), (-1, 33, 232, 39), (-1, 34, 233, 39), (-1, 36, 234, 36), (-1, 37, 235, 35), (-1, 38, 236, 34), (-1, 40, 237, 32), (-1, 42, 238, 30), (-1, 43, 239, 28), (-1, 46, 240, 22), (-1, 48, 241, 19), (-1, 50, 242, 15)], 'hashtag': '', 'sum_segment': 0} Rotation of photo 1385928409 of 30 degree temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0.jpg [, , , ] 30 remove_crop_border : True version de PIL : 9.5.0 Needs to change image size ! [[ 0.8660254 0.5 ] [-0.5 0.8660254]] 30 [[ 0.8660254 0.5 ] [-0.5 0.8660254]] shrink_image : True len(list_crops) : 4 time for calcul the mask position with numpy : 0.00044035911560058594 nb_pixel_total : 221 time to create 1 rle with old method : 0.0004565715789794922 On the border Smaller than minimal size ! time for calcul the mask position with numpy : 0.0004134178161621094 nb_pixel_total : 1155 time to create 1 rle with old method : 0.0018928050994873047 . crop are not in the shrunk photo ! crop are not in the shrunk photo ! len(list_crops_rotate) : 1 list_crops_rotate : : {'photo_id': -3, 'hashtag_id': 2087736828, 'type': 529, 'x0': 44, 'x1': 93, 'y0': 238, 'y1': 268, 'score': 1.0, 'id': None, 'points': ['86,264,74,268,61,265,50,260,44,253,43,245,50,240,61,237,75,239,87,245,93,251,93,259'], 'sub_photo_id': 0, 'rles': [(-1, 59, 238, 9), (-1, 55, 239, 17), (-1, 73, 239, 1), (-1, 51, 240, 27), (-1, 49, 241, 30), (-1, 48, 242, 34), (-1, 48, 243, 36), (-1, 46, 244, 40), (-1, 45, 245, 43), (-1, 44, 246, 45), (-1, 44, 247, 46), (-1, 44, 248, 47), (-1, 44, 249, 48), (-1, 44, 250, 49), (-1, 44, 251, 50), (-1, 44, 252, 50), (-1, 44, 253, 50), (-1, 45, 254, 49), (-1, 45, 255, 49), (-1, 47, 256, 47), (-1, 48, 257, 46), (-1, 48, 258, 46), (-1, 50, 259, 44), (-1, 51, 260, 43), (-1, 52, 261, 41), (-1, 54, 262, 37), (-1, 56, 263, 35), (-1, 59, 264, 31), (-1, 60, 265, 28), (-1, 63, 266, 21), (-1, 67, 267, 2), (-1, 70, 267, 11), (-1, 74, 268, 3)], 'hashtag': '', 'sum_segment': 0} Rotation of photo 1385928409 of 45 degree temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0.jpg [, , , ] 45 remove_crop_border : True version de PIL : 9.5.0 Needs to change image size ! [[ 0.70710678 0.70710678] [-0.70710678 0.70710678]] 45 [[ 0.70710678 0.70710678] [-0.70710678 0.70710678]] shrink_image : True len(list_crops) : 4 time for calcul the mask position with numpy : 0.0003342628479003906 nb_pixel_total : 143 time to create 1 rle with old method : 0.0002570152282714844 On the border Smaller than minimal size ! time for calcul the mask position with numpy : 0.0003426074981689453 nb_pixel_total : 1161 time to create 1 rle with old method : 0.0014934539794921875 . crop are not in the shrunk photo ! crop are not in the shrunk photo ! len(list_crops_rotate) : 1 list_crops_rotate : : {'photo_id': -4, 'hashtag_id': 2087736828, 'type': 529, 'x0': 69, 'x1': 121, 'y0': 258, 'y1': 286, 'score': 1.0, 'id': None, 'points': ['115,279,105,285,91,286,79,284,72,279,69,272,74,265,84,259,98,258,110,260,118,265,120,273'], 'sub_photo_id': 0, 'rles': [(-1, 96, 258, 5), (-1, 88, 259, 17), (-1, 84, 260, 26), (-1, 111, 260, 1), (-1, 82, 261, 31), (-1, 81, 262, 34), (-1, 78, 263, 38), (-1, 77, 264, 42), (-1, 75, 265, 45), (-1, 74, 266, 46), (-1, 73, 267, 47), (-1, 72, 268, 48), (-1, 72, 269, 49), (-1, 71, 270, 50), (-1, 70, 271, 51), (-1, 69, 272, 53), (-1, 70, 273, 52), (-1, 70, 274, 51), (-1, 70, 275, 50), (-1, 71, 276, 48), (-1, 71, 277, 49), (-1, 72, 278, 47), (-1, 71, 279, 47), (-1, 72, 280, 45), (-1, 73, 281, 41), (-1, 76, 282, 37), (-1, 77, 283, 34), (-1, 79, 284, 31), (-1, 82, 285, 25), (-1, 85, 286, 21)], 'hashtag': '', 'sum_segment': 0} Rotation of photo 1385928409 of 60 degree temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0.jpg [, , , ] 60 remove_crop_border : True version de PIL : 9.5.0 Needs to change image size ! [[ 0.5 0.8660254] [-0.8660254 0.5 ]] 60 [[ 0.5 0.8660254] [-0.8660254 0.5 ]] shrink_image : True len(list_crops) : 4 time for calcul the mask position with numpy : 0.0003304481506347656 nb_pixel_total : 414 time to create 1 rle with old method : 0.0005867481231689453 On the border Smaller than minimal size ! time for calcul the mask position with numpy : 0.0004127025604248047 nb_pixel_total : 1159 time to create 1 rle with old method : 0.0018057823181152344 . crop are not in the shrunk photo ! On the border Smaller than minimal size ! len(list_crops_rotate) : 1 list_crops_rotate : : {'photo_id': -5, 'hashtag_id': 2087736828, 'type': 529, 'x0': 102, 'x1': 152, 'y0': 269, 'y1': 300, 'score': 1.0, 'id': None, 'points': ['148,286,140,295,127,299,115,300,106,297,101,291,105,283,113,275,126,270,139,268,147,271,151,278'], 'sub_photo_id': 0, 'rles': [(-1, 132, 269, 1), (-1, 135, 269, 6), (-1, 126, 270, 18), (-1, 124, 271, 24), (-1, 121, 272, 28), (-1, 118, 273, 32), (-1, 116, 274, 34), (-1, 113, 275, 38), (-1, 112, 276, 39), (-1, 111, 277, 41), (-1, 111, 278, 42), (-1, 109, 279, 44), (-1, 108, 280, 44), (-1, 108, 281, 44), (-1, 106, 282, 45), (-1, 105, 283, 47), (-1, 105, 284, 46), (-1, 104, 285, 46), (-1, 104, 286, 46), (-1, 104, 287, 45), (-1, 104, 288, 44), (-1, 103, 289, 44), (-1, 103, 290, 43), (-1, 102, 291, 43), (-1, 102, 292, 42), (-1, 103, 293, 41), (-1, 104, 294, 38), (-1, 104, 295, 37), (-1, 105, 296, 33), (-1, 106, 297, 28), (-1, 107, 298, 27), (-1, 111, 299, 19), (-1, 112, 300, 1), (-1, 114, 300, 9)], 'hashtag': '', 'sum_segment': 0} Rotation of photo 1385928409 of 75 degree temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0.jpg [, , , ] 75 remove_crop_border : True version de PIL : 9.5.0 Needs to change image size ! [[ 0.25881905 0.96592583] [-0.96592583 0.25881905]] 75 [[ 0.25881905 0.96592583] [-0.96592583 0.25881905]] shrink_image : True len(list_crops) : 4 time for calcul the mask position with numpy : 0.0004374980926513672 nb_pixel_total : 1204 time to create 1 rle with old method : 0.0018575191497802734 On the border Smaller than minimal size ! time for calcul the mask position with numpy : 0.0004131793975830078 nb_pixel_total : 1157 time to create 1 rle with old method : 0.0019178390502929688 . crop are not in the shrunk photo ! time for calcul the mask position with numpy : 0.0003223419189453125 nb_pixel_total : 264 time to create 1 rle with old method : 0.0004410743713378906 On the border Smaller than minimal size ! len(list_crops_rotate) : 1 list_crops_rotate : : {'photo_id': -6, 'hashtag_id': 2087736828, 'type': 529, 'x0': 138, 'x1': 183, 'y0': 271, 'y1': 308, 'score': 1.0, 'id': None, 'points': ['182,285,176,295,164,303,153,307,144,306,138,302,139,293,145,283,156,275,168,270,177,271,183,277'], 'sub_photo_id': 0, 'rles': [(-1, 168, 271, 10), (-1, 165, 272, 14), (-1, 162, 273, 18), (-1, 160, 274, 22), (-1, 157, 275, 25), (-1, 155, 276, 28), (-1, 154, 277, 30), (-1, 153, 278, 31), (-1, 152, 279, 32), (-1, 150, 280, 33), (-1, 148, 281, 36), (-1, 148, 282, 36), (-1, 146, 283, 38), (-1, 145, 284, 38), (-1, 144, 285, 39), (-1, 144, 286, 39), (-1, 143, 287, 39), (-1, 143, 288, 38), (-1, 142, 289, 39), (-1, 141, 290, 40), (-1, 141, 291, 38), (-1, 140, 292, 39), (-1, 140, 293, 39), (-1, 139, 294, 39), (-1, 139, 295, 38), (-1, 139, 296, 38), (-1, 139, 297, 36), (-1, 139, 298, 35), (-1, 139, 299, 33), (-1, 139, 300, 32), (-1, 138, 301, 32), (-1, 138, 302, 30), (-1, 139, 303, 27), (-1, 140, 304, 25), (-1, 142, 305, 19), (-1, 143, 306, 17), (-1, 143, 307, 14), (-1, 150, 308, 1)], 'hashtag': '', 'sum_segment': 0} Rotation of photo 1385928409 of 90 degree temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0.jpg [, , , ] 90 remove_crop_border : True version de PIL : 9.5.0 Needs to change image size ! [[ 6.123234e-17 1.000000e+00] [-1.000000e+00 6.123234e-17]] 90 [[ 6.123234e-17 1.000000e+00] [-1.000000e+00 6.123234e-17]] shrink_image : True len(list_crops) : 4 time for calcul the mask position with numpy : 0.00041794776916503906 nb_pixel_total : 1389 time to create 1 rle with old method : 0.0017724037170410156 .time for calcul the mask position with numpy : 0.0003426074981689453 nb_pixel_total : 1157 time to create 1 rle with old method : 0.0014736652374267578 . crop are not in the shrunk photo ! crop are not in the shrunk photo ! len(list_crops_rotate) : 2 list_crops_rotate : : {'photo_id': -7, 'hashtag_id': 2087736828, 'type': 529, 'x0': 268, 'x1': 319, 'y0': 258, 'y1': 294, 'score': 1.0, 'id': None, 'points': ['279,287,270,279,268,269,271,262,281,258,294,259,307,265,316,273,319,282,315,290,305,294,292,293'], 'sub_photo_id': 0, 'rles': [(-1, 280, 258, 8), (-1, 278, 259, 18), (-1, 275, 260, 23), (-1, 273, 261, 27), (-1, 271, 262, 31), (-1, 271, 263, 33), (-1, 270, 264, 36), (-1, 270, 265, 38), (-1, 269, 266, 40), (-1, 269, 267, 41), (-1, 268, 268, 43), (-1, 268, 269, 45), (-1, 268, 270, 46), (-1, 268, 271, 47), (-1, 269, 272, 47), (-1, 269, 273, 48), (-1, 269, 274, 48), (-1, 269, 275, 49), (-1, 269, 276, 49), (-1, 270, 277, 48), (-1, 270, 278, 49), (-1, 270, 279, 49), (-1, 271, 280, 48), (-1, 272, 281, 48), (-1, 273, 282, 47), (-1, 274, 283, 45), (-1, 276, 284, 43), (-1, 277, 285, 41), (-1, 278, 286, 40), (-1, 279, 287, 38), (-1, 281, 288, 36), (-1, 283, 289, 33), (-1, 285, 290, 31), (-1, 287, 291, 27), (-1, 289, 292, 23), (-1, 291, 293, 18), (-1, 299, 294, 8)], 'hashtag': '', 'sum_segment': 0},: {'photo_id': -7, 'hashtag_id': 2087736828, 'type': 529, 'x0': 174, 'x1': 213, 'y0': 263, 'y1': 306, 'score': 1.0, 'id': None, 'points': ['213,275,210,287,201,297,191,304,182,306,175,303,174,294,177,283,186,272,196,265,205,263,212,267'], 'sub_photo_id': 0, 'rles': [(-1, 203, 263, 3), (-1, 199, 264, 9), (-1, 196, 265, 14), (-1, 194, 266, 18), (-1, 193, 267, 20), (-1, 192, 268, 21), (-1, 190, 269, 23), (-1, 189, 270, 24), (-1, 187, 271, 27), (-1, 186, 272, 28), (-1, 185, 273, 29), (-1, 184, 274, 30), (-1, 184, 275, 30), (-1, 183, 276, 31), (-1, 182, 277, 31), (-1, 181, 278, 32), (-1, 180, 279, 33), (-1, 179, 280, 34), (-1, 179, 281, 33), (-1, 178, 282, 34), (-1, 177, 283, 35), (-1, 177, 284, 35), (-1, 176, 285, 35), (-1, 176, 286, 35), (-1, 176, 287, 35), (-1, 176, 288, 34), (-1, 175, 289, 34), (-1, 175, 290, 33), (-1, 175, 291, 32), (-1, 175, 292, 31), (-1, 174, 293, 32), (-1, 174, 294, 31), (-1, 174, 295, 30), (-1, 174, 296, 29), (-1, 174, 297, 28), (-1, 174, 298, 27), (-1, 175, 299, 24), (-1, 175, 300, 23), (-1, 175, 301, 22), (-1, 175, 302, 20), (-1, 175, 303, 19), (-1, 177, 304, 15), (-1, 179, 305, 10), (-1, 181, 306, 4)], 'hashtag': '', 'sum_segment': 0} Rotation of photo 1385928409 of 105 degree temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0.jpg [, , , ] 105 remove_crop_border : True version de PIL : 9.5.0 Needs to change image size ! [[-0.25881905 0.96592583] [-0.96592583 -0.25881905]] 105 [[-0.25881905 0.96592583] [-0.96592583 -0.25881905]] shrink_image : True len(list_crops) : 4 time for calcul the mask position with numpy : 0.0003523826599121094 nb_pixel_total : 694 time to create 1 rle with old method : 0.0009226799011230469 On the border Smaller than minimal size ! time for calcul the mask position with numpy : 0.0003428459167480469 nb_pixel_total : 1162 time to create 1 rle with old method : 0.0014595985412597656 . crop are not in the shrunk photo ! crop are not in the shrunk photo ! len(list_crops_rotate) : 1 list_crops_rotate : : {'photo_id': -8, 'hashtag_id': 2087736828, 'type': 529, 'x0': 209, 'x1': 242, 'y0': 248, 'y1': 296, 'score': 1.0, 'id': None, 'points': ['241,257,242,269,236,281,228,291,220,295,212,294,209,285,209,274,215,261,222,252,231,247,238,249'], 'sub_photo_id': 0, 'rles': [(-1, 229, 248, 3), (-1, 233, 248, 1), (-1, 229, 249, 10), (-1, 226, 250, 14), (-1, 225, 251, 15), (-1, 223, 252, 17), (-1, 222, 253, 19), (-1, 221, 254, 21), (-1, 221, 255, 21), (-1, 220, 256, 23), (-1, 219, 257, 24), (-1, 218, 258, 25), (-1, 217, 259, 26), (-1, 216, 260, 27), (-1, 215, 261, 28), (-1, 215, 262, 28), (-1, 214, 263, 29), (-1, 214, 264, 29), (-1, 214, 265, 29), (-1, 213, 266, 30), (-1, 213, 267, 30), (-1, 212, 268, 31), (-1, 212, 269, 31), (-1, 211, 270, 32), (-1, 210, 271, 32), (-1, 211, 272, 31), (-1, 210, 273, 31), (-1, 210, 274, 31), (-1, 209, 275, 31), (-1, 209, 276, 31), (-1, 209, 277, 31), (-1, 209, 278, 30), (-1, 209, 279, 29), (-1, 209, 280, 29), (-1, 209, 281, 28), (-1, 209, 282, 28), (-1, 209, 283, 27), (-1, 210, 284, 25), (-1, 209, 285, 25), (-1, 209, 286, 25), (-1, 209, 287, 24), (-1, 210, 288, 22), (-1, 210, 289, 21), (-1, 210, 290, 21), (-1, 212, 291, 17), (-1, 212, 292, 15), (-1, 212, 293, 14), (-1, 212, 294, 12), (-1, 215, 295, 8), (-1, 219, 296, 1)], 'hashtag': '', 'sum_segment': 0} Rotation of photo 1385928409 of 120 degree temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0.jpg [, , , ] 120 remove_crop_border : True version de PIL : 9.5.0 Needs to change image size ! [[-0.5 0.8660254] [-0.8660254 -0.5 ]] 120 [[-0.5 0.8660254] [-0.8660254 -0.5 ]] shrink_image : True len(list_crops) : 4 time for calcul the mask position with numpy : 0.0003337860107421875 nb_pixel_total : 221 time to create 1 rle with old method : 0.0003757476806640625 On the border Smaller than minimal size ! time for calcul the mask position with numpy : 0.000339508056640625 nb_pixel_total : 1155 time to create 1 rle with old method : 0.0014460086822509766 . crop are not in the shrunk photo ! crop are not in the shrunk photo ! len(list_crops_rotate) : 1 list_crops_rotate : : {'photo_id': -9, 'hashtag_id': 2087736828, 'type': 529, 'x0': 238, 'x1': 268, 'y0': 227, 'y1': 276, 'score': 1.0, 'id': None, 'points': ['264,233,268,245,265,258,260,269,253,275,245,276,240,269,237,258,239,244,245,232,251,226,259,226'], 'sub_photo_id': 0, 'rles': [(-1, 251, 227, 10), (-1, 250, 228, 12), (-1, 249, 229, 13), (-1, 248, 230, 16), (-1, 247, 231, 18), (-1, 246, 232, 19), (-1, 245, 233, 21), (-1, 245, 234, 21), (-1, 244, 235, 22), (-1, 244, 236, 22), (-1, 243, 237, 24), (-1, 243, 238, 24), (-1, 242, 239, 25), (-1, 242, 240, 26), (-1, 242, 241, 26), (-1, 241, 242, 27), (-1, 240, 243, 28), (-1, 240, 244, 29), (-1, 240, 245, 29), (-1, 240, 246, 29), (-1, 239, 247, 29), (-1, 240, 248, 28), (-1, 239, 249, 29), (-1, 239, 250, 29), (-1, 239, 251, 28), (-1, 239, 252, 29), (-1, 238, 253, 30), (-1, 238, 254, 29), (-1, 238, 255, 29), (-1, 238, 256, 29), (-1, 238, 257, 29), (-1, 238, 258, 28), (-1, 238, 259, 28), (-1, 238, 260, 28), (-1, 238, 261, 27), (-1, 239, 262, 25), (-1, 239, 263, 25), (-1, 239, 264, 25), (-1, 239, 265, 24), (-1, 240, 266, 23), (-1, 240, 267, 22), (-1, 240, 268, 22), (-1, 240, 269, 21), (-1, 241, 270, 19), (-1, 241, 271, 18), (-1, 242, 272, 17), (-1, 244, 273, 13), (-1, 244, 274, 12), (-1, 245, 275, 11), (-1, 246, 276, 8)], 'hashtag': '', 'sum_segment': 0} Rotation of photo 1385928409 of 135 degree temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0.jpg [, , , ] 135 remove_crop_border : True version de PIL : 9.5.0 Needs to change image size ! [[-0.70710678 0.70710678] [-0.70710678 -0.70710678]] 135 [[-0.70710678 0.70710678] [-0.70710678 -0.70710678]] shrink_image : True len(list_crops) : 4 time for calcul the mask position with numpy : 0.0003783702850341797 nb_pixel_total : 143 time to create 1 rle with old method : 0.00030922889709472656 On the border Smaller than minimal size ! time for calcul the mask position with numpy : 0.0003399848937988281 nb_pixel_total : 1160 time to create 1 rle with old method : 0.0014798641204833984 . crop are not in the shrunk photo ! crop are not in the shrunk photo ! len(list_crops_rotate) : 1 list_crops_rotate : : {'photo_id': -10, 'hashtag_id': 2087736828, 'type': 529, 'x0': 258, 'x1': 286, 'y0': 199, 'y1': 251, 'score': 1.0, 'id': None, 'points': ['279,204,285,214,286,228,284,240,279,247,272,250,265,245,259,235,258,221,260,209,265,201,273,199'], 'sub_photo_id': 0, 'rles': [(-1, 272, 199, 2), (-1, 269, 200, 6), (-1, 265, 201, 1), (-1, 267, 201, 9), (-1, 277, 201, 1), (-1, 264, 202, 15), (-1, 264, 203, 16), (-1, 264, 204, 17), (-1, 263, 205, 18), (-1, 262, 206, 19), (-1, 262, 207, 20), (-1, 261, 208, 22), (-1, 260, 209, 23), (-1, 261, 210, 23), (-1, 260, 211, 25), (-1, 260, 212, 25), (-1, 260, 213, 25), (-1, 260, 214, 26), (-1, 260, 215, 27), (-1, 259, 216, 28), (-1, 259, 217, 28), (-1, 259, 218, 28), (-1, 259, 219, 28), (-1, 258, 220, 29), (-1, 258, 221, 29), (-1, 258, 222, 29), (-1, 258, 223, 29), (-1, 258, 224, 29), (-1, 259, 225, 28), (-1, 259, 226, 28), (-1, 259, 227, 28), (-1, 259, 228, 28), (-1, 259, 229, 28), (-1, 259, 230, 28), (-1, 259, 231, 28), (-1, 259, 232, 28), (-1, 260, 233, 27), (-1, 260, 234, 27), (-1, 260, 235, 27), (-1, 260, 236, 26), (-1, 261, 237, 25), (-1, 261, 238, 25), (-1, 262, 239, 23), (-1, 263, 240, 22), (-1, 263, 241, 22), (-1, 263, 242, 21), (-1, 264, 243, 20), (-1, 265, 244, 18), (-1, 265, 245, 17), (-1, 266, 246, 16), (-1, 267, 247, 15), (-1, 268, 248, 13), (-1, 270, 249, 8), (-1, 279, 249, 1), (-1, 271, 250, 5), (-1, 272, 251, 1)], 'hashtag': '', 'sum_segment': 0} Rotation of photo 1385928409 of 150 degree temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0.jpg [, , , ] 150 remove_crop_border : True version de PIL : 9.5.0 Needs to change image size ! [[-0.8660254 0.5 ] [-0.5 -0.8660254]] 150 [[-0.8660254 0.5 ] [-0.5 -0.8660254]] shrink_image : True len(list_crops) : 4 time for calcul the mask position with numpy : 0.0003314018249511719 nb_pixel_total : 414 time to create 1 rle with old method : 0.0005884170532226562 On the border Smaller than minimal size ! time for calcul the mask position with numpy : 0.0003647804260253906 nb_pixel_total : 1159 time to create 1 rle with old method : 0.0014612674713134766 . crop are not in the shrunk photo ! crop are not in the shrunk photo ! time for calcul the mask position with numpy : 0.0003142356872558594 nb_pixel_total : 1 time to create 1 rle with old method : 1.7404556274414062e-05 len(list_crops_rotate) : 1 list_crops_rotate : : {'photo_id': -11, 'hashtag_id': 2087736828, 'type': 529, 'x0': 269, 'x1': 300, 'y0': 168, 'y1': 218, 'score': 1.0, 'id': None, 'points': ['286,171,295,179,299,192,300,204,297,213,291,218,283,214,275,206,270,193,268,180,271,172,278,168'], 'sub_photo_id': 0, 'rles': [(-1, 278, 168, 2), (-1, 277, 169, 5), (-1, 283, 169, 1), (-1, 275, 170, 10), (-1, 273, 171, 14), (-1, 272, 172, 16), (-1, 271, 173, 18), (-1, 271, 174, 19), (-1, 271, 175, 20), (-1, 271, 176, 21), (-1, 270, 177, 24), (-1, 270, 178, 24), (-1, 270, 179, 25), (-1, 269, 180, 27), (-1, 269, 181, 27), (-1, 269, 182, 27), (-1, 269, 183, 28), (-1, 269, 184, 28), (-1, 269, 185, 28), (-1, 270, 186, 27), (-1, 270, 187, 29), (-1, 269, 188, 30), (-1, 270, 189, 29), (-1, 270, 190, 29), (-1, 270, 191, 30), (-1, 270, 192, 30), (-1, 270, 193, 30), (-1, 270, 194, 30), (-1, 271, 195, 29), (-1, 271, 196, 29), (-1, 272, 197, 28), (-1, 272, 198, 29), (-1, 272, 199, 29), (-1, 273, 200, 28), (-1, 273, 201, 28), (-1, 273, 202, 28), (-1, 274, 203, 27), (-1, 274, 204, 27), (-1, 275, 205, 26), (-1, 275, 206, 26), (-1, 275, 207, 25), (-1, 276, 208, 25), (-1, 277, 209, 23), (-1, 279, 210, 20), (-1, 279, 211, 20), (-1, 280, 212, 19), (-1, 282, 213, 17), (-1, 282, 214, 16), (-1, 283, 215, 14), (-1, 285, 216, 11), (-1, 289, 217, 5), (-1, 291, 218, 2)], 'hashtag': '', 'sum_segment': 0} Rotation of photo 1385928409 of 165 degree temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0.jpg [, , , ] 165 remove_crop_border : True version de PIL : 9.5.0 Needs to change image size ! [[-0.96592583 0.25881905] [-0.25881905 -0.96592583]] 165 [[-0.96592583 0.25881905] [-0.25881905 -0.96592583]] shrink_image : True len(list_crops) : 4 time for calcul the mask position with numpy : 0.0004143714904785156 nb_pixel_total : 1204 time to create 1 rle with old method : 0.0015032291412353516 On the border Smaller than minimal size ! time for calcul the mask position with numpy : 0.00033473968505859375 nb_pixel_total : 1158 time to create 1 rle with old method : 0.0014765262603759766 . crop are not in the shrunk photo ! time for calcul the mask position with numpy : 0.000324249267578125 nb_pixel_total : 264 time to create 1 rle with old method : 0.0004360675811767578 On the border Smaller than minimal size ! len(list_crops_rotate) : 1 list_crops_rotate : : {'photo_id': -12, 'hashtag_id': 2087736828, 'type': 529, 'x0': 271, 'x1': 308, 'y0': 137, 'y1': 182, 'score': 1.0, 'id': None, 'points': ['285,137,295,143,303,155,307,166,306,175,302,181,293,180,283,174,275,163,270,151,271,142,277,136'], 'sub_photo_id': 0, 'rles': [(-1, 276, 137, 4), (-1, 281, 137, 3), (-1, 276, 138, 11), (-1, 274, 139, 14), (-1, 274, 140, 17), (-1, 273, 141, 18), (-1, 272, 142, 22), (-1, 271, 143, 24), (-1, 271, 144, 26), (-1, 271, 145, 26), (-1, 271, 146, 27), (-1, 271, 147, 28), (-1, 271, 148, 28), (-1, 271, 149, 29), (-1, 271, 150, 30), (-1, 271, 151, 31), (-1, 271, 152, 31), (-1, 272, 153, 31), (-1, 272, 154, 31), (-1, 272, 155, 32), (-1, 273, 156, 32), (-1, 273, 157, 32), (-1, 273, 158, 32), (-1, 274, 159, 31), (-1, 274, 160, 32), (-1, 275, 161, 32), (-1, 275, 162, 32), (-1, 275, 163, 32), (-1, 276, 164, 32), (-1, 276, 165, 32), (-1, 277, 166, 31), (-1, 278, 167, 30), (-1, 279, 168, 29), (-1, 280, 169, 28), (-1, 280, 170, 29), (-1, 281, 171, 27), (-1, 281, 172, 27), (-1, 283, 173, 25), (-1, 283, 174, 25), (-1, 284, 175, 24), (-1, 285, 176, 23), (-1, 287, 177, 21), (-1, 289, 178, 17), (-1, 290, 179, 15), (-1, 292, 180, 13), (-1, 294, 181, 10), (-1, 301, 182, 2)], 'hashtag': '', 'sum_segment': 0} Rotation of photo 1385928409 of 180 degree temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0.jpg [, , , ] 180 remove_crop_border : True version de PIL : 9.5.0 Needs to change image size ! [[-1.0000000e+00 1.2246468e-16] [-1.2246468e-16 -1.0000000e+00]] 180 [[-1.0000000e+00 1.2246468e-16] [-1.2246468e-16 -1.0000000e+00]] shrink_image : True len(list_crops) : 4 time for calcul the mask position with numpy : 0.00034308433532714844 nb_pixel_total : 1389 time to create 1 rle with old method : 0.0017523765563964844 .time for calcul the mask position with numpy : 0.00034117698669433594 nb_pixel_total : 1157 time to create 1 rle with old method : 0.0014712810516357422 . crop are not in the shrunk photo ! crop are not in the shrunk photo ! len(list_crops_rotate) : 2 list_crops_rotate : : {'photo_id': -13, 'hashtag_id': 2087736828, 'type': 529, 'x0': 258, 'x1': 294, 'y0': 1, 'y1': 52, 'score': 1.0, 'id': None, 'points': ['287,41,279,50,269,52,262,49,258,39,259,26,265,13,273,4,282,1,290,5,294,15,293,28'], 'sub_photo_id': 0, 'rles': [(-1, 281, 1, 2), (-1, 278, 2, 7), (-1, 275, 3, 12), (-1, 273, 4, 16), (-1, 272, 5, 19), (-1, 271, 6, 20), (-1, 270, 7, 22), (-1, 269, 8, 23), (-1, 269, 9, 24), (-1, 268, 10, 25), (-1, 267, 11, 26), (-1, 266, 12, 28), (-1, 265, 13, 29), (-1, 265, 14, 30), (-1, 264, 15, 31), (-1, 264, 16, 31), (-1, 263, 17, 32), (-1, 263, 18, 32), (-1, 262, 19, 33), (-1, 262, 20, 33), (-1, 261, 21, 34), (-1, 261, 22, 33), (-1, 260, 23, 34), (-1, 260, 24, 34), (-1, 259, 25, 35), (-1, 259, 26, 35), (-1, 259, 27, 35), (-1, 259, 28, 35), (-1, 259, 29, 35), (-1, 259, 30, 34), (-1, 259, 31, 34), (-1, 259, 32, 33), (-1, 258, 33, 34), (-1, 258, 34, 33), (-1, 258, 35, 33), (-1, 258, 36, 32), (-1, 258, 37, 32), (-1, 258, 38, 31), (-1, 258, 39, 31), (-1, 258, 40, 30), (-1, 259, 41, 29), (-1, 259, 42, 28), (-1, 260, 43, 26), (-1, 260, 44, 25), (-1, 260, 45, 24), (-1, 261, 46, 23), (-1, 261, 47, 22), (-1, 262, 48, 20), (-1, 262, 49, 19), (-1, 264, 50, 16), (-1, 266, 51, 11), (-1, 268, 52, 4)], 'hashtag': '', 'sum_segment': 0},: {'photo_id': -13, 'hashtag_id': 2087736828, 'type': 529, 'x0': 263, 'x1': 306, 'y0': 107, 'y1': 146, 'score': 1.0, 'id': None, 'points': ['275,107,287,110,297,119,304,129,306,138,303,145,294,146,283,143,272,134,265,124,263,115,267,108'], 'sub_photo_id': 0, 'rles': [(-1, 271, 107, 6), (-1, 267, 108, 14), (-1, 266, 109, 19), (-1, 266, 110, 22), (-1, 265, 111, 24), (-1, 265, 112, 25), (-1, 264, 113, 27), (-1, 264, 114, 28), (-1, 263, 115, 31), (-1, 263, 116, 32), (-1, 263, 117, 33), (-1, 264, 118, 33), (-1, 264, 119, 34), (-1, 264, 120, 35), (-1, 264, 121, 35), (-1, 265, 122, 35), (-1, 265, 123, 36), (-1, 265, 124, 37), (-1, 266, 125, 36), (-1, 266, 126, 37), (-1, 267, 127, 37), (-1, 268, 128, 36), (-1, 269, 129, 36), (-1, 269, 130, 36), (-1, 270, 131, 35), (-1, 271, 132, 35), (-1, 271, 133, 35), (-1, 272, 134, 34), (-1, 273, 135, 33), (-1, 274, 136, 33), (-1, 276, 137, 31), (-1, 277, 138, 30), (-1, 278, 139, 29), (-1, 279, 140, 27), (-1, 280, 141, 26), (-1, 282, 142, 23), (-1, 283, 143, 22), (-1, 285, 144, 19), (-1, 289, 145, 15), (-1, 293, 146, 6)], 'hashtag': '', 'sum_segment': 0} Rotation of photo 1385928409 of 195 degree temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0.jpg [, , , ] 195 remove_crop_border : True version de PIL : 9.5.0 Needs to change image size ! [[-0.96592583 -0.25881905] [ 0.25881905 -0.96592583]] 195 [[-0.96592583 -0.25881905] [ 0.25881905 -0.96592583]] shrink_image : True len(list_crops) : 4 time for calcul the mask position with numpy : 0.0003807544708251953 nb_pixel_total : 727 time to create 1 rle with old method : 0.0009646415710449219 On the border Smaller than minimal size ! time for calcul the mask position with numpy : 0.0003399848937988281 nb_pixel_total : 1162 time to create 1 rle with old method : 0.0014543533325195312 . crop are not in the shrunk photo ! crop are not in the shrunk photo ! len(list_crops_rotate) : 1 list_crops_rotate : : {'photo_id': -14, 'hashtag_id': 2087736828, 'type': 529, 'x0': 248, 'x1': 296, 'y0': 78, 'y1': 111, 'score': 1.0, 'id': None, 'points': ['257,78,269,77,281,83,291,91,295,99,294,107,285,110,274,110,261,104,252,97,247,88,249,81'], 'sub_photo_id': 0, 'rles': [(-1, 256, 78, 15), (-1, 254, 79, 19), (-1, 253, 80, 22), (-1, 250, 81, 28), (-1, 249, 82, 30), (-1, 249, 83, 32), (-1, 249, 84, 34), (-1, 249, 85, 35), (-1, 249, 86, 36), (-1, 248, 87, 39), (-1, 249, 88, 39), (-1, 248, 89, 41), (-1, 248, 90, 43), (-1, 248, 91, 43), (-1, 250, 92, 42), (-1, 250, 93, 42), (-1, 250, 94, 43), (-1, 251, 95, 43), (-1, 252, 96, 42), (-1, 252, 97, 43), (-1, 253, 98, 43), (-1, 254, 99, 42), (-1, 256, 100, 40), (-1, 257, 101, 40), (-1, 258, 102, 38), (-1, 259, 103, 37), (-1, 260, 104, 36), (-1, 261, 105, 35), (-1, 263, 106, 32), (-1, 266, 107, 29), (-1, 268, 108, 27), (-1, 270, 109, 21), (-1, 271, 110, 1), (-1, 273, 110, 18), (-1, 275, 111, 9), (-1, 285, 111, 3)], 'hashtag': '', 'sum_segment': 0} Rotation of photo 1385928409 of 210 degree temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0.jpg [, , , ] 210 remove_crop_border : True version de PIL : 9.5.0 Needs to change image size ! [[-0.8660254 -0.5 ] [ 0.5 -0.8660254]] 210 [[-0.8660254 -0.5 ] [ 0.5 -0.8660254]] shrink_image : True len(list_crops) : 4 time for calcul the mask position with numpy : 0.0003349781036376953 nb_pixel_total : 250 time to create 1 rle with old method : 0.0004260540008544922 On the border Smaller than minimal size ! time for calcul the mask position with numpy : 0.0003445148468017578 nb_pixel_total : 1155 time to create 1 rle with old method : 0.0015006065368652344 . crop are not in the shrunk photo ! crop are not in the shrunk photo ! len(list_crops_rotate) : 1 list_crops_rotate : : {'photo_id': -15, 'hashtag_id': 2087736828, 'type': 529, 'x0': 227, 'x1': 276, 'y0': 52, 'y1': 82, 'score': 1.0, 'id': None, 'points': ['233,55,245,51,258,54,269,59,275,66,276,74,269,79,258,82,244,80,232,74,226,68,226,60'], 'sub_photo_id': 0, 'rles': [(-1, 244, 52, 3), (-1, 240, 53, 11), (-1, 252, 53, 2), (-1, 237, 54, 21), (-1, 233, 55, 28), (-1, 231, 56, 31), (-1, 230, 57, 35), (-1, 230, 58, 37), (-1, 228, 59, 41), (-1, 227, 60, 43), (-1, 227, 61, 44), (-1, 227, 62, 46), (-1, 227, 63, 46), (-1, 227, 64, 47), (-1, 227, 65, 49), (-1, 227, 66, 49), (-1, 227, 67, 50), (-1, 227, 68, 50), (-1, 227, 69, 50), (-1, 228, 70, 49), (-1, 229, 71, 48), (-1, 230, 72, 47), (-1, 231, 73, 46), (-1, 232, 74, 45), (-1, 233, 75, 43), (-1, 235, 76, 40), (-1, 237, 77, 36), (-1, 239, 78, 34), (-1, 242, 79, 30), (-1, 243, 80, 27), (-1, 247, 81, 1), (-1, 249, 81, 17), (-1, 253, 82, 9)], 'hashtag': '', 'sum_segment': 0} Rotation of photo 1385928409 of 225 degree temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0.jpg [, , , ] 225 remove_crop_border : True version de PIL : 9.5.0 Needs to change image size ! [[-0.70710678 -0.70710678] [ 0.70710678 -0.70710678]] 225 [[-0.70710678 -0.70710678] [ 0.70710678 -0.70710678]] shrink_image : True len(list_crops) : 4 time for calcul the mask position with numpy : 0.00036835670471191406 nb_pixel_total : 169 time to create 1 rle with old method : 0.0003139972686767578 On the border Smaller than minimal size ! time for calcul the mask position with numpy : 0.00034356117248535156 nb_pixel_total : 1161 time to create 1 rle with old method : 0.0014853477478027344 . crop are not in the shrunk photo ! crop are not in the shrunk photo ! len(list_crops_rotate) : 1 list_crops_rotate : : {'photo_id': -16, 'hashtag_id': 2087736828, 'type': 529, 'x0': 199, 'x1': 251, 'y0': 34, 'y1': 62, 'score': 1.0, 'id': None, 'points': ['204,40,214,34,228,33,240,35,247,40,250,47,245,54,235,60,221,61,209,59,201,54,199,46'], 'sub_photo_id': 0, 'rles': [(-1, 215, 34, 21), (-1, 214, 35, 25), (-1, 211, 36, 31), (-1, 210, 37, 34), (-1, 208, 38, 37), (-1, 207, 39, 41), (-1, 204, 40, 45), (-1, 203, 41, 47), (-1, 202, 42, 47), (-1, 201, 43, 49), (-1, 202, 44, 48), (-1, 201, 45, 50), (-1, 200, 46, 51), (-1, 199, 47, 52), (-1, 199, 48, 53), (-1, 200, 49, 51), (-1, 200, 50, 50), (-1, 200, 51, 49), (-1, 201, 52, 48), (-1, 201, 53, 47), (-1, 201, 54, 46), (-1, 201, 55, 45), (-1, 202, 56, 42), (-1, 205, 57, 38), (-1, 206, 58, 34), (-1, 208, 59, 31), (-1, 209, 60, 1), (-1, 211, 60, 26), (-1, 216, 61, 17), (-1, 220, 62, 5)], 'hashtag': '', 'sum_segment': 0} Rotation of photo 1385928409 of 240 degree temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0.jpg [, , , ] 240 remove_crop_border : True version de PIL : 9.5.0 Needs to change image size ! [[-0.5 -0.8660254] [ 0.8660254 -0.5 ]] 240 [[-0.5 -0.8660254] [ 0.8660254 -0.5 ]] shrink_image : True len(list_crops) : 4 time for calcul the mask position with numpy : 0.0003306865692138672 nb_pixel_total : 450 time to create 1 rle with old method : 0.0006511211395263672 On the border Smaller than minimal size ! time for calcul the mask position with numpy : 0.0003714561462402344 nb_pixel_total : 1159 time to create 1 rle with old method : 0.0014688968658447266 . crop are not in the shrunk photo ! crop are not in the shrunk photo ! time for calcul the mask position with numpy : 0.0003199577331542969 nb_pixel_total : 1 time to create 1 rle with old method : 1.7642974853515625e-05 len(list_crops_rotate) : 1 list_crops_rotate : : {'photo_id': -17, 'hashtag_id': 2087736828, 'type': 529, 'x0': 168, 'x1': 218, 'y0': 20, 'y1': 51, 'score': 1.0, 'id': None, 'points': ['171,33,179,24,192,20,204,19,213,22,218,28,214,36,206,44,193,49,180,51,172,48,168,41'], 'sub_photo_id': 0, 'rles': [(-1, 198, 20, 9), (-1, 208, 20, 1), (-1, 191, 21, 19), (-1, 187, 22, 27), (-1, 187, 23, 28), (-1, 183, 24, 33), (-1, 180, 25, 37), (-1, 179, 26, 38), (-1, 177, 27, 41), (-1, 177, 28, 42), (-1, 176, 29, 43), (-1, 175, 30, 43), (-1, 174, 31, 44), (-1, 173, 32, 44), (-1, 172, 33, 45), (-1, 171, 34, 46), (-1, 171, 35, 46), (-1, 170, 36, 46), (-1, 169, 37, 47), (-1, 170, 38, 45), (-1, 169, 39, 44), (-1, 169, 40, 44), (-1, 168, 41, 44), (-1, 168, 42, 42), (-1, 169, 43, 41), (-1, 170, 44, 39), (-1, 170, 45, 38), (-1, 171, 46, 34), (-1, 171, 47, 32), (-1, 172, 48, 28), (-1, 173, 49, 24), (-1, 177, 50, 18), (-1, 180, 51, 6), (-1, 188, 51, 1)], 'hashtag': '', 'sum_segment': 0} Rotation of photo 1385928409 of 255 degree temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0.jpg [, , , ] 255 remove_crop_border : True version de PIL : 9.5.0 Needs to change image size ! [[-0.25881905 -0.96592583] [ 0.96592583 -0.25881905]] 255 [[-0.25881905 -0.96592583] [ 0.96592583 -0.25881905]] shrink_image : True len(list_crops) : 4 time for calcul the mask position with numpy : 0.0004086494445800781 nb_pixel_total : 1237 time to create 1 rle with old method : 0.001567840576171875 On the border Smaller than minimal size ! time for calcul the mask position with numpy : 0.0003402233123779297 nb_pixel_total : 1158 time to create 1 rle with old method : 0.0014753341674804688 . crop are not in the shrunk photo ! time for calcul the mask position with numpy : 0.00032210350036621094 nb_pixel_total : 234 time to create 1 rle with old method : 0.0003921985626220703 On the border Smaller than minimal size ! len(list_crops_rotate) : 1 list_crops_rotate : : {'photo_id': -18, 'hashtag_id': 2087736828, 'type': 529, 'x0': 137, 'x1': 182, 'y0': 12, 'y1': 49, 'score': 1.0, 'id': None, 'points': ['137,34,143,24,155,16,166,12,175,13,181,17,180,26,174,36,163,44,151,49,142,48,136,42'], 'sub_photo_id': 0, 'rles': [(-1, 170, 12, 1), (-1, 164, 13, 14), (-1, 161, 14, 17), (-1, 160, 15, 19), (-1, 156, 16, 25), (-1, 155, 17, 27), (-1, 153, 18, 30), (-1, 151, 19, 32), (-1, 150, 20, 32), (-1, 149, 21, 33), (-1, 147, 22, 35), (-1, 146, 23, 36), (-1, 144, 24, 38), (-1, 144, 25, 38), (-1, 143, 26, 39), (-1, 142, 27, 39), (-1, 142, 28, 39), (-1, 142, 29, 38), (-1, 140, 30, 40), (-1, 140, 31, 39), (-1, 140, 32, 38), (-1, 139, 33, 39), (-1, 138, 34, 39), (-1, 138, 35, 39), (-1, 138, 36, 38), (-1, 137, 37, 38), (-1, 137, 38, 36), (-1, 137, 39, 36), (-1, 138, 40, 33), (-1, 137, 41, 32), (-1, 137, 42, 31), (-1, 137, 43, 30), (-1, 137, 44, 29), (-1, 139, 45, 25), (-1, 139, 46, 22), (-1, 141, 47, 18), (-1, 142, 48, 14), (-1, 143, 49, 10)], 'hashtag': '', 'sum_segment': 0} Rotation of photo 1385928409 of 270 degree temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0.jpg [, , , ] 270 remove_crop_border : True version de PIL : 9.5.0 Needs to change image size ! [[-1.8369702e-16 -1.0000000e+00] [ 1.0000000e+00 -1.8369702e-16]] 270 [[-1.8369702e-16 -1.0000000e+00] [ 1.0000000e+00 -1.8369702e-16]] shrink_image : True len(list_crops) : 4 time for calcul the mask position with numpy : 0.0003581047058105469 nb_pixel_total : 1389 time to create 1 rle with old method : 0.002351522445678711 .time for calcul the mask position with numpy : 0.00034046173095703125 nb_pixel_total : 1157 time to create 1 rle with old method : 0.0014786720275878906 . crop are not in the shrunk photo ! crop are not in the shrunk photo ! len(list_crops_rotate) : 2 list_crops_rotate : : {'photo_id': -19, 'hashtag_id': 2087736828, 'type': 529, 'x0': 1, 'x1': 52, 'y0': 26, 'y1': 62, 'score': 1.0, 'id': None, 'points': ['41,32,50,40,52,50,49,57,39,61,26,60,13,54,4,46,1,37,5,29,15,25,28,26'], 'sub_photo_id': 0, 'rles': [(-1, 14, 26, 8), (-1, 12, 27, 18), (-1, 9, 28, 23), (-1, 7, 29, 27), (-1, 5, 30, 31), (-1, 5, 31, 33), (-1, 4, 32, 36), (-1, 4, 33, 38), (-1, 3, 34, 40), (-1, 3, 35, 41), (-1, 2, 36, 43), (-1, 2, 37, 45), (-1, 1, 38, 47), (-1, 1, 39, 48), (-1, 2, 40, 48), (-1, 2, 41, 49), (-1, 2, 42, 49), (-1, 3, 43, 48), (-1, 3, 44, 49), (-1, 3, 45, 49), (-1, 4, 46, 48), (-1, 4, 47, 48), (-1, 5, 48, 47), (-1, 6, 49, 47), (-1, 7, 50, 46), (-1, 8, 51, 45), (-1, 10, 52, 43), (-1, 11, 53, 41), (-1, 12, 54, 40), (-1, 13, 55, 38), (-1, 15, 56, 36), (-1, 17, 57, 33), (-1, 19, 58, 31), (-1, 21, 59, 27), (-1, 23, 60, 23), (-1, 25, 61, 18), (-1, 33, 62, 8)], 'hashtag': '', 'sum_segment': 0},: {'photo_id': -19, 'hashtag_id': 2087736828, 'type': 529, 'x0': 107, 'x1': 146, 'y0': 14, 'y1': 57, 'score': 1.0, 'id': None, 'points': ['107,44,110,32,119,22,129,15,138,13,144,16,145,25,143,36,134,47,124,54,115,56,108,52'], 'sub_photo_id': 0, 'rles': [(-1, 136, 14, 4), (-1, 132, 15, 10), (-1, 129, 16, 15), (-1, 127, 17, 19), (-1, 126, 18, 20), (-1, 124, 19, 22), (-1, 123, 20, 23), (-1, 122, 21, 24), (-1, 120, 22, 27), (-1, 119, 23, 28), (-1, 118, 24, 29), (-1, 117, 25, 30), (-1, 116, 26, 31), (-1, 115, 27, 32), (-1, 115, 28, 31), (-1, 114, 29, 32), (-1, 113, 30, 33), (-1, 112, 31, 34), (-1, 111, 32, 34), (-1, 110, 33, 35), (-1, 110, 34, 35), (-1, 110, 35, 35), (-1, 109, 36, 35), (-1, 109, 37, 35), (-1, 109, 38, 34), (-1, 109, 39, 33), (-1, 108, 40, 34), (-1, 108, 41, 33), (-1, 108, 42, 32), (-1, 108, 43, 31), (-1, 107, 44, 31), (-1, 107, 45, 30), (-1, 107, 46, 30), (-1, 107, 47, 29), (-1, 107, 48, 28), (-1, 107, 49, 27), (-1, 108, 50, 24), (-1, 108, 51, 23), (-1, 108, 52, 21), (-1, 108, 53, 20), (-1, 109, 54, 18), (-1, 111, 55, 14), (-1, 113, 56, 9), (-1, 115, 57, 3)], 'hashtag': '', 'sum_segment': 0} Rotation of photo 1385928409 of 285 degree temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0.jpg [, , , ] 285 remove_crop_border : True version de PIL : 9.5.0 Needs to change image size ! [[ 0.25881905 -0.96592583] [ 0.96592583 0.25881905]] 285 [[ 0.25881905 -0.96592583] [ 0.96592583 0.25881905]] shrink_image : True len(list_crops) : 4 time for calcul the mask position with numpy : 0.00038361549377441406 nb_pixel_total : 727 time to create 1 rle with old method : 0.0009605884552001953 On the border Smaller than minimal size ! time for calcul the mask position with numpy : 0.0003476142883300781 nb_pixel_total : 1162 time to create 1 rle with old method : 0.0018985271453857422 . crop are not in the shrunk photo ! crop are not in the shrunk photo ! len(list_crops_rotate) : 1 list_crops_rotate : : {'photo_id': -20, 'hashtag_id': 2087736828, 'type': 529, 'x0': 78, 'x1': 111, 'y0': 24, 'y1': 72, 'score': 1.0, 'id': None, 'points': ['78,62,77,50,83,38,91,28,99,24,107,25,110,34,110,45,104,58,97,67,88,72,81,70'], 'sub_photo_id': 0, 'rles': [(-1, 101, 24, 1), (-1, 98, 25, 8), (-1, 97, 26, 12), (-1, 95, 27, 14), (-1, 94, 28, 15), (-1, 92, 29, 17), (-1, 90, 30, 21), (-1, 90, 31, 21), (-1, 89, 32, 22), (-1, 88, 33, 24), (-1, 87, 34, 25), (-1, 87, 35, 25), (-1, 86, 36, 25), (-1, 85, 37, 27), (-1, 84, 38, 28), (-1, 84, 39, 28), (-1, 83, 40, 29), (-1, 83, 41, 29), (-1, 82, 42, 30), (-1, 81, 43, 31), (-1, 81, 44, 31), (-1, 81, 45, 31), (-1, 80, 46, 31), (-1, 80, 47, 31), (-1, 79, 48, 31), (-1, 79, 49, 32), (-1, 78, 50, 32), (-1, 78, 51, 31), (-1, 78, 52, 31), (-1, 78, 53, 30), (-1, 78, 54, 30), (-1, 78, 55, 29), (-1, 78, 56, 29), (-1, 78, 57, 29), (-1, 78, 58, 28), (-1, 78, 59, 28), (-1, 78, 60, 27), (-1, 78, 61, 26), (-1, 78, 62, 25), (-1, 78, 63, 24), (-1, 78, 64, 23), (-1, 79, 65, 21), (-1, 79, 66, 21), (-1, 80, 67, 19), (-1, 81, 68, 17), (-1, 81, 69, 15), (-1, 81, 70, 14), (-1, 82, 71, 10), (-1, 87, 72, 1), (-1, 89, 72, 3)], 'hashtag': '', 'sum_segment': 0} Rotation of photo 1385928409 of 300 degree temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0.jpg [, , , ] 300 remove_crop_border : True version de PIL : 9.5.0 Needs to change image size ! [[ 0.5 -0.8660254] [ 0.8660254 0.5 ]] 300 [[ 0.5 -0.8660254] [ 0.8660254 0.5 ]] shrink_image : True len(list_crops) : 4 time for calcul the mask position with numpy : 0.000335693359375 nb_pixel_total : 250 time to create 1 rle with old method : 0.00045299530029296875 On the border Smaller than minimal size ! time for calcul the mask position with numpy : 0.0003457069396972656 nb_pixel_total : 1155 time to create 1 rle with old method : 0.0019397735595703125 . crop are not in the shrunk photo ! crop are not in the shrunk photo ! len(list_crops_rotate) : 1 list_crops_rotate : : {'photo_id': -21, 'hashtag_id': 2087736828, 'type': 529, 'x0': 52, 'x1': 82, 'y0': 44, 'y1': 93, 'score': 1.0, 'id': None, 'points': ['55,86,51,74,54,61,59,50,66,44,74,43,79,50,82,61,80,75,74,87,68,93,60,93'], 'sub_photo_id': 0, 'rles': [(-1, 67, 44, 8), (-1, 65, 45, 11), (-1, 65, 46, 12), (-1, 64, 47, 13), (-1, 62, 48, 17), (-1, 62, 49, 18), (-1, 61, 50, 19), (-1, 60, 51, 21), (-1, 59, 52, 22), (-1, 59, 53, 22), (-1, 58, 54, 23), (-1, 58, 55, 24), (-1, 57, 56, 25), (-1, 57, 57, 25), (-1, 57, 58, 25), (-1, 56, 59, 27), (-1, 55, 60, 28), (-1, 55, 61, 28), (-1, 55, 62, 28), (-1, 54, 63, 29), (-1, 54, 64, 29), (-1, 54, 65, 29), (-1, 54, 66, 29), (-1, 53, 67, 30), (-1, 53, 68, 29), (-1, 54, 69, 28), (-1, 53, 70, 29), (-1, 53, 71, 29), (-1, 53, 72, 28), (-1, 53, 73, 29), (-1, 52, 74, 29), (-1, 52, 75, 29), (-1, 52, 76, 29), (-1, 53, 77, 28), (-1, 53, 78, 27), (-1, 53, 79, 26), (-1, 53, 80, 26), (-1, 54, 81, 25), (-1, 54, 82, 24), (-1, 54, 83, 24), (-1, 55, 84, 22), (-1, 55, 85, 22), (-1, 55, 86, 21), (-1, 55, 87, 21), (-1, 56, 88, 19), (-1, 56, 89, 18), (-1, 57, 90, 16), (-1, 59, 91, 13), (-1, 59, 92, 12), (-1, 60, 93, 10)], 'hashtag': '', 'sum_segment': 0} Rotation of photo 1385928409 of 315 degree temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0.jpg [, , , ] 315 remove_crop_border : True version de PIL : 9.5.0 Needs to change image size ! [[ 0.70710678 -0.70710678] [ 0.70710678 0.70710678]] 315 [[ 0.70710678 -0.70710678] [ 0.70710678 0.70710678]] shrink_image : True len(list_crops) : 4 time for calcul the mask position with numpy : 0.0003459453582763672 nb_pixel_total : 169 time to create 1 rle with old method : 0.0003046989440917969 On the border Smaller than minimal size ! time for calcul the mask position with numpy : 0.0003426074981689453 nb_pixel_total : 1161 time to create 1 rle with old method : 0.0014793872833251953 . crop are not in the shrunk photo ! crop are not in the shrunk photo ! len(list_crops_rotate) : 1 list_crops_rotate : : {'photo_id': -22, 'hashtag_id': 2087736828, 'type': 529, 'x0': 34, 'x1': 62, 'y0': 69, 'y1': 121, 'score': 1.0, 'id': None, 'points': ['40,115,34,105,33,91,35,79,40,72,47,69,54,74,60,84,61,98,59,110,54,118,46,120'], 'sub_photo_id': 0, 'rles': [(-1, 48, 69, 1), (-1, 45, 70, 5), (-1, 41, 71, 1), (-1, 43, 71, 8), (-1, 40, 72, 13), (-1, 39, 73, 15), (-1, 39, 74, 16), (-1, 39, 75, 17), (-1, 38, 76, 18), (-1, 37, 77, 20), (-1, 37, 78, 21), (-1, 36, 79, 22), (-1, 36, 80, 22), (-1, 36, 81, 23), (-1, 35, 82, 25), (-1, 35, 83, 25), (-1, 35, 84, 26), (-1, 34, 85, 27), (-1, 34, 86, 27), (-1, 34, 87, 27), (-1, 34, 88, 28), (-1, 34, 89, 28), (-1, 34, 90, 28), (-1, 34, 91, 28), (-1, 34, 92, 28), (-1, 34, 93, 28), (-1, 34, 94, 28), (-1, 34, 95, 28), (-1, 34, 96, 29), (-1, 34, 97, 29), (-1, 34, 98, 29), (-1, 34, 99, 29), (-1, 34, 100, 29), (-1, 34, 101, 28), (-1, 34, 102, 28), (-1, 34, 103, 28), (-1, 34, 104, 28), (-1, 34, 105, 27), (-1, 35, 106, 26), (-1, 36, 107, 25), (-1, 36, 108, 25), (-1, 36, 109, 25), (-1, 37, 110, 23), (-1, 38, 111, 23), (-1, 38, 112, 22), (-1, 39, 113, 20), (-1, 40, 114, 19), (-1, 40, 115, 18), (-1, 40, 116, 17), (-1, 41, 117, 16), (-1, 42, 118, 15), (-1, 43, 119, 1), (-1, 45, 119, 11), (-1, 46, 120, 6), (-1, 47, 121, 2)], 'hashtag': '', 'sum_segment': 0} Rotation of photo 1385928409 of 330 degree temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0.jpg [, , , ] 330 remove_crop_border : True version de PIL : 9.5.0 Needs to change image size ! [[ 0.8660254 -0.5 ] [ 0.5 0.8660254]] 330 [[ 0.8660254 -0.5 ] [ 0.5 0.8660254]] shrink_image : True len(list_crops) : 4 time for calcul the mask position with numpy : 0.0003311634063720703 nb_pixel_total : 450 time to create 1 rle with old method : 0.0006511211395263672 On the border Smaller than minimal size ! time for calcul the mask position with numpy : 0.0003418922424316406 nb_pixel_total : 1159 time to create 1 rle with old method : 0.0014503002166748047 . crop are not in the shrunk photo ! On the border Smaller than minimal size ! len(list_crops_rotate) : 1 list_crops_rotate : : {'photo_id': -23, 'hashtag_id': 2087736828, 'type': 529, 'x0': 20, 'x1': 51, 'y0': 102, 'y1': 152, 'score': 1.0, 'id': None, 'points': ['33,148,24,140,20,127,19,115,22,106,28,101,36,105,44,113,49,126,51,139,48,147,41,151'], 'sub_photo_id': 0, 'rles': [(-1, 28, 102, 2), (-1, 27, 103, 5), (-1, 25, 104, 11), (-1, 24, 105, 14), (-1, 23, 106, 16), (-1, 22, 107, 17), (-1, 22, 108, 19), (-1, 22, 109, 20), (-1, 22, 110, 20), (-1, 21, 111, 23), (-1, 20, 112, 25), (-1, 21, 113, 25), (-1, 20, 114, 26), (-1, 20, 115, 26), (-1, 20, 116, 27), (-1, 20, 117, 27), (-1, 20, 118, 28), (-1, 20, 119, 28), (-1, 20, 120, 28), (-1, 20, 121, 29), (-1, 20, 122, 29), (-1, 21, 123, 28), (-1, 21, 124, 29), (-1, 21, 125, 29), (-1, 21, 126, 30), (-1, 21, 127, 30), (-1, 21, 128, 30), (-1, 21, 129, 30), (-1, 22, 130, 29), (-1, 22, 131, 29), (-1, 22, 132, 30), (-1, 22, 133, 29), (-1, 24, 134, 27), (-1, 24, 135, 28), (-1, 24, 136, 28), (-1, 24, 137, 28), (-1, 25, 138, 27), (-1, 25, 139, 27), (-1, 25, 140, 27), (-1, 26, 141, 25), (-1, 27, 142, 24), (-1, 27, 143, 24), (-1, 29, 144, 21), (-1, 30, 145, 20), (-1, 31, 146, 19), (-1, 32, 147, 18), (-1, 33, 148, 16), (-1, 34, 149, 14), (-1, 36, 150, 10), (-1, 37, 151, 1), (-1, 39, 151, 5), (-1, 41, 152, 2)], 'hashtag': '', 'sum_segment': 0} Rotation of photo 1385928409 of 345 degree temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0.jpg [, , , ] 345 remove_crop_border : True version de PIL : 9.5.0 Needs to change image size ! [[ 0.96592583 -0.25881905] [ 0.25881905 0.96592583]] 345 [[ 0.96592583 -0.25881905] [ 0.25881905 0.96592583]] shrink_image : True len(list_crops) : 4 time for calcul the mask position with numpy : 0.00037217140197753906 nb_pixel_total : 1237 time to create 1 rle with old method : 0.0015673637390136719 On the border Smaller than minimal size ! time for calcul the mask position with numpy : 0.00033974647521972656 nb_pixel_total : 1157 time to create 1 rle with old method : 0.012245416641235352 . crop are not in the shrunk photo ! time for calcul the mask position with numpy : 0.00032639503479003906 nb_pixel_total : 234 time to create 1 rle with old method : 0.00039386749267578125 On the border Smaller than minimal size ! len(list_crops_rotate) : 1 list_crops_rotate : : {'photo_id': -24, 'hashtag_id': 2087736828, 'type': 529, 'x0': 12, 'x1': 49, 'y0': 138, 'y1': 183, 'score': 1.0, 'id': None, 'points': ['34,182,24,176,16,164,12,153,13,144,17,138,26,139,36,145,44,156,49,168,48,177,42,183'], 'sub_photo_id': 0, 'rles': [(-1, 18, 138, 2), (-1, 17, 139, 10), (-1, 16, 140, 13), (-1, 16, 141, 15), (-1, 15, 142, 17), (-1, 13, 143, 21), (-1, 13, 144, 23), (-1, 13, 145, 24), (-1, 13, 146, 25), (-1, 13, 147, 25), (-1, 13, 148, 27), (-1, 13, 149, 27), (-1, 12, 150, 29), (-1, 13, 151, 28), (-1, 13, 152, 29), (-1, 13, 153, 30), (-1, 13, 154, 31), (-1, 13, 155, 32), (-1, 13, 156, 32), (-1, 14, 157, 32), (-1, 14, 158, 32), (-1, 14, 159, 32), (-1, 15, 160, 32), (-1, 16, 161, 31), (-1, 16, 162, 32), (-1, 16, 163, 32), (-1, 16, 164, 32), (-1, 17, 165, 32), (-1, 18, 166, 31), (-1, 18, 167, 31), (-1, 19, 168, 31), (-1, 19, 169, 31), (-1, 20, 170, 30), (-1, 21, 171, 29), (-1, 22, 172, 28), (-1, 22, 173, 28), (-1, 23, 174, 27), (-1, 24, 175, 26), (-1, 24, 176, 26), (-1, 26, 177, 24), (-1, 27, 178, 22), (-1, 30, 179, 18), (-1, 30, 180, 17), (-1, 33, 181, 14), (-1, 34, 182, 11), (-1, 37, 183, 3), (-1, 41, 183, 3)], 'hashtag': '', 'sum_segment': 0} About to upload 24 photos upload in portfolio : 27193598 init cache_photo without model_param we have 24 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1758729005_1864669 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first Unexecpected behavior in 07/2025 that can be generalized l287 : type_extension .jpg This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first Unexecpected behavior in 07/2025 that can be generalized l287 : type_extension .jpg This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first Unexecpected behavior in 07/2025 that can be generalized l287 : type_extension .jpg This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first Unexecpected behavior in 07/2025 that can be generalized l287 : type_extension .jpg This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first Unexecpected behavior in 07/2025 that can be generalized l287 : type_extension .jpg This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first Unexecpected behavior in 07/2025 that can be generalized l287 : type_extension .jpg This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first Unexecpected behavior in 07/2025 that can be generalized l287 : type_extension .jpg This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first Unexecpected behavior in 07/2025 that can be generalized l287 : type_extension .jpg This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first Unexecpected behavior in 07/2025 that can be generalized l287 : type_extension .jpg This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first Unexecpected behavior in 07/2025 that can be generalized l287 : type_extension .jpg This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first Unexecpected behavior in 07/2025 that can be generalized l287 : type_extension .jpg This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first Unexecpected behavior in 07/2025 that can be generalized l287 : type_extension .jpg This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first Unexecpected behavior in 07/2025 that can be generalized l287 : type_extension .jpg This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first Unexecpected behavior in 07/2025 that can be generalized l287 : type_extension .jpg This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first Unexecpected behavior in 07/2025 that can be generalized l287 : type_extension .jpg This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first Unexecpected behavior in 07/2025 that can be generalized l287 : type_extension .jpg This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first Unexecpected behavior in 07/2025 that can be generalized l287 : type_extension .jpg This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first Unexecpected behavior in 07/2025 that can be generalized l287 : type_extension .jpg This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first Unexecpected behavior in 07/2025 that can be generalized l287 : type_extension .jpg This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first Unexecpected behavior in 07/2025 that can be generalized l287 : type_extension .jpg This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first Unexecpected behavior in 07/2025 that can be generalized l287 : type_extension .jpg This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first Unexecpected behavior in 07/2025 that can be generalized l287 : type_extension .jpg This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first Unexecpected behavior in 07/2025 that can be generalized l287 : type_extension .jpg This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first Unexecpected behavior in 07/2025 that can be generalized l287 : type_extension .jpg This is a hack ! we have uploaded 24 photos in the portfolio 27193598 time of upload the photos Elapsed time : 6.605446100234985 map_filename_photo_id : 24 map_filename_photo_id : {'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_00.jpg': 1385928412, 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_015.jpg': 1385928413, 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_030.jpg': 1385928414, 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_045.jpg': 1385928415, 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_060.jpg': 1385928416, 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_075.jpg': 1385928417, 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_090.jpg': 1385928418, 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0105.jpg': 1385928419, 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0120.jpg': 1385928420, 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0135.jpg': 1385928421, 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0150.jpg': 1385928422, 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0165.jpg': 1385928423, 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0180.jpg': 1385928424, 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0195.jpg': 1385928425, 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0210.jpg': 1385928426, 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0225.jpg': 1385928427, 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0240.jpg': 1385928428, 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0255.jpg': 1385928429, 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0270.jpg': 1385928430, 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0285.jpg': 1385928431, 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0300.jpg': 1385928432, 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0315.jpg': 1385928433, 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0330.jpg': 1385928434, 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0345.jpg': 1385928435} Len new_chis : 24 Len list_new_chi_with_photo_id : 28 of type : 529 list_new_chi_with_photo_id : [, , , , , , , , , , , , , , , , , , , , , , , , , , , ] batch 1 Loaded 28 chid ids of type : 529 Number RLEs to save : 1197 TO DO : save crop sub photo not yet done ! batch 1 Loaded 28 chid ids of type : 529 ++++++++++++++++++++++++++++Number RLEs to save : 0 TO DO : save crop sub photo not yet done ! batch 1 Loaded 28 chid ids of type : 529 ++++++++++++++++++++++++++++Number RLEs to save : 0 TO DO : save crop sub photo not yet done ! batch 1 Loaded 28 chid ids of type : 529 ++++++++++++++++++++++++++++Number RLEs to save : 0 TO DO : save crop sub photo not yet done ! batch 1 Loaded 28 chid ids of type : 529 ++++++++++++++++++++++++++++Number RLEs to save : 0 TO DO : save crop sub photo not yet done ! batch 1 Loaded 28 chid ids of type : 529 ++++++++++++++++++++++++++++Number RLEs to save : 0 TO DO : save crop sub photo not yet done ! batch 1 Loaded 28 chid ids of type : 529 ++++++++++++++++++++++++++++Number RLEs to save : 0 TO DO : save crop sub photo not yet done ! batch 1 Loaded 28 chid ids of type : 529 ++++++++++++++++++++++++++++Number RLEs to save : 0 TO DO : save crop sub photo not yet done ! batch 1 Loaded 28 chid ids of type : 529 ++++++++++++++++++++++++++++Number RLEs to save : 0 TO DO : save crop sub photo not yet done ! batch 1 Loaded 28 chid ids of type : 529 ++++++++++++++++++++++++++++Number RLEs to save : 0 TO DO : save crop sub photo not yet done ! batch 1 Loaded 28 chid ids of type : 529 ++++++++++++++++++++++++++++Number RLEs to save : 0 TO DO : save crop sub photo not yet done ! batch 1 Loaded 28 chid ids of type : 529 ++++++++++++++++++++++++++++Number RLEs to save : 0 TO DO : save crop sub photo not yet done ! batch 1 Loaded 28 chid ids of type : 529 ++++++++++++++++++++++++++++Number RLEs to save : 0 TO DO : save crop sub photo not yet done ! batch 1 Loaded 28 chid ids of type : 529 ++++++++++++++++++++++++++++Number RLEs to save : 0 TO DO : save crop sub photo not yet done ! batch 1 Loaded 28 chid ids of type : 529 ++++++++++++++++++++++++++++Number RLEs to save : 0 TO DO : save crop sub photo not yet done ! batch 1 Loaded 28 chid ids of type : 529 ++++++++++++++++++++++++++++Number RLEs to save : 0 TO DO : save crop sub photo not yet done ! batch 1 Loaded 28 chid ids of type : 529 ++++++++++++++++++++++++++++Number RLEs to save : 0 TO DO : save crop sub photo not yet done ! batch 1 Loaded 28 chid ids of type : 529 ++++++++++++++++++++++++++++Number RLEs to save : 0 TO DO : save crop sub photo not yet done ! batch 1 Loaded 28 chid ids of type : 529 ++++++++++++++++++++++++++++Number RLEs to save : 0 TO DO : save crop sub photo not yet done ! batch 1 Loaded 28 chid ids of type : 529 ++++++++++++++++++++++++++++Number RLEs to save : 0 TO DO : save crop sub photo not yet done ! batch 1 Loaded 28 chid ids of type : 529 ++++++++++++++++++++++++++++Number RLEs to save : 0 TO DO : save crop sub photo not yet done ! batch 1 Loaded 28 chid ids of type : 529 ++++++++++++++++++++++++++++Number RLEs to save : 0 TO DO : save crop sub photo not yet done ! batch 1 Loaded 28 chid ids of type : 529 ++++++++++++++++++++++++++++Number RLEs to save : 0 TO DO : save crop sub photo not yet done ! batch 1 Loaded 28 chid ids of type : 529 ++++++++++++++++++++++++++++Number RLEs to save : 0 TO DO : save crop sub photo not yet done ! After datou_step_exec type output : time spend for datou_step_exec : 11.294659852981567 time spend to save output : 7.081031799316406e-05 total time spend for step 3 : 11.29473066329956 caffe_path_current : About to save ! 1 Inside saveOutput : final : True verbose : True saveOutput not yet implemented for datou_step.type : rotate we use saveGeneral [937852786, 937852786, '1385928409'] map_info['map_portfolio_photo'] : {} final : True mtd_id 243 list_pids : [937852786, 937852786, '1385928409'] Looping around the photos to save general results len do output : 24 /1385928412Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385928413Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385928414Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385928415Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385928416Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385928417Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385928418Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385928419Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385928420Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385928421Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385928422Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385928423Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385928424Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385928425Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385928426Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385928427Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385928428Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385928429Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385928430Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385928431Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385928432Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385928433Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385928434Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385928435Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('243', None, None, None, None, None, None, None, None) ('243', None, '937852786', None, None, None, None, None, None) ('243', None, None, None, None, None, None, None, None) ('243', None, '937852786', None, None, None, None, None, None) ('243', None, None, None, None, None, None, None, None) ('243', None, '1385928409', None, None, None, None, None, None) begin to insert list_values into mtr_datou_result : length of list_values in save_final : 75 insert ignore into MTRPhoto.mtr_datou_result (mtd_id, mtr_portfolio_id,mtr_photo_id,result,result_long,result_double,hashtag_id,proba, mtr_current_id) values (%s,%s,%s,%s,%s,%s,%s,%s,%s) on duplicate key update mtr_portfolio_id = mtr_portfolio_id list_values : [('243', None, '1385928412', 'None', None, None, None, None, None), ('243', None, '1385928413', 'None', None, None, None, None, None), ('243', None, '1385928414', 'None', None, None, None, None, None), ('243', None, '1385928415', 'None', None, None, None, None, None), ('243', None, '1385928416', 'None', None, None, None, None, None), ('243', None, '1385928417', 'None', None, None, None, None, None), ('243', None, '1385928418', 'None', None, None, None, None, None), ('243', None, '1385928419', 'None', None, None, None, None, None), ('243', None, '1385928420', 'None', None, None, None, None, None), ('243', None, '1385928421', 'None', None, None, None, None, None), ('243', None, '1385928422', 'None', None, None, None, None, None), ('243', None, '1385928423', 'None', None, None, None, None, None), ('243', None, '1385928424', 'None', None, None, None, None, None), ('243', None, '1385928425', 'None', None, None, None, None, None), ('243', None, '1385928426', 'None', None, None, None, None, None), ('243', None, '1385928427', 'None', None, None, None, None, None), ('243', None, '1385928428', 'None', None, None, None, None, None), ('243', None, '1385928429', 'None', None, None, None, None, None), ('243', None, '1385928430', 'None', None, None, None, None, None), ('243', None, '1385928431', 'None', None, None, None, None, None), ('243', None, '1385928432', 'None', None, None, None, None, None), ('243', None, '1385928433', 'None', None, None, None, None, None), ('243', None, '1385928434', 'None', None, None, None, None, None), ('243', None, '1385928435', 'None', None, None, None, None, None), ('243', None, '937852786', None, None, None, None, None, None), ('243', None, '1385928409', None, None, None, None, None, None)] time used for this insertion : 0.02087712287902832 save_final save missing photos in datou_result : After save, about to update current ! datou_cur_ids : [] len(datou.list_steps) : 3 output : {1385928412: ['937852786', 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_00.jpg', [, ]], 1385928413: ['937852786', 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_015.jpg', []], 1385928414: ['937852786', 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_030.jpg', []], 1385928415: ['937852786', 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_045.jpg', []], 1385928416: ['937852786', 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_060.jpg', []], 1385928417: ['937852786', 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_075.jpg', []], 1385928418: ['937852786', 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_090.jpg', [, ]], 1385928419: ['937852786', 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0105.jpg', []], 1385928420: ['937852786', 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0120.jpg', []], 1385928421: ['937852786', 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0135.jpg', []], 1385928422: ['937852786', 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0150.jpg', []], 1385928423: ['937852786', 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0165.jpg', []], 1385928424: ['937852786', 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0180.jpg', [, ]], 1385928425: ['937852786', 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0195.jpg', []], 1385928426: ['937852786', 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0210.jpg', []], 1385928427: ['937852786', 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0225.jpg', []], 1385928428: ['937852786', 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0240.jpg', []], 1385928429: ['937852786', 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0255.jpg', []], 1385928430: ['937852786', 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0270.jpg', [, ]], 1385928431: ['937852786', 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0285.jpg', []], 1385928432: ['937852786', 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0300.jpg', []], 1385928433: ['937852786', 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0315.jpg', []], 1385928434: ['937852786', 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0330.jpg', []], 1385928435: ['937852786', 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0345.jpg', []]} ret_da : {1385928412: ['937852786', 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_00.jpg', [, ]], 1385928413: ['937852786', 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_015.jpg', []], 1385928414: ['937852786', 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_030.jpg', []], 1385928415: ['937852786', 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_045.jpg', []], 1385928416: ['937852786', 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_060.jpg', []], 1385928417: ['937852786', 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_075.jpg', []], 1385928418: ['937852786', 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_090.jpg', [, ]], 1385928419: ['937852786', 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0105.jpg', []], 1385928420: ['937852786', 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0120.jpg', []], 1385928421: ['937852786', 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0135.jpg', []], 1385928422: ['937852786', 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0150.jpg', []], 1385928423: ['937852786', 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0165.jpg', []], 1385928424: ['937852786', 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0180.jpg', [, ]], 1385928425: ['937852786', 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0195.jpg', []], 1385928426: ['937852786', 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0210.jpg', []], 1385928427: ['937852786', 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0225.jpg', []], 1385928428: ['937852786', 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0240.jpg', []], 1385928429: ['937852786', 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0255.jpg', []], 1385928430: ['937852786', 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0270.jpg', [, ]], 1385928431: ['937852786', 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0285.jpg', []], 1385928432: ['937852786', 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0300.jpg', []], 1385928433: ['937852786', 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0315.jpg', []], 1385928434: ['937852786', 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0330.jpg', []], 1385928435: ['937852786', 'temp/1758728962_1864669_937852786_7d9a231a08a1c63d0868e56a5361bf67_0345.jpg', []]} list chi : [[, ], [], [], [], [], [], [, ], [], [], [], [], [], [, ], [], [], [], [], [], [, ], [], [], [], [], []] ############################### TEST flip ################################ t Inside batchDatouExec : verbose : True ##### chargement datou SELECT name, created_at,limit_max FROM MTRDatou.mtr_datou WHERE id=571 SELECT mtd.id, mtdt.`type`, mtd.`param`, mtd.param_json, mtdt.nb_input, mtdt.nb_output, mtdt.prod, mtdt.is_local, mtdt.is_datou_depend, mtdt.is_photo_id_local FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_step_types mtdt WHERE mtdt.`id`=mtd.`type` AND mtd.mtd_id=571 SELECT mtd.id, mtd.mtd_id, mdsdt.id, mdsdt.name, mdsdt.description, msid.output_or_input, msid.data_order_id, mdsdt.type FROM MTRDatou.mtr_datou_step mtd, MTRDatou.mtr_datou_steptype_io_datatypes msid, MTRDatou.mtr_datou_step_data_types mdsdt WHERE mtd.`type`=msid.`mtr_datou_step_type` AND mtd.mtd_id= 571 AND msid.data_type=mdsdt.id SELECT mts_id_output, id_output, mts_id_input, id_input FROM MTRDatou.mtr_datou_step_by_step WHERE mtd_id=571 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! no param json to modify List Step Type Loaded in datou : flip list_input_json : [] ##### fin chargement datou ##### chargement data ##### Call load_data_input : nb_thread : 5 origin SELECT photo_id, url FROM MTRBack.view_photos_with_urls_250912 ph WHERE photo_id IN (911785586) Found this number of photos: 1 ##### Call download_photos : nb_thread : 5 Exception in thread Thread-81: Traceback (most recent call last): File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner self.run() File "/usr/lib/python3.8/threading.py", line 870, in run self._target(*self._args, **self._kwargs) File "/home/admin/workarea/git/Velours/python/mtr/datou/datou_lib.py", line 294, in download_sub url_temp = url.split(".") AttributeError: 'NoneType' object has no attribute 'split' Command terminated by signal 15 41.14user 34.10system 15:23:27elapsed 0%CPU (0avgtext+0avgdata 6466532maxresident)k 4347816inputs+31552outputs (7026major+3102748minor)pagefaults 0swaps