<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Publishing DTD v1.3 20210610//EN" "JATS-journalpublishing1-3.dtd">
<article article-type="research-article" dtd-version="1.3" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xml:lang="ru"><front><journal-meta><journal-id journal-id-type="publisher-id">izvestswsu</journal-id><journal-title-group><journal-title xml:lang="ru">Известия Юго-Западного государственного университета</journal-title><trans-title-group xml:lang="en"><trans-title>Proceedings of the Southwest State University</trans-title></trans-title-group></journal-title-group><issn pub-type="ppub">2223-1560</issn><issn pub-type="epub">2686-6757</issn><publisher><publisher-name>ЮЗГУ</publisher-name></publisher></journal-meta><article-meta><article-id pub-id-type="doi">10.21869/2223-1560-2024-28-3-214-227</article-id><article-id custom-type="elpub" pub-id-type="custom">izvestswsu-1337</article-id><article-categories><subj-group subj-group-type="heading"><subject>Research Article</subject></subj-group><subj-group subj-group-type="section-heading" xml:lang="ru"><subject>Информатика, вычислительная техника и управление</subject></subj-group><subj-group subj-group-type="section-heading" xml:lang="en"><subject>Computer science, computer engineering and IT managment</subject></subj-group></article-categories><title-group><article-title>Разработка метода локализации в замкнутой  и насыщенной объектами среде</article-title><trans-title-group xml:lang="en"><trans-title>Development of a method for localizing objects  in a closed and saturated environment</trans-title></trans-title-group></title-group><contrib-group><contrib contrib-type="author" corresp="yes"><contrib-id contrib-id-type="orcid">https://orcid.org/0000-0002-3746-3493</contrib-id><name-alternatives><name name-style="eastern" xml:lang="ru"><surname>Мостаков</surname><given-names>Н. А.</given-names></name><name name-style="western" xml:lang="en"><surname>Mostakov</surname><given-names>N. A.</given-names></name></name-alternatives><bio xml:lang="ru"><p>Мостаков Николай Алексеевич, аспирант,  лаборатория Киберфизических систем,</p><p>д. 65, ул. Профсоюзная, г. Москва 117997.</p></bio><bio xml:lang="en"><p>Nikolay A. Mostakov, Post-Graduate Student, Laboratory of Cybernetic Systems, </p><p>65, Profsoyuznaya str., Moscow 117997.</p></bio><email xlink:type="simple">nikrus333@gmail.com</email><xref ref-type="aff" rid="aff-1"/></contrib><contrib contrib-type="author" corresp="yes"><contrib-id contrib-id-type="orcid">https://orcid.org/0000-0003-4221-7710</contrib-id><name-alternatives><name name-style="eastern" xml:lang="ru"><surname>Захарова</surname><given-names>А. А.</given-names></name><name name-style="western" xml:lang="en"><surname>Zakharova</surname><given-names>A. A.</given-names></name></name-alternatives><bio xml:lang="ru"><p>Захарова Алёна Александровна, доктор  технических наук, главный научный сотрудник, лаборатория Киберфизических систем, </p><p>д. 65, ул. Профсоюзная, г. Москва 117997.</p><p>ResearcherID: F-8209-2017.</p></bio><bio xml:lang="en"><p>Alena A. Zakharova, Dr. Sci. (Engineering), Chief Scientific Officer, Laboratory of Cybernetic Systems,</p><p>65, Profsoyuznaya str., Moscow 117997.</p><p>ResearcherID: F-8209-2017 </p></bio><email xlink:type="simple">zaawmail@gmail.com</email><xref ref-type="aff" rid="aff-2"/></contrib></contrib-group><aff-alternatives id="aff-1"><aff xml:lang="ru"><institution>Институт проблем управления им. В.А. Трапезникова РАН</institution></aff><aff xml:lang="en"><institution>V.A. Trapeznikov Institute of Control Sciences  of RAS</institution></aff></aff-alternatives><aff-alternatives id="aff-2"><aff xml:lang="ru"><institution>Институт проблем управления  им. В.А. Трапезникова РАН</institution></aff><aff xml:lang="en"><institution>V.A. Trapeznikov Institute of Control Sciences  of RAS</institution></aff></aff-alternatives><pub-date pub-type="collection"><year>2024</year></pub-date><pub-date pub-type="epub"><day>16</day><month>12</month><year>2024</year></pub-date><volume>28</volume><issue>3</issue><fpage>214</fpage><lpage>227</lpage><permissions><copyright-statement>Copyright &amp;#x00A9; Мостаков Н.А., Захарова А.А., 2024</copyright-statement><copyright-year>2024</copyright-year><copyright-holder xml:lang="ru">Мостаков Н.А., Захарова А.А.</copyright-holder><copyright-holder xml:lang="en">Mostakov N.A., Zakharova A.A.</copyright-holder><license xml:lang="ru" license-type="creative-commons-attribution" xlink:href="https://creativecommons.org/licenses/by/4.0/" xlink:type="simple"><license-p>Данная работа распространяется под лицензией Creative Commons Attribution 4.0.</license-p></license><license xml:lang="en" license-type="creative-commons-attribution" xlink:href="https://creativecommons.org/licenses/by/4.0/" xlink:type="simple"><license-p>This work is licensed under a Creative Commons Attribution 4.0 License.</license-p></license></permissions><self-uri xlink:href="https://izvestswsu.elpub.ru/jour/article/view/1337">https://izvestswsu.elpub.ru/jour/article/view/1337</self-uri><abstract><sec><title>Цель исследования</title><p>Цель исследования. Целью работы является исследование и разработка методов локализации сверхлегкого беспилотного летательного аппарата (БПЛА) в насыщенной объектами замкнутой среде, основанных на симантико-топологических данных, получаемых из окружения. Целью работы также явля-ется разработка программного обеспечения и выбор аппаратного комплекса для запуска и опробации разработанного решения. </p></sec><sec><title>Методы</title><p>Методы. Для реализации поставленной цели были проведен обзор и сравнение существующих решений. Оптимизация архитектуры нейронной сети для детектирования объектов. Разработка алгоритма составления графа объектов, отражающего их взаимосвязи. Разработка алгоритма сравнения графов для определения положения БПЛА. Внедрение решения по повышению точности определения геометрического центра задетектированных объектов. Использование методов определения ключевых точек (SIFT, SURF) для решения проблемы идентификации объектов одного класса.</p></sec><sec><title>Результаты</title><p>Результаты. Результатом работы является разработанный метод локализации на основе симантикотопологических данных, получаемых из окружения. Также разработан пакет программного обеспечения, основанный на платформе ROS2 humble, и реализованный на аппаратной части, основанной на плате Rockchip 3588. Эксперименты проводились на готовых наборах данных (KUM dataset) и с использованием БПЛА в помещении. </p></sec><sec><title>Заключение</title><p>Заключение. Разработанная система локализации представляет собой перспективный шаг в направлении создания эффективных и гибких систем, способных работать в сложных условиях. В будущем планируется интегрировать данный метод с другими датчиками для повышения робастности в динамичных условиях, добавить алгоритмы визуальной одометрии для повышения точности локализации БПЛА, и расширить применение системы на БПЛА, используемых в других отраслях (инспекция инфраструктуры, поиск и спасение.</p></sec></abstract><trans-abstract xml:lang="en"><p>The purpose of the work is to study and develop methods for localizing an ultra-light unmanned aerial vehicle (UAV) in a closed environment saturated with objects, based on semantic and topological data obtained from the environment. The purpose of the work is also to develop software and select a hardware complex for launching and testing the developed solution.</p><sec><title>Methods</title><p>Methods. To achieve this goal, a review and comparison of existing solutions were conducted. Optimization of the neural network architecture for detecting objects. Development of an algorithm for compiling a graph of objects reflecting their relationships. Development of an algorithm for comparing graphs to determine the position of the UAV. Implementation of a solution to improve the accuracy of determining the geometric center of detected objects. Use of keypoint detection methods (SIFT, SURF) to solve the problem of identifying objects of the same class.</p></sec><sec><title>Results</title><p>Results. The result of the work is a developed localization method based on semantic and topological data obtained from the environment. A software package based on the ROS2 humble platform and implemented on the hardware based on the Rockchip 3588 board was also developed. The experiments were conducted on ready-made datasets (KUM dataset) and using UAVs indoors.</p></sec><sec><title>Conclusion</title><p>Conclusion. The developed localization system is a promising step towards creating efficient and flexible systems capable of operating in complex conditions. In the future, it is planned to integrate this method with other sensors to improve robustness in dynamic conditions, add visual odometry algorithms to improve the accuracy of UAV localization, and expand the application of the system to UAVs used in other industries (infrastructure inspection, search and rescue).</p></sec></trans-abstract><kwd-group xml:lang="ru"><kwd>компьютерное зрение</kwd><kwd>локализация</kwd><kwd>детектирование объектов</kwd><kwd>построение графов</kwd></kwd-group><kwd-group xml:lang="en"><kwd>computer vision</kwd><kwd>localization</kwd><kwd>object detection</kwd><kwd>graph construction</kwd></kwd-group><funding-group><funding-statement xml:lang="ru">Работа выполнена при поддержке Российского научного фонда, грант № 23-19-00342, https://rscf.ru/en/project/23-19-00342/</funding-statement><funding-statement xml:lang="en">This work was supported by the Russian Science Foundation, grant No. 23-19-00342, https://rscf.ru/ en/project/23-19-00342/.</funding-statement></funding-group></article-meta></front><back><ref-list><title>References</title><ref id="cit1"><label>1</label><citation-alternatives><mixed-citation xml:lang="ru">Evaluating Multispectral Images and Vegetation Indices for Precision Farming Applications from UAV Images / S. Candiago, F. Remondino, M De Giglio, M. Dubbini, M. Gattelli // Remote Sensing. 2015. https://doi.org/10.3390/rs70404026.</mixed-citation><mixed-citation xml:lang="en">Candiago S., Remondino F., De Giglio M., Dubbini M., Gattelli M. Evaluating Multispectral Images and Vegetation Indices for Precision Farming Applications from UAV Images. Remote Sensing. 2015. https://doi.org/10.3390/rs70404026.</mixed-citation></citation-alternatives></ref><ref id="cit2"><label>2</label><citation-alternatives><mixed-citation xml:lang="ru">Duan T., Chapman S.C. Dynamic monitoring of NDVI in wheat agronomy and breeding trials using an unmanned aerial vehicle // Field Crops Research. 2017. URL: https://www.researchgate.net/publication/317423059_Dynamic_monitoring_of_NDVI_in_wheat_agronomy_and_breeding_trials_using_an_unmanned_aerial_vehicle</mixed-citation><mixed-citation xml:lang="en">Duan T., Chapman S.C. Dynamic monitoring of NDVI in wheat agronomy and breeding trials using an unmanned aerial vehicle. Field Crops Research. 2017. Available at:  https://www.researchgate.net/publication/317423059_Dynamic_monitoring_of_NDVI_in_wheat_agronomy_and_breeding_trials_using_an_unmanned_aerial_vehicle</mixed-citation></citation-alternatives></ref><ref id="cit3"><label>3</label><citation-alternatives><mixed-citation xml:lang="ru">Tan Yi., Li S. Automatic inspection data collection of building surface based on BIM and UAV // Automation in Construction. 2021. URL: https://www.sciencedirect.com/science/article/abs/pii/S0926580521003320?via%3Dihub</mixed-citation><mixed-citation xml:lang="en">Tan Yi., Li S. Automatic inspection data collection of building surface based on BIM and UAV. Automation in Construction. 2021. Available at:  https://www.sciencedirect.com/science/article/abs/pii/S0926580521003320?via%3Dihub</mixed-citation></citation-alternatives></ref><ref id="cit4"><label>4</label><citation-alternatives><mixed-citation xml:lang="ru">Mader D., Blaskow R., Westfeld P. Potential of uav-based laser scanner and multispectral camera data in building inspection // The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences. 2016. URL: https://isprsarchives.copernicus.org/articles/XLI-B1/1135/2016/</mixed-citation><mixed-citation xml:lang="en">Mader D., Blaskow R., Westfeld P. Potential of uav-based laser scanner and multispectral camera data in building inspection. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences. 2016. Available at:  https://isprsarchives.copernicus.org/articles/XLI-B1/1135/2016/</mixed-citation></citation-alternatives></ref><ref id="cit5"><label>5</label><citation-alternatives><mixed-citation xml:lang="ru">Ibrahim Motawa, Alexandra Kardakou Unmanned aerial vehicles (UAVs) for inspection in construction and building industry // The 16th International Operation &amp;amp; Maintenance Conference, 2018. URL: https://pure.ulster.ac.uk/en/publications/unmannedaerial-vehicles-uavs-for-inspection-in-construction-and-</mixed-citation><mixed-citation xml:lang="en">Ibrahim Motawa, Alexandra Kardakou. Unmanned aerial vehicles (UAVs) for inspection in construction and building industry. The 16th International Operation &amp;amp; Maintenance Conference. 2018. Available at: https://pure.ulster.ac.uk/en/publications/unmannedaerial-vehicles-uavs-for-inspection-in-construction-and-</mixed-citation></citation-alternatives></ref><ref id="cit6"><label>6</label><citation-alternatives><mixed-citation xml:lang="ru">Motawa I., Kardakou A. Unmanned aerial vehicles (UAVs) for inspection in construction and building industry // The 16th International Operation &amp; Maintenance Conference. 2018. URL: https://pure.ulster.ac.uk/en/publications/unmanned-aerial-vehicles-uavsfor-inspection-in-construction-and-</mixed-citation><mixed-citation xml:lang="en">Motawa I., Kardakou A. Unmanned aerial vehicles (UAVs) for inspection in construction and building industry. The 16th International Operation &amp; Maintenance Conference. 2018. Available at:  https://pure.ulster.ac.uk/en/publications/unmanned-aerial-vehiclesuavs-for-inspection-in-construction-and-</mixed-citation></citation-alternatives></ref><ref id="cit7"><label>7</label><citation-alternatives><mixed-citation xml:lang="ru">Handley D.D., Yu P.L., Wechsler S.P. GPS-Denied UAV Localization Using Satellite Navigation and Inertial Sensors // International Conference on Robotics and Autonomous Systems. 2019. URL: https://www.researchgate.net/publication/335140533_GPS-Denied_UAV_Localization_using_Pre-existing_Satellite_Imagery</mixed-citation><mixed-citation xml:lang="en">Handley D.D., Yu P.L., Wechsler S.P. GPS-Denied UAV Localization Using Satellite Navigation and Inertial Sensors. International Conference on Robotics and Autonomous Systems. 2019. Available at: https://www.researchgate.net/publication/335140533_GPS-Denied_UAV_Localization_using_Pre-existing_Satellite_Imagery</mixed-citation></citation-alternatives></ref><ref id="cit8"><label>8</label><citation-alternatives><mixed-citation xml:lang="ru">Hacohen S., Medina O. Improved GNSS Localization and Byzantine Detection in UAV Swarms // Sensors. 2020. https://doi.org/10.3390/s20247239.</mixed-citation><mixed-citation xml:lang="en">Hacohen S., Medina O. Improved GNSS Localization and Byzantine Detection in UAV Swarms. Sensors. 2020. https://doi.org/10.3390/s20247239.</mixed-citation></citation-alternatives></ref><ref id="cit9"><label>9</label><citation-alternatives><mixed-citation xml:lang="ru">Accurate Localisation Based on GNSS and Propagation Knowledge for Safe Applications in Guided Transport / J. Marais, S. Ambellouis, A. Flancquart, S. Lefebvre, C. Meurie, Y. Ruichek // Procedia - Social and Behavioral Sciences. URL: https://www.sciencedirect.com/science/article/pii/S1877042812027930</mixed-citation><mixed-citation xml:lang="en">Marais J., Ambellouis S., Flancquart A., Lefebvre S., Meurie C., Ruichek Y. Accurate Localisation Based on GNSS and Propagation Knowledge for Safe Applications in Guided Transport. Procedia - Social and Behavioral Sciences. Available at:  https://www.sciencedirect.com/science/article/pii/S1877042812027930</mixed-citation></citation-alternatives></ref><ref id="cit10"><label>10</label><citation-alternatives><mixed-citation xml:lang="ru">Past, present, and future of simultaneous localization and mapping: Toward the robustperception age / C. Cadena, L. Carlone, H. Carrillo, Y. Latif, D. Scaramuzza, J. Neira, I. Reid, J.J. Leonard // IEEE Trans. Robot. 2016. https://doi.org/10.1109/TRO.2016.2528477</mixed-citation><mixed-citation xml:lang="en">Cadena C., Carlone L., Carrillo H., Latif Y., Scaramuzza D., Neira J., Reid I., Leonard J.J. Past, present, and future of simultaneous localization and mapping: Toward the robust-perception age. IEEE Trans. Robot. 2016. https://doi.org/10.1109/TRO.2016.2528477</mixed-citation></citation-alternatives></ref><ref id="cit11"><label>11</label><citation-alternatives><mixed-citation xml:lang="ru">Khairuddin A.R., Haron H. Review on simultaneous localization and mapping // 2015 IEEE International Conference on Control System, Computing and Engineering (ICCSCE). URL: https://ieeexplore.ieee.org/document/7482163</mixed-citation><mixed-citation xml:lang="en">Khairuddin A.R., Haron H. Review on simultaneous localization and mapping. 2015 IEEE International Conference on Control System, Computing and Engineering (ICCSCE). Available at:  https://ieeexplore.ieee.org/document/7482163</mixed-citation></citation-alternatives></ref><ref id="cit12"><label>12</label><citation-alternatives><mixed-citation xml:lang="ru">Sumikura S., Shibuya M., Sakurada K. OpenVSLAM: A versatile visual SLAM framework // Proceedings of the 27th ACM International Conference on Multimedia. 2019.</mixed-citation><mixed-citation xml:lang="en">Sumikura S., Shibuya M., Sakurada K. OpenVSLAM: A versatile visual SLAM framework. Proceedings of the 27th ACM International Conference on Multimedia. 2019.</mixed-citation></citation-alternatives></ref><ref id="cit13"><label>13</label><citation-alternatives><mixed-citation xml:lang="ru">Labb´e M., Michaud F. RTAB-Map as an open-source lidar and visual simultaneous localization and mapping library for large-scale and long-term online operation // Journal of Field Robotics.</mixed-citation><mixed-citation xml:lang="en">Labb´e M., Michaud F. RTAB-Map as an open-source lidar and visual simultaneous localization and mapping library for large-scale and long-term online operation. Journal of Field Robotics.</mixed-citation></citation-alternatives></ref><ref id="cit14"><label>14</label><citation-alternatives><mixed-citation xml:lang="ru">Kinnari J., Verdoja F., Kyrki V. GNSS-denied geolocalization of UAVs by visual matching of onboard camera images with orthophotos // 2021 20th International Conference on Advanced Robotics (ICAR). URL: https://arxiv.org/pdf/2103.14381.pdf</mixed-citation><mixed-citation xml:lang="en">Kinnari J., Verdoja F., Kyrki V. GNSS-denied geolocalization of UAVs by visual matching of onboard camera images with orthophotos. 20th International Conference on Advanced Robotics (ICAR). 2021. Available at:  https://arxiv.org/pdf/2103.14381.pdf</mixed-citation></citation-alternatives></ref><ref id="cit15"><label>15</label><citation-alternatives><mixed-citation xml:lang="ru">Precise Visual-Inertial Localization for UAV with the Aid of A 2D Georeferenced Map. / J. Mao, L. Zhang, X. He, H. Qu, X. Hu // 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). URL: https://arxiv.org/ftp/arxiv/papers/2107/2107.05851.pdf</mixed-citation><mixed-citation xml:lang="en">Mao J., Zhang L., He X., Qu H., Hu X. Precise Visual-Inertial Localization for UAV with the Aid of A 2D Georeferenced Map. 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Available at: https://arxiv.org/ftp/arxiv/papers/2107/2107.05851.pdf</mixed-citation></citation-alternatives></ref><ref id="cit16"><label>16</label><citation-alternatives><mixed-citation xml:lang="ru">Klein L.C., Klein L.C., Klein L.C. A Machine Learning Approach to Robot Localization Using Fiducial Markers in RobotAtFactory 4.0 Competition // Sensors. 2023. URL: https://www.mdpi.com/1424-8220/23/6/3128</mixed-citation><mixed-citation xml:lang="en">Klein L.C., Klein L.C., Klein L.C. A Machine Learning Approach to Robot Localization Using Fiducial Markers in RobotAtFactory 4.0 Competition. Sensors. 2023. Available at:  https://www.mdpi.com/1424-8220/23/6/3128</mixed-citation></citation-alternatives></ref><ref id="cit17"><label>17</label><citation-alternatives><mixed-citation xml:lang="ru">Semantic loop closure detection based on graph matching in multi-objects scenes / Q. Cao, Y. Zhang, Y. Liu, G. Lv // Journal of Visual Communication and Image Representation. 2021. https://doi.org/10.1016/j.jvcir.2021.103072</mixed-citation><mixed-citation xml:lang="en">Cao Q., Zhang Y., Liu Y., Lv G. Semantic loop closure detection based on graph matching in multi-objects scenes. Journal of Visual Communication and Image Representation. 2021. https://doi.org/10.1016/j.jvcir.2021.103072</mixed-citation></citation-alternatives></ref><ref id="cit18"><label>18</label><citation-alternatives><mixed-citation xml:lang="ru">X-View: Graph-Based Semantic Multi-View Localization / A. Gawel, C., Del Don R. Siegwart, J. Nieto, C. Cadena // IEEE Robotics and Automation Letters. 2019. URL: https://ieeexplore.ieee.org/document/8281068</mixed-citation><mixed-citation xml:lang="en">Gawel A., Del Don C., Siegwart R., Nieto J., Cadena C. [X-View: Graph-Based Semantic Multi-View Localization. IEEE Robotics and Automation Letters. 2019. Available at:  https://ieeexplore.ieee.org/document/8281068</mixed-citation></citation-alternatives></ref><ref id="cit19"><label>19</label><citation-alternatives><mixed-citation xml:lang="ru">Object Detection on COCO test-dev Leaderboard. URL:https://paperswithcode.com/sota/object-detection-on-coco</mixed-citation><mixed-citation xml:lang="en">Object Detection on COCO test-dev Leaderboard]. Available at: https://paperswithcode.com/sota/object-detection-on-coco</mixed-citation></citation-alternatives></ref></ref-list><fn-group><fn fn-type="conflict"><p>The authors declare that there are no conflicts of interest present.</p></fn></fn-group></back></article>
