Research Article
BibTex RIS Cite

Konvolüsyonel Sinir Ağı Tabanlı YOLO Yöntemi ile Orta Kulak Otoskop Görüntülerinde Timpanik Membran Bölgesinin Belirlenmesi

Year 2020, Volume: 22 Issue: 66, 919 - 928, 22.09.2020
https://doi.org/10.21205/deufmd.2020226625

Abstract

Orta kulak iltihabından dolayı kulak zarında çeşitli deformasyonlar meydana gelmektedir. Hastalığın teşhis edilebilmesi için otoskop cihazı ile kulağa bakıldığı zaman zar bölgesine erişilmesi ve detaylı bir şekilde kulak zarının incelenmesi gerekmektedir. Son yıllarda derin öğrenme birçok alanda uygulanmış ve oldukça etkili sonuçlar elde edilmiştir. Derin öğrenmenin biyomedikal alanda da sık bir şekilde kullanıldığı ve oldukça iyi neticelere varıldığı bilinmektedir. Otomatik nesne tanımlamada da derin öğrenme tabanlı yöntemler başarılı bir şekilde kullanılmaktadır. Bu çalışmada otoskop cihazı ile elde edilen orta kulak imgelerinde zar bölgesinin otomatik tespiti için derin öğrenme tabanlı nesne algılama yöntemi olan YOLO kullanılmıştır. YOLO yöntemi ile ilgili alanın otomatik olarak tespit edilmesini sağlamak üzere, nesne önerileri için evrişimsel sinir ağı mimarilerinden olan AlexNet, VGGNet, GoogLeNet, ve ResNet ile deneysel çalışmalar yapılmıştır. Performans sonuçlarına göre ResNet ve VGGNet mimarileri ile en verimli sonuçlar elde edilmiştir. YOLO ile zar bölgesinin tespiti %93 başarı oranı ile tespit edildi.

References

  • D. K. Marcia Murphy, “A review of techniques for the investigation of otitis externa and otitis media,” Clin. Tech. Small Anim. Pract., vol. Volume 16, no. Issue 4, p. Pages 236-241.
  • T. A. Valdez et al., “Multi-color reflectance imaging of middle ear pathology in vivo,” Anal. Bioanal. Chem., vol. 407, no. 12, pp. 3277–3283, 2015.
  • H. S. a Thorbjörn Lundberg, Leigh Biagio, Claude Laurent and D. W. Swanepoel, “Remote evaluation of video-otoscopy recordings in an unselected pediatric population with an otitis media scale,” Int. J. Pediatr. Otorhinolaryngol., vol. 78, pp. 1489–1495, 2014.
  • T.-I. J. Yong Bin Ji, Hyeon Sang Barg, Dong Woo Park, Sam Kyu Noh, Seung Jae Oh, “Diagnosis Otitis Media Using teahertz Otoscope.”
  • M. Koçyiğit, S. G. Örtekin, and T. Çakabay, “Otitis Media , Sınıflandırma ve Tedaviye Yaklaşım Prensipleri Otitis Media , Classification and Principles of Treatment Approach,” vol. 8, no. 2, pp. 65–70, 2016.
  • N. Thone, M. Winter, R. J. García-Matte, and C. González, “Simulation in Otolaryngology: A Teaching and Training Tool,” Acta Otorrinolaringol. (English Ed., vol. 68, no. 2, pp. 115–120, 2017.
  • V. Wu and J. A. Beyea, “Evaluation of a Web-Based Module and an Otoscopy Simulator in Teaching Ear Disease,” Otolaryngol. - Head Neck Surg. (United States), vol. 156, no. 2, pp. 272–277, 2017.
  • M. Oyewumi et al., “Objective Evaluation of Otoscopy Skills among Family and Community Medicine, Pediatric, and Otolaryngology Residents,” J. Surg. Educ., vol. 73, no. 1, pp. 129–135, 2016.
  • Y. K. Huang and C. P. Huang, “A depth-first search algorithm based otoscope application for real-time otitis media image interpretation,” Parallel Distrib. Comput. Appl. Technol. PDCAT Proc., vol. 2017-Decem, pp. 170–175, 2018.
  • A. Ş. Ümit Budak, Ömer Faruk Alçin, Muzaffer Aslan, “Optic Disc Detection in Retinal Images via Faster Regional Convolutional Neural Networks,” pp. 3–5, 2018.
  • E. Deniz, A. Sengür, Z. Kadiroğuglu, Y. Guo, V. Bajaj, and Ü. Budak, “Transfer learning based histopathologic image classification for breast cancer detection,” Heal. Inf. Sci. Syst., vol. 6, no. 1, p. 18, 2018.
  • Z. Cömert, A. F. Kocamaz, and V. Subha, “Prognostic model based on image-based time-frequency features and genetic algorithm for fetal hypoxia assessment,” Comput. Biol. Med., vol. 99, pp. 85–97, Aug. 2018.
  • Z. Zhao, Y. Zhang, Z. Comert, and Y. Deng, “Computer-Aided Diagnosis System of Fetal Hypoxia Incorporating Recurrence Plot With Convolutional Neural Network,” Front. Physiol., vol. 10, p. 255, 2019.
  • Z. Cömert and A. F. Kocamaz, “Fetal Hypoxia Detection Based on Deep Convolutional Neural Network with Transfer Learning Approach,” in Software Engineering and Algorithms in Intelligent Systems, 2019, pp. 239–248.
  • Ö. Algur, V. Tümen, and Ö. Yildirim, “Dış Ortam Görüntülerindeki İnsan Hareketlerinin Hibrit Derin Öğrenme Yöntemleri Kullanarak Sınıflandırılması,” vol. 30, no. 3, pp. 121–129, 2018.
  • M. A. Al-antari, M. A. Al-masni, M.-T. Choi, S.-M. Han, and T.-S. Kim, “A fully integrated computer-aided diagnosis system for digital X-ray mammograms via deep learning detection, segmentation, and classification,” Int. J. Med. Inform., vol. 117, pp. 44–54, Sep. 2018.
  • S. Lu, Z. Lu, and Y.-D. Zhang, “Pathological brain detection based on AlexNet and transfer learning,” J. Comput. Sci., vol. 30, pp. 41–47, Jan. 2019.
  • H. Greenspan, M. Frid-Adar, E. Klang, I. Diamant, M. Amitai, and J. Goldberger, “GAN-based synthetic medical image augmentation for increased CNN performance in liver lesion classification,” Neurocomputing, vol. 321. pp. 321–331, 2018.
  • A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet Classification with Deep Convolutional Neural Networks,” in Proceedings of the 25th International Conference on Neural Information Processing Systems - Volume 1, 2012, pp. 1097–1105.
  • K. Simonyan and A. Zisserman, “Very Deep Convolutional Networks for Large-Scale Image Recognition,” Sep. 2014.
  • C. Szegedy et al., “Going deeper with convolutions,” in 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015, pp. 1–9.
  • K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778.
  • J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You Only Look Once: Unified, Real-Time Object Detection.”
  • C. L. ; Y. T. ; J. L. ; K. L. ; Y. Chen, “Object Detection Based on YOLO Network,” in 2018 IEEE 4th Information Technology and Mechatronics Engineering Conference (ITOEC), 2018.
  • Y. Altuntaş, Z. Cömert, and A. F. Kocamaz, “Identification of haploid and diploid maize seeds using convolutional neural networks and a transfer learning approach,” Comput. Electron. Agric., vol. 163, p. 104874, 2019.
  • M. Sarıgül, B. M. Ozyildirim, and M. Avci, “Differential convolutional neural network,” Neural Networks, vol. 116, pp. 279–287, 2019.
  • G. Zhao, G. Liu, L. Fang, B. Tu, and P. Ghamisi, “Multiple convolutional layers fusion framework for hyperspectral image classification,” Neurocomputing, vol. 339, pp. 149–160, Apr. 2019.
  • Y. Fu and C. Aldrich, “Flotation froth image recognition with convolutional neural networks,” Miner. Eng., vol. 132, pp. 183–190, Mar. 2019.
  • S. Wan, Y. Liang, and Y. Zhang, “Deep convolutional neural networks for diabetic retinopathy detection by image classification,” Comput. Electr. Eng., vol. 72, pp. 274–282, Nov. 2018.
  • Y. Seo and K. Shin, “Hierarchical convolutional neural networks for fashion image classification,” Expert Syst. Appl., vol. 116, pp. 328–339, 2019.
  • S. Deepak and P. M. Ameer, “Brain tumor classification using deep CNN features via transfer learning,” Comput. Biol. Med., vol. 111, p. 103345, Aug. 2019.
  • S. Lu, B. Wang, H. Wang, L. Chen, M. Linjian, and X. Zhang, “A real-time object detection algorithm for video,” Comput. Electr. Eng., vol. 77, pp. 398–408, Jul. 2019.
  • Y. Altuntaş, Z. Cömert, and A. F. Kocamaz, “Identification of haploid and diploid maize seeds using convolutional neural networks and a transfer learning approach,” Comput. Electron. Agric., vol. 163, p. 104874, Aug. 2019.
  • A. Soudani and W. Barhoumi, “An image-based segmentation recommender using crowdsourcing and transfer learning for skin lesion extraction,” Expert Syst. Appl., vol. 118, pp. 400–410, Mar. 2019.
  • H. Y. Hulin Kuang, Cairong Liu, Leanne Lai Hang Chan, “Multi-class fruit detection based on image region selection and improved object proposals,” Neurocomputing, vol. 283, no. 241, p. 255, 2018.
  • R. Y. Yoshimasa Kawazoe, Kiminori Shimamoto and H. U. M. F. and K. O. Yukako Shintani-Domoto, “Faster R-CNN-Based Glomerular Detection in Multistained Human Whole Slide Images,” J. Imaging, 2018.
  • N. B. Atakan Körez, “İnsansız Hava Aracı (İHA) Görüntülerindeki Yayaların Faster R-CNN Algoritması ile Otomatik Tespiti,” in 2nd International Symposium on Multidisciplinary Studies and Innovative Technologies (ISMSIT), 2018.

Determination of Tympanic Membrane Region in the Middle Ear Otoscope Images with Convolutional Neural Network Based YOLO Method

Year 2020, Volume: 22 Issue: 66, 919 - 928, 22.09.2020
https://doi.org/10.21205/deufmd.2020226625

Abstract

Due to inflammation of the middle ear, various deformations occur in the eardrum. In order to diagnose the disease, it is necessary to examine the tympanic membrane in detail with an otoscope. In recent years, deep learning has been applied in many areas including biomedical field and very effective results have been achieved. Deep learning based methods are used successfully in automatic object detection. In this study, a deep learning based object detection method namely You Only Look Once (YOLO), is used for automatic detection of tympanic membrane in eardrum images obtained using otoscope device. To enable automatic detection of tympanic membrane by YOLO, experimental studies were conducted with AlexNet, VGGNet, GoogLeNet and ResNet. According to the performance results, the most efficient results were obtained with ResNet and VGGNet architectures. Tympanic membrane region detection with YOLO, was performed with an accuracy rate of 93%.

References

  • D. K. Marcia Murphy, “A review of techniques for the investigation of otitis externa and otitis media,” Clin. Tech. Small Anim. Pract., vol. Volume 16, no. Issue 4, p. Pages 236-241.
  • T. A. Valdez et al., “Multi-color reflectance imaging of middle ear pathology in vivo,” Anal. Bioanal. Chem., vol. 407, no. 12, pp. 3277–3283, 2015.
  • H. S. a Thorbjörn Lundberg, Leigh Biagio, Claude Laurent and D. W. Swanepoel, “Remote evaluation of video-otoscopy recordings in an unselected pediatric population with an otitis media scale,” Int. J. Pediatr. Otorhinolaryngol., vol. 78, pp. 1489–1495, 2014.
  • T.-I. J. Yong Bin Ji, Hyeon Sang Barg, Dong Woo Park, Sam Kyu Noh, Seung Jae Oh, “Diagnosis Otitis Media Using teahertz Otoscope.”
  • M. Koçyiğit, S. G. Örtekin, and T. Çakabay, “Otitis Media , Sınıflandırma ve Tedaviye Yaklaşım Prensipleri Otitis Media , Classification and Principles of Treatment Approach,” vol. 8, no. 2, pp. 65–70, 2016.
  • N. Thone, M. Winter, R. J. García-Matte, and C. González, “Simulation in Otolaryngology: A Teaching and Training Tool,” Acta Otorrinolaringol. (English Ed., vol. 68, no. 2, pp. 115–120, 2017.
  • V. Wu and J. A. Beyea, “Evaluation of a Web-Based Module and an Otoscopy Simulator in Teaching Ear Disease,” Otolaryngol. - Head Neck Surg. (United States), vol. 156, no. 2, pp. 272–277, 2017.
  • M. Oyewumi et al., “Objective Evaluation of Otoscopy Skills among Family and Community Medicine, Pediatric, and Otolaryngology Residents,” J. Surg. Educ., vol. 73, no. 1, pp. 129–135, 2016.
  • Y. K. Huang and C. P. Huang, “A depth-first search algorithm based otoscope application for real-time otitis media image interpretation,” Parallel Distrib. Comput. Appl. Technol. PDCAT Proc., vol. 2017-Decem, pp. 170–175, 2018.
  • A. Ş. Ümit Budak, Ömer Faruk Alçin, Muzaffer Aslan, “Optic Disc Detection in Retinal Images via Faster Regional Convolutional Neural Networks,” pp. 3–5, 2018.
  • E. Deniz, A. Sengür, Z. Kadiroğuglu, Y. Guo, V. Bajaj, and Ü. Budak, “Transfer learning based histopathologic image classification for breast cancer detection,” Heal. Inf. Sci. Syst., vol. 6, no. 1, p. 18, 2018.
  • Z. Cömert, A. F. Kocamaz, and V. Subha, “Prognostic model based on image-based time-frequency features and genetic algorithm for fetal hypoxia assessment,” Comput. Biol. Med., vol. 99, pp. 85–97, Aug. 2018.
  • Z. Zhao, Y. Zhang, Z. Comert, and Y. Deng, “Computer-Aided Diagnosis System of Fetal Hypoxia Incorporating Recurrence Plot With Convolutional Neural Network,” Front. Physiol., vol. 10, p. 255, 2019.
  • Z. Cömert and A. F. Kocamaz, “Fetal Hypoxia Detection Based on Deep Convolutional Neural Network with Transfer Learning Approach,” in Software Engineering and Algorithms in Intelligent Systems, 2019, pp. 239–248.
  • Ö. Algur, V. Tümen, and Ö. Yildirim, “Dış Ortam Görüntülerindeki İnsan Hareketlerinin Hibrit Derin Öğrenme Yöntemleri Kullanarak Sınıflandırılması,” vol. 30, no. 3, pp. 121–129, 2018.
  • M. A. Al-antari, M. A. Al-masni, M.-T. Choi, S.-M. Han, and T.-S. Kim, “A fully integrated computer-aided diagnosis system for digital X-ray mammograms via deep learning detection, segmentation, and classification,” Int. J. Med. Inform., vol. 117, pp. 44–54, Sep. 2018.
  • S. Lu, Z. Lu, and Y.-D. Zhang, “Pathological brain detection based on AlexNet and transfer learning,” J. Comput. Sci., vol. 30, pp. 41–47, Jan. 2019.
  • H. Greenspan, M. Frid-Adar, E. Klang, I. Diamant, M. Amitai, and J. Goldberger, “GAN-based synthetic medical image augmentation for increased CNN performance in liver lesion classification,” Neurocomputing, vol. 321. pp. 321–331, 2018.
  • A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet Classification with Deep Convolutional Neural Networks,” in Proceedings of the 25th International Conference on Neural Information Processing Systems - Volume 1, 2012, pp. 1097–1105.
  • K. Simonyan and A. Zisserman, “Very Deep Convolutional Networks for Large-Scale Image Recognition,” Sep. 2014.
  • C. Szegedy et al., “Going deeper with convolutions,” in 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015, pp. 1–9.
  • K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778.
  • J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You Only Look Once: Unified, Real-Time Object Detection.”
  • C. L. ; Y. T. ; J. L. ; K. L. ; Y. Chen, “Object Detection Based on YOLO Network,” in 2018 IEEE 4th Information Technology and Mechatronics Engineering Conference (ITOEC), 2018.
  • Y. Altuntaş, Z. Cömert, and A. F. Kocamaz, “Identification of haploid and diploid maize seeds using convolutional neural networks and a transfer learning approach,” Comput. Electron. Agric., vol. 163, p. 104874, 2019.
  • M. Sarıgül, B. M. Ozyildirim, and M. Avci, “Differential convolutional neural network,” Neural Networks, vol. 116, pp. 279–287, 2019.
  • G. Zhao, G. Liu, L. Fang, B. Tu, and P. Ghamisi, “Multiple convolutional layers fusion framework for hyperspectral image classification,” Neurocomputing, vol. 339, pp. 149–160, Apr. 2019.
  • Y. Fu and C. Aldrich, “Flotation froth image recognition with convolutional neural networks,” Miner. Eng., vol. 132, pp. 183–190, Mar. 2019.
  • S. Wan, Y. Liang, and Y. Zhang, “Deep convolutional neural networks for diabetic retinopathy detection by image classification,” Comput. Electr. Eng., vol. 72, pp. 274–282, Nov. 2018.
  • Y. Seo and K. Shin, “Hierarchical convolutional neural networks for fashion image classification,” Expert Syst. Appl., vol. 116, pp. 328–339, 2019.
  • S. Deepak and P. M. Ameer, “Brain tumor classification using deep CNN features via transfer learning,” Comput. Biol. Med., vol. 111, p. 103345, Aug. 2019.
  • S. Lu, B. Wang, H. Wang, L. Chen, M. Linjian, and X. Zhang, “A real-time object detection algorithm for video,” Comput. Electr. Eng., vol. 77, pp. 398–408, Jul. 2019.
  • Y. Altuntaş, Z. Cömert, and A. F. Kocamaz, “Identification of haploid and diploid maize seeds using convolutional neural networks and a transfer learning approach,” Comput. Electron. Agric., vol. 163, p. 104874, Aug. 2019.
  • A. Soudani and W. Barhoumi, “An image-based segmentation recommender using crowdsourcing and transfer learning for skin lesion extraction,” Expert Syst. Appl., vol. 118, pp. 400–410, Mar. 2019.
  • H. Y. Hulin Kuang, Cairong Liu, Leanne Lai Hang Chan, “Multi-class fruit detection based on image region selection and improved object proposals,” Neurocomputing, vol. 283, no. 241, p. 255, 2018.
  • R. Y. Yoshimasa Kawazoe, Kiminori Shimamoto and H. U. M. F. and K. O. Yukako Shintani-Domoto, “Faster R-CNN-Based Glomerular Detection in Multistained Human Whole Slide Images,” J. Imaging, 2018.
  • N. B. Atakan Körez, “İnsansız Hava Aracı (İHA) Görüntülerindeki Yayaların Faster R-CNN Algoritması ile Otomatik Tespiti,” in 2nd International Symposium on Multidisciplinary Studies and Innovative Technologies (ISMSIT), 2018.
There are 37 citations in total.

Details

Primary Language English
Subjects Engineering
Journal Section Articles
Authors

Erdal Başaran 0000-0001-8569-2998

Zafer Cömert This is me 0000-0001-5256-7648

Yüksel Çelik 0000-0002-7117-9736

Subha Velappan This is me 0000-0002-4992-4090

Mesut Toğaçar 0000-0002-8264-3899

Publication Date September 22, 2020
Published in Issue Year 2020 Volume: 22 Issue: 66

Cite

APA Başaran, E., Cömert, Z., Çelik, Y., Velappan, S., et al. (2020). Determination of Tympanic Membrane Region in the Middle Ear Otoscope Images with Convolutional Neural Network Based YOLO Method. Dokuz Eylül Üniversitesi Mühendislik Fakültesi Fen Ve Mühendislik Dergisi, 22(66), 919-928. https://doi.org/10.21205/deufmd.2020226625
AMA Başaran E, Cömert Z, Çelik Y, Velappan S, Toğaçar M. Determination of Tympanic Membrane Region in the Middle Ear Otoscope Images with Convolutional Neural Network Based YOLO Method. DEUFMD. September 2020;22(66):919-928. doi:10.21205/deufmd.2020226625
Chicago Başaran, Erdal, Zafer Cömert, Yüksel Çelik, Subha Velappan, and Mesut Toğaçar. “Determination of Tympanic Membrane Region in the Middle Ear Otoscope Images With Convolutional Neural Network Based YOLO Method”. Dokuz Eylül Üniversitesi Mühendislik Fakültesi Fen Ve Mühendislik Dergisi 22, no. 66 (September 2020): 919-28. https://doi.org/10.21205/deufmd.2020226625.
EndNote Başaran E, Cömert Z, Çelik Y, Velappan S, Toğaçar M (September 1, 2020) Determination of Tympanic Membrane Region in the Middle Ear Otoscope Images with Convolutional Neural Network Based YOLO Method. Dokuz Eylül Üniversitesi Mühendislik Fakültesi Fen ve Mühendislik Dergisi 22 66 919–928.
IEEE E. Başaran, Z. Cömert, Y. Çelik, S. Velappan, and M. Toğaçar, “Determination of Tympanic Membrane Region in the Middle Ear Otoscope Images with Convolutional Neural Network Based YOLO Method”, DEUFMD, vol. 22, no. 66, pp. 919–928, 2020, doi: 10.21205/deufmd.2020226625.
ISNAD Başaran, Erdal et al. “Determination of Tympanic Membrane Region in the Middle Ear Otoscope Images With Convolutional Neural Network Based YOLO Method”. Dokuz Eylül Üniversitesi Mühendislik Fakültesi Fen ve Mühendislik Dergisi 22/66 (September 2020), 919-928. https://doi.org/10.21205/deufmd.2020226625.
JAMA Başaran E, Cömert Z, Çelik Y, Velappan S, Toğaçar M. Determination of Tympanic Membrane Region in the Middle Ear Otoscope Images with Convolutional Neural Network Based YOLO Method. DEUFMD. 2020;22:919–928.
MLA Başaran, Erdal et al. “Determination of Tympanic Membrane Region in the Middle Ear Otoscope Images With Convolutional Neural Network Based YOLO Method”. Dokuz Eylül Üniversitesi Mühendislik Fakültesi Fen Ve Mühendislik Dergisi, vol. 22, no. 66, 2020, pp. 919-28, doi:10.21205/deufmd.2020226625.
Vancouver Başaran E, Cömert Z, Çelik Y, Velappan S, Toğaçar M. Determination of Tympanic Membrane Region in the Middle Ear Otoscope Images with Convolutional Neural Network Based YOLO Method. DEUFMD. 2020;22(66):919-28.

Dokuz Eylül Üniversitesi, Mühendislik Fakültesi Dekanlığı Tınaztepe Yerleşkesi, Adatepe Mah. Doğuş Cad. No: 207-I / 35390 Buca-İZMİR.