About Biomimetics Aims Biomimetics (ISSN 2313-7673) is an open access journal regarding biomimicry and bionics, dedicated to research that relates to the most basic aspects of living organisms and the transfer of their properties to human applications. The journal aims to provide a forum and a survey for researchers and professionals in the fields of materials science, mechanical engineering, nanotechnology and biomedicine interested in exploiting biologically inspired designs in engineering systems, technology and biomedicine aimed to developing novel solutions that enable sustainable innovation. Scope Biomimetics invites submissions on a wide range of topics, including but not limited to: • Biomimetic mechanism and design • Biomimetic robotics • Biofabrication and characterization • Biomimetic and bioinspired chemistry • Biosensing • Nanotribology, nanomechanics, micro/nanoscale studies • Plant biomechanics • Synthetic systems • Self-organization and cooperative behavior • Tissue engineering • Bioinspired, biomedical and biomolecular materials Copyright / Open Access Articles published in Biomimetics will be Open-Access articles distributed under the terms and conditions of the Creative Commons Attribution License (CC BY). The copyright is retained by the author(s). MDPI will insert the following note at the end of the published text: © 2019 by the authors; licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/). Editorial Office Journal Contact Biomimetics Editorial Office MDPI, St. Alban-Anlage 66, 4052 Basel, Switzerland biomimetics@mdpi.com Tel. +41 61 683 77 34; Fax: +41 61 302 89 18 Ms. Ashlynn Wang Managing Editor ashlynn.wang@mdpi.com Dr. Katniss Wang Managing Editor katniss.wang@mdpi.com Ms. Delia Mihaila Publishing Manager mihaila@mdpi.com Editorial Board Editor Prof. Josep Samitier Editor-in-Chief 1. Nanobioengineering group, Institute for Bioengineering of Catalonia (IBEC), Baldiri Reixac 10-12, 08028 Barcelona, Spain 2. Department of Engineering: Electronics, Universitat de Barcelona, Barcelona, Spain 3. Networking Biomedical Research Center (CIBER), Spain Interests: Surface functionalization; Engineering cell–material interface; Biosensors and lab on a chips; Microfluidics; 3D Bioprinting and 3D Cell culture; Organ-on-a-chip engineering mailto:mihaila@mdpi.com mailto:katniss.wang@mdpi.com mailto:ashlynn.wang@mdpi.com mailto:biomimetics@mdpi.com Editorial Board Members (36) Dr. Rosalyn Abbott Biomedical Engineering Department, Carnegie Mellon University, Pittsburgh, Pennsylvania, USA Interests: silk biomaterials; adipose tissue; tissue engineering; disease models; tissue regeneration; bioreactors Prof. Andrew Adamatzky Director of the Unconventional Computing Centre, UWE, Bristol BS16 1QY, UK Interests: Reaction-diffusion computing; Cellular automata; Physarum computing; Massive parallel computation; Applied mathematics; Collective intelligence and robotics. Special Issues and Collections in MDPI journals: Special Issue in Biomimetics : Proto-Architecture and Unconventional Biomaterials Prof. Dr. Nurit Ashkenasy Department of Materials Engineering, Ben Gurion University of the Negev, P.O.B. 653, Beer-Sheva 84105, Israel Interests: charge transport through peptide molecular junctions; Electron and proton transport in self- assembled peptide nanostructures; Peptide templates for the formation of inorganic materials and electronic devices; the electronic behavior of peptide- inorganic interface; nanopore fabrication and use in sensing applications; Field effect transistor biosensors Dr. Vipul Bansal Ian Potter NanoBioSensing Facility, NanoBiotechnology Research Laboratory, School of Applied Sciences, RMIT University, GPO Box 2476, Melbourne VIC 3001, Australia Interests: Biological and biomimetic synthesis of functional nanomaterials; Ionic liquids mediated synthesis and self-assembly of functional nano(bio)materials; Multifunctional nanomaterials; Structure-function relationship of nanomaterials and composites; Applications of nanomaterials in biosensing, bioimaging, drug-delivery, antimicrobials, wound-healing, (photo)catalysis and flexible electronics Dr. Christopher J. Bettinger Departments of Materials Science and Engineering and Biomedical Engineering, Carnegie Mellon University, 5000 Forbes Avenue, Wean Hall 3325, Pittsburgh, PA 15213, USA Interests: smart biomaterials; medical devices; bioelectronics; polymers; brain–machine interfaces Prof. Dr. Bharat Bhushan Nanoprobe Laboratory for Bio- & Nanotechnology and Biomimetics (NLB2), Ohio State University, Columbus, OH 43210-1142, USA Interests: nanotechnology; bio/nanotechnology, biomimetics, bioinspired surfaces, nanotribology; AFM-based data recording; beauty care; aging in Li-ion batteries https://www.mdpi.com/journal/biomimetics/special_issues/proto-architecture Prof. Dr. Giuseppe Carbone Dipartimento di Meccanica-Matematica-Management DMMM, Campus, Via Orabona 4, 70125 Bari, Italy Interests: contact mechanics; tribology; wetting and interfaces; applied computational mathematics; automotive systems engineerin; rheology of materials Special Issues and Collections in MDPI journals: Special Issue in Biomimetics : Micro- and Nano-Structured Bio-Inspired Surfaces Special Issue in Applied Sciences : Viscoelastic Solids: Mechanical Behaviour, Contact Mechanics, Fracture and Wear Special Issue in Lubricants : Adhesion, Friction and Lubrication of Viscoelastic Materials Special Issue in Coatings : Anti-Adhesive Surfaces Dr. Federico Carpi Dipartimento di Ingegneria Industriale, Università degli Studi di Firenze, Via di S. Marta 3, 50139 Firenze, Italy Interests: biomimetics, smart materials, electrical and magnetic systems Dr. Jose Luis Chiara Department of Bioorganic Chemistry, Institute of General Organic Chemistry, Spanish National Research Council (CSIC), Juan de la Cierva 3, 28006 Madrid, Spain Interests: molecular hybrid nanoparticles; new methods and strategies for organic synthesis; design, synthesis, and evaluation of glycomimetic compounds Dr. Antonio Concilio Department of Adaptive Structures, Centro Italiano Ricerche Aerospaziali; 81043 Capua (CE), Italy Interests: shape memory alloys; piezoelectrics; magnetorehological fluids; shape memory polymers; fibre optics; smart materials; adaptive structures; morphing structures; deployable structures; smart structures; distributed actuator systems; distributed sensor networks; adaptive wings; smart landing gear; de-icing electromechanical systems; noise and vibration control; anti-seismic systems; acoustic antennas; structural health monitoring Special Issues and Collections in MDPI journals: Special Issue in Biomimetics : Morphing Aircraft Systems Special Issue in Actuators : Shape Memory Alloys Actuators https://www.mdpi.com/journal/actuators/special_issues/SMA_actuators https://www.mdpi.com/journal/biomimetics/special_issues/morphing_aircraft https://www.mdpi.com/journal/coatings/special_issues/anti-adhesive_surfaces https://www.mdpi.com/journal/lubricants/special_issues/viscoelastic_materials https://www.mdpi.com/journal/applsci/special_issues/contact_mechanics https://www.mdpi.com/journal/applsci/special_issues/contact_mechanics https://www.mdpi.com/journal/biomimetics/special_issues/bio_inspired_surfaces Prof. Dr. Marco d'Ischia Department of Chemical Sciences, University of Naples Federico II, Via Cintia 4, I-80126 Naples, Italy Interests: structure, synthesis, physicochemical properties, and reactivity of melanins; polydopamine and related bioinspired functional materials for underwater surface functionalization and hybrid nanostructures for bioelectronics and biomedical applications; design, antioxidant properties, and reactivity of bioactive phenolic and quinone compounds; free radical oxidations and nature-inspired redox-active systems for biomedical and technological applications; chemistry and physicochemical properties of natural or bioinspired heterocyclic compounds; bioorganic chemistry of organic sulphur and selenium compounds; model reactions and transformation pathways of polycyclic aromatic hydrocarbons and derivatives of astrochemical relevance Special Issues and Collections in MDPI journals: Special Issue in Biomimetics : Bioinspired Catechol-Based Systems: Chemistry and Applications Special Issue in International Journal of Molecular Sciences : Melanin Based Functional Materials Special Issue in Biomimetics : Selected Papers from NanoTech Poland 2018 and 1st Symposium on Polydopamine Special Issue in Biomimetics : Prebiotic Processes: Systems and Theories Topical Collection in International Journal of Molecular Sciences : Feature Papers in Materials Science Special Issue in International Journal of Molecular Sciences : Wet Adhesion: New Chemistries, Models and Translation to Materials Prof. Dr. Ir. J.M.J. (Jaap) Den Toonder Eindhoven University of Technology, P.O. Box 513, 5600 MB Eindhoven, The Netherlands Interests: micro-electro-mechanical systems; microfluidics; organ-on-chip; biomimetics Special Issues and Collections in MDPI journals: Special Issue in Biomimetics : Bioinspired Microfluidics Prof. Dr. Hermann Ehrlich Institute of Electronic and Sensor materials, TU Bergakademie Freiberg, Gustav-Zeuner Str. 3, 09599 Freiberg, Germany Tel. +00493731 392867; Fax: +0049 3731 394314 Interests: marine biomaterials; biominerals; biocomposites and biomimetics Special Issues and Collections in MDPI journals: Special Issue in Marine Drugs : Marine Biomaterials Special Issue in International Journal of Molecular Sciences : Frontiers of Marine Biomaterials Special Issue in Biomimetics : Extreme Biomimetics Special Issue in Marine Drugs : Marine Biomaterials II, 2017 Special Issue in Marine Drugs : Marine Biomaterials 2020 https://www.mdpi.com/journal/marinedrugs/special_issues/marinebiomaterials2020 https://www.mdpi.com/journal/marinedrugs/special_issues/marine_biomaterials_17 https://www.mdpi.com/journal/biomimetics/special_issues/Extreme_Biomimetics https://www.mdpi.com/journal/ijms/special_issues/frontiers_marine_biomaterials https://www.mdpi.com/journal/marinedrugs/special_issues/marine_biomaterials https://www.mdpi.com/journal/biomimetics/special_issues/bioinspired_microfluidics https://www.mdpi.com/journal/ijms/special_issues/wet_adhesion https://www.mdpi.com/journal/ijms/special_issues/wet_adhesion https://www.mdpi.com/journal/ijms/special_issues/featurepapers_materials https://www.mdpi.com/journal/biomimetics/special_issues/prebiotic_processes https://www.mdpi.com/journal/biomimetics/special_issues/nanotech_2018 https://www.mdpi.com/journal/biomimetics/special_issues/nanotech_2018 https://www.mdpi.com/journal/ijms/special_issues/melanin_materials https://www.mdpi.com/journal/biomimetics/special_issues/catechol Prof. Joel R. Fried J.B. Speed School of Engineering, University of Louisville, Louisville, KY 40292, USA Interests: membrane separation and transport; computational chemistry; molecular simulations; polymer blends and composites; biomimetic membranes; proton transfer in fuel cell membranes; ion and small molecule transport through membrane proteins Prof. Dr. Ille C. Gebeshuber Institute of Applied Physics (IAP), Vienna University of Technology (TU Wien), Wiedner Hauptstrasse 8-10/134, 1040 Vienna, Austria Tel. + 43 (0)1 58801 13483; Fax: +43 (0)1 58801 13499 Interests: tribology; nanotribology; green technology; positive technologies; systems approaches; complex systems Special Issues and Collections in MDPI journals: Special Issue in Lubricants : Friction and Lubricants Related to Human Bodies Special Issue in Biomimetics : Biomimetic Nanotechnology Special Issue in Lubricants : Green Nanotribology Prof. Dr. Stanislav N. Gorb Department Functional Morphology and Biomechanics, Zoological Institute of the University of Kiel, Kiel, Germany Interests: biological attachment; functional morphology and biomechanics; evolution of structure and functions; behaviour of arthropods, animal-plant interactions Dr. Juan P. Hinestroza Department of Fiber Science and Apparel Design, College of Human Ecology, Cornell University, Ithaca, NY 14853-4401, USA Tel. +1-607-255-7600 Interests: textiles; nanotechnology; nanofibers Special Issues and Collections in MDPI journals: Special Issue in Nanomaterials : Textiles Nanotechnology Prof. Dr. Zoran Jaksic Centre of Microelectronic Technologies, Institute of Chemistry, Technology and Metallurgy, University of Belgrade, Njegoseva 12, 11000 Belgrade, Serbia Interests: Mesoscopic and subwavelength electromagnetic optics / nanooptics and nanophotonics; Plasmonics; Nanophotonic and nanoplasmonic chemical and biological sensors; Optical metamaterials; Photonic band gap materials; Advanced solid-state sensors, optical sensors; Infrared photodetectors, including Auger-suppressed nonequilibrium devices; Narrow-bandgap semiconductors; Nanomembranes https://www.mdpi.com/journal/nanomaterials/special_issues/textile_nano https://www.mdpi.com/journal/lubricants/special_issues/green_nanotribology https://www.mdpi.com/journal/biomimetics/special_issues/biomimetic_nanotechnology https://www.mdpi.com/journal/lubricants/special_issues/Human-Bodies Prof. Dr. Tatsuo Kaneko Japan Advanced Institute of Science and Technology, Nomi, Japan Interests: high-performance polymers; Bio-based polymers; Liquid crystals; Gels; Biomolecules; Synthetic chemistry Special Issues and Collections in MDPI journals: Special Issue in Polymers : Aromatic Polymers Prof. Dr. Anne Kietzig Department of Chemical Engineering, McGill University, 3610 University Street, Montreal, QC H3A 0C5, Canada Interests: femtosecond laser micromachining; wetting; adhesion; friction; flow; icing; drop impact; structural colour Prof. Dr. Gijs J.M. Krijnen MESA+ Research Institute, University of Twente, 7500 AE Enschede, The Netherlands Fax: +31 53 489 2735 Interests: (biomimetic) transducers; nano-, micro-technology and additive manufacturing; nonlinear transduction. Special Issues and Collections in MDPI journals: Special Issue in Micromachines : Biomimetic Systems Prof. Haeshin Lee Department of Chemistry, KAIST, Daejeon, South Korea Interests: interface; adhesion; bio-inspired materials; polydopamine Special Issues and Collections in MDPI journals: Special Issue in Biomimetics : Selected Papers from NanoTech Poland 2018 and 1st Symposium on Polydopamine Prof. Dr. Guoqiang Li Department of Mechanical Engineering, Louisiana State University, Baton Rouge, LA 70803, USA Interests: advanced grid stiffened composites; composite joints; infrastructure composites; impact; mechanics of composite materials; particulate-filled composites; repair of composite structures; shape memory polymer; solid mechanics; smart self-healing composites https://www.mdpi.com/journal/biomimetics/special_issues/nanotech_2018 https://www.mdpi.com/journal/biomimetics/special_issues/nanotech_2018 https://www.mdpi.com/journal/micromachines/special_issues/biomimetic https://www.mdpi.com/journal/polymers/special_issues/aromatic_polymers Prof. Dr. João F. Mano Department of Chemistry, CICECO - Aveiro Institute of Materials, University of Aveiro, 3810-193 Aveiro, Portugal Tel. +351 234 370 733; Fax: +351 253 510 909 Interests: biomaterials; tissue engineering; controlled delivery of bioactive molecules; natural-based biodegradable polymers; biomimetic and nano/micro-technology approaches Special Issues and Collections in MDPI journals: Special Issue in Polymers : Biomimetic Polymers Special Issue in Polymers : Advance of Polymers Applied to Biomedical Applications: Cell Scaffolds Prof. Dr. Abraham Marmur Albert and Anne Mansfield Chair in Water Science and Technology, Chemical Engineering Department, Technion - Israel Institute of Technology, Haifa 32000, Israel Interests: contact angle theory and measurement; super-hydrophobic surfaces; nano-bubbles on surfaces; foam stability; thermodynamics of non-ideal solutions; equations of state Dr. Barbara Mazzolai Center for Micro-BioRobotics, Istituto Italiano di Tecnologia, Pontedera, Italy Tel. + 39 050 883444 Interests: plant-inspired robotics, self-growing robots, soft robotics, biomimetics, robotics for biology, variable stiffness soft actuators, plant-hybrid energy Special Issues and Collections in MDPI journals: Special Issue in Biomimetics : Soft Robotics Special Issue in Robotics : Soft Perceptive Robots Prof. Dr. Tony Prescott Director of Sheffield Robotics, The Adaptive Behaviour Research Group (ABRG), Department of Psychology, The University of Sheffield, Western Bank, Sheffield S10 2TN, UK Interests: active sensing; computational models of natural action selection; layered control architecture of the vertebrate brain; robot models of biological control systems; assistive and industrial robots; societal impacts of robotics. Dr. Bernhard Schuster Institute for Synthetic Bioarchitectures, Department of NanoBiotechnology, University of Natural Resources and Life Sciences, Muthgasse 11, 1190 Vienna, Austria Interests: supported lipid membranes; nanobiotechnology; membrane protein based biosensors Special Issues and Collections in MDPI journals: Special Issue in International Journal of Molecular Sciences : Membrane Protein Based Biosensors Special Issue in Membranes : Supported Lipid Membranes Special Issue in International Journal of Molecular Sciences : Membrane Protein Based Biosensors 2016 https://www.mdpi.com/journal/ijms/special_issues/based_biosensors_2016 https://www.mdpi.com/journal/ijms/special_issues/based_biosensors_2016 https://www.mdpi.com/journal/membranes/special_issues/lipid-membranes-2016 https://www.mdpi.com/journal/ijms/special_issues/based-biosensors https://www.mdpi.com/journal/robotics/special_issues/SPR https://www.mdpi.com/journal/biomimetics/special_issues/soft_robotics https://www.mdpi.com/journal/polymers/special_issues/polymer_cell_scaffolds https://www.mdpi.com/journal/polymers/special_issues/biomimetic-polymers Prof. Dr. Thomas Speck Plant Biomechanics Group Freiburg, Botanic Garden, Faculty of Biology, University of Freiburg, Schänzlestrasse 1, D-79104 Freiburg, Germany Interests: biomimetics; biomechanics and functional morphology of plants; evolution of growth forms and other functional parameters in plants; early evolution of land-plants; eco-biomechanics of plants in tropical rainforests; movements of plants and plant organs; functional aspects of pollination biology; movement at low Reynolds-numbers Prof. André R. Studart ETH Zurich, Department of Materials, Vladimir-Prelog-Weg 5 HCI G539, 8093 Zurich, Switzerland Interests: bio-inspired complex materials, medical implants, energy conversion systems, smart structures Prof. Dr. Bo Su Bristol Dental School, University of Bristol, Bristol BS1 2LY, UK Interests: biomaterials; biomimetic materials; bio-inspired materials; cell-instructive surfaces; ceramics; composites; micro/nanofabrication; surface patterning; nanotopography; implants; tissue engineering; scaffolds; bacteria; stem cells Prof. Dr. Candan Tamerler Department of Mechanical Engineering and Institute for Bioengineering Research，University of Kansas, 1530 W 15th St Learned Hall Lawrence, KS-66045, USA Interests: Bio-Nano Interfaces, Bio-nanotechnology, Surfaces, Biomaterials, Tissue Engineering, Nano-Biosensors, Biocatalysis Molecular Biomimetics, Bioengineering Prof. Dr. Andreas Taubert Institute of Chemistry, University of Potsdam, Building 26, Rm. 2.64, Karl-Liebknecht-Str. 24-25, D- 14476 Golm, Germany Tel. 0049 (0)331 977 5773; Fax: +49 331 977 5055 Interests: inorganic materials synthesis in ionic liquids; functional ionic liquids-hybrid materials; ionogels; biomimetic materials; hybrid materials; calcium phosphate; silica Special Issues and Collections in MDPI journals: Special Issue in International Journal of Molecular Sciences : Energy Technology for the 21st Century - Materials and Devices Special Issue in International Journal of Molecular Sciences : Functional Materials- From Functional Hybrid Materials to Functional Polymers Special Issue in Materials : Energy Technology for the 21st Century - Materials and Devices Special Issue in Materials : Advances in Materials Science Special Issue in International Journal of Molecular Sciences : Metal Containing Ionic Liquids Special Issue in International Journal of Molecular Sciences : Ionic Liquids 2014 & Selected Papers from ILMAT 2013 Special Issue in Inorganics : Polymer Controlled and Bio-inspired Mineralization of Inorganic Compounds Special Issue in International Journal of Molecular Sciences : Ionic Liquids 2016 and Selected Papers from ILMAT III Special Issue in International Journal of Molecular Sciences : Ionic Liquids 2018 and Selected Papers from ILMAT IV Prof. Oommen P. Varghese Department of Chemistry, The Ångström Laboratory, Uppsala University, Box 538, SE-75121 Uppsala, Sweden Interests: Tissue engineering; Hydrogels; Modified nucleic acids; Gene delivery; siRNA delivery; Drug delivery. Dr. Paul Verschure 1. Laboratory of Synthetic Perceptive, Emotive and Cognitive Systems, Center of Autonomous Systems and Neurorobotics, Pompeu Fabra University, Roc Boronat 138, 08018 Barcelona, Spain 2. ICREA, Institució Catalana de Recerca i Estudis Avançats, Passeig Lluís Companys 23, 08010 Barcelona, Spain Interests: Perception; Cognition; Behavior; Robotics Dr. Silvia Vignolini Department of Chemistry, University of Cambridge, Lensfield Road, Cambridge, CB2 1EW, UK Interests: biomimetic materials; bio-inspired photonics https://www.mdpi.com/journal/ijms/special_issues/ionic-liquids-2018 https://www.mdpi.com/journal/ijms/special_issues/ionic-liquids-2018 https://www.mdpi.com/journal/ijms/special_issues/ionic-liquids-2016 https://www.mdpi.com/journal/ijms/special_issues/ionic-liquids-2016 https://www.mdpi.com/journal/inorganics/special_issues/mineralization_inorganic_compounds https://www.mdpi.com/journal/inorganics/special_issues/mineralization_inorganic_compounds https://www.mdpi.com/journal/ijms/special_issues/ionic-liquids-2014 https://www.mdpi.com/journal/ijms/special_issues/ionic-liquids-2014 https://www.mdpi.com/journal/ijms/special_issues/metal-ion-liquids https://www.mdpi.com/journal/materials/special_issues/materials-science https://www.mdpi.com/journal/materials/special_issues/energy-technology-21st-materials https://www.mdpi.com/journal/ijms/special_issues/functional_materials https://www.mdpi.com/journal/ijms/special_issues/functional_materials https://www.mdpi.com/journal/ijms/special_issues/energy_technology_21st https://www.mdpi.com/journal/ijms/special_issues/energy_technology_21st Biomimetics — Indexing & Archiving Biomimetics is covered by following databases and archives: Indexing & Abstracting Services • Chemical Abstracts (ACS) • DOAJ - Directory of Open Access Journals • Emerging Sources Citation Index - Web of Science (Clarivate Analytics) • PubMed (NLM) • Web of Science (Clarivate Analytics) • Zetoc (British Library) Full-text Archives • CLOCKSS (Digital Archive) • e-Helvetica (Swiss National Library Digital Archive) • PubMed Central (NLM) Content Aggregators • Academic OneFile (Gale/Cengage Learning) • J-Gate (Informatics India) • ProQuest Central (ProQuest) • Science In Context (Gale/Cengage Learning) • WorldCat (OCLC) Table of Contents Biomimetics, Volume 5, Issue 1 (March 2020) • Gutiérrez-Muñoz, M.; González-Salazar, A.; Coto-Jiménez, M. Evaluation of Mixed Deep Neural Networks for Reverberant Speech Enhancement. Biomimetics 2020, 5(1), 1; https://doi.org/10.3390/biomimetics5010001. https://www.mdpi.com/2313-7673/5/1/1 • Ozalp, M.; Miller, L.; Dombrowski, T.; Braye, M.; Dix, T.; Pongracz, L.; Howell, R.; Klotsa, D.; Pasour, V.; Strickland, C. Experiments and Agent Based Models of Zooplankton Movement within Complex Flow Environments. Biomimetics 2020, 5(1), 2; https://doi.org/10.3390/biomimetics5010002. https://www.mdpi.com/2313-7673/5/1/2 • Dzhabieva, Z.; Shilov, G.; Avdeeva, L.; Dobrygin, V.; Tkachenko, V.; Dzhabiev, T. Biomimetic Water Oxidation Catalyzed by a Binuclear Ruthenium (IV) Nitrido-Chloride Complex with Lithium Counter-Cations. Biomimetics 2020, 5(1), 3; https://doi.org/10.3390/biomimetics5010003. https://www.mdpi.com/2313-7673/5/1/3 • Charpentier, V.; Adriaenssens, S. Effect of Gravity on the Scale of Compliant Shells. Biomimetics 2020, 5(1), 4; https://doi.org/10.3390/biomimetics5010004. https://www.mdpi.com/2313-7673/5/1/4 • Zobl, S.; Wilts, B.; Salvenmoser, W.; Pölt, P.; Gebeshuber, I.; Schwerte, T. Orientation- Dependent Reflection of Structurally Coloured Butterflies. Biomimetics 2020, 5(1), 5; https://doi.org/10.3390/biomimetics5010005. https://www.mdpi.com/2313-7673/5/1/5 • Biomimetics Editorial Office Acknowledgement to Reviewers of Biomimetics in 2019. Biomimetics 2020, 5(1), 6; https://doi.org/10.3390/biomimetics5010006. https://www.mdpi.com/2313-7673/5/1/6 • Stadler, A.; Schönauer, M.; Aslani, R.; Baumgartner, W.; Philippi, T. The Impact of a Flexible Stern on Canoe Boat Maneuverability and Speed. Biomimetics 2020, 5(1), 7; https://doi.org/10.3390/biomimetics5010007. https://www.mdpi.com/2313-7673/5/1/7 • Figueroa-Mata, G.; Mata-Montero, E. Using a Convolutional Siamese Network for Image- Based Plant Species Identification with Small Datasets. Biomimetics 2020, 5(1), 8; https://doi.org/10.3390/biomimetics5010008. https://www.mdpi.com/2313-7673/5/1/8 • Ogunka, U.; Daghooghi, M.; Akbarzadeh, A.; Borazjani, I. The Ground Effect in Anguilliform Swimming. Biomimetics 2020, 5(1), 9; https://doi.org/10.3390/biomimetics5010009. https://www.mdpi.com/2313-7673/5/1/9 • Weber, P.; Arampatzis, G.; Novati, G.; Verma, S.; Papadimitriou, C.; Koumoutsakos, P. Optimal Flow Sensing for Schooling Swimmers. Biomimetics 2020, 5(1), 10; https://doi.org/10.3390/biomimetics5010010. https://www.mdpi.com/2313-7673/5/1/10 biomimetics Article Evaluation of Mixed Deep Neural Networks for Reverberant Speech Enhancement Michelle Gutiérrez-Muñoz † and Astryd González-Salazar † and Marvin Coto-Jiménez *,† Escuela de Ingeniería Eléctrica, Universidad de Costa Rica, San José 11501-2060, Costa Rica; michelle.gutierrezmunoz@ucr.ac.cr (M.G.-M.); astryd.gonzalez@ucr.ac.cr (A.G.-S.) * Correspondence: marvin.coto@ucr.ac.cr † These authors contributed equally to this work. Received: 30 October 2019; Accepted: 16 December 2019; Published: 20 December 2019 ���������� ������� Abstract: Speech signals are degraded in real-life environments, as a product of background noise or other factors. The processing of such signals for voice recognition and voice analysis systems presents important challenges. One of the conditions that make adverse quality difficult to handle in those systems is reverberation, produced by sound wave reflections that travel from the source to the microphone in multiple directions. To enhance signals in such adverse conditions, several deep learning-based methods have been proposed and proven to be effective. Recently, recurrent neural networks, especially those with long short-term memory (LSTM), have presented surprising results in tasks related to time-dependent processing of signals, such as speech. One of the most challenging aspects of LSTM networks is the high computational cost of the training procedure, which has limited extended experimentation in several cases. In this work, we present a proposal to evaluate the hybrid models of neural networks to learn different reverberation conditions without any previous information. The results show that some combinations of LSTM and perceptron layers produce good results in comparison to those from pure LSTM networks, given a fixed number of layers. The evaluation was made based on quality measurements of the signal’s spectrum, the training time of the networks, and statistical validation of results. In total, 120 artificial neural networks of eight different types were trained and compared. The results help to affirm the fact that hybrid networks represent an important solution for speech signal enhancement, given that reduction in training time is on the order of 30%, in processes that can normally take several days or weeks, depending on the amount of data. The results also present advantages in efficiency, but without a significant drop in quality. Keywords: artificial neural network; deep learning; LSTM; speech processing 1. Introduction In real-environments, audio signals are affected by conditions such as additive noise, reverberation, and other distortions, due to elements that produce sounds simultaneously or are presented as obstacles in the signal path to the microphone. In the case of speech signals, communication devices and applications of speech technologies may be affected in their performance [1–4] by the presence of such conditions. In recent decades, many algorithms have been developed to enhance degraded speech; these try to suppress or reduce distortions, as well as preserve or improve the quality of the perceived signal [5]. Many recent algorithms are based on deep neural networks (DNN) [6–9]. The most common implementation is based on approximating a mapping function from the degraded characteristics of speech with noise, towards the corresponding characteristics of clean speech. Biomimetics 2020, 5, 1; doi:10.3390/biomimetics5010001 www.mdpi.com/journal/biomimetics http://www.mdpi.com/journal/biomimetics http://www.mdpi.com https://orcid.org/0000-0003-3313-8324 https://orcid.org/0000-0002-3444-0464 https://orcid.org/0000-0002-6833-9938 http://www.mdpi.com/2313-7673/5/1/1?type=check_update&version=1 http://dx.doi.org/10.3390/biomimetics5010001 http://www.mdpi.com/journal/biomimetics Biomimetics 2020, 5, 1 2 of 12 The benefits of achieving this type of speech signal enhancement can be applied to signal processing in mobile phone applications, voice over Internet protocol, speech recognition systems, and devices for people with diminished hearing ability [10]. In addition to the classical perceptron model, created in the 1950s, new types of neural networks have been developed, e.g., recurrent neural networks (RNNs). An example of RNNs are the LSTM neural networks. In previous efforts to enhance speech, spectrum-derived characteristics, such as Mel-frequency cepstrum coefficients (MFCC), have been mapped successfully between noisy speech to clean speech [11,12]. The benefits of using LSTM, as well as other types of RNNs, are superior for modeling of the dependent nature of speech signals. Among the drawbacks of LSTM are the high computational cost of its training procedures. In this work, we extend previous experiments with LSTM by evaluating deep neural networks, with a fixed number of three hidden layers, that combine LSTM layers (bidirectional) and simpler layers, based on perceptrons. Such deep neural network algorithms have been successful in overcoming the performance of classical methods based on algorithms for signal processing, which have considered several signal-to-noise ratios (SNR) [12–15], or reverberant speech [16–18]. Some recent work has explored the use of mixed neural networks to achieve a better performance in different tasks, such as classifying the temporary stages of sleep, analyzing the real-time behavior of an online buyer, or the suppression of noise in a MEMS gyroscope, in which good results were obtained for specific situations and configurations [19–21]. The combination of different types of neural networks have been successfully presented in [22], in the form of ensemble models to predict diseases in images. The wide variety of models applied in other fields, where regression, classification, and prediction are required, have also been analyzed [23,24], and show the multiple possibilities and the wide field of experimentation that is possible with deep neural networks. Our main focus is on reducing the training time of the networks without a significant reduction in the capacity of the network. To achieve improvement, we consider all the different combinations of layers for de-reverberation, with the goal of accelerating the training process due to fewer connections. Thus, the process can become more efficient. For this purpose, several objective measures were used to verify the results, which comparatively show the capacity of the BLSTM with three layers, and the combination with layers of perceptron, in improving speech conditions of reverberation. The rest of this document is organized as follows. Section 2 provides the background and context of the problem of improving reverberant speech and the BLSTM. Section 4 describes the experimental setup. Section 5 presents the results with a discussion. In Section 6, conclusions are presented. 2. Problem Statement In real-world environments where speech signals are registered with microphones, the presence of reverberation is common. It is caused by the reflections of the audio signal on its path to the microphone. This phenomenon is accentuated when the space is wide and the surfaces favor the reflection of the signals. It can be assumed that the reverberated signal x is a degraded version of the clean signal s. The relationship between both waves is described by [25]: x(n) = h>(n) ∗ s(n), (1) where h = [h1, h2, . . . , hL] > is the impulse response of the acoustic channel from the source to the microphone, L is the index of the discrete-time impulse response coefficient vector, > is the transpose of vector, and ∗ is the convolution operation. Biomimetics 2020, 5, 1 3 of 12 The degraded speech signal with reverberation is perceived as distant or as a very short type of echo. Consequently, this effect generally increases as the speaker’s distance to the microphone increases. Since this effect is not desired for proper recognition and analysis of the speech signal, new algorithms have been proposed to minimize it. Mainly, in the last few years, algorithms based on deep learning have stood out. By implementing deep neural networks, an approximation to s(n) can be estimated using a function f (·) between the data of the reverberated signal and the clean signal: ŝ(t) = f (x(t)) . (2) The quality of the approximation performed by f (·) usually depends on the amount of data and the algorithm selected. For the present work, we take as a base case the estimation of f (·) made by bidirectional LSTM (BLSTM) networks with three hidden layers. The main motivation in applying these deep neural networks is their recent success in speech enhancement related tasks, where they surpassed other algorithms applied to improve speech in noisy or reverberant conditions. In most of these experiences, it is noted the high computational cost of training the LSTM and BLSTM networks as a constraint to perform extended experimentation. In this model, we propose a comparison and statistical validation of results with mixed networks, which include combinations of BLSTM layers and perceptron. 3. Autoencoders of BLSTM Networks Since the appearance of RNNs, there are new alternatives to model the character dependent on the sequential information in applications where the nature of the parameters is relevant. These types of neural networks are capable of storing information through feedback connections between neurons in their hidden layers or another network that is in the same layer [26,27]. With the purpose of expanding the capabilities of RNNs by storing information in the short and long term, the LSTM networks shown in [28] introduce a set of gates into the memory cells capable of controlling access and storage and propagation of values across the network. The results obtained when using LSTM networks in areas that depend on previous states of information, as is the case with voice recognition, musical composition, and handwriting synthesis, were encouraging [28–30]. In addition to the recurring connections between the internal units, each unit in the network has additional gates for storing values: One for input, one for memory clearing, one for output, and one for activating memory. In this way, it is possible to store values for many steps or have them available at any time [28]. The gates are implemented using the following equations: it = σ (Wxixt + Whiht−1 + Wcict−1 + bi) (3) ft = σ ( Wx f xt + Wh f ht−1 + Wc f ct−1 + b f ) (4) ct = ftct−1 + it tanh (Wxcxt + Whcht−1 + bc) (5) ot = σ (Wxoxt + Whoht−1 + Wcoct + bo) (6) ht = ot tanh (ct) (7) where σ is the sigmoid activation function, i is the input gate, f is the memory erase gate, and ot is the exit gate. c is the activation of memory. Wmn is the matrix that contains the values of the connections between each unit and the gates. h is the output of the LSTM memory unit. Additional details about the training process and the implications of this implementation can be found at [31]. Biomimetics 2020, 5, 1 4 of 12 An additional extension of LSTM networks that has had a greater advantage in tasks related to temporal parameter dependence is the BLSTM. Here, the configuration of the network allows the updating of parameters in both directions of the process: One can convert the input parameters to the reference of the output, and vice versa. In this work, these units are used to make comparisons. The structure of a simple bidirectional network with input i, output o, and two hidden layers (h f and hb) is shown in Figure 1. it-1 hbt-1 hft-1 ot-1 it hbt hft ot it+1 hbt+1 hft+1 ot+1 Figure 1. Bidirectional Long Short-term Memory (BLSTM) network structure. Adapted from [32]. LSTM networks can handle information over long periods; however, using bidirectional LSTM (BLSTM) neural networks with two hidden layers connected to the same output layer gives them access to information in both directions. This allows bidirectional networks to take advantage of not just the past but also the future context [32]. One of the main architectures applied for regression tasks (including speech enhancement) using deep neural networks are the autoencoders. An autoencoder for speech enhancement is a neural network architecture that has been successful in various tasks related to speech [33]. This architecture consists of an encoder that transforms an input vector s into a representation in the hidden layers h through a f mapping. It also has a decoder that takes the hidden representation and transforms it back into a vector in the input space. During training, the features of the distorted signal (noise or reverberation) are used as inputs for the noise elimination autoencoders, while the features of the clean speech are presented as outputs. In addition, to learn the complex relationships between these sets of features, the training algorithm adjusts the parameters of the network. Currently, computers and algorithms have the ability to process large datasets, as well as networks with several hidden layers. 4. Experimental Setup To test our proposed mixed neural networks LSTM/Perceptron to enhance reverberated speech, the experiment can be summarized in the following steps: 1. Selection of conditions: Given the large number of impulse responses contemplated in the databases, we randomly chose five reverberated speech conditions. Each of the conditions has the corresponding clean version in the database. 2. Extraction of features and input-output correspondence: A set of parameters was extracted from the reverberated and clean audio files. Those of the reverberated files were used as inputs to the networks, while the corresponding clean functions were the outputs. 3. Training: During training, the weights of the networks were adjusted as the parameters with reverberation and clean were presented to the network. As usual in recurrent neural networks, the updating of the values of the internal weights was carried out using the back-propagation algorithm through time. In total, 210 expressions were used for each condition (approximately 70% of the total database) to train each case. The details and equations of the algorithm followed can be found in [34]. Biomimetics 2020, 5, 1 5 of 12 4. Validation: After each training step, the sum of the squared errors within the validation set of approximately 20% of the statements was calculated, and the weights of the network were updated in each improvement. 5. Test: A subset of 50 phrases, selected at random (about 10% of the total number of phrases in the database), was chosen for the test set, for each condition. These phrases were not part of the training process, to provide independence between training and testing. In the following subsections, more details of the experimental procedure are provided. 4.1. Database We used the Reverberant Voice Database created at the University of Edinburgh [35], which was designed to train and evaluate the methods of speech de-reverberation. The reverberated speech of the database was produced by convolving the recordings of 56 native English speakers with several impulse responses in various university halls. For this work, we randomly chose the following conditions: ACE Building Lobby 1, Artificial Room 1, Mardy Room 2, ACE Lecture Room 1, and ACE Meeting Room 2. 4.2. Feature Extraction The pairs of WAV files corresponding to clean and reverberated speech were processed using the Ahocoder [36] software to obtain the coefficients. Those were extracted with a frame size of 160 samples and a frame-shift of 80 samples. For each frame of speech, we extracted the spectrum parameters (39 MFCC), fundamental frequency ( f0), and the energy. For this work, neural networks were applied to improve the 39 MFCC coefficients, while the rest of the parameters remained invariant. During training, the parameters of the reverberated speech were presented as the inputs of the network, while the correspondent parameters of the clean speech were outputs. For the test set, the MFCC parameters of the reverberated speech were substituted with the enhanced version, and the evaluation measure was applied to the reconstructed WAVE file, also made with the Ahocoder system. 4.3. Evaluation For the evaluation of the results, the following objective measures were applied: • Perceptual evaluation of speech quality (PESQ): This measure uses a model to predict the subjective quality of speech, as defined in ITU-T P.862.ITU recommendation. The results are in the range [0.5, 4.5], where 4.5 corresponds to the signal enhanced perfectly. PESQ is calculated as [37]: PESQ = a0 + a1Dind + a2 Aind (8) where Dind is the average disturbance and Aind is the asymmetric perturbation. The ak were chosen to optimize PESQ in the measurement of general speech quality. • Sum of squared errors (sse): This is the most common metric for the validation set error during the training process of a neural network. It is defined as: sse(θ) = T ∑ n=1 (cx − ĉx) 2 (9) = T ∑ n=1 (cx − f (cx)) 2 , (10) where cx is the known value of the outputs and ĉx is the approximation made by the network. • Time per epoch: This refers to the time it takes for an iteration of the training process. Biomimetics 2020, 5, 1 6 of 12 Additionally, Friedman’s statistical test was used to determine the statistical significance of the results in the test sets. 4.4. Experiments Figure 2 shows the procedure followed for the comparison between the different architectures tested in this work. To analyze all the architectures that can be formed with a mixture of BLSTM layers and MLP layers, eight different neural networks were tested for each reverberation condition: • BLSTM–BLSTM–BLSTM • BLSTM–BLSTM–MLP • BLSTM–MLP–BLSTM • BLSTM–MLP–MLP • MLP–BLSTM–BLSTM • MLP–BLSTM–MLP • MLP–MLP–BLSTM • MLP–MLP–MLP The metrics were applied in each of these possibilities, which constitute all the possibilities that can be combined between the BLSTM and MLP layers in three layers. Inputs MLP Layer MLP Layer BLSTM Layer MLP Layer MLP Layer BLSTM Layer BLSTM Layer BLSTM LayerBLSTM Layer Comparison Outputs OutputsOutputs MLP Network BLSTM NetworkMixed Network Figure 2. Sample of three networks compared in this work: The purely multi-layer perceptron (MPL), a mixed network, and the purely BLSTM network. 5. Results and Discussion Table 1 shows the training results for all networks and all possible combinations of three hidden layers. The training of each set was repeated three times, and the average values are reported. Following previously reported results, the network with only BLSTM layers provides the best results in most cases of reverberation conditions. Biomimetics 2020, 5, 1 7 of 12 Table 1. Efficiency of the different combinations of hidden layers, by the condition of reverberation. * is the best value of sse in each condition. Condition Network (Hidden Layers) sse Time per Epoch (s) MARDY BLSTM–BLSTM–BLSTM 201.34 * 50.6 BLSTM–BLSTM–MLP 204.39 33.3 BLSTM–MLP–BLSTM 210.81 33.5 BLSTM–MLP–MLP 218.91 15.9 MLP–BLSTM–BLSTM 204.82 36.1 MLP–BLSTM–MLP 256.32 18.6 MLP–MLP–BLSTM 216.46 18.8 MLP–MLP–MLP 400.34 1.2 Lecture Room BLSTM–BLSTM–BLSTM 213.12 74.9 BLSTM–BLSTM–MLP 214.35 48.8 BLSTM–MLP–BLSTM 221.88 49.3 BLSTM–MLP–MLP 229.22 23.2 MLP–BLSTM–BLSTM 212.34 * 52.8 MLP–BLSTM–MLP 226.39 27.7 MLP–MLP–BLSTM 230.85 27.6 MLP–MLP–MLP 360.41 1.8 Artificial Room BLSTM–BLSTM–BLSTM 88.47 * 55.5 BLSTM–BLSTM–MLP 90.37 36.5 BLSTM–MLP–BLSTM 93.61 36.6 BLSTM–MLP–MLP 104.23 17.4 MLP–BLSTM–BLSTM 92.18 39.5 MLP–BLSTM–MLP 108.56 20.6 MLP–MLP–BLSTM 111.13 20.5 MLP–MLP–MLP 170.61 1.3 ACE Building BLSTM–BLSTM–BLSTM 207.32 * 73.8 BLSTM–BLSTM–MLP 210.17 45.8 BLSTM–MLP–BLSTM 214.29 46.1 BLSTM–MLP–MLP 212.54 21.6 MLP–BLSTM–BLSTM 208.04 49.2 MLP–BLSTM–MLP 221.28 25.6 MLP–MLP–BLSTM 220.13 25.8 MLP–MLP–MLP 333.60 1.7 Meeting Room BLSTM–BLSTM–BLSTM 197.37 69.9 BLSTM–BLSTM–MLP 199.03 45.7 BLSTM–MLP–BLSTM 204.68 45.8 BLSTM–MLP–MLP 217.52 21.6 MLP–BLSTM–BLSTM 196.90 * 49.6 MLP–BLSTM–MLP 206.03 25.7 MLP–MLP–BLSTM 214.28 25.9 MLP–MLP–MLP 363.19 1.7 For the five cases of reverberation considered in this paper, the network that stands out as a competitive alternative to the three-layer BLSTM network is the MLP–BLSTM–BLSTM configuration. In addition to presenting in two cases a better result between all the architectures (under the conditions “Lecture Room” and “Meeting Room”), the training time is almost 30% less per epoch in comparison to the BLSTM network. This is one of the main indicators sought in this work. Table 1 also shows how the training times are similar between those configurations consisting of two BLSTM layers and one MLP and those consisting of only one BLSTM layer and two MLPs. The MLP–MLP–MLP type networks, despite having very low training times per epoch, as expected, do not present competitive results in comparison to the others. In addition to the verification of the training efficiency of the networks, Table 2 shows the results in terms of the PESQ quality metric. This is of the utmost importance, since the analysis of the problem Biomimetics 2020, 5, 1 8 of 12 of de-reverberation of speech signals is what is being presented. Thus, improvements in efficiency and sse values must also be checked in terms of the quality of the signal achieved. Table 2. Objective evaluations for the different combinations of hidden layers, by the condition of reverberation. * is the best value. The p-value was obtained with the Friedman test, with a significance of 0.05. Condition Network (Hidden Layers) PESQ Significative Difference p-Value MARDY BLSTM-BLSTM-BLSTM 2.30 - - BLSTM–BLSTM–MLP 2.31 * no 0.715 BLSTM–MLP–BLSTM 2.27 yes 0.003 BLSTM–MLP–MLP 2.19 yes 6.648 × 10−8 MLP–BLSTM–BLSTM 2.28 no 0.147 MLP–BLSTM–MLP 2.08 yes 1.965 × 10−14 MLP–MLP–BLSTM 2.24 yes 0.000 MLP–MLP–MLP 1.94 yes 0.000 Lecture Room BLSTM–BLSTM–BLSTM 2.28 * - - BLSTM–BLSTM–MLP 2.21 no 0.095 BLSTM–MLP–BLSTM 2.22 yes 0.0034 BLSTM–MLP–MLP 2.20 yes 1.729 × 10−7 MLP–BLSTM–BLSTM 2.27 no 0.199 MLP–BLSTM–MLP 2.21 yes 9.635 × 10−5 MLP–MLP–BLSTM 2.20 yes 9.617 MLP–MLP–MLP 2.00 yes 0.000 Artificial Room BLSTM–BLSTM–BLSTM 3.18 * - - BLSTM–BLSTM–MLP 3.17 no 1.000 BLSTM–MLP–BLSTM 3.14 yes 0.002 BLSTM–MLP–MLP 3.12 yes 6.650 × 10−8 MLP–BLSTM–BLSTM 3.17 no 1.000 MLP–BLSTM–MLP 3.06 yes 1.965 × 10−14 MLP–MLP–BLSTM 3.08 yes 2.695 × 10−6 MLP–MLP–MLP 2.90 yes 0.000 ACE Building BLSTM–BLSTM–BLSTM 2.37 * - - BLSTM–BLSTM–MLP 2.35 no 0.068 BLSTM–MLP–BLSTM 2.35 no 0.147 BLSTM–MLP–MLP 2.32 yes 4.22 × 10−5 MLP–BLSTM–BLSTM 2.36 no 0.474 MLP–BLSTM–MLP 2.33 yes 0.026 MLP–MLP–BLSTM 2.33 yes 0.008 MLP–MLP–MLP 2.08 yes 0.000 Meeting Room BLSTM–BLSTM–BLSTM 2.28 - - BLSTM–BLSTM–MLP 2.29 * no 0.147 BLSTM–MLP–BLSTM 2.24 no 0.060 BLSTM–MLP–MLP 2.23 yes 0.002 MLP–BLSTM–BLSTM 2.28 no 0.474 MLP–BLSTM–MLP 2.25 no 0.715 MLP–MLP–BLSTM 2.20 yes 0.001 MLP–MLP–MLP 2.0 yes 1.960 × 10−14 In the last table, the differences obtained for the BLSTM–BLSTM–BLSTM base system are presented, in terms of statistical significance, according to the Friedman test. In each of the five reverberation conditions, the results of these tests can be summarized: • MARDY, Lecture Room and Artificial Room: Only two of the mixed configurations present results that do not significantly differ statistically with the base system. These mixed networks are BLSTM–BLSTM–MLP and MLP–BLSTM–BLSTM. Biomimetics 2020, 5, 1 9 of 12 • Ace Building: In this case, three combinations of hidden layers present results that do not differ significantly from the base case. • Meeting Room: This is a particular case, because the combination BLSTM-BLSTM-MLP is the one that presents the best result, although the improvement is not significant compared to the base system. On the other hand, MLP–BLSTM–BLSTM, BLSTM–MLP–BLSTM, and MLP–BLSTM–MLP present results that do not differ significantly from the base system. Figure 3 shows the spectrograms corresponding to clean speech, as well as those corresponding to speech with reverberation and to two of the proposed configurations: One based solely on BLSTM layers and the mixed network that obtained better results (MLP–BLSTM–BLSTM). One can appreciate the improvements introduced by the neural networks and the proximity that is perceived visually in this representation of the spectrogram of the mixed network in comparison to that of the base system. (a) (b) (c) (d) Figure 3. Spectrograms of a phrase in the database: (a) speak clean; (b) speak with reverberation (ACE Building Lobby); (c) enhancement result with the BLSTM network; and (d) enhancement result with the mixed MLP–BLSTM–BLSTM network. Considering the previous efficiency results and how these are reflected in the PESQ metric, it is emphasized that there are combinations of mixed networks, especially MLP–BLSTM–BLSTM, which reduce the times of training considerably, without significantly sacrificing the quality of results in the Biomimetics 2020, 5, 1 10 of 12 reverberation of the signals.However, to increase efficiency in further experiments, some processes can be parallelized and the proposal repeated in networks of greater depth. 6. Conclusions In this work, the use of mixed neural networks, consisting of combinations of layers formed by perceptron units, with BLSTM layers was proposed as an alternative for the reduction of training time of purely BLSTM networks. Training time has represented a limitation for extensive experimentation with this type of artificial neural network in different applications, including some related to the improvement of speech signals. One of the eight possible combinations of mixed networks presented competitive results, in terms of the metrics of the training system, and results that did not differ significantly from the purely BLSTM case in terms of the PESQ of the signals. The significance was determined with a statistical test. The reduction in training time is on the order of 30%, in processes that can normally take hours or days, depending on the amount of data. The results presented here open the possibility for simplifying some neural network configurations to be able to perform extensive experimentation in different applications where it is required to map parameters with similar characteristics, as in the case of autoencoders. Author Contributions: Conceptualization, M.G.-M., A.G.-S. and M.C.-J.; methodology, A.G.-S. and M.C.-J.; software, M.C.-J.; M.G.-M., A.G.-S. and M.C.-J.; formal analysis, M.G.-M. and M.C.-J.; investigation, M.G.-M., A.G.-S. and M.C.-J.; resources, M.G.-M., A.G.-S. and M.C.-J.; data curation, M.C.-J.; writing—original draft preparation, M.G.-M., A.G.-S. and M.C.-J.; and writing—review and editing, M.G.-M., A.G.-S. and M.C.-J. All authors have read and agreed to the published version of the manuscript. Funding: This research received no external funding. Acknowledgments: This work was made with the support of the University of Costa Rica, project 322–B9-105. Conflicts of Interest: The authors declare no conflict of interest. Abbreviations The following abbreviations are used in this manuscript: BLSTM Bidirectional Long Short-term Memory Neural Network DNN Deep Neural Network LSTM Long Short-term Memory Neural Network MEMS Microelectromechanical System MFCC Mel Frequency Cepstral Coefficients MLP Multi-Layer Perceptron PESQ Perceptual Evaluation of Speech Quality RNN Recurrent Neural Network SNR Signal-to-noise Ratio TTS Text-to-Speech Synthesis References 1. Weninger, F.; Watanabe, S.; Tachioka, Y.; Schuller, B. Deep recurrent de-noising auto-encoder and blind de-reverberation for reverberated speech recognition. In Proceedings of the 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Florence, Italy, 4–9 May 2014; pp. 4623–4627. 2. Weninger, F.; Geiger, J.; Wöllmer, M.; Schuller, B.; Rigoll, G. Feature enhancement by deep LSTM networks for ASR in reverberant multisource environments. Comput. Speech Lang. 2014, 28, 888–902. [CrossRef] 3. Narayanan, A.; Wang, D. Ideal ratio mask estimation using deep neural networks for robust speech recognition. In Proceedings of the 2013 IEEE International Conference onAcoustics, Speech and Signal Processing (ICASSP), Vancouver, BC, Canada, 26–30 May 2013; pp. 7092–7096. http://dx.doi.org/10.1016/j.csl.2014.01.001 Biomimetics 2020, 5, 1 11 of 12 4. Bagchi, D.; Mandel, M.I.; Wang, Z.; He, Y.; Plummer, A.; Fosler-Lussier, E. Combining spectral feature mapping and multi-channel model-based source separation for noise-robust automatic speech recognition. In Proceedings of the 2015 IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU), Scottsdale, AZ, USA, 13–17 December 2015; pp. 496–503. 5. Hansen, J.H.; Pellom, B.L. An effective quality evaluation protocol for speech enhancement algorithms. In Proceedings of the Fifth International Conference on Spoken Language Processing, Sydney, Australia, 30 November–4 December 1998. 6. Du, J.; Wang, Q.; Gao, T.; Xu, Y.; Dai, L.R.; Lee, C.H. Robust speech recognition with speech enhanced deep neural networks. In Proceedings of the Fifteenth Annual Conference of the International Speech Communication Association, Singapore, 14–18 September 2014. 7. Han, K.; He, Y.; Bagchi, D.; Fosler-Lussier, E.; Wang, D. Deep neural network based spectral feature mapping for robust speech recognition. In Proceedings of the Sixteenth Annual Conference of the International Speech Communication Association, Dresden, Germany, 6–10 September 2015. 8. Maas, A.L.; Le, Q.V.; O’Neil, T.M.; Vinyals, O.; Nguyen, P.; Ng, A.Y. Recurrent neural networks for noise reduction in robust ASR. In Proceedings of the Thirteenth Annual Conference of the International Speech Communication Association, Portland, Oregon, 9–13 September 2012. 9. Deng, L.; Li, J.; Huang, J.T.; Yao, K.; Yu, D.; Seide, F.; Seltzer, M.L.; Zweig, G.; He, X.; Williams, J.D.; et al. Recent advances in deep learning for speech research at Microsoft. In Proceedings of the 2013 IEEE International Conference on Acoustics, Speech and Signal Processing ICASSP, Vancouver, BC, Canada, 26–31 May 2013; Volume 26, p. 64. 10. Healy, E.W.; Yoho, S.E.; Wang, Y.; Wang, D. An algorithm to improve speech recognition in noise for hearing-impaired listeners. J. Acoust. Soc. Am. 2013, 134, 3029–3038. [CrossRef] [PubMed] 11. Coto-Jiménez, M.; Goddard-Close, J. LSTM Deep Neural Networks Postfiltering for Enhancing Synthetic Voices. Int. J. Pattern Recognit. Artif. Intell. 2018, 32, 1860008. [CrossRef] 12. Coto-Jiménez, M. Robustness of LSTM Neural Networks for the Enhancement of Spectral Parameters in Noisy Speech Signals. In Proceedings of the Mexican International Conference on Artificial Intelligence, Guadalajara, Mexico, 22–27 October 2018; Springer: New York, NY, USA, 2018; pp. 227–238. 13. Kumar, A.; Florencio, D. Speech enhancement in multiple-noise conditions using deep neural networks. arXiv 2016, arXiv:1605.02427. 14. Hinton, G.; Deng, L.; Yu, D.; Dahl, G.E.; Mohamed, A.R.; Jaitly, N.; Senior, A.; Vanhoucke, V.; Nguyen, P.; Sainath, T.N.; et al. Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups. IEEE Signal Process. Mag. 2012, 29, 82–97. [CrossRef] 15. Vincent, E.; Watanabe, S.; Nugraha, A.A.; Barker, J.; Marxer, R. An analysis of environment, microphone and data simulation mismatches in robust speech recognition. Comput. Speech Lang. 2017, 46, 535–557. [CrossRef] 16. Feng, X.; Zhang, Y.; Glass, J. Speech feature denoising and dereverberation via deep autoencoders for noisy reverberant speech recognition. In Proceedings of the 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Florence, Italy, 4–9 May 2014; pp. 1759–1763. 17. Ishii, T.; Komiyama, H.; Shinozaki, T.; Horiuchi, Y.; Kuroiwa, S. Reverberant speech recognition based on denoising autoencoder. In Proceedings of the Interspeech, Lyon, France, 25–29 August 2013; pp. 3512–3516. 18. Zhao, Y.; Wang, Z.Q.; Wang, D. Two-Stage Deep Learning for Noisy-Reverberant Speech Enhancement. IEEE/ACM Trans. Audio Speech Lang. Process. 2019, 27, 53–62. [CrossRef] [PubMed] 19. Dong, H.; Supratak, A.; Pan, W.; Wu, C.; Matthews, P.M.; Guo, Y. Mixed Neural Network Approach for Temporal Sleep Stage Classification. IEEE Trans. Neural Syst. Rehabil. Eng. 2017, 26, 4–5. [CrossRef] [PubMed] 20. Sakar, C.O.; Polat, S.O.; Katircioglu, M.; Kastro, Y. Real-time prediction of online shoppers’ purchasing intention using multilayer perceptron and LSTM recurrent neural networks. Neural Comput. Appl. 2018, 31, 1–16. [CrossRef] 21. Jiang, C.; Chen, Y.; Chen, S.; Bo, Y.; Li, W.; Tian, W.; Guo, J. A Mixed Deep Recurrent Neural Network for MEMS Gyroscope Noise Suppressing. Electronics 2019, 8, 181. [CrossRef] 22. Qummar, S.; Khan, F.G.; Shah, S.; Khan, A.; Shamshirband, S.; Rehman, Z.U.; Khan, I.A.; Jadoon, W. A Deep Learning Ensemble Approach for Diabetic Retinopathy Detection. IEEE Access 2019, 7, 150530–150539. [CrossRef] 23. Shamshirband, S.; Rabczuk, T.; Chau, K.W. A Survey of Deep Learning Techniques: Application in Wind and Solar Energy Resources. IEEE Access 2019, 7, 164650–164666. [CrossRef] http://dx.doi.org/10.1121/1.4820893 http://www.ncbi.nlm.nih.gov/pubmed/24116438 http://dx.doi.org/10.1142/S021800141860008X http://dx.doi.org/10.1109/MSP.2012.2205597 http://dx.doi.org/10.1016/j.csl.2016.11.005 http://dx.doi.org/10.1109/TASLP.2018.2870725 http://www.ncbi.nlm.nih.gov/pubmed/31106230 http://dx.doi.org/10.1109/TNSRE.2017.2733220 http://www.ncbi.nlm.nih.gov/pubmed/28767373 http://dx.doi.org/10.1007/s00521-018-3523-0 http://dx.doi.org/10.3390/electronics8020181 http://dx.doi.org/10.1109/ACCESS.2019.2947484 http://dx.doi.org/10.1109/ACCESS.2019.2951750 Biomimetics 2020, 5, 1 12 of 12 24. Babaee, E.; Anuar, N.B.; Abdul Wahab, A.W.; Shamshirband, S.; Chronopoulos, A.T. An overview of audio event detection methods from feature extraction to classification. Appl. Artif. Intell. 2017, 31, 661–714. [CrossRef] 25. Naylor, P.A.; Gaubitch, N.D. Speech Dereverberation; Springer Science & Business Media: New York, NY, USA, 2010. 26. Fan, Y.; Qian, Y.; Xie, F.L.; Soong, F.K. TTS synthesis with bidirectional LSTM based recurrent neural networks. In Proceedings of the Fifteenth Annual Conference of the International Speech Communication Association, Singapore, 14–18 September 2014. 27. Zen, H.; Sak, H. Unidirectional long short-term memory recurrent neural network with recurrent output layer for low-latency speech synthesis. In Proceedings of the 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), South Brisbane, Australia, 19–24 April 2015; pp. 4470–4474. 28. Hochreiter, S.; Schmidhuber, J. Long short-term memory. Neural Comput. 1997, 9, 1735–1780. [CrossRef] [PubMed] 29. Graves, A.; Jaitly, N.; Mohamed, A.R. Hybrid speech recognition with deep bidirectional LSTM. In Proceedings of the 2013 IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU), Olomouc, Czech Republic, 8–13 December 2013; pp. 273–278. 30. Graves, A.; Fernández, S.; Schmidhuber, J. Bidirectional LSTM networks for improved phoneme classification and recognition. In Proceedings of the International Conference on Artificial Neural Networks, Warsaw, Poland, 11–15 September 2005; Springer: New York, NY, USA, 2005; pp. 799–804. 31. Gers, F.A.; Schraudolph, N.N.; Schmidhuber, J. Learning precise timing with LSTM recurrent networks. J. Mach. Learn. Res. 2002, 3, 115–143. 32. Wöllmer, M.; Eyben, F.; Schuler, B.; Rigoll, G. A multi-stream ASR framework for BLSTM modeling of conversational speech. In Proceedings of the 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Prague, Czech Republic, 22–27 May 2011; p. 4861. 33. Coto-Jimenez, M.; Goddard-Close, J.; Di Persia, L.; Rufiner, H.L. Hybrid Speech Enhancement with Wiener filters and Deep LSTM Denoising Autoencoders. In Proceedings of the 2018 IEEE International Work Conference on Bioinspired Intelligence (IWOBI), San Carlos, CA, USA, 18–20 July 2018; pp. 1–8. 34. Greff, K.; Srivastava, R.K.; Koutník, J.; Steunebrink, B.R.; Schmidhuber, J. LSTM: A search space odyssey. IEEE Trans. Neural Networks Learn. Syst. 2017, 28, 2222–2232. [CrossRef] [PubMed] 35. Valentini-Botinhao, C. Reverberant Speech Database for Training Speech Dereverberation Algorithms and TTS Models; University of Edinburgh: Edinburgh, UK, 2016. [CrossRef] 36. Erro, D.; Sainz, I.; Navas, E.; Hernáez, I. Improved HNM-based vocoder for statistical synthesizers. In Proceedings of the Twelfth Annual Conference of the International Speech Communication Association, Florence, Italy, 27–31 August 2011. 37. Rix, A.W.; Hollier, M.P.; Hekstra, A.P.; Beerends, J.G. Perceptual Evaluation of Speech Quality (PESQ) The New ITU Standard for End-to-End Speech Quality Assessment Part I–Time-Delay Compensation. J. Audio Eng. Soc. 2002, 50, 755–764. c© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/). http://dx.doi.org/10.1080/08839514.2018.1430469 http://dx.doi.org/10.1162/neco.1997.9.8.1735 http://www.ncbi.nlm.nih.gov/pubmed/9377276 http://dx.doi.org/10.1109/TNNLS.2016.2582924 http://www.ncbi.nlm.nih.gov/pubmed/27411231 http://dx.doi.org/10.7488/ds/1425 http://creativecommons.org/ http://creativecommons.org/licenses/by/4.0/.