| dc.description.abstract | Machine learning with deep neural networks has been ubiquitous in solving various real-world problems with distinct human inputs, such as images, natural language, speech, and motion. As deep learning technology transitions from research environments to industry applications, in-the-wild challenges (e.g., scalability, reliability, efficiency, and usability) towards deep learning applications arise and become more prevalent. Frugality, a balance between computational cost and performance, is a relatively under-discussed question among those practical challenges. To achieve better model performance, researchers typically design a more complex model architecture with higher computational costs, which can be a problem in some scenarios, such as wearable devices with limited computing power.
In this dissertation, we will discuss potential frugality challenges deriving from the development of deep learning applications. Specifically, we will give an insight into practical challenges from three stages: I) Data generation. II) Single model optimization. III) Multi-model structure optimization. Discussions on improving frugality are based on three use cases, representing different stage challenges: a) The Sketch2Vis challenges. b) The ASVSpoof Challenge. c) SASV Challenges. We present the gap between each use case's current solution and frugality concerns and analyze corresponding challenges. To address the problem, we 1) propose DSL-based reverse captioning in Sketch2Vis challenges for scalable automatic dataset generation. Then, 2) present the ARawNet model structure in the ASVSpoof Challenge to reach state-of-the-art results with less model complexity. Finally, 3) describe the SA-SASV model to solve the SASV challenge with a single end-to-end model. | |
