Research

This page contains the brief introduction of my research of interests in machine learning.

NSF MoDL: Graph-Optimized Cellular Connectionism via Artificial Neural Networks for Data-Driven Modeling and Optimization of Complex Systems

In the realm of intricate problem-solving, a remarkable stride has been made with the integration of cutting-edge technology centered around graphs. These visual representations assist us in comprehending multifaceted real-world challenges and devising effective solutions. However, the methodologies employed in constructing and updating these interconnected graphs have exhibited limited evolution over time. The prospect of automating these processes, reducing the dependence on human intervention, and enhancing the resolution of critical real-world issues is the impetus behind our pioneering endeavor. Graph Neural Networks have displayed promising capabilities in addressing diverse challenges and hold untapped potential for further advancements. However, the adaptability of these tools to shifting scenarios predominantly relies on human intervention, a resource-intensive and time-consuming endeavor. Our objective is to forge a more intelligent application of these tools, enabling them to autonomously adapt in response to evolving circumstances.

The optimization of electric power distribution systems (EPDS) with distributed energy resources (DER) presents a pivotal arena for our aspirations. The emerging modern EPDS, the volt-var curve (VVC) optimization problem will be tackled by the graph-based modeling and optimization of the complex system given the insights generated by this project, which can influence practice in the modeling of diverse complex systems. This concerted effort aims not only to advance our problem-solving capacities but also to deepen our comprehension of the underlying power system mechanisms. Our mission is to develop these astute tools to enrich our lives, render our energy systems more sustainable, and unravel the intricacies of advanced technology. We validate our theoretical findings using the IEEE 8500-node test feeder and assess the scalability of the proposed approaches. The pursuit of decentralized and distributed optimization is dedicated to ensuring the smooth and efficient flow of electricity, especially in the context of harnessing renewable energy sources. This holds paramount importance, given the environmental benefits of sustainable energy sources.

NIH SocialBit: Establishing the Accuracy of a Wearable Sensor to Detect Social Interactions after Stroke

Stroke survivors are vulnerable to reduced social interactions. Reduced interactions are related to worse physical recovery after stroke. Enhancing social interactions after stroke may be one of the most powerful strategies to improve stroke recovery. Social interactions are defined as the synchronous interactions, commonly verbal, between individuals who are usually co-present in the same physical location. Current ways to detect social interactions rely on self-report, which cannot be performed reliably by patients with language or cognitive deficits. Patients with such deficits are most vulnerable to social isolation. This project introduces a new wearable social sensor, SocialBit, that can detect audio signatures of social interactions in real-world settings. Our preliminary data show that SocialBit can detect social interactions accurately (~95%), and it can do so by processing select audio features without storing raw audio data. Therefore, the technology detects and measures the duration of the social interaction while preserving the privacy of the content during the interaction. Based on these findings, we have developed a research plan to establish the usefulness of SocialBit in stroke survivors in the immediate post-stroke period. The post-stroke period is apt for such a study because 1) patients are vulnerable to social deprivation in this time period, and 2) the bounded nature of an inpatient setting provides an ideal environment to test SocialBit against a ground truth of directly observed social interactions. Our central hypothesis is that SocialBit can accurately detect social interactions in stroke survivors in inpatient settings. This project is primarily designed to establish the accuracy of SocialBit to detect social interaction in patients with varying deficits against the ground truth of video-assisted, real- time observation in the post-stroke period. First, we will examine the accuracy of SocialBit to detect the social interaction time against direct observation in 200 patients (Aim 1). Second, we will determine the association of social interaction time to social isolation and stroke outcomes at 3 months (Aim 2). Finally, we will determine the medical factors associated with social interaction time (Aim 3). This study will establish the key criteria of quantifying social interaction in stroke recovery research. The project will (a) identify automatic and unobtrusive methods to measure social interaction, (b) determine key design and outcome criteria for a future intervention trial, and (c) increase our understanding of underlying mechanisms in social changes after stroke. In so doing, this study will address the public health priority of building better behavioral modification strategies for patients with stroke.

NSF MLWiNS: Democratizing AI through Multi-Hop Federated Learning Over-the-Air

Federated learning (FL) has emerged as a key technology for enabling next-generation privacy-preserving AI at-scale, where a large number of edge devices, e.g., mobile phones, collaboratively learn a shared global model while keeping their data locally to prevent privacy leakage. Enabling FL over wireless multi-hop networks, such as wireless community mesh networks and wireless Internet over satellite constellations, not only can augment AI experiences for urban mobile users, but also can democratize AI and make it accessible in a low-cost manner to everyone, including people in low-income communities, rural areas, under-developed regions, and disaster areas. The overall objective of this project is to develop a novel wireless multi-hop FL system with guaranteed stability, high accuracy and fast convergence speed. This project is expected to advance the design of distributed deep learning (DL) systems, to promote the understanding of the strong synergy between distributed computing and distributed networking, and to bridge the gap between the theoretical foundations of distributed DL and its real-life applications. The project will also provide unique interdisciplinary training opportunities for graduate and undergraduate students through both research work and related courses that the PIs will develop and offer.

This project proposes to use concepts of federated learning and multi-agent reinforcement learning to provide optimal solutions for training DL models over wireless multi-hop networks that have communication constraints due to noisy and interference-rich wireless links. The main thrusts include: 1) developing a novel hierarchical FL system architecture with layered federated computation, semi-asynchronous model aggregation, and regularized objective function to significantly improve system scalability, communication efficiency, and stability; 2) fine-tuning the FL system via multi-agent reinforcement learning to maximize the FL accuracy with the minimum convergence time under the computing constraints of edge devices; 3) finding high-gain computation-light robust federated computing strategies for resource-constraint edge devices, including efficient DL model design and resource-aware model adaptation; and 4) developing an open-source wireless FL framework (OpenWFL) for fast prototyping, deploying, and evaluating the proposed FL algorithms in both an emulator and physical testbeds.

Interactive Reinforcement Learning

Reinforcement learning is often challenged by the lack of stability, robustness, and slow convergence, which lead to aptitude and alignment problems. To address these issues, interactive reinforcement learning expands the existing RL framework to account for human guidance. We investigate diverse form of human-AI interactions to enhance RL agent's capabilities for aptness and alignment.

Sparse Bayesian Reinforcement Learning (SBRL) / Interpretible Learning

Sparse Bayesian solution can suggest unnoticed discovery from input samples or experiences. Learning can store most significant experiences as human remembers the highlights of our experiences as snapshots. SBRL is supposed to provide an analysis solution for any blackbox learning. Thus, we investigate various memorization and interpretation through visualiztion for black box models such as Deep Reinforcement Learning.

Practice for Faster Reinforcement Learning

Practice is a type of transfer learning approaches that relax the requirements of transfer source tasks to non-RL tasks. Without any pre-knowlege on the target problem, through strategic or non-strategic practice, we expect to improve learning speed and reduce the necessary samples for training various types of reinforcement learning agents. Also, practices are expected to reach general intelligence.

Other Application Topics

Adaptive Robot Motion Control, Co-Creative Sketch Robots, Adaptive Networking, WiFi-Powered Biometry, Brain Computer Interfaces, Multi-task Learning, Dialogue Generation, Urban Data Analytics, Computational Fluid Dynamics

Past Projects

Pretraining Deep Networks for Faster Reinforcement Learning

Reinforcement learning requires a goal and a reward function to guide an agent to learn a good policy. However, sometimes, this training process can be expensive or slow. In supervised learning, deep learning pre-trains the networks to overcome slow and unstable learning. Similarly, this research suggests a way to train deep networks with state change predictions and without any goal or reward signal. The proposed approach shortens the learning time and improves the learned policies.

Continuous Action Search over Neural Networks

An octopus arm is a hard problem that requires a complex, high dimensional control to curl or release an arm to reach an object or prey. This research focuses on stable learning with continuous valued action control. For this, we extend Greedy-GQ for multi-layered neural networks and derived gradient descent for continuous action search through back-propagation.

Function Approximations for Robust Reinforcement Learning

Function approximation is one of the key components for successful reinforcement learning. Thus, choice of function approximation and its training algorithm are crucial. Especially in the continuous domain and without estimating the value functions, it is impossible to solve these problems. We examine various function approximations such as linear regression, neural networks, radial basis functions, restricted gradient descent, support vector machines, and relevance vector machines. I implemented for these algorithms in a general machine learning package, JML (available at https://bitbucket.org/lemin/jml).

Robust High Dimensional Clustering

Classical Gaussian mixtures are sensitive to outliers and fail to maintain true centers. This results in poor clustering and this problem becomes worse as the dimensionality grows. By adopting t-distributions for each dimension, we suggest robust clustering solutions for various high dimensional clustering problems.

Transfer Learning Survey

Not only for reinforcement learning, machine learning researchers have studied efficient ways to learn by exchanging learned knowledge. As part of research exam and exploration of dissertation topics, I surveyed transfer learning and categorized common algorithms. This study was developed to practice for reinforcement learning research.

Hurricane Detection from Global Forecasting System Data

The National Centers for Environmental Prediction (NCEP) Global Forecast System (GFS) is a global spectral model used for aviation weather forecasting. It produces forecasts of wind speed and direction, temperature, humidity and precipitation out to 192 hr every 6 hr over the entire globe. Although not specifically designed for tropical cyclones, the model solutions contain smoothed representations of these storms. One of the challenges in using global atmospheric models for hurricane applications is objectively determining what is a tropical cyclone given the three dimensional solutions of atmospheric variables. To address this issue, without manually selecting features of interests, the initial conditions from a low resolution version of the GFS are examined and compared with the known positions of tropical cyclones by using various classification tools.

Multi-agent Reinforcement Learning

Multi-agent reinforcement learning has been studied for either cooperating with or competing against other agents. Depending on the problem, different evaluation metrics have been used. Based on unified metrics for both problems, we suggest scalable and efficient multi-agent reinforcement learning algorithms that can learn how to compete or to cooperate with other intelligent, adaptable agents.

I3D2

I3D2 is an intelligent transportation system (ITS) that was supported by the Ministry of Science and Technology in Korea. I3D2 is an intelligent, integrated, interactive and distributed microscopic transporation simulation environment. Using discrete event system specification (DEVS) formalism, traffic network, vehicles with drivers, and traffic management sytems are modeled independently and hierarchically. I developed the HLA/DEVS simulation engine (DeSym/HLA) for distributed simulation, models, and the simulation interfaces.

SECUSIM

SEUCSIM is a cyber attack simulation tool that models malicious attacks and defense mechanisms and evaluates the consequences. I developed symbolic simulation engine (SDeSym) to generate autonomous attacks.