Zhengran Ji

I am currently a first-year PhD student in Computer Sceince at Duke University in Durham NC, United States. My advisor is Prof. Boyuan Chen. I recived my BS in Mathematics and Computer Science from University of California, Irvine (UCI), my advisor was Prof. Huolin Xin.

I am currently working on Human-guided Reinforcement Learning, Multi-Agent Reinforcemnet Learning, and RLHF.

Email  /  CV  /  Scholar  /  Twitter  /  Github

Research

I am generally interested in reinforcement learning, multi-agent/robot systems, human-in-the-loop learninng, and RLHF.

Collective intelligence depends not only on the capabilities of individual members, but also on how those members are organized. Yet artificial multi-agent systems are typically assembled using fixed organizational structures, even when the physical tasks they perform impose fundamentally different coordination requirements. Here we show that principles from human organization theory can be operationalized to organize large, heterogeneous collectives of embodied artificial agents. We introduce ORCH (Organizing Roles and Coordination Hierarchies), which constructs task-specific hierarchical organizations by combining pooled interdependence for work that can proceed concurrently with sequential interdependence for work governed by prerequisite relationships. Across 25 wildfire-response missions spanning reconnaissance, rescue, transportation, resource management, containment and suppression, we evaluated teams of up to 50 heterogeneous agents using eight large language models. Organizations constructed using these principles consistently outperformed four representative embodied multi-agent approaches across mission outcome, execution efficiency, exploration and computational resource use. Human-designed ORCH organizations improved final score by 63.97% and execution efficiency by 74.29% on average relative to the four prior frameworks. Organizations generated automatically by language models improved these measures by 43.63% and 52.53%, respectively. These advantages persisted across missions and underlying language models. Notably, collective performance was not monotonically determined by model scale. Analysis of long-horizon missions showed that hierarchical organization enabled teams to preserve concurrent activity within specialized groups while coordinating ordered transitions between mission phases. These results establish organizational design as a fundamental dimension of artificial collective intelligence and suggest that principles developed to understand human organizations can guide the construction of scalable embodied AI collectives.

We introduce Pref-GUIDE, a novel method that convert real-time human feedback into preference dataset for continual training after human left the loop. Pref-GUIDE further leverages the population power by aggregating feedback from multiple evaluators through voting to generate soft-label preference dataset to train more robust reward models.

We introduce HUMAC, a novel framework that enables multi-robot collaboration from single-human guidance. HUMAC leverages a novel human-robot interface that allows a single human to guide multiple robots simultaneously. We demonstrate the effectiveness of HUMAC in a challenging Multi-agent Hide and Seek game both in simulation and real-world setting.

We introduce GUIDE, a framework for real-time human-guided reinforcement learning that accelerates policy learning by integrating continuous human feedback into dense rewards. GUIDE also features a simulated feedback module that learns human feedback patterns, reducing the need for human input while enabling ongoing training. Tested on tasks with sparse rewards and visual observations, GUIDE achieves a 30% higher success rate with just 10 minutes of human feedback compared to standard RL methods. Our human study with 50 participants highlights the effectiveness of our approach.

We introduce a platform for Human-AI teaming research. CREW offers extensible environment design, enables real-time human-AI communication, supports hybrid Human-AI teaming, parallel sessions, multimodal feedback, and physiological data collection, and features ML community-friendly algorithm design.

We developed a deep learning model to decompose mixed Mn oxidation states from Mn L2,3 edges in EELS and XAS, without the need for reference spectra or calibration. Trained on a large synthetic dataset, the model achieves high accuracy and robustness, successfully validating against experimental Mn oxide data for automated analysis.

We propose a CNN-BiLSTM neural network to automate the detection and identification of core-loss edges in EELS spectra, overcoming challenges like low signal-to-noise ratios and weak edges in raw data. Trained on a synthesized spectral database modeled after real experimental data, the network achieves 94.9% accuracy, enabling fast, accurate core-loss edge recognition without manual preprocessing.

Website template from Jon Barron.