About Me

Robin Luo

Computer Science PhD ∈ Northwestern University

I am a second-year Ph.D. student in the Computer Science department at Northwestern University, advised by Professor Han Liu in MAGICS lab.

My primary research interest lies in the intersection between Machine Learning, Health Care, and Financial Data.

Topics I am currently working on (Machine/Deep Learning):

  • Large Foundation Models
  • Memory Retrieval and Memory Enhanced Foundation Models
  • Modern Hopfield Network
  • Hyperparameter Optimization

Topics I had worked on in the past (Before my PhD study):

  • Multimodality includes Video Question Answering and Visual Question Answering
  • Humman Robotic Interation
  • Reinforcment Learning
  • Natural Language Understanding
  • Question Answering

Besides the school and research work, I have found my interests in lots of things over time including photography, hiking and traveling.

News


Education

Northwestern University - MAGICS Lab

Computer Science PhD @ Northwestern University

Sept. 2022

Northwestern University

GPA 3.92/4.0, MSIT @ Northwestern University (Transfer to CS PhD program)

Sept. 2021

Georgia Institute of Technology

GPA 3.7/4.0, VQA-related research @ Machine Learning Lab

Augest. 2020 - May 2021
MS in Computer Science

Georgia Institute of Technology

Graduate with High Hornor, GPA 3.53/4.0, NLP-related research @ Machine Learning Lab, Dean List

Jan. 2018 - May 2020
BS in Computer Science

Michigan State University

Transfer to Georgia Institute of Technology with GPA 3.65/4.0

Sept. 2016 - Dec 2017
BS in Computer Engineering

My Research

Northwestern University

Integrated Modern Hopfield Networks into the Knowledge-augmented Language Model to address limitation in self-attention architecture and Foundation Model, e.g. no-op outlier in the training;Explored the interpretability of Jailbreak strategies in safety-aligned LLMs by employing in-context learning and outlier analysis to enhance the alignment and safety of superhuman AI system; Applied LLMs to address multivariate time-series challenges, including in-context time series forecasting, Limit Order Book dynamics prediction, and Cross-sectional Regression Time-series Analysis; Developed efficient Hyperparameter Optimization (HPO) methods to optimize models for time series predictions with Tree-structured Parzen Estimator (TPE) and Genetic Algorithm.

Dec. 2021 - Present
Graduate Research Assistant

Northwestern University

Work in Distributed Deep Reinforcement Leaning by implementing IQN model with Wasserstein GAN algorithm; Designed policy evaluation experiment with Atari Game Database using the fixed policy inputting and compare the result with the baseline of the fixed policy; Developed policy optimization on the IQN model with the Wasserstein GAN algorithm

Sept. 2021 - Nov. 2021
Graduate Research Assistant

Northwestern University

Worked on model-free reinforcement learning to show SOTA performance with our physical simulator; Implemented PPO, SAC algorithm to get and plot the reward of the reinforcement learning in our physical simulator; Published a paper in progress- “ RobLAX- A Differentiable Robotics Framework for Physics Augmented Reinforcement Learning”

May. 2021 - Sept. 2021
Graduate Research Assistant

Georgia Institute of Technology

Worked on the brat problem and built the pipeline of the NLP labeling application; Researched on the polysemy problem and used the BERT model to label the target words and phrase; Researched on the Operational Transformation Algorithm to make sure the several contributors modifying together in cloud.

Sept. 2020 - May 2021
Graduate Research Assistant

Georgia Institute of Technology

Worked on Continuous Neuro-Symbolic VQA. In particular researched on the problem of soft logic function; Designed and implemented the reasoning model based on Bert, Bi-direction LSTM and Stack-NMN. Evaluated the performance with other VQA models; Researched on the Video Question Answering and designed the model with R(2+1)D to detect the action and movement of objects in videos; Publishing a paper in progress - Differentiable End-to-End Program Executor for Sample and Computationally Ef;icient VQA.

Aug. 2020 - Oct 2020
Graduate Research Assistant

Georgia Institute of Technology

Researched the visual question answering problem. Visual question answering using NS-VQA algorithm to classify the relationship of different objects in images and give out the answer of the GQA problem. Researched on the reasoning work and used the bi-directional LSTM to make some improvement on the reasoning work in VQA;Understood what the function of the VQA works and what the latest technology on solving the VQA problem

Aug. 2019 - May 2020
Undergraduate Research Assistant

Georgia Institute of Technology

Developed a pipeline capable of analyzing the adsorption energy in chemical reactions by using Python and ASE API; Implemented a machine learning model capable to analyzing the structure of chemical molecular.

Jan. 2019 - May 2020
DFT Model Analyze Adsorption Energies Team

Georgia Institute of Technology

Developed a pipeline that collects data provided by the stadium networks and analyzes collected data to give out a precise result to stadium staffs; Delivered a website tool capable to setting up a remote database for the LoPT database team.

Jan. 2018 - Dec 2018
VIP Georgia Tech – LoPt Database Team

Publication

Outlier Efficient Modern Hopfield Model for Large Transformer-Based Models

Review by ICML2024

Jan. 2024
Jerry Yao-Chieh Hu*, Pei-Hsuan Chang*, Haozheng Luo*, Hong-Yu Chen, Weijian Li, Wei-Po Wang, Han Liu

Chain-of-action- Faithful and Multimodal Question Answering through Large Language Models

Review by ICML2024

Jan. 2024
Zhenyu Pan, Haozheng Luo, Manling Li, Han Liu

SWGA- A Distributed Hyperparameter Search Method for Time Series Prediction Models

Review by ICML2024

Aug. 2023
Weijian Li*, Haozheng Luo*, Chenwei Xu, Han Liu

SMUTF - Schema Matching Using Generative Tags and Hybrid Features

Review by VLDB2024

Aug. 2023
Yu Zhang*, Di Mei*, Haozheng Luo*, Chenwei Xu, Richard Tzong-Han Tsai

Hyperparameter Search via Cyclic Block Coordinate Optimization

Review by ICML2024

Jan. 2023
Chenwei Xu, Haozheng Luo, Shifeng Li, Zhenyu Pan, Han Liu

IGN - Implicit Generative Networks

ICMLA 2022 - https://ieeexplore.ieee.org/abstract/document/10069446

March. 2022
Haozheng Luo, Tianyi Wu, Feiyu Han, Zhijun Yan

MBGDT:Robust Mini-Batch Gradient Descent

preprint in arxiv - https://arxiv.org/abs/2206.07139

Dec. 2021
Hanming Wang *, Haozheng Luo *, Yue Wang *

RobLAX-A Differentiable Robotics Framework for Physics Augmented Reinforcement Learning

https://sites.google.com/u.northwestern.edu/roblax/home

Sept. 2021
Guo Ye, Qinjie Lin, Tim Tsz-Kit Lau, Wanxin Jin, Cheng Zhou, Haozheng Luo, Zhuoran Yang, Zhaoran Wang, Han Liu

IBERT- Idiom Cloze-style reading comprehension with Attention

preprint in arxiv - https://arxiv.org/abs/2112.02994

May. 2021
Ruiyang Qin, Haozheng Luo, Zheheng Fan, Ziang Ren

SciAnnotate- A Tool for Integrating Weak Labeling Sources for Sequence Labeling

preprint in arxiv - https://arxiv.org/abs/2208.10241

Jan. 2021
Mengyang Liu *, Haozheng Luo*,Leonard Thong*, Yinhao Li, Chao Zhang, Le Song

Open-ended multi-modal relational reason for video question answering

RO-MAN 2023 - https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10309342

Dec. 2020
Haozheng Luo, Ruiyang Qin

HOW TO DESIGN SAMPLE AND COMPUTATIONALLY EFFICIENT VQA MODELS

preprint in arxiv - https://arxiv.org/pdf/2103.11537.pdf

Oct 2020
Karan Samel*, Zelin Zhao*, Kuan Wang*, Haozheng Luo, Binghong Chen, Song Le

Question ClassiNication with Deep Contextualized Transformer

FICC 2021 - https://link.springer.com/chapter/10.1007/978-3-030-73103-8_32

Oct 2019
Haozheng Luo, Ningwei Liu, Charles Feng

My Career

Georgia Institute of Technology

I grade students’ projects and exams and provide feedback to facilitate improvement; Have a solid understanding of algorithms in KMM, Gradient Model and Neural-Networks; Provide students with the means to succeed by holding ofkice hours and responding to questions online

Augest. 2020 - May 2021
Graduate Teaching Assistant

Splunk

I automated the process of sending security alerts through Slack and Email by developing an app that would be deployed on all windows machines at Splunk; Developed a pipeline that detects sensitive kiles used by customer support and implements the required retention policies.

June. 2020 - Augest 2020
Backend Software Engineering Intern

Splunk

I designed and developed a customer support tool that automatically suggests answers to customer questions, by using Python/React.js/ Flask/Electron/TensorFlow and NLTK; Created a pipeline that updates the customer support knowledge base with customer support feedback from the previous tool, it potentially saved 35% of the original time.

May 2019 - Augest 2019
Backend Software Engineering Intern

Michigan Republican Party

I delivered a pipeline capable of predicting Twitter usernames from local residents' real names; Implemented a visualization tool that describes the analysis with several kinds of statistic chart

Augest 2018 - September 2018
Data Team Developer

R2.ai Inc.

I contributed work to the construction of a website tool which provides customers cloud-computing server; Designed a pipeline that tests machine learning functions which are inside web servers and compares results with common machine learning algorithms using Python and TensorFlow.

June 2018 - Augest 2018
Machine Learning Summer Intern

DBAPP Security

I developed a pipeline capable of scrawling data from the Common Vulnerability and Exposures website using Python and Selenium; Delivered the scrawled data into PostgreSQL

May 2017 - July 2017
Programmer Analyst Intern

Talks

  • Oral Presentation on IEEE-RO-MAN 2023
  • Group Reading Group Presentation about Autoformer paper
  • Group Reading Group Presentation about Gradient Boosting Algorithm
  • Oral Presentation on FICC 2021
    •

    My Skills

    My Projects

    Snake Robot Simulator

    This project is my project. I designed and developed a Snake robot simulator with the ROS API ans python programming code.

    ...   ...

    Worldwide Attacking & Protection Web Project

    This project is other's project. I designed and implemented a web capable of describing Cybersecurity attacking behavior in the world by HTML5; delivered a pipeline that scrawls the data from Kaspersky website and sets up a database to store scrawled data

    ...   ...

    Object Relational Rest

    This is another project. I designed and implemented a Restful API with the PostgreSQL, greatly convince the non-technology user can use the database without limitation; delivered a React application to show up our Restful API on the computer

    ...   ...

    Chatbot

    This is another project. I designed and implemented a Rasa project to make a clothes chatbot application.

    ...   ...

    Reward & Patent

    Patents

    • Movable infusion stand (201120560457X)
      Robin Luo
      December 2013 in Hangzhou

    Other

    • High Hornor of Graduateions
      May 2020 in Atlanta

    Miscellaneous

    30-30 Project

    Hopefully before I am 30-year-ago, I can


    Contact Information

    2233 Tech Drive
    Seeley Mudd Room 3410
    Evanston, IL 60208