Hi, my name is Richard Li

An undergraduate at Columbia studying Computer Science and East Asian Studies

ML / AI engineer & researcher

Richard eating pizza in New York

I’m an undergraduate at Columbia University, class of 2028, studying Computer Science and East Asian Studies. I love research Retrieval on GPUs @ IBM ResearchOnboard VLMs @ NASA JPLMedical VLMs @ MicrosoftRobot learning @ ColumbiaSingle-cell genomics @ UW, I like building things that solve real problems Comic translation @ ToonsutraCampus food delivery @ BiteAI-generated figures @ AstraSomnia, and I’m always exploring new tech Robot wushu · work in progress and finding ways to apply it to my own life.

At any given moment, you’ll find me working on my projects ClimbSimMars to Table. In my free time, I like doing wushu, rock climbing, singing & dancing, playing jazz, and visiting NYC’s museums.

Take a look at what’s on my mind →

Robotics & embodied intelligence

CAVR

Foundation-model representations for robotic control (work in progress)

Three frozen encoders“pick up the block”V-JEPAfills in masked video,as features, not pixelsWorld modelgenerates the framesthat come nextVLAturns a frame and wordsinto robot actions64 tokens256 tokensfrom depth 1 of 4from depth 2 of 4from depth 3 of 4from depth 4 of 4Shared policy headthe same for every encoder“put the block on the plate”no instructionactions✓ successthe camera frame goes back in, every step30 LIBERO tasks900 rollouts per model, scoredon task success and a calibratedrepresentation-similarity score16 settings per encoder4 depths × 2 token budgets × language,plus blind and random baselines;10,000-resample paired bootstrapThree frozen encoders“pick up the block”V-JEPAfills in masked video,as features, not pixelsWorld modelgenerates the framesthat come nextVLAturns a frame and wordsinto robot actions64 tokens256 tokensfrom depth 1 of 4from depth 2 of 4from depth 3 of 4from depth 4 of 4Shared policy headthe same for every encoder“put the block on the plate”no instructionactions✓ successthe camera frame goesback in, every step30 LIBERO tasks900 rollouts per model, scoredon task success and a calibratedrepresentation-similarity score16 settings per encoder4 depths × 2 token budgets × language,plus blind and random baselines;10,000-resample paired bootstrap
Which frozen representation is most useful for control? Every encoder stays frozen and feeds the same policy head, so the representation is the only thing that changes. The harness sweeps four depths, two token budgets and language on or off, against blind and random baselines, across 30 LIBERO tasks.

Video, world-model, and vision-language-action foundation models each represent the same robot actions in their own way, and it isn’t clear which of those representations actually helps a robot finish a task. I built a Python/PyTorch harness that feeds frozen V-JEPA, generative world-model, and vision-language-action representations into the same imitation-learning policy heads across 30 LIBERO tasks, so the representation is the only thing that changes. Each model is set up for 900 rollouts, with ablations over four representation depths, 64- vs. 256-token budgets, and language conditioning, plus blind and random baselines. I measure closed-loop task success and a permutation-calibrated representation-similarity score, using five-fold episode-grouped validation and a 10,000-resample paired bootstrap to quantify uncertainty.

ClimbSim

Scene reconstruction and contact modeling (work in progress)

mggLP: can these forceshold the climber?feasible ✓Checked onsynthetic scenes< 0.5%scale error< 0.5°off true gravity< 3 mmhold dimensions≤ 2%load-limit error24 testscalibration, contacts,segmentation, geometry1Filma phone sweepsthe wall2Rebuildphotogrammetry, ArUcoscale, gravity down3Collideholds segmented, cutinto convex pieces4Checkan LP over frictioncones, forces, limbsmggLP: can these forceshold the climber?feasible ✓1Filma phone sweepsthe wall2Rebuildphotogrammetry, ArUcoscale, gravity down3Collideholds segmented, cutinto convex pieces4Checkan LP over frictioncones, forces, limbsChecked onsynthetic scenes< 0.5%scale error< 0.5°off true gravity< 3 mmhold dimensions≤ 2%load-limit error24 testscalibration, contacts,segmentation, geometry
From phone video to a physics question. Photogrammetry rebuilds the wall, ArUco markers set its scale, and the scene is turned so gravity points straight down, with each hold split into convex pieces MuJoCo can collide with. Then a linear program asks whether forces inside every hold’s friction cone can keep the climber on the wall.

Vertical and overhanging surfaces show up in infrastructure inspection, planetary exploration, and rescue. A successful locomotive robot must move both horizontally and vertically. Climbing involves choosing which holds to use, in what order, without ever falling. I built a Python pipeline that turns phone video of a climbing wall into a metric, gravity-aligned MuJoCo scene, and a linear-programming solver that checks whether the forces on a set of holds can keep the body on the wall, the test every stance in a climb has to pass. The solver matched the analytical load capacity within 2% in a synthetic single-contact test, by modeling force and moment balance, friction constraints, and limb limits, with diagnostics for why a stance fails. Across five synthetic holds, the scenes came in under 0.5% scale error, under 0.5° gravity-alignment error, and under 3 mm dimensional error, using photogrammetry, ArUco-marker calibration, hold segmentation, and convex decomposition. The project passes 24 tests covering calibration, contact physics, segmentation, and MuJoCo geometry.

Robot Wushu

Work in progress

Wushu basics1Salutebaoquan li12Horse stancemabu, block and punch23Bow stancegongbu, punch34Snap kicktantui, with a punch45Knee lifttixi, arms open5Wushu basics1Salutebaoquan li12Horse stancemabu, block and punch23Bow stancegongbu, punch34Snap kicktantui, with a punch45Knee lifttixi, arms open5
What I’m working toward: a humanoid robot performing the basics of a wushu routine, from the opening salute through horse- and bow-stance strikes to a snap kick and a knee lift.

Robot Wushu is about understanding how robots can take part in arts and performance, starting with Wushu, the Chinese martial art I practice with Columbia Wushu. I’m working on getting a humanoid robot to perform the basics of a Wushu routine. It’s early days, so more soon.

Work & research

IBM Research

Fall 2026

questionpassageacrosswithinGPUIs water damage covered?1Warranty §4 · liquid damage2Warranty §2 · returns3Care guide · cleaningLLMYes, it’s covered under §4.Can I work abroad for a month?1Remote work §3 · abroad2Travel §1 · booking3Benefits · time offLLMYes, up to 30 days (§3).Where do failed uploads retry?1upload.py · retry_upload()2upload.py · validate()3README · setupLLMIn retry_upload(), with backoff.supportsupportpolicypolicycodecodeEnergy distancetwice the mean distance across,minus the mean within eachOn the GPUevery pair in parallel,with CUDAWhat it’s forfinding what an LLM readsbefore it answersquestionpassageacrosswithinEnergy distancetwice the mean distance across,minus the mean within eachGPUOn the GPUevery pair in parallel,with CUDAIs water damage covered?1Warranty §4 · liquid damage2Warranty §2 · returns3Care guide · cleaningLLMYes, it’s covered under §4.Can I work abroad for a month?1Remote work §3 · abroad2Travel §1 · booking3Benefits · time offLLMYes, up to 30 days (§3).Where do failed uploads retry?1upload.py · retry_upload()2upload.py · validate()3README · setupLLMIn retry_upload(), with backoff.supportsupportpolicypolicycodecodeWhat it’s forfinding what an LLM readsbefore it answers
Before an LLM answers, RAG retrieves the passages it will read, and the answer can only be as good as what it finds. Energy distance compares every embedding of the question with every embedding of a passage, as two distributions: twice the mean distance across them, minus the mean within each. That takes a distance for every pair, a whole matrix for each passage, and making that fast on GPUs with CUDA is our part. The same retrieval sits behind support assistants, company search and coding tools.

In RAG, retrieval decides which passages an LLM reads before it answers, in everything from support assistants to company search and coding tools. Energy distance has been theorized to beat cosine similarity at retrieval in early tests, but it needs a distance for every pair of embeddings, so it’s much slower. For my team project in Columbia’s High-Performance Machine Learning course, I’m working with IBM Research, mentored by Mr. Bhattacharjee, a Senior Technical Staff Member and IBM Master Inventor, to accelerate energy distance on GPUs with CUDA so it’s fast enough to use at scale. Results coming soon.

NASA Jet Propulsion Laboratory

Summer 2026

wildfireground stationTime to respondin simulationground in the loop72 minsatellites coordinate10 min7.2× fasterOnboard, every satellitedetecttriageplanGemma VLM agents in LangGraph85% triage accuracyLoRA fine-tune: +40% CLIPScorewhat a satellite seesinter-satellite linkwildfireground stationTime to respondin simulationground in the loop72 minsatellites coordinate10 min7.2× fasterOnboard, every satellitedetecttriageplanGemma VLM agents in LangGraph85% triage accuracyLoRA fine-tune: +40% CLIPScorewhat a satellite seesinter-satellite link
Three orbits from a simulated 10-satellite constellation, all passing over one fire. Each satellite runs a Gemma vision-language model onboard. The one overhead flags the fire and passes word back to the satellites behind it, so they arrive already tasked instead of waiting on the ground. In simulation, that cut the response from 72 minutes to 10.

When a satellite spots a complex disaster like a flood triggering a landslide, its images usually wait for a ground station pass, ML pass, and human review before anyone responds. A fast-moving disaster doesn’t leave that time. As a machine learning intern at JPL in Pasadena, advised by Dr. Steve Chien and Jake Lee, I built a LangGraph multi-agent system of onboard Gemma VLMs that detects, triages, and plans a response to disasters on the satellites themselves, so a constellation can act without waiting on the ground. Onboard coordination cut simulated disaster-response time from 72 to 10 minutes (7.2×), measured in a Python constellation simulator I built with inter-satellite links, downlink scheduling, and Contact Graph Routing. The system reached 85% triage accuracy across a simulated 10-satellite constellation. I improved Gemma’s CLIPScore on aerial imagery by 40% by fine-tuning it with two-stage LoRA and FSDP, and the adapter and its inference harness were handed off to JPL’s flight software team for a satellite mission. When BLEU/CIDEr scores proved misleading, I traced them to caption-length confounding and built an LLM-as-a-judge evaluation pipeline (Claude on AWS Bedrock) to score model outputs instead.

AstraSomnia

Summer 2026

{ "kind": "line", "x": "year", "y": "CO₂ (ppm)", "style": "paper"}✓ validhash it9f3c…e1misshit: serve the stored imageGeminiwrites a spec,not a pictureTyped specPydantic v2 schema,98% valid first tryCloud Storageeach image namedby its spec’s hashRenderermatplotlib, behinda feature flagSame byteson everyre-render{ "kind": "line", "x": "year", "y": "CO₂ (ppm)", "style": "paper"}✓ validhash it9f3c…e1misshit: serve the stored imageGeminiwrites a spec,not a pictureTyped specPydantic v2 schema,98% valid first tryCloud Storageeach image namedby its spec’s hashRenderermatplotlib, behinda feature flagSame byteson everyre-render
Gemini writes each figure as a typed spec instead of a picture. The spec is checked against a Pydantic v2 schema (98% pass on the first try) and hashed, and the hash is the image’s name in Cloud Storage. The matplotlib renderer only runs on a miss, and drawing the same spec twice gives the same bytes, so a cached image is always the right one.

Scientific figures made by an LLM are only useful if they’re valid and deterministic, which a model drawing pictures directly can’t promise. As a software engineer at AstraSomnia, I defined a typed Pydantic v2 schema that Gemini writes each figure in and built a deterministic matplotlib renderer for it, so every figure is checked before it’s drawn and the same spec always draws the same image. 98% of specs passed the schema on the first attempt (n=100), and re-renders were byte-identical. I cached rendered figures in Google Cloud Storage under a hash of each spec, so a figure is only drawn once, and shipped the renderer behind a feature flag with a regression test confirming nothing changed while the flag was off. I also built concurrency-safe session management for LLM agents in FastAPI/asyncio, deployed on Google Cloud Run with Firestore and Cloud Storage, and it sustained 100+ concurrent sessions in load testing.

Bite Campus Eats

Winter ’25

CBACBADEStudents’ phonesthousands of grad studentsReact Native and ExpoSupabase Authemail, magic link, Apple,Google, multi-factorprofilesone row per studentRow-level securityusing (auth.uid() = id)trigger on every changeAudit logappend-onlyCBACBADEStudents’ phonesthousands of grad studentsReact Native and ExpoSupabase Authemail, magic link, Apple,Google, multi-factorprofilesone row per studentRow-level securityusing (auth.uid() = id)trigger on every changeAudit logappend-only
Every query from a student’s phone carries who signed in, and Postgres checks it against an owner-only policy, so a request can only ever reach that student’s own row; anyone else’s is refused. A trigger copies every profile change into an append-only audit log.

Bite is a Columbia startup that delivers food from local restaurants to campus with no service fees, and its app has to keep each student’s account and data private. As a full-stack software engineer intern, I built Supabase Auth sign-in, Postgres row-level security, and trigger-based audit logging of every profile change for its React Native/Expo app, so each student can reach only their own data and every change is on record. It protects the accounts of thousands of graduate students that use the app.

Microsoft Research

Summer & Fall ’25

axial slicesFINDINGSLungs are clear. No nodules,consolidation or effusion.Heart size is normal.IMPRESSIONNo acute abnormality.3D CT volumeevery slice read at oncePatches3D featurespooledlearned, in 3DMLPGemma-3writes the report0.86 BERTScore F1on about 150 held-out volumes50 GB on disk at a time420× less disk21 TB archive21,000 CT-RATE volumesaria2c download → preprocess → train → evict8 × A100FSDPaxial slicesFINDINGSLungs are clear. No nodules,consolidation or effusion.Heart size is normal.IMPRESSIONNo acute abnormality.Patches3D featurespooledlearned, in 3DMLP3D CT volumeevery slice read at onceGemma-3writes the report0.86 BERTScore F1on about 150 held-out volumes50 GB on disk at a time420× less disk21 TB archive21,000 CT-RATE volumes8 × A100FSDParia2c download → preprocess → train → evict
Reading the whole scan, not one slice. Gemma-3 sees the CT volume through learned 3D pooling and an MLP projection, then writes the report (the one here is illustrative). Underneath, the data loader streams fixed-size chunks through a 50 GB window: download, preprocess, train, evict. That fit all 21,000 CT-RATE volumes, 21 TB, on one 8-GPU node.

Radiologists read CT scans in 3D, but most medical vision-language models that could draft their reports flatten the scan into 2D slices and lose that structure. As a machine learning intern at Microsoft Research in Redmond, advised by Dr. Ying Jin, I extended Gemma-3 with learned 3D pooling and an MLP projection so it reads the whole CT volume and writes the radiology report from it. Trained with an 8×A100 FSDP pipeline I built, it reached 0.86 BERTScore F1 on about 150 held-out volumes. I cut peak disk use 420× (21 TB to 50 GB) with a Python streaming pipeline (aria2c, PyTorch DataLoader) that downloads, preprocesses, and evicts fixed-size chunks, which made single-node training on all 21,000 CT-RATE volumes possible.

Toonsutra

Fall ’25

LOOK OUT!RUN!PHEW!VLMसावधान!Hindiসাবধান!Bengaliஜாக்கிரதை!Tamilజాగ్రత్త!TeluguA comic pagein English, per panelfind similar panelsTranslated beforethe closest panels come alongTranslatesreading the paneland its neighborsFour languagesLOOK OUT!RUN!PHEW!VLMसावधान!Hindiসাবধান!Bengaliஜாக்கிரதை!Tamilజాగ్రత్త!TeluguTranslated beforethe closest panels come alongfind similarpanelsA comic pagein English, per panelTranslatesreading the paneland its neighborsFour languages
Translating with the picture in view. Each panel’s art and words are matched against panels that were already translated, and the closest few go to the model along with the panel itself, so a line reads the way the art means it.

Toonsutra is a comics app serving 1M+ monthly active users, and in a comic the art changes what the dialogue means, so translating the words alone gets lines wrong that people then have to fix by hand. As an AI research intern, I helped develop image-conditioned RAG for comic translation in four South Asian languages, which gives the model each panel’s art along with its words.

University of Washington

Student researcher · 2023 – 2025

Hofbauer cellsfibroblaststrophoblastsendothelialreadsPlacentathree tissuesCell Rangerreads → countsSeurat, then SingleRquality control, clusters, cell typesTime to labelthe cellstens of thousands of themby hand, cluster by cluster7+ daysSingleR, from a referenceunder 12 hours90%+ less timedropped in quality controla cell, not yet labeledlabeled by SingleRHofbauer cellsfibroblaststrophoblastsendothelialreadsPlacentathree tissuesCell Rangerreads → countsSeurat, then SingleRquality control, clusters, cell typesTime to label the cellstens of thousands of themby hand, cluster by cluster7+ daysSingleR, from a referenceunder 12 hours90%+ less timedropped in quality controla cell, not yet labeledlabeled by SingleR
Seurat groups the cells; SingleR names each group from a labeled reference, not by hand.

Single-cell RNA sequencing shows what each cell in a tissue is doing, but a study’s conclusions are only as good as the data handling behind them, and technical noise in tens of thousands of cells can easily be read as biology. In Dr. Kristina Adams Waldorf’s lab at UW Medicine, I built a single-cell RNA-seq pipeline that removes that noise and labels cell types automatically. It became the backbone of our placenta atlas in BMC Genomics, where I’m co-first author. I filtered out empty droplets, dying cells, and doublets; normalized with SCTransform so differences in sequencing depth didn’t pass for differences in gene expression; and integrated the samples with Harmony so cells grouped by type rather than by the chip they were sequenced on. To compare cell-type proportions between groups, I used sccomp, a Bayesian compositional model that treats each sample, not each cell, as a replicate.

Publications & talks

AGU Fall Meeting 2026

Session IN021 · accepted for oral presentation

R. Li, J. H. Lee, J. Haug, S. A. Chien. Onboard Vision Language Models for Autonomous Multi-Agent Earth Monitoring.

BMC Genomics, 2026

*Equal contribution · preprint bioRxiv 2025

A. Li*, R. Li*, H. Huang, …, K. M. Adams Waldorf. A single-cell transcriptomic atlas of the pigtail macaque placenta in late gestation. doi:10.1186/s12864-026-13095-x

At Columbia

Courses

Data Structures and AlgorithmsData Structures and AlgorithmsOperating SystemsOperating SystemsHigh-Performance Machine LearningHigh-Performance Machine LearningComputational Aspects of RoboticsComputational Aspects of RoboticsDeep Learning for Computer VisionDeep Learning for Computer VisionDeep Learning for Robot ManipulationDeep Learning for Robot Manipulation8412261014846binary searchP1P2P3time →schedulingall-reducedata parallelism→ IBMmotion planningconvolutiongrasping→ CAVR

Courses I’ve taken include Operating Systems, High-Performance Machine Learning, Data Structures and Algorithms, Deep Learning for Computer Vision, Deep Learning for Robot Manipulation, and Computational Aspects of Robotics. Two of the projects on this page started in class: CAVR began as my project for Deep Learning for Robot Manipulation, and the IBM Research project is my team project for High-Performance Machine Learning.

Columbia Space Initiative

Sub-team lead, NASA SUITS

DINOv3: damageAR displayguides the moonwalk

In the Columbia Space Initiative, I lead a sub-team of our NASA SUITS mission. SUITS (Spacesuit User Interface Technologies for Students) is a NASA challenge in which student teams design and build augmented-reality displays to guide Artemis astronauts on moonwalks. We build the interface in Unity and C#, along with a DINOv3-based vision model that spots external damage on lunar terrain vehicles from the headset’s video.

ADI

Executive board, community co-lead

DevFestDevFestworkshopsworkshopsmentorshipmentorship

ADI, the Application Development Initiative, is the largest tech community at Columbia, serving 6.5K students. It runs DevFest, Columbia’s largest hackathon, along with workshops and mentorship programs for students at every level. On its executive board, I co-lead the community team, which builds spaces for students to meet, collaborate, and grow together.

Columbia Wushu

Vice president, now senior advisor

700+ businessesMaps API, PlaywrightGeminiwrites to eachOutreachpersonalized

Columbia Wushu is the largest collegiate Chinese martial arts club in New York City, and it performs on campus and around the city. I joined to mend my shoulders after years of swimming and weightlifting in high school, then served as the club’s vice president; now I’m a senior advisor. Along the way, I built the club’s website, columbiawushu.org, with embedded performance videos and automated email, and a sponsor-outreach pipeline (Python, Playwright, the Google Maps API, and Gemini) that found more than 700 businesses and personalized our outreach to each.

AWS Student Builder

Campus leader

workshopsworkshopseventseventsprojectsprojects

AWS Student Builder Groups, formerly AWS Cloud Clubs, are student-run communities for learning to build on the cloud. As a campus leader at Columbia, I bring students together for hands-on workshops, events, and projects on AWS.

Resident Advisor

My roomalongside my residents

As a Resident Advisor, I live alongside my residents: I help them settle into life at Columbia, build community where we live, and am someone they can come to when they need a hand.