Shimizu: Autonomous Site Survey & Mapping

At Carnegie Mellon’s AirLab, I’m developing autonomous aerial survey capabilities for Shimizu, a Japanese construction firm. The broader goal is to monitor excavation progress and turn aerial observations into useful site-level information. My work so far brings together coverage-driven flight planning, semantic 3D mapping, and real-world flight testing in CMU’s HighBay.

In the September 28, 2026 test flights, both drones observed 100% of their required coverage targets, with at least two camera frames per target and one battery per drone. The drones flew separately, and their maps were merged afterwards into one site map.

Combined HighBay semantic map with coverage targets and paths from two separate drone flights — Two flights, one map. Magenta and orange traces show the two drones' paths; green spheres mark coverage targets.

Project Videos

Flight Demonstration

Watch the flight video on Google Drive

Foxglove Visualization

Watch the Foxglove recording on Google Drive

AirStack Mesh Simulation

A simulation of the mesh in AirStack, complementing the real-world flight and mapping recordings above.

Watch the AirStack simulation on Google Drive

From Sensing to Autonomous Flight

The drone gathers camera images, geometry, and position data while a ground-station laptop updates the maps, selects the next viewpoint, and plans an obstacle-avoiding path. Data passes over Wi-Fi, and the drone executes the planned path. After each flight attempt, the system updates coverage and repeats until all required targets have been observed.

The flight platform is a ModalAI Starling 2 Max with an added Livox LiDAR. RayFronts provides open-vocabulary semantic mapping on the ground station. A pilot handles takeoff and landing and can take over during the mission; autonomous movement begins after the operator enables it.

Coverage-Driven Viewpoint Planning

The survey volume contains 132 coverage balls arranged in an 11 × 3 × 4 grid, with 5 m horizontal spacing, 1.2 m vertical spacing, and a 1 m radius. These balls represent places the camera needs to see; one camera view can observe several balls.

A target counts as covered only after it appears in at least two different camera frames. It must be in front of the camera, inside the image, within 8 m, and visible before an intervening surface. The planner scores candidate viewpoints by balancing new coverage against travel distance and a height preference, then builds a safe path to the chosen viewpoint.

Semantic 3D Mapping

The perception pipeline assigns labels from an 11-class vocabulary without training on this site: floor, wall, window, door, table, chair, cart, shelving, roof, steel beam, and light. Observations from within 8 m are combined into a 3D map made of 30 cm voxels.

Geometry-based rules refine the camera labels, particularly for floors and steel beams. In the comparison shown in the project presentation, the share labeled as floor increased from 27% to 41% after applying those rules; this measures the label distribution, not a validated accuracy score.

Before geometry refinement: RayFronts floor and steel-beam predictions in the fused HighBay map — **Before — RayFronts labels.** Yellow-green marks floor predictions and purple marks steel-beam predictions.

After geometry refinement: a more continuous floor region and refined overhead beams and columns, shown from the same viewpoint — **After — geometry and label propagation.** Shape rules extend labels along the floor plane and connected steel structures. These are model predictions, not ground-truth annotations.

Flight Results & Current Work

September 28 test	Required targets covered	Excluded targets	Battery use
Drone 1	131 / 131	1	1 battery
Drone 2	130 / 130	2	1 battery

Coverage is calculated over non-excluded targets, with two camera observations required for each. These flights demonstrated complete required-target coverage and a combined map from separate drone runs. Current work focuses on improving semantic label quality for roofs, steel beams, and lights, including testing image settings to reduce label noise.

View the Shimizu project presentation

Broader AirLab Research

At Carnegie Mellon’s AirLab, I research autonomous aerial systems at the intersection of physical AI and robotics. My work focuses on two main threads: extending the capabilities of AirStack — an open-source drone autonomy stack — and building real-world applications that bridge simulation and deployment.

AirStack

AirStack is an open-source drone autonomy platform built on NVIDIA Isaac Sim, designed to provide a full-stack foundation for autonomous flight — from low-level control to high-level mission planning. My contributions focus on:

Enhancing simulation fidelity and sim-to-real transfer pipelines
Improving system safety margins and flight operations robustness
Developing autonomy modules that generalize across drone platforms

The goal is to give researchers and developers a reliable foundation so they can focus on algorithm development rather than rebuilding infrastructure — the same philosophy that drove SWARM at Purdue.

Largest Drone Ever Built at CMU

As part of CMU’s Large Air Vehicle Development Course (Robotics Institute), I was part of the team that designed, built, and flew the largest drone in Carnegie Mellon history.

The platform: a dodecacopter (12-motor) with a custom carbon fiber airframe
Weight: ~140 lbs
Thrust: ~600 lbs
First flight: September 30th — on deadline

The project covered the full engineering stack:

Mechanical & Structural

Hands-on design and fabrication of a large-scale carbon fiber airframe, with structural analysis to validate load-bearing components and quality wire soldering throughout.

Propulsion

Twelve 6S LiPo batteries powering a high-current ESC and motor controller system delivering ~600 lbs of total thrust — requiring careful thermal management and power routing.

Controls

ArduPilot integration with dual-GPS redundancy, followed by systematic PID tuning to achieve stable flight for a multirotor of this scale and inertia. Multirotor dynamics at this size are fundamentally different from smaller platforms.

Autonomy Foundations

Developing a vehicle model inside AirLab’s AirStack on NVIDIA Isaac Sim, laying groundwork for full autonomous operations via sim-to-real transfer.

After 5–6 hour days in the lab leading up to the deadline, stable first flight was achieved on September 30th. The project continues with work on extended flight time and full autonomy.

Research Focus

My broader research at AirLab sits in physical AI — integrating large language models, transformers, and vision systems to build robots that can reason about and operate in the real world. Drones are a compelling testbed: they must fuse sensor data in real time, plan under uncertainty, and execute in dynamic environments where failure has real consequences.

CMU AirLab