IMULTRAKILLINGIT

The problem

Fast movement is only one constraint.

Completing a level requires correct progression, quick traversal, combat survival, and enough style for the target rank. A death can invalidate the attempt, while the nearest visible door may be locked behind an earlier switch or arena. The agent therefore has to reason about progression and risk while reacting at gameplay speed.

I organized the bot around explicit state ownership. Navigation controls movement during traversal; combat interrupts it and owns movement, camera, and weapons; recovery takes over when progress stalls. Each transition uses the current world state instead of assuming the previous plan succeeded.

Navigation

Semantic goals with physics-based traversal

A world scanner inspects loaded scene objects for progression cues such as doors, checkpoints, switches, elevators, arenas, and level exits. An objective selector scores them by completion state, availability, distance, reachability, progression value, and prior failures. This keeps a nearby locked exit from outranking the switch that actually unlocks it.

For local planning, the agent samples the Unity NavMesh, calculates and validates complete paths, and follows their corners as waypoints. It steers the player's Rigidbody velocity rather than teleporting. Doors and checkpoints use “through-targets” placed beyond their triggers, so reaching a waypoint also advances the game.

When no semantic objective is usable, frontier exploration samples reachable positions at several angles and distances, preferring unexplored spatial cells. NavMesh checks and downward raycasts reject unsafe ground before the bot commits to a route.

Hybrid demonstration-based routing

Human route intent, autonomous correction

Pure NavMesh geometry could not always express a level's intended progression: an object might be close through a wall while the real route loops through another arena. I added a second layer that records a human trajectory—position, view direction, and meaningful actions—rather than raw keyboard timing.

The recorded route supplies high-level progression guidance while the live movement controller handles physics and local variation. Playback localizes against the nearest compatible route point, so the bot can start mid-level. If combat moves the player away from the expected path, it searches forward for a suitable re-entry point and uses NavMesh recovery to reconnect.

Combat AI

Utility decisions under survival pressure

The combat system periodically scans active enemy components, filters invalid or occluded targets, and scores the remaining choices. Its utility function considers distance, threat, boss status, kill opportunity, style value, and the player's survival. The result is a changing answer to “which enemy matters most right now?” rather than a static nearest-target rule.

Combat coordinates camera rotation, weapon selection, firing, and movement as one action. The bot approaches distant targets, strafes near threats, jumps conditionally, and uses raycasts for basic obstacle avoidance. It discovers the weapons available on the current save and rotates among their existing game components instead of assuming every weapon has been unlocked.

Recovery and fault tolerance

Plans are disposable; current state is authoritative.

The agent compares intended movement with actual displacement and progress toward the active waypoint. When it detects a stall, it escalates from reversing and side-stepping to re-aiming, recalculating the path, selecting another target, and eventually abandoning an objective after repeated failures.

Failure counts are tracked per objective to prevent infinite retry loops. The same principle handles unreachable NavMesh points, route divergence, unsafe drops, bad semantic choices, and the large position changes caused by combat: re-localize, re-evaluate, and continue from where the player actually is.

Game integration

Working inside a compiled Unity game

The agent runs as a BepInEx 5 BaseUnityPlugin. I inspected ULTRAKILL's managed assemblies to integrate with internal components including movement, weapons, enemies, doors, and checkpoints. A late execution order lets the plugin react after normal gameplay systems update.

High-level decisions, scanning, aiming, and actions run in Update(); physics steering runs in FixedUpdate(). Keeping those responsibilities aligned with Unity's loops avoids mixing frame-rate decisions with fixed-step movement.

Route data is stored per scene in an invariant-culture, pipe-delimited format. I replaced an unreliable JsonUtility version after it silently omitted route points, then added a startup round-trip check to verify that saved values survive serialization.

Architecture

Separate owners, shared live state

Bot controller
├── Navigation
│   ├── Objective scoring
│   ├── NavMesh planning + frontier exploration
│   ├── Demonstrated route + re-localization
│   └── Movement, hazard checks, and recovery
└── Combat
    ├── Enemy scanning + utility scoring
    ├── Aim + weapon actions
    └── Target-relative movement

Key technical challenges

Where the design changed

What I learned

Robust autonomy starts with admitting uncertainty.

The most useful design decision was treating every plan as a hypothesis. Scene geometry, prior waypoints, and expected movement can all be wrong after the world changes. Clear control ownership, observable failure conditions, and re-localization made the system far more reliable than adding more special-case commands.

The project also reinforced that game AI is a real-time systems problem as much as a planning problem: good decisions still fail if physics, update timing, camera control, and subsystem handoffs are not coordinated.

Technical highlights

• Plans from the player's live position, including mid-level activation
• Combines semantic objectives, NavMesh planning, and demonstrated routes
• Scores combat targets by threat, distance, opportunity, style, and survival
• Recovers from stuck movement, failed objectives, combat displacement, and unsafe paths
• Controls real player physics, camera aim, and installed weapons through game systems
• Validates persistent route data with a serialization round-trip check

Visuals to add

Suggested diagrams and screenshots

• A route overlay showing demonstration points and NavMesh recovery
• The navigation → combat → re-localization state transition
• An objective-debug view with scores for a door, switch, and checkpoint
• A combat frame showing the chosen target and utility factors
• A short clip of the agent recovering after combat displacement