Client Application

The Hyperscape client is a React-based web application with Three.js WebGPU rendering. It provides a modern MMORPG experience with VRM avatars, responsive UI panels, and real-time multiplayer.

Client code lives in packages/client/src/. The rendering systems are in packages/shared/src/systems/client/.

Architecture Overview

┌─────────────────────────────────────────────────────────────────┐
│                       React Application                         │
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────────────────┐  │
│  │ Auth Flow   │  │  Screens    │  │    Game Panels          │  │
│  │ PrivyAuth   │  │ - Login     │  │ - Inventory  - Skills   │  │
│  │ Wallet Auth │  │ - CharSel   │  │ - Equipment  - Bank     │  │
│  │             │  │ - GameClient│  │ - Combat     - Chat     │  │
│  └─────────────┘  └─────────────┘  └─────────────────────────┘  │
├─────────────────────────────────────────────────────────────────┤
│                    Three.js WebGPU Renderer                     │
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────────────────┐  │
│  │ Scene Graph │  │  Cameras    │  │    Post-Processing      │  │
│  │ Terrain     │  │ Third Person│  │ - Bloom                 │  │
│  │ Entities    │  │ First Person│  │ - Tone Mapping          │  │
│  │ VRM Avatars │  │ RTS Mode    │  │ - Color Grading         │  │
│  └─────────────┘  └─────────────┘  └─────────────────────────┘  │
├─────────────────────────────────────────────────────────────────┤
│                      Client World                               │
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────────────────┐  │
│  │  Network    │  │    Input    │  │     Systems             │  │
│  │  WebSocket  │  │  Keyboard   │  │ - Graphics  - Camera    │  │
│  │  Reconnect  │  │  Mouse      │  │ - Audio     - Loader    │  │
│  │             │  │  Touch      │  │ - Health    - XP Drops  │  │
│  └─────────────┘  └─────────────┘  └─────────────────────────┘  │
└─────────────────────────────────────────────────────────────────┘

Entry Point

The client entry point is packages/client/src/index.tsx:

// Main entry point for the Hyperscape browser client
import { World, installThreeJSExtensions } from "@hyperscape/shared";
import React from "react";
import ReactDOM from "react-dom/client";

// Screens
import { LoginScreen } from "./screens/LoginScreen";
import { CharacterSelectScreen } from "./screens/CharacterSelectScreen";
import { GameClient } from "./screens/GameClient";

// Authentication (Privy + Wallet)
import { PrivyAuthProvider } from "./auth/PrivyAuthProvider";

// Embedded mode support for spectator views
import { EmbeddedGameClient } from "./components/EmbeddedGameClient";
import { isEmbeddedMode } from "./types/embeddedConfig";

Authentication Flow

Authentication uses Privy for wallet-based login:

Login Screen → Privy Modal → Wallet Signature → Character Select → Game

// auth/PrivyAuthProvider.tsx
export function PrivyAuthProvider({ children }: { children: React.ReactNode }) {
  return (
    <PrivyProvider
      appId={PRIVY_APP_ID}
      config={{
        loginMethods: ["wallet", "email", "google"],
        appearance: { theme: "dark" },
      }}
    >
      {children}
    </PrivyProvider>
  );
}

Screens

Wallet connection via Privy
Social login (email, Google)
Session persistence

Character Select Screen

List existing characters
Create new character
Character preview with VRM avatar

Game Client

Main game loop
World initialization
UI overlay rendering

// screens/GameClient.tsx
export function GameClient({ characterId }: { characterId: string }) {
  const worldRef = useRef<World | null>(null);
  
  useEffect(() => {
    const world = new World({
      isServer: false,
      systems: clientSystems,
    });
    
    world.connect({ characterId });
    worldRef.current = world;
    
    return () => world.destroy();
  }, [characterId]);
  
  return <CoreUI world={worldRef.current} />;
}

UI Components

Core UI (CoreUI.tsx)

The main UI wrapper that renders HUD elements and game panels:

export function CoreUI({ world }: { world: ClientWorld }) {
  const [ready, setReady] = useState(false);
  const [deathScreen, setDeathScreen] = useState(null);
  
  // Event handlers for loading, death, disconnect
  useEffect(() => {
    world.on(EventType.READY, () => setReady(true));
    world.on(EventType.PLAYER_DIED, handleDeath);
    world.on(EventType.DISCONNECTED, handleDisconnect);
  }, [world]);
  
  if (!ready) return <LoadingScreen />;
  
  return (
    <>
      <Sidebar world={world} />
      <StatusBars world={world} />
      <Chat world={world} />
      <ActionProgressBar world={world} />
      <XPProgressOrb world={world} />
      {deathScreen && <DeathScreen {...deathScreen} />}
    </>
  );
}

Game Panels

Located in packages/client/src/game/panels/:

Panel	Description
`InventoryPanel.tsx`	28-slot inventory with drag & drop
`EquipmentPanel.tsx`	Equipment slots with stat display
`SkillsPanel.tsx`	All skills with XP progress bars
`CombatPanel.tsx`	Combat style selection
`BankPanel.tsx`	Bank interface with tabs
`DialoguePanel.tsx`	NPC conversation UI
`StorePanel.tsx`	Shop/trading interface
`SettingsPanel.tsx`	Game settings
`LootWindow.tsx`	Ground item pickup interface

HUD Components

Located in packages/client/src/game/hud/:

StatusBars.tsx - Health, prayer, run energy bars
XPProgressOrb.tsx - XP tracking orb with skill icons
ActionProgressBar.tsx - Skilling action progress
EntityContextMenu.tsx - Right-click context menus
Minimap.tsx - RS3/OSRS-accurate top-down map view with location icons
Nametags.ts - Entity name/level labels

Minimap System

The minimap provides RS3/OSRS-accurate navigation with location icons and entity tracking. Visual Standards (OSRS-accurate):

White dots: Other players
Yellow dots: NPCs, mobs, and buildings
Red dots: Ground items and loot
White square: Local player (distinct from other players)
Red flag: Destination marker (RS3-style)

Location Icons: The minimap displays icons for key locations instead of generic dots:

Icon	Location Type	Visual
$	Bank	Gold coin symbol on gold circle
Bag	Shop	Open-top bag silhouette
Cross	Prayer Altar	White cross (religious symbol)
R	Runecrafting Altar	Purple circle with “R”
Anvil	Smithing Station	Dark anvil silhouette
Flame	Furnace	Orange circle with flame
Steam	Cooking Range	Brown circle with steam lines
Fish	Fishing Spot	Cyan circle with fish
Pickaxe	Mining Rock	Brown circle with pickaxe
Tree	Woodcutting	Green circle
!	Quest Available	Blue circle with white ”!”
?	Quest In Progress	Blue circle with white ”?”

Quest Icon States: Quest giver NPCs display state-aware icons based on quest progress:

// From Minimap.tsx
// Quest icon with state awareness
if (serviceTypes?.includes("quest")) {
  const questIds = npcConfig?.questIds;
  const statuses = questStatusesRef.current;
  
  if (questIds && questIds.length > 0 && statuses.size > 0) {
    let hasAvailable = false;
    let hasActive = false;
    let allCompleted = true;
    
    for (const qid of questIds) {
      const state = statuses.get(qid);
      if (state === "available") hasAvailable = true;
      else if (state === "active") hasActive = true;
      if (state !== "completed") allCompleted = false;
    }
    
    if (hasAvailable) {
      subType = "quest_available";  // Blue "!"
    } else if (hasActive) {
      subType = "quest_in_progress";  // Blue "?"
    }
    // If all completed, no quest icon (shows bank/shop icon if applicable)
  }
}

Quest Status Synchronization: The minimap fetches quest statuses from the server and updates icons in real-time:

// Fetch quest list from server
world.network?.send("getQuestList", {});

// Listen for quest status updates
world.network?.on("questList", (data) => {
  const mapped = data.quests.map((q) => ({
    id: q.id,
    state: mapStatus(q.status),  // "not_started" → "available", etc.
  }));
  
  // Update ref for synchronous access in entity loop
  questStatusesRef.current = new Map(mapped.map(q => [q.id, q.state]));
});

// Re-fetch on quest events
world.on(EventType.QUEST_STARTED, fetchQuestList);
world.on(EventType.QUEST_PROGRESSED, fetchQuestList);
world.on(EventType.QUEST_COMPLETED, fetchQuestList);

Icon Behavior:

Blue ”!”: NPC has at least one available quest (not started)
Blue ”?”: NPC has at least one quest in progress
No icon: All quests completed (shows bank/shop icon if NPC provides those services)
Icons disappear when all of an NPC’s quests are completed

NPC Configuration: Quest giver NPCs must have questIds field in their configuration:

{
  "id": "quest_giver_lumbridge",
  "name": "Quest Giver",
  "services": {
    "types": ["quest"],
    "questIds": ["cooks_assistant", "sheep_shearer"]
  }
}

The questIds field is passed through the spawn → network → client pipeline so the minimap can identify quest giver NPCs and display appropriate icons. Icon Detection: Icons are automatically assigned based on entity configuration:

// From Minimap.tsx
// Detect NPC service type for minimap icons
const npcConfig = entity.config;
const serviceTypes = npcConfig?.services;  // Fixed: services is string[], not { types: string[] }
if (serviceTypes?.includes("bank")) {
  subType = "bank";
} else if (serviceTypes?.includes("shop")) {
  subType = "shop";
}
// Quest icon detection (with state awareness) shown above

// Detect resource subtype for minimap icons
const resConfig = entity.config;
if (resConfig?.resourceType === "fishing_spot" || resConfig?.harvestSkill === "fishing") {
  subType = "fishing";
} else if (resConfig?.resourceType === "mining_rock" || resConfig?.harvestSkill === "mining") {
  subType = "mining";
} else if (resConfig?.resourceType === "tree" || resConfig?.harvestSkill === "woodcutting") {
  subType = "tree";
}

Bug Fix (PR #885): Fixed broken NPC service detection cast. The services field is string[], not { types: string[] }. This also fixed bank/shop icon detection which was previously broken.

Size Hierarchy:

Entity dots: 6px diameter (compact for clarity)
Location icons: 12px diameter (prominent for navigation)

Rendering:

3D terrain rendered via orthographic camera (throttled to ~15fps for performance)
2D overlay canvas for entity pips and icons (60fps for smooth interaction)
Cached projection-view matrix keeps pips synced with throttled 3D render
Pre-allocated vectors avoid GC pressure in render loop

Features:

Click-to-move on minimap
Scroll to zoom (20-1000 tile extent)
Rotates with main camera (RS3-style)
Destination marker persists until reached
Resizable and collapsible
Edit mode support for UI customization

3D Graphics System

WebGPU Renderer

The graphics system uses Three.js with WebGPU for high-performance rendering:

WebGPU Required: All rendering uses TSL (Three Shading Language) which only works with WebGPU. There is no WebGL fallback. Requires Chrome 113+, Edge 113+, or Safari 18+ (macOS 15+). Check compatibility at webgpureport.org.Breaking Change (Feb 2026): WebGL fallback was completely removed. All shaders use TSL which only compiles to WebGPU. Users on browsers without WebGPU will see an error screen with upgrade instructions.

WebGPU Initialization Timeouts: The renderer includes timeout protection to detect GPU driver issues:

// 30s timeout on adapter request
const adapter = await Promise.race([
  navigator.gpu.requestAdapter(),
  new Promise((_, reject) => 
    setTimeout(() => reject(new Error('WebGPU adapter request timed out')), 30000)
  )
]);

// 60s timeout on renderer initialization
await Promise.race([
  renderer.init(),
  new Promise((_, reject) => 
    setTimeout(() => reject(new Error('WebGPU renderer initialization timed out')), 60000)
  )
]);

These timeouts help diagnose misconfigured GPU servers where WebGPU initialization hangs indefinitely.

// systems/client/ClientGraphics.ts
export class ClientGraphics extends SystemBase {
  private renderer: WebGPURenderer;
  private postProcessing: PostProcessingComposer;
  
  async init() {
    // WebGPU is required - no fallback
    if (!navigator.gpu) {
      throw new Error('WebGPU not supported. Please use Chrome 113+, Edge 113+, or Safari 18+');
    }
    
    this.renderer = await createRenderer({
      powerPreference: "high-performance",
      antialias: true,
    });
    
    // Configure shadows (Cascaded Shadow Maps)
    configureShadowMaps(this.renderer, {
      cascades: 3,
      shadowMapSize: 2048,
    });
    
    // Post-processing (TSL-based, WebGPU only)
    this.postProcessing = createPostProcessing(this.renderer, {
      bloom: true,
      toneMapping: true,
      colorGrading: true,
    });
  }
}

Post-Processing Pipeline: The post-processing system uses TSL (Three Shading Language) for all effects:

// From ClientGraphics.ts
// All post-processing uses TSL node materials (WebGPU only)
this.composer.render();

// LUT color grading with intensity control
if (!this.colorGradingEnabled) {
  this.lutPass.intensity = 0.0;
}

Post-Processing Effects (TSL-based):

Bloom: Glow effects for magical items and particles
Tone Mapping: HDR to LDR conversion with auto-exposure
Color Grading: LUT-based color correction
Outline: Entity highlighting for hover/selection

Fix Applied (PR #829):

Issue: When LUT was “none” but hover triggered outline, stale LUT data leaked into rendering
Solution: Zero LUT intensity when disabled so outline-only rendering stays clean
Benefit: Entity highlights work correctly even when color grading is disabled

Rendering Pipeline

Pre-render: Update matrices, frustum culling
Shadow Pass: Render cascaded shadow maps (CSM)
Main Pass: Render scene with deferred lighting
Post-Processing: Bloom, tone mapping, color grading (TSL-based)
UI Overlay: Render 2D React UI on top

All rendering uses WebGPU - there is no WebGL fallback path.

Model Loading & Transform Baking

The ModelCache system handles GLTF model loading with transform baking to prevent rendering issues:

// From ModelCache.ts
private bakeTransformsToGeometry(scene: THREE.Object3D): void {
  // Ensure all matrices are up to date
  scene.updateMatrixWorld(true);

  // Apply transforms to each mesh's geometry
  scene.traverse((child) => {
    if (child instanceof THREE.Mesh && child.geometry) {
      // Clone geometry to avoid modifying shared geometry
      child.geometry = child.geometry.clone();

      // Apply world matrix to geometry (Three.js built-in method)
      child.geometry.applyMatrix4(child.matrixWorld);

      // Reset transform to identity
      child.position.set(0, 0, 0);
      child.rotation.set(0, 0, 0);
      child.scale.set(1, 1, 1);
      child.updateMatrix();
    }
  });
}

Why Transform Baking? GLTF files can have transforms stored in various ways:

Position/rotation/scale properties
Baked into matrices
Non-decomposable transforms (shear)

Baking all transforms into vertex positions guarantees correct rendering regardless of how the GLTF was exported from Blender or other 3D tools. Quaternion Normalization: Entity rotations use quaternions with all four components (x, y, z, w):

// From Entity.ts
quaternion: config.rotation
  ? [
      config.rotation.x,
      config.rotation.y,
      config.rotation.z,
      config.rotation.w,  // Uses actual w value, not hardcoded 1
    ]
  : undefined,

This prevents “squished” or incorrectly rotated models that can occur when quaternion components are not properly normalized.

Camera System

Supports multiple camera modes:

// systems/client/ClientCameraSystem.ts
export class ClientCameraSystem extends SystemBase {
  private settings = {
    minDistance: 2.0,        // Min zoom
    maxDistance: 15.0,       // Max zoom
    minPolarAngle: Math.PI * 0.35,  // Pitch limits
    maxPolarAngle: Math.PI * 0.48,
    rotateSpeed: 0.9,        // RS3-like feel
    zoomSpeed: 1.2,
    shoulderOffsetMax: 0.15, // Over-the-shoulder offset
  };
  
  // Camera spherical coordinates
  private theta = Math.PI;   // Horizontal angle (Math.PI = behind player)
  private phi = Math.PI * 0.4;  // Vertical angle (pitch)
  private radius = 5.0;      // Distance from target
}

Camera Initialization Fix (PR #829): The camera now correctly initializes with theta=Math.PI for standard third-person behind-the-player view. Previously, theta=0 placed the camera in front of the player, causing backwards movement on fresh load.

Mode	Controls
Third Person	Right-drag to rotate, scroll to zoom, click-to-move
First Person	Pointer lock, WASD movement
Top-down/RTS	Pan, zoom, click-to-move

VRM Avatar System

Characters use VRM format avatars with humanoid bone mapping:

// components/CharacterPreview.tsx
import { VRM, VRMLoaderPlugin, VRMUtils } from "@pixiv/three-vrm";
import { retargetAnimationToVRM } from "../utils/vrmAnimationRetarget";

export const CharacterPreview: React.FC<{ vrmUrl: string }> = ({ vrmUrl }) => {
  const vrmRef = useRef<VRM | null>(null);
  const mixerRef = useRef<THREE.AnimationMixer | null>(null);
  
  useEffect(() => {
    const loader = new GLTFLoader();
    loader.register((parser) => new VRMLoaderPlugin(parser));
    
    loader.load(vrmUrl, async (gltf) => {
      const vrm = gltf.userData.vrm as VRM;
      VRMUtils.rotateVRM0(vrm);  // Fix rotation for VRM 0.x
      
      // Retarget animations to VRM bones
      const mixer = new THREE.AnimationMixer(vrm.scene);
      const idleClip = await loadAnimation("idle.glb");
      const retargeted = retargetAnimationToVRM(idleClip, vrm);
      mixer.clipAction(retargeted).play();
      
      vrmRef.current = vrm;
      mixerRef.current = mixer;
    });
  }, [vrmUrl]);
};

VRM Bone Mapping

type VRMHumanBoneName =
  | "hips" | "spine" | "chest" | "upperChest" | "neck" | "head"
  | "leftShoulder" | "leftUpperArm" | "leftLowerArm" | "leftHand"
  | "rightShoulder" | "rightUpperArm" | "rightLowerArm" | "rightHand"
  | "leftUpperLeg" | "leftLowerLeg" | "leftFoot" | "leftToes"
  | "rightUpperLeg" | "rightLowerLeg" | "rightFoot" | "rightToes";

Client Systems

Located in packages/shared/src/systems/client/:

System	Description
`ClientGraphics.ts`	WebGPU rendering, shadows, post-processing
`ClientCameraSystem.ts`	Camera controls and collision
`ClientNetwork.ts`	WebSocket connection, reconnection
`ClientInput.ts`	Keyboard, mouse, touch handling
`ClientAudio.ts`	3D positional audio, music
`ClientLoader.ts`	Asset loading with progress
`HealthBars.ts`	Floating health bars over entities
`Nametags.ts`	Entity name labels
`DamageSplatSystem.ts`	Floating damage numbers
`XPDropSystem.ts`	XP gain notifications
`EquipmentVisualSystem.ts`	Equipment rendering on avatars
`TileInterpolator.ts`	Smooth tile-based movement

Embedded Mode

For stream overlays and spectator views:

// URL params for embedded mode
?embedded=true
&mode=spectator|free
&agentId=AGENT_ID
&followEntity=ENTITY_ID
&quality=low|medium|high
&hiddenUI=inventory,skills,chat

// components/EmbeddedGameClient.tsx
export function EmbeddedGameClient() {
  const config = window.__HYPERSCAPE_CONFIG__;
  
  return (
    <World
      mode={config.mode}
      followEntity={config.followEntity}
      hiddenUI={config.hiddenUI}
    />
  );
}

Overview

Engine Architecture

Client Application

Game Systems

AI Agents

Data Manifests

Technical Reference

Client Application

Client Application

Architecture Overview

Entry Point

Authentication Flow

Screens

Character Select Screen

Game Client

UI Components

Core UI (CoreUI.tsx)

Game Panels

HUD Components

Minimap System

3D Graphics System

WebGPU Renderer

Rendering Pipeline

Model Loading & Transform Baking

Camera System

VRM Avatar System

VRM Bone Mapping

Client Systems

Embedded Mode

Overview

Engine Architecture

Client Application

Game Systems

AI Agents

Data Manifests

Technical Reference

​Client Application

​Architecture Overview

​Entry Point

​Authentication Flow

​Screens

​Login Screen

​Character Select Screen

​Game Client

​UI Components

​Core UI (CoreUI.tsx)

​Game Panels

​HUD Components

​Minimap System

​3D Graphics System

​WebGPU Renderer

​Rendering Pipeline

​Model Loading & Transform Baking

​Camera System

​VRM Avatar System

​VRM Bone Mapping

​Client Systems

​Embedded Mode

​Related Documentation

Client Application

Architecture Overview

Entry Point

Authentication Flow

Screens

Login Screen

Character Select Screen

Game Client

UI Components

Core UI (CoreUI.tsx)

Game Panels

HUD Components

Minimap System

3D Graphics System

WebGPU Renderer

Rendering Pipeline

Model Loading & Transform Baking

Camera System

VRM Avatar System

VRM Bone Mapping

Client Systems

Embedded Mode

Related Documentation