> ## Documentation Index > Fetch the complete documentation index at: https://hyperscape-ai-mintlify-docs-update.mintlify.site/llms.txt > Use this file to discover all available pages before exploring further. # Client Application > React UI, 3D rendering, VRM avatars, and client architecture # Client Application The Hyperscape client is a React-based web application with Three.js WebGPU rendering. It provides a modern MMORPG experience with VRM avatars, responsive UI panels, and real-time multiplayer. Client code lives in `packages/client/src/`. The rendering systems are in `packages/shared/src/systems/client/`. ## Architecture Overview ``` ┌─────────────────────────────────────────────────────────────────┐ │ React Application │ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────────────────┐ │ │ │ Auth Flow │ │ Screens │ │ Game Panels │ │ │ │ PrivyAuth │ │ - Login │ │ - Inventory - Skills │ │ │ │ Wallet Auth │ │ - CharSel │ │ - Equipment - Bank │ │ │ │ │ │ - GameClient│ │ - Combat - Chat │ │ │ └─────────────┘ └─────────────┘ └─────────────────────────┘ │ ├─────────────────────────────────────────────────────────────────┤ │ Three.js WebGPU Renderer │ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────────────────┐ │ │ │ Scene Graph │ │ Cameras │ │ Post-Processing │ │ │ │ Terrain │ │ Third Person│ │ - Bloom │ │ │ │ Entities │ │ First Person│ │ - Tone Mapping │ │ │ │ VRM Avatars │ │ RTS Mode │ │ - Color Grading │ │ │ └─────────────┘ └─────────────┘ └─────────────────────────┘ │ ├─────────────────────────────────────────────────────────────────┤ │ Client World │ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────────────────┐ │ │ │ Network │ │ Input │ │ Systems │ │ │ │ WebSocket │ │ Keyboard │ │ - Graphics - Camera │ │ │ │ Reconnect │ │ Mouse │ │ - Audio - Loader │ │ │ │ │ │ Touch │ │ - Health - XP Drops │ │ │ └─────────────┘ └─────────────┘ └─────────────────────────┘ │ └─────────────────────────────────────────────────────────────────┘ ``` *** ## Entry Point The client entry point is `packages/client/src/index.tsx`: ```typescript theme={"theme":{"light":"github-light","dark":"tokyo-night"}} // Main entry point for the Hyperscape browser client import { World, installThreeJSExtensions } from "@hyperscape/shared"; import React from "react"; import ReactDOM from "react-dom/client"; // Screens import { LoginScreen } from "./screens/LoginScreen"; import { CharacterSelectScreen } from "./screens/CharacterSelectScreen"; import { GameClient } from "./screens/GameClient"; // Authentication (Privy + Wallet) import { PrivyAuthProvider } from "./auth/PrivyAuthProvider"; // Embedded mode support for spectator views import { EmbeddedGameClient } from "./components/EmbeddedGameClient"; import { isEmbeddedMode } from "./types/embeddedConfig"; ``` *** ## Authentication Flow Authentication uses [Privy](https://www.privy.io/) for wallet-based login: ``` Login Screen → Privy Modal → Wallet Signature → Character Select → Game ``` ```typescript theme={"theme":{"light":"github-light","dark":"tokyo-night"}} // auth/PrivyAuthProvider.tsx export function PrivyAuthProvider({ children }: { children: React.ReactNode }) { return ( {children} ); } ``` *** ## Screens ### Login Screen * Wallet connection via Privy * Social login (email, Google) * Session persistence ### Character Select Screen * List existing characters * Create new character * Character preview with VRM avatar ### Game Client * Main game loop * World initialization * UI overlay rendering ```typescript theme={"theme":{"light":"github-light","dark":"tokyo-night"}} // screens/GameClient.tsx export function GameClient({ characterId }: { characterId: string }) { const worldRef = useRef(null); useEffect(() => { const world = new World({ isServer: false, systems: clientSystems, }); world.connect({ characterId }); worldRef.current = world; return () => world.destroy(); }, [characterId]); return ; } ``` *** ## UI Components ### Unified Tooltip System (March 2026) **New Feature (PR #1102)**: Unified tooltip styling system with consistent behavior across all UI panels. The tooltip system provides centralized style utilities for consistent tooltip appearance: **Location**: `packages/client/src/ui/core/tooltip/tooltipStyles.ts` **Style Functions:** ```typescript theme={"theme":{"light":"github-light","dark":"tokyo-night"}} import { getTooltipTitleStyle, getTooltipMetaStyle, getTooltipBodyStyle, getTooltipDividerStyle, getTooltipTagStyle, getTooltipStatusStyle, } from '@/ui/core/tooltip/tooltipStyles'; // Title text (primary content) const titleStyle = getTooltipTitleStyle(theme); // Returns: { color, fontWeight: 700, fontSize: "13px", lineHeight: 1.2 } // Meta text (secondary info like level requirements) const metaStyle = getTooltipMetaStyle(theme); // Returns: { color: muted, fontSize: "11px", lineHeight: 1.3 } // Body text (descriptions) const bodyStyle = getTooltipBodyStyle(theme); // Returns: { color: secondary, fontSize: "11px", lineHeight: 1.45 } // Divider line with spacing const dividerStyle = getTooltipDividerStyle(theme, accentColor); // Returns: { borderTop, marginTop: "8px", paddingTop: "8px" } // Tag/badge styling const tagStyle = getTooltipTagStyle(theme); // Returns: { display, padding, borderRadius, background, border, color, fontSize } // Status indicators (success/danger/warning) const statusStyle = getTooltipStatusStyle(theme, "success"); // Returns: { marginTop, padding, borderRadius, background, border, color, fontSize, textAlign, fontWeight } ``` **Usage Example:** ```typescript theme={"theme":{"light":"github-light","dark":"tokyo-night"}} import { CursorTooltip } from '@/ui'; import { getTooltipTitleStyle, getTooltipMetaStyle } from '@/ui/core/tooltip/tooltipStyles'; function ItemSlot({ item }) { const theme = useTheme(); const [hoverState, setHoverState] = useState(null); return ( <> {hoverState && (
{item.name}
{item.description}
)} ); } ``` **Panels Using Unified Tooltips:** * Inventory Panel * Equipment Panel * Bank Panel * Skills Panel * Prayer Panel * Spells Panel * Store Panel * Trade Panel * Duel Panel * Action Bar * Loot Window **Benefits:** * Consistent visual language across all UI * Centralized style management * Easy theme customization * Reduced code duplication (\~500 lines eliminated) ### Core UI (CoreUI.tsx) The main UI wrapper that renders HUD elements and game panels: ```typescript theme={"theme":{"light":"github-light","dark":"tokyo-night"}} export function CoreUI({ world }: { world: ClientWorld }) { const [ready, setReady] = useState(false); const [deathScreen, setDeathScreen] = useState(null); // Event handlers for loading, death, disconnect useEffect(() => { world.on(EventType.READY, () => setReady(true)); world.on(EventType.PLAYER_DIED, handleDeath); world.on(EventType.DISCONNECTED, handleDisconnect); }, [world]); if (!ready) return ; return ( <> {deathScreen && } ); } ``` ### Game Panels Located in `packages/client/src/game/panels/`: | Panel | Description | | -------------------- | ---------------------------------- | | `InventoryPanel.tsx` | 28-slot inventory with drag & drop | | `EquipmentPanel.tsx` | Equipment slots with stat display | | `SkillsPanel.tsx` | All skills with XP progress bars | | `CombatPanel.tsx` | Combat style selection | | `BankPanel.tsx` | Bank interface with tabs | | `DialoguePanel.tsx` | NPC conversation UI | | `StorePanel.tsx` | Shop/trading interface | | `SettingsPanel.tsx` | Game settings | | `LootWindow.tsx` | Ground item pickup interface | ### HUD Components Located in `packages/client/src/game/hud/`: * `StatusBars.tsx` - Health, prayer, run energy bars * `XPProgressOrb.tsx` - XP tracking orb with skill icons * `ActionProgressBar.tsx` - Skilling action progress * `EntityContextMenu.tsx` - Right-click context menus * `Minimap.tsx` - RS3/OSRS-accurate top-down map view with location icons * `Nametags.ts` - Entity name/level labels ### Minimap System The minimap provides RS3/OSRS-accurate navigation with location icons and entity tracking. **Visual Standards (OSRS-accurate):** * **White dots**: Other players * **Yellow dots**: NPCs, mobs, and buildings * **Red dots**: Ground items and loot * **White square**: Local player (distinct from other players) * **Red flag**: Destination marker (RS3-style) **Location Icons:** The minimap displays icons for key locations instead of generic dots: | Icon | Location Type | Visual | | ----------- | ------------------ | ------------------------------- | | **\$** | Bank | Gold coin symbol on gold circle | | **Bag** | Shop | Open-top bag silhouette | | **Cross** | Prayer Altar | White cross (religious symbol) | | **R** | Runecrafting Altar | Purple circle with "R" | | **Anvil** | Smithing Station | Dark anvil silhouette | | **Flame** | Furnace | Orange circle with flame | | **Steam** | Cooking Range | Brown circle with steam lines | | **Fish** | Fishing Spot | Cyan circle with fish | | **Pickaxe** | Mining Rock | Brown circle with pickaxe | | **Tree** | Woodcutting | Green circle | | **!** | Quest Available | Blue circle with white "!" | | **?** | Quest In Progress | Blue circle with white "?" | **Quest Icon States:** Quest giver NPCs display state-aware icons based on quest progress: ```typescript theme={"theme":{"light":"github-light","dark":"tokyo-night"}} // From Minimap.tsx // Quest icon with state awareness if (serviceTypes?.includes("quest")) { const questIds = npcConfig?.questIds; const statuses = questStatusesRef.current; if (questIds && questIds.length > 0 && statuses.size > 0) { let hasAvailable = false; let hasActive = false; let allCompleted = true; for (const qid of questIds) { const state = statuses.get(qid); if (state === "available") hasAvailable = true; else if (state === "active") hasActive = true; if (state !== "completed") allCompleted = false; } if (hasAvailable) { subType = "quest_available"; // Blue "!" } else if (hasActive) { subType = "quest_in_progress"; // Blue "?" } // If all completed, no quest icon (shows bank/shop icon if applicable) } } ``` **Quest Status Synchronization:** The minimap fetches quest statuses from the server and updates icons in real-time: ```typescript theme={"theme":{"light":"github-light","dark":"tokyo-night"}} // Fetch quest list from server world.network?.send("getQuestList", {}); // Listen for quest status updates world.network?.on("questList", (data) => { const mapped = data.quests.map((q) => ({ id: q.id, state: mapStatus(q.status), // "not_started" → "available", etc. })); // Update ref for synchronous access in entity loop questStatusesRef.current = new Map(mapped.map(q => [q.id, q.state])); }); // Re-fetch on quest events world.on(EventType.QUEST_STARTED, fetchQuestList); world.on(EventType.QUEST_PROGRESSED, fetchQuestList); world.on(EventType.QUEST_COMPLETED, fetchQuestList); ``` **Icon Behavior:** * **Blue "!"**: NPC has at least one available quest (not started) * **Blue "?"**: NPC has at least one quest in progress * **No icon**: All quests completed (shows bank/shop icon if NPC provides those services) * Icons disappear when all of an NPC's quests are completed **NPC Configuration:** Quest giver NPCs must have `questIds` field in their configuration: ```json theme={"theme":{"light":"github-light","dark":"tokyo-night"}} { "id": "quest_giver_lumbridge", "name": "Quest Giver", "services": { "types": ["quest"], "questIds": ["cooks_assistant", "sheep_shearer"] } } ``` The `questIds` field is passed through the spawn → network → client pipeline so the minimap can identify quest giver NPCs and display appropriate icons. **Icon Detection:** Icons are automatically assigned based on entity configuration: ```typescript theme={"theme":{"light":"github-light","dark":"tokyo-night"}} // From Minimap.tsx // Detect NPC service type for minimap icons const npcConfig = entity.config; const serviceTypes = npcConfig?.services; // Fixed: services is string[], not { types: string[] } if (serviceTypes?.includes("bank")) { subType = "bank"; } else if (serviceTypes?.includes("shop")) { subType = "shop"; } // Quest icon detection (with state awareness) shown above // Detect resource subtype for minimap icons const resConfig = entity.config; if (resConfig?.resourceType === "fishing_spot" || resConfig?.harvestSkill === "fishing") { subType = "fishing"; } else if (resConfig?.resourceType === "mining_rock" || resConfig?.harvestSkill === "mining") { subType = "mining"; } else if (resConfig?.resourceType === "tree" || resConfig?.harvestSkill === "woodcutting") { subType = "tree"; } ``` **Bug Fix (PR #885)**: Fixed broken NPC service detection cast. The `services` field is `string[]`, not `{ types: string[] }`. This also fixed bank/shop icon detection which was previously broken. **Size Hierarchy:** * Entity dots: 6px diameter (compact for clarity) * Location icons: 12px diameter (prominent for navigation) **Rendering:** * 3D terrain rendered via orthographic camera (throttled to \~15fps for performance) * 2D overlay canvas for entity pips and icons (60fps for smooth interaction) * Cached projection-view matrix keeps pips synced with throttled 3D render * Pre-allocated vectors avoid GC pressure in render loop **Features:** * Click-to-move on minimap * Scroll to zoom (20-1000 tile extent) * Rotates with main camera (RS3-style) * Destination marker persists until reached * Resizable and collapsible * Edit mode support for UI customization *** ## 3D Graphics System ### WebGPU Renderer The graphics system uses Three.js with WebGPU for high-performance rendering: **WebGPU Required**: All rendering uses TSL (Three Shading Language) which only works with WebGPU. There is no WebGL fallback. Requires Chrome 113+, Edge 113+, or Safari 18+ (macOS 15+). Check compatibility at [webgpureport.org](https://webgpureport.org). **Breaking Change (Feb 2026)**: WebGL fallback was completely removed. All shaders use TSL which only compiles to WebGPU. Users on browsers without WebGPU will see an error screen with upgrade instructions. **WebGPU Initialization Timeouts:** The renderer includes timeout protection to detect GPU driver issues: ```typescript theme={"theme":{"light":"github-light","dark":"tokyo-night"}} // 30s timeout on adapter request const adapter = await Promise.race([ navigator.gpu.requestAdapter(), new Promise((_, reject) => setTimeout(() => reject(new Error('WebGPU adapter request timed out')), 30000) ) ]); // 60s timeout on renderer initialization await Promise.race([ renderer.init(), new Promise((_, reject) => setTimeout(() => reject(new Error('WebGPU renderer initialization timed out')), 60000) ) ]); ``` These timeouts help diagnose misconfigured GPU servers where WebGPU initialization hangs indefinitely. ```typescript theme={"theme":{"light":"github-light","dark":"tokyo-night"}} // systems/client/ClientGraphics.ts export class ClientGraphics extends SystemBase { private renderer: WebGPURenderer; private postProcessing: PostProcessingComposer; async init() { // WebGPU is required - no fallback if (!navigator.gpu) { throw new Error('WebGPU not supported. Please use Chrome 113+, Edge 113+, or Safari 18+'); } this.renderer = await createRenderer({ powerPreference: "high-performance", antialias: true, }); // Configure shadows (Cascaded Shadow Maps) configureShadowMaps(this.renderer, { cascades: 3, shadowMapSize: 2048, }); // Post-processing (TSL-based, WebGPU only) this.postProcessing = createPostProcessing(this.renderer, { bloom: true, toneMapping: true, colorGrading: true, }); } } ``` **Post-Processing Pipeline:** The post-processing system uses TSL (Three Shading Language) for all effects: ```typescript theme={"theme":{"light":"github-light","dark":"tokyo-night"}} // From ClientGraphics.ts // All post-processing uses TSL node materials (WebGPU only) this.composer.render(); // LUT color grading with intensity control if (!this.colorGradingEnabled) { this.lutPass.intensity = 0.0; } ``` **Post-Processing Effects (TSL-based):** * **Bloom**: Glow effects for magical items and particles * **Tone Mapping**: HDR to LDR conversion with auto-exposure * **Color Grading**: LUT-based color correction * **Outline**: Entity highlighting for hover/selection **Fix Applied (PR #829):** * **Issue**: When LUT was "none" but hover triggered outline, stale LUT data leaked into rendering * **Solution**: Zero LUT intensity when disabled so outline-only rendering stays clean * **Benefit**: Entity highlights work correctly even when color grading is disabled ### Rendering Pipeline 1. **Pre-render**: Update matrices, frustum culling 2. **Shadow Pass**: Render cascaded shadow maps (CSM) 3. **Main Pass**: Render scene with deferred lighting 4. **Post-Processing**: Bloom, tone mapping, color grading (TSL-based) 5. **UI Overlay**: Render 2D React UI on top **All rendering uses WebGPU** - there is no WebGL fallback path. ### Model Loading & Transform Baking The `ModelCache` system handles GLTF model loading with transform baking to prevent rendering issues: ```typescript theme={"theme":{"light":"github-light","dark":"tokyo-night"}} // From ModelCache.ts private bakeTransformsToGeometry(scene: THREE.Object3D): void { // Ensure all matrices are up to date scene.updateMatrixWorld(true); // Apply transforms to each mesh's geometry scene.traverse((child) => { if (child instanceof THREE.Mesh && child.geometry) { // Clone geometry to avoid modifying shared geometry child.geometry = child.geometry.clone(); // Apply world matrix to geometry (Three.js built-in method) child.geometry.applyMatrix4(child.matrixWorld); // Reset transform to identity child.position.set(0, 0, 0); child.rotation.set(0, 0, 0); child.scale.set(1, 1, 1); child.updateMatrix(); } }); } ``` **Why Transform Baking?** GLTF files can have transforms stored in various ways: * Position/rotation/scale properties * Baked into matrices * Non-decomposable transforms (shear) Baking all transforms into vertex positions guarantees correct rendering regardless of how the GLTF was exported from Blender or other 3D tools. **Quaternion Normalization**: Entity rotations use quaternions with all four components (x, y, z, w): ```typescript theme={"theme":{"light":"github-light","dark":"tokyo-night"}} // From Entity.ts quaternion: config.rotation ? [ config.rotation.x, config.rotation.y, config.rotation.z, config.rotation.w, // Uses actual w value, not hardcoded 1 ] : undefined, ``` This prevents "squished" or incorrectly rotated models that can occur when quaternion components are not properly normalized. ### Camera System Supports multiple camera modes: ```typescript theme={"theme":{"light":"github-light","dark":"tokyo-night"}} // systems/client/ClientCameraSystem.ts export class ClientCameraSystem extends SystemBase { private settings = { minDistance: 2.0, // Min zoom maxDistance: 15.0, // Max zoom minPolarAngle: Math.PI * 0.35, // Pitch limits maxPolarAngle: Math.PI * 0.48, rotateSpeed: 0.9, // RS3-like feel zoomSpeed: 1.2, shoulderOffsetMax: 0.15, // Over-the-shoulder offset }; // Camera spherical coordinates private theta = Math.PI; // Horizontal angle (Math.PI = behind player) private phi = Math.PI * 0.4; // Vertical angle (pitch) private radius = 5.0; // Distance from target } ``` **Camera Initialization Fix (PR #829)**: The camera now correctly initializes with `theta=Math.PI` for standard third-person behind-the-player view. Previously, `theta=0` placed the camera in front of the player, causing backwards movement on fresh load. | Mode | Controls | | ---------------- | --------------------------------------------------- | | **Third Person** | Right-drag to rotate, scroll to zoom, click-to-move | | **First Person** | Pointer lock, WASD movement | | **Top-down/RTS** | Pan, zoom, click-to-move | *** ## VRM Avatar System Characters use VRM format avatars with humanoid bone mapping: ```typescript theme={"theme":{"light":"github-light","dark":"tokyo-night"}} // components/CharacterPreview.tsx import { VRM, VRMLoaderPlugin, VRMUtils } from "@pixiv/three-vrm"; import { retargetAnimationToVRM } from "../utils/vrmAnimationRetarget"; export const CharacterPreview: React.FC<{ vrmUrl: string }> = ({ vrmUrl }) => { const vrmRef = useRef(null); const mixerRef = useRef(null); useEffect(() => { const loader = new GLTFLoader(); loader.register((parser) => new VRMLoaderPlugin(parser)); loader.load(vrmUrl, async (gltf) => { const vrm = gltf.userData.vrm as VRM; VRMUtils.rotateVRM0(vrm); // Fix rotation for VRM 0.x // Retarget animations to VRM bones const mixer = new THREE.AnimationMixer(vrm.scene); const idleClip = await loadAnimation("idle.glb"); const retargeted = retargetAnimationToVRM(idleClip, vrm); mixer.clipAction(retargeted).play(); vrmRef.current = vrm; mixerRef.current = mixer; }); }, [vrmUrl]); }; ``` ### VRM Bone Mapping ```typescript theme={"theme":{"light":"github-light","dark":"tokyo-night"}} type VRMHumanBoneName = | "hips" | "spine" | "chest" | "upperChest" | "neck" | "head" | "leftShoulder" | "leftUpperArm" | "leftLowerArm" | "leftHand" | "rightShoulder" | "rightUpperArm" | "rightLowerArm" | "rightHand" | "leftUpperLeg" | "leftLowerLeg" | "leftFoot" | "leftToes" | "rightUpperLeg" | "rightLowerLeg" | "rightFoot" | "rightToes"; ``` *** ## Client Systems Located in `packages/shared/src/systems/client/`: | System | Description | | -------------------------- | ------------------------------------------ | | `ClientGraphics.ts` | WebGPU rendering, shadows, post-processing | | `ClientCameraSystem.ts` | Camera controls and collision | | `ClientNetwork.ts` | WebSocket connection, reconnection | | `ClientInput.ts` | Keyboard, mouse, touch handling | | `ClientAudio.ts` | 3D positional audio, music | | `ClientLoader.ts` | Asset loading with progress | | `HealthBars.ts` | Floating health bars over entities | | `Nametags.ts` | Entity name labels | | `DamageSplatSystem.ts` | Floating damage numbers | | `XPDropSystem.ts` | XP gain notifications | | `EquipmentVisualSystem.ts` | Equipment rendering on avatars | | `TileInterpolator.ts` | Smooth tile-based movement | *** ## Embedded Mode For stream overlays and spectator views: ```typescript theme={"theme":{"light":"github-light","dark":"tokyo-night"}} // URL params for embedded mode ?embedded=true &mode=spectator|free &agentId=AGENT_ID &followEntity=ENTITY_ID &quality=low|medium|high &hiddenUI=inventory,skills,chat ``` ```typescript theme={"theme":{"light":"github-light","dark":"tokyo-night"}} // components/EmbeddedGameClient.tsx export function EmbeddedGameClient() { const config = window.__HYPERSCAPE_CONFIG__; return ( ); } ``` *** ## Related Documentation * [Engine Architecture](/wiki/engine/overview) * [Networking](/wiki/engine/networking) * [Movement System](/wiki/game-systems/movement)