Tool reference

Every tool the Holodeck computer uses on a scene. The server agent, the REST API (POST /api/v1/playground/scenes/:id/tools/:name), MCP (each tool takes an extra scene_id) and the Unity client all run this same code. Machine-readable: GET /api/v1/playground/tools.

Conventions: metres, y up, the ground is y = 0, positions are an object's base, angles are degrees, the default camera looks at the origin from +z.

get_scene

Describe the current scene: environment, object counts by name, and a list of objects (id, name, position, size).

ArgumentTypeDescription
name_filter string
group string

✱ required

search_assets

Search the 3D asset library (low-poly Synty-style prefabs and models; prefabs listed first). Returns asset_id, kind, name, pack_id, tags, size_m (metres) for each match.

ArgumentTypeDescription
query ✱ string short concrete noun phrase, e.g. 'pine tree', 'wooden fence', 'street lamp'
limit integer 1-24, default 8
pack string pack_id from an earlier result, e.g. '@synty/PolygonNature', to search only inside that pack
humanoid boolean true = only single humanoid character prefabs (for NPCs). Prefer this + query like 'SM_Chr' or 'cowboy' — never place pack 'Characters.fbx' atlases as NPCs.

✱ required

search_animations

Search animation clips for NPCs (idle, WalkFwd, …). Humanoid clips retarget across packs. Pass compatible_with = a character asset_id when you can.

ArgumentTypeDescription
query ✱ string e.g. 'idle', 'WalkFwd', 'walk'
compatible_with string character asset_id
limit integer

✱ required

make_npc

Turn a character prefab into an animated NPC (or place one with asset_id + position). Prefer search_assets(humanoid:true) SM_Chr_… prefabs — not Characters.fbx atlases. Auto-scales to ~1.7 m (kids ~1.25 m). Auto-picks idle/walk clips from the character's npc_clips, SiteVars, or the library. Wander + click-to-talk by default.

ArgumentTypeDescription
object_ids array exact object ids
group string folder path in the scene hierarchy, e.g. 'Nature/Trees', 'Village/Houses'
name_filter string every object whose name contains this text
asset_id string place this character prefab then make it an NPC
position object — fields: x, y, z
rotation_y number
scale number uniform; omit to auto-fit person height
name string
clip_id string idle animation asset id
walk_clip_id string
interact_clip_id string
wander_radius number metres to roam (default 5; 0 = stand still)
wander_speed number m/s, default 1.2
wander boolean default true when making an NPC
interact boolean clickable / F / VR trigger
message string what they say when interacted with
remove boolean

✱ required

set_simulation

Play / pause / restart the simulation clock (NPCs animate and wander while playing — default on). Use when the user says pause, freeze, stop, resume, play, or restart / reset people to their spawn.

ArgumentTypeDescription
action ✱ string (play, pause, restart) play = resume; pause = freeze; restart = snap NPCs back to spawn and play

✱ required

place_asset

Place one library asset in the scene. position is the centre of its footprint at its base. The result warns when it lands inside another object.

ArgumentTypeDescription
asset_id ✱ string
position ✱ object — fields: x, y, z
rotation_y number
scale number uniform, default 1
name string
group string folder path in the scene hierarchy, e.g. 'Nature/Trees', 'Village/Houses'
static boolean scenery that never moves (instanced, not clickable)
on_top_of string object id to stand this on (sets y to its top)

✱ required

scatter_assets

Scatter many copies of one or more assets naturally over an area with spacing, random rotation and scale. Best for forests, rocks, foliage, crowds of props.

ArgumentTypeDescription
asset_ids ✱ array 1-8 asset ids, picked at random per copy
count ✱ integer 1-150
area ✱ object Where to scatter: a circle {center:{x,z}, radius, inner_radius?} (inner_radius leaves a clearing) or a rectangle {min_x,max_x,min_z,max_z}. — fields: center, radius, inner_radius, min_x, max_x, min_z, max_z
min_spacing number metres between copies; default from asset size
scale_min number
scale_max number
random_rotation boolean
avoid_existing boolean default true
group string folder path in the scene hierarchy, e.g. 'Nature/Trees', 'Village/Houses'
static boolean default true — scattered scenery is static
max_slope number 0 flat … 1 steep; default 0.6 (keeps trees off cliffs)
min_height number
max_height number terrain height band in metres
allow_underwater boolean

✱ required

place_primitive

Build a simple shape when no asset fits (water pond = flat blue cylinder, glowing crystal = emissive cone, wall = box).

ArgumentTypeDescription
shape ✱ string (box, sphere, cylinder, cone, plane, torus, capsule)
size object metres {x,y,z} — fields: x, y, z
color string #rrggbb
emissive boolean
position ✱ object — fields: x, y, z
rotation_x number
rotation_y number
name string
group string folder path in the scene hierarchy, e.g. 'Nature/Trees', 'Village/Houses'
static boolean
on_top_of string

✱ required

add_light

Add a point light (fire, lamp, magic glow).

ArgumentTypeDescription
position ✱ object — fields: x, y, z
color string
intensity number default 4
range number metres, default 12
name string
group string

✱ required

modify_objects

Change existing objects, chosen by object_ids, group or name_filter.

ArgumentTypeDescription
object_ids array exact object ids
group string move into this folder
name_filter string every object whose name contains this text
position object absolute (single object only) — fields: x, y, z
offset object move by — fields: x, y, z
rotation_y number
rotate_by number
scale number
scale_by number
color string
name string
static boolean
intensity number

✱ required

delete_objects

Remove objects chosen by object_ids, group or name_filter — or everything with all: true. `area` limits it to a circle/rectangle (alone it removes everything there; static_only: true just the scenery).

ArgumentTypeDescription
object_ids array exact object ids
group string every object in this group
name_filter string every object whose name contains this text
all boolean
area object Where to scatter: a circle {center:{x,z}, radius, inner_radius?} (inner_radius leaves a clearing) or a rectangle {min_x,max_x,min_z,max_z}. — fields: center, radius, inner_radius, min_x, max_x, min_z, max_z
static_only boolean

✱ required

set_environment

Set sky/lighting and ground.

ArgumentTypeDescription
preset string (holodeck, day, sunset, night, overcast, space)
ground string (grid, grass, sand, snow, dirt, water, stone, asphalt, none)
ground_size number metres across, default 120
fog number 0-1
sky_color string
ground_color string

✱ required

focus_camera

Point the user's camera at objects or a spot.

ArgumentTypeDescription
object_ids array
target object — fields: x, y, z
position object camera position — fields: x, y, z

✱ required

add_camera

Place a scene camera the player can switch to (keys 1-9).

ArgumentTypeDescription
name string
position ✱ object camera position; y default 1.7 (eye height) — fields: x, y, z
look_at ✱ object point it looks at; y default 1 — fields: x, y, z
fov number degrees, default 55

✱ required

create_material

Create a scene material: colour + PBR values, optional glow, transparency and procedural pattern.

ArgumentTypeDescription
name ✱ string
color string #rrggbb base colour
roughness number 0 glossy - 1 matte, default 0.8
metalness number 0-1
emissive string #rrggbb glow colour
emissive_intensity number
opacity number 0.05-1; below 1 is see-through (glass, water)
pattern string (none, checker, stripes, bricks, tiles, noise, wood, marble, dots, grid)
pattern_color string second colour of the pattern
tile_size number metres per pattern repeat, default 2
tint boolean on library assets: keep their own texture and multiply by color

✱ required

generate_texture

Paint a new seamless, tileable image texture from a description (AI image, ~6 s) and make it a material.

ArgumentTypeDescription
prompt ✱ string the surface, e.g. 'mossy cobblestone', 'polished white marble with grey veins'
name string
tile_size number metres per repeat, default 2
roughness number

✱ required

set_terrain

Turn on (or reshape) the endless procedural terrain: rolling hills, mountains, islands… Grounded objects move onto the new ground.

ArgumentTypeDescription
enabled boolean
style string (flat, hills, plains, mountains, islands, dunes)
height number relief amplitude in metres (hills 8–20, mountains 30–80)
scale number feature size in metres (60 = bumpy, 300 = broad)
roughness number 0 smooth … 1 rugged
seed integer a different landscape
water_level number | null sea / lake level in metres; null = no water
flat_radius number keep the ground flat within this many metres of the origin (a building site)

✱ required

sculpt_terrain

Sculpt the terrain in a circle: raise or lower (strength = metres at the centre), flatten (to `height`, or the centre's height) or smooth (strength 0–1). Grounded objects follow.

ArgumentTypeDescription
mode ✱ string (raise, lower, flatten, smooth, noise, terrace) raise / lower (strength = metres), flatten / smooth / noise (roughen) / terrace (stepped slopes), strength 0–1
center ✱ object — fields: x, y, z
radius ✱ number metres, up to 80
strength number
height number flatten: target height · terrace: step height (default 2 m)
hardness number 0 soft edge … 1 hard edge (default 0)

✱ required

paint_terrain

Paint a ground layer in a circle (paths, beaches, rocky patches, snow caps). 'auto' restores the automatic colouring.

ArgumentTypeDescription
layer ✱ string (auto, grass, dirt, sand, rock, snow, path, moss)
center ✱ object — fields: x, y, z
radius ✱ number
hardness number 0 soft, dithered edge … 1 crisp edge (default 0.5)

✱ required

sculpt_path

Carve along a line of points: a road (flattened, following the ground, painted 'path'), a river or ditch (dug `depth` below the ground, sandy banks — dig below water_level to fill it), or a ridge / embankment (raised). Objects on it are re-settled.

ArgumentTypeDescription
points ✱ array 2–200 points along the line
kind ✱ string (road, river, ridge)
width number metres (default 4)
depth number river depth / ridge height in metres (default 2)
layer string | null layer painted down the middle (default path for roads, sand for rivers); null = none
strength number 0–1 (default 1)

✱ required

paint_by_rule

Paint a ground layer wherever the terrain matches, inside an area: snow above a height, rock on steep slopes, sand just above the water, moss in hollows.

ArgumentTypeDescription
layer ✱ string (auto, grass, dirt, sand, rock, snow, path, moss)
area ✱ object Where to scatter: a circle {center:{x,z}, radius, inner_radius?} (inner_radius leaves a clearing) or a rectangle {min_x,max_x,min_z,max_z}. — fields: center, radius, inner_radius, min_x, max_x, min_z, max_z
min_height number
max_height number
min_slope number 0 flat … 1 steep
max_slope number

✱ required

set_terrain_layer

Change how a ground layer looks everywhere it's painted (or auto-chosen): a colour, or a scene material — e.g. a generate_texture result for real cobblestones.

ArgumentTypeDescription
layer ✱ string (grass, dirt, sand, rock, snow, path, moss)
color string #rrggbb
material_id string
reset boolean back to the default look

✱ required

undo_terrain

Undo the last terrain sculpt / paint / path edit (repeat to go further back).

sample_terrain

Ground height and slope (0–1) at points — before placing something big or choosing where to build.

ArgumentTypeDescription
points ✱ array

✱ required

manage_group

Act on a folder of the scene hierarchy (and its subfolders): mark static, hide, lock, rename/move to another path, offset everything, or delete it.

ArgumentTypeDescription
group ✱ string folder path in the scene hierarchy, e.g. 'Nature/Trees', 'Village/Houses'
static boolean
hidden boolean
locked boolean
rename_to string
offset object — fields: x, y, z
delete boolean

✱ required

build_structure

Build a room or building from a pack's modular pieces on an exact grid — walls meet with no gaps or overlaps, doors and windows sit in wall slots, storeys stack, the roof closes it. Prefer build_from_recipe for cabins/shops/taverns (hardcoded shells). Use this when you already know the piece ids.

ArgumentTypeDescription
floor_asset_id ✱ string
wall_asset_id ✱ string
door_asset_id string a door / doorway wall piece (doorframe with door)
window_asset_id string optional window wall piece (used on alternate segments when windows: true)
roof_asset_id string optional; default: a layer of floor tiles
roof boolean false = open top
center ✱ object — fields: x, y, z
size ✱ object footprint in metres {x, z}; rounded to whole floor tiles — fields: x, z
rotation_y number
stories integer 1-4
doors array default one door in the middle of the south (-z) side
windows boolean
name string
group string folder path in the scene hierarchy, e.g. 'Nature/Trees', 'Village/Houses'
flatten_ground boolean default true — levels the terrain under it
interactive_doors boolean default true — doors open and close

✱ required

list_building_recipes

List hardcoded building recipes (cabin, shop, tavern, barn, house, outpost). Prefer these over inventing wall layouts.

build_from_recipe

Build a recipe building: resolves modular floor/wall/door/window from the asset lock, runs StructureBuilder, optionally furnishes the interior. LLM only chooses recipe, center, name, rotation.

ArgumentTypeDescription
recipe ✱ string (cabin, shop, tavern, barn, house, outpost)
center ✱ object — fields: x, y, z
name string
rotation_y number
size object optional override of recipe footprint — fields: x, z
stories integer
pack string optional pack_id to pull pieces from
furnish boolean default true — place furniture props inside
group string folder path in the scene hierarchy, e.g. 'Nature/Trees', 'Village/Houses'
flatten_ground boolean

✱ required

set_world_meta

Set world_depth (simple|standard|deep) and/or persist a world_plan (title, lore, buildings). Depth gates how many buildings you may add.

ArgumentTypeDescription
world_depth string (simple, standard, deep)
world_plan object { title, lore, depth, buildings: [{ id, recipe, name, x, z, rotation_y, npc_name, npc_message }] }

✱ required

build_world

Build a whole settlement from a goal: picks a depth (simple/standard/deep), lays out recipe buildings, flavours names/lore with the model, furnishes, places NPCs and scatter. Use for village/town/world goals instead of placing walls by hand.

ArgumentTypeDescription
goal ✱ string e.g. 'a deep lakeside village with a tavern'
depth string (simple, standard, deep) optional override
flavour boolean default true — one LLM pass to customise names/lore

✱ required

shop_buy

Buy stock from a shop NPC / counter (settings.shops[object_id]). Spends player credits.

ArgumentTypeDescription
object_id ✱ string
sku ✱ string
qty integer

✱ required

inventory

Show player credits/inventory and open shops on this scene.

place_container

Put a chest (container) on an object with a loot table. Treasure class common/dungeon/boss, or pass an explicit loot table {rolls, credits:[min,max], entries:[{sku,weight,min,max}]}. Opens into the player inventory.

ArgumentTypeDescription
object_ids ✱ array exact object ids
group string every object in this group
name_filter string every object whose name contains this text
class string (common, dungeon, boss) treasure class (or pass loot for a custom table)
title string chest name, default '<object> chest'
loot object explicit loot table

✱ required

open_container

Open a chest and collect its loot into the player's inventory (one-shot — marks it opened).

ArgumentTypeDescription
object_id ✱ string

✱ required

list_items

List the loot item catalog (sku, name, kind, rarity) and treasure classes.

spawn_mob

Turn an object (usually an NPC) into a mob — friendly / normal / elite / boss with a loot table. Mobs are 'defeated', not killed; a sim restart revives them.

ArgumentTypeDescription
object_ids ✱ array exact object ids
group string every object in this group
name_filter string every object whose name contains this text
name string
kind string (friendly, normal, elite, boss) friendly (won't fight) / normal / elite / boss
faction string
loot_class string (common, dungeon, boss)
loot object explicit loot table

✱ required

fight_mob

Fight a mob: deterministic exchange driven by your best weapon. Victory drops its loot + XP and advances kill-quests.

ArgumentTypeDescription
object_id ✱ string

✱ required

mob_list

List mobs and whether they're defeated.

story_list

List world_plan story beats and which are ready (trigger met) / done.

advance_story

Fire the next due story beat (grants XP + narrates it).

quest_list

List world_plan quests and their progress.

quest_advance

Advance a quest objective (talk / buy / visit / loot / kill / own). Completes objectives in order; the last one grants the reward.

ArgumentTypeDescription
quest_id ✱ string
event ✱ string
target string optional target match (e.g. NPC role)
note string

✱ required

request_continue

After this turn finishes, auto-start a follow-up agent goal (chained deep worlds). Max 3 hops.

ArgumentTypeDescription
goal ✱ string

✱ required

check_layout

Check the whole scene for objects inside each other, hanging in the air or sunk into the ground. Run it after building and fix what it reports.

make_interactive

Make doors (or lids, gates, hatches) open and close: the player clicks them in walk / fly / VR, or they open by themselves when the player comes near (auto). The moving part of a prefab is found automatically. Also opens / closes them (open: true / false) or removes the behaviour (remove: true).

ArgumentTypeDescription
object_ids array exact object ids
group string every object in this group
name_filter string every object whose name contains this text
kind string (hinge, slide, lift) hinge (swings), slide (sideways, sci-fi), lift (up, garage)
hinge string (left, right)
angle number hinge swing in degrees, default 90 (negative swings the other way)
direction string (left, right) slide direction
open boolean
auto boolean
part string prefab node name(s) that move, comma-separated; default auto
remove boolean

✱ required

look_at_scene

Take a picture of the scene through the user's editor and look at it yourself — check composition, gaps, things floating or buried, whether it matches the goal. Frames everything by default, or object_ids, or a camera position / target.

ArgumentTypeDescription
object_ids array
target object point to look at — fields: x, y, z
position object camera position — fields: x, y, z

✱ required

apply_material

Put a scene material on objects (object_ids / group / name_filter) and/or the ground. material_id null restores the original look.

ArgumentTypeDescription
object_ids array exact object ids
group string every object in this group
name_filter string every object whose name contains this text
material_id ✱ string | null
ground boolean apply to the ground

✱ required