Tool reference
Every tool the Holodeck computer uses on a scene. The server agent, the
REST API
(POST /api/v1/playground/scenes/:id/tools/:name), MCP
(each tool takes an extra scene_id) and the Unity client
all run this same code. Machine-readable: GET /api/v1/playground/tools.
Conventions: metres, y up, the ground is y = 0, positions are an object's base, angles are degrees, the default camera looks at the origin from +z.
get_scene
Describe the current scene: environment, object counts by name, and a list of objects (id, name, position, size).
| Argument | Type | Description |
|---|---|---|
name_filter |
string | |
group |
string |
✱ required
search_assets
Search the 3D asset library (low-poly Synty-style prefabs and models; prefabs listed first). Returns asset_id, kind, name, pack_id, tags, size_m (metres) for each match.
| Argument | Type | Description |
|---|---|---|
query ✱ |
string | short concrete noun phrase, e.g. 'pine tree', 'wooden fence', 'street lamp' |
limit |
integer | 1-24, default 8 |
pack |
string | pack_id from an earlier result, e.g. '@synty/PolygonNature', to search only inside that pack |
humanoid |
boolean | true = only single humanoid character prefabs (for NPCs). Prefer this + query like 'SM_Chr' or 'cowboy' — never place pack 'Characters.fbx' atlases as NPCs. |
✱ required
search_animations
Search animation clips for NPCs (idle, WalkFwd, …). Humanoid clips retarget across packs. Pass compatible_with = a character asset_id when you can.
| Argument | Type | Description |
|---|---|---|
query ✱ |
string | e.g. 'idle', 'WalkFwd', 'walk' |
compatible_with |
string | character asset_id |
limit |
integer |
✱ required
make_npc
Turn a character prefab into an animated NPC (or place one with asset_id + position). Prefer search_assets(humanoid:true) SM_Chr_… prefabs — not Characters.fbx atlases. Auto-scales to ~1.7 m (kids ~1.25 m). Auto-picks idle/walk clips from the character's npc_clips, SiteVars, or the library. Wander + click-to-talk by default.
| Argument | Type | Description |
|---|---|---|
object_ids |
array | exact object ids |
group |
string | folder path in the scene hierarchy, e.g. 'Nature/Trees', 'Village/Houses' |
name_filter |
string | every object whose name contains this text |
asset_id |
string | place this character prefab then make it an NPC |
position |
object | — fields: x, y, z |
rotation_y |
number | |
scale |
number | uniform; omit to auto-fit person height |
name |
string | |
clip_id |
string | idle animation asset id |
walk_clip_id |
string | |
interact_clip_id |
string | |
wander_radius |
number | metres to roam (default 5; 0 = stand still) |
wander_speed |
number | m/s, default 1.2 |
wander |
boolean | default true when making an NPC |
interact |
boolean | clickable / F / VR trigger |
message |
string | what they say when interacted with |
remove |
boolean |
✱ required
set_simulation
Play / pause / restart the simulation clock (NPCs animate and wander while playing — default on). Use when the user says pause, freeze, stop, resume, play, or restart / reset people to their spawn.
| Argument | Type | Description |
|---|---|---|
action ✱ |
string (play, pause, restart) | play = resume; pause = freeze; restart = snap NPCs back to spawn and play |
✱ required
place_asset
Place one library asset in the scene. position is the centre of its footprint at its base. The result warns when it lands inside another object.
| Argument | Type | Description |
|---|---|---|
asset_id ✱ |
string | |
position ✱ |
object | — fields: x, y, z |
rotation_y |
number | |
scale |
number | uniform, default 1 |
name |
string | |
group |
string | folder path in the scene hierarchy, e.g. 'Nature/Trees', 'Village/Houses' |
static |
boolean | scenery that never moves (instanced, not clickable) |
on_top_of |
string | object id to stand this on (sets y to its top) |
✱ required
scatter_assets
Scatter many copies of one or more assets naturally over an area with spacing, random rotation and scale. Best for forests, rocks, foliage, crowds of props.
| Argument | Type | Description |
|---|---|---|
asset_ids ✱ |
array | 1-8 asset ids, picked at random per copy |
count ✱ |
integer | 1-150 |
area ✱ |
object | Where to scatter: a circle {center:{x,z}, radius, inner_radius?} (inner_radius leaves a clearing) or a rectangle {min_x,max_x,min_z,max_z}. — fields: center, radius, inner_radius, min_x, max_x, min_z, max_z |
min_spacing |
number | metres between copies; default from asset size |
scale_min |
number | |
scale_max |
number | |
random_rotation |
boolean | |
avoid_existing |
boolean | default true |
group |
string | folder path in the scene hierarchy, e.g. 'Nature/Trees', 'Village/Houses' |
static |
boolean | default true — scattered scenery is static |
max_slope |
number | 0 flat … 1 steep; default 0.6 (keeps trees off cliffs) |
min_height |
number | |
max_height |
number | terrain height band in metres |
allow_underwater |
boolean |
✱ required
place_primitive
Build a simple shape when no asset fits (water pond = flat blue cylinder, glowing crystal = emissive cone, wall = box).
| Argument | Type | Description |
|---|---|---|
shape ✱ |
string (box, sphere, cylinder, cone, plane, torus, capsule) | |
size |
object | metres {x,y,z} — fields: x, y, z |
color |
string | #rrggbb |
emissive |
boolean | |
position ✱ |
object | — fields: x, y, z |
rotation_x |
number | |
rotation_y |
number | |
name |
string | |
group |
string | folder path in the scene hierarchy, e.g. 'Nature/Trees', 'Village/Houses' |
static |
boolean | |
on_top_of |
string |
✱ required
add_light
Add a point light (fire, lamp, magic glow).
| Argument | Type | Description |
|---|---|---|
position ✱ |
object | — fields: x, y, z |
color |
string | |
intensity |
number | default 4 |
range |
number | metres, default 12 |
name |
string | |
group |
string |
✱ required
modify_objects
Change existing objects, chosen by object_ids, group or name_filter.
| Argument | Type | Description |
|---|---|---|
object_ids |
array | exact object ids |
group |
string | move into this folder |
name_filter |
string | every object whose name contains this text |
position |
object | absolute (single object only) — fields: x, y, z |
offset |
object | move by — fields: x, y, z |
rotation_y |
number | |
rotate_by |
number | |
scale |
number | |
scale_by |
number | |
color |
string | |
name |
string | |
static |
boolean | |
intensity |
number |
✱ required
delete_objects
Remove objects chosen by object_ids, group or name_filter — or everything with all: true. `area` limits it to a circle/rectangle (alone it removes everything there; static_only: true just the scenery).
| Argument | Type | Description |
|---|---|---|
object_ids |
array | exact object ids |
group |
string | every object in this group |
name_filter |
string | every object whose name contains this text |
all |
boolean | |
area |
object | Where to scatter: a circle {center:{x,z}, radius, inner_radius?} (inner_radius leaves a clearing) or a rectangle {min_x,max_x,min_z,max_z}. — fields: center, radius, inner_radius, min_x, max_x, min_z, max_z |
static_only |
boolean |
✱ required
set_environment
Set sky/lighting and ground.
| Argument | Type | Description |
|---|---|---|
preset |
string (holodeck, day, sunset, night, overcast, space) | |
ground |
string (grid, grass, sand, snow, dirt, water, stone, asphalt, none) | |
ground_size |
number | metres across, default 120 |
fog |
number | 0-1 |
sky_color |
string | |
ground_color |
string |
✱ required
focus_camera
Point the user's camera at objects or a spot.
| Argument | Type | Description |
|---|---|---|
object_ids |
array | |
target |
object | — fields: x, y, z |
position |
object | camera position — fields: x, y, z |
✱ required
add_camera
Place a scene camera the player can switch to (keys 1-9).
| Argument | Type | Description |
|---|---|---|
name |
string | |
position ✱ |
object | camera position; y default 1.7 (eye height) — fields: x, y, z |
look_at ✱ |
object | point it looks at; y default 1 — fields: x, y, z |
fov |
number | degrees, default 55 |
✱ required
create_material
Create a scene material: colour + PBR values, optional glow, transparency and procedural pattern.
| Argument | Type | Description |
|---|---|---|
name ✱ |
string | |
color |
string | #rrggbb base colour |
roughness |
number | 0 glossy - 1 matte, default 0.8 |
metalness |
number | 0-1 |
emissive |
string | #rrggbb glow colour |
emissive_intensity |
number | |
opacity |
number | 0.05-1; below 1 is see-through (glass, water) |
pattern |
string (none, checker, stripes, bricks, tiles, noise, wood, marble, dots, grid) | |
pattern_color |
string | second colour of the pattern |
tile_size |
number | metres per pattern repeat, default 2 |
tint |
boolean | on library assets: keep their own texture and multiply by color |
✱ required
generate_texture
Paint a new seamless, tileable image texture from a description (AI image, ~6 s) and make it a material.
| Argument | Type | Description |
|---|---|---|
prompt ✱ |
string | the surface, e.g. 'mossy cobblestone', 'polished white marble with grey veins' |
name |
string | |
tile_size |
number | metres per repeat, default 2 |
roughness |
number |
✱ required
set_terrain
Turn on (or reshape) the endless procedural terrain: rolling hills, mountains, islands… Grounded objects move onto the new ground.
| Argument | Type | Description |
|---|---|---|
enabled |
boolean | |
style |
string (flat, hills, plains, mountains, islands, dunes) | |
height |
number | relief amplitude in metres (hills 8–20, mountains 30–80) |
scale |
number | feature size in metres (60 = bumpy, 300 = broad) |
roughness |
number | 0 smooth … 1 rugged |
seed |
integer | a different landscape |
water_level |
number | null | sea / lake level in metres; null = no water |
flat_radius |
number | keep the ground flat within this many metres of the origin (a building site) |
✱ required
sculpt_terrain
Sculpt the terrain in a circle: raise or lower (strength = metres at the centre), flatten (to `height`, or the centre's height) or smooth (strength 0–1). Grounded objects follow.
| Argument | Type | Description |
|---|---|---|
mode ✱ |
string (raise, lower, flatten, smooth, noise, terrace) | raise / lower (strength = metres), flatten / smooth / noise (roughen) / terrace (stepped slopes), strength 0–1 |
center ✱ |
object | — fields: x, y, z |
radius ✱ |
number | metres, up to 80 |
strength |
number | |
height |
number | flatten: target height · terrace: step height (default 2 m) |
hardness |
number | 0 soft edge … 1 hard edge (default 0) |
✱ required
paint_terrain
Paint a ground layer in a circle (paths, beaches, rocky patches, snow caps). 'auto' restores the automatic colouring.
| Argument | Type | Description |
|---|---|---|
layer ✱ |
string (auto, grass, dirt, sand, rock, snow, path, moss) | |
center ✱ |
object | — fields: x, y, z |
radius ✱ |
number | |
hardness |
number | 0 soft, dithered edge … 1 crisp edge (default 0.5) |
✱ required
sculpt_path
Carve along a line of points: a road (flattened, following the ground, painted 'path'), a river or ditch (dug `depth` below the ground, sandy banks — dig below water_level to fill it), or a ridge / embankment (raised). Objects on it are re-settled.
| Argument | Type | Description |
|---|---|---|
points ✱ |
array | 2–200 points along the line |
kind ✱ |
string (road, river, ridge) | |
width |
number | metres (default 4) |
depth |
number | river depth / ridge height in metres (default 2) |
layer |
string | null | layer painted down the middle (default path for roads, sand for rivers); null = none |
strength |
number | 0–1 (default 1) |
✱ required
paint_by_rule
Paint a ground layer wherever the terrain matches, inside an area: snow above a height, rock on steep slopes, sand just above the water, moss in hollows.
| Argument | Type | Description |
|---|---|---|
layer ✱ |
string (auto, grass, dirt, sand, rock, snow, path, moss) | |
area ✱ |
object | Where to scatter: a circle {center:{x,z}, radius, inner_radius?} (inner_radius leaves a clearing) or a rectangle {min_x,max_x,min_z,max_z}. — fields: center, radius, inner_radius, min_x, max_x, min_z, max_z |
min_height |
number | |
max_height |
number | |
min_slope |
number | 0 flat … 1 steep |
max_slope |
number |
✱ required
set_terrain_layer
Change how a ground layer looks everywhere it's painted (or auto-chosen): a colour, or a scene material — e.g. a generate_texture result for real cobblestones.
| Argument | Type | Description |
|---|---|---|
layer ✱ |
string (grass, dirt, sand, rock, snow, path, moss) | |
color |
string | #rrggbb |
material_id |
string | |
reset |
boolean | back to the default look |
✱ required
undo_terrain
Undo the last terrain sculpt / paint / path edit (repeat to go further back).
sample_terrain
Ground height and slope (0–1) at points — before placing something big or choosing where to build.
| Argument | Type | Description |
|---|---|---|
points ✱ |
array |
✱ required
manage_group
Act on a folder of the scene hierarchy (and its subfolders): mark static, hide, lock, rename/move to another path, offset everything, or delete it.
| Argument | Type | Description |
|---|---|---|
group ✱ |
string | folder path in the scene hierarchy, e.g. 'Nature/Trees', 'Village/Houses' |
static |
boolean | |
hidden |
boolean | |
locked |
boolean | |
rename_to |
string | |
offset |
object | — fields: x, y, z |
delete |
boolean |
✱ required
build_structure
Build a room or building from a pack's modular pieces on an exact grid — walls meet with no gaps or overlaps, doors and windows sit in wall slots, storeys stack, the roof closes it. Prefer build_from_recipe for cabins/shops/taverns (hardcoded shells). Use this when you already know the piece ids.
| Argument | Type | Description |
|---|---|---|
floor_asset_id ✱ |
string | |
wall_asset_id ✱ |
string | |
door_asset_id |
string | a door / doorway wall piece (doorframe with door) |
window_asset_id |
string | optional window wall piece (used on alternate segments when windows: true) |
roof_asset_id |
string | optional; default: a layer of floor tiles |
roof |
boolean | false = open top |
center ✱ |
object | — fields: x, y, z |
size ✱ |
object | footprint in metres {x, z}; rounded to whole floor tiles — fields: x, z |
rotation_y |
number | |
stories |
integer | 1-4 |
doors |
array | default one door in the middle of the south (-z) side |
windows |
boolean | |
name |
string | |
group |
string | folder path in the scene hierarchy, e.g. 'Nature/Trees', 'Village/Houses' |
flatten_ground |
boolean | default true — levels the terrain under it |
interactive_doors |
boolean | default true — doors open and close |
✱ required
list_building_recipes
List hardcoded building recipes (cabin, shop, tavern, barn, house, outpost). Prefer these over inventing wall layouts.
build_from_recipe
Build a recipe building: resolves modular floor/wall/door/window from the asset lock, runs StructureBuilder, optionally furnishes the interior. LLM only chooses recipe, center, name, rotation.
| Argument | Type | Description |
|---|---|---|
recipe ✱ |
string (cabin, shop, tavern, barn, house, outpost) | |
center ✱ |
object | — fields: x, y, z |
name |
string | |
rotation_y |
number | |
size |
object | optional override of recipe footprint — fields: x, z |
stories |
integer | |
pack |
string | optional pack_id to pull pieces from |
furnish |
boolean | default true — place furniture props inside |
group |
string | folder path in the scene hierarchy, e.g. 'Nature/Trees', 'Village/Houses' |
flatten_ground |
boolean |
✱ required
set_world_meta
Set world_depth (simple|standard|deep) and/or persist a world_plan (title, lore, buildings). Depth gates how many buildings you may add.
| Argument | Type | Description |
|---|---|---|
world_depth |
string (simple, standard, deep) | |
world_plan |
object | { title, lore, depth, buildings: [{ id, recipe, name, x, z, rotation_y, npc_name, npc_message }] } |
✱ required
build_world
Build a whole settlement from a goal: picks a depth (simple/standard/deep), lays out recipe buildings, flavours names/lore with the model, furnishes, places NPCs and scatter. Use for village/town/world goals instead of placing walls by hand.
| Argument | Type | Description |
|---|---|---|
goal ✱ |
string | e.g. 'a deep lakeside village with a tavern' |
depth |
string (simple, standard, deep) | optional override |
flavour |
boolean | default true — one LLM pass to customise names/lore |
✱ required
shop_buy
Buy stock from a shop NPC / counter (settings.shops[object_id]). Spends player credits.
| Argument | Type | Description |
|---|---|---|
object_id ✱ |
string | |
sku ✱ |
string | |
qty |
integer |
✱ required
inventory
Show player credits/inventory and open shops on this scene.
place_container
Put a chest (container) on an object with a loot table. Treasure class common/dungeon/boss, or pass an explicit loot table {rolls, credits:[min,max], entries:[{sku,weight,min,max}]}. Opens into the player inventory.
| Argument | Type | Description |
|---|---|---|
object_ids ✱ |
array | exact object ids |
group |
string | every object in this group |
name_filter |
string | every object whose name contains this text |
class |
string (common, dungeon, boss) | treasure class (or pass loot for a custom table) |
title |
string | chest name, default '<object> chest' |
loot |
object | explicit loot table |
✱ required
open_container
Open a chest and collect its loot into the player's inventory (one-shot — marks it opened).
| Argument | Type | Description |
|---|---|---|
object_id ✱ |
string |
✱ required
list_items
List the loot item catalog (sku, name, kind, rarity) and treasure classes.
spawn_mob
Turn an object (usually an NPC) into a mob — friendly / normal / elite / boss with a loot table. Mobs are 'defeated', not killed; a sim restart revives them.
| Argument | Type | Description |
|---|---|---|
object_ids ✱ |
array | exact object ids |
group |
string | every object in this group |
name_filter |
string | every object whose name contains this text |
name |
string | |
kind |
string (friendly, normal, elite, boss) | friendly (won't fight) / normal / elite / boss |
faction |
string | |
loot_class |
string (common, dungeon, boss) | |
loot |
object | explicit loot table |
✱ required
fight_mob
Fight a mob: deterministic exchange driven by your best weapon. Victory drops its loot + XP and advances kill-quests.
| Argument | Type | Description |
|---|---|---|
object_id ✱ |
string |
✱ required
mob_list
List mobs and whether they're defeated.
story_list
List world_plan story beats and which are ready (trigger met) / done.
advance_story
Fire the next due story beat (grants XP + narrates it).
quest_list
List world_plan quests and their progress.
quest_advance
Advance a quest objective (talk / buy / visit / loot / kill / own). Completes objectives in order; the last one grants the reward.
| Argument | Type | Description |
|---|---|---|
quest_id ✱ |
string | |
event ✱ |
string | |
target |
string | optional target match (e.g. NPC role) |
note |
string |
✱ required
request_continue
After this turn finishes, auto-start a follow-up agent goal (chained deep worlds). Max 3 hops.
| Argument | Type | Description |
|---|---|---|
goal ✱ |
string |
✱ required
check_layout
Check the whole scene for objects inside each other, hanging in the air or sunk into the ground. Run it after building and fix what it reports.
make_interactive
Make doors (or lids, gates, hatches) open and close: the player clicks them in walk / fly / VR, or they open by themselves when the player comes near (auto). The moving part of a prefab is found automatically. Also opens / closes them (open: true / false) or removes the behaviour (remove: true).
| Argument | Type | Description |
|---|---|---|
object_ids |
array | exact object ids |
group |
string | every object in this group |
name_filter |
string | every object whose name contains this text |
kind |
string (hinge, slide, lift) | hinge (swings), slide (sideways, sci-fi), lift (up, garage) |
hinge |
string (left, right) | |
angle |
number | hinge swing in degrees, default 90 (negative swings the other way) |
direction |
string (left, right) | slide direction |
open |
boolean | |
auto |
boolean | |
part |
string | prefab node name(s) that move, comma-separated; default auto |
remove |
boolean |
✱ required
look_at_scene
Take a picture of the scene through the user's editor and look at it yourself — check composition, gaps, things floating or buried, whether it matches the goal. Frames everything by default, or object_ids, or a camera position / target.
| Argument | Type | Description |
|---|---|---|
object_ids |
array | |
target |
object | point to look at — fields: x, y, z |
position |
object | camera position — fields: x, y, z |
✱ required
apply_material
Put a scene material on objects (object_ids / group / name_filter) and/or the ground. material_id null restores the original look.
| Argument | Type | Description |
|---|---|---|
object_ids |
array | exact object ids |
group |
string | every object in this group |
name_filter |
string | every object whose name contains this text |
material_id ✱ |
string | null | |
ground |
boolean | apply to the ground |
✱ required