@pithagoras/godot0.2.1

Godot for Pithagoras

@pithagoras/godot runs the real Godot 4.7.2 editor in a container on your portal's Docker, rendered on an NVIDIA GPU, and brings it into Pithagoras in two ways: the agent in every chat gets the editor's 30 tools, and you get the editor's live screen in a panel beside the chat, where you can watch the agent work and take the mouse and keyboard at any time. Every edit the agent makes happens in that editor, through its undo history.

Not made or endorsed by the Godot Foundation. "Godot" names what the extension runs.

What it needs

• Pithagoras 0.3.0 or later.
• Docker for extensions (pithagoras-docker 1.3.0 or later), which ships with the portal but is not on by default. Installing this extension from npm installs or switches it on along with it; installing it from a folder does not, and the extension then waits, saying so in Settings → Extensions, until you install Docker for extensions from Settings → Extensions → Discover.
• A Linux host with Docker Engine 28.0 or later: the server's network has no gateway, which older Docker cannot make.
• For the GPU, an NVIDIA card and the NVIDIA Container Toolkit. Without one, Godot opens Compatibility projects (the 2D template) with Mesa's software OpenGL and cannot render Forward+ projects.
• The portal over HTTPS, or opened as http://localhost. The editor's stream decodes with WebCodecs, which browsers give only a secure context, and the panel is one only if the portal's page is. Over plain HTTP on another name the panel says so and shows how to fix it (PORTAL_TLS_CERT and PORTAL_TLS_KEY, see the Pithagoras guide's HTTPS section).

Installing

In Settings → Extensions → Discover → Install by name:

From	Spec
npm	npm:@pithagoras/godot
A built folder, for development	the folder's path on the portal's machine, after npm run build in it

The first start

Settings → Godot → Install. The extension downloads its image (ghcr.io/prometheion/pithagoras-godot:0.2.1, about 0.6 GB unpacked), makes a volume for the projects and a network of its own (pithagoras-godot-net), makes the container on it and starts it, and opens the portal's way to it (below, under Security). The page shows the download, then the server starting, then the editor's own state ("Editor ready: starter"). The first start makes a 3D project called starter; the agent and you make more.

The card: the page lists the NVIDIA cards Docker can give a container, read in a short-lived probe container (pithagoras-godot-probe), with how much of each card's memory is in use. Automatic gives a new server the card with the most free memory and keeps it there; choosing a card moves a running server to it at once (the container is made again, the projects stay). A host owner can pin the card with PITHAGORAS_GODOT_GPU; a pinned card Docker cannot give (no card by that UUID or index, or no NVIDIA runtime) is said on the page, with Docker's words, and Install is refused rather than making a container that cannot start.

The screen: 1280 × 720, 1600 × 900 (the default) or 1920 × 1080. The editor fills it, and the panel scales all of it to fit, so a smaller screen is sharper in a narrow panel. Changing it restarts the editor.

The panel

The chat's header has a Godot button once the server is installed; it shows a dot while one of the Godot tools runs in that chat. The panel holds the viewer, served by the portal at /api/x/@pithagoras/godot/ui/ behind the portal's sign-in, so you never sign in to the server. The viewer's page and Selkies' web client (pages/ in the package) are the extension's own; what they ask of the server (its API, the downloads, the stream's WebSocket) is passed on to the container with its token. In it:

• the editor itself, streamed by Selkies: mouse, keyboard and the clipboard in both directions;
• the project menu (the project's name, top left): every project with its renderer, Open, New 2D / New 3D, Upload .zip, Download open project, and the open project's latest builds (export) and recordings (record) to download;
• Run and Stop: the project's main scene runs in the editor, in a window over it, as the editor's own Play button runs it;
• Activity: the agent's calls as they happen.

Fullscreen in the panel's head gives it the whole screen. Only one view of the stream is connected at a time: opening the panel in another chat, tab or device takes it over, and the one that lost it says so, with Reconnect here. Switching chats reconnects the stream (2-3 s).

The tools

The server's MCP tools are the extension's own tools in every chat, named godot_ and the server's name: godot_ping, godot_list_projects, godot_new_project, godot_open_project, godot_restart_editor, godot_list_files, godot_read_file, godot_write_file, godot_delete_file, godot_get_scene_tree, godot_open_scene, godot_new_scene, godot_add_node, godot_set_property, godot_delete_node, godot_attach_script, godot_save_scene, godot_reload, godot_run_project, godot_run_scene, godot_stop, godot_get_output, godot_game_input, godot_game_state, godot_run_playtest, godot_check_scripts, godot_screenshot, godot_record, godot_export and godot_execute_gdscript. Their descriptions, their schemas and the server's MCP instructions (said to the agent once) are the ones this version of the extension was released with, in server/tools.json, which the release's smoke test holds the image to: never what the running server says of itself, since the agent's code runs in that container. A tool the server lists that the file does not name is not relayed, and one it names that the server does not list (an older image) is not offered.

They are on in every chat by default. Settings → Tools lists them under @pithagoras/godot, where the group or a single tool can be switched off, or made lazy (offered through the tools tool instead of in every request: the 30 cost about 4k tokens a request), for the whole portal, an agent, a project or a mode. They leave every chat when the extension is switched off or removed, and while the server is not installed.

godot_screenshot and godot_record return a picture, as do godot_game_state with image and a visible godot_run_playtest (its checkpoints), which reaches the model as a picture (a model that sees pictures, input: ["text", "image"]). The chat's card of the call shows its text; the picture is what the panel shows.

Don't add an MCP server named godot beside this extension: its tools would have the same names.

Playing and testing the game

Three of the tools reach into the game the agent builds, where godot_execute_gdscript, which runs in the editor, cannot:

• godot_game_input plays the game the editor runs: it taps, presses or releases an action of the project's Input Map (move_right, jump, ui_accept) or a key, or clicks or moves the mouse at a point in the game's viewport. A tap holds for some physics frames (or seconds), lets go, and answers with the frames: {"action": "jump"} gives {"sent": "tap jump", "pressed_frame": 812, "released_frame": 813, "frames_held": 1}.
• godot_game_state reads the running game: its live node tree (each node's path, type and a few telling values such as position, velocity and text) and Godot expressions evaluated with the current scene as their base, such as get_node("Player").position or get_tree().get_nodes_in_group("enemies").size(), each value as JSON ([x, y] for a Vector2) and each failed expression in errors, optionally after some physics frames, and with image, the frame those values are from.
• godot_run_playtest runs the tests the agent writes in the project's res://tests/: files named test_*.gd that extend PlayTest, each test_* function a test run on the scene loaded afresh. Headless (the default), the project runs with no window at a fixed frame rate, so a run repeats exactly, and takes a fraction of a second for a short test; visible, it runs from the editor in its window, at the same rate and capped to it, so you watch it at normal speed, and its checkpoints come back as pictures. The answer says which tests passed, each failed check with its line and message, the game's errors and what it printed. Without a path, every test_*.gd in res://tests/ runs.

extends PlayTest

func test_jump() -> void:
    var player := node("Player")
    await wait_frames(10)
    var y: float = player.position.y
    await press("jump")
    await wait_frames(15)
    check(player.position.y < y - 40, "the player rises after jump")
    checkpoint("in the air")

PlayTest
await press(action, frames = 1), await hold(action), await release(action)	an input action: a tap, held down, let go
await key(name, frames = 1)	a key: Space, Enter, Left, A, ...
await click(Vector2, button = "left"), await move_mouse(Vector2)	the mouse, in the game's viewport
await wait_frames(n), await wait_seconds(s)	let the game run (physics frames; seconds are whole frames)
await wait_until(callable, max_frames = 600)	wait for a condition; giving up fails the test
node(path), scene, frame	a node of the scene (a path that names none fails the test), its root, the frames since the test began
check(condition, message), check_eq(got, want, message), fail(message)	a failed check fails the test at its line and the test goes on
checkpoint(name)	a picture of the game as it is (in a visible run)

A script error in a test fails it at its line; one in the game while a test runs fails it too, with the game's file and line. Frames are physics frames throughout: input is applied at the end of a physics frame, after the game's own physics, as a real device's input arrives between two frames, and the game sees it from the next frame on.

How it gets into the game. The editor plugin adds -s res://addons/godot_mcp_bridge/game.gd to the command line of each game the editor runs (EditorPlugin._run_scene, which every run from the editor goes through: the tools, the panel's Run and the editor's own Play buttons). That script is the game's SceneTree, its main loop, and adds no node to it: the game's main scene, autoloads and settings load as they always do. It talks to the editor over the debugger connection the editor already has with each game it runs (EngineDebugger in the game, an EditorDebuggerPlugin in the editor). A headless playtest is the project run by the server with --headless --fixed-fps 60 -s and the same script. Nothing is written to the project, and no export has it: nothing gives an exported game that command line, and the export presets the server writes leave the addon's scripts out (Godot still packs its plugin.cfg, an inert text file).

Security

• What the agent's code runs as. The container runs as the image's own user (godot, uid 10001), not root, with no Linux capabilities at all (CapDrop: ALL) and no-new-privileges: the server, Xvfb, Selkies, the editor, the games and the agent's GDScript alike. The image's files (the server, Selkies, Godot) are root's and cannot be changed from inside. The server runs Python in isolated mode (-I, so nothing written under its home is imported at its next start) and keeps its own process from the others of its user (PR_SET_DUMPABLE 0: none can attach to it or read its memory or its environment). It takes the stream's Unix socket only from the Selkies it started (the listener's process has to be in that Selkies' session), so code in the container cannot put a server of its own in Selkies' place.
• What it reaches. The container is on pithagoras-godot-net, which has no way out. What godot_execute_gdscript, a @tool script or a game reaches is the server and, through the forwarder's port, the server again; for a portal in a container of its own, also what is listened for on 0.0.0.0 in that container's network namespace: the portal's own port, which asks for its sign-in, and Understory's (3800), which shares the namespace and asks for its token. Not the host, its loopback, the local network or the internet. The server's token is in the container's environment, which code there can read (the init process is the same user's): with it, that code can call the server it already runs in, nothing more.
• Nothing it serves runs on the portal's origin. The viewer and Selkies' web client come from the package, and the server sends no page: what it answers is JSON, files to download (served with sandbox, and as attachments but for pictures and videos), and the stream's frames. Web builds are files to download, not pages served there: a web export runs the project's code, and Godot's JavaScriptBridge lets a game run any script in the page. The image's build stops unless the Selkies in it has the web client the package serves (server/selkies-web.sha256).
• One credential. A token made at random when the server is first installed (24 bytes), kept in the extension's own folder (settings.json, mode 0600, which only the portal's user can open), and given to the container. Every route of the server but /healthz asks for it as Authorization: Bearer. The portal's proxy adds it; no page is given it. There is no password, default or otherwise.
• A network of its own. The container is on pithagoras-godot-net, which Docker for extensions makes internal and with no gateway (gateway mode isolated): nothing on it reaches the host, the local network or the internet, and nothing but the portal reaches it. The server listens on 0.0.0.0 of that network. For a portal on the host's network, the forwarder pithagoras-godot-forward publishes port 3880 on 127.0.0.1 and passes it on; for a portal in a container of its own, that container is on the network. Selkies listens on a Unix socket in a folder only the server can open, and the X display is the container's own (-nolisten local). The editor's own listeners (its bridge plugin, with a secret of its own per start, the GDScript language server, the debug adapter and the debugger) take free loopback ports at each start.
• Through the portal. Every route of the extension is behind the portal's sign-in, and any of them that changes something (Install, Start, Stop, Remove, the card, the screen, and every write the viewer makes) is refused unless it comes from the portal's own pages, by its Origin or Sec-Fetch-Site: same-origin: the portal's cookie is SameSite=Lax, which a page on another port of the same address is still sent. The proxy sends the server the request without the portal's cookie and the page's own credentials, adds the token, lets only the portal frame the answers (frame-ancestors 'self'), adds nosniff, and passes no cookie the container sets.
• The tools run code, and say only what the package says. godot_execute_gdscript runs GDScript in the editor; written scripts load at once and @tool scripts run in the editor; every game is a program, which godot_game_input drives, godot_game_state's expressions run code in (they can call any method) and godot_run_playtest runs the agent's tests in. Every Godot tool's result is outside content to the guard, and once a conversation has read a result the guard flags as a likely prompt injection, it holds godot_write_file, godot_attach_script, godot_run_project, godot_run_scene, godot_game_input, godot_game_state, godot_run_playtest, godot_record and godot_execute_gdscript as uploads: it asks you first, or refuses where nobody can answer. What the model is told of the tools, and the instructions in every chat's prompt, are the package's (server/tools.json), never what the running server says of itself.
• Who can use it. Everyone signed in to the portal can use the panel, and drive the editor from it.
• The other direction. For a portal on the host's network, a process on the host reaches the server's port on 127.0.0.1; for a portal in a container of its own, the processes in that container do. Neither can use it without the token. With the sandbox on, agents run as users of their own and cannot read it; with it off, the agent runs as the portal's user and could.

Data

Projects, builds, recordings and the editor's settings are in the Docker volume pithagoras-godot-data at /data in the container:

In the volume
projects/<name>/	the projects (each gets the editor plugin addons/godot_mcp_bridge)
exports/<name>/web/, exports/<name>/linux/	the latest build of each kind
renders/<name>/<time>/	recordings: movie.mp4 and preview.jpg, or the PNG frames
home/	the editor's settings

To back it up, for example: docker run --rm -v pithagoras-godot-data:/data -v "$PWD":/out ubuntu tar czf /out/godot-backup.tgz /data.

Settings the portal's environment can change

Variable	Default
PITHAGORAS_GODOT_CONTAINER	pithagoras-godot	the container's name; the probe is <name>-probe, the network <name>-net, the forwarder <name>-forward
PITHAGORAS_GODOT_VOLUME	<container>-data	the projects' volume
PITHAGORAS_GODOT_PORT	3880	the server's port, and for a portal on the host's network the forwarder's on 127.0.0.1
PITHAGORAS_GODOT_IMAGE	ghcr.io/prometheion/pithagoras-godot:<version>	another image, such as one built locally
PITHAGORAS_GODOT_GPU		a card's UUID or index: the only card the server is given

They are for a second portal on the same Docker daemon, for tests, and for a host owner who keeps the server off a card or a port.

Removing it

Remove on the Settings page first, then uninstall the extension: the container is restarted by Docker unless stopped (unless-stopped), and outlives the extension, which has no say once it is uninstalled. Remove asks whether to delete the projects too, and also removes the network and the forwarder. The image stays on disk, as other containers may use it: docker image rm ghcr.io/prometheion/pithagoras-godot:0.2.1 deletes it.

Limits

• One editor serves every chat and you: agents in two chats work in the same open project, and nothing locks.
• One view of the stream at a time; the panel's frame reloads, and the stream reconnects, on every chat switch.
• A modal dialog in the editor blocks the agent's calls until someone answers it in the panel; godot_restart_editor clears it.
• No window manager: a game opens as a window over the editor, not in the editor's Game tab.
• A project that sets application/config/disable_project_settings_override ignores the -s the editor adds: its games run without the bridge's game side, and godot_game_input, godot_game_state and visible playtests say so. A project with a main loop of its own (application/run/main_loop_type) runs from the editor with the bridge's SceneTree in its place.
• A headless playtest draws nothing: its checkpoints give the frame, and pictures come with a visible run.
• The stream is encoded on the CPU (x264): NVENC wants the first GPU's device node, which a container that sees only another card does not have.
• For a portal on the host's network, port 3880 on 127.0.0.1 is the forwarder's: a port in use shows as the server failing to start, with Docker's "address already in use"; PITHAGORAS_GODOT_PORT moves it.
• An upload must arrive within Node's request timeout (300 s): about 3 GB at 10 MB/s, less on a slow link; the server takes up to 2 GB.
• When the portal's container is made again or restarted, the editor keeps running.
• Nothing in the container reaches the internet: the editor's Asset Library, and a game or a script that downloads, do not work there. Bring files in with the panel's Upload.
• Only Chrome is tested.

Licences

The extension's own code (TypeScript, the viewer pages/viewer.html, the stream page's shim pages/stream/__gs_shim.js, the server server/server.py, the end-to-end client server/client.py and the editor plugin server/addon/godot_mcp_bridge) is under the Apache License 2.0 (see LICENSE and NOTICE). The plugin is copied into each project the server opens, and is Apache-2.0 there; Godot is MIT, so a project and its plugins may be under any licence.

The npm package contains no Godot. It does contain Selkies 2.1.0's web client, unchanged (pages/stream: the files of selkies/selkies_web that the stream page loads, its index.html served with one script tag added for the shim), under the Mozilla Public License 2.0, whose text is beside it (pages/stream/LICENSE); its source is Selkies' own at https://github.com/selkies-project/selkies/tree/2.1.0.

The image it runs redistributes, unchanged: Godot 4.7.2 (MIT, with the third-party notices of its COPYRIGHT.txt), Selkies 2.1.0 (MPL-2.0) with the Python packages of its own environment, each under its own licence in its dist-info folder; among them pixelflux 2.2.1 (MPL-2.0, bundling libx264 and x265, GPL-2.0-or-later, so that combined binary is under the GPL, and libde265, LGPL-3.0-or-later) and pcmflux 2.2.0 (MPL-2.0, bundling AlmaLinux 8 builds of libpulse, libsndfile, libsystemd, libgcrypt and other libraries under LGPL-2.1-or-later, each a separate shared library); VirtualGL 3.1.5 (the wxWindows Library Licence 3.1); and Ubuntu 24.04 packages, each under its own licence. The image carries the licence texts and a NOTICE in /usr/share/doc/pithagoras-godot, and each release that publishes an image attaches the sources the GPL and the LGPL ask for that are not in Ubuntu's archive (pixelflux and the x264, x265 and libde265 it was built from; pcmflux and the AlmaLinux source RPMs of the LGPL libraries it bundles; VirtualGL), with the list of Ubuntu packages and versions in the image. See server/NOTICE.

Building it

npm ci                     # in the repository's root: the workspace's tools
npm run build -w godot     # web/ and lib/server.js
npm test -w godot          # the unit tests

The image: server/fetch-assets.sh fetches and checks what it is built from into server/assets/ (the Godot editor and export templates, Selkies and VirtualGL packages, licence texts; it needs curl and bsdtar, libarchive-tools on Debian and Ubuntu), then docker build server/. The release workflow does both, tests the image (server/smoke-test.sh, which runs it as the extension does, and holds its tools and instructions to server/tools.json), and publishes it before the package.

Selkies' web client in pages/stream is copied unchanged from Selkies' .deb (selkies/selkies_web), and server/selkies-web.sha256 lists its files: the image's build checks the Selkies it installs against the list, and a unit test checks pages/stream against it. Moving Selkies means copying the new client there and writing the list again.