Skip to content
Back to search

video-to-td-corridor

SkillWorks with Claude CodeGitHub

Turn a forward-moving video (dolly / walk / drive down a street or hallway) into a 3D point-cloud scan and build the "3DGS infinite corridor" treadmill effect in a running TouchDesigner 2025 session with POPs. Use when asked to make an infinite corridor, endless street, splat/point-cloud treadmill, or to scan a video into TouchDesigner.

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/jarvisrr--video-to-td-corridor/badge.svg)](/listings/jarvisrr--video-to-td-corridor)

HTML

<a href="/listings/jarvisrr--video-to-td-corridor"><img src="/api/listings/jarvisrr--video-to-td-corridor/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

Copy this folder to `.claude/skills/video-to-td-corridor/` in a project, or to `~/.claude/skills/video-to-td-corridor/` for all projects, then ask Claude for an infinite corridor from a video.

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

SKILL.md · Manifest · 153 lines
  1. ---
  2. name: video-to-td-corridor
  3. description: Turn a forward-moving video (dolly / walk / drive down a street or hallway) into a 3D point-cloud scan and build the "3DGS infinite corridor" treadmill effect in a running TouchDesigner 2025 session with POPs. Use when asked to make an infinite corridor, endless street, splat/point-cloud treadmill, or to scan a video into TouchDesigner.
  4. ---
  5. # Video → 3D scan → infinite corridor in TouchDesigner
  6. The effect: a strip of scanned points slides past a **fixed** camera, wraps with
  7. `mod`, and both ends dissolve into noise so the wrap seam is never seen. The scan
  8. is repeated `TILES` times (default 3) end to end ahead of the camera, and the
  9. camera sits at the near end, so the whole frame travels forward.
  10. Sky: the render is composited over a heavily blurred frame of the video, and
  11. distant points fade into a haze colour. Do **not** fill the distance with a sharp static still of the video: it covers most of
  12. the frame and makes the POV look frozen even though the scan is moving (the
  13. `Backdrop Brightness` parameter exists but defaults to 0 = off).
  14. Scripts are in `scripts/`. Reference result: a 25 s street dolly, 151 frames,
  15. 13 million points, 60 fps at 1280x720 on an Apple M4 Pro.
  16. ## 1. Talk to TouchDesigner
  17. TouchDesigner must already be open. Any way of running Python inside it works:
  18. - **Textport** (Alt+T): paste the `exec(open(...).read())` line from step 3.
  19. - **A TouchDesigner MCP / bridge** that exposes an "execute Python" tool.
  20. `scripts/td_bridge.py` is a small stdio JSON-RPC client for the TDMCP
  21. codex-bridge; adapt the server path to whichever bridge is installed:
  22. ```bash
  23. python3 scripts/td_bridge.py -q execute_code '{}' code /path/to/snippet.py
  24. ```
  25. - Bridge replies truncate long values. Have the snippet write JSON to a file and read that file.
  26. - Set `result = ...` in the snippet for a short return value.
  27. - The open project may hold other work. Only create/destroy your own component
  28. (`/project1/infinite_corridor`); never save the user's `.toe` without asking.
  29. Save your component with `comp.save('x.tox')` instead, and tell the user which
  30. `.toe` is open and where it lives, because their save goes to that location.
  31. - Check what you built by saving a TOP to disk (`op('out1').save('x.jpg')`) and looking at it.
  32. ## 2. Make the scan
  33. Needs a venv with `torch transformers pillow numpy opencv-python-headless`
  34. (about 600 MB) and the model `depth-anything/Depth-Anything-V2-Metric-Outdoor-Large-hf`
  35. (1.2 GB, safetensors, cached in `~/.cache/huggingface`). Ask before downloading.
  36. For interiors use the `Metric-Indoor` variant.
  37. ```bash
  38. mkdir -p scan/frames && cp scripts/depth.py scripts/fuse.py scan/
  39. ffmpeg -i video.mp4 -vf "select='not(mod(n\,4))',scale=1920:-1" -fps_mode passthrough -q:v 2 scan/frames/f_%03d.jpg
  40. .venv/bin/python scan/depth.py # about 1 s per frame on Apple silicon
  41. .venv/bin/python scan/fuse.py 151 24 0.05 # frames, metres of depth kept per frame, voxel size
  42. ```
  43. - **Use the whole shot, densely**: about 150 frames (every 4th frame of a 25 s
  44. clip). 25 frames left torn walls and holes. Drop frames that are not the shot
  45. (cuts, fades) with `lte(n\,LAST)` in the select filter.
  46. - `fuse.py` finds the focal length by trying values and keeping the one with the
  47. most PnP inliers, poses every frame with SIFT + PnP against the keyframe before
  48. it (keyframes every N/25 frames, to limit drift), rescales each frame's depth,
  49. then **carves**: a point that two other frames see straight through is a ghost
  50. or smear and is dropped. Survivors are de-duplicated on a voxel grid keeping
  51. the closest observation.
  52. - Two outputs, same coordinates (x right, y up, travel along −z, centred on
  53. z = 0, origin 1.7 m above the ground):
  54. - `street_scan.ply`: the detailed near layer (depth < kept value).
  55. - `street_far.ply`: building faces beyond that, up to 75 m, coarser (0.12 m),
  56. sky pixels removed, and removed wherever the near layer already has a
  57. surface. Without it tall buildings are cut off and the top of the frame is empty.
  58. - `street_scan.json`: `length`, `cam_x` (an obstacle-free lane for the camera),
  59. `focal_px`, `voxel`, `far_voxel`.
  60. - Look at `scan/preview_top.jpg`, `preview_side.jpg` and `preview_far_front.jpg`
  61. before going on. Ground markings and signs should be readable.
  62. - Tuning: far ground curling upward → lower the depth-kept value. Walls missing
  63. → raise `HALF_WIDTH` or the depth-kept value. Too many points → raise the voxel size.
  64. ## 3. Build the corridor
  65. Run inside TouchDesigner:
  66. ```python
  67. CORRIDOR_VIDEO = '/path/to/video.mp4'
  68. CORRIDOR_SCAN = '/path/to/scan/street_scan.ply' # optional
  69. exec(open('/path/to/scripts/build_infinite_corridor.py').read())
  70. ```
  71. It rebuilds `/project1/infinite_corridor` from scratch each time, so iterate by
  72. editing the script and re-running. The first-person view is `out1`; an outside
  73. view is `render_overview`.
  74. Controls are custom parameters on the component (select it, press `p`):
  75. `Travel Speed`, `Corridor Length`, `Edge Dispersion Zone`, noise and swirl
  76. strength, point sizes, `Sky Brightness`, `Haze Colour`, `Distance Haze`, and a
  77. **Record** page with a toggle that writes `out1` to a movie file.
  78. If no scan file exists it falls back to "Video frame" mode: one frame projected
  79. onto a ground + two walls box using the vanishing point. Quick, but far sections
  80. are blurry and everything is flat.
  81. ## Things that cost time (TouchDesigner 2025.33230)
  82. - **GLSL POP uniforms** default to type `float`. Set `vecNtype = 'vec4'` or
  83. `.y/.z/.w` fail with "vector swizzle selection out of range".
  84. - **Sequence parameters** need their block count set first:
  85. `g.seq.vec.numBlocks = 5`, `g.seq.attr.numBlocks = 2`.
  86. - **Output attributes**: list them in `outputattrs` (`'P Color PointScale'`).
  87. An attribute that does not exist on the input must be created in the
  88. Create Attributes sequence (`attr0name = 'color'`, `'pointscale'`). Creating
  89. one that already exists fails with "nameless block contains a member that
  90. already has a name".
  91. - Reading a missing input attribute (`TDIn_Color()` on a grid) is a compile error.
  92. That is why the two sources have separate shaders.
  93. - Compile errors are in the `<name>_info` DAT docked to the GLSL POP, not in `errors()`.
  94. - Textures in a compute shader: bind on the Samplers page and use `textureLod(s, uv, 0.0)`.
  95. - **Point File In POP**: PLY colours arrive as 0–255, so divide by 255. Set
  96. `colorfields = 'red green blue'` and clear `normalfields` to match the file.
  97. - **Point Sprite MAT**: `sizingmodel = 'perspective'` and
  98. `pointscaleattrib = 'PointScale'`. For a scan with voxel size v, a scale near
  99. `v * 1.6 * 0.35` fills gaps without turning blocky.
  100. - Grid POP needs `surftype = 'points'` to render as points.
  101. - Non-commercial licence caps every TOP at 1280×1280; render at 1280×720.
  102. - **Movie File Out TOP** records `out1`; the default codec writes large `.mov`
  103. files, so convert with `ffmpeg -i in.mov -c:v libx264 -crf 18 -pix_fmt yuv420p out.mp4`.
  104. One seamless loop lasts `Corridor Length / Travel Speed` seconds.
  105. - Compute the treadmill phase in the parameter expression
  106. (`(absTime.seconds * speed) % length`) so the shader never sees a huge float.
  107. - Put the camera in the scan's `cam_x` lane, not on the film camera's path, or
  108. the loop flies through walls where the film camera never went.
  109. - Tiling: one GLSL POP per copy (`uTile.x` = copy index) into a Merge POP; the
  110. dissolve is computed over the whole tiled length, not per copy. Three copies of
  111. 4.7M points still ran at 60 fps.
  112. ## Checking the result
  113. The test that matters is "does the POV travel forward": in consecutive frames the
  114. buildings ahead must grow and pass the camera across the whole frame, not only at
  115. the edges.
  116. Save `render1` at several moments a couple of seconds apart and look at all of
  117. them: one good frame says nothing about the rest of the loop. Also save
  118. `render_overview` to confirm both ends of the strip are dissolving.
  119. ## Known limits
  120. - This is a depth-fused point cloud, not a trained Gaussian splat: surfaces are
  121. rougher than a real 3DGS scan, thin objects can smear into rows of copies where
  122. the estimated camera motion drifts, and the coarse far layer looks blocky when
  123. it passes close to the camera.
  124. - A slow, straight dolly sees each surface from nearly one direction, so side
  125. walls are the weakest part. Footage with more parallax scans better.
  126. - Metric scale from the depth model can be off by about 2x; only proportions matter here.
README.md · README · 80 lines
  1. # video-to-td-corridor
  2. A Claude Code skill that turns a forward-moving video (a dolly, walk or drive
  3. down a street or hallway) into a 3D point-cloud scan and builds the
  4. "infinite corridor" effect in TouchDesigner 2025 with POPs: the scanned scene
  5. streams past the camera forever, appearing out of dust in the distance and
  6. flying apart as it passes.
  7. Inspired by keita.kuki's "3dgs infinite corridor" (`#touchdesigner #tdpops`).
  8. ## How it works
  9. 1. **Frames** — ffmpeg pulls about 150 frames from the whole shot.
  10. 2. **Depth** — Depth Anything V2 (metric) estimates a depth map for every frame.
  11. 3. **Camera motion** — SIFT feature matches plus PnP against the depth give each
  12. frame's camera position; the focal length is found by trying values and
  13. keeping the one that explains the motion best.
  14. 4. **Fusion** — every frame's pixels are pushed into 3D. Points that other
  15. frames can see straight through (ghosts, smears) are carved away, and the
  16. rest are merged on a voxel grid. The result is two coloured `.ply` files: a
  17. detailed near layer and a coarser layer of tall / distant building faces.
  18. 5. **Corridor** — in TouchDesigner, a GLSL compute shader slides the points
  19. toward a fixed camera and wraps them with `mod`. The scan is repeated three
  20. times end to end so there is scene all the way into the distance, both ends
  21. dissolve into noise to hide the wrap, far points fade into haze, and a
  22. blurred frame of the video is the sky.
  23. Everything is one generated component, `/project1/infinite_corridor`, with
  24. parameters for speed, dissolve, point size, sky, haze and recording.
  25. ## Requirements
  26. - TouchDesigner 2025 (POPs). The non-commercial licence works; output is capped at 1280x720.
  27. - Python 3.9+ with the packages in `requirements.txt` (about 600 MB), and ffmpeg.
  28. - The depth model `depth-anything/Depth-Anything-V2-Metric-Outdoor-Large-hf`
  29. (1.2 GB, downloaded from Hugging Face on first run). Use the `Metric-Indoor`
  30. variant for interiors.
  31. - Developed on Apple silicon (MPS); falls back to CPU elsewhere.
  32. ## Install as a skill
  33. Copy this folder to `.claude/skills/video-to-td-corridor/` in a project, or to
  34. `~/.claude/skills/video-to-td-corridor/` for all projects, then ask Claude for
  35. an infinite corridor from a video.
  36. ## Use by hand
  37. ```bash
  38. python3 -m venv .venv && .venv/bin/pip install -r requirements.txt
  39. mkdir -p scan/frames && cp scripts/depth.py scripts/fuse.py scan/
  40. ffmpeg -i video.mp4 -vf "select='not(mod(n\,4))',scale=1920:-1" -fps_mode passthrough -q:v 2 scan/frames/f_%03d.jpg
  41. .venv/bin/python scan/depth.py
  42. .venv/bin/python scan/fuse.py 151 24 0.05 # frame count, metres of depth kept per frame, voxel size
  43. ```
  44. Then in the TouchDesigner Textport:
  45. ```python
  46. CORRIDOR_VIDEO = '/path/to/video.mp4'
  47. exec(open('/path/to/scripts/build_infinite_corridor.py').read())
  48. ```
  49. View `out1` inside `/project1/infinite_corridor`.
  50. ## Files
  51. | File | Purpose |
  52. | --- | --- |
  53. | `SKILL.md` | Instructions Claude follows, including the TouchDesigner pitfalls |
  54. | `scripts/depth.py` | Depth maps for the extracted frames |
  55. | `scripts/fuse.py` | Camera poses, carving, fusion, `.ply` export |
  56. | `scripts/build_infinite_corridor.py` | Builds the TouchDesigner network |
  57. | `scripts/td_bridge.py` | Optional helper to run Python in TouchDesigner through an MCP bridge |
  58. ## Limits
  59. This is a point cloud from estimated depth, not a trained Gaussian splat.
  60. Surfaces are rougher than a real 3DGS scan, thin objects can smear where the
  61. estimated camera motion drifts, and a slow straight dolly gives weak coverage of
  62. side walls. Footage with more sideways movement scans better.

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.