Projects

Interactive & Installation

Poem Sight
AI generation, Voice interaction Temporal Convergence —— The Eternal Answer

Installation art, Futurology, AI, Go

Feeling the Feelings of Plants
Interactive App , Decentering Humanity Hao Bai Apartment, Building B1, Unit 1808
First-Person Narrative, Memory-Themed 简 JIAN
Interactive installation, Meditation Virtual Fencing
Fencing, AR BEACHCOMBERApp, User experience BUBBLE LANTERN
Interactive installation, Farewell

Film & Photography

Becoming
Descend
Dumpling Ad: Ring in the New Year with Wanchai Ferry
CHRONOS
Lost and Bound
Environmental PSA
A Non-Visual Film: The Complete Assassination of a Pianist

Experiments

Weaving Memories

About

Poem Sight (吟画)

AI Agent(Claude Code) & Voice Interaction System

Core Overview

This is a real-time interactive system based on an AI Agent workflow. Users read a line of poetry, and the system instantly presents an image matching the poetic imagery through voice recognition and AI image generation.

The core feature of this project is not just the final effect, but that the entire development process was completed through human-AI collaboration. From conception, planning, and coding to debugging, everything was achieved through conversations with Claude Code.

Technical Architecture

Voice Input → Whisper Voice Recognition → DALL-E Image Generation → TouchDesigner Real-time Display

Tech Stack

Claude Code: AI programming assistant, responsible for code generation and debugging.
TouchDesigner 2024: Real-time interactive environment and visual presentation.
OpenAI Whisper API: Voice-to-text.
OpenAI DALL-E 3 API: Text-to-image generation.
Python: Script control layer within TouchDesigner.

TouchDesigner Nodes Process Image

Audio Chain (CHOP): Microphone input → Audio recording → WAV file output.
Script Control (DAT): Core Pipeline script + Keyboard trigger.
Visual Output (TOP): Generated image display + Poem text display.

AI Collaboration Process

Not a single line of code was handwritten in this project. All code was generated through conversations with Claude Code.

Step 1: Describe the Vision

I described the project goals using natural language. Claude Code asked clarifying questions (API Keys, local environment, TD version) and then generated a complete technical plan.

Step 2: Approve the Plan

Claude Code entered "planning mode" and output a complete plan including architecture diagrams, file structures, node configurations, and API details. I reviewed and adjusted it.

Step 3: Step-by-Step Implementation

Claude Code sequentially generated the core scripts and step-by-step guidance for setting up the TouchDesigner nodes. I just followed the instructions to create nodes and paste code.

Challenges & Iterations

Here are three examples of human-machine collaboration solving problems during development:

Problem 1: API Failure

The original plan used Stability AI, but the API kept returning a 403 error. Claude Code suggested switching to DALL-E 3 using my existing OpenAI key, and rewrote the code in minutes.

Problem 2: Thread Conflict

Images generated but TouchDesigner threw a "THREAD CONFLICT" error. Claude Code decisively removed the multithreading design and switched to synchronous execution.

Problem 3: UX Optimization

The initial "hold-to-record" design made it hard to control timing. Claude Code changed the interaction to a toggle mode ("press once to start, press again to stop").

Final Results

The system successfully generated works based on classic poems.

Read My Original Tutorial