Software & Tool
Jan 2025 - Present
Sculpt3D
Multimodal Text & Sketch-to-3D Point Cloud Interactive Generation
Organization: IIT Mandi
Key Result & Impact Benchmark
Demonstrated precise region-level 3D point cloud spatial editing guided by natural language prompts.
Technical Overview
Sculpt3D is an advanced generative 3D spatial AI project that enables users to synthesize and edit 3D point cloud objects from multimodal inputs (textual descriptions, 2D sketches, and reference images). Building upon Signed Distance Function fusion (SDFusion), Sculpt3D introduces continuous natural language feedback loops powered by LLMs (LLaMA & GPT-4o). Users can iteratively prompt the model to modify localized parts of an existing 3D point cloud (e.g. 'elongate the chair legs' or 'smooth the surface') without regenerating the object from scratch.
Key Technical Highlights
- Mentored B.Tech thesis students at IIT Mandi on 3D generative diffusion and spatial point cloud manipulation.
- Introduced continuous feedback prompt editing allowing interactive localized 3D point cloud refinement.
- Integrated Open3D, PyTorch3D, and Fury.gl for real-time browser-based 3D visualization.
Technologies & Frameworks
SDFusion
LLaMA / GPT
Open3D
PyTorch3D
Plotly
Fury.gl
Python