Nestwork (Web App)
Web App for Personalized Residential Design via LLMs and Graph Generative Models
LLMs, Multi-modal Interface, Web App
Note: This project builds on the deep learning research nestwork by integrating it into a web-based interactive tool. It focuses on user interaction, multi-modal inputs (text + graph UI) and real-time generative feedback. The labor is roughly divided as : Shuhan Miao works on backend development using Flask, model inference implementation, LLM prompt engineering and deployment (Github Actions, Docker, AWS Elastic Beanstalk). Junling Zhuang works on front-end development using Next.js and Three.js, UI/UX design, conducting user interviews, and overseeing product design.
This project develops an interactive web-based design assistant that allows users to create personalized residential layouts by describing their living preferences in natural language. By integrating Large Language Models (LLMs) with graph-based generative models, the app translates user inputs into structured room adjacency graphs, which are then transformed into 3D furnished layouts. Inspired by traditional architectural workflows—where designers translate textual briefs into spatial diagrams—this system enables rapid design iteration and democratizes architectural design, making it accessible to a wider audience. The web app provides an intuitive interface for users to input lifestyle descriptions, explore AI-generated layouts, and iteratively refine their designs, merging computational power with human-centered design.
Workflow of the Web App
Technical Details
Backend: The backend is built with Flask, handling API requests and managing communication between LLM-based text processing and graph-based generative models.
Frontend: It is developed using Next.js for dynamic UI rendering and user interactions. It uses Three.js to visualize 3D layouts in real-time, enabling interactive exploration of generated designs.
Deployment & CI/CD: It uses Github Actions to automates CI/CD pipeline with Docker to containerize both frontend and backend environments. AWS Elastic Beanstalk (EB) is used to host the web app.
Learn more about the deep learning research behind this project: Nestwork - Conditional 3D Furnished House Layout Generation with Heterogeneous Graph Variational Autoencoders