OpenAI’s latest model allows users to create realistic 3D models from text inputs.
San Francisco-based AI powerhouse OpenAI appears to be on the rise. After launching the revolutionary ChatGPT and its follow-up version with advanced features, the company now appears to be preparing to unveil its latest innovation.
Since it is widely known that GPT-4 can generate answers from natural language prompts in the fastest and most efficient way, OpenAI seems to have taken a leap forward and introduced a text-to-3D model. Known as Shap-E, this model can transform text prompts into three-dimensional objects.
The latest generative AI tools are reported to provide far more accurate answers compared to a similar tool, Point E, launched by OpenAI in 2022. This is seen as another breakthrough by OpenAI following the success of Dall-E, a text-to-image generation tool.
What is Shape E?
OpenAI’s latest tools are designed to generate fine-grained textures and complex shapes from text input. Shap-E, which can create object contours, works with a feature known as Neural Radiance Fields (NeRF) to provide accurate answers. Interestingly, NeRF is a technology used in virtual and augmented reality that transforms three-dimensional images into photorealistic environments. Shap-E is reported to apply NeRF to a general diffusion model to create object shapes and textures from text prompts. Compared to point E, the process for shape E is significantly faster.
Shap-E is now freely available on GitHub and runs locally on your PC. Tomshardware.com reports that setting up and running the tool is a huge undertaking. Like existing models, the Shap-E offers a lot of potential, but its performance is currently passable. According to reports, most of the release was in low resolution and missing important details. Enter text and the output will be a color animated GIF or monochrome PLY file that can later be opened in a program like Paint 3D.
Features and use cases
The company reportedly demonstrates Shap-E’s potential with examples such as a bowl of food, a penguin, and a campfire. These 3D models are generated in seconds, allowing the user to create his 3D assets faster. In terms of applications, Shap-E is beneficial for movies, virtual reality experiences, movies, technical simulations and most importantly scientific research. A more accurate example of Shap-E’s 3D output is awaited, but the tool is touted as revolutionary in the field of 3D modeling and rendering.