Using AI to create 3D models

Resources
,
1

An exciting usage of generative AI would be to generate 3D models but at this point there seem to be no reliable tool to generate 3D models by prompts just yet.

This being said, CSM dropped a feature that gets us closer to it as you could now use AI to convert a 2D image to a 3D model which is easy to combine with images generated by AI.

Feature announcement: https://www.csm.ai/any-image-to-3d
Example of 3D models that have been created using CSM: CSM 3D Viewer

All in all, using generative AI to generate an image, to then use CSM to convert that image in 3D will fasten the process of generating 3D models in many cases.

août-08-2023 11-14-36

And that process is documented below just for you lil wizards:

  1. Use MidJourney to create an image (png)
  2. Use Apple Photos to remove the background (png) or you can just go to Remove.bg if you don’t have an apple device
  3. Use CSM to convert the image to 3D (glb)
  4. Use Blender to fix any issues
  5. Use Blender to export with texture (obj)
  6. Use Mixamo to auto-rig and animate the 3D object (fbx)

And there you go, you can now use you model in AR, VR and various 3D environments using Unity and Unreal

This process has been shared by Ben X Tan on his linkedin together with a Google slides documenting his pipeline

Let him know if you used his process, there’s no shame in the crediting game, all but the contrary!

Cheers,

Laz

3 Likes