Vertex displace shader (grayscale depthmap) for 3D-ish AR

Spark AR
#1

Hi all.
New here, and came after seeing a youtube video by Josh, of using a perlin noise texture to displace a texture.
I’m not coding savvy, and would like to ask for help in the doing this, if it’s possible

  • Creating a Spark AR shader and effect that uses the bottom half of a grayscale depth image to displace the vertexes of a plane mesh which would be textured by the half of the image.

The idea would be to create a 3Dish mesh/model using a “depth map” - sort of how Facebook does their 3d images, but this would be like a 3D model when viewed in Spark AR and placed on a detected plane.

20210102_104516.00_02_19_18.Still001
20210102_104516.00_02_19_18.Still0011920×1080 76.8 KB

Hoping it could be done,
I’m 1 day into SparkAR so lots to learn.

Best Regards.

#2

In most 3D software, you can use depth maps to displace vertices. For some reason, spark lacks the capability. All of the texture operations are forced into the fragment stage, so the pixel data can’t be used to displace vertices.

One way around this would be to paint vertex colors onto the mesh you want to use, and use those to displace the vertices (but I’m not sure what the point would be if you already have a 3d mesh).

Another option would be to convert the grayscale texture into a normal texture, and plug it into a PBR material. You’ll get a slight relief effect that way, but it’s generally very shallow.

I love Spark, but it definitely comes with some limitations…

2 Likes
#3

Thanks so much for explaining this, Josh.
Indeed, it’s a bit frustrating now that I understand the limitation. Likewise, the removal of external URL for texture streaming.

Would this vertex displacement idea work in Snap Lens Studio?

#4

I’m less familiar with LS, but in general it’s a bit more flexible than spark. My guess is that it would be possible, but I think it doesn’t have external texture streams either.

@finofinissimo has done some pretty cool volumetric 3d using depth maps in Spark, so maybe that could be a solution. The idea is to stack a bunch of planes and make a shader that applies alpha based on plane position and texture color.

1 Like