Store a sampled colour from camera texture?

Is it possible to sample and then store / fix a color to use in a material?

In other words, can you sample skin colour once and use that color as the material for (say) elf ears?

I think it can be done ‘live’ but I was hoping to avoid colour changes as the user moves around.

I’ve seen a stack overflow question where the answer appears to suggest that this is not possible (‘The texture data is stored on the GPU, and there is no way in Spark to read this data from the GPU to the CPU for scripting purposes’ - from Spark AR Studio: Is it possible to get the color of a screen pixel from script? - Stack Overflow).

Thanks for any help.

1 Like

You could try to sample a point or multiple points from the face based on their texture coordinate/uv and do some math to make a formula to try to balance or get consistent result first. and then I think you could use freeze frame technique to capture that exact color at that exact point in time apply it other materials as base colors.
The patch you’ll need to sample the point would be texture sampler or composition patch, where the top slot is where you plug your texture, and bottom is for your uv coordinate points, x and y from 0 to 1 range. And also shader render pass, delay frame and mix patch to do the freeze frame thing.

My advice would be:

  1. Try to choose a point on the face that won’t be covered by common stuffs like glasses, forehead hair, mustache or beard, mask.
  2. When you sample left side, try to make a counter sampler on the right side to even out the shades from highlight and shadows created by the real life light.
  3. Since you just need a base color, you don’t have a high res area to sample, so try to pass it to render pass first and set the scale to low values, if you want you cant also try to blur it afterward and down sampling it again . before you finally freeze the frame to sample it.

That’s probably how I would deal with it when I need to sample skin color for my project.
Let me know how it works out for you :slight_smile:
Good luck!

Hello and many thanks for your reply. I’m working on trying this out but - just to change the subject - I am having a problem freezing the frame, which I think may be a bug. I can get a blurry freeze frame effect to work at 0.9975 alpha but not at 1.00 alpha. I’ve submitted a bug report. I am on v 124 on an iMac.

In any case thank you for the excellent suggestions.

Yup me too. I’ve reported this long time ago since the update 123 if i remember it correctly where they introduce some changes in patch editor behavior.

In my case, the freeze frame is working but if something is updated in spark, like changing values of even something that has nothing to do with the freeze frame, the frame is still getting updated, and then refreeze it again. like this:

and if the first frame is not using the direct shader renderpass from the camtex (camera texture), but using the shader renderpass from the mix just like the input below it, it will turn to black or whatever the background color we set. super annoying.

*Note, but it only updated if there is a state or value changes made by user input, like typing, plug/unplug, copy paste, etc. if the value changes made by lets say runtime or loop animation or animation, then it behaves just like in previous versions.