Frame Cache from Sugar shop

Spark AR Questions
, ,
#1

I purchased the Frame Cache project/patches and there is something specific I’m hoping to do with it and I’m trying to figure the best way to go about it.

The sample project shows a setup where is a grid of 16 (offset in time and constantly updating) frames and setting the Index number on the FrameGrab patch lets you specify which of those (constantly updating) frames to “grab” - so let’s start with that as an example…

Basically I’d like to achieve a few things.

  • I would like to be able to send a pulse and have it use the current camera texture image at that time as frame 1 (no longer updating that frame but rather as a still “frozen” frame)
  • then another pulse sets frame 2 the same way (using the frozen still camera image from the time of that pulse)
  • and so forth for each frame in the grid (say 16 as in the demo project)
  • and I would want to have easy access to each of these 16 frozen still images within the patch editor (to do whatever the hell I want with them : )

optional ;

  • once all frames had been set it would be good if another pulse sent to that input could set frame 1 again using the frozen cam texture from the time of the 17th pulse (looping back through the 16 “slots”)
  • if it was easy to achieve I would prefer for a frame which had not yet been “set” to simply display the straight camera texture (so the pulse to set the frame would appear as the moving image “freezing” on the pulse)
  • maybe offer an input where a pulse could be sent to “clear” the set frames (so all 16 would show straight camera texture)

I’m going to start to look into this but would appreciate any suggestions etc.

Thanks.

#2

Instead of using the frame number counter from script, you can use your own tap (or other pulse) counter in patches, like this:

Screen Shot 2020-12-30 at 9.56.43 AM
Screen Shot 2020-12-30 at 9.56.43 AM1594×672 108 KB

There’s no built in way to use a fallback texture in case one hasn’t been stored yet, but you can use a bunch of if/else patches in conjunction with “less than” to check if the snapshot is ready or not.

To get each of those individual textures, you’d use the frame grab patch with a constant number as the index.

You don’t need to worry about resetting the counter since it automatically wraps back to the start (e.g. index 16 is actually index 0). It uses a modulo to keep things within the bounds. You could do something similar with your counter if you want to reset the texture fallback logic when the index goes back to zero… or you could just rely on the counter’s built in wrapping by setting maximum count to 16, but that value isn’t pipe-able so you’d need modulo if you are dynamically changing the number of snapshots.

As for clearing the frames, I think you can ignore the actual texture. Just update the current index and rely on your if/else statements to determine if they should show the camera feed. This way you can just send a pulse to the counter’s “jump to” to reset the whole experience.

1 Like
#3

Thanks for the thorough response. This has got me going and it looks like I’ll be able to use it in the way(s) I’d hoped.

#4

Here’s a separate question (but about the same Frame Cache project/patches)…

I find that in the simulator I can preview it at different sizes and aspect ratios and it looks correct, and when I export the effect to test in IG it looks correct while previewing BUT it stretches vertically as soon as I start to record video (and indeed the video records stretched) - this is true even just exporting the original frame cache project and whether you are outputting the Frame Cache grid or Frame Grab image - taking a photo does NOT display this stretched image but recording a video does… so is that likely an Instagram bug ? or is there a known issue that would explain it ?

UPDATE ; I’m seeing this on an Android phone but I did not see the same problem when I checked it on an iPhone just now… so maybe it is indeed an IG bug…

1 Like
#5

It certainly sounds like a bug. I know there’s a feature on iOS that will appear to zoom in the camera when you start recording, but it certainly shouldn’t stretch anything.

I just opened and tested (on my Android, I just get a black screen) the frame cache project and it looks like it’s using RGB for the render pass and delay frame. I was told directly by the spark team that there’s a bug in IG that will crash the effect if anything other than RGBA is used. I switched to use RGBA and now it’s working as expected. I can’t say if that will fix your stretching issue, but it’s worth a shot

1 Like