Looking for feedback on biteScript including collision with face-features

monogon · November 6, 2020, 10:23am

I expanded a little on the bounding box, and while I’ve moved on to learning shaders for the moment I’d really appreaciate some feedback and discussion on some of my methods. I think what someone else could potentially profit from is looking at the projectFacePointsToFocalPlane() thing, I feel like figuring that out helped me the most (and probably will continue to do so in the future). As for the utility of using this for actual collision with the mouth - I saw @josh_beckwith saying he’d just use the center point and compare the distance. That makes a lot of sense to me and is probably the better solution, but I learned a lot from writing this and the projected points of the mouth features moving on the focal plane looks kinda neat.

github.com

johannesrave/spar-ar-misc/blob/master/biteScript.js

// @ts-nocheck
const D = require('Diagnostics');
const S = require('Scene');
const R = require('Reactive');
const F = require('FaceTracking');
const P = require('Patches');

let counter = 0;
const tolerance = 0.02;

(async function main() {
    D.log("Script initialized");

    /** @type { SceneObjectBase[] } */
    const targets = getTargetsFromScene();

    /** @type { SceneObjectBase[] } */
    const lipMarkers = getLipMarkersFromScene();

    /** @type { PointSignal[] } */

This file has been truncated. show original

Edit: I should also mention I started out with that bite-tutorial on Medium, https://medium.com/@Julien_He/game-in-spark-ar-part-1-1ddefdfa9838, so thanks for that and credit to Julien. The approach has changed a little over the course of the project however, so don’t expect this to work exactly the same or even similarly.

josh_beckwith · November 7, 2020, 5:43pm

Wow, overall the script is super organized! Very cool typescript hints too. I learned something from this!

In terms of UX, I wonder if it’s clear that the user needs to bite the targets as opposed to catching with an open mouth. Most of the effects I’ve seen like this use an open mouth to catch. I suppose this could be demonstrated in the demo video, but I’m mostly wondering about if the action is intuitive.

One thing that I kind of struggled with in patches was disabling the eaten targets until they were reset. I could hide them or toggle visibility, but the targets were still active in the hit-tester. Since you are using script, you could remove them from the active targets array and put them somewhere else (or just throw them away). This would prevent the user from being able to bite the same object several times, although maybe it doesn’t matter in your game.

monogon · November 7, 2020, 11:00pm

Heyhey,
thank you for the detailed feedback and even rebuilding the scene, very cool.
I agree with your comments regarding script/patches and UX, and there were specific reasons for doing it this way. The bite action was part of the brief. The connection towards the patches was borne from the fact that someone else was doing something else with patches in the project and we agreed on this being the splitting line between our domains. If it had just been my thing I would’ve coded it all (the bite function is easily expandable), but it went okay the way it did.
Thanks again! … back to shaders :’( Trying to understand screen space refraction using the vertex attributes… it’s tough, man