ViewPort recognize what Object looks at

Spark AR
, ,
#1

Hi LAB friends,

Wanted to ask if you anyone know how to recognize what view port is looking at. Meaning if I use Device for rotating in a 360 sphere, can I recognize what angle is it looking at? Or if an object is being seen by the view port?

Thanks!

1 Like
#2

I don’t know if I understand the question, @josh_beckwith @Boris any idea?

#3

You can unpack Device Motion

1 Like
#4

I really had to dig in the FB group for this one. Here’s a script from Jamie Poole from 2018! I haven’t tested it, but maybe it will work for you. You’ll need to use the new syntax for finding scene objects, but other than that I think it will do the trick.

const Scene = require('Scene');
const Diagnostics = require('Diagnostics');
class WorldToPixels {
    constructor() {
        this.Scene = require('Scene');
        this.Patches = require('Patches');
        this.camera = this.Scene.root.child('Device').child('Camera');
        this.screenSize = this.Patches.getPoint2DValue('screenSize');
        this.widthTan = this.camera.focalPlane.width.div(2).div(this.camera.focalPlane.distance);
        this.heightTan = this.camera.focalPlane.height.div(2).div(this.camera.focalPlane.distance);
    }
    convert(objectToTrack) {
        let focalPlaneWidthAtObject = objectToTrack.worldTransform.z.mul(this.widthTan).mul(2);
        let focalPlaneHeightAtObject = objectToTrack.worldTransform.z.mul(this.heightTan).mul(2);
        let pixelX = this.screenSize.x.div(focalPlaneWidthAtObject).mul(objectToTrack.worldTransform.x).mul(-1).add(this.screenSize.x.div(2));
        let pixelY = this.screenSize.y.div(focalPlaneHeightAtObject).mul(objectToTrack.worldTransform.y).mul(-1).add(this.screenSize.y.div(2));
        return {
            x: pixelX,
            y: pixelY
        };
    }
}
let wp = new WorldToPixels();
let FaceMeshPixelPositions = wp.convert(Scene.root.find('FaceMesh'));
Diagnostics.watch('FaceMesh Pixel X', FaceMeshPixelPositions.x);
Diagnostics.watch('FaceMesh Pixel Y', FaceMeshPixelPositions.y);
2 Likes
#5

Wow thanks @josh_beckwith Ill try it!
Thanks!

Sorry for not being so clear on my question!

#6

@josh_beckwith also remember that the coordinate system changed! I’m not entirely sure about what the script tries to achieve as I haven’t looked too much into this topic so I can’t modify it (without ruining it). Just keep that in mind :wink:

1 Like