What are the best methods to fit SceneObject to screen in LS?

Lens Studio
1

Hey everyone,

Question regarding fitting an object to the screen.

I have a rectangular cube with ratio equal to average screen ratio (9:16), the goal is to write a func which would fit it to the screen. The cube will use screen texture as the texture, so it should fit the screen 1:1. What are the best pmethods and what resources to check for proper implementation?

Key issues IMO:

  • ratio differs depending on the device used and also on if the front or back camera is used;
  • I suppose it should also use the estimated depth of an object relative to the camera (?).

Would appreciate any advice.

2

I haven’t tested this at all but I think I came up with the beginning of a solution. This script should at least give you an idea of what needs to happen to scale the object. It uses the aabb min/max to determine the bounding box of the mesh visual, then uses some camera space conversion to determine how much the object needs to be scaled to fit in the viewport. It’s definitely not bulletproof yet, but I left some notes.


//@input Component.RenderMeshVisual obj
/** @type {RenderMeshVisual} */
var obj = script.obj

//@input Component.Camera camera
/** @type {Camera} */
var camera = script.camera

print('================================')

// we will leave the object in place (assuming it's in the center of the camera view) and scale it up

// determine size of the object on the screen
// var objMinScreen = camera.worldSpaceToScreenSpace(obj.worldAabbMin())
// var objMaxScreen = camera.worldSpaceToScreenSpace(obj.worldAabbMax())
// var objScreenSize = objMinScreen.add(objMaxScreen).mult(new vec2(.5, .5))

// get the distance from the camera to the object
// using center point of object bounding box might be inexact for determining what we need...
// but you could probably get a min from distance between aabb min and max instead
var camPosition = camera.getTransform().getWorldPosition()
var objBoundingBoxSize = obj.worldAabbMax().sub(obj.worldAabbMin())
var objBoundingBoxCenter = obj.worldAabbMax().add(obj.worldAabbMin()).mult(new vec3(.5, .5, .5))
var objDistFromCamera = camPosition.distance(objBoundingBoxCenter)

print(['objBoundingBoxCenter', objBoundingBoxCenter])
print(['objBoundingBoxSize', objBoundingBoxSize])
print(['objDistFromCamera', objDistFromCamera])

// determine the frustum box we need to fit the object into
var frustumTopLeft = camera.screenSpaceToWorldSpace(new vec2(0, 0), objDistFromCamera)
var frustumBottomRight = camera.screenSpaceToWorldSpace(new vec2(1, 1), objDistFromCamera)
var frustumSize = frustumTopLeft.sub(frustumBottomRight).mult(new vec3(-1, 1, 1))
var frustumSizeVec2 = new vec2(frustumSize.x, frustumSize.y)

print(['frustumTopLeft', frustumTopLeft])
print(['frustumBottomRight', frustumBottomRight])
print(['frustumSize', frustumSize])

var startScale = obj.getTransform().getWorldScale()
var scaleBy = frustumSize.div(objBoundingBoxSize).x
obj.getTransform().setWorldScale(startScale.mult(new vec3(scaleBy, scaleBy, scaleBy)))

print(['scaleBy', scaleBy])

snap-obj-fit2screen.zip (4.9 MB)

1 Like
3

Thanks a lot!

I’ll check it out and share what I’ll come up with later in this thread.

1 Like