user5839 user5839 - 3 months ago 29
Javascript Question

Three.js part of video as texture

I'm trying to use part of a video as a texture in a Three.js mesh.

Video is here, http://video-processing.s3.amazonaws.com/example.MP4 it's a fisheye lens and I want to only use the part with actual content, i.e. the circle in the middle.

I want to somehow mask, crop or position and stretch the video on the mesh so that only this part shows and the black part is ignored.

Video code

var video = document.createElement( 'video' );
video.loop = true;
video.crossOrigin = 'anonymous';
video.preload = 'auto';
video.src = "http://video-processing.s3.amazonaws.com/example.MP4";
video.play();

var texture = new THREE.VideoTexture( video );
texture.minFilter = THREE.NearestFilter;
texture.magFilter = THREE.LinearFilter;
texture.format = THREE.RGBFormat;

var material = new THREE.MeshBasicMaterial( { map : texture } );


The video is then projected onto a 220 degree sphere, to give the VR impression.

var geometry = new THREE.SphereGeometry( 200,100,100, 0, 220 * Math.PI / 180, 0, Math.PI);


Here is a code pen
http://codepen.io/bknill/pen/vXBWGv

Can anyone let me know how I'm best to do this?

Answer

In short, you need to update the UV-Map of the sphere so that the relevant area of your texture is assigned to the corresponding vertices of the sphere.

The UV-coordinates for each vertex define the coordinates within the texture that is assigned to that vertex (in a range [0..1], so coordinates (0, 0) are the top left corner and (1,1) the bottom right corner of your video). This example should give you an Idea what this is about.

Those UV-coordinates are stored in your geometry as geometry.faceVertexUvs[0] such that every vertex of every face has a THREE.Vector2 value for the UV-coordinate. This is a two-dimensional array, the first index is the face-index and the second one the vertex-index for the face (see example).

As for generating the UV-map there are at least two ways to do this. The probably easier way (ymmv, but I'd always go this route) would be to create the UV-map using 3D-editing software like blender and export the resulting object using the three.js exporter-plugin.

The other way is to compute the values by hand. I would suggest you first try to simply use an orthographic projection of the sphere. So basically, if you have a unit-sphere at the origin, simply drop the z-coordinate of the vertices and use u = x/2 + 0.5 and v = y/2 + 0.5 as UV-coordinates.

In JS that would be something like this:

// create the geometry (note that for simplicity, we're 
//   a) using a unit-sphere and 
//   b) use an exact half-sphere)
const geometry = new THREE.SphereGeometry(1, 18, 18, Math.PI, Math.PI)
const uvs = geometry.faceVertexUvs[0];
const vertices = geometry.vertices;

// compute the UV from the vertices of the sphere. You will probably need 
// something a bit more elaborate than this for the 220degree FOV, also maybe
// some lens-distorion, but it will boild down to something like this:
for(let i = 0; i<geometry.faces.length; i++) {
  const face = geometry.faces[i];
  const faceVertices = [vertices[face.a], vertices[face.b], vertices[face.c]];

  for(let j = 0; j<3; j++) {
    const vertex = faceVertices[j];
    uvs[i][j].set(vertex.x/2 + 0.5, vertex.y/2 + 0.5);
  }
}
geometry.uvsNeedUpdate = true;

(if you need more information in either direction, drop a comment and i will elaborate)