Shooting VR180 Content with the Canon EOS R7

Canon has been creating a number of dual fisheye lens upgrades for their existing line of cameras, specifically their mirrorless cameras, but including the C400 which also has an RF lens mount.  These dual lens attachments project both sides of the captured stereoscopic image onto the large single sensor, in a side by side configuration.  The resulting recordings can then be processed via a variety of methods, and viewed in a VR headset for an immersive experience.  180 degree VR only offers half of the potential view of 360 degree video, but is far more practical, because you only have to fill and record half of the potential perspective, and 360 video viewers rarely look backwards anyway.  (One of the biggest challenges of directing 360 content is directing the viewer to where you want them to look.)  Shooting VR180 instead means you don’t have to have a clean backside, allowing crew and other infrastructure to remain just out of sight behind the camera.  This includes the camera operator, who must be remote if they don’t want to be in the shot of a 360 recording.  But with VR180, you just hold the camera up to your eye and snap a picture or record a video of everything in front of you.  And with the advent of these dual lenses, the resulting records are stereoscopic, for even more immersive viewing than a standard 360 sphere recording.

Canon has multiple options for VR180 image capture, starting with the 5.2mm dual fisheye lens for full frame RF cameras like the EOS R5, R6 and C400.  This provides a greater than 180 degree field of view, similar to each side of a 360 degree image capture.  This is a high end option, which costs a total of about $5K to get started.  More recently they released the 3.9mm dual fisheye lens for APS-C sized crop sensor cameras, specifically the EOS R7, and the full solution is half that price, around $2500 for the camera and lens.  This lens offers a 144 degree field of view, which is less than 180 degrees, but more than enough to cover the view available in a VR headset.  With most VR headsets limited to at maximum a 110 degree field of view, this provides 10-15 degrees of extra content all the way around.  This is less than the 40 degrees available around the edges of a true 180 degree recording, but enough to fill the headset FOV with image to spare.  This new lens was announced in conjunction with Apple, so I would expect to see more tie-in with the Vision Pro headset as the products mature.

Canon was generous enough to send me an EOS R7 camera and dual fisheye lens for a few weeks, to test out the entire VR180 workflow.  My existing camera is an EOS 70D, about ten years old, so I haven’t used a new Canon camera in quite a while.  When the dual fisheye lens is attached, the camera is automatically ready to record stereoscopic VR content.  The lens projects the left and right views onto the sensor side by side, and the resulting recording has the right eye on the left and the left eye on the right. This is due to the natural image inversion from the lens, which is always corrected in the camera.  Each lens flips its part of the image, and then the camera flips the entire image as a normal part of the recording process.  The camera can record stills at up to 6/.9K or UHD video to HEVC files, at 60, 30, or 24 fps, to an SD card in one of two slots.  I recommend 60fps for VR recording, for smoothest viewing results.

The lens allows 30.5mm filters to be attached to the rear, since you clearly can’t put them over the outer bulging fisheye lens elements.  So you could add ND filters that way, or any other filter, although I don’t think most creative filters would be applicable to VR shooting.  Pretty much all Canon EOS cameras support remote control and shooting over WiFi, from a computer via the EOS Utility, or smart phone via the Camera Connect app.  That software is especially applicable when shooting VR180 content, because Canon has added support for live equirectangular conversion.  This can be helpful in confirming that you are getting the shot you want, instead of just seeing the two fisheye projections on the camera displays.  Remote or not, the live magnify feature is super important for verifying focus, since the recorded image is so small on the display (due to the ultra wide angle).  One software change that does kick in specific to having the VR lens attached, is that instead of magnifying the center of the sensor, which would be the gap between the two image circles, it magnifies the center of the left recording, and pressing Info swaps to the other eye.  This allows you to confirm focus, and easily compare the perspectives between the two eyes.  The lens also allows the user to focus the left lens separate from the right one, but that should only be necessary if something falls out of alignment. (Normally you will always want them focused exactly the same.)

Whenever you are recording a non-standard format, whether it is a new codec, or a new lens perspective like this, processing the image and outputting it to an audience can be a bigger challenge than usual.  In this case it involves VR viewing, which requires equirectangular conversion for editing and playback.  Canon has written their own software tools for this, which I have been quite impressed by.  Their EOS VR Utility can convert the fisheye recordings from the camera, both still and video, to equirectangular formats optimized for editing and processing, or for direct upload to the web.  They also created a simple EOS VR Plugin for Premiere Pro, to allow the footage to be used seamlessly in VR sequences, with no processing or transcoding required.

Once everything was installed, it worked together quite smoothly, with the Canon tools operating in the background, with no other user intervention required.  The Canon VR plugin for Premiere converts the footage from two reversed fisheye views filling a 3840×2160 (standard UHD) H.265 file, into an equirectangular 3840×1920 image, matted to 144 degrees, with the eyes swapped back.  My understanding is that it does this on the fly, based on GPU acceleration.  And Premiere can easily export MP4 files that play as VR videos in Windows or on Youtube.  I will explore the specific details of setting up and using that software, as well as editing VR180 content in Premiere Pro, in my next article, coming soon.

Leave a Reply

Your email address will not be published. Required fields are marked *