Looking Glass Documentation
Buy Looking GlassDiscord
  • Looking Glass User Guide
  • NEW! โœจLooking Glass Goโœจ
  • Displays
    • ๐Ÿ”ฎLooking Glass Go
      • โšกGet Started with Looking Glass Go
      • ๐Ÿ“ฑUsing the Go Mobile App
      • ๐Ÿ”†Liteformsโ„ข
      • ๐Ÿ–ฅ๏ธStandalone and Desktop Modes
      • ๐Ÿ”ŒPower and LED States
      • ๐Ÿช„Default Playlist
      • โญ๏ธWhat's Next?
      • ๐Ÿ‘œAccessories
      • โš™๏ธAdvanced Playlist Creation
      • โš™๏ธManual Firmware Update
      • โ“Common Troubleshooting Issues
    • ๐Ÿ”ฎLooking Glass 16" Light Field Displays
      • ๐Ÿช„Looking Glass 16" Light Field Display (Landscape)
      • ๐Ÿช„Looking Glass 16" Light Field Display (Portrait)
      • โ“Common Troubleshooting Issues
    • ๐Ÿ”ฎLooking Glass 32" Light Field Displays
      • ๐Ÿช„Looking Glass 32" Light Field Display (Landscape)
      • ๐Ÿช„Looking Glass 32" Light Field Display (Portrait)
      • โ“Common Troubleshooting Issues
    • ๐Ÿ”ฎLooking Glass 65" Light Field Display
    • ๐Ÿ”ฎLooking Glass Portrait
      • โšกGet Started with Looking Glass Portrait
      • ๐ŸŽ‡Demo Holograms
      • โ“Common Troubleshooting Issues
  • Software Tools
    • ๐Ÿ“šOverview
    • ๐ŸŒ‰Looking Glass Bridge
      • โœจUsing Looking Glass Bridge
      • ๐ŸชŸDisplay Settings on Windows
      • ๐ŸŽDisplay Settings on macOS
      • ๐ŸงDisplay Settings on Linux
      • ๐Ÿ‘ทTroubleshooting
    • ๐Ÿ“ฝ๏ธLooking Glass Studio
      • ๐Ÿ“šUsing Looking Glass Studio
      • RGB-D Photo / Video
      • Quilt Images / Video
      • iPhone Portrait Photos
      • Android Depth Photos
        • Samsung Portrait Mode
      • Light Field Photosets
      • Exporting Holograms
      • Troubleshooting
    • ๐ŸงŠLooking Glass Blocks
      • โœจConvert any image into a hologram
      • โฌ†๏ธUploading, editing and sharing Quilts
      • ๐ŸฅฝViewing your holograms on different platforms
    • ๐ŸŽฌHologram Video
    • ๐Ÿ”†Liteformsโ„ข
      • ๐ŸชŸGet Started on Windows
      • ๐ŸŽGet Started on MacOS
      • ๐Ÿง™Making Custom Liteforms
      • ๐ŸชConfigure Liteforms for Kiosk Setup
    • ๐Ÿ‘€Looking Glass Model Viewer
    • ๐ŸงชExperiments
      • ๐Ÿ“ฝ๏ธMedia Encoder
      • โฉLooking Glass Media Encoder
  • Key Concepts
    • ๐Ÿ”‘Hologram 101
      • ๐ŸŽ‰Making great holograms
      • ๐ŸŒŽHolograms in the real world
      • ๐Ÿ‘พHolograms, but virtual
    • ๐Ÿ”How does Looking Glass Work?
    • ๐Ÿ“ธVirtual Camera
    • ๐ŸงŠ3D Design Guidelines
      • Lighting
      • Post-Processing
      • Scene Composition
      • Materials & Textures
      • Performance
    • ๐Ÿ’กQuilts
      • ๐ŸŽฌQuilt Video Encoding
    • ๐Ÿ’กLight Fields
      • โšกLinear Light Field Capture
    • ๐Ÿ“ฝ๏ธFilming a Looking Glass
  • Creator Tools
    • ๐ŸŽจOverview
    • ๐Ÿ”ŒUnity
      • โšกUsing Unity with Looking Glass
      • Setting up your Development Environment
      • Example Scenes
      • Prefabs
        • Hologram Camera
        • 3D Cursor
        • Dual Monitor Application
      • Script Reference
        • HologramCamera.cs
        • LKGDisplaySystem.cs
        • QuiltCapture.cs
        • Cursor3D.cs
        • OrbitControls.cs
        • BlockUploader.cs
        • MobileDMAController.cs
        • iOS Sample Scene Scripts
          • ChangeInteractionModality.cs
          • XYSlider.cs
          • XYSpotlight.cs
          • AudioRotateControl.cs
          • ModelController.cs
          • DemoIOSUIController.cs
          • PageHandler.cs
          • PageConnectHandler.cs
          • PageCalibrateHandler.cs
          • PageTestHandler.cs
          • DynamicAlcove.cs
      • Package Integrations
        • Built-in Render Pipeline Post-Processing
      • Developing for iOS
      • Depth of Field and Other Post-Processing Effects
      • Quilt Video Audio and Playback
      • Developing for Ultraleap Hand Tracking on Looking Glass
      • Looking Glass Pro (Legacy)
      • Additional Support
      • Change Log
    • ๐ŸนBlender
    • ๐Ÿ”ŒUnreal Engine
      • ๐Ÿ—’๏ธChange Log and Old Versions
      • ๐Ÿ’ฝInstalling the Unreal Engine plugin
      • ๐ŸŽฅUsing the Unreal Engine Plugin
        • Using the HoloPlay Capture
        • Exporting Quilt Images and Video with Sequencer
      • Accessing HoloPlay Data with Blueprint Scripting
      • Packaging the Project
      • Unreal Engine 4 (Legacy Support)
        • Unreal Engine 4 Quick Start
          • Using the HoloPlay Capture
          • Exporting Quilt Images and Video with Sequencer
    • ๐ŸŒWebXR
      • ๐Ÿ”ถPlayCanvas
      • 3๏ธโƒฃ3๏ธโƒฃ Three.JS
      • โš›๏ธReact-Three-Fiber
      • ๐ŸŒˆSpline
  • Tutorials
    • ๐Ÿ“ฑRunning Looking Glass Displays with iOS Devices
    • โ“‚๏ธCapturing Light Fields in Maya
    • ๐Ÿง‘โ€๐Ÿ”ฌUsing ShaderToy with Looking Glass
    • ๐ŸŽจSculpting with Adobe Medium and Looking Glass
    • ๐Ÿ”Turn Videogames Holographic with Refract
    • ๐Ÿ–๏ธBuilding with Leap Motion
  • Developer Docs
    • โš™๏ธLooking Glass Bridge SDK
      • ๐Ÿช›Web Application Integration
      • ๐Ÿ”ŒIntegrating Native Applications
      • ๐Ÿ““Native Function Reference
    • ๐ŸฆLooking Glass Core Migration Guide
  • Third Party Software
    • ๐Ÿ—ก๏ธ3D Slicer
    • ๐Ÿ”ฌVTK (Visualization Toolkit)
    • ๐Ÿ—บ๏ธParaView
    • ๐Ÿ”ฉCommunity Made Tools & Projects
  • Legacy Software
    • ๐Ÿ“ก3D Model Importer
      • 3D Model Importer App Guide
      • 3D Model Importer App Reference
    • ๐ŸŽญDiorama
      • Change Log
    • ๐ŸŽž๏ธDepth Recorder
    • ๐Ÿ“ธLightfield Photo App
    • ๐ŸLooking Glass Core Overview
    • ๐Ÿฝ๏ธLooking Glass Core SDK
      • C API Overview
        • HoloPlayCore.h
          • State Functions
          • General-Query-Functions
          • Device-Query-Functions
        • libHoloPlayCore.h
        • HoloPlayShaders.h
    • ๐Ÿ”ฐLooking Glass Core JS
      • API
        • HoloPlayCore.Client
        • HoloPlayCore.Message
        • HoloPlay Service Related
        • examples
  • Legacy Hardware
    • ๐Ÿ”ปLooking Glass 8.9"
    • ๐Ÿ”ปLooking Glass 15.6"
    • ๐Ÿ”ปLooking Glass 8K
    • ๐Ÿ”ปLooking Glass 16"
    • ๐Ÿ”ปLooking Glass 32"
    • ๐Ÿ”ปLooking Glass Pro
  • ๐Ÿ’ฌContact Us
  • ๐Ÿ› ๏ธFAQ & Troubleshooting
Powered by GitBook

Follow us

  • Twitter
  • Instagram
  • LinkedIn
On this page
  • Requirements
  • Introduction
  • teps

Was this helpful?

  1. Tutorials

Using ShaderToy with Looking Glass

PreviousCapturing Light Fields in MayaNextSculpting with Adobe Medium and Looking Glass

Last updated 9 months ago

Was this helpful?

Requirements

Introduction

This post is written in collaboration with David Gallardo. You can find him on , or . is a popular website created by Pol Jeremias and Inigo Quilez, where shader programmers submit hundreds of shaders every day. In this tutorial, we'll explain how to convert those creations for the Looking Glass portrait. Some programming and basic linear algebra knowledge is needed.

โ€

Some programming and basic linear algebra knowledge is needed.

Fun Fact, some of the Holograms you've seen in the Looking Glass Portrait's Demo reel were actually made in Shadertoy! You can check out by here! ๐Ÿ’กMake sure to follow proper rules for crediting and attribution! Usually shaders on Shadertoy will have the license at the top of the main file!

teps

1. Rendering a quilt

In order to have the best visual result, you should directly render a and not an RGB-D image. Quilts allow for better handling of depth and occlusions, but also lighting and specular highlights. In these examples we'll be using a technique called ray tracing to generate our images and then combine them into a quilt. A quilt is an array of 2D images, where the bottom left is the left-most image, and the top-right is the right-most image.

2. Getting started with a shader built for VR

While only 0.5% of the shaders in Shadertoy support VR, they are easier to convert. For this tutorial, we'll start with VR Shaders. Converting a Shadertoy to work on the Looking Glass displays is complex. The goal is to generate a light field, but a ShaderToy offers a very minimal framework, which means that rendering code or camera management is coded from scratch inside the shader, so itโ€™s not possible to automatically convert a shader without minimal programming skills and motivation to understand how each shader works. Each shader will have its own challenge during the conversion. โ€ VR Shaders are easier because they are the only ones to have a 3d interface. A VR shader must implement a function like this, and this function is very easy to call from a light field generator.

void mainVR( out vec4 fragColor, in vec2 fragCoord, in vec3 fragRayOri, in vec3 fragRayDir )
  • fragColor is the final color for the pixel,

  • fragcoord (rarely used in VR) is the coordinate in pixel,

  • fragRayOri is the start position of the ray and

  • fragRayDir is the normalized ray direction.

โ€

In theory itโ€™s very simple - but there is a catch. In VR on Shadertoy, the objective is to have a panorama around you and the coordinate zero is the initial position of the VR headset. The software is tracking the headset and modifies the ray direction and position of each ray depending of the position rotation of the headset. On a Looking Glass we donโ€™t want a panorama, we want to put an object in a box (the screen). And itโ€™s important to have the object placed precisely where it should be.

โ€

Letโ€™s start with a simple shader that we created for this tutorial.

โ€

Comment out the original โ€œmainImageโ€ function and copy paste the following at the bottom of the source and replace it with the snippet here.

 #define QAA	2                                     

const float	sideAngle = radians(35.0);
const float	horizontalAngle = radians(14.0);
const float	cameraSize = 2.0;
const float	aspectRatio = 0.75;
const vec3  	VROffset = vec3(0, 0, 9);
float quiltColumns	=	8.;
float quiltRows		=	6.;
void mainImage( out vec4 fragColor, in vec2 fragCoord )
{
    vec4	accColor = vec4(0.);
    for( int m = 0; m < QAA; m++ )
    for( int n = 0; n < QAA; n++ )
    {
        // pixel coordinates
        vec2 o = (vec2(float(m), float(n)) / float(QAA) - 0.5) / iResolution.xy                                                                                 ;
		
		vec2	coord = (fragCoord / iResolution.xy  + o) * vec2(quiltColumns,quiltRows);
		vec2	fract = fract(coord);
		vec2	floor = floor(coord);
		float	imageId = floor.x + floor.y * quiltColumns;
		float	valueId = imageId / (quiltColumns * quiltRows - 1.);
		float	dist = (cameraSize / 2.) / tan(horizontalAngle / 2.);
		float	minCam = -(cameraSize / 2.) - tan(sideAngle / 2.) * dist;
		
		vec3	camPos = mix(vec3(minCam,0,dist),vec3(-minCam,0,dist),valueId);
		vec3	screenPos = vec3((fract.x-0.5)*cameraSize,(fract.y-0.5)*cameraSize/aspectRatio,0);
		
		vec3	dir = normalize(screenPos - camPos);
		vec4	color;
		mainVR(color, fragCoord, camPos - VROffset, dir);
		accColor += clamp(color, 0.0, 1.0);
	}
	fragColor = accColor / float(QAA*QAA);
}

The cameraSize is the horizontal size of the screen in the shaderโ€™s world space. It is a very important value and I probably should have called it screenSize instead. In our simple example we have a sphere with radius of 1 unit (So its diameter is 2) and to fit the whole sphere horizontally, you will need to set the cameraSize to at least two. If you find the sphere too big on screen you just have to increase the value.

The tricky part here is the offset: Here in the mainVR, itโ€™s easy to see that we have an offset of (0, 0, 9) applied, so we have two solutions here:

  • the first choice is to remove the offset here (in mainVR)

  • the second possibility is to counter the offset by setting the VROffset with the same value

Usually I prefer the latter because the less changes in the original code the better.

3. Converting shaders that weren't built for VR

Shadertoys that do not have VR support are more difficult and less intuitive to adapt, but most of the time itโ€™s not that hard even if there is no standard on how to write code on shadertoy. Most shaders are computing in the mainImage function the ray origin and the ray direction, so you have to add additional parameters to mainImage โ€œ_roโ€ and โ€œ_rdโ€ and override the value computed by the original shader by the new values in parameter, just before these values were used for the raymarching or raytracing. โ€ I converted a couple of famous shaders, I invite you to see how I converted them. You can click on the images to be redirected to the converted shadertoy.

โ€

4. Exporting a quilt video

ShaderBoi is a tool with lots of cool features and one of them is that itโ€™s very easy to export an โ€œonlineโ€ Shadertoy in video at any custom resolution.

To use it, you just have to open a command prompt and type command like this:

sboi -i Nlj3zm -s 2048x2048 -c rec -l 5 -y -h -o HyperDodecahedron.mp4

โ€œ-iโ€ is for the shadertoy ID, the example for this shader toy https://www.shadertoy.com/view/Nlj3zm the id is โ€œNlj3zmโ€

โ€œ-sโ€ is for resolution, we recommend 3360x3360 or higher for final holograms, but 2048x2048 works well for testing with the looking glass portrait.

โ€œ-cโ€ forces the cache to refresh, itโ€™s really important because without this flag, even if you modified your shader in shadertoy it will still use the one cached on your computer.

โ€œ-lโ€ is the length in seconds. I advise you to choose a small number when experimenting.

โ€œ-hโ€ is an optional parameter to export in H265 instead of H264. It gives you less compression artefacts with a smaller bandwidth, but some old video players wonโ€™t play it, so remove the option if you have issues.

You can use sboi help to know all the options and sboi rec help for all recording options.

Open this with your favorite browser and โ€œforkโ€ it. To do this, you will need a Shadertoy account and to be logged in. When you Fork the shader it will create a new copy of the original shader, you can rename it if you donโ€™t like the name (the name needs to be unique among every shader in shadertoy, sometimes finding a unique name may be a challenge).

on Github.

If you are making it for a landscape Looking Glass display, it's recommended to change the QuiltRows and QuiltColumns parameters to match the device you're targeting. You can find our standard quilt dimensions .

So if you followed all the steps you should have a shader equivalent to this . You can now directly jump to the โ€œHow to make a videoโ€ section to see how to generate a video to be able to view it on your holographic display.

You can download it from directly from the prerequisites section above, or from . You will also need to download FFmpeg, the instructions for this can be found in the readme file that is included with ShaderBoi.

๐Ÿง‘โ€๐Ÿ”ฌ
shader
view raw
shadertory_to_quilt.glsl
here
shader
DangerNoodle
Cloud SpikeBall
Diorama
ShaderBoi.net
Twitter
Youtube
Shadertoy
Protean Clouds
Nimitz
Quilt

Looking Glass Bridge

The connection between your computer and your Looking Glass.

Looking Glass Studio

A holographic media player.