VRChat tricks, part 1

Small shader tricks for VRChat avatars

My little bag of shader tricks for VRChat, from spotting mirrors and cameras to hiding meshes cheaply and keeping outlines one size at any scale.

  • Part 18 min dose
  • Intermediate

A shader can’t run scripts, but it’s a curious little thing all the same. It can find out whether it’s drawn into a mirror or a photo, for a headset or a screen, and at what size. It can also hide a mesh in some views for almost nothing. Here they are, from most to least used.

A bunny and its reflection
Drag to turn the view, or use the arrow keys

Turn on JavaScript to run this demo.

Which views draw the gold halo.
Which views draw the pink heart badge.
Fig. 1

Meet the test bunny. Its halo and its heart each have their own material and mode. Set Halo and Heart to Always, Hide in mirror or Only in mirror, then drag to peek from another spot. The mirror here is only a reflected ray. In VRChat it’s a whole second camera.

Mirror, camera or your own eyes

Here’s something fun to picture. A VRChat mirror is a second camera that draws the world again, your avatar included, with the same shaders. The handheld camera is one more. So how does a shader tell the views apart?

Before it draws each view, VRChat sets a few globals, values that every shader can read by name. Two of them, _VRChatMirrorMode and _VRChatCameraMode, tell you which view it is (from VRChat Shader Globals):

ValueMirror modeCamera mode
0not a mirrornormal view
1mirror, VR viewerVR handheld camera
2mirror, Desktop viewerDesktop handheld camera
3screenshot

A stream has no value of its own and counts as the handheld camera. In Unity, both are 0.

One bunny, three views
Drag to turn the view, or use the arrow keys

Turn on JavaScript to run this demo.

The heart's reveal mode, as in my MirrorReveal shader.
Who looks into the mirror. VRChat reports it as mirror mode 1 or 2.
Which kind of photo. VRChat reports it as camera mode 1, 2 or 3.
The heart's material multiplies itself by a blue tint in mirrors.
Fig. 2

The two numbers under each view are mirror mode, then camera mode, set the way VRChat would set them. Play with Mirror viewer, Camera and Heart mode, and look for a dashed outline where a heart is hiding. Drag to spin the bunny around.

The mirror gives itself away in the projection matrix

The mirror camera stands behind the glass. So that nothing behind the glass gets in the way, its near plane, the surface where drawing starts, is moved onto the glass. That changes its projection matrix, the 4 by 4 grid of numbers that turns 3D positions into screen positions.

A quick heads-up first. Shader code counts rows and columns from 0, so row 2 is the third row. For a normal camera, depth depends only on how far ahead a point is. That’s why the first two cells of row 2 are always 0. A mirror camera fills them in:

[2][0]=a Cx[2][1]=a Cy\begin{aligned} [2][0] &= a\,C_x \\ [2][1] &= a\,C_y \end{aligned}

CxC_x is how much the glass faces sideways (Mirror turn), CyC_y how much it faces up or down (Mirror tilt), and aa is a scale that also depends on Distance. Don’t worry about the exact values. For the check, all that matters is whether they’re 0.

Turn the mirror and watch row 2

Turn on JavaScript to run this demo.

On puts the near plane on the glass, as a mirror does. Off is a plain camera.
Turns the glass left or right, in degrees. The top view shows this.25
Leans the glass up or down, in degrees. A top view cannot show this.0
How far the glass is from the camera, in metres.2.0
A VR eye's view is shifted to one side. This is that shift.0.00
matrix.hlsl draws the demo above
Fig. 3

On the left you see it all from above, with the lilac glass and the hatched area that never gets drawn. Move Mirror turn and Mirror tilt and watch row 2’s pink cells light up. Eye off-centre shifts the view like one headset lens. It fills [0][2] (top row, third column), but it never wakes a pink cell that sits at 0.

My helper checks both the global and the matrix, unity_CameraProjection, because each one can miss. The global has been seen at 0 inside a mirror, and the matrix misses a mirror seen straight on. Two checks, fewer surprises.

DocMe_VRC.cgincRuns in Unity only
inline bool vrcInMirror()
{
    return _VRChatMirrorMode > 0.5 || unity_CameraProjection[2][0] != 0.0 || unity_CameraProjection[2][1] != 0.0;
}

Hiding a mesh before the pixels even start

An animator toggle hides a mesh in every view. A shader can be pickier and hide a mesh only in mirrors, or only in photos. Its vertex shader runs once per corner and places it on screen. Its pixel shader runs once per covered pixel and picks its colour. Pixels far outnumber corners, so the earlier you stop, the less work. Here are three ways to hide a mesh:

  • Alpha 0. Every pixel is still worked out in full, then blended into nothing.
  • Discard. The pixel shader runs, then throws its result away.
  • NaN. The vertex shader writes NaN, “not a number” (what 0 divided by 0 gives), into the corner’s screen position. The GPU drops any triangle with a NaN corner before it makes a single pixel. The condition is the same for every corner, so the whole mesh vanishes.
What each way of hiding costs
Click to choose, or use the arrow keys

Turn on JavaScript to run this demo.

How the mesh is hidden. With one corner, click a corner to pick it.
nan.hlsl draws the demo above
Fig. 4

Step through Hide by. With Alpha 0 and Discard the left shows only a faint outline, but the right is still full of work. With NaN position both sides go empty. Then pick NaN on one corner and click a corner to move the pink cross there. Every triangle touching it disappears. Poof.

DocMe_VRC.cgincRuns in Unity only
#define VRC_NAN asfloat(0x7fc00000u)
// ...
#define VRC_CULL_IF(clipPos, cond) { if (cond) (clipPos) = VRC_NAN; }

VRC_CULL_IF swaps the screen position for NaN when the condition holds. There’s one catch, though. A headset’s own view has been reported to read a wrong camera mode while streaming, so I only ask for it outside the stereo version (next section):

#if defined(USING_STEREO_MATRICES)
    bool photo = false; // a headset eye is never the camera
#else
    bool photo = _VRChatCameraMode != 0;
#endif
bool mirror = vrcInMirror();
bool hide = (_Mode == 1 &&  mirror)   // hide in mirror
         || (_Mode == 2 && !mirror)   // only in mirror
         || (_Mode == 3 &&  photo);   // hide from photos
VRC_CULL_IF(o.pos, hide);

VR or Desktop, and which eye

Unity builds a separate stereo version of every shader for headset eyes, and only that version defines USING_STEREO_MATRICES. This one trips people up, so take it slow. It tells you the person looking wears a headset, not whether the avatar’s wearer does. Mirrors and the handheld camera use the plain version, even for a headset viewer. There, mirror mode 1 or camera mode 1 says it (a VR viewer, from the table above).

In the stereo version, unity_StereoEyeIndex says which eye it is, 0 for left and 1 for right. In a VR mirror, _VRChatMirrorCameraPos is the point between the viewer’s eyes. Picture yourself as the right eye, and that middle point sits on your left. So the eye that sees it on its left is the right eye. Few shaders need the eye. One that shows each eye its own picture does, like a 3D photo stored as a left half and a right half.

UNITY_SETUP_STEREO_EYE_INDEX_POST_VERTEX(i);
#if defined(USING_STEREO_MATRICES)
    float eye = unity_StereoEyeIndex; // 0 left, 1 right
#else
    float3 c = mul(unity_WorldToCamera, float4(_VRChatMirrorCameraPos, 1)).xyz;
    float eye = (_VRChatMirrorMode == 1 && c.x < 0) ? 1 : 0; // right eye in a VR mirror
#endif

The same size at any avatar scale

A width set in the mesh’s own units grows and shrinks with the avatar. The fix lives in the model matrix, unity_ObjectToWorld, which places the mesh in the world. Each of its first three columns is one of the mesh’s axes, and that column’s length is the scale along that axis.

sx=∣ ([0][0], [1][0], [2][0]) ∣s_x = \left|\,([0][0],\ [1][0],\ [2][0])\,\right|

The bars mean “length of”. ss is the average of sxs_x, sys_y and szs_z. Divide a width by ss and it keeps one size at any avatar scale. Tiny avatar or giant avatar, same outline.

Same outline at any scale

Turn on JavaScript to run this demo.

The scale of the avatar, as the model matrix holds it.1.50
The outline width the material asks for.0.045
scale.hlsl draws the demo above
Fig. 5

Move Avatar scale. On the left, an outline of Outline width in mesh units grows with the avatar. On the right it’s divided by the scale and stays put. The lilac arrows are the matrix columns, and their length is the scale.

float3 s3 = float3(length(unity_ObjectToWorld._m00_m10_m20),
                   length(unity_ObjectToWorld._m01_m11_m21),
                   length(unity_ObjectToWorld._m02_m12_m22));
float s = (s3.x + s3.y + s3.z) / 3.0;
float width = _OutlineWidth / s;

A few more little ones

  • _VRChatFaceMirrorMode is 1 while VRChat’s face mirror draws.
  • Some cameras draw without perspective, so far things don’t shrink. unity_OrthoParams.w is 1 then, so an effect that needs perspective can switch itself off.
  • VRChat hands shaders the clock in _VRChatTimeEncoded1, in the viewer’s own time zone, never the wearer’s. The SDK’s VRCTime.cginc decodes it with VRC_GetLocalTime.
  • A number can be split over two colour channels to keep its precision. The CRT lesson shows how.
  • Never start your own property names with _VRChat. VRChat may add globals with those names at any time.

In practice

  • Declare the globals you read inside the shader code, for example float _VRChatMirrorMode; float _VRChatCameraMode;, never in Properties. A material property with the same name hides VRChat’s value.
  • All of this works only on PC. Players who block custom shaders see the material’s fallback instead, and it draws in every view. So give a mirror-only secret Tags { "VRCFallback"="Hidden" }, or it isn’t much of a secret.
  • Put the hide condition in every pass, shadow and outline too, or that part still shows. A lonely shadow gives the game away.
  • The model matrix also holds any import scale, so check ss at the avatar’s normal size. On a skinned mesh such as the body, Unity may scale the corners itself. Check that ss really changes before you divide by it.
  • In Unity every global is 0, so test the mirror and camera branches in VRChat itself.

Credits

Takeaways

  • VRChat’s mirror and camera globals, together with row 2 of the projection matrix, tell a shader which view it’s drawn for.
  • A NaN corner drops a triangle before any pixel work, which makes it the cheapest way to hide a mesh in some views.
  • USING_STEREO_MATRICES means the person looking wears a headset. In mirrors, mirror mode 1 tells you instead.
Search the lessons

Type any word from the lessons, like normalize or smoothstep. The start of a word is enough.