VRChat tricks, part 2

Let your shader see other players with Material Contacts

A VRChat contact receiver can now write the nearest hands and heads into your material every frame. Let's see how a shader reads them and keeps the effect steady.

  • Part 210 min dose
  • Intermediate

A shader could see its own mesh, the camera and the light. The hand reaching toward it? Never. Material Contacts change that. A contact receiver writes the nearest hands, fingers and heads straight into your material, every frame. Let’s look at what arrives, how a shader reads it, and the small habits that keep an effect steady.

Three players reach for one surface
Drag your hand, or press Tab to reach it and use the arrow keys

Turn on JavaScript to run this demo.

lib.hlsl, look.hlsl and hero.hlsl draw the demo above
Fig. 1

Your browser has no VRChat inside it (sadly), so these demos fake the contact data. You look down on one receiver, with your hand under the pointer and two players who move around on their own. Drag the blue hand. Each sender the shader receives presses a soft dent and lights a ring in its player’s color. The thin gray circles show where the hands really are, and the ones without a ring didn’t make it into the material.

A receiver that writes into your material

Every avatar carries small invisible shapes on its head, hands, fingers and more. These are contact senders, and creators can add their own too. A contact receiver is the VRChat component that notices them. I like to picture it as a net around part of your avatar:

  • Its shape, usually a sphere, is the net. Only senders inside it count.
  • Its Collision Tags (Hand, Finger, Head and so on) say which body parts it catches.
  • Allow Others catches other players’ senders, Allow Self your own.

Until now, a receiver on an avatar could only drive an animator parameter. Here’s the new part. Set its type to Proximity, turn on Write to Materials (the docs call it Write to Shaders) and list up to four renderers. From then on, every frame, VRChat writes a short list into those materials, one row per caught sender. I call each row a slot:

SlotWhat VRChat writes
0 to 7one sender each, telling where it is and which way it points, whose it is (the player id, a number VRChat gives each player in the instance) and which of the receiver’s tags it has
8the receiver itself, telling where it is (its Root Transform if set, without the shape’s offset) and how many of slots 0 to 7 are filled

In the shader, the list arrives as two arrays (numbered lists) of nine entries: _VRChatContact_Transforms_ and _VRChatContact_Data_, each followed by the receiver’s Parameter text.

Slot 0 holds the sender nearest the receiver’s center, and the others follow roughly in order (measured from the receiver’s center to the sender’s surface). Senders outside the receiver, and any past the eighth, don’t make the list. The tags arrive as tag bits, one on or off flag per tag in the receiver’s list. They are only filled when Write Matching Tag Bitflags is on, so don’t forget that one.

Which sender lands in which slot
Drag your hand, or press Tab to reach it and use the arrow keys

Turn on JavaScript to run this demo.

slots.hlsl draws the demo above
Fig. 2

Drag the mint hand around. Each sender that got a slot carries its slot number. A dashed one is inside the receiver but past the eighth slot, so it missed out; a plain lilac one is out of range. On the right, each bar is a sender’s proximity. It reads 1 when the sender reaches the receiver’s center and 0 at its edge. Proximity decides the order, but VRChat doesn’t write it. Box 8 shows the count as dots.

Reading the slots in a shader

VRChat ships a small helper file with the SDK. These are the lines I use most:

VRCContacts.cgincRuns in Unity only
#define VRC_CONTACT_MAX_TARGETS 8
#define VRC_CONTACT_MAX_TAGS 16
#define VRC_CONTACT_BUFFER_SIZE 9 // MAX_TARGETS, plus one extra for SELF
#define VRC_CONTACT_SELF_INDEX VRC_CONTACT_MAX_TARGETS
// ...
#define VRC_CONTACT_DECLARE(name) \
uniform float4x4 _VRChatContact_Transforms_##name[VRC_CONTACT_BUFFER_SIZE]; \
uniform float4 _VRChatContact_Data_##name[VRC_CONTACT_BUFFER_SIZE]
#define VRC_CONTACT_TARGETCOUNT(name) (asint(_VRChatContact_Data_##name[VRC_CONTACT_SELF_INDEX].y))
// ...
#define VRC_CONTACT_WORLDPOS(name, i) (_VRChatContact_Transforms_##name[i]._m03_m13_m23)
// ...
#define VRC_CONTACT_PLAYERID(name, i) (asint(_VRChatContact_Data_##name[i].x))
// ...
#define VRC_CONTACT_TAGS(name, i) (asuint(_VRChatContact_Data_##name[i].y))

That’s a lot of capital letters, so let’s go through it slowly. VRC_CONTACT_DECLARE creates both arrays of 9 entries; slot 8 is the receiver’s own. For name, type the receiver’s Parameter text exactly, the same name its animator parameter has. After that, four macros (named shortcuts that expand into code) cover most effects:

  • VRC_CONTACT_TARGETCOUNT(name) tells you how many slots are filled.
  • VRC_CONTACT_WORLDPOS(name, i) gives the position of the sender in slot i.
  • VRC_CONTACT_PLAYERID(name, i) says whose sender it is.
  • VRC_CONTACT_TAGS(name, i) shows which of your tags it has.

Each transform is a matrix, a 4 by 4 block of numbers that stores where something is, which way it points and how big it is. Its last column is the position, which VRC_CONTACT_WORLDPOS reads.

There’s one trap to watch for. The count is stored as the raw bits of a whole number. Read as an ordinary number, a count of 1 looks like a tiny fraction close to 0. So always read it through VRC_CONTACT_TARGETCOUNT, and never compare the value directly.

How far is the nearest contact?

VRChat sends where each sender is, not how big it is. So the simplest effect asks one question. How far is this pixel from the nearest one? It’s an old friend, the union of circles from Drawing shapes with distance, softened with smoothstep:

d=min⁡i∣p−ci∣f=1−smoothstep⁡(0,r,d)\begin{aligned} d &= \min_i |p - c_i| \\ f &= 1 - \operatorname{smoothstep}(0, r, d) \end{aligned}

pp is the point on the surface and cic_i the position in slot ii. dd is the distance to the closest filled slot (min⁡i\min_i means the smallest over all slots). rr (Reach r) is how far from a contact the effect fades to nothing, and ff is how strong the effect is. Taking only the nearest keeps every spot the same size; adding them all up lets close contacts melt together.

A soft falloff around each contact
Drag your hand, or press Tab to reach it and use the arrow keys

Turn on JavaScript to run this demo.

How far from a contact the effect fades to nothing.0.30
Use the closest contact, or add up the falloff of every contact.
falloff.hlsl draws the demo above
Fig. 3

Drag your hand and play with Reach r. Those small dots are all the shader receives. Set Combine to Add all and watch overlapping contacts add up and melt into smooth shapes.

A slot is a place, not a name

This one is sneaky, so let’s take it slow. Think of the slots as places in a race. Slot 0 is whoever is nearest right now, slot 1 the next. VRChat sorts the list again every frame, so when two fingers pass each other, they swap places. What stays the same is the name on the runner’s shirt. Here that’s the player id and the tag bits, which say whose sender it is and which body part.

Slot 0Slot 1
This framePlayer 2’s index fingerPlayer 2’s middle finger
Next frame, the middle finger came closerPlayer 2’s middle fingerPlayer 2’s index finger

So it comes down to two cases:

  • Only positions matter, as for a glow or a dent. Then you look at every filled slot. The order doesn’t matter, so the swapping can’t hurt you.
  • Something must stay with one sender, such as a random color or shape. Make it from the player id and the tag bits, never from the slot number. A random look grows from a number called a seed (the noise lesson shows how). Seed it from the slot, and the look hops over to the other finger at every swap.

One more catch is that tags only tell apart what you list. Both hands carry Hand, so also list HandL and FingerL to tell left from right. Senders that still match need a steady order of their own, such as their place along the palm.

Flowers seeded by slot or player
Drag your hand, or press Tab to reach it and use the arrow keys

Turn on JavaScript to run this demo.

What the random look of each contact is based on.
seeds.hlsl draws the demo above
Fig. 4

With Seed from on Slot number, watch the flowers when two senders swap slots. The petal count and the turn jump to another sender. Switch to Player and tag, and each sender keeps its own flower however the race goes. Fingertip flowers sit a little outside the fingertip so they stay readable.

Switching it off leaves a ghost

Here’s a surprise. Switching a receiver off is like pausing a video, because the material keeps showing the last picture. VRChat stops writing to the material, but it doesn’t clear it. The arrays keep the last frame they got, and the effect freezes where the hands were. The receiver’s own animator parameter drops to 0, but the shader never sees it.

So the effect needs its own off signal. One way is a material property, _On, that the menu toggle’s animation sets to 0 while it switches the receiver off. Another is to leave the receiver on and move it far away, so it keeps writing an empty list. That one also suits shaders that lock their material values at upload, like Poiyomi.

A ghost glow after switching off
Drag your hand, or press Tab to reach it and use the arrow keys

Turn on JavaScript to run this demo.

Turn the contact receiver on or off, as a menu toggle would.
The toggle also sets a material value that tells the shader to stop.
offsignal.hlsl draws the demo above
Fig. 5

Leave the blue hand where it is, turn off Receiver on, then drag it away. The glow stays behind like a ghost. Turn on Off signal and the frozen glow is gone. This demo can’t remember anything, so the other players stand still and the frozen frame is the one at the starting spot.

In practice

Here’s a setup that works, for a receiver named Glow:

  • The receiver on its own child object, so a toggle never touches the renderer.
  • Shape Sphere, its Radius the reach. Type Proximity.
  • Collision Tags Hand and Finger, and Head if heads should count.
  • Allow Others on, Allow Self only if your own hands should count.
  • Write to Materials on, your renderer in the list, Parameter Glow.

A minimal glow, in my own example code:

#include "Packages/com.vrchat.base/ShaderLibrary/VRCContacts.cginc"
VRC_CONTACT_DECLARE(Glow);
float _On;     // set by the same toggle as the receiver
float _Reach;  // in meters

float ContactGlow(float3 worldPos)
{
    int n = clamp(VRC_CONTACT_TARGETCOUNT(Glow), 0, VRC_CONTACT_MAX_TARGETS);
    float nearest = 1e5;
    [loop] for (int i = 0; i < VRC_CONTACT_MAX_TARGETS; i++)
    {
        if (i >= n) break;
        nearest = min(nearest, distance(worldPos, VRC_CONTACT_WORLDPOS(Glow, i)));
    }
    return _On * (1.0 - smoothstep(0.0, _Reach, nearest));
}

And that’s the whole glow. It’s one loop, a min and a smoothstep, all times _On.

One check helps. A matrix VRChat wrote always ends in 1 (_m33), and an array that was never written holds zeros. So VRC_CONTACT_TRANSFORM_SELF(Glow)._m33 < 0.5, another macro in the same file, means no data ever arrived. That happens in Unity’s edit mode, or for a viewer whose client doesn’t have the feature yet. A loop over the count like this one then draws nothing anyway, but a preview can use the check.

My Goo Strands and Skin Touch are built on this. Goo Strands stretches sticky strands between your palm and another player’s hand as you pull apart (yes, as gooey as it sounds). Skin Touch makes skin dent softly where a finger presses. Both use the full matrix in each slot, so they know which way a finger points, not only where it is.

A few things to watch for in Unity and VRChat:

  • One name per receiver. The Parameter becomes part of the property names, so use letters, digits and _ only. Two receivers with one name on one renderer overwrite each other every frame. Modular Avatar’s parameter renaming and a VRCFury Full Controller can rename the Parameter at upload. VRChat then writes under the new name and the shader receives nothing, so keep receiver names out of those lists.
  • Toggle the receiver, not the renderer. One case is known in the beta client. A target renderer started switched off, with Cast Shadows off, and its receiver stopped writing on the wearer’s own client until the avatar was reset. To be safe, keep target renderers on with shadows on.
  • Keep optimizers away. An empty entry in the renderer list switches writing off. Add the receiver’s Parameter as a Float to your FX controller so Avatar Optimizer keeps the receiver, and exclude target renderers from d4rk Avatar Optimizer’s merging.
  • A shader has no memory. It lives in the moment and sees only this frame’s eight slots, so it cannot know how long a hand stayed. Anything that must last after the hand leaves needs something that remembers. An animator state can keep counting a material value after the touch ends, or you can use a texture that remembers.
  • Both players must allow it. Avatar Interactions must let the other player touch you, and viewers only see the effect on a client that has the feature.

Credits

  • Material Contacts and VRCContacts.cginc are by VRChat, part of the VRChat SDK. Their beta documentation describes the receiver settings.
  • The random numbers in the flower demo come from Dave Hoskins’ Hash without Sine (MIT License); the noise lesson explains it.

Takeaways

  • A Proximity receiver with Write to Materials puts up to 8 nearby senders into your material every frame, roughly nearest first.
  • A slot is a place in a race that changes every frame. Seed anything random from the player id and tags, never the slot.
  • A switched off receiver leaves its last frame behind like a ghost, so give the effect its own off signal.
Search the lessons

Type any word from the lessons, like normalize or smoothstep. The start of a word is enough.