# Introduction to LPEs#

## What are LPEs?#

Light Path Expressions (LPEs) are a very flexible and powerful way to extract rendering information to output as AOVs. They are extremely powerful because they allow you to quickly create custom AOVs without being limited to builtin AOVs defined by materials.

LPEs use regular expressions to describe light transport paths. They define paths initiated from camera that are bouncing all around the scene until they reach light sources.

## Why are they useful?#

LPEs are extremely powerful as they can extract and output custom AOVs. They are extremely useful when it comes, for example, to extract per-light AOVs to relight in compositing or to output specific AOVs that are not made available by materials.

At first glance, LPEs can certainly look magical and tricky for new users. Indeed, in order to understand LPEs you must not only understand the basic principle of a path tracer but also the regex notation which is the standard syntax used in LPEs to write the actual expression.

Note

This document isn't specific to Clarisse: the implementation of LPEs in Clarisse/Angie is based upon Open Shading Language (OSL) implementation.

## How does a path tracer work?#

If you have no idea of how a path tracer works nor have any experience in writing regex, don't worry we will try to cover the basics in this guide so that you'll get up to speed in no time!

A path tracer is a method aiming at rendering a 3D Scene as an image. In its simplest form, a path tracer creates paths by launching rays from the camera that bounce in the scene according to material properties until they reach a light source.

Note that light source is used here in the general sense of the term referring to either a light, an emitting object or the background.

This succession of ray-bounces defines what is called a light path. In other words a light path is a path which starts from the camera and ends on a light source.

Contrary to the real world, a path tracer traces its paths backwards. The reason why paths are traced backward is for obvious performance reasons since a small minority of paths originating from light sources actually ends up to the camera.

Note

If you are interested to learn more about different path tracing methods we invite you to have a look here.

Another important thing to know about path tracers is that rays are bouncing all over the scene according to the properties of the materials and the maximum number of ray bounces set in the path tracer.

### Path Propagation and Materials#

Paths are propagated according to the properties of the materials that can be each a mixture of the 3 main properties: emission, reflection and transmission:

• Emission defines the amount of light emitted by the material.
• Reflection defines the amount of incoming light reflected by the material
• Transmission defines the amount of light transmitted through the material.

In the end, this mixture of properties is what's defining in which direction a ray should bounce or if it should terminate the path by sampling a light.

Let's put everything we've seen so far altogether and see how we can describe the following path to the renderer using an LPE:

1. The ray leaves the camera to the scene
2. The ray hits a reflective surface and gets reflected to the scene
3. The ray hits again a reflective surface and gets reflected to the scene
4. The ray hits a light source

In order to describe this path as an LPE we must first discuss the concept of Events.

## Events#

One of the most fundamental concepts of LPEs are events. Events are used to describe what's happening at each vertex/point along the path. There are 3 main events when it comes to define a path:

1. The Camera event describing when the path starts, when the ray leaves the camera.
2. Multiple Surface events describing when rays bounce on surfaces.
3. A Light event describing when the path ends, when the ray hits a light source.

### Event Syntax#

All LPEs events are always described in 3 parts such as:

<EventTypeScatteringType'Label'>

Where:

• EventType is a keyword declaring the type of the event (Camera/Surface/Light)
• ScatteringType a keyword declaring the scattering type of the event.
• 'Label' a user label used to identify a unique or a set of items.

Fortunately, LPE notation doesn't force us to specify Scattering Type or Label so we can just omit them for now and focus exclusively on Event types.

#### Basic Event Types#

Event types are always specified by a single letter keyword so that for example a Camera event will be written <C> where a Light event <L>.

Starting Event Description
C Camera event. All LPEs start with a Camera event. The camera event doesn't support any scattering types.
Intermediate Events Description
R Surface Reflection.
T Surface Transmission
Ending Events Description
L Light event. The light event terminates a path. The Light even doesn't declare any scattering types.
O Object Emission event. Object emission event terminates a path. The Object Emission event doesn't declare any scattering types.
B Background event when the ray hits nothing aka the background. It doesn't support any scattering types nor labels. This event can be used to generate masks for example.

Now that we saw the basic events types let's see how we can write an actual LPE.

## LPE Syntax#

Since Light Path Expressions always start from a Starting Event and ends with an Ending Event, they can be generalized like this:

<StartingEvent><IntermediateEvent1>...<IntermediateEventN><EndingEvent>

While the event type Camera is the only type of Starting event, there are multiple types of Intermediate and Ending events. Furthermore, Intermediate events are completely optional and thus they can be omitted from your LPEs.

Now that we've seen the syntax, let's finally write our first LPE!

If we look back at our previous example, each of the 4 steps are actually defining an event. The first event is an event of type camera. Then we have two successive reflection events to finally have an event hitting the light.

Now if we look at everything we've learned so far and considering that C defines a camera event, R a reflection event and L a light event...

...the path of our example can then be written like:

<C><R><R><L>


Here, as we can see, our AOV represents the indirect reflection of the light in our scene.

What if we want the first bounce of reflection?

It's actually very simple since we just have to use the following expression:

<C><R><L>


### Grouping#

Ok, let's see how we can now combine both <C><R><L> and <C><R><R><L> to get an AOV quite similar to the beauty with the exception of the light that won't be visible to the camera.

Let's take a few seconds to think about we want to achieve. What we would like is basically <C><R><L> + <C><R><R><L> to describe the following light paths:

As we can see there's some path branching occurring here. The green path is the direct path to the light whereas the blue one is an indirect path.

Assuming there's only one indirect bounce of reflection, the sum of the two paths leads to the full illumination in this render.

So how can we write down this expression?

Until now we intentionally omitted to speak about regex and now is the time! This when the power of regex comes into play since regex provides operators that are going to help us writing this expression.

What we are interested in here are the OR operator and parenthesis (). Using the OR and parenthesis <C><R><L> + <C><R><R><L> can be simply written:

<C>(<R><L>)|(<R><R><L>)


If you are perplexed with the OR operator, well, the OR operation is exclusive for a given path but inclusive for the scope of the expression. Our result is the union of <C><R><L> and <C><R><R><L> paths.

Note that it is possible to write a completely equivalent expression but written slightly differently:

<C>(<R>)|(<R><R>)<L>


Now let's see how we can only output the light seen by the camera. In that case what we would like is:

1. The Camera event which describes when the ray leaves the camera.
2. A Light event describing when the path hits a light source.

Such light path can be described this way:

<C><L>


As we can see now, we only have our light in our AOV as expected. Now let's combine this so that our AOV looks like the beauty. Okay, we have the following LPEs: <C><L>, <C><R><L>, <C><R><R><L>.

Using the parenthesis and the OR operator we've just seen, we can simply write:

<C>(<L>)|(<R><L>)|(<R><R><L>)


### Dot wildcard and syntax simplification#

Until now, we've seen that all LPE events have a mandatory Event Type and optional Scattering Type and Label. Well, there's slightly more to it.

For the sake of clarity we intentionally omitted to discuss about the dot wildcard concept and the syntax simplification.

What's important to know is that In regex the dot wildcard means ANY characters in such a way that, the regex a.c matches all the 3-letter-long strings that start with 'a' and end with 'c' such as for example, abc, aac, a1c, aZc...

Now if we look back at the event syntax, we've seen that an event is described like this:

<EventTypeScatteringType'Label'>.

So in fact, when we write <R> we are actually simplifying the notation <R..> which translates to any events of type reflection whatever the scattering type or label.

This simplification is possible since there's no ambiguity whatsoever.

Now that we've seen the dot wildcard, we can now finally see what are the Scattering Types.

### Scattering Types#

Path tracers don't compute the same way paths for diffuse, glossy or specular (mirror-like) reflection and transmission. It is then possible to use LPEs to isolate paths for each scattering component of a material.

In other words you can narrow down <R> or <T> events to output a specific scattering component.

Scattering Type Description
D Diffuse
G Glossy
S Specular for perfectly sharp (mirror-like) reflection or transmission
s Straight. Note the lower case. This is used when ray passes through a surface without changing its direction such as transparency/opacity for example.

The materials on our simple scene have each 3 components: one diffuse, one glossy and one specular. We've seen that the reflection event <R> outputs all components at the same time since the notation is equivalent to <R..> which translates to reflection event for ANY scattering types with ANY labels.

To identify a specific path for a given scattering type of reflection event we just have to write:

• <RD> for diffuse reflection
• <RG> for glossy reflection
• <RS> for specular reflection

To output the direct diffuse reflections of our light in our scene, we just have to write <C><RD><L>. For direct glossy reflections <C><RG><L> and <C><RS><L> to get the specular reflections.

So far we have only extracted the first bounce of reflection for each scattering component. Let's see how we can write an expression to extract all bounces.

#### Repetition operation#

We've seen that using the OR operator it's possible to concatenate multiple LPEs for path that have different depths. Indeed using <C>(<R>)|(<R><R>)<L> we output the first and the second bounce of reflection.

This notation is completely impractical to describe very long paths. Fortunately regex syntax provides {} which is a very useful operator to handle repetitions.

It is possible to specify a number of repetitions of specific patterns using {RepetitionNumber}. For example <RD>{1} corresponds to <RD> and <RD>{2} corresponds to <RD><RD>.

Then to extract the second bounce of diffuse reflection we can now just write <C><RD>{2}<L> instead of <C><RD><RD><L> which is very handy.

The great thing about the repetition operator is that it supports a range such as {minimumRepetitionNumber, maximumRepetitionNumber}.

It is then possible to write <RD>{1, 2} which translates to (<RD>)|(<RD><RD>). So basically we can now simplify <C>(<RD>)|(<RD><RD>)<L> by writing <C><RD>{1,2}<L>.

Another useful thing to know about the repetition operator is that it is possible to specify an infinite range by omitting the last number of the range.

For example, if we want to output the full indirect diffuse reflection which corresponds to the 2 bounces of <RD> and more, we can just write <C><RD>{2,}<L> regardless of the number of maximum indirect diffuse bounces set to the path tracer.

Indeed, <C><RD>{2,}<L> translates to something like <C>(<RD><RD>)|(<RD><RD><RD>)|(<RD> and so on...)<L>. To get the direct diffuse and the full indirect diffuse we can just write <C><RD>{1,}<L> which translates to <C>(<RD>)|(<RD><RD>)|(<RD><RD><RD>)|(<RD> and so on...)<L>.

The repetition operator also support ranges starting from 0 repetition which is actually super handy.

For example <C><RD>{0,}<L> corresponds to <C>(<L>)|((<RD>)|(<RD><RD>)|(<RD> and so on...)<L>) so that way not only we get our full diffuse component but also we can see the light.

If we wrap everything we've learned so far together then we can now finally write an expression to output the full beauty:

<C><...>{0,}<L>|<O>


or

<C><>{0,}<L>|<O>


or

<C>.{0,}<L>|<O>


Basically we get all paths starting from the camera and ending directly to the lights or emitting objects as well as all the intermediate paths with events such as <R> or <T> of any depth ending to the lights or to the emitting objects of the scene.

This corresponds indeed to our beauty!

##### Repetition range shortcuts#

Since the usage of repetition ranges are very common in regex, the regex syntax provides a couple of useful keywords to simplify the notation of expressions.

Keyword Description
* Equivalent to
+ Equivalent to

Using these shortcuts our beauty can be written like this:

<C>.*<L>|<O>


### Labels#

The final element of events we haven't covered yet are labels. Labels are extremely useful when it comes to apply an expression to a subset of items.

For example, labels can be used to identify a specific light or material or a group of lights and materials.

Let's consider the beauty expression <C><...>*<L>|<O>. Using labels we can easily output the beauty per light or material.

If in our scene the material of the ground has a 'ground' label and the material of our sphere has a 'sphere' label we can output the beauty of each material simply by specifying <C><..'ground'>*<L>|<O> and <C><..'sphere'>*<L>|<O>

In the same way, it is possible to output the beauty per light using labels in our LPEs. If our scene has two lights labelled each respectively 'point1' and 'point2', we can use <C>.*<L.'point1'> and <C>.*<L.'point2>.

You can see that we omitted the contribution of emissive objects. Indeed, since we are outputting the beauty per light, we must remove all paths ending on emitting objects for each of our light AOVs.

We then have to output a specific AOV for emissive objects: <C>.*<O>

### Matching operators#

The regex syntax offers a special matching operator which is very useful to simplify your expressions. Using [] operator it is possible to specify a set of possible matching scattering components.

This is very useful to simplify your expressions when you want for example to output both the glossy and the specular components of reflections for example.

Using the square brackets operator you can write <C><R[GS]>.*<L> instead of <C>((<RG>)|(<RS>)).*<L>. Please note that the two expressions are completely equivalent.

It is also possible to use the square brackets operator to exclude matching scattering components or labels. For example <C><R[^D]>.*<L> translates to <C><R[GSs]>.*<L> and <C><R[^DG]>.*<L> translates to <C><R[Ss]>.*<L>.

For labels, you can write <C><..[^'ground']>.*<L> to get everything else than 'ground' label. You can also write <C><..[^'ground''sphere']>.* to exclude both 'ground' and 'sphere'.

#### Further LPEs simplification#

It is possible to greatly simplify expressions. In this guide we intentionally kept using the <> event token since it is simpler to identify individual events when learning LPEs.

However, these tokens are entirely optional and you can directly write C instead of <C>. If we put everything we've learned together so far, you won't be surprised to read that the expression <C>(<L>)|(<R><L>)|(<R><R><L>) can be written CR{0,2}L.

Many tokens, and keywords can also be omitted when there are no ambiguities in your expressions. For instance, it is possible to write directly CD.*L|O to get the full direct and indirect diffuse component whether it's a reflection or transmission path.

In the same way you can write C'ground'*L|O instead of C<..'ground'>*L|O.

### Going further with LPEs#

Well, if you managed to go through all this guide, congratulations! The good news is that you should now have all the concepts to get you started in order to write your own LPEs.

However, just know that there's much more to learn about regex and we greatly recommend you to take a look at the Regular Expression page on wikipedia to learn more about the regex syntax.