• Page
• Discussion
• View source
• History

From Odwiki

(... very much "work in progress")

Contents 1 In The Beginning... 2 This Product Contains no Context 3 Don't Kill the Messenger 4 How to kill the messenger... properly!

In The Beginning...

...there was a function. And in The Library, it became Legion. (it was a really nice library, OK? :-)

That's all it is: a bunch'o functions -- but then, by extension, Houdini is nothing more than a heap'o parameters with a few dozen C++ operators for glue!... OK; so there's something more to it then, but what is it? A library has structure. <cue the choir on that last word>

This is a large topic, and there are many, many books written on the subject; but we can touch on some of the important points here. As a Houdini user, you've already been exposed to this kind of structure; or rather, the benefits you can get from a good version of such a structure: think SOPs. In Houdini you build geometry by combining small, well defined little units that are designed to do one thing (and only one thing) well. Additionally, these little units are as insulated from what goes on around them (the context) as possible. If their input is valid, then they'll happily go about their business and give you the right output; and if it's not valid, then they'll try really hard to simply ignore it (and raise a flag to let you know that your grand schemes are not so grand after all ;)

So; if we had to distill some of that into something useful; into a kind of "recipe" that we could follow when building our library, it would go something like this:

1. Write the initial version of your shader focusing on functionality. i.e. make sure it works exactly as you intended. If it's a bit of a mess, don't worry too much; "form" can come later (at least you'll hopefully be familiar with your own mess
2. Now that you know it intimately, step back, look at the overall structure, and try to spot all the bits that are truly generic: the sections that are purely functional and self-contained; things that could just as easily have been used in your previous shaders. (this is not as easy as it sounds... it takes a lot of practice to get this part right)
3. Extract all the generic chunks by stripping away all their contextual references. In other words, erase all remnants of their prior association with your shader, and turn them into pure functions.
4. Put each and every parameter through a "torture test". Feed it ridiculous values and see if it breaks. If you can't avoid a failure state (due to the nature of the algorithm), then decide how you'll handle the error: can you return an error code? should you take an extra parameter to write the error code to? or maybe just return some valid default value and "to hell with it"?
5. <Optional> For the purists: Here's where you stand even further back and compare your new function to the existing ones in your library, and look for a pattern. If your new function is really nothing more than a variation of something you already had, then you're likely looking at specializations of a larger concept.... and we're back at step 1. (although, if you're in this category, then you probably did this step before even writing the shader!
6. Finally; look at the DataTypes of both the parameters (inputs), and the retun value (output). If you have a one-dimentional function (float), can you extend it to two dimensions (2 floats)? how about three (vector)? four (vector4)? Did you notice that almost all runtime functions in VEX come in as many "flavours" as possible? There's a good reason for this, so try to do the same. Incidentally, inside VexBuilder (and in the programming world), these variations are called "signatures".

OK. Enough with the theory already; time to get dirty. Let's start with a few "Vex Gotchas" (with apologies to Stephen Dewhurst) that you want to be aware of as you write functions for your own killer vex lib.

This Product Contains no Context

Even though we're talking about writing shaders, we should keep in mind that some (hopefully a lot) of the functions we write for our shaders, could also be used in other Houdini contexts. After all, it's all VEX. It takes very little effort to write your library so that all its functions work in every context (or at the very least, fail graciously). But it takes a significant amount of effort to add this capability after the fact. So, to ensure that all context-dependence is gone, you need to:

• Never use context globals inside your library functions.

You can use them all you like inside the context function itself, of course, but refrain from using them inside your library functions. Why? Simply put: not all globals are defined for all contexts, and so using them in your library functions means you're unnecessarily restricting the usefulness of your library to only those contexts.

Prefer this:

float MyRings( vector pos; float amp,freq,phase,offset ) {
   return amp * ( sin(length(pos)*2.0*M_PI*freq + phase) + offset );
}

over this:

float MyRings( float amp,freq,phase,offset ) {
   float pos = length(set(s,t,0)) * 2.0*M_PI;
   return amp * ( sin(pos*freq + phase) + offset );
}

The globals s and t are not defined for all contexts, so the second version will fail when compiling a VEX COP, for example.

• Use compile-time symbols to do context switching.

This is known as "conditional compilation". The idea is to use vcc's Prepreocessor pass to choose different parts of your code to be included/excluded from compilation, based on whether one or more SymbolicConstants have been defined. Here's an example:

#if defined(VOP_SHADING)
   float curvature(...) {
      // calc surface curvature using derivatives
      return valid_result;
  }
#else
   float curvature(...) {
      // Non-shading contexts don't provide derivative info, so
      return 0.0; // 0.0 == no curvature... best we can do without derivs
   }
#endif

What this is saying is: "if the symbol VOP_SHADING has been defined (before reaching this line), then the first chunk will be included for compilation and the second chunk will never be considered, else the second chunk will get included and the first discarded". And when I say discarded, I mean it will be as though you had never written it -- vcc will never see it.
Fine; so where did that VOP_SHADING symbol come from?
It was defined (or not) automatically by VexBuilder as part of its compilation process. VexBuilder has the advantage of knowing the context for which it is compiling -- i.e. it knows that it is about to compile a surface shader SHOP instead of a COP, for example. And, knowing this, it can pass the information along to our code by defining a few symbols which our code can then use to branch accordingly (see the page on SymbolicConstants for a complete list of predefined symbols).

So that's how VexBuilder does it. But what if you're not using VOPs to build your OP, but using vcc directly instead? Then no one is going to define VOP_SHADING or any other symbol for you; you need to do it yourself. Luckily this is very easy to do: just create one include file per context (wherein you'll have a set of #define directives for that particular context), and remember to include it first, before any other #includes, in your main file (the one with yor shader). Putting it all together then, this is how we may go about modifying the curvature() example above so that you can include it from within VexBuilder, or in a VEX file being compiled directly with vcc:

First, a file (ctxtShading.h) that defines symbols common to all shading contexts:

//...

// The following is true for all shading contexts
#define WE_HAVE_DERIVATIVES

//...

Now we change our function file (say curvature.h) to take our own define into account:

#if defined(VOP_SHADING) || defined(WE_HAVE_DERIVATIVES)
   float curvature(...) {
      // calc surface curvature using derivatives
      return valid_result;
   }
#else
   float curvature(...) {
      // Non-shading contexts don't provide derivative info, so
      return 0.0; // 0.0 == no curvature... best we can do without derivs
   }
#endif

Now if we want to use it in a stand-alone compilation using vcc, we'd start our surface shader file like this:

// We're defining a shader so we include our "shading context" header first
#include <ctxtShading.h>

// And then the rest (which may use the info in ctxtShading.h)
#include <curvature.h>

// Then our shader code...
surface curvy(...) {
   Cf = curvature(P, ...);
}

And if we're building our shader inside VexBuilder, we don't need to do anything special because it will define VOP_SHADING for us, and our library already understands (and uses) it. We'll still need to include the file with the curvature function if we want to use it though. So we open up the OperatorTypeProperties dialog and in the Outer Code section, we write:

#include <curvature.h>

and in the Inner Code section we just use it:

float $localvar = curvature($pos, ...);

Both cases will work; and even if you were compiling a COP, you would still get no compilation errors (just a constant curvature of 0.0, that's all). Now; you may be thinking that this contradicts the whole "remove the context" bit I was going on about, but it doesn't: we're not adding context awareness to our functions, we're doing it to the build process -- our functions are still context-neutral.

(to be continued...)

Don't Kill the Messenger

All parameters in VEX are passed by reference. This means that if you assign a value to a parameter, you're actually modifying the variable that was used in the function call. It's easy to start talking in circles with this, so here's an example:

Don't try this at home:

01   float f(float x) {
02      x=0.5;
03      return x;
04   }

05   surface broken() {
06      float a = 0.2;
07      float b = f(a);
08      Cf = a*b;
09   }

What do you think the value of the variable a will be at the end of our "broken" shader? (no peeking!).

The answer is 0.5. What about b ? Also 0.5; and our output Cf will get the value a*b = 0.5*0.5 = 0.25. Surprised? If you've been exposed to other languages like C/C++ or even RSL, you should be (heck, you should be stunned that this thing compiled at all! :-). We explicitly assigned the value 0.2 to a during our declaration in line 06; so how did it get to be 0.5? It happened because we did something we should never have done to begin with: assigned a value to a parameter that wasn't explicitly declared as having read/write access. This happens in the function f() at line 02: x = 0.5. Function f() takes a parameter (x) and then changes its value to 0.5. And since we passed our variable a as the parameter to f(), what it actually did is change the value of our lowly a -- it just stomped all over it as though it owned it and didn't tell us anything about it.

OK; now that you are aware of how it can happen, here's the really important thing to remember: we have no way of telling function f() to not do that! (pretty please with a cherry on top). If f() wants to do it, it will, and there's nothing we can do about it (except rewrite f() and never write code like that again, of course!:). Now; the example above is pretty transparent, but it is not unusual for a library funtion to call another, which calls another, which calls another, and so on. And if you did something like this in a function that was at the end of a large call stack like that... well... you could spend weeks trying to find that bug.

The moral of the story: don't do it! ;)

How to kill the messenger... properly!

After all that, we're now going to talk about how to go ahead and do it; but carefully! Here's the thing: a function, by construction, can only really return one data type. In the case of VEX, this maps to one of int, float, vector, vector4, matrix3, matrix, or string. That's a really nice selection, but what if you needed to return an int, two vectors, one string, and a matrix? or how about 3 matrices?

Recall from the previous section that function parameters are passed by reference, and that we shouldn't write to them because this would modify the variable that was used in the call. Well; as it turns out, this is the only mechanism we have to enable us to return more than one item from a function. Which means that there are times when we absolutely must modify some parameters. But how can you tell, by simply looking at a function's signature, which parameters will be written to and which won't? Let me give you an example:

int Trace(vector pos, dir, Ph, Ch, Oh; float maxlen, Ah)
{
   //... a lot of stuf we don't want to look at goes here ...
}

Can you spot the parameters that the Trace() function modifies? No? Me neither. This hypothetical function traces a ray starting at pos, in the direction dir, for a maximum length maxlen, and returns a 1 for success, or a 0 for failure. It also "returns" information about the thing it hit: the location Ph, color Ch, opacity Oh and the alpha weight Ah, for a total of five parameters that are modified. I'll confess that I scrambled the parameters a bit to make it less obvious, but you see my point: It's hard to tell, at a glance, what's going on with this function. And after your library grows to hundreds of functions, you'll have a hard time remembering what you meant when you wrote it. So; what can we do?

One possibility is to document the function and state exactly what it does. This should be done regardless. But VEX also has an access modifier keyword that you might have come across: "export". You'll often see it used with parameters for context functions to signal the fact that the designated parameter can be modified (has write access). It's like a big sign that you can't possibly miss; and exactly what we need. Using this keyword and rearranging a bit, we can rewrite the Trace function like this:

int Trace(vector pos, dir; float maxlen; export vector Phit, Chit, Ohit; export float Ahit)
{
   //... a lot of stuf we don't want to look at goes here ...
}

I also adopt the convention that all modified parameters go at the end of the list; but that's a personal preference. The important thing is that now we can tell what's going on right away. Keep in mind however, that this is just a marker for our convenience; for legibility; having this keyword there doesn't suddenly make vcc impose any rules about what can be written to and what can't -- vcc still allows every parameter in a user function (not so with a context function) to be written to. So all the cautions from the previous section still apply; but now we at least have some way to keep the number of accidental errors down. For your own sanity: use it! :)

Incidentally... you may have noticed that a lot of functions in the VEX docs have parameter names that are preceded by a strange symbol '&'. This operator comes from C++ and, in this context, it's called the "reference operator" and carries the exact same meaning as our "export" keyword, i.e: "this parameter can be written to". In that world though, the compiler will only let you write to parameters that are marked that way - if it's not marked with a & and you try to write to it, it won't compile. In our case, the "export" keyword (in parameters for user functions) is just a placebo; vcc won't help you enforce it.

(to be continued...)

Predefined symbols for each context:

(Note: These should go in some Preprocessor page...)

In a stand-alone compilation:

• __vex is set to 1
• __vex_major is set to the major number of the Houdini release
• __vex_minor is set to the minor number of the Houdini release

Inside the Vex Builder environment:

For vop nets that define Sops, Cops, Chops, and Pops:

  #define VOP_OP

For all shading-related vop nets:

  #define VOP_SHADING

And context-specific:

       VOP_DISPLACE
       VOP_FOG
       VOP_LIGHT
       VOP_PHOTON
       VOP_SHADOW
       VOP_SURFACE

       (...)