Shaders

Ideas, enhancements, feature requests and development related discussion.

Re: Shaders

Postby rjh » Fri Dec 14, 2007 5:13 pm

Hey Sascha,

Looks like you are making great progress with the Renderman shaders. Could you possibly publish a rib file output from Jpatch (in polygons not sds) and the sl code for your shader ? I would like to test this against BMRT. While on the subject, take a look at http://www.renderman.org/. this is the RenderMan Repository and is a great resource for shaders.

Rob
rjh
 
Posts: 179
Joined: Thu Dec 30, 2004 10:33 pm

Re: Shaders

Postby sascha » Sat Dec 15, 2007 3:53 pm

Could you possibly publish a rib file output from Jpatch (in polygons not sds) and the sl code for your shader ?

Sorry, right now it can only export SDS. It's a bit of a hack, it's using JPatch's internal level-2 mesh data and exports it - this isn't optimal, it's got no normals at boundaries or creases and JPatch is more forgiving with non-manifold surfaces than e.g. Aqsis is. I also need to be able to export models at subdiv-level 3 or higher (as SDS or as polygons), so a better exporter is needed anyway.

I can still post the SDS RIB file and the shader source if you're interested. The shader is just an experiment, it's rather simplistic (the AO computation is done as a preprocessing step by JPatch, the shader just expects the occlusion values declared in a "vertex float [ ... ]" array, accompanying the vertex positions in the RIB file.

Is there a reason why you still stick with BMRT? It's for sure a nice piece of software, but it's closed-source and unmaintained, so I just wondered. If you need a RenderMan raytracer, you could give 3Delight or Pixie a try.

Talking about global illumination, I was under the impression that BMRT supported radiosity, do you have any experience with that?
sascha
Site Admin
 
Posts: 2792
Joined: Thu May 20, 2004 9:16 am
Location: Austria

Re: Shaders

Postby dcuny » Sat Dec 15, 2007 8:20 pm

sascha wrote:Talking about global illumination, I was under the impression that BMRT supported radiosity, do you have any experience with that?

I had a look at BMRT: A Global Illumination Implementation of the RenderMan Standard, where they describe the method used. It appears to be the same progressive radiosity method used in Blender, which no one really uses anymore. It's pretty straightforward to implement with a raytracer. You can certainly get nice results with it, though.

The problem with that approach is the same as the vertex-based AO - you've got to tweak the resolution of the mesh to get good results. In fact, Bunnell points out you can get the same sort of results by running an indirect lighting pass using his algorithm:
indirect_lighting.png
Direct lighting plus two bounces of indirect lighting
indirect_lighting.png (54.06 KiB) Viewed 8110 times
dcuny
 
Posts: 2902
Joined: Fri May 21, 2004 6:07 am

Re: Shaders

Postby sascha » Sun Dec 16, 2007 12:21 pm

I've just browsed through the docs of 3Delight, Pixie and Aqsis, and all three of them provide means of global or indirect illumination. 3Delight seems (not surprisingly) to offer the most sophisticated solutions, including raytraced occlusion and/or indirectdiffuse operations, as well as using precomputed point-clouds (or rather disk-clouds). Note that the point-clouds needed for the final AO pass can be computed by 3Delight using a special bake3d shadeop. Check out this section in the 3Delight manual. I'll have to play with that some day.

All in all this supports my view that global illumination is something that should be done by the renderer, and not by JPatch. What do you think?
This boils down to the the question whether my JPatch is not a renderer and never will be statement was premature or a violation of the Temporal Prime Directive :-)
sascha
Site Admin
 
Posts: 2792
Joined: Thu May 20, 2004 9:16 am
Location: Austria

Re: Shaders

Postby dcuny » Sun Dec 16, 2007 8:43 pm

sascha wrote:All in all this supports my view that global illumination is something that should be done by the renderer, and not by JPatch. What do you think?

Here's my thoughts in general:

  • JPatch should have a built-in renderer of some sort.
  • My preference is that the built-in renderer be software based, in order to support as many users as possible. There's nothing I hate so much as "free" software that requires the user to purchase expensive hardware to make it run. If I had money, I'd probably have purchased A:M years ago and been done with it.
  • Writing a renderer and adding features is fun, but time consuming. Time you spend of the renderer is time not spent on other JPatch features.
  • My nomination is Sunflow, because it's fast, has a nice interactive renderer, and produces nice pictures. Most importantly, it's written in Java and should be relatively easy to embed.
  • JPatch should support multiple renderers. No one renderer is going to have the features that everyone needs.
  • JPatch should not lock a user to a particular renderer. This means that - as much as possible - JPatch should generically support features such as "metal" and "glass". It should be capable of baking shader information to uv maps, or some sort of other generic method.
  • Users should be able to override the "generic" settings that JPatch provides. But the generic settings should, for the most part, be "good enough" that they wouldn't need to.

There are things that would be nice if they were automatic, at least from the user's point of view. For example, if a renderer used shadowmaps, there should be enough hooks inside of JPatch to support the automatic creation of shadowmaps without user intervention. Similarly, if the renderer supported texture maps, automatic creation of reflection maps would be nice to have. Of course, it would be up to the renderer to actually generate the maps - but from the user's point of view, they would "automatically" work.

All right - back to your specific question:
  • In general, I'd argue that JPatch shouldn't be doing any rendering. Find something else that can do the job!
  • However, the ambient occlusion calculation is actually a pre-process step, so it's got to be done outside the renderer anyway.
  • By putting it in JPatch, you can bake static objects, and run the calculation faster. For example, my OpenGL renderer wastes a chunk of time trying to figure what points are shared by a vertex, where JPatch already has that information.
  • Using disk-based ambient occlusion is actually better for animation than traditional AO, because it's got less noise associated with it.
  • JPatch can supply this AO information to other renderers.
The bottom line for me: the faster I can generate good looking frames, the better. :)
dcuny
 
Posts: 2902
Joined: Fri May 21, 2004 6:07 am

Re: Shaders

Postby rjh » Sun Dec 16, 2007 11:35 pm

Hey Guys,

Is there a reason why you still stick with BMRT?

lazyness ... I have become comfortable with the workflow I have developed with BMRT; it is second nature to me. I know that it will render the "experimental" bicubic patches that Jpatch outputs without arguing with me. Though I can say it has crashed BMRT before on some of the more complex scenes using polys as ouput. But, I can start up as many instances as my hardware can manage. I currently have 6 instances of Jpatch (with 6 corresponding instances of BMRT) up on 2 machines cranking out 700 frames of 944 x 400 resolution frames for some character animation that I am testing.

you could give 3Delight or Pixie a try.

I do have plans on digging into 3Ddelight ... I have dabbelled in it a long while ago. I susspect that I would have issues opening 6 instances of 3Delight ...
I also tried Pixie when it first became available ... tested the area light support with it using shadows ... gave interesting results, but quite a bit of hand tweaking had to take place before I could get a rib to render without errors.
I also plan to take a look at AIR. Its output looks very good to me, and they offer a low-cost entry point for a comercially supported renderer.

Talking about global illumination, I was under the impression that BMRT supported radiosity, do you have any experience with that?

Yes I do. David made some good points concerning this. One can get very good results, but quite a bit of tweaking is invloved, and they come at a high price ... radiosity in BMRT is very expensive. Even then the results are quite inconsistent for animation - rendering artifacts that look like the digital equivilant of grainy film. I'll try to dig up a radiosity animation test I did with BMRT ...

global illumination is something that should be done by the renderer, and not by JPatch.

Absolutely ... Jpatch should provide the geometry the renderer is expecting with the appropriate tags associated to that geometry, camera information appropriate for the renderer, and lighting information appropriate for the renderer.

David made some good points:

My nomination is Sunflow, because it's fast, has a nice interactive renderer, and produces nice pictures. Most importantly, it's written in Java and should be relatively easy to embed.

Here here ! (meaning I concur ... ) Sunflow does make some nice pictures. If you can integrate Sunflow as easily as you did Renderman (after I set it up in Jpatch, all I do is click Render Animation, specify the frames and away it goes)

JPatch should support multiple renderers. No one renderer is going to have the features that everyone needs.

Distributing Sunflow with Jpatch as an internal renderer (see ... I have already accepted that) and providing Renderman and POV support should appease most users. An SDK for a geometry parser could be provided to those interested in supporting other renderers through an outut plugin.

JPatch should not lock a user to a particular renderer. This means that - as much as possible - JPatch should generically support features such as "metal" and "glass". It should be capable of baking shader information to uv maps, or some sort of other generic method.

Yes, those do sound to be the logical attributes to provide as universal, definately the UV mapping, but at some point, Jpatch cannot be everything to everyone. In a sentence: "Jpatch must be an effective modeling, animation, and animation setup tool". If it ships as noted above (capable internal render - Sunflow; support for a professional industry standard Renderer - Renderman; support for one of if not the most prolific free renderers - POV), allows me to make the animation I do with it, and is efficient in doing so as it currently is, then it has served me well.

Users should be able to override the "generic" settings that JPatch provides.

Absolutely

And as always, I cannot wait to play with this once the Animation is there.

Rob
rjh
 
Posts: 179
Joined: Thu Dec 30, 2004 10:33 pm

Re: Shaders

Postby sascha » Tue Feb 05, 2008 11:15 pm

I've tried per-pixel AO with discs (passed to the shader as uniform arrays). 3Delight can handle that, but I'm not happy with the results.
So I've tried 3Delights built-in (raytraced) AO function:
The image uses my (fake) skylight, (fake) rim-lights, a single distant-light with raytraced shadows and raytraced ambient occlusion. It took about 140 seconds to render at 640x640 pixels:
PixelSamples: 4x4
ShadingRate: 1
Occlusion-Rays: 128 per pixel
(click for larger version)
3delight_raytraced_ao.png

Sorry for the "3Delight" overlay, I haven't installed the license server yet.

I think it looks very solid and real, almost as if it was carved out of stone. This is due to the AO - raytraced shadows add a bit to the realism, but without AO it looks more plastic like (funny, as none of the material parameters changed), and a lot less tangible (if this is the right term).

There is quite some noise, but that might be ok (adds some film grain ;-))
What I'm more concerned with are the artifacts (Head, above right eye, below nose, gloves, pullover,...). I'm not sure what's causing them, perhaps there are some parameters that can be adjusted. Both artifacts and noise get better with either more AO sample rays or a lower shading rate, but that translates to even longer render times.

Anyway, it looks much better than my earlier (disc/disc based) approach (rightmost image here).

Edit:
Here's a better one (click for larger version)
3delight_raytraced_ao2.png


The artifacts are still there, but the noise is gone - and it took only 90 seconds to render.
In the first version, I've subtracted AO from all channels, this time it's only subtracted from the ambient and the face-skylight channels, diffuse and specular are unaffected by AO.
sascha
Site Admin
 
Posts: 2792
Joined: Thu May 20, 2004 9:16 am
Location: Austria

Re: Shaders

Postby dcuny » Wed Feb 06, 2008 12:15 am

sascha wrote:I think it looks very solid and real, almost as if it was carved out of stone.

I think those words - solid and real - nail exactly why AO is compelling in a rendering.

One thing I don't like about the rendering is the use of yellow light. Anim8or also used yellow lights by default, and I think they look terrible. :?

There is quite some noise, but that might be ok (adds some film grain ;-))

I've always liked the noise, but the trick is keeping it low enough so that people don't notice it. have you considered adding a small random value to the approximated ambient occlusion to simulate the noise effect. :P

In the first version, I've subtracted AO from all channels, this time it's only subtracted from the ambient and the face-skylight channels, diffuse and specular are unaffected by AO.

Just to throw a spanner in the works, I think the first one looks a bit better than the second. :) (I think it has to do with more white light from above).

So clarify this for me: how does you AO shader work? It's the same as the per-vertex shader, but on a per-pixel basis?
dcuny
 
Posts: 2902
Joined: Fri May 21, 2004 6:07 am

Re: Shaders

Postby sascha » Wed Feb 06, 2008 9:59 am

One thing I don't like about the rendering is the use of yellow light.

I agree. I used a bluish skylight and therefor a yellow sun. This might be useful for "magic hour" lighting, but in general (especially for testing purposes) I'll use white lights in the future.

Just to throw a spanner in the works, I think the first one looks a bit better than the second.

Yes, the AO effect is stronger in this one. But you can't simply subtract AO from a direct lightsource, can you? This needs some fine-tuning.

So clarify this for me: how does you AO shader work? It's the same as the per-vertex shader, but on a per-pixel basis?

I've had a per-pixel shader that uses the occlusion disks working, but I wasn't satisfied with the results - and it was slow.
The images you see are computed using 3Delights occlusion() function in the shader - which simply shoots shadow rays in random directions (you have to make the objects visible to the raytracer though, which consumes memory). The first one used 128 rays per sample and a shading rate of 1. For the second one, I've lowered the rays per sample, but also the shading rate (I think it was 16 rays per sample and a shading rate of 0.25 - i.e. 4 samples per pixel). For some reason, this is faster and produces less noise, I'll have to play with it to find out why.

3Delight also offers true indirect lighting via raytracing (indirectdiffuse() shader funtion), subsurface-scattering, photon mapping, raytraced shadows and reflections and other quite coll features. All are optional and can be enabled per object, so you get a mix of speedy REYES rendering and cool raytracing effects. They still offer free licenses (the first license is free), even for commercial projects, so 3Delight is definitely worth a look.

When rendering the occlusion data to a separate channel, one could later average it over some frames (e.g. +/- 3 frames with some weighting funtion) to smooth it, like in the Pixar paper.
sascha
Site Admin
 
Posts: 2792
Joined: Thu May 20, 2004 9:16 am
Location: Austria

Re: Shaders

Postby dcuny » Wed Feb 06, 2008 6:49 pm

sascha wrote:I used a bluish skylight and therefor a yellow sun. This might be useful for "magic hour" lighting, but in general (especially for testing purposes) I'll use white lights in the future.

Part of the problem is the gray model, with a "plastic" sort of look. It's neutral, but unattractive. For a "clay" sort of render (i.e.:showing off lighting effects) it's better to have an object which is 100% white to start with. Using a gray object also reduces the contrast of the blue fill, which I think is the other essential feature of that the "magic hour" lighting.
Image
(Image from this lighting tutorial).

Yes, the AO effect is stronger in this one. But you can't simply subtract AO from a direct lightsource, can you?

You can do whatever you want, as long as it looks good. ;)

I've had a per-pixel shader that uses the occlusion disks working, but I wasn't satisfied with the results - and it was slow. The images you see are computed using 3Delights occlusion() function in the shader

OK, so it's standard raytraced AO. Other than the light noise and speed, there's nothing the matter with doing that.

3Delight also offers true indirect lighting via raytracing (indirectdiffuse() shader funtion), subsurface-scattering, photon mapping, raytraced shadows and reflections and other quite coll features. All are optional and can be enabled per object, so you get a mix of speedy REYES rendering and cool raytracing effects.

8)

When rendering the occlusion data to a separate channel, one could later average it over some frames (e.g. +/- 3 frames with some weighting function) to smooth it, like in the Pixar paper.

Yeah, I'd considered that. Occlusion data tends to change slowly. But note that in the Pixar paper, the animated objects have been removed from the scenes - only the static objects can amortize the lighting values.
dcuny
 
Posts: 2902
Joined: Fri May 21, 2004 6:07 am

Re: Shaders

Postby sascha » Thu Feb 07, 2008 12:10 am

After some fruitless experiments with my own (disc based) ambient occlusion shader, I found out that 3Delight supports this kind of ambient occlusion as well. It doesn't need any precomputed values, instead it creates a point-cloud (it's actually disks with normals and radius) in a first pass. This point-cloud is then used in a second pass to compute the per-pixel occlusion values.

Here are two images for comparison (click for larger versions):

ao_rt.png

ao_ptc.png


Both images show AO only, no other shader or lightsource is used.

The first one used raytraced AO. I've set the parameters to get rid of all artifacts and to keep the noise at a reasonable level. It took about 4.5 minutes to render a 640x640 image. Doubling the number of rays makes it almost noise-free, but also takes longer (9 minutes).

The second one uses the pointcloud-based approach. I didn't tweak any parameter (the pointcloud was also computed at 640x640), and both passes combined took about half a minute (9 times faster!).

The differences are very subtle, and although the raytraced version looks better in a side-by-side comparison, the pointcloud version is completely noise free (I feel an impulse to add some random noise to the shader ;-)) and renders about 10 times as fast, so I think this is the way to go when talking about animations. The contact shadows also look great in the pointcloud version, almost indistinguishable from the raytraced one (this is where all my tries failed miserably).

3Delight even supports pointcloud based color bleeding (i.e. simulated indirect diffuse light), I'll have to try that too.

All in all I'm very impressed, 3Delight really rocks! It's a pity it isn't open-source, but it's still cool that it can be used free of charge.

Edit:
Adding some subtle noise to the pointcloud-AO image IMHO improves it (as above, with noise added):
ao_ptc_noise.png
sascha
Site Admin
 
Posts: 2792
Joined: Thu May 20, 2004 9:16 am
Location: Austria

Re: Shaders

Postby dcuny » Thu Feb 07, 2008 1:36 am

The renderings look very nice! 8)

Blender's AO implementation also uses a point-based approach, modeled after the one in RenderMan:
Peach Blog wrote:The second approach I tried now approximates the sum of disks with spherical harmonics, as used in PRMan and explained in the Point-Based Graphics book, chapter 8.4 (sorry, again no direct link). This means we don’t have to cluster by normal anymore, though in the end it seems not much faster, but it does seem to avoid some artifacts at lower accuracy.

I'm going to have to take a good look at Blender's code - I haven't got $80 to spend on a book. But first, I'll have to get the rasterizer working. :|

Of course, I could just be sensible and use 3Delight to do the rendering. ;)

I ran across an interesting comment that the "dirty little secret" of the Gelato renderer was that they hadn't really been able to get that much hardware acceleration out of it. Considering that GPUs are memory constrained, and that most production scenes are massive, in retrospect that's not too surprising.
dcuny
 
Posts: 2902
Joined: Fri May 21, 2004 6:07 am

Re: Shaders

Postby sascha » Thu Feb 07, 2008 7:19 am

After I've discovered 3Delight's pointcloud AO feature, I remembered reading about it a year ago :roll:
I think this is strong support for my theory that the renderer should do the rendering :wink:
So I'll stop my AO attempts here and focus on modeling and animation again.
I haven't got $80 to spend on a book. But first, I'll have to get the rasterizer working.

$80 for a book - that's crazy. The information about ambient occlusion seems to be in a single chapter, so using Amazon's search inside feature might do the trick.
I ran across an interesting comment that the "dirty little secret" of the Gelato renderer was that they hadn't really been able to get that much hardware acceleration out of it. Considering that GPUs are memory constrained, and that most production scenes are massive

I'm not an expert here, but I think that the problem of current GPUs is that the length of shader code is constrained to a couple of kilobytes. This seems to be enough for games (and that's what these cards are built for, unless you spend some $1000 for a serious graphics card), but once this changes they should be able to render rather complex scenes in hardware.
The "secret" of REYES is that the algorithm doesn't need much memory. You pass geometry through the pipeline, and once it's on screen, the renderer can forget about it (much like OpenGL). I remember reading that even in early CGI feature films the renderer had to access several gigabytes of data just to render a single frame, and I doubt that any workstation had a GB of RAM available by that time. Texture images are also tiled, so the shader only has to load a few tiles into memory for efficient texture lookups.
I'm going to have to take a good look at Blender's code

Keep in mind that to do this efficiently, you'll need about one disk per pixel. One disk per pixel, one micropolygon per pixel, hmmm... du/dv information that comes for free (for computing the area of the micropolygon, err, disk)... Having a REYES renderer comes in quite handy here. This directly leads to my second theory: Whenever thinking about the best way to do rendering, sooner or later you'll end up at here.

I could just be sensible and use 3Delight to do the rendering.

It's not open source, but it's free (the watermark disappear once you connect it to the license server, and the first license is free - it's bound to your computer's MAC address). I'm not RMS (who'd argue that it's not free, but gratis), so I think I can live with that.
It's definitely a production quality renderer and has been used in production, so it should be fast, stable, reliable and produce high quality images (again proven in production, not test images) - and I guess it's a long way to get to that point.

Edit:
Btw, here's a list of 3Delight's features: 3delight_tech_specs.pdf
sascha
Site Admin
 
Posts: 2792
Joined: Thu May 20, 2004 9:16 am
Location: Austria

Re: Shaders

Postby dcuny » Thu Feb 07, 2008 9:37 am

sascha wrote:I think this is strong support for my theory that the renderer should do the rendering :wink:

I agree.


So I'll stop my AO attempts here and focus on modeling and animation again.

:D

I was all geared up to start working on the scanline renderer, but I've decided to back down from working on it. It just doesn't make sense when there's already a free alternative. However, I am bugging the Sunflow folk about adding it there. I'd love to have it available as an option for JPatch's "native" renderer. :P

$80 for a book - that's crazy. The information about ambient occlusion seems to be in a single chapter, so using Amazon's search inside feature might do the trick.

Heh. Yes, I had the exact same thought. Then again, I could also just look at the Blender code.

...but once this changes they should be able to render rather complex scenes in hardware.

You mentioned that a single frame could have gigs of data. A huge amount of this is often in the texture maps. I know there's work on getting the GPU to talk to the CPU, but for the moment, CPU is king.

The REYES algorithm scales really well to large amounts of geometry. As you said, that's really key. Speaking of which, I expected your link to show this image:
Image
dcuny
 
Posts: 2902
Joined: Fri May 21, 2004 6:07 am

Re: Shaders

Postby sascha » Thu Feb 07, 2008 11:06 am

I was all geared up to start working on the scanline renderer, but I've decided to back down from working on it. It just doesn't make sense when there's already a free alternative. However, I am bugging the Sunflow folk about adding it there. I'd love to have it available as an option for JPatch's "native" renderer.

I didn't want to talk you out of something - for the same reasoning I could stop working on JPatch and simply use Blender (with the additional killer argument that Blender actually is free software).
On the other hand, 3Delights feature list reads like a wish list for Christmas. Besides the REYES support, the raytracer looks pretty nifty too: ray-differentials, direct ray-shape intersection tests (without prior tesselation), etc. Then there's texture baking (2D and 3D), OpenEXR support, full RenderMan compliance, hierarchical SDS meshes, just to name a few.
It would be nice if it was opensource, but I'm confident that the opensource projects (Aqsis, Pixie, Blender's renderer, etc.) eventually will catch up. Until then, there's nothing wrong using gratis software, for what it's worth.
Speaking of which, I expected your link to show this image

Right :-)
sascha
Site Admin
 
Posts: 2792
Joined: Thu May 20, 2004 9:16 am
Location: Austria

PreviousNext

Return to Development

Who is online

Users browsing this forum: No registered users and 1 guest

cron