Builtin OpenGL Renderer (Again)

Ideas, enhancements, feature requests and development related discussion.

Builtin OpenGL Renderer (Again)

Postby dcuny » Sat Aug 02, 2008 1:12 am

I'm just sort of thinking out loud here...

JPatch already has an OpenGL renderer, which uses "generic" OpenGL. As a result, you can't do shadow or texture mapping.

In theory, you could simulate z-buffer shadows using multi-pass rendering. That is, you could render the scene from the point of view of the various lights, creating a zbuffer for each of them. You could then render the scene in multiple passes, one for every light. After each pass, you could mask off the portions of the screen that occluded by light's shadow buffer. Composite all the passes together, and you've got shadows.

That seems like a lot of work, though. :?

Another possible approach - when I was playing around with SDS, I wrote a stupid little renderer. After subdividing the mesh, I simply sorted resulting micropolygons (to simulate the painter's algorithm) and rendered them the the screen using Java2D. The result was surprisingly good, although a bit slow:
LowResGirl1_0001.png
LowResGirl1_0001.png (32.5 KiB) Viewed 11749 times
You've already mentioned that you could get fairly large models into JPatch and subdivided using your code. Once you've subdivided the mesh down to about the size of a pixel, you can apply texture mapping and shadow mapping directly to the micropolygon and render it directly to the screen.

You'd probably want to cull the backfacing polys before rendering and shading, but as things stand, you've already got the core of a micropolygon renderer. Since everything is SDS, you're going to dice things anyway. Add multi-pass rendering and you've got soft shadows, motion blur and focal blur.

Of course, rendering the mesh down to micropolygon size is going to make things a bit slow. But you said your dicer is pretty fast, right? :mrgreen:
dcuny
 
Posts: 2902
Joined: Fri May 21, 2004 6:07 am

Re: Builtin OpenGL Renderer (Again)

Postby sascha » Sat Aug 02, 2008 6:39 pm

As a result, you can't do shadow or texture mapping.

Both could be added to the realtime renderer. Texture mapping is trivial, and things like hadrware shadow-maps or stencil-buffer shadows can be done, at least on OpenGL 2.0 cards. Check out the JOGL demos.
Of course, rendering the mesh down to micropolygon size is going to make things a bit slow. But you said your dicer is pretty fast, right?

You bet it is ;-)
Add multi-pass rendering and you've got soft shadows, motion blur and focal blur.

I've too thought about a REYES like renderer that uses OpenGL for shading and rasterization. Basically it would
* dice the geometry to pixel-sized micropolygons (in software - although this could be done with a clever fragment shader or forthcoming geometry shaders in hardware too)
* use a vertex shader for displacement and surface/light shading
* use vanilla GL for rasterization, perhaps with full-scene AA or accumulation buffers for antialiasing.

The shortcomings of this approach are:
* Shaders are limited. For example, light and surface shaders have to be combined into a single shader.
* Shader complexity is rather limited.
* There's no RGB opacity, OpenGL provides only a single alpha channel.
There could be a fallback to software shading though for shaders that can no way run in hardware.

The question is: for what? For pre-viz this is clearly overkill, and for production rendering its too limited. That's why I've basically abandoned the idea for the time being.
My approach to pre-viz would be to either use the realtime renderer, but save the images to file so that they can be played back at a constant framerate, or - if you need shadows or shading - use a production renderer with speed settings (e.g. simplified shaders, lo-res shadow/env maps, a higher shading rate, etc.)
sascha
Site Admin
 
Posts: 2792
Joined: Thu May 20, 2004 9:16 am
Location: Austria

Re: Builtin OpenGL Renderer (Again)

Postby dcuny » Sat Aug 02, 2008 10:26 pm

sascha wrote:...and things like hadrware shadow-maps or stencil-buffer shadows can be done, at least on OpenGL 2.0 cards. Check out the JOGL demos.

On my crappy work machine, I couldn't get shadow mapping to work, so I don't know how common OpenGL 2.0 is. That's one of the reasons I dropped my OpenGL renderer. Working with OpenGL shaders was also a pain - I'd have to reload Java on a regular basis. I was probably doing something stupid in my code, but after a while, the coding process bogged down as I kept having to fight the API and struggle to figure out the difference between one buffer type and another. :(

I've too thought about a REYES like renderer that uses OpenGL for shading and rasterization.

As you've pointed out, there would be issues with the shaders being limited. I suspect you'll see that issue go away in the near future, though - the better hardware can do some pretty amazing stuff.

But I was thinking about keeping it software based, up to the point of actually rendering the micropolygon. That way, you can make the shaders as simple or complex as you'd like. You aren't as fast as a hardware renderer, but it's still pretty fast. Of course, you lose some of that when you use 16x multi-pass rendering for soft shadows and anti-aliasing.

One of the cool things about the Reyes approach is, because you can jitter micropolygons in time, you can do most of that in a single pass.

The main thing that's missing is ambient occlusion. I'm holding out some hope that someone comes up with a clever and believable screen based AO, but not much.

The question is: for what? For pre-viz this is clearly overkill, and for production rendering its too limited. That's why I've basically abandoned the idea for the time being.

Your time is better spent on JPatch. Still, it's a pretty cool hack.
dcuny
 
Posts: 2902
Joined: Fri May 21, 2004 6:07 am

Re: Builtin OpenGL Renderer (Again)

Postby sascha » Sun Aug 03, 2008 8:15 am

But I was thinking about keeping it software based, up to the point of actually rendering the micropolygon.

Hmmm... You'd still have issues with opacity, and the speed gain will be marginal. I'm not sure, but I think that quite some time is spent in splitting and dicing, and the majority of the time is spent shading the grid. Rasterization should be quite fast anyway, so I don't see a speed gain here.
sascha
Site Admin
 
Posts: 2792
Joined: Thu May 20, 2004 9:16 am
Location: Austria

Re: Builtin OpenGL Renderer (Again)

Postby dcuny » Sun Aug 03, 2008 10:39 am

sascha wrote:I'm not sure, but I think that quite some time is spent in splitting and dicing, and the majority of the time is spent shading the grid.

To deal with the overhead of shading, you can do deferred shading. Instead of calculating the color, write out the depth, normal, and a shader ID. Once all the elements have been rendered to the screen, run through it and calculate the shading.

Since you've deferred calculating the shading until after you've determined visibility, you're guaranteed that you're only running shading calculations for visible geometry, which is potentially a big win.

You can handle transparency the same as REYES - keep a list at the pixel/subpixel level. When you get a transparent sample that's in front of an opaque sample, add it to the list. Again, you don't have to deal with shading anything until after everything has been rendered.

I've looked into depth peeling, which is what the Gelato renderer used to handle transparency. But that's based on the premise that you're using a GPU so that rendering again and again and again is cheap. For a software-based approach, keeping a transparency list seems much simpler.

One thing I noticed was that, if the micropolygons were small enough, I could generally treat them as rectangles and use their bounding box instead of doing a more exact interior test. This only really works if you're taking a REYES approach and using a sample location within a pixel instead of some location like (0,0) - otherwise, you get an error like what I was seeing with my transparency code.

I was initially thinking that OpenGL would be useful here, but the real work is done in the subdivision and shading, both of which wouldn't be helped by hardware acceleration.
dcuny
 
Posts: 2902
Joined: Fri May 21, 2004 6:07 am

Re: Builtin OpenGL Renderer (Again)

Postby sascha » Mon Aug 04, 2008 7:27 pm

Since you've deferred calculating the shading until after you've determined visibility

Yeah, but you can't do displacement that way :(
sascha
Site Admin
 
Posts: 2792
Joined: Thu May 20, 2004 9:16 am
Location: Austria

Re: Builtin OpenGL Renderer (Again)

Postby dcuny » Tue Aug 05, 2008 2:11 am

sascha wrote:Yeah, but you can't do displacement that way :(

No? :?

All you're doing is deferring the shading until later. You should still be able to perform the rest of the operations normally.

What am I missing here?

I've been looking around at screen space ambient occlusion. It looks like this guy has been making some good progress. Since most of the SSAO is aimed at realtime performance, the main question I've got is if you can get acceptable results with a higher number of samples.
dcuny
 
Posts: 2902
Joined: Fri May 21, 2004 6:07 am

Re: Builtin OpenGL Renderer (Again)

Postby sascha » Tue Aug 05, 2008 6:05 am

All you're doing is deferring the shading until later.
Displacement shaders are shaders after all. But you're right, I guess you could run the displacement shaders, then the hider, and then the rest of the shaders.
sascha
Site Admin
 
Posts: 2792
Joined: Thu May 20, 2004 9:16 am
Location: Austria

Re: Builtin OpenGL Renderer (Again)

Postby dcuny » Tue Aug 05, 2008 6:14 am

The more I think about this, the more tempting it is to try this myself. But I've got to keep from getting distracted. :P

Various rendering "tricks" require that there's a single sample/pixel, rather than an average of the surrounding samples. Z buffer depth is one of them, and so is storing the surface normals. I'd like to play around with SSAO, and it needs both the z depth and the surface normal. This doesn't really map well to the REYES algorithm, but it works with a plain zbuffer nicely.
dcuny
 
Posts: 2902
Joined: Fri May 21, 2004 6:07 am

Re: Builtin OpenGL Renderer (Again)

Postby sascha » Tue Aug 05, 2008 6:17 am

I've been looking around at screen space ambient occlusion.
I think the point cloud AO of 3Delight is very fast, and the results are virtually indistinguishable from raytraced AO. The pointcloud is generated by dicing the scene, and you can of course set up the scene beforehand (e.g. position the camera). If you use the same camera settings as in the final image, you should theoretically get the same results as with SSAO. Then again, you could move the camera slightly backwards or use a wider angle lense to also get occlusion data from out of screen objects. You can also use a completele different camera angle to do the AO pointcloud generation, e.g. an orthographic projection, and you can even mix multiple pointclouds.
I have to experiment a little to find out what works best, but all in all this approach is extremely fast and seems to be a lot more flexible than SSAO. As you said, SSAO is aimed at realtime rendering, point cloud AO will take a few seconds to setup the pointclouds (although they can be reused later - you could e.g. generate a cloud with all static objects, and then only compute the cloud for moving objects and combine them before rendering), but that's negligible for production rendering - a few seconds vs. a few milliseconds doesn't make a difference when rendering takes a few minutes, and you'd gain a lot of flexibility.
sascha
Site Admin
 
Posts: 2792
Joined: Thu May 20, 2004 9:16 am
Location: Austria

Re: Builtin OpenGL Renderer (Again)

Postby dcuny » Tue Aug 05, 2008 7:04 pm

sascha wrote:I think the point cloud AO of 3Delight is very fast, and the results are virtually indistinguishable from raytraced AO.

I was surprised to see that there was very little difference in speed between a simple Blender scene that used AAO. (They use roughly the same algorithm as RenderMan). But I suspect you'll see a slowdown with larger scenes. It's still much faster than raytraced AO, though.

And I agree that it looks just about as good as raytraced AO. In fact, it's better in most cases, because it's free of noise artifacts.

I have to experiment a little to find out what works best, but all in all this approach is extremely fast and seems to be a lot more flexible than SSAO.
There are a lot of limitations with SSAO, but I think for large scenes it'll end up being muchfaster. You pay for this in terms of artifacts, and there's the additional problem that you can only generate SSAO based on what's visible - back faces aren't taken into account.

When you generate the point cloud using AAO, you can't cull back faces, because you need that data for the point cloud. So there's a bit more time spent in rendering when the point cloud is created. For large scenes, point clouds get really big, and need to be cached on disk.

Obviously, SSAO doesn't have this issue, but you pay for it in terms of quality. But while it's not production quality, given how simple it is to implement, I'll probably try it out in Marin at some point.
dcuny
 
Posts: 2902
Joined: Fri May 21, 2004 6:07 am

Re: Builtin OpenGL Renderer (Again)

Postby sascha » Tue Aug 05, 2008 7:39 pm

When you generate the point cloud using AAO, you can't cull back faces

With REYES, you can't cull backfaces anyway (again, because of displacement mapping).
back faces aren't taken into account.

Why not? - IMHO you think too much in OpenGL terms. You'll dice the geometry anyway, backfacing or not, so why not use the diced grid as AO disks? The dicer will output everything you need, positions, areas and normals (the latter two can be computed using the u and v tangents). No need to recreate any of these values from the z-buffer, let alone store any of them in a z-buffer.
And again, 3Delight does the same thing, with the additional flexibility that you can dice any scene into a point-cloud (e.g. with the camera somewhere else).
I'd rethink that SSAO approach, it makes a lot of sense for games, but for production rendering - I don't know...
sascha
Site Admin
 
Posts: 2792
Joined: Thu May 20, 2004 9:16 am
Location: Austria

Re: Builtin OpenGL Renderer (Again)

Postby dcuny » Tue Aug 05, 2008 8:59 pm

sascha wrote:With REYES, you can't cull backfaces anyway (again, because of displacement mapping).

You're right. You can't really do any culling if you want the full AO taken into account.

Why not? - IMHO you think too much in OpenGL terms.

If you're doing SSAO, you're only looking at the zbuffer.

For anything else, you can do anything else you want. You're right - you can generate the point cloud directly from the data from your dicer.

The reason for bringing up SSAO is because all that data from the diced points goes into the point cloud.

I'd rethink that SSAO approach, it makes a lot of sense for games, but for production rendering - I don't know...

I'm not arguing against point cloud AO - I'm planning on adding point cloud based AO to Marin at some point. But it's going to take a bit of work, and SSAO is pretty low- hanging fruit. There's a paper from NVidia that shows the current state of the art. It's still pretty noisy, but increasing the number of samples seems to lead to good results.

My motivation (besides laziness) is that I've got a suspicion that there's some demand for a free "good quality" renderer. Given that there are already very high quality renderers available for free - like 3delight - I could be mistaken. If you're going to spend hours modeling and animating, it doesn't make a lot of sense to take a shortcut at the end for a low-quality render when you could get a high-quality render in just a little bit more time. :?
dcuny
 
Posts: 2902
Joined: Fri May 21, 2004 6:07 am

Re: Builtin OpenGL Renderer (Again)

Postby sascha » Wed Aug 06, 2008 7:27 am

...there are already very high quality renderers available for free - like 3delight...

I've nothing against free beer, nevertheless I wouldn't trade it for free speech - so if there's an open source alternative to 3Delight, I'd embrace it.
Of course there are Aqsis and Pixie too, both of which I haven't used for some time now, so I can't really tell how close to 3Delight they are in terms of features and performance.
sascha
Site Admin
 
Posts: 2792
Joined: Thu May 20, 2004 9:16 am
Location: Austria

Re: Builtin OpenGL Renderer (Again)

Postby pndragon » Wed Aug 06, 2008 12:19 pm

I've nothing against free beer, nevertheless I wouldn't trade it for free speech

Also remember think of the possibilities... You can't use 3delight to create something commercially ( to help pay for all of the time and money that have been invested in JPatch ) without buying a license. I think that also applies to things created in fun and later sold but I'm not sure.

--- Jim
"We're so sorry, Uncle Albert,
But we haven't done a bloody thing all day."
--- Paul McCartney
pndragon
 
Posts: 591
Joined: Sun Dec 05, 2004 1:27 am
Location: North Carolina

Next

Return to Development

Who is online

Users browsing this forum: No registered users and 1 guest

cron