1) My ray stepping algorithm is now linear + binary search. What this means is I step my ray by large increments initially, detect when the ray's z-value is below the depth value, move my ray backwards, and decrease the step size. This is better than my previous approach of doing pure linear stepping because I can detect intersections faster and refine my results in less time.
2) Back-face rendering. I now use two framebuffer objects, one for front faces and one for back faces. Although this is somewhat slower than before, it makes reflections more accurate at certain angles.
3) Refractions. These were actually not that hard to do. Right now, the only difference between reflection and refraction in the shader is glsl refract vs reflect on the input ray. I might need to differentiate the two effects to be more physically accurate. I do not render refractive objects to the frame buffer because they would get in the way of the pixels behind.
4) Falloff. What I mean by this is the longer the ray must move to reflect an object, the more the reflected pixels fade. This lets me create objects that are cloudy or blurry, yet still reflective (not included in the demo below).
5) Lots of objects at once + physics (using Bullet)
What I need to do:
1) Improve visual quality. I've noticed that the reflected pixels get really jagged at around 45 degree angles to the surface. I'm still trying to figure out why this is happening. Also, I need to tweak my ray step algorithm to find the right balance between speed and accuracy.
2) Fresnel reflection. Reflection and refraction should adjust based on the view angle (assuming an object is both reflective and refractive).
3) I don't know if this is feasible for a couple reasons, but I would like to try out a linked list fragment structure for extremely accurate reflections. This would get rid of the ugly shadows when the reflective object covers the wall behind it. Unfortunately, this technique has high memory consumption. More details on this technique from Sean Lilley's blog: http://gamerendering.blogspot.com/
Anyway, there is still a good amount to work to do but I thought I would show a demo of my progress:
Note: The fps drops to around 40 when I'm really zoomed in, but otherwise it is 60+
Yesterday I got the framebuffers working and today I've been messing with the reflection shader. Right now the code is inefficient in many ways, but it is starting to resemble proper reflections. Anyway, here are some pictures I've taken throughout the process:
I'll be discussing the last three images. All of these have been running on my laptop's bad graphics card, so I expect a speedup on my dekstop. Speedup will also come with making the shader more efficient. I'm aware of at least one major slowdown at the moment, which is I'm advancing the ray in view space and converting to screen space every step, rather than converting the reflection direction to screen space from the start and only working in screen space from that point on. I was doing this initially, but I encountered some problems so I said screw it and did it the slower way. The jagged lines correspond to the ray step size. The smaller the steps, the more refined (but slower).
Also, you may notice in the final image that the reflections do not draw the backfaces of the spheres. This is because the framebuffer can only store one color value per pixel, the front pixel. Since we are accessing the framebuffer to get the reflected color, it will not find anything for the back pixel.
One interesting thing to point out is there is a cool shadowing effect on some of the walls. I'm not totally sure how this happens at the moment, but it is definitely cool.
Clearly there is a lot more work to be done in terms of visual quality and speed, but at least my results resemble proper reflections.