Preface:
“On the Bubble” is a work-in-progress project I’ve been working on during my third year at Breda University of Applied Sciences. I’ve decided to write it out here because I’m looking for an internship, and this serves as the most recent up-to-date example of my ability to collaborate with a team and contribute to a project as a graphics programmer. I am also incredibly proud of it. We’ve been told many times that we’re one of the best tech projects that BUas has had, and I can’t help but feel pride in that. But do keep in mind, we’re still under development so take what you read here with a grain of salt. I will update this page once we’ve finally completed all the work on this project and it is released. That’s expected to happen around the middle/end of June.
Project Description
Link to Steam Page: [TBA]
Link to GitHub: [TBA]
“On The Bubble” is a game inspired by Super Monkey Ball. We’ve decided to expand on the concept of the widely loved game by adding a community level editor and path-traced graphics.
It is made with the “Ball Engine” we built from the ground up specifically for this game.
The Engine, Graphics, Physics, and Gameplay are all coded by us.
Here’s a video of the most up-to-date look of our “game”. Keep in mind Denoising and parts of ReSTIR are missing, so a large amount of noise is to be expected, we currently combat that by relying heavily on previous frame data, therefore all the blurriness:
The Team
The Core Team behind “On the Bubble” currently consists of 12 people:
– 5 Engine Programmers
– 3 Graphics Programmers
– 3 Physics Programmers
– 1 Producer
– 1 Visual Artist
Full list of credits on our GitHub.
Timeline
8 Months: From September 2023 – July 2024
I have contributed to every phase of this project, from its inception to its expected completion.
Technologies Used
The project is fully written in C++
Graphics APIs:
Windows: DirectX 12 / DirectX Raytracing
PS5: PS5 Graphics API / PS5 Raytracing Graphics API
Helper Libraries used:
– Ultralight
– tinyGLTF
– OpenGL Mathematics
– FMOD Studio
– Dear ImGui
– stb_image
– mod.io
– Steamworks
– Catch2
– nlohmann/json
Personal Contributions:
During most of this project, I held the role of Graphics Lead. As such I have the following responsibilities in this order:
– Keep a strong overview of the project as a whole and where the Graphics fit.
– Make sure conflicts with Engine and Physics are quickly cleared.
– Create and maintain the feature list, milestone graph, and all graphics issues on the scrum board.
– Evaluate feature requests and decide on feasibility based on research and discussion.
– Establish, maintain, and overlook the structure of Graphics code.
– Review code often and spot potential issues early.
– Research and implement required rendering features.
The following are the more impressive features I’ve worked on:
BEAR: Back-End Agnostic Renderer
From day one when we received the project brief and we were tasked with creating a cross-platform game engine for Windows and PS5, I knew that we couldn’t get away with what we did in Hellbound. Due to the insane amount of graphics work that was ahead of us, we decided that we would like to minimize the amount of platform-specific API work and create a wrapper for everything called BEAR (programmers like acronyms). We also decided to only wrap the compute dispatches and ray-tracing libraries, as that was 90% of our rendering, ImGui handles debugging, and Ultralight handles UI.
In retrospect, creating BEAR turned out to be a great decision that saved us a ton of time and allowed us more thorough debugging on both platforms as we could use debug tools from both platforms. The sacrifice we’re making is that we cannot do hardware-specific optimizations, but for only a 1-year project, we decided that this would be out of scope with the features we intend to have.
To summarize the process, it began with Andrei Bazzaev and I sitting down together and for 1 week we exclusively did planning, research, creating design diagrams, comparing the differences between the 2 APIs, and deciding on sacrifices we must make. The following week I worked remotely and we wrote the implementation and wrapper code based on the system that we thought of. Next week we got the prototype working with a basic compute shader which clears and displays a texture to the render target. In the final and 4th week of intense BEAR development, we focused on the ray-tracing libraries and wrapping the code required for us to do hardware-accelerated ray-tracing. This included the creation of BLAS, TLAS, wrapping shader function calls, etc.
Windows vs PS5 Final Image with common compute calls:
I want to note that this whole process would’ve been way more difficult without the help of  Jesper Enschede who agreed to join our team to take on the challenge of creating a transpiler that converts HLSL shader code to the format PS5 expects. Also, The co-creator of BEAR, Andrei Bazzaev wrote a more in-depth blog post, I recommend you check it out: [LINK HERE].
Both are insanely talented developers, and I can’t recommend them enough.
ReSTIR
For the longest time, I wanted to challenge myself by implementing a feature from an academic paper. I believe being able to understand, implement, and optimize an algorithm from a modern paper is the highest form of showcasing mastery in graphics programming. Especially in a game/product as that comes with its plethora of additional restrictions and specific requirements.
As of writing this, I am still very much in deep work and research. I’ve already implemented “Resampled Importance Sampling” with reservoirs which is the key element to the puzzle but I’m still working on reusing that data through space and time. Here’s the most up-to-date video of the algorithm working.
The most important part of ReSTIR is that it remains unbiased during implementation. If there is bias that can quickly add up due to the reuse of the weights. I’m proud to showcase that mine is (mostly) unbiased as you can see from the images below:
4096 Samples with Monte Carlo Sampling:
4096 Samples (* 16 RIS) with Resampled Importance Sampling:
Representation of Models in Bindless Heap
The way we currently dispatch compute shaders is by binding the relevant resources and keeping all global resources that could be required in a Resource Heap. For example, in the shader below we bind all information that is relevant for reading the rays we need to shade and for writing new shadow rays. But resources that might be needed in all shaders, for example, the data of all of the 3D models we might have in the scene, are kept in the m_ResourceHeap.
I was responsible for setting up how all models are represented within this Resource Heap. This system answers how we get from a ray and some Model ID, Instance ID, and Triangle IDÂ information that it returns when it intersects a scene to any resource that we might need in a shader like any buffer or texture. I plan to later write a whole article about a deep dive into the “Ball” renderer so for now have these structs as hints to how it might work:
Finally, for thoroughness’s sake, here’s a list of important PRs I’ve authored:
Lessons Learned
These are some key takeaways I have from this project. All of them have a story behind them, but I’ll skip it for now:
- Design backend graphics architecture with multithreading in mind from earlier. Especially when working with modern APIs like DirectX12 and PS5.
- The order in which we did the work was really good. It was a great idea to invest 2 months in doing our separate research into how a fully path-traced game, the problems we want to solve, and further issues we might encounter as opposed to directly jumping into development work in the project.
- Plan the Header of the main Graphics API that the rest of the engine will interact with and implement the simplest OpenGL renderer to not block other developers by not having visuals.
- If something seems like a stupid idea, that’s because it most likely is and a better solution is available, it might just take more time.
What’s Next?
As we’re moving onto the final 2 months of the project we’re going to be finishing up ReSTIR and Denoising, doing optimizations so the game can run decently on PS5, and tweaking the settings and assets to give the game the feel we want.
Hopefully, with this, we will have some great images and gameplay to showcase all of our hard work on this project. Stay tuned for the release!
About the Author
If you want to learn more about my work, please check my “About Me” page or any of my other blog posts about my projects on my Portfolio page.
Thank you for reading my blog post. Feel free to reach out to me for any comments/suggestions.