James Graham

Interactive Graphics & Audio Specialist

My Projects

Click on a project!

Paddles Demo

Lua

Renoise API

A stylish update to
a classic game

Paddles

Paddles is a game written in Lua, that was met with praise upon its release, and is currently featured on the front page of Renoise's Tools website.

Write-Up

The Concept

Being a musician (drums, synths), one of my favorite programs is Renoise. After learning to program, I remembered that Renoise offers a Lua API to create extensions of the program! I decided to make a game within Renoise. Wanting something classic, I decided on Pong. The only other game made using the Renoise API is a port of Nibbles ("Nibbles" is a Snake clone included in FastTracker2).

Nibbles Gameplay

Renoise port of "Nibbles"

The Development

Learning Lua

I learned Lua using the official guide and documentation. I found my favorite feature of the language to be tables. Tables are so simple in contrast to C++'s structs, arrays, enums, vectors, hashmaps, etc. If you understand how they work under the hood, they can be very performant, too!

Engineering a Display

The Renoise API doesn't really offer a way to directly control pixels. Instead, it offers elements like sliders, knobs, text boxes, etc. However, I found that I could control a grid of tiny bitmaps (loaded from the filesystem) to create a "virtual screen"! Each frame, I had to manually update bitmaps, rather than blanking the whole screen and redrawing everything as graphics engines typically do, because Renoise isn't very fast at updating large amounts of bitmaps (it can update ~100 bitmaps at 25 frames/second).

Renoise Controls

Renoise ViewBuilder API controls & my bitmaps

The Launch

After some development, the game had a rainbow that trailed behind the ball, an AI opponent, a 2-player mode, sound effects, and more! Within 24 hours of its release, it was featured on the front page of the Renoise website's "Tools" section (where it still remains today), and even received recognition from Renoise's lead programmer and CEO; Eduard Müller (aka "taktik").

Early Paddles Demo

Early release of Paddles

The Update

Upgrading the Display

After the game's release, I discovered that Renoise provided another GUI element capable of creating a "virtual screen". Buttons can be scaled, and colored via a HEX color-code. This change offered a full 8-bit color display—capable of displaying 16,777,216 colors! To top it off, Renoise updates them 9x faster than bitmaps! I also implemented a double frame buffer system that only pushed updates to buttons that needed it. This allowed for more complex graphics, as I could now blank the screen each frame!

Button Demonstration

A button

Utilizing the New Display

I had been studying some water simulation algorithms for use in my 3D graphics projects, and decided to implement a water simulation in my Pong game for some nice visuals! The update was released and received more positive feedback from the Renoise community!

Paddles V3.0 Demo

Latest release of Paddles

Reflecting

If I were to continue working on this project, I would enhance the sound effects, and add a Breakout-style game mode. I would also like to write a 3D renderer for Renoise's Lua API someday, and create a small 3D game using it, though other projects are currently higher priorites.

Hopes

I sometimes worry that Renoise will cease development. Its keyboard–only tracker-style approach was common from the 1980's to the early 2000's, but most users today prefer a piano–roll interface. I hope that my Renoise tools will attract more like-minded users to the program, and keep its development alive for years to come.

Piano Roll vs. Tracker

The same MIDI sequence in a piano roll and a tracker

Curves Demo

Lua

Renoise API

A Bézier Curve
editor

Curves

Curves is a pixel-perfect, weighted, infinite-degree Bézier curve editor written in Lua, created as a testing ground for a feature being developed for Reform.

Write-Up

The Purpose

During development on Reform, the need arose for the calculation and rasterization of various curves. My first solution was to use logarithmic functions. They worked, but implementing discrete mathematical functions for different curve shapes wasn't a very robust solution. I ended up going down a rabbit hole of math and 2D rasterization that yielded the desired result. This project was the testing ground where I isolated this task.

The Development

Infinite-Degree Bézier Curves

After a lot of digging, I found Bézier Curves. Compared to Linear, Quadratic, and Cubic Bézier curves, I wanted my curves to support an infinite number of control points. I found this recursive definition of a Bézier curve. After implementing this—along with the associated functions for Binomial Coefficients and Bernstein Basis Polynomials—my curves supported an infinite number of control points! A very proud moment indeed!

Curves of Varying Degrees

Curves of varying degrees

Tension

I wanted each control point to have a variable "weight" (or "tension") to control its influence on the curve. I found an amazing resource called A Primer on Bézier Curves by Mike Kamermans (aka "Pomax"), which contains a chapter on adding this "weight"! After implementing this new math, I now had infinite-degree, weighted, Bézier curves!

Curve Tension

Adjusting weight

Rasterization

Bézier curves are rendered in a unique way. Rather than solving for X / Y coordinates per–pixel, you have to solve for t—which represents an interpolation between the curve's endpoints.

Showing t interpolation

t being interpolated

Naiveté

A naive approach is to just sample a high number of points, and fill the pixels where those points lie. The result of this approach is a curve that's too thick in high–tension segments, and breaking apart in low–tension segments. A better approach was clearly needed.

Curve sample sizes

Uneven t distribution

Pixel-Perfect

My solution was to sample a moderate amount of points, and connect them with line segments, which are easier to rasterize (which I later discovered is called "flattening" the curve). More robust solutions have been documented (such as in Chapter 4 of A Rasterizing Algorithm for Drawing Curves by Alois Zingl), but this solution would be sufficient, and quicker to implement. Wanting pixel–perfect, aliased lines, I found the Bresenham algorithm, which yielded a beautiful result!

Flattening the curve

Flattening a curve

Finished!

Being modular/loosely coupled, my finished code was easily implemented into Reform, with 2 curve types available now, and a custom curve editor to come in a future update. I will definitely be utilizing the knowledge from this project in other future projects as well! I can already imagine the possibilities of what can be created using these techniques!

Reform Curve Demo

Bézier curves implementation in Reform

Reform Artwork

Lua

Renoise API

A music composition tool for Renoise

Reform

Reform is a music composition tool for Renoise, that provides users with granular, interactive control over the transformation of selected groups of notes, performing processes such as scaling, bending, shifting, volume/panning/fx remapping, and more.

Write-Up

The Concept

After the initial release of Paddles, I got another idea for a Renoise tool; a tool that makes it quick and easy to powerfully fine–tune strums. The idea was inspired by FL Studio's "Strumizer" tool. It offers knobs and sliders to quickly and easily edit timings and velocities. You see the changes happen to the notes in realtime. Pressing the spacebar allows you to quickly hear the strum. This allows you to create and fine–tune strums very quickly.

FL Studio's Strumizer

FL Studio's Strumizer

The Problem

Being a tracker, Renoise represents musical notes as text on a spreadsheet. By default, each cell on the spreadsheet corresponds to a 16th note. To have notes trigger in–between this 16th–note grid, you must type a hexadecimal value 0x00–0x80 into the "delay" column next to the note. This results in an inefficient process for creating strums that are not in sync with the grid's time division, requiring the user to manually type individual hexadecimal values for each note over and over again.

Manual Strum Editing in Renoise

Strum editing in Renoise

My Solution

I decided to create a tool for Renoise that resembled FL Studio's Strumizer. Some priorites of the project included... a playful UI, intuitive controls, notes updating in realtime, easy preview playback, strumming across pattern boundaries, and more. This all added up to be quite a big project for a tracker–based environment.

WarioWare D.I.Y. Music Studio

WarioWare D.I.Y. inspired Reform's UI

The Development

It would take much too long to discuss every stage of Reform's development (be my guest to have a look at the source code though if you'd like), but I will cover some of the noteworthy stages of development. Just because something isn't discussed here doesn't mean it wasn't a significant stage of development, either.

Caching

To allow manipulated notes to overflow into other patterns, avoid colliding with other notes, and wrap from the end of the song to the beginning, I needed to know pattern lengths, song lengths, and more. When you're dealing with a large amount of notes spanning across multiple patterns, getting all of that data from Renoise's API becomes very slow. My solution was to cache the data Lua–side after retrieving it from Renoise. Each time I accessed data, I would first check to see if I had it cached, and if that cached data was valid. If the "valid" flag was false, I would get the data from the Renoise API. I would set the cached data's "valid" flag to false when changes were detected using the Observer–style notifiers that Renoise provides. This resulted in a dramatic increase in performance, while still ensuring that calculations are done with accurate data, that is only retrieved when needed.

Caching Diagram

Basic Caching Diagram

Curving

Through the development of my Curves tool, I studied Beziér curves, and implemented them in Reform to allow selections of notes to have their timings redistributed according to a curve. Note timings are treated as if they are distributed linearly upon selection, and can then be redistributed along a Quadratic Beziér curve, or an S-shaped Cubic curve. I could easily add more curves, and even allow users to create custom curves, but for the majority of use–cases, these two curve types are robust.

Reform's Curve Function Demo

Reform's Curve Function

The Result

This is my most organized, in–depth project to date, requiring a lot of learning, problem–solving, and optimizations to get the accurate, performant result that was achieved. I have some features I plan to add later on, such as undo/redo support, note–length preservation, quantize/humanize control, granular anchor placement, and a custom curve editor (based on my Curves editor). Despite this to–do list, I consider this project to be very full–featured and polished to a highly professional degree. I decided to provide the tool for free, out of love for Renoise and the many good times I've had with it.

Reform Demo

Demonstrating some of Reform's functions

SNEK Demo

C++

WinCon.h

FMOD

An ASCII game for the Windows Console

SNEK

SNEK is an ASCII-rendered game for the Windows console, and uses the FMOD Studio framework for procedural sound effects, and beat-synced music playback.

Write-Up

Starting Out

This was my first substantial programming project. I chose C++ as my first programming language because I wanted a low–level foundation for my understanding of programming, and because I find working with computers at a low level to be fun and interesting (for these reasons, I've also dabbled in GBZ80 assembly, and the circuitry of GameBoy modding). I chose the Windows Console for my first project because it has a very simple API that would allow me to focus on learning fundamental programming concepts.

Rendering

The game was first rendered using cout statements, but this didn't result in a framerate–capable runtime (although it did have a cool film–like effect). So I learned and refactored my code to write directly to the console buffer using the WriteConsoleOutputCharacter() function, which resulted in a stable image capable of faster framerates. It also allowed me to easily add some color to the game, using the WriteConsoleOutputAttribute() function.

SNEK Old Version vs New Version

Rendering comparison: cout vs WriteConsoleOutput()

Sound

As the project matured, I wanted to add sound to the game, and decided to learn FMOD Studio and the FMOD Studio API. From this endeavor I learned how to read API documentation, which assured me that I would be able to learn and use any libraries I want for future projects. I did successfully add sound effects, as well as music that is synced to the player's movement (each frame that the snake moves corresponds to an 8th note in the music). The music also reacts when the player eats a fruit exactly on a downbeat, gets a new high score, and more.

Music from the game
(Song sections progress as
the player's score increases)

Reflecting

Now that I've matured as a developer, I can see there are areas in this project where code can be cleaned up, and more elegant solutions can be implemented for certain features. But, I've decided to leave it as–is for the time being, and learn new technologies with more potential for a wider impact. If I were to return to this project, my first priority would be to make it cross–platform with Windows, Linux, and Mac. Then I would add more gameplay features and visual effects.

HTML

CSS

JS

Three.js

Portfolio (this website)

Portfolio

This is my portfolio website, created with HTML5, CSS3, JavaScript, and Three.js.

Write-Up

Design

I love 3D graphics. My favorite 3D effect is refraction, particularly when used to create water. I thought an interactive water effect would make a website a lot of fun.

Adaptive Performance

To accomodate a wide range of devices, Three.js's rendering resolution gets reduced on-the-fly if the current framerate is too low. This allows for an HD experience on high-end devices, while still maintaining a smooth framerate on lower-end devices.

Adaptive Geometry

The Quadratic formula is used to ensure the water's geometry has a consistent density of vertices regardless of the aspect ratio of the HTML document. This means the water simulation moves at the same speed regardless of where or how the website is being viewed.

To-Do

I have plans to work on the 3D scene soon. I've also considered adding a dark mode button, and some options to control 3D rendering or disable it entirely.

About Me

Picture of me!

James Graham

Hello!

I'm a software/web developer, specializing in interactive graphics & audio.

I love designing and developing high-level features, but I also love developing clever solutions in low-level code to make things run fast and smooth.

I'm currently open to work! Talk with me on LinkedIn, AngelList, or by email!

Skills

Languages

C++

C#

JavaScript

CSS3

HTML5

Lua

Python

Frameworks

Three.js

FMOD

Unity

Godot

Arduino

Tools

Git

Visual
Studio

VS Code

Neovim

Powershell

MSYS2

Other

ZBrush

Blender

Cinema 4D

Photoshop

Premiere

Aseprite

Renoise

FL Studio

LSDJ

Links