Mini-project: proto-slots

I’ve just released proto-slots, a mini-project I created as part of Ambienome. From the README:

proto-slots provides a macro for defining prototypal accessor methods so that CLOS instances will support protoypal inheritance. Prototypal inheritance means that an instance’s slots can inherit values from another instance, known in these docs as the “base object” (but more commonly known elsewhere as the “prototype”).

I wrote the first version of what eventually because the def-proto-slots macro to solve a problem I described in my previous post:

But, for each shape instance to have its own color (or other such data in the future), each shape instance must have its own job, because the color would be a uniform contained in that job. If I set the color in the base job, all instances of that kind of shape would have the same color. But if each instance has its own job, I lose the benefit of having the vertex data stored in one place.

I solved this by letting a job have a “parent”, from which it inherits anything it’s missing, such as a program or elements. For vertex attribs and uniforms, the inheritance works by merging the parent’s list of attribs/uniforms with the child’s list, with any of the child’s attribs/uniforms taking precedence when it has the same name as one of the parent’s.

I realized later that I was going to need prototypal inheritance for other parts of Ambienome, too. For example, I’m pretty sure the creature class will use this pretty heavily. So, I took my ad hoc macro, made it more flexible (it now supports different “inheritance strategies”), and polished it up. Along the way I decided to release it as a separate library, to maybe save someone else the trouble of implementing this same sort of thing.

proto-slots is available at GitHub, released under the X11/MIT license. If you find a bug, make an issue. To submit a patch, fork my repo and send a pull request.

Posted in Lisp, Projects | Leave a comment

Colored Shapes

I got a bit sidetracked while working on creature components, but still ended up making important progress for the overall system.

First, I created a new transform class by abstracting and cleaning up the shape class’s position, angle, and size attributes and methods. I also created a transformable “mixin” class, which can be used by any class that has a transform. (Lisp doesn’t have mixins in the same sense that Ruby does, but since Lisp supports multiple inheritance, you can get more or less the same effect by designing a class in a mixin-ish style.) The transformable class provides accessors for the transform’s position, angle, and size, so that you can treat them as if they were direct slots of the object holding the transform.

Next, I decided to enhance the shape class so that shapes can have a color. This turned out to be somewhat challenging, because of how I had designed the OpenGL wrapper classes.

Read More »

Posted in Ambienome | Leave a comment

Shape Mesh Progress

Yesterday, I completed the code to generate triangle and circle meshes:

Screenshot of a rectangle mesh, a triangle mesh, and a circle mesh.

The triangle looks like it’s too small, but it fits perfectly in the circle, and the circle fits perfectly in the square. Yay, geometry! Of course, when building a creature, they can be scaled to whatever size you like, and the algorithms can produce meshes at any level of detail. Eventually, I’ll make it so the game automatically adjusts the level of detail based on the shape’s size on the screen.

As I mentioned in the previous post, these aren’t the only shapes that will be available, but they are enough to let me move on to other things. One of the core concepts of Ambienome is that I can program new shapes later, even after the game has been released. If I’m feeling really clever, I could probably leverage Lisp’s power to allow users to program new kinds of shapes, too. That’s not a high priority, though.

Today I’ll start code-sketching the concept of components and creatures. A creature is built from one or more components, which are basically shapes with some associated behavior. Components can be grouped to move together, and groups can contain other groups, so it’s essentially a “tree” (hierarchy) of transformation nodes, with each “leaf” being a shape. This is the same concept as grouping in SVG or Flash, or parenting in 3D software like Blender or Maya.

After weeks of just studying and debugging OpenGL, it sure is nice to be making visible progress again!

Posted in Ambienome | Leave a comment

Adventures with the OpenGL shader pipeline

For the past several weeks, I’ve been learning “modern” OpenGL programming practices, by which I mean using a GLSL shader pipeline with vertex and fragment shaders.

Even before starting Ambienome, I was already somewhat familiar with the old OpenGL “fixed function pipeline”, using glBegin/glEnd, glColor, glVertex, etc. Ambienome is going to be visually simple enough that I probably could have used the fixed function pipeline, but I decided to learn the shader pipeline to improve my knowledge and skills, to allow nicer visual effects, and to leverage the GPU’s number-crunching power as much as I can.

Grokking the shader pipeline was a challenge. There are many separate concepts to learn, and then you must understand how they fit together. Also, there are fewer learning resources for the shader pipeline than there are for the older fixed function pipeline. I relied mostly on Joe Groff’s “An intro to modern OpenGL” article series, but if I were doing it over again, I might try to find a good book. Unfortunately, the selection is somewhat thin. I’m considering OpenGL Shading Language (the so-called “Orange Book”) or the OpenGL ES 2.0 Programming Guide. (OpenGL ES 2.0 is conceptually very similar to the OpenGL shader pipeline, and also very similar to WebGL.) Unfortunately, the reviews suggest those two are not very well written, organized, or proofread. Real-Time Rendering looks like it might provide a good high-level understanding of shaders and rendering, but it’s not specific to OpenGL or GLSL, and I’m not really interested in advanced photorealistic shader effects. Maybe if there are any book stores still open in this town, I’ll see if any of these books are on the shelf so I can assess them before I buy.

Anyway, even after I had scraped together a modest understanding of the concepts behind the shader pipeline, applying that knowledge in practice took longer than I anticipated. It was fairly frustrating at times, partly due to some (minor) shortcomings of cl-opengl and lispbuilder-sdl, and partly due to me misinterpreting some vague parts of the OpenGL documentation. There were many times when I felt like putting the shader pipeline aside, and just using the fixed function pipeline for a while, so I could keep my momentum up. But I stubbornly stuck with it, and now I’ve got a working, reusable rendering framework built atop the shader pipeline. Huzzah!

My next task is to write code to algorithmically generate mesh data for several geometric primitives. Yesterday, I finished the code for a rectangle mesh built from triangle strips (well, actually one long strip with degenerate triangles connecting each row). Here’s a screenshot of a 10×10 rectangle mesh rendered in wireframe mode:

Screenshot of a rectangle mesh

It’s not much to look at in itself. I could have achieved that in an afternoon using the fixed function pipeline! But that humble mesh, rendered here using the simplest possible vertex and fragment shaders, represents weeks of learning. Now that I have the basics working, I can write more interesting shaders to morph the shape and create cool visual effects. For example, Ambienome takes place underwater, so I’ll probably write a shader to make things sway and ripple as if they were being affected by a water current.

Next, I’ll be writing the code for a triangle mesh, and then a circle mesh. I have plans for more shapes, like teardrop, leaf, ring (circle with a hole in the middle), and tentacle (a Bézier curve with thickness controlled by another Bézier curve). But, I’m going to save those for later. Once I’ve got rectangle, triangle, and circle, I’m going to take a step back from the low-level foundation for a while, and start fleshing out higher-level concepts like components, creatures, and scenes.

Posted in Ambienome | Leave a comment

Ambienome – Technology

In my previous post, I described the concept and roadmap for my new project, Ambienome. In this post, I’ll be describing the technology (programming language and libraries) I’m using, and why I chose them.

Even though I have seven years of experience with Ruby, and my own game library, Rubygame, already available and ready to use, I’ve decided not to write Ambienome in Ruby. Instead, I’m writing it in Common Lisp, using OpenGL and OpenAL for graphics and audio.

Libraries

OpenGL and OpenAL were natural choices, since they’re free, cross-platform, and widely supported with pretty much any programming language. I would have chosen these no matter what language I settled on.

OpenGL doesn’t provide window management or input event processing, though, so I’ll need another library to handle that. The main choices for that are SDL, GLFW, and GLUT.

GLUT is a very popular and widely supported framework, but I don’t like how you have to shape your entire program structure around GLUT’s expectations, so I quickly dismissed it. SDL is also popular, but my years of experience with it have made me keenly aware of its shortcomings. So, I decided to try GLFW first.

GLFW is pretty nice, like a less obtrusive version of GLUT. I played around with it long enough to create an abstraction layer for opening and closing the window, but then discovered that GLFW currently has no API for setting the window’s icon. That may seem like a minor detail, but it was enough to nudge me into reconsidering SDL.

Two other points in SDL’s favor are that I’m already familiar with it — unlike everything else about this project! — and that SDL 1.3 will have support for multi-touch devices, and possibly multiple mice, both of which could provide some cool ways of interacting with the game. SDL 1.3 hasn’t been released yet (after many years…), but I might try taking a snapshot of the development repository, and working with that.

So, I ported the window abstraction layer from GLFW to SDL (1.2), and added an input event abstraction layer. I’m not entirely happy with it, so I might reevaluate GLFW (or even GLUT, since I’ve read that freeglut and Apple’s GLUT let you avoid the glutMainLoop function). But for now, SDL will be enough while I figure out the challenging new stuff with OpenGL and OpenAL.

One final library worth mentioning is Clutter, which I’m considering using to create the in-game GUI. When it gets to that point, I’ll try Clutter, then look into alternative libraries if Clutter doesn’t work out. As a last resort, I could just roll my own GUI widgets, but I’d rather avoid that if I can.

Lisp

OpenGL and OpenAL are pretty popular, “safe” choices of technology for game development. There are countless web articles, tutorials, books, and other learning resources available, and many experienced programmers I could turn to for help if needed. Both libraries are available on Linux, Mac, and Windows, and have bindings for many programming languages. There’s nothing particularly surprising or unusual about using OpenGL or OpenAL.

The Lisp Alien Mascot; Made with secret alien technology

The choice to use Common Lisp, on the other hand, is likely to raise a few eyebrows. After all, it’s secret alien technology powered by excessive parentheses!

Lisp, despite its elegance and power, is not nearly as commonly used for game programming as the C language family (C++, C#, Java, etc.), Python, or probably even Ruby. If I had to speculate as to why Lisp has a low adoption rate among game programmers, I’d cite the usual factors that affect Lisp’s adoption rate in general: difficulty getting started (choosing and installing a Lisp implementation and IDE, finding good tutorials, etc.), the “strange” syntax, Lisp’s reputation as being more academic than practical, and the Lisp community’s reputation (whether deserved or not) for being somewhat elitist and unfriendly. Also, many game programmers are obsessed with raw performance, as if brute-force number crunching was the key ingredient in making a fun game. Those programmers will usually stick with C or C++, and dismiss anything else as being “too slow”.

But, I don’t particularly care about Lisp’s popularity or why other people like or dislike it. I care about having an interesting, powerful language that fits the project well.

  • Like Ruby, Python, etc., Lisp promotes rapid prototyping with concise and flexible code, and sustains creative flow in a way “stop-and-compile” languages like C cannot.
  • Common Lisp can be compiled to native executables, and I can declare optimizations for those situations where number crunching efficiency is needed, such as generating custom audio data.
  • Lisp’s notion of “code as data” should work well with the concept of expressing behaviors as objects, and saving each creature’s behavior in a file.
  • Lisp’s macro and package systems should make it really easy to create a safe, simple, and efficient domain-specific language to allow users to define new creature parts and behaviors.
  • Lisp is a novel language to me, so this project is sure to be an interesting learning experience!

Before settling on Common Lisp, I also considered CoffeeScript and Clojure. CoffeeScript (using HTML5 Canvas and Audio elements) was very tempting because of the ease of deployment and cross-platform compatibility. But, HTML5 does not yet have a workable API for generating and manipulating audio data, and running in a browser would also limit my options for player-to-player file sharing and multiplayer gameplay.

As for Clojure: it’s a really interesting Lisp, and the ability to use Java libraries and compile to Java bytecode could be useful. But, learning to write a game using only immutable data structures and STM/agents would be a bit too much of a novel learning experience, and this project is going to be a lot to wrap my head around already. Also, Clojure’s strengths, such as safe multithreading, don’t provide much of a practical benefit to this particular project.

So far, I’m really enjoying Common Lisp. I’ve been reading Practical Common Lisp, which is a really excellent resource for learning Common Lisp, and available online at no cost. I’m certainly not a Lisp guru yet, but I can already feel the amazing expressive power and freedom Lisp provides. This is sure to be an interesting project!

Posted in Ambienome | 1 Comment

New Project: Ambienome

I’ve started a new project: Ambienome. This is a game (or, rather, a toy) I’ve wanted to create for years — I blogged about it in May 2007 and November 2009 — and now I’m finally making a serious attempt.

I’ll be blogging about my development progress, both to solidify my ideas by putting them into words, and to keep myself “accountable” for maintaining progress. I’m starting off with a two-part announcement and introduction to the project. In this part, I’ll describe Ambienome’s concept and a tentative list of progress milestones. In the second part, I’ll write about the tools (programming language and library) I’m using, and why I chose them.

Concept

Ambienome will be a freeform creative toy/playground, where players design and interact with colorful, musical, AI-driven aquatic creatures. The concept is heavily inspired by Toshio Iwai’s Electroplankton, and by Will Wright’s Spore.

I loved Electroplankton, but wished there was a way to create new types of creatures, to have multiple types of creatures in the same scene, and to save and share scenes with my friends. Spore’s easy-to-use method for creating new creatures out of parts, and integrated system for saving, uploading, and downloading creations online, suggested a workable model for creating and sharing Ambienome creatures and scenes.

Roadmap

This is a very novel kind of project for me, and it will have many challenges to overcome and problems to solve. I can’t even guess how long it might take to finish, so I’m not going to bother speculating about a project timeline. But, I have plotted out some of the major milestones for the project, in terms of functionality to be implemented:

  1. Players can interact with a variety of built-in creatures, and watch/hear them interact with each other.
  2. Players can assemble new types of creatures from a variety of built-in parts.
  3. Players can import and export creations (creatures and scenes), to share with other players.
  4. Players can upload and download creations from a central site (a la Sporepedia), and rate and comment on other players’ creations.
  5. Players can design custom parts, and share them with other players.
  6. Multiple players can build and interact with the same scene simultaneously, over the network.

I consider 1-3 to be essential, and the rest to be icing on the cake. I’ll do at least one release per milestone. I haven’t decided yet whether this will be an open source project or not. If it seems to be turning out well, I might try to make some money off it.

Currently, I’m still laying the foundation, implementing low-level functionality. For reasons I’ll discuss in the next post, I’ve chosen not to use Ruby or Rubygame for this project. So, in many ways I’m starting from square one, reimplementing the basics: opening a window, handling input events, drawing to the screen, managing the game loop, etc. Well, not exactly square one — I’m building atop other libraries, and I can borrow the basic concepts from Rubygame, so I don’t have to solve those problems all over again.

In the next post, I’ll be writing about the programming language and libraries I’ll be using, and why I chose them.

Posted in Ambienome | Leave a comment

Sup email client keybindings and filters

Lately I’ve been playing with Sup, a geeky console-based email client written in Ruby. I’ve used it enough to understand it and settle in a bit, but unfortunately also enough to realize that it’s probably not right for me. Along the way, though, I’ve done some work with keybindings and message filters that I’d like to share with you.

Before I get to that, though, a brief review of Sup. What first attracted me to Sup was the fact that it’s written in Ruby (with fairly clean code, from what I’ve seen), and has a fairly extensive system of hooks for customizing and extending the client behavior (the hooks are also written in Ruby). Also, it has a nice label-based filing system and powerful search utilities (a la Gmail), rather than the tired old folders metaphor. It runs in the console and is entirely keyboard driven, the workflow is pretty quick and easy, and you compose messages in the text editor of your choice (emacs, vim, etc.).

Those are the strong points, for me. Unfortunately, there are a number of issues that made me decide to look elsewhere. Foremost is the console-based interface using Curses, which holds it back in a variety of ways. One issue I ran into was that there’s no way to define keybindings that use the Alt key; Curses simply doesn’t support that. Sup also doesn’t seem to be able to render non-Latin characters, which is occasionally an issue for me. And naturally, being console-based, modern luxuries like HTML mail and previews of attached images are out of the question.

So, Sup turns out not to be my cup of tea after all. But, maybe it is yours, so I’ll share the fruits of my brief foray.

  • The first obstacle was the fact that Sup, like many Unix-style “mail user agents”, has no facilities for the actual fetching or sending of email over a network. Instead, it leverages programs like fetchmail, offlineimap, and sendmail to handle those things. This Unix-style mail system was something I had no prior experience in, so I had a lot to learn. After a lot of research and a fair bit of frustration, my advice is to forget the traditional fetchmail and sendmail, and instead use getmail and msmtp. They are a lot simpler to set up, and do everything most people need.

  • After I got my email imported into Sup, I wanted to change some key bindings to be more comfortable. In particular, being an Emacs user, I am used to Ctrl-n/Ctrl-p to move the cursor up and down lines, while Sup defaults to the vi-style hjkl bindings (which make no sense for me, using the Dvorak keyboard layout). Unfortunately, I could find very little documentation about how to change the keybindings, other than the fact that there was a “keybindings.rb” hook that I could write… if I knew what to write in it.

    Undeterred, I dove into the source code and learned all about the keybindings system. Then, I turned that knowledge into a comprehensive CustomizingKeys wiki page, so that the next guy wouldn’t have to dig through the code like I did. That was a fun afternoon project.

  • Next, I realized I needed a system to filter my incoming mail, automatically apply appropriate labels, and so on. Sup doesn’t have a filtering system like that built in, but it’s fairly straightforward to create one by taking advantage of Sup’s hook system. There is some documentation about this on the AutoAddLabelsToNewMessages wiki page, but as I started writing my filters, I began to realize that, although writing ruby code for every filter gives you a lot of flexibility and power, it also involves a lot of redundant code and is not very easy to read or maintain.

    So, I started writing my own filter system, which loads rules from a YAML file (see the example filters file). I’ve posted the result to my sup-filters repository on GitHub. Actually, what I’ve posted there is even more polished and better documented than what I used myself. Even after I had decided I wasn’t going to use Sup myself, I spent a couple days cleaning it up for other people, to call it a job well done. That’s Just How I Roll ™.

Anyway, if you use Sup, I hope you will find this stuff useful. As for myself, I will miss Sup’s labels and easy searching, but it’s time to continue my quest for a well-fitting email client. Next on my list to try is Wanderlust. If you have any recommendations for other clients to try, leave a comment!

Posted in Ruby, Snippets | Tagged , | 2 Comments

Nice-FFI 0.4 released

Nice-FFI 0.4 is now available. You can install the new version with gem install nice-ffi, as usual. Please note that Nice-FFI is still not considered API-stable, so if you aren’t willing to update your code when a new version comes out, you shouldn’t use it until version 1.0.

Version 0.4 fixes several major bugs related to finding and loading dynamic libraries:

  • PathSet now properly expands globs (“*”) in file and directory names, like it was supposed to.
  • PathSet now works even when a directory path has no trailing separator (“/” or “\”).
  • Fixed a typo which caused NameError when using load_library with the default paths.

You can get Nice-FFI:

Please send bug reports and feature requests to the issue tracker on Github.

Posted in Ruby | Tagged | Leave a comment

Nice-FFI 0.3 (and 0.2)

I’ve just released a new version of Nice-FFI, my convenience library for Ruby FFI libraries. You can install the new version with gem install nice-ffi, as usual. Please note that Nice-FFI is still not considered API-stable, so if you aren’t willing to update your code when a new version comes out, you shouldn’t use it until version 1.0.

Version 0.3 has a few changes:

  • The default PathSet has a less restrictive filename search pattern, so it is better at detecting libraries.
  • Struct.new can now accept FFI::Buffers. Buffers are like MemoryPointers, but with improved performance on JRuby in some situations.
  • Struct.new will now create a Buffer instead of a MemoryPointer when you create a new struct instance by value from an array, hash, bytestring, or another struct.

Also, since I never posted an announcement for 0.2, here are the changes for that version:

  • Added automatic memory management to the Struct class, if you define a “release” class method. It can be disabled per-instance by passing :autorelease => false to MyStruct.new.
  • Added new OpaqueStruct class. This class is useful for when the struct layout is not declared in the C library header. It also features automatic memory management like Struct.
  • Added Struct.typed_pointer, as a convenient alternative to TypedPointer.new(MyStruct). You can also pass :autorelease => false to disable memory management for all structs created via that TypedPointer, which is useful when using the TypedPointer as a function return type.
  • Structs can now be dumped to a bytestring (Struct#to_bytes), and also created from a bytestring (by passing a String to Struct.new).
  • Revamped the PathSet class. It now has separate patterns for paths (directories to search) and files (filename patterns), to be more flexible and easy to customize (e.g. you can add a new directory without caring about the filename pattern).
  • Removed Pathset#<< and Pathset#>>. Use append and prepend instead.
  • Renamed Library::DEFAULT_PATHS to PathSet::DEFAULT.
  • Added Library.optional, for when you want to continue loading the library even if a certain function or variable doesn’t exist in the library (e.g. because it’s not defined in old versions). It takes a code block, and will rescue and emit a warning if there is a FFI load error.
  • Also added Library.opt_func, which is the same as doing attach_function within an optional block.
  • Added some convenience aliases in Library: func = attach_function, var = attach_variable.
  • Nice-FFI no longer depends on the “need” library. We don’t need no stinking need!

You can get Nice-FFI:

Please send bug reports and feature requests to the issue tracker on Github.

Posted in Ruby | Tagged | Leave a comment

Snippet: ls_r (recursively list and process directories)

Here’s a little Ruby function I came up with today. I thought it might be useful to other people, so here it is.

Think you can do better? Fork me and prove it! ;)

Posted in Ruby, Snippets | Leave a comment