Leetless Coding Blog

Keepalive

2010-09-07T09:52:00

Hey guys.

I know it was very quiet here in the past 2 months. Good news for me is that I really like my new job. Bad news for grail is that I don't have much time for it at the moment. That does not mean its dead though. For my part, the commit rate hos "only" slowed down remarkably.

zdctb also mentioned that he might have some more time to work on grail in the near future. However we'd prefer a more regular stream of updates. So if you're a C++ developer who always wanted to participate in an adventure game engine (or you know of one), don't hesitate to contact us!

For the holy grail!

IRC Channel

2010-07-03T17:45:00

Just got over it in my previous post: There is an IRC channel for Grail on Freenode now, its called #grail (creative, eh?). Feel free to join there for any questions/suggestions regarding grail or any other stuff related to me or this site.

Site updates

2010-07-03T17:24:00

Hey guys. I'm glad to announce that I finished my diploma thesis a few weeks ago and am now working at the TU Braunschweig as PhD student.

As I'm not sure yet how the regulations regarding publication are, I will not publish my thesis here yet (probably its not of much interest to any usual leetless reader anyways).

There are also some changes in the VIM Section: I removed lots of the older themes in order to focus on the real good ones. Also I'm proud to present a new theme, called PhD, hopefully its useful to someone.

Another thing, which I neglected to post for months now is that the Giga Mansion 2 adventure game project is dead. They used the Indiana engine (the predecessor of Grail) which I developed in parallel to the game. At first I wanted to wait until they announce the end of the project themselves which seemingly never happened. Nevertheless it had been fun working with these guys and really pushed the adventure game engine stuff forwards a lot.

And while I'm at it: Because of the diploma thesis there was some pause, but I managed to include OpenGL into Grail, which makes it run about 10 times faster (at least on my machine). Check out the OpenGL Branch if you want to try it. If during compiling there is no OpenGL found, the engine will be compiled with plain old SDL, so this can not come to anyones disadvantage.

Going to github

2010-05-03T16:15:00

Currently data is being pushed to GitHub. I plan to let this run in parallel to the current self-hosted solution for a while and if it is good I'll shot down the self hosted git repository.

Reasons for github in favor of hosting the repository myself:

Wider audience
Integrated issue tracking system
My git web CSS is broken and I'm too lazy to fix it ;)

Check it out at The grail github repository.

Oh by the way, you may have noticed the latest commit is already a few days ago. That's because I'm currently very busy with university stuff. But don't you be afraid, I have no intent to give up this project. If you want to contribute though, it will probably advance somewhat faster :)

First Grail screenshot

2010-02-16T19:09:00

Whats new:

Parallax scrolling of background/foreground layers
Animations don't require annoying to produce "stripe" format anymore, additionaly theres a script now that extracts layers from an xcf (Gimp format) file into a directory so its useable by grail.

Edit: I substituted the .png with a .jpg so the page load times should be lots less annoying.

First steps

2010-02-02T22:13:00

Luabind is so cool. Wrapping really is the easy part now, so I finally focus on implementing stuff again. The main character already can walk around on the screen, now I'm concentrating on a user interface and then on interactions with other scene actors.

Thats sounds like the biggest part but keep in mind, theres still a lot to do:

User interface already is a big topic (buttons, inventory display, etc...)
Loading/Saving (will hopefully be easy due to boosts serialization lib)
Scene/walkpath stuff (the "algorithmic" corner yay)
Loading scenes directly from atlantis files
Dialogs (talking)
Sound/Music/Voice output
From time to time try to get the stuff to compile & run under windows ;)

Feed problems

2010-01-18T16:20:00

Sorry for the feed problems lately (wrong urls, complete feed gets pulled twice etc...). Should now be fixed.

In general: When you discover something doesn't work well/correctly, please report, I can only keep the site up with proper feedback.

Good bye NIH, hello luabind & boost!

2010-01-18T15:45:00

I have to confess something. To you all.

I am ill. What I have is called the NIH syndrome. For those of you who never heard of it: It means preferring to code stuff yourself instead of learning how to use a library somebody else wrote. So its all about reinventing the wheel. [1]

For example I was inventing a DSL for describing c++/lua bindings together with a handcrafted (and ugly) parser which generated c++ code from that DSL etc... although I knew there was luabind.

Now that I tried it, let me say: luabind rocks. Don't dare to invent something yourself until you tried that one.

Same goes for boost::shared_ptr. It is fun to invent a reference counting mechanism, but seriously: Use shared_ptr, its really cool, luabind supports it already and it simply makes your life so much easier.

I also plan to use/learn more parts of boost such as the serialization library which seems to be even more flexible than the serialization stuff in D I wrote and was kinda proud of.

Stay tuned for more.

[1]

Note that this can be a good idea actually. Say you decide to use $big_library and it works quite well. You save valuable time in your early project phases by not having to write all that stuff yourself.

But then, as you project evolves you have more special needs and demands to that library. Depending on how its designed and what it is about you might have to actually patch the library which suddenly requires a huge amount of time, because you have to learn the internals of $big_library for just one little change.

libregamewiki.org linking

2009-12-12T18:57:00

Grail is now findable on the Libregamewiki thanks to BAM, so another reason for me to spend more time on grail in the future.

Somebody feel like making CSS?

2009-12-10T13:38:00

My current gitweb CSS looks terrible, the standard stuff doesn't fit very well into the rest of the page so I started this but I didn't really finish and now lost fun on it ;)

So if somebody feels like making the existing CSS more pretty/useable (or, if you prefer, creating a complete new set of CSS rules for the gitweb area) your help is very welcome, just drop me a mail.

A few thoughts on memory management in grail

2009-12-10T09:42:00

In D we had a garbage collector that made things really easy: Just allocate an object and never think about what happens to the memory of that object again, the gc does it for you. In C++ there is no garbage collector built in so you have to delete old objects yourself. That per se is not so much a problem as long as the question of ownership is always cleared.

Ownership means: Object a has a reference to object b and is responsible for deleting it. Normally with a little thought its easy to say which object should own which one(s). However, if you let the user allocate and play around with objects in lua, things can get a little more complicated.

Current non-solution

Currently I'm using a thingy I call "registry". This holds references to all objects allocated in lua and destroys them when they aren't useful anymore. To find out when that is, you tell it to the registry. There are two possible scopes: CHAPTER and APPLICATION. Objects with CHAPTER scope exist until the current chapter runs. This is most useful for scenes and the actors running around in them.

The APPLICATION scope holds objects as long as the program runs. This can be useful for the main menu or other user interface stuff.

However this approach now leads to some Problems:

Currently it can happen that a Scene "s" holds an actor "a" and its not in general known if "a" is in the registry. In this case its not known if the scene should destroy "a" when its destroyed itself or if the registry would do it. Also it might be that the user wants a longer lifetime for "a" than "s" has.
The user has a lot of ways to fuck up everything when creating an object with chapter scope. For example what happens when a chapter-scope object is created outside of any running chapter? What if the user by accident holds a reference to a chapter-object after the object has been destroyed?
Its annoying to always have to write "chapter" or "application" for each object you create.

Possible solution 1: Register everything

Just force everything to be in the registry for example by giving the user no way to create an object without registering it. Especially when objects internally create other objects, put them in the registry too, "inheriting" scope from the parent object. Then no object would ever use delete, that would completely be Registry s job.

Problem: This still allows the user to reference objects beyond their lifetime and to produce similar fuckups. And its still annoying to always tell the scope when creating an object.

Possible solution 2: Register only certain object types

We could only register Scene s and let them have ownership for things like actors etc... However there is no guarantee that the user doesn't try to use the same image in two different scenes and there we have the double free corruption again. So this one is really a no-go

Possible solution 3: Use reference counting

Build a simple reference counting mechanism so each object knows how many places reference to it. This would mostly make the registry unecessary, except maybe for holding scenes for the current chapter and giving names to objects for the user. The only problem with reference counting is that it can't handle reference circles well (ie A has a pointer to B and B has a pointer to A), so I have to think about the whole structure a little more to guarantee there won't be such circles (although I'm already pretty sure there won't).

Below is a little examplary code. I didn't even try to compile it so forgive me if its not correct, it should just illustrate the idea.

class ReferenceCounted {
  public:
    unsigned int referenceCount;

    ReferenceCounted() : referenceCount(0) { }
};

template <typename T>
class Reference {
    T* obj;

    // Forbid copying of references
    Reference(const Reference<T>& other) { }
    Reference<T> operator=(const Reference<T>& other) { }

  public:
    Reference() : obj(0) { }

    Reference(T* obj) : obj(obj) {
      (obj->referenceCount)++;
    }

    virtual ~Reference() {
      if(obj) {
        (obj->referenceCount)--;
        if(obj->referenceCount == 0) {
          delete obj;
        }
      }
    }

    void ref(T* o) {
      if(obj) {
        (obj->referenceCount)--;
        if(obj->referenceCount == 0) {
          delete obj;
        }
      }
      obj = o;
      (obj->referenceCount)++;
    }

    T& operator*() { return *obj; }
    T* operator->() { return obj; }
};

// Usage:

class Foo : public ReferenceCounted {
  public:
    void foo() { ... }
}

void f() {
  Reference<Foo> r;

  r.ref(new Foo());
  r->foo();

} // "r" is removed from the stack here
// that reduces the reference count for the Foo
// object and thus deletes it! Nice isn't it?

Compiles under Windows!

2009-12-05T10:48:00

BAM reported yesterday that he managed to make Grail compile under windows, good work BAM :)

Gas gas gas

2009-12-04T19:59:00

I have been a little lazy regarding Grail in the past week (8 days without a commit! Thats near a desaster!). Ill accelerate development as good as I can in the next time, promised.

To give you an update, currently a friend of mine which I'll only call "BAM" here is working on compiling grail under windows with CMake and MingW. I wish I had more such tasks that one can delegate a little but except for the build system currently I don't think there is much to do than port the ole Indiana code. I think that'll change once all classes are basically ported so there is a structure with only holes that need to be filled.

By the way we already have an animated sprite that walks around depending on where you click, so there is at least something already ;)

Stay tuned!

What kind of GUI-Editor for grail?

2009-11-27T14:13:00

I'm currently thinking about how the future GUI for grail will look. Basically there are 2 approaches that I would say are worth considering at the moment:

Step-by-step fix Atlantis to become the new grail editor. The first step would be to have it generate lua code instead of D code, in the future it would be extended with a lua editor component etc...
As grail is basically starting from scratch this would be the perfect opportunity to include some editor capabilities directly into grail.
- (-) There would never be a code-editing component, I would keep the additional GUI elements to a bare minimum. I guess the control would be over a one-line shell-like command line at the bottom where you enter commands like "create polygon" or something. No checkboxes, no dialogs.
- (+) Having the editor directly combined with the engine allows some interesting things like running parts of the game while working on it. Although I must confess this point is very weak: Due to the lua scripting engine one could realise similar behaviour by sending lua commands to the runtime via a network socket or similar approaches.
- (+) Another good thing would be: Having a command line and the capabilities for displaying polygons etc... inside the engine allows for advanced debugging utilities, that could be really cool.
- (-) Probably this approach would mean more work :(

Comments welcome!

"Relaunch"

2009-11-21T14:24:00

I dunno why but I hate the word "relaunch". Maybe it's because it makes more fuss about the thing than its worth it.

Nevertheless, you might have already noticed some of the changes. To sum up:

Everything on the page except for the comment function is now static html, ie I post with creating text files in my git repository and running a html generation script, which I think is very cool/geeky.
I tried a new style, hope this one looks a little more serious/professional/friendly.
I cleaned up a bit, since I'm now moving towards "Grail" instead of "Indiana" (see previous post). So things like the indiana screenshots etc... are missing now partly because of lazyness and partly for a reason: I want to avoid yielding more confusion about Grail vs. Indiana then necessary. Some blog posts are missing too because I was to lazy to convert them, if you really miss something lemme know, but I think the most important ones should be there.

So that's it for now, stay tuned!

The thing with D

2009-11-21T10:31:00

Suddenly...

Some of you might already know, I experienced some severe/general problems with D compilers lately, namingly I ran into a situation where I was not able to find a D compiler(version) that would have no compiler bugs that influenced indiana and would run with phobos/d1. Another constraint was that we need one compiler on windows and one on linux (preferrably x86_64), both naturally being able to compile & link the same code.

I wont get in any further detail here, I just say: I really tried for about 2 full days and at least 8 compiler versions on 3 different platforms and I came to the conclusion that these were my options:

An old version of gdc I used until then had only one compiler bug that nulled a pointer in some circumstances, rendering "only" the serialization module (and thus the whole savegame system) unusable. I could have tried to stick with the old version and ship the code around the bug.
llvmdc would have been interesting to try, unfortunately it doesn't seem to support phobos, so for this maybe-solution a switch to tango or at least tangobos would have been necessary.
Take the newest available gdc/dmd and fix indiana that seemed to actually lay behind the current D1.0 "stable" language definition so the compilers would work with it. Note that the most current gdc releases (from sourceforge/goshaw) both couldnt even build phobos correctly (at least that day).
Do a complete rewrite in a different language with more stable toolchain.

I decided for the last one for a lot of reasons:

The incident that GDC didn't receive updates for at least a year (or so) while DMD did shows that the D community is not yet "mature"/"stable"/"big" whatever you wanna call it. I wouldn't call the D community "unreliable", but at least for gdc you never know what you get.
A complete rewrite offers some interesting opportunities of restructuring the code, kicking some old mistakes that werent worth the effort to do it up to now.
When I started with indiana in D, I didn't plan the lua part from the beginning, now when rewriting I can keep it in mind and/or do it on the way.

So which language did I choose?

Well it had to be a language that runs at least on windows and linux, allowed relatively easy usage of SDL and lua on those platforms, could somehow produce binaries for easy distribution on windows and needless to say that I'm now looking for a big community.

This, and my knowledge of languages in mind (learning a complete new one would cost lots of additional time), there weren't many options:

C#/other .NET stuff, possible, but I'm really not experienced with it, nor do I have knowledge about how reliable the linux toolchain is, how easy you can bind to SDL etc...
Java: No wai. For a "dynamic" language the syntax is way too clumsy, distribution of small binaries isnt possible (at least not that I know of), performance can be a problem. Plus a personal dislike + lacking experience ;)
Python: Confessed, I like python. Naturally with python I wouldn't have built a lua interface, I'd just have the games be written in python. The windows executable would be made with py2exe, which afaik just stuffs the complete python interpreter and all libraries into one big exe file. All in all, a good candidate, development speed has been always fast in python (at least for me), but definetely we would run into performance problems sooner or later and thus would need to have some parts in C/pyrex/whatever.
C++: There you are. Long time I ran around and told people: The only reason for choosing C++ over D is popularity/interfacing with existing code. I still believe so (except for some little language parts maybe), but this is all about community, so no need to lie to myself here. Also it might not be too hard to find a few helpers when I feel I need them.

So what happens next?

At the moment I'm coding what I will call "grail" (you know, the holy grail from indiana jones III / last crusade) in C++ right along with lua support. I'm not sure yet what I will do about the gui part (ie if I just alter atlantis to generate lua-code, if I extend it to be a more full-fledged editor, or if I build something new).

But one way or another in the end there will be

libgrail - Similar to what indiana is now, just in C++. This will theoretically allow you to build your adventure games directly in C++ without using/linking lua at all, although regarding documentation and "making it easy/fun to use", I'll focus on the runtime first as I think most game developers will rather use lua than C++.
grail-runtime - A binary application that takes a game as input and runs it. Where a game is a directory full of lua code, graphics, sounds, etc... Naturally later on it will be possible to stuff the game into a zipfile and append it to the grail-runtime so you can distribute you game in a single binary. Also I'm thinking about an open lua interface that developers can enable, so you (or the gui) can control the engine while it is running allowing cool debugging/testing things.
The GUI, as said, I'm not sure yet what it will be. Maybe not much more than atlantis is now (for first at least), just that it generates lua code. Maybe a full-fledged visionaire/wintermute - like editing system that does most of the work you have to do.

Later, I'll put up my grail git directory, don't judge me for the code yet, it has been a while since I did real c++ coding. And naturally at this state you can't expect something to look at really. When I have it that far that a character can walk around again, I'll post some screenies and inform you!

It's a trap - Multilingual web applications

2009-11-21T10:26:00

Say you're designing a multilingual web application. Where would you put the language information?

Carry it over with each request (generate hidden inputs)
Into the users session
Into the URL

The first approach is obviously quite tedious and more or less just included for completeness. No doubt you don't wanna do this except for a really small app or some rather unusual constraints (no cookies, framework that has hard constraints on URL scheme).

The second one looks like a really good idea:

You don't need to find out the users language with each request with url parsing or similar things,
You can freely choose your URLs after whatever scheme you like, language doesn't have to be a part of it.
You URLs can be exchanged between users of different languages, they will be displayed in each users favourite language to them.

So wheres the TRAP? Here a hint: Think of search engines.

Still don't see it?

Well took me some time, too. The problem is: If a search engine from some country indexes your page its bot will most probably not emulate/have cookie support. So all search engines in all countries will see and thus index the page in its default language (that one that appears when a user has no cookies enabled). This way the search engines won't find your site when the users enter keywords in their own language. What a crap.

So the answer is obvious: Put that information into your url. Or if you want detect it from the domain name (if you have one domain for each country/language <- distinguish carefully btw!)

It's a trap - Python object construction / reference semantics

2009-11-21T10:21:00

I guess this is a common trap. I personally am python coder for years now and still, from time to time I run into it. As the syntax for it is very concise you might sometimes be tempted to give default values for members in a classes definition. Code says more then thousand words. This works fine:

class Foo(object):
  x = 7

f1 = Foo()
f2 = Foo()

f2.x = 42

print f1.x # prints 7
print f2.x # prints 42

Now say we have something else for x, say it will contain a list most of the time and thus by default it should be the empty list:

class Foo(object):
  x = []

  def append(self, y): self.x.append(y)

f1 = Foo()
f2 = Foo()

f1.append("Take evasive action!")

print f1.x # prints ["Take evasive action!"]
print f2.x # prints ["Take evasive action!"] <- WTF?

The somewhat experienced programmer immediately sees what happens. Let's look at this step by step.

What does python do when you access f1.x?

It first looks if the object has a member x, if it can not find any, it accesses the class member x.

So what exactly happens in the first example?

Directly after construction the objects f1 and f2 point both to Foo.x that is at the same x. Thes with f2.x = 42 we add a member x to the object f2 so that we have a situation like the following:

Foo.x points to the value 7
f1 does not have a member x, so access to f1.x will be redirected to Foo.x
f2.x points to the value 42

Note that you never "change" an integer in python, you can only reference a different integer, ie integers are immutable.

What exactly happens in the second example?

As in the first example, after construction, both f1 and f2 don't even have a member x, they just redirect to Foo.x. Whats different here is that in this example we don't ever create an x-member in one of the objects!

Instead of integers, lists are mutable. In our append call, we actually change Foo.x instead of creating a new member. As both f1 and f2 still pass through to Foo.x, we see the change on both objects.

Yet another trap!

Now you might think "Okay, then I do it like this:"

class Foo(object):
  def __init__(self, x=[]):
    self.x = x

f1 = Foo()
f2 = Foo()

But this is basically the same problem, with a slightly different background. The default value for x you type there is only created once. So what you do here would be creating members f1.x and f2.x but still having them point to the same object which leads to the same unwanted result.

So what should I do?

Don't use this as a way for giving default values for object members. Instead do something like:

class Foo(object):
  def __init__(self, **kwargs):
    self.x = kwargs.get('x', [])

This way you have the default value defined inside __init__, which guarantees that each defaultly-initialized object gets a different "[]".

Depending on your actual needs you might also do something like:

class Foo(object):
  def __init__(self, x=None):
    self.x = x
    if self.x is None: self.x = []

Or you could copy every received value:

class Foo(object):
  def __init__(self, x=[]):
    self.x = x[:] # assumes, passed x is always a list!

For those of you that are still very new to python: Try to always keep one eye on what references what. Be aware that everything in python has reference semantics although for immutable values that can't lead to this problem (and in lots of situations this "feels" like value semantic). Also note that my approaches are far from perfect too. And: Don't use any of that code if you do not fully understand it, play around with everything in the python interpreter, you can learn a lot of things there.

It's a trap!

2009-11-21T10:16:00

I'm hereby creating a new kind of postings on this site: I want to post about common programming traps, you know that kind of stuff you think at first "Obviously it works this way" and later on you feel... trapped!

For general reference to this topic see:

Admiral Ackbar, Sheldon and/or Captain Picard

Traps ahead, be prepared!