Leetless Coding Blog

Path Finding in Grail

2012-07-15T00:22:00

A topic thats using up my (rare) Grail-time lately is the topic of how to define where an actor is allowed to walk and how it will find its way there. I found this question quite fun to work on, as it poses an interesting mix of theoretical (geometry) and practical (coding) considerations.

So what is the problem exactly? The answer is: A combination of things:

Restrict the actors movement to a certain area that will be defined by the game programmer, I call this the walkable area.
Come up with a mechanism that allows the actor to walk from one point in the walkable area to another one. Thereby the actor should stay in the walkable area at all times, so a path inside the area has to be found.
When the user requests to walk to a point outside the walkable area, the actor should instead walk to some near point inside the walkable area, whereas near is not exactly defined.

It turned out I had to go through a number of idea-implementation iterations before coming up with something that works really well. Let me give you a short intro on what we did with respect to the first two points:

Discarded approaches (& why)

Pixels

A pretty intuitive idea to describe areas in a 2D game is always by just noting the pixels which belong to the area and those who don't (for example you could save them as a black/white image). This approach has the nice practical advantage that its awesomely easy to find out for any given point whether or not it is in the walkable area. There is of course no principal necessity of those pixels to be the same size as the actual displayed pixels.

But how large should those pixels be? If we choose the pixel size too big, the walkable area will have stair-shaped edges and, even worse, walking along these blocks will look awkward as the actor will seemingly walk a zig-zag style when being requested to walk angular lines. If we choose smaller pixels however, path finding inside the walkable area will be drastically more expensive (even though, techniques like A* are applicable).

So even though there might be a sweet-spot for some situations it always feels a little dodgy working with these virtual pixels. After all, one of the aims of Grail is to provide an adventure game engine that scales well with the users screen resolution.

Walls

The next idea that came to mind was: Just define a number of walls (in forms of lines) the actor is not allowed to cross. If we just ensure the walls define a closed shape (e.g. by obtaining them from the edges of a polygon), there should be no chance of escape for the actor. Indeed, that is the case, at least if you have a keen eye on the corner cases of line intersection calculations.

However, path finding inside the walkable area again is a little more problematic. Take for example the picture to the left: The shortest path inside the walkable area that connects the two given points (red) goes directly along some of the edges of the polygon. That implies, that if we send the actor along that path it will walk directly along some of the walls. In the case shown here, where the target point is well inside the polygon that isn't a big problem: We can just "turn off" the wall-crossing detection for as long as the actor walks its path and turn it back on afterwards.

The problem occurs however when the target point lies on an edge or a corner of our polygon or the walk is interrupted halfway through because the user meanwhile chose a different target: In that case the actor will stop on a point directly on a wall and it will depend on rounding whether the character is considered to be inside or outside the walkable area. Worse, if the walkable area contains a hole (like P2 or P3 in the figures below), we might end up in a situation where we don't even know which of the two adjacent polygons (the hole and the surrounding polygon) is the walkable area the actor should return to.

Concluding, two things are missing in this approach:

A way to identify which polygons actually belong to the walkable area and which not to avoid getting trapped outside of the walkable area if we leave it for some reason
For positions near the border of the walkable area we also need a mechanism that can bring back the actor to a point that is guaranteed to be inside the WA.

The working approach

Components

In order to tackle the first problem, we define the walkable area by a set of nested polygons. As those polygons are never overlapping we can represent them by a tree in which polygon A is a child of polygon B if it is contained in B completely. See for example the walkable area figure and the according tree. This way, we can easily check whether or not any polygon is to be considered walkable area: The outermost polygon (= root of the tree) is assumed to be walkable, everything outside of that of course is assumed not to be walkable.

Any nested polygons inside the outermost polygon are considered holes, i.e. not walkable (see P2 and P3). Polygons inside holes are again walkable areas, and so on. In the tree view that means: If the root is considered to be at level 0, all polygons on even levels are walkable and all uneven are not.

Where am I?

For actually finding a path given this tree of polygons, we first have to find out, in which polygon of the tree we actually are. Assume, the actor is located at the red dot in the figure. We lay a horizontal line through that position and count the number of intersections with all polygons. If the line intersects a polygon an uneven number of times on both sides, we can be sure that the point is located in the interior of that polygon. However, as there are holes, it might also be in a nested polygon. Given our polygon tree we thus conduct a binary search for the innermost polygon that contains the given point (see figure, here P4).

Having found the polygon we are navigating in, we create a map (as shown in the figure in Components for P1), that connects all area corners that can reach each other. We also connect the start and target positions accordingly (not shown in the picture). Based on that map we can conduct a Dijkstra search for the shortest path [1].

Don't you escape!

That still leaves one problem open: What do we do when the actor is located slightly outside of a walkable polygon (e.g. in P3)? (See above for when&why this might happen). As we know we are only slightly off and Grail is designed to work with discrete (although more fine granular than screen resolution) positions, we can basically just check all of the near positions until we found one that is located inside the walkable area and gently shift the actor there.

We do this by simply checking the positions on a spiral around the actors current position until we found a position that is located inside a walkable area. To avoid a very long or even endless search in case the application programmer deliberately put us into non-walkable area, we limit our efforts to a certain amount of points that are to be tested.

A possible optimization (that is not yet implemented but should be a total no-brainer), is to not check every virtual pixel, but make larger steps (e.g. 10 virtual pixels) on the spiral to save computing time (on my machine the delay for this mechanism is not noticeable even with a pixel-wise spiral though).

Conclusion

Seems this post has become one of the somewhat longer ones, hope you enjoyed it anyway and got a little insight on what is currently happening in grail development, stay tuned!

[1]	Theoretically it might be possible to use A* or similar approaches here to speed up the search for a lot of nodes, however I doubt any game will ever construct maps complex enough that Dijkstra will be considered too slow (if it is at one point, we can still easily replace Dijkstra with A*).

Coming up with a Backup/Synchronization Strategy: Subgoals and Tools

2012-05-20T21:48:00

In my previous post I stated a little what I would want/need of a nice backup/synchronization strategy. Now lets structure these into more precisely defined features and see which programs out there cover them. As a disclaimer upfront: Of course I only know a certain subset of backup software. If you have a noteworthy candidate in mind that I didn't mention, please don't hesitate to comment.

Features of backup/synchronization solutions

Tool	GUI	Versioning	Auto-Add	Incremental Storage	Rotation	Direct Access	Crypto
Duplicity					?
rsnapshot				(file-level)	Count
rdiff-backup					?	(only latest)
bup		, Git	?		N/A	?	?
Back In Time				(file-level)	Various conditions
Unison					N/A	N/A
Git					N/A		N/A
Git-Annex	?			N/A	N/A	N/A
Dropbox, Ubuntu One, etc...		, auto		?	?	?	?

GUI: Tool comes with a graphical user interface.
2-Way: Tool allows 2 (or more)-Way sync between two machines, i.e.: Changes happened on both sides, and $tool is able to bring both sides to the same state somehow (possibly with some user interaction).
Versioning: This term refers to the usual commit/merge mechanics of revision control systems. I.e.: The system at hand will be able to record diffent states of a number of files and allow going back to any of them. Additionally it will track a hierarchy of these states that allow branches and all that good jazz. Usually those systems allow for easy merging of text-based files one way or the other.
Auto-add: Files in a certain directory will be automatically tracked, this works not only for editing but also for adding/deleting files. Useful if file changes are happening very often and informing the system manually is considered a tedious task. Note: No arguing here that "Auto-add" is not always something desirable (e.g. there are good reasons why this "feature" is not enabled/available by default for revision control systems usually).
Incremental Storage: Backups based on older backups avoid storing data redundantly (i.e. re-using the data that is common for multiple versions of the backup)
Rotation: For tools that do incremental backups this describes how rotation of backups is done. I.e. will it keep the latest $count backups, or keep backups until the harddisk is full, etc...
Direct Access: For systems with incremental storage it sometimes is the case that data must be compiled from various diff's before it can be accessed. With "direct access" I mean: The system allows accessing older versions of the data easily without any time-intensive precomputation.
Meta-Sync: Some systems (here: one) allow to communicate not all the files or patches between the files directly but only the information about which file is where in which state and then giving the user a way to synchronize certain file version from remote on-demand, I call this feature "meta-sync" here for brevity.
Crypto: Does the tool (by itself) support encrypted storage? Note: Even if not for most tools it should of course be possible to use an encrypted filesystem or similar underneath.

Conclusions

Back In Time seems to be a promising candidate for handling protection against dumb things as well as protection against hardware failures. What I especially like is that there is a notification in the systray whenever it does something and the various ways of specificing conditions for when to remove old backups. This way the tool is (I assume) much less annoying in terms of sudden surprise i/o, full harddisks or unnecessarily rotated backups (than e.g. rsnapshot). I'll definitely give this one a try :)
2-Way syncing, especially concerning the MP3-problem mentioned in the previous post is still an enigma to me. I found no tool that would handle MP3s in a special/smart way. E.g.: I rename a MP3 file and change its id3 tags, the tool should handle that as a rename only, i.e. I don't want to end up with 2 versions of the same file at the other end, neither do I want to have to explicetely tell the tool what happened.
Independently of the MP3 problem, concerning 2-Way sync I'm still not sure which approach is best: Unison tends to get slow which lots of data, the Git-based approaches seem to all require too much manual intervention. And I don't like the idea of syncing my private or semi-private data through a commercial third-party hoster. I found a list of Dropbox alternatives though, which sounds promising (especially the git-based tool SparkleShare), maybe I'll give that approach a try.
For syncing config-files I'll definitely stay with git for the time being, as for that use case I really want the safety and expliciteness of git.

EDIT: It turned out BackInTimes GUI just tricked me into believing it can do remote backups, it seems that is not the case. I'll continue my search a little and keep you guys posted when I have found something cool.

Coming up with a Backup/Synchronization Strategy: Problem Statement

2012-04-16T17:18:00

For years now I'm constantly changing my backup plans, always a little unsatisfied with the current solution. So let me try to write down what I actually want first and then see what options are there. I know there are plenty of tools to deal with those (I know e.g. unison, git for config files, bup, etc..), the thing is: Without a little planning the best tool won't help anything.

Synchronization of config files. I.e. dotfiles in my home directory. As I work on multiple computers and regularly change things, this is a quite essential element. Versioning/diff-/mergemechanics would be nice, but is not a must. The mechanism used here should not automatically include all sorts of files as some are machine-specific and beyond the dot-directories there are lots of caches etc... which I don't want to explicetely exclude, rather I'd like to include a certain subset of my config files for syncing.

Synchronization of data files. I.e. presentations, wallpapers, papers, all that stuff. No need to do versioning here, as I'll practically never want to go back to an earlier version of those files (as I more or less horde them anyway). It will quite often happen though that these files move or are being renamed (as they are sorted automatically). For those files, everything under a certain subdirectory should always be included in the syncing process, manually adding things here would get quite tedious.

Synchronization of music. This is a tough one as my music collection is quite large (e.g. its infeasible to sync with unison). To make things harder: My music player automatically sorts my music by artist, etc... and these change from time to time as I discover some files that are not or not correctly tagged. Having to tel a synchronization program that a file has moved and changed every time i correct a track name in my player is certainly not a thing I want to do. On the other hand I don't want/need the same state on all machines (it probably wouldn't hurt, but my internet connection at home is not the fastest). I wouldn't mind carrying a usb disk back and forth between home/work for syncing my music, however I do not want to manually select files for syncing, a high degree of automatism is a must here.

Protection against dumb things. Especially premature deletions of files. This even holds for checked-out repositories in the ideal case (when deleting a changed file accidentially that is not yet checked in). Ideally a mechanism to achieve this would allow going back a selection of time intervals (e.g. like rsnapshot does). However: Say I'm on vacation for the longest time interval and come back, then discover I accidentially deleted file X the day before I left, but in the oldest backup version its still gone. So the solution here should not do anything if files didn't change.

Protection against hardware failures. Assume my laptop explodes and the harddisk is completely destroyed. In this scenario there should be a backup somewhere off-site that is not more than a week old. However I don't want an automated network sync that I can't see or immediately abort (I might be sitting in a train and be connected via GSM or simple need every bit of the bandwidth for something urgent).

The sum of the solutions should be maintainable. I.e. it would be nice not to have 5 different tools running for solving these problems.

In my next post about this topic I'll list a few tools I know/use up to now and maybe say a few words on how well they solve some of these problems.

Quick Update

2012-02-14T11:20:00

Hey guys, as I should be working at this time of the day, I'll keep this update very brief.

At beginning of the month I was at the FOSDEM 2012 which was quite interesting. Beyond others I heard a talk about Data-Driven and Component-Based Game-Entities which was quite interesting
As you might have noticed, there is some activity recently on github. Thanks to feedelli who started working on the dialog system, we now have a plan for 0.1
There where some problems for I don't know how long with commenting on articles, it should work again now, if not please drop me a mail.

Tiny Fonts

2011-08-21T21:25:00

Most recently, I fell for the idea of having a real tiny font for my working environment. Some observations immediately popping up were:

Almost unecessary to mention that only monospaced fonts are candidates.
Pixel-oriented / Bitmapped fonts have the advantage of using space more efficently while still being easy to read (because there is no need for antialiasing so things don't get "blurry").
Especially with such small fonts it is important to be able to easily distinguish letters, eg. lowercase "L" vs. capital "I" vs. the number "1".
I want the same font in my URxvt terminal emulator as in GVim. That is noteworthy as I experienced some fonts didn't seem to work equally well in both these environments (especially considering the following point).
I need normal and bold font weight in GVim as well as in URxvt, and of course it should still be readable and not look "blurry".

Before I bore you to death with a detailed story of my findings, here is a summary of my results. The screenshots are a GVim with the Pyte colorscheme as this uses as well bold as italic font styles. Note the different sizes of the screenshots resulting from the different space usage of the fonts.

Don't spoil yourself by jumping to the Conclusion ;)

Fonts

DejaVu Sans Mono 7

Antialiased font, still astonishingly clear and readable
Very small, thus does not come with bold variant

Edit:

I advertised this as "FreeMono" in the first version of this article, the correct name of the font is "DejaVu Sans Mono" though, probably available for your distro.

DejaVu Sans Mono 8

A surprising amount larger than DejaVu Sans Mono 7
Looks very nice, but wastes pixels with antialiasing

Fixed 8

Should be available in most distros
Letter "O" and number "0" too similar, captial lettel "I" and lowercase letter "l" too

Fixed 9

Letter "O" and number "0" too similar, captial lettel "I" and lowercase letter "l" too (similar to size=8 case)

Fixed 10

Too large for my taste

Gohu

URL: http://font.gohu.eu

Captial letters are a little higher than in Montecarlo.
"D" and "O" look very similar:

Erus

URL: http://aur.archlinux.org/packages.php?ID=37946

Only a slight variation on Gohu

Montecarlo

URL: http://www.bok.net/MonteCarlo/

The "l" (ell) looks a little weird
Lowercase "A" and lowercase "O" look very similar
Less squilles than Gohu/Erus, thus looks clearer to the eye

Proggyfonts

URL: http://www.proggyfonts.com/

This is a nice collection of bitmap fonts for programming and similar uses, I'll only look into a few of them here, check out their site for more.

Note that I only look at the true type versions of the fonts here, as I had problems with getting the bitmap versions with bold typeface support running in GVim (not sure if proggys or archlinux's/packagers or maybe even my own fault though).

ProggyClean

The bold parts look ugly/blurred, thats because the font engine generates the bold font from the truetype font. As noted above, this probably is only a configuration or packaging issue, with bitmap versions (tested in urxvt) it looks fine.
I like the style of proggy, just before deciding to go for a really small font, I used this one for coding, for what I want now, this is too large.

ProggySquare

Not much to say about this, a little smaller than proggyclean

ProggyTiny

A very good candidate, especially very clean (similar to MonteCarlo)
The bold-issue was a no-go though and I didn't care to try very hard to fix it yet (if somebody has, please comment)

Terminus

Didn't get bold version to work at all :(
A little larger (too large for what I'm looking for currently)
Very readable, clean font though, although "I", "l" and "1" could be a little more different:

Conclusion

Currently, I prefer using Gohufont, as the D/O problem rarely matters (in contrast to the a/o problem with montecarlo). There are some interesting alternatives listed here which would be worth trying for a longer time too. And of course theres a lots more such fonts on the web which I didn't test here.

Twitter feed

2011-06-29T18:28:00

There is now an "inofficial" twitter bot for this blog in order to keep you updated on my (far too few) postings. Thanks dear anonymous creator ;)

Background Music

2011-05-09T20:02:00

Background Music

As I'm still mostly focusing on my job currently rather than coding for these spare time projects, I still cannot announce any remarkable progress on those. However I've come to finally find some cool music that is suited for programming/working (I've searched for that for like ever). Here is a list for you:

Some (little) distraction, cool sound

Beats Antique - (Mostly) text/voice-less, draws some (little) attention to the listener, a slight oriental touch, some very cool songs
Daft Punk in general
The tron legacy soundtrack by Daft Punk - Electronic, nearly distractionless
Dub FX - Contains text/voice but still suited for background, very nice beat/sound (try Love Someone or Society Gates)
RMB - Mostly a little heavy and towards rave for working, although e.g. Unreality is perfect for that task
The Animatrix soundtrack
The Starcraft II Soundtrack
http://kohina.com - Chiptunes radio stream, I love the sound but can be pretty distracting after some time
last-fm tag chiptunes - See kohina above
http://feeds.feedburner.com/GroovetekElectronicaMusic - Electronic music podcast
Muse - The cool songs are somewhat distracting, but I think not too much
last-fm tag the presets
last-fm artist boards of canada nice not-too-distracting electronic music

Chilly/Soft ambient music, zero distraction

Blind Foley
Brian Eno
Portishead - With text, but very chilly, beautiful voice
http://cloudcycle.net/log/feed/podcast - Chillout music podcast
last-fm tag lounge

Varying degree of distraction (from song to song)

Maxstack (Singularity Soundtrack) - Cool electronic background music
Pink floyd - Degree of distraction actually can vary within a song very much, you have to find the right songs. Echoes and Shine on you crazy diamond are cool for example.
The Portal 1 and Portal 2 Soundtracks - Most of those are pretty not-distracting, although naturally there are phases in the game when the soundtrack tries to support some more action
http://di.fm - Internet radio with stations of differing degree of electronicness and distractionlessness
http://www.stereomood.com/ - HTML5-browser playlist-based radio with stations sorted by the mood/activity of the music (mostly chilly)

Some general notes:

Its not easy to categorize music like this on the fly, so don't rely on those captions too hard
I found (included in the gtk2 based music player banshee - which is very cool btw.), http://www.miroguide.com/ which is an index of music and video podcasts (also the source of the podcasts I presented here), definitely worth a look if you are looking for some special kind of music
All radios where I didn't say anything else are real streams and thus can be played with mplayer/vlc/almost any music player

Some status updates

Scala

Oh and btw. I already played around a little with scala (and read a good part of the book) - The impression so far: Its a really cool language, however seems the amount of work you save by not having to worry about threading and locks (when coding the immutable style) is just about as much as the amount of work you need to invest when designing your stuff in an immutable style. However that was one single (simulation-) application I coded, so don't put too much weight in these early impressions yet, more to come.

Grail

I hope to squeeze in a little more work on grail into my spare time in the future, however the job always has precedence. As always, if you care to contribute, you are more than welcome.

The hunt for the perfect programming language (ppl)

2010-09-11T16:07:00

Basically since I started programming, I'm kinda looking for a programming language that is perfect in a way. First problem with that is that I don't even have a precise definition of what perfect in this context exactly means, but it should go along those lines (in no particular order):

Type system

Take popular object oriented programming languages like Java or C++. Why on earth are "primitive" types treated so way differently than object oriented types? Why would a number not be an object? In most popular programming languages its also clumsy to really abstract types. E.g. in C++ you can use Templates to express things like "This is a list of arbitrary element type" but its lots harder to express things like "If B is a subtype of A than a List<B> is a subtype of a List<A>". (We will come to programming languages which allow such things and much more)

Community support

Not necessarily as huge as that of C++ or alike but productive, helpful and able to fix bugs and port compilers/runtime environments. Naturally this includes the availability of libraries for all kind of stuff. I had this problem with D (a programming language I really like for lots of other aspects) where there was a window when there would be no D-compiler which compiled my (according to the spec correct) code on my 64 bit Linux. Not to mention that compiling under Windows was somewhat inconvenient too when linking to lots of different libraries.

Multiprocessing

Note that I didn't say "multithreading". Multithreading is fine but I really don't want to worry about complicated locking mechanisms for every damn variably access like a caveman. A lot more scalable idea is to spawn multiple processes (e.g. one per CPU) and have a good way of communicating between them. If this is implemented nicely, you can even spawn the processes on different machines and have communication between them run via network. This is really what you want when you code a scalable app. Again: Not to say that threads would be useless or so in any way, they have their uses and places, but I want a language that can do multiprocessing (or something similar).

Not too verbose

And/or has absurd restrictions. E.g. write a hello world program in Java and compare the result to a minimal hello world program in python. Point taken, most of your programs will be more complex than hello world anyway, but you get the picture.

Not too many ways to do the same thing

Imagine a programming language that allows to write the same simple thing in a functional style or an imperative object oriented style or a logical style or whatever and its not at all obvious which one is better in this situation. What you end up with is either only one style prevails or you have very mixed programs and projects that are hard to understand. Naturally this is only sensible to a certain degree: Its OK (and for most programming models unavoidable anyway) to be able to write the factor function recursively or with an iterative loop. But having functional variants, structs/records, and object oriented classes as different concepts that do basically the same thing is redundant.

Cool syntax

Obviously we're now deep in the matter-of-taste region, but anyway: I never tried LISP. Why? Not because its so old or whatever, just because I was afraid of the ugly syntax. I'd really like something that is in between the C++'ish brace-style and the python'ish indentation style, i.e. a programming language that will understand your indentation but allows braces or whatever for clarification. Needless to say the syntax should be at least decidable so people have a chance to write usable tools for it.

No preprocessor

At least not in the C++ sense that doesn't understand the syntax at all. Syntactic macros are OK, but in general the language itself should be flexible enough to just let you do whatever you want without that additional step.

Performance

In terms of short run times of the programs, low memory footprint, etc...) is naturally desirable. Not for every application but the language wouldn't be "perfect" if it would produce darn slow programs.

Thats what I can think of off the top of my head right now. I'll write more about this topic in the future. E.g. I'll try to take a look at programming languages like Scala, Ocaml, Python, Dylan and some others. I don't know yet if I will really say anything about C++ and/or Java as you probably know enough about them anyway, but we'll see.

Keepalive

2010-09-07T09:52:00

Hey guys.

I know it was very quiet here in the past 2 months. Good news for me is that I really like my new job. Bad news for grail is that I don't have much time for it at the moment. That does not mean its dead though. For my part, the commit rate hos "only" slowed down remarkably.

zdctb also mentioned that he might have some more time to work on grail in the near future. However we'd prefer a more regular stream of updates. So if you're a C++ developer who always wanted to participate in an adventure game engine (or you know of one), don't hesitate to contact us!

For the holy grail!

IRC Channel

2010-07-03T17:45:00

Just got over it in my previous post: There is an IRC channel for Grail on Freenode now, its called #grail (creative, eh?). Feel free to join there for any questions/suggestions regarding grail or any other stuff related to me or this site.

Site updates

2010-07-03T17:24:00

Hey guys. I'm glad to announce that I finished my diploma thesis a few weeks ago and am now working at the TU Braunschweig as PhD student.

As I'm not sure yet how the regulations regarding publication are, I will not publish my thesis here yet (probably its not of much interest to any usual leetless reader anyways).

There are also some changes in the VIM Section: I removed lots of the older themes in order to focus on the real good ones. Also I'm proud to present a new theme, called PhD, hopefully its useful to someone.

Another thing, which I neglected to post for months now is that the Giga Mansion 2 adventure game project is dead. They used the Indiana engine (the predecessor of Grail) which I developed in parallel to the game. At first I wanted to wait until they announce the end of the project themselves which seemingly never happened. Nevertheless it had been fun working with these guys and really pushed the adventure game engine stuff forwards a lot.

And while I'm at it: Because of the diploma thesis there was some pause, but I managed to include OpenGL into Grail, which makes it run about 10 times faster (at least on my machine). Check out the OpenGL Branch if you want to try it. If during compiling there is no OpenGL found, the engine will be compiled with plain old SDL, so this can not come to anyones disadvantage.

Going to github

2010-05-03T16:15:00

Currently data is being pushed to GitHub. I plan to let this run in parallel to the current self-hosted solution for a while and if it is good I'll shot down the self hosted git repository.

Reasons for github in favor of hosting the repository myself:

Wider audience
Integrated issue tracking system
My git web CSS is broken and I'm too lazy to fix it ;)

Check it out at The grail github repository.

Oh by the way, you may have noticed the latest commit is already a few days ago. That's because I'm currently very busy with university stuff. But don't you be afraid, I have no intent to give up this project. If you want to contribute though, it will probably advance somewhat faster :)

First Grail screenshot

2010-02-16T19:09:00

Whats new:

Parallax scrolling of background/foreground layers
Animations don't require annoying to produce "stripe" format anymore, additionaly theres a script now that extracts layers from an xcf (Gimp format) file into a directory so its useable by grail.

Edit: I substituted the .png with a .jpg so the page load times should be lots less annoying.

First steps

2010-02-02T22:13:00

Luabind is so cool. Wrapping really is the easy part now, so I finally focus on implementing stuff again. The main character already can walk around on the screen, now I'm concentrating on a user interface and then on interactions with other scene actors.

Thats sounds like the biggest part but keep in mind, theres still a lot to do:

User interface already is a big topic (buttons, inventory display, etc...)
Loading/Saving (will hopefully be easy due to boosts serialization lib)
Scene/walkpath stuff (the "algorithmic" corner yay)
Loading scenes directly from atlantis files
Dialogs (talking)
Sound/Music/Voice output
From time to time try to get the stuff to compile & run under windows ;)

Feed problems

2010-01-18T16:20:00

Sorry for the feed problems lately (wrong urls, complete feed gets pulled twice etc...). Should now be fixed.

In general: When you discover something doesn't work well/correctly, please report, I can only keep the site up with proper feedback.

Good bye NIH, hello luabind & boost!

2010-01-18T15:45:00

I have to confess something. To you all.

I am ill. What I have is called the NIH syndrome. For those of you who never heard of it: It means preferring to code stuff yourself instead of learning how to use a library somebody else wrote. So its all about reinventing the wheel. [1]

For example I was inventing a DSL for describing c++/lua bindings together with a handcrafted (and ugly) parser which generated c++ code from that DSL etc... although I knew there was luabind.

Now that I tried it, let me say: luabind rocks. Don't dare to invent something yourself until you tried that one.

Same goes for boost::shared_ptr. It is fun to invent a reference counting mechanism, but seriously: Use shared_ptr, its really cool, luabind supports it already and it simply makes your life so much easier.

I also plan to use/learn more parts of boost such as the serialization library which seems to be even more flexible than the serialization stuff in D I wrote and was kinda proud of.

Stay tuned for more.

[1]

Note that this can be a good idea actually. Say you decide to use $big_library and it works quite well. You save valuable time in your early project phases by not having to write all that stuff yourself.

But then, as you project evolves you have more special needs and demands to that library. Depending on how its designed and what it is about you might have to actually patch the library which suddenly requires a huge amount of time, because you have to learn the internals of $big_library for just one little change.

libregamewiki.org linking

2009-12-12T18:57:00

Grail is now findable on the Libregamewiki thanks to BAM, so another reason for me to spend more time on grail in the future.

Somebody feel like making CSS?

2009-12-10T13:38:00

My current gitweb CSS looks terrible, the standard stuff doesn't fit very well into the rest of the page so I started this but I didn't really finish and now lost fun on it ;)

So if somebody feels like making the existing CSS more pretty/useable (or, if you prefer, creating a complete new set of CSS rules for the gitweb area) your help is very welcome, just drop me a mail.

A few thoughts on memory management in grail

2009-12-10T09:42:00

In D we had a garbage collector that made things really easy: Just allocate an object and never think about what happens to the memory of that object again, the gc does it for you. In C++ there is no garbage collector built in so you have to delete old objects yourself. That per se is not so much a problem as long as the question of ownership is always cleared.

Ownership means: Object a has a reference to object b and is responsible for deleting it. Normally with a little thought its easy to say which object should own which one(s). However, if you let the user allocate and play around with objects in lua, things can get a little more complicated.

Current non-solution

Currently I'm using a thingy I call "registry". This holds references to all objects allocated in lua and destroys them when they aren't useful anymore. To find out when that is, you tell it to the registry. There are two possible scopes: CHAPTER and APPLICATION. Objects with CHAPTER scope exist until the current chapter runs. This is most useful for scenes and the actors running around in them.

The APPLICATION scope holds objects as long as the program runs. This can be useful for the main menu or other user interface stuff.

However this approach now leads to some Problems:

Currently it can happen that a Scene "s" holds an actor "a" and its not in general known if "a" is in the registry. In this case its not known if the scene should destroy "a" when its destroyed itself or if the registry would do it. Also it might be that the user wants a longer lifetime for "a" than "s" has.
The user has a lot of ways to fuck up everything when creating an object with chapter scope. For example what happens when a chapter-scope object is created outside of any running chapter? What if the user by accident holds a reference to a chapter-object after the object has been destroyed?
Its annoying to always have to write "chapter" or "application" for each object you create.

Possible solution 1: Register everything

Just force everything to be in the registry for example by giving the user no way to create an object without registering it. Especially when objects internally create other objects, put them in the registry too, "inheriting" scope from the parent object. Then no object would ever use delete, that would completely be Registry s job.

Problem: This still allows the user to reference objects beyond their lifetime and to produce similar fuckups. And its still annoying to always tell the scope when creating an object.

Possible solution 2: Register only certain object types

We could only register Scene s and let them have ownership for things like actors etc... However there is no guarantee that the user doesn't try to use the same image in two different scenes and there we have the double free corruption again. So this one is really a no-go

Possible solution 3: Use reference counting

Build a simple reference counting mechanism so each object knows how many places reference to it. This would mostly make the registry unecessary, except maybe for holding scenes for the current chapter and giving names to objects for the user. The only problem with reference counting is that it can't handle reference circles well (ie A has a pointer to B and B has a pointer to A), so I have to think about the whole structure a little more to guarantee there won't be such circles (although I'm already pretty sure there won't).

Below is a little examplary code. I didn't even try to compile it so forgive me if its not correct, it should just illustrate the idea.

class ReferenceCounted {
  public:
    unsigned int referenceCount;

    ReferenceCounted() : referenceCount(0) { }
};

template <typename T>
class Reference {
    T* obj;

    // Forbid copying of references
    Reference(const Reference<T>& other) { }
    Reference<T> operator=(const Reference<T>& other) { }

  public:
    Reference() : obj(0) { }

    Reference(T* obj) : obj(obj) {
      (obj->referenceCount)++;
    }

    virtual ~Reference() {
      if(obj) {
        (obj->referenceCount)--;
        if(obj->referenceCount == 0) {
          delete obj;
        }
      }
    }

    void ref(T* o) {
      if(obj) {
        (obj->referenceCount)--;
        if(obj->referenceCount == 0) {
          delete obj;
        }
      }
      obj = o;
      (obj->referenceCount)++;
    }

    T& operator*() { return *obj; }
    T* operator->() { return obj; }
};

// Usage:

class Foo : public ReferenceCounted {
  public:
    void foo() { ... }
}

void f() {
  Reference<Foo> r;

  r.ref(new Foo());
  r->foo();

} // "r" is removed from the stack here
// that reduces the reference count for the Foo
// object and thus deletes it! Nice isn't it?

Compiles under Windows!

2009-12-05T10:48:00

BAM reported yesterday that he managed to make Grail compile under windows, good work BAM :)

Gas gas gas

2009-12-04T19:59:00

I have been a little lazy regarding Grail in the past week (8 days without a commit! Thats near a desaster!). Ill accelerate development as good as I can in the next time, promised.

To give you an update, currently a friend of mine which I'll only call "BAM" here is working on compiling grail under windows with CMake and MingW. I wish I had more such tasks that one can delegate a little but except for the build system currently I don't think there is much to do than port the ole Indiana code. I think that'll change once all classes are basically ported so there is a structure with only holes that need to be filled.

By the way we already have an animated sprite that walks around depending on where you click, so there is at least something already ;)

Stay tuned!

What kind of GUI-Editor for grail?

2009-11-27T14:13:00

I'm currently thinking about how the future GUI for grail will look. Basically there are 2 approaches that I would say are worth considering at the moment:

Step-by-step fix Atlantis to become the new grail editor. The first step would be to have it generate lua code instead of D code, in the future it would be extended with a lua editor component etc...
As grail is basically starting from scratch this would be the perfect opportunity to include some editor capabilities directly into grail.
- (-) There would never be a code-editing component, I would keep the additional GUI elements to a bare minimum. I guess the control would be over a one-line shell-like command line at the bottom where you enter commands like "create polygon" or something. No checkboxes, no dialogs.
- (+) Having the editor directly combined with the engine allows some interesting things like running parts of the game while working on it. Although I must confess this point is very weak: Due to the lua scripting engine one could realise similar behaviour by sending lua commands to the runtime via a network socket or similar approaches.
- (+) Another good thing would be: Having a command line and the capabilities for displaying polygons etc... inside the engine allows for advanced debugging utilities, that could be really cool.
- (-) Probably this approach would mean more work :(

Comments welcome!

"Relaunch"

2009-11-21T14:24:00

I dunno why but I hate the word "relaunch". Maybe it's because it makes more fuss about the thing than its worth it.

Nevertheless, you might have already noticed some of the changes. To sum up:

Everything on the page except for the comment function is now static html, ie I post with creating text files in my git repository and running a html generation script, which I think is very cool/geeky.
I tried a new style, hope this one looks a little more serious/professional/friendly.
I cleaned up a bit, since I'm now moving towards "Grail" instead of "Indiana" (see previous post). So things like the indiana screenshots etc... are missing now partly because of lazyness and partly for a reason: I want to avoid yielding more confusion about Grail vs. Indiana then necessary. Some blog posts are missing too because I was to lazy to convert them, if you really miss something lemme know, but I think the most important ones should be there.

So that's it for now, stay tuned!

The thing with D

2009-11-21T10:31:00

Suddenly...

Some of you might already know, I experienced some severe/general problems with D compilers lately, namingly I ran into a situation where I was not able to find a D compiler(version) that would have no compiler bugs that influenced indiana and would run with phobos/d1. Another constraint was that we need one compiler on windows and one on linux (preferrably x86_64), both naturally being able to compile & link the same code.

I wont get in any further detail here, I just say: I really tried for about 2 full days and at least 8 compiler versions on 3 different platforms and I came to the conclusion that these were my options:

An old version of gdc I used until then had only one compiler bug that nulled a pointer in some circumstances, rendering "only" the serialization module (and thus the whole savegame system) unusable. I could have tried to stick with the old version and ship the code around the bug.
llvmdc would have been interesting to try, unfortunately it doesn't seem to support phobos, so for this maybe-solution a switch to tango or at least tangobos would have been necessary.
Take the newest available gdc/dmd and fix indiana that seemed to actually lay behind the current D1.0 "stable" language definition so the compilers would work with it. Note that the most current gdc releases (from sourceforge/goshaw) both couldnt even build phobos correctly (at least that day).
Do a complete rewrite in a different language with more stable toolchain.

I decided for the last one for a lot of reasons:

The incident that GDC didn't receive updates for at least a year (or so) while DMD did shows that the D community is not yet "mature"/"stable"/"big" whatever you wanna call it. I wouldn't call the D community "unreliable", but at least for gdc you never know what you get.
A complete rewrite offers some interesting opportunities of restructuring the code, kicking some old mistakes that werent worth the effort to do it up to now.
When I started with indiana in D, I didn't plan the lua part from the beginning, now when rewriting I can keep it in mind and/or do it on the way.

So which language did I choose?

Well it had to be a language that runs at least on windows and linux, allowed relatively easy usage of SDL and lua on those platforms, could somehow produce binaries for easy distribution on windows and needless to say that I'm now looking for a big community.

This, and my knowledge of languages in mind (learning a complete new one would cost lots of additional time), there weren't many options:

C#/other .NET stuff, possible, but I'm really not experienced with it, nor do I have knowledge about how reliable the linux toolchain is, how easy you can bind to SDL etc...
Java: No wai. For a "dynamic" language the syntax is way too clumsy, distribution of small binaries isnt possible (at least not that I know of), performance can be a problem. Plus a personal dislike + lacking experience ;)
Python: Confessed, I like python. Naturally with python I wouldn't have built a lua interface, I'd just have the games be written in python. The windows executable would be made with py2exe, which afaik just stuffs the complete python interpreter and all libraries into one big exe file. All in all, a good candidate, development speed has been always fast in python (at least for me), but definetely we would run into performance problems sooner or later and thus would need to have some parts in C/pyrex/whatever.
C++: There you are. Long time I ran around and told people: The only reason for choosing C++ over D is popularity/interfacing with existing code. I still believe so (except for some little language parts maybe), but this is all about community, so no need to lie to myself here. Also it might not be too hard to find a few helpers when I feel I need them.

So what happens next?

At the moment I'm coding what I will call "grail" (you know, the holy grail from indiana jones III / last crusade) in C++ right along with lua support. I'm not sure yet what I will do about the gui part (ie if I just alter atlantis to generate lua-code, if I extend it to be a more full-fledged editor, or if I build something new).

But one way or another in the end there will be

libgrail - Similar to what indiana is now, just in C++. This will theoretically allow you to build your adventure games directly in C++ without using/linking lua at all, although regarding documentation and "making it easy/fun to use", I'll focus on the runtime first as I think most game developers will rather use lua than C++.
grail-runtime - A binary application that takes a game as input and runs it. Where a game is a directory full of lua code, graphics, sounds, etc... Naturally later on it will be possible to stuff the game into a zipfile and append it to the grail-runtime so you can distribute you game in a single binary. Also I'm thinking about an open lua interface that developers can enable, so you (or the gui) can control the engine while it is running allowing cool debugging/testing things.
The GUI, as said, I'm not sure yet what it will be. Maybe not much more than atlantis is now (for first at least), just that it generates lua code. Maybe a full-fledged visionaire/wintermute - like editing system that does most of the work you have to do.

Later, I'll put up my grail git directory, don't judge me for the code yet, it has been a while since I did real c++ coding. And naturally at this state you can't expect something to look at really. When I have it that far that a character can walk around again, I'll post some screenies and inform you!

It's a trap - Multilingual web applications

2009-11-21T10:26:00

Say you're designing a multilingual web application. Where would you put the language information?

Carry it over with each request (generate hidden inputs)
Into the users session
Into the URL

The first approach is obviously quite tedious and more or less just included for completeness. No doubt you don't wanna do this except for a really small app or some rather unusual constraints (no cookies, framework that has hard constraints on URL scheme).

The second one looks like a really good idea:

You don't need to find out the users language with each request with url parsing or similar things,
You can freely choose your URLs after whatever scheme you like, language doesn't have to be a part of it.
You URLs can be exchanged between users of different languages, they will be displayed in each users favourite language to them.

So wheres the TRAP? Here a hint: Think of search engines.

Still don't see it?

Well took me some time, too. The problem is: If a search engine from some country indexes your page its bot will most probably not emulate/have cookie support. So all search engines in all countries will see and thus index the page in its default language (that one that appears when a user has no cookies enabled). This way the search engines won't find your site when the users enter keywords in their own language. What a crap.

So the answer is obvious: Put that information into your url. Or if you want detect it from the domain name (if you have one domain for each country/language <- distinguish carefully btw!)

It's a trap - Python object construction / reference semantics

2009-11-21T10:21:00

I guess this is a common trap. I personally am python coder for years now and still, from time to time I run into it. As the syntax for it is very concise you might sometimes be tempted to give default values for members in a classes definition. Code says more then thousand words. This works fine:

class Foo(object):
  x = 7

f1 = Foo()
f2 = Foo()

f2.x = 42

print f1.x # prints 7
print f2.x # prints 42

Now say we have something else for x, say it will contain a list most of the time and thus by default it should be the empty list:

class Foo(object):
  x = []

  def append(self, y): self.x.append(y)

f1 = Foo()
f2 = Foo()

f1.append("Take evasive action!")

print f1.x # prints ["Take evasive action!"]
print f2.x # prints ["Take evasive action!"] <- WTF?

The somewhat experienced programmer immediately sees what happens. Let's look at this step by step.

What does python do when you access f1.x?

It first looks if the object has a member x, if it can not find any, it accesses the class member x.

So what exactly happens in the first example?

Directly after construction the objects f1 and f2 point both to Foo.x that is at the same x. Thes with f2.x = 42 we add a member x to the object f2 so that we have a situation like the following:

Foo.x points to the value 7
f1 does not have a member x, so access to f1.x will be redirected to Foo.x
f2.x points to the value 42

Note that you never "change" an integer in python, you can only reference a different integer, ie integers are immutable.

What exactly happens in the second example?

As in the first example, after construction, both f1 and f2 don't even have a member x, they just redirect to Foo.x. Whats different here is that in this example we don't ever create an x-member in one of the objects!

Instead of integers, lists are mutable. In our append call, we actually change Foo.x instead of creating a new member. As both f1 and f2 still pass through to Foo.x, we see the change on both objects.

Yet another trap!

Now you might think "Okay, then I do it like this:"

class Foo(object):
  def __init__(self, x=[]):
    self.x = x

f1 = Foo()
f2 = Foo()

But this is basically the same problem, with a slightly different background. The default value for x you type there is only created once. So what you do here would be creating members f1.x and f2.x but still having them point to the same object which leads to the same unwanted result.

So what should I do?

Don't use this as a way for giving default values for object members. Instead do something like:

class Foo(object):
  def __init__(self, **kwargs):
    self.x = kwargs.get('x', [])

This way you have the default value defined inside __init__, which guarantees that each defaultly-initialized object gets a different "[]".

Depending on your actual needs you might also do something like:

class Foo(object):
  def __init__(self, x=None):
    self.x = x
    if self.x is None: self.x = []

Or you could copy every received value:

class Foo(object):
  def __init__(self, x=[]):
    self.x = x[:] # assumes, passed x is always a list!

For those of you that are still very new to python: Try to always keep one eye on what references what. Be aware that everything in python has reference semantics although for immutable values that can't lead to this problem (and in lots of situations this "feels" like value semantic). Also note that my approaches are far from perfect too. And: Don't use any of that code if you do not fully understand it, play around with everything in the python interpreter, you can learn a lot of things there.

It's a trap!

2009-11-21T10:16:00

I'm hereby creating a new kind of postings on this site: I want to post about common programming traps, you know that kind of stuff you think at first "Obviously it works this way" and later on you feel... trapped!

For general reference to this topic see:

Admiral Ackbar, Sheldon and/or Captain Picard

Traps ahead, be prepared!