Remove filter Showing only articles with tag "Programming"

Links on all kinds of things

Posted: Sunday, 2020-02-02 17:41 | Tags: Programming, Python, MachineLearning

Some more links in particular about ML & Python topics

Source: rosiejuliet

CourseDuck has a really nice overview of free Machine Learning tutorials with ratings & pros/cons; spanning YouTube, Coursera and so forth. Definetely worth checking out!

Codi looks like the "iPython Notebook in VIM" I always wanted, did not yet take the time to install it but its number one of my list of things to try.

Altair is yet another Python data visualazition lib (think Matplot and the like). What positively sets it apart though is that it has a really nice declarative language which is particularly interesting for more explorative work.

Broot gives you an overview of a directory tree in your shell, offering search (similar to fzf & thelike), cd'ing, moving files around, and seeing what takes space. Some overlap/competation to ranger it seems?

Magic: The (somewhat) Deep Learning

Posted: Sunday, 2019-08-04 08:33 | Tags: MachineLearning, Programming, MTG

In the previous posts in this series we dug into some data analysis on Magic the Gathering cards and tried to model the mana cost of cards in order to answer questions like "how much does 'flying' cost?".

We found a linear model that could explain the costs to some degree, however it struggled to get right the more complex cost structure around eg. 0-mana or many-colered cards. In this post we make an attempt at a neural network model of MTG mana costs in order to address these nonlinearities.

Wizards of the Coast, Magic: The Gathering, and their logos are trademarks of Wizards of the Coast LLC in the United States and other countries. © 2009 Wizards. All Rights Reserved. This web site is not affiliated with, endorsed, sponsored, or specifically approved by Wizards of the Coast LLC.


Some random-ish Python Links

Posted: Saturday, 2019-06-08 22:58 | Tags: Programming, Python

Quick notes of some Python-related links of various topics I thought I share!

Source: aoifecahill

Random stuff

Awesome collection of all kinds of robotic / driving related algorithms incl. Python implementations:


Magic: The Machine Learning

Posted: Monday, 2018-07-16 19:17 | Tags: MachineLearning, Programming, MTG

In the previous post on this topic, we raised the question on how much particular abilities such as "flying" cost in the widely known trading card game Magic the Gathering. We found luckily there is a nice, usable free database out there that should be helpful to answer this and while trying to take a look into the data we learned about Prinicipal Component Analysis and how it can be helpful to visualize high-dimensional data.

Somewhat unsatisfactory we ended up without getting an answer to the "flying" question, lets try to come up with a model now that can help with that.

Wizards of the Coast, Magic: The Gathering, and their logos are trademarks of Wizards of the Coast LLC in the United States and other countries. © 2009 Wizards. All Rights Reserved. This web site is not affiliated with, endorsed, sponsored, or specifically approved by Wizards of the Coast LLC.


Magic: The Data Gathering

Posted: Sunday, 2017-04-30 16:00 | Tags: MachineLearning, Programming, MTG

Magic: The Gathering is a trading card game I happen to like. It also so happens that there is a database of cards available online. Additionally I (like many programmers) kind-of got into Machine Learning recently. Lets see if we can combine these ingredients to come up with something interesting, shall we? Have you ever wondered whether cards got "cheaper" (in terms of Mana) in the past years or how much more a creature costs because it can fly? How much cheaper is a creature that has Defender? I'll be toying around with some card data and do some basic data analysis for practice, education and fun. There shall be some (somewhat hacky) Python code using Pandas, SciKit-Learn and Matplotlib in the end that you can use to reproduce (or extend) what I did (you'll have to wait for the second part for that though).

In this first part we will do some general analysis of the data and see what we can learn about the game with a simple Principal Component Analysis.

Wizards of the Coast, Magic: The Gathering, and their logos are trademarks of Wizards of the Coast LLC in the United States and other countries. © 2009 Wizards. All Rights Reserved. This web site is not affiliated with, endorsed, sponsored, or specifically approved by Wizards of the Coast LLC.


These days there are some damn cool things out there to build up your Unixoid (development) environment. This is a medium-sized elaboration on the stuff I like.

Spoilers: Including an editor that can be considered better than VIM in some regards and the single most awesome shell there is (no, not ZSH). I structured it a bit though so you can skip through the parts that are boring to you :-).


Python notes

Posted: Sunday, 2017-01-22 11:02 | Tags: Programming, Python

I've been watching some PyCon talks online lately and stumbled across a few interesting things, these are mostly notes for myself but feel free to consider them useful ;-)


Implementing state-machines: An evaluation

Posted: Sunday, 2015-10-18 10:15 | Tags: Programming, C++

In the previous article I raised the question on whether -- from a performance standpoint -- one should implement state machines using switch-statements or function pointers.

We saw that switch-statements are compiled to a number of jump/compare instructions that grow at least logarithmically with the number of case-labels. I had to close with the observation that a too contrived example however leads to our compiler actually analyzing our state machine good enough to be able to rewrite our code into unrolled instructions and loops.

In this post I want to discuss a slightly less contrived example that is a (stupid and useless) "parser" for XML data. We will observe that even when the state logic is more complex and transitions depend on unforeseeable input data, the compiler can do smart transformations with switch-statements.


Implementing state-machines: switch() vs. method pointers

Posted: Saturday, 2015-10-17 15:51 | Tags: Programming, C++

Finite-state machines are useful for many purposes: Say we have a long running computation that goes back and forth between different phases and is executed chunk-by-chunk in a main loop that does other things in between (e.g. updating some user interface). Yes, you scream "threads", but there are some situations where those are just not necessary or wanted for whatever reason.

Another common use is building a lexer/parser that reads some kind of input and interprets the incoming characters differently depending on "in what state we are" e.g. whether we just read a '' or not might influence how we interpret the current character.

Most state-machine code I've seen so far is built with giant switch()-blocks and recently I've come to wonder whether this is always the way to go.


Path Finding in Grail

Posted: Sunday, 2012-07-15 00:22 | Tags: Grail, Programming, Pathfinding

A topic thats using up my (rare) Grail-time lately is the topic of how to define where an actor is allowed to walk and how it will find its way there. I found this question quite fun to work on, as it poses an interesting mix of theoretical (geometry) and practical (coding) considerations.

So what is the problem exactly? The answer is: A combination of things:

  • Restrict the actors movement to a certain area that will be defined by the game programmer, I call this the walkable area.
  • Come up with a mechanism that allows the actor to walk from one point in the walkable area to another one. Thereby the actor should stay in the walkable area at all times, so a path inside the area has to be found.
  • When the user requests to walk to a point outside the walkable area, the actor should instead walk to some near point inside the walkable area, whereas near is not exactly defined.

Quick Update

Posted: Tuesday, 2012-02-14 11:20 | Tags: Site, Programming

Hey guys, as I should be working at this time of the day, I'll keep this update very brief.

  • At beginning of the month I was at the FOSDEM 2012 which was quite interesting. Beyond others I heard a talk about Data-Driven and Component-Based Game-Entities which was quite interesting
  • As you might have noticed, there is some activity recently on github. Thanks to feedelli who started working on the dialog system, we now have a plan for 0.1
  • There where some problems for I don't know how long with commenting on articles, it should work again now, if not please drop me a mail.

Background Music

Posted: Monday, 2011-05-09 20:02 | Tags: Music, Programming, PPL

Background Music

As I'm still mostly focusing on my job currently rather than coding for these spare time projects, I still cannot announce any remarkable progress on those. However I've come to finally find some cool music that is suited for programming/working (I've searched for that for like ever). Here is a list for you:


The hunt for the perfect programming language (ppl)

Posted: Saturday, 2010-09-11 16:07 | Tags: PPL, Programming

Basically since I started programming, I'm kinda looking for a programming language that is perfect in a way. First problem with that is that I don't even have a precise definition of what perfect in this context exactly means, but it should go along those lines (in no particular order):


First steps

Posted: Tuesday, 2010-02-02 22:13 | Tags: Grail, Programming

Luabind is so cool. Wrapping really is the easy part now, so I finally focus on implementing stuff again. The main character already can walk around on the screen, now I'm concentrating on a user interface and then on interactions with other scene actors.

Thats sounds like the biggest part but keep in mind, theres still a lot to do:

  • User interface already is a big topic (buttons, inventory display, etc...)
  • Loading/Saving (will hopefully be easy due to boosts serialization lib)
  • Scene/walkpath stuff (the "algorithmic" corner yay)
  • Loading scenes directly from atlantis files
  • Dialogs (talking)
  • Sound/Music/Voice output
  • From time to time try to get the stuff to compile & run under windows ;)

Good bye NIH, hello luabind & boost!

Posted: Monday, 2010-01-18 15:45 | Tags: Programming, Grail

I have to confess something. To you all.

I am ill. What I have is called the NIH syndrome. For those of you who never heard of it: It means preferring to code stuff yourself instead of learning how to use a library somebody else wrote. So its all about reinventing the wheel. [1]

For example I was inventing a DSL for describing c++/lua bindings together with a handcrafted (and ugly) parser which generated c++ code from that DSL etc... although I knew there was luabind.

Now that I tried it, let me say: luabind rocks. Don't dare to invent something yourself until you tried that one.

Same goes for boost::shared_ptr. It is fun to invent a reference counting mechanism, but seriously: Use shared_ptr, its really cool, luabind supports it already and it simply makes your life so much easier.

I also plan to use/learn more parts of boost such as the serialization library which seems to be even more flexible than the serialization stuff in D I wrote and was kinda proud of.

Stay tuned for more.


Note that this can be a good idea actually. Say you decide to use $big_library and it works quite well. You save valuable time in your early project phases by not having to write all that stuff yourself.

But then, as you project evolves you have more special needs and demands to that library. Depending on how its designed and what it is about you might have to actually patch the library which suddenly requires a huge amount of time, because you have to learn the internals of $big_library for just one little change.

A few thoughts on memory management in grail

Posted: Thursday, 2009-12-10 09:42 | Tags: Programming, Grail

In D we had a garbage collector that made things really easy: Just allocate an object and never think about what happens to the memory of that object again, the gc does it for you. In C++ there is no garbage collector built in so you have to delete old objects yourself. That per se is not so much a problem as long as the question of ownership is always cleared.

Ownership means: Object a has a reference to object b and is responsible for deleting it. Normally with a little thought its easy to say which object should own which one(s). However, if you let the user allocate and play around with objects in lua, things can get a little more complicated.

Current non-solution

Currently I'm using a thingy I call "registry". This holds references to all objects allocated in lua and destroys them when they aren't useful anymore. To find out when that is, you tell it to the registry. There are two possible scopes: CHAPTER and APPLICATION. Objects with CHAPTER scope exist until the current chapter runs. This is most useful for scenes and the actors running around in them.

The APPLICATION scope holds objects as long as the program runs. This can be useful for the main menu or other user interface stuff.

However this approach now leads to some Problems:

  • Currently it can happen that a Scene "s" holds an actor "a" and its not in general known if "a" is in the registry. In this case its not known if the scene should destroy "a" when its destroyed itself or if the registry would do it. Also it might be that the user wants a longer lifetime for "a" than "s" has.
  • The user has a lot of ways to fuck up everything when creating an object with chapter scope. For example what happens when a chapter-scope object is created outside of any running chapter? What if the user by accident holds a reference to a chapter-object after the object has been destroyed?
  • Its annoying to always have to write "chapter" or "application" for each object you create.

Possible solution 1: Register everything

Just force everything to be in the registry for example by giving the user no way to create an object without registering it. Especially when objects internally create other objects, put them in the registry too, "inheriting" scope from the parent object. Then no object would ever use delete, that would completely be Registry s job.

Problem: This still allows the user to reference objects beyond their lifetime and to produce similar fuckups. And its still annoying to always tell the scope when creating an object.

Possible solution 2: Register only certain object types

We could only register Scene s and let them have ownership for things like actors etc... However there is no guarantee that the user doesn't try to use the same image in two different scenes and there we have the double free corruption again. So this one is really a no-go

Possible solution 3: Use reference counting

Build a simple reference counting mechanism so each object knows how many places reference to it. This would mostly make the registry unecessary, except maybe for holding scenes for the current chapter and giving names to objects for the user. The only problem with reference counting is that it can't handle reference circles well (ie A has a pointer to B and B has a pointer to A), so I have to think about the whole structure a little more to guarantee there won't be such circles (although I'm already pretty sure there won't).

Below is a little examplary code. I didn't even try to compile it so forgive me if its not correct, it should just illustrate the idea.

class ReferenceCounted {
    unsigned int referenceCount;

    ReferenceCounted() : referenceCount(0) { }

template <typename T>
class Reference {
    T* obj;

    // Forbid copying of references
    Reference(const Reference<T>& other) { }
    Reference<T> operator=(const Reference<T>& other) { }

    Reference() : obj(0) { }

    Reference(T* obj) : obj(obj) {

    virtual ~Reference() {
      if(obj) {
        if(obj->referenceCount == 0) {
          delete obj;

    void ref(T* o) {
      if(obj) {
        if(obj->referenceCount == 0) {
          delete obj;
      obj = o;

    T& operator*() { return *obj; }
    T* operator->() { return obj; }

// Usage:

class Foo : public ReferenceCounted {
    void foo() { ... }

void f() {
  Reference<Foo> r;

  r.ref(new Foo());

} // "r" is removed from the stack here
// that reduces the reference count for the Foo
// object and thus deletes it! Nice isn't it?

Compiles under Windows!

Posted: Saturday, 2009-12-05 10:48 | Tags: Programming, Grail

BAM reported yesterday that he managed to make Grail compile under windows, good work BAM :)

What kind of GUI-Editor for grail?

Posted: Friday, 2009-11-27 14:13 | Tags: Grail, Atlantis, Python, Programming

I'm currently thinking about how the future GUI for grail will look. Basically there are 2 approaches that I would say are worth considering at the moment:

  • Step-by-step fix Atlantis to become the new grail editor. The first step would be to have it generate lua code instead of D code, in the future it would be extended with a lua editor component etc...
  • As grail is basically starting from scratch this would be the perfect opportunity to include some editor capabilities directly into grail.
    • (-) There would never be a code-editing component, I would keep the additional GUI elements to a bare minimum. I guess the control would be over a one-line shell-like command line at the bottom where you enter commands like "create polygon" or something. No checkboxes, no dialogs.
    • (+) Having the editor directly combined with the engine allows some interesting things like running parts of the game while working on it. Although I must confess this point is very weak: Due to the lua scripting engine one could realise similar behaviour by sending lua commands to the runtime via a network socket or similar approaches.
    • (+) Another good thing would be: Having a command line and the capabilities for displaying polygons etc... inside the engine allows for advanced debugging utilities, that could be really cool.
    • (-) Probably this approach would mean more work :(

Comments welcome!

The thing with D

Posted: Saturday, 2009-11-21 10:31 | Tags: Programming, Site, Giga, Atlantis, Releases, Indiana, Grail


Some of you might already know, I experienced some severe/general problems with D compilers lately, namingly I ran into a situation where I was not able to find a D compiler(version) that would have no compiler bugs that influenced indiana and would run with phobos/d1. Another constraint was that we need one compiler on windows and one on linux (preferrably x86_64), both naturally being able to compile & link the same code.

I wont get in any further detail here, I just say: I really tried for about 2 full days and at least 8 compiler versions on 3 different platforms and I came to the conclusion that these were my options:

  • An old version of gdc I used until then had only one compiler bug that nulled a pointer in some circumstances, rendering "only" the serialization module (and thus the whole savegame system) unusable. I could have tried to stick with the old version and ship the code around the bug.
  • llvmdc would have been interesting to try, unfortunately it doesn't seem to support phobos, so for this maybe-solution a switch to tango or at least tangobos would have been necessary.
  • Take the newest available gdc/dmd and fix indiana that seemed to actually lay behind the current D1.0 "stable" language definition so the compilers would work with it. Note that the most current gdc releases (from sourceforge/goshaw) both couldnt even build phobos correctly (at least that day).
  • Do a complete rewrite in a different language with more stable toolchain.

I decided for the last one for a lot of reasons:

  • The incident that GDC didn't receive updates for at least a year (or so) while DMD did shows that the D community is not yet "mature"/"stable"/"big" whatever you wanna call it. I wouldn't call the D community "unreliable", but at least for gdc you never know what you get.
  • A complete rewrite offers some interesting opportunities of restructuring the code, kicking some old mistakes that werent worth the effort to do it up to now.
  • When I started with indiana in D, I didn't plan the lua part from the beginning, now when rewriting I can keep it in mind and/or do it on the way.

So which language did I choose?

Well it had to be a language that runs at least on windows and linux, allowed relatively easy usage of SDL and lua on those platforms, could somehow produce binaries for easy distribution on windows and needless to say that I'm now looking for a big community.

This, and my knowledge of languages in mind (learning a complete new one would cost lots of additional time), there weren't many options:

  • C#/other .NET stuff, possible, but I'm really not experienced with it, nor do I have knowledge about how reliable the linux toolchain is, how easy you can bind to SDL etc...
  • Java: No wai. For a "dynamic" language the syntax is way too clumsy, distribution of small binaries isnt possible (at least not that I know of), performance can be a problem. Plus a personal dislike + lacking experience ;)
  • Python: Confessed, I like python. Naturally with python I wouldn't have built a lua interface, I'd just have the games be written in python. The windows executable would be made with py2exe, which afaik just stuffs the complete python interpreter and all libraries into one big exe file. All in all, a good candidate, development speed has been always fast in python (at least for me), but definetely we would run into performance problems sooner or later and thus would need to have some parts in C/pyrex/whatever.
  • C++: There you are. Long time I ran around and told people: The only reason for choosing C++ over D is popularity/interfacing with existing code. I still believe so (except for some little language parts maybe), but this is all about community, so no need to lie to myself here. Also it might not be too hard to find a few helpers when I feel I need them.

So what happens next?

At the moment I'm coding what I will call "grail" (you know, the holy grail from indiana jones III / last crusade) in C++ right along with lua support. I'm not sure yet what I will do about the gui part (ie if I just alter atlantis to generate lua-code, if I extend it to be a more full-fledged editor, or if I build something new).

But one way or another in the end there will be

  • libgrail - Similar to what indiana is now, just in C++. This will theoretically allow you to build your adventure games directly in C++ without using/linking lua at all, although regarding documentation and "making it easy/fun to use", I'll focus on the runtime first as I think most game developers will rather use lua than C++.
  • grail-runtime - A binary application that takes a game as input and runs it. Where a game is a directory full of lua code, graphics, sounds, etc... Naturally later on it will be possible to stuff the game into a zipfile and append it to the grail-runtime so you can distribute you game in a single binary. Also I'm thinking about an open lua interface that developers can enable, so you (or the gui) can control the engine while it is running allowing cool debugging/testing things.
  • The GUI, as said, I'm not sure yet what it will be. Maybe not much more than atlantis is now (for first at least), just that it generates lua code. Maybe a full-fledged visionaire/wintermute - like editing system that does most of the work you have to do.

Later, I'll put up my grail git directory, don't judge me for the code yet, it has been a while since I did real c++ coding. And naturally at this state you can't expect something to look at really. When I have it that far that a character can walk around again, I'll post some screenies and inform you!

It's a trap - Multilingual web applications

Posted: Saturday, 2009-11-21 10:26 | Tags: ItsATrap, Programming, Web

Say you're designing a multilingual web application. Where would you put the language information?

  • Carry it over with each request (generate hidden inputs)
  • Into the users session
  • Into the URL

The first approach is obviously quite tedious and more or less just included for completeness. No doubt you don't wanna do this except for a really small app or some rather unusual constraints (no cookies, framework that has hard constraints on URL scheme).

The second one looks like a really good idea:

  • You don't need to find out the users language with each request with url parsing or similar things,
  • You can freely choose your URLs after whatever scheme you like, language doesn't have to be a part of it.
  • You URLs can be exchanged between users of different languages, they will be displayed in each users favourite language to them.

So wheres the TRAP? Here a hint: Think of search engines.

Still don't see it?

Well took me some time, too. The problem is: If a search engine from some country indexes your page its bot will most probably not emulate/have cookie support. So all search engines in all countries will see and thus index the page in its default language (that one that appears when a user has no cookies enabled). This way the search engines won't find your site when the users enter keywords in their own language. What a crap.

So the answer is obvious: Put that information into your url. Or if you want detect it from the domain name (if you have one domain for each country/language <- distinguish carefully btw!)

I guess this is a common trap. I personally am python coder for years now and still, from time to time I run into it. As the syntax for it is very concise you might sometimes be tempted to give default values for members in a classes definition. Code says more then thousand words. This works fine:

class Foo(object):
  x = 7

f1 = Foo()
f2 = Foo()

f2.x = 42

print f1.x # prints 7
print f2.x # prints 42

Now say we have something else for x, say it will contain a list most of the time and thus by default it should be the empty list:

class Foo(object):
  x = []

  def append(self, y): self.x.append(y)

f1 = Foo()
f2 = Foo()

f1.append("Take evasive action!")

print f1.x # prints ["Take evasive action!"]
print f2.x # prints ["Take evasive action!"] <- WTF?

The somewhat experienced programmer immediately sees what happens. Let's look at this step by step.

What does python do when you access f1.x?

It first looks if the object has a member x, if it can not find any, it accesses the class member x.

So what exactly happens in the first example?

Directly after construction the objects f1 and f2 point both to Foo.x that is at the same x. Thes with f2.x = 42 we add a member x to the object f2 so that we have a situation like the following:

  • Foo.x points to the value 7
  • f1 does not have a member x, so access to f1.x will be redirected to Foo.x
  • f2.x points to the value 42

Note that you never "change" an integer in python, you can only reference a different integer, ie integers are immutable.

What exactly happens in the second example?

As in the first example, after construction, both f1 and f2 don't even have a member x, they just redirect to Foo.x. Whats different here is that in this example we don't ever create an x-member in one of the objects!

Instead of integers, lists are mutable. In our append call, we actually change Foo.x instead of creating a new member. As both f1 and f2 still pass through to Foo.x, we see the change on both objects.

Yet another trap!

Now you might think "Okay, then I do it like this:"

class Foo(object):
  def __init__(self, x=[]):
    self.x = x

f1 = Foo()
f2 = Foo()

But this is basically the same problem, with a slightly different background. The default value for x you type there is only created once. So what you do here would be creating members f1.x and f2.x but still having them point to the same object which leads to the same unwanted result.

So what should I do?

Don't use this as a way for giving default values for object members. Instead do something like:

class Foo(object):
  def __init__(self, **kwargs):
    self.x = kwargs.get('x', [])

This way you have the default value defined inside __init__, which guarantees that each defaultly-initialized object gets a different "[]".

Depending on your actual needs you might also do something like:

class Foo(object):
  def __init__(self, x=None):
    self.x = x
    if self.x is None: self.x = []

Or you could copy every received value:

class Foo(object):
  def __init__(self, x=[]):
    self.x = x[:] # assumes, passed x is always a list!

For those of you that are still very new to python: Try to always keep one eye on what references what. Be aware that everything in python has reference semantics although for immutable values that can't lead to this problem (and in lots of situations this "feels" like value semantic). Also note that my approaches are far from perfect too. And: Don't use any of that code if you do not fully understand it, play around with everything in the python interpreter, you can learn a lot of things there.