Subjectively

Who do we trust

2010-10-06T02:19:00.001+02:00

In the old days of the internet (10-15 years ago) there used to be about 10-20 trusted root ca's installed on my operating system. On my windows-xp machine at work i have hundreds.

For those of you who don't know what i'm talking about, here's a basic intro:

Root Certificate Authorities

When you connect to certain websites you will see a "Secure lock" icon appear in your browser in an address that starts with https.
What this is supposed to mean is that the connection is "secure".
This security is provided by a number of protocols, most importantly each website has a certificate that says "the company X owns this address".
Now the problem is that you can't just take someone at their word - you need some trusted third party that verifies that this person is who they are.

This is a similar idea to personal id- your government serves as a trusted authority that provides a certificate (id) that verifies someone is who they say they are.
In the world of the internet, this authority is called a Certificate Authority (CA).

The difference between the government and the internet in this case is that we implicitly assume that the government is a trusted issuer of id's.
In the world of the internet, there is no accepted, trusted authority that we can count on to produce these id's. Several commercial organizations then took this role and has been accepted as "trust worthy".
This makes them what is known as Trusted Root Certificate Authority.
What this means is that certain organizations were accepted as trusted and are allowed to ascertain the identity of others.
The root ca is responsible for the validity of the certificates it provides and holds the power to revoke them if they are misused or stolen.

The problem is that once a certificate authority is accepted as root, it holds tremendous power for abuse.
So when you find yourself with hundreds of them installed on your machine - then something is very wrong.
What could a rouge root certificate authority do with it's power?

First of all, a rouge root ca is in a sense unstoppable - once a certificate is accepted as trusted on your machines local certificate store there is no "higher authority" that can revoke the certificate.

If someone gains control of a root certificate authority they can use it to fake the identity of anyone.
This opens up all "secure" traffic to an undetectable man-in-the-middle attack.

Main In The Middle

in a man-in-the-middle attack works something like this:
say that you and i talk to each other on the phone.
we both assume that you are listening to me when i say something, and that when you speak i am hearing you talk.
now let's assume we've never met and neither of us knows what the other one sounds like.
I call you and i assume you answer, but in fact some other person is on the other end and he is on a separate call with you.
They repeat most of what i am saying to you, and most of what you are saying to me.
But since they are in the middle of the line, they can change what is being said.
Neither one of us is even aware that something is wrong...

This is the essence of a man in the middle attack. On the internet, the equivalent to you and me knowing what we sound like is the certificates given to us by the root certificate authority.

What could someone do if they were able to impersonate a root ca?
They could monitor any secure channel you have - you email. your Facebook account, basically anything you log into. They could also act on your behalf and you would not even suspect something is wrong, no software would alert you, no anti virus.

DONT PANIC

My intention is not to alarm anyone about someone reading your emails because of compromised root ca, but to point to the fact that the more root ca's are installed by default on our operating system, the grater the chance that one of them is compromised.
And that my computer has root ca's installed by certificate authorities around the world, many of which i do not consider the least bit trustworthy.

Introduction To Programming - What is Code?

2010-05-22T05:13:00.065+03:00

In the most technical sense computer code (or source code) is a series of instructions that can be translated through a process that varies from trivial to complicated* into machine "language".

While technically correct, the previous definition is pretty boring. It is also very far from what actual code is like. It certainly doesn't help us understand what it's for and why it works the way it works - which I think are the more interesting questions.

Before I try to talk more about what real world code is like and some more interesting technical aspects, I would like to say that there are many different programming languages and there are many things that can be considered computer code, from the structural language called HTML that describes the page you are reading to the assembly language that is used to write processor instructions.

It's hard to cover the huge variety of what can be considered "code" in a single, broad definition so instead of doing that I will try to explain how code is used in some of the common programming languages and the why's, how's and what's of this code.

Why do we need it?

Computer code is a tool we use to describe ideas, concepts and processes in a kind of functional way that can be converted into machine language.

We need this tool because there is a dividing line between what can be easily and efficiently done in hardware - the physical components of a computer, primarily in the processor - and what can be efficiently done in software. Code begins pretty much where the hardware leaves off.

Processors are good at things like adding one number to another number, moving bits from one place to another, multiplying, subtracting and going through these instructions in sequence. They are also good at performing logical decisions like "if this number is a zero move to the next instruction, if it's not zero jump 5 instructions"

Machine language is these basic instructions, there is a language called Assembly which exists directly "above" machine code. It allows giving instructions in a way that translates almost immediately into these instructions. It is rarely used for complicated things and is considered one of the toughest and most respected specializations in computer programming. Its proximity to the actual hardware gives the programmer the most control of any language over what goes on in the processor. It can allow for some of the most efficient code that can be written.

This control comes at a cost - expressing complex ideas is very hard, writing even the simplest programs requires a lot of thought and training and even with training reading Assembly is a tedious job. A good knowledge of Assembly is one of the essential tools for hackers.

Assembly is rarely used in software development. languages like Java, c and c++ are far more popular as they allow describing more complicated structures and expressing complicated ideas in simpler ways then assembly.

A 2 lines of code in java, like this one counting to 100 and printing each number:

for (int i=0; i<100; i++)
   system.out.println("i = "+i);

Will translate into hundreds of instructions in assembly.

When programmers speak of computer code, they rarely talk about Assembly.
The thing is, using other more "high level" languages like Java or c++ requires some kind of translation into this computer code as this is the only thing that can run on the physical hardware. This is why a process called "compilation" was introduces, which translates higher level languages into machine code. This abstraction allows us to create far more complicated software in exchange for giving up control of the specific instructions given to the hardware.

Higher and more specialized languages exist that are even further away from the processor.
The common thing to all programming languages is that they exchange some level of control over what really happens on the machine for clarity and ease of use.

The building blocks

We discussed the why, so now let's talk about what code is made out of.

The goal of any higher level programming language is to provide the syntax or "semantics" to express complicated ideas in a coherent, manageable and extendable way.

There are some concepts that are shared between most languages, these are:
variables, conditionals, loops, procedures (or methods/functions) and mathematical operations. Another important common concept is the Comment. Slightly less common are classes.

To better understand what code is let's look at the building blocks.

Variables - variables are a way of giving a name to some of the computer's memory in a way we can easily use in the code.
For example, we can define:

int myVariable = 100;

now we can refer to "myVariable" in other places in the code, we could change its value, add to it, subtract from it or compare it to other values.

Conditionals - conditionals are a way of asking for one thing or another to happen, depending on some logical condition.
For example this rule:

if (myVariable<100)
   system.out.println("My variable is less than 100");
else
   system.out.println("My variable is equal to or larger than 100");

States that the line "My variable is less than 100" should be printed if the value within "myVariable" is less than 100, and the line "My variable is equal to or larger than 100" otherwise.

Loops - loops are a tool we use when we want to perform repeated operations. One of the useful properties of computers is that they can do certain things many times and very fast. Loops are a way to facilitate this property. A loop generally consists of one or more instructions we want to perform and some condition that will make the loop stop.
For example:

for (int i=0; i<100; i++)
   system.out.println("i = "+i);

This code says something like this:
"For as long as the variable "i" - which starts out at zero - is smaller then 100, increase "i" and perform the next instruction"
put another way - do the next line 100 times.
This next line says write to the screen the text "i = " followed by the value of i.
the output would look like :
i = 1
i = 2
...(and so on)

Procedures/Functions/Methods - Procedures are one of the mechanisms that allow us to create new tools and break apart complicated logic into smaller, reusable and more manageable parts.

It is an essential mechanism when we want to collaborate with other programmers and logically separate different parts of our program.

One example of a function is the "system.out.println("some text");" that i was using in other examples.
In Java, this call hides all the complicated logic required to tell the computer to print out the text into a very simple, easy to use line of code.

Because someone else did the work of writing this method for us we don't need to worry about how exactly it was done or how to do it - we only need to know what it's supposed to do, and how to use it - which is far simpler then writing it in the first place.

The Comment - a comment is just as it sounds, commenting about the code that was written.
Code files are no different than other text files - that is, they are simply text. The compiler takes our text and converts it into machine code. By marking certain lines as comments we can tell the compiler to ignore these lines. This allows us to communicate with the reader and provide more insight into the code.

Comments are a vital component of programming languages and are one of the few tools that are dedicated to communicating our intentions to other programmers.

A comment in code looks very much like other text, but it contains a marker that tells the compiler to ignore
//This is a comment, it can contain any text we want and many special characters and we can use it
//To help other programmers understand the code we are writing
//For example
//The following code computes 2 * 2 and sets the result in myVariable
//Assign the value 2 to myVariable
int myVariable = 2;
//Multiply myVariable by itself, and set it to myVariable
myVariable = myVariable*myVariable;

Classes - These are more complicated elements and harder to explain without more technical background. In general terms classes are a way of creating useful metaphors and using them in code. They are the foundation of what is called "Object Oriented Programming" which is the ruling paradigm in software development for many years.

What is code?

Computer code and its components serve a dual function:
The first is to help us to describe our ideas in terms that allow us and other programmers to read understand the ideas behind the code - the purpose of what we want to achieve with the code.
The second is that the code must translate into machine code that can perform the function for which it was written.

This is duality is important to understand - all working code essentially complies with the second requirement, it will run and perform its function. The first and more important requirement - to provide a coherent picture of what the writer of the code intended is an often overlooked** and vitally important role of code. This requirement to be clear and coherent is central to the job of a programmer. It is the hardest to achieve is rarely taught.

The compiler imposes certain rules, for example it requires that lines be terminated with a semicolon. It requires that a parenthesis that opened must also be closed. It also requires the user of specific "keywords" to be used to express specific ideas. A conditional must always take a certain form (as demonstrated above). Many compilers are "case sensitive", this means that MyVariable is a different variable then myVariable simply because one uses a different capitalization on the "m".

Despite all of these constraints, the compiler doesn't particularly care about the "shape" of things or about the names of things. that is - it doesn't care what names you use, as long as you follow the rules of syntax.
In this sense a compiler is like a strangely strict teacher that will accept any answer as long as it is spelled and punctuated correctly.

The ability to name things, comment on them and use more coherent and reusable structures in code exists so that programmers can impart meaning to the code that would not exist otherwise. This meaning exists only in the minds of the programmers who read the code - the computer does not "understand" the intentions of the programmer. It does not even "see" the original code as written by the programmer.

The vast majority of the lines of code written and much of the effort in writing them is dedicated conveying an idea. Of course, it's important that they actually perform the function for which they were written, but it's at least just as important and often far more important that they do so in a way that describes ideas simply, coherently and on some rare occasions - beautifully.

* - The complexity of the translation depends primarily on the language, compiler and hardware involved.

** - When I was in high school my teacher would often complain that my code was completely incoherent. I would argue that it works. Only after trying in my second year of high school to re-read some code I wrote in the previous year that I really understood why writing well formatted, well commented and coherent code is so vitally important. Writing incoherent code means that to use it or change it you must first spend a long amount of time understanding it.

Introduction to programming - What is programming?

2010-05-17T03:17:00.000+03:00

I've been wanting to write something like this for a long time.
My goal is to try and explain computer programming and more general things about computers in a way that i hope will be interesting even to those who are not particularly interested in learning programming.

So, Let's get started.

What is computer programming?

Well, generally speaking it's about making computers doing whatever you want them to.
This is not to say that I or any other programmer CAN make them do anything we want, actually we work under pretty limited constraints. It is by using these rules, constraints and limitations and manipulating them that the richness of software you see around you can be achieved.

In a way, practically any interaction with a computer system is about understanding, testing and manipulating it's capabilities and its bounds. Every interaction with a computer system carries within it the essence of programming.

The basic job of a programmer is to use their understanding of the internal workings of computers and provide a more natural and intuitive interface so that other people can take their programs and make some use of them.

It's easier to use an example, so let's talk about MS Paint. I am sure pretty much everyone saw or used MS Paint or something that resembles it at some point.
Let's try and take a look at the basics of MS Paint through the eyes of a developer.
We'll be working under an operating system (i.e. windows).

To a programmer an operating system is an environment that provides many many services.
In the same way that when someone wants to draw simple drawings they can open paint, choose a color and click their mouse, programmers receive tools and services from the operating system.
We don't have to worry about controlling the movement of the mouse cursor, controlling the various complicated mechanisms required to draw something on the screen, managing the memory, hard disk or any one of the many components needed to make computers work.

So what do we have to do?
Well, the operating system, along with our programming environment already cover many of the things we want to do. they will provide us with a window to work in, buttons to click and they will let us know when the user clicks the mouse inside our window, they will also tell us where he clicked it, and which button he clicked.
We also get something that can be thought of as a canvas that we can draw things on. To draw something we change the color of a pixel (a pixel is the smallest possible dot we can draw on the screen) to whatever color we want.

A simple recipe for a program that lets you draw with the mouse might look something like this:
1. draw a window 200x200 pixels (there is literally something called a Window or a Form and you can set its size)
2. when the mouse is clicked, change the pixel in the same position as the mouse to the color Black. (you can "ask" the window to let you know whenever someone clicks on it, and where it happened - then you can tell the window to change the color of the pixel at that point)

Doesn't look all that complicated right? that's mostly because it's really not all that complicated. it's not so far from opening paint and clicking the mouse.
Of course, I am oversimplifying certain aspects of what needs to be done - and there are languages and operating systems where doing what I just described is far more complicated.

But it's like saying that it's easier to dig a hole with a shovel then it is with your bare hands - True, but not very interesting :)

In its essence, computer programming is about creating tools for other people. I doubt if any of the people who wrote MS Paint ever really used it, personally i rarely use (other than during development) any of the software I write. What we did in our little programming thought exercise was to take an idea - drawing pictures with your mouse, and used our imagination (albeit limited) and the tools at our disposal, and created something for someone else to draw with.

Now that we know how to manipulate the pixels by changing their color, we don't need something like a mouse if we want to draw things on the screen, but by finding a simple interface other people can easily use we created a simple tool.

So what is computer programming? For me it is about taking our idea's, knowledge and understanding of what computers are and how they work and using it to create tools for (mostly) others to use.
Most professional software developers writing code today do exactly this - they spend their days thinking and writing tools and solving problems so that others can become more productive.

I'd be happy to get your thoughts/questions/comments.

No Gaming For Old Men

2010-04-23T03:31:00.005+03:00

I know this might sound like the complaints of a grumpy old* gamer (because they are :) ) but i feel like i haven't played a truly new game in ages.

I grew up in the 80's and 90's when "electronic" games were constantly reinvented and evolved, there was always something new coming out. some better, some worse but never boring.

Today gaming is a multi-gazillion dollar industry and despite this, very little happened in the past decade of gaming. Especially when we compare them to the 90's and the 80's. I think probably more major genres died then evolved in recent years.

I don't know why, perhaps the developers are too far from the hardware, It's very easy to keep making first person shooters when you have all these great engines out there. But when ID did wolfenstine and then reinvented the genre with doom, they worked with the bits and bytes - they even invented special number for Quake 3. There is no doubt we tend to be limited by the tools we have. Sometimes having no tools means more freedom of thought and more wild experimentation. Perhaps its the very high cost of entering the game - there are some very powerful players in the market and perhaps its easier to make a well defined game in a well defined genre then to try something new..

I think most of all what bothers me is that i feel that there has been little or no evolution in any of my favorite genres, and in most cases there haven't even been any serious attempts to change anything.

As i have some more details complaining to do, i thought i would direct my complaints at the individual genres.

Real Time Strategy

Many would disagree but frankly i feel that there haven't been anything new in RTS's since the genre was truly defined by westwood with dune 2 and to a lesser extent Blizzard with Warcraft.

What's even more depressing is that since Starcraft - which was nothing innovative, but simply a masterpiece of balance - there have been no (interesting/worthwhile) developments in the genre. And from what i am reading Starcraft 2 doesn't seem to be bringing anything new to the table either. I think the only game to try and bring something new to the table was the bizarre Perimeter by CodeMasters. Sure, it was innovative for the genre - but it really didn't work.

Role Playing Games

I love role playing games, i loved them since i first played Eye of The Beholder. Not only that, but i love most sub-genres of RPG's. My problem with this genre is that i feel it is loosing it's way recently. Western rpgs have been dying since around Gothic 2 (which together with Gothic 1 are at the top of my list). Bethesda created a beautiful game in Oblivion but was too lazy to bother with any Gameplay. Why do they leave those endless piles of crap in the game world i will never understand. I also don't see why they can't be bothered to balance their game before releasing it - If someone at Bethesda happens to read this, try to understand Auto Leveling means you never advance, which is the cornerstone of role playing gameplay. And no - your pathetic attempt at "capped" auto leveling in Fallout 3 was not "good". (but at least there was *some* point to leveling). A bigger problem is possibly that others began to follow - Boarderlands seems to use a similiar misguided system, and while it was fun to play it wasn't fun to power-play.

And last but certainly not least are Pirhana bytes, makers of the excellent but unjustifiably relatively-unknown Gothic and Gothic 2 (and inexcusably also Gothic 3 and Risen). Gothic (1&2 - i am not counting the dissapointing 3) was a wonderful series. It focused on a very simple and very clear damage system. This, along with interesting battles that truely required varied tactics for each creature, a class system that created truly separate gameplay (to the extent of changing some plot elements) and a perfectly balanced world where you really wanted to slay every possible enemy and gain every bit of experience you can. The system was so rewarding that you always tried to get every tiny bit of advantage while still keeping the gameplay challanging.

So gothic 3 was a disappointment - even if we disregard the crappy and crashing game engine, the gameplay which made the gothic series was gone. it seems to have been "simplified" for the unknown masses, and it lost the balance and challenge that made the previous games so great. Pirhana bytes moved on to create Risen, which is a pale unbalanced shadow of gothic with very little plot and far too easy Gameplay.

Finally, there's the KOTOR series which was excellent and a fine example in the genre and i would love to play another title in that series.

Simulation

My favorite genre in simulation is racing and i must say i have no complaints in this area - a good simulation game aspires to, well, simulate reality. Titles like Fortze Morotsport and Dirt and the first serious Need for Speed - Shift are excellent titles, the genre is constantly becoming more and more realistic with an incresingly accurate level of simulation. Sure, i can hope for some faster loading on the xbox 360 (in some titles loading time is proportional to racing time) but other then that - love it, keep up the good work.

First Person Shooters

Well, i could say that first persons shooters haven't really changed since the days of Wallenstein and doom, and in some respects that's true, but in most respects they are climbing my genre-preference ladder.

Games like Farcry (excellent gameplay and excellent AI), Call of Duty - Modern Warfare (1&2) - Bringing a truely cinematic experience to FPS gameplay. In a sense FPS's and simulation have a lot to do with technology, the level of detail and quality of graphics are the first and for many the most critical factors in FPS's. Call of duty demonstrated that a great plot is just as important. All in all the genre is doing what it has always done - keep pushing the limits of the hardware to show more realistic ways of simulating shooting people with a wide variety of weapons..

Quests (aka** Adventure games)

Quests are dying, and its a real shame. Actually, the genre is all but dead. Telltale games are the only ones that seem to be keeping the genre alive these days, and the success of their latest monkey island show that there are enough of us around to appreciate the genre (i also recommend Strong Bad's Cool Game For Attractive People, if you read this far - this game is definitely for you). As one of the most successful genres of the 90's its surprising that almost no one is still around. Lucas Arts moved on to other pastures and Sierra basically died...very sad. Oh, and i just remembered there's also the excellent Machinarium by Amanita Design.

Turn based Strategy/Managements

Nothing much to say there, its hard to talk about change or advancement in this genre. Graphics are slowly improving over time, and the notable titles (Say Civilization) are changing and tweaking the gameplay, but somehow nothing exciting happen there in years. Some new and interesting title would be good. I would also settle for ...actually, i would be really excited about bringing back say Master of Orion or Ascendancy.. You know, thinking about it i can't think of any worthwhile recent Turn based 4X game besides the Civilization series, and even that one has taken a turn for the worst with their latest "Civ Revolutions" (although civ 5 should be coming out soon, so at least they didn't abandon the fans).

Casual*** Gaming

I have a love-hate relationship with casual gaming. On the one hand this is one area of gaming where there is still a lot of innovation. there are many new kinds of addictive puzzle games,castle defense, tower defense and notably World Of Goo with other less-defined categories of casual gaming. On the other hand you have all those social games on Facebook and other platforms. It's not that i mind what people play, it's that i feel they are lowering the bar that game creators have worked so hard to raise all these years.

MMORPGs

finally the last genre on my list - MMORPG - my nemesis.

As i mentioned before - I Love RPG's and because of this i have never played an MMORPG and never will.

So why have i never played an MMORPG? The simple answer is addiction.

When i started playing gothic 2 for the first time, i played it for about a week straight.

By "a week straight" i mean that i barely ate and i slept when i could no longer concentrate on the game. I did almost nothing else. The reason i allowed myself to live with such a level of addiction untreated is that its short-lived. Once every few years there's a title good enough to warrant this level of addiction, but the experience is bound to end after a very short time. Even the most search-every-corner-twice-and-then-go-back-to-make-sure gameplay of the most complex RPG will still end after 40-100 hours. MMORPG's never end - it's the whole point of them. I can't allow something to completely take over my life, so i can't even try them.

Why are they my nemesis? because a lot of excellent titles in my favorite genre are going MMO which is leaving less and less non-MMO titles for me to play...

Also i am pretty sure MMO's are the first phase in the evil plan of the machines to enslaves us by making us live in a virtual world.

* - well - old in gamer terms

** - they are mostly known as Advanture games, but in israel we call them "Quests" (or questim in Hebrish)

*** - how can anyone call spending 10 hours a day on farmville "casual"?

Strange New Spam

2010-03-23T16:44:00.012+02:00

I just got a new comment on one of my articles (Estimating The Cost of Game Piracy) saying:

"Well your article helped me altogether much in my college assignment. Hats off to you dispatch, will look audacious in the direction of more related articles promptly as its anecdote of my favourite subject-matter to read."

Every other day i get a message from some bot selling one of the spammers greatest hits.

Naturally when i saw another "New Message from Anonymous" i immediately suspected spam.

Reading the comment i wasn't sure - it starts out reasonably coherent - but "Hats off to your dispatch" really caught my eye, as well as "its anecdote of my favorite..".

The text reads very much generated - i don't know if it's some kind of place holder message for spammers or some CS student somewhere is running an experiment with generation of comments. (its an excellent way to do a low-cost turing test. most blog comments are moderated one way or another - if you managed to post a message and it wasn't deleted, you could say that the writer is reasonably estimated as a human)

So i ran a goolge search for the strange expression "Hats off to your dispatch" - currently it returns 3 results - all of them look generated, and all from the past week or so.

One of them is identical to the comment above, the other are:

"Approvingly your article helped me truly much in my college assignment. Hats off to you dispatch, choice look audacious in the direction of more related articles in a jiffy as its anecdote of my choice subject-matter to read."

"Correctly your article helped me terribly much in my college assignment. Hats off to you dispatch, wish look ahead in behalf of more cognate articles without delay as its united of my choice topic to read."

looking at the comments there are two major connecting elements:

"Your article helped me" and "College Assignment"

Running this through google returns about 80 hits at the moment, from my surface scan the comments are from the past week and each and every one of them seem to have been generated in the same way.

I wonder who was the source of this - and if it is indeed some experiment, and the experimenter happens to stumble over this post - let me know :)

Update:

I scanned a few more of those comments (now google lists about 230 hits)

Seems to be related to "Debt Relief" some of the comments are named Debt Relief and where they can they link - So looks like it's just another spam flying around.

I guess they didn't make sure the blogs allow links before posting..

Estimating The Cost of Game Piracy

2010-02-28T00:28:00.040+02:00

The Motivation

A few years ago we had an idea on how to create a fairly hard to crack DRM.

Coming from a security background (among other things) we knew what most people in the field know - All defenses can be breached. It is simply a matter of resources.

In the case of game copy protection we knew that there are considerable resources invested by hackers and crackers to break copy protection. A good[1] copy protection can buy whoever is using it a few months at most. We figured that if we can make a sufficiently flexible and complex copy protection we can buy about 6 months for a given game before the protection is broken.

To see if this was a worthwhile cause we wanted to know what is the real value of a good copy protection. We also wanted to know if the 6 months we believed we can get would be good enough[2].

I believe that the value of a good copy protection should be the number of additional copies of a given game that would be sold if the game cannot be copied minus the number of copies that would have been sold if those resources would have gone to improving the game (or investing in marketing - in other words - the alternative cost).

It is very hard to say how many copies of a game could be sold if some additional funds are put into any aspect of making a game. It is also very hard to tell how much is lost due to the additional inconvenience introduced by some of the more intrusive protections[3].

On the other hand it might be possible to find out how many games could be sold if there was an unhackable copy protection.

Looking For Numbers

In my quest for numbers i ran a search to see if there were any research that could reasonably estimate how many copies of a given game could have been sold had it been protected.

Unfortunately none were available. What i found was plenty of articles citing the BSA and IIPA for pirated software numbers and estimates for the loss of income due to those pirated copies.

This might be considered good evidence but after taking a more in depth look at the numbers and methodology i decide i don't really trust the results (i might go more into that in a seperate article - but i am certainly not the first to question those surveys) .

I also found this very comprehensive article about game piracy in general. I recommend it to anyone who wants to get a good understanding of the different aspects of game piracy.

So the question remained - How do you measure the actual loss due to piracy?

If you want to compare things, finding something that is reasonably similar is a good start.

Like The Economist's Big Mac Index - you want a product that is arguably of the same quality, make, reputation etc' as possible if you want to make a good comparison.

I think that i found such a product for piracy loss research - Cross platform titles.

A Different Approach

There are two[4] 7th generation consoles on the market - Microsoft's Xbox 360 and Sony's Playstation 3.

These consoles are arguably[5] the same in terms of interface, level of graphics and the type quality and number of titles released for these consoles.

The Xbox 360 has been hacked many years ago (i think sometime in 2006).

The Ps3 on the other hand had not yet been hacked. Sony has made considerable efforts in securing the PS3 and so far - 4 years and counting - The PS3 had not been hacked.

My argument is that since the Xbox 360 had been hacked and the PS3 had not been hacked, the difference in sales of various titles should reflect the loss of revenue due to piracy.

In recent years more and more titles are being released to both consoles. These games constitute a comparable (identical?) product. Most cross platform titles look the same on both consoles, distributed enjoy the same publishing houses, have the same level of publicity and the same kind of gameplay, story and whatever other aspect you may choose to consider. They are also - for the most part (and for all games considered in this research) - released at the same time.

A good comparison should consider alternative or competing products. This aspect is much harder to account for. I cannot accurately or reasonably claim that it doesn't make any difference or that i can accurately measure or correct for it. I hope i can reasonably persuade you that it is of reduced importance because of two things:

1. If you look at the current (at the time of writing) top sellers lists of both the Xbox 360 and the PS3 you'll find that 5 of the top 10 sellers on the PS3 lists are cross platform titles (3 of them are the top sellers) and 6 of the top 10 sellers for the Xbox 360 are also cross platform.

2. To further reduce the impact of console specific titles on the overall numbers i tried to include as many games as i could find in the research.

Still, I acknowledge that competing exclusive titles have an impact on the overall results. But i think the numbers themselves demonstrate that this impact is not likely to be too significant.

Correcting For Console Sales Difference

The final significant difference between the consoles is the numbers of consoles sold.

Currently Wikipedia (no. for dec 2009) cites 33.5 million PS3 consoles sold vs. 39 million Xbox 360 consoles sold. For the sake of transparency i am showing the adjusted and unadjusted numbers.

It should be noted that without correcting for the number of consoles sold, the total number of copies sold from all titles i looked at are significantly higher on the xbox 360.

Commutative unadjusted game sales for 56 leading titles by console (unadjusted):

Without correcting by the number of consoles sold there is a predictably higher number of copies sold on the Xbox 360.

The Commutative game sales from 56 leading titles by console, adjusted by the number of consoles sold:

As you can see - once we adjust to the number of consoles soled the commutative number of copies sold is almost identical. The green line at the bottom shows the difference between the consoles.

Estimating The Cost Of Piracy

The following graph shows the ratio of adjusted PS3 sales and Xbox360 sales.

I believe that part of this ratio can, at least in part, be explained by piracy.

According to these results the estimated loss due to piracy is around 5-10%.

However it should be noted that this is the case if (and its a huge IF) all of the difference between the sales of these titles between consoles is solely due to piracy.

There are many other factors that may effect game sales.

If we make the assumption that game piracy incurs a significant loss and is by far the most significant possible factor effecting sales of these titles between consoles than this number may be accurate.

Personally i do not believe this to be the case. I believe that there are so many possible different contributing factors to the differences in sales between consoles that they must contribute (positively or negatively) to the sales of games between consoles.

In other words - i do not believe that there is enough of a difference in the number of copies sold to reasonably claim that piracy incurs a measurable loss on game sales.

[1] - Good at being a copy protection. The most successful copy protections tend to be pretty evil in what they actually do.

[2] - We were wrong btw. it seems that games sell mostly during the 2 months before Christmas. This is probably why many releases are targeted around that time.

[3] - Sony BMG's care is perhaps the most famously intrusive and destructive of those.

[4] - Well, Nintendo would have you believe that there are 3 but you can't really compare the level of graphics or the gameplay in Wii to the other consoles. Xbox360 and PS3 are consoles for gamers. Wii is the casual gamer console. You also can't reasonably claim that the titles released to the Xbox 360 and Ps3 are the same as the ones released on the Wii.

[5] - This is the subject of many arguments. For the sake of full disclosure i own an Xbox 360. I (no longer) claim it has better graphics than the PS3. Only that it both consoles are practically the same.

This is the list of title's whose sales numbers i used:

Army of Two

Assassin's Creed

Bad Company

Borderlands

Bound in Blood

Brutal Legend

Burnout Paradise

Carbon

Dark Sector

Dead Space

Devil May Cry 4

DiRT 2

Fallout 3

Far Cry 2

FIFA Soccer 09

FIFA Soccer 10

Fight Night Round 3

Fight Night Round 4

Grand Theft Auto IV

GRID

Guerrilla

Guitar Hero III

Hell's Highway

Los Angeles

Madden NFL 09

Madden NFL 10

Mirror's Edge

Modern Warfare 2

Modern Warfare

Mortal Kombat vs DC Universe

NBA 2K9

Need for Speed Carbon

out.txt

PES 2009

Prince of Persia

Project Origin

ProStreet

Prototype

Pure

Saints Row 2

Sid Meier's Civilization Revolution

Sonic Unleashed

SoulCalibur IV

Street Fighter IV

Tales of Vesperia

The Force Unleashed

The Simpsons Game

Tiger Woods PGA Tour 09

Top Spin 3

Undercover

Wet

World at War

World in Flames

World Tour

WWE SmackDown vs Raw 2009

WWE SmackDown vs. Raw 2010

Aspect Based INotifyPropertyChanged Implementation

2010-02-17T01:44:00.012+02:00

There are quite a few articles floating around about how to implement an aspect oriented version of INotifyPropertyChanged Interface.

My version is based on an article by Tamir Khason Posted on CodeProject.

The original version had some problems - placing the Attribute on both a Parent and Child class caused an exception to be thrown and because the interface was implemented at compile time by PostSharp* it could not be used without an unsafe cast.

My version requires that the class implement the interface directly, adds an interface that reports which property that also provides the pre-change value (useful when you want to save undo for example) and checks with any implementing class to see if anyone is registered (and skips all other checks if no one is listening...). We then check to see if there's any modification and only fire notification on change - this might not be the required behavior in all cases so the check can be easily removed.

So, performance is probably better, usage is pretty straight forward - implement INotifyPropertyChanged and IFirePropertyChanged and mark your class with the NotifyPropertyChanged attribute.

If this is implemented on a base class all children will also fire modification notification on all properties. I am adding a sample base class implementation as well.

Any class that extends this base class gets a free change notification for all properties.

Example Base Class:


using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using AutoPropertyChangedWiring;
using System.ComponentModel;

namespace AutoPropertyChanged
{
/// <summary>
/// base class for any class that wishes to get INofityPropertyChanged for "free"
/// any class that extends this base class will get a PropertyChanged events, auto-implemented for all properties
/// </summary>
[NotifyPropertyChanged]
public abstract class PropertyChangeNotifier : INotifyPropertyChanged, IFirePropertyChanged
{
public event PropertyChangedEventHandler PropertyChanged;

public void FirePropertyChanged(string propertyName, object oldValue)
{
if (PropertyChanged != null)
PropertyChanged.Invoke(this, new PropertyChangedEventArgs(propertyName));
}

public bool HasPropertyListeners
{
get
{
if (PropertyChanged == null)
return false;
else
return true;
}
}
}
}

Aspect Implementation and base class:


using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Diagnostics;
using System.Linq;
using System.Reflection;
using System.Text;
using PostSharp.Extensibility;
using PostSharp.Laos;

namespace AutoPropertyChangedWiring {

public interface IFirePropertyChanged
{
void FirePropertyChanged(String propertyName,Object oldValue);
Boolean HasPropertyListeners{get;}
}
[Serializable,  AttributeUsage(AttributeTargets.Assembly | AttributeTargets.Class, AllowMultiple = false, Inherited = true),
MulticastAttributeUsage(MulticastTargets.Class, AllowMultiple = false, Inheritance = MulticastInheritance.Strict, AllowExternalAssemblies = true)]
public sealed class NotifyPropertyChangedAttribute : CompoundAspect {
public int AspectPriority { get; set; }

public override void ProvideAspects(object element, LaosReflectionAspectCollection collection) {
Type targetType = (Type)element;
foreach (var info in targetType.GetProperties(BindingFlags.Public | BindingFlags.Instance | BindingFlags.DeclaredOnly).Where(pi => pi.GetSetMethod() != null)) {
collection.AddAspect(info.GetSetMethod(), new NotifyPropertyChangedAspect(info.Name) { AspectPriority = AspectPriority });
}
}
}

[Serializable]
internal sealed class NotifyPropertyChangedAspect : OnMethodBoundaryAspect {
private readonly string _propertyName;
private object oldValue;
private Boolean fireEvent;
public NotifyPropertyChangedAspect(string propertyName) {
if (string.IsNullOrEmpty(propertyName)) throw new ArgumentNullException("propertyName");
_propertyName = propertyName;
}

public override void OnEntry(MethodExecutionEventArgs eventArgs) {
var instance = eventArgs.Instance as IFirePropertyChanged;
fireEvent = true;
//check if anyone is listening.
if (!instance.HasPropertyListeners)
{
//no one is listening, no point in moving on.
fireEvent = false;
return; //no need for the other checks
}

var targetType = eventArgs.Instance.GetType();
var setSetMethod = targetType.GetProperty(_propertyName);
if (setSetMethod == null) throw new AccessViolationException();
oldValue = setSetMethod.GetValue(eventArgs.Instance,null);
var newValue = eventArgs.GetReadOnlyArgumentArray()[0];

if (IsEqual(oldValue,newValue)) //nothing changed, nothing to do..
eventArgs.FlowBehavior = FlowBehavior.Return;
}
protected static Boolean IsEqual(Object left, Object right)
{
//both are null == both are equal.
if (left == null && right == null)
return true;

//at least one is not null, if its left - lets use it for the "Equals" method.
if (left != null)
return left.Equals(right);
else //left is null, but both are not null - which means right is not null and they are not equal.
return false;
}
public override void OnSuccess(MethodExecutionEventArgs eventArgs) {
if (!fireEvent)
return; //no need to fire the event.
var instance = eventArgs.Instance as IFirePropertyChanged;
instance.FirePropertyChanged(_propertyName,oldValue);

}
}

}

* - This probably goes without saying - Using aspects requires Postsharp to be installed (v1.5)

Simple Technologies For Better Car Safety

2010-02-15T01:52:00.006+02:00

I've been thinking about car accidents recently, or rather, about ways to prevent them.

I believe that at some point in the coming decades cars or whatever form of personalized transport we use will be fully computerized. Ultimately i do not think that regular public transportation will ever replace personalized transportation, but i do think we might have some form of automated point to point mass transportation system.

There are many different technologies currently being developed in the area of driving assistance, and some systems have even been implemented. Usually the way this works is that advancements are first implemented in luxury cars, and some of these technologies eventually become more widespread.

Some of these advancements involve various driving assistance systems, systems that "see" the road and can provide warnings and braking assistance in case the driver gets too close.

These technologies have a long way to go before they can be truly effective in preventing car accidents in common road cars. But i think that it is possible to implement some simpler technologies that may help us get closer to this future, and help us reduce car accidents in all cars.

Enhanced Brake Warning:

The first idea is to implement a slightly more sophisticated brake light system.

Current braking lights are pretty much boolean - on or off.

I think it shouldn't bee too complicated to have levels - say 3 levels (maybe even three lights) indicating "slow" brake "medium" brake and "hard" brake.

Modern cars have computers that control and monitor many of the car and engine functions, it shouldn't be too hard to implement a system that controls the brake light by the actual deceleration of the car instead of their current boolean implementation.

Why will this help? The way things work at the moment, when you see a brake light you don't know how quickly and how hard you need to brake. Your brain has to watch the car ahead and calculate the deceleration before you can decide how hard to brake. I think this period of time can be reduced and help prevent or reduce the damage in many accidents.

Inter-Car Signaling:

The Second idea is a little more complicated, but still in the not-too-hard category.

There are various radar and vision based technologies used to measure the distance and velocity of nearby vehicles. These are pretty expensive at the moment and don't work well enough to depend on.

It should be relatively cheap to implement the following:

If a standardized radio transmitter , broadcasting a low power signal with a known, well defined (and fixed) frequency and power. It should be relatively simple (and cheap) to roughly determine the distance between a receiver and a near by source. The idea is to optimize such a system to detect the distance between the receiver - your car - and the transmitter of the car ahead of you, behind you, and in your general area. to be affective and reduce interference the signal should be relatively weak and pretty directional.

By placing the transmitters in the corners of the car and designating a specific signature for each transmitter, it is possible to get a pretty good idea of the relative position of adjacent cars.

If we also broadcast the speed of the car using these transmitters we can very accurately know if a car ahead of us is braking, if there might be a car coming fast in our direction when crossing an intersection.

It should be pretty cheap to manufacture if standardized, and more importantly if mandated by governments. this kind of device can be easily installed on existing cars.

The system can be used for anything from advanced warnings for dangerous situations to actively engaging the brakes in dangerous situations - this last option would probably be only applicable to computer controlled cars.

I believe that the future of transportation is in more advanced version of computer controlled transportation. I think that computer controlled transportation can be far more efficient in traffic direction and possibly make car accidents a thing of the past.

WeakReference for lazy loading and releasing

2009-12-04T00:38:00.005+02:00

One thing that always bothered me when working with nhibernate and lazy objects is the following problem:
hibernate will automatically generate a wrapper proxy for your objects and fill them only when a data is requested. so far so good, but what about when i want to get rid of them?

Lets start at the beginning.
Many applications require processing large datasets, these datasets are often way too large to fit in memory.
Another thing that is common is that many applications do not require all of the records all the time.
Because of this ORMs like hibernate support lazy loading, you pay with an extra fetch for the benefits of a faster initial query and less memory usage.
When a proxyed object is accessed, their data is loaded into memory and remains there until some explicit action is taken.
in any case, a proxied object will remain in memory as long as their session is active, and they are attached to it.

Now, lets look at the case of an application like the one i am currently implementing, i have a very large dataset that i wish to present to my users.
They can only see and use a small portion of the dataset at a time, but they need to be able to access all of it.
Our interface of choice is a single data table, and our record counts are hundreds of thousands for the average case and millions for the larger sets.

Loading 100k records into memory, while possible takes about 24 seconds on my machine, and consumes about 100mb of ram.
Obviously 24 seconds is too long for a user to wait until they get their data (There are several sets of 100k, which we switch between) and 150 mb is waay to much as it doesnt scale for our worst case, on an average user's machine.

Its not hard to imagine that many other applications have similar requirements - handle large volumes of data without requiring insane amount of memory and ages to load.

The classical solution to this problem is pagination - we divide our dataset into segments and execute a query to load the required page.
This solution works pretty well in many cases, but in my particular case unless the pages are very small, page loading will be visible (And annoying) to the user.

So the problem is this - how do i load data only when it's required and also allow it to be released when its not currently being used.
I would also like the solution to be as simple as possible, and require as little management as possible.

So, after thinking about it for a while i came up with this:
Create an object that will hold some way of retrieving the full record (non lazy).
Hold a WeakReference in that object pointing to the actual object.
Expose a property that will check the reference, return the instance or reload it if its not there.

The solution looks like this:


class WeakWrapper<WrappedData>
{
long id;
WeakReference reference;

public WrappedData Data
{
    get
    {
        WrappedData result;
        //check if something is set in our reference
        if (reference != null)
        { //we have - lets see if it is valid
            result = reference.Target as WrappedData;
            if (result!=null) 
               return result; //return the data.
        }
        //no reference, load our data from the db.
        result = LoadData();
        //set the reference
        reference = new WeakReference(result);
        //return the result
        return result;
    }
}
/// <summary>
/// This method loads the data from an nhibernate session 
/// using the record id.
/// </summary>
/// <returns></returns>
private WrappedData LoadData()
{
    ISession session = ActiveRecordMediator
   .GetSessionFactoryHolder()
   .CreateSession(typeof(EntityType));
    //load all the entities
    
    IList entities =
        session.CreateCriteria(typeof(EntityType))
  .Add(Expression.Eq("Id", id))
  .List();
    //release the session
    ActiveRecordMediator
 .GetSessionFactoryHolder()
 .ReleaseSession(session);
    
 return (WrappedData)entities[0];
}
}

This solution works relatively well, but when the DataGrid that displays this data is scrolled down it takes a noticable amount of time for elements to load.
The main reason for this is that elements are loaded individually.

To avoid loading them individually, we group the calls and then load all the elements in a single statement.
To acheive this i used a nice trick i found on Tomer Shamam's blog.
Tomer did implemented a similar mechanism using a custom collection and a caching mechanism.
He used the Dispatcher Thread's prioritized InvokeLater method to deffer the invocation of the fetch until several calls have been made, and only then load the data.
This method works much better then a single fetch method.
My version, using The WeakReference mechanism is implemented via a modified version of the WeakWrapper, and a Bulk Loader that fetches the record via the deferred loading operation.

My particular implementation uses nhibernate, and an "int" id.
these can be easily changed for any kind of fetch (in the loader's load method) and any kind of "id".

The modified wrapper looks like this:


/// <summary>
///  A class that holds a weak reference to an entity.
///  it requires an entity that implements from IEntity 
///  (which contains a single property int Id)
/// </summary>
/// <typeparam name="EntityType"></typeparam>
public class WeakEntityWrapper<EntityType> : INotifyPropertyChanged
   where EntityType : IEntity

{
private static BulkEntityLoader<EntityType> loader
                          = new BulkEntityLoader<EntityType>();
public event PropertyChangedEventHandler PropertyChanged;
private WeakReference _objectReference = null;
private int _objectID;

/// <summary>
/// create a new wrapper with the provided id.
/// </summary>
/// <param name="objectID"></param>
public WeakEntityWrapper(int objectID)
{
    this._objectID = objectID;
}
public WeakEntityWrapper(EntityType entity)
{
    //get the id
    this._objectID = entity.Id;
    //save the reference
    this._objectReference = new WeakReference(entity);
}
public EntityType Entity
{
    get
    {
        //check if we have an initialized reference
        if (_objectReference!=null)
        {
            EntityType result = 
      _objectReference.Target as EntityType;
            //check if we have a valid target
            if (result!=null) 
                return result; //return the target.
        }
        loader.LoadEntity(this);
        return null; //nothing for now, but soon! muahahahah
        
    
    }
    set
    {  //the loader got back to us - 
    //set the entity and tell whoever is listening
        _objectReference = new WeakReference(value);
        //notify anyone who is listening
        if (PropertyChanged!=null)
            PropertyChanged
     .Invoke(this,new PropertyChangedEventArgs("Entity"));
    }
}
/// <summary>
/// the persistant id of the object.
/// </summary>
public int ObjectId
{
    get { return _objectID; }
    set { _objectID = value; }
}


}

As you can see, the main difference is that the Load method has been externalized
and that we now have a "set" method (that will be called by the loader.
This method will notify the viewer that it needs to update itself again, and this time it will get a non-null reference.

The BulkEntityLoader uses ActiveRecord and NHibernate and looks like this:



/// <summary>
/// uses deffered actions to group loading operations.
/// this class is NOT thread safe. 
/// it expectes to be called by a single (UI) thread.
/// 
/// </summary>
/// <typeparam name="EntityType"></typeparam>
class BulkEntityLoader<EntityType>
 where EntityType : IEntity
{
protected static readonly ILog Logger = 
                 LogManager.GetLogger(typeof(BulkEntityLoader<EntityType>).Name);
//here we save all the deferred wrappers for later
private Dictionary<long,WeakEntityWrapper<EntityType>>  deferredWrappers 
                         = new Dictionary<long, WeakEntityWrapper<EntityType>>();

//flag to mark the that the deffered action should only happen once
private volatile bool _isDeferred = false;

/// <summary>
/// NOT THREAD SAFE. FOR UI THREAD ONLY
/// 
/// mark an entity for future loading.
/// </summary>
/// <param name="wrapper"></param>
public void LoadEntity(WeakEntityWrapper<EntityType> wrapper)
{
    //check if a deffered action was already set in place
    if (!_isDeferred)
    { 
        //mark our flag
        _isDeferred = true;
        //no deffered action, create a new one.
        Dispatcher.CurrentDispatcher.BeginInvoke(
           DispatcherPriority.Render,
           (Action)LoadEntities);
        
    }
    //check if the wrapper is in our collection, if not - add it.
    if (!deferredWrappers.ContainsKey(wrapper.ObjectId))
        deferredWrappers.Add(wrapper.ObjectId, wrapper);
}
/// <summary>
/// Load all the entities from the wrapper dictionary.
/// </summary>
public void LoadEntities()
{
    Logger.Debug("Starting deffered action.\n"
             +" deferring "+deferredWrappers.Count+" elements");
    using (new SessionScope(FlushAction.Never))
    {
        //open a session
        ISession session = ActiveRecordMediator
       .GetSessionFactoryHolder()
       .CreateSession(typeof (EntityType));
        //load all the entities
        IList entities =
            session.CreateCriteria(typeof (EntityType))
                          .Add(Expression.In("Id", deferredWrappers.Keys))
     .List();
        //release the session
        ActiveRecordMediator
                .GetSessionFactoryHolder()
  .ReleaseSession(session);


        foreach (EntityType entity in entities)
        {
            //set the values
            deferredWrappers[entity.Id].Entity = entity;
            //remove all entities that were set
            deferredWrappers.Remove(entity.Id);
        }
        //mark the end of the deffered action.
        _isDeferred = false;
    }
}
}

So basically we collect all the calls we get until the Dispatcher decides to invoke us.
Once the dispatcher has time to invoke us, we process all the calls in a single query and set the property for everyone that requested an update.
Once set, the property will fire a PropertyChange event, that will inform the UI that it needs to reload these properties.

So now all that remains is loading a list of ids from the database, and setting the collection in a DataGrid.

The Strong Type Inheritance Pattern

2009-11-27T01:01:00.013+02:00

Generics were originally designed to help avoid very common mistakes while coding with various simple structures. The classic examples for this are Lists and HashMaps.
Employing these objects would result in confusion in the best case, and abuse in the form of multi-type elements in the worst case.
Generics are a an elegant solution for this problem - A Simple List<MyClass> makes a world of difference in both readability and usability of the code. It will save you both a cast and the occasional ClassCastException.

So what else are generics good for?
I think they open up new and interesting design patterns, one of may favorites is the Strong Type Inheritance Pattern (i am unaware of a different name for it).

The Strong Type Inheritance pattern is a simple pattern that allows you to make use of an inheriting type's "Type" to create typesafe methods that would otherwise require more complex, less efficient and more error prone reflection.

The example below shows a simple serializer class that provides serialization to inheriting types in a typesafe way. it also uses the type name as the file name.


/// <summary>
/// This is a save/load from xml facility.
/// it uses the class type as the file name, 
/// and so can only persist a single instance of each type.
/// </summary>
/// <typeparam name="InheritingType"> 
/// The type of a class inheriting from this base class
/// </typeparam>
public abstract class SelfSerializingBase<InheritingType>
where InheritingType : SelfSerializingBase<InheritingType>
{
/// <summary>
/// Load the inheriting type from an xml, 
/// use the type name as the file name
/// </summary>
/// <returns></returns>
public static InheritingType LoadFromXml()
{
    //create a serializer for the inheriting type
    XmlSerializer serializer = 
        new XmlSerializer(typeof(InheritingType));
    //get a stream to load from
    FileStream stream = 
        new FileStream(typeof(InheritingType).Name, FileMode.Open);
    //deserialize and return the result
    return (InheritingType)serializer.Deserialize(stream);
}
/// <summary>
/// Save the current instance to an xml.
/// Use the type name as the file name.
/// </summary>
public void Save()
{
    //create a serializer for the inheriting type
    XmlSerializer serializer = 
        new XmlSerializer(typeof(InheritingType));
    //create a new file
    FileStream stream = 
        new FileStream(typeof(InheritingType).Name, FileMode.CreateNew);
    //serialize ourselves to the file.
    serializer.Serialize(stream, this);
}
}

/// <summary>
/// A freeloading child that gets everything for free 
/// and doesn't do anything itself.
/// The code for the child is even shorter than our summery!
/// :)
/// </summary>
public class SerializingChild : SelfSerializingBase<SerializingChild>
{
public int SomeProperty { get; set; }
}

static class TestClass
{
/// <summary>
/// An example of how we would use the methods from the parent.
/// </summary>
public static void TestSerialization()
{
    //create an instance
    SerializingChild child = new SerializingChild();
    //set our property
    child.SomeProperty = 10;
    //serialize the instance
    child.Save();
    //load the instance
    child = SerializingChild.LoadFromXml();

}
}

Lets examine the base class definitions, which is what this pattern is all about:



public abstract class SelfSerializingBase<InheritingType>
where InheritingType : SelfSerializingBase<InheritingType>

So what we have is a base class that has a seemingly cyclic definition.
It expects a generic type , and it also expects this generic type to be an extender of itself.

The intention here is to say that inheriting types should pass their own type if they wish to use use these facilities.
An important part of this pattern is the limiting "where" clause that requires any type that wishes to use this base class to use a child which inherits from this base class. This limit is not required by the functionality - this class could easily have been a "serializer" facility that provides a load/save functionality, but in our design we wish to mandate that ONLY inheriting types are allowed to have access to this functionality.
The Generic magic here is that we get a static method of any extending type which is type-safe and implemented just one.

Ok, so we can use the type of an inheriting class and we force this type to inherit from the base - but what is it really good for?
A good example for using this pattern is in castle's ActiveRecord. ActiveRecord provides various database access facilities to inheriting classes, it is effectively a thin wrapper across NHibernate's functionality ( - in this implementation, ActiveRecord itself is an interesting and useful pattern for persistent objects).

This is only a partial example however, because castle's ActiveRecord source code does not include the "where" clause, which should be a part of the pattern - since the purpose of the clause is to inform (and force) the developer using this class that the intention of the framework is that you extend this class.

If a developer misunderstands the design of a base class implementing a Strong Type Inheritance they will get a complier error and hopefully read the documentation and change their implementation to comply with the intended design.

One important point we must pay attention to when we use this pattern is that the generic type is expected to be the FINAL type of the entity, if we wish to extend the base class with another class that adds functionality but is still abstract - we must add the same InheritingType to its class signature. once we set an actual type to the generic definition, we sealed all future generations from using the base class's generic facilities.

For example - if we happen to extend SerializingChild with say YetAnotherChild, we are exposing a "save" method that can only save the components of a SerializingChild. This is a strange an unexpected behavior for a developer using YetAnotherChild and will likely lead to abuse and inevitably to increased cost of maintenance.

Typesafe abstractions with .net generics

2009-07-31T01:14:00.043+03:00

The common example for generics in .net (and java) is usually for typesafe collections and typesafe operations. Safety in the workplace is very important so it's a good idea to use generic collections whenever possible. However generics, especially in .net (java generics are mostly fake) can be very powerful things to incorporate as part of our design (Made up fact - did you know that catching a ClassCastException is the no.2 cause of head injury among developers? The no.1 cause is NullPointerExeption!).

So the first example i want to talk about is of using generics to help us define a clear relationship between an abstract base class and it's children.

Generally speaking, we use an abstract base class when we have some common responsibilities for all the children, but the base class itself lacks some functionality and cannot be instantiated.

We can use a generic type definition for the abstract base to further refine this relationship and provide the inheriting classes with a type specific method signature they need to implement.
Without generics we'd be stuck with the lowest common denominator (just like on the tonight show).

So here is an example of some abstract base class that does some work, and delegates the type specific work to the children:


public abstract class AbstractProductCreator <SourceType,TargetType>
   where SourceType : Material
   where TargetType : Product
{
   public TargetType CreateProduct(SourceType material)
   {   //let's call this "buisness logic" :)
       preprocessMaterial(material);
       //get the product from the inheriting child
       TargetType product = ProcessMaterial(material);
       //do some more "buisness logic"
       finalizeProduct(product);
       //return the completed, type specific product!
       return product;

   }
   private void PreprocessMaterial(Material material)
   {
       //umm..lets say we melt it...
   }
   private void FinalizeProduct(Product product)
   {
       //put it in a shiny box?
   }

   //the only method our children will have to implement
   public abstract TargetType ProcessMaterial(SourceType source);
   //without generics this method signature would be:
   //public abstract Product ProcessMaterial(Material material);

}

So our AbstractProductCreator does some type specific work with the generic level he knows and hands it to the child for the specifics, and then finalizes it before returning.

our child only needs to do what it is supposed to, and to define its own rule (what gets converted into what).
Here's an example of a famous inheritance of this base class:


class IPhoneCreator : AbstractProductCreator <RainbowsAndBunnies,IPhone>
{
public IPhone ProcessMaterial(RainbowsAndBunnies rnb)
{
   //this is the actual implementation of the IPhone creation process.
    return new IPhone(rnb.GetBunnies(),rnb.GetBunnies());
   //its, um..copyrighted i think..so..um..dont tell anyone you saw it here..
};

}

Once we define our child, we completely define the scope of our responsibility.
Visual studio's Intellisense will also implement the skeleton of the method for us if we perform the right ceremony. (right click on the abstract class name+ implement methods if i recall correctly).

The downside of this pattern, and its an unfortunate common pitfall of generics in general - is that once you start defining something in generic terms, it tends to infect your application.
Anyone who uses an instance of our children doesn't have to worry about the generic type definition, but usually some other layer of our application that orchestrates a more fundamental mechanism doesn't really care about the specific type and wants to treat everything in more abstract terms.

This layer may want to have access to the AbstractProductCreator, but cant because this layer will have to define a variable that looks something like:


AbstractProductCreator <SomeMaterialtype,SomeOtherMaterialType> creator;

This sort of thing is really very far from what we originally intended.
So what we want is some way of abstracting things for the lower layers of the application into the most general terms they need to know about while maintaining type specific implementations in the higher layers.

So, how do we solve this problem?

There's a nice method i came up with after thinking about this problem for a few days.(challenge: without reading on, try to think of a solution. code it (the first few attempts are likely to reach some dead end). if it can compile and run, let me know how long it took you to solve it).

The solution:
Our abstract base class can already invoke the typesafe methods of it's children. now we only need some way of invoking the abstract base without all the generic additions.
To do that we add a new interface!


public interface ProductMaker
{
    public Product MakeProduct(Material material);
}

//Our base class can implement this interface and

public abstract class AbstractProductCreator <SourceType, TargetType>
    where SourceType : Material
    where TargetType : Product
{

    public Product MakeProduct(Material material)
    {
        //make sure someone is not trying to hand us the wrong kind of material.
        if (!material.GetType().Equals(typeof(SourceType)))
            throw new Exception("Type " + material.GetType() + " not supported  by" + this.GetType());
        //the only cast ugliness we introduce.
        return this.CreateProduct((SourceType)material);
    }


    public TargetType CreateProduct(SourceType material)
    {
        //same as before..
    }

}

Now the lower layers can refer to the abstract base through the interface, and not worry about the Generic type definitions at all!

i.e.


maker.MakeProduct(product);

Without generics we'd be lost. No generics means we can either do the type specific implementation and lose the abstraction, or use the abstraction but then no one can use the type specific and all the inheriting classes have to do the casts themselves.

I hope you enjoyed this article,any comments/questions/criticism/praise/award nominations are welcome as usual.

On the next episode i'll give an example of how to extend this concept into a constructing a binding layer that allows both the lower and upper layers to be separate and type specific.
I will also be exposing the manufacturing process of the mac book air.

Thinking outside the checkbox

2009-07-28T00:32:00.016+03:00

I was talking to dan - a good friend of mine, and he was telling me a horror story about purchasing a shiny new netbook. He was overall very pleased with the hardware: the size, noise level, price and performance all suited him perfectly for his application - a battery backed webserver he can put in a closet and forget about. The problem was with the operating system, or rather the flood of popups, warnings, wizards and welcome messages that took a good 15 minuets to clear.

You see, dan is by no means a novice when it comes to using computers. He spent a few years doing system administration and is one of the best developers i know. But he spent the last couple of years using a macbook and probably more then that since his last windows installation. He has long forgotten what it feels like to configure a windows machine after it was installed.

When you install office 2007 there is some live plugin that can be installed. if it is installed you are faced with a welcome screen. you have a checkbox you can check if you dont want to see this message again. then you have two options - click ok, which supposedly also takes you through some configuration, or click cancel.

What do you do?

If i check the checkbox and cancel, will it show up next time? if i click ok, do i have to go to another annoying popup?
In my case it turns out neither option workes to disable this popup because a bug in the installation process failed to create the registry entry for this plugin. Since the checkbox was never saved it kept popping up no matter what combination of checkbox+ok/cancel i used.

I am a great believer in keeping things simple. it doesn't matter if you are designing a framework, designing your UI or writing a document. Things should be as simple as possible.
Solving a complicated problem in a complicated way is easy. solving a simple problem in a complicated way is easier. As a common rephrasing* of H.L Mecken's quote goes -"Complex problems have simple, easy to understand, wrong answers". This is why i try to keep things as simple as possible. The challenge is to keep the simple things simple and complex things as simple as possible.

Why am i telling you all this? Because i think that it's the lazy and indecisive designer that leaves all the options open and visible. I think they are lazy because they didn't take the time to think about how they can simplify things down to their core essentials. I think they are indecisive because they make someone else choose instead of researching and thinking about the problem enough to reach a decision themselves.

If i was designing an application with a "welcome" or "splash" or any other annoying popup screen, and 99% of the users would - as their very first user experience with my application - click on a checkbox that means "stop annoying me" and cancel, i didn't go that extra mile.Or lets be honest, that extra inch. I think that if every developer who creates such a page received a dollar whenever someone actually read the welcome screen and lost one whenever they check the box and cancel we'd have some very poor developers and very few of these messages.

I think the key is to design for simplicity.To try not to popup to many windows, make as many decisions as you can for your users. Use everything at your disposal to limit the information presented to the user to the minimum, relevant information to the particular view/function of the frame they are looking at. If a view deals with more then one "topic" or "issue", it should probably be separated. I also think one must always consider the limited nature of a checkbox before using it as the means to answer a question.
Oh, and try to avoid the "we'll make it floating/detaching/docking/toolbox" type solutions - these are usually the lazy indecisive type solutions. most users will never change your default setup. they don't care that they can configure and change almost anything. they just want to use it without spending hours configuring it to be usable.

Another thing i think desktop application developers should do is take some notes from the online world. These days many websites track very carefully how users get to their site, where do they go, where do they exist at,they follow every user's clickpath through their website, they track ad clicks, and any other statistic they can receive. This information is easy to get and can be very useful.

In the offline world this sort of thing is a bit more complicated. i did a surface search to see if there are any common frameworks for tracking this sort of thing, or any common methodology for this and i couldn't find one.
Wrapping all your controls in UI usage statistics gathering can be a bit of a hassle, but i would expect the leading providers of UI controls to pick up the glove and implement such mechanisms, along with their analysis tools. Frameworks that help create clean and usable UI will add great value to application developers.
I think developers of desktop applications should use their QA and beta testers to gather this sort of statistics btw - don't ask your users- it annoys them, and it makes them wonder how exactly you will be tracking them.

I'd like to add that usage tracking is to UI usability as profiling is to performance - it helps you solve problems and improve the situation, but it only solves a problem you already created.

As for dan, well - he did what he had planned from the start - he installed ubuntu. you can visit his netbook here.

* - the actual quote is "There is always an easy solution to every human problem—neat, plausible, and wrong."

Blast From The Past

2009-03-31T00:07:00.015+03:00

A long long time ago, in the days before the Weird World Web. In a time of BBS’s and modems. A time when viruses roamed every floppy disk and computers were IBM Compatible.
In that faraway past just before the last decade of the 20th century a game was created by Activision.

This game was far ahead of it’s time. So far in fact that since its creation it was never again duplicated. Many tried, none succeeded.

I am speaking of course about Death Track (cue insane laugh).

You may find yourself thinking: "Death Track?... Death... Track? Really? I mean Death and Track?? "
Yes. Really!

I know the name is pretty lame (lame is also pretty lame)* but the game is not. You see, Death Track had everything. It had cars with cool and exciting weapons, it had contract killing and it had upgrades.

Ok, let me explain:
Death Track was the first Racing+Shooting game i ever played. I have played all racing+shooting games i could get ever since. No game was ever like Death Track. It may just be the nostalgia talking but i really think that at least until this point in time, it has never been surpassed in it's genre.

Many games since added some element of weaponry or destruction, but no racing game i know was solely dedicated to killing off your opponents. To facilitate this the game offered a wide array of weapons:
Machine guns, Lasers, Mines, Missiles, Ram Spikes and more. All the weapon systems in the game were upgradable. This was a very important element in the gameplay. you always wanted more: better weapons, better armor, faster engine. There were plenty of things you wanted to upgrade. Some helped you make money by winning races while others helped you win races by killing off the competition.

Of course, the competition didn't wait around for your to kill it. They would go after you as well. The game also featured contract eliminations that won you large sums of money if you succeeded. These were substantial enough to focus on killing your target even if you didn't get 1'st prize from winning the race.

I think that the upgrade elements were an important part of what made it so addictive. The game managed to always keep you wanting more, and you got to feel a real sense of acheivement when you got that next upgrade. And it payed off too. There is nothing more satisfying than blowing up your nemesis with your new tracking missiles.

I wonder why this was never done since. There are plenty of racing games around, and plenty of racing game engines. Many of which feature arcade to ultra-realistic damage systems. I would really love to see an arcady Need For Speed based game combining weapons, upgrades and total destruction.

On that note i would like to say that many of the console racing games have this element of certain upgrades opening up only after you played part of the game. I think this is really the wrong way to keep players interested. Money is a great way to achieve this: Its clear and its a simple way to give you multiple alternatives. If you feel that the only thing stopping you from getting to that next upgrade is X amount of $ its a whole other thing then knowing you need to complete the next stage or circuit. The fact that you have to comparmise and think about what to upgrade adds certain elements longer term strategy.

In many games you don't even know what's available. Give me the option to buy things and make the purchase worth-while. Let me feel i get an edge, a sort of "cheating" advanted over my oponents that really makes a difference in the game. The best way to keep gamers interested is by dangling the possibility right in front of them, and allowing them to choose how they get it.
If i want to play that same level 10 times on an easier setting to get the money for the upgrade i want - give me that option.

Its now twenty years since Death Track came out, and it seems like i should maybe let go of my dream of playing a game like that again on modern hardware, engine and graphics. But if there ever was an industry that keeps recycling the classics its the gaming industry.

My only fear is that it comes out exclusively for PS3...

Addendum:

Mmmm....I have just been informed that i am an idiot. turns out that in the very link i added was the link to the New version of Death Track!
I am downloading the demo now. Thanks Dan, I Can't tell you how exciting it is to finally play Death track with modern graphics.

* - That's a triple lame sentence. This is the first time such a fete has been attempted. Do not try this at home.

Trivial Statistics

2009-03-30T03:02:00.008+03:00

So we got the new Scene It? Box Office Smash for the xbox 360.
We really like trivia games and we especially enjoy the Scene It series (we also have Lights, Camera, Action).
We like the subject, but much more than that we enjoy the presentation. Questions like anagrams, pictogram, skewed images that gradually clear, sound bites that require that you pay attention to the details etc'. All in all its a well made game with original question presentation. It also has the advantage of being one of the too few 4 player games (on the same xbox).

However, there is a serious downside to the game and it has to do with the way questions are selected.
In Scene It? Lights,Camera, Action there were 1,800 new questions (For some reason this information on Box Office Smash is hard to track down). Unfortunately we never got to experience all these questions because relatively quickly we started noticing questions repeating. This got me thinking: there are 1,800 questions and there's no way we played long enough to see 1,800 questions, why are we seeing repeats?

The cause as it turns out, is statistics. Specifically a phenomenon similar to the birthday paradox.
Quick review for those who are not familiar with the birthday paradox and can't be bothered to read the wiki page:
If you sit in a room with a group of 23 randomly selected people (like a classroom) there's a 50 percent chance 2 people in the room have the same birthday.
for 57 people there's >99% chance for two people to have the same birthday.
To get the general idea why your intuition (that this is sounds wrong) is wrong, think of the 2 kids in your elementary school or high school class that had the same birthday.
Oh, and think of how many people you know that also had 2 kids with the same birthday in their class (hint: you went to school with them :) )*.

How is this the same for a game that randomly selects questions?
Without going into the math too much lets assume you played the game for a little while and you saw 180 questions (10%).
At this point, the probability that the next question is a question you already answered is ~1/10. This already is high enough to be discouraging. Worse yet, the expected number of repeat questions until you reach 180 questions is ~8.4! (because of the birthday paradox).
Now lets assume you played long enough to answer 600 (~33%) questions in the game. Out of the 600 question ~80(!!) are expected to be repeats. Needless to say, if you saw a third of the questions (you need to answer more then 600 questions for that) then one in every three questions will be one that you saw already.
At 1/10 its annoying. at 1/3 the game is unplayable. So everybody loses. You enjoy only about a third of the value you thought you were getting from the game, and the game developer works very hard to create 1,200 more questions that you will never see.

The good news is that there is a pretty simple solution for this problem that works out well for everyone.
You see, the problem stems from the method of randomly selecting the questions. Fortunately there is a different method that both guarantees randomly selected questions and no repetitions. instead of "rolling the dice" every time we need to select a game we select a permutation of the sequence of questions (in simple terms - we randomly rearrange the sequence of questions) and save this permutation. This guarantees that you will see all 1,800 questions before you see a question you know.
Since 1,800 is a decent number of questions there's a good chance that you will forget the earlier questions in the sequence. But even if you don't - at least you get to enjoy all of the questions in the game.

BTW: On xbox.com Feature listing for the game mentions a "minimal repeat" feature for the game which keeps track of questions answered to minimize repeats.
I don't know how that feature is implemented, but i can tell you from experience that it doesn't seem to work very well as we saw plenty of repeats. Besides, there's no reason the number of repeats should be anything other then zero.

Oh, and as for the issues of online play and playing against different players etc'. There are solutions to all of these problems (generate a permutation for the unseen subgroup on all player's lists, you can also add a timestamp to the question to ensure a 'long' time between repeats). I know it's not ideal and i know that there are details to work out. But i believe that this is a core issue for any trivia game and a great game needs to give the very best possible solution.

One final point, and i may be way off on this one. I think if i knew that i was running out of questions or if there was some way of indicating to me that i saw most of what's available, it would encourage me to get the question packs. It might be nice if this was done automatically (you only have 200 questions left, why not try the XYZ expansion).

* - yes, yes, i know *technically* its statistical lie. But what better way to fight faulty statistical intuition than faulty statistical intuition?

Amazing!!! The Future of cell phones is here now!!!!

2009-03-23T02:20:00.005+02:00

Check this out, it’s this amazing invention! The MODU!
It’s a cell phone, right?

And, get this – it comes in different shapes!!!!

Its so cool and innovative!!!!
I mean, think of the possibilities: you can have like , a yellow slider pone, or like an angular little phone, or this tiny thing with no numbers on it, or like a crazy psychedelic phone with like really small buttons!!!

They will make millions! Billions even! Of possibly dollars!
And you will never believe the specs! Its 2.5 Gen (like the amazing non 3g version of the iphone 1.0)
It has Bluetooth (like the iphone)
It doesn’t have a built in GPS (like the iphone non-3g)
And it has 2 Gig of flash (less (thus lighter) then the cheapest version of the first iphone)!!!
You know, it amazingly also costs like half of what the iphone costs, plus you get 2 jackets! So it’s like 4 times better! Actually you can use the modu without the jacket, so its like 6 times better! So its even mathematically proven to be superior! With numbers!

And if that’s not enough, they have a touch screen. But unlike the silly iphone, it’s not going to do things when you touch the screen, only when you touch the buttons. I mean, the iphone hardly HAS any buttons, the modu has 7 buttons on the tiny version alone! It’s like, obviously if you want to do something you click the BUTTONS.

To show how amazing and cool the modu is, I have made this table of comparison.
Using this table you can clearly see the superiority of the Modu!

	Amazing Modu	Silly Iphone
Jackets	4	0
Buttons	7	2
Guinness world records	1	0
Waste battery on WiFi	0	-1
Waste battery on GPS	0	-1
Can become a car radio	1	0
Total awsomness	>1,000,000*	0

* - math is for silly iphone geeks.

Oh, full disclosure - i have one of those silly iphone things.

War of the Worlds

2009-03-19T02:24:00.010+02:00

H.G Wells got the title right. Indeed there is a war going on. A War between Worlds. War between the world of man and the aliens that live amongst us.

This is the story of one man’s struggle in a battle against the odds to defeat the invaders from another world.

It all started years ago when I was living in an apartment not very far from the botanical gardens in Jerusalem. i suspected nothing when I moved in. There was no sign. No declaration of war. No warning shot. They always came at night, just as I was getting to sleep. They waited, sticking to the walls, biding their time. Suddenly – bang! One of them would create a diversion. I did not realize this at the time, but they wanted to get me out of the protection of the covers. They wanted me to bring back the light so they could mark their target.

As soon as the light came on they would start flying around it, circling it in widening and narrowing circles. Looking for an opening to strike. When the light was off again they would send their kamikaze fighters to crash into me as I was trying to hide. There was nothing I could do, they were too many.

I decided to beat them at their own game, they will not outwit me with their mind games. I will prepare a trap for them! I opened a window, and lured them out with the porch light. As soon as they got out I would shut the window and turn off the light.

This worked for a while, but they finally figured out my little ruse. Instead of all going at once they would send a scout. If he got trapped they would immediately counter attack, smashing into everything!

Then I tried keeping the window shut all the time. They really didn’t like that. I was sitting in my room one day, minding my own business, when suddenly I was hit in the back of the head. In the middle of the day no less!

I tought maybe i should try and reason with them: “Try to see it my way” I said, “Do I have to keep talking ‘till I can’t go on?”

They circled the lamp to show their agreement, so I continued:

“Think of what I’m saying, We can work it out, and get it straight or say good night”.

They obviously didn’t like the idea of night, as one of them did a flyby as a warning. I figured they must be angry about something. Perhaps because they were only here for a short time. Maybe they feared night time as every night brought them closer to their inevitable demise. I thought it was worth a shot , so i looked them strait in the eyes and in a calm and direct manner I told them:

“life is very short, and there’s no time for fussing and fighting my friend”.

“This has to stop”. I was shocked! They can talk! Moths can talk!

“what do you mean” I said hesitantly.

“It’s bordering on copyright infringement”.

“huh?” I was baffled.

“The Beatles song. Listen, we’re from the RIAA. “

“The RIAA??”

“We are here to monitor file-sharing activity.”

“Shouldn’t you be monitor internet traffic then?” For some reason this seemed like the proper response.

“We tried that for a while, but they told us we should get out more, do some field work”

“Field work?” this was getting stranger by the minuet.

“Packet sniffing”.

“Packet sniffing?”

“We have a very keen sense of smell, we are very small, and we can get into many places. Naturally, we are the best candidates for the job”. They sounded very proud.

"I see." i really didn't. "Well, why here then?"

"We heard reports about music coming from this location"

"Mmmmm... ok, But music is not illegal right?"

"You are replicating Copyrighted materials"

"No i am not, i am just listening to music!"

"So you admit it! The analog replication of digital music is prohibited under copyright law"

"But..But.....um...I am pretty sure listening to music is protected under fair use"

"Many do. They are wrong."

"So you're saying i am not allowed to listen to music?"

"Under U.S Copyright law making ten copies or more is considered a felony"

"So i can listen to a song ten times?"

"Technically nine, ten would be a felony."

"And this doesn't seem ridiculous to you?"

"We are talking Moths working for the RIAA, what do you think?"

On the importance of Garbage Collection

2009-03-17T16:21:00.042+02:00

Update (21/10/2011):
It's been more then 2 years and it seems Infragistics have done nothing about this. They made a very idiotic design decision to sacrifice a fundamental behavior of managed code and then insist on not fixing it. Reader Jarrett posted an update that extends the fix below for win7 and Infragistics v10.3 and possibly 11.1 - I've added it to the code below.
Thanks Jarret!

---------

About two years ago we found a nice UI package for WinForms called NetAdvantage by infragistics.
A very nice pack, lots of useful controls and not too expensive. Maybe a bit slow, but this is not a problem in most applications.

For a while all was good and well, we built our application and our clients were happy.
Then one day - disaster!
Our application started crashing!
This was after a rather long development cycle where we added support for many new features in the hardware (this was a management application for specialized hardware).
Worse yet, we only started noticing it in QA. The reason was that it only happened after sending configuration to the hardware about 10-15 times without restarting the app - something we almost never do in dev.

Well, when your app crashes after repeated anything its very natural to suspect a memory leak. So i carefully watched memory consumption while calling send configuration. Memory consumption was increasing on task manager, but this is to be expected in managed environments (the simple explanation is that new memory is allocated as long as possible before old memory is reacquired. the actual explanation is not very simple :) ) . Since the machine had 2 gigs of memory and the app only grew from about 150 MB to 200 MB before crashing i figured it was something else.

What i did notice during my exhilarating time with Task Manager was that User Objects were growing rapidly. Being originally of the java persuasion i was unfamiliar with these User Objects. Turns out it was to do with windows handles and the silly way win32 api does UI.
It also turns out that windows has a limit of about 10,000 open handles. That seemed like a lot to me and i didn't understand how we could even approach this limit so fast. Sure, we have a lot of complex UI elements, but still - firefox was no.2 in the object count on my machine and only had 600 handles. Our app had~3,000 just after startup. This didn't seem likely.

The thing was handle leak and memory leak were very similar under managed environment. we used the excellent dot Tracer to locate the source of our leak. After a few hours staring at rather complicated object graphs it turned out that the leak came from...well...i guess my opening was a dead giveaway... - Infragistics Controls!

Anyone who is a developer in a managed environment is well aware of the crimes associated with keeping references to One's object when they have been removed from scope. It's very easy to do it by accident. a Hashmap that wasn't cleared correctly, an errant thread that keeps holding your objects or misunderstood relationships with event handler. All these and more were explored during my search for the leak. All these were not the case with Infragistics.

No - they did it on purpose.
Huh? you say. How, or why, would anyone do this on purpose?
The short answer is "i don't know". The longer answer is that they wanted a mechanism that will allow them to change the look and feel if the user changed the windows theme while the program was running. To do that they hooked into some theme change events and maintained links to every control. The short answer still holds tho , because its a really bad excuse. As i pointed out to Infragistics, you can easily use WeakReference to completely solve this problem.

During this ordeal i was talking to Mr. Vince McDonald, the manager of dev support at Infragistics. it took me a very long time and many lengthy emails and an almost religious debate until i finally got him to admit it was indeed a design flaw. He informed me that "..we will not be able to apply any changesfor it as part of a hotfix" as "..any such changes would have a very high chance to destabilize our current systems". he also promised that "..changes we may make can only safely be done as part of a volume release".
Well, this sounded reasonable to me so i decided to wait with this post. However 3 volumes have been released since my original conversation with him and the problem has not been fixed.

Despite a complete lack of support from Infragistics, we were finally able to resolve the issue ourselves. We had to use a rather nasty trick to bypass their mechanisms.
Basically we reflected their objects and forcefully cleaned all the references.
The code to resolve the problem is posted below.
Important note - This fix was prepared for v7.3 and may work with 10.3 (Thanks Jarrett) I dont know if it will work for any other version. I Suspect it might, and i would appreciate it if you tried it on a different version and it worked for you.

using System;
using System.Collections.Generic;
using System.Text;
using System.Windows.Forms;
using Infragistics.Win;
using System.Reflection;
using Infragistics.Win.AppStyling;
using System.Diagnostics;
using Snoops.UI;
using System.Collections.Specialized;
using System.Collections;

namespace CleaningServices
{

///  IMOPORTANT NOTICE:
///  THIS HACK WORKS ONLY WITH Infragistics V7.3
///  AND POSSIBLY 10.3 (Thanks Jarrett)
///  IT WAS CREATED FOR THIS VERSION ONLY.
///  IF IT WORKS WITH ANY OTHER VERSION, ITS BY CHANCE ALONE.
///
///
/// <summary>
///  ReleaserControl is a special control designed to rid our code of the ugliness introduced by using the Infragistics UI Package.
///  Unfortunately someone in Infragistics made the "Design Descision" (according to their support personal) to turn all their UI objects into unmanaged object.
///  They acheived this by strong binding every control to static events held by static classes deep within the infrastructure.
///  As a result, if a UI class holds an infragistics control within it's Controls list, it will never get garbage collected.
///  Infragistics controls NEVER get garbage collected, unless Dispose is explicitly called on the control.(the Dispose method releases the even bind).
///
///  WTF??? you say, well, yes. since every control holds a strong reference to Parent, and the control itself is held
///  by a static event handler, your controls will not get collected when they get out of scope, if they hold an infragistics control.
///  this will naturally cause a memory leak. this in itself is not the worst of it. since all controls hold a Windows Handle,
///  your app will crash with the lovely "Cannot Create Window Handle" exception somewhere around 10,000 objects.
///
///  So, how did we solve this problem? well, we didnt REALLY solve it. we did a hack.
///  this seems to work for our current usage of the package, and i did not notice any unexpected behavior in the UI controls.
///  the limitation of this fix is that it was created for the Infragistics package v7.3 and might not work for any other version.
///  that's the way hacks go i'm afraid..
///
///  What does our hack hack do? well, it forces unregistration of infragistics controls from all the static places we could find them binding to.
///  calling the InfragisticsCleaner.ClearEventBinding(); causes all events for all controls to be unregistered, allowing them to get GCed as they should.
///  the price we pay is that some windows events will not be handled this way,(like changing themes) , but we are willing to live with that.
///  There might be other implications..we dont know.
///
///  for ease of use we added the ReleaserControl that extends UserControl. all it does is add an event handler
///  which calls the InfragisticsCleaner.ClearEventBinding(); method whenever a control is added.
///  extend this class in your user control to allow your controls to get GCed again.
///
///  a once in a while thread calling the ClearEventBinding method would probably work just as well, but it might not get called
///  exactly when you need it...
///
/// </summary>

public class ReleaserControl : UserControl
{
public ReleaserControl() : base()
{
this.ControlAdded += new ControlEventHandler(ReleaserControl_ControlAdded);
}
void ReleaserControl_ControlAdded(object sender, ControlEventArgs e)
{
InfragisticsCleaner.ClearEventBinding();
}
};
public class InfragisticsCleaner
{
private static StaticPropertyHolder[] properyHolders = new StaticPropertyHolder[]
{
new StaticPropertyHolder(typeof(StyleManager), "styleChangedDelegate", "StyleChanged"),
new StaticPropertyHolder(typeof(Office2007ColorTable), "colorSchemeChanged", "ColorSchemeChanged"),
new StaticPropertyHolder(typeof(Office2007ColorSchemeChangedNotifier), "colorSchemeChanged", "ColorSchemeChanged"),
new StaticPropertyHolder(typeof(RoleSelectionUI),"queryComponentRoleDelegate","QueryComponentRole"),
new StaticPropertyHolder(typeof(XPThemes), "themeChangedDelegate", "ThemeChanged") 
};

public static void ClearEventBinding()
{
foreach (StaticPropertyHolder holder in properyHolders)
holder.ClearEvents();
AcessibleTextManagerCleaner.CleanDictionaries();
}

class AcessibleTextManagerCleaner
{
ListDictionary[] dictionaries;
static AcessibleTextManagerCleaner cleaner;
//initialize the static cleaner instance.
static AcessibleTextManagerCleaner()
{
String[] propertNames = new String[]{ "SubclassList", "ControlList", "EditorList" };
cleaner = new AcessibleTextManagerCleaner(GetAccesibleTextManagerType(), propertNames);
}
public static void CleanDictionaries()
{
foreach (ListDictionary dictionary in cleaner.dictionaries)
dictionary.Clear();
}
private AcessibleTextManagerCleaner(Type accessibleTextManagerType, String[] dictionaryPropertyNames)
{
//retrieve the singleton instance.
PropertyInfo instanceField = accessibleTextManagerType.GetProperty("Instance", BindingFlags.NonPublic | BindingFlags.Static);
object instance = instanceField.GetValue(null, new Object[0]);
//get the properties
dictionaries = new ListDictionary[dictionaryPropertyNames.Length];
for (int i = 0; i < dictionaryPropertyNames.Length; i++)
dictionaries[i] = ReflectListDictionary(dictionaryPropertyNames[i], instance, accessibleTextManagerType);
}
//get the dictionaries to clear.
private static ListDictionary ReflectListDictionary(String propertyName, Object instance,Type type)
{
PropertyInfo instanceField = type.GetProperty(propertyName, BindingFlags.NonPublic | BindingFlags.Instance);
ListDictionary list = (ListDictionary)instanceField.GetValue(instance, new Object[0]);
return list;
}

private static Type GetAccesibleTextManagerType()
{
//we need the infragistics assembly that holds our type (its a private type so we cant access it directly)
Assembly infragisticsAssm = Assembly.GetAssembly(typeof(EditorWithMask));
Type[] types = infragisticsAssm.GetTypes();
Type accessibleTextManagerType = null;
//search through the assembly until we get what we want.
foreach (Type t in types)
{
if (t.Name.IndexOf("AccessibleTextManager") != -1)
{
accessibleTextManagerType = t;
break;
}
}
return accessibleTextManagerType;
}

};
class StaticPropertyHolder
{
EventInfo eventInfo;
FieldInfo field;

public StaticPropertyHolder(Type type, String delegateFieldName, String eventName)
{
this.field = type.GetField(delegateFieldName, BindingFlags.Static | BindingFlags.NonPublic);
eventInfo = type.GetEvent(eventName);

}
public void ClearEvents()
{
Delegate eventDelegate = field.GetValue(null) as Delegate;
if (eventDelegate != null)
{
Delegate[] invocationList = eventDelegate.GetInvocationList();
foreach (Delegate del in invocationList)
{
eventInfo.RemoveEventHandler(null, del);
}
}

}

};


};
}

Mocking The Monkeys

2008-11-11T01:51:00.011+02:00

So I was listening to this lecture about developmental psychology, and the lecturer was talking about the experiments they were doing with young children. Every once in a while she would go back and compare the performance of the children to that of monkeys. Now, I am not sure who should be insulted, but I am fairly sure someone should. There seems to be a good case for the kids since the research equates their mental capacity to that of a monkey. The monkeys on the other hand also seem to have a strong case as no one seems to try and challenge babies with climbing a tree or living with Jane Goodall.

You hear stuff like monkeys can do this , monkeys can do that all the time, and I would always say it sounds interesting…sounds interesting, But is it really? Is it interesting? Its not like there's some actual work we are planning on having monkeys perform, right? Its not that we can gain some deep and meaningful understanding of life, the universe and everything by having monkeys perform these weird tasks.

Now, this sort of thing presumably took quite a bit of work to get to, you sometimes have to live with the monkeys or train them to perform or understand certain rules, and you have to do controls…very hard work. And its not just limited to monkeys, there's the whole story with Skinner and his pigeons, rats and mazes, and generally speaking a whole lot of experiments involving animals performing all kinds of strange tasks.

So I had to ask myself,"why?" , "why go to all this trouble?" ,why would anyone spend all this time and money to show that a monkey has the mental capacity of a two year old? And then it hit me – why does Jay Leno ask random pedestrian to locate Iraq on the world map(obviously its somewhere in Australia), or what the DC stands for in Washington DC (don’t believe all that district of Columbia stuff. It obviously stands for De Capital). Also, why are there shows like "are you smarter then a 5'th grader" or "beauty and the geek"? simple – to show how people are stupid so that you can sit at home and mock their ignorance.

Sure, mocking people isn’t "nice", its entertaining, but its not nice. Still, must we mock the monkeys? i mean, there are so many things humans are ignorant about, surely our time and resources are better spent mocking one another.

So please, if there's one thing you take from this post, let it be that "Prison Break" stopped making sense once they broke out of prison.

Computer Voodoo

2008-04-24T02:12:00.017+03:00

The following is a true story, and unlike most stories that begin with this kind of statement, it actually is true!

I know this for a fact, as I was personally involved with this voodoo magic.

For the past few days, my computer has been acting strangely with everything USB. Windows started throwing annoying messages about how my "USB Device is not functioning" claiming that "USB Device Not Recognized" it was all very annoying, even more so because when I connect a mouse or a webcam, it’s very obvious to ME that they are a mouse or a webcam. For some reason windows disagreed with me...even stranger, it recognized these devices just fine before and I couldn't see any reason it might suddenly forget. I tried reasoning with Windows, explaining to it that it makes no sense for it to claim that whatever was working not 2 minutes ago is suddenly malfunctioning beyond recognition. This explanation left little impression on windows and it insistently repeated that “USB Device Not recognized”.

Of course, it's a well known fact that computers are, for the most part, malfunctioning piles of junk which for some reason we care deeply about and take good care of only to have them reject us at times of grate need. In some sense, they are very much like ungrateful teenagers (who, btw are probably the largest group of computer users. coincidence? I don’t think so!).

So anyway, back to my story.
Since for the most part the devices were such that I used rarely, I did what most users would do, ignore the problem and hope for the best. Unfortunately, the problem didn't think that was such a good idea and got worse until the system decided to reboot (it rarely asks my opinion of such things. i guess it assumes it knows better than I do what I should do with the next 5-10 minutes of my life, namely, stare blankly at the screen while as it boots). Even more unfortunately was that my previously functioning webcam stopped being recognized. So again, I did what most users would do - I connected it to a different USB port...windows wasn't impressed, so I resorted to Google (fortunately my USB based cable modem decided to work for the time being).

After a few minutes of browsing the various forums I found a collection of solutions from removing all the devices in safe mode, to reinstalling windows. Well, I didn't much feel like either, so I kept reading...and one unlikely solution kept popping up. This solution involved what is technically known as "Computer voodoo".

"Computer voodoo” is a highly complex technical term that loosely means "doing something really weird in the hopes that it will somehow fix your problem". This voodoo is usually accompanied by an unlikely explanation, targeted at the computationally (and when it comes to computers – rationally) challenged. A regular voodoo solution would look something like: "your windows keeps crashing because of a virus! Bang your mouse on the table tree times and left click then right click". While the voodoo explanation would be something like: "banging your mouse encourages the virus to migrate into the mouse's memory. left clicking + right clicking then erases the mouse's memory , killing the virus!" (I am sure that I could convince at least some of you of this in other context).

voodoo solutions, as well as voodoo explanations exist for the same reasons superstition exist - people will do and believe anything if said in a convincing, authoritative tone! They will believe even more if it’s written down.

The thing that separates regular Voodoo from Computer voodoo is the absurdity of the proposed solutions, and the incredibly large amount of otherwise intelligent people that are willing to participate in strange rituals if anything electronic is involved (Reboot voodoo is the most common, replacing the ancient bang-it-twice-on-the-side voodoo used by my ancestors)

In the case of my strange problem the voodoo solution involved the following - shut down your computer, disconnect all devices , disconnect main power, wait for half an hour. The voodoo explanation given was something about static charge accumulating on the USB hubs. If we analyze this voodoo, we can see that there are 4 separate voodoos involved:

Shutdown the computer – this is a variation of the restart voodoo, but separating the voodoo into its ingredients, first the shut down, and later, the start up.
Disconnect main power- another common voodoo. It is believed that electricity holds magical powers and can infiltrate through the sturdiest of switches. Disconnecting a device from the main power helps to convince it that it’s really off, forcing the electrons to leave the possessed device.
Disconnect all devices – follows from the main power voodoo, the electrons might not truly leave but take residence in the attached electronic devices, and then return once the main power is connected. Only way to be sure is disconnect all of them.
Wait for half an hour – all things must rest; this is the basic assumption behind this voodoo. Your computer has been hard at work and its acting up because you never let it get a good rest. Leave it alone for a while. When you come back it will be well rested and eager to work.

In any case, I decided I would try this solution, I didn't really expect it to work in spite of the various user comments like "I don't believe it, but it worked" (seriously , who are you kidding, this is EXACTLY the sort of thing claimed by people who are trying to trick you into buying things on the shopping channel)

Why did I try it you ask? I am a firm believer in the least effort approach. Other solutions required that I DO stuff, this one merely involved my NOT DOING things. I consider not doing things to be considerably easier and much less damaging than doing things, so I shut down, disconnected the power cable, and waited.

...
...

Obviously, you all expect me to say that I turned my computer back on and, that to my surprise everything worked.

Well, that's exactly what happened (had you there didn't I)!

As you all know, I believe in computer voodoo as much as the next ex-technician/support guy/programmer. As the creator of many voodoo explanations myself, I can usually tell when someone is trying to get me to do something really silly, just for the perverse pleasure of knowing that somewhere out there people are turning off their computers and just sitting around waiting for things to fix themselves...but I have to tell you. It really does work.

Finally, I recommend that next time you have a problem, not just computer related. Shut down and wait for things to sort themselves out. The passive act of waiting facilitates the flowing of cosmic rays and realigns the stars and atoms along the energy lines in the universe, smoothing the flow of good Chi and flushing out the bad Chi.

true story.

Silence of the blogs

2007-09-24T03:11:00.000+02:00

Some say silence is golden.

Of course, they didn't even bother following their own golden rule, so we shouldn't pay any attention to them. I, however, was not lost for words, but an overload of work left very little time for some of my favorite things. One of them is writing here, but others are watching TV, and of course, playing computer games (how's that for segue of the year?).

Most ‘real’ adults I know don’t actually play computer games (no , soliter and minesweeper don’t count). I am talking about “hard core” games. RPG’s like gothic (one of the best RPG’s ever. Well, two actually. Gothic 1 and 2 where amazing. Gothic 3 was an awful mistake that I am willing to overlook if they change it back and fix the engine in gothic 4) and Never winter nights. One might also expect me to mention oblivion, but the truth is it’s a terrible game with an amazing graphics engine. The problem with oblivion btw is that it takes the whole fun of RPG’s out of the game by creating a game system that upgrades your enemies according to your current level…and this means you simply can’t progress in the game. Doing a beginner quest later on is just as hard as if you would have done it in the beginning….it doesn’t “keep the game challenging” as some misguided game designer might think, it keeps you from getting anywhere. Imagine a life where things never get harder or easier…they are just the same all the time. Sure, the graphics are great, but the plot leaves something to be desired, and you can never achieve anything.

Hmm…it seems I strayed from my point there….what I was saying is that “true adults” don’t seem to play computer games, and I think it’s a shame. I bought myself an xbox in the beginning of the year. I’v always wanted a game console as a kid but my parents couldn’t afford one. They also considered it to be a waste of money (‘why would you spend that much money on a game anyway’). They were, however, willing to buy a computer because we managed to convince them it was ‘educational’. Turns out it was. Wanting to play computer games motivated me into learning how to tweak the mighty Autoexec.bat and the holy Config.Sys. Words cannot describe the joy of getting your conventional memory up to a free 620k (and I could even manage 632k in some cases). It was almost as satisfying as playing the game. So the years passed, and it turns out that playing all these games and learning how to tweak things, finding new ways to cheat at games by freezing and changing memory addresses and hex editing save files really played off. These days I can afford to do all the things I wanted to do as a child thanks to these computer thingies.

The problem is that our society is structured in a very annoying way, now that I can afford to do all these cool things I am considered too old for them (which doesn’t usually bother me…) but mostly, it doesn’t leave you time to do all the things you CAN do. Time also takes away some of the energy, although I have an untested theory that you actually gain energy by running around, playing with your friends and doing fun and exciting things. they also took all the fun junk food away. When you’re a kid, your parents limit your junk food consumption because it’s not healthy for you (you don’t care , because you’re a kid and you know what’s fun) , but as you grow older , you start to believe that junk food is actually bad for you. Hell, it’s not even just junk food anymore..as time passes it seems food in general is just out to get you and is looking for new ways to kill you (mad cow, cancer , allergies, free radicals, confined radicals, toxins, chemicals, monosodium glutamate to name a few). The only things that are good and healthy are things you don’t really want to eat. And even if you find something that you like to eat and is both good and healthy, then you still have to watch out for the ominous cholesterol creeping through your veins…really, it seems some people can’t enjoy life , and somehow they manage to convince the rest of us that we shouldn’t either. Even if we still do things that are fun , they find all kinds of ways to make us feel guilty about it.

So I bought myself an Xbox. I don’t have as much time as I did when I was a kid to play games. Usually I prefer spending time with my girlfriend, watching TV, reading or having a drink with my friends. But every now and then I still find the time to play a new computer game (maybe I’ll buy that ultimate gaming pc I always wanted..which will be outdated a month later J ) or a cool game on the xbox (last one was BioShock… Unreal engine 3 is truly Unreal… ) . And I wish to say to you : buy your xbox, travel to that place, do that thing you wanted to do as a kid. Maybe people will tell you it’s silly or a waste of time and money. Don’t listen to them - I promise you, it’s worth it.

File sharing vs, Copyrights

2007-08-20T02:07:00.000+03:00

Copyright laws have been the subject of much debate in recent years. With the passing of USA's Digital Millennium law (DMCA) many individuals have become the target of lawsuits from large and powerful media corporations. These corporations are now testing the boundaries of the mandate given to them by the U.S. legislator. On the other side of the debate (and perhaps the law) are the File sharing networks, allowing the mass distribution of digitized content, often copyrighted. This discussion is an attempt to make the case for both sides.

FileSharing – The file sharing industry have been a major pain for the RIAA , they have been trying to shut down any network they can. They got Napster, Kazaa, and recently Edonkey as well. Of course, the donkey network is still alive thanks to the Emule team. And the torrent network is very active. Also, the newsgroups are a source of content – its still file sharing, just not bandwidth/disk space/p2p sharing. While the law may unjustly support the prosecution of people who distribute copyrighted content, it has nothing to do with p2p file sharing itself. This technology has many different uses.

Copyrights – you can't seriously argue that the file sharing networks are a harmless tools for sharing original or non-copyrighted content. File sharing network transfer consists almost entirely of copyrighted materials – movies, music, games and other software. A tiny percentage of the data transferred is free information, such as open source software distribution.

FileSharing – well, I’m sure you know the "guns don’t kill people…" argument. The fact that file sharing network CAN be used for transferring copyrighted content is not reason to eliminate those networks. I’m sure you would also love to see rewritable digital media eliminated, along with hard drives, TiVo, flash-memory based devices and practically anything else that can be used to copy copyrighted content. But you can't start banning technology just because it might be used for illegal activities.

Copyrights - guns are used primarily to kill people. Much in the same way file sharing networks are used primarily to "share" copyrighted content. The difference is that guns are heavily regulated in most countries. The same goes for drugs – they can be used legally and illegally. Legal use of drugs is heavily regulated – you need a prescription to get any drug that might be addictive, and you are monitored while using it. None of this can be said for the file-sharing networks. They are completely unregulated. In fact, they are almost impossible to regulate – you need to penetrate many layers of anonymity before you can find out who is the person sharing material you have copyrights for. The RIAA does not have the legal authority to regulate file sharing networks, and even if the laws where changed to include regulation such regulation would probably be technologically impossible. The only real option is to seek legal action against providers of file-sharing services and against users of such networks who share material they don’t own the copyrights for.

FileSharing – so your argument is that if we can't regulate something, then we should ban it completely? This is a very problematic argument – first of all how do we define the limits of file sharing, is it the applications? The protocols? The concept? Second, many things cannot be regulated – the internet itself cannot be regulated. Should we bad website construction tools or website hosting because these things can be used to publish child pornography?

Copyrights – well, first of all, I am not saying that anything that cannot be regulated should be banned. Second, I am not saying that anything that might be used for immoral or illegal purpose must be banned or regulated – but I do think that in the particular case of file sharing networks, the primary use of existing technology is the illegal, immoral sharing of copyrighted material. Should the situation change – either by finding a way to regulate file-sharing networks, or by a comprehensive change in the type of materials shared on these networks, then I would have no problem with the technology itself. But as long as the primary use of these networks remains what it is today, I consider the providers of such services to be accomplices to these illegal activities.

FileSharing – this brings us to another major point of the argument – I can't argue with the statement that sharing copyrighted material is an illegal activity – there are laws in many countries that define such acts to be illegal. As soon as you manage to create a strong lobby supporting such laws - you make such activities illegal. In the 1920's the United States banned the production, transport and sale of alcohol. By instituting prohibition, the US government made a very popular activity illegal, and in the process made many people felons, giving rise to powerful criminal organization that still exist today. The argument made against the use of alcohol was its immoral nature, in the same way that copying copyrighted material is said to be immoral today.

Copyrights – the fact that many people wish to consume alcohol or share copyrighted material does not make such activities morally correct. I do not wish to defend prohibition, as I do not consider the consumption of alcohol to be immoral or to be a catalyst of immoral activities. I do wish to point out that alcohol use IS regulated - in many countries there is a limitation on the legal drinking age. As for stealing copyrighted martial – I don’t see how the immorality of such activities can be questioned.

FileSharing – by using the word "stealing" you effectively called millions of people world wide "thieves". I doubt that many members of the file sharing community would agree with that categorization. And yes – I do believe actual thieves would agree to call themselves "thieves". Theft and stealing is a concept generally reserved for physical property. By replicating copyrighted material the owner of the media in question does not lose anything – unlike actual theft – the theoretical offense is made here not against the owner of the physical media (if such media exists) , but against the owner of the copyrights to the material recorded on the media. When talking about theft you should demonstrate that one side has gained something, while the other side lost something. If no one lost anything, then how can you argue that something was stolen?

Copyrights – I do not agree that there was a "theoretical" offense, and that no one lost anything. If a person wants to use or own materials created by other people , it is reasonable to require that this person be given permission by the author to do so, if by paying for the permission, or by some other means acceptable to the author of the work in question. The loss here is not theoretical , its very real – when you download copyrighted material and enjoy the product without paying for the right to do so – a right other people DID pay for – you are denying the author appropriate compensation for the time, effort and creativity he invested.

I feel I cannot end this discussion with a clear conclusion, as there are valid points to be made for both sides. I also can’t keep arguing with myself forever :)

Join us next time for another argument of me vs. myself. And please feel free to suggest subjects for discussion.

What the $&@^&*#???

2007-08-18T00:43:00.000+03:00

This article was written by Adina Lederhendler:

Over the years several people have asked me for my opinion on the film "what the bleep do we know". I hadn`t seen it and assumed it was another popular-science film trying to explain quantum mechanics. I saw the film today and really feel like I have to set the record straight. The film is not, as it seems, a scientific documentary, but is actually a promotional film for a spiritual movement based upon the teachings of JZ Knight (who appears herself in the film as one of the interviewees), a woman who founded "Ramtha`s School of Enlightenment", Ramtha being the spirit who Knight claims to channel.

My objection to the film does not stem from the ideas that are portrayed in it. In fact I agree to some extent with the claims that our emotions and past experience effect our reality and that there are psycological effects on biological phenomena (the placebo effect, for instance, is an example which is widely accepted as valid). Even the more radical ideas that the universe is actually created by our thoughts and that our behavior is dictated solely by our bodies "addiction" to certain emotions didn`t bother me. I think people should be free both to believe whatever they want and be able to present their beliefs to whatever public agrees to hear them.

What I did find disturbing, however, was the presentation of these ideas as scientific fact. This was done so cleverly that it took me about half an hour to realize that what I was watching wasn`t just another pop-science film. We tend to regard an expert presented as a "scientist" to be both objective and trust-worthy. While many (but not

all) of the interviewees in the film were in fact scientists, they were far from objective. The most impressively credential-ed scientist in the film is John Hagelin, who tells us about both his Harvard degree and his work at CERN. What he fails to mention, however, is that since 1984 he has been the head of the physics department at the Maharishi University of Management, founded by an Indian guru. His research concentrates on trying to link particle physics to Transcendental Meditation. (He has also run for President three times as the leader of the Natural Law Party).

The ideas presented in the film rely heavily on "facts" taken from quantum mechanics, specifically that the state of a system is not determined untill it is measured by an "observer". While the mathematical formulation of quantum mechanics can be "translated" into words in that way, like in any translation something of the substance is lost. In any case, attributing literal meaning to this formulation even with regard the sub-atomic world which it describes is controversial within the scientific community, and certainly no one would claim it applies to the world of people and basketballs and such.

But all this is just a manifestation of a much bigger problem. Our society relies heavily on science and technology. However, informed knowledge of these fields is held only by a small minority. The fact that scientific theories cannot be fully explained without the use of mathematics leaves many people in awe of science and they tend to credit it with disproportionate authority.

Another factor is a general misconception of what science is all about. The goal of science is to find the best way to describe nature, not to find an ultimate truth as to "what it is". For example, gravity is the best way we have found to describe how things fall to the ground, but even the most advanced theories of gravity will not tell you what it is! Most importantly, scientists don`t care. They might ponder philosophical implications of various theories as they look up at the stars on their way home from the lab, but it is very clear that these are purely the realm of metaphysics and not scientific research.

Finally (yes finally...) there is a dangerous lack of understanding of the scientific method. The scientific method can perhaps be summed up

as: "When I see it, I still won`t believe it". In fact, a true scientific theory cannot be proven, and a good scientist spends most of his time trying to disprove his theory. As silly as that sounds, it is the only way to ensure that only the best ideas survive. A scientific theory is only accepted if it can accurately predict the results of many different experiments and the results of an experiment are only accepted if they can be replicated. Even then, scientists continue to come up with ideas to test the theory, to find the cracks.

Otherwise there is no progress.

So here`s where you come in. Do not (ever!) accept ideas or facts simply because they were presented by someone claiming to be a scientist or using scientific-sounding terminology. Approach these ideas or facts with caution: Is this person objective? Is he basing spiritual or philosophical claims on science? Are his methods scientific? Google it!!!!

I know I sound a bit dramatic here, but there are many people out there who abuse the standing of science in our society to promote their own agendas, and people are many times taken in on false pretenses.

But don`t take my word for it, check it out for yourselves

Planet of the spoons - Part 2

2007-08-14T01:58:00.002+03:00

Previously on planet of the spoons.
The spoons have constructed a machine that allows them to take a Quantum Leap and appear on distant lands, but the machine had one critical fault - those pesky quantums are always so uncertain. The spoons where debating who should be assigned to watch the machine, when the forks tried to take control of the machine. The spoons quickly realized that true unison is achieved by fighting against others. And so, the story of the spoons continues.

Many historians still debate how the spoons won the war against the violent forks, as it was not in their nature to be instruments of war. Some believe the forks where overpowered by brilliant tactical maneuvers. General Spoony , leader[1] of the spoon army explains : "Wars are not won by numbers or training, wars are won by kicking the enemy in the silverware when their back is turned."
However, other historians say the defeat could be best attributed to the fact that the forks didn't know where the quantum machine was, and where actually on their way to the annual fork convention.

Wars are known around the world as the leading cause of activists, and the Spoon War was no different. Even though the war was very short and was considered by all[2] to have been a clear victory, some still manage to find fault, as some always do.

The anti-war movement was led by Made-In-China - foreign ambassador to the Spoon Council, who criticized the war for being a half measure that would only lead to further conflict. The activists believed that the only way to avoid future wars was to get rid of all the enemies. Their argument was that wars are fought against enemies, so: No enemies = No wars. This would have been a popular bumper sticker, but none of the spoons owned cars (or bumpers).

A spoon mathematician later proved that the equation was, in fact, false. The anti war movement retaliated by defining said mathematician as "the enemy" and proved the opposite. The incident helped the activists realize that by naming new enemies and getting rid of them they can advance the coming of ultimate peace. unfortunately for the anti-war movement (but fortunately for everyone else) someone pointed out to them that by defining enemies, they were becoming enemies of their enemies themselves, and the anti-war movement was forced to eliminate itself.

....To (maybe) be continued...

[1] - no one could remember how he became general, nor could they agree on what it was exactly he did during the war, therefore they assumed it was the job of a general to take credit for victory.

[2] - All spoons. No one else seems to have noticed this war.

Planet of the Spoons

2007-08-09T03:53:00.001+03:00

A recent study proved that spoons are rapidly disappearing from our planet. some suggestions are made by the researchers as to where the spoons are disappearing to, the most promising suggestion is based the theory from The Guide - The Theory of Plane of the Spoons!(imagine hearing ominous notes)

As you all know, The Guide is the most reliable source of information and is widely regarded by the scientific community as infallible. With such a well established source of information as basis for this theory, i decided to go forth and extend the original paper. The results of my research will be presented in an n part series (n>=1). While the spoons eventually migrated into a utopian planet where they live a happy spoonful life, much controversy and conflict preceded their arrival at their new home planet. This part tells the story of how it all began.

Planet of the spoons resides in an undisclosed[1] location. it enjoys a cool average temperature of about 15 degrees and long walks in the park. It’s one of those rebellious planets that never joined up with a stable star system. it just flies carelessly through space refusing to revolve around a sun or trim it's pointy mountains. It likes to hang around asteroid belts and sometimes pretends to collide into a passing sun to scare the natives. The spoons didn't really want to settle on this planet, but the leaders of the exodus convinced everyone that its really a nice planet and he just got a bad rap.

When word got out that spoons are planning to migrate from earth into their own planet, other cutlery where skeptical of this move. Everyone knew that the knives were on the verge of achieving teleportation and that the forks have almost completed work on their faster than light drive, but no one imagined that the spoons would be the first to leave the planet. only when the first spoons started their migration did the cutlery world finally realize that the spoons were on to something. a committee was founded and a joint task force was sent to the leaders of the spoons in an effort to persuade them into letting everyone escape from the clutches of the evil humans[2]. The spoon leaders refused the request and were given an ultimatum by the knives to disclose the location of their planet and means of transport "or else...".

"or else what?" asked the spoon leaders. "um....ni?" said the knives. The spoons were not impressed[3]. Nevertheless, fearing the knives might come up with something more imposing to threaten them with, the spoons expedited their efforts to leave this planet. they were about to begin a massive quantum leap removing all spoons at once from the planet, when it was discovered in the nick of time[4] that this may result in all spoons being here and there at the same time (bloody quantums and their superior positions.) it was then decided to forgo the original plan and instead have someone stand by the machine to watch it and make sure the quantums behave themselves. This meant that someone had to stay behind. Since no one wanted to stay some method was needed for choosing the operators of the machine. at first it was suggested that everyone draw straws and this was generally agreed upon by everyone as the best method with the least amount of complications. However, the straws strongly opposed this method so a different solution had to be found. A vote was suggested, everyone could vote for the person they did not wish to see operate the machine. Naturally, the idea failed when everyone voted for themselves. This effectively destroyed all future hope for a democracy amongst the spoons as this attempt at voting clearly demonstrates that everyone only thinks about themselves and shouldn't be allowed to make decisions for other people.

After democracy was finally proven to be completely useless, a succession of regime changes followed as the spoons tried them all to decide which method of government might be the best one for their utopic planet, and more practically - to decide who stays to operate the machine. Unfortunately the lack of government prevented the spoons from coordinating their search for a better regime and all regimes were attempted simultaneously. Many spoons were lost during this dark period in spoon history. But fortunately for the spoons it didn't last very long. The forks were secretly trying to capture the machine and the spoons quickly united against them and blocked the attack. In the aftermath the spoons realized that the best leadership is not leadership from within, but from without! Instead of uniting with each other, they could unite against others!

....To be continued...

[1] - i have been sworn to secrecy and may not reveal the true location. i am only allowed to say that its most definitely NOT somewhere in the vicinity of betelgeuse.

[2] - almost all humans were considered evil and clutching , except for the Chinese since the chop stick population was widely ignored in those days*.

* - while usually the past is considered better then the present, some aspects of it are considered by most to be less then optimal. these days chop sticks are regarded as only slightly less equal then other kitchenware (the word cutlery is now considered a racial slur)

[3] - some suggest that it was actually the insidious forks behind the attempt to threaten the spoons. it is widely accepted that while ominous looking, the knives rarely work without the forks.

[4] - deceptively called nick, but actually played by johny

...And patents for all*

2007-08-08T01:52:00.000+03:00

I read an article which claims that "Software patents help heighten the standard by supporting the re-use of the code of established and tested systems" the theory behind this idea is very nice - if you give people patents, then they can afford (by selling the patented software) to build higher quality software and thus be able to provide the market with a high quality less error prone product. to me this article represents a deep misunderstanding of the software world. patents are granted for 20 years while software is outdated in 2-5 years at most (and 5 years is REALLY stretching it) things change quickly in the software world and that is the reason people code quickly and are forced to build and rebuild things. sure, some software still contains very old code, some systems are continuously developed over long periods of time. but every now and then you have to rewrite things. knowledge has progressed. the tools, languages , operating systems , design patterns and coding methods change over time. when enough time has passed your code becomes so outdated and inefficient that you're just better off starting from scratch. this doesn't mean you haven't learned anything - you gathered alot of knowledge on the way the software should function and look, and writing the system again is usually faster than writing it the first time. the added benefit of the world around you progressing usually results in new and faster more reliable ways of doing things. patents don't help improve the quality of software, they prevent people from using good ideas that were patented. you can easily write software today using advanced languages and support from the OS , that would have taken you months to write just 10 years ago. and the software you end up with will be more reliable , better looking and support more functionality that it would have 10 years ago. but to do that you must use new methods, languages and tools.

I was recently entertaining the idea of starting up a software company. its not very hard coming up with interesting ideas that can make alot of money. its also possible to get funding if you have a serious business plan and a good team. the problem is that if anyone is thinking of something similar to the thing you wanted to do, odds are they have patents in the field. this means that even if you are more capable then whoever else is doing approximately the same thing, you still won't be getting any money from investors - and for good reason , you might find out that you cant market your product because you might be sued for patent infringement. this goes against the very ideology behind a patent - if the idea is so innovative , how come so many people came up with it again and again? of course, you could say that with no patents to begin with, someone would just copy your idea. well, if your idea is easy to duplicate and trivial to come up with - i guess its not patent worthy to begin with. i know its pretty hard to give a general definition of what is a trivial idea, its also hard to agree on what idea is trivial and what idea is innovative. the thing is , you know it when you see it. if you know something about a certain field , you can say if something is truly innovative or just something a lot of other people can (and will) come up with (or may have already thought of). the one-click buy is such an example. the idea is so trivial i would have never in a million years thought it's possible to patent. amazon where issued their patent in sept 1999. they sued B&N by October. i don't know when B&N added the feature to their website, but i think no one can seriously claim they started work in September - when they saw the patent approval - and were done and launched the feature by October. it was probably in the works long before that, which only helps to demonstrate how obvious the idea is. really, this sort of thing should probably be grounds for revoking the patent rights. but then again - it should never have been issued in the first place.

* well, maybe not *all* - all , more like "if you have the money and are the first to ask for them" kind of all.

2007-08-05T04:09:00.000+03:00

The Israeli version of the 'American dream' has become in recent years the 'start-up' dream. as far as everyone is concerned you can come up with some silly idea get a patent for it and get rich. the concept is so wide spread that the expression 'that's a startup(company/idea)' meaning 'a trivial/obvious/stupid idea' has come to replace a previously wide spread expression 'that's a patent' - meaning the same thing . everyone here has some startup idea. mostly these ideas are impractical/impossible/useless/done-to-death/done-and-failed/already-exist[1].

So, following in the footsteps of my people i wish to present my new 'startup' idea - the amazing 'one click patent' software:
In this revolutionary new product you can enter your patent application and then get it reviewed and approved in just one click! with patent pending patent issuing technology our application can run through thousands of previously issued patents and cross check the validity of the patent pending patent request. With the new patent pending Insta-Pent™ algorithm we can scan all current human knowledge and judge the LOI (Level Of Innovation) in your patent vs. the current level of background innovation. once your application has surpassed the LOI required by US patent law[2] , our software automatically confirms your patent. if your patent request is below the minimal required LOI you are displayed a unique Insta-Ject™ screen showing our friendly (Copyrighted) message : "A thousand monkeys can come up with Shakespeare and all you can come up with is this? please hire more monkeys and try again later". So hurry and get your copy of "One click patent".
In the words of our advertising agency : "Get a patent today, before all the good ideas are taken!"

All kidding aside, the reason people think of a patent as something easily attainable is because it actually is. of course, i could be wrong, it might take a true genius to come up with one-click-purchase. personally, i think the conversation leading to the patent went something like this:

Marketing : "our users are not buying things fast enough!"
Development : "why not?"
Marketing : "well, i've done some extensive research and it seems that people are clicking on the wrong things in our website. they have to click again and again..and they get confused. i conducted this research on myself."
Development : "hmm...so , what do you suggest?"
Marketing : "well.. i dont know. this is a real though one. maybe we can add more flash animations, pop up windows and commercial banners?"
Development : "but wouldn't that cause the exact opposite of what you are trying to achieve?"
Marketing :"let me think about this for a minute....hmm.....yes..i see your point. ok , well then how about ..."
Development : "listen, why don't we just reduce the number of clicks a user has to go through when buying something? maybe we can use some sort of account"
Marketing :"oh, stop it with your technical terms, this is a real world problem. let me think about this scientifically. well, the minimal number of clicks a user can possibly make is two.."
Development :"what? why two?"
Marketing :"You see - thats what happens when they make me talk to technical people , everything has to be explained to you people - you have to click on things twice to get them to do stuff. it's known in the marketing world as 'twicely clickable' "
Development :"you mean double click?"
Marketing :"if you must use layman's terms.."
Development :"..um...you know, you don't actually have to click everything twice.."
Marketing :"of course you do."
Development :"no , you don't, look (click) "
Marketing :"what? let me try that, you're probably unconsciously clicking twice. ....(click)...hmm...(click again)....interesting. hey listen. this gives me an idea. why! this is brillant! what if we had a button on all our products that supports this single click technology? i'll have to research it, but im pretty sure no one has ever thought about this before. we should really patent this one"
Development :"patent? clicking on something once to buy it? your not serious?"
Marketing :"of course i am. you technical people cant appreciate real-world creativity when you see it."

[1] - pick any combination you like. as unlikely as it may sound, some of the things i heard are impossible and impractical and still have been done to death , and of course , failed because they are impractical , impossible and completely useless.

[2] - due to the level of innovation in our product we have advanced far beyond current patent application laws (apparently no one else seem to agree about the need and validity of our LOI concept..a mere technicality).therefore, predicting the coming change and definition of LOI we use the Amazon One-Click-purchase patent as a benchmark for all other applications. if you are at least as innovative as one-click , you are automatically granted the application.

The (role playing) games we play

2007-07-31T22:40:00.001+03:00

RPG games have been around for years now, and with the onset of computerized RPGs they have evolved beyond games once played by groups of spotty faced geeks. with the invention of the MMO world , RPGs are more popular today than ever in human history[1]! There are many elements that make RPGs fun, but i want to focus on one impotent aspect of the role playing world - the classes and levels system. when you play a role playing game you need to increase your experience to gain more power and magic. you have to learn to use your powers wisely if you are to progress. this crucial concept- experience-based power-wielding ability - is missing from the real world. especially in the software world.

I have been a programmer for a while now, and I've seen horrors coded in many languages. I've seen all kinds of dangerous 'language features' that should not be left unattended for the inexperienced programmer to abuse.
In the same way you will not let a ten year old operate heavy machinery while intoxicated(well, maybe for some harmless fun), you really shouldn't let computer science students use parameter overloading in c++ when they can barely comprehend the concept of functions.

This is why i think we should learn from the RPG world and implement a class and level system for the coding world. the details of such a system are presented here:

First you start apprenticing (Levels 1-3) and you are only allowed to use non languages like LOGO or HTML, after a while when you have proven yourself worthy, you might be allowed to progress to more advanced scripting[2] (Levels 4-7).you can use java/vb scripting but only in a web based environment, where the browser prevents you from causing too much harm. once you gathered more experience (Level 8-10) you will be allowed to choose a VB programmer class. experts[3] agree that VB is not really a programming language, so anyone who chooses to use VB is forever forbidden from writing code in any other language.
Anyone who tries to code in Perl will lose all their execution privileges, not that it matters since it seems Perl programmers are more concerned with how many "kewl" things they can do in a single incomprehensible line of code, they wont actually notice that the code doesn't execute.

if you decide to program in a real language you will be allowed to select one of the real languages (c, c++, java, c#, assembly is reserved to levels 50+) each language will have it's own restrictions for a beginner.

C :
beginner c programmer can only use if, for, while, switch, single parameter function (you can choose between return param or function param), and 3 global variables (any more and the beginner gets confused). loops cannot be nested until level 13, you get a nesting level for each level above 13. altho, if you need beyond 3-4 nesting levels for anything you will be converted into a VB programmer. function within function calls have a limited stack of 3 calls. this effectively disables the beginner from using recursion, and also helps reduce their confusion with all these functions calling functions. the use of pointers is only allowed from level 15, void pointers from level 20, you get a new level of pointer nesting beyond the first for every level beyond 21, function pointers starting level 35, malloc is disabled until you reach level 30. you get 1 ifdef per level starting with level 12, programmers above level 20 can use as many as they like.
syntactic sugar like using ++ is only allowed from level 14 and on.
copy and paste functionality is disabled in the IDE until level 18 to prevent use of the poor man's code reusability (lazy programmers that cant refactor things into functions).

C++ :
all limitations for c also apply to c++, with extra limitation for operator overloading (level 50+). the required use of classes lowers the required level for use of pointers to level 10 (entry level for c++) but only class and primitive pointers. the use of function pointers is forbidden in c++ (object orient damn it!) until level 50+ those who practice the black art of function pointers and operator overloading will be periodically reviewed by the great council :the ICCC[4]. any transgressions will cause immediate demotion to either VB programmer or perl coder.

Java:
well, no malloc and pointers, so that makes life a little easier, also use of classes is enforced by the language, and no param overriding or ifdefs so you can do less harm to begin with.
the limitations for java programmers like function call stack size, loop nesting etc. is the same as in c++ and c. the catching of Exception is forbidden for programmers under level 23, RuntimeExceptions can only be caught by programmers level 26+. NullPointerException cannot be caught by anyone (except by catching more global exceptions) attempt to catch a NullPointerException will result in immediate demotion to VB programmer. the use of Vectors is allowed starting level 20 but allows only for storage of primitive wrappers and strings. When you reach level 25 you can choose other things, but you must take a test on garbage collection before you can advance to level 25. passing the test also allows the use of HashMaps. reflection can only be used from level 30+. the use of threads is allowed from level 40 and on, attempt to use the double lock pattern will result in a hearing before the great council which may result in a 3 week transfer to customer support. inheritance depth will be limited to 2 until level 15, you get an inheritance depth level for every level beyond 15. the use of anonymous classes is allowed starting level 18.
using JNI is allowed starting level 45 and requires knowledge of either c/c++ of at least level 20.

C# :
mostly the same as java, but with the reintroduction of pointers and limited operator overloading. unmanaged code + operator overloading require a level 50 c# programmer.
c#/java programmers get a bonus level in the other language for every 2 levels they get their native language.

You too can help improve the future of coding, coders, and software in general by joining the great council and adding new, more draconian restrictions on low level programmers (only level 40+ programmers are allowed to join (but not vote). you get one vote[5] per level starting with level 50). Join today by leaving your email address in any public website or forum and get free Viagra commercials, exciting business opportunities from Nigeria and more!

[1] - well, known human history, or rather ... human history that is known to me...which is a somewhat limited scope..but , well, it happened so long ago that no one* could seriously say they care about it..

*- no one except old people. it seems that at a certain age a lot of people start caring about history. i think it might be driven by their quest to prove everything was better in the good old days, and everything sucks now. then again, it could be that they are simply bored...

[2] - OS scripting not included. if you wanna write OS scripts you can change your class to Sysadmin, of course, you'll have to apprentice as annoying user, then progress to user, windows installer, super user, computer tech, and finally sysadmin. level 1 sysadmin can only maintain windows servers, only starting level 10 are you allowed to even see a linux/unix system. attempting to do so earlier can result in sever shock and sometimes death. the lack of a mouse based gui system has caused many inexperienced sysadmins to lose their minds and start clicking on random monitors. before the LCD days this often ended in electrocution .

[3] - an expert is anyone programming in any of the real languages (assembly, c, c++, java, c#)

[4] - - International Council of Computer Coders (take that, I triple E) - not associated with the other ICCC

[5] - The votes don't count

The rythm of writing

2007-07-28T01:57:00.000+03:00

I sometimes find something lacking in the written word.
When i write there's a certain rhythm to the writing, i sort of expect the text to be read at a certain pace and intonation. all of this is lost in the written word and i always wonder how would this text 'sound' to someone else? can my particular way of expression cross over through the text?
I assume that people who know me and are aware of my style of conversation might be able to more accurately reproduce my rhythm from the text. but then again, i am not sure i can do this when i read their writings.

When i was in high school we where learning a Hebrew poem by Leah Goldberg called 'mi shirei ha ben haoved' (From the songs of the lost son. loosely translated) . Our teacher was reading the poem in a kind of empty voice, she was reading the words and sentences but she was ignoring the rhythm and intonation that to me where screaming from the page. it was just so obviously there. and the poet was obviously using it - the poem was build from three separate 'sub poems' and each had it own rhythm. eventually, when my teacher reached the last of the sub poems , which was the most obviously structured and had the most meaningful rhythm to it i couldn't take it any more. i stopped her reading and i asked why wasn't she reading the poem the way it was meant to be read? why was she leaving out the rhythm and intonation? she didn't understand what i was talking about and asked me to read it. there was a deep silence when i finished.

In a Hebrew bible you can find markings that details the precise way of expression for a word and a sentence. the precise tone , rise fall, rhythm and pace are described by numerous markings decorating the regular alphabet characters. certainly these symbols are not required in order to read the text and understand it and while i can read the bible , i do not know the meaning of all the different symbols that decorate it. the reason these markings are there is that the people who wrote the bible aspire to preserve not only the words but also the accurate 'sound' of reading the bible. there where no means of recording at the time, and since people wanted to preserve everything very accurately , they constructed a very elaborate and accurate method of conveying both the words and their pronunciation , rhythm, everything they could think of. add to that constant repetition and very strict methods of transferring the accurate knowledge of reading both the words and their decorations , and you get something very close to a recoding from two thousand years ago.

it would have been nice to have this sort of decoration, that way i could know exactly how my words would 'sound' to a reader. but maybe leaving the 'sound' to the reader provides some of what is called the 'power of the written word'. perhaps when you read the text to yourself, with your own internal voice , the words become more personal, more persuasive. maybe a bit more like your own thoughts than like someone else's writings.

For the love of Embedded databases

2007-07-26T01:19:00.001+03:00

mmm...there are a lot of dev blogs out there that focus on information for developers. they have guides, reviews and code samples.
i might do that (review, guide, code samples etc' ) from time to time, but really there's so much out there that i find it somewhat redundant to have yet another post on some tech or another.
the reason i am still posting this one is to share the results of some research, and to share what might happen when you fall in love with some crazy idea of embedded databases.

so , anyhow, today's topic is my experience with setting up O/R mapping and it's relational DB using .net.

first a short intro:
O/R Mapping stands for 'Object - Relational mapping' and it's a concept that makes databases a more useful , easier to work with thing. (it gets very close to making them worth while ;) ).
basically, it defines a way to easily map objects into database tables and structures. its a rather complicated problem when you start thinking about how to represent parent/child relations, or one to many ..or collection...joins...and other weird things db people like to do.
if you want more details, please check out the source of all knowledge for more details.

in the java world, there's only one O/R mapper worth its while - Hibernate. its a very powerful , well written , well maintained , open source O/R mapper.
the grate thing about .net is that its so much like java that all the cool projects out there started porting to .net (Spring, Log4Net, NUnit to name a few). its so much like java that you only have to learn how they renamed the classes and packages. the rest is pretty much the same.

anyhow, in the .net world there seem to be a few others. but i know the way the hibernate people do the job and i trust them more then some other projects. especially since they are amongst the few that are open source for .net . also, they already solved alot of the problems when they where writing the java version, so i guess they can do it better then other inexperienced dev's.
One other project i particularly liked the idea of was db4O. it used to be an open source, and is still free for non commercial use. its not actually an o/r mapper , but an embeddable object based database.
unfortunately , i am a commercial user , so we preferred the open and free option.

at first i thought i wanted an embedded database. it sounded so cool - no installation for the user, no DB administration for the user, none of the annoying hassle of a full blown DB.
sure, it wouldn't have the same performance , and it might have trouble working with too many concurrent users , but our project doesn't need either. but, as bono once said - love is blindness*.

since i already decided on hibernate , there was only the question of which db to use. this means we need a db with a dialect hibernate understands (one might think that SQL is enough of a standard , but as you soon find out, its very far from being standard. ). this gave me a very short list to work with. some databased i already knew and knew i didn't want :
Oracle - expensive and annoying
MS SQL - expensive and annoying, tho probably not as much as oracle
Access - not really a database
DB2 - expensive , annoying and made by IBM(to relieve and doubt - being made by ibm is a bad thing. one might think that its a big prestigious company , and it is. but their products suck. badly.)
so that left the databases i never heard about , and MySql. i'v always heard people say that its a grate open source database , and that everyone is using it. i'v never met someone who told me they where actually using it (they all use Oracle...those poor bastards)
now, MySql was such an obvious choice that i had to go with something else. and besides i was inlove with the concept of a the three no's of embedded databases : no-installation , no-configuration , no-administration.
so i checked out:
SqlLite - looked awful , no smooth integration with C# as far as i cared to look.
PostgreSQL - seemed nice but wasn't really embeddable.
Firebird - seemed nice and was also embeddable.
so i decided to give firebird a go. everything was fine for a while..it was a bit annoying to install it's driver and it took some effort to properly configure nhibernate due to some mistakes in their guide (which is very much unlike them , particularly since the corrections where present as user comments on the bottom of the configuration page) , but anyway, i got it to work. i was very happy with my shiny new embeddable database, and then DISASTER! i started getting : Internal gds software consistency check (cannot find tip page (165)).
hmmm... i thought to myself...this cant be a major thing, i just finished setting up the db, just committed my first tiny bit of data...i probably configured something wrong. quick google search turned up this page ... and...oh my...not only is this not a configuration error, its a critical DB error....database is completely destroyed....
Wait a minuet...when i started reading up on firebird ..all this backup, repair and fix tools where mentioned...1 .... + ..... 1 = wtf???
well, i guess i am fortunate it failed so soon. it would have been far more disastrous if these crashes only happened after we delivered it to our customers.

for the random google searching user i will state this plainly :
Don't use Firebird. don't use it with .net . don't use it with NHibernate. Just don't use it at all.

i ran some more searches, looking for some obscure database i might have missed but no go.
i also took a short look at the embeddable version of MS SQL express edition, but it only supports a max 4 gig database (it has other annoying limitations as well)..so it was useless for our app.

eventually i came back to MySQl... as it turns out , if you want a free, open source real DB...its the only serious option out there.

EDIT: read update at the end of this post , turns out MySql is not actually free.

it took half an hour to install and configure it to work with hibernate. and aside for an annoying problem i had starting it up (i wanted to use UTF8 as the charset because our users might need internationalization ..to set it up in MySQL you need to enter a freetext string, so i put in UTF8...turns out you have to put in lower case utf8....the the exception you get is just perfect "key not found in dictionary" worse yet - there's a gui and a ""test connection" which tells you everything is ok even tho you configured a nonexistent charset. how i figured it out i still don't know..i guess you come to expect silly things like that when you do this sort of work long enough) but after that everything seems to be working.. no complete database crashes for no reason , fast responses, and some nice db management tools.

As for my initial infatuation with the concept of embedded db, well, they** say "you never forget someone you loved"..so i still think some day embedded db's might be something we take for granted in every application. in the mean time we can all tell everyone that everyone is using mysql.

Join us next time for "Transparent Controls or: how i learned to stop trying and hate windows forms"

UPDATE (8/10/2007): German Viscuso mentioned that MySql isn't actually free. its distributed by a dual license - the dirty trick fake open sources use (free means free. not 'free under the following conditions') - a GPL infective**** or a Commercial license. so we turned to the only available alternative : PostrgreSQL. supported by NHibernate, works flawlessly and most impotently , doesn't infect my company with an evil licenses (PostgreSQL is distributed under BSD license - almost completely free. basically all they want is the credit for the db they wrote...) .

* - i told you it was the source of all knowledge :)
** - 5 points for anyone*** who guesses which happened first - i thought up the line, or i saw the quote in that link.
*** - only one vote per ip , the points don't matter.
**** - GPL is like a virus - if you want to use something licensed under GPL, you have to use GPL yourself. ad infinitum. they even have a philosophy as to why they have to make you be just as 'free' as they claim they are. so , in the eyes of the GPL people - software should be 'free' as long as they get to define the meaning of freedom.

Don't panic

2007-07-25T01:55:00.002+03:00

at first i thought i might title this "the evolution of ui" but im not a UI expert, im mostly a UI user , and sometimes a UI designer.
What i would like to say tho, is that i like the direction new UI is heading. back in the DOS days, you where happy if someone was kind enough to give you a text based screen with some text boxes you can tab through. then this new "windows" fad came up. and alot of people where quite skeptical of this concept. it's sad to say that even in these modern times , when even the most dedicated linux* , who compiles his own kernel and in the mean time (it could take days , seriously ) argues which editor is better, VI or EMACs (if they had just asked me i could have ended the discussion and tell them the simple truth - they both suck :) ) have probably seen, if not used , a mouse controlled GUI based operating system , there are still people out there who refuse to join the civilized world of modern UI.

See, when people got their hands on GUI's and some GUI dev tools, they started going crazy. its like the first days of the internet, some companies built html authoring tools and suddenly everyone had a "homepage" , and they where ugly. i would have supplied links to some , but i oppose that sort of thing and i refuse to increase it's google score. for interesting result, try searching for "virtual reality glove" (yeh, i know...whatever happened to the whole VR thing?) and click on the first link..check out the wonderful 'scrolling' back button at the bottom of the page, and follow twice for a particularly vivid reminder of how most of the web used to look.
anyhow, people did some terrible things on the old GUI's, they started putting textboxes and images and other ugly things all over the place, and it only got worse as time passed. UI's became more cluttered , more buttons more menus with more options. have you ever tried enabling all the possible toolbars in word 2003? not to mention try and reach every possible setting available on the menus, submenus, and subsubmenus (maybe there are subsubsubmenus also, i never had the patience to find out). its not the worst of it. word is a "user" application, application for programmers or other professionals are much much worse , try your average CAD software for a taste of how horribly complicated UI can be. but this new trend of minimalist UI is wonderful.
people finally realized that cluttering their users with useless options in hundreds of unreachable submenus is not such a good idea. that maybe there are more useful features and , god forbid, LESS useful features, or even features that are completely USELESS.
it's kind of like a revolution, the people have been abused for years with annoying cluttered overcomplicated UI and they finally rose up and said : no more! we will not have your irrationally placed options! we will not use your incomprehensibly selected icon toolbars! and we will never again surrender to your annoyingly popping windows asking us silly "Yes-No-Cancel-Abort" questions!
well, it would have been nice if it actually came from the users, but i suppose in a way it did. programmers, designers and on the rare occasion even software dev execs are all horrible UI users...i'd like to think they actually got fed up with all this cluttered nonsense.
its like adams wrote about in hitchiker's guide : if you just have the words "dont panic" in big friendly letters on the cover, you will sell a lot more books :)
i dont know if adams ever considered this analogy but having a nice , clean , simple UI with big friendly buttons and large friendly fonts is a lot like having "Don't Panic" printed in big friendly letters on the cover.

* - while usually refers to an operating system, i find that its more of a way of life, almost a religion, for most people who use it (the OS) , so in the same way one may be a Buddhist one may also be a linux (plural linuxs or maybe linuxi ? any linguists out there care to comment on this one?)

Welcome to my new blog

2007-07-25T01:35:00.000+03:00

Hi everyone (well, future everyone...).
Welcome to my new blog.
this is jumping ahead abit, but i was really impressed with this blogging platform..
im not very well versed in the blogging biz, but i do have another blog in hebrew, and this blogging platform is just so nice...but more about that later.

i suppose the first post should have to do with a sort of mission statement, so here goes:
this is intended to be a blog about science, technology, programming, software in general and probably some other random things i think about.

back to the blogging platform, now the random reader (U(0,1)) might think that perhaps this is some kind of commercial for Google , or for blogger.com or whatever, but really its not.
i just like the "feel" of everything, i love that it took me 5 mins to set up the blog, that it has nice features like autosave for posts while writing, many nice looking and tastefully done templates (so what if i went with the first one ..i like the "word press like" feeling) but mostly i like the nice friendly clean uncluttered interface.

but enough about that.

anyhow, i hope you enjoy your stay, and please feel free to feed on my RSS , it has 20% of the daily recommended dosage of news and updates!

Yossi.