Author: Bas Westerbaan

Shiftings in Architectures

Cell, the hype

Sony is working on a new kind of processor which they call Cell. A single Cell processor is said to have a 1/75th of the power of the Blue Gene/L supercomputer in teraflops, and that for 400 dollars.

The new Cell processor is hyped, a lot. People are shouting that the performance of the PlayStation 3, which will feature a Cell processor, will blow away those of the competers: the XBox and the Revolution, both using claimed to be slow PowerPC processors.

Teraflops

So what are these Teraflops? Why aren’t they talking about Hertzes anymore?

A TeraFLOPS stands for 1,000,000,000,000 Floating point Operations Per Second. (TeraFLOPS on wikipedia).

A normal computer has a few GigaFLOPS (one TeraFLOPS is 1000 GigaFLOPS).

The Cell processor in the PS3 will have 1,8 TeraFLOPS. It is very easy to believe that the PS3 will inheritly be about 500 times faster than a normal computer. But that is false.

The performance of a computer isn’t all about how fast a computer can calculate with floating point numbers. The performance of a computer has got to do with a lot more than that. Simply doubling the memory performance (not specificly memory bandwidth, but the whole) would have a bigger impact than doubling the amount of TeraFLOPS, even on applications that heavily rely on floating point operations.

So why have a lot of TeraFLOPS then? It is quite simple, the new platforms haven’t got seperate GPU’s (Graphical Processing Units), their task is now intergrated into the CPU. And what was their task? Calculating, a lot, with floating points.
The amount of TeraFLOPS on the new platforms are the sum of the amount of TeraFLOPS of both the GPU and the CPU. A GPU needs to calculate a lot with numbers for 3d rendering.

Your normal computer has got a lot more than those few GigaFLOPS in your CPU in your GPU already. Although the leap to over the one TeraFLOPS is certainly impressive.

Marketing

Super Computers need to do a lot of FLOPS too, they calculate models which got a lot of floating points. So why put the Cell processor in the Ps3 instead of directly in some supercomputer? Simple, marketing. Almost everyone that is superficially following the computer news is telling me over and over again that the Cell processor is the most incredible thing there is. It is hyped.

Linux on Ps3

Also the Cell processor uses a whole new architecture which still has to be tested a lot to mature. Sony will let you install Linux on your Ps3? Why? Simple, because they want the Linux community to add decent support for the Cell architecture.

New Architectures & CLI

I guess it will be a matter of time before new architectures will come. A problem with a lot new architectures is that there isn’t a lot of support for them yet. The solution? CLI’s like Java and .Net could bring the solution.

Microsoft rumoured to have MSIL running (semi)natively on their XBox360 and making Longhorn more and more rely on .Net (the ontop applications, not the kernel offcourse) means a lot less architecture dependency.

I wonder what will happen, but one thing is sure.. things are shifting.

Python Url Encoding

I looked in the Python Library Reference for a function to encode special characters in strings to be able to be put in Urls (and Uri’s), but found none.

Maybe I missed it?

Anyways, here is an implementation I wrote based on this article:

HexCharacters = "0123456789abcdef"

def UrlEncode(s):
    r = ''
    for c in s:
        o = ord(c)
        if (o >= 48 and o <= 57) or \
            (o >= 97 and o <= 122) or \
            (o >= 65 and o <= 90) or \
            o == 36 or o == 45 or o == 95 or \
            o == 46 or o == 43 or o == 33 or \
            o == 42 or o == 39 or o == 40 or \
            o == 41 or o == 44:
            r += c
        else:
            r += '%' + CleanCharHex(c)
    return r

def CleanCharHex(c):
    o = ord(c)
    r = HexCharacters[o / 16]
    r += HexCharacters[o % 16]
    return r

note: I used the character numbers instead the characters to compare with so I could do greater than and lesser than for the alphanumeric numbers. Maybe testing with a bitmask would be more efficient.

I have to write almost everytime I work with python my own something to hex function which doesn’t add the ‘0x’ in front the built-in hex does.

Either I can’t search or Python hasn’t got enough batteries included. I guess the first, if not I`ll submit my batteries.

BSD

BSD is Unix.. said to be the professional cousin of Linux.

This piece of propoganda for BSD against linux gives as reason that linux is bad for it is told to be just hacked together. This because of the people who develop linux are just people from the community who put a little bit of time in it to get a feature (they probably want) added into Linux and don’t really concern about making it perfect, was said.

They took some of the sarcastic ‘todo’ comments in the kernel as example, blaming that if that stuff is in the kernel linux can’t be trusted at all.

But why does BSD hasn’t got the widespread hardware support Linux has? They blame the big company’s like IBM for instance. I just wonder whether their 60 dedicated BSD programmers could code all the hardware drivers that the thousands of contributers of Linux have coded, even if it were as bad as it is now according to them.

I bet that *BSD has got more than enough of those comments in their code too. If they haven’t they are just hiding the truth and hiding points of improvement, for these TODO’s and FIXME’s are fixed in the end. And even if they get rid of all the TODO’s and FIXME’s before they release any of it they waste a lot of time it could have been used already (less efficiently though.. but it usualy doesn’t make such a big difference).

Services, a different approach

A Server is a computer (or a group of them) that offers services.

Such services could be a webmarket, a website, a game server and so on.

These services tend all to be implemented quite differently, hardly interacting at all. All these services require a different approach, a different way to manage them. This creates both overhead and chaos.

I`m working on a small framework intented to host services which is very flexible.

Framework
A service isn’t a single process as it usualy is, although it could be put in its own seperate process. A service is rather just an instance of its servervice type in the framework somewhere, meaning you can have multiple instances of one service with different configurations scattered accross processes and even machines as you like.

You can edit the settings and manage the services using the same API (and likely the same configuration tool).

A service isn’t aware where it is hosted, from its perspective it is just in a framework with other services and service modules with which it can interact in always the same way.

You as administrator however are aware of the services and its allocation. You can allocate a service (even runtime) to different locations which haven’t neccessarily got to be processes, but also can be lightweight processes found in .net like appdomains or on different machine’s processes.

Having different processes for each service decreases performance but increases stability, and visa versa.

Interaction
Services shouldn’t be restricted to their own process but should also be able to interact with other services by using their exposed interfaces.

An example could be a webserver-service which you provide with the name of the logger-service you want to be used for that webserver. The webserver could just interact by casting an object in the public interface dictionary of the logger to an log interface. The framework takes care of the required underlying communication like the remoting in case they are hosted on different processes or even different machines.

A more tighter type of interaction would be modules. A module of a webserver could be a webmarker. The webmarket would just access the object of the webserver which handles hooking to certain URI, add the hooks and await a visit. Other services could do the same but a module is specificly bound to the service it attached to.

Futhermore modules would be an easier way for administrators to manage extra functionality which is more module like than filling in the instance paths to use in the settings of a certain service.

Portability
I`m still in doubt whether I should make it cross-language. It would be a good thing but it would certainly create havoc to make everything run smooth and alike on all languages. Not even to mention the overhead to load each languages’ runtime.

Power PC dumped

Apple to ditch IBM, switch to Intel chips

When Apple will switch to x86 and ia64 it will be possible to run mac osX on your own home bought computer, given that they don’t avoid it by changing some opcodes.

One other things is sure; they will loose some power. As long as they provide binaries and provide one set for all apple computers they will have to add support for both ppc and x86 in one binary distribution which will undoubtely be a lot bigger if not slower too.

It will be interesting to see how this issue will evolve.

Subversion

I’ve recently been using CVS a lot, more specificly subversion. It makes it a lot easier to share source code. Where it wasn’t possible to work on the same projects or even on another project that depends on the first because sharing source code is tricky, subversion was the solution.

mount..?
As far as I know there isn’t a program that allows you to mount a subversion repositry into the linux filesystem. This would make managing a subversion reposity a bit easier for at the moment you need to use svn add to add every single file; svn cp to copy, svn rm to remove, and so on which could all be intergrated when wrapping a repositry up.

It is possible I haven’t been searching in the right place. Searching for it just gave me an enourmous list of other people graving for it.

pruning
To my knowladge there is no really easy way to get rid of older versions. Offcourse it would be against the basic idea behind the system storing every single version, but sometimes it just isn’t practicly. In case you are dealing with extra not sourcecode files, like images or data files, which usualy are quite large in comparison with source code, you’ll get a really big and slow repositry.

Adding a feature that will store all files before version x if it isn’t the top version in a compressed archive of some type would be really nice. Although it would drasticly decrease the access time for the older versions it does decrease the space used and increases the access time for the top versions which are used a lot more than the older ones.

XBox 360

Microsoft released the specifications of the XBox 360..
it is a beast..

With 3 CPU cores running on 3.2Ghz each with 2 hardware threads and 1 mb l2-cache each it is a powerhouse.

A remarkable feature of the XBox 360 is that it has got processor level support for MSIL (the bytecode used by .net applications).

The best thing of the XBox 360 is that it’ll be quite cheap (~150$), this probably lets the XBox360 deliver the most processor capability per dollar.

Maybe I’ll buy a few and make my own little linux server cluster with it, when the guys behind xbox-linux.org have added support for the XBox360 :-).

Music Fill Out Thingy

Zef handed me the music baton, meaning I got to fill out a form ’bout music. So well… here it is:

Total volume of music files on my computer
3.2 GB.. that isn’t a lot.. but in contrary to some with huge collections I do listen to them all.

The last CD I bought was
I got myself some albums of Children of Bodom recently. I`m trying to get the Lamentations DVD of Opeth.

Song playing right now
Opeth – Still Life – The Moor

Five songs I listen to a lot/mean a lot to me
I don’t really listen to a certain song a lot.. although recently I listened to these a bit more than the others:

– Opeth – The Drapery Falls
– Children of Bodom – Needled 24/7
– Nirvana – You Know You’re Right
– Opeth – Dirge for November
– Muse – Citizen Erased

Lets be creative and add another question:

Artists you like
Opeth, Children of Bodom, Nightwish, Nirvana, Linkin Park and last but not least Muse.

The five people where this meme will go on
– Kaja Fumei
– Noud Aldenhoven
– Gumuz
– you!
– (yes, i was out of idea`s)

Safe web authentication

The major problem with security of web applications is that the client sends the login name and password in plain text if https isn’t available. A nasty person with access to the network could use ARP poisening alongside packet sniffing to acquire the login, which wouldn’t really be desirable.

I stumbled accross a very interesting piece javascript which implements the md5 hash algorithm: http://pajhome.org.uk/crypt/md5/.

Using a hash makes it impossible to reverse engineer a password and makes authentication safer. An issue with this is that you only require the hash, not the password to get in. To prevent this the password should be salted before hashed.

Basicly a secure authentication via http would look like this:

Client sends request for login to server.
Server sends the login form which includes a login id and salt to the client.
Server stores the login id and salt it sent to the client.
Client sends the hash of the filled out password and received hash alongside the login id from the server to the server.
Server checks whether the hash of the password in the database and the received hash combined with the login id are valid.
Server sends whether authentication was a success.

Maybe I’ll implement an example application :-). In any case I hope that this will be employed.

Update, most authentication system used by webbased software are still vulnerable and would almost neglect the use of this by being able to hijack a session by just getting the session key. The client however could also implement javascript to use a similar method with a salt to protect the session key. The problem still is that it is extra overhead on the client and that not every client has got javascript enabled.

Assembly ‘programmers’ suck

Some programmers claim writing assembly is the solution to every single programming issue, for it would be great for being down to the basics, or it creates small programs, or even be fast. For all I care they are just trying to brag for not a lot of people know how to program in assembly and it is generally seen as difficult to do.

Programming in assembly is almost everytime a bad idea.

So first, what exactly is assembly?
Assembly is a text language with which you can write opcodes and arguments of machine code processed by the processor by hand. This gives you a very high amount of control of what the processor does. Whereas higher languages generate the machine code for you as they deem it to fit you can choose which machine code would fit.

Assembly isn’t difficult to learn. Assembly is very straight forward. It just is hard to program for you don’t have functions, you don’t have high level structures, you don’t have all the aid of a high level languages.

The reason programmers claim that assembly is superior is for with assembly you can write faster and smaller code. This is only partially true.

When you know all opcodes and tricks you can pull on all different architectures you can beat a compiler.. The problem here is that there are tens of architectures with totally different opcodes and even more subarchitectures with each a slightly different implementation and different performance characteristics of each opcode or way to do something.

So to truely beat a compiler which compiles one piece of code to assembly, you have to create for each different architecture a seperate piece of assembly source.

You’d also have to learn all the opcodes of one cpu, that aren’t just a few hundred which would suffice to get it working, but thousands which are required to get it working as fast as possible.

Compilers know all these extra opcodes and tricks for each architecture and would therefore a higher level programmer would do a better job on creating an executable than an experienced assembly programmer in the same amount of time. If the assembly programmer would want to beat the higher level programmer he would require not just 2 times more time but at least 10 times.

Also assembly isn’t very portable. If you want to change one little thing you got to change it for all optimalizations for all different processors. I haven’t seen a big application written in assembly. And that has got a reason.

Most programmers that claim to be able to write quick assembly don’t have got an idea how everything works exactly and are just thrilled that they made a little program work in assembly.

Assembly though can be usefull. An example is an arbitrary length integer class like GMP which uses optimized assembly for certain operations. It aren’t a lot of operations done in assembly, but it certainly has got a lot of assembly. And it is worth it.

Sticking with a good compiler and a high low level language like C is always the best option for good applications.

Gentoo Linux

3 days ago I came accross gentoo linux.

Gentoo Linux is a linux redist that provides you with a livecd from which you are able to build a whole new linux installation practicly from scratch.

Yes, that means compiling everything yourself.

Normally compiling means reading through tons of documentation, trying to find dependencies, trying to get stuff working, making tweaks, and reading again through tons of documentation, trying to find dependencies, hoping version won’t collide, and again, and again.

Luckily gentoo provides a very neat tool called portage.

Portage allows you to simply install for instance gnome by just typing:

emerge gnome

Portage will look up which dependencies are required, download them, configure them, compile them, and clean up garbage. The only thing you got to do is hang back and relax.

Portage also automaticly implements your preferenced optimalization settings to create the best possible build for every single application you compile.

Gentoo seems to have resulted in a lot faster linux installation, although there is one little issue, it takes time.. A lot of time…

Compiling gnome (with all dependencies) took about 8 hours…

Maybe gentoo will be the first linux to kick windows from my desktop :-).

Extending using Wrapping

Ever had a problem, not being able to add more functionality to an existing class? Some languages seem to support adding functionality to an existing class but it does make it less secure and leaves a lot of dangers of maluse.

A solution could be to create a new class which inherits the original class although it can only be instanciated using a cast from an instance of the original class or instanciating providing the original instance. Also it won’t be able to override anything otherwise it couldn’t act as its base object.

A possible implementation of this idea could be:

wrapper Int_IsOdd : int {
  public bool IsOdd() {
    return this % 2;
  }
}

And it could be used in several ways:

int myInt = 100;
print ((Int_IsOdd)myInt).IsOdd();
print myInt.IsOdd();
print new Int_IsOdd(myInt).IsOdd();

The first way would be the easiest for both a reader which isn’t familiar with the extension and for the compiler to guess errors. The second would be neat to use but it would be confusing for people that cannot access the rest of the source or do not know what function is meant. The third way would be very compatible.

I wondered by myself whether normal inheritance could provide this functionality.. and it does. The only problem is that polyforism only works up and not down as the wrapper requires. (when you got a class a and b inherits of a, you can with inheritance treat an instance of b as a but you cant treat an instance of a as b which is required for the wrapper)

Also a wrapper would be more cached and not really a new instance as suggested by the way it seems to work and can already be slightly implemented in existing languages.

Maybe it’ll be put in paradox.

update – I talked with Kaja, and it will most likely be a feature of paradox

It’s all about the model

With all those strange new physics theories and models which seem to conflict eachother it seems all rather too awquard to be true.

Like Quantum Mechanics stating that something can be two things at the same time, which seems impossible, but certainly is when keeping in mind that Quantum Mechanics states that something is what it seems to be when measured where measurement always has to interact which means that very unstable states of an object don’t exist but still can be applied on an object being two states at the same time with a third set of properties… great…

What people don’t seem to realize is that most physics theories apparently conflict for they are formulated rather awquard, but only formulated like that for then they are reasonably understandable as a model. Also one given theory isn’t superior above another, usualy one theory just stresses a certain part of apparent reality more by the way it represents everything.

There just isn’t the theory that explains how the universe works, there are only models covering a part in a certain way.

I don’t care if someone likes to represents an atom as a little UFO, as long as the conclusions and predictions in that theory are usefull in being able to predict behaviour and characteristics.

This is also where philosophie makes a fatal mistake, a philosopher would tend to use metaphores, scientificly called models, literaly assuming that reality is just a metaphore. And there ishardly anything you can do to persue them otherwise anyways :-).

Apparently it seems, assuming the string theory is an accurate model, that we only experience an infinite small amount of the whole universe. It maybe could be seen that we are just a little (end) fractal in a big fractal (inside another).

But why are we as we are? It would be logical to assume that we are nothing more than a logical result of one chaos like formula for the whole universe seems to be pseude randomly as many chaos formula’s seem. We’re just one of the infinite possibilities (stressing possibilities).

That would seem rather depressing and our lives would seem rather useless.. and actually.. they are rather useless for the whole static everything. But actually that makes just one more reason to maintain our human species for otherwise we couldn’t enjoy our uselessness :-).

And even if we would have to formula to predict everything (which maybe even has been found already by some mathematician, but is only experienced differently by us as models differ from eachother), we would only be interested in derived models for our perception of it to aid our uselessness :-).