Archive for the ‘General’ Category

Codeyard 2008

Tuesday, February 12th, 2008

Codeyard is a project of the RU to stimulate high school students for Free and Opensource Software in the Netherlands. Primarily they offer hosting and guidance for setting up projects. They also organize (great!) monthly meetings at the university where participants meet.

To motivate people to join, they organize a yearly award sponsored by Capgemini for the best project. With a friend, Noud, and my brother Bram we participated in their first year (2006) and won the award. This year, I have the honor to join as a member of the jury.

If you’re interested, you can visit one of the “community days”. The next one is this Saterday. There will also be a presentation by the people who cracked the “OV-kaart”, which should be quite interesting.

Bye bye Reiser4

Wednesday, January 23rd, 2008

A few days ago my root partition (formatted Reiser4) corrupted on my notebook. [ the usual IO hangups and nasty output in dmesg ]. Probably due to the usual wear and tear a notebook has to suffer or a faulty suspend cycle causing bogus IO. Something I suffered a few times before and didn’t think it would be a great deal. This time, though, fsck.reiser4 said it was all ok. That meant I was pretty screwed, for I knew it didn’t work correctly.

I lended a USB hdd, booted up to a fallback installation on a separate ext2 partition and tried to copy over everything to the USB hdd. It was quite tricky to copy over as much as I could and remembering the point where it started to crash when reading it. Luckily, I salvaged my whole /home. /var, /bin, /usr/share and a lot of other trees weren’t that lucky.

Formatted to XFS, copied everything I got back to the HDD and copied a Gentoo stage 3 tarball over it. A stage 3 tarball contains a minimal installation to which can be chrooted and then booted and from which the rest of the system can be build: the usual method to install Gentoo. I didn’t lost my world nor /etc/make.conf file. A small script later I got portage re-emerging every package I had installed on the system. Still 200 to go at the moment, but at least I’m now in a partially functioning gnome desktop, which is a lot more usable than TWM (ugly default WM of Xorg).

XFS performs quite well. It’s latency under load is a lot smaller than Reiser4’s. (It’s a pity I haven’t yet come to try the new patches in mm to help Reiser4 a bit with that problem. And also becasuse Reiser4 seems so close to inclusion, reading Andrew’s merge plans). In contrast, XFS sucks at handling a lot of small files compared to Reiser4. This is all just a feeling though. I haven’t tested anything. The most important characteristic of a FS, though, is only apparent after long use: the influence of fragmentation. Having looked around a bit, btrfs seems interesting.

On a sidenote on latency: my mom runs Ubuntu with EXT3 and even though EXT3 sucks in practically every single performance benchmark it has seem to got a superb responsiveness. Ah, 150 packages to go.

2008

Tuesday, January 1st, 2008

Happy newyear!

Tickle

Friday, December 28th, 2007

Tickle is a small Python serializer like Pickle. It however aims at generating smaller output:

>>> len(tickle('hello'))
7
>>> s = StringIO.StringIO()
>>> pickle.dump('hello', s)
>>> len(s.getvalue())
13

Though the difference is and remains quite small, this alone is useful for serialization of small things in the case of for instance RPC. However, usually you already know what kind of data to expect and you don’t really bother about the type information. This can be done by specifying a template:

>>> obj = []
>>> for i in xrange(100):
       obj.append((i, str(i)))
>>> len(tickle(obj))
629
>>> len(tickle(obj, template=(tuple, \
   ((tuple,((int,), (str,))),)*100)))
390

(Instead the *100 an iterator could be constructed, but that would clutter the example even more than it already is.) In comparison:

>>> s = StringIO.StringIO(); pickle.dump(obj, s)
>>> len(s.getvalue())
1680

One big disadvantage of Tickle is speed. Pickle has got a nice C implementation, which is quite fast. Psyco helps a bit but not really enough for really big things. Even more so pickle is a bit smarter: it builds a LUT for instances to avoid duplicate data. However, in the situations where Tickle will be used (by me at least) that isn’t too big of an issue.

You can download tickle.py via gitweb.

Merry Christmas

Monday, December 24th, 2007

For happy a christmas!

Java crashing on xcb_xlib_unlock.

Monday, December 17th, 2007

When Java applications crash on linux with xcb_xlib_unlock: Assertion `c->xlib.lock' failed you should upgrade to libxcb-1.1 and add LIBXCB_ALLOW_SLOPPY_LOCK=1 to the environment.

Virtual packages in python

Saturday, November 10th, 2007

When writing an application in python, it can be very convenient to be able to import a module from the top of your module tree. Eg. import myapp.config instead of (python 2.5 only) relative imports: import ....config. To do this one would have to make myapp a package. The normal way to do this is to put your application directory (which now has to be named myapp) somewhere in python’s package search path. This isn’t all to convenient.

The solution: manually set up your package module:

import os
import sys
import imp
def setup_virtual_package(name, path=os.curdir):
    """ Sets up a package at the given path with a given
     name """
    modulePath = os.path.abspath(path)
    f, fn, suffix = imp.find_module('__init__',
         [modulePath])
    imp.load_module(name, f, fn, suffix)
    sys.modules[name].__path__ = [modulePath]

Now import myapp.something works like a charm.

Rome

Monday, October 22nd, 2007

I’m visiting the city of Rome next week.

19

Thursday, August 30th, 2007

Nuff said.

Section 202c of the German computer crime laws

Monday, August 13th, 2007

This section has come into effect over the weekend. It makes it illegal to create, possess, obtain, provide access to, yield, distribute or otherwise allow access to lots of widespread tools that can be used to breach security. Take for instance nmap.

This law does not only impede our freedom (of speech), research, decrease security and allow for misuse, but more importantly it won’t even stop the real criminals.

Stefan of the Month of PHP Bugs Project writes:

The law does not affect our freedom of speech to report and inform about security vulnerabilities and how to exploit them.

We are just not allowed to create/distribute/use software that could be used as “hacking tools”.

Where would they draw the line between reporting/informing about a vulnerability and how to exploit it and the actual source code to do it. Would pseudocode be illegal? Would literate code be illegal? Also there would be no way for security researchers to try out their work.

What will happen in the worst case if similar laws are accepted in other countries and enforced, is that vendors will rather cover up all vulnerabilities using these laws instead of securing it. That there are lots of ready-to-use exploits is good. It’s a very good incentive for security.

That there will always be a leak in a piece of software that someone will be able to find on his own will not be changed by this law. Also there will be no way to stop the real criminals from creating and distributing tools underground. Now everyone still knows what kind of tools are around and will know what to expect.

Wacken

Friday, July 27th, 2007

I’ll be up early tomorrow, Saturday, to take the train with lots of friends to the little town of Wacken in Germany. (Actually, there is no train station in Wacken, so we need to hire a cab someway for the last dozen kilometers)

Sunday morning the camping of the Wacken Open Air festival will open. The festival itself will start on Thursday. I hope to be home again the first Sunday of August.

“Nothing to hide”

Wednesday, July 11th, 2007

In this short essay, written for a symposium in the San Diego Law Review, Professor Daniel Solove examines the “nothing to hide” argument. When asked about government surveillance and data mining, many people respond by declaring: “I’ve got nothing to hide.” According to the “nothing to hide” argument, there is no threat to privacy unless the government uncovers unlawful activity, in which case a person has no legitimate justification to claim that it remain private. The “nothing to hide” argument and its variants are quite prevalent, and thus are worth addressing. In this essay, Solove critiques the “nothing to hide” argument and exposes its faulty underpinnings.

“I’ve Got Nothing to Hide” and Other Misunderstandings of Privacy

Not only is its subject very relevant, the Essay is very well written and a pleasure to read.

Xgl with RandR 1.3 and Ati

Monday, June 25th, 2007

You have to add Option "DesktopSetup" "single" to your ati’s Device section in xorg.conf otherwise Xgl will think you’ve got two screens if you’ve got two adapters even if you just use one (which is the case with my Thinkpad) and will miserably fail on the second unattached screen.

Graduated

Friday, June 15th, 2007

Today I received a long anticipated phone call from my mentor who took away the doubt and let me know that I graduated for my VWO exams at the Stedelijk Gymnasium Nijmegen, which is a matriculation exam.

With mixed feeling I look back at seven long years. It certainly shouldn’t have took 7 years, nor 6, not even 3. But espacially in my last few years I came to appreciate the tons of very different people with whom I spend the days. A diversity I won’t find in that extend within the students of Maths and Physics which I intend to study next year at the Radboud University in Nijmegen.

For one thing I certainly wouldn’t have want to have missed these last few years. Some say they’re the best of your life. I do really look forward to university, though.

For those interested, I’ve planned an “Examenfeest” with some others in a nice club in Nijmegen. Send me an e-mail if you happen to be in the neighborhood and care to join.

Upgrading wordpress with git

Thursday, June 7th, 2007

I didn’t like upgrading wodpress much. Everytime I did it, I needed to re-apply all my little tweaks to the new wordpress. It took too much time.

I tried to diff -uNr on the current version I was running and the newer version and then applying the resulting diff to the current version, but it seems wordpress has been backporting changes so I got conflicts, quite a lot of them.

Because I was quite tired of porting my changes, I’ve tried git, the Source Code Managment tool used by the linux kernel, to do it for me:

I did this all in the parent directory of the root of blog.w-nz.com. This folder contains:

  • htdocs current installation (2.1.2)
  • 2.1.2 the unmodified wordpress
  • 2.2.0 the new wordpress I want to upgrade to

First, I created an empty git repository:

mkdir git; cd git; git init-db; cd ..

Then I copied over the unmodified version of wordpress I was running, and commited them:

cp 2.1.2/* git -R
cd git
git add *
git commit -a -s
cd ..

Then I copied over my current installation:

cp htdocs/* git -R
git status # lets see what changed

There are lots of files like uploads I want git to ignore, so I edit .gitignore to make git ignore them. There weren’t any files I added though, otherwise I’d had to run git add to let git know.

And let commit my changes:

git commit -a -s

Now, lets go back to the original commit — the clean 2.1.2 wordpress — and start a branch from there:

git checkout HEAD^ # HEAD^ means parent commit of HEAD: the previous commit
git checkout -b tmp # create a new branch tmp from here

Now I’m in a branch without my own changes, which was forked from the master branch. Lets apply the new wordpress on this branch:

cd ..
cp 2.2.0/* git -R
cd git
git status # see what changed

git-status showed me that there are a few new files in wordpress 2.2.0, I git-add-ed all of these new files. And then committed it all:

git commit -a -s

Now I’ve got two branches:

  • master which contains wordpress 2.1.2 with my own changes on top as a commit
  • tmp which is forked from the wordpress 2.1.2 from the master branch without my own changes but with the 2.2.0 changes on top

What I want to do is to reapply the 2.2.0 changes on top of my current changes’ commit instead of on top of the 2.1.2 commit. To do this, git has a very powerfull util called git-rebase:

git rebase master

This will search down the tree until the point where the current branch (tmp) forked from the target branch (master). Then it will re-apply all commits in between on the latest commit of the target branch.

Just like if I’d use diff/patch I get a merge conflict. git rebase lets me know this and git status shows me which one are these. The one little difference with the diff/patch approach is, that there are way less merge conflicts (git is smarter) and that the merge conflict are way easier to identify and they’re inline in the original files. Not to mention that when I would have fucked up I’d always have a way back.

After I fixed the merge conflict, I git update-index each conflicted file (to tell git it’s resolved) and git rebase --continue-ed.

Now I’ve got my updated wordpress in the git folder. Then I backuped the current, copied over from git and visited wp-admin/upgrade.php and I’m done :) .

By the way: “I didn’t say Subversion doesn’t work. Subversion users are just ugly and stupid.” — Linus on this Google tech talk.

Sidenote, I switched from Hashcash to Akismet. Hashcash didn’t work anymore and Akismet theoretically should be the best solution because it isn’t based on security by obscurity.

This is How We Catch You Downloading

Sunday, April 15th, 2007

torrentfreak.com has acquired a document how a british company is tracking down illegal use of P2P: This is How We Catch You Downloading.

Basically they use a modified P2P client which searches for infringing content, download it and if that works and is indeed is the content then they do a whois on your IP and send a infringement notice to your ISP. The best thing is that they claim that this provides enough proof that you really are infringing.

They probably never heard about botnets.

Ruby and parenthesis

Wednesday, April 11th, 2007

Do you prefer

p = Pathaname.new('.').realpath

or

p = (Pathname.new '.').realpath?

C&C 3 on Linux

Sunday, April 1st, 2007

I’ve got Command and Conquer 3: Tiberium Wars running on my Gentoo Linux installation with wine 0.9.34 by following the instructions here. I had to first install it on windows though, and copy the folder for the installer didn’t work, even with Crossover Office.

Except for (very glitchly) video and sometimes a crash everything seems to run. (Didn’t try multiplayer yet though). I had to put all quality settings to lowest, which makes me wonder whether that is my radeon X1400 being not so good as I expected or wine just being slow in emulating Direct3D.

Watermarking media

Thursday, March 1st, 2007

It seems the new trend of the music industry against piracy is watermarking movies/audio/etc.

Content is water-marked by adding very small (unnoticeable) changes that could store something like a rsa based certificate to identify a given audiotrack.

Originally I thought they’d use it to track down the source of an illegal download. It sounds illogical to me because it’s hard to keep watermarks when format is changed (mp3, ogg and others really do mess up slight unnoticeable differences because otherwise they wouldn’t compress as good). And when someone has got two versions of the same audiotrack one can compare them and find out how something is watermarked.

Maybe the scheme of the industry isn’t that stupid, but the other way around (and a lot more evil). Maybe just sue everyone who hasn’t got a watermark on their movies or mp3.

Luckily a Fair Use bill was passed which they say (haven’t checked) allows fair-use conversion of format of media.

Reiser4 for linux 2.6.20

Sunday, February 25th, 2007

reiser4-for-2.6.20-0.patch.gz

Big fat warning, when hibernating using suspend2 it seems to corrupt some memory here and there. So don’t hibernate.