Intrepid Blog » Computer Science

GStreamer: accurate duration

Bas Westerbaan — Sun, 19 Apr 2009 13:26:01 +0000

When decoding, for instance, a variable-bitrate MP3, gstreamer reported durations are, to say the least, estimates. I’ve tried to get a better result in a few ways. First off, some files yield a duration tag, but even if you’re lucky and it is there, there are no guaranties about precision. After that I tried seeking to the end (GST_SEEK_END) of the stream and querying the position, which gstreamer didn’t like. Finally, routing the audio into a fakesink, waiting for the end of stream and then querying for the position gives the right result. It’s not the prettiest method, but it works.

This is a Python script that prints the duration of a media to stdout.

The Cuckoo Hashtable

Bas Westerbaan — Sun, 21 Dec 2008 22:22:41 +0000

A Hashtable algorithm is a specific algorithm to implement key-value pair datastructure with efficient by-key look-ups using hashing of the keys. A hashtable contains a list of buckets. In a simple implementation, the i-th bucket, contains the key-value pairs in a list of which the key has a hash that is i modulo the amount of buckets. The hash-table would increase the amount of buckets if any bucket contains more than a fixed amount of pairs. This results in a constant-time look-up, but an insertion might invoke a expensive rebuild.

There are a few variations on Hash Tables. I’d like to share a really smart one: the Cuckoo Hashtable.

The Cuckoo hashtable expects two different hash-functions for the keys. Instead of storing a list of pairs in each bucket, the Cuckoo Hash table stores a single pair in each bucket. When inserting a pair, it is inserted in one of the two possible locations. If it happens to be occupied, the old pair is replaced. Then the replaced pair is inserted at its other possible location, potentially kicking out another pair. This step is repeated, until there is no displaced pair or a loop is detected. When a loop is detected, the hash-functions can be changed, if the buckets are for the most part empty -or- when the table is almost full, the amount of buckets can be increased. It can be shown that an insertion has a amortized constant time. (The buckets can be resized in-place, and each entry is repositioned as if it was displaced by another.)

If I’d own a botnet… (1)

Bas Westerbaan — Wed, 26 Nov 2008 21:36:47 +0000

…and didn’t want to loose it if my control servers got shut down, I’d let every orphaned zombie randomly connect to hosts in a given IP range, and challenge them to give a preimage of a hardcoded hash. [ detail: add a salt to prevent replay attacks ]. With a sufficiently safe casu quo large range, it also might be helpfull to allow zombies to forward still orphan zombies.

git‘s versus svn‘s storage efficiency

Bas Westerbaan — Tue, 08 Jul 2008 00:43:47 +0000

At Codeyard we maintain a git and a subversion repository (which are synced with each other) for each of the >115 projects. The following graph shows the repositories plotted logarithmically according to the size of their whole server side subversion repository horizontally and their git repository size vertically:

To make more sense of the logarithmic nature of the graph, I’ve added three lines. The first (solid black) indicates the points of which both sizes are equal. The second course dashed line indicates the points of which the subversion repository is twice as large as the git repository. And lastly, the third finely dashed line indicates the points of which the subversion repository is five times as large as the git repository.

All projects for which git is less storage efficient, are smaller than 100Kb. The projects for which git is most storage efficient (up to even 6 times for a certain C# project), are all of medium size (10–100MB) and code-heavy. For the other projects, which are blob heavy (eg. images), git and subversion are close (git beats svn by ~20%).

One notable disadvantage of huge (someone committed a livecd image) git repositories, is an apparent memory usage of git repack even if I tell it not to with --window-memory.

ssmtp segfault in sendmail on Gentoo

Bas Westerbaan — Tue, 26 Feb 2008 20:39:52 +0000

Disabling the md5sum useflag might fix it.

Thanks to Bram for the tip.

Evolving the Object Paradigm

Bas Westerbaan — Thu, 31 Jan 2008 12:39:02 +0000

Kaja is writing a series of articles on the shortcomings and solutions to the current object paradigm. Very interesting.

Linux 2.6.24-bw-r34

Bas Westerbaan — Sun, 20 Jan 2008 00:24:50 +0000

Again an update for my bw-tree. There isn’t a tree that includes reiser4 and TuxOnIce without a lot of other bloat, so I created one myself.

Download the big diff or the seperate patches broken out.

Note I pulled in new patches for reiser4 from the -mm tree against -rc8 which should fix the Reiser4 flush problem a bit.

Update Patched against 2.6.24 stable. New TuxOnIce patched added and the genpatches. Please note that I haven’t tested Reiser4 thoroughly enough on this version.

linux 2.6.24-bw-r12

Bas Westerbaan — Thu, 20 Dec 2007 22:07:05 +0000

There isn’t a (stable-ish) tree that includes reiser4 and TuxOnIce (suspend2), so I made one myself, based on 2.6.24~~-rc5~~-rc67.

You can grab it as one big patch, bw-r12-for-2.6.24~~-rc5~~-rc67.diff.bz2, or broken out: bw-r12-for-2.6.24~~-rc5~~-rc67.tar.bz2.

rebuild-scm-ebuilds

Bas Westerbaan — Sat, 08 Dec 2007 18:51:37 +0000

Two very simple ruby scripts (I love ruby!) and a little bash script that searches all ebuilds with 9999 in the name, orders them by dependencies and then emerge -a them. Very useful when you got a lot of 9999 ebuilds.

CaCert.org

Bas Westerbaan — Sat, 08 Dec 2007 12:20:46 +0000

CaCert is a Certification Authority that works with a web of trust: people meet and assure (similar to keysigning) eachother. If you’ve been assured by enough people you’ll be able to let your ssl server key be certified by cacert. It’s a lot more secure than other CA’s who just give anyone a certificate who pays enough.

Still a hierarchical system with a CA is flawed. When the CA is compromised, the whole system fails. PGP’s web of trust hasn’t got this weakness.

(Got a nice shiny cacert certified ssl certificate on my webserver now)