Any way good luck on the project and I will contribute a patch when I have the time to work on it.

I just edited my test script to use a preinitialized Struct. PyPy does not optimize it as well as calling just struct.unpack. That is a shortcoming of PyPy.

I agree that code should be cleanly designed without worrying about a specific implementation. Only where performance is critical, we should fiddle with toys as the jitviewer. The performance of msgpack on PyPy is critical for me, hence the effort.

I still am very doubtful that cffi would increase performance, for the same reasons I gave before. If you would contribute a cffi backend to msgpack-python, that would be great.

The mspack spec was designed to avoid this copying and to avoid type conversions. Although in Python you can only avoid the copying by using the unpack_from and pack_into methods. Although maybe PyPy can also avoid the type conversion but I don’t know if it is smart enough to do so at this time.

The only issue related to strings that I had with the struct.pack and struct.unpack was the parsing of the format string. I just wanted to make sure it was done only once for each msgpack type. It looks like PyPy is able to clean this up but I don’t feel like its a good idea to code with a poorer design knowing that PyPy is able to clean it up. You should be able to write good clean Python code that will execute fast on PyPy rather than writing code based on your knowledge of how PyPy is able to make improvements, as what it’s able to optimize will always be a moving target. Although in this particular case I don’t believe PyPy would make a regression in this area that would remove the optimizations that are helping you at this time.

Also before the loop is JiITed it has to run in the interrupter so the earlier operations of each msgpack type will be slower due to parsing the format string. Also what about those who may want to use this library in CPython and decide not to use cython. The performance for them is going to truly suck big time as the format string will have to be parsed every time. So that’s another reason to convert the code to use struct objects.

We have been discussing how to improve the existing code but I like to just remind you that it’s very likely that the best performance you going to see from PyPy will likely come from using cffi and the native c library. That’s basically the same approach that the cython solution is relying on.

Also, just to try to make this point a little stronger. The jitviewer should be mainly used by PyPy core developers and those building PyPy VMs. A normal developer writing Python code to run on PyPy shouldn’t have a need to use it. They can use it to point out an inefficiency that PyPy has to the core developers but it should not be used as a way to get you to write Python code in a way that has a better chance of being optimized under PyPy except for very rare occasions and even then it should only be made by those who follow closely and understand PyPy’s development.

You can personally benefit much more from learning about good design and the benefits it provides over learning how to write obscure Python code that runs fast in PyPy. In general a better design will also run faster in PyPy.

So, for PyPy my approach seems optimal. However, for plain CPython your approach could certainly be superior to mine.

It turns out performance issues you have are due to the algorithms you are using. You really need to change the architecture to see an real big performance gains. That’s why I suggested you don’t create all these sub strings from your buffer and instead keep track of your position in the message as you process the message and use the unpack_from and pack_into methods.

Not creating all the sub strings will reduce the memory overhead as well as reduce the pressure on the garbage collector while also giving you a significant boost in performance.

Now I’m not sure how much of the overhead created by all your if and elif statements PyPy are able to remove but I know implementing a more efficient algorithm that gets rid of all the if and elif will be even faster.

I haven’t seen the output of jitviewer for this code. As a matter of fact I haven’t even run the code. By looking at the code I can see that its design could be improved in a number of areas. I also see the code is written in the style of C code, which in general, Python code written like C will have poor performance issues as the run time characteristics of C and Python code are quite different. For example, function calls in C have very little overhead compare to the overhead they have in Python. So in C where calling out to lots of functions in a tight loop does not have much of an impact it does so in CPython.

When I come across code that is slow I generally do the following
* Perform a quick code review to see is there are any obvious issues
* Fix the obvious issues
* If it still too slow I run a profiler. In Python I use cProfiler and look at the top few big hitters or view the output of the profiler using a tool call runsnakerun which gives you a great way to visualize the profile data and helps you answer quickly many questions you may have while looking at the profile data.
* I then make updates to the code based on the profiller results and then repeat the profilling and code changes as necessary.
* Next for a PyPy project and only in rare cases I would look at the jitviewer if I still needed more performance.

The jitviewer should only be used in the **rare** cases where you need that little extra performance and only after you know you have already implemented good algorithms and have reviewed the profiller results.

From looking at the code I would say your nowhere near the point where you should consider using the jitviewer. Of course this is just my opinion.

Hopefully you find this useful.

What was your idea for minimizing memory usage on CPython?

should have been

# define a struct for each type of msgpack type
float_struct = struct.Struct(“>f”)
double_struct = struct.Struct(“>d”)
…

Change was s/float_struct/double_struct/

By looking at the code there appears to be many opportunities to make it faster. So here are a few suggestions.

There are a couple of ways in which you can use the struct module. You can use the functions or the Struct class. When you use the functions it always has to parse the format strings where as if you create a struct object the format is parsed just once and you can call its methods as often as you like.

so instead of doing this inside a loop
struct.unpack(“>f”, some_string)

do something like

# outside the loop
float_struct = struct.Struct(“>f”)

# inside the loop
float_struct.unpact(some_string)

The idea is to create one struct object for each type of struct you need to deal with.

Another area to increase performance is to reduce all the string copying that is going on. The msgpack spec was designed in a way to reduce the need of copying data around. Unfortunately Python can’t take full advantage of this like you can in say C in which you can do pointer arrhythmic and a cast. Although maybe PyPy has something to help in this regard although I don’t know off the top of my head. What you can do in Python at least is keep track of the position in the message as you process it and use the pack_into and unpack_from methods provided by the struct module. These methods take a buffer and an offset which is your current position.

By making this change you will also eliminate the need to use PyPy’s StringBuilder.

Another area for improvement is to eliminate the heavy use of if and elif statements such as the ones used in the _fb_unpack method. Since the types in msgpack can be determined by reading one byte I would allocate a list of 256 tuples that would be used to aid in the unpacking process. Each tuple would contain two values. The first would be a reference to the struct object used to pack/unpack and the second for TYPE_IMMEDIATE, TYPE_RAW, TYPE_ARRAY, etc. So you would do something like this to set up the list

# define a struct for each type of msgpack type
float_struct = struct.Struct(“>f”)
float_struct = struct.Struct(“>d”)
…

# create a list to hold 256 tuples of unpack operations
unpack_operations = [None]*256

# create the tuples for the operations
unpack_operations[0x00] = …
…
unpack_operations[0xca] = (float_struct, TYPE_IMMEDIATE)
unpack_operations[0xcb] = (double_struct, TYPE_IMMEDIATE)
…
unpack_operations[0xff] = …

For types such as Positive FixNum, FixMap, FixArray, FixRaw, Negative FixNum you could set up one tuple for each one of them and use a for loop to add the reference to these tuple to the list. You can also do the same thing for the reserved types.

Then inside the _fb_unpack message you could do something like

struct_object, typ = unpack_operations[b]
struct_object.unpack_from(buffer, position)

# then do processing for the typ

That way you can eliminate all those if elif statements and increase the speed of the code as a lot less work would need to be done.

Another further improvement would be code up each of the typ operations as new private methods and instead of the 2nd item in the tuple being TYPE_IMMEDIATE, TYPE_RAW, TYPE_ARRAY, etc it would instead have a reference to these additional methods.

I suspect there are quit a few additional improvements that could be made but this should be a good starting point. I didn’t really take a good look at all the code so these suggestions were just from a quick look. Of course these suggestions also apply to the pack side.