A gnome desktop and 3000 printers

We have something like 3000 printers. They are named something like “MF2301″ and “MF2302″. The printer properties luckily show the location of the printer.

So let’s fire up system-config-printer and search for the right printer. First thing is that it takes ages for system-config-printer to start with 3000 printers. Somewhere close to 20 seconds. What’s happening? Is it requesting the status for each printer? Is the delay caused by 3000 icons (Nautilus is faster when displaying a directory with 3000 files)? Then we have the search field. Hmmm it allows only searches on the name, not by location or description. Unfortunately our users are humans and not computers, so they usually know the location where they are, but not the number of the printer. So I first have to walk to the printer, write the name down and walk back to my thin-client to select the correct printer and click “set as default”.

Now I want to print something. So I hit <ctrl><p> in openoffice.org, and it shows MF00001 as printer? That was not my default printer! Openoffice.org shows a dropdown with all printers, so I have to scroll through the 3000 printers to select the right printer again.

So far for printing for now.

Update: let me be clear, I don’t want to bash the developers of the printer settings (I’m very glad it exists!!!!) but I just want to show some of the issues that arise in large desktop deployments.

memory leak debugging

For long running applications it is important that there are no memory leaks. For an application that runs for a short time the memory will be freed after the application quits, but for an application that runs for days or weeks or more, any memory that is allocated by the application should be free’ed by the application, otherwise I will not be available for other programs for a long time.

A very useful tool for memory leak debugging is valgrind. It makes your program run a lot slower (10X?) but it will resturn all interesting memory allocations that have not been explicitly free’ed. GTK does one thing that valgrind doesn’t like: the slice allocator does it owns memory management. Memory seems to be leaking, but it is ready to be used by the slice allocator. Luckily you can turn that off with the environment variable G_SLICE=always-malloc.

$ G_SLICE=always-malloc valgrind --tool=memcheck src/bluefish

valgrind will now report if you have memory leaks. To see where the leaking memory is allocated use

$ G_SLICE=always-malloc valgrind --tool=memcheck --leak-check=full --num-callers=32 src/bluefish

Valgrind will show several false positives, that is memory that gtk is allocating that is not supposed to be free’ed.

Sometimes the origin of the memory leak is not good enough, because it is a reference counted gobject, and you can’t find where the reference is increased that-should-have-been-decreased or another bug like that. A useful tool to debug that is the gobject-lifetime debugger library http://cgit.collabora.co.uk/git/user/danni/gobject-list.git/tree/ or the similar refdbg http://gitorious.org/refdbg/refdbg which is packaged for Debian.

The gobject-list library is used like this:

LD_PRELOAD=~/libgobject-list.so src/bluefish

I get a lot of output here, and there seem to many false positives (I hope, because valgrind doesn’t report them!?!). I have to play with this a little more to learn how to use it effectively.

2 Comments

New desktop GUI’s and terminal servers

Gnome shell, but also Unity, make extensive use of modern video hardware possibilities. Which is a good thing. The downside is that they do not function anymore without access to the modern video hardware. In an organisation that uses thin-clients and terminal servers over a wide area network this becomes a bit of a problem. Protocols like NX (Nomachine) and VNC (many products, such as ThinLinc) that can handle the high latencies on wide area networks do not provide access to these functions of the video hardware.

This means that the thin-clients are limited to the “fallback” gnome desktop. But how long will that be maintained? When will the first open source product decide to drop support for the oldfashioned gnome desktop? What if Empathy or Networkmanager will not work anymore with the fallback desktop? Does that make our thin-clients worthless?

What is your strategy regarding thin-clients?

On GList anti-patterns and good documentation

Several people posted GList anti-patterns, calling the code completely broken. Although I agree such code is broken, it must be said that the glib documentation for a very long time didn’t even specify if g_list_append() would return the first or the last item of the list. I know for sure that some of the early Bluefish code called append() in a loop. Currently there is a note in the glib documentation that calling g_list_append() has to traverse the list every time. That note is a major improvement, but for a beginning programmer it would be better to show an example that g_list_prepend() in a loop followed by a g_list_reverse() is actually faster than g_list_append() in a loop.

There are quite a few linked list implementations that look like the glib GQueue implementation (so they keep a pointer to the head and the tail) in which append() is equally fast to prepend(), so programmers coming from another language or library might not even think about the difference between the two. Perhaps we should  recommend beginning programmers to use a GQueue instead of a GList ?

Bluefish scanning engine: DFA state table limit reached

The Bluefish syntax scanning engine uses a DFA state table. To move from one state to the other we use a 16 bit unsigned integer. That means we can switch between 65536 states. But we now have a 100% complete HTML5 + SVG (+ all attributes of all tags in both languages) language definition, which needs about 70000 states.

A state table only needs to be able to switch to other states in the same context. Right now we keep all contexts in the same state table. A possible design change would be to create separate state tables for each context. That would raise the limit to 65536 states per context. Right now I cannot image which language file will hit this limit (but then, 640K memory is enough for a computer, right?)

Update: the new design is working, we have a  65536 state limit per context now, and right now no single language is getting even close to that limit.

Leave a Comment

Optimising the editor widget: the results

After a couple of redesigns the optimisations are almost done now. In almost all cases everything works perfect. It’s now down to good testing before this branch is merged with trunk.

Memory usage for the XML file with a million matches is now only 19% of the original memory usage (from 437Mb down to 85Mb).  Scanning a complete file is 10% faster now. But the best is that a small change in the file will no longer cause the complete file to be rescanned: most of the time only a small region needs to be rescanned.

 

Leave a Comment

saving memory: redesign of some editor widget internals

After my previous post Valentijn suggested to use a tree structure with pointers instead of a stack. That indeed reduces a lot of redundant information in memory. With the memory usage in mind I redesigned some of the internal datastructures and the first results are promising.The queue is now replaced by pointers to the parent structure, I reduced the number of pointers and added a state which is stored bitwise in a single integer.

The syntax scanning cache for the 12Mb XML file with 1 million matches is now back to 24Mb for the cache on 40Mb of QSequenceNodes, 7Mb blocks and 13Mb for context changes.  That’s only 22% of the original memory usage! Code is in a separate branch:  http://bluefish.svn.sourceforge.net/viewvc/bluefish/branches/bluefish_optimise_bftextview/src/

On normal small to medium sized files the difference in memory usage is much smaller, about 80% of the original memory usage. The scanning speed on small to medium sized files also hasn’t changed much  (~1.1X faster). Only on very large files the memory usage makes a difference in the scanning speed.

The other big improvement I want to add in this branch is more reuse of previous scan results. Currently everything following a change is always re-scanned. But many of the previous results are probably still valid.

Leave a Comment

The next challenge in bluefish optimization

I’m still playing with the 12Mb XML file. The syntax scanner matches more than a million times on a pattern in this file (1038246 to be precise). Each time we have a match, we store the context stack and the blockstack.The cache structures consume 56Mb and it needs 40Mb of GSequenceNodes.  The items on the stack are refcounted, so their memory consumption isn’t spectacular. There are 221348 blocks in this file, that use 5.1Mb and 570036 context changes that use 8.9Mb memory. But the stack itself is a GList. Actually, we have about 2 Million GList’s here (for every cache item two stacks), and together they consume a whopping 277Mb of memory for this file!!!

What to do: don’t store the complete blockstack and contextstack every time we find a match.That would help to reduce the 277Mb for the stacks. That still leaves the 96Mb for the cache structures and the balanced tree..

Leave a Comment

1) 16 bit limit overflow

A theoretical scalable design can be implemented with limitations, as I found out today. while fixing https://bugzilla.gnome.org/show_bug.cgi?id=637580

The file referred to is a 12Mb XML file, with about 200000 XML blocks.This file showed two problems in the bluefish editor widget implementation.

The bluefish editor widget used a 16 bit integer (a glib guint16 type) to keep the reference count of found blocks and found context changes. As you can image, the reference count overflowed on this XML file with 200000 blocks.

The solution: use 32 bit integers.

2) clearing GtkTextTag’s

Every 100ms scanning run bluefish starts by clearing any leftover GtkTextTags from old syntax highlighting. However, the GtkTextView widget uses > 100ms to clear the formatting for 12Mb of data. And thus the scanning for syntax didn’t even start (the total loop may take 100ms). The syntax scanning thus never finished.

The solution: only clear old syntax highlighting once, and use a boolean to track when we have to clear old syntax highlighting again. The first run it only clears old syntax highlighting, but the next run immediately starts scanning new syntax.

Theoretical performance and real scalability

Leave a Comment

Scale of the enterprise desktop – users, accounts, groups and permissions

Large corporations have many employees in many departments. And many of them will have an account. 16 bits for the UIDnumber is not big enough for some enterprises (but luckily the kernel handles 32bit numbers fine – but does your app?). All those employees in different projects and different departments means there are lots of different authorizations, meaning lots of groups, again possibly beyond the 16bit limit. And you may guess that the traditional scheme with owner/group/others might not do it – ACL’s are needed.

what does that mean for a GUI? For example a GUI to set file permissions:

• Ever thought of a dropdown with groups or users? Does that work with 50000 groups or 70000 users?
• Does it have a search field to select the right user/group?
• Does it display ACL’s in the GUI?

You can image that lots of users also means lots of users that forget their passwords. One solution to that is kerberos. Log on with your password, you receive a kerberos ticket, and you log on to every service using your kerberos ticket – never using your password again. Or better: logon using your PKI smartcard (with pkinit), you receive a kerberos ticket, and you never use a password at all! But this implies that all clients and all services support kerberos. The basics work well with Linux. Kerberos init on logon works and firefox understands it (so most internal web servers will work). But what about instant messaging (empathy?), voip and email clients? Lets make it worse: log on with dual factor authentication: a PKI smartcard with PIN code. Again the basics work, pkinit works perfectly on Linux, so you get a Kerberos ticket using your PKI smartcard. And even programs like gnome-screensaver can ask for your PIN code instead of a password. But GDM doesn’t understand it completely, you’re asked for a username while you enter your smartcard (that’s already passed with your certificate!). And your default gnome keyring won’t unlock anymore without a password (would be great if we could unlock it with the PKI certificate as well!).

• does your app work with Kerberos?
• will it work with dual factor authentication?

To manage a situation with this number of users, accounts and groups will be in a directory server, probably LDAP. In large enterprises all accounts are mostly in one level in the directory server. Smaller organizations sometimes try to organize accounts in their departments, but in large organizations there are so many people that move around to different departments, so many people that work in multiple departments, that they usually keep the departments as attributes in the account, and keep all accounts in one level. So what can go wrong. Image a ldap browser that lists all accounts per level: listing 50.000 of them won’t fir on your screen, and probably will take ages to load. You would think that most ldap browsers are designed for these situations, but they almost all suffer from this problem.

• can your app handle 50.000 results on a ldap query?

The good thing about the LDAP server is that all users have the same account on all systems, with the same permissions, same address, etc. So once you know the email address, you know their jabber and voip account as well. But oh: my email client knows how to look up names in a directory, but my jabber client doesn’t. And I cannot start a VOIP call from my email client – even if I know that the address is the same, I have to copy & paste it into another program.

• does your app support ldap directory lookups?

So there is some room for improvement here. And don’t get me wrong – I really like it that most things already work out of the box and how easy this is. It’s just the small things that could be improved.