How to get good user-testing feedback

One of the powerful aspects of open source software development is the fantastic end-user engagement. Many users use the bleeding edge code straight from the version control system for production work. Because of this, bugs in new code are quickly detected and solved long before a release. In the case of Bluefish I guess that more than 60% of the bugs is detected before release.

However, because these users do no formal testing but do production work, there is no guarantee that a new feature is actually used, and thus no guarantee that a new feature is actually tested. So you cannot conclude that the code is bug free if there are no bugs reported.

I would like to improve on the information position by collecting inforation what the users have actually used.

To collect this information for example a web application could be used where users can select which features of an application have been used. If a feature has been used, the user should be able to select some additional detail. In the case of Bluefish a user could for example select that the “toggle comment” feature has been used, after which the user can give more detail: added a line comment, removed a line comment, added a block comment, removed a block comment, with a selection, without a selection, etc.

I was searching for a web application to support this, but I haven’t found one. I expected something like this to exist already. Does anybody have a pointer how to collect this kind of end user testing information?

3 Comments

Bluefish on the Raspberry Pi

After ordering in September, I finally received my Raspberry Pi a few weeks ago (the upside of the long time between order and delivery is that mine is the new revision with 512Mb RAM).

I have no specific plans with the device other than playing a bit around with it. One of the things I obviously had to try was to run Bluefish as editor on the Pi. Installing all the build dependencies and compiling takes a few hours, but Bluefish was running as expected. Entirely true? No, some bits were slow, most notably the auto completion popup. So I dug into the code to find out why.

In the auto-completion popup, Bluefish has a “reference pane”. This shows some rerefence information about the item you are trying to auto-complete. For an HTML tag this might show the valid attributes, for a C function it might shown the arguments and the return codes etc. This is implemented with a hash-table and the “changed” signal on the GtkTreeSelection: if the selection changes, a lookup in the hash table is performed to see if there is reference information available. On the next key-press, bluefish re-calucalates the possible auto-completion candidates, and re-fills the GtkListStore that lies underneath the GtkTreeView. And this is where the problem was: before filling the list of items, Bluefish has to clear the old items. And the selection changed signal is called for each item that is removed from the GtkListStore, which in it’s turn does a hash table lookup and renders the reference information in the reference pane. Do that for 15000 items and you’ll have 100% cpu load for a second on the Raspberry Pi.

So what is improved now: first, the number of items in the auto-completion popup is limited to 500 items. Second a boolean is added that is set to TRUE whenever the popup is clearing or filling items. As long as that boolean is TRUE, the selection changed signal will do nothing at all.

The result: even on the Raspberry Pi, Bluefish auto-completion is again much faster than you can type, and every bit of sluggishness is gone. We’re close to the 2.2.4 release, and this fix will be part of 2.2.4!

3 Comments

Written manual or video howto’s? (and how to make them)

In time, the number of features in Bluefish has grown to a very large number. More and more we receive requests for features that are somewhat superfluous: a similar feature is already present in Bluefish. Often if we explain the submitter how to use a certain existing feature he/she is quite satisfied. This means that some Bluefish features are (too) hard to find.

So what is  the best way to improve this? Is written documentation with screenshots the best way to introduce features to the users nowadays, or are video tutorials (screencasts) better?

From the community the support for our manual project has been declining over the last years (I must admit that I personally think manual writing is not the most rewarding work). Currently the Bluefish manual is mostly improved by a single volunteer, and therefore continuously behind. Is this a sign that written manuals are considered less important nowadays? At least the Bluefish screencasts on YouTube do well (194000 views, but is that high or not?).

So what do you think? Written manual or screencast?

Second subject for this post: what is the best way to create screencasts? Previously I used the built-in screen recorder in gnome-shell: excellent quality with a high compression. Merging the sound, however, always was lousy (tried Pitivi and Openshot): the quality decreased while the file size usually grew a lot. And it was a lot of work. But with the last Ubuntu release (gnome-shell 3.6) the feature seems to be missing (Ubuntu bug or Gnome-shell bug?). What is the best way to create screencasts (with sound) in Gnome-shell?

11 Comments

Generating language definition files from xml schema’s

The Bluefish syntax scanner / auto-completion engine can deal pretty well with xml style languages. You can create a language definition file such that only the correct attributes for a tag are highlighted and auto-completed, and only the correct hierarchy of tags is highlighted / auto-completed. Not everything is supported however: entities are for example not expanded, and the syntax scanner is not tag-order-aware (and xml is tag-order-aware).

A while ago Daniel Leidert came up with the idea to generate language definition files on demand from DTD’s, relax-ng or xml schema files. I started prototyping some ideas, and I have some basics scripts in python now that can handle simple DTD’s and slightly more advanced relax-ng files. However, the scripts are not good enough to parse the XHTML DTD and XHTML Relax-NG definition (yet).

For the impatient, the current scripts can be found here: http://bluefish.svn.sourceforge.net/viewvc/bluefish/trunk/testcode/conversion_scripts/

Leave a Comment

Secure boot – but what about secure distribution!

Chain of trust

There is a lot of talk about secure boot recently – mostly because of the windows 8 certification requirements. Secure boot gives makes it harder (but not impossible) for malware such as rootkits to start at boot time and then control the operating system. It creates a chain of trust from the hardware to the bootloader to the operating system, so criminals cannot break that chain.

However, if we want to protect open source software against criminals there is a different chain of trust that we need to protect: the chain from upstream developers to the end-user. There are quite a few bits and pieces in place already, but some essential parts are missing. In august 2009, for example, the web server from the squirrelmail webmail project was hacked, and two plugins were compromised with malware. It was lucky that the hack was discovered very quickly so little harm was done, but that could have been much worse.

Most open source projects use a version control system for their source code that involves some form of crypto authentication. Some very basic, like a login over ssh to a subversion or cvs server, other require every patch to be signed with a PGP key. Most distributions use some form of signing to protect the integrity of their packages. But the step from upstream development to the distribution is often not secured at all. Many upstream developers offer a md5 or sha1 sum of the downloads – but hey, once the criminals hack your webserver, they can change both the md5 and the tarball with the source code!

So what should we do?

If all upstream developers would sign their releases with PGP, and distributions should check if the source tarball is correctly signed, and signed with a trusted key, it would be much harder for criminals to interfere with this step. The level of trust could be very minimal (just check if the source tarball is signed with the same key as the previous time we downloaded the package) or very high (require a web of trust where keys have to signed after an official government issued ID has been checked such as Debian requires), just depending on the importance of the package.

Bluefish source tarballs have been PGP signed for a while already. Now it’s time for the distributions to automatically check these signatures when building a package.

Bluefish 2.2.3 released

Last months we have worked on lots of small things in your favorite programming editor Bluefish, resulting in the 2.2.3 release. There are only few major changes: a corrupted state in the syntax scanner that could (very rare) lead to a segfault was fixed, code folding had major fixes and improvements, search had a few major fixes, printing with syntax highlighting was added, and a lorem ipsum generator was added.

Then we had lots of small things. The GUI was restructured in some areas, most notably the preferences dialog, but alsi in the the Tools menu and the HTML Tags menu, and some shortcut key combinations were added.

As I pointed out in one of my previous posts we did work on some visibility features, such as a bigger cursor and cursor highlighting, and some options were improved such as zoom and the custom colors.

External commands had some changes such as better cursor positioning after a filter has
been used, user supplied arguments for external commands (for example define a command chmod %a %f so you can add the mode when you activate the menu entry), and an option to restore the default commands. While writing this post I just found the first regression in 2.2.3: the help text that explains all options for external commands has disappeared…

[edit: after opening the dialog again I realized that the help text is moved into a tooltip to save screen space, so it did not just dissapear]

On the multi-platform front: the broken shortcut key S was fixed on OSX, and file recovery was fixed on Windows.

On the web front some dialogs were added for HTML5, the thumbnail generator was fixed, and insert color, path and relative path have been added.

Many language files were improved, and more user configurable options have been added to most language files. For example if you want functions to auto-complete with a semi-colon appended, or if you want a block from square brackets to be foldable. Unfortunately these options require a Bluefish restart, because the language files need to be reloaded and re-compiled into a DFA table for these settings to take effect.

A new feature: you can now select a block of text by dragging the mouse in the margin, and move the selected block with <ctrl><up> and <ctrl><down>.

Now get Bluefish here!

1 Comment

Install Gnome3 on Ubuntu 12.04

You like the nice configuration defaults of Ubuntu but not Unity?

Add this to your /etc/apt/sources.list

deb http://ppa.launchpad.net/gnome3-team/gnome3/ubuntu precise main 
deb-src http://ppa.launchpad.net/gnome3-team/gnome3/ubuntu precise main
deb http://ppa.launchpad.net/webupd8team/gnome3/ubuntu precise main 
deb-src http://ppa.launchpad.net/webupd8team/gnome3/ubuntu precise main

Now run

sudo apt-get update && apt-get install gnome-shell

and you have an up-to-date gnome-shell. Even many of the pretty extensions are packaged as well.

As a bonus I found out that Gnome 3 runs much snappier than Unity on an old IBM T43 laptop (which has 6 years old Centrino technology).