Better polygon rendering in LibreOffice's Gtk3 Support

Above is how LibreOffice's "svp" backend rendered rotated text outlines in chart where the text is represented by polygon paths. Because the gtk3 backend is based on that svp backend that's what you got with the gtk3 support enabled.

After today's work above is how the svp backend now renders those paths when rendering to a cairo-compatible surface such as the gtk3 support provides.

If we mandate that "svp" only operates on cairo compatible surfaces, then we can get this niceness into android and online too, and can ditch our non-cairo text rendering code paths.

Insert Special Character in Spelling Dialog

LibreOffice 5.1 spelling dialog now has a little toolbar to enable inserting special characters into the spelling editing widget. Also Added paste, so the insert icon isn't lonely.

finding UI crashes by fuzzing input events with american fuzzy lop

As mentioned previously I've been experimenting using afl as a fuzzing engine to fuzz a stream of serialized keyboard events which LibreOffice reads and dispatches.

Performance is still pretty poor, but by tweaking our headless mode to allow dialogs to be created, then using that headless mode for eventtesting and then hacking out the actual rendering of the UI to the headless backend I've got something that performs reasonably well enough to enable me to set a far higher limit of 50 input events per cycle and start to discover real bugs in impress.

1. 5.1 only crash in impress sidebar under some circumstances
2. null marked obj still in impress mark cache
3. another null deref in impress sidebar panels
4. crash in impress if you exit while the annotation window is open
5. divide by zero in an impress sidebar panel
6. another annotation window null deref issue
7. crash on sending a keystroke to an empty impress page list widget
8. missing dispose on annotation windows
9. missing dispose on alive by unshown panels
10. crash if frame is destroyed before keystroke gets to it
11. crash if you close impress main frame while slideshow is running

all of which is encouraging, though some of these are possibly very unlikely in real world use. But the prized find is

intermittent crash on undo of insert slide

because I've seen that happen plenty in the real world, and is the problem I was hoping to find.

Turns out its been so difficult to track down because there's a timer involved which is triggered by earlier modifications to the document. To get it to crash by undoing insert slide you have to modify an object in the document, which triggers an object-modified timer, and then very quickly, before the modified-timer fires, undo insert slide. Which has the effect of impress not registering that the slide has been deleted. Some time later, in far away code, impress will crash on use of the deleted slide.

afl-eventtesting was able to find a sequence of keystrokes (which isn't a huge surprise seeing as I primed it with some insert and undo slide sequences so it didn't have to mutate things too far before it modified a document object after an insert and before the undo) to trigger the crash and the eventtesting + headless modes of LibreOffice gave a reproducible platform where the same events happen in the same sequence without any intermediate system-ui events to throw off the delicate timing. Once the thing is reliably reproducible then its just a matter of grinding through the debugging.

fuzzing LibreOffice input events with american fuzzy lop

This is a demo of using the american fuzzy lop fuzzer as an engine to drive LibreOffice UI testing by fuzzing serialized keyboard input events.

For demo purposes the UI is visible here, but it can be run headlessly too. Given enough time afl can fuzz the initial input text of hello into keyboard shortcuts to enter menus and launch dialogs.

I think the concept is pretty neat and fun, but this is still experimental stage stuff that I'm playing with and trying to up the performance.

view/edit shape control points

In LibreOffice 5.1 the shape properties dialog for enhanced shapes now lists and enables editing the control points. This is in addition to the preexisting mechanism of selecting with the mouse the yellow control handle of the shape, but enables viewing and fine control over the control values

impress save background image

Impress has a "Set Background Image" option in its slide context menu for a while. For 5.1 I've added a matching "Save Background Image" to save the current background image to file.

crash testing and coverity, conference report

Slides for this morning's Crash Testing and Coverity numbers presentation. Summary, all ok, numbers ~0. If I'm analysing this right, then the highest quality is achieved at the height of the holiday season.

vertical text columns preview

My small contribution to last night's LibreOffice conference hack-fest. In vertical text mode, the column view for pages now previews in the correct direction.

guadec 2015 porting LibreOffice to gtk3 slides

Presented our porting LibreOffice to GTK3 presentation at GUADEC yesterday. Here, as hybrid pdf, are those slides with a rough guide to our architecture there and current wayland progress.

It was pointed out after the presentation (by jrb), that our gtk3-themed spinbuttons had the up and down buttons in +,- order instead of the correct -,+ order. So that's fixed now.

crash testing, 80000 documents, 0 export failures, 0 import failures

The last LibreOffice crashtesting run reports our goal of 0 import/export crash/asserts. This is on a refreshed up to date 80000 docuent corpus from various bugzillas and other sources.

Earlier runs had been over a static collection of 76000 documents, hopefully now we have zeroed the dials we can refresh the corpus far more frequently, perhaps on every run, and actively trawl for crasher documents.

RHEL 6.7 upgrades LibreOffice from 4.0.4.2 to 4.2.8.2

The freshly announced RHEL 6.7 upgrades LibreOffice from 4.0.4.2 to 4.2.8.2. RHEL 7.2 will upgrade from 4.2.6.3 to 4.3.7.2. Fedora 23 will have 5.0.0 in it.

LibreOffice on wayland

Hacked LibreOffice a bit more today towards wayland support via the gtk3 vclplug. Good news is that it launches, displays and you can interact with it mostly as expected.

Under F22 with a gnome-on-wayland session, GDK_BACKEND=wayland ./soffice.bin gives me...

Which is encouraging as this morning I had no window contents at all. Downside is that I can't resize the window and the menubar is displayed behind the title bar. Maybe a client side decoration issue.

gtk auto-mnemonics support

Thanks to Simon Long over at raspberrypi, we now have auto-mnemonics support in LibreOffice under gtk3 and gtk2. So the underlines appear in dialogs when alt is pressed and disappear when released, while menus activated from the keyboard show underlines while those activated from the mouse don't. Gives a more native flavour to the UI. Only in 5.1 for the moment.

crash testing, 1 export failure, 0 import failures

I graphed our crashtesting improvement on importing documents a while back, and mentioned that while import failures had improved dramatically that the export figures weren't as shiny. But there's been some great progress there too, especially with the work mstahl has been putting in, so today the crashtesting has reported effectively three consecrative 0 import failures and with the first drop to 1 reported export failure (an assert) so I present the graph of export failure progress

 And an updated import crashtesting graph.

This is on our corpus of 76000+ documents sucked down from various bugzillas and other sources.

Equalize Width/Height

In LibreOffice 5.1 I've added an equalize width/height pair of adjustments to the "shapes" submenu when multiple objects are selected. Equalize Width and Equalize Height which adjusts the width/height of the selected objects to the width/height of the last selected object.

So if you need to adjust the shapes of a bunch of little images and shapes. Adjust one, then select the lot, selecting the reference one last, and use these to update the rest of the sizes.

Impress Slide Design

Selecting multiple slides in normal view and using the slide design dialog will now affect all the selected slides as opposed to the single last selected slide in 5.1 onwards.

gtk3 clipboard support implemented

Our LibreOffice gtk2 vclplug inherits from our generic X11 vclplug and so in lots of places we just continued to use our historic X11 vclplug for various things, one big example being clipboard support.

To do the same with the gtk3 vclplug would work for the case where gtk3 is backed by X11, but not if backed by wayland. So we needed to implement cut and paste with the gtk3 apis.

X clipboard/selection/cut and paste is errr... "tricky", so it was a bit of a death march to drag myself to the keyboard to go about this. But it turns out the gtk clipboard apis are really good and its reasonably easy to get it up and running. So the LibreOffice gtk3 vclplug now has clipboard support.

Last major thing is to get gtk3 gstreamer integration working for video playback and then it's mopping up territory.

impress, format, character, font-effects, Capitals

Added All-Caps character effects to impress/draw (for 5-1 onwards).

impress, right click, insert image

Added "insert image" to right click context menu in impress.

crash testing, 1 import failure

moggi described here our crash testing infrastructure. Basically we have a document horde mostly populated through get-bugzilla-attachments-by-mimetype which downloads all the attachments from our bugzilla (and a whole bunch of other bugzillas) that are in formats which LibreOffice can open. We then import the lot of them with the above testing harness looking for crashes and aborts. A new report tends to appear every 1-3 days.

These documents are filed in bugzillas. In many cases they were filed specifically because they were causing some sort of trouble for someone, so there are a lot of hostile documents in there.

We currently have 76221 documents in the horde, the most recent run reports one, one single solitary failure (an assert in a .doc about invalid positioning of a cross-reference bookmark in a document with change-tracking enabled).

Here's a graph over time of our failure rate. Where failure is either a straight forward crash, or a triggered assert. The builds are dbgutil, extra-debugging, extra-checking, assert-enabled, exception-specification-enabled builds.

You get temporary local peaks every now and then when either a new assert is added or someone introduces a bug. We have two purposes here, immediate regression discovery and historic bug removal.

We also have export crash testing, where the numbers aren't as shiny yet, but are on an equivalent downward trend. More on that in a while when we figure out how to fix this final import stinker.

gtk3 native theming menubar

After something of a struggle I appear to have the right gtk3 menubar theming combination for the selected item now after image...

before image...

more gtk3 theming

Continuing on the Gtk3 theming work. Now got the combobox and editbox rendering and sizes correct along with new gtk3-alike focus rectangles. Here's the after...

Here's the before of what the gtk3 effort looked like in 4-4

Here's the equivalent 4-4 gtk2 effort. Note that now in the above gtk3 theming we have a single focus rectangle for the full combobox rather than a focus rectangle around the non-button part of the widget and that, as in a normal gtk3 combobox, the background isn't set to blue when selected. I always hated that out of character blue listbox/combobox selection color. So certain elements of the gtk3 theming now slightly surpass the gtk2 one which is nice. Though clearly the spinbuttons are still effectively imaginary ones as they look nothing like the native gtk3 ones.

I also fixed (for both gtk2 and gtk3) that notorious checkbox issue where unchecking a checkbox would leave a portion of the check still drawn outside the checkbox rectangle.

new area fill toolbar dropdown

The GSOC 2014 Color Selector is in LibreOffice 4.4, but it's not used for the "area fill" dropdown in impress or draw. So I spent a little time today for LibreOffice 5.0 to hack things up so that instead of using the old color drop down list for that we now have the new color selector in the toolbar instead. Gives access to custom colors, multiple palettes, and recently used colors all in one place.

LibreOffice 5.0

And here's the old one for reference, I've backported the above change to Fedora 22's 4.4.X to address some in-house vented frustration at selecting colors in impress.

LibreOffice 4.4

gtk3 notebook theming

Starting to work on the gtk3 theming now. Here's a before and after shot of today's notebook color and font theming improvements.

Before:

After:

And a a random native gtk3 notebook for comparison

gtk3 vclplug, some more gesture support

Now gtk3 long-press support to go with swipe

With the demo that a long-press in presentation mode will bring up the context menu for switching between using the pointer for draw-on-slide vs normal slide navigation.

gtk3 vclplug, basic gesture support

gtk3's gesture support is the functionality I'm actually interested in, so now that presentations work in full-screen mode, I've added basic GtkGestureSwipe support to LibreOffice (for gtk3 >= 3.14) and hooked it up the slideshow, so now swiping towards the left advances to the next slide, to the right for the the previous slide.

gtk3 vclplug, full-screen presentation canvas mode

Newly added simple support to the gtk3 vclplug for "canvas" support which is the thing we draw onto for presentations. Which means the gtk3 vclplug now supports full screen presentations. Which required a whole massive pile of reorganization of the existing canvas backends to move them from their own per-platform concept in canvas to the per-desktop concept in vcl.

So now rather than having only one cairo canvas backend based on the xlib apis which is for "Linux" we have a cairo canvas for each vcl plug. The old school xlib one is moved from inside its #ifdef LINUX in canvas to the shared base of the gtk2, kde, etc backends in vcl, and there is now a new one for gtk3

Presumably there are lots of performance gains to be made to the new canvas backend seeing as I'm just invalidating the whole slide window when the canvas declares that it's flush time but slides appear to appear instantaneously for me and fly ins and move around a patch effects are smooth even in -O0 debug mode so I'll hold back on any optimizations efforts for now.

gtk3 vclplug,

I've been hacking the gtk3 vclplug for LibreOffice recently, here's the before image after scrolling up and down a few times. UI font not rendered the same as the rest of the desktop, bit droppings everywhere, text missing from style listbox, mouse-wheel non-functional

 Here's today's effort. Correct UI font, scrolling just works, mouse-wheel functional, no bit droppings.

After making it possible to render with cairo to our basebmp surface initially for the purposes of rendering text, I tweaked things so that instead of re-rendering everything in the affected area on a "draw" signal we do our initial render into the underlying basebmp surface on resize events and then trust that our internally triggered paints will keep that basebmp up to date and gtk_widget_queue_draw_area those areas as they are modified in basebmp and just blit that basebmp to the gtk3 cairo surface on the resulting gtk_widget_queue_draw_area- triggered "draw". This is pretty much what we do for the MacOSX backend.

The basebmp is now cairo-compatible so the actual LibreOffice->Gtk3 draw becomes a trivial direct paint to the requested area in the gtk surface from our basebmp surface

With our cairo-compatible basebmp surface the gtk3 native rendering stuff for drawing the buttons and menus etc can then render directly into that basebmp at the desired locations removing a pile of temporary surfaces, conversion code and bounds-checking hackery.

Further under the hood however the headless svp plug that the gtk3 inherits from had a pair of major ultra-frustrating bugs which meant that while it looked good in theory, in practice it still was epically failing wrt bit dropping in practice. Now solved are the two underlying clipping-related bugs. One where an optimization effort would trigger creating an overly clipped region, and another where attempts to copy from the surface were clipped out by the clip region.

Still got some glitches in the impress sidebar and of course the above theming engine is still missing a pile of stuff and slide-show/canvas mode needs implementing, but I'm heartened. Its not complete, but its now less traffic accident and more building site.

gtk3 vclplug, text rendering via cairo

The LibreOffice gtk3 vclplug is currently basically rendering everything via the "svp" plugin code which renders to basebmp surfaces and then blits the result of all this onto the cairo surface belonging to the toplevel gtk3 widget

So the text is rendered with the svp freetype based text rendering and looks like this...

With some hacking I've unkinked a few places and allowed the basebmp backend to take the same stride and same same rgbx format as cairo, so we can now create a 24bit cairo surface from basebmp backing data which allows us to avoid conversions on basebmp->cairo and allows us to render onto a basebmp with cairo drawing routines, especially the text drawing ones. So with my in-gerrit-build-queue modifications it renders the same as the rest of the gtk3 desktop.