Category Archives: Uncategorized

Accessibility 3.0

Last week I attended a week-long meeting at the Sun offices in Dublin. The purpose of the trip was to hammer out issues with the D-Bus AT-SPI port, so that everyone can feel more confident about its readiness for Gnome 3.0.

What a great week we ended up having. I should first say thanks to all that attended, Willie Walker, Mike Gorse, Li Yuan, Ke Wang, Brad Taylor, and Rob Taylor. Sun, Novell & Codethink also need thanking. They paid for everyone to be there, some flying from China, with no foundation money involved. The Gnome community is extremely lucky to have Willie devoting his time to accessibility. I dread to think what Gnome a11y would be like without his organization and direction

Willie organized the event, I imagine more out of fear at what he found in the D-Bus AT-SPI code-base than anything else. All told I think everyone was pleasantly surprised. The project will take a-lot of work to get to the level of maturity required, but loads has already been done. What is required of us for Gnome 3.0 isn’t our of our reach.

We started the week with a very brief overview of the design for AT-SPI D-Bus. Only Mike and I had much experience with the project and we wanted to get everyone up to speed.  This very quickly turned into a Q&A session and we figured out what the main issues were on Monday afternoon. Tuesday was spent on the important task of checking the D-Bus protocol specification against both the code and the IDL spec. This bought up a number of bugs, issues and improvements. It was well worthwhile. Amazingly Wednesday & Thursday were spent hacking by everyone, which is a great feat considering that four of us had only peeked at the code previously. So much got done. Orca is now running with a fair degree of success and we can move on to performance issues and application-specific bugs. For a full run-down of the still-to-dos and what-got-dones take a look at the weeks wiki.

Dublin

Outside of work was great fun also. Dublin really is a beautiful city. Lots of green spaces nice buildings and extremely friendly people. The business park is pretty swanky also, and luckily the Oracle buildings are across the road from Sun. We went out to eat and drink in Temple bar, which apparently isn’t a bar. Wednesday involved a trip to the very very small Novell offices in Dublin. They are a way out of town, Brad and I were lucky to get there at all given the distinct lack of signs and road-names that have changed since Google scraped them. We met up with Alan McGovern, a Moonlight hacker, and along with a meal and a few drinks geeked the night away.  Alberto Ruiz took us all out for a nice meal on Thursday night. I forget where it was. You’ll have to ask him for the recommendation. Later we attended the Sun Pub Quiz. Luckily for me, our arrival was just as the Quiz part was ending. I’m not so good with the questions.

Me for 3.0

What am I doing for 3.0 I hear you ask? I’m sure you didn’t but I’m going to blurt it out anyway. I’ll be finding time any-where I can to work on D-Bus accessibility. Codethink has already dedicated lots of my time to the project, and there may be a little more in the future. The accessibility plan looks a little sketchy in the road-map, but I believe that with some hard work it can become a big success for the big release.

me3

After ranting about my work I’d love to hear what everyone else us up-to. There must me more than a few semi-secret 3.0 charges taking place.

‘Do-ifying’ Gtk (and other applications)

Many thanks to racarr for posting about this earlier today.  I had seen posts on the ‘Do-ifying’ of GTK but they had passed me by. ‘Do-ifying’ applications is a fantastic idea. We shouldn’t limit ourselves to GTK apps though.

The bit that caught my interest was that racarr was thinking of exposing a UI heirarchy over D-Bus to allow Gnome Do to find key-bindings and make them available. I suggested that AT-SPI would be a better way to do this, as all key-bindings are already exposed. I come to no conclusions over whether A11y is the best way. I’m biased. :) It did get me thinking about how A11y technologies could be used to provide general application assistance with Gnome Do.

Context sensitive ‘Do’ commands

Orca, the gnome screen reader, uses a11y mainly by responding to ‘focus’ events which are emitted when input focus moves to a new widget. Gnome Do could use ‘focus’ events to provide context-sensitive commands. By knowing which application, and which part of an application the user is currently focused on it might be possible to provide a more tailored selection of commands.

Perhaps the choice of ‘Do’ commands for each application would depend on finding out what D-Bus interfaces it supports.

Application generated ‘Do’ commands

Obviously for application generated commands we need methods of exposing these to Gnome Do. My preference would be to do this via D-Bus, which again a11y could help with. All ATs (Assistive technologies) need to know when new accessible applications appear on the desktop. Towards this end, D-Bus AT-SPI has a daemon that acts as an application registry and informs ATs when new applications are added. This registry could be made more generic so that Gnome Do would listen for new applications and register their ‘Do’ commands when they are started.

I can’t describe the registration interface here. I don’t know Gnome Do. I’m imagining a list of actions with a command name and descriptions, but it could well be much much more complicated.

Key Bindings

As racarr suggests, it should be possible to inspect the applications UI hierarchy to find all the key bindings and present them and their descriptions to Gnome Do. GOK, the gnome on screen keyboard, already does something VERY similar to this by inspecting an application and presenting all the actions of the currently focused window in a simple manner. There may be other things we can infer from the UI hierarchy, but I wouldn’t want to get too ambitious here.

I’ve often thought that a11y technology was under utilized. As applications on our desktop are already merging together a little by providing and accessing more D-Bus services, a11y seems like it could be well placed to enable some really innovate interfaces for the general user. If this happens it could really help experiences for accessibility users too. More bugs fixed, more descriptions added to widgets with key-bindings. :)

Gnome, KDE & Mono A11y

Congrats Mono A11y

Many congratulations to Mono Accessibility, team for getting their first release out into the open.  The Mono A11y team must be one of the largest open-source A11y groups out there and I’m really excited about the work they are doing. WinForms and Moonlight are not yet my thing, but if Silverlight takes off the UIA provider they have created will undoubtedly form an essential part of Linux accessibility.

I don’t believe that it will be in the first release, but I’m really keen to see work start on the UIA client library for Mono. C# and Mono sound like a great place for developing new ATs.

AT-SPI D-Bus on freedesktop.org

For people who don’t know about Gnome accessibility or AT-SPI D-Bus:

AT-SPI D-Bus is a project which aims to use D-Bus instead of ORBit/CORBA as the IPC mechanism for Linux accessibility. For anyone interested in finding out about the Gnome accessibility architecture the developers page has some good information. Oddly enough KDE has a very good Gnome A11y overview, and Sun has a good diagram. Long story short the AT-SPI D-Bus aims to write a new, D-Bus based adapter for ATK, a registry daemon, and client libraries that are API compatible with the existing cspi and pyatspi.

The project has a new home on the freedesktop.org servers.

The code-base exists at:  git://anongit.freedesktop.org/git/at-spi2/at-spi2-core.git.

We are keeping a page on the linux-foundation wiki updated with all our progress. Unfortunately I’d say that the code is not yet ready for a first release. For reasons soon evident the code isn’t currently getting the love it deserves. (Help MUCH appreciated)

The reason we chose freedesktop.org and the Linux Foundation instead of Gnome hosting is that we wanted to emphasize the cross-desktop possibilities of a D-Bus based accessibility architecture.

Gnome, KDE & Mono: How it all fits together

The Mono A11y architecture diagram is missing something important that the AT-SPI D-Bus project can add – QT accessibility.

The drive to D-Bus accessibility came from ORBit deprecation, the embedded community and an ideal of cross-desktop accessibility. Its the last motive that has me most excited right now. QT currently has a D-Bus framework based heavily off AT-SPI, but unfortunately it has never been taken far enough to be compatible with existing AT-SPI ATs. The reason that the ATK, cspi and pyatspi libraries are not getting my attention right now is that I really want to get started on bringing QT into the mix.

A QT adapter for AT-SPI D-Bus will certainly round-out the Accessibility infrastructure on Linux. Not being involved in the KDE community I don’t have much say on how they do A11y, but I hope to make it as easy as possible for them to choose AT-SPI D-Bus. Along with the Mono work this could mean that QT, GTK, ATK, Winforms & Swing apps are accessible, using the same ATs, in both KDE and Gnome. I think that would be a fantastic achievement. If we work hard enough accessibility could be one of the big success stories of a joint Akademy/Guadec next year.

The book meme

I like this meme, its a fantastic window into how depressingly geeky we all are.

  • Grab the nearest book.
  • Open it to page 56.
  • Find the fifth sentence.
  • Post the text of the sentence in your journal along with these instructions.
  • Don’t dig for your favorite book, the cool book, or the intellectual one: pick the CLOSEST.

“Some learned people are intelligent.”

From “Mathematics for the Nonmathematician” by Morris Kline. This is actually an excellent book supposedly for non-science majors to learn about mathematics. In reality its a very nice read on the history of mathematics, with some problems thrown in.

The choice of this book was a grey area according to the rules. The closest was a copy of Nature Genetics, but this is published in volumes and so started at page 700. The next closest was a stack of books, the top one being the latest Buffy comic. I’m not sure comics count. Can anyone make a firm decision about that?

D-Bus Accessibility – Its alive(ish)

This may not look like much more than another screen-shot of Accerciser. You’ll have to trust me that is Accerciser inspecting firefox-2 over D-Bus. Its the first visible step in a long project aiming to replace ORBit with D-Bus for the AT-SPI accessibility protocol.

accerciserliverpool

Mike Gorse and I have been hacking on this for a little over four months now, and its been pretty slow going. I’m very happy to finally have something to show for it.

So much to do…

What we have however isn’t quite the finished article. One of the main goals of the project is to help bring together Gnome and KDE accessibility. I believe that QT already has an ATK implementation, but I think it would make more sense for the QT accessibility interfaces to meet directly with the AT-SPI D-Bus protocol, a project that has not-yet begun.

On the ATK-Bridge / Pyatspi side there is still a huge amount of testing and bug fixing to do, along with a fair few missing features.

So much to test…

Its testing thats really occupying my thoughts at the moment. A while back, on a linux-foundation conference call, I remeber a conversation about how to test the pyatspi and cspi interfaces.

The conclusions were:

  • Have lots of small applications, each exposing only a few accessible objects and concentrating on a single accessible interface, such as the Component or Image interface.
  • Have a unit test written for cspi or pyatspi that is partnered with one of the mini applications, and inspects it over the AT-SPI protocol.

I have made a poor attempt to unit test pyatspi in this way, but I found it was a-lot of work to get any coverage. I think this is down to my choice of implementing a dummy ATK framework to create the applications. Mini GTK applications would have done just as well.

The advantage of lots of these mini applications and unit tests is that when someone comes to implement the AT-SPI protocol with a new widget set, or with new client side bindings they have something simple to validate against.

Where to go

The repository for the D-Bus AT-SPI work is still on the Codethink servers. It won’t be there for long though, there are plans for the project to be hosted at freedesktop.org soon.

A week in the life of a braindead hacker

Help needed

There was some discussion this week on desktop-devel about turning ‘Assistive technologies’ on by default in Gnome. See Willie Walkers original e-mailRob Taylor and I had discussed this previously, and at the time I didn’t have much of an opinion either way. After reading Willie’s e-mail though I was firmly in favor.

However Willie stumbled across a possible performance problem when accessibility is turned on. Using gtk-demo to test memory usage ‘top’ shows that accessibility can increase the ‘DATA’ memory section by 15mb, or over double the memory consumption. This was pretty shocking to me, so I tried to use massif to find the cause.

Thing is, massif reports that the memory allocation is not greatly increased by turning accessibility on.  The maximum stack size is not increased either. The ‘DATA’ field of ‘top’ apparently reports the DRS, or data resident set. The number of pages used by both the stack and heap. Rob suggested that fragmentation could be the problem, but in general I’m pretty stumped as to what is causing the discrepancy.

Help and suggestions greatly appreciated :)

Conduit conversations

We have John Carr in the office now. (Infact he’s been there more than Rob and I this week.) This means lots of conduit discussions. I was treated to a crash course on Conduit internals that left my brain feeling like a squeezed sponge, wrung cloth, and juiced orange all at once. This syncing thing is terribly hard and John has some difficult use cases that he wants to make work. (Tranferring contacts from his laptop go to GMail, and from GMail to the N810 and from that to his laptop. Whew.) Good luck to him. I think its the ‘Brain the size of texas’ award if he manages it.

The rest of the week…

Rob probably can’t blog to tell you this because he is gibbering somewhere in the corner muttering “Reference, Reference, where art thou?”. He’s been debugging lifecycle issues for a fortnight now. Its left us vowing never to touch ‘C’ again whenever humanly possible. (Which is not very often).

I have been working hard on the D-Bus version of pyatspi. Its going much slower than hoped, but i’ll get there in the end. I have a holiday coming this week, a trip to Finland for a friends wedding. I’m really looking forward to it. Were going to have a week of saunas in Lapland.

Memories of GUADEC & Istanbul

I had the pleasure of attending my first GUADEC last week. I doubt I’ll have a more enjoyable, tiring, embarrassing, or geek-filled time for the rest of the year.

I started the week locked up in my hotel room. Rob Taylor, John Carr and I were desperately trying to finish a demo that did not make it. Our trips out until Tuesday were mostly visiting restaurants, which would have been enough for me. Lamb Kebab and rice is just about my perfect meal, and during my entire stay in Istanbul I ate like a very happy king.

Once I got myself set up at the conference things got even better, there were some great talks. ‘UXD? WTF!‘ by Leisa Reichelt, ‘Introduction to the desktop data model‘ by Owen Taylor and ‘Syncronization and the GNOME deskop‘ by John Stowers all had me thinking.  It was a real shame that more people didn’t turn up to ‘Accessibility happenings in Novell‘, although it was the very last talk of the conference. Rob and I did a talk, ‘Wizbit a distributed revisioning file system‘, on Thursday morning. It was well attended, and I had a good time speaking.

The whole thing was smaller, and far friendlier, than I had expected. I was fortunate enough to meet Stormy just before, and soon after she had been announced as the GNOME executive director. She was just as lovely to me both times, so I assume power didn’t corrupt too much. :) Thanks to everyone I met for creating a thousand interesting conversations.

I can’t properly express my thanks to the people in Istanbul. They were incredibly helpful, patient and friendly. Even the ‘flexible prices’ that everyone seemed to experience only got bad when it came to taxis and drinks.  Thanks to Ali Sukru Goksu for his organization, and for showing us a great night out on Thursday. The conduit team and I went to an electronica festival on Friday night, which we never would have made it to without being looked after. Thanks Tuçe, Thanks Elvin.

The party prize and the most ‘flexible’ price prize go to John Stowers, for his 5-in-a-row 6am parties and for a certain vodka red bull.

I’ve come back really enthused about GNOME, about the people and the technology. Hopefully I’ll be able to put that enthusiasm to good use over the next year. See you all in Gran Canaria.

Python tidbits

During work on client bindings for AT-SPI this week python managed to surprise me in a couple of ways.

  1. Lambda doesn’t work as I think it should.

    lambda x: del(x) results in a syntax error. This is clearly stated in the python language reference: lambda_form ::= "lambda" [parameter_list]: expression The del statement is not an expression; unsurprisingly its a statement. Its still pretty non-intuitive.

  2. You cant weak reference a weak reference.

    This must be an implementation issue. I’m sure a weak reference of a weak reference makes sense. It did to me at the time!

  3. __slots__ is the most useful thing in the world.

    Just take a look at this graph. Its the results of a totally scientific, completely well researched, test into the size of different structures in python. They are all holding roughly the same amount of information.

    Python memory use

    Structure Type Memory use (kB)
    Slot Object 18988
    Tuple 19252
    List 19476
    Dict 42768
    Object 44320

D-Bus AT-SPI is well underway

Work is starting on the port of the AT-SPI framework to D-Bus. Codethink are, graciously, being funded by Nokia to undertake the work. Mike Gorse is working on the project full time, as part of his work for Novell.

Mike has got alot done and there are some initial D-Bus specifications, as well as alot of code on the ATK to AT-SPI adapter side. For the moment we’ll both be working in a git repository on Codethink. Hopefully we will be able to move the code into a repository on freedesktop one day.

The AT-SPI D-Bus work is important as it is attempting to eliminate the last vestiges of CORBA from Gnome and at the same time help with cross desktop accessibility. I’m hopeful that despite the transition we will be able to create a flexible and performant D-Bus protocol for AT-SPI.

I’ve written before about our results on D-Bus vs CORBA performance, done in preparation for this project. The results were a bit shocking for the D-Bus team so there need to be some significant changes to the IPC for things to remain snappy.

A week of head scratching and ranting

I’ve taken part in a couple of phone calls this week, a quick conversation with Michael Meeks and a conference call with the open a11y group. Both about AT-SPI D-bus. There was a week of silence after my initial e-mail to lots of accessibility lists, and after that something of a storm. It was good to actually hear everyones opinion, I think people care, and AT-SPI D-Bus will probably happen sooner rather than later.

Thats pretty good news as Rob and I have spent LOTS of time discussing this one, with some of it being rather heated. Its all about “standardising” the AT-SPI interface, what constitutes a “component” system, and whether accessibility really needs object lifecycle management. :) Could anyone really have raised voices in a conversation about that?

I’m going to put together our thoughts / proposals for everyone to go through. (With heavy input / modification from Rob Taylor) Hopefully it will be up to scratch.