Posted tagged ‘Protocols’

Google’s Wireshark Dissectors for NFC

May 25, 2012

Earlier, I noticed that @hiro99ma had ReTweeted a post from @eggman stating the following:

欲しかったやつだ。Googleの作ってるね。 / “wireshark-nfc – NFC dissectors for Wireshark. – Google Project Hosting” 

The Japanese text roughly means something like “Fellows wanted. People inside Google are making this”, from what I understand.

That aside, after cloning the Git repository into my local Wireshark SVN plugins directory, my initial attempt at building the code failed with:

However, I was quickly able to rectify the problem by exporting some environment variables:

export WIRESHARK_INCLUDE=$HOME/wireshark/
export WIRESHARK_LIB=$HOME/wireshark/lib/

Under my VirtualBox-based Ubuntu installation, the plug-in binary ( was installed in /home/tyson/.wireshark/plugins, after running make install again.

However, after starting Wireshark using sudo, it appears that the plug-in itself was undetected – since the aforementioned path isn’t the default plug-in search path for the root user.

When the dissector plug-in is unavailable, it is possible to open an LLCP trace file – but packets are displayed in a generic manner:

After moving the binary to /usr/local/lib/wireshark/plugins/1.7.2/, and restarting Wireshark, I was successfully able to dissect the packets in the example trace file:

Hopefully, Google will work with the upstream Wireshark developers in order to integrate this functionality into mainline, so that I can investigate integration of the NDEF payload dissector into my FeliCa and MiFare dissectors; and also see if it’s possible to integrate the main LLCP dissector with my NXP PN532 chipset-specific protocol one.

Attempting to install MOST4Linux 1.0.0 under Ubuntu – Part 1

May 24, 2012

Earlier on today, I decided to see if I could build the Media-Oriented Systems Transport protocol stack from the MOST4Linux project under Ubuntu 11.04.

However, it was originally designed for earlier versions of the Linux kernel than 2.6.38; and is no longer actively maintained by its developers – which makes compiling, and using it a challenge.

When I initially attempted to build the code, after downloading and extracting the most recent source code archive, the build process failed with:

By making a number of modifications to the source code and build scripts, I was eventually able to reach this stage:

Unfortunately, I’m unsure of the best way to continue – so I’ve decided to dump my modified code on BitBucket.

Those curious about my modifications to achieve the aforementioned result can deduce them from the “reversed” difference list – which was created by adding my modified version of the code into a special branch in a new Mercurial repository (“Initial_Ubuntu_Port_Attempt“), and then retroactively importing the original version of the code into yet another branch (“Original_Code“).

When I get the chance, I’ll probably document how I arrived at that conclusion, and share some resources that I discovered along the way.

Thoughts on Distributed Window Systems

October 12, 2011


In this post, I’ll propose Amadeus – a design for an experimental highly-distributed, extensible, and network-transparent windowing system that supports resolution-independence through the use of vector graphics technology, and scales from a single PC (or other device) to a large cluster of devices.

This will hopefully be achieved via the meticulous usage of compression, and adaptive data structures; the adoption of a multicast/broadcast architecture, and orchestration from federated or standalone Registrars.

My decision to write this proposal was inspired by interesting discussions with a friend regarding implementations of other systems (Display PostScript, NeWS, and the many window servers of Symbian OS); and reading various tirades against X11, plus documentation regarding various proprietary systems (especially Photon).

However, the system itself has a brand-new design (as far as I’m aware) – and probably plenty of stupid design mistakes that were rectified by others in the past…

High-Level Architecture

The Amadeus architecture consists of 4 main components:

  • The Rendering Surface – responsible for accepting hardware events, and rendering graphics data received from Applications via private channels or network broadcasts.
  • The Registrar – responsible for coordinating the activities of the rest of the architecture.
  • Applications – either built directly against a library that generates vector graphics data, and sends it to the Rendering Surface; or against a ported GUI widget toolkit (or compatibility layer for an existing windowing system) designed to do so.
  • Hardware Event Providers – responsible for collecting and dispatching hardware events.
These roughly fit together, like so:

Responsibility for font rasterisation and management is either directly accounted for by the Rendering Surface implementation, and a suitable rasterisation library (e.g. FreeType or MonoType iType) – in the case where text is embedded in SVG data, or by applications designed to deliver pre-rendered text as vector shapes and paths.

Descriptions of other window system architectures, such as Microsoft’s GDI and various successors, X11, Apple’s Quartz, QNX’s Photon, NitPicker,’s Wayland, and variants of Display PostScript are best found elsewhere.

Authentication and Confidentiality

Authentication and confidentiality of signalling and graphical data network traffic is beyond the scope of this proposal – as other parties have more expertise in that area, and I feel that I cannot immediately improve upon existing designs. Those interested in such functionality should investigate IPSec (which should work with multicasting, according to this paper from Cisco), and SSL/TLS (or SSH) for private channels.

The Registrar

The Registrar announces its availability upon launch via network broadcasts (or multicasts); and tracks Rendering Surfaces, HWEPs and client applications interested in either listening for hardware events, or displaying windows on a per-machine, cluster, or network basis by unique name.

It also stores client configuration data (e.g. connected display resolutions, the characteristics of connected HIDs (keyboards, mice and game controllers), and preferred data structure widths), in addition to orchestrating the window management activities of machines within a cluster.

Whilst the intention is obviously to eventually support large-scale clusters and networks of “screens and machines”, with application windows freely distributed amongst them (in the case of X11-style invocation and display of applications installed on other machines); multiple Registrar instances (supporting multiple clusters or standalone PCs) shouldn’t conflict with each other, and it should always be possible for users to decide upon levels of isolation, and appropriate network topologies.

Multicast DNS (as implemented in Apple’s Bonjour/mDNSResponder, and Avahi), or D-BUS might be feasible technology options for implementing parts of this functionality.

The Rendering Surface

The Rendering Surface implementation accepts compressed SVG data from applications, and hardware events from HWEPs, received via either a private channel (e.g. a transient UDP socket with a port number known by the Registrar, a UNIX domain socket, or a platform-specific IPC mechanism), or a network broadcast/multicast transmission.

Instances thereof must register themselves using a unique name to the Registrar (and optionally specify a private channel), and accept any relevant events from Hardware Event Providers, such as mouse clicks and movements, and keystrokes.

The SVG data that applications generate should accurately render using any quality SVG implementation – regardless of its ultimate bitmap data destination (e.g. a raw framebuffer, or Qt’s SVG rendering widget), post decompression.

Compression Algorithms and Data Structures

Data structures used by Amadeus for signalling are designed to be extensible and easy to parse (explicitly identified using 64-bit field type IDs, and accompanied with content length fields); and capable of being dynamically resized to accommodate network connections with varying levels of quality (i.e. latency and bandwidth).

If so desired, it may be theoretically possible to adapt data structures from other protocols, to provide additional functionality – although in the case of some (e.g. the X11 Clipboard protocols), it would probably be a better idea to design new ones, in the long run.

Although it was originally intended that “plain” SVG 1.1 data be compressed according to the WBXML 1.3 specification from the Open Mobile Alliance, it should be possible to support alternative serialisations of an SVG data stream (e.g. transformed JSON), and additional compression algorithms (e.g. those implemented by ZLib/GZip) via the aforementioned extension mechanism.

An Open Source WBXML parsing and generation library (released under a variant of the BSD License, with a promotional clause), written in C++ is available from the Sybyx project on SourceForge. Although I haven’t attempted to use it (yet), I received the impression that its API is reasonably clean and well-designed.

Bilal Siddiqui’s article also provides a rough idea of a conversion technique, and an archive containing a set of  XML files in both encoded, and non-encoded forms.

I have attempted to register the SVG 1.1 XML Document Type Definition ID (-//W3C//DTD SVG 1.1//EN) with the OMA, using their online form – although the process failed, due to a configuration error of the Web server.

Compression – State of the Art

Whilst I’m unaware of other, entire window system architectures utilising SVG for drawing, the technology has been successfully utilised as a significant part of KDE’s Plasma family of desktop environments; for rendering icons in the AVKON/S60 user interface framework for Symbian OS (and its successors); and for a multitude of other applications.

However, most implementations tend to assume that graphics files are stored as plain text XML files within a local (or remote/removable) file system, and can be quickly accessed prior to rendering – which works well for most applications, but is likely to be extremely inefficient within Amadeus, due to its network-bound, distributed nature.

The most obvious solution to this problem would be to use one of the aforementioned compression techniques (or another) – which is reinforced by this paper (circa 2003) from the seemingly defunct developers of the X-Forge game development toolkit, concluding that their proprietary compression technology compares favourably to PNG and JPEG bitmaps, in terms of resulting file size.

X-Forge’s developers were also able to achieve a bitmap-equivalent level of apparent image quality for a static set of textured game graphics (and further size reductions when cramming all of their resources into a ZLib-compressed archive), in addition to resolution independence – although in the case of Amadeus, it is highly likely that graphics will consist of both vector widgets, and wrapped bitmap images (e.g. photographs, Web graphics, and pre-rendered widgets from applications built against non-native toolkits), which may affect efficiency.

From very brief testing with GZip’s -9 argument, I was able to compress a copy of the SVG file (exported from InkScape in “Plain SVG” format) corresponding to the architecture diagram to 1.00 KB (1,024 bytes), from 3.97 KB (4,075 bytes). The original, uncompressed version of the file, exported without optimisations to remove InkScape-specific metadata was 5.18 KB (5,314 bytes) in size.

As previously mentioned, I have not yet tested a WBXML implementation against these files, for comparison.

Bitmap Image Support

Through the use of data: URIs, and Base64-encoded image files, it is possible to embed bitmap graphics data into SVG content, so that the aforementioned common use cases (e.g. display of Web image content, and graphics from other windowing systems) can be supported.

This technique could be combined with a hybrid local-distributed image caching system that is integrated with the Registrar, delta encoding of SVG payloads, and checksum-based URIs for referencing extracted bitmap image data within the cache.

More sophisticated approaches to this problem are described in a technical reference document from NoMachine, related to their NX unicast display/window sharing architecture (which in turn is closely entwined with the X11 architecture).


In isolation, distributed windowing systems; multicast architecture-based distributed systems in general; and vector-based GUIs are nothing new, conceptually. What Amadeus brings to the table is an amalgamation of these disparate concepts in the form of a modern, flexible design, that should be suitable for supporting multiple high-resolution displays, and users with wildly varying requirements.

Whilst I’ve left issues related to high-performance rendering of 3D graphics and video content, latency, security, and various other architectural aspects unresolved; I’ve hopefully provided an interesting starting point for debate on the design of such systems.

An ISI/PhoNet-over-USB dissector for Wireshark

December 25, 2010

Whilst working on Project Iris, I have found Sebastian Reichel‘s Wireshark dissector plug-in invaluable for identifying the content of ISI packets generated by my handset.

However, it relies upon the ability to use the Linux PhoNet stack – which isn’t always possible under certain circumstances.

For example, the stack may not be available at all under the running Linux kernel version; or the USB device generating ISI traffic may be connected to a virtual machine running a Windows-based application – which is obviously invisible to the host’s network stack.

With that in mind, I’ve decided to release a modified version of the aforementioned plug-in on BitBucket (in source code form only, at present), and I’ve uploaded a sample trace file to test it against, here.

Rough instructions for building it against an SVN release version of Wireshark under Fedora are provided in the repository; as are a copy of my colouring rules for working with USB and ISI traffic.

At present, the dissector has the following features:

  • Basic support for dissection of ISI/PhoNet packets encapsulated in USB framing (AKA “CDC PhoNet”) – for USB CDC_DATA class packets
  • Basic support for dissecting ISI GPS and SIM Authentication packets (inherited from the original version of the dissector)
  • Basic support for identifying specific types of CDC_DATA packets (works for ISI, PPP and AT/Hayes commands)

However, there are also a number of limitations and bugs – especially when compared to the original version:

  • ISI packets encapsulated in Linux Cooked framing are currently unsupported
  • Due to lack of heuristics, this dissector will override the PPP dissector (and the ISO/IEC 13818-1 dissector) when working with USB trace files
  • The length indicator may not always be accurate – although a lot of effort was spent on attempting to make it work

When working with this dissector, I recommend either using the isi.usbtype == 0x1b display filter, or individually filtering out various other types of USB packets, in order to avoid confusion.

For curious folks, a screenshot of the dissector in action is provided:

I hope that others find this useful for something.

That aside, I’d like to thank the following:

  • Chris Maynard for his USB patches (especially the CDC Ethernet one), which were useful for figuring out how to integrate with the USB dissector
  • Sebastian for providing the initial version of the dissector
  • William Roberts for providing the Nokia N73 that’s serving me well as my primary handset (and its USB cable, of course), and for persisting with me whilst I grappled with various stupid mistakes during learning C and C++

I wish readers a happy Christmas, and all of the best for 2011! 🙂

Project Iris: Affordable, Instant Connectivity for Syborg/QEMU

November 1, 2010

Apologies for not updating here as often as I wanted – although in order to keep things concise, I won’t detail the reasons for my hiatus in this post.

That aside, whilst I can remember the details, I’d like to share a proposal for a novel (in my humble opinion – but I’m prepared to be corrected) method of potentially using unmodified, off-the-shelf Nokia handsets as a modem under Symbian OS running on QEMU.

Please note that I have so far been unable to implement this, or test certain individual components (e.g. the Linux PhoNet stack); although I believe from the research that I’ve done that individual components should work in isolation.

Additionally, this isn’t intended to be a competitor to the excellent Wild Ducks project, or the ad-hoc efforts surrounding getting regular modems utilising Hayes/AT commands to work, either. (It’s for folks who for whatever reason either can’t afford to acquire a fully fledged Wild Ducks set-up, don’t want to commit themselves for the long-term, or just want a quick-‘n’-dirty way to test stuff that requires network connectivity).

With that in mind, I’ll introduce the architecture diagram, and hopefully try to provide further details – because a picture is apparently worth a thousand words:

Click to view full size

The system itself consists of the following components, in no specific order:

  • A version of QEMU with customisations specific to the Symbian Platform, as detailed in my ancient post on the Symbian Blog – and a few others, since then!
  • Two brand new components, which will be described in further detail later (the TI SSI bus “pseudo-modem” and the raw PhoNet-to-SSI bridge)
  • The Linux PhoNet protocol stack, which was contributed to the mainline Linux kernel by Nokia on behalf of members of what was once known as the “Maemo Computers” department (if memory serves correct)
  • Your favourite Nokia device, providing that it supports USB connectivity and the “PC Suite” profile – since that’s how we can access certain baseband services via PhoNet! (A well-kept secret, so it seems)…
  • The Symbian Platform (which consists of the Symbian OS, UI framework, middleware and other components) and the baseport – Syborg, in the case of Project Iris
  • Nokia’s baseband “TSY” (telephony support plug-in), which should work in conjunction with a well-designed TI SSI bus “pseudo-modem” and the raw PhoNet-to-SSI bridge to simulate the presence of a real Nokia baseband by proxy 🙂

The most interesting components are the TI SSI bus “pseudo-modem” and the raw PhoNet-to-SSI bridge, which are pivotal to making this thing work.

The raw PhoNet-to-SSI bridge can potentially either be integrated into QEMU, or left standalone –  although designing the IPC mechanism for the latter use-case is left as an exercise for the reader.

Communication with the device could occur via either a /dev/phonet0 device node (if such a thing existed, but according to this IRC log, it seems that it doesn’t under certain circumstances), or directly bound low-level datagram/pipe sockets to communicate with the user’s handset via raw PhoNet/ISI packets encapsulated in USB frames.

Obviously, the raw PhoNet-to-SSI bridge will encapsulate and decapsulate PhoNet packets that are transmitted/received by the handset into Texas Instruments-proprietary SSI frames for consumption by the “pseudo-modem”.

The “pseudo-modem” works in conjunction with the Nokia TSY (as mentioned earlier) and the raw PhoNet-to-SSI bridge; and will be a brand new, integral component of QEMU. It has minimal state of its own; and other than creating the illusion of a genuine Nokia/TI modem’s presence, it serves solely to transport packets between the bridge and the TSY.

Finally, the interaction between the TSY, network and telephony stacks and other parts of Symbian OS are extensively documented elsewhere.

For those curious about the title, the “instant” bit refers to the fact that as of recent versions of the Linux kernel and NetLink stuff, things should Just Work™ when a PhoNet device is connected (according to this page and this presentation from 2009), and that limited hardware knowledge is necessary to use one – just plug it in and switch it on.

The “affordable” bit refers to the fact that Nokia devices are relatively low-cost, easy to obtain, and plentiful (unlike specialist hardware such as the BeagleBoard and standalone GSM modems – as great as they are, for example).

Reverse-engineering an IrDA/USB Multiplexing Protocol : Part 1

August 1, 2010

Last month, I acquired a USB IrDA transceiver based upon a chipset from a seemingly defunct vendor (either the S110 or S110-SG1 from Tanic Electronics Ltd.), since it was fairly cheap (just under £12), and I had a spare phone with a washed-out display and an IrDA port sitting in my drawer to test it against.

Unfortunately, it seems that not only does this chipset/device not comply with the USB IrDA Bridge Device spec, but there is also no public data sheet, which makes writing a driver for non-Windows operating systems difficult. Theoretically, the Windows driver could be used under a modified version of NDISWrapper, or a similar NDIS API/ABI implementation – although that’s suboptimal in many ways (which should be obvious to some readers).

Those caveats aside, the device works fairly well when coupled with Windows XP’s primitive IrDA stack and associated utilities…

However, being the curious sort; and since the state of IrDA and USB debugging/tracing under Windows is dire, I decided to use Wireshark for analysis, coupled with the USB tracing functionality in recent versions of LibPCap and the Linux kernel – but in order to keep this post concise, I’ll provide more details in a follow-up post.