System A2

Content Loading...
September 2002
System A2 is an inconspicuous persistent computing platform. While not being worn, the system can be operated remotely, and is able to autonomously adjust itself depending on the relative physical position of its user.

While searching for a wireless controller for System A1, it was found that some of the more recent Sony-Ericsson (SE) phones are able to interact with a computer over a bluetooth connection. The usefulness of this was immediately apparent; a mobile phone is a commonplace appliance, allowing it to be used as a controller (with the computer stashed in a backpack) without drawing attention to the user.

Once built, the system proved enormously useful, and is still being used by the author as a portable music player on a daily basis. It is seen that this will be used the base platform for future systems to be built on. New developments in small form factor computing will also increase the effectiveness and portability of this system.
This system is an attempt to change the relationship between the user and the computer - instead of relying on the traditional 'seated desktop' environment, the user can interact with the machine on his/her terms. A typical scenario is illustrated below.
At 5pm, user 'B' finishes the last of his work for the day before folding up his laptop and placing it in his bag. Donning a pair of wireless headphones, he uses his phone to start some music playing for the long walk to his car. On the journey, he uses his phone to switch to a nice peaceful playlist suitable for the serene walk. Once in his car, the music continues playing uninterrupted, transitioning from playing via his headphones, to playing through the car's stereo.

At home, a similar transition takes place, with the music now playing on his home stereo system. Taking the laptop from his bag, he checks his email and decides to run down to the shops, leaving his laptop still running. As soon as he's out the door, the system detects that he's left, turns the music off and switches to power saving mode. Upon his return, the music and laptop both wax back to life. As he retires for the evening, 'B' uses his phone to turn the music off, and sets his phone's alarm to go off in the morning.

At 7am, the system detects the phone's alarm, and brings the music back to life, wakening 'B' and automatically performs several operations (such as opening predefined URLs and checking email) preparing 'B' for the day ahead.
At this point it is important to point out that this concept was driven by what we already knew was possible - it is the result of bringing together the features of available hardware, and not the outcome of an unresearched 'wishlist' session.

As in SA1, SA2 is based around music playback; this allows us to focus on the interface and issues that can arise in making a continuous system like this; specific applications can be added as part of subsequent systems (for example, SA3 will be a real-world implementation of the GeoCode service).

It was seen that the system would consist of a laptop (in a backpack while worn) with wireless audio capabilities to allow it to pipe its audio to wireless headphones and stereos. Input/Output operations would be performed through the Bluetooth phone. The use of wireless equipment would make the laptop largely transparent - the system is available anywhere within the wireless radius, taking it beyond 'portable'.

At regular intervals, the computer could 'ping' the phone; if it remains unresponsive, it can assume that the user has left the local area and can pause music playback and perform other operations (such as activating a power-saving mode or locking the workstation).

This raises the idea of automatic personal networks - if several machines in separated locations were networked, the user's desktop could seamlessly travel to and from each machine, following the user's movements. While systems like this exist in corporate environments, there are other applications in non-computing fields, for example you could imagine each user's lighting and airconditioning settings follow them around the house. However, this is a deep field of study, and is perhaps best left alone for the time being.
Problems were immediately apparent upon further investigation into wireless audio - most RF wireless audio devices require a mains-powered base station, and there were no suitable Bluetooth headphones on the market (only low audio quality headsets). Using a portable FM transmitter together with some Sony SHR-M1 FM headphones seemed like the only viable solution, and it would have the side effect of allowing broadcast to the car and home stereo. Unfortunately, when tested, it turned out that FM transmitters are horribly underpowered (partially due to broadcasting regulations in Australia). This, combined with the weak reception capabilities of the headphones, made it useful only as a proof of concept system; at times it was near impossible to hear any music through the hail of static. In the end we fell back on using wired headphones, deferring the use of wireless audio until proper hardware is available (see 'The Future', below).

In coding the phone-PC interface, the SE t39m developer guide proved useless. Although after analysing the traffic going to and from the phone, it was found that the developer guide for a completely different phone (the R320) contained the appropriate reference. These new-found commands allowed a multitude of input styles - menus, text input fields and simple keypress-relaying, together with the ability to write back to the t39m's screen allowed the phone to be easily used as a dumb terminal, and raised more possibilities than originally anticipated (for example, it would be possible to use the phone as a tiny telnet client).

As before, a heirachical menu system was used to navigate between the different applications. In this case, only a music-control application was made available, which, once selected, switched the phone to keypress-control mode, with different keys on the phone mapped to different playback functions as depicted in Figure 1.
phone navigation
Figure 1. Music control interface.
The software framework (coded in Python) was built with expansion in mind, making it easy to add functionality at a later date. While there were some minor bugs (particularly when stopping and restarting threads), none adversely affected the running of the project, and so were left to be fixed in a future revision.
In initial tests, the mobile phone performed as intended - although we had one issue where it would cease responding if we sent it too much data. Changing the 'song currently playing' message (Figure 2) more than two or three times a second would trigger this lack of response, so we made that update itself once a second, rather than whenever the song was changed. Unfortunately, this latency introduced a usability concern regarding lack of instant visual feedback when rapidly switching between several songs.

phone navigation
Figure 2. Screen showing song currently playing.
The bluetooth connection between the phone and laptop was mostly reliable; in a stable situation it stayed uninterrupted for 48 hours. In portable use, connection would occasionally be lost for no identifiable reason, although this occured on average only once every two days. Occasionally the laptop would need to be rebooted, as the PC bluetooth drivers seem to still be in their infancy.

As a stationary system (with speakers replacing the headphones), the system performed like a personal stereo, with the advantage of having a larger music library (in mp3 format) and a remote that worked through walls and floors. The most positive feedback however, was in regard to the system's control of the music upon arrival and departure of the user, automating and removing the 'enable/disable' requirement in the human/machine relationship.

For wearable use, the laptop was carried in a Crumpler bag, with the headphone leads trailing from an opening near the user's shoulder. The same bag provided a convenient mobile-pocket on the chest-strap, allowing easy use and storage of the phone. Visually, there is nothing that would alert an observer to the fact that the wearer is using a computer. The system was used on an hour long daily commute (involving bus, train, tram and walking) for several weeks. The following concerns were noted:
  • Having to remove the mobile from its pocket and flip open the lid before use became annoyingly repetitive in situations where minimal interaction is required.
  • The headphone wire caused some problems on the first few days, with the user becoming entangled every time the bag was put on or removed.
  • The laptop became quite hot in the backpack, and while it never overheated to the point of shutting itself down, it did cause some discomfort in the hot weather.
  • The laptop was still quite big and heavy.
  • The software provided no 'turn off' function, so the laptop had to be manually shut down after use.
  • Interacting with people while wearing headphones isn't socially acceptable, so they frequently have to be removed. This has the unfortunate effect of returning the user to an 'enable/disable' relationship with the system.
It is seen that many of the problems experienced during the test are solvable, with many being addressed through the use of newer hardware. For example, wires causing induced entanglement can be replaced by bluetooth, Laptop issues (portability/heat) can be resolved by replacing it with a device such as the OQO - a full-featured portable personal computer the size of a packet of cigarettes.

system in use
Figure 3. System in use.
More thought should go to solving the greater issue of the 'enable/disable' relationship induced by the social factors surrounding headphone use. Preliminary research points to using single-ear bluetooth phone headsets, which are seeing increased social acceptance (especially in corporate markets). While this solution was dismissed in the concept stage as BT headsets are unsuited to music playback, their monaural output may be enough for other applications less dependant on hi-fidelity stereo streams of audio. The inclusion of a microphone also raises the possibility of implementing voice control, thereby dispensing with the requirement for a handheld controller. However, until the OQO/BT headset becomes available (and until we can afford it), we are partially offsetting the headphone social problem by using earbuds - they are small and inobtrusive, and can be worn in one ear in most situations without offending.

We are inspired by the lessons learned and the potentials presented by the A2 System; it has shown that cheap wearable computing is a reality. With a basic hardware and software framework in place, we can now start to focus on creating applications to best take advantage of the possibilities on offer. System A3 is already in development, focusing on location-based services.
Source Code
The source code for this project is not available for download at this time.

Base System:Dell Inspiron 4100 Laptop
Interface:Sony-Ericsson T39m Mobile Phone
Software Platform:Windows XP, Python

Key Features:
  • Non-specific input hardware.
  • Range-based user detection.
  • Usable while not being worn.

People Involved
Glen Murphy

back to systems

printable version