Towards an artificial phonological loop : An assistive device for working memory and attentional control

We describe the initial development of an artificial phonological loop (APL), a new technology to assist individuals with impairment of the working memory system. The phonological loop, along with the visuospatial sketchpad, is one of the two slave short-term memory subsystems that comprise working memory, a cognitive function closely associated with the control of attention. In the phonological loop, phonological (speech) information lasting for 1–2 second is maintained active by repetitive, subvocal (silent speech) rehearsal. Deficits in working memory, specifically in the phonological loop, occur in many disorders, including attention-deficit disorder and Alzheimer’s disease. In these disorders, it appears that the ability for phonological rehearsal is intact, but the regulation or triggering of the rehearsal process is inadequate, thus causing the contents of working memory to be lost. The purpose, then, of the APL is to facilitate the phonological loop by artificially extending the duration of phonological rehearsals. The APL mimics the natural phonological loop by providing audible vocal echoes to take the place of subvocal rehearsals. In this system, the user talks to him/herself in short (1–2 second) phrases; the device records these phrases, stores them in electronic memory, and then repeats— i.e., echoes—the phrases multiple times over an extended period. Two versions of this device have been developed: the Echo-APL and the Rearticulation-APL. In the Echo-APL, only echoing is involved. In the Rearticulation-APL, however, the user re-vocalizes (rearticulates) the phrase in response to an audible cue. The device repeats the cue until it detects (hears) the re-vocalization. Future research and development of the APL will require extensive testing and careful evaluation of possible echo-schedules: the predefined program controlling inter-echo time intervals and echo-amplitude (echo loudness). The APL essentially exteriorizes the silent phonological loop and makes it audible. It is a system that helps the user to “talk to him/herself” to keep ideas in mind.


INTRODUCTION
In this paper we describe the initial development of an artificial phonological loop (APL), a new technology to assist individuals with impairment of the working memory system.Deficits in working memory may be associated with developmental disabilities, head injury, neurodegenerative diseases, brain tumors, strokes, and the so-called dementias of the elderly, which include Alzheimer's disease.The proposed technology is a device which supports and/or substitutes for the "internal dialog" of the subvocal (silent) verbal rehearsal system which is active in working memory.The APL essentially exteriorizes the silent phonological loop and makes it audible.It is a system that helps the user to "talk to him/herself" to keep ideas in mind.
There are many conditions associated with impairments of working memory, and such impairments can diminish quality of life, interfere with employment and activities of daily living, and lead to loss of independent living.At the same time, there is very little available in the way of treatment, rehabilitation, or assistive technology for deficits in working memory.Technology to improve working memory function represents a new direction in the general field of "prosthetic memory", which has previously focused on D. Bogen prospective and declarative memory functions: issues such as storage/retrieval capacity, rather than attentional and regulatory aspects of working memory.
In the following, we will provide some background on related technology, working memory, the concept of the phonological loop, the role of vocalization in the development of the working memory, and then the basic concept of the APL.Next, we will describe the design and construction of working APL devices.Finally, we will describe some early indications about the use of the APL, as well as challenges for future research and development.

BACKGROUND
This work focuses on only one specific aspect of memory: working memory.Although other investigators have developed technology to assist memory, previous work has been directed at enhancing other kinds of memory function.In this section, we will first provide some context for the present work, briefly summarizing prior efforts to develop assistive technology to aid memory.Next, we will explain the rationale for the APL by reviewing the concept of working memory, highlighting insights derived from the literature on self-directed speech, and then briefly discussing some of the disorders associated with impairment of working memory.

Context: assistive technology for memory
Previously developed memory aids, both experimental and commercial, fall into three basic categories: aids for declarative memory, aids for prospective memory, and aids for episodic memory.Aids for declarative memory compensate for difficulties in storing and/or retrieving information from long-term memory storage by supplying an electronic storage medium.Examples include commercial electronic pocket organizers, "Palm"-type personal digital assistants (PDAs), as well as experimental systems which strive to provide context-appropriate information (Lamming and Flynn 1994;Lamming et al. 1994;Rhodes 1997;DeVaul et al. 2000).Aids for prospective memory facilitate "remembering to do something", or daily scheduling.Examples include "personal electronics" with alarm functions, electronic pocket organizers, and PDAs.Such devices have been used in cognitive rehabilitation (see Hersh and Treadgold 1994;Kapur 1995;Goldstein et al. 1998 for reviews).However, mass-market electronic pocket reminders have often been judged as "too complex", in terms of use, programming, and data entry (Hersh and Treadgold 1994;Goldstein et al. 1998).Thus, several systems have been developed to make the "user-interface" simpler.One of these systems is the NeuroPage system (Hersh and Treadgold 1994;Wilson et al. 1997), in which the reminders are delivered to the patient by a radio-frequency paging system.Aids for episodic memory construct a record of personal experience.Examples include automated electronic diary systems (Eldridge et al. 1992).
Research into memory-enhancing devices remains an active area, with on-going efforts to develop small, wearable, memory augmentation systems at a number of academic and industrial centers (Lamming and Flynn 1994;Lamming et al. 1994;Rhodes 1997;DeVaul et al. 2000); there is also an interest in using small, commercially available audio "memo-recorders" as assistive devices, particularly with patients with brain injury (LoPresti and Willkomm 1997;Willkomm and LoPresti 1997;Van Den Broek et al. 2000).However, both the research and clinical communities are mostly directed towards assisting longterm storage/retrieval and/or problems with prospective memory, rather than developing technology specifically directed towards working memory.

Working memory
Working memory, a concept developed and refined over the last 25 years by Baddeley and coworkers, refers to ". . .a system for the temporary holding and manipulation of information during the performance of a range of cognitive tasks such as comprehension, learning and reasoning."(Baddeley 1986).Beyond this general concept of working memory, Baddeley has also advanced a more specific model of working memory, which, while consistent with many research findings, is still incomplete and under discussion in the research community.
As envisioned by Baddeley, working memory consists of two kinds of short-term memory store, plus mechanisms for: (1) maintaining and extending the memory stores; and (2) supervising the manipulation and transfer of information that is being temporarily held.In developing a model of working memory, Baddeley sought to explain: (1) how subjects could simultaneously hold small amounts of information in short-term storage (i.e., digit span), while at the same time performing cognitive tasks; and (2) how the apparent interference between short-term storage and concurrent tasks depended on exactly what was being stored and on the nature of the concurrent task.The model that Baddeley eventually devised was one with three components: a central executive that supervises two slave systems-the phonological loop and the visuospatial sketchpad.The phonological loop is the system that permits short-term storage of speech-related information; and the visuospatial sketchpad permits short-term storage of visual and spatial information.The phonological loop, in turn, is considered as a system with two subcomponents: the phonological storage unit itself, which can store speech-based information for 1-2 seconds; and an articulatory control process.The latter Baddeley likens to "inner speech" and is able to maintain the phonological store by rehearsal-by repeating the contents of the store subvocally.This is illustrated in Figure 1.
The central executive is seen as a supervisory entity that coordinates the operation of the two slave systems.Working memory is seen as a system with limited capacity; so the central executive is that entity which controls what information is to be "held" and what information is to be "dropped" over the short time span.There is a close association between the central executive and the operation of selective attention: in fact, Baddeley refers to the central executive as an attention-controlling system.

Self-directed, private, speech
The phonological loop is seen as a system in which inner speech occurs subvocally-i.e., silently.The literature on self-directed, or private, speech deals with a related concern-audible speech that is not directed outward to other persons.Many investigators now understand selfdirected speech as a self-regulatory process which is active during the development of the child and which later becomes internalized: that is, vocal speech becomes subvocal (Bivens and Berk 1990).Research has also shown that individuals with cognitive disabilities employ private speech as a strategy for self-regulation.Berk (Berk and Potts 1991;Berk and Landau 1993), for instance, observed that children with attention-deficit hyperactivity disorder (ADHD) or learning disabilities used more self-regulating private speech than normal children and further suggested that such children needed access to environments that would allow them to use this strategy.

Disorders of working memory
There is considerable evidence that many "memory problems" are not memory storage problems per se, but problems in regulating memory through attentional control.Thus, these problems are best understood in terms of deficits in working memory.In fact, deficits in working memory have been described in three major diseases or populations: traumatic brain injury, Alzheimer's disease, and attentiondeficit disorder (ADD).
Individuals with deficits in working memory may demonstrate normal performance on tests of short-term memory capacity, but nevertheless complain of memory difficulties and have difficulty with activities of daily living.This is true for some Alzheimer's patients, who appear to have a deficit in central executive function, with difficulty in coordinating information from slave systems (Morris and Baddeley 1988;Baddeley et al. 1991Baddeley et al. , 1999)).Thus, Alzheimer's patients do especially poorly on complex tasks in which the central executive would need to regulate and supervise competing demands on a working memory with limited capacity.
Another large population with impairment in working memory is persons with ADD.Barkley (1998), referring to working memory, states: "Such remembering is crucial to timeliness and goal-directed behavior; it provides the means for hindsight, forethought, preparation and the ability to imitate the complex, novel behavior of othersall of which are impaired in people with ADHD."Barkley further goes on to propose that ADHD is a disorder resulting from delayed maturation of executive function (Barkley 1997(Barkley , 1998)).

Goal
Reduced to its essentials, an operative phonological loop requires a mechanism for phonological storage, a mechanism for rehearsing this phonological information, and a mechanism for repetitively triggering the rehearsal.There are three distinct capabilities; and the previous discussion of Disorders of Working Memory suggests that the repetitive triggering of rehearsals is the "weak link" of the system.Because the working memory system serves as an attentionaldirector or task-director, then failure to rehearse would lead to premature loss of stored phonological information and consequent loss of attentional control: that is, an individual would go "off-task".
Thus, the goal of the APL is to address the weak link of working memory: to facilitate phonological rehearsals in situations where this natural process regularly fails to occur.At the present time, there seems to be no way to directly interface with, or access, inner speech and the neurological articulatory control system.Nevertheless, it is possible to interface with speech itself, and hence, selfdirected speech.Although, in normal development, the subvocal phonological loop appears to replace the immature mechanism of self-directed speech, there is no reason to believe that the capability for a vocal phonological loop function is lost.It should therefore remain as a vestigial, if unused, function.
Our working hypothesis is that facilitating the vestigial vocal loop will lead to improved working memory function.Thus, the APL is understood as a device that helps a person to talk to him/herself.In the following section, we will introduce two different approaches for APL devices: the Echo-APL and the Rearticulation-APL.For each approach, we will describe the basic concept (conceptual design) and discuss methods of operation and requirements for the devices.We will also give an example how the device would be used and describe how the APL is expected to interact with the natural phonological loop.In the later Design of APL Devices section, we will then describe the detailed design and construction of working APL units.

Concept
The Echo-APL is one approach taken towards the goal of facilitating working memory via vocal rehearsals.The basic concept is a device, modeled after the natural phonological loop, in which audible, vocal echoes take the place of subvocal rehearsals.In this system, the user talks to him/herself in short (1-2 second) phrases (the same duration of the natural phonological store); the device records these phrases, stores them in electronic memory, and then repeatedly plays back-i.e., echoes-the phrases over an extended period.A cartoon showing the basic concept of this APL is shown in Figure 2.

Requirements
Requirements for an Echo-APL device can be broken down into three categories: functional requirements, userinterface requirements, and form requirements.
Functional requirements.The basic functional requirements for the device are that it must include: a voice sensor (a microphone); a voice recorder (digital storage module); a voice producer (a speaker or earphone); and an echo-scheduler (electronic timing and control system).The first three components already exist within many commercial electronic products such as memo-recorders and telephone answering machines; but the echo-scheduler is a new concept.The echo-scheduler constitutes the artificial central executive/articulatory control, and regulates the device via an echo-schedule: a program for delivering the echoes.An example of an echo schedule is "echo every two minutes for ten minutes."To be useful, however, the echo-schedule must satisfy the conflicting needs for both simplicity and flexibility.An inflexible schedule, like "echo every two minutes for ten minutes", might not be acceptable because there may be situations in which the memory phrase is only needed for 3 minutes, or other situations where the memory phrase needs to be maintained for 15 minutes.On the other hand, if there is no underlying schedule of echoes (i.e., the device only plays the memory phrase on demand, like a memo-recorder), then the device will not useful for its intended purpose.This is because our target population with working memory dysfunction lacks exactly the kind of attentional control that is required for completely volitional (in contrast to automatic) playback of the stored phrase.A memo-recorder can only be used as a working-memory aid when the user has good central executive function; but central executive function requires working memory.Thus, it would seem that the echo-scheduler should be based on a pre-defined, automatic schedule, but that the user should still have some control over the echo process, specifically to: Record a spoken phrase into the APL; Stop the echoes when the phrase is no longer needed; and Play the phrase on demand, to allow the user to access the contents of the artificial loop at any time.
User-interface.By its very nature, the APL requires audio (specifically, voice) input and output.But the previous discussion indicates that the user must also be able to exert three actions on the APL: Record, Stop, and Play.This poses a problem, in that a general requirement for the user-interface is that it be exceedingly simple, and that it makes little physical or mental demand (because the APL is designed for people who are easily distracted, or taken off-task).Ideally, these user-actions should not require any (or much) shifting of eye gaze or interruption of manual activities.Thus, although the three user actions could be accomplished by three pushbuttons, some simplification seems highly desirable-to reduce the number of switches or to replace them with some other input modality.Many possibilities come to mind, including an entirely voice-operated APL, to accommodate a wide range of user abilities and preferences.
Form.The physical form of the APL certainly must incorporate audio input and output, as well as the three control functions.The form, therefore, will be largely influenced by whether audio output is speaker or earphone, and by the control interface: e.g., by whether it has three pushbuttons or one.In general, though, to be practical, the APL device needs to be small and lightweight enough to be easily carried around.Preferably, the APL would be pocket-sized; and, still better, it would be wearable.
Summary.The requirements for the APL are that it be a small, possibly wearable, device with voice recording and playback capability, along with a simple three-function user-interface requiring very little physical or mental effort.The APL should be programmed so that it can capture a single 1-2 second spoken phrase, and then echo that phrase back following a standard schedule.

Example
It will be helpful here to give a realistic example of howand when-such a device would be used.Take as an example a child with significant ADD who is asked to put on her socks and shoes before leaving the house.This situation is typical of the tasks that are problematic for children with ADD.Indeed, it is challenging for many young children.This is because it is an extended task with many subtasksfinding the shoes, finding the socks, finding a place to sit down, manipulating the socks, manipulating the shoes, tying or fastening the shoes.This ordinary activity of daily living is actually quite complex, involving sequences of subtasks, planning, decision-making, physical dexterity, and problem-solving.During the performance of this task, there are many opportunities for distraction.Thus, if the child is sent upstairs to put on socks and shoes, it might not be completely surprising to find her 10 minutes later sitting on the floor of her room, playing with a toy or reading a book.
The same child using the Echo-APL device would look like this.After being instructed to put on her socks and shoes, she would record the short phrase, "put on socks and shoes" (or just, "socks and shoes") into the Echo-APL and then start towards her room.Let's imagine that this child has great difficulty in staying on task; and, therefore, the Echo-APL has a standard echo-schedule set at "echo every two minutes for 20 minutes", 20 minutes being the very longest activity that the child might be asked to undertake.
After 2 minutes the child might be upstairs in her room.She may have started to look for socks in her drawer, but is now looking at bathing suits.The 2-minute echo will then cue her to look for socks rather than bathing suits.The successive echoes will repeatedly push her back on track in the same way that a parent (with enough time, energy, and diligence) will monitor and supervise children's chores and activities.Finally, after 11 minutes, the child has her socks and shoes on.She then presses the Stop button and the echoing ceases.

Integration with natural phonological loop
The Echo-APL is designed to interact with and support the operation of the natural phonological loop.The Echo-APL operates in parallel with the natural subvocal and vocal loops, as shown in the conceptual diagram in Figure 3, and interacts with the natural system.The nature of this interaction is perhaps best explained by making an analogy between the natural phonological loop and dynamic random access memory (dynamic RAM) used in computer electronics.Dynamic RAM holds information as stored charge in small capacitors.This charge will leak out (i.e., the information will decay) unless the dynamic RAM is periodically and frequently refreshed: that is, the capacitors (memory) must be continually recharged.The phonological store is similar.Its contents will also dissipate unless refreshed by continual rehearsal.The APL, however, is similar to static random access memory (static RAM), also used in computer electronics.Static RAM holds the same kind of information as dynamic RAM, but does not require refreshing.
When the APL is in use, the dynamic phonological RAM (i.e., natural phonological store) is refreshed by both the natural phonological rehearsals and the artificial rehearsals.In essence, the refresh rate is increased by the APL.However, when the natural system fails to supply the refresh signals (i.e., rehearsals), the artificial system will continue to supply the refresh signals.Thus, the APL effectively operates as a redundant or backup system.The APL is capable of redundant operation because it incorporates the same three basic functions as the natural system: i.e., mechanisms for phonological storage, rehearsal, and repetitive triggering.

Concept
One possible limitation of the Echo-APL concept is that, while it does involve rehearsal of phonological information, it does not involve articulatory rehearsal.That is, the user does not actually have to rearticulate the phrase.Perhaps, rehearing of the phrase is enough to facilitate working memory: whether the natural phonological loop requires rearticulation, as opposed to rehearing only, is a matter of controversy.Gupta and MacWhinney (1993) have concluded that rehearing is paramount, and that rearticulation is not necessary for the operation of the loop.However, because this issue has not been completely resolved, we consider the possibility that rearticulation has advantages over rehearing: that a more active process has benefits over a relatively passive process.
This leads us to a second proposal: the Rearticulation-APL.In the Echo-APL, the device delivers an echo according to a programmed schedule.In the Rearticulation-APL, the device instead delivers an auditory cue according to a programmed schedule.This cue is a signal to the user to repeat the phrase aloud.This cue could be a "beep", or a verbal command (e.g., "say it"), or it could be an echo of the actual phrase.
A cue, by itself, will not necessarily bring about the rearticulation.The user simply could choose to ignore it.Therefore, the device must take some other action in order to ensure that a rearticulation actually does occur.Because the APL contains an "electronic ear", this suggests having the APL listen for the rearticulation, and then respond to its presence or absence.After the device has determined whether the user has repeated the phrase, there are many possible device responses.However, a standard response algorithm for the device is to repeat (echo) the phrase until the user rearticulates it.This is the approach typically used by a parent or teacher who says, "repeat after me . .." and then does not give up until the child actually repeats the phrase.
Thus, if the Rearticulation-APL device were used with the socks-and-shoes task described previously, the programmed rehearsal schedule might be: "cue every two minutes for 20 minutes; at each cue time, listen for the This APL requires the user to repetitively articulate the phrase to be held in working memory.The Rearticulation-APL periodically cues the user to re-vocalize (rearticulate) the stored phrase.This rearticulated phrase is sensed by a voice detector within the APL.Should the user not rearticulate the phrase, the APL will repeat the cue.Like the Echo-APL, the Rearticulation-APL serves as a redundant system to the natural phonological loop.However, in this case there is more active involvement of the user.This may be advantageous in supporting working memory function.
vocal rehearsal and keep on repeating the cue (up to 10 times) until the rehearsal is heard."The same three-action user-interface (Record, Stop, Play) used for the Echo-APL interface can also be applied to the Rearticulation-APL.

Requirements
The requirements for a Rearticulation-APL device are essentially identical to those for an Echo-APL device.The same three user-actions are needed, as well as the same needs for form.The Rearticulation-APL can thus employ the same construction as the Echo-APL, with one important addition: a hardware/software system to detect the rearticulated phrase.An Echo-APL device needs only the ability to record and play back a short phrase; the Rearticulation-APL device also needs the ability to identify a short, spoken phrase.

Example
To return to the socks and shoes example, the child using the Rearticulation-APL would look like the Echo-APL child, except that one would also hear the child actually speaking to herself every 2 minutes.At, say, 10 minutes, one might hear the following dialog, in which the child needs multiple cues before rearticulating: • APL: Socks and shoes.

Integration with natural system
The Rearticulation-APL acts as a redundant system, refreshing the dynamic phonological store, in a fashion similar to that of the Echo-APL.However, the interaction between the artificial and natural systems may be somewhat different for the two types of APL systems.In all cases, the artificial and natural systems interface through the artificial ear-to-natural voice and artificial voice-to-natural ear connections.However, with the Echo-APL there is no direct indication that the natural central executive is involved with this interaction.However, for the Rearticulation-APL, the user must be aware of the echo phrase because the device requires him/her to vocalize it.Thus, the conceptual diagram in Figure 4 shows a direct connection between the artificial voice and the natural central executive.

DESIGN OF APL DEVICES
The previous discussion of the Echo-APL and Rearticulation-APL described the conceptual design and general operation of these devices, but did not indicate specifically how they could be implemented.We now move on to the detailed design and construction of working APL devices.The first step in this process is to select the actual features and performance of the devices to be achieved in this design: i.e., to set the specifications for this implementation of the APL concept.

Design decisions
The design of an actual APL device requires the following decisions, or selections, to be made: 1. Selection of form (e.g., headphone style device vs. wristwatch style device).2. Selection of user-interface (e.g., three-button interface vs. voice-controlled interface).3. Selection of the electronic hardware (e.g., selection of particular voice recording modules, microcontrollers, etc.).4. Selection of the echo/rearticulation schedules (e.g., fixed inter-echo intervals, vs. progressively lengthening interecho intervals).
The basis for these decisions was the immediate goal at this stage: to produce a working APL device that could be used as an exploratory "test-bed".This test-bed should allow us to evaluate the concept of the APL and to explore different kinds of echo and rearticulation schedules.The purpose of this version of the APL is to generate experience with the APL prior to clinical testing or commercial product development.Therefore, the following decisions were made.

Form
The first decision was to construct a hand-held version of the device, using speaker output.The unit would thus resemble a memo-recorder or calculator in size and form.This decision was made because it afforded the simplest construction and would seem to be acceptable to the widest range of users.Although a much smaller and more aesthetic package is readily achievable, a hand-held size (like a calculator) was considered to be "good enough" for the first version: easily transported, carried in a briefcase or handbag, or clipped onto a belt.The focus here was realizing the basic functionality of the APL.

User-interface
A second important decision was to develop the userinterface using two pushbuttons.A two-button interface was judged to be operable "by feel", in the same way that a keychain-remote for an automobile door-lock does not require the user to actually look at the buttons.The left and right buttons are found easily and virtually automatically by the user.Although hands-free interfaces are possible, including those based on voice-control, it was judged that hands-free operation for the APL was a design refinement that was not necessary for the initial evaluation of the APL.On the other hand, the ability to operate the device without looking at it was thought to be a crucial requirement.The decision to limit the user-interface to two buttons was made early in the design, thereby constraining the range of user-control to what could easily be achieved with this minimal interface.

Hardware
The decision about hardware components was heavily influenced by prior experience.Electronic design was based on a Microchip PIC-series microcontroller and an ISD (Interactive System Design, Winbond, Inc.) ChipCorder  series voice record/playback module.Microcontrollers and voice modules from other manufacturers could have been used in this design, but the developing laboratory had previous experience and device-programmers for the PIC and ISD chip series.A decision was also made concerning the degree of hardware refinement to undertake.Because the goal was not a market-ready product, minimizing parts count, minimizing cost, minimizing device size, and maximizing battery life-though not ignored-were given lower priority than functionality.

Echo and rearticulation schedules
Because our initial goal was to gain experience with the APL, for the purpose of identifying those aspects which are worthy of further development and exploration, we decided to consider a wide range of echo and rearticulation schedules, not dismissing too many a priori.Accordingly, a variety of echo/rearticulation schedules were programmed.In particular, the programs allow investigation of such parameters as the inter-echo interval and echo volume.

Rearticulation cue
Because we had no prior information about whether rearticulation would be best cued by a "beep", a voice command, or the echoed phrase itself, the Rearticulation-APL was designed to allow all three types of rearticulation cues.

Rearticulation detection
The development of a hardware-software system to determine whether a rehearsed vocal phrase "matches" an initial spoken phrase is potentially challenging, with the possibility for great complexity.However, at this point we are only interested in proof-of-concept, and therefore have chosen to implement the very simplest rearticulation detection method: to examine the amplitude of the audio input signal (microphone) and determine whether it is in the same general range as the amplitude of the original spoken phrase.This is far from an optimal approach, but it is sufficient to explore the concept of the Rearticulation-APL.

Hardware design
Based upon the design decisions described above, two APL prototype devices were constructed.The second version is described below.The first version was virtually identical in design, except that it had a slightly different chip set (PIC 16F628 instead of 16F819, ISD1016 instead of ISD1416).This earlier device was capable of only Echo-APL function, while the second device was capable of both Echo-APL and Rehearsal-APL function.The basic hardware organization of the APL is shown in Figure 5.The device consists of three major subsystems: (1) an audio subsystem; (2) a controller subsystem; and (3) a user-interface.
The audio subsystem is comprised of an electret microphone, a microphone preamp, a digital voice record/ playback module, an audio amplifier, a speaker, a digitally controlled volume control, and an analog-to-digital converter (for voice detection).The controller subsystem is comprised of a microcontroller, and a real-time clock.Finally, the user-interface consists of two pushbutton switches.
The ISD1416 voice recorder chip (Winbond Electronics Corporation of America, San Jose, CA) was chosen as the digital voice record/playback module.This particular device is capable of keeping up to 16 seconds of voice data in nonvolatile storage and incorporates a control structure that allows multiple voice records to be acquired, stored, and played back under digital control.The voice module also includes an accessible preamp stage, so a separate preamp was not required.An LM4864 (National Semiconductor, Santa Clara, CA) was used as the audio amplifier.The volume control was realized using a DS1267 digital pot (Maxim Integrated Products, Inc., Sunnyvale, CA) as the gain-setting resistor of the LM4864 amplifier.
The controller subsystem efficiently employed a PIC 16F819 microcontroller.The real-time clock was realized using the Timer2 submodule of the 16F819 in conjunction with an external 32 kHz crystal and an interrupt-driven software routine.The 16F819 also contains the analog-todigital converter needed in the audio subsystem.
Standard electronic construction techniques using through-hole components were used on a two-layer 7.5 cm × 10 cm printed circuit board.The microphone, speaker, and switches were all mounted directly onto the board.The board was installed into a plastic case, measuring 3 cm × 8 cm × 15 cm, with cut-outs for the speaker, microphone, and switches.The case also included a battery compartment for two AA batteries.

User-interface
The user-interface for the Echo-APL consists of the speaker, microphone, and two pushbuttons.The left button is labeled "Record" and the right button is labeled "Play".The buttons can be located by feel and can be manipulated by one or two hands.When not in use, the device is in "Standby" mode.Pressing the Record button allows the working-memory phrase to be recorded and initiates the echoing.Pressing the Record button at any time will cause the device to record a new working-memory phrase.If the Record button is pressed, but the user does not speak into the device, then the device returns to Standby mode, terminating any echo schedule in progress.Thus, the "Stop" function is achieved by using the Record button: (Record + silence = Stop).The Play button is used to play back the phrase at any time.Although this Play function seems desirable, it is not clear that it is absolutely essential.Thus, a single pushbutton might be adequate for the user-interface.
The user-interface for the Rearticulation-APL is essentially the same as that for the Echo-APL.The Record and Play buttons have the same effects in both devices.However, in the Rearticulation-APL, when it is time for the D. Bogen user to re-vocalize the phrase, it is not necessary to press the Record button.

Types of echo/rearticulation schedules
A variety of echo programs-schedules and rules for generating the echoes-were developed for the APL.These were programmed using the PicBasic Pro compiler, version 2.43 (microEngineering Labs, Inc., Colorado Springs, CO), and programmed into the PIC 16F819 using an EPIC programmer (microEngineering Labs, Inc., Colorado Springs, CO).Controlled parameters included-the number of echoes produced, the interval between echoes, the time between the initial recording and the first echo, and the amplitude (loudness) of the echoes-as illustrated in Figure 6.The rationale for irregularly spaced echoes and varying echo volume is as follows.One might hypothesize that it is more difficult to initiate the rehearsal process required by the phonological loop, than it is to maintain the rehearsal process.Thus, the APL might be more effective if it delivered more echoes, and louder echoes, at the beginning of the schedule.On the other hand, one might hypothesize the reverse: that it is especially difficult to maintain the rehearsals for an extended period of time; and, therefore, more (and louder) echoes should be delivered later during the echo schedule.

Program operation: state-diagram structure
The basic operation of the programs is explained in terms of the state diagram shown in Figure 7.When there is no echo program underway, the program is in STANDBY state.Pressing the "Record" button puts it into the RECORD state, storing the phrase, and then starting the echoing se-quence.While the system is waiting for the next echo to occur, it is in the TIME state, where it times the inter-echo interval by periodically polling the real-time clock until it is time for an echo (ECHO state).This triggers playback of the echo phrase.After the echo finishes playing, the system generally sets up for the next echo and re-enters the TIME state.However, if the end of the echoing period has been reached ("last echo" condition), then echoing is discontinued and the system returns to STANDBY.If the record button is pressed while in the TIME state, then a new phrase will be recorded and a new echo sequence initiated.However, if the record button is pressed, and no phrase is actually spoken (a "null phrase"), the system interprets this is as a "stop" signal: if an echo sequence is underway, it will be terminated; if no sequence is underway, then the null phrase will not be echoed.

DISCUSSION: POTENTIAL LIMITATIONS, EARLY EXPERIENCE, AND FUTURE DIRECTIONS
This paper has presented the overall concept of an APL, the rationale for it, underlying design concepts, and the detailed implementation of actual devices.The construction and programming of the APL has only recently been completed, and its evaluation is just underway.While it is too early to make any certain statements about its effectiveness and applicability, nevertheless, we have already learned much from our early experiences with the device.The following discussion first presents some of the anticipated limitations or difficulties of the APL-challenges that were anticipated at the beginning of the conceptualization and design of the APL.We then discuss some of our early experience with the APL.Finally, we present directions for future research and development.

Hypothetical limitations of the APL
There are several potential, or hypothetical, limitations of the APL.One hypothetical limitation is that it would actually interrupt the user and interfere with task performance.Although this is a valid concern, it is likely to be outweighed by other factors.First, if the user is highly distractible, then many of the echoes will occur when the user is already off-task; in this situation, the echo should have a positive effect-to reorient the user to stay on-task.Second, verbal interruptions are commonly used by teachers, coaches, and parents in providing encouragement and coaching to children.For example, in a school situation, teachers often walk around the classroom where students are engaged in projects (e.g., art, shop, science lab, classwork) and provide encouragement and reinforcement by a few words to each student: e.g., "that's good", "keep going", "don't stop now".It is generally recognized that these short verbal messages help, rather than deter, performance.Third, the argument can be made that a vocalized loop has a greater focusing effect than a subvocal loop.Many adults resort to self-directed speech when trying to achieve great focus: for instance, when self-coaching themselves in athletics to "keep your eye on the ball!"Nevertheless, it is recognized that if the echoes are delivered too frequently (say, every 30 seconds), then the APL could be interfering-or if not interfering, at least annoying.The optimal echo frequency, then, must balance the need to prevent the user from going off-task against the possibility of actually interfering with the task.The optimal balance point will depend upon an individual's capacity for attentional control, the nature of the task, and environmental factors.
Another, perhaps more fundamental, limitation of the APL is that its usefulness could be short-circuited if the user decides to stop every echo stream with the Stop button.But this is the same problem as with all assistive technology.If the potential user is not motivated to use it, then it will be of limited benefit.The APL is likely to be effective only with users who want to stay on task.
Finally, another significant concern is that potential users of the APL would be self-conscious or embarrassed to use it.This is certainly a possibility.There may also be situations in "quiet environments" where vocalization would not be acceptable.Nevertheless, there may be home, school, and office settings where self-directed speech is less of a problem.In some cases, settings could be restructured to make it easier for an individual to use an APL.In an office environment, where people are continually on the telephone, the user of an APL might not call attention to him/herself.This is especially true now that many people are using telephone headsets (earphone with microphone boom).The APL could readily be incorporated into a telephone headset.In this case the APL and its use would be virtually undetectable.The increasing use of such headsets with cell phones further expands the settings in which people can acceptably hold conversations with unseen persons.Cell-phone users now regularly converse in animated fashion to unseen persons while walking down the street, on buses, on trains, and in offices.Thus, the user of a head-mounted APL may easily blend into the population of cell-phone users.In the future, one might possibly combine a cell phone and APL into a single unit.The cell phone already contains most of the hardware necessary to realize an APL.

Early experience
The effectiveness of the APL certainly depends upon the validity of its underlying concept.But the real effectiveness of an APL device also depends upon the choice of echo schedules, the physical construction of the device, and the user-interface.Thus, if the echo-schedules are not optimally chosen for a given user, or if the industrial design aspects of the APL are poorly executed, then the APL may not achieve its intended effect.Consequently, we have begun an initial testing program to learn as much as we can about the use of the APL before clinical testing, with the intention of providing a more refined version for clinical testing.The current field tests, informal in nature at this point, have involved a small number of laboratory associates using the APL in home and work situations.Of course, this experience in no way substitutes for formal testing in a population with a bona fide impairment in working memory or attentional control, such as individuals with ADD.With the above caveat, our first observations follow: 1.The APL appeared to be useful in some circumstances in helping the user to stay on-task, particularly when the user: (a) was involved in tasks that he/she really did not want to do; (b) was attempting to carry out a task while subjected to frequent interruptions; (c) needed to move locations or assemble additional materials in order to carry out the task; (d) needed to carry out a relatively minor task while being concerned with other larger tasks; or (e) needed to carry out a task that required the user to break from well-established habits or expectations.2. The Play button was not used.This suggests that only a single pushbutton would be required for the userinterface, thus making the operation of the APL extremely simple.However, it remains to be seen whether someone with impaired working memory would use the Play button.3. The Echo-APL device was preferred over the Rearticulation-APL device.4. When the APL was programmed with a fixed inter-echo interval, the interval seemed too short at times: i.e., the echoes came too frequently and were annoying.But at other times, the echo frequency seemed fine; and the APL helped to maintain task concentration.This suggests that the user should have some means of adjusting the echo frequency, according to need.Perhaps, the adjustment could be as simple as a selector with several settingslow, medium, and high-to designate occasional echoing, medium echoing for general use, and frequent echoing for situations where the user finds it difficult to stay ontask.5.The APL in its present form, the size of a scientific calculator, is too large.Although it can clip onto a belt and is transportable, it is not conveniently transportable.Thus, miniaturization is a priority for future versions.6.Although the original intent of the APL was to facilitate the operation of working memory, the echoes also seem to have a coaching or motivational effect.Although this was unintended, it is a positive effect and suggests a dual-use for the APL device.7. The biggest problem experienced with the APL was forgetting to use it in the first place.That is, the user would initiate a task without thinking to record the taskdirecting phrase into the APL.This kind of "forgetting to remember" problem would likely be even more significant for users with impaired working memory.This observation therefore suggested modifying the software so that it would periodically remind the user to use the APL.Thus, the software was modified so that the APL "beeps" periodically while in Standby mode.This appears to be an improvement in some circumstances.

Future directions
The greatest priority for future work on the APL is testing and evaluation.Related to this is the development and optimization of echo/rearticulation schedules.The schedules presented here are examples of possible schedules, but it is not clear which of these are more effective and useful, or even whether there might not be other more useful schedules.Future versions of the APL would also benefit from a mechanism that allowed the user some control over echo rate.It seems likely that the user's level of attentional control could vary, depending on many factors including time of day, environment, nature of immediate task, and medication.Thus, there may be times when the user needs frequent rehearsal echoes to stay on-task, and other times where the echoes can be less frequent.Future hardware development is likely to improve the effectiveness of the APL.The effectiveness of the APL undoubtedly will be influenced by the user's motivation; and this in turn will be influenced by the user's perception of the APL as an acceptable product.The present form of the APL lacks the high-quality finish of current consumer products.Therefore, it will be necessary to produce a smaller and more aesthetic package for the APL.This in turn will require electronic miniaturization and greater attention to industrial design and ergonomics.An attractive option is the incorporation of the APL into a headset unit (earphone with boom microphone).
Another area for exploration is the user-interface.The present version of the APL employs two pushbuttons to control three input functions.Although this arrangement can be operated "by feel", it still requires the user to interrupt a manual task in order to operate the APL.Even if the Play button turns out not to be essential (thus reducing the device to a one-button interface), a hands-free userinterface still might be advantageous.Possibilities include the use of gesture-control (for example, head movement) and voice-control.
Finally, it is not only possible to combine the APL with other consumer appliances, such as the cell phone, but it is also possible to combine the APL with other memoryassisting technologies, such as those described earlier in the Background section.

CONCLUSION
The devices described here represent the first steps towards a useful assistive technology to aid working memory and attentional control.Much work remains, and the current version of the APL, while promising, requires continued testing and development.Greater effort towards developing the APL is justified by the prevalence of working memory impairment in many medical disorders, and the profound effect such impairment has on cognitive and behavioral function.

Figure 1
Figure1Baddeley's model of the phonological loop.The loop consists of a short-term phonological store.Maintenance of the store is achieved by silent rehearsal, under articulatory control.The loop itself is further under the control of the central executive.

Figure 3
Figure3Conceptual diagram showing operation of the artificial phonological loop.The APL operates in parallel with the natural phonological loop, with artificial echoes occurring along with natural subvocal and vocal rehearsals.The timing of natural and artificial rehearsals is depicted by a pulse train.The dotted pulse-train (lower portion of the figure) indicates that the triggering of the natural rehearsal process has failed, which would ordinarily lead to loss of information held in the phonological store.However, because the APL continues to generate rehearsals (echoes), the contents of the phonological store can be replenished and maintained.

Figure 4
Figure4Conceptual diagram showing operation of the rearticulation-artificial phonological loop.This APL requires the user to repetitively articulate the phrase to be held in working memory.The Rearticulation-APL periodically cues the user to re-vocalize (rearticulate) the stored phrase.This rearticulated phrase is sensed by a voice detector within the APL.Should the user not rearticulate the phrase, the APL will repeat the cue.Like the Echo-APL, the Rearticulation-APL serves as a redundant system to the natural phonological loop.However, in this case there is more active involvement of the user.This may be advantageous in supporting working memory function.

Figure 6
Figure 6 Example of echo programs.Shown are six diagrammatic examples of echo programs.The black vertical bars represent the instant at which a short 1-2 second phrase is recorded.The vertical gray bars represent the echoes, or playback instances.The horizontal dimension corresponds to time; and, hence, the horizontal spacing between the bars represents the time interval between echoes.The vertical extent of the bars indicates the amplitude (loudness) of the echoes.Shown are: (a) fixedinterval, fixed-amplitude echoes; (b) lengthening-interval echoes; (c) shortening-interval echoes; (d) increasing-amplitude echoes; (e) decreasing-amplitude echoes; and (f) lengthening-interval with decreasing amplitude.

Figure 7
Figure 7 Simplified state diagram for Echo-APL device.Ovals indicate internal states and arrows indicate transitions between states.Each transition is labeled with its associated condition.Bracketed labels indicate external user-interface conditions, such as pressing the Record or Play buttons.