![[Rainbow]](./images/rainbow_hline.gif)
MANRAY LABS
8th Annual X Technical Conference
Boston, MA
January 24-26, 1994
Extending X For Recording
Martha Zimet-
Abstract
Over the last few years, several attempts have been made to stan-
dardize record and playback, or journaling, in X. Although input
synthesis, or playback, is currently implemented in the XTest
extension, no standard currently exists for recording, and server
vendors are required by their customers to support a wide variety
of recording extensions and methodologies. In addition, none of
the current recording solutions adequately addresses the issue of
synchronization: some impose a particular methodology, others
ignore the issue altogether.
This paper describes the proposed Record R6 extension that is
being developed by members of the xtest working group. Record
provides support for the recording of all core X protocol and
arbitrary X extension protocol and provides support for a broad
range of synchronization methodologies. The paper presents a
brief historical context of journaling in X, the goals of the
Record extension, and explores recording alternatives and issues
as well as Record implementation details and decisions.
Introduction
Diverse approaches that support record and playback in the X Win-
dow System [Scheifler92] environment have been implemented by
various vendors in recent years. These mechanisms range from
``wire tools,'' that sit between the server and a set of clients,
to X extensions. The XTest extension defines a standard for
input synthesis, or playback, however no standard for recording
has yet emerged. As the currently available mechanisms are lim-
ited in the range of X protocol that can be recorded and provide
incomplete solutions to the issue of synchronization, a new
recording extension, ``Record,'' is being developed that provides
support for the recording of all core X protocol and arbitrary X
extension protocol. Furthermore, Record addresses the issue of
-----------
- Martha Zimet is the software quality architect
at Network Computing Devices, Inc.
Page 1
synchronization without requiring previous knowledge of the
application being recorded.
Historical Context
The idea of journaling is not new and this technology is used in
applications such as automated software testing, software product
demonstrations, computer-based training and software that
addresses the needs of users with movement and sensory impair-
ments. This section provides a brief survey of the currently
available approaches and how they compare to Record.
The ability to allow a client to generate user input actions in
the server is provided by the XTestExtension1 [Woestman91], which
was used in the test suite developed by the X Testing Consortium.
While this extension is expansive in its journaling capabilities
of user input actions, allowing user input actions to be diverted
to a client or copied to a client, and providing playback of
these actions, it does not provide any functionality for the
recording of the consequences of device events, which include
protocol requests, replies and non-device events.
A related approach was taken by XTrap [Annicchiarico91] which
also provides the capability to record and playback user actions.
XTrap additionally allows the monitoring of graphics output from
the X server to the user, which can provide the basis for syn-
chronizing on certain objects. A trapping client could express
interest in certain output requests, such as ImageText8 or Poly-
Text8, and parse the request data to extract the text strings
that would be displayed on the screen. This textual output pro-
vides control points during playback. However, as XTrap does not
support the recording of all core X protocol or X extension pro-
tocol, synchronization control points are limited to objects that
can be parsed from core requests and device events. In addition,
XTrap does not allow the ability to record just one client or a
subset of clients and is limited in its utilization of X Input
devices.
Input synthesis, or playback, has been recently standardized by
the X Consortium in the XTest [Drake91] extension. Both XTrap
and XTestExtension1 duplicate, in part, the functionality of
XTest. However, XTest does not support recording, nor was it
ever intended to do so. XTest was designed to support the X Test
Suite by providing access to write-only server operations and
opaque client-side data structures, and to simulate the user when
verifying functionality such as server grabs and keycode mapping.
Wire, or byte-stream, recording tools have been proven to be
incomplete solutions since they only capture a subset of the
user's actions; events that clients have not expressed interest
in are excluded from the recorded session. In addition, since
these tools sit between the server and clients, they are intru-
sive and can significantly degrade performance.
Page 2
Current Work
Discussions regarding the importance of a recording standard
began last year. Originally, it was envisioned that XTrap would
provide the basis for the new extension, with the functionality
that duplicates XTest simply removed. However, as the require-
ments became clearer, the design of a new extension limited to
recording was begun by the test working group of the X Consor-
tium.
Although the requirement to provide synchronization of contextual
information comes from the playback mechanism, contextual infor-
mation must be captured during recording. There must be a way to
interleave device or user events with their consequences. With
this information captured during the recorded session, human
input can be simulated in a predictable and reproducible manner.
In the scenario when the user moves the mouse and presses and
releases a mouse button in an iconified window to deiconify the
window, the device or user events are MotionNotify, ButtonPress
and ButtonRelease. Given that X follows an event-driven model,
there are consequences resulting from the user actions, which are
in the form of X protocol. From the previous user actions,
requests such as ImageText8 and PolyLine could be sent from the
client, and non-device events such as Expose and MapNotify could
be sent to the client.
Since it is impossible to predict in advance what synchronization
information is required by a particular application, the Record
takes a ``broad brush'' approach, making no assumptions about the
intended use of the recorded data. Facilities exist to both
record and filter core X protocol and arbitrary X extension pro-
tocol. As such, Record does not enforce a specific synchroniza-
tion methodology (eg. window mapping/unmappping, cursor changes,
text strings, etc.).
Objectives
The primary goals of the Record extension are:
o To implement an X Consortium standard for recording and
synchronization, whereby both device events and contex-
tual synchronization information in the form of device
event consequences are recorded.
o To record contextual information used in synchronized
playback without prior knowledge of the application which
is being recorded.
o To provide the ability to work with an arbitrary set of X
extension protocol.
The secondary goals of the Record extension are:
Page 3
o To be independent of other clients and extensions for its
operation.
o To have no significant impact on performance.
o To support multiple client connections and per-client
filtering.
Design
The mechanism used by Record is to intercept all core X protocol
and arbitrary X extension protocol entirely within the X server
itself. The extension provides a mechanism for capturing all
events, including input device events that go to no clients.
This mechanism is analogous to a client ``expressing interest''
in all events, in all windows, including the root window.
There are two event filtering models which can be used in record-
ing:
1. User input actions are copied -- one copy sent to the
server for normal processing and one copy going to a
client.
2. User input actions are diverted to a client before
being processed by the server, the effect being as if
the user had performed no input action.
Only the first is currently supported in Record; the second model
is discussed in the Futures section.
Protocol Interception
Recording clients can specify, or express interest in, the range
of protocol values that are intercepted by the extension. This
range can be specified for only one client or a set of clients,
including all currently existing and future clients. When the
extension has been to intercept specific protocol by one or more
clients, the protocol data is formatted and returned to the
recording clients. Returning data to clients via events is a
common scheme among X extensions. However, given the event size
limit of 32 bytes, two other techniques were evaluated also.
In the first technique, the server would store intercepted proto-
col data in an internal buffer. As new data is intercepted on
behalf of the recording client, it is appended to the buffer.
When the buffer changes from empty to non-empty, the server sends
a notification event to the recording client. Similar to how
GetProperty currently works in the server, the recording client
requests to read data from the buffer, specifying the number of
bytes it wishes to read. The data is removed from the buffer and
sent in a reply back to the client, along with the number of
Page 4
bytes remaining in the buffer.
In the second technique, the paradigm used is for a recording
client to open two connections to the server -- a control connec-
tion and a data connection. The use of two connections is dis-
cussed in detail in the section Recording Clients. In addition,
the notion of a request having multiple replies, exemplified by
ListFontsWithInfo is utilized. It was decided to to use the sec-
ond technique since the size of internal buffers, one for each
recording client, could become quite large, and potentially, data
would be have to be discarded when the buffer grew too large.
Protocol data intercepted on behalf of the recording client and
sent to the recording client may, therefore, be split across sev-
eral reply packets. These packets provide information to enable
clients to unpackage the data, including the ``direction'' of the
protocol data:
+-----------------------------------------------------+
|Direction Type |
+-----------------------------------------------------+
|client -> server xRequest |
|client <- server xReply xError xEvent |
+-----------------------------------------------------+
Additionally, since there is no way for arbitrary X extensions to
register byte-swapping routines with the Record extension, two
techniques were considered. In the first, all intercepted proto-
col data would be returned in the byte-order of the recorded
client. Therefore, recording clients would be responsible for
all byte-swapping, if required. In the second, intercepted pro-
tocol data would be returned in the byte-order of the recording
client for core X protocol only. Recording clients would then be
responsible for byte-swapping X extension protocol only, if
required. It was decided that all intercepted protocol data
would be returned in the byte-order of the recorded client, in
order to be consistent among core and extension protocol.
Therefore, reply packets will specify whether or not the byte-
order of the client being recorded is swapped relative to the
server.
Recording Clients
In Record, a Configuration resource is added by the extension to
the set of resources managed by the server. Although any program
with multiple paths open to the server is viewed as multiple
clients by the X protocol, the recommended communication model
for a recording client is to open two connections to the server
-- one connection for configuration control and one connection
for reading intercepted protocol data.
Page 5
The ``control'' connection is used by the recording client to:
o Obtain information about the supported protocol version
o Create and destroy configurations
o Specify protocol ranges to be intercepted
o Query the current state of a configuration
o Stop interception and reporting of protocol data
The ``data'' connection is used by the recording client to:
o Request the start of interception and reporting for a
particular configuration
o Receive intercepted protocol data
At the Xlib level, the data connection could potentially use the
asynchronous reply mechanism that has been implemented in R6 for
GetAtomNames, InternAtoms, GetWindowAttributes, etc. At a higher
level, since the X Toolkit model supports a single application
context with multiple display connections and multiple alternate
inputs, the control connection could create the application con-
text and then register the file descriptor of the data connection
as an input source with XtAppAddInput. A callback procedure
would then be executed when input is available on the given file
descriptor. From an application viewpoint, no resources would be
allocated over the data connection and no event processing would
take place over this connection.
Protocol Requests
The following protocol requests are provided by the extension:
QueryVersion
This request specifies the Record extension protocol
version the client would like to use. Clients should
use this request before other Record extension
requests. If this request is not the first extension
request executed, the protocol version used by the
extension is implementation dependent. Typically, this
request is executed by a recording client over the con-
trol connection.
CreateConfiguration
This request is executed by a recording client over the
control connection to create a new Configuration object
within the server. The newly created configuration is
given a resource identifier that specifies the
Page 6
recording client is the owner of the resource. The
ranges of protocol values to be intercepted can be
specified when the configuration is created. Configu-
rations can be shared among clients.
FreeConfiguration
This request is executed by a recording client over the
control connection to delete the association between
the resource identifier and the configuration, and then
destroy the configuration. Any data transfer taking
place between the extension and data connections is
discontinued and those connections released.
ChangeConfiguration
This request specifies the configuration parameters to
be modified for one client, or a set of clients, and
associates both the parameters and the client(s) to the
configuration. Typically, this request is used by a
recording client over the control connection to express
interest in specific core X protocol and X extension
protocol to be intercepted by the extension.
Page 7
GetConfiguration
This request is used by a recording client over the
control connection to query the state of a configura-
tion. The reply to this request returns the current
configuration state and characteristics, specific to a
recorded client or a set of recorded clients.
EnableConfiguration
This request specifies whether or not to enable data
transfer between the recording client and the exten-
sion, and returns the protocol data the recording
client has previously expressed interest in. Typi-
cally, the request to enable data transfer comes from
the data connection, while the request to disable data
transfer comes from the control connection. As previ-
ously described, the protocol data reported to the
recording client over the data connection may be split
across several reply packets.
In summary, utilizing the model of two client connections, Record
protocol requests are executed as follows:
+-------------------------------------------------------------+
|Control Connection Data Connection |
+-------------------------------------------------------------+
|QueryVersion |
|CreateConfiguration |
|FreeConfiguration |
|ChangeConfiguration |
|GetConfiguration |
|EnableConfiguration(False) EnableConfiguration(True) |
+-------------------------------------------------------------+
Futures
At the moment, Record sample implementation, protocol version
1.0, is nearing completion. The extension as well as client-
side library and sample clients will be included in R6 Beta. The
protocol specification needs to be updated to revision 1.3 and
distributed once again for technical review. It is hoped that
the extension will be approved as a standard by the X Consortium
in the near future.
One feature not provided in the current Record extension is the
mechanism for diverting device events and in-place, synchronous
substitution or modification. Event filtering currently only
provides a mechanism for feeding device events to recording
clients by copying them. It has not yet been determined if this
feature will be provided in another extension or in a later
Page 8
version of Record.
Returning Intercepted Data
At the time of this paper, it has not been proven that returning
intercepted protocol data via multiple replies works sufficiently
well under all circumstances. If it does prove inadequate, a
future version of Record could implement the model where inter-
cepted data is stored in an internal buffer until a recording
client requests the data. Additionally, functionality in the
Synchronization extension [Glauert93] could be utilized to keep
track of the state and size of the internal buffer and to send
recording clients notification when a condition in the buffer was
reached.
References
[Annicchiarico91] Dick Annicchiarico, Robert Chesler, Alan Jamison.
XTrap Architecture. Digital Equipment Corporation, July
1991.
[Drake91] Kieron Drake. Some Proposals for a Minimal X11 Testing
Extension. UniSoft Ltd., April 1991.
[Glauert93] Tim Glauert, Dave Carver, James Gettys.
X Synchronization Extension, Protocol Version 2.0, June
1993.
[Scheifler92] Robert W. Scheifler and James Gettys. X Window System.
Digital Press, 1992.
[Woestman91] Larry Woestman. X11 Input Synthesis Extension Proposal.
Hewlett Packard, November 1991.
Acknowledgements
Thanks to Jim Fulton, Kieron Drake, Robert Chesler, Marc Evans
and Ken Miller for their valuable input and review of this work.
Thanks especially to Bob Scheifler for asking the hard questions.
Thanks to Bill Crane and Susan Maxwell at NCD for their support.
Author Information
Martha Zimet is the software quality architect at Network Comput-
ing Devices, Inc. She currently leads the X Consortium efforts
with regards to journaling and testing. She can be reached at
zimet@ncd.com.
Page 9