"Image Reading Teaching Conference Support System:
Analysis, Design, Implementation and Evaluation"

Akihiro Toshimitsu, MS
Olivia R. Liu Sheng, PhD
Paul Jen-Hwa Hu, MS

Toshiba Medical Engineering Laboratory
1385 Shimoishigami
Otawara, Tochigi 324, Japan
toshimit@mel.nasu.toshiba.co.jp

Department of Management Information Systems
College of Business and Public Administration
The University of Arizona
Tucson, Arizona 85721
sheng@bpa.arizona.edu
phu@bpa.arizona.edu

Abstract

Of the research and development issues of Picture Archiving and Communication Systems (PACS), investigations on what healthcare-related activities could be better supported with such systems have become an interesting research area that requires more studies. In addition to their generally known capabilities of facilitating individual tasks and activities, PACS also present opportunities to improve cooperative group work, a common form of activity taking place within or across healthcare institutions.

Our research has investigated probable applications of groupware technologies, which have been known for their support of cooperative activities in business and government domains, in medical education. This study, in particular, focuses on routine face-to-face teaching conferences, one of the most important resident-training activities through which radiologists pass valuable image reading skills accumulated from years of practices over to residents.

Paper

Among research and development issues related to Picture Archiving and Communication Systems (PACS), investigations of healthcare-related activities that could be better supported is an area that requiring further study. In addition to their generally known capabilities of facilitating individual tasks and activities, PACS also present opportunities to improve cooperative group work, a common form of activity taking place within or across healthcare institutions. Our research has investigated application of groupware technologies, which are widely used to support cooperative activities in business and government domains, to medical education. This study, in particular, focuses on routine face-to-face teaching conferences, a critical resident-training activity through which radiologists impart to residents the valuable image reading skills they have accumulated from years of practice.

Recently, many technologies and products, collectively known as groupware, have been designed to support such cooperative activities as meetings, group software development, and group writing (Ellis, et al., 1991; Ishii, 1990; Lai, et al., 1988; Mantei, 1989; Mantei, et al., 1991; Stefik, et al., 1987; Watabe, et al., 1990.) Some empirical and field studies have shown that use of these tools improved the efficiency and productivity of groups engaging in collaborative activities and/or tasks (Yakemovic and Conklin, 1990), particularly conventional face-to-face meetings in business environments (Nunamaker, et al., 1989; Post, 1992) and academic settings (Norman, 1990). However, little attention has been directed toward supporting such group activities as diagnostic or teaching conferences in healthcare institutes (Martinez et al., 1993; Cox, et al., 1992; Hoshi et al., 1992; Jensch et al., 1992.)

In an initial study, we focused on the the potential for providing better support for radiological image reading teaching conferences. This paper presents the design and implementation of a groupware system aiming at better facilitation of these conferences and discusses an evaluation plan of the system. The organization of the paper is as follows. Section two discusses the research questions and design, followed by a description of the teaching conference flow in section three. Requirements and design of a prototype system supporting teaching conferences more effectively and efficiently than the current conference mode is summarized in section four. Some interesting preliminary evaluation results and a future evaluation plan are presented in section five. The paper is concluded with some remarks and proposed future research direction.

Research design

The main purposes of this research have been to explore the applicability of groupware systems in medical fields, to measure resulting incremental changes in service level, and to investigate differences between groupware applications in the medical domain and in businesses and other organizations. To explore these issues, we selected as our research setting the routine resident teaching conferences held at the Department of Radiology of University Medical Center (UMC) at the University of Arizona.


The research questions addressed some design issues critical to the success of applying groupware technologies in the targeted medical group activity, resident teaching conferences such as:
- What factors impede the efficiency of the current conference mode?
- What functional requirements must be supported to facilitate teaching conferences?
- What data sources and formats should be investigated and included?
- Which communication media could provide a higher level of conference support?
- What user interface characteristics must be addressed? In particular, how can user interface design deal with such issues as
- windows arrangement,
- monitors arrangement, and
- input and pointing devices.

Field observations and interviews were the primary techniques employed to identify problems with the current teaching conference mode and functional requirements essential to effective teaching (Toshimitsu et al., 1994.) Observations were made on the entire life cycle of the conference, from pre-conference preparation to post-conference interactions between lecturers and residents. Interviews, both structured and unstructured, provided clarification, verification and answers to questions that had arisen in observations.

In the system requirements analysis phase, factors reducing the service level of the current teaching conference process were identified, and the applicability of groupware systems in these conferences was investigated. In addition, requirements analysis also addressed such issues as what functions, data sources and formats, and communication means would need to be included in the proposed system. System design dealt with issues such as groupware tool selection, data support, user interface, and so on. A prototype system was designed and its implementation is currently under way.

Radiological image reading teaching conference

Teaching conferences provide valuable opportunities for residents in radiology departments to develop radiographic images reading skills. At UMC, a teaching conference lasting approximately one and a half hours is held every day. In each conference session, two lecturers are allocated about 45 minutes each to present and discuss some interesting and educational cases of a specific modality or disease/abnormality. Conference lecturers are usually radiologists from the Radiology Department of the study site, with occasional inclusion of outside speakers. Because of clinical obligations and work site constraints, not all of the residents can attend the conference on a daily basis; however, they are encouraged to do so. The number of the residents attending a conference is usually between ten and twenty, with some of the attendees unable to stay through the entire conference session because of other commitments.

The following sequences depict the entire life cycle of the teaching conferences. In advance of the session, the lecturing radiologist searches and selects a set of interesting cases from teaching files (individual or departmental), and prepares relevant data and information which will be used during the conference. Data/information may be in various formats such as films, slides, transparencies, patient clinical history, or anatomy of a specific anatomic part, and may be from various sources. At the conference, after showing the images of one case to the residents, the lecturer will ask one of the residents to interpret the images shown. While the resident does this, the radiologist may raise questions concerning some key points regarding quality interpretation of the current case. A clinical reading report will be given at the end of the case discussion. The radiologist at this point will highlight how to interpret the current case and answer questions from the residents. The lecturer usually discusses three to five cases, depending on the actual pace of the conference. Figure 1 summarizes the described conference process.
Click here for Picture


Figure 1 Process of Teaching Conferences

Because of the well-defined life cycle of the teaching conference, use of Group Decision Support Systems (GDSS) to conduct them has a natural appeal. GDSS have specific tools to address and facilitate each conference phase, from pre-conference preparation to post-conference activities. However, the nature of the teaching conference is quite different from that of ordinary meetings in that no decision needs to be reached as a final deliverable (because the diagnosis or examination reading report is well documented and accessible.) A GDSS session begins a with planning session, which is equivalent to pre-conference preparation. During the session, ideas and comments on the discussed issue can be generated, classified, sorted, and prioritized using some GDSS tools. Other GDSS tools can be utilized to facilitate the discussion of comments having high priority in a more detailed manner. GDSS support for comment generation, classification, sorting, and prioritization therefore gives the technology obvious potential to improve image reading teaching conferences.

Teaching conference support system

We next describe the purposes and functions of a Medical Image Reading Teaching Conference Support System (MedTeaConS), a system supporting image reading teaching conferences, and show how these functions have been implemented as a prototype system.

Purposes of the system

The goals of MedTeaConS are to improve the accessibility of needed data before, during, and after an image reading teaching conference, and to stimulate the dynamics and synergy of discussion during a session. From observation of teaching conferences we found that conventional conferences suffered from the following drawbacks:

1) Difficulty in accessing data required during the conferences because of the scattered location of sources.

2) Inability to corrrielate diagnosis and images making it difficult for residents to review dung the conference how images were used to diagnose the discussed case.

3) Serial discussion deprives a majority of the residents of participation in case discussion. Residents who tend to be passive or have not been asked to explain their interpretations seldom, if ever, get feedback from the lecturing radiologist.

To address the limitations commonly found in the current conference mode, the requirements for the proposed system needed to include the following.

1) To provide convenient access to all data/information relevant to image interpretation as they are needed.

2) To give all residents the opportunity to "speak out" their interpretations and questions while making accessible comments and interpretations generated during the discussion session by all the residents, as a group.

3) To encourage the participants to generate as many comments as possible, with no reservations or concerns.

4) To record the data used and the comments generated during the conference and save them in a conference minute file, accessible to residents for post-conference use.


System functions

In this section, we identify the functions of the MedTeaConS, which has been designed based on the requirements described above.

Preparation support

The main function of conference preparation is to provide a friendly interface to PACS and reference databases to facilitate the search and selection of educational and interesting cases to be used in the conferences. The lecturer should be able to locate appropriate cases by querying the reference database using such predicates as diseases/abnormalities, modalities, dates, and physician or reading radiologist names. Given that the data to be used in image reading are multimedia in nature, content-based information search/retrieval may be more appropriate and deserves more future research. Data selected at the conference server should be readily transmitted to the other workstations on the network. The system also should have a function that allows a lecturer to sequence the data/information to be presented or made accessible during the conference in such a way that he or she can literally, if desired, hide some images and/or data from the residents.

Comment generation support

Image commenter:

The "image commenter" is a tool which allows participants to enter comments for radiographs and displays those comments on the monitors of all the workstations during an image reading teaching conference. If a comment is correlated with a specific anatomic part on an image, the user can link the comment to that specific anatomic part with an arrowhead.

The user interface of the image commenter has three kinds of window: "comment entry window," "public image window," and "public text window." The user enters a comment through the comment entry window. When he or she needs to indicate which specific anatomic parts of an image are being used to make his or her diagnosis, the comment is connected to the image by clicking a mouse button to point to the specific anatomic parts on the image of a public image window. The comment is then sent to the conference server and the server distributes it to all the workstations, where comments referring to the image are displayed in the public image window, which consists of an image and a comment window. Comments not referring to the image are displayed in a public text window. During comment generation, users can control their windows independently; that is, they can browse and select images they want to read and can scroll the generated comments. Although, by default, comments are connected by links to specific anatomic parts to which they refer, the user can stop displaying links when an image becomes messy with links.

Since all participants using this tool can enter comments at the same time, the lecturer can get comments (i.e., image interpretations) from all the residents participating in the conference and obtain a collective view on their image reading skills. This is a major improvement over conventional conferences. Also, because comments entered by other residents are displayed on each workstation, participants may be inspired, stimulated, or encouraged to create additional comments.

Discussion support

Image reading idea organizer:

Even though perusing the comments in the public windows lets a lecturer know in general how well the participants read images, the lecturer can get more specific information by reorganizing the comments. The "comment organizer" finds and gathers comments having common characteristics. The tool allows the lecturer to sort comments by entering a keyword such as a disease, a specific anatomic part, or symptom, by means of a function similar to the "find" command in editor software. In addition, the tool provides a function to gather the comments referring to the same specific anatomic part in an image. By drawing a line surrounding the specific anatomic part in which the lecturer is interested, the comment organizer finds and gathers the comments referring to that specific anatomic part and displays them together. By looking at the comments gathered with this tool, the lecturer can verify whether the residents have correctly interpreted the important anatomic part and identify what kind of images they have difficulty in reading. The lecturer can also "trap" comments in which he/she might be interested. If there is a a specific anatomic part which is the key to interpreting an image, the lecturer can designate that part by drawing lines surrounding it in advance. As soon as comment generation starts, all comments which refer to the specific anatomic part are selected and displayed together.

Cursor synchronization:

Since the users of the system need to interpret images and to enter comments independently during comment generation, the system allows users to change the arrangement of windows on their workstations and to select an image they want to read. After comment generation, they move to a phase in which they verbally discuss images and make comments on those images. In this phase, the system provides functions to synchronize the windows arrangement and cursor movement of all the workstations so that all the participants can look at whatever subject one of them points to with the cursor. The system provides only one cursor for a group of users and the right to control the cursor and windows arrangement is passed among them on a first-come-first-served basis.

Review support

The annotations entered by residents are stored as minutes, which record images used in a conference and other conference information including the date, time, the names of the lecturer and participants so residents can look at these data to review a conference. The minutes can be found by using one of keywords listed above, or a word in annotations.

Implementation

The prototype system is under development. The image commenter, image idea organizer, and part of the preparation and review support function, which allows to select images, save conference minutes, and display minutes, have been implemented on DEC 3100 stations as shown Figure 2. Functions for conference preparation and review are installed as conference server functions and sorting annotations with the image idea organizer is performed on the conference server. The image commenter function is installed as client software. Although it is possible to implement the communication functions between the conference server and other workstations by using X protocol, TCP/IP is used for these functions because it can be adapted to a future expansion to voice and video communications.
Click here for Picture


Figure 2 Prototype system configuration

The user interfaces described in previous sections are X window-based and written in Motif, which is the most sophisticated user interface tool on UNIX-based operating systems. Figure 3 shows an example of the user interfaces of the image commenter function. We decided to use a pull down menu to invoke the functions the system provides because the main target users are residents who might be relatively familiar with such user interfaces.

Evaluation

Initial Evaluation Results

A preliminary evaluation of MedTeaConS was performed prior to conducting any formal or large-scale evaluation in clinical settings. The purpose of the initial evaluation was to ascertain the appropriateness of the current research direction and design approaches. A senior radiologist to whom the prototype system was demonstrated for a critique. His comments were mostly positive and encouraging. He agreed that the effectiveness of the teaching conference could be improved by a system which allowed residents to assume more active participation than in the conventional conference setting in which the majority of the residents assume a rather passive participation role.

In addition, the subject made some recommendations, from a medical educator and clinician's perspective, on both user-interface and probable system applicability. Given that the nature of generated comments (or diagnoses) tend to be disease-, reason-, and/or anatomic-part-oriented, a more structured user-interface design is desirable. That is, the design of public windows probably needs to include multiple sections, each specifically dedicated to collecting comments of the same category (e.g., disease). Thus, lecturers could , at any point in time during the conference, closely trace residents' comment trajectory, as a group, by category.

Quality assurance, a procedure to ensure that the service rendered by the organization has been maintained at a satisfactory level, was another area where MedTeaConS may have potential for considerable improving the effectiveness of teaching conferences. Expansion to support virtual teaching conferences in which residents and/or lecturers are spacially separated is another promising application area for MedTeaConS. However, the subject showed reservations about the potential value of the system in supporting diagnostic conference, because of its unstructured and case-dependent nature.



Future evaluation plan

Environment:

Complying with the UMC teaching culture and its mainstream practice, the study focused particularly on conferences in which approximately 15 to 20 medical residents are to be taught and trained by a radiologist using several pre-selected teaching cases of targeted modality and/or disease. Restricted by the current level of hardware support, the targeted conference size to be supported may shrink. In addition, because current technologies in image processing and achievable resolution have not reached the level required to perform image reading on monitors (i.e., workstations), original films will be made accessible throughout the conference in case the image quality on monitor is not readable.

Characteristic parameters:

To objectively evaluate incremental changes in service level realized and/or restricted, some system considerations such as input and environment will be investigated. Group and task characteristics constitute the main system input considerations. Characteristics such as number of participating residents, individual domain knowledge, typing skill, and level of physical dispersion will be some of the important aspects describing the "group," whereas attributes such as case complexity, discussion time, completeness and clarity of the case collectively will be used to describe "task characteristics" (teaching characteristics). In addition, consideration of the teaching environment (context), which reflects characteristics of both the organization and selected groupware tools, must include organizational culture, conference objectives, process style, and task structure. A. R. Dennis has examined the results from prior Electronic Meeting System (EMS) studies (Dennis, et al., 1991) and concluded that the processes and outcomes of group activity are dependent on variables that can be categorized into four groups: organizational context, group characteristics, task characteristics, and EMS environment. Table 1 provides a complete listing of the identified variables and their classifications.

Table 1 Variables to describe the EMS studies

Organizational context; Group characteristics; Task characteristics; EMS environment

Organizational culture, Incentives, Same objectives, Interrelated problems; Subjects, Task familiarity, History/future, Formal hierarchy, Experience with EMS, Group size, Logical size; Task type, Complexity, Meeting length, Task clarity, Information symmetry, Information sources, Meeting minutes, Previous meeting; EMS name, Process style, Process structure, Task structure, Facilitation

From observation of conventional teaching conferences, we have already assumed some of these parameter values as follows:

- Organization characteristics
- Compliance with organization culture and conference objectives. The proposed system will not change organization culture or redefine conference purposes.
- Group characteristics
- Homogeneity. Participating residents are considered homogeneous in terms of domain knowledge, image reading skills, typing skill, familiarity with cases to be discussed and groupware tools to be used.
- Peer-to-peer. No formal organization hierarchy or negative inter-personal relationships exist among the participating residents.
- Information symmetry. All participating residents have equal access to data and information on the cases discussed.
- Task characteristics
- Unlimited information sources. The information sources for the conference are not limited to those included in the system; that is, outside information sources are accessible as they are needed.
- Process characteristics
- No lead figure. Each resident is to be treated equally and can make any image interpretation he/she wishes in parallel to those of others.
- Anonymity. The identity of the individual resident making a comment will not known to others, including the radiologist in charge of the conference.
- Least possible level of outside facilitation support. No facilitator will be present to guide the conference proceeding. The radiologist in charge of the conference is expected to assume the role of conference facilitator.
- Same time and location only. The system, at the current stage of its development, is focusing on the support of teaching conferences held in one location, with participants showing up at the same time.

The level of support in all three conference phases identified and discussed in an earlier section will be assessed. Some measurements have been studied and selected for each activity phase. For instance, measurements on the ease and flexibility of case selection, diversity of the case pool, time required to select a case, and degree of relevant information will be included as part of the pre-conference evaluation to provide a clear picture of the effectiveness of the system in supporting conference preparation. The level of participation, individual and group productivity in contributing image interpretation, quality of the attempted interpretations, and overall satisfaction are some examples of measurements that will be used to indicate during-conference process support whereas the frequency of the use of virtual conference minutes, for both educational and clinical purposes, will provide a good indication of the post-conference reference values of the system.

Conclusion

The goals of this research were to explore the applicability of groupware systems in a medical field and to identify the requirements of systems designed to support group activities in radiology departments. To this end, we focused our investigation on one of these activities, face-to-face radiological teaching conferences, and derived system requirements of the conference using both field observation and interview techniques. From this study we concluded that a system supporting such conferences should facilitate access to the various types of data to be used, encourage synergy in interaction between participants, and record discussions so that residents can review the conferences. According to these derived requirements, we designed a system based on electronic meeting support technologies and implemented some of the system functions, comment generation support and post-conference support, as a prototype system. A senior radiologist's first impression of the system was supportive and we will continue to evaluate the effectiveness of the system in real settings.

Acknowledgments

The authors would like to thank doctors in the Radiology Department at the University Medical Center of the University of Arizona for their support and cooperation during the requirements analysis. Special thanks are due to Drs. Terry Ovitt and Laurie Fajardo for their coordination of the project. Thanks are also due to Toshiba Corporation for its continuous support for and dedication to PACS research.
References

J.R.Cox, G.J.Blaine, M.W.Dubetz, K.Krieger, R.G.Jost, S.M.Moore, W.D.Richard, J.S.Turner, and A.Winterbauer (1992). "A Demonstration of Medical Communications Based on an ATM Broadband Network Technology," SPIE vol.1654 Medical Imaging VI, pp.44-53.

A.R.Dennis, J.F.Nunamaker, and D.R.Vogel (1991). "A Comparison of Laboratory and Field Research in the Study of Electronic Meeting Systems," Journal of Management Information Systems, Vol.7, No.3, pp.107-135.

C.A.Ellis, S.J.Gibbs, and G.L.Rein, (1991). " Groupware: Some Issues and Experiences," Communications of the ACM, Vol.34, No.1, pp.39-58.

T.Hoshi, K.Mori, Y.Takahashi, Y.Nakayama, and T.Ishizuka (1992). "B-ISDN Multimedia Communication and Collaboration Platform Using Advanced Video Workstations to Support Cooperative Work," IEEE J.Select.Areas Commun., vol.10, no.9, pp.1403-1412.

H.Ishii (1990). "TeamWorkStation: Towards a Seamless Shared Workspace," Proceedings of the Conference on Computer-Supported Cooperative Work(CSCW'90), Los Angeles, CA, pp.13-26.

P.Jensch, A.Hewett, and A.Barth (1992). "ISDN-PACS Concepts and Groupware Essentials for Tele-Diagnosis," SPIE vol.1654 Medical Imaging VI, pp.89-96.

K.-Y.Lai, T.W.Malone, and K.-C.Yu (1988). "Object Lens: A "Spreadsheet" for Cooperative Work," ACM Transactions on Office Information Systems Vol.6, No.4, pp.332-353.

M.M.Mantei (1989). "Observation of Executives Using a Computerized Supported Meeting Environment," International Journal of Decision Support Systems 5, pp.153-166.

M.M.Mantei, R.M.Baeker, A.Sellen, W.A.S.Buxton, T.Milligan, and B.Wellman (1991). "Experiences in the Use of a Media Space," Proceedings of CHI'91, pp.203-208.

R.Martinez, J.Kim, and J.Nam (1993). "Remote Consultation and a Diagnosis in a Global PACS Environment," SPIE vol.1899, Medical Imaging V PACS: Design and Evaluation.

K.L.Norman (1990). "The Electronic Teaching Theater: Interactive Hypermedia and Mental Models of the Classroom," Current Psychology: Research & Reviews, Vol.9, No.2, pp.141-161.

J.F.Nunamaker, D.Vogel, A.Heminger, B.Martz, R.Grohowski, and C.McGoff (1989). "Experiences at IBM with Group Support Systems: A Field Study," Decision Support Systems, Vol.5, No.2, pp.183-196.

B.Q.Post (1992). "Building the Business Case for Group Support Technology," Proceedings of the Hawaii International Conference on System Sciences, Vol.IV, pp.34-45.

M.Stefik, G.Foster, D.G.Bobrow, K.Kahn, S.Lanning, and L.Suchman (1987). " Beyond the Chalkboard: Computer Support for Collaboration and Problem Solving in Meetings," Communications of the ACM, Vol.30, No.1, pp.32-47.

A.Toshimitsu, O.R.L.Sheng, and P.Hu (1994a). "Design and Implementation of Computer-aided Support for Teaching Conferences in Radiology," forthcoming in Proceedings of SPIE Medical Imaging VI.

K.Watabe, S.Sakata, K.Maeno, H.Fukuoka, and T.Omori (1990). "Distributed Multiparty Desktop Conferencing System: MERMAID," Proceedings of the Conference on Computer-Supported Cooperative Work(CSCW'90), Los Angeles, CA, pp.27-38.

K.C.B.Yakemovic, and E.J.Conklin (1990). "Report on a Development Project Use of an Issue-Based Information System," Proceedings of the Conference on Computer-Supported Cooperative Work (CSCW'90), Los Angeles, CA, pp.105-118.