Mathematical Problems in Engineering

Volume 2013 (2013), Article ID 932586, 8 pages

http://dx.doi.org/10.1155/2013/932586

Research Article

Exploring the Effects of Multidimensional Context on Human Performance in Personal Bookmark Management

Siu-Tsen Shen

Department of Multimedia Design, National Formosa University, 64 Wen-Hua Road, Hu-Wei 63208, Taiwan

Received 9 September 2013; Accepted 14 October 2013

Academic Editor: Stephen D. Prior

Copyright © 2013 Siu-Tsen Shen. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

This paper presents an exploratory study in comparing the gender difference of how Taiwanese university students interacted and organized their bookmarks under 2D and 3D conceptual prototyping. Participants were asked to execute a set of online tasks for bookmark management and take screenshots to further ground the findings. The results showed that 62% of the female participants preferred the 2D context and there was no significant difference between the 2D and 3D tasks in terms of their time taken, which indicated that they maintained the consistency of their task performance strategy and were not subject to the change of spatial dimension. In contrast, the male participants performed poorly in the 3D context, as opposed to the 2D environment, even though they stated that they preferred the 3D context. A possible explanation for this might be that most of the male participants (mean age 21.8 years, ) are regular 3D game players. Therefore, they tend to unconsciously prefer 3D over 2D. Although the null hypothesis was not disproved, the task time results showed a degree of significance in terms of gender and spatial context differences, which requires further investigation.

1. Introduction

Internet interfaces are no longer a series of static pages linked together by HTML code. A huge range of websites now offer different forms of interaction to engage users, and these can take the form of search engines, online video conferencing, instant messaging, music sharing, personalized blogging, and online shopping and bookings, to name but a few. Users expect websites to meet their needs by collecting and categorizing their data autonomously. This requires web browser developers to generate smarter and more intuitive programs; at present, the best that they can offer is a strictly defined bookmark management system to help users organize data.

In this paper, the author investigates the possibilities of developing 2D and 3D bookmark management prototypes based on the Garden Metaphor [1] and conducts a comparative evaluation of user’s performance and preference between 2D or 3D interfaces.

2. Theoretical Background

2.1. Bookmark Management Systems

2.1.1. Browser-Based System of Folders

Bookmarks, known as favorites, or internet shortcuts in Internet Explorer, help users to save their desired links and are an essential component of modern web browsers. The function of bookmarks is to provide the user with the ability to create numerous folders (directories) and subfolders (subdirectories) in order to store, retrieve, and organization information sources. However, the maintenance of bookmarks requires continuous and intensive efforts by naming, sorting, refining, and updating taxonomies. The majority of users find that this process is time consuming and still does not guarantee later information retrieval. To alleviate this problem, the leading browsers such as Google Chrome and Internet Explorer added a search function within the system, whereby typing partial keywords or Uniform Resources Identifiers (URIs) would display an a-priori assumption list.

2.1.2. Tag-Based Systems

There is another kind of bookmark management system called social bookmarking, also known as tagging or folksonomies. This tag-based system is mainly for sharing information within special interest groups either for private or for public purposes. Famous examples are Flicker and Delicious (http://delicious.com).

2.1.3. Other Experimental Alternatives

Other web browser developers have extended the concept of the bookmark function in saving links further. PowerBookmarks developed by Li et al. allow bookmarks to be indexed and classified by query [2]. Live bookmarks from RSS feeds have been introduced by Mozilla Firefox since 2004 and could be automatically updated by news sites or blogs. Bookmarklets, which were coined by combining the terms bookmark and applets, are run by JavaScript programs and stored as bookmarks which can be installed by browsing a website and clicking to launch an action [3]. Other types include Weave, Xmarks, and Diigo [4].

These various bookmark management tools aim to help users organize bookmarks more efficiently and effectively. However, they still cannot alleviate the problems of massive piles of unorganized bookmarks and lost bookmarks for unforeseen reasons.

2.2. Related Work on 2D and 3D Desktop Management

2.2.1. 2D Desktop Management

Most current bookmark management systems rely on a hierarchical structure, as well as file management systems. Various examples are different from their visualization presentation and functionality [5–7]. The 2D Lifestreams interface developed by Freeman was a time ordered stream of documents much like a diary of your life, whereby you could store your documents that you created and others sent you. Lifestreams had five major functions, that is, New (for creating documents), Clone (for duplicating documents), Transfer (copying from your stream to others), Find (for a search query and creating a substream), and Summary (by taking a substream and compressing it to a single overview) [8]. More recent documents appeared to be in the front of the screen, while the older ones stayed in the back of the screen. It provided a personal collection of saved documents that could be viewed through filtering based on time.

Ringel et al. had a similar idea for the use of time-centric visualization to display search results from personal emails, files, and bookmarks provided by both public (news headlines, holidays) and personal (date appointments, digital photos) landmark events [9].

Linking File System Browse (LiFSBrowse) used customizable graphical structures in the file system and allowed users to view their files and web pages through querying [10].

Scopeware Vision developed by Mirror World Technologies (NWT) integrated documents and Email messages into addictive streams [11]. Its default screen could show a maximum of 100 of the most recent files and the users could type in keyword search terms such as the file’s name, format style, date, and size in thumbnail previews. However, Scopeware was discontinued in 2004 [12], possibly because of its similar capabilities to the update version of Microsoft’s Longhorn release.

Inxight StarTree, also known as the Hyperbolic Tree, provided users with the capability to navigate and explore visual hierarchical relationships for a large collection of different file formats, and drill-down further information of interest within a circular boundary [13].

We may conclude that 2D interfaces have not radically evolved since the introduction of Graphical User Interfaces (GUIs) in the 1980s. The linear, hierarchical structure has been the standard and has been applied extensively to the whole operating system and web applications, such as the file system and bookmark management system. The exploration of alternative 2D information visualization still needs more empirical research to support further development. Therefore, the standardized system has remained largely unchanged and underdeveloped in comparison to 3D systems.

2.2.2. 3D Desktop Management

Over the decades, there have been several predictions of the coming trend in three-dimensional (3D) user interfaces for the desktop. Nevertheless, 3D visualization seems to be restricted to home entertainment, especially the game industry, and its preponderance for first person gaming.

Several attempts to develop 3D bookmark management systems include work by Card et al.—WebBook, and Roberson et al.—Data Mountain. WebBook offered users the ability to save web pages as a book collection in a 3D environment, where users could turn and view the pages and even store them in a bookcase [14]. Data Mountain made use of spatial memory for document management and allowed users to arrange the files in their desired location with 2D iconic viewing layout, that is, thumbnails of web pages, under a 3D desktop environment [15]. It has been said that spatial cognition could aid in information management.

Further, examples include Robertson et al. Cone Tree, and Agarawala and Balakrishnan’s Drag’n’Cross. Corn Tree displayed a hierarchical tree structure of information in order to maximize the screen space and could overview the whole structure [16]. Drag’n’Cross simulated a series of our physical actions such as dragging, crossing, and scrubbing on desktop environment to help organize information [17].

Unfortunately, these efforts lack further validation and improvement results. Partially, this might be because of a lack of major leading software developers being not keen to move from 2D to 3D. Therefore, 3D user interface desktop applications have not yet been able to compete with developments in the 3D games industry.

2.3. User Performance and Preference of 2D versus 3D Interface Design

User performance and preference of 2D and 3D interfaces have been discussed in various research papers. Levy et al. found that 3D graphics were preferred in the need of describing and memorizing details, whilst 2D graphics were preferred for the instant feedback and general impressions [18]. Ware and Franck showed that 3D visualization performed better and more comprehensively than 2D, in terms of information presentation [19]. Cockburn and McKenzie examined the 2D and 3D versions of desktop management based on the concept of Robertson et al. [15], Data Mountain, and indicated that there was a significant preference for 3D, even though the subjects were slightly faster using 2D in storing and retrieving tasks. However, 2D retrieval time was not significant when compared to 3D [20].

Ozok and Komlodi conducted an experiment with 20 subjects who had regular online shopping experience by providing product representations in 2D and 3D (3D low interaction and 3D high interaction). The results showed that the 3D representations, both in low and high interaction, were preferred over the 2D ones, in that they contained more product details, were easy to use, and more fun than the 2D representations [21].

Despite 3D interfaces having certain advantages in terms of the browsing depth of the views and presenting more semantic information, it has been said that they require more computer hardware such as memory cards and a higher resolution of image cards. Perhaps more importantly, they take additional cognitive loading with more complicated instructions in navigation and manipulation [22–25]. 3D spatial semantic mapping provides no advantage and may result in poor performance compared to 2D mapping [26].

In short, the choice of 2D or 3D is highly dependent on which types of tasks are being conducted, which in turn leads to different browsing performance and mapping between 3D input devices, displays, and interaction styles. In the following user studies, we investigate the potential advantages and disadvantages of using 2D and 3D prototypes for a series of bookmarking tasks, which is the extension of Shen and Prior’s work on the i-Pot conceptual bookmark management systems [1].

3. User Study Trials

3.1. Test Conditions

Each participant (mean age 21.4 yrs, ) was rewarded with 1000 NT$ (approx. 33 US$) for their time. All the participants had previously qualified with intermediate certificate of General English Proficiency Test (GEPT) issued by the Language Training and Testing Center (LTTC) in Taiwan. The computer used during the tests was running Windows XP SP3 with CPU Intel Pentium 4 (3.20 GHz) 1 GB Ram, using the Google Chrome browser.

3.2. User Study in 2D Prototype

There were a total of 26 students (13 males and 13 females) who took part in this study which was undertaken during 10–31 May, 2012. After a brief introduction and demonstration, the participants were asked to do a series of tasks including creating, deleting, naming a new pot, and changing the color and texture of a new pot (see Supplementary Material at http://dx.doi.org/10.115/2013/932586). They were each given eight (English) websites to bookmark (creating a new leaf) into four specific headline categories, that is, work, social, travel, and health according to a controlled vocabulary protocol. At the end of the test, the participants were asked to find a particular website from the History tree. The time taken for each user’s performance was measured and graded on how well they accomplished the tasks. This was followed by a five-point Likert scale satisfaction questionnaire, and the participants were asked to provide their feedback on what they thought of the i-Pot 2D version and which elements they liked most and liked least about the interface and give their reasons for this. Further questions included the user’s acceptance of the web browser and participants were asked to provide further suggestions on changes to the design of the 2D i-Pot (see Figure 1).

Figure 1: 2D i-Pot bookmark management interface.

3.2.1. Results

The mean task time for male participants to complete the test was 191.4 s (SD = 28.5), whilst the mean task time for females was 229.3 s (SD = 57.5). The vast majority of the participants (92%) agreed that the concept of gardening was familiar to them and the information displayed was in a natural and logical order. Most of the participants (88%) thought that the visual icons and layout fitted appropriately to the gardening context, which allowed users to adapt frequent actions and to speed up the interaction. There were 81% of the participants who thought that the visual icons and layout contained relevant information and that the overall design was attractive. The majority of the participants (92%) agreed that the pots, plants, and leaves are logical representational equivalents for folders, files, and documents.

With regard to the functionality of the prototype, most of the participants (73%) found that it was easy to create or delete a new pot. Nevertheless, there was one participant who disagreed (4%) and six participants (23%) who were neutral. Most of the participants (73%) found that it was easy to name a pot. However, there were two participants who disagreed (8%) and five participants (19%) who were neutral. The majority of the participants (93%) considered that it was easy to change the color and texture of the pot. Almost all (96%) of the participants found that it was easy to add a new leaf, while there was one participant (4%) who disagreed. More than half of the participants (62%) thought that the use of the History tree was easy to view their history records, whilst there were two participants (8%) who disagreed and eight participants (31%) who were neutral on this point.

3.2.2. Discussion

In general, the 2D i-Pot prototype had a high satisfaction rate. In terms of functionality, there was one participant who found that it was not easy to create or delete a new pot, whilst two participants thought it was not simple to name a pot. The reason for this might be that the function of creating and deleting actions required the user to click on certain buttons on the screen, rather than click on the right key from the mouse directly. The function of naming a pot required the effort of the user to go to the “Pot Edit” to type the name, compared to the existing bookmark management systems that allow the users to directly name/rename the folders or subfolders (see Figure 2). Furthermore, there were two participants who found that it was not easy to view their history records. This could be explained by the fact that the visualization of the History tree was new to them, rather than the traditional lineal lists (see Figure 3).

Figure 2: 2D i-Pot customizable control panel.

Figure 3: 2D History Tree.

Most of the participants liked the concept of gardening with its 2D horizontal screen mobility, the selection of different pot styles, plant shapes, colours, and textures, because of its novelty, visual presentation of realization, and the sense of freedom to customize the elements. Few participants disliked the limitation of 2D horizontal viewing angle, because the limited screen could only show two or three pots, rather than the whole pot collection. The majority of the participants would consider using the beta version of i-Pot web browser, if it was made available. The reasons included the use of multiple colours and textures that allowed the users to distinguish between their bookmarks, the fun visual look, and the novelty factor.

Further suggestions related mostly to technical issues, which included improving the Internet connection speed, adding a blue status bar when linking to the Internet websites, and attaching a background of the screen that makes it more like a real garden setting. Other suggestions involved the adjustment of the web view proportion into a full screen; adjust the My Favorite with pull-down menus, and add more selections of pot styles, plant shapes, colours and textures.

3.3. User Study in 3D Prototype

The same group of 26 students (13 males and 13 females) that took part in the first study also took part in this study which was undertaken in the computer lab during 10–31 May, 2012. After a brief introduction and demonstration, the participants were asked to do a series of tasks including creating, deleting, naming a pot, and changing the colour and texture of a pot (see Supplementary material). They were each given eight (English) websites to bookmark into four specific headline categories, that is, work, social, travel, and health according to the controlled variable theory. At the end of the test, the participants were asked to find a particular website from the History tree and review their History tree. The time of each user’s performance was measured and graded on how well they accomplished the tasks. This was followed by a five-point Likert scale satisfaction questionnaire, and the participants were asked to provide their feedback on what they thought of the i-Pot 3D version, which elements they liked most, and which they liked least about the interface, and explain the reasons for this. Other questions included the user’s acceptance of this interface, if the web browser based on the Garden Metaphor was made available to them and further suggestions on changing the design of the 3D i-Pot based on the Garden Metaphor.

3.3.1. Results

The average task time for the male participants was 257.5 s (SD= 83.5), whilst the average task time for females was 235.7 s (SD= 58.5). Most of the participants (88%) agreed that the concept of gardening was familiar to them and that the information display was in a natural and logical order. The vast majority of the participants (92%) regarded the visual icons and layout fitted appropriately to the gardening context, which allowed users to adopt frequent actions and speed up the interaction. There were 81% of the participants who thought that the visual icons and layout contained relevant information and the overall design is pleasant. However, there were five participants (19%) who remained neutral on this point. The majority of the participants (96%) agreed that the pots, plants, and leaves are logical representational equivalents for folders, files, and documents.

With regard to the functionality of the prototype, more than half of the participants (58%) found that it is easy to create or delete a new pot. Nevertheless, there were eleven participants who remained neutral (42%). Most of the participants (73%) thought that it is easy to name a pot. However, there were two participants who disagreed (8%) and five participants (19%) who were neutral. The majority of the participant (93%) considered that it is easy to change the colour and texture of the pot. There were 73% of the participants who found that it is easy to add a new leaf, while there was one participant (4%) who disagreed. Most of the participants (73%) thought that the use of History tree is easy to view their history records, whilst there was one participant (4%) who disagreed and six participants (23%) who were neutral.

3.3.2. Discussion

In general, the 3D i-Pot prototype had a high satisfaction rate. In terms of functionality, there were two participants who thought that it was not easy to name a pot. The reason for this might be that the function of naming a pot had required the effort of the user to go to “Pot Edit” to type the name, compared to the existing bookmark management systems that allow the users to directly name/rename the folders or subfolders. Furthermore, there was one participant who found that it was not easy to view their history records. This could be explained by the fact that the visualization of the History tree was new to them, rather than the traditional lineal lists (see Figure 6).

Most of the participants liked the concept of gardening with its 3D easy to manipulate multiple angles (see Figure 4), intuitive spatial management (see Figure 5), the selection of choosing different pot style, colour, and texture, because of its novelty, visual presentation of realization, and the sense of freedom to customize the elements. A few participants disliked the 3D prototype due to the fact that they thought it required a great effort to organize their bookmarks, especially when there was more than twenty pots. The majority of the participants would consider use beta version of i-Pot web browser if it was made available. The reasons included the use of multiple colours and textures that allow the users to locate their bookmarks, the fun visual look, and novel features.

Figure 4: 3D i-Pot bookmark management interface.

Figure 5: 3D i-Pot browsing window split screen.

Figure 6: 3D History tree.

Further suggestions were mostly technical in nature and including improving Internet connection speed, adding a blue status bar when linking to the Internet websites, and attaching a background of the screen that makes it more like a real garden setting such as sky and pebble grounds. Others were the adjustment of the Web View proportion into a full screen, My Favorite with pull-down menus, and more wide selections of pot styles, plant shapes, colour, and texture.

3.4. Overall Summary

The purpose of this study was mainly to observe the participants’ capability to adapt to a new visually oriented bookmark management system, and to see how they migrate and manipulate the elements on the screen, based on their experiential cognitive association knowledge. Even though the researchers did not constrain time, they made sure that each participant had successfully completed the task before the end of the session.

Analysis of the overall results shows some interesting findings as follows (see Figure 7).

(1)Males performed the fastest overall in the 2D tasks.(2)Males performed the slowest overall in the 3D tasks.(3)Females performed broadly the same in both the 2D and 3D tasks.

Figure 7: Task time analyses for gender and spatial context.

Figure 8 shows a box plot of the task time performance experimental data across the four subject groups (gender spatial context).

Figure 8: Box plot of task time experimental data.

3.4.1. One-Way ANOVA Analysis of Task Times

We analyzed the task times of the subjects using a one-way ANOVA test (Table 1); this was conducted using four groups :

 A: female (3D), B: male (3D), C: female (2D), D: male (2D).

Table 1: Results of ANOVA statistical analysis.

Source of variation
Sum of squares
d.f.
Mean squares

Between

3
9838
2.711

Error

48
3629

Total

51

Each group consisted of 13 time data points (). The null hypothesis, denoted by , for the overall F-test for this experiment would be that all four levels of the factor produce the same response, on average. Therefore, there would be no significant difference between gender or the 2D or 3D environments.

The probability of this result, assuming the null hypothesis, is 0.055. In this case, at α = 0.05. Since F = 2.711 < 2.80, the results are on the borderline of significance at the 5% level. One would marginally accept the null hypothesis, concluding that there is evidence that the expected values in the four groups differ and exist on the borderline of statistical significance as can be seen in Figure 9.

Figure 9: F-distribution graph.

3.4.2. User Preferences

When asked about preferences for either the 2D or 3D environment again, the results highlighted some interesting findings as follows.

(1)The vast majority of males (85%) say that they prefer the 3D environment.(2)The majority of females (62%) say that they prefer the 2D environment.

So although most of the male participants say that they preferred using the 3D environment, they were actually much faster when using the 2D environment. They were even faster than the female participants in this context. The reason to explain this might be that most of the male participants (age 20-21) are the regular 3D game players. Therefore, they tend to prefer 3D rather than 2D unconsciously.

The females on the other hand seem to recognize the benefit of simplicity that the 2D environment gives. Overall, the 2D tasks were completed on average 81 s faster than the 3D tasks across both genders.

Several technical problems were identified such as direct manipulation in pointing and clicking on the specific objects which were not properly been selected both for the 2D and 3D prototype and the overlapping pots on the same location which occur unexpectedly when adding a new pot in 3D. These issues would be rectified in the next stage of the development.

4. Conclusion

This study extends previous work by the authors, demonstrating benefits of the Garden Metaphor when participants performed a series of bookmark management tasks requiring direct manipulations of objects in both the 2D and 3D prototype interfaces.

The i-Pot conceptual prototype aims to provide a personalized organization and bookmark management system. The notion of bookmarks is extended by introducing the concepts and functionality of gardening, such as browsing view, navigation, selection of colors and textures, the plant species, and history tree records.

In general, participants performed faster and with a much reduced standard deviation in the 2D tasks as opposed to the 3D tasks. This finding agreed with previous research as mentioned above [21, 24–26]. It is suggested that this may be due to the familiarity of the users’ daily interaction with 2D applications, even with 3D applications such as games, AutoCAD and 3DMax that still rely on 2D direct manipulation of the WIMP.

Results of the testing showed that the male participants performed poorly in the 3D environment, as opposed to the 2D environment, even though they stated that they preferred the 3D environment. This might be due to incongruence between the spatial mapping capability and their presubjective perception. In contrast, 62% of the female participants preferred the 2D environment. However, there was no significant difference between the 2D and 3D tasks in terms of their time taken. This suggests that the female participants’ cognitive ability was found not to be influenced between the 2D and 3D context. This may suggest that the female participants maintained the consistency of their task performance strategy and were not subject to the change of spatial dimension.

It may be argued that this does not represent the whole target users, based on such a small sample of participants. However, it is worth noting that these participants all major in multimedia design, a course which demands a highly skillful ability in both 2D and 3D relevant software. Therefore, even though this was a small sample of users, it still represents valuable data on the subject of user interaction.

Although the null hypothesis was not disproved, the task time results showed a degree of significance in terms of gender and spatial context differences, which requires further investigation. This research indicates a promising and useful application of integrating novel bookmark management systems together with the Garden Metaphor. Many features provided by the i-Pot concept demonstrate creative human computer interaction and rich functionalities.

4.1. Further Work

As stated above, it is intended to further investigate the gender issues raised in this paper. In order to achieve an answer to the central hypothesis (users performance and preference of 2D and 3D interfaces), we plan to involve a larger group of participants. We anticipate that a group size of 100–200 participants would suffice for this purpose.

References

S. Shen and S. D. Prior, “A fresh look at graphical web browser revisitation using an organic bookmark management system,” in Cognitively Informed Intelligent Interfaces: Systems Design and Development, E. M. Alkhalifa and K. Gaid, Eds., pp. 41–55, Information Science Reference IGI Global, Hershey, Pa, USA, 2012.

W.-S. Li, Q. Vu, D. Agrawal, Y. Hara, and H. Takano, “PowerBookmarks: a system for personalizable web information organization, sharing, and management,” Computer Networks, vol. 31, no. 11–16, pp. 1375–1389, 1999.

Wikipedia, “Bookmark (World Wide Web),” 2012, http://en.wikipedia.org/wiki/Bookmark_(World_Wide_Web)

J. Fitzpatrick, “Five best bookmark management tools,” 2010, http://lifehacker.com/5540019/five-best-bookmark-management-tools

R. Pircher, “Personal knowledge management: individual, organizational and social perspectives,” Library Review, vol. 61, no. 2, pp. 165–167, 2012.

S.-T. Shen and S. D. Prior, “Re-thinking bookmark management—less choice is more efficient,” in Design, User Experience, and Usability: Web, Mobile, and Product Design, vol. 8015 of Lecture Notes in Computer Science, pp. 304–312, Springer, Berlin, Germany, 2013.

T. M. Skare and B. Bilinski, “System and method for managing bookmark buttons on a browser toolbar,” US Patent 8,402,375, 2013

E. T. Freeman, The lifestreams software architecture [Ph.D. thesis], Yale University, New Haven, Conn, United States, 1997.

M. Ringel, E. Cutrell, S. Dumais, and E. Horvitz, “Milestones in time: the value of landmarks in retrieving information from personal stores,” in Proceedings of the 9th IFIP TC13 International Conference on Human Computer Interaction (INTERACT '03), IOS Press, Zürich, Switzerland, Septemberm 2003.

S. Ames, “LiFSBrowse: a visual, user environment for the linking file system,” Storage Systems Research Center, Baskin School of Engineering, University of California, Santa Cruz, Calif, USA, 2007.

E. Grevstad, “Scopeware vision review bring clutter and chaos (and maybe your PC) to their knees,” 2003, http://www.smallbusinesscomputing.com/biztools/article.php/2193081/Scopeware-Vision-Review.htm

Scopeware and Mirror Worlds Technologies, 2004, http://web.archive.org/web/20040607060111/, http://www.scopeware.com/

Inxight, “Inxight VizServer,” Inxight Software Inc, 2006

S. K. Card, G. G. Robertson, and W. York, “The WebBook and the Web Forager: an information workspace for the World-Wide Web,” in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '96), pp. 111–117, ACM, Vancouver, Canada, April 1996.

G. Robertson, M. Czerwinski, K. Larson, D. C. Robbins, D. Thiel, and M. van Dantzich, “Data mountain: using spatial memory for document management,” in Proceedings of the 11th Annual ACM Symposium on User Interface Software and Technology (UIST '98), pp. 153–162, ACM Press, San Francisco, Calif, USA, November 1998.

G. Robertson, J. D. Mackinlay, and S. K. Card, “Cone trees: animated 3D visualizations of hierarchical information,” in Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI '91), ACM Press, New Orleans, La, USA, 1991.

A. Agarawala and R. Balakrishnan, “Keepin' it real: pushing the desktop metaphor with physics, piles and the pen,” in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '06), pp. 1283–1292, ACM, Montréal, Canada, April 2006.

E. Levy, J. Zacks, B. Tversky, and D. Schiano, “Gratuitous graphics? putting preferences in perspective,” in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '96), pp. 42–49, ACM, Vancouver, Canada, April 1996.

C. Ware and G. Franck, “Evaluating stereo and motion cues for visualizing information nets in three dimensions,” ACM Transaction on Graphics, vol. 15, no. 2, pp. 121–139, 1996.

A. Cockburn and B. McKenzie, “3D or not 3D? evaluating the effect of the third dimension in a document management system,” in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '01), ACM, Seattle, Wash, USA, 2001.

A. A. Ozok and A. Komlodi, “Better in 3D? an empirical investigation of user satisfaction and preferences concerning two-dimensional and three-dimensional product representations in business-to-consumer e-commerce,” International Journal of Human-Computer Interaction, vol. 25, no. 4, pp. 243–281, 2009.

S. J. Westerman, “A comparison of the cognitive demands of navigating two-versus three-dimensional spatial database layouts,” Ergonomics, vol. 41, no. 2, pp. 207–212, 1998.

M. M. Sebrechts, J. V. Cugini, S. J. Laskowski, J. Vasilakis, and M. S. Miller, “Visualization of search results: a comparative evaluation of text, 2D, and 3D interfaces,” in Proceedings of the 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '99), ACM, Berkeley, Calif, USA, August 1999.

A. Leuski and J. Allen, “Improving interactive retrieval by combining ranked lists and clustering,” in Proceedings of the 6th International Conference on Computer-Assisted Information Retrieval (RIAO '00), ACM, Paris, France, April 2000.

I. Barshi and A. F. Healy, “The effects of mental representation on performance in a navigation task,” Memory and Cognition, vol. 30, no. 8, pp. 1189–1203, 2002.

S. J. Westerman, J. Collins, and T. Cribbin, “Browsing a document collection represented in two- and three-dimensional virtual information space,” International Journal of Human Computer Studies, vol. 62, no. 6, pp. 713–736, 2005.