CoScribe

Integrating Paper and Digital Documents / TU Darmstadt

Even though computing is more than ever part of our private and professional activities, paper is still ubiquitous and plays an important role in knowledge work. This is mainly due to the intuitiveness of writing on paper and to the fact that paper is cheap, lightweight, and can be laid out in physical space. However, information on paper can be less easily searched and shared with co-workers than electronic content. The simultaneous work with paper and digital documents, which is typical for today’s knowledge workers, leads to disruptive transitions, particularly because interaction techniques differ in both worlds.


This research project therefore seeks to extent computing into the physical world and to bridge the gap between paper and digital documents. Based on extensive field studies, we develop novel interaction techniques that seamlessly integrate printed and digital documents and enable users to collaborate with each other in co-located and distant settings. Moreover, we develop general guidelines that guide the design of pen-based user interfaces for cross-media work.

Cross-media Annotation

CoScribe supports users in taking handwritten notes on documents and sharing them with other learners. The same digital Anoto pen is used on paper to annotate printed documents and on a tabletop display to annotate digital documents. CoScribe currently supports PDF and PPT documents as well as web pages. Users can collaborate at the same place and moreover electronically share their annotations with co-workers over the distance. A document viewer and a plug-in for Mozilla Firefox provide for accessing a digital version of the document including own annotations and shared annotations of other users. Moreover, CoScribe provides means for the paper-based classification of annotations with semantic types and visibility levels for sharing.


Cross-media Hyperlinks

Users can easily create own hyperlinks between printed documents and web pages (in arbitrary combinations). It suffices to connect both documents or document passages with an intuitive pen gesture. Once created, hyperlinks can be activated by tapping with the pen on the link hot-spot on a printed document or on a digital document which is displayed on the tabletop. A novel visualization integrates all documents and focuses on the hyperlinks between these documents. This supports users in structuring the entire document space.


Cross-media Bookmarking and Tagging

Bookmarks provide an efficient and well established means for structuring, indexing and tagging documents, all important processes for successful knowledge work. Digital Paper Bookmarks are a novel interaction technique which combines the intuitiveness of paper bookmarks with the powerful capabilities of electronic bookmarking. The physical manipulation of tangible stickers automatically creates electronic bookmarks. Document can be tagged both with a set of predefined types and with freely-chosen handwritten labels.


Digital Paper Bookmarks are electronically accessible in several visualizations. A collaborative visualization of own bookmarks and shared ones of other users allows contrasting the own understanding with those of others.


Further paper-based interaction concepts for tagging include Tag Buttons for tagging with predefined types and Tag Menu Cards for tagging with user-defined labels.

Selected Publications

CoScribe: Integrating Paper and Digital Documents for Collaborative Knowledge Work

Jürgen Steimle, Oliver Brdiczka, Max Mühlhäuser: CoScribe: Integrating Paper and Digital Documents for Collaborative Knowledge Work, IEEE Transactions on Learning Technologies, 2 (3), pp. 197-204, IEEE Press, 2009. [DOI]

Designing Pen-and-Paper User Interfaces for Interactions with Documents

Jürgen Steimle: Designing Pen-and-Paper User Interfaces for Interactions with Documents,Proceedings of the 3rd International Conference on Tangible and Embedded Interaction (TEI’09), pp. 197-204, ACM Press, 2009. (Full Paper) [DOI]

Pen-and-Paper User Interfaces

Jürgen Steimle: Pen-and-Paper User Interfaces, Springer: Heidelberg, 2012.

Awards

  • Computer science dissertation award 2009 by the German Informatics Society (GI), the Swiss Informatics Society (SI), the Austrian Computer Society (OCG) and the German Chapter of the ACM
  • Finalist of the European E-Learning Award (eureleA) 2009
  • Outstanding Full Paper Award at the 8th IEEE International Conference on Advanced Learning Technologies (ICALT 08)
  • Best Paper Award at the German national e-Learning Fachtagung Informatik (DeLFI 07)

People

Dr. Jürgen Steimle
Oliver Brdiczka
Prof. Dr. Max Mühlhäuser