On Thursday February 28 two test sessions for the third case study of sub project B took place. In previous workshops and meetings, a VR application was defined for case study 3 that aims to “facilitate the identification, validation and evaluation of use issues in the early stages of the product development process of the company”. The prototype of this VR application has been developed over the last two months, and tested in the test session reported here. The purpose of this test session is to verify the usefulness of the application, and refine requirements with respect to its implementation.

Application overview

The “Virtual Annotation” application supports three early stage design tasks. The starting point of this process is a preliminary definition of a product concept, for instance an initial product description or a product opportunity identified by marketing activities.

1. Create a coarse product model. The model can be made using existing CAD components and/or primitive shapes
2. Review and annotate the product model in a realistic 3D environment
3. Document and share the knowledge gathered during the creation and annotation stages

The first step is to create a 3D representation of this product concept. This step can be achieved with existing tools, but it is also supported by the VR prototype application. Given this model, the second step consists of a collaborative product review facilitated by the VR application. The application allows a design team (ideally including external stakeholders) to review, discuss and modify the 3D representation of the product and annotate specific parts of it through text or free-hand drawings. The third step that is supported by the application is the documentation of all annotations, discussions and feedback generated during the collaborative session. The application automatically collects this data and presents it in a report.

The test session focusses on step 2 and 3. In an earlier session, engineers from the company were asked to create a coarse 3d model of the product concept that is featured in the use case.

Test setup

The VR application consists of a 3d interactive virtual environment projected on a large screen. The environment can be navigated (walk-through or free camera) and modified in real-time.

The main screen showing the virtual environment (pixelised here) and the list of annotations on the left.

One of the key features of the application is the support for text and sketch annotations, which are stored in an annotation database. The application targets group collaboration, and therefore supports input from multiple remote ‘client devices’ (tablets, smartphones or laptops) in addition to being operated by a dedicated moderator.

The sketch application allows participants to sketch on top of the virtual environment

The text app allows participants to add text notes to specific parts of the virtual environment

Two multi-disciplinary groups of engineers, sales representatives and assembly engineers from the company participated in two sessions. Both groups were introduced to the use case topic (a specific product concept currently in development within the company) and asked to discuss current and expected issues related to the anticipated use of the concept. In addition to this primary task, the participants were also asked to try out and compare two different forms of interaction with the VR application.

1. In the first part of the group session, participants engage in a traditional group discussion. A dedicated moderator (in this case the researcher) adds all issues, remarks and questions to the virtual environment. The moderator is not involved in the discussion itself.
2. In the second part of the group session, participants use individual input devices to add text or sketches to the virtual environment. In the test session we used iPads, but smartphones and laptops are also supported by the client app.
3. In the final part of the session participants were free to choose their input method; either through the moderator or by using their own input devices.

Each session (both sessions used this format) took 1 hour and 15 minutes, including an introductory presentation and filling out a post-session evaluation form.

Results

Both sub sessions managed to create a useful and detailed discussion about the product concept. Compared to application tests in the earlier case studies (1 and 2), there was a significant difference in how the participants engaged in the discussion; this time participants were not obstructed or distracted by the VR application and managed to reach a deeper layer of discussion (about the product concept rather than the VR application).

The setup as used in the test sessions, showing the project leader in front of the virtual environment, other participants are seated close to the screen.

After the session however it was still possible to reflect on the application, also supported by the post-session evaluation forms. As a result, both the researcher and the participants had quite an effective meeting.

Some results related to the VR application are worth mentioning;

• It was noted that the virtual environment or the virtual objects were almost never modified or even moved (which was supported by the application and briefly demonstrated prior to the discussion). Apart from adding some objects to the scene, the scene remained relatively static throughout the sessions.
• There was a difference in how both groups responded to the introduction of individual input devices; the first group was quite distracted, leading to a less active group discussion. To prevent this from happening (to much) in the second session, a more explicit request to also keep the discussion going was made prior to handing out the iPads.
• There was a clear preference for the moderated discussion form, though several participants indicated (verbally or on the evaluation form) that both forms are useful, either combined in one session or used individually.
• Based on what was produced during the session, text annotations are more popular than free-hand sketches. While this can partly be due to the prototype nature of the sketch apps, even in the moderated session participants did not often request a sketch to be made.