Tag Archives: cross company evaluation

Cross Company Evaluation 2

Posted on by
Reply

The second Cross-Company Evaluation (CCE2) took place on Wednesday September 5, 2012 in the VR-Lab of Twente University. It was attended by representatives from the three primary industrial partners as well as two representatives from AgentschapNL.  Just like CCE1, this session has been an important and successful milestone in the research of Sub Project B. The researcher would once again like to thank all the participants for attending the session.

Session Structure

The session consisted of two parts.

  1. Presentation – In the first part of the session, the researcher presented a summary of the second case study covering the main events in the case study, such as the group workshop and the test sessions. The presentation also introduced the topic of the case study, namely Virtual Personas. Personas are detailed (written) descriptions of a user, and intended to inform designers about who they are designing for. Virtual personas are digital representations of personas (in 3d avatars) that can act out use scenarios in a fully virtual world.
  2. Use Case – A prototype of this application that had been developed during the case study (see figures) was demonstrated to the audience. After this quick introduction, the participants were asked to carry out a simple use case with the tool, featuring a design assignment similar to the one carried out during the ‘real’ case study. This allowed participants to experience what it is like to use the Virtual Persona tool.

Participants operating the Virtual Persona tool

Review

The topic of ‘personas’ turned out to be quite interesting for all parties involved. One of the other two companies has been using personas quite successfully and shared insights about how they managed to do so. Other participants, who had only limited experience with personas (but are interested in using them in their design process) benefit from hearing these experiences.

The Virtual Persona tool in the VR lab

During the interactive part of the session the two non-case study companies participated in a mini use case to experience the tool themselves. Both groups were joined by people from the case study company, who helped out with getting to know the tool and with providing additional insights from their own use case. Most of the time was spent on understanding how to control the avatars and the virtual world. As a result, the mini use cases were somewhat more limited in terms of generating ideas, but gave the participants an idea of what it is like to use the Virtual Persona tool.

After the interactive part of the session, the participating groups were asked to describe how their company (or product design process) could benefit from Virtual Personas; what benefits are anticipated, would the application be similar to the one developed during the case study, and what kind of tools would be available to realise this application? The results of these brainstorms were shared during a group discussion in which all companies participated.

Preliminary Findings

The following is concluded from the CCE2 session:

  • Personas (non-virtual) were well received among the participants; positive and negative experiences with personas were shared even before discussing virtual personas.
  • Designers with prior experience with ‘traditional’ personas identified several advantages of virtual personas; it helps with focussing designers on specific personas, provides a richer visualisation and allows for personas to be placed in various ‘situations’ (e.g. different rooms, locations, events, etc.)
  • The tool (in particular the avatar controls) requires sufficient training before virtual personas can really be used in a session. This level of training was achieved during the case study (as more time was available), but not during the mini use-cases carried out during the CCE.

In addition to the above preliminary findings, insights will be derived by reviewing session recordings and the forms that were filled out during the session.

 

Cross-Company Evaluation of VR Solution

Posted on by
1

The first Cross-Company Evaluation (CCE) took place in the VR-Lab of Twente University. It was attended by representatives from the various companies involved in the project, as well as the REPAR research team. The primary purpose of the meeting was to see how well the results of the first case study (e.g. the resulting VR solution) can be translated to other design domains.

The session consisted of two parts.

  • The first 90 minutes were used to present the results of the case study carried out in Sub Project B. The presentation explained the various events in the case study (e.g. workshops, demonstrations, evaluations) that contributed to the final result, the ‘Virtual Printshop’. The Virtual Printshop is an interactive 3D virtual environment in which designers can conduct user studies in realistic use contexts (e.g. a printshop or office). After presenting a demonstration of this application, company representatives presented their experiences with using tools to create the Virtual Printshop. These presentations provided the audience (the other companies) with first-hand experiences with VR tools and gave clear insights in opportunities  and pitfalls. The presentations concluded with an outlook on a follow-up study, carried out in collaboration with sub project C.
  • In the second part of the session, the participating companies were asked to ‘translate’ the VR solution (as presented in the first part) into a useful solution for their own company or design domain. The purpose of this part of the session is to identify similarities and differences between how various companies could benefit from VR. It is the first step towards connecting VR technologies to specific design domains or companies. The participants were divided into company groups. Each group was also supported by case study representative to exchange experiences from the case study and the use of VR in general.

The groups were asked how they would translate the presented VR solution into something useful to their company, and present these results in short final presentations to the other groups.

A company representative presenting his experiences with VR tools

A company representative presenting his experiences with VR tools

Results

While the actual session data still needs to be analysed, the results of the final group presentations show that most of the groups are able to translate some of the aspects of the original VR solution into opportunities for their own domain. In addition to translating the application, the participants also identified opportunities and bottlenecks for re-using existing tools and skills already available within their company. Using the insights from the presentations, they were provided with an initial benchmark to better estimate their requirements (e.g. ‘we need something more/less realistic than what we’ve seen in the case study’) and connect them to their own situation (e.g. ‘we could use our current CAD tools for this part of the tool chain’). In general, the resulting presentations show that while a lot of 3D modelling skills could be re-used for VR applications, it remains difficult to properly define ‘behaviour’ and interaction of models and context.

Overall, the session has been a fruitful meeting for the companies as well as the researchers. The presentations by company representatives have been very useful in providing first-hand practical experiences with VR and triggered a lot of interaction between the attending companies. The results of the second part of the session will be further analysed, and hopefully contribute to identifying guidelines for successfully using VR across design domains.

Next Steps

On the short-term the focus is with the second Case Study, where we recently started the second case study of Sub Project B. On a side-track I will be providing technical support for a follow-up study with the first company, carried out within Sub Project C.