Cross-Company Evaluation of VR Solution

The first Cross-Company Evaluation (CCE) took place in the VR-Lab of Twente University. It was attended by representatives from the various companies involved in the project, as well as the REPAR research team. The primary purpose of the meeting was to see how well the results of the first case study (e.g. the resulting VR solution) can be translated to other design domains.

The session consisted of two parts.

  • The first 90 minutes were used to present the results of the case study carried out in Sub Project B. The presentation explained the various events in the case study (e.g. workshops, demonstrations, evaluations) that contributed to the final result, the ‘Virtual Printshop’. The Virtual Printshop is an interactive 3D virtual environment in which designers can conduct user studies in realistic use contexts (e.g. a printshop or office). After presenting a demonstration of this application, company representatives presented their experiences with using tools to create the Virtual Printshop. These presentations provided the audience (the other companies) with first-hand experiences with VR tools and gave clear insights in opportunities  and pitfalls. The presentations concluded with an outlook on a follow-up study, carried out in collaboration with sub project C.
  • In the second part of the session, the participating companies were asked to ‘translate’ the VR solution (as presented in the first part) into a useful solution for their own company or design domain. The purpose of this part of the session is to identify similarities and differences between how various companies could benefit from VR. It is the first step towards connecting VR technologies to specific design domains or companies. The participants were divided into company groups. Each group was also supported by case study representative to exchange experiences from the case study and the use of VR in general.

The groups were asked how they would translate the presented VR solution into something useful to their company, and present these results in short final presentations to the other groups.

A company representative presenting his experiences with VR tools

A company representative presenting his experiences with VR tools

Results

While the actual session data still needs to be analysed, the results of the final group presentations show that most of the groups are able to translate some of the aspects of the original VR solution into opportunities for their own domain. In addition to translating the application, the participants also identified opportunities and bottlenecks for re-using existing tools and skills already available within their company. Using the insights from the presentations, they were provided with an initial benchmark to better estimate their requirements (e.g. ‘we need something more/less realistic than what we’ve seen in the case study’) and connect them to their own situation (e.g. ‘we could use our current CAD tools for this part of the tool chain’). In general, the resulting presentations show that while a lot of 3D modelling skills could be re-used for VR applications, it remains difficult to properly define ‘behaviour’ and interaction of models and context.

Overall, the session has been a fruitful meeting for the companies as well as the researchers. The presentations by company representatives have been very useful in providing first-hand practical experiences with VR and triggered a lot of interaction between the attending companies. The results of the second part of the session will be further analysed, and hopefully contribute to identifying guidelines for successfully using VR across design domains.

Next Steps

On the short-term the focus is with the second Case Study, where we recently started the second case study of Sub Project B. On a side-track I will be providing technical support for a follow-up study with the first company, carried out within Sub Project C.

Exploring advanced VR modeling tools

Following the Do It Yourself Virtual Environments (DIYVE) workshop, where non-experts modeled their own virtual environments using SweetHome3D, the final step in the first case study of Sub Project B is to look at ‘Behaviour Modeling’ of virtual environments. By now we have seen that even non-experts are able to create realistic virtual use contexts. The next step, Behvaiour Modeling, is done by a more dedicated designer who is more experienced with programming and modeling (refered to as ‘prototyper’). Consequently we can try to use advanced tools to achieve higher levels of complexity, flexibility and interactivity in virtual environments.

Approach

Unlike the DIYVE workshop, where participants were guided through the tool and worked on an assignment during the workshop, the advanced modeling tool is introduced to the dedicated designer in a one-on-one meeting. It was decided to use Blender as a starting point for this part of the case study. Blender was also used for developing the VR demonstration projects, as well as the more advanced prototype shown during the Interactive Demonstration Session for the company. As such, it has prooven itself as a versatile tool for modeling and real-time virtual environments, it is an open piece of software, and well-known to the researcher.

The tool was introduced to the prototyper by starting with one of the virtual environments created during the DIYVE workshop. The virtual environment was imported to Blender, after which the tool was used to add ‘interactivity’. We decided to implement

  • A ‘first person shooter’ perspective (e.g. so you can walk around in the environment)
  • A working door, so the user can open and close a door, and
  • The ability to pick up virtual objects, move them and put them back

These exercises are considered feasible and relevant for behaviour modeling and are a good way to get to know the modeling tool. While working on these topics the prototyper noticed several positive and negative aspects of the tool, it’s user interface and the overall ‘way of working’. In general, it was concluded that Blender offers a lot of (and probably too much) functionality, but has quite a threshold for new users. Despite the experience of the prototyper in the field of programming, gaming and modelling, it was still difficult to grasp the modeling concepts that are used in Blender. Nevertheless, the above functionality could succesfully be added to the scene, with the exception of the third one.

Blender is used for behaviour modeling. Here one of the virtual environments created in the DIYVE workshop is imported.

Preliminary Conclusions

After the introductory session the prototyper worked with Blender individually over a period of several weeks (as a ‘friday afternoon’ activity). Spending more time on using the tool, the aforementioned conclusion was confirmed. The tool offers a lot of functionality, but fails to do this intuitively. Especially if such tools would be used once every other month or so, it is important to quickly (re)grasp the use of the tool rather than having to spend a week of training on it each time you use it.

There are several ways to address the challenges that emerged during this period of use. Firstly, the user interface of the tool could be adapted to the specific tasks of VR modeling (e.g. leave out unused functions, or hide them in deeper ‘UI layers’). The prototyper suggested to look at visual programming solutions; simple ‘if-then-else’ statements could be used to define the behaviour of a VR scene. Secondly, additional training could be provided to properly explain the prototyper the basics of the tool before diving into the complexity of VR modeling.

With plenty of solutions for any of these directions available, the real question is how do designers (or the company as a whole) treat the trade-off between investements in time and tools (e.g. training, specific UI’s) and the benefits of using VR they gain in return. In order to understand this trade-off, the company (and its designers) first need to experience the use of VR as prescribed during the case study. This experience should clarify the benefits of applying VR, and add insights about the use of tools (e.g. see where additional investements are required and/or justified).

DIY Virtual Environments Workshop

The Do It Yourself Virtual Environment workshop (DIYVE) is one of the last workshops held in the first case study of Sub Project B. In the previous meeting (Interactive Demonstration Session) we described a VR tool chain that integrates with the tool chain of the case study company. One of the critical elements of the proposed tool chain is the creation of virtual environments by untrained (i.e. no specific 3D modeling training) designers. The main aim of the DIYVE workshop is to determine how well these untrained designers are able to create virtual environments. The workshop particularly investigates the designer’s translation of input material (e.g. photos, a floorplan or from memory) into output (the virtual environment). Creating this virtual environment is facilitated by an existing software tool.

Session Outline

After an introductory presentation a fifteen minutes tutorial of SweetHome3D, the tool used in the workshop, was carried out. After this, the participants (4 people from the design department) started to work on individual assignments. Each assignment requires a specific approach for the environment modeling. The first two assignments are relatively structured; participants translate concrete input into a virtual environment. The other two assignments require a more creative approach and rely on memory and imagination rather than concrete input
materials.

The participants worked on their assignment for about 90 minutes. While working on the environments, questions about the software applications were answered every now and then, but apart from that the process was mostly autonomous. Near the end of the workshop the results were collected and presented on a beamer. Each participant presented a short ’walk-through’ through their virtual environment, depicting specific parts of the environment and explaining some of their design decisions (see figure 2(b)). After these presentations and brief discussions the participants filled out the evaluation form.

A participant presenting her virtual environment in a 3D walkthrough

A participant presenting her virtual environment in a 3D walkthrough

Results

It was shown that from a multi-disciplinary group of design department representatives all four participants were able to create a virtual environment based on photos, floorplans, memory or imagination. The authoring tool used in the workshop turned out to provide sufficient support for the participants to quickly define room parameters, add walls, floors and basic furniture. Furthermore, the limited list of available 3D objects was sufficient for basic furnishing of the room. While various detailed user interface issues were identified, for instance related to 2D/3D navigation and access to the 3D object library, the tool was generally found to be appropriate for the task.

A virtual environment created by a participant

A virtual environment created by a participant

An important finding is that all participants, regardless of their workshop assignment, at some point expressed the need for more accurate 3D models. While the participants were able to solve these issues (e.g. by using on-line resources or alternative models), they indicated the need for 1) the ability to do simple object modelling tasks within the authoring tool, or 2) the ability to modify the objects from the provided model library. This is interesting, because during the previous interactive demonstration session the participants indicated that detailed (i.e. 1:1 match with reality) modelling is not really required for these environments.

What’s Next?

The next step in the case study is to further embed the virtual environment modelling task in the company’s design process. During the DIYVE workshop this modelling task was well received by the designers, in the sense that they recognise the added value of using the environments, and see how it could fit their future tool chain. The 3D models resulting from the DIYVE workshop will be used as input for a behaviour modelling task. This task, also part of the envisioned VR tool chain, adds behaviour and interaction to the created environments.

VR Prototype Session

In the group workshop several applications of virtual reality were identified for Case Study 1. For instance, VR could be used as a communication/marketing application, or as a virtual use context for usability evaluations. While the workshop provided these insights, the details about each application were still quite vague. This is partly because the company participants have no prior experience with VR, which makes it difficult to say “I really need/want this type of technology” or “We need this type of virtual environment” without experiencing it first.

To allow participants to provide more concrete/founded feedback on the details of the application, an interactive demo session was organised, in which four functional VR ptototypes were introduced. The four prototypes, partly based on earlier versions from the first demo session, allowed the participants to experience different types of virtual environment (Mixed and fully virtual reality) and different levels of realism (visual, audio, interactions, etc.).

Results

The interactive demo session managed to achieve refinement of the VR application for the case study as well as a first step towards the tool selection for this application. With the results of the session it is now clear where and how to connect the VR tool chain to the tool chain currently used in the company.

 

Case Study 1: Workshop Results

Last week the workshop (explained here) took place. The aim of the workshop was to determine the direction for the development of a first VR tool prototype. On the whole the workshop is considered a succes because it achieved the goals that were set in the proposal. It has been an efficient and effective way to get concrete knowledge, expertise and expectations from a multidisciplinary group of participants. Participants used storyboards to express their ideas regarding VR applications in the company;

  1. First, four prepared animated storyboards were shown to inspire the participants
  2. Each participant was asked to create his/her own individual storyboard, based on the examples shown.
  3. In groups of 3, participants presented and discussed their storyboards
  4. Each group created one final storyboard by combining or choosing individual storyboards. These group storyboards were presented and discussed to wrap up the workshop

The storyboards created during the workshop made clear to both the researcher and the participants how VR could be used in the design process. After analysing the outcomes of the workshop, a concrete functional description of the VR design tool was created. This tool fits both the expectations of the company (expressed during the workshop) and addresses the main research questions relevant for the research project.

Participants working on the individual and group storyboards

One of the group storyboards

Case Study 1: Workshop Structure

Following the kick-off meeting, the next step in the first case study is a workshop. The aim of this workshop is illustrated below.

Aim of the workshop: exchange information, knowledge and experiences between designers and researcher.

The illustration shows the designer’s perspective and the researcher’s perspective. All the sessions up to now were very useful in providing these two parties with insights in to each other’s domain; the company internships allowed the researcher to look around in the designers’ context, the VR demo session provided designers with an overview of VR technologies. The aim of the workshop is to let both sources of knowledge, information and experience come together and discuss explicit combinations of design tasks and VR technologies, in order to identify VR tool opportunities.

To do this, I decided to follow a participatory design approach, in which I am the designer (of the future VR design tool) and the designers are the users (end users of the VR design tool). As you may know, participatory design actively involves end users in the (conceptual) design process. For the upcoming workshop the plan is to use a method described by Iacucci et al (2000) and more recently by Urnes et al (2002), involving a miniature environment in which users ‘act out’ scenarios using puppets, objects and other artefacts.

The animated storyboards presented in a previous post will be used as examples to show my perspective; how do I envision VR to be useful for design tasks? The explicit visualisation method makes it more easy to get positive or negative feedback. Similarly, the workshop participants will be asked to do the same from their perspective; act out the  design tasks that could be supported by VR tools. By discussing the resulting scenarios and performances, more detailed requirements and expectations are expected to be found.

Case Study 1: Idea Generation

The design reviews that were discussed during the Kick-off meeting will be used as (on of the) inputs for the group workshop. The group workshop aims to get into more detailed requirements and expectations for VR design tools. It should be a follow-up or more detailed version of the VR demo session held earlier. To prepare the examples for this workshop I started to document some of the ideas. The sketches shown here are used to make some more refined representations of the tool examples.

Continue reading

Case Study 1: Kick-Off

The first step in the first phase of the first Case Study! As explained in the Approach and Planning pages, the case study starts with “Phase 1: Specify Applications”. Phase 1 is split into two meetings, the Kick-Off meeting and a Workshop. During the kick-off meeting, we’ll make a first selection of design reviews in which we expect VR tools could be useful. The reviews are to be characterized in terms of who, where, why, when, etcetera. Also, in preparation for the workshop, the aim is to get some example material illustrating the inputs and outputs of such reviews.