| 1. |
- Backman, Anders, et al.
(författare)
-
easyADL – Wearable Support System for Independent Life despite Dementia
- 2006
-
Ingår i: ACM CHI 2006 Workshop onDesigning Technology for People with Cognitive Impairments.
-
Konferensbidrag (refereegranskat)abstract
- This position paper outlines the easyADL project, a two-year project investigating the possibility of using wearable technology to assist people suffering the dementia disease in performing Activities of Daily Living (ADL). An introduction to the egocentric interaction modeling framework is provided and the virtual reality based development methodology is discusssed.
|
|
| 2. |
- Kalpouzos, Grégoria, et al.
(författare)
-
Neurocognitive systems related to real-world prospective memory.
- 2010
-
Ingår i: PLOS ONE. - San Francisco : Public library of Science. - 1932-6203. ; 5:10, s. e13304-
-
Tidskriftsartikel (refereegranskat)abstract
- BACKGROUND: Prospective memory (PM) denotes the ability to remember to perform actions in the future. It has been argued that standard laboratory paradigms fail to capture core aspects of PM.METHODOLOGY/PRINCIPAL FINDINGS: We combined functional MRI, virtual reality, eye-tracking and verbal reports to explore the dynamic allocation of neurocognitive processes during a naturalistic PM task where individuals performed errands in a realistic model of their residential town. Based on eye movement data and verbal reports, we modeled PM as an iterative loop of five sustained and transient phases: intention maintenance before target detection (TD), TD, intention maintenance after TD, action, and switching, the latter representing the activation of a new intention in mind. The fMRI analyses revealed continuous engagement of a top-down fronto-parietal network throughout the entire task, likely subserving goal maintenance in mind. In addition, a shift was observed from a perceptual (occipital) system while searching for places to go, to a mnemonic (temporo-parietal, fronto-hippocampal) system for remembering what actions to perform after TD. Updating of the top-down fronto-parietal network occurred at both TD and switching, the latter likely also being characterized by frontopolar activity.CONCLUSION/SIGNIFICANCE: Taken together, these findings show how brain systems complementary interact during real-world PM, and support a more complete model of PM that can be applied to naturalistic PM tasks and that we named PROspective MEmory DYnamic (PROMEDY) model because of its dynamics on both multi-phase iteration and the interactions of distinct neurocognitive networks.
|
|
| 3. |
- Larsson, Clara, et al.
(författare)
-
Visualizing Invisible Environmental Data in VR: Development and Implementation of Design Concepts for Communicating Urban Air Quality in a Virtual City Model
- 2023
-
Ingår i: Communications in Computer and Information Science Volume 1819 CCIS, Pages 253 - 267. - : Springer. - 1865-0929 .- 1865-0937. ; , s. 253-267
-
Konferensbidrag (refereegranskat)abstract
- As cities continue to grow, the desire to combine densification with sustainability and greenery may present a challenge to air quality, resulting from reduced ventilation caused by dense buildings and vegetation. To support the careful urban planning required, effective and interactive tools that can visualize and communicate information about air quality to stakeholders are essential. In a transdisciplinary research project aiming to explore such visualizations a prototype pedagogical virtual reality tool was developed, allowing users to explore the impact of aspects of the built environment upon urban air quality. The tool was evaluated with adolescents in upper secondary school through interviews and observations, as well as with the general public through a questionnaire study. This paper provides insights, potential solutions, and initial assessments relevant to data visualization in 3D and immersive analytics in urban planning and stakeholder communication. Identified challenges include difficulties with color association and data distinguishability, and as well as tool complexity relating to the many features requested by experts involved in a transdisciplinary project.
|
|
| 4. |
- Lorenz, M., et al.
(författare)
-
Presence and User Experience in a Virtual Environment under the Influence of Ethanol: An Explorative Study
- 2018
-
Ingår i: Scientific Reports. - : Springer Science and Business Media LLC. - 2045-2322. ; 8
-
Tidskriftsartikel (refereegranskat)abstract
- Virtual Reality (VR) is used for a variety of applications ranging from entertainment to psychological medicine. VR has been demonstrated to influence higher order cognitive functions and cortical plasticity, with implications on phobia and stroke treatment. An integral part for successful VR is a high sense of presence - a feeling of 'being there' in the virtual scenario. The underlying cognitive and perceptive functions causing presence in VR scenarios are however not completely known. It is evident that the brain function is influenced by drugs, such as ethanol, potentially confounding cortical plasticity, also in VR. As ethanol is ubiquitous and forms part of daily life, understanding the effects of ethanol on presence and user experience, the attitudes and emotions about using VR applications, is important. This exploratory study aims at contributing towards an understanding of how low-dose ethanol intake influences presence, user experience and their relationship in a validated VR context. It was found that low-level ethanol consumption did influence presence and user experience, but on a minimal level. In contrast, correlations between presence and user experience were strongly influenced by low-dose ethanol. Ethanol consumption may consequently alter cognitive and perceptive functions related to the connections between presence and user experience.
|
|
| 5. |
- Mubin, O., et al.
(författare)
-
Towards an Agenda for Sci-Fi Inspired HCI Research
- 2016
-
Ingår i: 13th International Conference on Advances in Computer Entertainment Technology (Ace 2016). - New York, NY, USA : ACM. - 9781450347730
-
Konferensbidrag (refereegranskat)abstract
- Science fiction media has had a long lasting influence on the progression of interactive technology, however recently contradictions are emerging in the development of the two disciplines. Therefore, in this exploratory position paper we report on the insights attained through a day long workshop amongst scientists and researchers on how the collaboration between science fiction and Human Computer Interaction (HCI) can be advanced. Discussions in the workshop focused on detailing the relationship between HCI and science fiction. In conclusion, as our main contribution an action plan and agenda is presented for facilitating deeper influences amongst the two disciplines.
|
|
| 6. |
- Sjölie, Daniel, 1977-
(författare)
-
Adaptive games for cognitive training : Lessons measuring arousal with EEG
- 2013
-
Konferensbidrag (refereegranskat)abstract
- Computerized cognitive training is an area where the use of computer games technology and methods has a great potential, for example, to address cognitive decline in an aging population. Adaptive games, in particular, are of great interest as the level of training has often been suggested as important for efficient training. An important part of any adaptive application is measuring and interpreting whatever the game should adapt to. In this paper we describe our work on using the Emotiv Epoc commercial EEG headset in order to measure and adapt to the user's level of arousal in two different applications. The first application is an adaptation of a classic cognitive training task (N-back) using game technologies to create a dynamic and (relatively) realistic version in a 3d-environment. The second application is a simple version of the classic space invaders game. In both applications EEG measurements recorded during initial training are used in a later phase to adapt the difficulty of the game automatically. While we managed to get this setup to work to a limited degree for some individuals, we failed to create a system where this method worked reliably across subjects and trials. In this paper, we describe what we tried, what worked, and some of the lessons we learned.
|
|
| 7. |
- Sjölie, Daniel, 1977-, et al.
(författare)
-
Capturing neural correlates of disrupted presence ina naturalistic virtual environment
- 2013
-
Rapport (övrigt vetenskapligt/konstnärligt)abstract
- The concept of presence is commonly related to whether or not a user feels, acts, and reacts as if he/she were in a real familiar environment when using a virtual reality (VR) application. Understanding the neural correlates of presence may provide a foundation for objective measurements of presence and important constraints for theoretical explanations of presence. Discussions about the neural basis for presence are relatively common, but brain imaging has rarely been applied to investigating this issue. Previous studies have focused on detecting average differences between conditions that correlate with differences in reported presence. In this study we focused on breaks in presence and associated periods of disrupted presence as an important complement to previous work. Specifically, we measured brain activity with functional magnetic resonance imaging (fMRI) during execution of an everyday task in a naturalistic virtual environment (VE). Time periods of disrupted presence were identified by subject reports indicating something strange in the current environment, interpreted as a violation of expectations related to the sense of presence. Disrupted presence was associated with increased activity in the frontopolar cortex (FPC), lateral occipito-temporal cortex (LOTC), the temporal poles (TP), and the posterior superior temporal cortex (pSTC). We relate these areas to integration of key aspects of a presence experience, relating the (changing) situation to management of task and goals (FPC), interpretation of visual input (LOTC), emotional evaluation of the context (TP) and possible interactions (pSTC). Modulation of the activity level in these brain areas is consistent with an interpretation of disrupted presence as a re-evaluation of key aspects of a subjective mental reality, updating the synchronization with the virtual environment as previous predictions fail. Such a subjective mental reality may also be related to a self-centered type of mentalization, providing a link to accounts of presence building on the self.
|
|
| 8. |
- Sjölie, Daniel
(författare)
-
Cognitive Training and the Need for Reality-Based Brain-Computer Interaction: Theoretical Background and Potential Applications
-
Annan publikation (övrigt vetenskapligt/konstnärligt)abstract
- Computer-aided cognitive training has the potential to be an important tool in the increasingly important fight against dementia and cognitive decline. In order for this potential to be realized a number of challenges must be addressed. This paper presents an overview of the current state of related research with a focus on theoretical results that motivate the use of realistic and adaptive computer applications to address these challenges. This includes consideration of previous research on cognitive training within cognitive neuroscience, recent developments within human-computer interaction (HCI) and previous work relating to brain activity in virtual reality environments. The increasing popularity of reality-based interaction (RBI) and brain-computer interfaces (BCIs) is discussed and the unified concept of reality-based brain-computer interaction (RBBCI) is presented to capture the principles needed for the development of realistic and adaptive systems for cognitive training and to serve as a cornerstone concept for cross-disciplinary development.
|
|
| 9. |
- Sjölie, Daniel, et al.
(författare)
-
Effects of interactivity and 3D-motion on mental rotation brain activity in an immersive virtual environment
- 2010
-
Ingår i: Proceedings of the 28th international conference on Human factors in computing systems. - New York, NY, USA : Association for Computing Machinery (ACM). - 9781605589299 ; , s. 869-878
-
Konferensbidrag (refereegranskat)abstract
- The combination of virtual reality (VR) and brain measurements is a promising development of HCI, but the maturation of this paradigm requires more knowledge about how brain activity is influenced by parameters of VR applications. To this end we investigate the influence of two prominent VR parameters, 3d-motion and interactivity, while brain activity is measured for a mental rotation task, using functional MRI (fMRI). A mental rotation network of brain areas is identified, matching previous results. The addition of interactivity increases the activation in core areas of this network, with more profound effects in frontal and preparatory motor areas. The increases from 3d-motion are restricted to primarily visual areas. We relate these effects to emerging theories of cognition and potential applications for brain-computer interfaces (BCIs). Our results demonstrate one way to provoke increased activity in task-relevant areas, making it easier to detect and use for adaptation and development of HCI.
|
|
| 10. |
- Sjölie, Daniel, 1977-
(författare)
-
Human brains and virtual realities : Computer-generated presence in theory and practice
- 2013
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- A combined view of the human brain and computer-generated virtual realities is motivated by recent developments in cognitive neuroscience and human-computer interaction (HCI). The emergence of new theories of human brain function, together with an increasing use of realistic human-computer interaction, give reason to believe that a better understanding of the relationship between human brains and virtual realities is both possible and valuable. The concept of “presence”, described as the subjective feeling of being in a place that feels real, can serve as a cornerstone concept in the development of such an understanding, as computer-generated presence is tightly related to how human brains work in virtual realities.In this thesis, presence is related both to theoretical discussions rooted in theories of human brain function, and to measurements of brain activity during realistic interaction. The practical implications of such results are further developed by considering potential applications. This includes the development and evaluation of a prototype application, motivated by presented principles.The theoretical conception of presence in this thesis relies on general principles of brain function, and describes presence as a general cognitive function, not specifically related to virtual realities. Virtual reality (VR) is an excellent technology for investigating and taking advantage of all aspects of presence, but a more general interpretation allows the same principles to be applied to a wide range of applications.Functional magnetic resonance imaging (fMRI) was used to study the working human brain in VR. Such data can inform and constrain further discussion about presence. Using two different experimental designs we have investigated both the effect of basic aspects of VR interaction, as well as the neural correlates of disrupted presence in a naturalistic environment.Reality-based brain-computer interaction (RBBCI) is suggested as a concept for summarizing the motivations for, and the context of, applications building on an understanding of human brains in virtual realities. The RBBCI prototype application we developed did not achieve the set goals, but much remains to be investigated and lessons from our evaluation point to possible ways forward. A developed use of methods and techniques from computer gaming is of particular interest.
|
|
| 11. |
- Sjölie, Daniel, 1977-, et al.
(författare)
-
Integration of Research on Immersive Learning Environments and Education in Welding
- 2024
-
Ingår i: Sustainable Production through Advanced Manufacturing, Intelligent Automation and Work Integrated Learning. - : IOS Press. - 9781643685106 - 9781643685113 ; , s. 660-671
-
Bokkapitel (refereegranskat)abstract
- The integration of virtual reality (VR) laboratories into welding education presents an array of potential advantages. It can be used at campus or in distance, and it offers an alternative when access to traditional laboratories is challenging. The economic benefits, including savings on material preparation and energy costs, along with the environmental, health and safety advantages of mitigating exposure to welding fumes, arc radiation, and electrical hazards, add further value and contribute to sustainability in welding education. The work presented here is an example of the integration of education in the areas of welding and informatics and research on immersive learning. A multidisciplinary team worked on the development of an immersive learning environment, including virtual laboratory areas for welding processes as well as for microstructural inspection of welds.During the project, this learning environment, and the contained virtual laboratories, have been implemented by the researchers with the support from IT students, and tested, and improved with the feedback provided by students in welding technology, materials science, and manufacturing courses. Overall, more than twenty students from Informatics have been involved throughout the project, resulting in five bachelor theses, three master theses, three course projects in Immersive computing, and two course projects focusing on web development. The involvement of IT students has not only supported the development of the virtual learning environment, but it has also created new avenues for future research and developments in immersive computing.
|
|
| 12. |
- Sjölie, Daniel
(författare)
-
Measuring presence in the simulating brain
- 2014
-
Ingår i: Interacting with Presence: HCI and the sense of presence in computer-mediated environments, Editors: Giuseppe Riva, John Waterworth, Dianne Murray. - : De Gruyter Open. - 9783110409697 ; , s. 46-56
-
Bokkapitel (refereegranskat)abstract
- A description of the brain as an organ for simulating its environment can inform and illuminate a discussion about brain measurements in connection with the sense of presence. Developing theories of how mental simulations and continuous predictions underlie brain function have implications for how presence can, or cannot, be associated with brain measurements. Here, the simulating brain is accepted as a working hypothesis. The resulting implications are discussed and briefly related to how brain measurements have been used to investigate presence in previous studies.
|
|
| 13. |
- Sjölie, Daniel, et al.
(författare)
-
Mind the brain : The Potential of Basic Principles for Brain Function and Interaction
- 2013
-
Rapport (övrigt vetenskapligt/konstnärligt)abstract
- The prevalence and complexity of human-computer interaction makes a general understanding of human cognition important in design and development. Knowledge of some basic, relatively simple, principles for human brain function can significantly help such understanding in the interdisciplinary field of research and development Human-Computer Interaction (HCI) where no one can be an expert at everything. This paper explains a few such principles, relates them to human-computer interaction, and illustrates their potential. Most of these ideas are not new, but wider appreciation of the potential power of basic principles is only recently emerging as a result of developments within cognitive neuroscience and information theory. The starting point in this paper is the concept of mental simulation. Important and useful properties of mental simulations are explained using basic principles such as the free-energy principle. These concepts and their properties are further related to HCI by drawing on similarities to the theoretical framework of activity theory. Activity theory is particularly helpful to relate simple but abstract principles to real world applications and larger contexts. Established use of activity theory as a theoretical framework for HCI also exemplifies how theory may benefit HCI in general. Briefly, two basic principles that permeate this perspective are: the need for new skills and knowledge to build upon and fit into what is already there (grounding) and the importance of predictions and prediction errors (simulation).
|
|
| 14. |
- Sjölie, Daniel, 1977-, et al.
(författare)
-
Mind tricks for presence
- 2019
-
Ingår i: ACM International Conference Proceeding Series. - New York, NY, USA : ACM Publications. - 9781450372176
-
Konferensbidrag (refereegranskat)abstract
- As virtual and mixed reality (VR/MR) technology moves steadily towards general availability accessible descriptions of the surrounding theory is desirable. An initial focus on high-level concepts can provide common language for diverse teams, including artists, designers and engineers, helping them to quickly get a sense of basic principles and gain a familiarity with related research for further study. The concepts of synchronized reality and grounded simulation are introduced as helpful starting points for thinking about the design and development of mixed reality systems with optimal presence. This paper provides case studies where recent commercial VR applications are analyzed with the proposed principles in mind, in an attempt to illustrate to developers how to think about design of mixed reality games for optimal presence. © 2019 ACM.
|
|
| 15. |
- Sjölie, Daniel, et al.
(författare)
-
Neural correlates of disrupted presence: strange disruptions in a naturalistic virtual environment
- 2014
-
Ingår i: ICAT-EGVE 2014. - : Eurographics Association. ; , s. 21-28
-
Konferensbidrag (refereegranskat)abstract
- The concept of presence is commonly related to whether or not a user feels, acts, and reacts as if he/she were in a real familiar environment when using a virtual reality (VR) application. Understanding the neural correlates of presence may provide a foundation for objective measurements and important constraints for theoretical explanations. Here, we focus on the importance of expectations and their violation in several theoretical descriptions of presence in order to investigate neural correlates of disrupted presence. Functional magnetic resonance imaging (fMRI) was used to measure brain activity during execution of everyday tasks in a naturalistic virtual environment (VE). Time periods with disrupted presence were identified using subject reports of strange situations, interpreted as violations of expectations related to the sense of presence. Disrupted presence was associated with increased activity in the frontopolar cortex (FPC), lateral occipito-temporal cortex (LOTC), the temporal poles (TP), and the posterior superior temporal cortex (pSTC). We relate these areas to integration of key aspects of a presence experience, relating the (changing) situation to management of task and goals (FPC), interpretation of visual input (LOTC), emotional evaluation of the context (TP) and possible interactions (pSTC). These results are consistent with an interpretation of disrupted presence as a re-evaluation of key aspects of a subjective mental reality, updating the synchronization with the virtual environment as previous predictions fail. Such a subjective mental reality may also be related to a self-centered type of mentalization, providing a link to accounts of presence building on the self.
|
|
| 16. |
- Sjölie, Daniel, 1977-
(författare)
-
Practical Mixing of Virtual and Real
- 2019
-
Ingår i: VILÄR 5-6 december 2019, University West, Trollhättan. - Trollhättan : University West. - 9789188847430 - 9789188847447 ; , s. 19-20
-
Konferensbidrag (övrigt vetenskapligt/konstnärligt)abstract
- Virtual reality (VR) has the potential to provide office working environments with essentially limitless customizations, including screens, whiteboards or tables of any size and number, and the ability to store and switch between environments. Having an arbitrarily furnished office space for each project enables improved workflows as activities and trains of thought can be resumed based on the rich state of the surrounding environment, saved without effort from the last session. Still, adoption of VR in office working environments remains limited and the issue of how workers may be supported in this development is of interest.A likely reason for the limited adoption is the isolation from the real world that is inherent to traditional head-mounted VR. Initially, augmented reality (AR) seems well positioned to address this limitation, combining a primary view of the real world with augmented virtual screens, whiteboards, etc, but pure AR has limitations of its own. With AR, it becomes difficult or impossible to hide away distractions of the real world and the potential for immersive AR (not restricted to a narrow field of view) is very limited with current technology. What we suggest, and want to bring up for discussion in this session, is that we should strive to enable deliberate mixing of the real and virtual, freely picking which parts to incorporate into our practical working environment.We have experimented with techniques to create interactive and pragmatic mixed reality (MR) where objects, furniture, and people from the real world may be mixed into the virtual environment as desired. Using a Kinect depth camera we captured a view of the real world in full 3D and integrated select parts into a virtual environment. The goal is to give the user full control of this space and make it possible to draw on the full advantages of VR while simultaneously feeling safe and aware of your physical surroundings.
|
|
| 17. |
- Sjölie, Daniel, 1977, et al.
(författare)
-
PRACTICAL USE OF OFF-THE-SHELF VIRTUAL REALITY FOR DISTANCE EDUCATION - A LOOK AT THE FEASIBILITY OF NON-EXPERT USE
- 2020
-
Ingår i: Learning and Teaching. - : IATED. - 1755-2273. ; 2020
-
Konferensbidrag (refereegranskat)abstract
- As the potential value of online learning and distance education becomes increasingly clear (considering, e.g., global health and climate change) we are motivated to push for practical use of emerging technologies at an accelerated pace, to further facilitate rich and flexible distance education. In the current scenario, the changes forced into the classic educational system need to be executed quickly. While the pedagogical value of lectures has been often questioned, it remains a common method of instruction, making them relevant to investigate within digitalisation. Virtual Reality (VR) affords valuable embodied experiences and is currently at a point where it is within reach for non-experts, but the threshold may still be perceived as too high. By focusing on the use of off-the-shelf hardware and software to give virtual lectures, a larger number of educators can start experimenting within their comfort zone. The purpose of the current paper is to contribute to the acceleration of this process by describing challenges encountered in such an attempt to quickly employ readily available VR technology to give a lecture in VR. Is it possible for educators without previous expertise in VR to start using this technology now? What factors can be considered to make the experience positive to both educators and students? The setup of the VR lecture in this case study had the lecturer entering a virtual environment remotely (from another city) using the free VR application Bigscreen VR with students and three observing co-teachers entering the same environment being co-located in one physical room. The lecturer used the Oculus Rift S headset and a VR-capable laptop while the students used the simpler Oculus Go headsets. A predetermined view of the lecturer in the VR-environment was also shown via a projector, as a general fallback. The study was performed as an action research intervention, and the results were documented with ethnographic observations and a focus group. Compared to established tools for distance education, such as video meetings, the preparation time was significantly longer but the VR setup with off-the-shelf VR hardware and software worked relatively well for the lecture itself. The primary problems encountered concerned the student headsets and the preparation of and interaction with the students in the physical room. In addition to practical issues with managing a larger number of headsets (there were 5 headsets for the students) there were significant problems in getting the audio to work well as well as technical problems with Internet access. During the lecture more behavioral and social issues came to the forefront, for example students being uncertain about how to behave in the unfamiliar environment. Out of five participating students, two experienced discomfort, but were uncertain about how to act and whether it was OK to take off their headsets or not. Some issues encountered here may be avoided or minimised by raising awareness beforehand and additional preparation. Based on the present study, technical and ethical recommendations are given for which issues should be prioritized and how they may be dealt with, regardless of the educators level of expertise, to be able to successfully conduct a VR lecture.
|
|
| 18. |
- Sjölie, Daniel
(författare)
-
Presence and general principles of brain function
- 2012
-
Ingår i: Interacting with computers. - : Oxford University Press (OUP). - 0953-5438 .- 1873-7951. ; 24:4 SI, s. 193-202
-
Tidskriftsartikel (refereegranskat)abstract
- Recent developments in general theories of cognition and brain function make it possible to consider the concept of presence from a new perspective, based in general principles of brain function. The importance of interaction with reality for the development and function of the brain and human cognition is increasingly emphasized. The brain is explained as implementing a generative model of the current environment. Whether this environment is real or virtual does not matter. Mental simulations are created for whatever one interacts with, when possible. This view provides a basis for relating human experiences in virtual environments to several theories that explain cognition and brain function on many levels, from ultimate evolutionary motivations to plausible neural implementations. The purpose of this paper is not to provide yet another definition of presence but to suggest explanations of phenomena commonly related to presence, with a basis in general principles of brain function. Such principles are employed to explain how, and why, interaction with our environment, and internalization of objects and tools therein, play an essential role in human cognition. This provides a rich basis for further analysis of how central aspects of presence, such as breaks in presence or the perceptual illusion of non-mediation, may work on a fundamental level. More general descriptions of such phenomena have advantages such as being easier to relate to new contexts and technologies, and opening up for additional inspiration and confirmation from other disciplines such as cognitive neuroscience. In addition to an account of general principles for brain function and a discussion about the concept of presence in light of these, this paper also relates this discussion to a number of previous accounts of presence, and to practical implications and applications for interaction design.
|
|
| 19. |
- Sjölie, Daniel
(författare)
-
Realistic and adaptive cognitive training using virtual characters
- 2014
-
Ingår i: The 10th international conference on disability, virtual reality and associated technologies (ICDVRAT 2014), 2-4 September, 2014, Gothenburg, Sweden. - 9780704915466 ; , s. 385-388
-
Konferensbidrag (refereegranskat)abstract
- Computer-aided cognitive training has the potential to be an important tool in the fight against dementia and cognitive decline but many challenges remain. This paper presents an example of how realistic and adaptive training may address these challenges. Virtual characters were used as stimuli in a dual n-back working memory task in a realistic 3d-environment. Support for continuous adaptation was a priority, including adaption based on affective states such as arousal.
|
|
| 20. |
- Sjölie, Daniel
(författare)
-
Reality-based brain-computer interaction
- 2011
-
Licentiatavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Recent developments within human-computer interaction (HCI) and cognitive neuroscience have come together to motivate and enable a framework for HCI with a solid basis in brain function and human reality. Human cognition is increasingly considered to be critically related to the development of human capabilities in the everyday environment (reality). At the same time, increasingly powerful computers continuously make the development of complex applications with realistic interaction easier. Advances in cognitive neuroscience and brain-computer interfaces (BCIs) make it possible to use an understanding of how the brain works in realistic environments to interpret brain measurements and adapt interaction in computer-generated virtual environments (VEs). Adaptive and realistic computer applications have great potential for training, rehabilitation and diagnosis. Realistic interaction environments are important to facilitate transfer to everyday reality and to gain ecological validity. The ability to adapt the interaction is very valuable as any training or learning must be done at the right level in order to optimize the development of skills. The use of brain measurements as input to computer applications makes it possible to get direct information about how the brain reacts to aspects of a VE. This provides a basis for the development of realistic and adaptive computer applications that target cognitive skills and abilities. Theories of cognition and brain function provide a basis for how such cognitive skills develop, through internalization of interaction with the current environment. By considering how internalization leads to the neural implementation and continuous adaptation of mental simulations in the brain it is possible to relate designed phenomena in a VE to brain measurements. The work presented in this thesis contributes to a foundation for the development of reality-based brain-computer interaction (RBBCI) applications by combining VR with emerging BCI methods based on an understanding of the human brain in human reality. RBBCI applications can be designed and developed to interact directly with the brain by interpreting brain measurements as responses to deliberate manipulations of a computer-generated reality. As the application adapts to these responses an interaction loop is created that excludes the conscious user. The computer interacts with the brain, through (the virtual) reality.
|
|
| 21. |
- Sjölie, Daniel
(författare)
-
The brain and interaction in a multimodal reality
- 2009
-
Rapport (övrigt vetenskapligt/konstnärligt)abstract
- This paper reviews the neural correlates of multimodal integration and the role it plays in the creation and maintenance of perception of reality. These issues are illuminated by reviewing concepts and research from a number of related subjects and we explore some of the relevant cognitive models, such as the memory-prediction framework. We further focus on how multimodal integration affects reality-based interaction (RBI) in general and virtual reality (VR) in particular. In this case the reality in question is generated by a computer and perception of reality may be unstable. In VR-related research the quality of the perception of reality is commonly referred to as presence and a review of the conditions for and effects of varying degrees of presence is presented. An increased understanding of the role of multimodal integration in the creation and maintenance of presence is one of the primary goals of this paper. The hope is that this will help us to understand and improve presence, something that we will show to be of great value. The effect of disturbances and failure in the multimodal integration on the perception of reality and presence is of particular interest. This is related to the concept of breaks in presence and prediction errors, to provide some framework for understanding. Also, the importance of understanding the neural correlates of these cognitive functions is related to the possible use of VR in combination with brain imaging, exemplified with functional magnetic resonance imaging (fMRI). Finally, we discuss possible future work and possibilities to advance the understanding of the brain and reality in the context of human computer interaction.
|
|
| 22. |
- Sjölie, Daniel, et al.
(författare)
-
Tying activity theory to brain function: theoretical foundations for reality-based brain-computer interaction
-
Annan publikation (övrigt vetenskapligt/konstnärligt)abstract
- Recent developments within human-computer interaction and cognitive neuroscience point to the need for interaction theory with a solid basis in theories of brain function, and present the opportunity to develop such foundations. The rising popularity of brain-computer interfaces (BCIs) in particular calls out for such theoretical grounding. Activity theory has proved itself capable of supporting human-computer interaction (HCI) research and development over the last decades. Relating activity theory to modern theories of brain function emerges as a desirable goal. The inherent importance of complex realistic contexts in activity theory supports the development of applications combining BCIs with realistic interaction environments. The developmental focus of activity theory is also valuable when constructing applications for training or rehabilitation. In this paper we show how activity theory can be related to modern frameworks for brain function such as grounded cognition and the free-energy principle.
|
|
| 23. |
- Sjölie, Daniel, et al.
(författare)
-
Using brain imaging to assess interaction in immersive VR
- 2009
-
Ingår i: Challenges in the evaluation of usability and user experience in reality based interaction. - Boston, MA, USA : ACM. - 9782917490099 ; , s. 23-27
-
Konferensbidrag (refereegranskat)abstract
- We have developed a system where the combination of functional brain imaging (fMRI) and Virtual Reality (VR) can be used to study and evaluate user experience based on brain activation and models of cognitive neuroscience. The ability to study the brain during natural interaction with an (ecologically valid) environment has great potential for several areas of research and development, including evaluation of Reality-Based Interaction (RBI). The RBI concept of tradeoffs is of particular interest since we want to further explore the relation between how the brain works with an accepted reality and what happens when this reality is disrupted. We present the system with an overview of conducted studies to illustrate capabilities and feasibility. In particular, feasibility is supported by the fact that the brain activations seen in these studies match expectations based on existing literature. Further discussion elaborates on the relation to RBI and evaluation; and finally some possible future work is presented.
|
|
| 24. |
- Somanath, Sanjay, 1994, et al.
(författare)
-
Towards Urban Digital Twins: A Workflow for Procedural Visualization Using Geospatial Data
- 2024
-
Ingår i: Remote Sensing. - 2072-4292. ; 16:11
-
Tidskriftsartikel (refereegranskat)abstract
- A key feature for urban digital twins (DTs) is an automatically generated detailed 3D representation of the built and unbuilt environment from aerial imagery, footprints, LiDAR, or a fusion of these. Such 3D models have applications in architecture, civil engineering, urban planning, construction, real estate, Geographical Information Systems (GIS), and many other areas. While the visualization of large-scale data in conjunction with the generated 3D models is often a recurring and resource-intensive task, an automated workflow is complex, requiring many steps to achieve a high-quality visualization. Methods for building reconstruction approaches have come a long way, from previously manual approaches to semi-automatic or automatic approaches. This paper aims to complement existing methods of 3D building generation. First, we present a literature review covering different options for procedural context generation and visualization methods, focusing on workflows and data pipelines. Next, we present a semi-automated workflow that extends the building reconstruction pipeline to include procedural context generation using Python and Unreal Engine. Finally, we propose a workflow for integrating various types of large-scale urban analysis data for visualization. We conclude with a series of challenges faced in achieving such pipelines and the limitations of the current approach. However, the steps for a complete, end-to-end solution involve further developing robust systems for building detection, rooftop recognition, and geometry generation and importing and visualizing data in the same 3D environment, highlighting a need for further research and development in this field.
|
|
| 25. |
- Somanath, Sanjay, 1994, et al.
(författare)
-
Towards Urban Digital Twins: A Workflow for Procedural Visualization Using Geospatial Data
- 2024
-
Ingår i: Remote Sensing. - 2072-4292. ; 16:11
-
Tidskriftsartikel (refereegranskat)abstract
- A key feature for urban digital twins (DTs) is an automatically generated detailed 3D representation of the built and unbuilt environment from aerial imagery, footprints, LiDAR, or a fusion of these. Such 3D models have applications in architecture, civil engineering, urban planning, construction, real estate, Geographical Information Systems (GIS), and many other areas. While the visualization of large-scale data in conjunction with the generated 3D models is often a recurring and resource-intensive task, an automated workflow is complex, requiring many steps to achieve a high-quality visualization. Methods for building reconstruction approaches have come a long way, from previously manual approaches to semi-automatic or automatic approaches. This paper aims to complement existing methods of 3D building generation. First, we present a literature review covering different options for procedural context generation and visualization methods, focusing on workflows and data pipelines. Next, we present a semi-automated workflow that extends the building reconstruction pipeline to include procedural context generation using Python and Unreal Engine. Finally, we propose a workflow for integrating various types of large-scale urban analysis data for visualization. We conclude with a series of challenges faced in achieving such pipelines and the limitations of the current approach. However, the steps for a complete, end-to-end solution involve further developing robust systems for building detection, rooftop recognition, and geometry generation and importing and visualizing data in the same 3D environment, highlighting a need for further research and development in this field.
|
|
| 26. |
|
|
| 27. |
- Stahre Wästberg, Beata, 1974, et al.
(författare)
-
Visual conflicts - Challenges in combining rich volumetric 3D-data in a realistic VR city model
- 2021
-
Konferensbidrag (övrigt vetenskapligt/konstnärligt)abstract
- This paper presents a work in progress on how to visualize the impact of vegetation on urban air quality in a virtual city model, explorable using virtual reality (VR). The transdisciplinary project CityAirSim - Visualizing and modelling urban air quality - influence of vegetation, building morphology and traffic emissions develops a modelling framework to simulate the effects of vegetation, buildings and traffic strategies on air pollution, in order to understand the synergies and conflicts between these factors. The resulting data will be visualized in a VR-application allowing interactive exploration. Two primary target groups for the VR-application are high school students and urban planners. We aim for the tool to be used in teaching, e.g., in environmental science, as well as in communication with the public in exhibition environments, such as science centers. This paper focuses on how we should visualize invisible air pollution in an otherwise realistic urban setting to facilitate understanding. One important aspect to consider is to create a visual balance between data visualization and the visualization of the built environment in the model. Another aspect is to define the appropriate levels of data visualization and abstraction of representation to use, without losing richness of information.
|
|
| 28. |
- Stahre Wästberg, Beata, 1974, et al.
(författare)
-
Visualizing environmental data for pedestrian comfort analysis in urban planning processes
- 2017
-
Ingår i: Proceedings for CUPUM 2017 - 15th International Conference on Computers in Urban Planning and Urban Management.
-
Konferensbidrag (refereegranskat)abstract
- Digital tools are being developed for involving stakeholders in urban planning and transformation processes. One challenge is how to visualize and act upon all parameters that are relevant for dealing with complex plan-ning problems, such as environmental factors. Dialogue tools involving vis-ualization can bridge the distance between planners and citizens. This paper focuses on the problem of representing invisible environmental parameters affecting the urban climate such as wind, solar radiation, air pollution and noise, in a city model. The aim of the paper is to discuss challenges for rep-resenting and communicating environmental data in city models. In this pa-per, we have accounted for literature studies; our own conceptual modelling and prototype studies in three projects; as well as a survey with 16 urban planners. We conclude with defining design criteria for dialogue tools cre-ating a comprehensible base for communication in urban transformation processes.
|
|
| 29. |
- Surie, Dipak, 1981-, et al.
(författare)
-
Activity recognition based on intra and extra manipulation of everyday objects
- 2007
-
Ingår i: Proceedings of the 4th International Symposium on Ubiquitous Computing Systems. - Berlin/Heidelberg : Springer Berlin/Heidelberg. - 3540767711 - 9783540767718 ; , s. 196-210
-
Konferensbidrag (refereegranskat)abstract
- Recognizing activities based on an actor's interaction with everyday objects is an important research approach within ubiquitous computing. We present a recognition approach which complement objects grabbed or released information with the object's internal state changes (as an effect of intra manipulation) and the object's external state changes with reference to other objects (as an effect of extra manipulation). The concept of Intra manipulation is inspired by the fact that many everyday objects change their internal state when manipulated by the human actor, while extra manipulation is motivated by the fact that humans commonly rearrange the spatial relations between everyday objects as part of their activities. A detailed evaluation of our prototype activity recognition system in virtual reality (VR) environment is presented as a "proof of concept". We have obtained a recognition precision of 92% on the activity-level and 81% on the action-level among 15 everyday home activities. Virtual reality was used as a test-bed in order to speed up the design process of our activity recognition system, allowing us to compensate for the limitations with currently available sensing technologies and to compare the contributions of intra manipulation and extra manipulation for activity recognition.
|
|
| 30. |
- Surie, Dipak, 1981-, et al.
(författare)
-
Activity recognition using an egocentric perspective of everyday objects
- 2007
-
Ingår i: Proceedings of the 4th International Conference on Ubiquitous Intelligence and Computing. - : Springer Berlin/Heidelberg. ; , s. 246-257
-
Konferensbidrag (refereegranskat)abstract
- This paper presents an activity recognition approach based on the tracking of a specific human actor’s current object manipulation actions, complemented by two kinds of situational information: 1) the set of objects that are visually observable (inside the “observable space”) and 2) technically graspable (inside the “manipulable space”). This “egocentric” model is inspired by situated action theory and offers the advantage of not depending on technology for absolute positioning of neither the human nor the objects. Applied in an immersive Virtual Reality environment, the proposed activity recognition approach shows a recognition precision of 89% on the activity-level and 76% on the action-level among 10 everyday home activities.
|
|
| 31. |
- Unger, Jonas, et al.
(författare)
-
Foreword to the Special Section on the 23rd ACM Symposium on Virtual Reality Software and Technology 2017
- 2018
-
Ingår i: Computers and Graphics. - : Elsevier BV. - 0097-8493 .- 1873-7684. ; 77, s. A3-A4
-
Tidskriftsartikel (övrigt vetenskapligt/konstnärligt)abstract
- VRST is the top international symposium to see AR/VR/MR (XR) research results, systems, and techniques. VRST brings together the main international research groups working in XR. VRST is supported and joined by many of the world's leading XR companies. VRST enables XR researchers to interact, share new results, and show live demos. VRST event is sponsored by ACM SIGCHI and SIGGRAPH.
|
|
