Agile UX Work Challenges

Setup: Academic researcher at Jeppesen Systems, Boeing, five months, 2015

Summary: I performed a generative field study for four months to identify main challenges and success factors concerning UX work in an Agile setting as experienced by UX architects, business analysts, product owners, and developers.

Methods

During this field study, I conducted in-depth interviews (19), Shadowing (5), Observations (7), Focus group (2) as well as an extensive archival analysis of the product and project documents from the last two decades. I performed 19 in-depth interviews across the three units of the organization in two steps. In the first step, I conducted 13 open and semi-structured interviews. The initial interviews were open interviews since I aimed to discover the main events (circumstances, activities, and decisions) that affected UX work over the years in the case company and their enabling or prohibiting role. As I explored and learned more about the enabling and prohibiting factors, I performed additional interviews to gather detail information about them.

To better understand and gather more information about the ongoing UX practices and integration initiatives in the company,I also observed several of main meetings performed by the UX expert (e.g. design studio, impact mapping workshop), and weekly meetings of the UX guild. The above meetings were both audio-recorded and transcribed for coding and analysis.

Because I aimed to investigate the attitudes of various internal stakeholders towards the role of the UX expert and current integration efforts, in these meetings, I mainly focused on the collaboration and communication between the UX expert and the meetings participants (and their reflections, reactions, and perceptions) rather than the outcome of the meetings, i.e. the UX artifacts.

After both meetings, I gathered the participants’ feedback via short interviews to understand how they perceived the work of the UX expert and, in particular, the tools and methods she introduced. During this field study, I was also located in the same room as the UX expert to be able to closely observe various UX practices performed by this role. Regular discussions with this role also naturally happened due to this co-location.

Beside the UX expert, the UX guild was another rich source of information about the current UX work in the company. Hence, I also observed five guild meetings. The meeting discussions were recorded and the summary of the main points was distributed among the participants for validation. These observations provided rich data on how the UX guild members perceived the factors that in their view prohibited or enabled UX integration.

I also investigated the UX artifacts generated from 1992 to 2015. The aim of this document analysis was mainly triangulation of the interview data, i.e. verify the existence of the artifacts mentioned by the interviewees or discover the ones that were missing from our data. Other sources of data were two focus group workshops that I held at the company.

The first workshop involved participants representing the three units. The workshop aimed to reflect on the events identified through interviews and their effect on UX integration in the company (i.e. enabling and prohibiting factors). I held a second focus group workshop at the end of the data gathering.

Different methods that I applied in this research work

Outcome

In my data synthesis, I created 143 tags (codes). The data resulted in 27 insights and 18 actionable recommendations which the organization used to improve their Agile UX processes.

These findings were also published in form of peer-reviewed journal and conference papers.

The main three themes (high-level insights) and their interrelation

Three main identified themes of challenges and success factors in UX integration, and their interrelation. There is a complex interrelation also among the sub-themes although since the focus of the study has not been on identifying these detailed connections, here we only show the connections among the themes. We have formulated a number of questions related to each theme, based on our understanding of the identified challenges and success factors. Practitioners require to address these questions in their UX integration efforts while taking the context of their organization into account. Questions that concern how the process of UX integration is handled in the organization are marked with a star (x).
The pitfalls I identified in this study concerting UX integration
Note

In all of the included user research assignments, I was the sole researcher and responsible for the whole process: design, recruiting, data gathering, data analysis (tagging/coding and pattern/thematic analysis), creating an insight repository, and discussing the findings with the stakeholders in a series of workshops (e.g. to prioritize the insights or plan for taking actions based on the research recommendations). In addition, all of the interviews (often 1-2 hour) and workshops (often 2-3 hour) were audio/video recorded and transcribed for data analysis and generating evidence-based insights and recommendations. The research assignments were conducted in close collaboration with the stakeholders and in an Agile setting (iteratively) for fast delivery and feedback.