Digital eXperience of Release Process

Setup: Contractor at Volvo Group, 4 months, 2019-2020

Summary: During a four-month period, I conducted generative research on processes and tools applied by engineers in a recently introduced SAFe and CI/CD context. I closely collaborated with product and portfolio management and business process improvement leaders to provide input to their roadmaps.

Details omitted! As the project is confidential, the page only includes a high-level description of the research and its findings

In this research work, my focus was on the first half of the double-diamond: exploring the problem area

Methods

I conducted 20 in-depth interviews and task analysis to generate qualitative data. I also designed and run a survey to triangulate the findings with quantitative data.

Below is a summary of my main tasks and methods applied:

  • Conducted 20 in-depth interviews and task analyses
  • Designed and ran a survey to triangulate the findings, in particular, to validate the identified pain points and to rank them based on their perceived impact on quality or efficiency of the engineers' daily work
  • Facilitated 5 workshops to discuss and prioritize the insights
In this generative research, I applied a variety of methods to gather and analyze the data

Outcome

My data analysis resulted in 18 process- or tool- related insights that included pain points and actionable recommendations to overcome them. I facilitated 5 workshops with management stakeholders, to prioritize the insights based on their complexity and business impact. During these workshops we also decided on which insights to act on first. Below, is a list of my main deliverables:

  • Synthesized data into 18 actionable process- or tool- related insights
  • Developed user journey maps and personas
  • Provided actionable input to product and process roadmaps
  • Created a user research process and framework to complement the
    traditional change request approach within the organization which
    led to executive management buy-in
  • Presented a summary of the findings and the recommended next steps to the excetuvie management (VPs) in 2 sessions
I synthesised data into 18 actionable insights presented in 5 workshops to product and portfolio management also executive management. The interviews and workshops were all audio-recorded and transcribed to minimize bias in the analysis. The transcripts were then analyzed using thematic analysis (segmenting the data and coding/tagging).
Using service design methods and tools, the employees’ actual work processes were mapped in journey maps. These maps included the actual work steps, and pain points on the work processes and tool usage.

Reflection

I was the first user researcher in the organization and the tools to facilitate research work were not available. In addition, the client has restrict regulations on using third-party tools and services. Consequently, I had to use less optimal available tools for recording, transcribing and tagging (e.g. excel). I manually transcribed the recordings, segmented and tagged the data in excel to use its filtering options to generate patterns and insights. To generate the user journeys and personas I had to use Visio and manually create the visuals and templates. Such an approach had a big impact on efficiency of research, One other major challenge was to get access to users. As the users are Volvo employees with high work load they required permission from their managers to participate in the study. If I were to do the project again, I would request more suitable research tools and an agreed-up number of participants to increase validity of findings, in particular, enabling more quantitative analysis.

Note

In all of the included user research assignments, I was the sole researcher and responsible for the whole process: design, recruiting, data gathering, data analysis (tagging/coding and pattern/thematic analysis), creating an insight repository, and discussing the findings with the stakeholders in a series of workshops (e.g. to prioritize the insights or plan for taking actions based on the research recommendations). In addition, all of the interviews (often 1-2 hour) and workshops (often 2-3 hour) were audio/video recorded and transcribed for data analysis and generating evidence-based insights and recommendations. The research assignments were conducted in close collaboration with the stakeholders and in an Agile setting (iteratively) for fast delivery and feedback.