Shape some questions and a process as part of user research tied to binderhub-service

[damianavila]

Context

We believe we need a UX/UI user research validation step before implementing a general solution.
We are going to deploy tljh-repo2docker (#1596), which is an opinionated way to deliver the experience and conduct user research on top of it.

We need to come up with a process and the right set of questions we are going to ask in user (admins and final users) interviews.

@sgibson91 mentioned she has a contact who can help us with this process.

Proposal

@sgibson91 already contacted Meag Doherty and there are some ideas about the next step in the following message: #1577 (comment) (copying it below for simplicity):

Conversation with Meag Doherty RE user research

Meag is onboard to help us shape some questions we'd like to ask as part of user research, and has shared two resources for designing a user research study which I have added to our shared GDrive here: https://drive.google.com/drive/folders/1KTDi6fF3M0p8ldXo1tD0LjD1ZBZNncoL?usp=sharing

She would also be happy to join one of the meetings for a user research focussed agenda. Next week (Aug 9th) isn't possible, but @damianavila could we include her availability when scheduling a meeting in week 3? (w/c Aug 15).

Additionally, Meag is another SSI Fellow whose fellowship focus is on UI/UX for research software. One of her goals is to capture case studies of people who work in research software that might perform user research as part of feature development (feeding into a longer strategy of pioneering a Research Software Design role to accompany RSEs). I think it would be a great in-kind contribution for us to be a case study in return for her expertise (I can work with her on that if preferred by the team).

cc @meagdoh

Let's continue this specific conversation on this very same issue.

Updates and actions

• Drafting interview questions: https://docs.google.com/document/d/1RSrixqZ3k_as9YW8eyWw84pLsxl9JboPGSp5-Nu3yBE/edit?usp=sharing
• Any UX-UI research would be based on the binderhub-service work, see: Shape some questions and a process as part of user research tied to binderhub-service #1597 (comment) for more details.

[damianavila]

Meag is onboard to help us shape some questions we'd like to ask as part of user research, and has shared two resources for designing a user research study which I have added to our shared GDrive here: https://drive.google.com/drive/folders/1KTDi6fF3M0p8ldXo1tD0LjD1ZBZNncoL?usp=sharing

Super, that is great!!

She would also be happy to join one of the meetings for a user research focussed agenda. Next week (Aug 9th) isn't possible, but @damianavila could we include her availability when scheduling a meeting in week 3? (w/c Aug 15).

Yep, that makes sense.
@meagdoh, can you add your availability for the 3rd week here: https://www.when2meet.com/?16262802-AGkDQ

I think it would be a great in-kind contribution for us to be a case study in return for her expertise (I can work with her on that if preferred by the team).

That is totally fair and fine with me!

[sgibson91]

Actions points for me from the last meeting:

• Ping Meag, make sure she knows where and how to provide her availability for next week
• Create a document to scope a draft interview script. (See a previous script I have put together here.) Should have the following sections:
  • Context: What do we want to find out?
    • Rough things. I would like to crowd source this section with @damianavila @consideRatio and @yuvipanda
  • Aims: Distilled, concrete items from the context section
  • Questions: What we will actually ask participants. Should be clear which aim each question answers.
• Share this document with Meag and ask for feedback, improvements, tips on how to run the sessions. Maybe a walkthrough of the documents she shared with us, how to use, how to convert our draft document into those formats

[sgibson91]

I have created a document for drafting the user research questions here: https://docs.google.com/document/d/1RSrixqZ3k_as9YW8eyWw84pLsxl9JboPGSp5-Nu3yBE/edit?usp=sharing

@damianavila @consideRatio @yuvipanda I invite you to add what you hope to learn from this research to the "Context" section. Please do not worry about these being fully-formed, I will do some refinement once I have everyone's input 😉

[damianavila]

Some interesting and related content from another issue: #1598 (comment) (in case you did not see it yet).

[sgibson91]

Scheduling proved difficult so I ended up having a 1-to-1 chat with Meag on Friday. My takeaway from that meeting was that I'm maybe overthinking all this and we should just go with a semi-structured approach. Deploy tljh-repo2docker, get people in front of it, and ask them what they think. Let them drive the conversation more than having some specific questions.

Another alternative was a "diary study" where we give participants a list of tasks and they record themselves (actions, opinions, whatever) as they complete the tasks. Probably more scalable than zoom-based conversations with one of us present.

We could have a separate, longer discussion with Arnim about what other composable pieces need to be built in order to fully replicate the pbhub experience on top of a z2jh-compatible version of the tljh-repo2docker plugin.

[damianavila]

I think I would be biased toward the semi-structured approach instead of the diary study.

We could have a separate, longer discussion with Arnim about what other composable pieces need to be built in order to fully replicate the pbhub experience on top of a z2jh-compatible version of the tljh-repo2docker plugin.

Yep, that seems a sensible approach to me. Decoupling both activities (semi-structured/diary study on one side and fetching pbhub experience on the other side - would allow us to make in-parallel progress on both of them.

[arnim]

Just give me a ping :)

[damianavila]

@arnim @sgibson91, this is the when2meet link so we can agree on a day and time: https://www.when2meet.com/?16784142-sgYX7. Please drop your availability there so I can fill you with the meeting details as soon as we agree on one slot. Thanks!

[damianavila]

Copying over a comment from @arnim from another issue (#1598):

We had some User Stories for the pBHub.

[damianavila]

The meeting invitation was sent, folks.

[damianavila]

In the future, some UX-UI research will be needed but now based on the new binderhub-service described in #2120.
Updating the title and top message accordingly.