Cognitive Screeners

Introduction

If we use our own frame of reference as a starting point, we end up with products designed specifically for people of a specific age, language ability, tech literacy, learning style, and ability. Including people in the process means understanding and having empathy for people who are different from us.

There are different ways to learn from, co-create with, and explore the diverse ideas of others.

In addition to having teammates with diverse lived experiences, we recommend recruiting additional colleagues or external participants with diverse lived experiences to provide additional perspective. If you’re at the stage of recruiting, you’ll need to have a clear idea of why you’re bringing people in, how you want to use their time, and how you want to structure the experience.',

Here is our suggested recruiting process:

1. Figure out your objective
2. Write a research plan
3. Build a recruitment screener

  1. Figure out your objective

1a. Review the cognitive areas

What cognitive area(s) are needed to achieve tasks in your experience?

1b. Use the worksheet to understand where you are optimizing today and where you need to expand your thinking.

Who is excluded from your experience today and how might you bring more diversity into your recruiting?

  2. Write a research plan

This should include goals, methodology for the session, number of participants, and etc.

  3. Build a recruitment screener

Below are a few example questions for the cognitive demands featured in the Inclusive Design for Cognition Guidebook.

Remember that this is just a starting point and is meant to be inspiration for research teams. Many of these questions have been tested extensively and have yielded positive outcomes. Some of these are new and will require iteration over time.

We invite the inclusive design community to help us improve and add to this over time. Reach out to us with your suggestions at inclusivedesign@microsoft.com

Tips

• Create a mandatory requirement for diverse demographics such as: age, ethnicity, gender spectrum, income, education, physical ability, hybrid work, fully remote work, in-person work, and more dimensions of diversity.
• Include no more than 3-4 cognitive spectrums total (across one or multiple cognitive areas) and 1-2 open-ended questions in your screener. Anymore and the recruit becomes too complex.
• Pick and choose what works for you and your project. Some areas have multiple questions for the same concept. Select what works best for you.
• Nominate a workshop or session leader to speak with all potential co-creators before the experience to ensure it’s a match.
• Not everyone who has a medical diagnosis will feel they fit within the excluded population. Do not exclusively recruit on medical diagnosis. Rather, recruit using the spectrum. For example, if your cognitive area is Focus and you’re interested in people who struggle with interruptions… if someone has ADHD and experiences trouble with interruptions they are a fit. If they have ADHD but no trouble with interruptions, they may not be fit.

Learn

Recruit the following:

1. People who are excluded today based on the spectrums in the questions below.
2. People who are excluded today based on the spectrums in the questions below and fall into one of the “people with” group below:

• People with high stress and anxiety about learning something new.
• People with ADHD, ADD, Autism
• People with learning disabilities like Dyslexia, Dysgraphia, and Dyscalculia.

Please answer the following question on a 1-7 scale – scored 1 (strongly disagree) to 7 (strongly agree).

Q1: My first step in learning new technology is experimenting and tinkering with it.
TIP: If you are looking for guided learners as an example, you’d want people who select 1-3 on this question.

Q2: How do you approach learning something new for the first time?

• I like tinkering around. When trying to do an unfamiliar task or learning something new, I like to start trying until I figure it out.
• I want to lean on guidance to support me before starting something new like how-to video’s or documents.
• I want structure and a lot of support before trying something new like taking a class or having an expert at the ready.

Q3: (Open-ended) If you were to approach cooking a complicated new meal for the first time, what would you do?
(Note: If you are looking for guided learners, you are looking for responses like taking a class, asking an expert --- any answer that would indicate structured, guided, learning)

Q4: (Open-ended) Say you for some reason needed to learn something new on a computer – learning how to code or learning more deeply about excel – talk to me about how you’d approach that?
(Note: If you are looking for guided learners, you’d want to understand more about them. Would you take a class? Would you get a book and try it yourself? Would you download the program and play around? You want people who need the utmost structured learning)

Q5: (Open-ended) If you had trouble with the touch pad on your laptop, how would you approach solving the problem?
(Note: People will answer from a spectrum: experimenting and tinkering with it to asking for help or looking for guidance. If you are looking for guided learners you want people who would watch a help video, look online, etc.)
TIP: Asking about a non-technology task can help balance and diversify the mix of participants.

Decision Making

Recruit people who are excluded today based on the spectrums in the questions below:',

Please answer the following question on a 1-7 scale – scored 1 (strongly disagree) to 7 (strongly agree).

Q1: I spend a lot of time doing research before making important decisions.

Q2: When a decision needs to be made, it is important to me to gather relevant details before deciding, to be sure of the direction I’m headed.

Q3: I want to get things right the first time, so before I decide how to act, I gather as much information as I can.

Q4: I want to be aware of the consequences first, before deciding or acting with technology.

Q5: When there are too many steps involved to decide or too much information presented, I get overwhelmed and often ask for help or avoid making a decision.

Q6: I want to understand tradeoffs on the choices I am about to make to help me make a decision headed.

Focus

Recruit the following:

1. People who are excluded today based on the spectrums in the questions below
2. People who are excluded today based on the spectrums in the questions below and fall into one of the “people with” group below:

• People with ADHD, ADD, Autism, Anxiety, Sensory Processing Disorder.
• People with impaired ability to focus due to injury​.
• People with partial hearing loss, who use assistive technology.
• People with partial vision loss, who use assistive technology.​

Q1: What best describes how you feel about technology interruptions?

• I require complete isolation to do my best work; interruptions are difficult to recover from.
• When I’m interrupted by product features; I can experience negative emotions like frustration and anxiety.
• I don’t mind technology/software interruptions if they are useful to me.
• I don’t mind technology/software interruptions, they don’t negatively impact me.

TIP: If you are recruiting for people who struggle with interruptions, you may want to exclude anyone who only selects the last two responses.

Q2: (Open-ended) How do you maintain sustained concentration while focusing on an important project?
(Note: If you are looking for folks who struggle with concentration, you are looking for responses like “I struggle with this”)

Q3: (Open-ended) What are alerts and interruptions in your day today; which are disruptive vs. helpful?

Please answer the following questions on a 1-7 scale – scored 1 (strongly disagree) to 7 (strongly agree).

Q: Task switching (moving from tab to tab or page to page) with technology is hard and takes me out of my focus.

Q: I feel protective over my “focus time” while working on a project and struggle tuning people and technology out.

Q: I struggle tuning interruptions out when I want to focus on one specific task for a long period of time (like working or playing a video game).

Q: I struggle with selective attention: focusing on one activity during many activities (like listening to a colleague in a loud office).

Q: For my job or personal hobbies, it’s important to me to have the ability to focus on one task for a long time without distraction.

Q: I have a hard time selecting what to pay attention to while there’s a lot of stimuli around (like trying to work in a noisy office).

Q: I can easily ignore external (e.g. noise, technological interruptions) and internal (thoughts) distractions.

Q: Technology notifications make focusing difficult.

TIP: This is a lot of statements – choose 3-4 that work the best for your project.

Recall

Recruit the following:

1. People who are excluded today based on the spectrums in the questions below.
2. People who are excluded today based on the spectrums in the questions below and fall into one of the “people with” group below:

• People with short or long-term memory loss​.
• Friends, family and people who assist people with memory loss.​
• People with ADHD, ADD, Autism

Q1: Select a response.

• I always forget where my keys are.
• I sometimes have trouble remembering where I put my keys.
• I rarely have trouble remembering where I put my keys.
• I always remember where I put my keys.

Q2: Select a response to complete a statement:

To remember where I put my key takes me...

• More than 10 tries
• 5-10 tries
• 2-5 tries
• 1 try (I instantaneously memorize where I put my keys)

Please answer the following question on a 1-7 scale – scored 1 (strongly disagree) to 7 (strongly agree).

Q3: How would you approach the following situation: Imagine you are starting a new job and have to create multiple new logins with different usernames and passwords.

• I often reset my passwords and usernames.
• I write down my passwords and usernames.
• I use similar passwords and usernames to easily remember them.
• I use similar passwords and usernames to easily remember them.
• I remember my passwords and usernames and don’t use any assistance.

Q4: Select a response.

• I never remember my usernames and passwords and must reset it every time.
• I sometimes have trouble remembering my passwords and usernames.
• I rarely have trouble remembering my passwords and usernames.
• I always remember my usernames and passwords.

Q5: (Open-ended) What assistance, if any, do you use to remember your passwords and usernames?

Communication

Recruit the following:

1. People who are excluded today based on the spectrums in the questions below.
2. People who are excluded today based on the spectrums in the questions below and fall into one of the “people with” group below:

• People with high stress and anxiety about communicating with others.
• People with ADHD, ADD, Autism.
• People who need more than the average amount of time to process information.
• People without use of lower extremities, who use assistive technology​.
• People without use of upper extremities​, who use assistive technology.
• People who are left-handed​.
• People with complete blindness.
• People with complete hearing loss.​
• People with aphasia.
• People with a difficult time remembering past events, new events, or both.

Q1: (Open-ended) How do you prefer technology communicate with you for important information? (tips, updates, suggestions, guidance)',
(Note: You may consider selecting people with varied communication preferences from those who prefer real-time visual pop-ups to those who want infrequent multi-modal content.)

Q2: (Open ended) What do you want technology (like an AI assistant) to get to know about you over time to personalize an experience? (communication preferences about content format, tone, facts about your style, etc.)

Q3: Please answer the following question on a 1-7 scale – scored 1 (strongly disagree) to 7 (strongly agree).

• I don’t want technology (AI assistant, software) to know me or use any of my information to personalize an experience. I often reset my passwords and usernames.
• If I’ve used a particular technology multiple times and it doesn’t remember or leverage information about me to make the experience easier, I can get frustrated.
• I want technology to adapt to my communication preferences (for example, short and concise versus added context) over time as it gets to know me.
• I want technology (software, AI assistants) to get to know me over time and use information about me to make my experience easier, faster, better.
• I don’t want technology to assume a communication preference (like how I want suggestions surfaced) without asking.

Self-efficacy

Self-efficacy and attitudes toward risk aren’t cognitive areas by themselves though are important contextual considerations. If relevant to your project, please use the screening questions below.

Q1: Please answer the following question on a 1-7 scale – scored 1 (strongly disagree) to 7 (strongly agree).

• I can use unfamiliar technology features when I only have the internet as a reference.
• I am comfortable using technology.

Q2: How would you approach a new challenge with technology?

• It would feel like a fun puzzle. I’d get started solving it.
• I’d feel it was challenging but possible.
• Many challenges with technology feel impossible to me.

Q3: (Open ended) When there’s a significant problem with your computer – how effective do you feel that you can solve it yourself?

(Note: If you are looking for low self-efficacy, you are looking for people with a low ability to feel they can solve something themselves. For computer issues, they have lower confidence. They might tend to blame themselves vs. the technology for challenges they have.)

Attitude Toward Risk

Recruit people who are excluded today based on the spectrums in the questions below:

Q1: Please answer the following question on a 1-7 scale – scored 1 (strongly disagree) to 7 (strongly agree).

• I avoid using new apps or features in products before they are well-tested.
• I worry that new features will make it harder to get my job done.
• I avoid new software features until/unless I feel certain they’ll be useful.
• I enjoy trying new features even if I’m not fully confident they’ll help.
• It’s better to try a new feature and see it does not work out as intended than possibly miss information that could have been useful.
• I avoid using new apps or technology before they are well-tested. I avoid running software updates because I am worried the update will break something.
• I enjoy exploring new features when I’m not familiar with seeing what they do and learn.
• I enjoy finding the lesser-known features and capabilities of the software I use.

TIP: This is a lot of statements – choose a few that work the best for your project.

Q2: How do you feel about the adoption of new features?

• I enjoy trying new features even if I’m not fully confident they’ll all help.
• I avoid new software features until/unless I feel certain they’ll be useful.

Acknowledgements

Thank you to Dr. Margaret Burnett who’s Gender Mag work was the basis for creating several of these questions and the disability community who helped create this document.

This was created by: Doug Kim, Margaret Price, Christina Mallon, Anna Tendera, Puja Pandya