AIRELI Persona Taxonomy
This web app is the companion to the full research paper. It includes a concise overview of participant demographics, persona clusters and detailed tables, an annotated participant list with profiles and task summaries, and a replication package to export the underlying data and task codebase.
anonymized

Participant Demographics #

A concise profile of the participant sample (counts, age, experience, programs, and gender distribution).
Participants
22
Avg age ± SD
24.0 ± 2.0
Professional exp (yrs) ± SD
1.4 ± 1.9
Total exp (yrs) ± SD
4.6 ± 3.7
Study programs
MSc (12)BSc (10)
Gender
Man (18)Woman (4)

Persona Clusters #

Each rectangle represents a participant; chips summarize observed characteristics. Hover a cluster to peek more chips; use the button to expand.
Participant clusters by persona
Self-Sufficient (3)
P2
SCEXP
P4
SCEXP
P22
SCEXP
Understanding-Gated Delegators (10)
P1
SCAO
P11
AODD
P15
SCNOVQ&A
P21
SCNOV
P8
SCAWFNOV
P16
SCNOVQ&A
P17
SCAONOV
P19
SCAWF
P20
SCQ&A
P9
AODD
AI-Steered (8)
P5
IAIHAIAODD
P6
IAIHAISC
P3
AWFNOV
P7
IAIAWFAODDNOV
P12
IAIHAIAODDNOV
P13
DDNOV
P14
HAIAODD
P18
HAIAODD

Persona Details #

Sortable participant metrics. Click on a participant to expand and view persona reasoning: justifications for the assigned persona and why other personas were not assigned.
Self-Sufficient Developers — Can plan & ship without AI
IDGroupProfileTask 1Task 2
IDGroup
AI usage(1–5)
 
Backend(1–5)
 
T1 Coding(0/1)
 
T1 Comp Score(0–1)
 
T1 Conf.(1–5)
 
T2 Coding(0/1)
 
T2 Comp Score(0–1)
 
T2 Conf.(1–5)
 
Characteristics
Avg ± SD (3)3.67 ± 0.584.00 ± 1.0052.0 ± 9.5 min0.76 ± 0.074.67 ± 0.5823.7 ± 5.9 min0.44 ± 0.104.67 ± 0.58
P2 AI
4
5
51 min
0.69Questions: Q1: 2 · Q2: 0 · Q3: 4 · Total 6/12 Sketch: C 1.5/2 · R 2/2
5
28 min
0.50Questions: Q1: 4 · Q2: 2 · Q3: 0 · Total 6/12
5
SCEXP
P4
4
4
43 min
0.75Questions: Q1: 2 · Q2: 4 · Q3: 0 · Total 6/12 Sketch: C 2/2 · R 2/2
4
26 min
0.50Questions: Q1: 2 · Q2: 4 · Q3: 0 · Total 6/12
4
SCEXP
P22
3
3
62 min
0.83Questions: Q1: 2 · Q2: 4 · Q3: 2 · Total 8/12 Sketch: C 2/2 · R 2/2
5
17 min
0.33Questions: Q1: 2 · Q2: 2 · Q3: 0 · Total 4/12
5
SCEXP
Understanding-Gated AI Delegators — Progress is gated by understanding: build a mental map, then hand off execution to AI.
IDGroupProfileTask 1Task 2
IDGroup
AI usage(1–5)
 
Backend(1–5)
 
T1 Coding(0/1)
 
T1 Comp Score(0–1)
 
T1 Conf.(1–5)
 
T2 Coding(0/1)
 
T2 Comp Score(0–1)
 
T2 Conf.(1–5)
 
Characteristics
Avg ± SD (10)3.30 ± 0.672.40 ± 0.7070.7 ± 30.4 min0.76 ± 0.122.40 ± 1.0742.0 ± 0.0 min0.25 ± 0.122.50 ± 2.12
P1 AI
4
2
37 min
0.69Questions: Q1: 2 · Q2: 4 · Q3: 0 · Total 6/12 Sketch: C 2/2 · R 1.5/2
2
42 min (almost)almost finished task 2, login worked, only last subtask was missing
0.17Questions: Q1: 0 · Q2: 2 · Q3: 0 · Total 2/12
1
SCAO
P11
4
3
96 min
0.54Questions: Q1: 2 · Q2: 4 · Q3: 4 · Total 10/12 Sketch: C 1/2 · R 0/2
3
DDAO
P15 AI
4
2
0.75Questions: Q1: 0 · Q2: 4 · Q3: 2 · Total 6/12 Sketch: C 2/2 · R 2/2
1
SCNOVQ&A
P21
4
3
79 min
0.83Questions: Q1: 2 · Q2: 4 · Q3: 2 · Total 8/12 Sketch: C 2/2 · R 2/2
4
0.33Questions: Q1: 2 · Q2: 2 · Q3: 0 · Total 4/12
4
SCNOV
P8
3
2
0.77Questions: Q1: 0 · Q2: 4 · Q3: 4 · Total 8/12 Sketch: C 1.5/2 · R 2/2
1
SCAWFNOV
P16 AI
3
2
0.92Questions: Q1: 2 · Q2: 4 · Q3: 4 · Total 10/12 Sketch: C 2/2 · R 2/2
3
SCNOVQ&A
P17
3
3
0.67Questions: Q1: 4 · Q2: 0 · Q3: 0 · Total 4/12 Sketch: C 2/2 · R 2/2
1
SCNOVAO
P19
3
3
0.92Questions: Q1: 4 · Q2: 4 · Q3: 2 · Total 10/12 Sketch: C 2/2 · R 2/2
3
SCAWF
P20 AI
3
1
0.83Questions: Q1: 2 · Q2: 4 · Q3: 2 · Total 8/12 Sketch: C 2/2 · R 2/2
3
SCQ&A
P9 AI
2
3
0.67Questions: Q1: 2 · Q2: 4 · Q3: 4 · Total 10/12 Sketch: C 1/2 · R 1/2
3
DDAO
AI-Steered Developers — The AI leads, the developer follows.
IDGroupProfileTask 1Task 2
IDGroup
AI usage(1–5)
 
Backend(1–5)
 
T1 Coding(0/1)
 
T1 Comp Score(0–1)
 
T1 Conf.(1–5)
 
T2 Coding(0/1)
 
T2 Comp Score(0–1)
 
T2 Conf.(1–5)
 
Characteristics
Avg ± SD (8)4.13 ± 0.642.38 ± 0.9263.0 ± 12.3 min0.36 ± 0.181.88 ± 0.830.17 ± 0.002.00 ± 1.00
P5 AI
5
2
73 min
0.29Questions: Q1: 0 · Q2: 0 · Q3: 4 · Total 4/12 Sketch: C 0/2 · R 1/2
2
3
DDAOIAIHAI
P6 AI
5
2
71 min
0.75Questions: Q1: 4 · Q2: 0 · Q3: 2 · Total 6/12 Sketch: C 2/2 · R 2/2
2
SCIAIHAI
P3
4
2
0.40Questions: Q1: 0 · Q2: 0 · Q3: 2 · Total 2/12 Sketch: C 1.5/2 · R 1/2
1
AWFNOV
P7
4
4
0.13Questions: Q1: 0 · Q2: 0 · Q3: 0 · Total 0/12 Sketch: C 1/2 · R 0/2
1
DDAWFNOVAOIAI
P12 AI
4
1
0.33Questions: Q1: 2 · Q2: 4 · Q3: 2 · Total 8/12 Sketch: C 0/2 · R 0/2
2
DDNOVAOIAIHAI
P13
4
3
0.33Questions: Q1: 0 · Q2: 4 · Q3: 4 · Total 8/12 Sketch: C 0/2 · R 0/2
1
DDNOV
P14 AI
4
2
62 min
0.27Questions: Q1: 2 · Q2: 0 · Q3: 0 · Total 2/12 Sketch: C 1.5/2 · R 0/2
3
1
DDAOHAI
P18 AI
3
3
46 min
0.38Questions: Q1: 0 · Q2: 4 · Q3: 2 · Total 6/12 Sketch: C 1/2 · R 0/2
3
0.17Questions: Q1: 2 · Q2: 0 · Q3: 0 · Total 2/12
2
DDAOHAI
Other (failed to adhere to instructions)
IDGroupProfileTask 1Task 2
IDGroup
AI usage(1–5)
 
Backend(1–5)
 
T1 Coding(0/1)
 
T1 Comp Score(0–1)
 
T1 Conf.(1–5)
 
T2 Coding(0/1)
 
T2 Comp Score(0–1)
 
T2 Conf.(1–5)
 
Characteristics
Avg ± SD (1)5.00 ± 0.003.00 ± 0.000.58 ± 0.005.00 ± 0.000.50 ± 0.005.00 ± 0.00
P10 AI
5
3
0.58Questions: Q1: 0 · Q2: 4 · Q3: 4 · Total 8/12 Sketch: C 2/2 · R 0/2
5
0.50Questions: Q1: 4 · Q2: 2 · Q3: 0 · Total 6/12
5
IAIHAI

Participant List #

Detailed participants list including profile, Task 1 and Task 2 summaries.
Participant P1AI Group
Task 1 completedSoPra
23 yrs
BSc studentBSc student in Computer Science (or related field)
Participant Profile
AI usageHow a participant uses generative AI (e.g. Copilot, ChatGPT) to generate new code
Frontend
Javascript
Typescript
ReactJS
Backend
ExpressJS
SQL
Web dev experience
SoPra
AI in workflow
If I have an idea how to do things, then ask AI whether it is possible, how the syntax looks like
Task 1
Coding
Comprehension Quest.
Q1: 2 · Q2: 4 · Q3: 0 · Total 6/12 · Sketch: C 2/2 · R 1.5/2
Subjective ReportingParticipant's subjective reporting after working on the first task
Task Difficulty
Easy (2)
Mental effort
(4) Rather low mental effort
Confidence
Slightly confident (2)
What were the biggest difficulties you encountered when completing your task (if any)?
Understanding the connections between the different files, but once I had a mental graph of the connections I could get started with changes and also use ChatGPT to help me, with it doing all of the work in the end it feels like to me.
Task 2
Coding
Comprehension Quest.
Q1: 0 · Q2: 2 · Q3: 0 · Total 2/12
Subjective ReportingParticipant's subjective reporting after working on the second task
Task Difficulty
Difficult (4)
Mental effort
(6) Rather high mental effort
Confidence
Not at all confident (1)
What were the biggest difficulties you encountered when completing your task (if any)?
Creating the correct SQL statement which I probably did not succeed and is the reason why I did not finish. Also the problem how function calls seemingly did not work but I was just looking in the wrong place for console logs.
Participant P2AI Group
Task 1 completedTask 2 completed
23 yrs
MSc studentMSc student in Computer Science (or related field)
Participant Profile
AI usageHow a participant uses generative AI (e.g. Copilot, ChatGPT) to generate new code
Frontend
Javascript
Typescript
ReactJS
Backend
ExpressJS
SQL
Web dev experience
5-6 projects. Expert
AI in workflow
Generate documents and descriptions
Task 1
Coding
Comprehension Quest.
Q1: 2 · Q2: 0 · Q3: 4 · Total 6/12 · Sketch: C 1.5/2 · R 2/2
Subjective ReportingParticipant's subjective reporting after working on the first task
Task Difficulty
Easy (2)
Mental effort
(4) Rather low mental effort
Confidence
Extremely confident (5)
What were the biggest difficulties you encountered when completing your task (if any)?
Understanding the codebase as it is just a test code base a lot of stuffs are done in the way that it shouldn't be.
Task 2
Coding
Comprehension Quest.
Q1: 4 · Q2: 2 · Q3: 0 · Total 6/12
Subjective ReportingParticipant's subjective reporting after working on the second task
Task Difficulty
Very Easy (1)
Mental effort
(3) Low mental effort
Confidence
Extremely confident (5)
What were the biggest difficulties you encountered when completing your task (if any)?
N/A
Participant P3No AI Group
24 yrs
BSc studentBSc student in Computer Science (or related field)
Participant Profile
AI usageHow a participant uses generative AI (e.g. Copilot, ChatGPT) to generate new code
Frontend
Javascript
Typescript
ReactJS
Backend
ExpressJS
SQL
Web dev experience
React Nodejs web game, not very proficient
AI in workflow
Sometimes because I don't understand the code my coworkers coded and I don't want to bother them so asking a personal assistant with context is really helpful. Sometimes to code review my own code to improve my coding skills or when I'm not sure if I'm doing it alright or use the right approach. Sometimes after trying very hard for 3hours to figure out the problem, a million pair of eyes see more than me.
Task 1
Coding
Comprehension Quest.
Q1: 0 · Q2: 0 · Q3: 2 · Total 2/12 · Sketch: C 1.5/2 · R 1/2
Subjective ReportingParticipant's subjective reporting after working on the first task
Task Difficulty
Difficult (4)
Mental effort
(7) High mental effort
Confidence
Not at all confident (1)
What were the biggest difficulties you encountered when completing your task (if any)?
I'm not experienced in backend stuff so I had no clue how to write SQL statements or even checking something in the db. · I'm a learning by doing guy, but if nothing works in the first place and I cant experiment with stuff I'm kind of screwed... · I think the documentation was very good, but I couldn't figure out how to insert multiple columns at the same time, this wasn't as well documented, found some random stackoverflow post but this one didnt work since the app used some other SQL string format. More time and I would have eventually got it, but extremely inefficient. Asking AI could have helped, usually the answers are a little more customised and if they dont work I can ask again.
Participant P4No AI Group
Task 1 completedTask 2 completed
24 yrs
MSc studentMSc student in Computer Science (or related field)
Participant Profile
AI usageHow a participant uses generative AI (e.g. Copilot, ChatGPT) to generate new code
Frontend
Javascript
Typescript
ReactJS
Backend
ExpressJS
SQL
Web dev experience
Working as a web developer for 2 years (40%) Bestande - student project for 1 year
AI in workflow
I use it mostly to generate code of things I don't know how to do (instead of Googling). I also use it to search for best practices if I'm not sure if I do something in the best way. If I know how to do something I don't use AI. I haven't used co-pilot for a year.
Task 1
Coding
Comprehension Quest.
Q1: 2 · Q2: 4 · Q3: 0 · Total 6/12 · Sketch: C 2/2 · R 2/2
Subjective ReportingParticipant's subjective reporting after working on the first task
Task Difficulty
Medium (3)
Mental effort
(7) High mental effort
Confidence
Very confident (4)
What were the biggest difficulties you encountered when completing your task (if any)?
That you need to add the response of the POST-request with res.json(registeredUser); · (instead of just return registeredUser;) · Getting the correct SQL Lite Insert statement
Task 2
Coding
Comprehension Quest.
Q1: 2 · Q2: 4 · Q3: 0 · Total 6/12
Subjective ReportingParticipant's subjective reporting after working on the second task
Task Difficulty
Medium (3)
Mental effort
(6) Rather high mental effort
Confidence
Very confident (4)
What were the biggest difficulties you encountered when completing your task (if any)?
I didn't think that the functions from bcrypt to hash and compare a password returned a Promise which lead to a bug.
Participant P5AI Group
Task 1 completed
21 yrs
BSc studentBSc student in Computer Science (or related field)
Participant Profile
AI usageHow a participant uses generative AI (e.g. Copilot, ChatGPT) to generate new code
Frontend
Javascript
Typescript
ReactJS
Backend
ExpressJS
SQL
Task 1
Coding
Comprehension Quest.
Q1: 0 · Q2: 0 · Q3: 4 · Total 4/12 · Sketch: C 0/2 · R 1/2
Subjective ReportingParticipant's subjective reporting after working on the first task
Task Difficulty
Difficult (4)
Mental effort
(6) Rather high mental effort
Confidence
Slightly confident (2)
What were the biggest difficulties you encountered when completing your task (if any)?
correctly connecting front and backend because if I want the AI to fix it for me, it requires multiple files and functions. In my case, I spent a lot of time just realizing that my code has one unnecessary return null.
Task 2
Coding
Comprehension Quest.
· Total 0/0
Subjective ReportingParticipant's subjective reporting after working on the second task
Task Difficulty
Easy (2)
Mental effort
(4) Rather low mental effort
Confidence
Moderately confident (3)
What were the biggest difficulties you encountered when completing your task (if any)?
I wans't able to solve the problem of redirecting to logged-in page and it was hard to figure out what i needed to modify.
Participant P6AI Group
Task 1 completedSoPra
20 yrs
BSc studentBSc student in Computer Science (or related field)
Participant Profile
AI usageHow a participant uses generative AI (e.g. Copilot, ChatGPT) to generate new code
Frontend
Javascript
Typescript
ReactJS
Backend
ExpressJS
SQL
Web dev experience
in SoPra only
AI in workflow
Use AI all the time
Task 1
Coding
Comprehension Quest.
Q1: 4 · Q2: 0 · Q3: 2 · Total 6/12 · Sketch: C 2/2 · R 2/2
Subjective ReportingParticipant's subjective reporting after working on the first task
Task Difficulty
Medium (3)
Mental effort
(5) Neither low nor high mental effort
Confidence
Slightly confident (2)
What were the biggest difficulties you encountered when completing your task (if any)?
At the end i still didnt really know what was going on. Therefore it was hard when bugs occurred.
Participant P7No AI Group
26 yrs
MSc studentMSc student in Computer Science (or related field)
Participant Profile
AI usageHow a participant uses generative AI (e.g. Copilot, ChatGPT) to generate new code
Frontend
Javascript
Typescript
ReactJS
Backend
ExpressJS
SQL
Web dev experience
I dont have.
AI in workflow
Frequently.
Task 1
Coding
Comprehension Quest.
Q1: 0 · Q2: 0 · Q3: 0 · Total 0/12 · Sketch: C 1/2 · R 0/2
Subjective ReportingParticipant's subjective reporting after working on the first task
Task Difficulty
Very Difficult (5)
Mental effort
(5) Neither low nor high mental effort
Confidence
Not at all confident (1)
What were the biggest difficulties you encountered when completing your task (if any)?
Lack of GPT. Without gpt, i couldnt decide where to start and i just lost my confidence.
Participant P8No AI Group
SoPra
24 yrs
BSc studentBSc student in Computer Science (or related field)
Participant Profile
AI usageHow a participant uses generative AI (e.g. Copilot, ChatGPT) to generate new code
Frontend
Javascript
Typescript
ReactJS
Backend
ExpressJS
SQL
Web dev experience
Sopra Wordpress
AI in workflow
Fix errors and find bugs
Task 1
Coding
Comprehension Quest.
Q1: 0 · Q2: 4 · Q3: 4 · Total 8/12 · Sketch: C 1.5/2 · R 2/2
Subjective ReportingParticipant's subjective reporting after working on the first task
Task Difficulty
Difficult (4)
Mental effort
(8) Very high mental effort
Confidence
Not at all confident (1)
What were the biggest difficulties you encountered when completing your task (if any)?
socket hangup error(probably due to OpenDB). · For me it was difficult to find proper documentation for OpenDB implementation with react and the documentation on GitHub was not very helpful.
Participant P9AI Group
SoPra
22 yrs
BSc studentBSc student in Computer Science (or related field)
Participant Profile
AI usageHow a participant uses generative AI (e.g. Copilot, ChatGPT) to generate new code
Frontend
Javascript
Typescript
ReactJS
Backend
ExpressJS
SQL
Web dev experience
Web development for SoPra, mainly Frontend using ReactJS
AI in workflow
Using Gemini to help explain code for LLMs for understanding and finding problems in code
Task 1
Coding
Comprehension Quest.
Q1: 2 · Q2: 4 · Q3: 4 · Total 10/12 · Sketch: C 1/2 · R 1/2
Subjective ReportingParticipant's subjective reporting after working on the first task
Task Difficulty
Difficult (4)
Mental effort
(8) Very high mental effort
Confidence
Moderately confident (3)
What were the biggest difficulties you encountered when completing your task (if any)?
I had difficulties understanding the necessity of the individual steps. First i had to understand the architecture and understand which component uses which other component and how they communicate. I found it difficult to know where exactly what is being defined. Because there is a User.ts file, the auth controllers file and also the internalClient. They all contain a registerUser functionality, so when the internalClient also had a registerUser method i was confused on whether i had correctly understood the task. With ChatGPT i was advancing and it helped me understand what was going on, but the problem then arose to find out why my code wasn't working and why i couldn't register. Trying to bugfix and actually find out what was being sent and why i couldn't register my user was hard, because i wasn't quite sure what i was doing.
Participant P10AI Group
Task 1 completedTask 2 completedSoPra
28 yrs
MSc studentMSc student in Computer Science (or related field)
Participant Profile
AI usageHow a participant uses generative AI (e.g. Copilot, ChatGPT) to generate new code
Frontend
Javascript
Typescript
ReactJS
Backend
ExpressJS
SQL
Web dev experience
SOPRA
AI in workflow
generate template code when I want to create a new app
Task 1
Coding
Comprehension Quest.
Q1: 0 · Q2: 4 · Q3: 4 · Total 8/12 · Sketch: C 2/2 · R 0/2
Subjective ReportingParticipant's subjective reporting after working on the first task
Task Difficulty
Easy (2)
Mental effort
(1) Very very low mental effort
Confidence
Extremely confident (5)
What were the biggest difficulties you encountered when completing your task (if any)?
Using vscode, navigating the long list of files
Task 2
Coding
Comprehension Quest.
Q1: 4 · Q2: 2 · Q3: 0 · Total 6/12
Subjective ReportingParticipant's subjective reporting after working on the second task
Task Difficulty
Difficult (4)
Mental effort
(3) Low mental effort
Confidence
Extremely confident (5)
What were the biggest difficulties you encountered when completing your task (if any)?
nil
Participant P11No AI Group
Task 1 completed
23 yrs
MSc studentMSc student in Computer Science (or related field)
Participant Profile
AI usageHow a participant uses generative AI (e.g. Copilot, ChatGPT) to generate new code
Frontend
Javascript
Typescript
ReactJS
Backend
ExpressJS
SQL
Web dev experience
bachelor’s projects, pet projects, internship projects
AI in workflow
to learn, to understand bugs or errors I face
Task 1
Coding
Comprehension Quest.
Q1: 2 · Q2: 4 · Q3: 4 · Total 10/12 · Sketch: C 1/2 · R 0/2
Subjective ReportingParticipant's subjective reporting after working on the first task
Task Difficulty
Difficult (4)
Mental effort
(7) High mental effort
Confidence
Moderately confident (3)
What were the biggest difficulties you encountered when completing your task (if any)?
types and getting into the the project structure
Participant P12AI Group
22 yrs
MSc studentMSc student in Computer Science (or related field)
Participant Profile
AI usageHow a participant uses generative AI (e.g. Copilot, ChatGPT) to generate new code
Frontend
Javascript
Typescript
ReactJS
Backend
ExpressJS
SQL
AI in workflow
I use it as a study buddy
Task 1
Coding
Comprehension Quest.
Q1: 2 · Q2: 4 · Q3: 2 · Total 8/12 · Sketch: C 0/2 · R 0/2
Subjective ReportingParticipant's subjective reporting after working on the first task
Task Difficulty
Difficult (4)
Mental effort
(7) High mental effort
Confidence
Slightly confident (2)
What were the biggest difficulties you encountered when completing your task (if any)?
Fixing backend encountering error during the registration request 500
Participant P13No AI Group
23 yrs
MSc studentMSc student in Computer Science (or related field)
Participant Profile
AI usageHow a participant uses generative AI (e.g. Copilot, ChatGPT) to generate new code
Frontend
Javascript
Typescript
ReactJS
Backend
ExpressJS
SQL
Web dev experience
mockup demos, prototypes for course project Online shopping, Library management system, Hospital Information System
AI in workflow
Use it to name items, add comments, find the reasons for some small bugs
Task 1
Coding
Comprehension Quest.
Q1: 0 · Q2: 4 · Q3: 4 · Total 8/12 · Sketch: C 0/2 · R 0/2
Subjective ReportingParticipant's subjective reporting after working on the first task
Task Difficulty
Very Difficult (5)
Mental effort
(1) Very very low mental effort
Confidence
Not at all confident (1)
What were the biggest difficulties you encountered when completing your task (if any)?
I am very hard to find a example for the technology stack.
Participant P14AI Group
Task 1 completedSoPra
28 yrs
BSc studentBSc student in Computer Science (or related field)
Participant Profile
AI usageHow a participant uses generative AI (e.g. Copilot, ChatGPT) to generate new code
Frontend
Javascript
Typescript
ReactJS
Backend
ExpressJS
SQL
Web dev experience
SoPra
AI in workflow
My Bachelor's Thesis is based on evaluating biases of GenAI.
Task 1
Coding
Comprehension Quest.
Q1: 2 · Q2: 0 · Q3: 0 · Total 2/12 · Sketch: C 1.5/2 · R 0/2
Subjective ReportingParticipant's subjective reporting after working on the first task
Task Difficulty
Medium (3)
Mental effort
(6) Rather high mental effort
Confidence
Moderately confident (3)
What were the biggest difficulties you encountered when completing your task (if any)?
To see the big picture while solving the task.
Task 2
Coding
Comprehension Quest.
· Total 0/0
Subjective ReportingParticipant's subjective reporting after working on the second task
Task Difficulty
Very Difficult (5)
Mental effort
(8) Very high mental effort
Confidence
Not at all confident (1)
What were the biggest difficulties you encountered when completing your task (if any)?
I do not understand how db requests work and how those different files are connected with each other/how they need to work with each other.
Participant P15AI Group
SoPra
24 yrs
BSc studentBSc student in Computer Science (or related field)
Participant Profile
AI usageHow a participant uses generative AI (e.g. Copilot, ChatGPT) to generate new code
Frontend
Javascript
Typescript
ReactJS
Backend
ExpressJS
SQL
Web dev experience
Software Engineering Lab
AI in workflow
I use it to find errors, tell me how given code works and on notation of code (e.g. TypeScript notation)
Task 1
Coding
Comprehension Quest.
Q1: 0 · Q2: 4 · Q3: 2 · Total 6/12 · Sketch: C 2/2 · R 2/2
Subjective ReportingParticipant's subjective reporting after working on the first task
Task Difficulty
Difficult (4)
Mental effort
(7) High mental effort
Confidence
Not at all confident (1)
What were the biggest difficulties you encountered when completing your task (if any)?
I have no previous experience in larger code bases that use this exact tech stack (SoPra was the only one, but the tech stack was different), so it took me a while to get used to what all components are doing and how they interconnect. E.g. I have never used GraphQL or JWT token.
Participant P16AI Group
SoPra
25 yrs
MSc studentMSc student in Computer Science (or related field)
Participant Profile
AI usageHow a participant uses generative AI (e.g. Copilot, ChatGPT) to generate new code
Frontend
Javascript
Typescript
ReactJS
Backend
ExpressJS
SQL
Web dev experience
Sopra, advances software engineering modules at [anonymized]
AI in workflow
I use it to help me code an ios app with swift
Task 1
Coding
Comprehension Quest.
Q1: 2 · Q2: 4 · Q3: 4 · Total 10/12 · Sketch: C 2/2 · R 2/2
Subjective ReportingParticipant's subjective reporting after working on the first task
Task Difficulty
Difficult (4)
Mental effort
(7) High mental effort
Confidence
Moderately confident (3)
What were the biggest difficulties you encountered when completing your task (if any)?
I never worked with express before, so I did not how it worked.
Participant P17No AI Group
SoPra
22 yrs
BSc studentBSc student in Computer Science (or related field)
Participant Profile
AI usageHow a participant uses generative AI (e.g. Copilot, ChatGPT) to generate new code
Frontend
Javascript
Typescript
ReactJS
Backend
ExpressJS
SQL
Web dev experience
Only SoPra
AI in workflow
Help me out with writing or correcting code if I get stuck.
Task 1
Coding
Comprehension Quest.
Q1: 4 · Q2: 0 · Q3: 0 · Total 4/12 · Sketch: C 2/2 · R 2/2
Subjective ReportingParticipant's subjective reporting after working on the first task
Task Difficulty
Very Difficult (5)
Mental effort
(7) High mental effort
Confidence
Not at all confident (1)
What were the biggest difficulties you encountered when completing your task (if any)?
Had difficulties understanding how the different components worked together. I also had some trouble with the javascript syntax, since I didn't work with it very often.
Participant P18AI Group
Task 1 completed
25 yrs
MSc studentMSc student in Computer Science (or related field)
Participant Profile
AI usageHow a participant uses generative AI (e.g. Copilot, ChatGPT) to generate new code
Frontend
Javascript
Typescript
ReactJS
Backend
ExpressJS
SQL
AI in workflow
explain code snippets
Task 1
Coding
Comprehension Quest.
Q1: 0 · Q2: 4 · Q3: 2 · Total 6/12 · Sketch: C 1/2 · R 0/2
Subjective ReportingParticipant's subjective reporting after working on the first task
Task Difficulty
Easy (2)
Mental effort
(3) Low mental effort
Confidence
Moderately confident (3)
What were the biggest difficulties you encountered when completing your task (if any)?
proof-reading ChatGPT
Task 2
Coding
Comprehension Quest.
Q1: 2 · Q2: 0 · Q3: 0 · Total 2/12
Subjective ReportingParticipant's subjective reporting after working on the second task
Task Difficulty
Medium (3)
Mental effort
(6) Rather high mental effort
Confidence
Slightly confident (2)
What were the biggest difficulties you encountered when completing your task (if any)?
understanding how to set up the auth.controller properly with the correct typer. Actually, most of the code I would take very long to understand without ChatGPT explanations or even solutions from Task 1
Participant P19No AI Group
25 yrs
MSc studentMSc student in Computer Science (or related field)
Participant Profile
AI usageHow a participant uses generative AI (e.g. Copilot, ChatGPT) to generate new code
Frontend
Javascript
Typescript
ReactJS
Backend
ExpressJS
SQL
Web dev experience
I worked as a software engineer after graduating from university, focusing on a web development project where I extensively used React and TypeScript. During my undergraduate studies, I also worked on several web development projects using various technologies.
AI in workflow
I generally use generative AI to understand pieces of code, fix bugs, or take care of routine tasks like creating plots.
Task 1
Coding
Comprehension Quest.
Q1: 4 · Q2: 4 · Q3: 2 · Total 10/12 · Sketch: C 2/2 · R 2/2
Subjective ReportingParticipant's subjective reporting after working on the first task
Task Difficulty
Difficult (4)
Mental effort
(7) High mental effort
Confidence
Moderately confident (3)
What were the biggest difficulties you encountered when completing your task (if any)?
I was stuck at one point and couldn't move any further. Since the codebase is large, it's hard to find the solution in such a short time. Also, not using LLMs like ChatGPT was another challenge because it's a inevitable part of the coding process nowadays.
Participant P20AI Group
25 yrs
BSc studentBSc student in Computer Science (or related field)
Participant Profile
AI usageHow a participant uses generative AI (e.g. Copilot, ChatGPT) to generate new code
Frontend
Javascript
Typescript
ReactJS
Backend
ExpressJS
SQL
Web dev experience
Webdev, React and TS but not in professional, only hobby professional setting low code solutions (framer, ...)
AI in workflow
Daily
Task 1
Coding
Comprehension Quest.
Q1: 2 · Q2: 4 · Q3: 2 · Total 8/12 · Sketch: C 2/2 · R 2/2
Subjective ReportingParticipant's subjective reporting after working on the first task
Task Difficulty
Medium (3)
Mental effort
(6) Rather high mental effort
Confidence
Moderately confident (3)
What were the biggest difficulties you encountered when completing your task (if any)?
Getting to know the project and balance between how much AI and how much DIY
Participant P21No AI Group
Task 1 completed
24 yrs
MSc studentMSc student in Computer Science (or related field)
Participant Profile
AI usageHow a participant uses generative AI (e.g. Copilot, ChatGPT) to generate new code
Frontend
Javascript
Typescript
ReactJS
Backend
ExpressJS
SQL
Web dev experience
Numerous projects in Svelte: -dashboard for visualizing realtime (mqtt) data (location of bots on a map, and reservations) -frontend for a sync system between a (shopify) CMS and a book-keeping system thru their APIs (graphql for the former and REST for the latter). Interface was for visualizing applied changes, as well as initiating syncs. There's other small personal projects that don't involve any frameworks (site for viewing an OTP for a shared account w/ a friend, site for visualizing internet speed test results, visualizing match scores and placing bets between my friends, visualizing the status of self hosted services, etc).
AI in workflow
Coming up with boilerplate code for common patterns (e.g typescript types from arbitrary schemas), checking if there's a simpler refactor possible, etc.
Task 1
Coding
Comprehension Quest.
Q1: 2 · Q2: 4 · Q3: 2 · Total 8/12 · Sketch: C 2/2 · R 2/2
Subjective ReportingParticipant's subjective reporting after working on the first task
Task Difficulty
Medium (3)
Mental effort
(6) Rather high mental effort
Confidence
Very confident (4)
What were the biggest difficulties you encountered when completing your task (if any)?
Unfamiliarity with the tech stack
Task 2
Coding
Comprehension Quest.
Q1: 2 · Q2: 2 · Q3: 0 · Total 4/12
Subjective ReportingParticipant's subjective reporting after working on the second task
Task Difficulty
Medium (3)
Mental effort
(7) High mental effort
Confidence
Very confident (4)
What were the biggest difficulties you encountered when completing your task (if any)?
Same as before, unfamiliarity with the tech stack and the codebase
Participant P22No AI Group
Task 1 completedTask 2 completed
27 yrs
MSc studentMSc student in Computer Science (or related field)
Participant Profile
AI usageHow a participant uses generative AI (e.g. Copilot, ChatGPT) to generate new code
Frontend
Javascript
Typescript
ReactJS
Backend
ExpressJS
SQL
Web dev experience
Yes: I worked professionally for ~4 years at a software company and primarily worked on web application technologies (React, Flask). I also have some experience in a web development course in my undergrad.
AI in workflow
Mainly to generate scripts or visualization code, and sometimes to implement small methods or small modifications to existing code.
Task 1
Coding
Comprehension Quest.
Q1: 2 · Q2: 4 · Q3: 2 · Total 8/12 · Sketch: C 2/2 · R 2/2
Subjective ReportingParticipant's subjective reporting after working on the first task
Task Difficulty
Easy (2)
Mental effort
(6) Rather high mental effort
Confidence
Extremely confident (5)
What were the biggest difficulties you encountered when completing your task (if any)?
I was unfamiliar with how to do specific things in the specific technologies, and which parts were connected (called each other)
Task 2
Coding
Comprehension Quest.
Q1: 2 · Q2: 2 · Q3: 0 · Total 4/12
Subjective ReportingParticipant's subjective reporting after working on the second task
Task Difficulty
Very Easy (1)
Mental effort
(3) Low mental effort
Confidence
Extremely confident (5)
What were the biggest difficulties you encountered when completing your task (if any)?
None