Overview
My team tested the Grubhub and DoorDash mobile applications (iOS). The main objective of this study was to test the usability of both food delivery applications with users. Our tasks covered browsing and searching for restaurants, finding deals, and updating account information. We will present detailed tasks objectives in the procedure section below.
The comparison of these two mobile applications highlighted the effective elements of each app and the areas that are falling short compared to their competitor. We summarized the test data, provided conclusions about these applications through statistics, listed both usability problems and positive findings, and offered actionable recommendations to improve the applications’ user experience.
METHODOLOGY
PARTICIPANTS
We tested 15 participants. The participant profile was as follows:
Age range: 18-40
Mix of genders
All subjects are iPhone users
All subjects are inclined to use technologies and mobile applications
Novice level for using Grubhub and DoorDash phone application
Goal is to recruit participants with a range of experience with ordering food deliveries
STIMULI
We tested two iOS mobile applications: Grubhub and DoorDash. Grubhub and DoorDash are food delivery applications. The scope of our study is browsing a local restaurant, using filter to search or sort, searching for a specific restaurant and get details, editing account information and finding a deal.
Each users interacted with both Grubhub and DoorDash. Each application testing session has 6 tasks, and each tasks was addressed by a Likert satisfaction scale. The Likert satisfaction scale was presented in Task part.
PROCEDURE
Before testing users, our group wrote a test plan, screener, and moderator guide to assist us with running tests. First, we recruited users and screened users to ensure they fit the profile we were looking to test. The study is within-subjects design, so 15 users were asked to complete 6 tasks in each application and provide their rating of the task using the Likert satisfaction scale. After users completed all task, they were asked to answer wrap-up questions. Half the users completed tasks in Grubhub first, and half the users completed tasks in DoorDash first.
TESTING RESULTS
HIGHLIGHTS
Of the 6 tasks, only 1 task showed a significant difference between Grubhub and DoorDash. The below information highlights the differences in the tasks.
Task 4 Prompt: You forgot to eat lunch, and it is already late at night. You want to find food that will get to your house quickly. What would you do?
Grubhub Results
Task completion rate: 40.00% (6 out of 15)
Time to complete task: Task completion time ranged from 20 seconds to 90 seconds, with an average of 43.00 seconds.
Average satisfaction rating (out of 5): 3.50.
User Comments: User 3 said it was hard for them to clear their search and save the changes. User 14 commented that they attempted to select ‘Deliver ASAP’, then ‘Delivery Min’, and then ‘Delivery Estimate.’ This user said Grubhub could clarify the names of these filters more effectively.
DoorDash Results
Task completion rate: 86.67% (13 out of 15)
Time to complete task: Task completion time ranged from 7 seconds to 27 seconds, with an average of 14.16 seconds.
Average satisfaction rating (out of 5): 4.85.
User Comments: User 1 felt DoorDash has better filters. User 2 felt unsure about the restaurants’ hours and if they will still be open. User 3 was happy to see the app has delivery time beneath each restaurant. User 9 liked that DoorDash listed on the restaurant if they offered free delivery. User 14 was trying to further filter the result by cuisine after the “fastest delivery” but was not able to do so.
Statistical Results:
We conducted a Fisher’s Exact Test to compare the task completion success of each application. For Task 4, Grubhub had 6 of 15 successful users and DoorDash had 13 of 15 successful users. The 2-tailed p-value is .02 and the 1-tailed p-value is .01. Using an alpha rate of .05, the completion rates are less than .05, so there is a significant difference. More participants completed the task through DoorDash than through Grubhub at a significant rate.
Conclusion and Discussion:
DoorDash has a significant difference on this task in terms of task success rate, completion time and satisfaction rate over Grubhub. Thus users are able to complete this task with significantly less time, more successful rate and better experience.
The main reason for users who failed this task on Grubhub is that they are unable to locate the “delivery time” since it is not visible in the filter without tapping on ”sort by”. There are other reasons including the users thought they should select ‘Delivery Minimum’ or ‘Distance’ before they found the correct filter, or selected ‘Deliver my Order ASAP’ which is a feature that allows users to schedule their delivery and does not give them results based on lowest delivery time.
Overall Satisfaction Rating:
After the user completed all 6 tasks in the two applications, they were asked to rate their satisfaction with Grubhub and DoorDash on a scale of 1-5 (with 5 being the most satisfied).
Grubhub Results
Average satisfaction rating (out of 5): The average rating is 3.90 (SD=0.93). 5 users rated their overall satisfaction with Grubhub as a 5 and 4 users rated a 4, which means 60% (9/15) users were satisfied with this application.
DoorDash Results
Average satisfaction rating (out of 5): The average rating is 4.27. All users rated their overall satisfaction with DoorDash as 4 or 5, and one user rated a 3. 93.33% (14/15) users were satisfied with this application.
Statistical Results
We conducted a paired t-test to compare the overall satisfaction rating between Grubhub(M=3.90, SD=0.93) and DoorDash(M=4.27, SD=0.59). The p-value was .15 which indicates that there is no significant difference in the overall satisfaction of these applications among users.
POSITIVE FINDINGS
HIGHLIGHTS
Grubhub Positive Finding
Task 2: You want to get some Mediterranean food and your friend recommended Pita Heaven to you. What would you do to learn more about this restaurant?
Restaurant Recommendations: In task 2, users were asked to locate a specific restaurant. Some users were unable to locate the restaurant, and Grubhub recommended other similar restaurants near their location in the search results. This feature allows users to browse the similar restaurants without having to restart their restaurant search. Many users commented on how they liked this feature.
Task 4: You forgot to eat lunch, and it is already late at night. You want to find food that will get to your house quickly. What would you do?
Filter Options: Several users commented that the filter options on Doordash were simple and clear to understand. In task 4, the majority of the users utilized the sorting by ‘Fastest Delivery’ option when asked to select a restaurant that would get to their house quickly. One user also commented that she preferred the ‘Top Rated’ filter option over Grubhub’s filtering by number of stars.
PROBLEMS AND RECOMMENDATIONS
HIGHLIGHTS
DoorDash Negative Finding and Recommendation
Observation: While using DoorDash for Task 1, a number of users initially missed the filters section.
Source of Problem: The filters section did not stand out from the rest of the content enough that users missed it when initially looking for filters.
Recommendation: Our recommendation would be to use a brightly colored button to help the filters section stand out from the rest of the content on the main page.
Severity Rating: High
Grubhub Negative Finding and Recommendation
Observation: When users were looking for the fastest delivery times in Task 4, there were many issues that they encountered. Some users did not know they could filter the way results were sorted and gave up before finding the ‘Delivery Estimate’. Some users thought they should select ‘Delivery Minimum’ or ‘Distance’ before they found the correct filter. Finally, many users selected ‘Deliver my Order ASAP’ which is a feature that allows users to schedule their delivery and does not give them results based on lowest delivery time. Due the problems mentioned above, Grubhub had a 40% task success rate for this task while DoorDash has 86.6%.
Source of Problem: This problem came from poor labels within the filter section that did not clearly communicate with users what the filters allowed the users to do.
Recommendation: Our recommendation would be to rename ‘Deliver My Order” to “Schedule My Order”, so users understand what this section is capable of. We also recommend adjusting the filter ‘Delivery Minimum’ to ‘Delivery Price Minimum’ and adjusting ‘Delivery Estimate’ to ‘Delivery Time Estimate’ so users do not confuse the two.
Severity Rating: High
