State Of Linux Usability 2020

State Of Linux Usability 2020

TL;DR

We’ve carried out a series of daily tasks on TOP 20 Linux distros as well as Windows and macOS to test whether Linux has a chance to compete in daily use space. And the answer is - Yes since neither Mac nor Windows came on the top. There is still a very long way to go for all of them, however.

Motivation

I’ve been a Linux evangelist and a strong user (as in regular usage, not just for servers) for almost two decades since I’ve been introduced to Mandrake Linux with a mysterious penguin in the background.

That’s why I’ve been the one who consistently annoys family, friends, and colleagues until they’re forced to give up their bellowed non-Linux life.

Of course, the change is often difficult since they are thrown in a never-before-seen world with never-before-understood rules. Being a CLI-first person, I often overlook what is so hard about it.

But over the years, I’ve received fewer and fewer complaints, and since there have been almost no cases of people going back unless it’s for work, I’ve decided to do a proper test to see whether someone who’s not a technician can live a proper Linux life.

The Subject

I wanted the subject to be someone with little-to-no technical skills/background as well as available for close supervision during the tests (and quarantine) to get accurate results.

So the background of the chosen subject is: a human, an illustrator, a long-time Mac-only user, spends 40% of work drawing on paper, 30% on Adobe suite and 30% on social media, saw me working numerous times and referred to the terminal as a “mysterious black window,” never installed an OS before, required assistance towards installing Adobe software, not a native English speaker.

What I want to emphasize is that the subject is not a tech-friendly person at all.

So before we started I had to teach the subject a few simple things: what ISO is, how to useEtcherand how to select USB as a bootable device.

No further advice or assistance has been given.

Of course, we need to put a disclaimer here - the subject is human after all and humans learn over time so that test results in similar desktop environments or derivatives can be influenced by previous experience, but from the obvious dissatisfaction that has been witnessed, I can confirm that previous experience has not benefited too much.

Rules

General rules for making the results more relevant:

Tasks

There are 16 tasks in total:

Each task is measured by the length of time it took to complete with additional subject scores for general UX (system intuitiveness, tool availability, etc.) and UI (out-of-box interface esthetics).

This will split evaluation into 4 total categories:

Evaluation

Each system is ranked from the best to the worst in each category and given a point for the respective reverse position (1-22). If the scores are equal, the maximum is given for the respective placement and continues from the minimum (e.g. 18, 17, 16, 16, 16, 13, 12, 11).

Tools

To be completely consistent with the test results, the same machine and network were used for all tests.

Machine: Thinkpad T480s (i5-8250U, 256GB Samsung 970 Plus NVMe, 16GB RAM). For macOS installation late-2017 MBP (i5, 128GB, 8GB RAM) was used.

Network: A dedicated (all other devices were disconnected) 100mbps, pretty stable, always through LAN.

Data Set

The data set consisted of Windows and macOS as a control group, then picked up TOP 20 distros fromDistroWatchover 12 months, removed Arch as it would require CLI knowledge and includedRegolithinstead (just because it’s my go-to distro).

So the final list is macOS, Windows, MX Linux, Manjaro, Mint, Debian, Ubuntu, Elementary, Solus, Fedora, Zorin, Deppin, Antix, KDE Neon, OpenSUSE, CentOS, Pop OS, ArcoLinux, PCLinuxOS, Kali, ReactOS, Regolith.

I am aware that most of the tasks are software-specific activity, so the test is mostly about OS compatibility with day-to-day apps and means to obtain them.

What Went Wrong

Since all tasks were expected to go smoothly by default, I have only noted failures/pain points when using a particular system that I noticed when observing the subject.

I know there are workarounds for some problems, different paths and so on. But if they have not been discovered by the subject in time, they are too hidden from the user and are therefore considered invalid.

So the results are as follows:

Failed Tests:

Antix
openSUSE
CentOS
ArcoLinux
Kali
ReactOS

Successful Tests:

macOS
Windows
MX Linux
Manjaro
Mint
Debian
Ubuntu
Elementary
Solus
Fedora
Zorin
Deppin
KDE Neon
PCLinuxOS
Regolith

Final Results

I have decided to provide two sets of results - for overall score and score without installation evaluation for cases where the long-term use is more important than the time to get it running.

Bear in mind that this study only affected the general light-use cases of daily software consumption. Therefore, I will not give any further opinions as to why one is better than the other, and vice versa, since my personal insights may not be consistent with the general needs of everyday users.

The key problem tends to be a basic UX of software installation and management. Proper section naming or broader default repositories would solve most of the problems that day-to-day users are facing if sticking to Linux.

A thought regarding Windows/macOS score - there may be some issues with the way the test was designed, but I think that these results are pretty accurate.

Overall Score

Distro/OSVersionDEScore
Pop OS20.04Gnome81
Mint19.3Cinnamon75
Ubuntu20.04Gnome72
Deppin20 BetaDeppin71
Zorin15.2Gnome65
Manjaro20XFCE61
MacOSCatalinaDefault59
Regolith1.4i358
Elementary5.1.3Pantheon54
KDE Neon20200430KDE53
Windows10Default53
Debian10.3Gnome53
Solus4.1Budgie51
Fedora32Gnome50
PCLinuxOS2020.03MATE42
MX Linux19.1XFCE34
ArcoLinux20.4.11XFCE15
CentOS8.1Gnome12
Kali2020.1bGnome10
openSUSE15.1Gnome5
Antix19Fluxbox0
ReactOS0.4.13Explorer0

Overall Score Without Install

Distro/OSVersionDEScore
Deppin20 BetaDeppin64
Pop OS20.04Gnome61
Ubuntu20.04Gnome61
Zorin15.2Gnome56
MacOSCatalinaDefault56
Mint19.3Cinnamon53
Windows10Default49
KDE Neon20200430KDE47
Debian10.3Gnome45
Manjaro20XFCE42
Regolith1.4i337
Solus4.1Budgie37
Elementary5.1.3Pantheon36
Fedora32Gnome33
PCLinuxOS2020.03MATE26
MX Linux19.1XFCE21
ArcoLinux20.4.11XFCE0
CentOS8.1Gnome0
Kali2020.1bGnome0
openSUSE15.1Gnome0
Antix19Fluxbox0
ReactOS0.4.13Explorer0

Raw results are available here:linuxusability/2020/results

Final Thoughts

In the end, one of the key reasons why we have a handful of Linux distributions available is that each of them is designed to solve a specific problem and serves a specific function for a specific audience.

A lot of multi-purpose distros aim to be Windows-like or MacOS-like to ease the Linux transition. Although there is nothing wrong with it, the true strength of Linux lies in its uniqueness, not its similarity, and I think that this uniqueness could be “sold” to the consumer as long as it is presented/explained correctly (i.e. recall your first move from Symbian to Android or Windows to macOS).

However, I do believe that the Linux ecosystem as a whole is certainly ready to replace Windows / macOS not just for day-to-day users, but also for educational/corporate/government use.

On a final note, I’d like to encourage you to try using a Linux system for a week and if you’re already a Linux user then go and make someone else use Linux for a week!

$ echo '#preachingpenguins'
$ exit