Confidence in Status app - weekly poll


#1

TL:DR - we’re planning to send out a 5 minute weekly pulse survey asking you to share your feedback on how the app worked (or didn’t) for you.

As a user of the Status app, your experiences of the highs and lows of using the app are a valuable source of feedback. Especially because we’ve made a huge leap in switching off Slack and dogfooding the app together. We know it hasn’t been easy - and we want to make sure we’re making the best of this opportunity.

Experienced app data loss? Or maybe you had a joyful experience using a new feature? The survey will be an avenue enabling you to report back each week. We’ll email you each Friday with this survey, and ask you to answer 9 optional questions to give us as much or as little detail about using the app as you choose. The questions we ask will be around issues encountered and your habits when using Status.

Why we’re doing this

Communication is a challenge that’s raised by multiple people, and we want to get more frequent actionable data on how people interact with Status to better understand specific problem areas.

Why your survey responses are important

This is our primary communication tool and we want to make the experience of using it as pleasant and engaging as possible. We want to track if improvements to the app are being felt over time by users (you!)

What will we be using survey responses for

Troubleshooting common issues and prioritising fixes/enhancements for the app. The results will be made available to everyone.

Suggestions?

Take a look at the survey and feel free to suggest any changes you’d like to see (either to the question set or the survey plan itself). Cheers!


#2

The results are in

Thanks to the 31 people who gave us some insight into their interactions with Status app last week! Amazing turnout :raised_hands:t2:

Raw survey data here, and below are some summaries (apologies for mismatched-looking copy pastes, I tried to embed the charts in this post directly, but ran up against a 413 error with too much html):


17

36
52

Other issues identified as per ^:

  • Yes, other issue: hitting the ‘Retry’ button when mail severs fail became a habit of mine now

  • The desktop app doesn’t work on my mac from last week.

  • Syncing across devices. My own messages on one platform either do not show up or trigger a Push Notification on others.

  • too many notifications

  • Desktop app is loosing mailserver connection when i switching off/on VPN

  • I am still unable to run desktop. I havent been able to in 3 weeks. It crashes upon opening and i get an error message that my data cannot be decrypted. Ive tried uninstalling and removing from apps as volodymy instructed but it just again when i re-install (stable version or nightly_

  • I haven’t been able to obtain mailserver messages on Android.

00

Is there anything in the app that you think should be fixed as a priority? [17 responses]

  • mailserver / offline messaging

  • Performance and message reliability

  • @mentions

  • message reliability in status desktop, battery drain on mobile

  • Yes, my confidence in receiving all messages is low, either we fix that or make an ephemeral messaging app where messages get deleted after 24h and don’t bother

  • Speed to app entry either from tap icon from home screen or from push notifications, needs to be super fast

  • The sync feature doesn’t sync my contacts and messages. Can find only other’s messages.

  • Mail servers and message reliability

  • people mentions in the chat, like @Jarrad -> he gets a ping

  • Syncing, which is specific to our use and therefore not a priority IMO.

  • switching between tabs (screens) time, visually it MUST be instant

  • notifications spam

Any other feedback to share? [9 responses]

  • Would love to not get battery warnings and want faster synchronisation

  • This week’s townhall was meaningful.

  • It’s not been fun trying to find a mailserver that works from Africa…

  • Each swarm should review suggested priorities for how critical they are to us and relevant to others. My own rules for prioritization: 1) Blocker for core contributors 2) Retention of non-core contributors 3) Adoption blockers for new users

  • dark mode for status desktop, my eyes are BLEEDING

  • I find that the mobile app has improved massively in the past months. Well done everyone!

  • Mobile app works fine, had mailserver connection issues on Desktop

  • File sharing, or at least image sharing should be added. Mentioning should be added as first priority

  • can anyone please help me with desktop? Ive tried working with the desktop team but maybe im just doing something wrong…not a dev :wink:


#3

Changing the screen’s colour temperature should help:
http://jonls.dk/redshift/
https://justgetflux.com/
https://github.com/stefantalpalaru/iccloader


#4

thanks for the insightful data.

Would it be possible to include a link to report issues in the survey (or ask a follow up question “Have you reported the issue on github” )? A few issues have been reported in the survey, but they haven’t been reported before or we lack of logs, which makes debugging and therefore fixing them really hard.
Thanks!


#5

that is a great initiative!

I have 2 points of feedback:

  • it is a bit hard is that Desktop and Mobile are mixed together in the “issues” section, e.g. I had data losses on Desktop but never on mobile, same with memory usage, it is irrelevant on mobile. I’d love to see 2 sections there;
  • I’m so used to the scale where larger number means better, that I almost filled the values wrongly :slight_smile:

/ Igor


#6

Hi @cammellos and @igor - thanks for your input! :slight_smile:

I made the following updates to the survey:

  • Added a field where the respondent can share a link to any GH issues reported
  • Differentiated with two different fields issues experienced in Desktop versus Mobile.

I didn’t invert the 1-5 scale for issues on the basis that doing that would make it a little bit tricky to compare future results against what we already collected, but lmk if you feel strongly about it and I will change them.

PTAL at the survey here and feel free to amend / add questions as you see fit :slight_smile:

Cheers!


#7

I like this one better! thanks a lot!

tbh, it will be tricky to compare anyway, just because the previous ones were joint for Mobile and Desktop and the new are split. but I don’t have a super strong opinion on that one.


#8

Could you add email as a mean of communication? Looks like 3 different people used 3 different spelling for it (mail, email, Mail)


#9

Great initiative @ceri , super insightful and definitely needed :slight_smile:


#10

Good suggestion, thanks @yenda! I added a few commonly-answered selections (e.g. email, Telegram etc.) as options. :slight_smile:


#11

W/e 15 March results

  • Responses: 27
  • Raw data here

33
43 51 56
58

  • The Desktop app sometimes stops receiving messages, probably after a laptop standby. Also heard from other people that they only received my messages much later on.
  • Didn’t use desktop app since I’m on Windows and the windows build hasn’t been updated in a while
  • Not syncing other device names (shows No Info on other devices)
  • Performance of switching between chats

59

  • Bandwidth usage when I was not interested in messages at all but my wallet
  • Not syncing other device names (shows No Info on other devices)

26
33
11

Is there anything in the app that you think should be fixed as a priority?

  • reliability, ui

  • Currently it is bugging me that if I’m typing and hit “sync” on top, the buttom bar is only displayed after hiding the text bar (ios). That is annoying.

  • Message Reliability, Speed at entry points to Status

  • Resource usage

  • Performance switching channels in Desktop, messages silently not being received after the app has been running for a long time

  • contact permissions, browser

  • https://github.com/status-im/status-react/issues/6812 - facing this issue each day. Doesn’t look hard to be fixed and will improve UX for desktop!

  • Having a stable Windows build would be the major part of me using Status on Desktop

  • Message reliability & sync

  • Code blocks are a plus, keyboard navigation (Cmd+K to quickly switch around)

  • https://github.com/status-im/status-react/issues/7722

Any other feedback to share?

  • hard to answer questions because mobile and desktop are of very different quality

  • Getting better each week, excited for reactions to come along!

  • We’re on the right path! Yay!!

  • For desktop, and for team efficiency, I think what we need most are mentions and search

  • No


#12

thanks for the data!

For whoever has syncing problems, looks like you are using the release version of desktop, which does not propagate info (the feature has been introduced later, so it will show as No info). In any case, contact me here on status directly (and anonymously if you wish) and we can have a look, it’d be helpful to see if there’s an actual problem or just a matter of versioning, thanks!

0x0424a68f89ba5fcd5e0640c1e1f591d561fa4125ca4e2a43592bc4123eca10ce064e522c254bb83079ba404327f6eafc01ec90a1444331fe769d3f3a7f90b0dde1


#13

I recommend another way of visualizing the time spent in the other app vs status as the pie chart is hard to read with the data set up as is.

Perhaps some sort of point system where the time spent in status vs other app is a scale of 1 to 10 further weighted by the respondent’s engagement with Status in the first question and second question (whichever is more frequent counts), i.e. if hourly x1.2, if daily x1, etc. Then summarize the points, calculate percentage from theoretical maximum excluding the 0.2 bonus on the “hourly” and you get a kind of “preference” score. Where the answer for the distribution vs the other app is missing, count 10 points to Status if the answer in the Q before it was “Only used status”, and count 0 points to Status if the answer was anything but.

Here is my attempt at this. I hid some columns in my copy of the data to make it clearer and the functions are hacked up in an external google script. In this version, we get 75% preference for Status over anything else, which seems realistic to me given the number of answers we got and accounting for the fact that only those really interested in providing data will have participated in the survey, and are thus considered more enthusiastic and, by extension, Status power users. Of course, the calculations and factors would need to be further tweaked, but personally I’d find this preference score more revealing and useful than the current chart for this particular data.


#14

Hi @bruno - nice idea! Check out the updated question with the answer as a linear 0-10 scale - wdyt?


#15

Yup, that seems better, but it still needs to be weighted against hourly / daily etc when calculating imo. Just a little bit of a variance of 0.2x in either direction.


#16

Sounds good! I’ll try to add the calculation to the results when they come in :slight_smile: