User growth and retention

Below are planned projects for phases of acquisitions and retention. These projects are on the roadmap and actively being worked on or planned to kick off. Taking in mind the resources we have, I think it’s key we stick to these and focus on building.

IMO any other projects should answer these 3 questions and should follow after planned projects have been delivered:

  1. Does it make it easier to invite friends?
  2. Does it incentivize to invite more friends?
  3. Would more people use Status more often?

Special highlight: :clap: Group chat invite links :clap:
Any technology adoption model grounded in research ever relies heavily on social norm and change agents. Friends are more powerful in a social network than strangers in a crowded chat room.

Projects on roadmap

Acquisition

  • Referral program
  • Chat key sharing (incl Group chat invite link and better social sharing of QR code and key)
  • Onboarding improvements

Day 1 retention

Hygiene features that people might expect from other messengers

  • Images
  • Mentions
  • Emoji Reactions
  • Audio messages
  • Channel discovery (hardcoded categorized list)

Trigger

  • Notifications on Android
  • Notifications on iOS
  • Sounds

Non-functional

Performance, attention point, but no dedicated project at the moment

Activation

  • Starterpack (part of referral program)
  • Friend connections (follows chat key sharing)
  • Read only channel (through bridged bot)

Day N retention

Differentiators and privacy specific hygiene features

  • Disappearing messages
  • Tribute to talk
  • SNT reactions
  • Contact management (incl. Trusted contacts)
  • Personalization (adding a profile image)

Another element here is how user acquisition works different in different markets. For example, in LatAm perhaps they are more interested in using Status as a “bank”, as opposed to as a messenger. This would also impact how we measure DAU and what stickiness would look like.

Agree. Marketable proposition is sending funds directly through chat. That implies, most of the roadmap should work initially, but Day N retention should focus more on defi. Deeplinking to dapps seems the safest path.

Two other data points would be:

  • dap.ps use
  • tx volume
7 Likes

I have created a dashboad for DAU and MAU here:

Which has some issues:

  • MAU metrics are counted only for slices every 30 days, not progressively for each day
  • DAU/MAY is done the same way, and metrics for all other days are 0
  • These metrics include peers generated by end-to-end tests

The last point is being addressed by use of eth.staging fleet for e2e tests:

But the other ones are caused by how ES queries work. And I don’t know how to change this behavior currently.

The other issue with ElasticSearch is that I can’t publish those metrics on https://metrics.status.im/ because adding it to data sources in Grafana will make it possible for anyone with access to the site - which in this case is public - to make arbitrary queries to our ES cluster. Which is clearly a problem, and unless solved we can’t have these metrics public in Grafana.

One way would be write some kind of automation that would make its own queries to ES Cluster and generate graphs as images and make those public. Not sure how much work writing something like that would be, but it would give us more control over how those metrics are displayed and calculated.

3 Likes

Metrics was discussed in Core dev call yesterday: https://notes.status.im/core-dev-call_33# - some details and action points can be found there.

should treat the “Unique Whisper Peer IDs” as DAU? How much are these due to testing or not? If not, how should we approximate DAU?

This is the best and most straightforward metric we have for DAU in an open p2p network where we don’t use in-app analytics when it comes to messaging. Once E2E tests (hopefully today) are removed from the metrics they should give us a reasonable estimate. We can complement it with other forms of transactional data such as SNT txs, dap.ps, ENS usernames, etc.

Here’s what things look like today. We expect these numbers to be adjusted downward once we remove E2E tests from the metrics.

From https://grafana.status.im/d/aLIffmGGk/status-usage-metrics?from=now-5M&to=now&orgId=1 Kudos to @jakubgs for adding this so quickly. Obviously needs some tweaks for the DAU/MAU calculation, but it is a good start.

DAU

image

Stickyness - DAU/MAU

image

This current graph isn’t exactly ideal, but what it basically means is that our stickiness was ~1-2% in February and March, 3% in April and 4-5% in May-June.

This is really bad and suggests that we have a serious retention problem, whether this is due to onboarding the wrong users, a poor user experience/product, or a mix of both.

DAU Growth Week-Over-Week

To visualize growth and drivers of it here are a few attempts.

The first shows DAU with launches highlighted:

image

This doesn’t really tell us the week to week growth rate though, which is a good metrics to measure and action on every week. It gives you something to fail at on a weekly basis (http://paulgraham.com/growth.html).

To fix this I made some basic charts here to visualize growth rate https://docs.google.com/spreadsheets/d/1BfXCnZ66sbcZ-IV8Is-O2z8qXXpBf32kvYETYq5KR9k/edit?usp=sharing

This shows Sunday-to-Sunday DAU. Ideally this should be done with a weekly average, but since I did this manually and quickly this will have to do. Using Sundays also gets rid of some biased testing. Feel free to edit this sheet/workflow as you see fit, it is open for CCs.

image

image

image

Conclusions

What conclusions can be made from this data?

  1. We have a severe retention issue for a messaging app. It is getting a bit better, but it is currently hard to draw conclusions without long lag times.
  2. Growth is extremely tied to launches. Without launches growth is basically non existent.
  3. We have failed to meet any reasonable sustained weekly growth target for the last 3 months. If this happens a single week (say, not growing 5-10%) this should be a flaming emergency.

A challenge

How can we make weekly growth go up 5-10% next Monday, and the Monday after that, without relying on a big-bang launch?

7 Likes

My sense from this is we are not giving users enough reason to stay engaged. If we breakdown Status as an app, it’s really 3 separate apps packaged together: a messenger, a wallet and a dapp browser. On their own there are alternatives to each, the unique value proposition of Status is being able to compose them.

An example is allowing dapps to leverage Status to enhance their own experience. We have heard from Aragon in the past that they would love to be able to allow Aragon organizations to communicate in real time using Status and Sythetix has been searching for a decentralized “troll box” to integrate with their exchange. Both of these integrations today are not possible because we don’t enable developers to build such integrations by not exposing the chat api to the dapp browser.

Other examples are using Status to help maintain privacy and decentralization for L2 solutions. Something @ricardo3 has expressed interest in as the ideal solution for Topic democracy. But clearly any scaling solution stands to benefit.

The power of cross pollinating with other crypto communities is something being witnessed in the DeFi space. As new protocols are built by composing interactions with existing protocols they experience growth by merely importing a slice of the existing protocol’s user base.

Measuring the impact of such a feature would also be possible simply by counting the number of dapps that integrate. If this works then every major dapp that has an integration should create a new high watermark in residual DAU. If verified new integrations becomes a growth strategy.

proposed spec:


more examples:
7 Likes

Thanks for the write up Oskar. Agree, we have bee flying blind and have been relying on fairly opaque information from the metrics dashboard.

This stands out to me most. We need a better method of actively tracking specific initiatives with retention. We now have a few different teams working across different areas of growth, and we need a better way to measure what is working and what is not.

In Q3 we have a few initiatives which we need to be tracking closely:

LatAm Go-to-Market (BR/ARG)

  • And influencer heavy and PR driven approach (top of funnel designed at high distribution) followed by meetup, ambassador engagement, and referral program (mid/bottom funnel designed for retention).
  • Positioning for these markets leans more towards defi, banking, and obtaining/holding more stable assets (than local fiat currencies).
  • With this and while maintaining end user privacy, im still curious how we can bets measure the impact of this specific campaign. We can monitor activity in localized channels. We can monitor installs by region. But this still doesn’t give us a really accurate idea of what retention looks like in a specific region right? Can we measure mailservers by region?
  • Lastly, we need to be better at implementing drip or nurture campaigns in which we identify the desired action, and guide the user through to it. In Latam, we have defined that action as sending a crypto payment to a friend in a direct message.

Simple post-registration chatbot

  • We have an on-boarding issue for sure (which is evident in our retention rates). I think this is due to a number of issues. The network effect most notably (i cant use a messenger if my friends are not on here to message with), and also the large product offering. There is a lot to convey to a first time user.
  • Without the network off the bat, we need to prompt people through the experience, and a bot could help people with this. We need to start with a very specific scope, measure, and refine as needed. Measuring this seems a bit more straightforward, as we should be able to see who is interacting with the bot correct?

Community Management and Engagement

  • This has been challenging for us since day 1 to be fair. Re-activating and converting our fringe community into DAU is priority
  • This is both a top of funnel initiative in growing the community as well as a middle/bottom funnel initiative in incentivizing people to keep public channels active.
  • @Jinho lets sync asap on the roadmap for community and ambassador program.

Brand Initiative

  • I say brand, but this a quite broad scope. We have some tactics outlined here which are aimed at both top of funnel inspiration to install, as well as middle/bottom of funnel participation.
  • We need to provide “branded” in app experiences that give people a reason to come back to Status (dapp partnerships, public channels on specific topics, principles, etc)

I see the “marketing work” for the next 3 months spanning across these 4 top of funnel initiatives (programmatic ad spend spread across all). With retention mechanisms (chat bot, channel management) relevant to all. With this, we need to be able to effectively measure the value each one of these efforts is having on DAU/MAU. Marketing team will set clear KPI’s and actionable metrics to each one of these by end of week in prep for Q3.

How can we make weekly growth go up 5-10% next Monday, and the Monday after that, without relying on a big-bang launch?


Looking at the data, there isa clear need to follow up product launches/major releases with in app actions for users. This should happen both in app (bot, channel management) and outside from a comms perspective (i.e. a tweet or blog that encourages people to register an ENS name - lots we can do here)


Things like this, and other extensions seem like the most unique way to port over existing communities. It truly makes Status a window into Ethereum or operating system but understand this is a ways out. Id like to work with communities to create Status public channels, but the UX is not ideal. Tagging, pinning, moderation seems like crucial elements to realistically expect a Telegram community, for example, to jump over to Status. However, I think we can start the process now for those accepting of a less than optimal experience.


I agree. There should be a simple way for anyone to view our growth. Just one distinction (that is not clear) - analytics.status.im is currently measuring the SNT utility in the network. Would it make sense to make a more integrated dashboard that accounts for peers, network activity and SNT utility?

6 Likes

Voices from our users

I’ve collected some feedback on retention and user growth from the Status community and UX designers. There were a lot of pleasing feedback and compliments on Status. However, for the development of Status, let me share some scathing ones this time. I’ve included my thoughts in the middle as well.

  1. First and foremost, it is difficult to find my friends or someone to hang out in Status.

    • People are not familiar with the concept of chat keys and ENS registrations yet. Asking a chat key of my friends or having them register ENS names is a barrier to use the app. The list of public chats is not localized (users who are not good at understanding English cannot jump into public chats) and doesn’t suggest topics that I’m interested in.
  2. Convenience vs Security

    • One of the most compelling features of Status is that it is SUPER secure and private. Status is a lifeboat for people who live in a controlled society suffering from censorship and surveillance. But the problem is many people tend to value convenience of a service more than security. The level of security is not visible until a user gets exploited but the convenience of a service is so easy to be found at first glance.
  3. Private messaging

    • Private and secure messaging is one of the core values of the Status app. But some users don’t feel the demand for the secure messenger sacrificing the convenience of the existing messengers. (If we can offer private messaging features with better convenience, it’s gonna be a totally different story though). People mostly use messengers for chitchats or casual things and these kinds of chats don’t require a super high-level of privacy. There are few reasons to move to Status from major messengers offering better usability yet. Even in some countries, although this is kind of an extreme example, people consider private messengers as an origin of crime. (This year, Telegram was a victim being an origin of child porn in Korea)
  4. Lack of content to attract users

    • People find services for mainly 3 reasons. They are fun or they help me or they give me money. A good thing is Status can be used for all the 3 points. A bad thing is there are a number of alternatives that are specialized in something with better UX and popularity. People use Youtube and Instagram for fun, banking apps for financial support, and e-commerce apps to make money. Status needs more focus and attractive content to retain users.

A UX designer I know said Status is a monumental realization of the decentralized technology showing off one of the best DApp experiences but a naive university graduate who just jumped into the harsh reality. There are so many major players outside the emerging dapp ecosystem.


Status Community for the growth

For better retention and the growth of the ecosystem, users should easily find someone to hang out and experience a welcoming atmosphere. This is where the community and the ambassador program can play a significant role. Ambassadors welcome newcomers, educate Status to others, and create informative content for Status.

The ambassador program has been steadily growing until now(82 completed tasks in Q2, 57 in Q1, and 53 in Q4 2019) but a lot of ambassadors and ambassador candidates left the Status community at the same time losing their interest in Status. Well, we(mostly I) should make the ambassador program more fun and productive but the fact that some ambassador candidates became inactive saying there are other things or projects they’d like to focus on means we didn’t provide them with enough values.


What’s next?

Status needs a focus. Considering the decentralized technology is kind of experimental yet and the resources are not infinite, Status needs the focus(I’ve received this feedback many times). A super app combining a lot of services could also mean that comms can be confusing and none of the features are superior to its competitors. Some features should be so overwhelming that Status should make its users desperately want to use Status. Otherwise, the app can be something for early adopters or tech enthusiasts only.

There should be a definite competitive advantage over other apps. We don’t need another Whatsapp or Telegram, which are doing well in their fields. Many people want Status to focus on private, secure communications with better UI/UX and user-friendly features, like the headline of our website. Status has the potential to be a holy land for anonymous communities which are constructive. For example, Blind network, anonymous professional network (https://www.teamblind.com/) has built an amazing community for employees across the world who want to stay anonymous.

On my end, I will do my best to make the ambassador program more fun and productive as I mentioned before and have Status translated into as many languages as possible catching up the development of our fantastic dapp with remarkable potential. Let me come up with of some ways to achieve these with more details later. Feel free to ping me if you have any opinions or suggestions!

14 Likes

I agree with absolutely everything @Jinho . I’m just a user who was enchanted with Status and saw potential in the app. I even gave my opinion as a user and also opened a request about the possibility of creating a name and profile picture without involving the registration at ENS.

I’ve been trying to contribute to Status with what I can: ideas to make Status attractive in features.

Status offer a very high level of security and privacy. No other messenger today can do the same. What I would hate to see is a setback in these aspects. I am particularly resistant to giving up the security and privacy offered to make Status more convenient, but I think there may be a way to reconcile convenience and security / privacy to attract users.

There are quite solid and well-known options out there, from WhatsApp, Telegram to Signal, but again I say, none as secure and private as Status and this is a big difference right from the start. One of the points that I believe is essential for Status to grow in numbers of users is precisely to invest in the development of unique and useful features.

When I bring a friend to use Status with me, I always get the question: How do I send a video? How do I attach a PDF to send you? I can send an audio message? And make a video call?

These features today, so common in messengers, are absent from the Status for obvious reasons that I understand. Some of them are already being worked on and will soon be available to users, but I would just like to record this situation.

The developers and contributors of Status are incredible and have been dedicated a lot to this project and I believe in the ability to become a reference in this segment of private messages and cryptocurrencies.

That is just my humble opinion, I hope that no one is offended. :grin:

10 Likes

Thank you for clearly stating “without this we are basically flying blind”, @oskarth. We all have opinions, all of which are welcome, but only data confirms theories. I’d love to see more data, especially metrics on regional app marketplace installs overlaid with marketing spend as applicable (visibility in to what marketing efforts attract new users, even if users don’t complete onboarding and/or return) and direct downloads. On that note, precisely determining onboarding friction is crucial to retention.

Status is user-friendly to us, but most people reading this thread aren’t average messaging app users. I love precise technical terms such as ENS, Waku, and DApps… but they primarily benefit users with specific knowledge, and can be obfuscated away through design. Argent and other apps are elegantly simple, yet don’t give up precision. Sadly, most users have no interest in learning about what powers their app… they just want to create an account, and trust the app to be secure and helpful. Less is more.

I also noticed gaps in the advertised Status metrics, and this makes me suspicious about conclusions being drawn. As an obvious example, https://analytics.status.im/sticker-market/ appears to be missing a significant amount of data (there is none in some parts) on at least half of the sticker packs. As privacy is a core Status principle, core contributors are forced to do more with less… and it’s going to take nearly all of the information available to them to determine stickiness.

On an unrelated note, I want to thank all current and past Status core contributors for achieving the monumentally difficult task of building an incredibly useful and robust privacy-first product while never compromising on principles. More and more is asked of all of Status core contributors as adoption ramps up… despite that the product is free and open source, and that no data is collected and sold. Your efforts towards saving us from ourselves are sincerely appreciated, even if it isn’t often stated.

9 Likes

Adding some more data for consideration. Using the dump of peers overtime we started constructing a cohort analysis to start understanding retention. Full notebook here: https://colab.research.google.com/drive/1Osl3EXbKR2_iwAhOHmAiCum9F6IquRX2?authuser=1#scrollTo=KW9qY7lwELYx

weekly_cohorts

The first thing we notice is a big drop off after the first week, however there are some interesting cohorts such as peers that appeared the week of 02-16 and 02-23 had long retention histories than the other cohorts.

Lets look at the distribution of peers based on number of days they were seen.

The number of peers seen only one day is so much larger than the rest that we should cut this group out for the purpose of being able to see the rest of the distribution.

all_peers

ex_1day

ex_5day

ex_20days

As we go from cutting off 1, 5 and then 20 days from the graph we can see more of the distribution as a result of a less steep drop off rate. One can infer from this that the long a peer has been on the network the longer they will continue to be on the network. Put another way if a peer has been on the network 1 day there is a ~2.5% probability they will be on the network in 20 days, but if they have been on the network 20 days there is ~25% probability they will be on the network another 20 days, a 10x difference.

A possible growth experiment could be to get new users to use Status for a month and see what the retention rate is in that cohort.

3 Likes

Thanks everyone so far for all your thoughtful comments, ideas and contributions! There are a lot of good ideas here for sure, and it is great to see so many people outside of Core having ideas for how to make our main product work better and retain more users. Special kudos to Barry for getting us sweet actionable retention rate graphs!

Weekly update

As of July 1, per post by Jakub earlier, we no longer count end to end tests in metrics. However, this didn’t have as a big an impact as one might suspect.

Unfortunately the dashboard at Grafana seems to be down. However, we can still the latest in Kibana.

image

DAU: Sunday to Sunday - 2658 on Jul 5, 2985 on Jun 28. This is -11% negative growth. Since this is on Sunday, the e2e test impact is negligible.

Retention rate: No new data here, see post by Barry above for latest here. However, since no releases or specific targeted marketing has occured in the last week (to my knowledge), there’s little reason to believe things would’ve changed here.

Android install numbers

A good of checking accuracy of metrics is to cross-check it with different sources. Here are the Android installs per week. These are exported from Play Store and you can see them in the Status Growth Metrics spreadsheet.

image

How to action on retention

This question was brought up earlier here. We obviously won’t be able to get accurate segmentation or anything of the sorts. But we can make do.

What the retention rates in Barry’s post above allow us to do is to have hypotheses about cohorts and campaigns. For example: this feature will increase D2-D30 retention. Then we can check the cohort for when that version is released.

We can also use it to test campaigns. Say there’s a big spike in installs a specific week. Then we can check how users from this cohort are retained. If they retain worse than previous cohorts, perhaps they aren’t our target audience.

In the future, we should be able to distinguish Android, iOS and desktop peers here. Though I’d leave that kind of sophistication as out of scope for now and more of a bonus until we get basics in place.

A good marketing/onboarding campaign here would be: how can we get N users to use Status for a week or a month? In the Saas world, this is a very common strategy for onboarding and increasing retention.

What’s good?

Since last week, a lot of things have improved when it comes to the state of metrics.

  1. We have clear DAU and retention metrics that tell us where we succeed and fail.
  2. We have a dashboard that clearly shows DAU and stickiness for everyone.
  3. We have increased confidence in our metrics by removing end to end testing from metrics collection.
  4. We have a very clear understanding of how bad the situation is right now.
  5. We have retention rate graphs that can be used for experiments and analysis on a time scale of ~days.

Outside of metrics and visibility, we have also seen a lot of good ideas on how to improve our growth and retention in this thread. And, of course, we have seen a lot of product development happening that we believe will result in better retention, e.g. images, push notifications, and other initiatives.

What still needs improvement?

I’ll divide this into two parts, one is general and the other is specific.

General:

  1. The hard part: growth and retention is still terrible. This week went backwards.
  2. Mindset: I am not convinced we are doing our best here. See section below.
  3. Theory of retention: I still don’t see it, and I’m not convinced by our reasoning here. This needs clear thinking where we know exactly who our users are and why they keep using Status, or not.

Specifics:

  1. Methodology, mentioned in OP, still needs work. This needs to be crystal clear and requires Core involvement to be correct. See https://notes.status.im/metrics-method?both# for a start. Marketing is an example user here.
  2. Retention rate dashboard and/or script. With Barry’s notebook we have actionable retention metrics. It’d be useful if we could script this, so a CSV export is run and we can update these graphs on a daily basis.
  3. Solve mystery 1. In mid-April we had a spike of installs through a performance marketing campaign. Question: Why does this not translate into a spike in DAU?
  4. Solve mystery 2. We see DAU of around ~3k, but installs per week recently is around ~100. Question: How does that add up?
  5. Different types of metrics. It’d be useful to see other, wallet/dapp-browser-oriented metrics reported in this thread.

Mindset

A word of warning - this section might be controversial and you might not agree with it. I have been having this conversation with a few people privately over the last week, and I think it is important to say publicly for everyone.

For us to do this, we need to have the right mindset. This means keeping growth and retention front and center every day. And I’m largely talking about Core here, as that’s the product team with the biggest impact.

It is also important that there’s a feeling of a sense of agency from the product team in this. Growth isn’t some magic sauce that another team solves. What (performance) marketing does is adding fuel to the fire. But there has to be a fire to begin with.

If we don’t currently feel this sense of agency or control, then that’s a bug to fix.

I heard in the Core retro last week that we shouldn’t let ourselves get distracted by metrics. I said it then but I’ll say it again - we should let the metrics guide us, and constantly ask ourselves if what we are making will improve retention or not. We have the data now, so there are no excuses not to do this.

I also heard that we shouldn’t let discussions take away from building. That seems dangerous to me. We are still a startup, and we don’t know exactly what the right path forward is. We need to be open and flexible. The more reflection we can do here, as shown in this thread, the better. To me this is where something like a theory of retention comes in, because it allows us to think more deeply than just “let’s get to feature parity with every other messenger out there”. This is a slight strawman position, but as if that’s the reason people would switch to Status. It doesn’t personally resonate with me as a user anyway.

There are a lot of smart and thoughtful individuals in this organization, and it’d be a shame not to capitalize on that. Six hours to chop down a tree and spend first four sharpening the ax kind of thing.

Obviously this should be complemented by ruthless execution once we are confident in our choices. But it shouldn’t stop us from breathing and making sure we are on the right path. Judged on a weekly growth basis, we are not, and haven’t been for the last three months.

It might seem like we have to do a lot of things at the same time. I don’t think that’s the case. As a core product, we mostly have to do one thing: improve retention. Usually only very few actions have a big impact. And that starts by taking a deep breath and emptying the cup in your mind. Take a long walk and a big breath.

Over to you!

9 Likes

Thank you for the install numbers. :slight_smile: No one has ever done what Status is doing, thus no one can predict how the product will be used once it’s ready for the masses. New users can (almost) always be acquired through paid advertising, but retention requires a killer feature set. Push notifications may feel like a silver bullet, but a solid use case for Status is also critical… and that means clear messaging on how its components stack up against other apps, and how its core features are better together.

If the Status app is marketed as a secure messaging app, users of other popular secure messenger apps will expect Status to have features comparable to the other apps they used. If the Status app is marketed as a decentralized app gateway, the Ethereum wallet should function with the most popular decentralized apps. If this functionality isn’t present, only the users who fully comprehend its potential will likely keep using it daily. Most others will make an account, and may revisit with each new release.

Marketing is obviously waiting for additional features to go live before acquiring users. The roadmap indicates a solid plan to build a robust messaging app. Core contributors build useful features quickly, and are respected for both the transparency and quality of their collective works. Stay the course towards a feature complete app, refine the messaging, continue to never compromise on principles, and measure (always!) all you can to fail fast if users don’t love something that’s built. Data will light the way.

6 Likes

This was me, so let me respond and clarify. To preface my response though: Thank you @oskarth, for bringing structure to the data we have available and starting the conversation about what we’re building. It is much needed. As a UX researcher I would love to do more research to understand drivers of DAU. My response below is to the sum of where we are now, the insights we do have and what I believe are our strengths and weaknesses in product development.


My key point on the Core retro was not to ‘not let ourselves to get distracted by metrics’. My point is that we have 2 months, a solid roadmap to work from and very little data to go on. Let’s focus on delivering, which historically is an area we have a lot to improve. It’s not about a mindset of staying away from metrics. It’s about having a mindset of delivering and learning. Decreasing time to delivery, scoping MVP’s is critical here so we can learn fast and pivot if needed. Building gives us data.

I 100% agree we shouldn’t be building blindly. I don’t believe we are, in
spite of the little data we have to go on. We do have valid data from the past where we polled among a decent number of people in surveys and qualitative studies (including Status and non-Status users) and we have feature requests. Reading the input above, only strengthens earlier findings and increases my confidence in earlier data.

Insights from past studies

2018 report on a study in UK and Iran:

  • Network effect is identified as key driver (page 34)
    epics: referral program, group chat invites
  • Main reason for using a messenger ‘Being able to stay in touch with friends
    and family’ (63%) (page 31)
    epics: referral program, group chat invites, notifications
  • Details on Network effect (page 35)

Survey respondents who love their current messaging app cite
the ease with which they could connect with family and friends.
For this reason, Status would need to have features that enables
it to onboard new users quickly based on their existing networks.

epics: referral program, group chat invite links, invite from contact management

  • Expectations of basic functionality ‘Easy means to share and interact with photographs
    and videos, Emojis & stickers’ (page 38)
    epics (images, emoji reactions, audio messages)
  • Perception of Status includes main concern ‘Not socially acceptable’ (page 41, 61)
    epics: referral program, group chat invite links, invite from contact management

Study in South-Korea

  • Focused more on crypto investment use cases (SNT holding) driven by socio-economic climate and investment regulations.
  • Identified “Privacy Ambassadors” as early adopters. Private, secure messaging for business purposes driving Status use.

Study among community members,

  • We found ‘texting friends’ to be one of the highest Performing features in a KANO analysis. Over public channels/groups, user stories/updates

There is much more anecdotal data here that all point to supporting basic ‘friends and family’ functionality and onboarding. This is what the roadmap is geared to.


8 Likes

Many excellent points have been made here. So I’d like to only share the following (hopefully) inspirational quotes:

When you can measure what you are speaking about, and express it in numbers, you know something about it; but when you cannot measure it, when you cannot express it in numbers, your knowledge is of a meagre and unsatisfactory kind.” - Lord Kelvin

Not everything that can be counted counts. Not everything that counts can be counted.” - Albert Einstein or William Bruce Cameron

None. It’s not the consumers’ job to know what they want.” - Steve Jobs (when asked what market research went into the iPad)

6 Likes

Jumping back in here after a few conversations on the topic this week.

How can marketing help drive retention? We need to be marketing to the right people first and foremost. With such a wide product offering, it not only makes the product positioning tricky (and in some instances diluted), but makes it difficult to identify the right tactics, channels, and end deliverables.

At this point there is so much inertia behind product development, that it almost as though we need to find the market that the product fits, instead of building a product that our market wants. Fortunately, we have two very clear markets in mind. The problem is, I suspect they want quite different things (and at times, they have conflicting interests)

Crypto Community: This is our audience first and foremost. They invested in the token sale. They have been following development longer than any group. The product is designed for them and early product positioning even stated “Ethereum OS” or “Window onto Ethereum”. Based on some conversation with community members, messaging is simply one element of our product offering - and often times, not even the most important (as there are other options). Access to crypto and a smooth mobile experience with dapps are equally if not more important. DeFi and the growth of those protocols speak for themselves. This then means we need to ensure dap.ps works well, and we offer the same accessibility to dapps as other ethereum mobile dapp browser/wallets. Privacy is clearly a desire for this group, but it has been mentioned many times, that the very nature of ethereum is public (and in stark contrast with one of our core value props)

Privacy Enthusiasts: We have built a product based on a set of principles - privacy and security being two paramount ones. Private communication is s clear desire of this market group and the messenger we are building fits the needs. However, in conversation with community, crypto/ethereum/blockchain, often gets in the way of this and is deemed-non-essential and sometimes and token is a major turnoff. Take the recent explosive growth of Signal with a team of cryptographers dedicated to solving this issue, and only this issue.


Some could say we have an identity crisis. Is Status a “window onto ethereum” or is it a tool for ultra private messaging? Or are we truly both? I would genuinely like to know others opinions on this and hopefully we can align as it will help the marketing team out.

From a marketing perspective - we need to test and be brutally honest with our findings. As mentioned above we have a few campaigns in the works:

LatAm G2M - primarily targeted at crypto users. Leading with wallet/browser functionality and messaging.

How can we measure the effectiveness of this campaign in terms of DAU given our primary goal is not messages sent but rather use of wallet and dapps?

Brand - This initiative needs to be split into two cohorts for effective measuring:

  • Crypto / Ethereum: in which we push the wallet and browser and raise awareness of all the other tools and products you can use inside of Status (other dapps)
  • Privacy / Messenger - push the messenger as a means of privacy

It is still unclear to me how we measure the success of one campaign over the other without in app analytics. Seems as though we will not be able to run simultaneous campaigns. We could push people into separate public channels and manually observe which ones are more active?

Next Steps:

  • Alignment between product and marketing as to what our goals are and who we are building for and marketing too @hester @JohnLea @andre
  • Come up with clear measurement plans for each campaign mentioned above. Timebox each campaign
  • Review data and base evaluate marketing efforts / audiences
9 Likes

Alignment between product and marketing as to what our goals are and who we are building for and marketing too

In short, (this won’t be new to you @jonathan ) my thinking is to target existing crypto and privacy oriented communities and proactively offer to facilitate their transition. Even if they choose to not transition to Status, the feedback will be valuable. We’d be validating product fit with them. Initial thoughts on communities to reach out and structure to the approach by @JohnLea and myself here.

I see this as a parallel grassroots approach to complement planned efforts through campaigns. Members of existing communities can thereafter act as change agents for wider friends and family adoption focused more on private messaging and relying on the referrals program.

Organization channels seem a promising feature to add upon reaching base level feature parity to help onboard communities. This would be good to validate when reaching out to communities.

4 Likes

I guess my point is that when you really get into the details on this, these two communities want different things.

Take for example the crypto art community. They want access to dapps like superrare, rarible, open sea, easy ability to purchase and bid on NFTs, and then be able to showcase and display those NFTs in a wallet. Or the defi community - they want seamless experience with defi dapps and protocols to earn on their holdings. The messenger comes second.

This is what we need to sort out. I feel like maybe there is a misalignment on what crypto and privacy communities are/want. Or perhaps we market a messenger to the crypto community?

1 Like

How can marketing help drive retention? We need to be marketing to the right
people first and foremost. With such a wide product offering, it not only makes
the product positioning tricky (and in some instances diluted), but makes it
difficult to identify the right tactics, channels, and end deliverables.

Thank you for asking the right questions @jonathan!

Some could say we have an identity crisis. Is Status a “window onto ethereum”
or is it a tool for ultra private messaging? Or are we truly both? I would
genuinely like to know others opinions on this and hopefully we can align as it
will help the marketing team out.

Back to the future

I’ll answer this question by going back into the future. Specifically, by looking at this answer from a Slack AMA with Status 3 years ago, before the ICO:

Question:

How do you get status to the billion Users out there
What will be ur arguments
Most people dont know cryptos. even dont care for security or privacy (facebook)

Answer:

Ultimately we’re building Status for the general public, but there’ll be
a number of market segments that we’ll focus on first as we work towards this
goal (not to mention Ethereum’s network needs to be ready for this first).

The logical progression from the Ethereum community would be to the greater
crypto space, followed by privacy conscious groups and libertarian groups. While
this is far from ‘mass adoption’, the success of apps like Signal and Telegram
(each with tens of millions of users, and whose core marketing messaging is
privacy) serves as a testament to how deeply this resonates with a small, but
growing percentage of the population in many markets.

Beyond this, we can begin creating custom marketing campaigns and custom
on-boarding sequences into Status for specific DApps we believe we can get users
for (DApps are really the core value-proposition). It’s worth noting here from a
user acquisition perspective there are only a few key metrics - retention rates
are the big one early on, and realtime bidding platforms (DSPs) make this very
much a data driven approach. In our experience once you find some pockets of
high converting users you can scale very quickly.

Looking further ahead I’d love to see Status bringing access to financial tools
and services to emerging markets, but these are complex problems and often
mobile data costs are a hurdle to adoption. I suspect this would require working
more closely with communities and mobile carriers, which also we’re prepared to
do.

In the years since then, I haven’t seen a good reason for us to deviate from
that basic plan. We need to convince the immediate Ethereum community community
first and foremost. After that we can branch out.

I have heard a lot of talks about moms using Status, and I wonder if this is something that is lost in translation (“so easy so my mom can use it”?). It seems like a weird target audience to me.

This is also why I wrote the following user litmus test two years ago:

[What would it take for] 50-80% of folks at Devcon4 [to] use Status daily?
Building from first principles and some litmus tests

That we aren’t focusing on this is worrying to me. I’m sure more research and thinking has been done since then, but nothing that has convinced me more than the original vision.

Effectiveness of campaigns

How can we measure the effectiveness of this campaign in terms of DAU given
our primary goal is not messages sent but rather use of wallet and dapps?

First, re DAU. DAU measures any user, not just people using chat. There is one caveat for this: they have to either be on WiFi OR have enabled sync in the chat.

There might be other data here that we can use, but nothing that has been explicitly made visible in this thread. E.g. D.apps data. I defer to @andre and @hester for this, and I encourage people to post that kind of data to this thread.

There is a WIP methodology doc here https://notes.status.im/metrics-method that @andre and rest of Core is helping fill out. (Thanks @cammellos for checking some basic assumptions).

Please shout out if there are more questions on this.

Measuring multiple campaigns and retention

Second, re multiple campaigns and retention. The way I would do this is the following. Have a campaign start a day and run over a few days. Then a week later start another campaign and do the same thing. Outside of install stats, this will then result in different retention rates for the various cohorts. This way we can judge the effectiveness w.r.t. retention after a week lag time. That might be sufficient, especially if we can do binary search / AB testing over the possible campaign space.

A specific campaign I’d love to see that Barry suggested is a challenge to use Status for a week straight, or a month straight.

2 Likes

My key point on the Core retro was not to ‘not let ourselves to get distracted
by metrics’. My point is that we have 2 months, a solid roadmap to work from
and very little data to go on. Let’s focus on delivering, which historically is
an area we have a lot to improve. It’s not about a mindset of staying away from
metrics. It’s about having a mindset of delivering and learning. Decreasing
time to delivery, scoping MVP’s is critical here so we can learn fast and pivot
if needed. Building gives us data.

I 100% agree we shouldn’t be building blindly. I don’t believe we are, in spite
of the little data we have to go on. We do have valid data from the past where
we polled among a decent number of people in surveys and qualitative studies
(including Status and non-Status users) and we have feature requests. Reading
the input above, only strengthens earlier findings and increases my confidence
in earlier data.

Based on these surveys etc, what is the expected change in [D2/D30/DX] retention
rate for [specific target group] once we have implemented [set of features from our roadmap]? Can we make this clear and testable, so we know if we are right
or not in terms of what drives retention? Let’s be precise here.

Additionally, you say “mindset of delivering and learning”. I agree. How many learning opportunities would this give us? From what I can tell, it sounds like we’ll only know after we have implemented a big part of our roadmap. Would that give us enough time to do any learning, or would we have run out of money at that point?

There is a reason startups like to measure on a week to week basis. And that’s with having an absolute minimum of six months in the bank.


I think I’ve mentioned it before, but as n=1 sample size, none or very few of these features personally have an impact on my retention. What matters for me (empirically) is (1) community (i.e. Status moving to Status) (2) desktop/sync support. I realize everyone is different and for some people, say, audio messages or SNT reactions might be what matters for retention, but I’ve also heard n=2 in private conversations. This significantly decreases my subjective probability that we have a clue about really what drives retention.

PS Here’s an interesting (but long) article on what makes social networks successful, looking specifically at building social capital (but also mentioning utility and entertainment). It may or may not be relevant to Status, but some readers might find it interesting. DS

Monday again, which means it is time for a weekly metrics update! Not much new here. Most of what was said here User growth and retention still applies.

Two specific thing that have improved in last week:

  1. @jakubgs setting up automated retention graphs here https://retention.status.im/
  2. @cammellos confirming the two following things (see https://notes.status.im/metrics-method#):
  • peer id confirmed is the same across restarts
  • same across multiple IPs

I’ve also heard @andre is working on some cool stuff, but I’ll let him post that.

DAU

image

Up 12%, however hard to know if this is significant as it puts us back at two weeks ago baseline. If we keep it up for next week that’d be a good sign.

Retention

Unchanged (seems to be lagging a week, and only weekly for now). No reason to believe this would have changed.

image

4 Likes

leak? let’s add another sentence to make discuss happy.