Gitcoin Hackathon Retention Analysis

Month over month, we expect our retention curves to move up. This means with every new cohort, we're doing a better job of retaining them on Gitcoin.

We're investigating the lows and highs of our hackathon cohorts, specifically the bright green line, which was our June 2020 hackathon registrant cohort. For 1-month retention, we hit the all time low in this specific cohort. This took place during the Protect your Privacy hack, which was correlated to some of the co-marketing that we did with AngelHack. We got quite a few signups, but our start and submit work conversion rates were low (~10%), which could have been an explanation for low engagement.

Taking the opposite view, we can see that back in 2019, we were doing much better in terms of retention. What changed?

Let's add on our entire user base, not just the hackathon cohorts, to check out cumulative performance.

I'm honing in on the top retention curve (2019-12), along with the peaks of retention that we see during 2019-09 and 2019-10. What sorts of actions led to higher engagements for the top retention curve, and can explain the peaks of engagement?

Let's break it down by a frequency histogram of user actions.

We can see for these highs, that quests and grants play a large role in the dominant action, if we exclude logins and visits. Start works are still there, but at a lower frequency.

We know from previous analyses that Grants induce a huge spike in Gitcoin traffic, and usually brings the base level of engagement to a higher set point. Grants offers a way to engage for a very concentrated period of time, usually via donations during CLR round. It's the doorway to the greater suite of Gitcoin products that allows developers and supporters to flock to Grants, and then stick around for some of the other actions and contributions they can help with on the platform. The latter, these "actions" and "contributions" is what we want to focus on in order to induce stronger developer loops. New user onboarding (which is currently in progress as of 2020-09-15) is also a viable method to improve retention.

Taking this from a product perspective and perhaps thinking from a more qualitative standpoint, it makes sense that larger actions such as "start work" or "submit work" might not be sufficient enough to create strong developer loops. Starting or submitting work is a singular action that is only repeated on a project by project basis. However, I caution to say that we should simply focus on products like Grants and Quests that have a tighter, more "micro" engagement basis simply to boost retention numbers. Retention is simply, "a cohort is doing something on our platform at frequent intervals because they find our platform inherently useful (and fun)."

1. Maintainers stay if the quality of developers is high, and they can easily connect with them. We've seen this play out over and over again within our hackathons product.
2. Developers stay if they can easily learn, earn, and connect.
3. Both sides of the market stay when there's a strong, active community, and multiple opportunities to engage.

The conclusion of this analysis makes me question what the unit of engagement is for the Gitcoin platform. The number one priority is still providing enough value on our platform that developers find it useful to engage with us. The root is still making sure that the quality of the Gitcoin network stays high.

1. Do we need to restructure any of our products or how we view Gitcoin in order to cycle through this flow?: great developers -> happy maintainers -> more work and actions that serve developers -> more great developers
2. What is the entry point for a successful developer loop? Right now, it begins at bounties or a hackathon, but what happens if we structure the unit of engagement around a project (either inside or outside a hackathon)? What about around a community?