Lessons from Duolingo's failed innovation experiment

And what worked instead.

Engineering at Duolingo is known for rapid experimentation. Whether the goal is to update a button or ship a new feature, engineering gathers data from iterative user experiments — sometimes hundreds over the course of a week.

Many companies wait to launch features until they've been perfected and limit any experiments to small user pools. Duolingo takes a different approach, rolling out experiments to as many users as possible (with careful monitoring). This helps engineers make more informed decisions about experiments, faster.

Leadership has fostered innovation in a similar manner: test new initiatives across the organization, gather data, and iterate. Experiments have ranged from hackathons and productivity frameworks to cross-functional teams, yielding many successes and failures.

One of the most prominent failed initiatives? Google's famous "20% time" policy.

Today, we'll dive into Google's 20% time policy, why it didn't work for Duolingo, and what has worked instead.

What is Google's 20% time policy?

When Google went public in 2004, founders Sergey Brin and Larry Page wrote in their IPO letter:

We encourage our employees, in addition to their regular projects, to spend 20% of their time working on what they think will most benefit Google. This empowers them to be more creative and innovative.

The letter credits the 20% policy for a few key advancements, including AdSense and Google News.

It's not totally clear if this policy is still in use. In 2020, a Google spokesperson said it's an active program. However, multiple employees have described barriers to using 20% time, including requirements for managerial approval and rigid productivity metrics.

Some employees have even referred to the policy as "120% time." They say it's impossible to scale back regular, day-to-day work to 80% while hitting company goals. If you want to innovate, it has to be done on top of your full-time responsibilities.

Despite questions about the feasibility of 20% time, it's an alluring idea for companies wanting to spark creativity. Many have tried to implement it, with mixed results.

In 2008, Atlassian tested the 20% policy for a year. End-of-year data showed that it was more like 1.1% time; still, 48 projects were started or completed. According to developer survey data, the biggest hurdle was carving out time for "side projects" while continuing to deliver on expected work.

Atlassian found greater success with a new iteration of 20% time: Innovation Week, first piloted by the Product Security team when they shifted to remote work. Instead of leaving it up to individuals to make time during development cycles, the team set aside 1 week every 5 weeks to form ad-hoc teams and tackle different problems. 80% of team members felt they got more quality time to work on their side projects during Innovation Week, as compared to previous implementations.

20% time at Duolingo

By the time Duolingo tried 20% time, there was a wealth of data from other companies to inform their decision. While some companies had reported successful outcomes, most had mixed reviews at best. Given Duolingo's commitment to rapid, large-scale experimentation, they decided any learnings from the experiment would be worth potential inefficiencies. 

Duolingo's verdict?

According to CTO and cofounder, Severin Hacker, 20% time was not a good fit for their organization.

Here's why:

Not enough time to start meaningful projects.

Innovation requires deep concentration and sustained effort. Duolingo conducts so many experiments already that it was difficult to carve out uninterrupted work time between daily obligations.

Plus, high-impact projects are rarely the work of one team. They often involve multiple dependencies and ongoing cross-functional collaboration. Duolingo developers could barely enter flow states themselves, let alone coordinate across multiple teams to ramp up their 20% time projects.

Vague framework discouraged participation.

Google's 20% policy is very high-level, providing little guidance around what constitutes a suitable project or how to allocate time. Duolingo developers were unclear on many details about the framework, including the extent to which projects should align with the roadmap.

As a result, many Duolingo developers were hesitant to invest in projects that could be seen as unrelated to their primary responsibilities. While side projects can lead to innovative solutions that move the company forward, there's always a fear that they'll be perceived as diversions — which might hurt developers' productivity metrics. 

3 initiatives that worked

Like many companies that have tried 20% time, Duolingo developers struggled to balance innovation with expected work. Even when they did manage to secure time for innovation, it just wasn't enough to gain momentum on collaborative, high-impact projects.

Atlassian addressed similar issues by evolving 20% time into "Innovation Week" events. When projects showed promise during Innovation Week, they were moved out of "side project" status and added to the roadmap.

Modifying 20% time to suit your organization is one approach. Duolingo ultimately went a different route, instead deciding to pilot new innovation initiatives. Here are their 3 biggest successes:

1. Innovation Awards

Each year, Duolingo recognizes team members who move the product forward with boundary-pushing solutions. This incentive has generated significant buy-in from developers. However, Duolingo recognizes that the awards work best as one facet of a rich culture of innovation. It doesn't provide a robust framework for getting creative projects done, but it shows that the company is committed to supporting experimentation outside of day-to-day work.

2. Hackathons

Duolingo involves all members of the company in hackathons, not just developers. Participants take two full days to collaborate on and demo projects that excite them (and that align with the company's mission or business goals). The benefit of hackathons is their structure. Developers don't have to search for time in their schedules, write proposals, or worry about project guidelines. All they have to do is show up and go.

Most projects don't end up taking off, but some of Duolingo's biggest features started as hackathon projects! The Duolingo English Test and avatar builder are just two success stories.

3. Special Projects

The best side projects solve a problem that no other team could tackle — at exactly the right time. It's an elusive combination, which is why most hackathon projects don't move beyond the demo phase.

Duolingo set out to create optimal conditions for innovation with the DARPA program (Duolingo Advanced Research Project Agency). At any time, one or more team members can pitch an idea. If decision makers agree on its value (and if it's not in the domain of another team), DARPA forms an interdisciplinary group to bring the idea to life.

DARPA boasts some pretty successful outcomes, including the famous Duo and Lily mascot suits that you've likely seen on socials. While it can be tricky to transfer ownership of projects once ad hoc teams disband, DARPA provides the structure that developers need to make innovation a reality.