A startup in the making (post #1)
A glimpse into what it's like building Strong Yes, our process, and the lessons we are learning along the way.
Q: Why are we writing this?
To provide an honest look at where we’re coming from, what we’re doing, and where we want to go. Cause we're not there yet.
It’s easy to find examples of startup success stories and the journeys that gave them their success in hindsight. For example, we’re big fans of Y Combinator and thankful for them sharing so many insights with anyone interested. We like them for being truthful, how starting a company is not easy, and how real progress means providing real value to a set of customers, not the glamour and glitz you often hear.
But it’s hard finding examples of people building their companies in “real time”; going from an idea to a solution that works, and all the trials and tribulations on the way. Three examples come to mind:
Startup (a podcast about building a podcasting company)
Kevin Rose (building the meditation app Oak together with his Facebook audience)
Sahil Lavingia (sharing board meetings, and Gumroad improvements on Youtube).
There’s something refreshing about this honesty of building in public. We’ve learned a lot from the examples above, and even found ourselves routing for their success.
This is now our chance to share what we’re learning about hiring, and the frameworks and processes we are using along the way. We’re always looking on how to make this more useful. Send us your ideas here.
Q: What are we building?
For the longer story of why we are building Strong Yes click here.
TL;DR - we are trying to solve two problems: 1) getting more qualified candidate leads, and 2) getting a clearer and faster signal of a candidate’s fit.
The market is filled with tools. Most are focused on helping teams track the application process. But we haven’t found one that actually helps hire better.
We want to solve that.
Our first month
We circled through three different ideas in March. Here’s how we came up with each idea, our way of testing if they work, and what we learned.
Idea #1 - Replace screening calls with video recorded work-samples from candidates.
The goal was to provide recruiters with more signal of a candidate’s fit in a fraction of the time. Screening calls usually take around 30 minutes, not including the scheduling and management of calendars, and often provide little information on a candidate’s fit. Time is spent on checking the required logistical boxes, and reviewing a candidate’s experience based on their CV.
We were inspired by Adam Grant who provides evidence on why work samples provide a better signal than traditional CV’s. Reasons include:
Most people lie on their CV’s (at least state a better version of themselves)
Asking to speak about one’s past experience on a high-level only favours candidates who are good talkers
When people rely on shared interests and employer names it quickly creates a homogenous culture
Work-samples on the other hand focus less on what a candidate says, and more on what they do, and can therefore be used as a simulation of the job in real time.
We thought why not have candidates record a minute long video showing a past project they are most proud of. We decided to start the role of the Product Manager (PM), as this was the role we knew best.
What we wanted to test
There were two hypotheses:
would candidates even record themselves and their work samples, and
would recruiters benefit from them (being faster, and a stronger talent signal)
If candidates would not record themselves — well, then we would have an obvious problem. We didn’t really buy the argument that that would only filter out poor candidates with nothing to share. We knew that strong candidates have more employment options, and therefore would be less likely put in any extra work. Tapping a button on LinkedIn, or uploading a CV is still easier than recording yourself speak about your work.
What we did
We found 20 PMs in a Slack group that had recently asked the community to provide CV feedback, and asked them whether or not they wanted feedback on past work-samples. In our message we said what work samples are, how we would provide them feedback on three important PM skills (“Communication”, “Strategy”, and “Customer Empathy”), and a link to our VideoAsk where they could easily record and send us their snippet.
What we discovered
Out of the 20 we received three submissions. We didn’t know if this number was good or not, but told ourselves that the number would surely be higher with real candidates. The urgency and impact of applying for a job is greater than getting feedback by two no-names.
The uncertainty around conversion was then overshadowed with a bigger problem: being able to work with the recordings the sent, and judge them based on our predefined competencies. To make it easy for us we asked ourselves the following “yes” or “no” questions:
It quickly became clear that watching a video and being able to provide easy evaluations in less time than jumping on a call was harder than we thought.
We were happy to realise in just under 2 weeks that this solution did not fix the problem. So we decided to move on to idea #2.
Idea #2 - Ask candidates a set of questions using prerecorded video to evaluate specific skillsets
Instead of making the task open-ended as we did with work-samples, we thought that asking more specific questions would help us evaluate the responses better. A set of focused questions would not require the candidate to structure their answers as much. But we were okay with that. It was more important for us to be able to use the work samples they sent, and drill deeper into what they were saying.
We chose video as a format, and not text or audio. The reason being that video provides such a closer and more intimate look into who the candidates are. Since so many communication tools are jumping to video as a way to engage, including Loom, VideoAsk, and now Slack, we were optimistic that we could use this behavior for hiring as well.
What we did
After reading the book “The Right It” by Alberto Savoia we decided to test our hypothesis using a framework called “pretotyping.” The idea here is to test assumptions as early as possible before even building out a full prototype. We went with the format “Pinocchio” aka “dog fooding” where entrepreneurs use the product themselves to see whether or not they would continue to use the product, and see value in it.
We decided to simply interview each other. The two assumptions we wanted to test:
how easy did we find it recording ourselves answering interview-style questions, and
are we able to evaluate each other faster and easier with these recordings
What we discovered
There were three things we learned.
That the comfort level of recording oneself can vary. Stephen felt more comfortable recording himself, whereas I took 2-3 takes.
Even with the best questions, it’s not that easy to evaluate the responses. We had done our homework, and read the WHO framework (which we wrote a little about here), Topgrading by Bradford Smart, Julie Zhuo (VP Design at Facebook), Brent Tworetsky, and Lenny’s Newsletter. But it still felt more of art to quantify the responses.
We weren’t convinced we were solving the problem. Perhaps the hardest thing to do is change someone’s behavior. Until now recruiters and candidates would simply jump on call. By replacing that with asynchronous video we were solving the problem around scheduling the calls, but solving that didn’t seem to justify changing one’s entire habit. We wanted our solution to be solid, 10x better in some way, and make someone’s existing function easier. And this solution was not that.
So we took a step back to drawing board, and decided to take a different approach. And this is where we are right now:
Idea #3 - Attract stronger talent using employee referrals.
At our previous company we hired two people that came via referrals. It kinda worked for several reasons:
the candidate already knew arguably the most important part of their new job: the people. This really helped in the onboarding.
the people making the referrals were great teammates which gave us the hiring team confidence in who they referred
jobs and referral bonuses were handed out
This process was really manual. Teams were informed about our referral program once every couple months. Individuals would have to proactively seek out what roles were open, and ask the hiring managers or recruiters for the status of their referrals. Despite them not being easy to use, they were still being used. We knew from LinkedIn Research that with referrals companies can hire faster, cheaper, and retain employees more than via passive recruiting. Seeing the potential, we listed three areas where we thought we could improve referrals:
1. The incentive
How might we make incentives more visible to teams
How might we make them more dynamic based on the talent scarcity of a given role, or the urgency within a company
2. The evaluation
How might we make referrals more informative to recruiters and hiring managers, and
gather more information from the person that knows them best: the referrer.
In this video both Stripe founders Patrick and John Collison, and Pinterest founder Ben Silbermann talk about what it’s like hiring early on in their companies. Early on Pinterest hired using references from current colleagues, and one way they quantified the evaluation was by asking colleagues if the candidate was in their top 1%, or top 5%.
3. The candidate experience
How might we be able to provide a unique (or shorter) candidate experience for the referee, depending on the information provided during the referral
How might we be able to keep the evaluation fair and objective also for those applying without a recommendation
We decided to map out reasons why referrals might not be used by teams on two axes: (x) our confidence in our ability to solve, and (y) the size and impact of the problem.
For example one reason why referrals might not be used is because employees simply don’t like their employer. Well, that may be a huge reason, but there’s also nothing we can do to solve that. On the other hand, not making time for sending out referrals may be a problem we can solve.
The value in making time for culture and people in organisations can be best seen at Airbnb, with rituals like “new hire tea time” to welcoming new hires.
Starting last week we’ve been talking with recruiters about their biggest problems, hoping to shine light on the following:
What are the greatest pain points recruiters face, and how do referrals compare to passive sourcing and active sourcing at the top of the funnel?
Do they see the upside in referrals, and if not why not?
How much effort have they put in implementing a referral program, what’s been the biggest challenge, and how are they going about these challenges?
We’ve been tapping our existing network as much as we could in order to build a catalogue of recruiters to speak to. One helpful tip we’re using is to always ask at the end of the interview who else we should talk to. This question has made it so we never run out of people to talk to, and have on average five lined up per week.
We’ll continue to use the Mom Test to talk to customers (recruiters). A snippet of a good and bad question:
"Would you buy a product which did X?"
Bad question. You’re asking for opinions and hypotheticals from overly optimistic people who want to make you happy. The answer to a question like this is almost always “yes”, which makes it worthless.
Let’s fix it:
Ask how they currently solve X and how much it costs them to do so. And how much time it takes. Ask them to talk you through what happened the last time X came up. If they haven’t solved the problem, ask why not. Have they tried searching for solutions and found them wanting? Or do they not even care enough to have Googled for it?
Rule of thumb: Anything involving the future is an over-optimistic lie.
Stay tuned for our next newsletter, as we’ll report on the outcome of these interviews.
Thanks for reading. We hope this has been fun.