Technical Interviews at BJSS: The Format, Criteria and Approach We Use

By Oliver Hathaway, Engineer at BJSS

Technical interviews are a staple of technical jobs that conjure images of being asked to implement an algorithm on a whiteboard or perform a trivial coding challenge as some kind of rite of passage before a “real” interview. At BJSS, we’ve taken a close look at the technical interview process and ensured that we’re getting the most value out of the little time that we have with a candidate before a decision is made.

Broadly speaking, a technical interview is an assessment of a candidate by their equally technical peers, used to determine whether they meet the level of the business’ standards. It’s always performed by at least two engineers to ensure that the evaluation doesn’t hinge on one person’s opinion and to reduce the pressure on the interviewers, with one able to simply observe when the other is talking.

Whilst a CV provides an indication of how long a candidate has worked in the industry, experience has shown us that where someone has worked and what they did there has a greater influence on capability than simply the number of years served. One engineer with only a couple of years in the industry may accrue a great deal of experience, whereas another may have spent a decade in a role that never gave them the opportunity to go beyond the basics.

Where a candidate claims experience in a particular technology, a technical interview provides the opportunity for engineers familiar with that technology to assess whether the candidate is truly proficient or if they’re just talking the talk. The ability to write a great looking CV is not something that we generally hire for.

As technologists with real lives, our enthusiasm runs the gamut from a keen follower of the cutting edge of industry advancements to leaving work at work. We’re always keeping an eye out for candidates who show a passion for tech, and technical interviews give them an opportunity to discuss their personal projects with people who will appreciate them as more than CV padding.

Ultimately, whilst all skills and abilities have their place, compensation reflects technical ability and not number of years worked, level of education, or charisma (which is all you can tell from a CV or traditional interview).

The BJSS Technical Interview Format

Our technical interview format follows a straightforward two-hour format:

Greetings and introductions
45-minute paired programming exercise
30-minute high level “whiteboard” session
General questions
Candidate questions.

In times gone past, we would ask candidates to perform a take home test – a coding exercise for a candidate to show off their programming skills that we’d discuss in the interview – but asking candidates to take time out of their lives is more likely to cause talented people to go somewhere else. Our present interview process is based around the idea that a candidate should be able to walk in off the street with zero preparation beyond a CV.

Each step in the interview process has a particular purpose, and the greetings and introductions step is no exception. During the first few minutes of the interview, we establish a relaxed tone, have each interviewer provide as much information about themselves as the candidate has given, and set out the plan. Candidates often come into an interview having been given little information from a recruitment agency beyond a time and place or Zoom link – this simple step removes the fog of war and makes everything feel a bit less like a mystery.

The name paired programming exercise isn’t a euphemism for programming test, it’s designed to be exactly what it sounds like. The emphasis of the exercise is not on whether the candidate can perform a simple coding test in a time limit, but to see what it would be like to work alongside them and understand how they approach problems. Almost nobody ever completes the brief within the time limit, but seeing how someone approaches problems, and the subconscious reflexes and practices that are developed from years in the industry, are generally more telling of someone’s ability than seeing what they produce.

The next step is for the candidate to talk through a high-level architecture or solution that they’ve worked on in the past. We’ve found that if we discuss something that the candidate has done, rather than proposing a theoretical problem for them to solve, it’s substantially less stressful. Ultimately, the purpose is to have a chat about big picture development rather than the nitty gritty of coding that we’ve already covered in the paired programming section. Not all candidates will have been in a position to design an entire system from scratch, but everyone will have worked on a large system and generally developed an understanding and opinion of the big picture.

After these two planned sections, we enter into free-form general questions. This gives us an opportunity to chat about interesting things that may have come up earlier in the interview. Often these will be questions about things that a candidate has included in their CV, about Agile methodologies, or their approach to testing. The question section is the catch-all for exploring topics that don’t fit into any of the previous sections.

Finally, “Any questions?” is a good deal more than a prompt to wrap up, it’s an opportunity to give a candidate insight into what it’s like to work here. There are a lot of reasons aside from a pay cheque for people to want to join one company over another and this section of the interview is always given enough time for the candidate to ask about them. It also lets candidates understand the nature of consulting and figure out if it’s for them.

Our Assessment Criteria

All parts of the interview and all actions the interviewers take can be boiled down to one thing - getting a read on the candidate. There is no score nor pass/fail questions, it’s a matter of two experienced engineers sizing up another’s technical abilities based on their experience in order to answer a single question - would you work alongside this person?

This question holds many facets: If you were put on a team with the candidate, would you have confidence in their ability to write good code? Do you think that they would rush and miss things? Do they understand the principles that underpin software engineering? It also explains why we have the first exercise as paired programming - it gives an impression of what it would actually be like to work in a team alongside the candidate.

A CV suggests, based on experience, how good an engineer a candidate may be. The technical interview process should be sufficient for us to understand whether those assumptions are correct. This works in both directions; some candidates perform at an ability level well above what their experience would suggest, some fall well short, but it’s part of the interviewer’s role to make that judgement.

It’s crucial to understand that, in a technical interview, there are more things that we absolutely do not care about or judge than things that we do. A more traditional interview will help to inform whether the candidate would be a good cultural fit, so during the technical interviews we neither form, nor feed back, an opinion on this unless it’s a staggeringly obvious no. Similarly, things like gaps in CVs or why the candidate is looking to move jobs doesn’t matter in the slightest. We only care about technical ability.

There are no right or wrong answers in a technical interview. The paired programming exercise has a time limit only so that it doesn’t take up too much time - if the interviewers haven’t managed to get a good read after 45 minutes, a further half an hour won’t help. One of the reasons for publishing this where it can be read by both potential candidates and BJSS interviewers is to demonstrate there are no secret red flags to look out for, nor five stock phrases that someone should use in an interview. It’s a qualitative assessment, because no two candidates are the same.

A brilliant jerk is still a jerk, and a charismatic fool is still a fool. The central question of the technical interview isn’t whether a candidate is technically brilliant, it’s whether we want to work alongside them. If they are difficult to work with, being a programming savant won’t help. Similarly, if they would let down any project team they were put on, it doesn’t matter how well they get on with the interviewers.

How We Keep Interviews Relaxed and Calm

Getting a read on the candidate is the primary goal of a technical interview, but 90% of interview technique isn’t about reading a candidate, it’s about making them feel comfortable and relaxed. This is why we start off with explaining what’s coming up - it will make the candidate feel like the interview is less unpredictable. It’s also why the whiteboard section revolves around a topic the candidate is familiar with but the interviewer isn’t - it gives them a sense of control over something that would otherwise feel entirely out of their hands.

For most candidates, interviewing is something they will do rarely but has potentially stressful consequences if they fail. For some, being offered a role may represent a life altering opportunity. Whatever the stakes, nobody attends an interview for a position that they don’t care about at all, so all candidates are going to feel some degree of pressure to make a good impression.

This pressure manifests as adrenaline and cortisol, the fight or flight hormones that induce panic, hamper higher cognition, and alter someone’s personality. It’s incredibly difficult to figure out if a mistake is borne out of ignorance or just stress - not knowing what you’re doing looks almost the same as panic. The other reason that we try to make an interview a calm and relaxing experience is because how someone feels during their interview is going to make up a large part of their impression of BJSS.

We’re not trying to judge someone’s ability to keep calm under pressure. A technical interview is so far removed from day-to-day work that if someone goes to pieces during a paired programming exercise, it doesn’t factor into our decision in the slightest. Even when work is high pressure or high stress, we always face those difficulties as a team.

Ensuring a Diverse and Inclusive Approach

The emphasis on getting a read on a candidate suffers from one glaring weakness: the easier you can put yourself in someone else’s shoes, the easier you can judge their strengths and abilities. This leads to a subconscious bias towards people like the interviewers. Our candidates come from a wide range of backgrounds, cultures, and experiences - and judgement within a technical interview should only reflect technical ability.

It's also important to look beyond someone’s educational background as a sign of their skill. Of course, there are plenty of computer science graduates who make great engineers, but we've also seen incredibly talented candidates come through with no formal qualification in coding whatsoever, let alone a university degree. It’s why we don't have standardised technical questions - raw talent trumps specific knowledge or experiences every time.

BJSS prides itself on having a great culture, but none of it depends on what candidates look like. There’s a certain (some might call it old-school) way of thinking that considers things like dyed hair, piercings and tattoos to be markers of unprofessionalism, so anyone with dyed hair, piercings or tattoos must be unprofessional. We believe that this way of thinking has no place in hiring - whether someone is professional is based on if they act professionally, not their appearance.

At the end of an interview, the interviewers write up feedback that is used by the recruitment team to help make a decision on whether to make an offer. Deciding what goes into this feedback can take a lot of thought, but deciding what to exclude takes only a second. Anything that indicates gender, ethnicity, or first language is irrelevant to a candidate’s technical ability or whether they’d be someone that you’d happily work alongside.

Summary

The technical interview is a technical process led by technical people
We’re trying to evaluate what the candidate can do, and whether their CV matches their ability
The ultimate question being answered is: would you work alongside them?
Providing a calm and relaxing experience is critical to being able to read someone
During the interview, the interviewers are ambassadors for BJSS
Anything outside of whether they can do the job is not something that needs to be considered as part of a technical interview