Teach LA’s Intern Search Experience

Some of our lovely interns and board members!

written by: Bonnie and Sophie • Dec 16, 2020 • category: internship • 2139 words • ~ 11 minute read •

We just wrapped up Teach LA’s internship recruitment process for 2020, and are welcoming twelve wonderful interns to our team! In this post, we aim to be as transparent as possible and reflect on our process.

ACM Internship Program Overview

Applicants apply through the general ACM internship program application, where they can indicate which committees and positions they are interested in. The application itself has three parts: general background questions, general ACM questions, and committee-specific questions. This program is led by our wonderful ACM internship directors Sagloria (Sahen and Gloria), and this was the first year Teach LA accepted interns through this process. So you’re probably wondering …

What do Teach LA interns do?

Interns work closely with our board on a wide variety of projects to level the CS education playing field. We divide our interns into cohorts that focus on similar goals/projects (e.g., curriculum, logistics & outreach, dev team, etc.). Above all, we want to help interns grow (as a teacher, developer, and person)!

What is the difference between a Teach LA general member and a Teach LA intern?

General members can teach as an instructor, develop curriculum, or join the dev team. They attend curriculum/dev team meetings.

Interns help lead curriculum meetings, make sure Teach LA runs smoothly, or lead projects on the dev team, depending on their role. They are included in both ACM officer/intern events and general curriculum/dev team meetings.

The Written Application Review Process

This year, ACM gave each committee the option of reviewing internship applications blind, removing all identifying information that would make it possible to assign scores based on demographic information, rather than merit. Teach LA ultimately made the decision to review each application three times: once blind, once non-blind, and once (non-blind) by Matt, our amazing club president who spent six hours looking over all 71 written applications in one sitting. (Please don’t hesitate to check in with Matt about his horrible sleep schedule at any time; he loves getting negative feedback about his unhealthy habits!) By assessing the difference in blind vs. non-blind scores, we were able to give each candidate a fair chance while simultaneously assessing the effectiveness of the blind procedure for future use.

In terms of the information omitted when reviewing applications blind, we were not given access to the applicant’s name, gender, sexuality, work authorization, address, email, or phone and were also prevented from accessing links to their github/portfolio/socials. We could, however, review their activities, hobbies/interests, GPA, and high school.

The blind vs. non-blind application review procedures were ultimately put into place to prevent inherent bias. For a bit more background, studies have shown that when identical resumes are submitted to job postings, one with a traditionally white name and one with a traditionally black name, the resume with the “white-sounding” name is significantly more likely to be chosen. In fact, a black applicant is about as likely to be chosen as a white applicant with a criminal record when the resumes are otherwise identical. Although we trust that our board members who reviewed each application would not favor applicants based on demographic information, Teach LA is committed to equity, diversity, and inclusion (EDI), so we collectively agreed to test out this new application review procedure, thereby ensuring that each applicant was given a fair shot.

Each written application was given a score in several categories, each of which was weighted differently. Overall, we were looking for applicants who demonstrated clear passion for Teach LA’s mission (providing CS education to all students) and also took into account teaching and leadership experience, as well as commitment to EDI. The exact score breakdown is as follows:

Passion + Effort (10 points)
Interest + Experience (10 points)
Diligence + Commitment (8 points)
Leadership (5 points)
EDI (2 points)

As previously mentioned, each candidate was scored by three reviewers: a blind reviewer, a not-blind reviewer, and Matt. To account for internal consistency within each reviewer, we normalized the scores that each reviewer gave. We noticed that there was a slight (but not overtly) symmetric bimodal nature to the scores for each reviewer; to test the rigidity of our hypothesis, we examined two different normalizing measures:

(current point - mean)/standard deviation
(current point - min)/(max - min)

Interestingly, the top 21 candidates did not differ regardless of the normalizing measure that we used. Thus, to extend the offer to interview for our final 21 candidates, we simply picked the candidates with the top 21 total scores. We equally weighted the individual score given by each reviewer.

After looking at the entire written application process, there were several interesting results:

There was a correlation between written application score and final acceptances, in particular with the applicants with significantly higher written application scores (> 1 stdev) - we offered all of these candidates final offers.
There is a correlation between being involved in Teach LA prior to the application process, and a higher written score. Out of the 21 interview applicants, 11 (52%) of them were already involved in Teach LA in some capacity; in contrast, less than 20 of the remaining 50 applicants (40%) had interacted with Teach LA at all.
In general, there was not much disagreement between blind and not-blind reviewers. In particular, each reviewer’s subset of the top 21 candidates was at most two candidates off from their internal top 10 list.
Matt was the only reviewer who reviewed every application. In general, Matt agreed with his other reviewers; the 21 interview candidates were within his top 24 candidates in total.
We were unable to collect demographic data about our applicants due to a logistical error. Unfortunately, this discounts our ability to evaluate if our written application scoring particularly favoured a specific group of applicants.

Given the relatively small sample size of reviewers, it is tricky to make definitive conclusions from our data. And, of course, we need to recall that correlation does not imply causation. Still, we value reflection and introspection for our process, and look to make it better next year!

The Interview Process

After reviewing all 71 written applications, we moved onto the interview stage of the internship search process, scheduling 15-minute interviews with 21 candidates. Though the written application questions were quite general, the interviews consisted of both introductory questions and position-specific ones: interviewers were given a list of intern positions and descriptions, and were asked to come prepared with the one they wanted to apply for. Although there was a very limited period of time during which we were able to schedule interviews, the time constraints of such short meetings admittedly made it quite difficult to get to know some of our more shy candidates especially.

Tips for Future Applicants

If you’re considering applying next year, we’d love to have you join our team! Here are some tips:

Don’t be afraid to let your personality come across in your written application and interview! You obviously want to make applications for jobs, internships, etc. as professional as possible, but whoever is reviewing your application is a person too. The applications I found the most memorable were not necessarily the ones that showcased the most technical experience or the ones with the longest answers. Just make sure your answer reflects some sort of quality that would make other people want to work with you.
Be as thorough as possible when answering interview questions. One-word or single-sentence answers do not give your interviewers a great sense of your personality, and if you were able to get to the interview stage, you definitely have a lot of great experience they would love to hear about! When answering a question, think about why it was asked. For example, “Describe a time you had conflict when working in a group.” is being asked to determine how you communicate with others and deal with obstacles. When answering, you should not only describe the conflict, but also talk about what you did and how you demonstrated qualities that are desirable in the role for which you are interviewing.
If your interviewer(s) can tell that you are nervous, they don’t hold this against you! We all know that the ability to answer a ton of super-specific questions about yourself and your experience is not always the best indicator of how good you are going to be in the role to which you are applying. If you feel bad about an interview experience because of how nervous you were at the time, remember all of the amazing information you were still able to provide, especially when you also submitted a resume/written application!
If you are applying for a leadership position in an organization that meets regularly and is open to the public, attend general meetings if you can! We loved to see familiar faces among our applicants, and I also noticed how much more comfortable people were during interviews when we had already spoken to/worked with them previously. You’ll also be able to get to know members and the ~vibe~ of an organization before applying, which is very important too!

What We Learned

As an organization, we learned a lot throughout our internship search. The following is a list of ways in which we can improve upon this process in the future:

Ask at least one specific question about each applicant’s commitment to EDI in the written application. We gave each written applicant a score assessing their commitment to EDI (for more information, see the section “The Written Application Review Process) without asking any specific questions about equity, diversity, and inclusion in the application itself. It is a bit unfair to assess candidates in a category of which they were not even aware.
Get a better sense of our applicants’ experiences and personalities during our interviews. The most obvious answer would be to cut down the number of interview invitations we send out, but we can also address this issue by making our interviewees feel more at ease from the start. We asked each candidate about their passions at the beginning of every interview, which gave us an opportunity to get a sense of their personality, but some people felt pressured to make their answer academic or computer science-related. In the future, I think we should ask people about hobbies/passions beyond CS or come up with another question to make our applicants more comfortable and willing to open up.
Ask about positions of interest in the written application. We decided that interns would have defined positions (e.g. “curriculum intern,” as opposed to the more general role of “Teach LA intern”) but failed to make this choice clear to our applicants. The decision to omit this information from the application was intentional, as we did not want to discourage passionate people from applying just because they didn’t find a role that fit them. However, this made the deliberation process difficult, especially when scheduling interviews. Although we wanted to make sure we had appropriate interviewers for each interview (e.g. someone with technical expertise for dev team roles, the logistics director for a logistics intern role, etc.), it was hard to match people given that we didn’t know what specific role(s) each applicant was interested in until their interview.
Clearly define the Teach LA intern role. When answering “Why do you want to join Teach LA?”, a lot of applicants said they wanted to teach or build technical skills, which could be done by joining as an instructor or dev team member. To avoid this misunderstanding in the future, we could work on clearly defining the Teach LA intern role (both in the application and in general) and make it clear that anyone can be a part of Teach LA without applying.
Consider a more distinct dev team application process. The classes, curriculum, and events team within Teach LA require a significantly different skill set than our dev team; in particular, teaching experience and event planning is vastly different from software development. In this written application cycle, we did not discuss this difference at length or spend a significant amount of time evaluating our dev team interns for technical ability. We believe that making this distinction more clear may increase the number of applicants for both dev and non-dev roles, as it makes it clearer to applicants what types of skills that they may need.

Conclusion

Thank you for reading this post! We hope this provided more insight for the inner workings of our internship program.

Got any feedback? Feel free to reach out to us, we’d love to hear what you have to say!

If you’re interested in joining Teach LA, you don’t have to be an intern!! See our join us page to get started!

Authors

Bonnie Lee (she/her)

Advisor (ACM)

Sophie Schoenmeyer

Sschoenmeyer