CodePath EES2024 Closing Keynote VIVEK RAVISANKAR - AI and the Evolution of Software Development

Welcome, welcome, welcome. I don’t know about you all, but I’m actually getting a little sad because it sounds like we’re at day three of the Emerging Engineering Summit here at CodePath EES 2024. I’m your host, Bobby D.

And this morning, I got a special one for you because I think we got a CEO for a company that y’all be using the platform all the time. And you’re going to get some really cool insights. But before we jump into that, let me first tell you day number one, day number two, if you missed it, I don’t know where you was at.

I mentioned it yesterday. You better have a good excuse. And remember, all of the sessions that we recorded or all of the sessions prior have been recorded, including this one coming up.

There are some logistic things that I just want to talk to you all about just really quick. So, first of all, I want to say thank you all for coming to our last day here at the Emergent Engineers Summit at CodePad. We have more chances for you to network with your fellow peers.

Remember, networking and connecting, it’s going to be your net worth. So, make sure you remember that, right? And another set of breakout sessions by Google, Warner Brothers, and Shell Games and more.

This afternoon, we’ll also have three hours of opportunities for you to connect with companies in their booth. But right now, it’s time for us to move into our last conversation, our last keynote for the summit. Like I said, I’m getting a little sad, but yeah, y’all see me come back up a little bit more.

We are thrilled to be hosting a conversation with Vivic Robin Sakar, the CEO of Hacker Ranking. Yeah, I said it. Of Hackering, and our very own co-founder here at CodePath, Tim Lee.

Vivic and Tim will be discussing AI and the evolution of software engineering. I hope that this closeout keynote session leaves you ready to leave a mark on our industry. So, thank you again for joining us.

You’ll see me a little bit later for question and answers, but let me have Vic and Tim come to the stage. Give them a round of applause. Welcome.

Welcome to the stage. Thank you, Bobby. Really happy to be here everyone.

We have a great session prepared for you and I think that we want to make it as concrete for you as possible. So this is going to be hopefully very actionable, very insightful and of course we are very pleased to have VC speaking with us. I want to see if we can get this is the last keynote.

Let’s get some chat going. Can 100% of us say hello in the chat. It’s a It’s on a about a 20 second delay for us.

So, you know, we’ll see. We won’t be able to respond instantly, but we want to, you know, get it there. So, if we could just like slam a hi, a hello in the chat.

Let’s just get, let’s just get some activity going in here. And then what I’d like you to say is, have you interviewed before? Yes or no?

You know, have you interviewed before? Yes or no? I know that some of us are kind of like pre- a technical interview, some of us are post techchnical interview.

So we’re going to have a great conversation today. The topics that we’re going to hit that I think will be really interesting is how is AI impacting software engineering careers. For example, I mean, hey, will we even need engineers anymore?

Or or maybe we do, but we’ll just need a lot less. Or or maybe even the reverse, there’ll actually be even more. Maybe everyone will be an engineer because you just kind of guide around this AI coder to to do the work and you kind of instruct it.

Verbally. So we’re going to talk about like what that evolution looks like and what that means for you. We’re going to talk about how AI is evolving the skill set of engineers.

So what skill sets aren’t going to be as relevant anymore? And what new skills will be necessary? I think this is really important because is it fair to say that you know your schools as wonderful as they are, it’s hard for them to keep up.

So, it’s kind of up to you to track this so that when you leave, you’re as prepared as possible. And then finally, we’re going to talk about how how this impacts interviewing and your interview prep, right? Because like, you know, we’ve been doing it a long the same way for a long time now.

Actually, literally, believe it or not, 20 years ago, people were asking the same questions of engineers that they’re asking for you today. That tells you how like static that is. So, I think that’s finally shifting though.

So those are the topics and that we’re going to like touch on. But before we dive into that, I want to bring VC onto the stage. VC as you know is CEO of HackerRank.

HackerRank actually started about 13 years ago as a YC company. Since then they’ve raised over hund00 million. They are deeply widespread and they have a lot of insight on what companies are looking for in interviews.

So VC, I before we kind of dive into our conversation, I’d love to get to invite you to share your personal background a little bit. Like how did this can you bring us back to when you were a college student like like our audience is today up until, you know, where you are now. Yeah.

Hey, thank you Tim and thank you CodePath team for inviting me. Super excited to be here. Super excited to be talking to all of you.

Yeah, 13 years. I’ve never really counted. It does feel it does feel long.

I I certainly feel old because it’s it’s how long since I since it has been since I graduated. So I just a little bit of background on me. I am I’m 36 years old.

I came to the Bay Area about 12 13 years ago for the Y Combinator. Prior to that I was born and brought up in India, grew up in India, went to college in India. And I studied computer science and right after that I got a job at Amazon again back in India.

I worked at Amazon for a year and I got an opportunity to interview people as well as shadow people who are interviewing. And when I was doing all of these technical interviews, I realized that résumés did not have a great correlation to your skills. Which means I had interviewed people who looked really good on your resume but completely bombed the interview and vice versa.

So something was wrong, something felt off on just putting so much trust on a piece of paper. So that was the motivation for us to start the company to put skills over pedigree and it’s been a core ethos of us from the start until now and we started with a simple product called screen which allows companies to build take-home challenges which a lot of you might have might have attended or or or or attempted it and and really understand and interview people on skills based on the code that they write as opposed to what’s present in a piece of paper. So that’s how we got started.

But then over time we realized that this whole skillover pedigree can be applied to every aspect of the life cycle journey of a developer whether how you prepare for your job, how you get hired, how you continue to upskill yourself once you join an organization internally and yeah now we’re building a whole suite of products across the entire developer life cycle all based on this. It’s taken 13 years. It feels a lot.

I think we should have done this in five but but but here we are. You know, that’s an interesting story and and you know, one thing that’s interesting is like audience, correct me if I’m wrong, but working in Amazon, I mean, that’s the brass ring. I mean, that’s something that a lot of us are working for, working really hard.

I remember, you know, striving for that, you know, back in back in my day, and that was just some that was kind of the end goal. And you had it and you were there and yeah, I get that you saw this problem, but like what what made you get off that that track because that’s that’s the track that we were a lot of us are are hoping to get on to. Yeah.

This is this is very nostalgic u because this this is the exact conversation that I had with my mom who said the same thing. Why are you quitting Amazon? That’s what everybody wants to get into.

Why do you want to do this? I mean they were certainly very supportive. I will just tell you our our u so I started this with my co-founder his name is Hari.

We went to the same college. And he he was working at IBM. So I quit Amazon.

He quit IBM. And frankly the thought was very simple. Look we’re just like 22 23 year old guys single living in a in a very small apartment.

Super low burn rate. So why don’t we just try this out for two years? If it takes off, great.

If it doesn’t, what’s there to lose? Like you can always go back to Amazon or IBM or like one of the companies and and and get going. So there was nothing really to lose and everything was upside with a very low burn.

And yeah, that that was it. And so so we just jumped in and and wanted to test this out. And you know, you’re only a year into your career.

So basically you’re you’re only just at that time a couple years older than the audience now. And so what made you think that you could start a company? Like I I get that like okay I get the risk analysis but like how did you even know how did you even know that you would be able to to to do this?

Did you have any insight? Because you’re in India too. You’re not you’re not in Silicon Valley where you’re surrounded by you know hundreds or thousands of like founders doing the same thing.

Like how did you have the knowledge of you know that you would be able to do this you know naivity and ignorance it is a bliss if if I actually I think somebody asked this to Jensen from Nvidia and he gave a very honest response which I was very surprised by in the sense that you know given how good Nvidia is doing somebody asked him like hey if you knew everything about how hard it how hard it is to a company like Nvidia, would you have actually started it? And he said, "Probably not." And this is Nvidia, which is like right now the hottest company in the planet. And yes, it’s probably it it’s it’s probably very very similar in the sense that if I knew how hard it was, I don’t know if I would have really started the company.

But all we thought was okay, all you need is to just like put up a product, buy a domain, put it on the web, have a pricing page, people will swipe your credit cards, they will start to buy the product. And I mean like how hard can it be? And of course it turns out to be much harder.

But like once you once you jump in you you’ve jumped into the world pool. So so it’s just going to continue to you just have to continue to learn continue to update and your thought on that. Yeah I’m sure that you have a million war stories of going through YC and the you know learning how to raise rounds and and and growing to your scale.

But let’s let’s actually now shift. Let’s let’s talk about AI and let’s talk about how you’re seeing the evolution of AI. Maybe let’s make it personal for now.

Let’s talk about like the HackerRank engineering team and how you yourselves are using like how is how what is the before and after of your day in the life of an engineer kind of like a year ago compared to today? Yeah, it’s a great question. I think if I think about if I think about like the job of a developer I think it’s well encapsulated in this concept called SDLC the software development life cycle and it has six stages you gather requirements on what you need to do you design your system you write code you test your code you deploy the code and you maintain like right I mean like those are the six things and that that’s like the task of the developer or the job of developer.

But what you see happening is a lot of these jobs have what I call as like schle tasks like things that you have to do but you don’t really like doing and you either grudgingly do it but a majority of the developers don’t even do it. That’s the reality. For example, writing documentation for your code is a shle task like but most people actually don’t really do a great job at it.

Writing test cases and coverage is an important one, but it’s always like, oh, the QA team will take care of it or my code is so good that I don’t need to write test cases and and things of those lines. So, so you constantly are ignoring parts of the parts of your job which I think you’re supposed to do but you’re not doing because it has like a lot of the grunt work present present there or reviewing other person’s code man like you know why do I have to review this like really poorly code and things. So we actually looked at all of these what you can consider as the toil that is present in the software development and see how much AI can really help make this better.

The two big shifts has been our own internal AI agents for writing test cases and for reviewing code. We actually trained on our own rubric on our own code base to actually say hey this is this is this is what we consider as a good quality code. This is how you need to give comments on the pull request.

And so today every pull request is first reviewed by this AI agent and then the developer actually goes and fixes it before you actually hand it over to your peer reviewer. And it’s first off, you’re able to just your cycle time is really fast because if I give pull requests to you, I don’t know if you’re going to be online right now. I have to wait for for a day and things of the science.

The cycle time is dramatically quick because I’m able to get like comments faster and you’re actually able to do the actual job that a software developer needs to do. So I think that’s been a big shift. So your your the code has gotten better quality, it’s gotten more robust because like you’re using AI engine to generate and write test cases and people are just like more productive but importantly people are happier.

So if you’re productive and happy that that seems like a magic drug that you want to take on. So so that’s that’s been the before and the after. Yeah, you bring up a really interesting point because you know this is a this is going to be a crowd interaction moment.

So, I’m going to ask the crowd a question, which is you know, how many lines of code do you think a professional software engineer with a decade of training writes a day? So, write in the chat, how many lines of code do you think a professional software engineer writes in a day? I mean, they they’re seasoned.

They’ve been doing it for a long time. What do you think? What do you think that number is?

So I think VC probably knows where I’m going. I think that stat that’s been shared publicly is something like less than 10 lines of code a day. Something like seven lines of code a day.

And there’s good reasons for that because like you know hey actually the job of an engineer is a lot more than coding. It’s requirements. It’s system design.

It’s testing. It’s debugging. It’s it’s it’s engagement.

It’s collaboration. But also there is a lot of shleep work in engineering. You know, you you have 10% of your brain, you know, you you know, as is your high-end work and then like and then you do a lot of water carrying, you know, to get to to do the stuff.

So, let me ask you a pointed question, VC, then now that you have this great efficiency and and and the schle work is being cut in half, you know, are you going to let go of half the HackerRank engineering team? I don’t know if the hacker team is watching. So, if they’re watching, no, the answer is no. there is a there is an interesting principle in economics.

It’s called the lump of labor fallacy. And I think that is probably the best encapsulation of what I think will likely happen. It’s a fancy term for saying that you are your demand will always continue to grow.

Which is human beings will will never be satisfied with what you have. So, you had taxis and then and then like you said like, "Oh, I want to push a button and I have to get a car in front of me." And then now you said like, "Oh, no. I want to push a button and I I I want to get like a SUV. Oh, I want to push a button.

Why is it taking seven why is it taking seven minutes? It should come in two minutes." and and your your your needs never get satiated. And that’s like a that’s like a good thing u because we’re serving to humanity.

So, human beings needs are never satiated. So if your demand is never is always going to continuously change, you are going to continue to build more and more new things. And you ask, you go and ask any company, any engineering team, they have like a backlog of like a thousand items that they’ve never been able to get to because they just don’t have time to to to work through.

So I think you’re going to only get more you’re going to do you’re going to be able to do more things. And I don’t I I think in a world where if you think your demand is constrained, I think this makes sense. Okay, my pi is fixed.

My engineers are twice as productive, so I only need half. But when you can actually make the pie bigger, then you have the argument actually you have the counter argument to say maybe I need to hire more because now I can actually do more. There’s like the possibilities are endless.

So So that’s how I think about it. Yeah, I I think that’s really interesting. I think another way of saying it like correct me if I’m wrong is I think if your engineers can do more work there’s more money to be made from for HackerRank I think there’s more the revenue that can be increased if you can do more and I think the customer expectations are also rising so that if you don’t do those things too another company might come in and be the next HackerRank and kind of eat your lunch like remember when blackberry was omniresent and everyone had a blackberry and then they kind of got sleepy and they got you know complacent And then now you know who who knows where they who who knows where they are.

So that’s I I think I think there’s one aspect in terms of certainly being more productive and doing more. Then there’s the other aspect of like hey what are all the things that were not possible before that are possible right now with AI. Now let me actually give you a very concrete example with hackerang.

So today when you think about somebody writing code on HackerRank we automatically evaluate and run it against a set of test cases and give you like a score and output and things in those lines. But with AI now you can actually go ahead and we’ve actually done this training training an agent where it’s it’s actually able to go ahead and like look at the code and even say how close was this candidate to the actual answer. So now you you’re actually I mean our our one of the core values and principle for us it’s a it’s a core value for the company is developer first.

We want to always showcase the best version of the developer. So sometimes when you actually are going to give like a very binary kind of a score hey this is 25 out of 50 or like 40 out of 50. It’s very hard to determine how close was this person.

Sure yeah you were 25 points off but is it like one for loop away? Is it like one small optimization away? How do you actually determine that?

But now an AI agent can literally look at a code and be able to say, "Yeah, this is how close you were." And that’s like a new kind of possibility that was not even possible before that’s going to add more value to customers and help showcase developers in their best version. So there’s one aspect in terms of becoming more productive from an output perspective and there’s a whole other aspect of like new things that were not possible that you’re able to do right now. And the way you’re describing the way your engineers are using AI, it’s it’s the visual that I have is like, okay, before the engineer, you know, had say 80% schle work and 20% like high intellect kind of like work.

And then you’re also saying, well, and there were just some things that were just like plain impossible, right? And now, hey, I have this thing. And so the vision here is like there it’s kind of like an Iron Man vision where like you know Tony Stark is working with Jarvis and there’s this like partnership you know and so in that vision you need the human and the AI but just the pursuit of this line is are we going to a world where you don’t need the human where you don’t need the Tony Stark and you just have the Jarvis and in that case you know HackerRank can go and you know explore all these things with with only the AI agents like when is that world arriving?

Yeah, there are a lot of lot of possibilities. A lot of possibilities. I’ll say maybe like two things.

One is it’s very hard to imagine what we don’t have or what is what are the new things that we can actually create in the future with technology. And so every time when we think about jobs, every time when we think about work, we always think about from in the current context of the world. When there are like going to be new completely new things that are possible that’s just going to open up a whole set of different types of jobs or different types of work.

So again when we think about predicting the future it’s always like in the current context if there’s a new technology comes what’s going to happen. Yes there’s going to be disruption but like you know you also don’t know what’s going to what’s going to be the new ones. The second thing that I wanted to say is we’ve actually seen this play out in one of the professions that a lot of you might be familiar with which is the airline industry.

If you look at if you trace the evolution of aviation and like the composition of the cockpit it started with four people. You had a pilot, you had a co-pilot, you had a navigator, you had a flight engineer. This was this was the composition like 60 years ago.

And as AI started to get better and better or was called machine learning back then, you didn’t need the navigator. You had the pilot, you had the co-pilot, you had the flight engineer, and then AI continue to get better. Then you had the pilot and the co-pilot.

And today you just you you know, you have the pilot and the co-pilot, but in reality, 80% of the flight is really flown by AI. And the pilot and co-pilot really do the takeoff and the landing and like any other things in in the middle. But this hasn’t really reduced the demand for pilots.

This hasn’t reduced the need for them to know the fundamentals of aeronautical engineering. But this has absolutely changed their jobs. But now that AI is becoming mainstream because nobody has really flown or not not a lot of people have really flown planes and you don’t have access to clockpits and stuff.

But now AI is mainstream. It’s it’s on your iPhone. It’s on your app.

Open up Charg. So now that AI is mainstream, you’re going to see this kind of a human plus AI transformation happen in every industry. And you’re seeing that in co-pilots with developers.

You’re seeing that in customer support with like agents included in your support tools. So yeah, you’re going to see a similar kind of a transition like what the pilots have gone through in all all these professions with an asterisk. There might be some professions that may not exist or might look completely different with AI.

And yeah, we need to we need to navigate we need to navigate through this. What I don’t probably subscribe to and it’s just like my own thing is this whole super intelligence that will just like come and rule rule rule humanity, rule the world and and we’ll just be out of jobs and we just don’t lose purpose in life and things of those lines. Not not really.

I not a not a subscriber of that. I I I agree. I think I think that could be could be a hundred plus years away or or or more maybe a thousand years away.

And I I’ll answer my own question which is like I I think that you you’re right your vision of like this evolving role and this and this partnership between humans and AI will continue for a long time and I think that we’ll the takeaway here I want to capture you know for audiences I think your careers are safe you know your your careers are there but to Vic Beck’s point and this will be our next transition it’s going to look different than VC and my career and the way that we worked and and you know the pilots of tomorrow and to you know UI we we sometimes call this crowd the first generation of AI native coders y you know and so can we talk about that shift because you know in what people may not realize is that this is not the first time that you know skill sets shift software engineering because it’s it’s it’s very young and dynamic field there are things that you had to do 20 years ago that you don’t have to do now and there’s been a change in you know cert certain skill set. So like you know in the now with this new world of human AI coding what skill are kind of because skills a stack rank right so what skills are like drifting down in the list of like the essentials and what new skills are being added in order for you to be an effective you know AI coder. I have a lot of thoughts.

I’m trying to compartmentalize. So maybe I’ll just try to try to think about it in like three different buckets. The first one is I’ll talk about like how the AI native engineers build things differently.

So one of the observations has been let’s say if you’re going to go ahead and get started on a project or something prior to AI you’re literally going to write code as your first thing okay I’m going to like go ahead and start building this to-do app or whatever thing that you want to go ahead and build out a weather app application those are like some of the canonical examples today you actually your first line code in quotes is not the actual programming language code. It’s really a natural language. Your your first thing is like your it’s an interaction with an AI saying look this is what I want to build.

This is the spec. Give me the scapolding and then like I’ll go ahead and edit based on that. That’s very very different mentality like on how you’re even beginning a project and and then you have this superpower intern at least right now who can potentially be very very smart over time the AI co-pilot where you’re just like constantly interacting it’s like literally pair programming in as you’re as you’re starting to build the project.

So the way that you’re starting the project, the way that you’re continuing to build the project is completely different and and very very different from what what it used to be in the nonInative engineers for lack of a better term. So that’s like one one big shift. The second one which I think we all need to be careful is again if you were to trace the evolution of programming languages.

You started with assembly where you literally had to say like move this from this memory register to that memory register and and and you literally had to program like moving from one thing to another and then like things got more and better abstraction over time like cobalt was an abstraction on it. Then you had like C++, C and C++. Then you had Python which almost sort of like resembled natural language and and you actually see all of these technologies getting stacked one over the other.

There was a programming language thing, then there was a cloud thing. Right now like everybody was probably using AWS or GCP or one of the cloud services. You don’t need like distributed.

You don’t need knowledge of distributed systems to actually go ahead and deploy the app because it’s abstracted away from you. So AI is also going to abstract a lot of things from you and there’s probably a little bit of a risk because the more you abstract away the more you can get away with not knowing the fundamentals but when you really need it it’s going to like come and come and like really hurt you. So so so so that’s that’s something that we need to like really think about which is why interestingly going back to the pilots the interview process of pilots to test their aeronautical engineering the fundamentals of aeronautical engineering has not changed over the last 40 years regardless of how much the AI technology has actually evolved because you do need to know the fundamentals what if the AI doesn’t work like how are you going to actually navigate so you do need to like make that happen so the second one is it is you are going to work at a higher degree of abstraction your job will be more of like reviewing code and things on those sides as opposed to writing in terms of the split of the time.

But be careful to not lose the fundamentals. And the third one is I also think this is actually going to generally give rise to more people who can actually build things. I know so many people who really wanted to build things but who just got very intimidated with a black screen and a blinking cursor also known as terminal.

What am I going to do? And like and then like you write your first program, it gives like a cryptic dependency error. You can’t even do this.

So, a lot of people actually just like gave up like you know I don’t know how to actually build it. But you’re going to get a lot of people to fall in love with programming because AI is actually going to make it easier. Think about the number of video or filmmakers YouTube and iPhone has has created.

If you have an iPhone and you have an internet connection, you can actually make a short movie and put it on YouTube. You’re now a filmmaker. So, you don’t need to be like, you don’t need to go and study a course in LA or whatever and like to to become maybe maybe if you want to become Steen Spielberg, maybe you need to, but like you can become a filmmaker with an iPhone and a YouTube.

So, think about like how many such engineers AI can actually go ahead and create. So yeah. So I think I think all of these things of the AI native way of starting a project and continuing is going to change.

You’re going to work at a different level of abstraction, but be careful not to miss out on the fundamentals. And I think overall it’s just going to create more engineers. So that’s how I I imagine the the landscape to shift.

Yeah. And you know a couple things I’m hearing out of that like well one you know it’s not like your CS degree is total garbage. You should you should still pay attention to your fundamentals.

You’ll need that when you know AI fails you and you’re like it does something and you’re like oh I bet there’s a stack overflow or I bet there’s such and such thing going on or a race condition. But you’re kind of adding on to the skill of saying hey the interaction is going to be different. You said you you use the word natural language which kind of means for many of us English.

In other words you’re going to be coding not in Python but you’re going to be coding in English. And a lot of you are thinking oh fantastic I already know English. Like I’m good to go.

But I’ll I’ll okay do this is another crowd interaction moment, right? So you know speaking to your in designing a AI in the way that VC is is saying is something that a seasoned engineer can do because like believe it or not despite our reputation, engineers are actually amazing communicators when they’re at that kind of like 20 year level. They’re actually really good writers, really good communicators of architecture.

So, here’s here’s the crowd interaction kind of like challenge to you to illustrate how hard this is. Think about a project that you’ve worked on in the last year. Just any coding project, any school project, something that you can kind of pull pull it up in your head, visualize it, and I want what I want you to do is type in the chat a sentence summarizing the architecture of your program.

So, I’m going to let you do that challenge. Like, pull up, you know, in your brain the project that you’ve worked on and then try to make a really great recap of the architecture. Go ahead and type it in.

And and I think maybe what I’m the point that I’m trying to illustrate is that that’s actually really hard to do. You know, it sound maybe it sounds easy and maybe some of you are slamming it out and it’s and it’s it’s actually brilliant. In which case, hey, you’re way ahead of the curve.

But I talked to a lot of engineering managers. I asked them, hey, what’s the biggest weakness of your engineering team? And they said, well, they can code.

They get the code working, but they kind of like pointed the code in grunt is literally what he said. This is this is the engineering engineering manager at Stripe which is a great company. And so it just tells you like today’s early engineers actually don’t have the English speaking like the language skills that VC is talking about is actually really hard and it takes time and intentional practice to develop.

So I guess VC like what would you do if you were back in college like how would you because you know like once again like how do I practice that? How do I balance that? What do I do practically because my classes aren’t necessarily going to teach me to do that.

So I’m gonna focus on my classes and make sure I know the fundamentals, but then what what would you do now, especially if you you have to delete your knowledge and experience too. Yeah. You know, and then and then how would you kind of like what would you do as a student to kind of make sure that you’re starting to muscle up on this skill?

Yeah. I think there are two key aspects and I think one of them you touched upon which is hey when the AI goes wrong or does not give you the right answer you need to have a very strong understanding of the fundamentals to be able to go ahead and correct this or course correct it. There’s another aspect in terms of AI which is over time I think humans are going to get intuitively better at determining when to use AI and when when to not use it or or how much you can actually trust certain things and how much you don’t like you just it’s just going to be it’s just going to become part of part of you and in order for you to even like develop that level of intuition or one is suddenly you need to start to like work better with with all the AI tools and co-pilots and the other is you do need to have like a pretty strong fundamentals.

So now to to to go a level deeper into your question I would say yes fundamentals are important probably you’re learning that a lot from schools and and courses and other things in terms of how do you balance this I think the tools tool stack is changing I don’t know what people actually use but like there are lots of AI first idees that have come up cursor is a great example of that replet is another example of that u lots of AI first IDE ides that have really come up. So getting familiar with that to actually build your projects in an AI first IDE will be a good concrete first step to really understand what is possible and what is not. So yes, a combination of the fundamentals and getting your AI first toolkit would be a would be a good would be a good start.

Great suggestion. I use cursor. I love it.

It’s fantastic and they’re releasing new features every day. So, here’s the pro tip for students which is like how do I how do I get started? Step one, because you’re probably using you might be using chat GBT to engage with code and that’s a great route.

And you might be using GitHub copilot but the suggestion here is explore cursor. Cursor unfortunately is a paid app so it’s $20 a month. Kodium.

C O D I EU M Kodium is free. And that’s another neat thing. But like what VC is saying like is take it out for a spin.

Do maybe do the project manually or and then build another thing using this coding tool just to kind of like compare and contrast and do do it both ways. The oldfashioned geyser way and you know the new fangled way and and start to kind of like just dip your toes in. And I think the key here is like don’t don’t boil the ocean.

Just baby step. Download cursor or download kodium and then just make an app, you know, using that and just try it out. Try out Vzero by Versel.

Go search V0 right now. It’s a really cool, you know, AI tool where you can actually implement design. All right, let’s let’s shift gears.

Let’s let’s now talk about interviewing because I think people know you’re the pro at understanding what companies are looking for in interviewing. So can you talk about the evolution there like what is going to can you make a prediction of what is in the next year or two or three u for the for internship interviews and new college grad interviews? What is the evolution there going to look like and how should we prepare for that?

Yeah, I think if you were to generally look at the evolution of technical interview in itself, maybe 20 years ago, developers were asked questions like, "Hey, how many golf balls fit in a 747 had got nothing to do with coding, but really tested or assessed your computational thinking skills, problem solving skills, and others was a big reward from developers. Hey, this has got nothing to do with coding." So then then people move into what is now known as the algorithm data structures kind of interview. And there there’s like a little bit of a revolt or undertone that you can actually go and see on Reddit.

Maybe Reddit has undertones of revolt and everything but who knows. But certainly on this one which is like hey I don’t I don’t use merge sort or I don’t use this graph algorithm on my day-to-day work. Why are you actually interviewing me on this?

And then there’s there’s been an evolution of trying to move your interviews to resemble real world projects at least during the interview phase, not on the take-home assessment. And I think if you were to just like look at the evolution and if you just like plot the graph, where you’re going to see is your whole interviewing or the hiring experience is going to be almost indistinguishable from your real world work. That’s what it’s all going.

So that’s the direction. And I think in terms of the order of operation or the sequencing in which this is actually going to happen is likely the following. You’re going to see the interviewing process for the senior engineers resemble real world work first like very soon.

Like and and we’re already seeing that happen which means like your the interviewing questions have changed to more like here is a code repo here is a bug how would you actually go and fix it customers are getting comfortable with even like giving them AI tools and others and so that’s where it’s going to happen first and then you’re going to start to see that trickle down into juniors during the interviewing phase on it in terms of the take-home assessment which is predominantly used mostly in the early career to junior seniors don’t generally do take-h home assessments because I don’t know maybe they have like a big ego like why are you testing me and kind of things but like whatever on the early career to junior in terms of the take-home I do think it’s going to change to become more and more real world but the change would be much slower than we anticipate whenever we talk to customers customers are more interested in understanding the fundamentals of computer science from this cohort and I think they’re more concerned with of with the usage of AI tools in this process which is hey you are going to get an AI assisted co-pilot when you come to work maybe we’ll even like give that to you during the interview when the hiring manager or the engineering manager is looking at you and your pair programming that but for take-home I would really recommend or prefer not to use any kind of AI tools and because I really want to test your fundamentals of computer science without any AI knowledge And so what we’re starting to see is a much is the problem statements and the structure of the take-home assessments for junior remaining pretty much the same but with a higher degree of sensitivity to plagiarism and proctoring and and other aspects of it where like discouraging the use of AI tools in the in the setup. I like I mentioned I do think that will change but I think the change in this cohort or in this mode will happen much slower than what we all expect. And so you know what should I as a student do to prepare because like I still have to grind leak code probably so that part of my prep at least in the next six months or a year.

You know and I want to acknowledge and all not all companies use leak code style questions right I don’t know what I don’t do do you know the percentage by chance of like you know internship like what would you guess like is it 75% is it 50% is it 90% who use code it’s probably closer to the 75% in terms of the traditional algorithm data structure like I mentioned I think it was much larger like a couple of years back but that transition is slow it’s certainly much slower than what you would actually see as you as you become become more senior. In terms of like the real prep, u, we’re, this might sound like an advertisement, but but I’m just like, this was the top of mind. We’re actually, we have actually built an AI, interviewer, a mock interviewer.

And this is like this is one of those examples of things that were that was not possible before that you that that you can actually do right now. U, it’s actually available for free at at hackrank. You can go log in and try this out or u or you can send a note to support.com yeah feel free to send it and like we’ll actually activate that for you and you can go practice with an AI interviewer who’s actually going to give you only hints not going to give you the code and work through it so practicing that certainly practicing lead code certainly trying to understand the fundamentals to be really good taking a lot of practice assessments before before you take the real one.

Those are those are some of the things. I think I think maybe people will be familiar with Gail Lman who wrote the crackling the code interview. She’s actually coming up with a sequel.

We’re actually going to host like a a book launch conference of sort on HackerRank. So you can stay tuned for that. Like so practicing some of the some of the challenges or questions there.

Those are those are some of the top things u in my mind. Okay. Well, your support team might not like this, but like so I just want to review this because there’s an exciting interesting new free HackerRank, you know, thing that you can they can trial.

They go to HackerRank.com and if I’m logged in, is it will how what do they say to the support? Like what what do they say exactly to get this feature activated? Yeah, you can activate the new AI mock interview feature for me.

I just heard it from the bank in the conference. Yeah, do do it. All right.

So, you got supporthacker rank.com. Poor support. You should probably send them an email telling them, warning them, but they want you can try this AI, you know, mock interview feature, which which sounds really interesting.

So I think that let me think here. We’re right about that time that we’re going to go to Q&A. So maybe Bobby, we should make that shift now.

Well, I want to say thank you so much both of you Vivc first of all giving us some insight and Tim helping draw this conversation to where we at right now. We have our first question that came in and it’s really talking about software engineers developing prioritized skills u for the next five years in AI and what would they be? If either of you could answer the rate at which AI is developing five years I don’t know I don’t know what kind of u capabilities that AI thing will have here’s what I would say there’s there’s a common denominator of skills that I think will always be valuable in the long term and here are they in no particular order the first one is to truly understand the problem that you’re trying to solve for your customer.

I mean ultimately what is engineering? Engineering is trying to solve a problem for the customer and software engineering is just like doing it through software. So the first important skill for any engineer would be to really understand the problem that you’re trying to solve for the user in as much of clarity as possible.

The second one is ability to then decompose the problem into multiple sub items. A third one is to go ahead and make sure that you can implement this in the most robust way possible. These are like some very core fundamental skills and AI is going to allow you to elevate or enhance in each of these aspects.

So understanding getting better and sharper on the fundamentals and making sure that you’re always in the frontier of using AI tools to un to to develop a sense of intuition on where you can use it, where you can’t use it, what is it good, what is not good as new models come up, as new ideas come up. I think a combination of these two things is what u will likely stand the test of time. And you know, I want to answer because I want to steal VC’s answer from earlier, but I just want to double click on it because I like to make things like really simple, right?

Which is I think I think in the next five years, it’s still VEX answer. It’s still which is like how do I iron man how do I Tony Stark this LLM or this this AI and and how do I code practically in it which is going to be a combination of fundamentals and this new way of programming which is going to be a lot of English speaking. But here’s here here’s what Vivec solution which I I 100% agree with.

Go and download cursor or vzero and try to build something. You’re going to find out really quickly. It actually makes all kinds of it doesn’t it it won’t work the first time.

You’ll say, "Hey, make me a clone of HackerRank." And it will have bugs. It won’t work. And I don’t think you need to do anything special to practice your fundamentals because you’re going to have to apply your fundamentals right away because it’s not going to work.

Go to Vzero, which is super cool by the way. Go to Vzero, have it implement a mock and you’re going to say, "Oh, well, I got like 75% of the way there." And you need to kind of like you need a Tony Stark game. Be like, "Oh, try this." Remember remember how many times he worked with Jarvis to get Iron Man?

There’s this like back and forth. And so what you think you need to develop as an engineer via your fundamental school skills is your meter for AI. Because when AI starts going off the rails, you have to be able to smell it, you know.

So one, I’ll leave with a final tip, which is like when you have the AI do something, don’t say, "Hey, how do I do authentication for this app?" Say, "Hey AI, give me three options for implementing authentication in this system right now." And so and and give me the pros and cons. So when you do that then you start to apply your fundamentals. You start to be like okay well oh okay I could do I could go I could use passport I could use octa I could do it myself.

Oh what are my what are the trade tradeoffs? So that’s the final tip I’ll say do vive solution where you try to build stuff with AI. You’ll you’ll find the edges really quickly.

You’ll find that it’s it is awesome. I never want to code without AI again but it’s also like it’s deeply flawed still and it’s deeply limited in a lot of ways still. And then and you’ll have to bring your fundamentals to bear to to actually have it do what you want it to do.

So sorry VC for stealing your answer but I I want to keep focused which is like just do that keep on doing that. Yeah makes sense. Yeah.

And the next question up is actually going to give a Vivc and and both you Tim also a chance to to elaborate a little bit more and more specifically when I’m thinking about when I’m hearing the engaging potential that having AI inside of the interview process is me showcasing how I’m using this tool. The IDE is normally the way that we showcase that tool, but more specifically AI potentially could be. What do you all feel would be something that would be very noteworthy in terms of showcasing the skill utilizing the tool called AI during the interview process or maybe even on in an online assessment.

Sorry what is the skill that you would want to focus you’d want to showcase during an online assessment with AI? Is that the question? Yeah, I would say definitely.

I think I I I think again like online assessments using AI it’s not going to happen soon. I do think it’s going to be much slower in terms of the trans slower transition than than what we think. But some of the early adopters who are using AI in online assessments the even the type of questions are very different and again this is what I mentioned like you know whenever we think about the future we always think about the current context and then like superimpose like the technology and what can happen when like you know your core fundamentals are different so let me actually give you an example so today if you were to think about an online assessment it has very rigid spec very rigid constraints means very clear answer on whether you got it right or wrong.

For a small set of customers who are embracing AI and online assessments, it’s literally the opposite. You have a very open-ended problem. You don’t have a rigid spec.

It can have multiple different solutions and you’re actually just evaluating how this person works with AI to go ahead and solve the problem. So even the whole way of how you would think about a problem and interview completely changes in the new landscape. So yeah, I think it’s going to be a slower transition, but we’re going to start to see that see that maybe over like two to three years, but not not in the immediate term.

Awesome. Awesome. And and thank you so much for that insights.

I do have another question for you, Vivic. This one’s more specifically around what are the critical skills that you’re looking for when developing a team as a CEO and what can students be preparing theelves and more or less to achieve those goals? I I think it certainly varies.

I mean this this certainly a common denominator of of skill sets or values that we look for in every team member at Hackerank. We’ve codified that in terms of core values and we really think about our core values not just as posters on the wall but something that we really want to want to embody within the organization. So that that that’s probably one common denominator but then I’ll answer the question more more specific to to to students and this answer might sound really flippant or even like simple whatever but this this this is true which is I think there are like two really high two two really strong skills that you can develop.

One is a very high degree of reliability. Which is to say that if you’re going to say something and if you’re going to say that you’re going to do something, just do it. And u if you’re going to show up, like show up on time.

Like if you’re going to do Yeah. If you’re going to say that you’re going to get something done, like you have to go and get get something done. That’s like one is like very very high reliability.

The second one is and again it’s a very simple hack. Go to your manager and ask what can you take off their plate? It’s a very simple thing.

What what can I what can I take take off your plate and and when you actually take take something off your plate firstly you’re allowing your manager to work at a higher degree of abstraction but you’re also automatically expanding scope a lot of people when they think about like career progression and things of the lines think of like hey they go and search for career ladders that’ll actually tell you hey this is what you need to do at IC2 IC3 whatever those things are okay but I’ve never really been a fan of that the simplest way to grow your career is to just like continue to take more of your from your manager’s plate and like expand your scope So really if you do these two things I’m I’m highly reliable and I will just get take care of the thing that you don’t want to or or you’re full that is like probably the best way to continue to grow in your career. Yeah, for sure. And and I actually have a question for Tim here.

With the partners that we have what are you seeing in terms of utilization of AI is starting to become more commonly mentioned by the partners that we engage with including with Hacker Inc. Utilization of AI in the context of interviewing or utilization of AI as a in the context of a software developer as a software developer you know it it’s it’s it’s widespread but it’s also not as fast as you would think like for you know a lot of us has been coding one way for a long time and I think that right now it’s broad in the sense that everyone’s probably using chat GPT as like kind of a sounding board and a rubber duck and maybe like doing some limited code generation or limited test generation. But the vision that VC has where you know 80 like all of our sle work is being done all of a sudden our documentation is all complete our test coverage is going to the moon that is actually happening slower because it’s actually non-trivial to use these AI tools you know so they’re they’re they’re still early generation I mean they’re they’re awesome by the way but it takes a decent amount of skill to and and exploration to do it.

So I would say it’s widespread but it’s only deep I feel like in you know smaller pockets right now. I think that’s going to change in a few years and and I think you know I by the way I still meet engineers today who was like ah I can code this it’s not really helping me code. I’m like you’re not using it right you know but that actually is a fairly widespread idea right now.

So I think I think in 5 years they’re going to face trouble because all these you know you know young pups are going to be eating their lunch and producing you know 100 times as much stuff. So widespread but it’s still fairly early. It’s still fairly early and that’s why you can get ahead of things by like starting to learn the tools now their strengths or weaknesses how really to utilize it.

Awesome. Awesome. And thank you so much for that insight.

We’re getting to our last question here and I’m going to get Vivic to close us out in terms of giving or I say actually both of you all. So Vbeck up first but giving the the cohorts the students you know your top advice for them as they’re going on their journey and pursuing you know careers in this field of engineering and technology top advice care more about the work and the people and your compensation and title in your early parts of your career. So interesting work and smart people matters way more.

You said top I have like probably a few. Second is try to optimize for any company that offers in person working than remote. Remote isn’t the greatest way to actually go ahead and like learn in your early early career.

And the other one is, yeah, just like make sure that you’re reliable and, you’re able to take things off from your manager’s plate. Those are like some of the ways that you can really optimize for a very steep slope early in your career, which is going to help. Awesome.

And Tim, you know, I think that we’re all ambitious here as a student group, right? And I think, you know, VC, Bobby, and I are also still ambitious, right? We have goals.

We actually know what to do. Dex told you. I think this is a great idea.

He’s told you what to do. But here, I want to acknowledge something that we have as humans, which is when we go alone, we fail. How many of you here have made a New Year’s resolution before?

How have those gone? Have you succeeded in all of your New Year’s resolutions? Have you abandoned making them entirely because everyone knows they don’t work?

So, it’s kind of weird, right? Like I have a goal. I this is this is this is not the resolution that school forced on me.

My my parents didn’t force me. I had this goal. I want to work at Amazon.

I want to start a company you know and I know what I need to do. I need to do this thing and then what happens you know fear procrastination probably due to fear you know hard to get started right so like we all 100% of us tend to fail when we set an individual goal and pursue it individually so what I’ve always done and you know and I’m the worst man I’m the worst procrastinator on the face of the planet like shout out to the procrastinators out there anyone anyone with me you know I know what I need to do to reach my goal. I just need to take those steps, but I don’t do it right.

And so I think here’s what will not work. Saying to yourself, oh, you you you jerk, do better next week. I’m going to redouble my you know what’s going to happen next week.

The exact same thing’s going to happen. I predict. Well, that’s maybe I’m just telling you more about myself than you all right now, but I’m probably going to fail next week, too.

So you have to change the fundamental way that you’re pursuing your goals. The thing that worked for me is I do really well in force with forcing functions. So for example signing up for a hackathon like are you is anyone here actually going to go and download Kodium or cursor vzero you know I think you want to but then how many of you will actually go do it?

So sign up for a hackathon. It’s a it’s a forcing function and then and then I’m do I’m in the hackathon anyway and I’m like ah well I might as well might as well use that IDE that was talking about give it a try right and do it with a group so like what I did back in the day is I found can I say this word still I found the nerdiest guy in the class I kind of peeped peeped him and I was like okay you you’re going to be my best friend so now every time you study I’m going to study we’re going to study together you know so like that idea of because you know I’ll like I’ll wind this up by saying like, you know, I flake on myself all day long. Like if I I’ll set a goal, I’ll say, I’m going to work out.

I’m going to go study. I’m going to go grind le code and I’ll flake on myself all day long. But if I make a commitment with a friend who also has a similar goal and we set a time and a schedule, I’m very I at least at least for me, I don’t really flake on other people.

So I I I won’t do that. So that’s that’s my advice, my closing advice, which is like if things are not working for you right now, if things are working for you now, great. Ignore everything I just said.

But if things are if you’re not meeting your expectations for your goals, you have to change the way you’re pursuing it. And not just by saying, "Hey, just do better, kid." You know, you have to think about how to set up an environment and a community and a group for you to be successful. Anyway, that’s my that’s my closing piece of advice.

Wow. Wow. That that’s so amazing and and I want to say thank you to both of you again for really enlightening the students, the cohorts here the attendees of the EES 2024.

This is our opening keynote. This is our last day, day three, so I know we’re getting a little bit close to the end, but remember what I talked about at the beginning. You can go and rewatch all of the pre-recorded but also this session here because I heard some gems from Vivic and I definitely want to encourage you all to listen in utilize these tools including AI for you to become efficient and definitely be ready for the next five years because the game is changing for sure.

I definitely want to give you all a round of applause for being here. So inside the chat send a thank you to our guests Vivic and also Tim. Send some love to your peers and definitely those that are here.

Thank you so much for hanging out with us for this last hour or so. But after this session here, you’ll get a feedback survey. Remember how important feedback is, right?

We want to know what we could do better and definitely all the things that we’ve been doing great. So definitely let us know. And yeah, I’m your host, Bobby D.

I’ll check you out in the lounge. I’ll see you in other sessions. Holl at your boy.

It’s Bobby D. Thank you, Bobby.