Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
Ask HN: Does the valley really have Engineer shortage?
55 points by nojvek on May 5, 2015 | hide | past | favorite | 62 comments
For the past few months I have been interviewing at various big tech firms in the bay area. Ones that start with F, G, U, A.

Have a CS degree, > 5 years experience. Full stack engineer and have a good paying job. I want to interview to basically test the waters and see if something good comes up.

It seems the tech companies will spend calling you, paying for your flights and meals, spending entire days interviewing you but at the end say "sorry we will not go ahead with you, we don't give feedback"

May be the more time you spend writing code on an editor, the worse you get at writing code on a whiteboard.

I can't quite figure it out. Anyone else feel the same. May be the valley has shortage of "cheap" engineers.



I was interviewing at Google, I don't know about the other companies. Paying for good engineers was never a problem. Finding engineers with good algorithmic skills is hard. So maybe the problem was not with your coding skills, but your algorithmic skills. That's what an interviewer is looking for, it's not important if you make some syntax mistakes. Anyways it's better to prepare without an IDE and using a simple editor (sublime, vim, emacs...). At Google anyways there are internal languages without IDE support, so you'll need this skill. If you really want to work at Google, it's best to study some algorithmic books and write lots of testing code. My favorite site was UVA online judge: http://uva.onlinejudge.org/

Solve half of the problems there (500 from 1000). It will take couple of months, but you learn a lot from the experience and get to understand what the interviewers look for.


Finding engineers with good algorithmic skills could be hard. But it's not the point. There are tons of undergraduate students in developing countries working hard on those puzzles day and night. Some of them can even squash a whole program into a single function with more than 1000 lines and still be able to make it bug free and quite micro-optimized. The really rare charms, in my opinion, are those who are able to design. That is, making good choices among the trade-offs. I don't think Chris Lattner, Rob Pike and Linus Torvalds would shine at the quickness of puzzle solving or inventing the fastest algorithms. I even doubt if Don Knuth can do better than you on UVA online judge.


Exactly. The most valuable skill for a working programmer is designing good abstractions. The ability to cleanly break up a complex problem up into manageable parts is more valuable in day-to-day coding work than any of the skills companies usually screen for. I'll concede there are probably some roles for which algorithmic chops are paramount but they're absolutely the exception.


Actually from the design and speed of git, I think Linus knows a lot about data structures to be good at ACM. Don Knuth: the same, he goes to instruction level speed in his books..is that really general design? Did you really read his books actually?

Rob Pike: sorry, but he was the ,,distinguish engineer'' guy making our lives hard with his half-baked languages that were full of as many inconsistencies as possible. I never liked the Pike language and maybe go is a lot better, but still it doesn't even have a generic type system.

Chris Lattner: I would rather bet on him than Rob Pike. Putting together a modern language with modern infrastructure is hard. Putting together a half-baked language is easy (especially if you have engineers working under your management).

Still what's more important: if you understand the speed of basic data structures, you can reason about performance of more specialized data structures (like memory management in the linux kernel) as well.


Hope you put your personal preferences in the most far corner if your brain, when interview candidates. If you wouldn't hire Rob Pike because of personal preferences it would be one of the biggest mistakes of interviewer. I don't like lack of generics in Golang, but it's too ridiculous reason to think Rob Pike is distinguish engineer or not talented enough.


I tried to be as factual as possible when writing my interview feedback. I wasn't on the interview committee, so you can rest assured that I wasn't in deciding position :)


According to their website, only 4 people have achieved 500 problems solved. That seems like a pretty high bar to set.


The bar for resistance to boring tasks? Probably that would really be a good indicator for a valuable employee who would work on whatever you throw at him.

Don't get me wrong, I love that kind of puzzles, but only for a while.

They don't seem to be too hard either. I randomly found this: http://uva.onlinejudge.org/index.php?option=com_onlinejudge&... which is literally just translating the given algorithm into a programming language of your choice and adding a counter.


It's true..maybe 100 is enough. Still, it's an awesome tool.


Where did you get that information from? It looks like there are actually hundreds of people who have solved over 500 problems[1].

Which is not to say it is easy, but putting it at only 4 successes makes it much more intimidating.

http://uva.onlinejudge.org/index.php?option=com_onlinejudge&...


I clicked "Site Statistics" -> "Overall rankings"

http://uva.onlinejudge.org/index.php?option=com_onlinejudge&...


I used to be a Director in Engineering team at Box. I now run a unique bootcamp in the valley (http://InterviewKickstart.com), where we train engineers to do better at technical interviews. In my perspective, a combination of following is happening:

0. The companies the OP interviewed at, can afford to not give a benefit of doubt. If they hire a few less engineers, their revenues are not going to see a significant dip. Not true of startups. e.g. If an e-commerce startup misses a holiday season, they are done forever. They'd hence be more forgiving.

1. Valley thinks it is a meritocracy, but it is not. Valley is full of biases when it comes to engineering talent. (It's true for all meritocratic systems). They want certain kinds of engineers.

2. Regardless of the hiring market, the companies OP interviewed at, are not forgiving in their interview processes. e.g. use of substring() instead of passing indices in a string recursion will kill an interview with most interviewers. So will use of pow() in atoi(). And you won't know ever.

3. Interviewing is not a test. Interviewing is closer to a date, than it's to a test. There are no "failures" on a date. It's about whether that relationship clicks. "Hiring market" has little bearing on a date.

4. Even if the interviewing panel and hiring committee knows that there is a shortage of engineers, they cannot knowingly give a benefit of doubt to a candidate, or that starts to affect morale in the company.

5. They cannot afford to give a lot of time in interviewing. e.g. they cannot spend a week working with you and knowing you better. If they do that, they will never be able to hire enough engineers. They need them in hundreds a month. They hence have to have an interview process that's short. And short interview processes are bound to be imperfect.


So if you need an Algorithm you look one up in Kunth or do some research - or for a lot of cases do what we did when I worked at cutting edge RnD buy it in from people like NAG.

Recently I needed to develop tool to optimse some PPC I did not know the Algorithms But i knew how to research and building the tool took me less than 2 days to implement.


It's absolutely stupid to ask candidate to work in uncomfortable environment (without favorite editor/IDE) and expect to see full potential of this developer. Even if Google have some internal langs, even if this particular developer will have to work with them time to time, developer will spend some time to get used to it and become comfortable (more or less), not like in the interview.

For more absurd, ask them to dictate their code.

All my anger here is not pointed to you as interviewer, although I see some try to defend this style of interviewing with argument about "internal langs". Google invest a lot into IDE support of their home-made langs (Go, Dart), so I doubt it's a valid point.


It's not about making an uncomfortable environment; it's a recognition that a program is 90% written before you should ever fire up your editor.

What is meant by "full potential" here? You are not expected to write a hundred lines of code, or get something working end-to-end. The goal is to see if you can see to the core of something more complex than a for loop. If you can, you ought to be able to explain that to a colleague in a person-to-person manner, including some syntactically-passable code. Believe it or not, a whiteboard is pretty much to most efficient way to do that.


Actually my first question as an interviewer was always a for loop. Generally reversing a string in place. For some people it took the whole 45 minutes of the interview to get it done. They felt accomplished at the end. I had to wait through it with straight face...I couldn't tell them that it was only a warm-up question, because I was not allowed to say anything that might hurt their feelings. My task was to gather data about the person.


You understand that those kinds of questions are so well known? If someone provides you solution immediately then he/she probably learned it before, as the other 100 typical questions. So you basically testing memory instead of programming/algorithmic skills.

Also I think that you asking for working code, not a general description of the algorithm, right?


This person never said that passing this first question was in any way a positive indicator of skill, though. For better or worse, when you have a lot of people coming in, sometimes you need to throw a negative filter in front. Something which you can say "If they can't do this, our conversation is over, and I can stop wasting time right away."

As much as I've been frustrated by someone asking me to write FizzBuzz, I greatly sympathize with the people who end up asking it or some variant of a very, very simple coding problem. I hope that at some point, there really will be a better way.


Maybe they think "it can't be so simple question, it's GOOGLE, maybe they ask for the most fast and efficient algorithm of existing..."


You can ask how efficient it needs to be. The interview is always a 2-way conversation. For example you can tell that you can check whether the string size is divisible by 4/8 and use built-in CPU instructions that convert between big-endian and little-endian ordering. Still, generally what's more important for me to see that the person who is interviewed checks all the edge cases (sees the difference between odd and even length strings, and makes sure that trivial cases, like 0 or 1 byte/character long string work as well)


I would be wary about that too, but I would rather answer with the first algorithm that comes to my mind, without worrying about efficiency yet, and if the interviewer ask me for something more efficient, well, then I would start digging.


Couldn't such a simple question be asked on a phone screen ?


If them the rules of the game then play the game. If you are interviewing at these places then spend time practicing answering questions on whiteboards. If companies are going to use proxys for identifying programming talent then just practice for whatever proxy is being tested. Yes it is silly, but until you are in a position to change anything your only choice is to play to win or not play at all.


No, there is another option: work with companies without silly rules of recruiting. And benefits sometimes are even better:)


That is the don't play option.


That's why he said syntax errors don't really matter.


Tried it just now. Site is woefully slow, which is ok to just load questions and submit. Problem is only supporting C/C++, Java, and Pascal. I'm trying out making Scala my 'go to' language.


The only problem — you must provide working solution, not algorithm but working program. That's a completely different thing.


> simple editor (sublime, vim, emacs...)

> simple editor

> simple

> vim

> emacs


Maybe by "simple" he only means "without IntelliSense" :)


> without IntelliSense

> vim


500 from 1000. Ha ha.

I solved more than 100 of them, and there's absolutely no way it will take "couple of months" to solve 500, unless you're one of the best programmers on the planet and are working on solving them full time.


Sorry, I wanted to write that I was in the interviewer's position at Google.


I looks like they rejected you after interview so you didn't even go to the part where you receive an offer and reject it for being too low so the part about "cheap" engineers is just a cheap shot. Pardon the pun.

The companies you did interview at do have a high standard for hiring but generally speaking at the very least you should have an idea how well you did during the interview without them spelling it out for you.

If you couldn't answer the questions or write the code they asked you to write then that's why they didn't hire you.

If you think that you did brilliantly then maybe you're not coming across as someone the interviewer would like to work with.


> I looks like they rejected you after interview so you didn't even go to the part where you receive an offer and reject it for being too low so the part about "cheap" engineers is just a cheap shot.

Even more so, if they were worried about cheap engineers, I assure you that they wouldn't be spending money to fly you around the country, buying you flights and meals (and often drinks and hotels and taxi/uber rides and sending you away with swag). The companies you listed pay the best amongst those in SV. I don't think price was the issue


Not sure if that logic holds. Seems like a viable strategy to show your best side while trying to lure a candidate. Once they are hired, they probably won't be showered with swag and limo rides anymore. In fact I would consider it a warning sign if too much fancy stuff is going on before the interview (even though I know Joel from FogCreek does it, whom I generally like).


I kind of agree with you. They don't provide any interview feedback so there's something key I am failing that I don't realize.


Re: your last line, almost any situation described as a labor shortage is actually just a complaint about the price of said labor. Offer a $10 million/year salary, and see how easy it is to get the engineers you're keen on. "That would be a ridiculously high salary", you say? Great, you've established my point; now we're just haggling...


It's true that there's rarely a shortage of anything if you're willing to pay arbitrarily high prices for it, but that's not really what the word means. That like saying there isn't a water shortage in California as long as anyone, anywhere in the world is selling bottled water at any price - that's obviously absurd, and no, it's not just haggling.

If you can only hire qualified software engineers at $250k, then there is reasonably some sort of shortage. That's not the same as suggesting that very rich firms shouldn't pay up (and they are), but let's not rob perfectly good words of their meaning just to make a point.


There are many high paid careers that earn that much, but they never get the amount of publicity and discussion about shortage than engineers do.

It's artificially generated.


So you haven't heard about the teacher shortage, the nurse shortage or the doctor shortage? They usually get way more publicity than the engineering shortage.


Oh wow, I really disagree. The engineer shortage gets a tremendous amount of press.


I tend to agree with you - if you define shortage as "unavailable at any price", then you have essentially written the word out of the English language. So obviously when people say "shortage" they mean something else.

I don't really agree with you about the $250k, depending on what you mean by qualified. Keep in mind, the median salary for a lawyer in silicon valley is just shy of 200k a year (check US News Best Jobs for salary info). For a physician, it's typically much higher.

Those are tough jobs. But does your definition of "qualified" include a graduate degree from an elite school? Keep in mind, the majors required even to apply have high attrition rates and lower GPAs than most other majors on campus. To get into an elite graduate engineering or CS school, you still need to get very high standardized test scores and have a strong GPA, and elite engineering or science programs typically have vastly higher attrition rates than law or med school (no joke, elite PhD programs have attrition rates between 35%-50%, whereas elite MD or JD programs have attrition rates below 0.5% (yes, one half of one percent). It's harder to get info on MS program attrition rates, the only one I found was behind a paywall and the summary didn't distinguish between elite and non-elite programs, but MS attrition rates overall were about 30%. MS programs are a bit shorter than Law school, for instance, but I consider the coursework to be considerably more rigorous (I actually did attend an elite law and an elite graduate engineering program, mine is only one data point, but I'd say law is like running 70 minutes at a 7mm, whereas engineering BS+MS is like running 50 minutes at a sub 6 mm. The first is pretty tough no doubt. But while the second is shorter, it is probably harder).

Then add in the really difficult interview process at elite software companies (to me, this is like taking a subset of the bar exam every time you change jobs), and overall?

Honestly, 250K is probably about where mid-career salaries would need to be for graduate degrees from strong programs in engineering to be competitive with the options available to students academically talented enough to do it.


I agree with you. A claim that "there is a shortage of X" typically amounts to a claim that "X is too expensive", albeit with slightly different connotations. This is the point I was making.


My sense is that the Valley (and other little Valleys) has a serious shortage of the same 100 engineers.

With HN as the only sample, companies don't seem to be very interested right now in taking good engineers and molding them into great ones, or in building a business structure that can benefit from good engineers.

Instead, they're all competing for the top 1% of the top 1% of the top 1% -- and complaining that there aren't enough of those to go around, so the visa program needs to be restructured to draw that talent from the rest of the world.

shrug Honestly, I think the best thing you can do in that kind of environment is focus on looking outside the Valley, or do some freelancing if you think that would be a good fit, or start your own business if you think you can and maybe see about hiring the good talent that nobody else wants right now.


> With HN as the only sample, companies don't seem to be very interested right now in taking good engineers and molding them into great ones, or in building a business structure that can benefit from good engineers.

There's a few reasons why this is. The first is that startups are often, if not always, extremely bandwidth limited. It's hard to spend time doing the training, allowing a person to experiment, and giving the freedom and mentorship that is needed to take a good engineer to a great one, or even a mediocre engineer into a good one, when you have only a limited amount of time and limited amount of funds to do so with. Molding engineers is a very long term investment, which suits companies like Google well, but doesn't fit as well into much of the rest of SV.

The second is that even where you want to do such molding, you need the base set of great engineers to help with that. Harder still is finding the set of great engineers you know how to teach and train, and are interested in doing so. While there are some people who enjoy teaching as well as engineering, a lot of people in engineering don't like to or want to spend their time training someone else.


That all makes fine sense, but it isn't without its disadvantages. Those top engineers come with a price tag, and the harder everyone competes for them, the more they realize they're worth. For startups working on particularly intractable problems, the cost is easier to justify, if you have enough money available.

But -- again, with HN as my only sample these days -- this hiring strategy seems to be spreading beyond the relatively small number of startups working on technically challenging problems.


I personally avoid this hiring strategy and hire people I can mentor and build up. Companies that don't want to do this do our society a disservice by only hiring people who already are at the position they want (or less). In addition, it signals to me that they are not likely to promote from within or compensate fairly as time goes on.

I have no interest in working for a company not willing to do this.


There's a lot of competition for the big-name companies. Think of it like applying to an Ivy League school. They can afford to be selective and completely arbitrary. It's the nature of competition. You can either a) learn to play the game better than everybody else or b) decide it's not a worthwhile game and select a different path.


Especially for more experienced positions, strong written communication is often "table stakes," positioning you as a competent engineer who can work well with others.

One way to start: using "maybe" instead of "may be", "a shortage" instead of "shortage" and "Valley" instead of "valley."


I would be shocked if a talented engineer didn't get an offer because he didn't capitalize Valley in his cover letter.


So you've interviewed for the ones that have engineers from all over the world begging for a job and who's recruitment process suffers from the issues that come with scale, and from that you draw conclusions about the entire industry?

Also, you really think you're that good an actor interviewers can't figure out you're just testing the waters and aren't really motivated?

Since you also don't have a clue as to why you've been rejected, I suggest you're not very good picking up communication signals that aren't very explicit and verbal. Hell, that might actually be the reason why you get rejected in some cases.


I'm not in the valley, but I have the same impression: companies are not overly desperate for developers. I'm also over 40, perhaps they are only desperate for developers fresh out of school who are still passionate.


It's not actually a shortage of engineers. I can post a job and have an endless stream of interviewees in the door. There is a shortage of engineers with the requisite amount of experience and the right skill set for many jobs.

This holds especially true with startups that don't have the time or money to spend 6+ months getting you up to speed from zero on whatever they're working on. Hiring people based mostly on intelligence (what big firms like Google try to do) works when you have the capital to spend 6 months training them towards being productive.


i've found attitude is sometimes different when "testing the waters" vs "looking for the next gig". this can lead to your situation in certain instances.


Could be due to https://news.ycombinator.com/item?id=9180229 (depressed at my job). Interviewers can catch on if you're unhappy and it can also effect your interview performance.

Ironic, trying to leave a job because you're blue makes it harder to get another.


You are right. I am blaming too much on external circumstances. Will work on it.


I know a few examples how major companies have rejected a very smart experienced developers. Also a friend of mine was interviewed in a few major companies and every single interview was extremely stupid (by his words) because they didn't ask any question about his real experience but instead spammed with typical tasks from glassdoor.


Are you interested in startups at all? We're looking for a full stack engineer.


See if you like my work at https://nojvek.com (contact at bottom)


What starts with U?


Uber.

Stumped me at first, too...not used to seeing it in this list but I guess I should get used to it :)




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: