Category: Rant

Business of Software

Survivorship Bias

XKCD Survivorship Bias CartoonThis week, I was listening to a business podcast and the guest was describing the “techniques for success” that they had used and how they now teach those techniques to others. As I began to wonder how much of this (and all such advice) was attributable to Survivorship Bias and planning this post in my mind, I started to think about what image should go with it. Of course there is a relevant XKCD comic, Randall Munroe thinks of everything first.

I’ve been accused of some Survivorship Bias in the past, too. My ideas around how individuals can become successful programmers through individual learning and not through college is how I got started, but am I establishing too much personal credibility on the fact that I made it through a minefield without stepping on a mine? I didn’t give it too much thought until this week.

If you aren’t sure what Survivorship Bias is, the XKCD above does a great job of explaining it. But, the short “word only” explanation is that Survivorship Bias is a kind of selection bias. In this case, the bias places a great deal of attention and credence on things that made it past some gate or obstacle and ignores the less visible things that didn’t make it past.

One example that I used this week is with “Guaranteed Winners for Monday Night Football”. I don’t know how prevalent this is any more, but it used to be advertised everywhere there was sports content. The basic idea is that you call in and they give you the winner of an upcoming football game, ostensibly so you can bet on it and win. The scam is that they give some portion of the callers one team and one portion of the callers the other team.

One of those teams will win and then those callers, bolstered by their win, will call back. The next time, they pay for a “Guaranteed Pick”. Again, the callers are given both teams fairly evenly. This goes on and on until you have people who have won 10 games in a row thinking that this phone line is worth its weight in gold. They might crow loudly about the powers of prognostication that the picker has. The problem is that this entire scenario forgets about the ever-growing portion of callers who have received a losing pick along the way. That’s Survivorship Bias.

So, is all advice bad? Is all advice a by-product of Survivorship?

As in all things, I think the answer is Yes and No.

In my opinion, the way around this is twofold. First, be very wary of a “lone survivor”. If there is only one voice crying out about a path, look for the 90+% who might have followed the same advice to a bad end. Secondly, if people who follow particular advice seem to always improve and you diligently look and you can’t find a significant population of people that failed with the advice, then it is likely good advice.

The problem is that advice can’t be too specific. Now, I believe that through hard work anyone can improve their circumstances. However, what does that even mean? Not everyone starts on a level playing field, with the same advantages or disadvantages, and from the same starting place. You have to know what success looks like for you.

For example: Yes, you are following the email list building advice, but your email list is 1/1000th the size of the person who sold you that course. Consider the advice and consider the circumstances. First, their market is huge. They are selling business products to wannabe entrepreneurs nationwide (or even worldwide). If you are selling backgammon supplies to the greater Sheboygan, Wisconsin area (nothing against Sheboygan), you aren’t going to be able to reach the same number of people.

As for my personal bias about becoming a self-taught programmer, this also has a caveat. You can only do it if you are the kind of person who will dedicate hours to learning and also to practice. You have to be able to learn from just watching videos or reading rather than interaction. You have to create so many side projects and put so much sample code on GitHub. You have to be diligent to study and work even when you get home from your full-time job and your family is fighting for your time. You have to promise to work after people have gone to bed or before they wake up… something. It is hard to know what to do when no one is telling you what to do.

That leads to the other bias secretly at play. There are the concepts of Self-Serving Bias and Attribution Theory. They are related and the relevant part I want to pick out is that people often attribute positive events to their own character but attribute negative events to external factors (Psychology Today). Likewise, if someone else messes up, it is common to ignore external factors and blame the person’s character or drive or something internal to them. We don’t cut them any slack, but we can cut ourselves all kinds of slack.

What can we do? Just be aware. For certain, tell your story. If you see someone successful, listen to their story. Then, in all things, listen critically. Think critically. Take advice generally and apply it to your specific situation where it fits, throw out what doesn’t. If that doesn’t work, try something else. Rapid iteration and prototyping isn’t just for software, it works for approaches to life’s problems and obstacles, too.

Podcasts

Podcast Episode 41 – Show Update and the Module Dependency Fiasco

DominoesLast time, I was looking for some feedback about how this show should go in the future (and if it should go on at all). You guys responded and I talk about that response and what to expect from me in the future. I also start talking about the left-pad NPM module that “broke the Internet” and a blog post I found about it. Then, I get a little worked up about it and go on a little rant of my own 😉


Links Mentioned in this Show:
Haney – Have We Forgotten How to Program?
Azer – I’ve Just Liberated My Modules
Kik – A Discussion About the Breaking of the Internet

You can also subscribe to the podcast at any of these places:
iTunes Link RSS Feed Listen on Stitcher

Thanks to all the people who listen, and a special thanks to those who have rated me. I really appreciate it.

The episodes have been archived. Click Here to see the archive page.

Podcasts

Podcast Episode 38 – A Stream of Consciousness Rant: The Development Community

A StreamIt has been a little while since I posted a new episode, so when inspiration struck me while waiting in the car, I didn’t pass up the opportunity to record Episode 38. I used my phone as a voice recorder and shared a kind of stream-of-consciousness rant. Some things had really piled up on me recently, not the least of which was my disgust with much of the development community at large. Major targets of my focus include judgement of new developers, judgement of people by their technology of choice, my dislike of “Why I’m Leaving X” posts, and how being yourself doesn’t mean that you need to be a douche when interacting with others.


Links Mentioned in this Show:
21 Management Things I Learned at Imgur
Asciinema.org

You can also subscribe to the podcast at any of these places:
iTunes Link RSS Feed Listen on Stitcher

Thanks to all the people who listen, and a special thanks to those who have rated me. I really appreciate it.

The episodes have been archived. Click Here to see the archive page.

Business of Software

Time to Leave Your Job?

The Future Awaits - from http://us.123rf.com/400wm/400/400/rjmiz/rjmiz0804/rjmiz080400085/2836581-business-slogans-on-a-road-sign-exit.jpg
An article was posted over at Mashable titled 8 Signs It’s Time to Leave Your Job. I can see their perpective on all of their points, but to me their 8th point was the main point.

You are no longer passionate about your work and dread going to the office each day.

That is my number one criteria for looking for new employment. Do I dread going into work? Am I excited about the kinds of things that I get to do at my job? Not every day is a mountaintop, but if you have a prolonged period of dreading work, then you should consider moving on. All of the other points roll into that one. They talk about politics and high profile work, etc. Different strokes for different folks and some people aren’t affected by the same things that affect others. It all comes down to whether or not you die a little inside when the alarm goes off in the morning.

Assuming that you’re getting paid basically what you can make elsewhere (give or take a few percent) and that your benefits are fine, the only other consideration in my mind is whether or not your skills are being allowed to grow, or whether they are stagnating.

For example, if you are a .Net developer and your workplace is still working on the .Net Framework 2.0 and you are only using Web Forms and you are on SQL Server 2000 with no chance of an upgrade, you should think about going even if you are happy. Your alternative is to get very involved in “new technologies” in your spare time so that you don’t become easily expendable.

The days of working somewhere for 40 years and retiring with a gold watch and a pension are basically extinct. If you allow a job to manage your career for you, you will find a time where it is 2013 and you are looking for work and you have been doing FoxPro 6 or Classic ASP for 15 years and now you can’t find a job that will pay you anywhere near what you need to support yourself and possibly a family.

So it is all summed up very simply as to whether or not you like going to work every day. If you love the place and it is feeding your career, then you already know that you shouldn’t be looking anywhere else.

If you love the place and it is strangling your career, consider whether or not you have the time or inclination to put in the work “off hours” in a very big way to gain the requisite experience in “new technologies” that you may or may not use professionally. If you don’t have that time or inclination, look to move on, even if you love your job.

If you don’t love waking up in the morning to go to work, get out. Life is too short for almost anything else to matter.

Rant

How Deep is the Rabbit Hole

Programming is Hard, Let's go Shopping. From http://memegenerator.net/instance/30286782Rob Conery tweeted about a blog post that “Kakubei” made back in May of 2012. You can read about it over here.

I find it interesting that the author starts off the post making a comparison between himself and his brother. That his brother’s academic success and his own academic failings (and subsequent self-taught programming skills) are at odds and that the industry would view one of them as a “real programmer” and the other one as not a “real programmer”.

As I often like to point out, every programmer is a self taught programmer. You may get the fundamentals at college and learn some things that can make you more efficient or a better designer of computer systems, but with the pace that technology is changing, you have to constantly learn to keep up.

That being said, Kakubei’s main complaint is that Rails is hard to learn. He compares the canonical “15 minute blog” demos to what it takes to do “real world” Rails development and he comes up with a big differential.

What I find funny is that he seems to have come from a PHP background. That means that he had to know the PHP language, possibly a PHP framework (like Cake), HTML, CSS, JS, probably MySql, etc. I say that to say this, he had to know a lot to work and he wasn’t coming to Rails from zero.

The author complains that to use Rails, you have to know Ruby and that you have to know MVC, Gems, RVM, Active Record, HomeBrew, HTML, CSS, JavaScript/jQuery, Rake, Capistrano, etc. My point is that he probably already knows HTML, JS, and CSS. If he owns a Mac, he has probably already encountered HomeBrew (it isn’t just for developers). If not, this was a fine time to learn.

I think this all boils down to something else, though. What does it take to be productive or at least “mildly useful” in a language/framework?

To me, I was able to follow a simple Rails tutorial and starting using Rails almost immediately. Know what I didn’t need? RVM. Just starting out, I just used one version of Ruby and Rails. As for Gems, in order to do my tutorial, I had to say “gem install rails”, so I already had an idea how it worked. After that, you can pretty much avoid using it if you don’t mind rewriting code that is already a “solved problem”.

What about HomeBrew? Don’t need it. Active Record? All you need to know to actually make stuff happen is in the “15 minute” video. Same with the MVC pattern. I didn’t know anything about making MVC websites and I was up and moving with minimal instruction from the video. Rake? Didn’t need anything special there, either, because I hadn’t created a very complicated system yet. Capistrano? I need an automated deployment on day one?

My point is that every rabbit hole is very deep. .Net? WebForms or ASP.Net MVC? HTML, CSS, JS/jQuery. Dependency Injection with AutoFac/AutoMapper/Ninject. ORMs like Entity Framework, NHibernate, etc. Don’t forget the language (C# or VB.Net or F# or whatever). How about Nuget for package management? Don’t forget source control, like TFS or Git. What about knowing testing like NUnit, xUnit, MSTest? Deploying with MSBuild and MSDeploy or one of the bajillion open source tools?

I could follow the same path to describe getting started in web development with Node.js or Python or whatever you could think of. It is silly to think that you can go from 0 to Amazing in 15 short minutes. That pattern doesn’t work.

What you have to know is how to learn. You have to know how to Just-In-Time the information so that you can be productive from the jump off. Yes, that means that you’ll always be learning, but if that concept doesn’t excite you, then you are probably in the wrong field.

I believe that that sort of complaining (“The development stack is too deep/rich”) shows a certain level of professional immaturity.

I realize that after Rob tweeted, Kakubei added an update after people started commenting, saying:

“I’ve been using Rails for more than a year and I still use it. In fact, I’m in the process or rewriting our main application’s API in Padrino with ActiveRecord, Thinking Sphinx and deploying to Heroku and it has been a primarily positive experience despite some teething issues. The admin for our database I’ve written in Rails 3.2 and there is a lot to like there. The idea was to write another post after this one titled “Why I love Rails” but I never got around to it. Even though this post is a rant, I still feel that way about Rails, so keep those comments coming, but don’t feel you have to convince me that Rails is good, I just take umbrage with some of its complexities and with the people who sell it as being simple and easy to learn. In the meantime, I’ll keep ranting.”

I understand his response and understand that he was probably frustrated when he wrote the post and he is not really apologizing for it. That leaves me to still consider my criticism valid that I believe that he faced the problem all wrong (or at least described a method that did just that).

To me, you learn the basics of the thing. You see what skills you have that you can already reuse (HTML, CSS, JS, SQL, whatever). Then, you pick up what you need, working with the “training wheels” that the frameworks or languages give you. Then, steadily over time you start to grow and move deeper in the tooling until one day you are someone that others look to for advice.

That’s the only way we’re going to make it in this crazy development world.