All software engineers can program, but not all programmers can engineer software
jsComplete.com/pro-programmer
Update: This article is now part of my book ?The Professional Programmer?.
Read the updated version of this content, and more programming advice at jscomplete.com/pro-programmer.
Some people don?t like the term Software Engineer because of the engineering metaphor. This article is not about that term. If you don?t like it you can substitute it with Software Author, Software Craftsperson, or Software Artist!
By Software Engineer, I mean a person who looks at writing quality software as their profession. A person who applies science and statistics to that profession and does not look at it as just a job that earns money.
Knowing how to program does not make you a software engineer.
Anyone can learn to program. It?s easy. Anyone can create simple programs that work for them on their machines but that would not guarantee that the same programs will work for others.
My favorite analogy about this is that everyone can sing and entertain themselves in the shower, but when it?s party time you do not play recordings of yourself singing. You go with the pros.
More analogies? Sure:
- We learned Math and Writing in school but that did not make us Mathematicians and Authors.
- Most of us can easily learn to cook but when it?s time to feed a lot of people we hire a Chef.
- You do not call the neighborhood handyman to build a house from the ground up.
The main message I want to share in this article is that simple programs are much different than engineered programs.
The act of programming, in its simplest definition, is giving computers instructions to do something with some input in order to produce some output.
The act of engineering software is about designing, writing, testing, and maintaining computer programs with the purpose of solving problems for many users. It is about creating robust and safe solutions that will withstand the test of time and will work for some of the unknown problems around the original obvious ones.
Software engineers understand everything about the problems they solve, the solutions they provide, the limitations of those solutions, their privacy implications, and their security implications.
If someone does not understand the problem, they should not be allowed to program a solution for it.
The Solution Mentality
Software engineers do not think of their career as just writing programs. They think in terms of satisfying needs and solving problems. This is important because not every problem needs a program. Some problems can be solved by existing programs or by putting together multiple programs. Some problems can be totally prevented by acting early. Designing good programs often involves planning to prevent future problems.
?Intellectuals solve problems, geniuses prevent them. ?
? Albert Einstein
Complicated problems usually require writing multiple programs. Some problems need programs that run in parallel while others need the programs to run sequentially. Some problems can be solved by educating users.
Before writing a program, a software engineer asks the questions:
- What problems am I trying to solve?
- What else besides writing code can be done to solve them?
- What can I do to make these problems easier to solve with code?
Code Quality
Great programs are clear and readable, they can be easily extended, they work great with other programs, and maintaining them is not a nightmare. The quality of the code is not a negotiable thing, using sloppy shortcuts because of a deadline or emotion is never acceptable.
One of the most important aspects of engineering software is to design anything from the ground up ready for extendibility. Modifying software is a fact of life. Users will demand more features and easier ways to use software.
A piece of software is usually not very useful on its own. Useful software features start when multiple pieces of software communicate with each other, exchange their data, and collaborate on the task of presenting data and interfaces to users.
Programs have to be designed with that in mind. What messages do they accept? What events are monitored? What messages are emitted? How do we authenticate and authorize communications?
Another important aspect of great programs is the clarity of the code, not how many tests there are or the number on the test coverage report. It is the simple question of is this code readable to someone else? Or better, would I, the writer of code today, understand this code a few weeks from now?
?There are only two hard things in Computer Science: cache invalidation and naming things.?
? Phil Karlton
Code readability matters a lot more than you think. Unfortunately, there are no good metrics for code clarity. Memorizing good software patterns and practices might help but are often not enough. Good software engineers just develop an eye for code clarity with experience and intuition. The writing metaphor here is perfect: just knowing a big list of words will not help you write concise and clear content.
?I didn?t have time to write a short letter, so I wrote a long one instead.?
? Mark Twain
Things will go wrong with programs. Being able to easily fix them when they do is a key attribute of good software. Errors happening in programs should have clear messages and be logged centrally somewhere to be monitored. When a new error is reported, the person who needs to fix it should be able to debug that error. They should be able to hook into the system and read information about the execution context at any point in time. They should be able to easily verify expectations about any part of the system.
Environments and Testing
When software engineers write programs, they make sure their programs will work in many different environments, on differently-resourced machines, and at different time zones. The software needs to work on many different screen sizes and orientations. It also needs to handle being forced to use limited memory or processing power.
When creating software for a web browser, for example, it needs to work in all the different major browsers. When creating desktop software, it needs to work for Mac and Windows users in most cases. When creating applications that depend on data, the software needs to work for the case when the connection to retrieve that data is slow or completely off for a while.
To write a piece of software, software engineers try to think of every possible scenario they can imagine and they plan to test these scenarios. This starts with what they call the happy path where nothing unexpected happens but more importantly they document every issue that is likely to happen and plan a test for that. Some software engineers start by writing code, which they call test cases, that simulate these scenarios. They then write the desired code that passes all these test cases.
Software engineers understand software requirements which are usually ambiguous and incomplete. The unique skill of a talented software engineer is not about how to write the solution but rather about identifying what should go in the solution.
Cost and Efficiency
Software engineers can solve problems fast in most cases. If you think that hiring experienced programmers means higher costs, think again. The more experienced the programmer you hire is, the faster they can provide robust, accurate, reliable, and maintainable solutions. This means lower costs overall in the long term.
You need to also consider the cost of running the program. Every program will use computer resources and those do not come free. Software engineers will write efficient programs that do not use computer resources unnecessarily. For example, caching frequently-used data is one strategy that applies here, but it is only one of maybe thousands of tools and variations that can make a program faster and more efficient.
A beginner programmer might give you a cheap solution, but running that solution might end up costing you and your clients a lot more than if you had an experienced programmer create an efficient solution in the first place.
Usability
Good programs are designed with the User Experience (UX) in mind. Human-computer interaction is a big topic with countless research studies and findings. The more these findings are embraced, the better the software would be.
Let me give a few examples here just for you to get a taste of this big domain:
- When designing input forms where users are expected to enter data, such as, their email address, a good receiver program would ignore the letter case used for the email address. It would also trim any extra spaces around it. Do not give the user a hard time because their CAPSLOCK key is on, an email is unique in its lowercase format. If the program is accepting new email addresses, validate that early to give the user a clear message that they probably used the wrong address. This includes obvious validation problems like not having an @ sign but it should also include the not so obvious validation problems like using a misspelled ?gmail.ocm.?
- When redirecting a user to do something, a good program would remember their original location and redirect them back to that location when they are done. A good program would also remember any already-defined data and interactions that need to be associated with future steps the user is asked to do. For example, let?s say you have been searching for flights as a guest on Expedia. You then decided to create an account. All your previous search would be saved into the new account and you could access them from entirely different machines.
- A good program is designed with user scenarios in mind. Put yourself in your users? shoes. Don?t just add features! The other day I booked a United flight forgetting to include my frequent flyer number. After I got the confirmation, I went to the United website to add my FF# to the flight and it took me a good TEN minutes to figure that out. There was no obvious path so I had to explore all links that could lead to that feature. I visited the page where the feature was available and I could not see it the first time because it was buried deep in a big form. It turned out that I had to edit traveler information, scroll past about 20 input elements on that form, select the type of FF# I wanted to use, and also enter the required phone number to make the whole form submit. This is an example of a program that was not designed by thinking from the point of view of the user.
Reliability, Security, and Safety
These are probably the most important points that set software professionals apart from the amateurs. They know they are responsible for writing safe and secure solutions.
A piece of software has to be resilient to bad input, bad states, and bad interactions. This is VERY hard to accomplish and it is the main reason why we hear stories about people dying because of software mistakes.
Users are going to use the software with bad or wrong input. Some will do that intentionally to try to break software and hack into resources represented by that software. The person who was allegedly responsible for the recent Equifax fiasco was accused of not doing their job, which is to engineer resiliency to bad and malicious input in all software that is publicly exposed.
The security story is also not only about bad and malicious input but sometimes normal input as well. If users forget their passwords, how many times can they be allowed to try? Do you lock them out after? What if someone else is trying to get them locked out? Do you allow your users to submit their password over a not-encrypted connection? What if an attempt to login to an account came from an unusual place? What do you do if the login seemed automated?
What do you do to protect your users from cross-site scripting and request forgery, man in the middle attacks, and simple social phishing? Do you have a backup strategy if you get a DDoS attack on your servers? These questions are just to name a few of the many concerns to be planned for.
Secure programs do not store sensitive information as clear text but rather as one-way encrypted data with very-hard-to-break algorithms. This is a backup strategy in case the program and data get compromised. Hackers would find encrypted data that is mostly useless to them.
The software will go into bad states and will need to be corrected. Unexpected problems will occur to the best of programs. If you are not aware of that and you are not planning for that, you are not a software professional, you are just a writer of unsafe programs.
Software defects are invisible. Our intellectual ability to predict and prevent known defects are limited. This is why software engineers understand the value of good tools that can help them write correct and safe software.
Embracing Tools
There is no doubt that we need more and better tools. Tools make a big difference and they are often under-appreciated.
Imagine if we still need to FTP files to deploy! Imagine debugging network and performance problems without Chrome DevTools! Imagine how inefficient it would be today to write JavaScript without ESLint and Prettier!
If you are a JavaScript developer and, for some reason, you are forced to pick only one plugin for your code editor, you should pick ESLint.
Any tool that shortens the feedback loop while you write code should be a welcomed addition. Bret Victor?s argument about inventing immediate visual representations to what we create was an eye-opener for me. Embracing and improving tools is one way to get us to that bright future. Go watch Bret?s talk right now if you have not seen it before.
When I find a great new tool, my only regret is not using that tool earlier. Better tools will help you be a better programmer. Find them, use them, appreciate them and, if you can, improve them.
The choice of language matters. Type-safety matters. The best thing that has happened to JavaScript is TypeScript (and Flow). Code static analysis is a bigger deal than you think. If you are not doing it you are basically making yourself vulnerable to future unknowns. Do not code without a static typing system. If your language of choice does not have static typing, either change languages or find a transpiler for it. Transpilers today are smart enough to work by just reading comments in code, which I think is the future of type-checking for languages that do not support it natively.
The Evolution of Software Engineering
No one can learn software engineering in two months, or six, or even a year. You do not learn to be a software engineer in a bootcamp. I have been learning for the past 20+ years and I am still learning today. I became confident enough to call myself an experienced programmer only after about a decade of learning and after designing, building, and maintaining applications that are used by thousands of users.
Software engineering is not for everyone, but everyone should learn to solve their own problems with computers. If you can learn to write simple programs you should. If you can learn to use generic software services you should. If you can learn to use open-source software you will have a lot of power.
Problems evolve and so should software engineering. The future of this profession is to enable regular computer users to use their computers without needing to study five years to do so. Enable users to solve the easy problems on their own with easy-to-use tools. Software engineers would then move on to create better tools, solve bigger known problems, and do their best to prevent unknown ones.
Thanks for reading.
samerbuna.com
