You are going to commission a large software project, but you have little experience doing so. How can you ensure success?
Table of Contents
- What qualifies me to pontificate on this topic?
- What must you do before you begin?
- How can you ensure that you get what you want?
- Feeling lost? Find a technical advisor
- What are the best technologies, languages, or architectures?
- How do you find the right development team?
- How do you know you are paying the right price?
- What should you do to manage the project?
- You want to add a feature, but your team says no. Why?
- These developers are too slow. Can’t they go faster?
- How can you protect your software investment?
- What about open sourcing the software?
- How can you ensure security?
- How should you maintain the software over time?
I have been participating in and leading software development projects for two decades. I’ve worked on large software projects for Kaiser Permanente, Sprint-Nextel, XM Satellite, The US Army, and many more organizations. I’ve worked on high speed trading, embedded systems, robotics, cybersecurity, financial fraud, consumer facing auctions, financial reporting, data warehouses, cloud, on-prem, and many other projects. I’ve owned companies executing on millions of dollars worth of software projects. I’ve personally created software that has generated hundreds of millions of dollars of revenue for others. I’ve seen well funded projects fail, and underfunded projects blast past all expectations. I’ve been the villain. I’ve been the hero. This is my confession.
Before you begin, you must ensure that you are on a path towards creating value for yourself and your users. Valuable software creates meaningful outcomes for both you and your target users. Your users will not continue to use the software if it does not create value for them. You will not sustain the software if you reap too few benefits from the time and money you spend creating it.
All software is built on top or is derived from existing software. There is likely no 100% buy solution for your specific needs. But before you begin building your own software, you should thoroughly research your options to buy and customize software.
Buying software pushes risks away from you and onto the vendor from which you are purchasing. This is tremendously valuable, and should not be overlooked. If you have the option to purchase software and then customize it - you should probably take that deal. Your greatest risk in buying software is that the software can not perform as advertised, or is not customizable to suit your needs. You can mitigate this risk by negotiating a delayed payment for licensing until after the major customization risks are overcome. That delayed licensing essentially provides you a time-loss only risk to achieve your goals - and you should thoroughly consider it.
If you have thoroughly reviewed the market, and find that there is not suitable “buy option”, then building software may be your only option. Building software is not cheap. It is a very risky endeavour - and there is a non-trivial chance you will not succeed. This guide is intended to help you overcome the risks of building software.
Meaningful software reflects the world around us, and blends into our lives. The very best software is invisible in our memory because it is not the focus of our attention, but the tool by which we accomplish our goals. While your software will consume most of your attention, it will likely not be the focus of your users. Remember that your users must derive value from your software - and they define what that value is. You are simply on a journey to discover what your users define as valuable.
Pause for a moment and consider what you did last week. Did you catch up with friends? Did you go to a movie? Did see your family? The things you remember are what you accomplished - and software was probably involved in how you accomplished them. You might have ordered the movie tickets online, and used a ride hailing app go to the theatre. You might have used a video chat to stay in touch with your family.
Consider your software in the context of your user’s lives. Write down your answers to the following questions.
What are your user’s goals? (Not your goals)
How does your software help your users accomplish their goals?
What did your user do before and after engaging with your software?
What does your user need from your software? (Note: there is a difference between what they need, and what you want them to do)
Create “pencil on napkin” screen flows and wireframes of how your software solves the problems above.
If you feel lost, engage a User Experience (UX) expert. They will help guide you through the context and initial ideation of your software journey.
You should have specific measurable goals from the software. An example measurable goal might be “we will receive $X of revenue in Y time”. If you can’t easily describe those goals, spend more time exploring what you want. Do you want money? Do you want to save your users (employees) time? Do you want measurements from tests? Write these goals down and measure your success by those metrics.
Do not begin your journey until you can define your measurable goals.
Software is expensive. You will expend significant amounts of time and money on your software commissioning journey. Be absolutely certain that you should be on this journey. Define your resources in terms of money and time. You should commit at least one if not two full time people from your business team to support requests from the developers. You will not get what you want from the developers, unless you support them - and it is a full time task.
Have your assumptions checked by your technical advisor and software development teams (see below). At a minimum double all of your estimates (see below for why). For safety, triple your estimates. Even experienced software developers dramatically underestimate the amount of time they’ve spent on previous projects.
Do not begin your journey until your entire team agrees you have the resources to be successful.
You are probably imagining a broad panacea of solving many problems. Do not follow that instinct. Instead, identify one single very small problem to solve. Solve that single problem - maybe even without creating new software. Did that produce your desired results? What did you learn? Then take another small step. And continue until you have reached your larger goals.
Another way to say this is “Deliver early and often”. Waiting until the end of a very long software project to discover whether you’ve created something of value is a recipe for disaster. Your first attempts at creating value will fail. The more often you test your assumptions (that you are creating value), the more likely you are to create something valuable.
This business concept is referred to as the Lean Startup method. Understanding the Lean Startup method is a valuable use of your time. This concept also works well in the ever popular Agile development method of short specific sprints. You will work with your team on the development, testing, and deployment cycles - see more below.
In your brain is a dream of software that does not yet exist. Your task is to usher that dream out of your head and into the brains of the people who must collaborate as a team to turn your dream into a reality. Software creation is the birth of a dream. And like having a baby - this will not be easy, but the results can be quite rewarding.
The single best thing you can do to ensure you get what you want, is to draw or have created high quality visual mockups of your ideas. When you create visual representations of your ideas, you are forced to understand them more deeply. Your visual representations will convey far more detail and context than words or spreadsheets.
Are you not a visually creative person? Don’t worry, there are many people and tools to help you. Find User Interface (UI) experts that can help you generate high quality mockups from your UX wireframes.
Spreadsheets and documents describing features are not sufficient. You may believe your words to be accurate, but you are not in control of the context those words have in the brains of your development team. Saying “I need a pie chart that shows my sales by vendor” - seems straight forward. But dive deeper. Is it an exploded pie chart? Is it above or below the related information? Is it used by the administrator or the end user? Providing written context on every single feature of your ideas will be exhausting for you, and for the development team. Using only long documents, you will waste many hours simply setting context, and reworking failed concepts.
Find a 3rd party hands-on-keyboard experienced advisor to be your coach. You should pay them for their time and attention - they can’t do this as a volunteer. Involve them in the team selection process, and throughout the project cycle. You will have many questions. It is important your technical advisor be an actively hands-on-keyboard experienced software developer, and not just someone who previously commissioned software. This person will have held titles like “VP of Engineering”, or “Software Development Manager”. The title “CTO” is often not indicative of hands-on-keyboard experience, and will require further proof of experience. Non-technical startup founders, ex-CEOs, marketing consultants, and business strategists are great for other advisory roles - but not as your technical advisor.
Be sure that your technical advisor is fully aware of your goals. This will require many conversations. This person should be financially incentivised to be on your team.
A good technical advisor recognizes that they are not in control of the technology or the business. But they can raise their hands when they perceive danger. When your advisor and foreman (see below) are in disagreement about technical aspects you don’t comprehend, ask your advisor to clearly state the business impact a technical decision will have on your company. A good advisor will do so in a way that you can comprehend and ultimately make your own decision.
Remember, technology selections should be left up to your foreman and their team. Your technical advisor only holds a veto card that should be used rarely.
Stop. Think about that question. Imagine you hired a plumber to install a dishwasher. You’ll likely define the brand of appliance that you want - but you’d never specify the brand of the wrench he’ll use to install it. The same goes for software. Rather than focusing on technologies, you should focus on engaging and retaining a team you’ll need to trust for the long term.
Do not chase buzzwords. There is no block chain solution for you. Big data will not give you unknown insights. AI will not solve all of your business execution problems. You will not create value by chasing buzz words.
Buzz words exist because technology vendors make echo chambers at conferences and in marketing materials. Smaller vendors are forced to conform the marketing of their products and services to fit the latest buzz words - so that they can remain relevant from a marketing perspective. The echo chamber grows louder until every vendor’s message appears the same as every other vendor (even when their offerings are very different) - and they are collectively forced to find a new differentiating buzz word.
Instead of chasing buzzwords, focus on your business processes and allow your technical teams to select appropriately measured technologies to achieve your goals. Perhaps your employees are dissatisfied performing repetitive mundane tasks - AI might be able to help with employee retention by removing mundane categorization tasks. Or your marketing team needs to be able to measure ROI more accurately to fulfill contractual obligations - a Big Data solution might be able to help.
Side Note: There is no commercial (outside of crypto finance) use for blockchain. Ignore all blockchain pitches
When a technical team does make a technology recommendation, you can evaluate the business impact by using a few simple tools. As a general rule, you should avoid the latest new hotness, and ensure you’re not being trapped in old and busted. Your sweet spot are technologies that have been proven to be reliable over a long period of time, and that have a talent pool from which you can sustain your project.
The Gartner Hype Cycle is one of the best tools for understanding the state of technical solutions. Knowing the current state of the hype curve for a given technology will help you tremendously. You should target technologies that are past inflated expectations, and somewhere between the trough of disillusionment and into sustained productivity.
When your team makes a technology selection, they will also be choosing a code language (java, python, c++, ruby, etc) for both the server side and client side software. Ask them what languages they’ll be using, and then review the popularity of those languages. It is important that they choose languages that are widely used. If they choose arcane or very nascent languages - you will have a much harder time finding replacement teams. One of the best ways to determine the state of code languages is GitHub’s yearly Octoverse report on world wide language use. You should be concerned if your teams target languages are not in that report.
Before your team begins coding, you should request - but not define - a system architecture document. Producing this document will ensure your team has clearly thought out the frameworks and systems they’ll use during their work. This document should address the following topics.
- Hosting (compute resources)
- Data storage (where will the data be stored?)
- Security (How will the software be monitored?)
- Continuous Integration (build every commit)
- Continuous Deployment (auto-deployment of new updates)
- Unit tests (testing done by a computer at build time)
- User testing feedback (how will you report bugs?)
- Monitoring (logs, compute resources, etc)
Commissioning software will require long term strategic relationships that are not easily severed. Your development team is your dance partner. They have to feel comfortable on that dance floor - or they’ll leave. You will need them as much or more than they will need you. Focus on recruiting and retaining a team that satisfies your needs.
Your most important developer team relationship is with your “foreman” - not your project manager. The foreman is the person on the development team that knows, at a deep technical level, how your software actually works. Like a foreman on a construction site, this person has been “hands on job tools” for many years and has gained the first-hand knowledge of what it’s like to build software. For reasons explained below, you should be skeptical of teams lead by project managers without deep technical experience. The project managers will give you useful information about the status of the project - but your foreman is the person on whom you rely on to get the job done right.
When interviewing teams, ask for concrete experience. This should include probing questions like “how many people worked on that project?”, “what are the skills of the people that were involved?”, and “what challenges did you encounter?”. The team lead should be able to sound confident about accurately describing the full course of the project. This isn’t a hard ask for talented teams, and will help you sniff out teams to avoid.
Do not use “industry experience” alone as your measure of development team fit. A team that has developed websites for healthcare clients will not be useful to your mobile application project in the same industry. Do not fall for the “industry experience” pitch. Move past that into the technical experience to save yourself much heartache.
If you want a mobile application, then you will have the best outcome engaging a mobile app team. If you want a high speed trading system, then find a team with high speed trading experience. If they happen to have industry experience that overlaps with your project, that should be considered only a strong bonus.
You can have success with teams that do not have your industry experience. In fact, you might find their fresh minds are open to more ideas and innovation than a team that’s been pigeonholed into one industry lane. Unless your goal is software to service business processes in a highly regulated environment, you’ll be ok picking a team without deep industry experience.
Gaining referenceable customers in large software development projects is not easy. If your team can provide happy referenceable customers, they should be commended. Teams that can not provide concrete references may still be a viable option.
Large software development projects often include extremely restrictive non-competes and NDAs. A team you are interviewing may say “We worked on a big project for a large bank”. You should ask them about the experience on that project, and not press for a named reference - because they probably can’t give you one.
Asking for references is a little bit like asking for references to ex-relationship partners on a first date. There’s a reason your software team is no longer fully engaged on that prior project. Maybe everything went great, and the relationship ended naturally. Or maybe not, and they’re still a perfectly viable team. If you probe too deeply, you will be stuck trying to determine who’s truth you’ll accept. Your team and their references will always give you a rosy view because they probably like you and want to make you happy!
The production of working software requires no particular skill. High schoolers with no prior training can create a website that looks decent. But software is like a house that you buy without really knowing what is on the inside. Is it well built? Are there cracks in the walls? Were the plans written down in a way future vendors can understand them? Was it built to standards? Is it secure? When you go cheap, you risk losing not only the time and money you’ve allotted, but also the business impact those failures will cause. And in the end, you will need to go back and re-build - which is never cheaper the second time.
The production of secure, reliable, and useable software is a skill that takes decades of experience. Large projects require the deep skills of multiple types of developers (front-end, back-end, operations, etc). Valuable software isn’t thrown together by a few people part time, or made by novice developers.
The right price is the one with which you are continually comfortable. Instead of focusing on price, create a budget and then focus on risk management. Your risks are that the team will not or can not create valuable, secure, and maintainable software. You can control these risks by selecting an appropriately skilled team and providing them with the resources they need.
Select a team that you feel has the right experience and a price range that leaves room in your budget for them to fail. This requires that you have a known budget and a flexible timeline.
Do not ask for reductions in price based on your available budget or the pricing offered by another team. When you do this, you are unknowingly asking them to squeeze by pressuring their employees (who then become non-motivated), or secretly reducing scope (removing features without telling you). Arbitrary price reductions are not in your interests or theirs.
If your team can not provide a price with which you are comfortable, you can identify potential scope reductions. If you still can not find a price that works for you, you can find another team, reconsider your budget, or elect not to execute the project. These are better options than arbitrary price negotiations.
Your budget should include room for the team to fail - especially related to delivery timelines. Your team is group of fallible humans. Those humans are solving puzzles on your behalf (more on that below). Your goal is to ensure those humans have all of the resources necessary to deliver what you want. They will not deliver every feature promptly, and you will not provide 100% of what they need on time.
Now that your project is underway, you’ll want to ensure that you reach the finish line in budget and on generally on time. But software isn’t like other projects - you’ll need to understand why even little road bumps can throw off software projects in a big way.
You have chosen to commission software because there’s no acceptable commercially available off-the-shelf option. This inherently means you are asking developers to create and assemble a puzzle without a fully complete template. Not all of the pieces of the puzzle exist yet. Software Architects perform the function of designing the system to ensure the pieces that do exist, and the pieces that are not yet created will fit together. The architect will then pass the plan to the developers who interpret that architecture and attempt to solve the puzzle on your behalf.
Imagine you have a puzzle with an unknown number of pieces and only part of the final picture is complete (because if the full picture was available, you’d have bought rather than built). How long will it take you to put that puzzle together? You could probably guess accurately about wide ranges of time, but specific time commitments would be impossible. Custom software development is like putting together puzzles without the complete template. Attempting to predict the full puzzle solving time is counter productive. You could spend months attempting to guess exactly how long a few week project will take - or you could spend that time executing the project. Do not spend time trying to predict puzzle solving times.
Software developers often use an Agile or Kanban (Just-In-Time) method of project management. These methods split larger projects into short periods of time (usually two weeks) called sprints. The team will predict what they can accomplish in the next sprint, perform that work, and then evaluate what they did accomplish during that sprint. They’ll use that review process to better inform their guesses for what they’ll finish in the next sprint. After a few sprints, the team will have a velocity - and that will guide the rest of the timeline for the project.
After a month or so, you should expect your team to be able to accurately (but not completely) guess what they will accomplish in each sprint. Evaluate the team’s velocity with your project manager, and you will have a better understanding of long term delivery timelines.
Project scope is defined by what features are to be or not to be included in your software project. A feature is anything related to the form or function of your product. When you ask your team to change something like a block of text, the color of a button, or the functionality of a screen - you are asking them to change scope. It is important to understand that you have no inherent reference to how a scope change request will affect the project timeline. Do not rely on your own assumptions of the difficulty of a change request.
When you commission a physical building, you can tangibly view the progress being made, and you can gain an immediate sense of the status of the project. In addition to input from your construction team’s project manager, that tangible first hand experience allows you to better understand what effects changes to scope might have on the project status and timeline. The same is not true for software. You will rely completely on your team to help you understand the impact scope changes have on the project.
The best thing you can do to keep your project on time is to control your own desire to change the scope of the project. Before making requests, be absolutely certain that the change is necessary to the core purpose of the business. Then ask yourself why you didn’t know that fact before you started the project. It is important to close the gap and know your final requirements before the developers begin creating your software.
Building valuable software systems is deceivingly difficult. Uber’s ride sharing app looks super simple, but it probably cost more than a hundred million dollars to create. Google’s search bar is super simple - and has hundreds of billions of dollars of investment behind it.
Just because a feature seems simple to you, does not mean it is simple to create. Just because another company has a feature, does not mean you can afford it. Just because your developer team says they can create a feature, does not mean you have the resources to actually make it happen.
Be very cautious about requesting and including features you do not understand thoroughly. Before you commission a feature, go through both the business and technical implications of adopting that feature. Consider the long term resources required to maintain that feature, and whether you or your users will derive sufficient value from that feature.
First, be sure that you understand the puzzle solving analogy above. Then be aware of the following.
Building reliable, secure, and maintainable software requires foundational work that takes significant time. These tasks - such as code build pipelines, and frontend testing frameworks - must be completed before major work begins. You won’t derive a tangibly direct benefit from this work, but you do need to accept that it is important work. If your team jumps directly into building user interface elements, you should ask them to pause and ensure they’ve built the foundations for long term productivity.
The beginning of a major software project should feel slow. There won’t be much visible progress in the user interface early in the project. Most of the features of the software will begin to become visible during the middle of the project. And those features will become well polished near the end of the project.
At the end of the project, you should see rapid additions of features and resolutions of bugs.
Each member of the development team must have familiarity with the use cases, technology stack, and project workflow before they can effectively collaborate. Adding additional team members requires existing team members stop working and provide knowledge transfer to the new members. For this reason, adding team members can actually slow down a project. Be very cautious about adding team members late in a project.
After a few weeks, your team should be able to demonstrate rapid releases of updates. One sign of a lagging team is the inability to quickly turn around very small changes. Ask your team to change one word on a screen, and then measure the amount of time between your request and when you see the result in the test system. If that time is longer than one day, you should be concerned. The ability to push out new changes is fundamental to the velocity of your project.
If your team is not able to turn around very small changes quickly, have them pause all work and focus only on this core issue. They should be able to create a continuous integration and continuous deployment (CI/CD) pipeline that quickly pushes out the latest changes. They should not be relying on a single release manager. And they should not require human intervention after approving code changes.
Resolving issues around update cadences will make your team more productive, and you will receive better software.
Software is easily duplicated and therefore is protected with ownership rights via the intellectual property (IP) legal framework of your country. Just because you have access to source code or can extend a piece of software, does not mean you have the right to do so.
The owner or creator of software defines the licensing terms. A license defines who has rights to access to the source code of the software, who has rights to re-use or extend that software code, and who has the right to sell that software. A terms and conditions agreement (also called an end-user license agreement) defines the terms under which the software may be used.
The most important thing to understand is that you can lose rights or be penalized for unknowingly basing your software on the work of upstream projects that have rights limiting licenses. All software depends on other lower level frameworks or pre-created libraries of functionality. These foundational packages for your project are called “dependencies”. Those dependencies have licenses. And by the nature of software licensing, you are subject to the terms and conditions set forth in those licenses.
If you plan on retaining exclusive rights to your software’s source code, you will need to avoid dependencies that require you to make your software open source, or restrict you from selling your software. A specific license to avoid in this situation is GPL. There are ways to incorporate GPL code into your product, but they require special consideration and handling.
Every time you click “I agree” when signing up for an online service, you’re agreeing to the end-user license agreement (EULA) or terms and conditions of the software owner. These seem mundane, but are important. These terms define the conditions under which you may use the software, what rights you have to use the software, and what recourse you have in the case that the software fails to perform adequately. Imagine if you could sue Apple because iTunes failed to play a song and that ruined your $10k party. You can’t because the terms and conditions limit your rights.
It is important your software have a EULA, and that it is customized to your business use case. You can adopt a common EULA (usually for open source projects), purchase EULA templates online, or engage a software IP rights lawyer to have one generated.
You may have a business plan that can benefit from allowing others to directly interact with your software code. For example, creating a city wide reporting tool for your local government would benefit from adoption by other cities. You would gain traction by having more than one city use this software.
Even if your business model requires exclusivity of code rights, you should consider finding some way to engage a wider developer audience. For example, if you are trying to create an online marketplace, you may wish to open source some or all of your “APIs” (Application Programing Interface) to allow other developers to easily access and create value using that API.
If you choose to open source your software, you will have several licensing options - or you can have a lawyer create a custom license to fit your needs. Typically software from commercial entities are open sourced under the MIT (no restrictions), Apache (some restrictions), or BSD (range of options requiring direct attribution).
Security is a journey, not a destination. The best time to start worrying about security is now. You will never stop worrying about security - so here are some tips on how to feel better about security.
First, define the risks that a security related incident could cause for your business. How much revenue would you lose if your systems were compromised? Who would be held responsible for security lapses? What would your users do when they learn about a security lapse?
Be open with your team about these risks, and make them aware of the potential impacts. This will help your team internalize the security risks, and ensure their attention through the project cycle.
As noted above, your team should define the security framework before they begin. If they haven’t defined that security framework, have them stop all work and focus only on the security framework. You’ll lose project time - but that’s better than losing your business because of a security breach.
A good security framework is a lengthy document that details the security measures implemented in each layer of your software. This document should include detailed descriptions of at least the following.
- Management training and education (informing management about security best practices)
- Organizational security policies and training (everyone in your organization should understand the actions they should take)
- Security owners (who holds the keys, and has the access)
- Data encryption methods (in transit and at rest)
- Secret key management (where the keys are stored, and how they are secured)
- Automated code security scanning
- Automated systems security scanning
- Server and Systems policies (Where they are defined, and how they are enforced)
- Monitoring and reporting (should send automated alerts)
- Automatic updating and patching of servers and systems
- Global CVE monitoring (human review of newly discovered critical vulnerabilities)
- Security incident plan (written steps for when a lapse occurs)
Work with an insurance broker to create a policy that will cover your business in case of a security lapse. An appropriate policy will cover your potential businesses losses, and provide you with tools to recover data if necessary.
You should also insist on professional liability and security lapse coverage from your development team.
Software, like a house, requires maintenance and upkeep after the initial construction phase is complete. Security requires ongoing monitoring and maintenance. As a rule of thumb, you should expect to spend at least 20% of the initial development costs every year in basic maintenance and monitoring of the software.
You will also likely want to expand the features over time, and planning for that expansion is necessary. A long term software road map will help you and your team understand these goals.
You should plan to perform routine maintenance over time. This maintenance will have minor immediate costs. But like maintaining a home, performing maintenance earlier rather than later will dramatically reduce your long term running costs. Here is a list of common maintenance tasks and periods.
- Quarterly dependency upgrades
- Quarterly, semi-annually Architecture updates
- Weekly or monthly operations reviews
Software security standards require ongoing maintenance that is critical to ensuring your software remains secure. The following is a list of security maintenance items you should ensure are being completed. You should engage a team specifically performing security maintenance and monitoring.
- Daily automated security report reviews
- Weekly security patch reviews
- Monthly global security assessments
- Semi-annual security improvements (continuous improvement)
As your software journey progresses, your users will identify feature wishes and you will identify new needs. Gather these features into your “backlog” - which can be a spreadsheet or a purpose built product management system. Periodically, you should prioritize the features in your backlog into high level “phases” and assign those phases to future dates. This is your roadmap. Share this road map with your users and seek their feedback.