Project Management 101

When working on projects its helpful to know in which direction you are headed.  Its good to place goals, do a bit of planning, and get things right to save time in the future.  Unfortunately however, there is little to no information online on how to successfully manage and keep a big software project going.  Sure there are management books on how to motivate people, or how to lead coworkers, or even how to plan every step of the project with careful diagrams, flow charts, and process graphs.  As far as I could find a year ago there was nothing on how to get down and dirty and manage things like a build process, or an automatic unit test suite, or how to deal with different platforms in your build system.  For the past year I have been working on a big project that required all these things and whenever I fell into a trap, or discovered a new idea, or came up with a new operation I wrote down little snip-its to be eventually published in this article.  I hope others looking for information on how to manage a project effectively and efficiently will find some of these ideas very helpful.

The very first obstacle that any project can face is most certainly compiling on any platform other than the “primary” operating system.  Putting developers outside their development comfort zone can breed resentment for the new platform.  This resentment manifests itself into very sub-par code for the platform in question and only breeds more resentment as the project progresses and developers start complaining about how the other platform “just is not working.”  I see this all the time when developers start using the Linux tool chain when used to Visual Studio, and when a Linux guru tries to use platforms like Windows.  What eventually ends up happening is a bunch of smelly code gets written for the platform no one likes and is put together using a terrible build system that only further breeds discomfort and anger against the offending OS.  This reflects in the final product where users of the accursed OS are shunned for their complaints, new feature requests are not taken seriously, and one by one new developers that were brought in to work on the platform leave the company because even they get shot down when offering advice to the “project gurus” who hate the operating system they do not understand.

Fortunately scenarios such as this can be quickly remedied before they even start.  As a general rule, never put someone on a project unless they understand all the pieces involved.  When a Linux developer is put in charge of a Windows build, their first instinct is going to want to build a set of recursive nmake files.  Nmake is sort of like a command line make tool for Windows for those unaware.  Development of the tool was canceled after Visual Studio 2005, but it still works and some Linux developers still try and use the woefully inadequate tool as a replacement to gnu make.  As the Linux developer learns about the limitations of nmake he starts building up his resentment for the platform in general.  The final product being a nmake build system that is full of quirky “hacks” to work around nmakes limitations and emulate some of the functionality of gnu make.  For example, with gnu make you can setup automatic dependency scanning so if a developer modifies a header file all .c and .cpp files that depend on it are recompiled.  If the Linux developer does not find similar functionality with nmake, he might just decide to setup nmake  to delete all object files every run and recompile the whole project each build.  This in turn causes other developers to not want to use the Windows compiler because Linux compiles in half the time and they can make quick changes without waiting hours for a new build.  This in turn causes development on Windows to slow to a trickle and the platform falls behind on enhancement requests as Windows developers start leaving for better prospects.  And thus the downward spiral continues.

In a different scenario lets put a Windows developer in charge of the Windows build.  He is most likely fluent in the best tool for compiling on Windows, Visual Studio.  He knows that nmake is a disaster and instead builds easy to maintain Visual Studio solutions which allows other developers to open the project and compile effortlessly.  He then utilizes Microsoft’s MSBuild tool to setup automated build and packaging processes and the project continues moving forward drawing in the best Windows development talent.

Sometimes due to lack of budget or development talent it is not always possible to put the best talent on the right job.  In my recent project I had to build both a Linux and Windows build system.  At the time I knew about Linux make and even maintained a couple make files, but other than that I had no experience writing one.  At these times its important to stick to the core principles of software development.  KISS ( Keep It Simple and Stupid ) and DRY ( Don’t Repeat Yourself ).  Using these two techniques developers on an unfamiliar system can sacrifice time for quality.  Building the gnu make system took me longer than someone who knew Linux make backwards and forwards, but the end result is about the same.  As an added benefit, the developer gains insight into how the new system works and in the future will not take so long to create new make files.

Learning how to effectively apply KISS and DRY is a skill worth developing as it takes quite a few mistakes and failures to get right.  The important questions to ask yourself while working is “Is this as simple and concise as it could be?” and “Is this same concept used more than once in my project?”  Google is most definitely your friend as you struggle with new software or design.  Basically, how I go about using a new system is imagine what I want it to do, and Google for others who got it to work that way.  In some cases people will instead tell you that your idea is dumb and is not what the software was designed to do.  Which may make poorly trained developers insert a hack or two into the system to make it work like he wants.  But this is the important point!  If you are using KISS and DRY you should never allow any hacks of any kind into your work.  Work with the software, do not make the software work for you.  The world is a much nicer place when software is doing the job it was designed to do as it was designed to do it.  Do not make software work how your warped perspective says it should work.  If you take nothing else from this article take that bit of advice.

When developing large systems it sometimes becomes apparent that some changes require large amounts of copy and pasting.  In my project I had 20-30 individual build projects with 20-30 individual makefiles and visual studio projects.  At one point each makefile held a lot of duplicate information.  If I wanted to add a CFLAG to the whole project I would have to edit each makefile or if I was lucky, have sed do it for me.  The whole system was one big violation of the DRY principle, and I was wasting valuable time writing sed scripts rather than doing real work.  Recognizing the problem I quickly redesigned the system to correctly leverage the capabilities of the makefile include directive.  Ideally when working on a project that contains many individual builds its advantageous to keep the actual build logic out of the makefiles.  The only thing needed for an individual build should be a list of source files and any extra flags the build might need.  By using the include directive you can separate the build and link implicit rules into different makefiles that your project makefiles include and use.  Changes to the global project are then applied to a single makefile and are included by all projects.

Same goes for Visual Studio projects.  In a solution containing 20-30 project files, changing something as simple as the release flags takes a long time.  Thankfully Visual Studio allows you to use property sheets which are basically a collection of properties that can be applied to many projects.  I use them for specifying all properties in the project except build type and source files.

I have uploaded the current version of my makefiles and property sheet to bitbucket, feel free to check them out.

Moving on to compilers and source files.  I think it is more important to pay attention to KISS and DRY when writing code then when writing a build system.  Build systems can be changed, code changes can get complicated, especially if you care about backwards compatibility.  When writing code start simple and get more complex.  Think of an easy representation to shoot for and adjust your expectations as the requirements get more specific.  Pay attention to things like namespaces, type names, capitalization, and other minor style issues.  Do not throw all your types into a single namespace as that gets confusing when you start dealing with fields of 30 or more types.  Pick a standard coding style and stick to it.  New users may have to get used to it, but the last thing you want is a code base covered in five different standards.  This obviously breaks programmer flow and causes severe loss of likability as users cannot adapt to your project and code based on their expectations rather then reading straight from the doc.

My next post will contain tips specifically for coding.  Related to GCC, Pimpls, smart pointers, and unit testing.

Be Sociable, Share!

•