The Mythical Man-Month, Chapters 18 - 19

The Mythical Man-Month, Chapters 18 - 19
by Frederick P. Brooks, Jr.

       In Chapter 18 of The Mythical Man-Month, Brooks Jr. provides a summary of all of the major ideas of his book, which I will summarize below:

• Chapter 1: Programming is rewarding because it is a creative task and involves critical thinking, but also is painstaking and can become obsolete quickly. Programming for a system takes much more time than programming a single component.
• Chapter 2: The main reason programming projects go wrong is because of calendar time and scheduling issues; the man-month shouldn't be used for measurement and is misleading.
• Chapter 3: A small team that resembles a surgical team is best, including a head programmer and a small group of others that assist him (editor, tool builder, documentation writer, etc).
• Chapter 4: Your design much have conceptual integrity, and the best way to ensure that is to have a small number of heads involved in the design.
• Chapter 5: The tendency in designing a second system is to over-design it. Instead, try to fix the most important issues without becoming ambitious and redesigning the whole thing.
• Chapter 6: The architecture must be completely specified, both formally and prose-like, and needs to be in the hands of everyone on the team.
• Chapter 7: Communication is very important, and must be done both formally and informally. A product workbook is essential to have all documentation in one place.
• Chapter 8: Estimate your tasks individually and not as a whole, because some tasks are more complex than others. Using a high-level language increases productivity.
• Chapter 9: Representation is the most important thing in programming. When designing to conserve memory, have standards but do not reduce functionality because of this standard.
• Chapter 10: A small base of documents should be the manager's best tool, but is used only sparingly; most of his time is spent out communicating with the team.
• Chapter 11: Design to throw one away, because you learn through doing and not just planning.
• Chapter 12: Keep one person to build tools, and choose wisely, because tools directly influence productivity and efficiency.
• Chapter 13: Test each part individually before putting them together. Design top-down and continually define and reduce abstraction until you can start coding.
• Chapter 14: Always keep track of schedule, so you can catch schedule slippage and do something about it as soon as possible.
• Chapter 15: When writing documentation the user will see, make sure it's technical enough but is written in easy-to-understand prose.

       In Chapter 19, Brooks talks about the effect that The Mythical Man-Month has had over twenty years, and how the ideas inside have changed with the times. He identifies the issues that affect software development and that are relevant no matter how much technology has changed:

• Conceptual Integrity: When the product is designed, it must have a conceptual image that pervades the whole system. This is important for the programming team because it gives a basis for everything that is included, including features and user interface. This is also important for the user because it lets them know the system has a clear use, has clearly-defined features, and that everything included has its own specified place in the system. The example Brooks gives is the WIMP interface, or "Windows, Icons, Menus, Pointing Device," which is the user interface system that Mac and PC computers still use today.
• Featuritis: Software tends to have too much functionality added on. The easiest to use program is one that has a clear use, and doesn't try to do everything or be too ambitious. Programmers are always trying to make the system more useful for users, but often adding too many features makes the program unwieldy and actually less useful because users don't understand the software.
• Incremental Design: The system should be designed in incremental parts, adding functionality each time. When this happens, there is always a deliverable to show to a customer, and could always be released early if funds run out (albeit missing some features). It also makes testing and debugging easier because only a few changes are made in code at one time, and you have less of a place you have to look for problems.
• Shrink-Wrapped Software: Nowadays, there are SDKs, libraries, and standards for programming languages. This helps with standardization of interfaces and allows programmers to reuse code. Code reuse makes the system easier to build and understand, and allows the programmers more time for testing, user studies, and means that the software gets to the user faster.

Although the examples in the book are outdated, many of the issues explained still exist today and the solutions given are still viable and relevant solutions for this programming problems.

       In all, I thought this book was very interesting because of how similar it was to Extreme Programming Installed.  Even though Brooks Jr. was working on operating system and computer software before GUIs and window-based interfaces were common, he understood the problems inherent in scheduling software projects, communication, documentation, design, testing, and debugging. I'm pretty sure that many of these ideas inspired Agile development and will continue to inspire revolutions in software engineering paradigms in the future.

The Mythical Man-Month, Chapters 16 - 17

The Mythical Man-Month, Chapters 16 - 17
by Frederick P. Brooks, Jr.

       In Chapter 16 of The Mythical Man-Month, Brooks Jr. talks about the "silver bullet." Computer hardware continues to grow by leaps and bounds, but software is unable to grow at the same speed. Software projects continually get behind schedule, are not as efficient, and never make the deadline. Brooks argues that there is no silver bullet; "there is no single development, in either technology or management technique, which by itself promises even one order of magnitude improvement in productivity, in reliability, in simplicity." There are two types of difficulties in software development that plague projects: essential difficulties (those inherent to the problem itself, such as algorithms or development techniques) and accidental difficulties (those that concern the development process, such as development machines, machine time, and programming language). Major essential difficulties include the complexity of software as compared to hardware, conformity (that software products might be controlled by other parties), changeability, and invisibility (that the product is not easily visualizable). Many of the accidental difficulties have actually been resolved through developments in technology, through high-level programming languages, time sharing of machines, object-oriented programming, and artificial intelligence.  While many advances have helped with the essential difficulties, such as rapid prototyping and incremental development, there is yet to be a silver bullet that allows for a large growth in software development efficiency.
       In Chapter 17, Brooks refines his original statement about the silver bullet with an update almost ten years later. After critics have issued rebuttals to his theory, he has decided that there actually is a silver bullet. First of all, visualization techniques and diagramming heavily help the process. But, the most important thing is to focus on quality of software instead of just productivity in the software development process. When you focus on quality, you learn how to produce great software and find techniques and shortcuts that end up helping the productivity of the team.  This whole system in called the "Vanilla Framework." In all, although a lot of things have been developed to help software development be more efficient, but we shouldn't sit around waiting for a major breakthrough. Instead, we should focus on the incremental upgrades that can be made and continue to look for small ways to be more efficient.

      I think for the most part that I agree with Brooks about software development and efficiency. It's easy to continue to make advances in the hardware for a system, but hard to have a large leap in efficiency in a software product. Because of the complexity of the system and how software is always changing (to keep up with the new hardware), it's hard to keep up. I definitely agree more with Brooks' revision of "No Silver Bullet," in that instead of focusing on large-scale order of magnitude developments, we should focus on the small improvements that can be made. This includes higher-level languages, new debugging techniques, version control, more useful and complex IDEs, web development, and new forms of wireless and touch-based computing. If we continue to incrementally upgrade our software development tools and methodologies, we can still see great improvements over the development processes of the past.

The Mythical Man-Month, Chapters 13 - 15

The Mythical Man-Month, Chapters 13 - 15
by Frederick P. Brooks, Jr.

       In Chapter 13 of The Mythical Man-Month, Brooks Jr. talks about building a program to work; putting all the little pieces together, testing and debugging, and integrating into the larger system. The first important step is to make sure your design and specification is "bug-proof." Everything major in the project should be defined clearly, and should be looked at by a third party to find errors and gaps that might be there. Some of the hardest errors to find are those when programmers of interacting systems assume different things about the code, whether it be the structure of the class or the way a message is formed. Top-down design is also very important; this procedure starts with high-level abstractions and sketches, and continues to refine the design down to the level needed for the programmers and implementers. Lastly, debugging is really important during the programming process. Whether it's done on-machine, in a batch setting, through memory dumps or snapshots, or interactively, careful attention should be made to debug as you go. When building a larger system, make sure that the smaller components have been debugged individually so we can rule out component-specific bugs.
       In Chapter 14, he addresses the issue of schedule slippage; why projects can slowly get behind schedule, and tools to keep this from happening. The most important thing of a programming project is it's schedule. In that schedule, we have a list of milestones, with concrete dates. These milestones must be completely specified so not a single person can argue about what it means, and have measurable events and a specific date they must done by. The more specific it is, the harder it is to lie and say that it's finished. PERT charts are also important because they allow you to lay out the schedule, with probabilities that events will be finished by the due date. It will also give you a concrete diagram of the dependencies between different parts of the system. Another problem is "the rug," meaning that whenever a small team slips behind, it won't always come to the attention of the main boss. Even though that manager may think they can fix the problem, this should always be told to the boss because they can use contingency plans to fix the issues. Lastly, the PERT chart should be used to make reviews throughout the project and find issues of schedule slippage as soon as they happen.
       In Chapter 15, Brooks Jr. emphasizes the importance of good documentation, and explains the most important documentation needed for a software product. First of all, the user needs an easy to read description of the program, including the purpose of the software, what kind of input it takes, the format of the output, different options the user has, exactly how to do those options, and how to make sure the program is working correctly. The user also needs some documentation that helps them believe the program works correctly, including examples that they can run, test cases, etc, that show different inputs and their corresponding outputs. When something goes wrong with the program, the user needs a description on how to modify and fix the issue. Usually, full detail for the user is required, which could include a flow chart, descriptions of the algorithm used, or an explanation of the file structure of the program. He spends the rest of the chapter talking about self-documenting programs, which really just means "comments." These comments in the code allow for the programmer to analyze the code and also see the documentation at the same time, not having to worry about looking back and forth between code and another document.

      Again, the stuff that Brooks has to say makes sense, but is kind of obsolete in our day. In his time, this information on high-level languages, interactive debuggers, and comments inline in source code must have seemed radical but extremely useful. It is interesting to read this book as a history lesson, because it accurately describes the techniques used before, and the modern-day solutions to those problems. Comments in source code definitely help, and having someone check over your design to check for gaps before implementing is definitely a good idea. I think the most important thing was the "top-down design" approach. It definitely helps to get a good picture of exactly what you want your code to do before bothering yourself with possible implementation strategies and how you will test the code.

The Mythical Man-Month, Chapters 10 - 12

The Mythical Man-Month, Chapters 10 - 12
by Frederick P. Brooks, Jr.

       In Chapter 10 of The Mythical Man-Month, Brooks Jr. addresses the issue of how much documentation that a manager should keep for themselves, and what important documents are needed. The concerns of any manager are the "what", "when", "how much", "where", and "who". In a software development setting, these correspond to objectives of the software and produce specifications, schedule, budgeting, space allocation, and an organization chart that assigns tasks to different programmers and managers. It's important to have these kinds of documents because writing down helps inconsistencies and gaps in the design process to come out. Those documents also serve as a way to communicate to everyone on the team the decisions that have been made about these issues, because oral communication and emails may not have circulated around to everyone. But, the manager must realize that these documents are only about 20% of the management tasks, and that the other 80% is spent guiding and encouraging the programmers on the team.
       In Chapter 11, Brooks Jr. talks about the very first part of the software project that is built. Usually, after some design and meetings, the programmers start working on the project and writing code. Algorithms are proposed and refined, and they find out that the functionality that they wrote works and is ready to be written into a larger-scale program. Brooks says that the teams should always "plan to throw one away." This means that whenever you first start programming, you should have in mind that this first release is tentative and will get thrown away. Then, you can start over with a better idea of what you want to accomplish, and know what works and what doesn't. If you stick with that first draft, you'll end up releasing a slow, buggy, and hard to use system to the users and end up having to redesign the product anyway. Brooks talks about how each system starts out "taking two steps forward and one step back." This means that continued releases add functionality but are hindered just a bit by bug fixes. But after users get more familiar with the program, more and more bugs are found, and the system is just taking "one step forward and one step back." This means that no matter what functionality you add, the bug fixes end up destroying that functionality and the structure of the system. This is why you should throw away the first draft and start over, and not waste time upgrading software that is essentially already dead.
       In Chapter 12, he talks about the tools that a programming team will use, and how their organization and selection can greatly increase the efficiency of the project. First, he says that each programmer shouldn't use their own set of tools, because they hamper communication and can cause differences in the code produced. As each team has different things they're working on, specialized tools per team are necessary, with a toolmaker designated per team that decides what tools to use and trains the team in how to use them. One important tool is the target machine and the vehicle machine for the program. The target machine is what the product will be run on, and the vehicle machine is the machine that the development will be done on. Programming libraries, source control, text-editors for documentation, and the language used are all important tools vital to the success of the project.

     Again, a lot of the ideas in this book are fairly outdated. In keeping with the XP tradition, I do like that Brooks proposes a smaller set of documents instead of the giant product book that he talked about in earlier chapters. This way you can spend more time programming and less time making changes in all of the documentation when design issues come up. I agree that the first draft of a product should usually be thrown out, and it's always good to start again. The problem is that whenever you work that hard on a project, it's hard to throw that away, and also hard to start a new project without doing the exact same work that you just finished. Lastly, he spends a lot of time talking about scheduling for machines, which isn't as relevant today. Maybe there are still groups that do research on supercomputers, but today usually each person has their own computer to program on and possibly their own server to test on as well.

The Mythical Man-Month, Chapters 7 - 9

The Mythical Man-Month, Chapters 7 - 9

by Frederick P. Brooks, Jr.

     In Chapter 7 of The Mythical Man-Month, Brooks talks about effective communication within a programming team. He gives the example of the Tower of Babel, in which the workers had leadership, the materials, and sound engineering design yet failed because they couldn't communicate with each other. In a programming project, it's important to have many types of communication, including informal (on the telephone, walking over to their cubicle), meetings (which should be daily), and through documentation. Brooks suggests that the project use a "Project Workbook" as the primary method for the documentation. This workbook contains all of the documentation that was ever produced for the project, allowing for daily changes and displayed so that the programmers can see what changes were made and a description of the changes. He also talked about different organizational roles within the project, and the two most important: the producer (who handles assigning work, making sure everything is getting done, tracking, etc) and the technical director (makes executive-level decisions about technical aspects of the system). In small projects,  both of these roles can be the same person. In really large projects with large teams, it's best to let the producer be in charge, and allow the technical director to be out of the way of management roles, while still having authority over decisions. In smaller teams, it's useful to let the technical director be in charge, and have the producer answer to them.

     Chapter 8 talks about estimating the time to finish a programming task, and general programming productivity. Brooks in his experience has found that using estimates for smaller programming task is ineffective whenever estimating for a large project. For example, if a single programmer estimates that the task will take about 2 weeks, they will almost always need 4 weeks, or double the time, to finish the task. This is because not all of your working time is spent programming; the programming portion of the task will still take 2 weeks as estimated, but the other 2 weeks is spent in machine downtime, sickness, meetings, paperwork, setting up system, and the like. Brooks also presents more data about the relationship between programming complexity and the time to complete the task. As expected, more complex problems that required interactions between program modules and between different programmers took longer. And, in the last cited study, programming in a higher-level language versus machine code made the programmers five times as productive.

     Chapter 9 talks about program space, its association of cost, and how to reduce the space you use when building a program. The size of a program is directly related to cost; whether it is the "memory-rental" cost per month to run a system in the past, or the cost of memory in the user's computers that the program will be run on, size plays a factor. Size inherently isn't bad, but unnecessary size is bad. When designing a software product, there should be space budgets built in. Not just for the core and the operating system, or the core memory that the program will use, but instead budgets for the disk, for the cache, for the stack, and everything included. It's also important to look at the features required for a section of code or module. If you specify that a module is going to be very big, you also need to have an exact specification of everything that the module will do. Usually, you'll find some memory that you're using that is not necessary to meet these goals. And lastly, you need to look at the representation of your data. Usually, you can find things that could be stored more efficiently using a different data structure, or that you store the same data more than once.

     I agree with Brooks when he talks about estimation; usually when you estimate two weeks, you don't take into account the time for breaks, for time off, for meetings, and for dealing with problems in the development system. When Brooks explains techniques to reduce space, it just doesn't seem as relevant today. Now that all systems have gigabytes and more of memory, it's hard to spend all your time making everything memory efficient. Yes, faster algorithms run faster on slower machines than do slow algorithms on faster machines, but we have so much computing power today that it's not as relevant. And lastly, I do not agree with the Project Workbook. Seems like too much effort to keep all of the changes, etc when you are constantly making those changes. Something like the XP method of programming first and designing later would be more effective.

Extreme Programming Installed, Chapters 22 - 24

Extreme Programming Installed, Chapters 22 - 24
by Ron Jeffries, Ann Anderson, and Chet Hendrickson

     Chapter 22 of Extreme Programming Installed is about defects and bugs in your code. The most important thing about finding defects is the way they are reported; whether you include an automated defect tool within the software or allow the customer to email them, they need to be quickly communicated and dealt with. There is also the issue of priority. If the defect is a minor fix, it can be written on a card and scheduled for the next iteration as if it was a regular user story. If the defect is a little more serious, the customer and programming team need to coordinate about priority. When a programmer is fixing a defect, they're not working on new code for the iteration, and so a few small features may need to be dropped to accommodate this new time requirement. It's also really important to try to prevent future defects. If a defect makes it to the customer, that means that it made it past the programmers' unit tests and also past the customer's acceptance tests. Examine your current tests and write a test that will make the defect appear. Then fix the defect, and now you know that future defects related to this defect will be caught before they make it to the customer.
     Chapter 23 summarizes all of the concepts of Extreme Programming that make software projects more efficient and on time. XP stresses communication between programmers, managers, and customers, and mandates that a customer be on-site for in-person communication. Testing is a heavy part of the development process, and any code that is going to be released must pass all of the unit tests 100%. Programmers code in pairs, with one person working on the code and the other watching over their shoulder and asking questions to make sure they understand the meaning and also catch as many errors as possible. Planning should be minimal at the beginning, and just consist of taking features and converting them into user stories, and making initial estimates. XP stresses iterative design and programming where you are constantly tracking your project and updating design paradigms and coding estimates along the way.
     Chapter 24 is about how to communicate to the customer and manager about the programmer's estimates. A lot of the time, management doesn't completely understand the difficulty and complexity of the software that is being built. As the authors explain, the programmers are usually forced to say "I'll try," and then end up working overtime and stressing themselves out to reach the deadline. Usually, even with all the hard work and stress, the deadline still isn't made. Instead, you should sit down with your customer and managers and explain the estimates. Explain that this is the amount of points you team can finish in a week, show how many points each task is estimated, and then allow them to prioritize and remove some less important stories from the iteration. Although they'll probably be frustrated and mad, you now will have a reachable deadline and can work towards that without stressing the whole team and managers out.

     In all, I thought that Extreme Programming Installed was a very interesting book, and seemed to address many of the issues with programming and software development paradigms used today. I agree that design should be done along the way, because as you work on a software project you learn more about possibly implementations and get better estimates along the way. I also like the idea of open communication - waiting for an email from someone else will just hold you back and keep you from continuing your work on important issues. It's a lot easier to just walk down the hallway and get a clear answer right away so you can continue your work. Extreme Programming, if adopted by more development teams, would mean that we haves more useful software on the market that meets deadlines and has less defects and bugs, all while allowing the programming team and managers to do their job efficiently, without stress, and enjoy it along the way.

The Mythical Man-Month, Chapters 4 - 6

The Mythical Man-Month, Chapters 4 - 6
by Frederick P. Jones, Jr.

     In Chapter 4 of The Mythical Man-Month, the author presents the concept of "conceptual integrity". This means that your system should be of one unifying design principle, and not just a lump of varying functionality. Conceptual integrity is extremely important to ease-of-use, and it is better to leave out some functionality than sacrifice this. It is also important to separate "architecture" of the system from the "implementation" of the parts. The ones that ensure the integrity of the system and it's design are the architects. It should be a small group of people so as to keep the ideas one of track, and they should not bother themselves with how each section might be implemented. If too many of the programmers who are implementing the system get involved in the discussion of the architecture, not enough time is spent on the implementation, which means the system might not be built as efficiently. The programmers still get to have a part in the creative process because they can decide the implementation process and come up with elegant and efficient solutions to the specifications given by the architects.
     In Chapter 5, the authors address the issue of the 'second system effect'. Usually, the first system that a group of programmers design is simple, efficient, and functional, because they have made sure they kept conceptual integrity. But, when designing the second system, they try to fix all the mistakes that they made in the first system. The problem with this is that it usually results in feature overload. Even though the new system will be more functional and useful to the users, it is usually an unwieldy monster that feels bloated and inelegant. Another problem is that there might be a lot of focus on a feature that wasn't quite right in the first iteration, even though that feature is outdated. For example, the programmers might spend a lot of time making sure that the FIFO scheduler of the operating system is efficient and uses less memory, when they should have spent time working on the next generation of scheduling algorithms that would be faster and more efficient.
     In Chapter 6, they talk about how it's possible to ensure conceptual integrity when you may have one hundred different programmers working on the project. The important thing that the architect releases is the manual; this document must lay out all of the functionality that will be provided, including everything the user will see, but also must not provide any of the implementation details and leave that to the programmers. The architect can also specify the system using a formal definition, most usually described with the implementation. The problem with this is that sometimes it will over specify how things work in the system. And, sometimes defining by implementation means that developers view errors and bugs in the system as defined and standard output, causing problems with previously written programs when you update and patch the system. Meetings are also very important, but should have a structure and formal method for communicating and resolving disputes. Keeping a telephone log of all questions to the architect and their responses provides a history to look up whenever there are design issues again.

     I didn't like this section as much as the first three chapters. It talked a lot about conceptual integrity, system architecture and design, and communicating these design paradigms to each member of the programming group. While the ideas are good, and it is important to have the architect formally define the design so you haves coherence, I think the design paradigm in XP is a much better process. Design should be minimal at the get-go, because usually it's not possible to know all the features and constraints at the beginning of the project. Yes, an architect (or customer, in XP) should define features they want, but those things shouldn't be set in stone. Instead, they should be looked at continually throughout the development process and changed to fit estimates and progress.