The Mythical Man-Month, Chapters 7 - 9

The Mythical Man-Month, Chapters 7 - 9

by Frederick P. Brooks, Jr.

     In Chapter 7 of The Mythical Man-Month, Brooks talks about effective communication within a programming team. He gives the example of the Tower of Babel, in which the workers had leadership, the materials, and sound engineering design yet failed because they couldn't communicate with each other. In a programming project, it's important to have many types of communication, including informal (on the telephone, walking over to their cubicle), meetings (which should be daily), and through documentation. Brooks suggests that the project use a "Project Workbook" as the primary method for the documentation. This workbook contains all of the documentation that was ever produced for the project, allowing for daily changes and displayed so that the programmers can see what changes were made and a description of the changes. He also talked about different organizational roles within the project, and the two most important: the producer (who handles assigning work, making sure everything is getting done, tracking, etc) and the technical director (makes executive-level decisions about technical aspects of the system). In small projects,  both of these roles can be the same person. In really large projects with large teams, it's best to let the producer be in charge, and allow the technical director to be out of the way of management roles, while still having authority over decisions. In smaller teams, it's useful to let the technical director be in charge, and have the producer answer to them.

     Chapter 8 talks about estimating the time to finish a programming task, and general programming productivity. Brooks in his experience has found that using estimates for smaller programming task is ineffective whenever estimating for a large project. For example, if a single programmer estimates that the task will take about 2 weeks, they will almost always need 4 weeks, or double the time, to finish the task. This is because not all of your working time is spent programming; the programming portion of the task will still take 2 weeks as estimated, but the other 2 weeks is spent in machine downtime, sickness, meetings, paperwork, setting up system, and the like. Brooks also presents more data about the relationship between programming complexity and the time to complete the task. As expected, more complex problems that required interactions between program modules and between different programmers took longer. And, in the last cited study, programming in a higher-level language versus machine code made the programmers five times as productive.

     Chapter 9 talks about program space, its association of cost, and how to reduce the space you use when building a program. The size of a program is directly related to cost; whether it is the "memory-rental" cost per month to run a system in the past, or the cost of memory in the user's computers that the program will be run on, size plays a factor. Size inherently isn't bad, but unnecessary size is bad. When designing a software product, there should be space budgets built in. Not just for the core and the operating system, or the core memory that the program will use, but instead budgets for the disk, for the cache, for the stack, and everything included. It's also important to look at the features required for a section of code or module. If you specify that a module is going to be very big, you also need to have an exact specification of everything that the module will do. Usually, you'll find some memory that you're using that is not necessary to meet these goals. And lastly, you need to look at the representation of your data. Usually, you can find things that could be stored more efficiently using a different data structure, or that you store the same data more than once.

     I agree with Brooks when he talks about estimation; usually when you estimate two weeks, you don't take into account the time for breaks, for time off, for meetings, and for dealing with problems in the development system. When Brooks explains techniques to reduce space, it just doesn't seem as relevant today. Now that all systems have gigabytes and more of memory, it's hard to spend all your time making everything memory efficient. Yes, faster algorithms run faster on slower machines than do slow algorithms on faster machines, but we have so much computing power today that it's not as relevant. And lastly, I do not agree with the Project Workbook. Seems like too much effort to keep all of the changes, etc when you are constantly making those changes. Something like the XP method of programming first and designing later would be more effective.

Extreme Programming Installed, Chapters 22 - 24

Extreme Programming Installed, Chapters 22 - 24
by Ron Jeffries, Ann Anderson, and Chet Hendrickson

     Chapter 22 of Extreme Programming Installed is about defects and bugs in your code. The most important thing about finding defects is the way they are reported; whether you include an automated defect tool within the software or allow the customer to email them, they need to be quickly communicated and dealt with. There is also the issue of priority. If the defect is a minor fix, it can be written on a card and scheduled for the next iteration as if it was a regular user story. If the defect is a little more serious, the customer and programming team need to coordinate about priority. When a programmer is fixing a defect, they're not working on new code for the iteration, and so a few small features may need to be dropped to accommodate this new time requirement. It's also really important to try to prevent future defects. If a defect makes it to the customer, that means that it made it past the programmers' unit tests and also past the customer's acceptance tests. Examine your current tests and write a test that will make the defect appear. Then fix the defect, and now you know that future defects related to this defect will be caught before they make it to the customer.
     Chapter 23 summarizes all of the concepts of Extreme Programming that make software projects more efficient and on time. XP stresses communication between programmers, managers, and customers, and mandates that a customer be on-site for in-person communication. Testing is a heavy part of the development process, and any code that is going to be released must pass all of the unit tests 100%. Programmers code in pairs, with one person working on the code and the other watching over their shoulder and asking questions to make sure they understand the meaning and also catch as many errors as possible. Planning should be minimal at the beginning, and just consist of taking features and converting them into user stories, and making initial estimates. XP stresses iterative design and programming where you are constantly tracking your project and updating design paradigms and coding estimates along the way.
     Chapter 24 is about how to communicate to the customer and manager about the programmer's estimates. A lot of the time, management doesn't completely understand the difficulty and complexity of the software that is being built. As the authors explain, the programmers are usually forced to say "I'll try," and then end up working overtime and stressing themselves out to reach the deadline. Usually, even with all the hard work and stress, the deadline still isn't made. Instead, you should sit down with your customer and managers and explain the estimates. Explain that this is the amount of points you team can finish in a week, show how many points each task is estimated, and then allow them to prioritize and remove some less important stories from the iteration. Although they'll probably be frustrated and mad, you now will have a reachable deadline and can work towards that without stressing the whole team and managers out.

     In all, I thought that Extreme Programming Installed was a very interesting book, and seemed to address many of the issues with programming and software development paradigms used today. I agree that design should be done along the way, because as you work on a software project you learn more about possibly implementations and get better estimates along the way. I also like the idea of open communication - waiting for an email from someone else will just hold you back and keep you from continuing your work on important issues. It's a lot easier to just walk down the hallway and get a clear answer right away so you can continue your work. Extreme Programming, if adopted by more development teams, would mean that we haves more useful software on the market that meets deadlines and has less defects and bugs, all while allowing the programming team and managers to do their job efficiently, without stress, and enjoy it along the way.

The Mythical Man-Month, Chapters 4 - 6

The Mythical Man-Month, Chapters 4 - 6
by Frederick P. Jones, Jr.

     In Chapter 4 of The Mythical Man-Month, the author presents the concept of "conceptual integrity". This means that your system should be of one unifying design principle, and not just a lump of varying functionality. Conceptual integrity is extremely important to ease-of-use, and it is better to leave out some functionality than sacrifice this. It is also important to separate "architecture" of the system from the "implementation" of the parts. The ones that ensure the integrity of the system and it's design are the architects. It should be a small group of people so as to keep the ideas one of track, and they should not bother themselves with how each section might be implemented. If too many of the programmers who are implementing the system get involved in the discussion of the architecture, not enough time is spent on the implementation, which means the system might not be built as efficiently. The programmers still get to have a part in the creative process because they can decide the implementation process and come up with elegant and efficient solutions to the specifications given by the architects.
     In Chapter 5, the authors address the issue of the 'second system effect'. Usually, the first system that a group of programmers design is simple, efficient, and functional, because they have made sure they kept conceptual integrity. But, when designing the second system, they try to fix all the mistakes that they made in the first system. The problem with this is that it usually results in feature overload. Even though the new system will be more functional and useful to the users, it is usually an unwieldy monster that feels bloated and inelegant. Another problem is that there might be a lot of focus on a feature that wasn't quite right in the first iteration, even though that feature is outdated. For example, the programmers might spend a lot of time making sure that the FIFO scheduler of the operating system is efficient and uses less memory, when they should have spent time working on the next generation of scheduling algorithms that would be faster and more efficient.
     In Chapter 6, they talk about how it's possible to ensure conceptual integrity when you may have one hundred different programmers working on the project. The important thing that the architect releases is the manual; this document must lay out all of the functionality that will be provided, including everything the user will see, but also must not provide any of the implementation details and leave that to the programmers. The architect can also specify the system using a formal definition, most usually described with the implementation. The problem with this is that sometimes it will over specify how things work in the system. And, sometimes defining by implementation means that developers view errors and bugs in the system as defined and standard output, causing problems with previously written programs when you update and patch the system. Meetings are also very important, but should have a structure and formal method for communicating and resolving disputes. Keeping a telephone log of all questions to the architect and their responses provides a history to look up whenever there are design issues again.

     I didn't like this section as much as the first three chapters. It talked a lot about conceptual integrity, system architecture and design, and communicating these design paradigms to each member of the programming group. While the ideas are good, and it is important to have the architect formally define the design so you haves coherence, I think the design paradigm in XP is a much better process. Design should be minimal at the get-go, because usually it's not possible to know all the features and constraints at the beginning of the project. Yes, an architect (or customer, in XP) should define features they want, but those things shouldn't be set in stone. Instead, they should be looked at continually throughout the development process and changed to fit estimates and progress.

Extreme Programming Installed, Chapters 19 - 21
by Ron Jeffries, Ann Anderson, and Chet Hendrickson

Chapters 19 - 21 of Extreme Programming Installed is about steering a project and making changes to schedules and estimates along the lifecycle of the software. Whenever you make estimates at the beginning of planning, it's not always easy to be completely accurate because you just don't know how much time is required until you start coding. When steering your Extreme Programming project, you need to focus on projected estimates versus actual coding time, changing priorities in the code, and roadblocks that keep a programmer from completing their tasks.

According to the authors, it's important to keep tabs on and track the project so you can steer the project to success. First of all, the most important thing is to get user stories done. Instead of almost completing all of the tasks, it's better to finish as many of you can and either postpone or throw out the tasks that aren't as important. You should also take the time as you're programming to improve your estimates. If a task that you estimated at 2 weeks ends up taking 3 weeks, you can use that knowledge to help when you're estimating future similar tasks. To keep track of these estimates and progress within the team, it's useful to have a team member who's job is to "track" the rest of the team. They are responsible for checking in with each team member on the status of their tasks, and be there to make decisions if there is a problem with development. For example, if a team member is struggling with a problem in their code, the tracker can make a decision about whether to assign the task to another person or change priorities.

And lastly, it's important to continually steer the project towards release. Keep track of how much work is getting done each iteration; if you can only finish 32 points in an iteration, don't plan to continue with the 40 points scheduled for the next iteration. Instead, you need to take out some user stories and adjust the task so it's on track for a successful release. And you always make sure that you choose the most valuable user stories to complete - in this way, you always have a successful project that could be released and be useful at any point in the stage.

I definitely agree with this process for updating your tasks and schedule along the way. It's not possible to always plan and design everything before you start doing the programming; it's so much easier to start programming and get a better understanding of the project and possible implementation avenues and then adjust your estimates. Being flexible makes it really easy to adjust and make sure your project is successful. And I definitely agree that when a programmer gets stuck on something that other people should step in. A lot of the time, you just need another set of eyes there that might be able to look at the problem from another perspective.

The Inmates Are Running the Asylum, Chapters 1 - 2

The Inmates Are Running the Asylum, Chapters 1 - 2
by Alan Cooper

     In the beginning of The Inmates Are Running the Asylum, Alan Cooper explains that any item crossed with a computer equals a computer. An airplane has lots of mechanical parts, including the engines, the seatbelts, and the bathrooms, but the total behavior of the system is controlled by the behavior of the computer inside. No matter if everything else that is mechanical or human is working correctly, the computer can have an error and halt the progress of everything else in the system. This is because current computer systems don't look at interaction with humans first. Whether it is an embedded computer system inside a submarine or airplane, or a desktop computer that you use at home, the programming and interaction was designed by a programmer. Because of this, systems fail and don't give proper warning messages, are hard to use in general, or do things and you can't understand why it's happening. Cooper argues that the solution is "interaction design" before programming design. We need to design the interaction of the system with its human users and how it communicates and behaves with them before we design how the system will be programmed.
     Cooper also introduces the concept of "cognitive friction." Cognitive friction results whenever the human brain and intellect has problems understanding the complex system of rules that is constantly changing in computer systems. For example, the numeric buttons on the microwave don't have one single function; when in "cook time" mode, the numbers correspond to the amount of time you want your food to be cooked, but when in "cook power" mode the numbers correspond to the amount of cooking power to be applied during that time. It is interaction paradigms like this that cause cognitive friction to the average user. The problem is that the people who design this interaction are programmers themselves; they understand how complex systems work because they have studied this field, and do not think about how users will understand the reaction. They are the "apologists," the ones who fight for software and explain all the good things that come from it, and their users are the "survivors," those who learn just enough to get by but never enjoy using the system. Cooper calls for an interaction designer that is not a programmer and that represents the public and possible users. Since software today is like a "dancing bear," meaning that it is amazing the software can do this task, but it does the task very poorly, having an interaction designer makes sure that systems are protected from the design of the programmers and can be made easier for average users.

     I really don't agree with a lot of things that Cooper says. I know that there are lots of error in programs that we write, and lots of times they can be devastating and fatal. But having more interaction design in the airplane navigation system that didn't alert the pilot of the fatal course change wouldn't have solved the problem. Extensive testing is more important for errors like that, where a key facet of the underlying functionality did not give correct feedback. But, I do agree that having an outside force in the construction of interactions in desktop and other low-maintenance software can be effective. This is because this person doesn't know the low-level details and can look at the system without bias. But, I really don't like the way that Cooper says programmers never understand real interaction. I am a programmer, but when I'm not programming, I don't view the world as a programmer. I am not able to understand some appliances or new things that I am exposed to just like normal humans. It has more to do with immersion and experience, and that a user needs to learn how to use the system. This is because not every complex system can be made a lot easier for all users.
    

The Mythical Man-Month, Chapters 1 - 3

The Mythical Man-Month, Chapters 1 - 3
by Frederick P. Brooks, Jr.

     In the first chapter of The Mythical Man-Month, Frederick Brooks, Jr., introduces the field of programming and the inherent benefits and problems with the field. Programming is rewarding and fun because we're building something tangible and something that users in the real world will experience. The act of creation of complex interactions and interwoven parts is intellectually stimulating, as is the learning process that takes place along the process of program development. But, programming large-scale products and systems usually encounter the "tar pit." Large-scale projects get sucked down and do not meet calendar deadlines, have to drop features, or may fail completely. Even the most ambitious and insightful programming projects can be claimed victim by the programming tar pit.
    Brooks explains that more programming projects fail because of calendar deadlines than any other reason. This is because estimating progress is extremely hard within a software team. Usually the team will confuse actual effort and time with progress, both because the teams are working hard but not actually producing useable code or because tracking scheduling and progress is done poorly. And when the project gets far behind schedule, the first thing that a programming manager will do is add more manpower. Brooks argues that men can not be interchangeable with months, and using the "man-month" to estimate software development is a dangerous and ineffective process. Because most large-scale systems have sequential dependencies, meaning that one part of the system or testing another part of the system is dependent on the progress of another worker, just adding more men isn't going to solve the problem. You will have to take time to train the new workers in the task you are completing, and the additional communication required (especially in a complex system) will actually make your task later than if you hadn't added any new programmers. The number of months required for a task depends on its sequential constraints, and you can't make that number any smaller by adding more programmers.
    The obvious solution is to do all programming tasks on smaller teams, to reduce the time needed for training and communication. But, with large-scale products that might need the equivalent of 5000 man-months to complete, this is not feasible. Brooks presents a model to cut a large system into separate tasks, and reduce the number of people that are in charge and have to communicate. The "surgery team" system is built upon the interactions in a medical surgery room: the surgeon is the main programmer, who does most of the work. The co-pilot is his right-hand man, ready there to give him advice and sometimes do some coding. We also have secretaries, language lawyers to give advice on possible implementation schemes with the programming language, and editors that manage the product's documentation. In all, the surgery team paradigm can reduce communication between 200 works to only 20 "surgeons," making the project more efficient.

     I thought the first 3 chapters were particularly interesting and insightful. These are the kinds of things that are written in the Extreme Programming Installed book that we're reading as well, but this book was written in 1975, when there weren't a ton of software companies. I agree that adding more people to a team makes a project much harder, because I am always more happy when I'm working on a small team or by myself. With larger teams, it's really hard to coordinate who is working on what, meeting times, and questions about code. I really like the idea that in a large-scale system we have lots of small groups that are assigned tasks. So that group can focus on their own task with minimal communication with other groups, and one person can be put in charge as a delegate between the other teams.