A project model for the FreeBSD Project

UNIX, originally developed at Bell Labs in 1969, was rewritten in C, rather than the original assembler, in 1973 by Dennis Richie and Kenneth Thompson. Since its presentation, professors at Berkeley had taken a strong interest in UNIX. With their first running UNIX machine early 1974, strong links were built to the UNIX communities at Bell Labs. Old ties to batch processing systems made the UNIX running machine have to run a batch job system for 16 hours and UNIX for 8 every day. With the success of Berkeley's Ingres database written for UNIX and increased demand by students, the computer science department ordered a new machine to run UNIX only.

Together with the new UNIX machine, Bill Joy and Chuck Haley started as graduate students. They wrote a Pascal system and a number of utilities and editors such as 'em' and 'vi'. The Pascal system became popular among the computer science students because of its good error handling system. Joy and Haley took an interest in the kernel when installing a tape with 50 changes to the kernel supplied by Bell Labs. These changes together with the Pascal system and their utilities was put together to form the "Berkeley Software Distribution" by Joy early 1977. This feedback and changes together with new features such as extensive support for terminals led to the second version, known as 2BSD in 1978.

As the need for larger address spaces for user programs came, the computer science department ordered the newly announced VAX-11 in 1978. Since the department was used to UNIX, and UNIX with virtual memory support was not available for the VAX, Ozalp Babaoglu and Bill Joy ported the virtual memory support to V/32, the VAX UNIX version. Seeing that this architecture would obsolete the older computers in the department, Joy began porting the 2BSD software to the VAX. The new kernel enhancements and utilities led to 3BSD, the first VAX distribution from Berkeley.

With 3BSD, Berkeley had proved its ability to create and maintain a working system and was able to land an 18-month contract with DARPA to create the operating system that should run on multiple hardware platforms so that DARPA would have a unified platform at the operating system level. This provided funds to employ Laura Tong to do the project administration. She set up a distribution system that could ship a much larger number of copies: 150 for 4BSD and 400 for 4.1BSD. Being satisfied with the results, DARPA continued its funding and a steering committee was set up to guide the design work and ensure that the needs of the DARPA research community were met.

Apart from being a credible actor, the research community needed the CSRG at Berkeley and BSD. Bell Labs and its UNIX had long had the role as a clearing house for computer science research. Many researchers were interested in UNIX which made their research results often either UNIX specific or easily integratable into UNIX. But with Bell Labs' commercialisation of UNIX in 1979, Bell Labs could no longer play this role, and the CSRG soon stepped into this role. [McKusick, 1992]

The work to implement the requests required much redesign and implementation of TCP/IP based network services. This led to an internal intermediate release, called 4.1a. However, many people grew impatient waiting for 4.2BSD, and this led to many copies of the system. Based on the feedback from 4.1a, a proposal for the new system, called "4.2BSD System Manual", was made. It contained a concise description of the proposed user- and application programming interfaces.

When Joy left for Sun Microsystems in the late spring of 1982, Sam Leffler was given the responsibility to finish the project that had been promised to the DARPA community by Spring 1983. To conform to this deadline, the remaining projects were evaluated and strict priorities were set. April 1983, an intermediate release, 4.1c was released, and after reworking the I/O system and making the install process easier, 4.2BSD was released in August 1983. [McKusick, 1985]

As with 4BSD, many people were critical to 4.2BSD because of performance issues. As with 4.1BSD to 4BSD, 4.3BSD, released in June 1986, would resolve the issues causing the stir.

A requirement for using BSD was that a System V license was held. AT&T had increased their fees much and vendors who wanted to use the TCP/IP-based products that Berkeley had pioneered, lobbied CSRG to provide them under terms that did not require such a license. In response to this, Networking Release 1 (Net/1), consisting of the original networking code and supporting utilities, was released in June 1989 under a liberal license. While Berkeley charged a $1000 fee to provide a tape, the code could be redistributed by anyone as long as the copyrights in the source code was kept and the use of Berkeley's source was acknowledged in the documentation. This led many sites to provide public access to the source through the network, and this made the release very popular. Also, institutions regarded this as a way of supporting CSRGs research, and many hundred tapes were bought.

Inspired by the popularity by Net/1, Keith Bostic suggested that making a release that included more BSD code. However, Marshall Kirk McKusick and Mike Karels noted that this would require hundreds of utilities and libraries to be rewritten and the kernel to be closely examined and partly rewritten. Rather than giving up by the size of the effort, Bostic pioneered large-scale net-based development by approaching people to rewrite Unix utilities based only on their published descriptions. With his continued effort to solicit people, within 18 months almost all the utilities and libraries has been rewritten. Keeping McKusick and Karels to their promise, CSRG went through the kernel code file by file and removed code originating from 32/V. This effort culminated in the release of Network Release 2 that included a fully working system except for 6 files (out of 18.000) that had been deemed to difficult to rewrite.

As with Net/1, Net/2 became widely spread. Within 6 months, a version bootable on the Intel 386 Architecture had been made by Bill Jolitz who had rewritten the 6 files. This sparked communities that would lead to the major BSD open-source projects at the time of writing: NetBSD, OpenBSD, FreeBSD and Darwin.

Based on Jolitz six files, the company BSD Incorporated started shipping a commercially supported version. They marketed it as being a system as costing only 1% of what System V cost with source and binaries. Unix System Laboratories (USL), which was mostly owned by AT&T and owned the Unix trademark, quickly demanded they stop marketing their product as Unix. BSDI complied, but still unhappy about the competition, USL filed suit to stop BSDI from selling their product, claiming that it contained USL code and trade secrets. BSDI defended that they only used code publicly available from Berkeley and 6 other files, which led USL to include Berkeley and its Net/2 in the suit.

After six weeks of advisement, the judge ruled in January 1993 that only two of the complaints should be brought to court. These complaints should be heared in a state court rather than a federal court. Berkeley followed up with a counter-suit against USL for using code from Berkeley without stating so in their documentation. Soon after the filing in state court, USL was bought by Novell whose CEO, Ray Noorda, stated publicly that he would rather compete in the marketplace than in court. A settlement was reached in January 1994 that resulted in three files being removed and numerous minor changes and added copyright notices. With the settlement, USL promised to not sue any organisation that used the new release called 4.4BSD-Lite. [McKusick, 1999]

4.4BSD was the final release from CSRG. The three reasons were

[McKusick, 1994]

March 29th, 1992, Karl Lehenbauer published the FAQ for the 386BSD, which is by many recognised as the start to make the Berkeley Net/2 release available on the Intel architecture. Version 0.1 was made available in June and a community grew quickly. However, the author, Bill Jolitz, did not maintain the software much. This led Jordan K. Hubbard, Nate Williams and Rod Grimes to create the "Unofficial 386BSD Patchkit". The maintenance of this kit, however, came to a halt when Jolitz withdrew his sanction from the project. In response to this, the FreeBSD Project, name coined by David Greenman, was started.

The first FreeBSD distribution was released in December 1993, based on 4.3BSD-Lite but also included work from 386BSD and by the Free Software Foundation [Kolstad, 1994]. After being updated to version 1.1 in May, 1994, the the project was a success. However, at this time the lawsuit between AT&T and Berkeley had been settled, and the project had to switch to the 4.4BSD-Lite code base from Berkeley, leaving the encumbered 4.3BSD-Lite code behind. Because the Intel version of 4.4BSD was highly incomplete, it was not until November 1994 that the project could release version 2.0 [Hubbard, 1994].

Since version 2.1, FreeBSD has enjoyed a reputation as a stable, feature-full UNIX system for the Intel platform. Kirk McKusick describes the evolution of FreeBSD as a random path walk. This is because new features have been developed on the initiative from the developers rather than through desicions by the core team. The project has since its early days been split it two main branches. FreeBSD-STABLE is the branch that maintains the current working release that is offered to the public for production use. FreeBSD-CURRENT is the branch where the main development happens.

The FreeBSD Project was hosted and to a large degree sponsored by Walnut Creek CDROM during its initial years. In March 2000, BSDI Merget with Walnut Creek CDROM to form a united front for the BSD opertaing systems. In April 2001, Wind River bought Walnut Creek CDROM to aquire the rights to BSD/OS, and desided in October 2001 to end its relationship with FreeBSD, a relationship it had got when aquiring Walnut Creek CDROM [Reed, 2001]. The FreeBSD trademark and related rights were transferred to FreeBSD Mall Inc in January 2002.

During requirements analysis, the requirements are interpreted by the developer and an alignment of the understanding of the problem by the developers compared to the formal definition of the problem laid out in the requirements. This stage often needs the transformation of requirements written in a human language (for instance English) to a common technical and unambiguous notation, often one that can be interpreted by computers. Such models will not only align the understanding of the problem at hand between developers, but also determine whether the requirements are complete, contradictory or too ambiguous.

Design consists of forming an overall architecture for the project and reducing parts of the model to sub-problems that are tractable and simple enough in their complexity to have the full scope of the sub-problem in a developers mind at a time. For this, technical figures, design document, interface descriptions, user interfaces and such are created. A very much used tool for the design stage that includes all of these are use cases.

The only phase that is certain to be within a successful development effort is the implementation. Even with the most sophisticated Computer Aided Software Engineering (CASE) tools used for design, there are always parts that have to be implemented in code.

Verification is the verifying that the developed system does as designed, that the design completely covers the requirement and that the system developed thus completely covers the requirements. Also, testing that the program does not misbehave and behaves predictably is a part of this phase.

Integration is the deployment of the system at the customer, seeing that it integrates in the environment of the customer.

The software engineering skill that is probably least taught at university to computer science student is software maintenance. The absence of this process in many software process models shows the way it is regarded in the software industry. [Boehm, 1973] states in his article that almost 40% of the software effort in Great Britain in 1973 was maintenance, and he predicted the figure to rise.

According to [Canning, 1972], causes for initiating software maintenance are "program won't run," "program runs but produces wrong output," "business environment changes", and "enhancements and optimisation". [Swanson, 1976] classifies maintenance into corrective maintenance, adaptive maintenance and perfective maintenance.

Corrective maintenance is due to processing failures (i.e. abnormal termination, erroneous output in reports or files), performance failures (not meeting the performance criteria specified) and implementation failures (implementation not following coding standards, not adhering to design etc).

Adaptive maintenance happens due to environmental changes. Especially, change in data (i.e. a logical restructuring of the database used) or processing environments (new hardware or software architectures implemented) lead to adaptive maintenance.

Perfective maintenance is done to improve the maintainability of the code, processing inefficiency (i.e due to poor use of the operators time) and performance enhancement (increasing readability of reports, adhering to new data presentation standards).

In his article, [Swanson, 1976] writes about the importance of keeping a maintenance database. A maintenance database is a database that keeps track of all problem reports, preferably both submitted by users and developers. This database should cover all issues that are interesting for the project to track. Such items are usually the problem description and a detailed description of how it occurred, preferably with debugging information. But it can also include who is responsible for the bug, where in the code it was found, how long time it took to be resolved and so forth.

[Canning, 1972] has written on the "maintenance iceberg" that is due to bad coding practises. Since then, people have been taught what is considered better coding practises and more high level programming languages have become available, thus ideally making software more maintainable. However, the maintenance issue has not been reduced. But just as [Canning, 1972] dared not measure the iceberg in 1972, I have not found articles saying how large the iceberg is today and how it has evolved since 1972.

What is true is that the software engineering field has become more maintenance aware and authors like [Sommerville, 2001] have included it in the description of modern software process models. Indeed, many agile software process models are said to be maintenance centric, seeing every change as a maintenance change.

The world of open source is also aware of the maintenance issue. [Tatham, 1999] writes on reporting bproblems effectively, enabling the author of a program to find and resolve problems. The open source community has been good at using maintenance databases like GNATS and Bugzilla. Indeed, open source hosting sites, such as Sourceforge, often come with bug reporting, version tracking and user discussion systems allowing for ongoing maintenance in all active projects.

With the introduction of assemblers and compilers for higher level programming languages, it became possible to write large systems. Problems of testing and documenting the systems led to the introduction of the "stepwise model". This model has eight stages that software development needs to go through in a linear order. The model is heavily document-driven: Royce gives the example of 15.000 pages for a system consisting of 100.000 instructions. [Benington, 1987]

The "waterfall model" is a refinement of the stepwise model. The two primary enhancements are feedback loops between the stages and the building of a prototype. The feedback loops is the realization that the current stage is an evaluation of the work done at the previous stage, and findings in this stage can force us to go back to a previous stage to do changes. The "build it twice" philosophy of the waterfall model builds a prototype to explore difficulties such as technical uncertainties or to gain understanding of how the user wants the interface. The main problem with the waterfall model is the criteria of the completion of fully elaborated documents. While this criterion is good for products that can easily be formalised, such as compilers and spacecraft controllers, this does not work well for end-user applications. Since the uses and interfaces of end-user applications are often poorly understood by the development staff in the beginning of a project, there is no purpose to writing the specifications in great detail. [Royce, 1970]

The "evolutionary model" assumes that there is software out there that with modifications can become the software the user wants. This approach matches well with situations where the user knows he or she needs a product but is not certain about how this product should be. This kind of development has been made less costly with the introduction of fourth-generation languages. There are mainly three problems with evolutionary programming:

[Boehm, 1988]

The transform model approaches the problem of spaghetti code that easily results from the many of the before mentioned models. It assumes that formal specifications can be transformed into code. It is an iterative process where the first goal of the first iteration is to make a formal specification with the best initial understanding of the system. The following development iterations are spent changing the specifications based on the customers input and optimising the code by giving more detailed guidelines to the tool converting the specifications into code. When the program is deployed, further iterations are used to adjust the specifications based on operational experience. In this way the transform model bypasses the problem of poorly structured code that is hard to modify. While this sounds very promising, it has proved hard to build tools that will convert a specification into complete code for other than just small projects. [Saers, 2002]

The "incremental model" [Mills et al, 1980] gives the customer the ability to delay decisions until he has some experience with the system on which he can make the decision. The customer identifies which parts of the project are the most important to them, and these parts get developed in early increments. Each increment is a delivery that the use customer can put into production. Each increment is a small waterfall model, and this way specification, development, verification and validation can be processes that come over and over. The customer will get the necessary documents to keep an overview of the project status. The use of increments, the system being put into production and the customer overview minimise the risk of the project failing due to miscommunication. One major problem with the incremental model is that it is difficult to estimate how large an increment should be. Also, since the requirements do not come all at once, it is difficult to identify and build general structures that will make it easier to code the system.

The "spiral model", known from [Boehm, 1988], is an incremental model focused on risk. It is illustrated as a cycle going outwards where each cycle involves risk analysis, prototyping, simulation, design, validation, planning the next cycle and evaluating alternatives. The risk analysis, prototyping, simulation, validation, and alternative evaluation all work towards minimising risk of project failure, and the cycles are short enough that if the result of the analysis is that the project should be halted, then there is an opportunity to halt it at the end of every cycle. As with the evolutionary model, prototyping is used heavily, ending in an operational prototype given to the customer. [Saers, 2002]

The most resent addition to our repertoire of models are "agile models". These models focus mainly on handling change and the most well known of these, extreme programming, defines change as the norm and stability as an abnormality. Extreme programming is a set of 13 practises that together reduce project risk through improving the response to change and improves productivity through making building software in teams more fun. It can be summarised as a series of many stepwise models where the time to go through the model is at most a couple of days, usually about a day. In practise, this means that design, implementation and verification are ongoing processes. [Saers, 2002]

Third party software that is intended to run on the system should not be unnecessarily encumbered, as such encumbrance can lead to it dropping support for the system. Such an act would lead users that require new releases of this software to prefer using systems that are supported.

Does much third party software run on the system?

To measure whether third party software runs on the system, we will look at how much software is officially said to be supported by the system, and how much of this that actually runs at the time of writing.

As the technology evolves, so will the needs of the users and developers as their technological fantasy will continue to suggest new uses that they think should be possible. Some of these new uses require new features to be added. These should be developed, or users will move to systems that provide these features and the momentum of the project will decrease.

Are new features implemented? Do new features deprecate old features?

To measure if new features are implemented, we will investigate release announcements and interview some of the developers in the sub-projects that created the new features.

In order to stay visible at conventions, to give an impression of progress for people worried about version numbers and in order to supply new users with the most up-to-date version, releases should be made regularly.

How often are releases made?

The release history will be examined and how many days there are between the different releases will be examined.

In order to stimulate to further recruitment to the project, commit privileges should be given to active contributors.

Is there a steady flow of new committers coming into the project?

This will be measured by monitoring how many committers have been added per month during the past year.

An inactive committer, that is a committer that has not performed a commit during the past 18 months, should have his commit privileges revoked. Commit privileges are also a security risk as they can be used to cause harm to the code. Dormant privileges can be picked up by intruders, and thus be a security problem for the project.

Are commit privileges regularly reviewed and revoked for inactive committers?

This will be measured by monitoring how many committers have their commit privilege removed and how regularly this is done.

For the project to be able to maintain its momentum, it is imperative that it is easy for developers to add their code.

How many commits happen? How easy is it for new committers to make commits?

To determine how many commits happen, we will watch commit logs for a year and split them into fit categories. To determine how easy it is for new committers to make commits, we will watch commit statistics for new committers for a three month period after they receive their commit privileges.

One of the main features of the FreeBSD project is its frequent releases. These releases are distributed by CVS, and users synchronise with these updates through a program called CVSup. This requires CVSup servers they can connect to, which in turn requires an infrastructure to handle the requests.

The CVSup system is by far the FreeBSDs project's largest logistical system. In addition, the project provides CD-ROM images that are used to produce installation CD-ROMs, either by companies that sell them for a profit or by individual users.

How many CVSup servers are there available? How long do users have to wait in order to use a CVSup server? Are CD-ROMs and DVDs easily available to users?

CVSup servers will be counted over time. An access simulation will be performed in order to estimate the availability of the servers. Sales statistics will be gathered and aggregated to measure the availability of discs.

There will inevitably be problems in any software. Therefore there should be ways that those who have not written a particular piece of software but experience trouble with it can alert the author about the problem, and possibly submit a patch.

With most of FreeBSDs users not being developers, it is very likely that they will identify problems that the developers do not know about. It is therefore important to have a standard way of error reporting that simplifies the task of duplicating the error so that the author can find and fix it.

In addition, there is little use in problem reporting without the problems being reviewed and solved. Ideally they should be resolved quickly so that the report originator sees that it is useful to report errors and is thus be stimulated to send further reports about other errors.

Are problem reports sent to the author? If so, how are they sent to the author? Are problems reported given attention? Are the problems resolved? If so, in what time frame are they resolved? How many problem are being worked on? How many problems are being resolved? How long does it take from a problem report is received until it is assigned to a committer? How long does it take before a problem is given attention? How long does it take before the problem is resolved?

To measure whether problem reports are sent, we will look at the frequency of problem reporting. Through interviewing a sample set of committers we will determine if it is common to receive problem reports in other ways than the standard format. To measure whether problem reports are given attention, we will see how quick the status of the report is changed from its initial state. We will also check how many have reached the state "closed". To measure how long it takes before the problem is assigned to the correct committer, the submit date and the first time the responsibility for the report is changed will be measured. To measure how long it takes before the problem is given attention, we will check the first time the state was changed from "open". To measure how long it takes before the problem is resolved, the time the report is submitted will be compared to the time the report is closed.