The PostgreSQL Development Process
Bruce Momjian
PostgreSQL Core Team member / EnterpriseDB
Abstract
The Postgres development process is both typical of other open source projects, and different based on the complex requirements of creating a reliable database system. This paper discusses the Postgres development model and explains why the development processes are used.
Introduction
Since 1996, PostgreSQL has been developed by an increasingly large group of volunteers. Made up of some of the best database developers world-wide, they use the Internet to coordinate their activities in creating the world's most advanced open source database.
The community development process is difficult to understand because it does not have the traditional structures associated with development teams --- it has no single authority, no bug tracking system, no development roadmap, and no release target dates. Some of these are absent because of the open source nature of the project, and others are lacking because an informal structure has been found to be more efficient and realistic for the project.
No Single Authority
There is no single authority among the Postgres developers. Rather, all discussion happens on the developer email list, and anyone can post their opinion. This allows for a much larger pool of people to be involved in complex decisions. This has two advantages --- it encourages experts in the discussed topic to give their input, and it allows new people to feel involved because there are no barriers to their involvement in the project. Often complex decisions are swayed by people who are not regular contributors to the project because they have insight or experience that the more estabilished developers lack.
No Bug Tracking System
Postgres has no bug tracking system. Typically, bug tracking systems are part of a larger development ecosystem which involves users submitting bug reports to the bug tracking system, bug tracking managers culling the submitted bug reports, removing invalid bug reports, and directing developers to work on specific items. In their worst incarnation, bug tracking systems are a dumping ground for bug reports that are recorded but rarely resolved.
Postgres uses the email lists to collect bug reports, which allows the bug tracking manager job to be distributed among the email subscribers. Because only a small percentage of bug reports are actual bugs, email subscribers can work with bug submitters to clarify the database behavior they are seeing. Legitimate bugs are usually easily seen by experienced developers and either fixed or added to the TODO list.
No Development Roadmap
As a community project with no assets, it is impossible to reliably direct development efforts. While the development team can try to sway developers to work on specific items, feature development is very closely aligned to the developers who contribute their time to the project, and employers who sponsor development. Having no development roadmap allows the project to continually adjust requirements as user community demand changes.
Development primarily happens via email communication on the hackers email list. The list receives about 85 postings a day relating to feature discussion, patch review, and bug fixing. The master source code repository is stored using CVS. There are about 15 active people who process patch submissions from the larger community.
No Release Target Dates
A database must be reliable, or it's useless. While the primary developers attempt to have major releases every 12-14 months, the reliability of the code determines when the code is ready for release.
The Development Process
A typical development week consists of discussion by perhaps ten long-time contributors working on major features, several first-time contributors posting their patches for review, and several dozen bug reports that have to be diagnosed and possibly fixed or added to the TODO list.
Every two months, a CommitFest processes all patches that have been submitted but not yet applied. CommitFests usually last for one month. After roughly nine months, the CommitFest process stops and all outstanding issues are gathered so they can be addressed and the first major beta released. There are usually several beta releases until the code is considered sufficiently bug-free that a release candidate is produced. If the release candidate receives good field test reports, a final major release is made, and a few weeks later, the development process starts again.
Supporting Tools
For complex database source code, regression testing is considered insufficient to discover most bugs. Rather, patch review is considered to be the primary means to make sure code additions will fix bugs, rather than add new ones.
Several tools help keep the code organized and reliable. The Postgres Buildfarm highlights unportable code constructs that need fixing, and developer tools help to keep the code consistently indented. There are also many documents to help new developers get started. The Developer FAQ is the best place for new developers to start.
New Developers
Postgres development often frustrates new developers because of its stringent requirements for SQL standards compliance, code cleanliness, and reliability. The Postgres developers inherited a ten-year-old code base in 1996 from the University of California at Berkeley. Inheriting a mature code base at the outset reinforced the importance of these requirements for long-term code maintenance. Unlike most development organizations, the Postgres developers are not focused on the immediate future, but rather how things done today will affect the database years from now.
Conclusion
Some of the Postgres processes are shared be other open source projects. Others are unique, based on our reliability requirements, long-term code maintenance requirements, and the complexity of database software. Though the Postgres process surprises some new people, it works well, and many people have come to appreciate the system that has evolved over 15 years of development.