IN MY first article on Extreme Scoping, I recommended an Agile software release approach for EDW/BI projects. I also mentioned that the project management function is performed by a four to five member core team and not by a single project manager. In this article, I will elaborate on the different team structures on EDW/BI (enterprise data warehousing/business intelligence) projects and their interactions with each other. But first, let's start with a quick review of a typical traditional project team.
Traditional project teams depend on the project manager to coordinate and assign tasks to individual project team members and to track and report the progress of the project. The project manager also reviews the work deliverables and makes project-related decisions.
The individual project team members are responsible for their own task deliverables, which they hand off at the appropriate time. For example, the requirements analyst gathers the application requirements, which are then handed off to the data modeller who creates the logical data model, which is then handed off to the database administrator who produces the physical data model and builds the database structures, which are then handed off to the developer who codes the application.
The only collaborative interaction that involves all team members at the same time happen once a week during the status review meeting. This approach may have worked on very large, multi-year projects that use traditional methodologies and have 20 to 30 people on a team, but it is completely ineffective for Agile EDW/BI projects that must deliver in 30 to 90 days.
The core team should be staffed by a seasoned project manager; a business representative (end-user or subject matter expert) who has the authority to make business decisions and to set policies; an enterprise information management (EIM) person like a data administrator who is trained in data administration disciplines, such as logical data modelling, normalisation rules, data standards, taxonomy; and at least one senior technical person (lead developer, database designer, or technical architect) with strong programming and technology skills.
All core team members together comprise the project management team, which replaces the single project manager function. The core team members manage all software releases of an application, thus ensuring continuity and knowledge transfer. Core teams are self-organising "SWAT teams" where all members of the team (together as a team) coordinate and assign tasks to each other an to members on the development track teams.
They review each other's work, collaborate on project issues, and make project-related decisions. The entire core team meets every day for a status review, and the core team members take over each other's tasks when needed, for example, when a team member is out sick.
A core team must be very small; a team size of three to five people (per project) is optimal. Project core teams should never exceed seven people per project.
Development Track Teams
As Figure 1 implies, development track teams are extensions of the core team, but they do not participate in making daily project management decisions. They do, however, function similar to the core team within their own tracks.
The most common development tracks are ETL, front-end BI application development, and metadata repository. If metadata is not a deliverable, then at least two development track teams work in parallel, namely ETL and BI application development.
On the other hand, there could also be more than three development track teams, such as an additional data mining team.
The development track teams are led by the lead developer, database designer, or technical architect from the core team. You may also choose to have each development track team be led by a different lead developer, if that better supports your organization and if you have the resources.
Development track team members should be developers (or a senior systems analyst) who have the skills needed for their particular track. They only participate in track-specific tasks for the duration of their track activities.
A development track team is even smaller than a core team; a team size of two to three people (per track and per project) is optimal. Development track teams should never exceed five people per track per project.
The extended team members play traditional roles and participate on the EDW/BI projects on an as-needed basis, not on a full-time or daily basis. Members of the extended team may be heavily involved only at certain times during the project, or they may be called intermittently for their expertise and contributions.
Extended team members include technicians and business people who contribute in much the same way they do on operational projects. Extended teams include support roles such as technical support, operations, security officer, IT auditor, the business sponsor, and other stakeholders.
The core team, the development track teams, and the extended team make up the complete EDW/BI project team structure.
BI steering committee
The BI steering committee is an advisory body of business executives and senior business managers, who understand EDW/BI, who support enterprise-wide activities, and who provide collective sponsorship.
They meet on a regular basis to discuss, plan, staff, and fund business initiatives, such as EDW/BI, MDM, CRM, CDI, DG, and so on. They stand behind an EDW/BI strategy that supports the business drivers. They fund an EIM group to perform enterprise-wide data governance activities. They identify data owners and direct the data owners to identify data stewards in their line of business.
They also free up business people from their operational responsibilities so that they can participate as full-time members on the EDW/BI project core teams.
BI programme manager
The EDW/BI programme management office is led by a BI programme manager (director) who works directly with - if not for - the BI steering committee. The BI programme manager performs periodic readiness assessments to identify new information needs and to ascertain end-user satisfaction with the EDW and the BI applications.
With the backing of the BI steering committee, the BI programme manager creates a BI strategy and enforces the common technical and non-technical infrastructure components in the EDW/BI environment.
Working with the BI steering committee, the BI programme manager prioritises EDW/BI projects, determines the project interdependencies, and coordinates the project resources and activities around these interdependencies.
Team interaction and communication
The dynamics of the core team or the development track teams are not the same as that of the extended team or a traditional project team.
The core team is like the SWAT team at the US space agency Nasa. Although they have their assigned cubicles and offices, they do not spend most of their time in them. Instead they collaborate, brainstorm, solve problems, and make decisions together in a "war room" dedicated to their project.
For example, the core team meets every day to review status and deliverables and to discuss roadblocks and solutions. Sometimes these meetings last 10 minutes, at other times working sessions could be scheduled for half a day. During these sessions, individual assignments are distributed to the appropriate team members who will work on them right after the meeting and report back the following day.
If major roadblocks are encountered, they are addressed immediately and alternative solutions and contingency plans are prepared. The project manager and the business representative will take the alternative solutions and contingency plans to the business sponsor for negotiation and/or a decision.
The development track teams function similarly to the core team, only on a smaller scale. They also meet every day to review the status and deliverables within their own track.
They collaborate, brainstorm, solve problems, and make decisions together on their track-specific issues. In fact, they function similar to Scrum or XP teams, sharing the prototyping activities of analysis, design, coding, and testing.
Communication with the core team is automatically established through the lead developer, database designer, or technical architect, who is managing the development track team(s).
The extended team meets with the core team once a week, usually on the same day and same time of the week. The core team members set up the meeting schedule at the beginning of the project, and the business sponsor sends out the notification to the extended team members.
The meetings are scheduled for one hour, but could be as short as 10 minutes if no major issues are to be discussed. The purpose of the weekly meetings is to keep all extended team members current with activities, status, and issues of the project.
In summary, if you want to go Agile with your EDW/BI projects you must also convert your traditional project teams into Agile, self-organising SWAT teams.
The key is to keep your project teams very small. Transfer the project management activities to the core team members (collectively). Be sure you have a business representative on the core team.
Your development track teams can have as few as one developer in each track (per project) or as many as four or five.
And finally, include the extended team members on your project on an as-needed or as-scheduled basis.
(Larissa T. Moss is founder and president of Method Focus Inc. She has over 30 years of IT experience, with over 20 years in Data Warehousing and Business Intelligence.)