Background
Project: Implementation of Project Management and Quality Monitoring across Organization
Duration: 24 Months
Project Sponsor: Quality Team
Stakeholders: Quality Team, Operations' Team, IS Team (DevOps)
Team Profile: 8 members with average 2 years of experience in DevOps
Objective of Project:
1) Implement Enterprise wide solution to capture accurate data for Quality Audits
2) Monitor Improve operational efficiency of the organization
3) Reduce billing leakage in case of T&M projects
The Organization primarily comprised of two category of teams:
1) Software Development(SD) - Primarily responsible for design and development of Hardware + Software products used in various electronic components in Automobile, Aerospace, Medical Equipment and Locomotives.
2) Engineering Work (EW) - Primarily responsible for design and POC validation of mechanical parts used in manufacturing.
Img1: Stakeholders for providing inputs to Project
Quality Team:
There were two quality teams for each group: 1) Process Definition 2) Quality Metrics Group
Operations:
There was no separate Operations team for Software Development (SD) and Engineering Work (EW). Operations team had to provide inputs required for Operational Metrics.
DevOps Team:
The DevOps team worked in a War Room with an old white board and a Projector. The DevOps team was responsible for implementing Project Management and Quality Monitoring tool across the Organization. It received inputs from Operations and Quality Team. So as opposed to standard perception of moving releases to production, the DevOps team was also made responsible to test the releases, do product configurations and work as a coordinating team between the internal stakeholders and Vendor.
These inputs were provided via email or excel circulated via email. DevOps team like most of the teams, worked on Dev Server, was delivered on QA server for Acceptance Testing (AT) and then moved to production. The project was planned for three phases. The first phase was relatively smooth. The first phase was of 6 Months and comprised of basic tools which would allow Project Managers to create projects, form teams for the project, record time to be billed and be able to submit time for billing to customer. The second phase was of 12 Months and comprised of empowering the project managers with standard processes for specific project types (e.g. Development Projects, Maintenance Projects and Support Projects). The third phase was of 6 Months and comprised of generating Quality and Operations Metrics.
Challenges in Executing the Project
At the end of 6 months Phase I was delivered. Project managers started using the system. It was a completely new system for the Organization. Quality and Operations team started receiving critical feedback regarding usability and need for additional features. This was during Phase II. A decision was taken by Quality Team that new features to be rolled out to the system in incremental mode instead of waiting for something BIG to happen. Due to this DevOps team started receiving continuous inputs for Phase II features and feedback on Phase I features from Quality and Operations Team. These feedback/inputs were provided to DevOps team via eMails and in many cases the eMails were being circulated directly to the DevTeam. Many a times the Project Lead was completely unaware of what has to be done. This lead to chaos in prioritizing the features to be released.
Img2: Multiple inputs provided to DevOps Team by Quality and Operations Team
That was not end of the misery for the team, deploying the features involved implementation of the Code Changes + Database Changes + Application Configuration + XML File Configuration. This led to long testing cycle and delay in the go live of the new features for end users. Sometimes the team missed to test a piece of functionality and its released to users. To bring the situation in control the team started using inhouse Issue tracking tool. But, things still slipped resulting in customer dis-satisfaction and frustration. The IS Head started receiving multiple complains from Quality Team regarding Quality and Efficiency of DevOps Team. Certain issues were escalated to IS Head and he was unaware of many issues. The team got scared and was unable to communicate when IS Head started inquiring about individual items and what each person was working on.
Solution
Kanban Implementation 1.0
An experiment was conducted for a smaller release to implement Kanban. A basic Kanban board was implemented with buckets of Priority, WIP, DONE. The board looked dirty as there was limited space available for Kanban Experiment. The white board had to be used for discussions!!!
An experiment was conducted for a smaller release to implement Kanban. A basic Kanban board was implemented with buckets of Priority, WIP, DONE. The board looked dirty as there was limited space available for Kanban Experiment. The white board had to be used for discussions!!!
Img3: Kanban Implementation 1.0
Each team member was notified that anything that they are working on should be one of the sticky notes. Color coding was implemented to keep track of Phase/issues. Green/ Yellow was for Phase-II features. Blue was for feedback. Purple was to indicate that information is needed by the team to progress. And most important was the Pink notes. They indicated issues or defects reported by Customer. Initially there was resistance from the team. Team was not comfortable with publicly displaying tasks they worked on and the progress of the tasks. But, IS Head was happy to see transparency achieved from the Kanban board. The escalations reduced as the Red Corner was given higher importance and resolved ASAP. Tasks were prioritized and tracked efficiently.
Kanban Implementation 1.1
After seeing significant benefits of Kanban, team decided to use online Kanban Tool (SwiftKanban). Many cards were created and the implementation project was moved to SwiftKanban for Tracking. The keenness to use new tool for tracking was exciting. But gradually the charm faded as it was cumbersome to log-in to the online tool to update the card status and get back to work. The major downside of the online tool for the team was that the Red Corner was not always visible. The promptness to resolve issues reduced. Things started getting slow. By this time, the team had better understanding of the significance of Kanban in Project Execution.
Kanban Implementation 2.0
Dedicated white board was provided to create Kanban Board. Resource utilization was tracked across the track and each step of Development, Testing and Production Deployment was created on the board.
Img4: Kanban Implementation 2.0
The width of each lane was carefully selected so as to limit the number of cards in that specific lane. The person on the top was relatively new to the team hence was given limited space. She was able to validate/test maximum 2 features at a time while other team members were able to test 3 or 4 features at a time. Issues related to any features were moved together and the corresponding cards were stuck together. The pre-production done lane contained all the features that were ready for the current release and can be moved to production. Once the pre-production lane gets significant number of cards, a production release was made.
Img5: Cards waiting in Pre-Production for Production Deployment
Since the scope and priority was clearly defined team was able to focus on the task to be executed and tasks were getting completed quickly. The time from pre-production to production reduced to 1 day as against pre-kanban time of 3 days (66% reduction in deployment time). This was because team members did not juggle between different tasks and the scope was clearly visible in the pre-production lane. Errors were reduced as nothing got missed in deployment.
After 2.0 implementation of Kanban, IS Head didn't have to ask individuals about the progress of the project as everything was visible on the board. Issues got resolved quickly and morale of the team was boosted for good.
