Mitel Automation Case Study

Mitel: We Stay with XBOSoft for Its Dependability, Agility, and Responsiveness

Customer satisfaction is a top XBOSoft priority. Now in our 10th year of providing quality software testing services around the globe, we want to ensure we continue to provide our customers with what they need when they need it.

We recently reached out to Peter Menhart, Senior Lead Software Developer at Mitel, to get his feedback on his 7-year relationship with XBOSoft. In Spring 2016, Menhart’s division at Benbria was bought by Mitel, but, because of his positive experience with XBOSoft, our company’s commitment to continue the partnership, and our depth of experience, XBOSoft continues to support Menhart’s product, Mitel Mass Notification, for the unit’s software quality assurance needs.

Based in Ottawa, in Ontario, Canada, Mitel is the world’s fastest-growing provider of cloud communications, making more than 33 million cloud connections daily. Mitel serves more than 60 million users in more than 100 countries.

Here’s what Peter told us about his experience with XBOSoft as a trusted testing partner:

“A CEO who’ll make an effort to visit you once a year to make sure you are happy with their services is at the top of my personal list [of recommendations].”

— Peter Menhart, Senior Lead Software Developer, Mitel

Q: Why have you continued to use XBOSoft’s services over the years?

Peter: There are several factors that interplay:

XBOSoft delivers what we need.
Flexibility: We can ask for urgent tasks on short notice.
They can tackle new requirements (e.g. new type of tests or new technology). We can rely on their work.
Amount of know-how accumulated over time. Also very good knowledge-passing mechanism when there are personnel changes in their team.
Price seems to be right.
Attentive CEO Philip Lew, committed to customer satisfaction. Over the years, our working relationship grew into a personal friendship.

Our product was acquired by Mitel last April, and there was pressure to switch QA to internal resources. Considering all the factors, the 7-year relationship with XBOSoft, and their quality of work, a decision was made that continuing with XBOSoft was the best solution.

Q: What was the quality issue that you faced when you initially connected with XBOSoft?

Peter: Our cooperation started in April 2010. XBOSoft was informally referred to us via some networking we did in New York City. We were a startup at the time with frequent releases, but less frequent testing. We prepared a “test plan” with very brief descriptions of test cases, but we didn’t have the manpower to execute them fully before each release. This led to quality issues (e.g. regression bugs where new features negatively affected “stable” areas). These factors led us to have a need for a company like XBOSoft

Q: Did XBOSoft immediately resolve your issues, or was there a learning curve? If the latter, how did XBOSoft’s team adjust to better address your issues?

Peter: We initially requested manual testing following our developed test plan, and XBOSoft delivered what we needed. As the process developed, adjustments were needed, as our requirements grew over time. XBOSoft had to develop detailed steps, add performance and stability tests, automated front-end tests using Selenium/WebDriver, and API tests using JMeter.

In addition, they had to learn how to install/ deploy/maintain/troubleshoot our software, which was different from the typical premise-based deployment model. Usually, developing a new type of activity requires a lot of guidance on our side. But, once the XBOSoft team learns the skills, they are pretty good at maintaining the knowledge, or applying it to similar types of problems. This eliminated the need for internal folks to spend time guiding XBOSoft.

Q: Most of XBOSoft’s testers are in a different time zone from you — did this have any effect on your work with them? How did you overcome the timing/distance factor?

Peter: Time and distance turned out to be an advantage. We could prepare the requirements toward the end of our day and if there is any urgent problem or clarification, I can still communicate with XBOSoft in the evening. Then XBOSoft would have test results ready for us when we came in the next morning. After that, we would analyze/provide feedback the next day.

We tried communicating through voice conversations first, but found written communication to work more efficiently. We are always interfacing with the team leader (although there have been a few personnel changes over the years). However, XBOSoft has a good knowledge-passing mechanism that works well.

Q: If you were to recommend XBOSoft to another company, what three attributes about XBOSoft would you highlight?

Peter: Choose from the list above… Having a CEO who’ll make an effort to visit you once a year to make sure you are happy with their services is at the top of my personal list. I’d also pick dependability, agility, and responsiveness.

Platform or Tools used	What We Did	Type of Testing	Results
Browsers	Design and maintain test cases Ensure the new functionalities Report and track bugs	Manual Testing	Almost 2000 test cases designed and maintained (~1800 Web UI tests and 200 API Manual Test Cases) Test Cases for new functionality periodically added (e.g., Swagger)
Linux System	Update and maintain server Analysis server log Solve server issue, if has	Manual Testing	For January through July 2017, XBO has tested over 30 patches for Mitel’s Linux server running OS. Server status reports are compiled and delivered daily, ensuring timely addressing of identified issues.
Linux System + java script	Performance result data (attached an instance) Generate chart, if necessary	Performance Testing	Performed general performance testing of critical service parameters (throughput of SMS and email) as well as priority functions (contact management, creating and sending of broadcasts, loading, and processing of campaign data). Results were used largely to identify differences in performance between various releases.
JMeter + Linux & Windows System	Design API script Execute API script Optimize API script Maintain API script issues	API Automation	In ongoing support of migration of key functional areas from Selenium automation to API automated testing, XBO has: developed over 250 manual API test scenarios, and developed scripts covering over 220 API automated test scenarios.
Selenium + Eclipse	Maintain selenium script Execute selenium script	Selenium Automation	Over 400 test case scenarios were automated using Selenium. Even with the focus toward API testing, Selenium tests will still be seen as value added. Not all Web UI capabilities will be ported to API.

Download This Case Study!