A.I. in QA: More Than Just a Bot I Kobiton

A.I. in QA: More Than Just a Bot I Kobiton

Hello everyone, my name is Drew Williams. I am the Marketing Director for Kobiton Bhutan. Thank you for joining us today on this webinar entitled "Artificial Intelligence and Testing." We are joined today by the Kobiton CTO Frank Moyer, who will talk about the different ways you can leverage AI and machine learning in QA, both holistically and on a granular level regarding test execution and optimization.

Webinar Structure The webinar will be a fireside chat-style discussion. We invite you to submit questions through the question submission box, and we'll answer them in a timely fashion if possible. Otherwise, we'll address them at the end.

Understanding AI and Machine Learning in Testing Frank Moyer differentiated between AI and machine learning, noting that machine learning is a subset of AI. AI comprises any computer program that accomplishes smart tasks to relieve human tasks, whereas machine learning involves training computers through human interaction to make them as adept as humans in specific tasks. Another concept is "self-healing," which allows test scripts to fix themselves between releases due to changes in selectors or the DOM, significantly reducing flaky tests.

Application of AI in Testing Moyer outlined various applications of AI and machine learning in QA:

• Anomaly Detection: AI identifies anomalies in system performance by comparing current execution against historical trends.
• Predictive Analytics: AI uses trends to predict future performance, including defect likelihood and capacity planning.
• Image Comparison: Comparing two application states to identify UI changes using convolutional neural networks (CNNs). CNNs enable more semantic comparisons, such as identifying button differences.
• Self-healing capabilities: These allow tests to adjust dynamically to changes in the DOM or selectors without manual intervention.

Important Models and Concepts An anomaly detection system, predictive analytics, and image comparison using CNNs are discussed in detail. A "taste model," which assesses predicted defects against actual outcomes, helps identify true positives, false positives, true negatives, and false negatives.

Key Players in AI and Testing Moyer highlighted various companies advancing AI in testing, including:

• Testim
• Mabel
• Functionize
• Applitools
• Testcraft
• Katalon

Each of these companies offers different functionalities, from bot execution and scriptless test authoring to visual comparison and self-healing.

AI for Web vs. Mobile Testing AI applications differ significantly between web and mobile testing due to the inherent complexities of mobile environments, such as varied device manufacturers rendering XML differently, less CPU and RAM, and device fragmentation.

Kobiton’s Journey and Innovations Moyer talked about Kobiton's efforts over the past three years, focusing on solving critical challenges in mobile environments, such as cross-device script compatibility and leveraging machine learning to enhance mobile testing.

Conclusion Moyer closed by addressing the inevitability and necessity of AI in QA, emphasizing that the role of testers will shift to subject matter experts who can harness AI tools to increase efficiency.

Keywords

• Artificial Intelligence (AI)
• Machine Learning
• Quality Assurance (QA)
• Anomaly Detection
• Predictive Analytics
• Image Comparison
• Self-healing
• Convolutional Neural Networks (CNN)
• Web Testing
• Mobile Testing
• Kobiton

FAQ

Q1: What is the difference between AI and machine learning in the context of testing? AI involves making computers perform smart tasks that typically require human intervention, while machine learning is a subset that enables computers to learn from data and human interaction to perform tasks efficiently.

Q2: How is anomaly detection used in testing? Anomaly detection leverages historical data to identify deviations in system behavior, such as performance, compared to what is considered normal based on past trends.

Q3: Can AI completely eliminate the role of the human tester? No, AI aims to complement the role of human testers by enhancing efficiency and capacity. Testers will remain crucial as subject matter experts who utilize AI tools.

Q4: Why is mobile testing more challenging compared to web testing? Mobile testing is more complex due to the varied rendering of XML by different device manufacturers, limited CPU/RAM, device fragmentation, and additional features like location and dynamic content.

Q5: What is the role of convolutional neural networks (CNNs) in image comparison? CNNs allow for more semantic image comparisons, identifying differences like color variations in buttons, making them superior to pixel-matching methods.

Q6: Who are some of the leading companies in AI-powered testing? Leading companies include Testim, Mabel, Functionize, Applitools, Testcraft, and Katalon, each offering unique AI-driven features for different aspects of testing.

Q7: What are the key benefits of using AI in QA? AI can significantly improve test execution by automating repetitive tasks, self-healing scripts, and providing predictive analytics to foresee potential issues.

Q8: What innovations has Kobiton introduced in AI and mobile testing? Kobiton has focused on solving mobile testing challenges, particularly in cross-device script compatibility, using machine learning to enhance the accuracy and efficiency of mobile tests.